Files
crearte-monorepo/docs/plans/2026-09-30-ci-test-baseline.md
T

404 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# P2 CI + 测试基线 实现计划
> **对人工审查者:** 必需的子技能:使用 subagent-driven-development(如果子智能体不存在则用 executing-plans)逐任务实现此计划。每个任务都由一个新鲜、有能力的工程师完成,他们可以将你的检查清单作为提示保留。不要并行执行两个实现任务。
**目标:** 三仓获得与其风险相称的 CI:后端 gofmt/vet/build/test + 真 Postgres 集成层(静默 skip 视为失败)+ 真栈 Playwright 全链路首次进 CI;部署仓获得 compose 守卫正反验证。
**架构:** 每仓一个 `.github/workflows/validate.yml`(对齐 crearte 既有文件的风格);e2e-stack job 落在 crearte-server 仓(私有/公开仓凭据不对称,见 spec §2.2 修订),用双 checkout 拼出脚本要求的兄弟仓布局并以 `CREARTE_STACK_BACKEND_REF=github.sha` 直测 PR 后端;crearte 侧只加两个脚本 env 开关(GO/REQUIRED),封死 runner 上的版本地板假绿与 CI 静默 skip 假绿。
**技术栈:** GitHub Actions YAML、Go 1.24(gh actions setup-go)、Node 24 + Playwright、docker compose v2、bash。
## 全局约束
- 分支:三仓各自新建 `feat/ci-test-baseline`(从各仓 master HEAD);**严禁直接 commit master**;合并(`git merge --no-ff`)+ 推送只在任务 5 由控制者执行。各仓提交前 `git branch --show-current` 核验。
- CHANGELOG 格式(三仓 AGENTS.md 逐字):同一条目英文行紧跟中文行(两行之间无空行);不同条目间空一行;高版本置顶。
- 本仓后端一切 go 命令走 docker:宿主 go 1.18 禁用。模板(缓存全热,首跑 1-3 分钟属正常):
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte-server/src
docker run --rm -e GOPROXY=https://goproxy.cn,direct -e GOSUMDB=sum.golang.google.cn -e GOCACHE=/gocache -v crearte_gocache:/gocache -v crearte_gomod:/go/pkg/mod -v "$PWD:/src" -w /src golang:1.24-alpine sh -c '<命令>'
```
- 长命令用 exec background=true + process poll(timeout≥15000) 续等,勿因无输出判死。
- YAML 校验统一用:`python3 -c "import yaml,sys; yaml.safe_load(open(sys.argv[1])); print('yaml ok')" <path>`。
- 不修改任何 `_test.go`;不动 `git.yoresee.cc`;GitHub 分支保护/Actions 启用查证属 owner 手动尾巴(spec §2.4),不入任务。
## 涉及文件:各仓库角色
| 仓库 | 文件 | 操作 |
|---|---|---|
| crearte | `src/scripts/e2e-stack.sh` | 修改(两个 env 开关) |
| crearte | `.github/workflows/validate.yml` | 修改(push 触发) |
| crearte | `docs/CHANGELOG.md` | 0.16.1 |
| crearte-server | `.github/workflows/validate.yml` | 新建(三 job) |
| crearte-server | `docs/CHANGELOG.md` | 0.11.1 |
| crearte-deploy | `.github/workflows/validate.yml`、`docs/CHANGELOG.md`、`README.md` | 新建/更新(0.5.1) |
| wrapper | `docs/specs/2026-09-30-ci-test-baseline-design.md`(已写)、ROADMAP、`docs/CHANGELOG.md` | 任务 5 收尾 |
---
### 任务 1:crearte — e2e-stack.sh 开关 + push 触发 + CHANGELOG 0.16.1
**文件:**
- 修改:`src/scripts/e2e-stack.sh`(两处,块 1.2)
- 修改:`.github/workflows/validate.yml`(块 1.4)
- 修改:`docs/CHANGELOG.md`(块 1.6)
- [ ] **步骤 1.1** 开分支并断言起点:
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte
test -z "$(git status --porcelain)" && git checkout -b feat/ci-test-baseline && git rev-parse HEAD
```
预期:HEAD 为 `eb5fbed`(master),分支创建成功。node_modules 已在,勿 npm ci。
- [ ] **步骤 1.2** 编辑 `src/scripts/e2e-stack.sh`。第一处——原文(约 27-28 行,逐字):
```bash
GO_BIN="/usr/local/go/bin/go"
command -v "$GO_BIN" >/dev/null 2>&1 || GO_BIN="$(command -v go 2>/dev/null || true)"
```
替换为:
```bash
# ⚠️ CI runner 的 /usr/local/go 不随 setup-go 变(可能低于 go.mod 地板 → skip 模式假绿)。
# CREARTE_STACK_GO 显式指定 go 二进制;不设即旧行为(本地零变化)。
GO_BIN="${CREARTE_STACK_GO:-/usr/local/go/bin/go}"
command -v "$GO_BIN" >/dev/null 2>&1 || GO_BIN="$(command -v go 2>/dev/null || true)"
```
第二处——原文(约 125-129 行,逐字):
```bash
else
echo "[stack] dependencies missing — running in SKIP mode (frontend only, full-loop will skip)"
(cd "$FRONT_ROOT" && npm run build:e2e) || exit 1
fi
```
替换为:
```bash
else
echo "[stack] dependencies missing — running in SKIP mode (frontend only, full-loop will skip)"
# ⚠️ CI 真栈 job 必须设 CREARTE_STACK_REQUIRED=1:skip 模式在 CI 里等于假绿(full-loop 全 skip 仍 exit 0)。
if [ "${CREARTE_STACK_REQUIRED:-0}" = "1" ]; then
echo "[stack] CREARTE_STACK_REQUIRED=1 — refusing silent skip, failing fast" >&2
exit 1
fi
(cd "$FRONT_ROOT" && npm run build:e2e) || exit 1
fi
```
- [ ] **步骤 1.3** 语法与行为探针(快,不建栈):
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte/src
bash -n scripts/e2e-stack.sh && echo SYNTAX-OK
mkdir -p /tmp/p2fakebin && printf '#!/bin/sh\nexit 127\n' > /tmp/p2fakebin/docker && chmod +x /tmp/p2fakebin/docker
PATH="/tmp/p2fakebin:$PATH" CREARTE_STACK_REQUIRED=1 bash scripts/e2e-stack.sh >/tmp/p2req.log 2>&1; echo "req rc=$?"; grep -c 'refusing silent skip' /tmp/p2req.log
```
预期:`SYNTAX-OK`;`req rc=1` 且 grep 输出 ≥1。控制组(不设 REQUIRED,同一 fake docker,15 秒即掐——skip 模式会去 build:e2e,属预期):
```bash
timeout 15 env PATH="/tmp/p2fakebin:$PATH" bash scripts/e2e-stack.sh >/tmp/p2skip.log 2>&1; echo "skip rc=$?"; grep -c 'SKIP mode' /tmp/p2skip.log
```
预期:grep ≥1(rc 可为 124=timeout,skip 分支确实到达即可)。
- [ ] **步骤 1.4** 编辑 `.github/workflows/validate.yml`:`on:` 块(逐字原文)
```yaml
on:
pull_request:
paths:
- 'src/**'
- '.github/workflows/validate.yml'
```
替换为:
```yaml
on:
pull_request:
paths:
- 'src/**'
- '.github/workflows/validate.yml'
push:
branches: [master]
```
两 job 与其余内容零改动。
- [ ] **步骤 1.5** `python3 -c "import yaml,sys; yaml.safe_load(open(sys.argv[1])); print('yaml ok')" .github/workflows/validate.yml` 预期 `yaml ok`。
- [ ] **步骤 1.6** `docs/CHANGELOG.md` 在 `# Changelog` 前言块之后、`## [0.16.0]` 之前插入(逐字,注意英中相邻):
```markdown
## [0.16.1] - 2026-09-30
### CI / 持续集成
- `scripts/e2e-stack.sh` gains `CREARTE_STACK_GO` (explicit go binary for the backend build — a CI runner whose `/usr/local/go` predates the backend's go.mod floor used to trip the version gate into silent SKIP mode, a false green) and `CREARTE_STACK_REQUIRED=1` (fail fast when the stack cannot start instead of skipping full-loop silently); `validate.yml` now also runs on pushes to `master`.
- `scripts/e2e-stack.sh` 新增 `CREARTE_STACK_GO`(显式指定编译后端的 go 二进制——runner 上 `/usr/local/go` 低于后端 go.mod 地板时曾掉进静默 SKIP 假绿)与 `CREARTE_STACK_REQUIRED=1`(栈起不来直接失败,不再静默跳过 full-loop);`validate.yml` 追加对 `master` 推送的触发。
```
- [ ] **步骤 1.7** 回归:`git diff --stat` 仅 3 文件;`grep -c '\[0.16.1\]' docs/CHANGELOG.md` = 1。本任务不触碰 TS 源,vitest 由任务 4 全量覆盖。
- [ ] **步骤 1.8** 覆盖提交:
```bash
git add src/scripts/e2e-stack.sh .github/workflows/validate.yml docs/CHANGELOG.md
git commit -m "ci: e2e-stack GO/REQUIRED env knobs + push:master trigger (P2)"
```
---
### 任务 2:crearte-server — validate.yml 三 job + CHANGELOG 0.11.1
**文件:**
- 创建:`.github/workflows/validate.yml`(块 2.2 全文逐字)
- 修改:`docs/CHANGELOG.md`(块 2.4)
- [ ] **步骤 2.1** 开分支:
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte-server
test -z "$(git status --porcelain)" && git checkout -b feat/ci-test-baseline && git rev-parse HEAD | head -c 7
```
预期:`e3da246`。
- [ ] **步骤 2.2** 写 `.github/workflows/validate.yml` **全文**(逐字;目录不存在则创建):
```yaml
name: validate
on:
pull_request:
paths:
- 'src/**'
- '.github/workflows/validate.yml'
push:
branches: [master]
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: src
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: src/go.mod
cache-dependency-path: src/go.sum
- name: gofmt
run: test -z "$(gofmt -l .)"
- name: vet
run: go vet ./...
- name: build
run: go build ./...
- name: test
run: go test ./...
integration:
runs-on: ubuntu-latest
needs: check
timeout-minutes: 15
services:
postgres:
image: postgres:17-alpine
env:
POSTGRES_USER: crearte
POSTGRES_PASSWORD: crearte
POSTGRES_DB: crearte
ports: ['5432:5432']
options: >-
--health-cmd "pg_isready -U crearte"
--health-interval 5s
--health-timeout 5s
--health-retries 10
defaults:
run:
working-directory: src
env:
TEST_DATABASE_URL: postgres://crearte:crearte@localhost:5432/crearte?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: src/go.mod
cache-dependency-path: src/go.sum
- name: integration tests (empty DB, serialized -p 1, verbose)
run: set -o pipefail && go test -v -count=1 -p 1 ./... 2>&1 | tee integration.log
- name: fail on silent skips
run: '! grep -q "skipping postgres integration test" integration.log'
e2e-stack:
runs-on: ubuntu-latest
needs: integration
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
with:
path: crearte-server
- uses: actions/checkout@v4
with:
repository: XingfenD/crearte
path: crearte
- uses: actions/setup-go@v5
with:
go-version-file: crearte-server/src/go.mod
cache-dependency-path: crearte-server/src/go.sum
- uses: actions/setup-node@v4
with:
node-version: 24
cache: npm
cache-dependency-path: crearte/src/package-lock.json
- name: Install Playwright Chromium
working-directory: crearte/src
run: npx playwright install --with-deps chromium
- name: Real-stack full-loop e2e
working-directory: crearte/src
env:
CREARTE_STACK_BACKEND_REF: ${{ github.sha }}
CREARTE_STACK_GO: go
CREARTE_STACK_REQUIRED: '1'
run: npm ci && npm run e2e:stack
```
设计要点(审查透镜,勿改):e2e job 双 checkout 的 `path` 必须保持 `crearte` 与 `crearte-server` 同级,否则脚本的兄弟仓路径解析失败(见脚本头部 ⚠️ 注释);`CREARTE_STACK_GO: go` 走 setup-go 的 PATH;`CREARTE_STACK_REQUIRED: '1'` 封死静默 skip。
- [ ] **步骤 2.3** 校验:yaml ok;且 `grep -c "path: crearte-server" .github/workflows/validate.yml` =1、`grep -c '\-p 1' .github/workflows/validate.yml` =1、`grep -c 'skipping postgres integration test' .github/workflows/validate.yml` =1。
- [ ] **步骤 2.4** `docs/CHANGELOG.md` 前言后、`## [0.11.0]` 前插入(逐字):
```markdown
## [0.11.1] - 2026-09-30
### CI / 持续集成
- New `validate.yml`: `check` (gofmt/vet/build/test), `integration` (empty Postgres 17 service, serialized `-p 1`, plus a guard that fails the job on any silent "TEST_DATABASE_URL not set" skip — empty-DB migration bootstrapping is exercised every PR), and `e2e-stack` (real-stack Playwright full-loop: sibling public crearte frontend + this PR's backend SHA, `CREARTE_STACK_REQUIRED=1` forbids false greens).
- 新增 `validate.yml` 三 job:`check`(gofmt/vet/build/test)、`integration`(空库 postgres 17 service、`-p 1` 串行、外加"任何 TEST_DATABASE_URL 静默 skip 即判失败"的守卫——每次 PR 都在走空库完整迁移)、`e2e-stack`(真栈 Playwright full-loop:兄弟公开仓 crearte 前端 + 本 PR 后端 SHA,`CREARTE_STACK_REQUIRED=1` 杜绝假绿)。
```
- [ ] **步骤 2.5** 覆盖提交:
```bash
git add .github/workflows/validate.yml docs/CHANGELOG.md
git commit -m "ci: validate workflow — check/integration/e2e-stack with skip guards (P2)"
```
---
### 任务 3:crearte-deploy — compose config 正反两路 + 0.5.1 + README 边界小节
**文件:**
- 创建:`.github/workflows/validate.yml`(块 3.2 全文)
- 修改:`docs/CHANGELOG.md`(块 3.4)、`README.md`(块 3.5)
- [ ] **步骤 3.1** 开分支:
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte-deploy
test -z "$(git status --porcelain)" && git checkout -b feat/ci-test-baseline && git rev-parse HEAD | head -c 7
```
预期:`4d53d58`。
- [ ] **步骤 3.2** 写 `.github/workflows/validate.yml` **全文**(逐字):
```yaml
name: validate
on:
pull_request:
paths:
- 'docker-compose.yml'
- '.env.example'
- '.github/workflows/validate.yml'
push:
branches: [master]
jobs:
compose:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: compose parses under every profile
env:
POSTGRES_PASSWORD: ***-a-real-secret
MINIO_ROOT_USER: ci
MINIO_ROOT_PASSWORD: ***
AUTH_TOKEN_SECRET: ***
BUNDLE_KEK_k1: ci-not-a-real-secret
run: |
for profile in dev prod mock debug; do
docker compose --profile "$profile" config -q
done
- name: guards refuse an empty POSTGRES_PASSWORD
env:
POSTGRES_PASSWORD: ''
run: |
if docker compose --profile dev config -q 2>/dev/null; then
echo 'compose accepted an empty POSTGRES_PASSWORD — :? guard bypassed' >&2
exit 1
fi
```
- [ ] **步骤 3.3** 本机跑通两 job 的等价命令(与 CI 完全同构,compose 只解析不建容器):
```bash
POSTGRES_PASSWORD=*** MINIO_ROOT_USER=ci MINIO_ROOT_PASSWORD=*** AUTH_TOKEN_SECRET=*** BUNDLE_KEK_k1=*** bash -c 'for p in dev prod mock debug; do docker compose --profile "$p" config -q || { echo "PROFILE-FAIL $p"; exit 1; }; done; echo CONFIG-ALL-OK'
POSTGRES_PASSWORD='' bash -c 'docker compose --profile dev config -q >/dev/null 2>&1; echo "empty-pw rc=$?"'
```
预期:`CONFIG-ALL-OK`;`empty-pw rc=` 非 0。
- [ ] **步骤 3.4** `docs/CHANGELOG.md` 前言后、`## [0.5.0]` 前插入(逐字):
```markdown
## [0.5.1] - 2026-09-30
### CI / 持续集成
- New `validate.yml`: parses `docker compose config -q` under dev/prod/mock/debug with placeholder secrets, and asserts the `:?` guards refuse an empty `POSTGRES_PASSWORD`. Single-repo CI validates parsing only — deployable cross-repo build contexts live in the monorepo layout (README note added).
- 新增 `validate.yml`:用占位 secret 对 dev/prod/mock/debug 四 profile 跑 `docker compose config -q`,并断言 `:?` 守卫在 `POSTGRES_PASSWORD` 为空时拒绝解析。单仓 CI 只验解析——可部署的跨仓 build 上下文存在于 monorepo 布局(README 已加边界说明)。
```
- [ ] **步骤 3.5** `README.md`:在「### prod 栈写侧(本机自包含演练)」小节标题行**之前**插入新小节(逐字):
```markdown
### 单仓 checkout 能验什么(CI 边界)
本仓 CI(`validate.yml`)只校验 compose 解析与各 profile 的 `:?` 守卫——单仓 checkout 没有 `../crearte-server`、`../crearte` 兄弟目录,build context 无法成立。可部署性与跨仓联调以 monorepo(wrapper `docs/` 登记)演练为准:`../../` 级 `./clone_all.sh` 布局 + `docker compose up`。
```
- [ ] **步骤 3.6** yaml ok + `git diff --stat` 仅 3 文件。
- [ ] **步骤 3.7** 覆盖提交:
```bash
git add .github/workflows/validate.yml docs/CHANGELOG.md README.md
git commit -m "ci: compose config validation with guard negative test (P2)"
```
---
### 任务 4:本机等价模拟(只验不改;live 证据留档)
**文件:** 创建:`/root/.openclaw/workspace/coder/crearte-monorepo/.superpowers/sdd/p2-task-4-report.md`(该目录 gitignored)。零代码改动。
前置:任务 1-3 审查通过并合入各自 feat 分支 HEAD(未合并 master 也可跑,模拟的是 job 内容而非触发)。
- [ ] **步骤 4.1** check job 等价(docker 模板见全局约束):`gofmt -l .` 空断言 + `go vet ./...` + `go build ./...` + `go test ./...`,记录输出尾部。
- [ ] **步骤 4.2** integration 等价:起一次性 pg(**5434 端口,避免与演练栈 5433 冲突**):
```bash
docker rm -f p2-ci-pg >/dev/null 2>&1
docker run -d --name p2-ci-pg -e POSTGRES_USER=crearte -e POSTGRES_PASSWORD=crearte -e POSTGRES_DB=crearte -p 5434:5432 postgres:17-alpine
for i in $(seq 1 30); do docker exec p2-ci-pg pg_isready -U crearte >/dev/null 2>&1 && break; sleep 1; done
```
然后在 golang:1.24-alpine 容器里带 `--add-host=host.docker.internal:host-gateway` 与 `TEST_DATABASE_URL=postgres://crearte:crearte@host.docker.internal:5434/crearte?sslmode=disable` 跑 `sh -c 'set -o pipefail && go test -v -count=1 -p 1 ./... 2>&1 | tee /tmp/integration.log; ! grep -q "skipping postgres integration test" /tmp/integration.log && echo SKIP-GUARD-PASS'`(tee 到容器内文件后 `docker cp` 或 tail 取证;守卫须打印 SKIP-GUARD-PASS)。**跑两遍**:第二遍证明空库迁移幂等共存。记录 `--- PASS` 计数与 `ok` 包列表。跑完 `docker rm -f p2-ci-pg`。
- [ ] **步骤 4.3** 真栈 e2e(本机 monorepo 天然满足兄弟仓布局;宿主 go 1.26.8 过地板检查):
```bash
cd /root/.openclaw/workspace/coder/crearte-monorepo/crearte/src
npx playwright install chromium 2>&1 | tail -1
CREARTE_STACK_REQUIRED=1 npm run e2e:stack 2>&1 | tee /tmp/p2e2e.log | tail -12
grep -c '\[stack\] ready:' /tmp/p2e2e.log
```
预期:日志含 `[stack] ready:`(grep ≥1)且 playwright 输出 `N passed`、0 failed、无 skipped 的 full-loop 用例。若浏览器版本不匹配 `--with-deps` 需要 sudo——本机 root 可直接 `--with-deps`。全程 background+poll 耐心(首跑编译后端 1-3 分钟 + 前端 build)。
- [ ] **步骤 4.4** deploy 两 job 等价:复用任务 3 块 3.3 两条命令,确认仍 `CONFIG-ALL-OK` / 非 0。
- [ ] **步骤 4.5** 三仓 workflow 终稿 yaml 解析全过(从各 feat HEAD 读文件)。
- [ ] **步骤 4.6** 清理与现场核验:`docker ps -a` 无 crearte-stack-*/p2-ci-pg 残留;`git -C ../crearte-server worktree list` 只剩主树;`ls fixtures/generated/stack.json` 不存在;`rm -f /tmp/p2*`;三仓 `git status --porcelain` 全空、仍停在各自 feat 分支。报告写全(p2-task-4-report.md)。
---
### 任务 5:三仓合并推送 + wrapper 收尾(控制者直接执行)
- [ ] **步骤 5.1** crearte-server:`git checkout master && git pull --ff-only origin master && git merge --no-ff feat/ci-test-baseline -m "Merge branch 'feat/ci-test-baseline' — CI + test baseline (P2)" && git push origin master && git branch -d feat/ci-test-baseline; git push origin --delete feat/ci-test-baseline 2>/dev/null; git log --oneline -1`。
- [ ] **步骤 5.2** crearte、crearte-deploy 同款 merge message 尾缀 `(P2)`。
- [ ] **步骤 5.3** wrapper:ROADMAP P2 行 → `**完成**(server \`<sha>\` / crearte \`<sha>\` / deploy \`<sha>\`,2026-09-30 合并推送;e2e-stack 宿主仓修订见 spec §2.2;Actions/分支保护 owner 手动)`;索引表加 P2 行;CHANGELOG 0.2.4 双语;commit+push。
- [ ] **步骤 5.4** 终态核验:三仓 `git pull --ff-only origin master` 均 Already up to date + `./clone_all.sh` 零失败。
- [ ] **步骤 5.5** `--ff-only` 拉来冲突时 `git merge --abort` 上报,禁强推。
---
## 自检记录(写计划后内联核过)
1. **规格覆盖**:spec §2.1→任务 2(check/integration+守卫);§2.2→任务 1(GO/REQUIRED 开关、crearte push 触发)+ 任务 2(e2e-stack job);§2.3→任务 3;§3 验证→任务 4 步骤 1-6;§2.4 owner 尾巴→任务 5 步骤 3 记入 ROADMAP 行。✔
2. **占位符扫描**:无 TODO/待定;`ci-not-a-real-secret` 是刻意占位值(CI config -q 不消费真值,P1 教训「占位冒充真值」不适用于此处——已按 P1 教训改用无歧义占位串并在注释声明,且 deploy 反路测试证明空值守卫仍在)。✔
3. **命名一致**:`CREARTE_STACK_GO/CREARTE_STACK_REQUIRED/CREARTE_STACK_BACKEND_REF`(任务 1 定义→任务 2 使用,拼写逐字);`integration.log` 守卫消息取 `migrate_test.go:22` 等 7 处公共子串 "skipping postgres integration test"(已 grep 全量核对,无第八处变体);路径 `crearte`/`crearte-server` 兄弟布局与 `e2e-stack.sh:11-15` 实际解析逐字对齐。✔