diff --git a/superpowers/plans/2026-08-11-yoresee-doc-sdk-migration.md b/superpowers/plans/2026-08-11-yoresee-doc-sdk-migration.md new file mode 100644 index 0000000..10267b0 --- /dev/null +++ b/superpowers/plans/2026-08-11-yoresee-doc-sdk-migration.md @@ -0,0 +1,584 @@ +# Yoresee Doc SDK Migration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Publish versioned Go and Node protobuf SDK repositories on Gitea and migrate backend, collab-go, frontend, and collab to consume them. + +**Architecture:** `proto` remains the only source of the protobuf contract. The two SDK repositories commit generated artifacts: Go stubs in the Go module root, and both ESM/Connect and CommonJS/gRPC outputs in separate Node subdirectories. Services consume tagged Gitea dependencies; existing local generation scripts and Dockerfiles remain unchanged. + +**Tech Stack:** Protobuf, Buf 1.72.0, `protoc-gen-go` 1.36.12, `protoc-gen-go-grpc` 1.6.2, `protoc-gen-es`, `protoc-gen-connect-es`, `grpc-tools`, Go modules, npm Git dependencies, Gitea Git over SSH. + +## Global Constraints + +- The protobuf source remains `proto/yoresee_doc/v1/yoresee_doc.proto`. +- The Go module path is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go`. +- The Node package name is `@yoresee-doc/sdk-node`. +- The initial SDK release is `v0.1.0`; consumers pin that tag. +- The protobuf `go_package` is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb`. +- Do not modify `deploy/script/gen_proto.sh`, Dockerfiles, Docker Compose, `clone_all.sh`, or CI/CD configuration. +- Do not change protobuf messages, fields, services, or runtime behavior. +- Preserve unrelated pre-existing changes, including untracked files under `backend/`. +- Private Gitea module/package reads require working SSH access; Go consumers use `GOPRIVATE=git.yoresee.cc/YoreseeDoc`. + +--- + +### Task 1: Point The Proto Contract At The Go SDK + +**Files:** +- Modify: `proto/yoresee_doc/v1/yoresee_doc.proto:5` +- Test: proto diff and source-format checks + +**Interfaces:** +- Produces the canonical Go import path used by generated SDK code and both Go services. + +- [ ] **Step 1: Replace the Go package option** + +Change the only `go_package` option to: + +```protobuf +option go_package = "git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb"; +``` + +Do not alter any other proto line. + +- [ ] **Step 2: Verify the focused proto diff** + +Run from `proto/`: + +```bash +git diff --check +git diff -- yoresee_doc/v1/yoresee_doc.proto +``` + +Expected: one changed `go_package` line and no whitespace errors. + +- [ ] **Step 3: Commit and push the proto source change** + +Run from `proto/`, staging only the proto file: + +```bash +git add yoresee_doc/v1/yoresee_doc.proto +git diff --cached --check +git commit -m "feat: point Go protobuf package at SDK" +git push origin master +``` + +Record the resulting source commit with: + +```bash +git rev-parse HEAD +``` + +The commit hash is recorded in both SDK READMEs. + +### Task 2: Generate And Validate The Go SDK + +**Files:** +- Create: `yoresee_doc_sdk_go/go.mod` +- Create: `yoresee_doc_sdk_go/go.sum` +- Create: `yoresee_doc_sdk_go/buf.gen.yaml` +- Create: `yoresee_doc_sdk_go/README.md` +- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc.pb.go` +- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc_grpc.pb.go` + +**Interfaces:** +- Consumes the proto directory from `yoresee_doc/proto`. +- Produces Go package `yoreseedocpb` at import path `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`. + +- [ ] **Step 1: Initialize the empty SDK repository on `master`** + +Run from `my-repos/`: + +```bash +cd yoresee_doc_sdk_go +git init -b master +git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git +``` + +If the repository already has a local Git initialization, keep it and only add the missing remote. + +- [ ] **Step 2: Create the Go module metadata** + +Create `go.mod` with: + +```go +module git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go + +go 1.24 + +require ( + google.golang.org/grpc v1.79.2 + google.golang.org/protobuf v1.36.10 +) +``` + +Create `buf.gen.yaml` with: + +```yaml +version: v2 +plugins: + - local: protoc-gen-go + out: . + opt: + - paths=source_relative + - local: protoc-gen-go-grpc + out: . + opt: + - paths=source_relative +``` + +- [ ] **Step 3: Generate both Go protobuf outputs** + +Run from `yoresee_doc_sdk_go/`: + +```bash +export PATH="$HOME/go/bin:$PATH" +buf generate ../yoresee_doc/proto --template buf.gen.yaml +``` + +Expected files: + +```text +yoresee_doc/v1/yoresee_doc.pb.go +yoresee_doc/v1/yoresee_doc_grpc.pb.go +``` + +- [ ] **Step 4: Resolve and test the Go module** + +Run from `yoresee_doc_sdk_go/`: + +```bash +go mod tidy +go test ./... +go vet ./... +``` + +Expected: the generated package compiles with no test or vet failures. + +- [ ] **Step 5: Write the Go SDK README** + +Document the module path, generated package import example, source proto path, recorded proto commit hash from Task 1, generator versions, and the fact that generated artifacts are committed. + +- [ ] **Step 6: Commit the Go SDK artifact** + +Run from `yoresee_doc_sdk_go/`, staging only SDK files: + +```bash +git add README.md buf.gen.yaml go.mod go.sum yoresee_doc +git diff --cached --check +git commit -m "feat: add generated Go protobuf SDK" +``` + +### Task 3: Generate And Validate The Node SDK + +**Files:** +- Create: `yoresee_doc_sdk_node/package.json` +- Create: `yoresee_doc_sdk_node/package-lock.json` +- Create: `yoresee_doc_sdk_node/buf.gen.yaml` +- Create: `yoresee_doc_sdk_node/README.md` +- Create: `yoresee_doc_sdk_node/esm/index.js` +- Create: `yoresee_doc_sdk_node/cjs/index.js` +- Create: `yoresee_doc_sdk_node/cjs/package.json` +- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_pb.js` +- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_connect.js` +- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_pb.js` +- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js` + +**Interfaces:** +- ESM consumers import `@yoresee-doc/sdk-node/esm/yoresee_doc/v1/...`. +- CommonJS consumers require `@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/...`. + +- [ ] **Step 1: Initialize the empty Node SDK repository on `master`** + +Run from `my-repos/`: + +```bash +cd yoresee_doc_sdk_node +git init -b master +git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git +``` + +If the repository already has a local Git initialization, keep it and only add the missing remote. + +- [ ] **Step 2: Create the Node package contract** + +Create `package.json` with this contract: + +```json +{ + "name": "@yoresee-doc/sdk-node", + "version": "0.1.0", + "type": "module", + "files": ["esm", "cjs", "README.md"], + "exports": { + ".": { + "import": "./esm/index.js", + "require": "./cjs/index.js" + }, + "./esm/*": "./esm/*", + "./cjs/*": "./cjs/*" + }, + "dependencies": { + "@bufbuild/protobuf": "^1.10.0", + "@connectrpc/connect": "^1.7.0", + "@grpc/grpc-js": "^1.12.0", + "google-protobuf": "^4.0.2" + } +} +``` + +Create `cjs/package.json` with: + +```json +{ + "type": "commonjs" +} +``` + +Create `buf.gen.yaml` with: + +```yaml +version: v2 +plugins: + - local: protoc-gen-es + out: esm + opt: + - target=js + - import_extension=.js + - local: protoc-gen-connect-es + out: esm + opt: + - target=js + - import_extension=.js +``` + +- [ ] **Step 3: Generate the ESM/Connect outputs** + +Run from `yoresee_doc_sdk_node/`: + +```bash +buf generate ../yoresee_doc/proto --template buf.gen.yaml +``` + +Expected files: + +```text +esm/yoresee_doc/v1/yoresee_doc_pb.js +esm/yoresee_doc/v1/yoresee_doc_connect.js +``` + +- [ ] **Step 4: Generate the CommonJS/gRPC outputs** + +Run from `yoresee_doc_sdk_node/`: + +```bash +mkdir -p cjs +grpc_tools_node_protoc -I ../yoresee_doc/proto \ + --plugin=protoc-gen-grpc="$(command -v grpc_tools_node_protoc_plugin)" \ + --plugin=protoc-gen-js="$(command -v protoc-gen-js)" \ + --grpc_out=grpc_js:cjs \ + --js_out=import_style=commonjs,binary:cjs \ + ../yoresee_doc/proto/yoresee_doc/v1/yoresee_doc.proto +``` + +Expected files: + +```text +cjs/yoresee_doc/v1/yoresee_doc_pb.js +cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js +``` + +- [ ] **Step 5: Add package entry points and lock dependencies** + +Create `esm/index.js`: + +```js +export * from './yoresee_doc/v1/yoresee_doc_pb.js' +export * from './yoresee_doc/v1/yoresee_doc_connect.js' +``` + +Create `cjs/index.js`: + +```js +module.exports = { + messages: require('./yoresee_doc/v1/yoresee_doc_pb.js'), + services: require('./yoresee_doc/v1/yoresee_doc_grpc_pb.js') +} +``` + +Run `npm install` in the SDK root to create `package-lock.json`. + +- [ ] **Step 6: Validate both Node module formats** + +Run from `yoresee_doc_sdk_node/`: + +```bash +npm pack --dry-run +node --input-type=module -e "import('./esm/yoresee_doc/v1/yoresee_doc_pb.js').then(() => console.log('esm ok'))" +node -e "require('./cjs/yoresee_doc/v1/yoresee_doc_pb.js'); require('./cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js'); console.log('cjs ok')" +``` + +The ESM check must load the generated ESM file without a module-format error. The consumer package-path checks are performed after the service dependencies are installed in Tasks 5 and 6. + +- [ ] **Step 7: Write and commit the Node SDK artifact** + +Document the source proto commit, generator versions, ESM/Connect import path, CommonJS/gRPC import path, and Gitea tag in `README.md`. Then run: + +```bash +git add README.md buf.gen.yaml package.json package-lock.json esm cjs +git diff --cached --check +git commit -m "feat: add generated Node protobuf SDK" +``` + +### Task 4: Publish Both SDK Releases + +**Files:** +- Modify: Git history and refs of `yoresee_doc_sdk_go` +- Modify: Git history and refs of `yoresee_doc_sdk_node` + +**Interfaces:** +- Produces `master` commits and immutable `v0.1.0` tags consumed by the services. + +- [ ] **Step 1: Push each SDK commit to `master`** + +Run separately in each SDK repository after checking the staged diff and clean working tree: + +```bash +git push -u origin master +``` + +- [ ] **Step 2: Tag and push the initial release** + +Run separately in each SDK repository: + +```bash +git tag -a v0.1.0 -m "Release v0.1.0" +git push origin v0.1.0 +``` + +- [ ] **Step 3: Verify remote refs** + +Run from `my-repos/`: + +```bash +git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git +git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git +git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git +git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git +``` + +Expected: `master` and `v0.1.0` exist in both repositories. + +### Task 5: Migrate Go Service Consumers + +**Files:** +- Modify: all backend Go files importing `github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1` +- Modify: `backend/go.mod`, `backend/go.sum` +- Modify: all collab-go Go files importing `github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1` +- Modify: `collab-go/go.mod`, `collab-go/go.sum` +- Do not modify: `backend/AGENTS.md`, `backend/docs/`, Dockerfiles, or generation scripts + +**Interfaces:** +- Both services import `yoreseedocpb` from `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`. + +- [ ] **Step 1: Replace backend import paths** + +In every backend source file returned by: + +```bash +git grep -l 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go' +``` + +replace only the import path with: + +```go +git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1 +``` + +Preserve existing aliases such as `pb`. + +- [ ] **Step 2: Add the pinned Go SDK dependency to backend** + +Add this direct requirement to `backend/go.mod`: + +```go +git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go v0.1.0 +``` + +Run from `backend/`: + +```bash +GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy +``` + +- [ ] **Step 3: Replace collab-go import paths** + +In every collab-go source file returned by: + +```bash +git grep -l 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go' +``` + +replace only the import path with: + +```go +git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1 +``` + +Preserve the existing `yoreseedocpb` alias. + +- [ ] **Step 4: Add the pinned Go SDK dependency to collab-go** + +Add the same direct requirement to `collab-go/go.mod`, then run from `collab-go/`: + +```bash +GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy +``` + +- [ ] **Step 5: Verify old paths are gone and Go services compile** + +Run: + +```bash +(cd backend && git grep 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go') || true +(cd collab-go && git grep 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go') || true +(cd backend && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...) +(cd collab-go && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...) +``` + +Expected: no old import matches and both modules compile against the remote SDK tag. + +### Task 6: Migrate Node Service Consumers + +**Files:** +- Modify: `frontend/package.json`, `frontend/package-lock.json` +- Modify: `frontend/src/services/grpc_client.js` +- Modify: `collab/package.json`, `collab/package-lock.json` +- Modify: `collab/src/grpc/client.js` +- Modify: `collab/src/grpc/document.js` +- Modify: `collab/src/grpc/system.js` +- Do not modify: Node Dockerfiles or generation scripts + +**Interfaces:** +- Frontend uses the SDK ESM/Connect subpaths. +- Collab uses the SDK CommonJS/gRPC subpaths. + +- [ ] **Step 1: Add the pinned Git dependency to frontend and collab** + +Add this dependency to both `package.json` files: + +```json +"@yoresee-doc/sdk-node": "git+ssh://git@git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_node.git#v0.1.0" +``` + +Run `npm install` separately in `frontend/` and `collab/` to update their lockfiles. The command requires SSH read access to Gitea. + +- [ ] **Step 2: Replace frontend generated imports** + +In `frontend/src/services/grpc_client.js`, replace: + +```js +@/gen/yoresee_doc/v1/yoresee_doc_connect.js +@/gen/yoresee_doc/v1/yoresee_doc_pb.js +``` + +with: + +```js +@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_connect.js +@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_pb.js +``` + +- [ ] **Step 3: Replace collab generated requires** + +In the three `collab/src/grpc/*.js` files, replace the relative `../gen/yoresee_doc/v1/` prefix with: + +```js +@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/ +``` + +Keep the existing generated filenames and CommonJS `require` calls. + +- [ ] **Step 4: Verify Node imports and builds** + +Run: + +```bash +(cd frontend && npm run build) +(cd collab && node -e "require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_pb'); require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb'); console.log('collab SDK imports ok')") +``` + +Expected: no service imports point to local generated directories, the frontend build succeeds, and collab loads both CJS modules. + +### Task 7: Commit And Push Service Migrations + +**Files:** +- Modify: `proto` repository history already prepared in Task 1 +- Modify: `backend` repository history +- Modify: `collab-go` repository history +- Modify: `frontend` repository history +- Modify: `collab` repository history + +- [ ] **Step 1: Review each component diff and exclude unrelated changes** + +Before every commit, run in that component: + +```bash +git status --short +git diff --check +git diff +``` + +In `backend`, do not stage the pre-existing untracked `AGENTS.md` or `docs/` files. + +- [ ] **Step 2: Commit each service migration separately** + +Use these commit messages and stage only the listed migration files: + +```text +backend: feat: consume generated Go SDK +collab-go: feat: consume generated Go SDK +frontend: feat: consume generated Node SDK +collab: feat: consume generated Node SDK +``` + +- [ ] **Step 3: Push each service `master` branch** + +Run from each service repository after its focused commit: + +```bash +git push origin master +``` + +Do not update unrelated root gitlinks or modify deployment scripts in this task. + +### Task 8: Final Cross-Repository Verification + +**Files:** +- Verify: both SDK repositories, `proto`, `backend`, `collab-go`, `frontend`, and `collab` + +- [ ] **Step 1: Verify all remotes and release refs** + +Run: + +```bash +git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git +git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git +git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git +git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git +``` + +- [ ] **Step 2: Verify service dependency metadata** + +Run: + +```bash +(cd backend && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go') +(cd collab-go && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go') +(cd frontend && git grep '@yoresee-doc/sdk-node' -- package.json src) +(cd collab && git grep '@yoresee-doc/sdk-node' -- package.json src) +``` + +- [ ] **Step 3: Verify clean intended worktrees** + +Run `git status --short` in each changed component and confirm only explicitly preserved unrelated files remain. Confirm no generated `node_modules`, secrets, or build output is staged.