docs/superpowers/plans/2026-08-11-yoresee-doc-sdk-migration.md

# Yoresee Doc SDK Migration Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Publish versioned Go and Node protobuf SDK repositories on Gitea and migrate backend, collab-go, frontend, and collab to consume them.

**Architecture:** `proto` remains the only source of the protobuf contract. The two SDK repositories commit generated artifacts: Go stubs in the Go module root, and both ESM/Connect and CommonJS/gRPC outputs in separate Node subdirectories. Services consume tagged Gitea dependencies; existing local generation scripts and Dockerfiles remain unchanged.

**Tech Stack:** Protobuf, Buf 1.72.0, `protoc-gen-go` 1.36.12, `protoc-gen-go-grpc` 1.6.2, `protoc-gen-es`, `protoc-gen-connect-es`, `grpc-tools` 1.13.1, `protoc-gen-js` 3.21.4-4, Go modules, npm Git dependencies, Gitea Git over SSH.

## Global Constraints

- The protobuf source remains `proto/yoresee_doc/v1/yoresee_doc.proto`.
- The Go module path is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go`.
- The Node package name is `@yoresee-doc/sdk-node`.
- The initial SDK release is `v0.1.0`; consumers pin that tag.
- The protobuf `go_package` is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb`.
- Do not modify `deploy/script/gen_proto.sh`, Dockerfiles, Docker Compose, `clone_all.sh`, or CI/CD configuration.
- Do not change protobuf messages, fields, services, or runtime behavior.
- Preserve unrelated pre-existing changes, including untracked files under `backend/`.
- Private Gitea module/package reads require working SSH access; Go consumers use `GOPRIVATE=git.yoresee.cc/YoreseeDoc`.

---

### Task 1: Point The Proto Contract At The Go SDK

**Files:**
- Modify: `proto/yoresee_doc/v1/yoresee_doc.proto:5`
- Test: proto diff and source-format checks

**Interfaces:**
- Produces the canonical Go import path used by generated SDK code and both Go services.

- [ ] **Step 1: Replace the Go package option**

Change the only `go_package` option to:

```protobuf
option go_package = "git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb";
```

Do not alter any other proto line.

- [ ] **Step 2: Verify the focused proto diff**

Run from `proto/`:

```bash
git diff --check
git diff -- yoresee_doc/v1/yoresee_doc.proto
```

Expected: one changed `go_package` line and no whitespace errors.

- [ ] **Step 3: Commit and push the proto source change**

Run from `proto/`, staging only the proto file:

```bash
git add yoresee_doc/v1/yoresee_doc.proto
git diff --cached --check
git commit -m "feat: point Go protobuf package at SDK"
git push origin master
```

Record the resulting source commit with:

```bash
git rev-parse HEAD
```

The commit hash is recorded in both SDK READMEs.

### Task 2: Generate And Validate The Go SDK

**Files:**
- Create: `yoresee_doc_sdk_go/go.mod`
- Create: `yoresee_doc_sdk_go/go.sum`
- Create: `yoresee_doc_sdk_go/buf.gen.yaml`
- Create: `yoresee_doc_sdk_go/README.md`
- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc.pb.go`
- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc_grpc.pb.go`

**Interfaces:**
- Consumes the proto directory from `yoresee_doc/proto`.
- Produces Go package `yoreseedocpb` at import path `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`.

- [ ] **Step 1: Initialize the empty SDK repository on `master`**

Run from `my-repos/`:

```bash
cd yoresee_doc_sdk_go
git init -b master
git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
```

If the repository already has a local Git initialization, keep it and only add the missing remote.

- [ ] **Step 2: Create the Go module metadata**

Create `go.mod` with:

```go
module git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go

go 1.24

require (
	google.golang.org/grpc v1.79.2
	google.golang.org/protobuf v1.36.10
)
```

Create `buf.gen.yaml` with:

```yaml
version: v2
plugins:
  - local: protoc-gen-go
    out: .
    opt:
      - paths=source_relative
  - local: protoc-gen-go-grpc
    out: .
    opt:
      - paths=source_relative
```

- [ ] **Step 3: Generate both Go protobuf outputs**

Run from `yoresee_doc_sdk_go/`:

```bash
export PATH="$HOME/go/bin:$PATH"
buf generate ../yoresee_doc/proto --template buf.gen.yaml
```

Expected files:

```text
yoresee_doc/v1/yoresee_doc.pb.go
yoresee_doc/v1/yoresee_doc_grpc.pb.go
```

- [ ] **Step 4: Resolve and test the Go module**

Run from `yoresee_doc_sdk_go/`:

```bash
go mod tidy
go test ./...
go vet ./...
```

Expected: the generated package compiles with no test or vet failures.

- [ ] **Step 5: Write the Go SDK README**

Document the module path, generated package import example, source proto path, recorded proto commit hash from Task 1, generator versions, and the fact that generated artifacts are committed.

- [ ] **Step 6: Commit the Go SDK artifact**

Run from `yoresee_doc_sdk_go/`, staging only SDK files:

```bash
git add README.md buf.gen.yaml go.mod go.sum yoresee_doc
git diff --cached --check
git commit -m "feat: add generated Go protobuf SDK"
```

### Task 3: Generate And Validate The Node SDK

**Files:**
- Create: `yoresee_doc_sdk_node/package.json`
- Create: `yoresee_doc_sdk_node/package-lock.json`
- Create: `yoresee_doc_sdk_node/buf.gen.yaml`
- Create: `yoresee_doc_sdk_node/README.md`
- Create: `yoresee_doc_sdk_node/esm/index.js`
- Create: `yoresee_doc_sdk_node/cjs/index.js`
- Create: `yoresee_doc_sdk_node/cjs/package.json`
- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_pb.js`
- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_connect.js`
- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_pb.js`
- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js`

**Interfaces:**
- ESM consumers import `@yoresee-doc/sdk-node/esm/yoresee_doc/v1/...`.
- CommonJS consumers require `@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/...`.

- [ ] **Step 1: Initialize the empty Node SDK repository on `master`**

Run from `my-repos/`:

```bash
cd yoresee_doc_sdk_node
git init -b master
git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```

If the repository already has a local Git initialization, keep it and only add the missing remote.

- [ ] **Step 2: Create the Node package contract**

Create `package.json` with this contract:

```json
{
  "name": "@yoresee-doc/sdk-node",
  "version": "0.1.0",
  "type": "module",
  "files": ["esm", "cjs", "README.md"],
  "exports": {
    ".": {
      "import": "./esm/index.js",
      "require": "./cjs/index.js"
    },
    "./esm/*": "./esm/*",
    "./cjs/*": "./cjs/*"
  },
  "dependencies": {
    "@bufbuild/protobuf": "^1.10.0",
    "@connectrpc/connect": "^1.7.0",
    "@grpc/grpc-js": "^1.12.0",
    "google-protobuf": "^4.0.2"
  }
}
```

Create `cjs/package.json` with:

```json
{
  "type": "commonjs"
}
```

Create `buf.gen.yaml` with:

```yaml
version: v2
plugins:
  - local: protoc-gen-es
    out: esm
    opt:
      - target=js
      - import_extension=.js
  - local: protoc-gen-connect-es
    out: esm
    opt:
      - target=js
      - import_extension=.js
```

- [ ] **Step 3: Generate the ESM/Connect outputs**

Run from `yoresee_doc_sdk_node/`:

```bash
buf generate ../yoresee_doc/proto --template buf.gen.yaml
```

Expected files:

```text
esm/yoresee_doc/v1/yoresee_doc_pb.js
esm/yoresee_doc/v1/yoresee_doc_connect.js
```

- [ ] **Step 4: Generate the CommonJS/gRPC outputs**

Run from `yoresee_doc_sdk_node/`:

```bash
npm install -g protoc-gen-js@3.21.4-4
mkdir -p cjs
grpc_tools_node_protoc -I ../yoresee_doc/proto \
  --plugin=protoc-gen-grpc="$(command -v grpc_tools_node_protoc_plugin)" \
  --plugin=protoc-gen-js="$(command -v protoc-gen-js)" \
  --grpc_out=grpc_js:cjs \
  --js_out=import_style=commonjs,binary:cjs \
  ../yoresee_doc/proto/yoresee_doc/v1/yoresee_doc.proto
```

Expected files:

```text
cjs/yoresee_doc/v1/yoresee_doc_pb.js
cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js
```

- [ ] **Step 5: Add package entry points and lock dependencies**

Create `esm/index.js`:

```js
export * from './yoresee_doc/v1/yoresee_doc_pb.js'
export * from './yoresee_doc/v1/yoresee_doc_connect.js'
```

Create `cjs/index.js`:

```js
module.exports = {
  messages: require('./yoresee_doc/v1/yoresee_doc_pb.js'),
  services: require('./yoresee_doc/v1/yoresee_doc_grpc_pb.js')
}
```

Run `npm install` in the SDK root to create `package-lock.json`.

- [ ] **Step 6: Validate both Node module formats**

Run from `yoresee_doc_sdk_node/`:

```bash
npm pack --dry-run
node --input-type=module -e "import('./esm/yoresee_doc/v1/yoresee_doc_pb.js').then(() => console.log('esm ok'))"
node -e "require('./cjs/yoresee_doc/v1/yoresee_doc_pb.js'); require('./cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js'); console.log('cjs ok')"
```

The ESM check must load the generated ESM file without a module-format error. The consumer package-path checks are performed after the service dependencies are installed in Tasks 5 and 6.

- [ ] **Step 7: Write and commit the Node SDK artifact**

Document the source proto commit, generator versions, ESM/Connect import path, CommonJS/gRPC import path, and Gitea tag in `README.md`. Then run:

```bash
git add README.md buf.gen.yaml package.json package-lock.json esm cjs
git diff --cached --check
git commit -m "feat: add generated Node protobuf SDK"
```

### Task 4: Publish Both SDK Releases

**Files:**
- Modify: Git history and refs of `yoresee_doc_sdk_go`
- Modify: Git history and refs of `yoresee_doc_sdk_node`

**Interfaces:**
- Produces `master` commits and immutable `v0.1.0` tags consumed by the services.

- [ ] **Step 1: Push each SDK commit to `master`**

Run separately in each SDK repository after checking the staged diff and clean working tree:

```bash
git push -u origin master
```

- [ ] **Step 2: Tag and push the initial release**

Run separately in each SDK repository:

```bash
git tag -a v0.1.0 -m "Release v0.1.0"
git push origin v0.1.0
```

- [ ] **Step 3: Verify remote refs**

Run from `my-repos/`:

```bash
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```

Expected: `master` and `v0.1.0` exist in both repositories.

### Task 5: Migrate Go Service Consumers

**Files:**
- Modify: all backend Go files importing `github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1`
- Modify: `backend/go.mod`, `backend/go.sum`
- Modify: all collab-go Go files importing `github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1`
- Modify: `collab-go/go.mod`, `collab-go/go.sum`
- Do not modify: `backend/AGENTS.md`, `backend/docs/`, Dockerfiles, or generation scripts

**Interfaces:**
- Both services import `yoreseedocpb` from `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`.

- [ ] **Step 1: Replace backend import paths**

In every backend source file returned by:

```bash
git grep -l 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go'
```

replace only the import path with:

```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1
```

Preserve existing aliases such as `pb`.

- [ ] **Step 2: Add the pinned Go SDK dependency to backend**

Add this direct requirement to `backend/go.mod`:

```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go v0.1.0
```

Run from `backend/`:

```bash
GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy
```

- [ ] **Step 3: Replace collab-go import paths**

In every collab-go source file returned by:

```bash
git grep -l 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go'
```

replace only the import path with:

```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1
```

Preserve the existing `yoreseedocpb` alias.

- [ ] **Step 4: Add the pinned Go SDK dependency to collab-go**

Add the same direct requirement to `collab-go/go.mod`, then run from `collab-go/`:

```bash
GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy
```

- [ ] **Step 5: Verify old paths are gone and Go services compile**

Run:

```bash
(cd backend && git grep 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go') || true
(cd collab-go && git grep 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go') || true
(cd backend && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...)
(cd collab-go && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...)
```

Expected: no old import matches and both modules compile against the remote SDK tag.

### Task 6: Migrate Node Service Consumers

**Files:**
- Modify: `frontend/package.json`, `frontend/package-lock.json`
- Modify: `frontend/src/services/grpc_client.js`
- Modify: `collab/package.json`, `collab/package-lock.json`
- Modify: `collab/src/grpc/client.js`
- Modify: `collab/src/grpc/document.js`
- Modify: `collab/src/grpc/system.js`
- Do not modify: Node Dockerfiles or generation scripts

**Interfaces:**
- Frontend uses the SDK ESM/Connect subpaths.
- Collab uses the SDK CommonJS/gRPC subpaths.

- [ ] **Step 1: Add the pinned Git dependency to frontend and collab**

Add this dependency to both `package.json` files:

```json
"@yoresee-doc/sdk-node": "git+ssh://git@git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_node.git#v0.1.0"
```

Run `npm install` separately in `frontend/` and `collab/` to update their lockfiles. The command requires SSH read access to Gitea.

- [ ] **Step 2: Replace frontend generated imports**

In `frontend/src/services/grpc_client.js`, replace:

```js
@/gen/yoresee_doc/v1/yoresee_doc_connect.js
@/gen/yoresee_doc/v1/yoresee_doc_pb.js
```

with:

```js
@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_connect.js
@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_pb.js
```

- [ ] **Step 3: Replace collab generated requires**

In the three `collab/src/grpc/*.js` files, replace the relative `../gen/yoresee_doc/v1/` prefix with:

```js
@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/
```

Keep the existing generated filenames and CommonJS `require` calls.

- [ ] **Step 4: Verify Node imports and builds**

Run:

```bash
(cd frontend && npm run build)
(cd collab && node -e "require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_pb'); require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb'); console.log('collab SDK imports ok')")
```

Expected: no service imports point to local generated directories, the frontend build succeeds, and collab loads both CJS modules.

### Task 7: Commit And Push Service Migrations

**Files:**
- Modify: `proto` repository history already prepared in Task 1
- Modify: `backend` repository history
- Modify: `collab-go` repository history
- Modify: `frontend` repository history
- Modify: `collab` repository history

- [ ] **Step 1: Review each component diff and exclude unrelated changes**

Before every commit, run in that component:

```bash
git status --short
git diff --check
git diff
```

In `backend`, do not stage the pre-existing untracked `AGENTS.md` or `docs/` files.

- [ ] **Step 2: Commit each service migration separately**

Use these commit messages and stage only the listed migration files:

```text
backend: feat: consume generated Go SDK
collab-go: feat: consume generated Go SDK
frontend: feat: consume generated Node SDK
collab: feat: consume generated Node SDK
```

- [ ] **Step 3: Push each service `master` branch**

Run from each service repository after its focused commit:

```bash
git push origin master
```

Do not update unrelated root gitlinks or modify deployment scripts in this task.

### Task 8: Final Cross-Repository Verification

**Files:**
- Verify: both SDK repositories, `proto`, `backend`, `collab-go`, `frontend`, and `collab`

- [ ] **Step 1: Verify all remotes and release refs**

Run:

```bash
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```

- [ ] **Step 2: Verify service dependency metadata**

Run:

```bash
(cd backend && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go')
(cd collab-go && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go')
(cd frontend && git grep '@yoresee-doc/sdk-node' -- package.json src)
(cd collab && git grep '@yoresee-doc/sdk-node' -- package.json src)
```

- [ ] **Step 3: Verify clean intended worktrees**

Run `git status --short` in each changed component and confirm only explicitly preserved unrelated files remain. Confirm no generated `node_modules`, secrets, or build output is staged.