Compare commits

..
3 Commits
Author SHA1 Message Date
XingfenD e71dfd0c0e docs: record Node codegen prerequisite 2026-08-11 19:18:55 +08:00
XingfenD 01ab8399bb docs: add SDK migration implementation plan 2026-08-11 19:15:00 +08:00
XingfenD db6ba9c8d0 docs: record SDK migration design 2026-08-11 19:09:39 +08:00
2 changed files with 723 additions and 0 deletions
@@ -0,0 +1,585 @@
# Yoresee Doc SDK Migration Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Publish versioned Go and Node protobuf SDK repositories on Gitea and migrate backend, collab-go, frontend, and collab to consume them.
**Architecture:** `proto` remains the only source of the protobuf contract. The two SDK repositories commit generated artifacts: Go stubs in the Go module root, and both ESM/Connect and CommonJS/gRPC outputs in separate Node subdirectories. Services consume tagged Gitea dependencies; existing local generation scripts and Dockerfiles remain unchanged.
**Tech Stack:** Protobuf, Buf 1.72.0, `protoc-gen-go` 1.36.12, `protoc-gen-go-grpc` 1.6.2, `protoc-gen-es`, `protoc-gen-connect-es`, `grpc-tools` 1.13.1, `protoc-gen-js` 3.21.4-4, Go modules, npm Git dependencies, Gitea Git over SSH.
## Global Constraints
- The protobuf source remains `proto/yoresee_doc/v1/yoresee_doc.proto`.
- The Go module path is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go`.
- The Node package name is `@yoresee-doc/sdk-node`.
- The initial SDK release is `v0.1.0`; consumers pin that tag.
- The protobuf `go_package` is `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb`.
- Do not modify `deploy/script/gen_proto.sh`, Dockerfiles, Docker Compose, `clone_all.sh`, or CI/CD configuration.
- Do not change protobuf messages, fields, services, or runtime behavior.
- Preserve unrelated pre-existing changes, including untracked files under `backend/`.
- Private Gitea module/package reads require working SSH access; Go consumers use `GOPRIVATE=git.yoresee.cc/YoreseeDoc`.
---
### Task 1: Point The Proto Contract At The Go SDK
**Files:**
- Modify: `proto/yoresee_doc/v1/yoresee_doc.proto:5`
- Test: proto diff and source-format checks
**Interfaces:**
- Produces the canonical Go import path used by generated SDK code and both Go services.
- [ ] **Step 1: Replace the Go package option**
Change the only `go_package` option to:
```protobuf
option go_package = "git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb";
```
Do not alter any other proto line.
- [ ] **Step 2: Verify the focused proto diff**
Run from `proto/`:
```bash
git diff --check
git diff -- yoresee_doc/v1/yoresee_doc.proto
```
Expected: one changed `go_package` line and no whitespace errors.
- [ ] **Step 3: Commit and push the proto source change**
Run from `proto/`, staging only the proto file:
```bash
git add yoresee_doc/v1/yoresee_doc.proto
git diff --cached --check
git commit -m "feat: point Go protobuf package at SDK"
git push origin master
```
Record the resulting source commit with:
```bash
git rev-parse HEAD
```
The commit hash is recorded in both SDK READMEs.
### Task 2: Generate And Validate The Go SDK
**Files:**
- Create: `yoresee_doc_sdk_go/go.mod`
- Create: `yoresee_doc_sdk_go/go.sum`
- Create: `yoresee_doc_sdk_go/buf.gen.yaml`
- Create: `yoresee_doc_sdk_go/README.md`
- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc.pb.go`
- Create: `yoresee_doc_sdk_go/yoresee_doc/v1/yoresee_doc_grpc.pb.go`
**Interfaces:**
- Consumes the proto directory from `yoresee_doc/proto`.
- Produces Go package `yoreseedocpb` at import path `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`.
- [ ] **Step 1: Initialize the empty SDK repository on `master`**
Run from `my-repos/`:
```bash
cd yoresee_doc_sdk_go
git init -b master
git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
```
If the repository already has a local Git initialization, keep it and only add the missing remote.
- [ ] **Step 2: Create the Go module metadata**
Create `go.mod` with:
```go
module git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go
go 1.24
require (
google.golang.org/grpc v1.79.2
google.golang.org/protobuf v1.36.10
)
```
Create `buf.gen.yaml` with:
```yaml
version: v2
plugins:
- local: protoc-gen-go
out: .
opt:
- paths=source_relative
- local: protoc-gen-go-grpc
out: .
opt:
- paths=source_relative
```
- [ ] **Step 3: Generate both Go protobuf outputs**
Run from `yoresee_doc_sdk_go/`:
```bash
export PATH="$HOME/go/bin:$PATH"
buf generate ../yoresee_doc/proto --template buf.gen.yaml
```
Expected files:
```text
yoresee_doc/v1/yoresee_doc.pb.go
yoresee_doc/v1/yoresee_doc_grpc.pb.go
```
- [ ] **Step 4: Resolve and test the Go module**
Run from `yoresee_doc_sdk_go/`:
```bash
go mod tidy
go test ./...
go vet ./...
```
Expected: the generated package compiles with no test or vet failures.
- [ ] **Step 5: Write the Go SDK README**
Document the module path, generated package import example, source proto path, recorded proto commit hash from Task 1, generator versions, and the fact that generated artifacts are committed.
- [ ] **Step 6: Commit the Go SDK artifact**
Run from `yoresee_doc_sdk_go/`, staging only SDK files:
```bash
git add README.md buf.gen.yaml go.mod go.sum yoresee_doc
git diff --cached --check
git commit -m "feat: add generated Go protobuf SDK"
```
### Task 3: Generate And Validate The Node SDK
**Files:**
- Create: `yoresee_doc_sdk_node/package.json`
- Create: `yoresee_doc_sdk_node/package-lock.json`
- Create: `yoresee_doc_sdk_node/buf.gen.yaml`
- Create: `yoresee_doc_sdk_node/README.md`
- Create: `yoresee_doc_sdk_node/esm/index.js`
- Create: `yoresee_doc_sdk_node/cjs/index.js`
- Create: `yoresee_doc_sdk_node/cjs/package.json`
- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_pb.js`
- Create: `yoresee_doc_sdk_node/esm/yoresee_doc/v1/yoresee_doc_connect.js`
- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_pb.js`
- Create: `yoresee_doc_sdk_node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js`
**Interfaces:**
- ESM consumers import `@yoresee-doc/sdk-node/esm/yoresee_doc/v1/...`.
- CommonJS consumers require `@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/...`.
- [ ] **Step 1: Initialize the empty Node SDK repository on `master`**
Run from `my-repos/`:
```bash
cd yoresee_doc_sdk_node
git init -b master
git remote add origin git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```
If the repository already has a local Git initialization, keep it and only add the missing remote.
- [ ] **Step 2: Create the Node package contract**
Create `package.json` with this contract:
```json
{
"name": "@yoresee-doc/sdk-node",
"version": "0.1.0",
"type": "module",
"files": ["esm", "cjs", "README.md"],
"exports": {
".": {
"import": "./esm/index.js",
"require": "./cjs/index.js"
},
"./esm/*": "./esm/*",
"./cjs/*": "./cjs/*"
},
"dependencies": {
"@bufbuild/protobuf": "^1.10.0",
"@connectrpc/connect": "^1.7.0",
"@grpc/grpc-js": "^1.12.0",
"google-protobuf": "^4.0.2"
}
}
```
Create `cjs/package.json` with:
```json
{
"type": "commonjs"
}
```
Create `buf.gen.yaml` with:
```yaml
version: v2
plugins:
- local: protoc-gen-es
out: esm
opt:
- target=js
- import_extension=.js
- local: protoc-gen-connect-es
out: esm
opt:
- target=js
- import_extension=.js
```
- [ ] **Step 3: Generate the ESM/Connect outputs**
Run from `yoresee_doc_sdk_node/`:
```bash
buf generate ../yoresee_doc/proto --template buf.gen.yaml
```
Expected files:
```text
esm/yoresee_doc/v1/yoresee_doc_pb.js
esm/yoresee_doc/v1/yoresee_doc_connect.js
```
- [ ] **Step 4: Generate the CommonJS/gRPC outputs**
Run from `yoresee_doc_sdk_node/`:
```bash
npm install -g protoc-gen-js@3.21.4-4
mkdir -p cjs
grpc_tools_node_protoc -I ../yoresee_doc/proto \
--plugin=protoc-gen-grpc="$(command -v grpc_tools_node_protoc_plugin)" \
--plugin=protoc-gen-js="$(command -v protoc-gen-js)" \
--grpc_out=grpc_js:cjs \
--js_out=import_style=commonjs,binary:cjs \
../yoresee_doc/proto/yoresee_doc/v1/yoresee_doc.proto
```
Expected files:
```text
cjs/yoresee_doc/v1/yoresee_doc_pb.js
cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js
```
- [ ] **Step 5: Add package entry points and lock dependencies**
Create `esm/index.js`:
```js
export * from './yoresee_doc/v1/yoresee_doc_pb.js'
export * from './yoresee_doc/v1/yoresee_doc_connect.js'
```
Create `cjs/index.js`:
```js
module.exports = {
messages: require('./yoresee_doc/v1/yoresee_doc_pb.js'),
services: require('./yoresee_doc/v1/yoresee_doc_grpc_pb.js')
}
```
Run `npm install` in the SDK root to create `package-lock.json`.
- [ ] **Step 6: Validate both Node module formats**
Run from `yoresee_doc_sdk_node/`:
```bash
npm pack --dry-run
node --input-type=module -e "import('./esm/yoresee_doc/v1/yoresee_doc_pb.js').then(() => console.log('esm ok'))"
node -e "require('./cjs/yoresee_doc/v1/yoresee_doc_pb.js'); require('./cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js'); console.log('cjs ok')"
```
The ESM check must load the generated ESM file without a module-format error. The consumer package-path checks are performed after the service dependencies are installed in Tasks 5 and 6.
- [ ] **Step 7: Write and commit the Node SDK artifact**
Document the source proto commit, generator versions, ESM/Connect import path, CommonJS/gRPC import path, and Gitea tag in `README.md`. Then run:
```bash
git add README.md buf.gen.yaml package.json package-lock.json esm cjs
git diff --cached --check
git commit -m "feat: add generated Node protobuf SDK"
```
### Task 4: Publish Both SDK Releases
**Files:**
- Modify: Git history and refs of `yoresee_doc_sdk_go`
- Modify: Git history and refs of `yoresee_doc_sdk_node`
**Interfaces:**
- Produces `master` commits and immutable `v0.1.0` tags consumed by the services.
- [ ] **Step 1: Push each SDK commit to `master`**
Run separately in each SDK repository after checking the staged diff and clean working tree:
```bash
git push -u origin master
```
- [ ] **Step 2: Tag and push the initial release**
Run separately in each SDK repository:
```bash
git tag -a v0.1.0 -m "Release v0.1.0"
git push origin v0.1.0
```
- [ ] **Step 3: Verify remote refs**
Run from `my-repos/`:
```bash
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```
Expected: `master` and `v0.1.0` exist in both repositories.
### Task 5: Migrate Go Service Consumers
**Files:**
- Modify: all backend Go files importing `github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1`
- Modify: `backend/go.mod`, `backend/go.sum`
- Modify: all collab-go Go files importing `github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1`
- Modify: `collab-go/go.mod`, `collab-go/go.sum`
- Do not modify: `backend/AGENTS.md`, `backend/docs/`, Dockerfiles, or generation scripts
**Interfaces:**
- Both services import `yoreseedocpb` from `git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1`.
- [ ] **Step 1: Replace backend import paths**
In every backend source file returned by:
```bash
git grep -l 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go'
```
replace only the import path with:
```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1
```
Preserve existing aliases such as `pb`.
- [ ] **Step 2: Add the pinned Go SDK dependency to backend**
Add this direct requirement to `backend/go.mod`:
```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go v0.1.0
```
Run from `backend/`:
```bash
GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy
```
- [ ] **Step 3: Replace collab-go import paths**
In every collab-go source file returned by:
```bash
git grep -l 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go'
```
replace only the import path with:
```go
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1
```
Preserve the existing `yoreseedocpb` alias.
- [ ] **Step 4: Add the pinned Go SDK dependency to collab-go**
Add the same direct requirement to `collab-go/go.mod`, then run from `collab-go/`:
```bash
GOPRIVATE=git.yoresee.cc/YoreseeDoc go mod tidy
```
- [ ] **Step 5: Verify old paths are gone and Go services compile**
Run:
```bash
(cd backend && git grep 'github.com/XingfenD/yoresee_doc/pkg/gen/yoresee_doc/v1' -- '*.go') || true
(cd collab-go && git grep 'github.com/XingfenD/yoresee_doc/collab-go/pkg/gen/yoresee_doc/v1' -- '*.go') || true
(cd backend && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...)
(cd collab-go && GOPRIVATE=git.yoresee.cc/YoreseeDoc go test ./...)
```
Expected: no old import matches and both modules compile against the remote SDK tag.
### Task 6: Migrate Node Service Consumers
**Files:**
- Modify: `frontend/package.json`, `frontend/package-lock.json`
- Modify: `frontend/src/services/grpc_client.js`
- Modify: `collab/package.json`, `collab/package-lock.json`
- Modify: `collab/src/grpc/client.js`
- Modify: `collab/src/grpc/document.js`
- Modify: `collab/src/grpc/system.js`
- Do not modify: Node Dockerfiles or generation scripts
**Interfaces:**
- Frontend uses the SDK ESM/Connect subpaths.
- Collab uses the SDK CommonJS/gRPC subpaths.
- [ ] **Step 1: Add the pinned Git dependency to frontend and collab**
Add this dependency to both `package.json` files:
```json
"@yoresee-doc/sdk-node": "git+ssh://git@git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_node.git#v0.1.0"
```
Run `npm install` separately in `frontend/` and `collab/` to update their lockfiles. The command requires SSH read access to Gitea.
- [ ] **Step 2: Replace frontend generated imports**
In `frontend/src/services/grpc_client.js`, replace:
```js
@/gen/yoresee_doc/v1/yoresee_doc_connect.js
@/gen/yoresee_doc/v1/yoresee_doc_pb.js
```
with:
```js
@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_connect.js
@yoresee-doc/sdk-node/esm/yoresee_doc/v1/yoresee_doc_pb.js
```
- [ ] **Step 3: Replace collab generated requires**
In the three `collab/src/grpc/*.js` files, replace the relative `../gen/yoresee_doc/v1/` prefix with:
```js
@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/
```
Keep the existing generated filenames and CommonJS `require` calls.
- [ ] **Step 4: Verify Node imports and builds**
Run:
```bash
(cd frontend && npm run build)
(cd collab && node -e "require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_pb'); require('@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/yoresee_doc_grpc_pb'); console.log('collab SDK imports ok')")
```
Expected: no service imports point to local generated directories, the frontend build succeeds, and collab loads both CJS modules.
### Task 7: Commit And Push Service Migrations
**Files:**
- Modify: `proto` repository history already prepared in Task 1
- Modify: `backend` repository history
- Modify: `collab-go` repository history
- Modify: `frontend` repository history
- Modify: `collab` repository history
- [ ] **Step 1: Review each component diff and exclude unrelated changes**
Before every commit, run in that component:
```bash
git status --short
git diff --check
git diff
```
In `backend`, do not stage the pre-existing untracked `AGENTS.md` or `docs/` files.
- [ ] **Step 2: Commit each service migration separately**
Use these commit messages and stage only the listed migration files:
```text
backend: feat: consume generated Go SDK
collab-go: feat: consume generated Go SDK
frontend: feat: consume generated Node SDK
collab: feat: consume generated Node SDK
```
- [ ] **Step 3: Push each service `master` branch**
Run from each service repository after its focused commit:
```bash
git push origin master
```
Do not update unrelated root gitlinks or modify deployment scripts in this task.
### Task 8: Final Cross-Repository Verification
**Files:**
- Verify: both SDK repositories, `proto`, `backend`, `collab-go`, `frontend`, and `collab`
- [ ] **Step 1: Verify all remotes and release refs**
Run:
```bash
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_go.git
git ls-remote --heads git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
git ls-remote --tags git@git.yoresee.cc:YoreseeDoc/yoresee_doc_sdk_node.git
```
- [ ] **Step 2: Verify service dependency metadata**
Run:
```bash
(cd backend && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go')
(cd collab-go && git grep 'git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go')
(cd frontend && git grep '@yoresee-doc/sdk-node' -- package.json src)
(cd collab && git grep '@yoresee-doc/sdk-node' -- package.json src)
```
- [ ] **Step 3: Verify clean intended worktrees**
Run `git status --short` in each changed component and confirm only explicitly preserved unrelated files remain. Confirm no generated `node_modules`, secrets, or build output is staged.
@@ -0,0 +1,138 @@
# Yoresee Doc SDK Migration Design
## Goal
Publish the protobuf-generated dependencies as two independent repositories under
the `YoreseeDoc` Gitea organization and migrate the four services to consume
those repositories:
- `yoresee_doc_sdk_go` for Go protobuf and gRPC stubs.
- `yoresee_doc_sdk_node` for browser ESM/Connect RPC stubs and Node.js CommonJS
gRPC stubs.
The existing protobuf repository remains the single source of the `.proto`
contract.
## Scope
This is a dependency migration, not a code-generation or CI/CD redesign.
Included:
- Generate and commit Go, ESM, and CommonJS artifacts in the two SDK repos.
- Use the Gitea repositories as versioned service dependencies.
- Change service imports to the SDK package paths.
- Change the protobuf `go_package` to the canonical Go SDK module path.
- Add package documentation describing the generated outputs and source proto.
Explicitly unchanged:
- `deploy/script/gen_proto.sh`.
- Existing Dockerfiles and Docker Compose files.
- Existing local generated-code directories and their fallback generation.
- CI/CD workflow and registry setup.
The existing local generation may continue to produce unused compatibility
artifacts during builds. It is intentionally left in place for this migration.
## Public Package Contracts
### Go
The Go SDK module path is:
```text
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go
```
Generated files live at:
```text
yoresee_doc/v1/yoresee_doc.pb.go
yoresee_doc/v1/yoresee_doc_grpc.pb.go
```
The protobuf `go_package` is changed to:
```text
git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_go/yoresee_doc/v1;yoreseedocpb
```
The initial SDK release is `v0.1.0`. `backend` and `collab-go` pin this
version instead of depending on the moving `master` branch.
### Node.js
The package name is `@yoresee-doc/sdk-node`, installed from:
```text
git+ssh://git@git.yoresee.cc/YoreseeDoc/yoresee_doc_sdk_node.git#v0.1.0
```
Generated output is split by runtime:
```text
esm/yoresee_doc/v1/yoresee_doc_pb.js
esm/yoresee_doc/v1/yoresee_doc_connect.js
cjs/yoresee_doc/v1/yoresee_doc_pb.js
cjs/yoresee_doc/v1/yoresee_doc_grpc_pb.js
```
The package uses conditional exports and explicit `esm/*` and `cjs/*`
subpaths. The `cjs/` directory declares CommonJS module semantics so the
root package can remain ESM-compatible.
Service import contracts are:
- `frontend`: `@yoresee-doc/sdk-node/esm/yoresee_doc/v1/...`
- `collab`: `@yoresee-doc/sdk-node/cjs/yoresee_doc/v1/...`
Runtime dependencies for both generated formats are declared by the SDK:
`@bufbuild/protobuf`, `@connectrpc/connect`, `@grpc/grpc-js`, and
`google-protobuf`.
## Generation and Repository Boundaries
The generated files are committed to the SDK repositories because they are the
published dependency artifacts. The source proto remains in the existing
`proto` repository and is not duplicated into the SDK repositories.
Generation for this initial migration is performed from the checked-out proto
source using the existing installed toolchains. The migration does not alter
the existing project generation scripts. Each SDK README records the source
proto commit and the generator versions used for the artifacts.
## Service Migration
`backend` and `collab-go` replace local generated-package imports with the Go
SDK import path and add the pinned Gitea module dependency. The services set
`GOPRIVATE` in their build environment documentation where required by the
private module.
`frontend` and `collab` add the pinned Git dependency and replace aliases or
relative paths that point to local generated files with the Node SDK exports.
No service behavior or protobuf message/service definitions change.
## Verification
The implementation is accepted only after all of the following checks pass:
- Go SDK generated package compiles and its module resolves at `v0.1.0`.
- Node SDK can be installed from the Gitea tag and both module formats load.
- `backend` and `collab-go` compile with the remote SDK dependency.
- `frontend` production build resolves the ESM/Connect exports.
- `collab` can load the CJS protobuf and gRPC exports.
- SDK repositories have clean commits pushed to `master`, with `v0.1.0`
tags.
## Risks and Rollback
- Private Gitea dependencies require read access from developer and Docker/CI
environments. A failed authenticated fetch is an environment issue, not a
protobuf generation issue.
- The old local generation remains redundant and may continue to fail because
its existing buf templates are outside this migration scope.
- Rollback consists of reverting service dependency/import changes and the
protobuf `go_package` change; SDK repositories remain harmless additive
artifacts.