docs(readme): rewrite run/dev/migrate instructions for dev/release deploy spec

- scripts/smoke.sh: idempotent re-run (reuse existing member/library on 409)
- web client.ts: probe localStorage methods, not typeof (node 22+ stub regression)
This commit is contained in:
2026-09-07 21:09:11 +08:00
parent 0de824cc6a
commit 95317a0108
4 changed files with 189 additions and 38 deletions
+125
View File
@@ -0,0 +1,125 @@
# AGENTS.md
@github.com/XingfenD/AGENTS.md:content/basic_agents_md/AGENTS.md
## Docker deployment (dev & release)
### Deployment mode
Mode differences (dev = `deploy/docker-compose.dev.yml`, release = `deploy/docker-compose.yml`):
- dev: source dirs bind-mounted into containers (`../backend:/app`, `../frontend:/app`, `../collab*`), `target: dev` images, workers run `go mod download && go run ./cmd/...`, hot reload without rebuild; also exposes infra ports to the host (postgres 5432, redis 6379, ES 9200, rabbit 5672) and a delve debugger on 2345, and gates startup on healthchecks.
- release: multi-stage `*.prod` Dockerfiles (`target: runner`) with compiled binaries, no source mounts — Go services only get config files mounted; infra ports stay on the internal network only.
### Volume mounts
- Named volumes (both modes, shared): `{$project_name}_{postgres,redis,consul,rabbitmq,minio,elasticsearch}_data` persist infra state to Docker volumes — identical names across the two compose files, so dev and release see the same data. Only `down -v` (i.e. `rebuild`/`clear`) wipes them.
- Config/log bind mounts (both modes): rendered outputs `deploy/nginx/nginx.conf`, `deploy/nginx/conf.d/default.conf`, `deploy/redis/redis.conf`, `deploy/rabbitmq/rabbitmq.conf` (`:ro`) plus `deploy/logs/{nginx,rabbitmq}` are mounted from the host — these come from `prepare.sh`, so stale files in the container mean you forgot to rerun it.
- Source mounts (dev only): `../backend:/app` (which run `go mod download && go run ./cmd/...`), `../frontend:/app` with an anonymous volume on `/app/node_modules` (so the image's deps survive the mount — delete containers with `down`, not `rm`, to avoid orphaning it).
- The File Storage mounts: if the backend stores uploaded files directly instead of in an object storage, mount `deploy/{$service_name}/storage` as the storage path into the container.
## Project structure
Basically, the project consists of the directories `docs`, `backend`, `frontend`, `deploy`, plus optional `collab*` services:
```plaintext
.
├── docs # project documentation (see Documents layout)
├── backend # golang services (see backend layout)
├── frontend # node web app, built and served via nginx (see frontend layout)
├── collab* # optional collaboration services, same layout as backend
├── deploy # compose stacks, rendered configs, host-side logs (see deploy layout)
├── .gitignore # repo-wide ignores only; module-specific ignores live inside each module
├── AGENTS.md # agent guidance for this repo
└── LICENSE
```
### Documents layout
The documents' layout is expected to be like below:
```plaintext
docs
├── CHANGELOG.md # general (non-WebUI) changes; highest version on top
├── CHANGELOG_web.md # WebUI changes only; highest version on top
├── README.md # primary (English) entry doc
└── README_zh.md # Chinese mirror of README.md — keep in sync
```
Constraints:
- Every user-visible change gets a changelog entry: WebUI changes → `CHANGELOG_web.md`, everything else → `CHANGELOG.md`; never duplicate an entry across both.
- `README.md` and `README_zh.md` stay content-equivalent; update them in the same change.
- Documentation lives only under `docs/` — no stray `*.md` at the repo root besides `AGENTS.md` and `LICENSE`.
### backend (golang)
```plaintext
backend
├── cmd # executable entry points, one dir per binary
├── internal # private packages: importable only within this module
├── pkg # public packages: intentionally shared with other repos
└── .gitignore # .gitignore inside backend
```
Constraints:
- `cmd`: each subdirectory holds exactly one `main` package; `main.go` only bootstraps — load config, wire dependencies, start the server/CLI, handle graceful shutdown. No business logic here, and nothing outside `cmd` imports `cmd`.
- `internal`: default home for all business code (domain, services, storage, config). If code is not meant to be imported by other repos, it goes here, not in `pkg`.
- `pkg`: opt-in public API surface — keep it small and stable, and never leak `internal` types through its APIs.
```plaintext
cmd
├── cli # command-line binary
│ ├── commands # one file/package per subcommand: flag parsing + dispatch into internal
│ └── main.go # arg parsing and subcommand dispatch only
└── webui # HTTP server binary
├── api # route registration + request/response DTOs — the endpoint contract, no logic
├── constant # webui-only constants (routes, keys, limits)
├── handlers # thin HTTP handlers: bind/validate input, call internal services, map errors
├── main.go # boot the HTTP server
├── static # assets served as-is by webui
└── templates # server-side HTML templates rendered by webui
```
Constraints:
- Handlers never touch the database or implement domain rules — they delegate to `internal`; SQL and business rules live behind `internal` boundaries.
- `api` is the single source of truth for the endpoint contract; the frontend aligns with it.
- Code shared between `cli` and `webui` belongs in `internal` (or `pkg`), never imported from one command by the other.
### frontend (node)
```plaintext
frontend
├── src # all hand-written app source: pages, components, state, API client
├── public # static assets copied into the build output as-is
├── package.json # dev/build/lint/test scripts, runnable unchanged inside the container
└── .gitignore # node_modules/ and build output (e.g. dist/) never committed
```
Constraints (framework-agnostic — any stack that keeps the layout above):
- The app talks to the backend only over the HTTP APIs defined in `backend/cmd/webui/api` — no direct access to infra (DB, ES, rabbit) from the frontend.
- Build output is disposable and git-ignored; committed sources live only in `src`/`public`; nginx serves the built assets in release.
- Dev runs from the source mount (`../frontend:/app`) with `node_modules` provided by the image's anonymous volume — install new deps inside the container and commit the lockfile.
### deploy
```plaintext
deploy
├── docker-compose.yml # release stack: multi-stage *.prod images, infra internal-only
├── docker-compose.dev.yml # dev stack: source mounts, target: dev, exposed ports, delve 2345
├── prepare.sh # renders the config files below; rerun after changing templates
├── nginx
│ ├── nginx.conf # main config, mounted :ro
│ └── conf.d/default.conf # site config, mounted :ro
├── redis/redis.conf # mounted :ro
├── rabbitmq/rabbitmq.conf # mounted :ro
├── logs # host-side dirs bind-mounted into containers ({nginx,rabbitmq})
├── {$service_name}/storage # file-storage path for services not using object storage
└── .gitignore # ignores actual/rendered configs and logs/; tracks only *.example files
```
Constraints:
- Only config examples (e.g. `*.conf.example`) are tracked by git; actual configs (rendered by `prepare.sh`) and log files are git-ignored.
- Everything under `deploy/` that containers mount (`*.conf` and `prepare.sh` outputs) is config, not app code: edit it here, rerun `prepare.sh`, restart — never edit configs inside a running container.
- Infra ports are exposed to the host only in `docker-compose.dev.yml`; release keeps them internal-network-only.
- Persisted state lives only in the named `{$project_name}_*` volumes; bind-mounts are reserved for source, config, logs, and file storage.
- Service-specific host dirs (storage, config) go under `deploy/{$service_name}/`, never loose at the repo root.
+56 -34
View File
@@ -1,53 +1,75 @@
# Book & Comic Library # Book & Comic Library
个人书库/漫画库:Go+Gin 后端(扫描/上传入库、多用户 JWT、阅读进度、磁盘+Redis 缓存)+ Docker Compose 部署。设计见 `docs/superpowers/specs/2026-09-04-book-comic-library-design.md`。 个人书库/漫画库:Go+Gin 后端(扫描/上传入库、多用户 JWT、阅读进度、磁盘+Redis 缓存)+ Docker Compose 部署。设计见 `docs/superpowers/specs/2026-09-04-book-comic-library-design.md`;部署规范见 `docs/superpowers/specs/2026-09-07-docker-deploy-spec-design.md`。
## Deploy mode
- release:`deploy/docker-compose.yml` — 多阶段 `backend|web/Dockerfile.prod`(target runner)编译产物,不挂源码;infra 端口只在容器网络。
- dev:`deploy/docker-compose.dev.yml` — 四服务全容器化,源码挂 `../backend:/app`、`../web:/app`(web 带匿名 `node_modules` 卷),`target: dev` 镜像;api 跑 `go mod download && dlv debug ./cmd/server`(热重启不 rebuild,delve :2345);infra 暴露宿主 PG 5432 / Redis 6379;healthcheck 门控。
## Volume Mount
- 共享 named volumes:`booklib_postgres_data`、`booklib_redis_data`(两个 compose 同名,dev/release 看到同一份数据;仅 `down -v` 清除)。
- 配置/日志绑定挂载:`deploy/nginx/{nginx.conf,conf.d/default.conf}`、`deploy/redis/redis.conf`(`:ro`)与 `deploy/logs/nginx` —— 全部来自 `deploy/prepare.sh`(模板在各产物同目录的 `*.tpl`),**容器里配置不对/缺失 = 忘了重跑它**。
- 文件存储:`deploy/api/storage` → 容器 `/data/books`(缓存写 `/data/books/cache`)。
## 跑起来(生产形态) ## 跑起来(生产形态)
```bash ```bash
cp .env.example .env # 填 JWT_SECRET、ADMIN_USER、ADMIN_PASSWORD(≥8 位,低于 8 位 seed 会跳过并 log) cp deploy/.env.example deploy/.env # 填 JWT_SECRET、ADMIN_USER、ADMIN_PASSWORD(≥8 位,低于 8 位 seed 会跳过并 log)
docker compose up -d --build deploy/prepare.sh
./scripts/smoke.sh # 端到端验收(登录、建库、上传、扫描、进度、删除、immutable 头) docker compose -f deploy/docker-compose.yml up -d --build
bash scripts/smoke.sh && bash scripts/smoke-web.sh
``` ```
- web: `http://localhost:8080`,API 走 nginx `/api/` 前缀反代到无状态 api 副本(`--scale api=N`)。 - web: `http://localhost:8080`(`WEB_PORT` 可改),API 走 nginx `/api/` 前缀反代到无状态 api 副本(`--scale api=N`)。
- 原始书放在 `./library/`(挂到 `/data/books`),scanner 周期入库(默认 60s)。 - 原始书放进 `deploy/api/storage/`(挂到 `/data/books`),scanner 周期入库(默认 60s)。
- nginx access/error 日志:`deploy/logs/nginx/`。
## 开发
```bash
deploy/prepare.sh
docker compose -f deploy/docker-compose.dev.yml up -d --build
```
- 前端: http://localhost:5173(vite,HMR 直接生效;`/api` 代理到容器内 api)。
- Go 改码后:`docker compose -f deploy/docker-compose.dev.yml restart api`(重编译挂载源码,无需 rebuild)。
- 断点调试:delve headless 在 `localhost:2345`(VSCode launch:`{"type":"go","request":"attach","mode":"remote","host":"localhost","port":2345,"substitutePath":[{"from":"${workspaceFolder}/backend","to":"/app"}]}`;命中断点后用 dlv 命令继续)。
- 直连基础设施跑测试:PG `localhost:5432`(lib/lib/lib)、Redis `localhost:6379`:
```bash
cd backend
export DATABASE_URL='postgres://lib:lib@localhost:5432/lib?sslmode=disable'
export REDIS_URL='redis://localhost:6379'
go vet ./... && gofmt -l .
go test -p 1 -count=1 ./...
```
`-p 1` 是必须的:集成测试共用同一个 PG 库,各自 `DELETE FROM ...` 清表——并行跑会互相删数据导致随机失败。dev 栈起停用 `down`(不是 `rm`),否则匿名 node_modules 卷成孤儿。无 PG/Redis 时依赖它们的测试自动 skip;Redis 挂掉不影响功能(全链路降级为 miss/放行,见 spec §9)。
前端门槛:`cd web && npm run check`(tsc + vitest + vite build)。
## 旧卷迁移(一次性,升级自上一版部署)
```bash
# 老 PG 数据 → 新共享卷
docker run --rm -v book-comic-library_pgdata:/from -v booklib_postgres_data:/to alpine cp -a /from/. /to/
# 老 cache 卷是封面/解压派生数据,直接丢弃(自动重建)
docker volume rm book-comic-library_pgdata book-comic-library_cache
```
书库文件:原宿主 `./library/` 的内容移入 `deploy/api/storage/`。
## 可信代理与限流 ## 可信代理与限流
- nginx 在 compose 网络内,api 的 `ClientIP` 只信 `TRUSTED_PROXY_CIDRS`(逗号分隔 CIDR,默认 `172.16.0.0/12`,即 compose 网段)。外部伪造 `X-Forwarded-For` 换不掉限流桶;换部署网络时改这个 env。 - nginx 在 compose 网络内,api 的 `ClientIP` 只信 `TRUSTED_PROXY_CIDRS`(逗号分隔 CIDR,默认 `172.16.0.0/12`,即 compose 网段)。外部伪造 `X-Forwarded-For` 换不掉限流桶;换部署网络时改这个 env。
- 登录限流 5 次/分钟/IP **按尝试计数,成功登录也计**——爆破和正常高频登录同账。 - 登录限流 5 次/分钟/IP **按尝试计数,成功登录也计**——爆破和正常高频登录同账。
## 开发 / 测试
```bash
docker compose -f deploy/docker-compose.dev.yml up -d # PG :5433, Redis :6380(避开本机默认端口)
cd backend
export DATABASE_URL='postgres://lib:lib@localhost:5433/lib?sslmode=disable'
export REDIS_URL='redis://localhost:6380'
go vet ./... && gofmt -l .
go test -p 1 -count=1 ./...
```
`-p 1` 是必须的:集成测试共用同一个 PG 库,各自 `DELETE FROM ...` 清表——并行跑会互相删数据导致随机失败。
无 PG/Redis 时依赖它们的测试自动 skip;Redis 挂掉不影响功能(全链路降级为 miss/放行,见 spec §9)。
## 改 schema 前必读 ## 改 schema 前必读
`db.Migrate` 只执行 `schema.sql` 的 `CREATE TABLE IF NOT EXISTS`——对已存在的库**加列/改列不会生效**。任何列变更之前,必须先引入 `schema_migrations` 版本表 + 有序迁移脚本,否则老部署会静默跑在旧结构上。 `db.Migrate` 只执行 `schema.sql` 的 `CREATE TABLE IF NOT EXISTS`——对已存在的库**加列/改列不会生效**。任何列变更之前,必须先引入 `schema_migrations` 版本表 + 有序迁移脚本,否则老部署会静默跑在旧结构上。
## 前端开发(web/) ## PWA
- `cd web && npm install`;`npm run dev`(:5173,`/api` 代理到 :8080)。 不可变资源(封面/CBZ 页/原文件)SW cache-first,读过的内容离线可翻;登出会清 SW 缓存。
- 后端起法:`docker compose -f deploy/docker-compose.dev.yml up -d --wait` 起 PG(:5433)/Redis(:6380),
然后 `cd backend && DATABASE_URL=postgres://lib:lib@localhost:5433/lib?sslmode=disable \
REDIS_URL=redis://localhost:6380 JWT_SECRET=dev go run ./cmd/server`。
- 门槛:`npm run check`(tsc + vitest + vite build)。
## 生产部署(含前端)
- `.env` 配 `JWT_SECRET/ADMIN_USER/ADMIN_PASSWORD` 后 `docker compose up -d --build`;
打开 http://localhost:8080(web=SPA+nginx,/api 反代 api:8080)。
- PWA:不可变资源(封面/CBZ 页/原文件)SW cache-first,读过的内容离线可翻;登出会清 SW 缓存。
- 冒烟:`bash scripts/smoke.sh`(后端直连)、`bash scripts/smoke-web.sh`(经 nginx 全栈,需 :8080 空闲)。
+5 -2
View File
@@ -21,14 +21,17 @@ TOK=$(tokfor "$ADMIN_USER" "$ADMIN_PASSWORD")
AUTH="authorization: Bearer $TOK" AUTH="authorization: Bearer $TOK"
say "member user + role enforcement" say "member user + role enforcement"
curl -fsS "$API/users" "${J[@]}" -H "$AUTH" -d '{"username":"smoke","password":"smokepw123","role":"member"}' >/dev/null || die "create member" curl -fsS "$API/users" "${J[@]}" -H "$AUTH" -d '{"username":"smoke","password":"smokepw123","role":"member"}' >/dev/null || say "member smoke 已存在(重跑),复用"
MTOK=$(tokfor smoke smokepw123) MTOK=$(tokfor smoke smokepw123)
[ -n "$MTOK" ] || die "member login"
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$API/users" "${J[@]}" -H "authorization: Bearer $MTOK" -d '{"username":"x","password":"xpw12345","role":"member"}') code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$API/users" "${J[@]}" -H "authorization: Bearer $MTOK" -d '{"username":"x","password":"xpw12345","role":"member"}')
[ "$code" = 403 ] || die "member write not blocked ($code)" [ "$code" = 403 ] || die "member write not blocked ($code)"
say "library + bad-ext upload rejected + good upload + scan" say "library + bad-ext upload rejected + good upload + scan"
mkdir -p deploy/api/storage/smoke-books mkdir -p deploy/api/storage/smoke-books
LID=$(curl -fsS "$API/libraries" "${J[@]}" -H "$AUTH" -d '{"name":"smoke","root_path":"/data/books/smoke-books"}' | sed -E 's/.*"id":([0-9]+).*/\1/') LID=$(curl -sf "$API/libraries" "${J[@]}" -H "$AUTH" -d '{"name":"smoke","root_path":"/data/books/smoke-books"}' | sed -E 's/.*"id":([0-9]+).*/\1/' || true)
[ -n "$LID" ] || LID=$(curl -fsS "$API/libraries" -H "$AUTH" | grep -oE '"id":[0-9]+,"name":"smoke"' | cut -d: -f2 | cut -d, -f1)
[ -n "$LID" ] || die "no library id"
printf 'x' > "$WORK/f" printf 'x' > "$WORK/f"
curl -fsS -o /dev/null "$API/libraries/$LID/upload" -H "$AUTH" -F "file=@$WORK/f;filename=virus.exe" && die "bad ext upload must fail" || true curl -fsS -o /dev/null "$API/libraries/$LID/upload" -H "$AUTH" -F "file=@$WORK/f;filename=virus.exe" && die "bad ext upload must fail" || true
printf 'hello smoke book' > "$WORK/f" printf 'hello smoke book' > "$WORK/f"
+3 -2
View File
@@ -3,9 +3,10 @@ import type { Book, Library, Me, ProgressRow, User } from "./types";
export const TOKEN_KEY = "booklib.token"; export const TOKEN_KEY = "booklib.token";
export const LOGOUT_EVENT = "booklib:logout"; export const LOGOUT_EVENT = "booklib:logout";
// 无 localStorage 环境(vitest node)退化为内存,仅测试路径生效 // 无 localStorage 环境(vitest node)退化为内存,仅测试路径生效。
// 探测方法而非 typeof:node 22+ 的 experimental webstorage 会暴露无方法的 localStorage 桩。
const mem = new Map<string, string>(); const mem = new Map<string, string>();
const hasLS = typeof localStorage !== "undefined"; const hasLS = typeof globalThis.localStorage?.setItem === "function";
export function getToken(): string | null { export function getToken(): string | null {
return hasLS ? localStorage.getItem(TOKEN_KEY) : mem.get(TOKEN_KEY) ?? null; return hasLS ? localStorage.getItem(TOKEN_KEY) : mem.get(TOKEN_KEY) ?? null;