diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 55a6e63..31493cc 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -8,6 +8,49 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab ## [Unreleased] +### Added / 新增 + +- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with a periodic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk). +- API:新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,定期清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。 + +- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent). +- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。 + +- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization. +- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。 + +- Tests: router contract test (`TestRouterContract`) pins the full route table — any added, removed or renamed route fails the test until the expectation is updated deliberately. +- 测试:新增路由契约测试(`TestRouterContract`),锁定全量路由表——任何路由的增删改名都会使该测试失败,必须显式更新预期。 + +- Tests: hand-written in-memory fakes (`internal/ports/portsfake`) cover all port interfaces, enabling handler unit tests with no PG or Redis. Error semantics mirror the real store (`pgx.ErrNoRows`, `store.ErrLastAdmin`, `*pgconn.PgError{Code:23505}`), so the fakes exercise the same 404/409/400 branches as the database. +- 测试:新增手写内存 fake(`internal/ports/portsfake`),覆盖全部 port 接口,使 handler 单测无需 PG/Redis 即可运行。错误语义与真实 store 一致(`pgx.ErrNoRows`、`store.ErrLastAdmin`、`*pgconn.PgError{Code:23505}`),因此 fake 走到的是与真库相同的 404/409/400 分支。 + +- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions. +- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。 + +### Changed / 变更 + +- Backend restructured around hexagonal ports: HTTP handlers now depend only on small consumer-side interfaces (`internal/ports`) instead of concrete `*store.Store` / `*redispkg.R` / `*scanner.Scanner`. Domain logic moved out of handlers into `internal/media` (cover/page extraction, page index cache, atomic cache writes) and `internal/upload` (chunked session lifecycle). `cmd/webui/main.go` is the composition root; `api.NewRouter` accepts pure interfaces. +- 后端按六边形端口重构:HTTP handler 现在只依赖 `internal/ports` 中的小口径消费端接口,不再直接持有 `*store.Store` / `*redispkg.R` / `*scanner.Scanner` 等具体类型。域逻辑从 handler 下沉到 `internal/media`(封面/页抽取、页索引缓存、缓存原子写)与 `internal/upload`(分片会话生命周期)。`cmd/webui/main.go` 作为装配根,`api.NewRouter` 只收接口。 + +- Upload session sweep moved off the request path onto the scanner's ticker cycle (B16), so `POST /upload/init` no longer pays for a directory walk. +- 上传会话清理从请求路径移到扫描器的定时周期(B16),`POST /upload/init` 不再顺带付出一次目录遍历的开销。 + +- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward. +- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。 + +- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader. +- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。 + +- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`. +- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。 + +- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/` (no client-supplied paths, validated at creation). +- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。 + +- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked. +- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`. + ### Fixed / 修复 - Rate limiter `IncrWindow` uses atomic Lua script for INCR+EXPIRE, preventing permanent IP lockout on EXPIRE failure (B1). @@ -36,45 +79,9 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab - 上传处理器在 `O_EXCL` 冲突时重试,处理并发同名上传(B12)。 - Bookmark methods check `err` before `RowsAffected` to avoid invalid reads on query failure (B13). - 书签方法先检查 `err` 再读 `RowsAffected`,避免查询失败时的无效读取(B13)。 - -### Changed / 变更 - -- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward. -- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。 - -### Fixed / 修复 - - Serve goroutine `log.Fatalf` replaced with channel-based shutdown to preserve graceful teardown (B14). - 服务 goroutine 中的 `log.Fatalf` 改为 channel 通知方式,确保优雅关停流程不被绕过(B14)。 - `DATABASE_URL` is now validated at startup (required, parseable); empty `REDIS_URL` logs a clear "redis disabled" message (B15). - `DATABASE_URL` 在启动时校验(必填、可解析);空 `REDIS_URL` 记录明确的 "redis disabled" 日志(B15)。 - `scripts/smoke.sh` aligned with current API contract, removed ignored `root_path` field (B17). - `scripts/smoke.sh` 对齐当前 API 契约,移除被忽略的 `root_path` 字段(B17)。 - -### Added / 新增 - -- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions. -- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。 - -### Added / 新增 - -- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent). -- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。 - -- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization. -- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。 - -- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with 24h opportunistic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk). -- API: 新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,超 24h 顺手清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。 - -### Changed / 变更 - -- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader. -- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。 - -- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`. -- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。 -- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/` (no client-supplied paths, validated at creation). -- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。 -- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked. -- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`. diff --git a/docs/README.md b/docs/README.md index 5f065b3..c4f096f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -78,6 +78,22 @@ Schema changes go through the ordered migration system in `backend/internal/db/m Local gate before each batch merge: `go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...` with dev PG+Redis running. +## Backend structure + +The backend is organized around consumer-side port interfaces (hexagonal style): + +- `cmd/webui` — binary entry point and composition root: `main.go` builds concrete implementations (`store.Store`, `redispkg.R`, `scanner.Scanner`, `media.M`, `upload.U`) and hands them to `api.NewRouter`, which only accepts the port interfaces. +- `cmd/webui/handlers` — HTTP layer: request binding, auth/authz, error → status mapping. No SQL, no archive/file logic. +- `internal/ports` — the small interfaces handlers depend on (`UserStore`, `LibraryStore`, `BookStore`, `ProgressStore`, `BookmarkStore`, `RateLimiter`, `Scanner`, `Media`, `UploadSessions`) plus shared sentinel errors. Interfaces live on the consumer side, implementations satisfy them. +- `internal/ports/portsfake` — hand-written in-memory fakes for every port, with error semantics mirroring the real store (`pgx.ErrNoRows`, `ErrLastAdmin`, PgError 23505). Handler unit tests run against these with no PG/Redis. +- `internal/media` — cover/page extraction, page-index cache (Redis-backed), atomic cache writes. +- `internal/upload` — chunked upload session lifecycle (init/part/status/complete/sweep). +- `internal/store` — all SQL, one place. +- `internal/scanner` — library walk, ingest (add/update/delete in one pass), sweep riding the scan ticker. +- `internal/bookfile` — shared file utilities (`SafeName`, `Contains`, `Hash`, `FormatFromExt`, cache dir layout). + +Testing is two-tiered: integration tests hit a real PG+Redis via the full router (`handlers/*_test.go` with `setupAPI`), unit tests hit the same router with `portsfake` injected (`handlers/*_unit_test.go`). The route table itself is pinned by `TestRouterContract` in `cmd/webui/api`. + ## CI - Workflow: `.github/workflows/ci.yml` (standard GitHub Actions syntax, Gitea Actions compatible). diff --git a/docs/README_zh.md b/docs/README_zh.md index e49e612..17a719e 100644 --- a/docs/README_zh.md +++ b/docs/README_zh.md @@ -78,6 +78,22 @@ Schema 变更通过 `backend/internal/db/migrations/` 中的有序迁移系统 每批合入前的本地门禁:`go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...`(需启动 dev PG+Redis)。 +## 后端结构 + +后端按消费端接口组织(六边形风格): + +- `cmd/webui` —— 二进制入口与装配根:`main.go` 构造具体实现(`store.Store`、`redispkg.R`、`scanner.Scanner`、`media.M`、`upload.U`)并交给 `api.NewRouter`,后者只接受 port 接口。 +- `cmd/webui/handlers` —— HTTP 层:参数绑定、认证/鉴权、错误→状态码映射。没有 SQL,没有压缩包/文件逻辑。 +- `internal/ports` —— handlers 依赖的小口径接口(`UserStore`、`LibraryStore`、`BookStore`、`ProgressStore`、`BookmarkStore`、`RateLimiter`、`Scanner`、`Media`、`UploadSessions`)与共享 sentinel 错误。接口定义在消费端,实现方来满足它们。 +- `internal/ports/portsfake` —— 全部 port 的手写内存 fake,错误语义与真实 store 一致(`pgx.ErrNoRows`、`ErrLastAdmin`、PgError 23505)。handler 单测无需 PG/Redis。 +- `internal/media` —— 封面/页抽取、页索引缓存(Redis)、缓存原子写。 +- `internal/upload` —— 分片上传会话生命周期(init/part/status/complete/sweep)。 +- `internal/store` —— 所有 SQL,集中一处。 +- `internal/scanner` —— 书库遍历、ingest(增/改/删一趟完成)、会话清扫搭扫描 ticker 顺风车。 +- `internal/bookfile` —— 共享文件工具(`SafeName`、`Contains`、`Hash`、`FormatFromExt`、缓存目录布局)。 + +测试分两层:集成测试走真实 PG+Redis、过完整 router(`handlers/*_test.go` 的 `setupAPI`);单测注入 `portsfake`、过同一个 router(`handlers/*_unit_test.go`)。路由表本身由 `cmd/webui/api` 的 `TestRouterContract` 钉死。 + ## CI - 工作流:`.github/workflows/ci.yml`(标准 GitHub Actions 语法,兼容 Gitea Actions)。