107 lines
7.6 KiB
Markdown
107 lines
7.6 KiB
Markdown
# Book & Comic Library
|
|
|
|
Personal book/comic library: Go+Gin backend (scan/upload ingestion, multi-user JWT, reading progress, disk+Redis cache) + Docker Compose deployment. Design: `superpowers/specs/2026-09-04-book-comic-library-design.md`; deployment spec: `superpowers/specs/2026-09-07-docker-deploy-spec-design.md`. Chinese mirror: `README_zh.md`.
|
|
|
|
## Deploy mode
|
|
|
|
- release: `deploy/docker-compose.yml` — multi-stage `backend|frontend/Dockerfile.prod` (target runner) compiled artifacts, no source mounts; infra ports stay on the internal network.
|
|
- dev: `deploy/docker-compose.dev.yml` — all four services containerized, source mounted as `../backend:/app` and `../frontend:/app` (frontend with an anonymous `node_modules` volume), `target: dev` images; api runs `go mod download && dlv debug ./cmd/webui` (hot restart without rebuild, delve :2345); infra exposed to host PG 5432 / Redis 6379; healthcheck-gated startup.
|
|
|
|
## Volume Mount
|
|
|
|
- Shared named volumes: `booklib_postgres_data`, `booklib_redis_data` (identical names in both composes, so dev/release see the same data; only `down -v` clears them).
|
|
- Config/log bind mounts: `deploy/nginx/{nginx.conf,conf.d/default.conf}`, `deploy/redis/redis.conf` (`:ro`) and `deploy/logs/nginx` — all rendered by `deploy/prepare.sh` (templates are the `*.tpl` files beside each output); **wrong/missing config in the container = you forgot to rerun it**.
|
|
- File storage: `deploy/api/storage` → container `/data/books` (cache writes to `/data/books/cache`).
|
|
|
|
## Run it (production shape)
|
|
|
|
```bash
|
|
cp deploy/.env.example deploy/.env # set JWT_SECRET, ADMIN_USER, ADMIN_PASSWORD (>= 8 chars; seed skips and logs below 8)
|
|
deploy/prepare.sh
|
|
docker compose -f deploy/docker-compose.yml up -d --build
|
|
bash scripts/smoke.sh && bash scripts/smoke-web.sh
|
|
```
|
|
|
|
- web: `http://localhost:8080` (change via `WEB_PORT`); the API goes through the nginx `/api/` prefix reverse proxy to stateless api replicas (`--scale api=N`).
|
|
- Drop raw books into `deploy/api/storage/` (mounted at `/data/books`); the scanner ingests periodically (default 60s).
|
|
- nginx access/error logs: `deploy/logs/nginx/`.
|
|
|
|
## Development
|
|
|
|
```bash
|
|
deploy/prepare.sh
|
|
docker compose -f deploy/docker-compose.dev.yml up -d --build
|
|
```
|
|
|
|
- Frontend: http://localhost:5173 (vite, HMR works directly; `/api` proxied to the in-container api).
|
|
- After editing Go code: `docker compose -f deploy/docker-compose.dev.yml restart api` (recompiles the mounted source, no rebuild).
|
|
- Breakpoint debugging: delve headless at `localhost:2345` (VSCode launch: `{"type":"go","request":"attach","mode":"remote","host":"localhost","port":2345,"substitutePath":[{"from":"${workspaceFolder}/backend","to":"/app"}]}`; continue via dlv commands after hitting a breakpoint).
|
|
- Run tests against directly reachable infra: PG `localhost:5432` (lib/lib/lib), Redis `localhost:6379`:
|
|
|
|
```bash
|
|
cd backend
|
|
export DATABASE_URL='postgres://lib:lib@localhost:5432/lib?sslmode=disable'
|
|
export REDIS_URL='redis://localhost:6379'
|
|
go vet ./... && gofmt -l .
|
|
go test -p 1 -count=1 ./...
|
|
```
|
|
|
|
`-p 1` is required: integration tests share one PG database and each clears tables with `DELETE FROM ...` — running in parallel deletes each other's data and fails randomly. Bring the dev stack down with `down` (not `rm`), or the anonymous node_modules volume becomes an orphan. Tests that need PG/Redis skip automatically when absent; Redis downtime doesn't break functionality (the whole chain degrades to miss/passthrough, see spec §9).
|
|
|
|
Frontend gate: `cd frontend && npm run check` (tsc + eslint + prettier + vitest + vite build). E2E is a separate manual pre-release gate against the dev stack: `cd frontend && npm run e2e` (requires `E2E_ADMIN_USER`/`E2E_ADMIN_PASSWORD`, or source `deploy/.env`; one-off browser install inside the web container: `npx playwright install --with-deps chromium`). Bundle report: `npm run analyze` → `dist-stats/stats.html`.
|
|
|
|
## Old-volume migration (one-off, upgrading from the previous deploy layout)
|
|
|
|
```bash
|
|
# old PG data → new shared volume
|
|
docker run --rm -v book-comic-library_pgdata:/from -v booklib_postgres_data:/to alpine cp -a /from/. /to/
|
|
# the old cache volume is derived data (covers/unzipped pages), just drop it (auto-rebuilt)
|
|
docker volume rm book-comic-library_pgdata book-comic-library_cache
|
|
```
|
|
|
|
Book files: move the contents of the old host `./library/` into `deploy/api/storage/`.
|
|
|
|
## Trusted proxies & rate limiting
|
|
|
|
- nginx lives inside the compose network, so api's `ClientIP` only trusts `TRUSTED_PROXY_CIDRS` (comma-separated CIDRs, default `172.16.0.0/12`, the compose subnet). Spoofed external `X-Forwarded-For` can't bypass rate-limit buckets; change this env when the deploy network changes.
|
|
- Login is limited to 5 attempts/min/IP **counted per attempt, successful logins included** — brute force and normal high-frequency login share the budget.
|
|
|
|
## Read this before changing the schema
|
|
|
|
Schema changes go through the ordered migration system in `backend/internal/db/migrations/`:
|
|
|
|
1. Create a new file: `NNNN_description.sql` (four-digit sequence number, lowercase snake_case).
|
|
2. Never modify an already-applied migration file — they are immutable.
|
|
3. No down migrations: rollback via database backup, fix-forward.
|
|
4. Existing databases are auto-baselined on first startup (0001 marked applied without re-running DDL).
|
|
5. Migrations run with `pg_advisory_lock` so `--scale api=N` replicas serialize safely.
|
|
|
|
Local gate before each batch merge: `go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...` with dev PG+Redis running.
|
|
|
|
## Backend structure
|
|
|
|
The backend is organized around consumer-side port interfaces (hexagonal style):
|
|
|
|
- `cmd/webui` — binary entry point and composition root: `main.go` builds concrete implementations (`store.Store`, `redispkg.R`, `scanner.Scanner`, `media.M`, `upload.U`) and hands them to `api.NewRouter`, which only accepts the port interfaces.
|
|
- `cmd/webui/handlers` — HTTP layer: request binding, auth/authz, error → status mapping. No SQL, no archive/file logic.
|
|
- `internal/ports` — the small interfaces handlers depend on (`UserStore`, `LibraryStore`, `BookStore`, `ProgressStore`, `BookmarkStore`, `RateLimiter`, `Scanner`, `Media`, `UploadSessions`) plus shared sentinel errors. Interfaces live on the consumer side, implementations satisfy them.
|
|
- `internal/ports/portsfake` — hand-written in-memory fakes for every port, with error semantics mirroring the real store (`pgx.ErrNoRows`, `ErrLastAdmin`, PgError 23505). Handler unit tests run against these with no PG/Redis.
|
|
- `internal/media` — cover/page extraction, page-index cache (Redis-backed), atomic cache writes.
|
|
- `internal/upload` — chunked upload session lifecycle (init/part/status/complete/sweep).
|
|
- `internal/store` — all SQL, one place.
|
|
- `internal/scanner` — library walk, ingest (add/update/delete in one pass), sweep riding the scan ticker.
|
|
- `internal/bookfile` — shared file utilities (`SafeName`, `Contains`, `Hash`, `FormatFromExt`, cache dir layout).
|
|
|
|
Testing is two-tiered: integration tests hit a real PG+Redis via the full router (`handlers/*_test.go` with `setupAPI`), unit tests hit the same router with `portsfake` injected (`handlers/*_unit_test.go`). The route table itself is pinned by `TestRouterContract` in `cmd/webui/api`.
|
|
|
|
## CI
|
|
|
|
- Workflow: `.github/workflows/ci.yml` (standard GitHub Actions syntax, Gitea Actions compatible).
|
|
- **Gitea**: register an `act_runner` instance, enable Actions in repo settings. Works out of the box.
|
|
- **GitHub**: works out of the box.
|
|
- Until a runner is registered, run the local gate manually before merging.
|
|
|
|
## PWA
|
|
|
|
Immutable assets (covers/CBZ pages/raw files) are SW cache-first; read content stays available offline; logging out clears the SW cache.
|