Compare commits

...
6 Commits
Author SHA1 Message Date
XingfenD 8305af9d5c docs: changelog + README backend structure for batch C (Task 29)
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
- CHANGELOG [Unreleased]: consolidated the duplicated Added/Changed/Fixed
  groups left by earlier batches into one group each (no entry dropped);
  added batch C entries — port-based restructure, media/upload domain
  packages, sweep moved to scanner ticker (B16), router contract test,
  portsfake unit-test layer. Upload-sweep wording no longer promises the
  old 24h opportunistic request-path behaviour.
- README.md / README_zh.md: new 'Backend structure' section documenting
  the port/fake layout (cmd/webui composition root, handlers as HTTP-only,
  internal/ports + portsfake, media/upload/store/scanner/bookfile
  responsibilities) and the two-tier testing approach (real PG+Redis
  integration vs fake-injected unit, route table pinned by contract test)

Gate: gofmt clean, go vet clean, go test -p 1 all pass (0 skip),
scripts/smoke.sh ALL SMOKE TESTS PASSED against a live webui on :18080
2026-09-14 23:37:36 +08:00
XingfenD 85c61d9f24 refactor: consolidate isUnique and path validation helpers (Task 28)
- seed.Admin: hand-rolled pgconn.PgError 23505 check → store.IsUniqueViolation
  (single predicate for unique violations across the codebase)
- books.absBookPath: local hasPrefixDir (filepath.Rel-based) removed in favor
  of bookfile.Contains, which also resolves symlinks — stricter escape check
- handlers isUnique alias + libraries.go local prefix check were already
  folded into ports.IsUniqueViolation / bookfile.Contains in Task 25;
  scanner.inside was folded in Task 24 — this commit closes the last two

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 23:26:16 +08:00
XingfenD cc1470f6e4 test: portsfake + router contract test + handler unit tests (Task 27)
- internal/ports/portsfake: hand-written in-memory fakes for all 9 ports
  (Users/Libraries/Books/Progress/Bookmarks/RateLimiter/Scanner/Media/Uploads);
  error semantics mirror the real store exactly — pgx.ErrNoRows for misses,
  store.ErrLastAdmin guard, and *pgconn.PgError{Code:23505} for unique
  violations (ports.IsUniqueViolation only accepts the PgError shape, so the
  fakes must produce it to exercise the 409 branch without touching prod code)
- Media/Uploads fakes are hook-programmable: one field per error branch, so a
  test can force e.g. CompleteErr=ErrIncomplete without stubbing the rest
- Scanner fake is mutex-guarded + WaitForScan: handler fires ScanLibraryByID
  in a goroutine, tests stay deterministic
- router_test.go: TestRouterContract pins all 27 routes — any route table
  change now fails the test explicitly
- handler unit tests (~30 cases, no PG/Redis): users CRUD branches (self-delete
  400, last-admin 400, dup 409, 204 ok), library reserved names (contract:
  code=bad_request message=reserved_name, per original impl), upload sentinel
  mapping (413/400/404 per branch)
- NewRouter takes pure port interfaces; main.go distributes *store.Store
  across the 5 store ports at the composition root

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 23:23:10 +08:00
XingfenD 7baadedeb4 refactor(handlers): consume port interfaces, wire media+upload assembly (Task 25+26)
- H now holds only small port interfaces (UserStore/LibraryStore/BookStore/
  ProgressStore/BookmarkStore/RateLimiter/Scanner/Media/UploadSessions);
  NewRouter is the composition root distributing *store.Store and *redispkg.R
- ports.Media gains EnsurePage; ChaptersOf returns ports.Chapter (media's
  local duplicate dropped); *media.M now provably satisfies ports.Media;
  ports.UploadSessions gains LibraryID for root validation before Complete
- content.go: duplicated page-index cache + cover self-heal + page extract
  logic removed in favor of media service — same redis keys, same contract;
  path traversal check stays in handler (403 semantics preserved)
- getLibrary/getLibRow merged into getLib(c, id); idParam helper dedupes
  :id parsing; isUnique replaced by ports.IsUniqueViolation (Task 28 partial)
- main.go assembles media + upload and passes upload.U as scanner Sweeper

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 22:56:15 +08:00
XingfenD b9b7022212 refactor(scanner): merge add/update into ingest, use shared utilities
- add/update consolidated into ingest(isNew) — persist branch differs,
  error handling and cover path fully shared (Task 24)
- zipIndex/cover use bookfile.OpenReaderAt instead of hand-rolled
  open+stat pairs
- root containment check uses bookfile.Contains (local inside removed)
- cover write goes through media.WriteAtomic (now exported, log-free —
  callers own context); media no longer logs inside the atomic helper
2026-09-14 22:12:30 +08:00
XingfenD fbd5cd243d feat(upload): extract upload subsystem + move sweep to scanner ticker (B16)
- internal/upload owns chunked-upload domain logic (fingerprint resume,
  part tmp+rename, assemble, session sweep) and the shared UniquePath
  helper used by both single-file and chunked completion paths
- handlers/uploads.go is now HTTP-only: bind params, call upload.U, map
  sentinel errors to the unchanged status/code/message contract
- B16: session sweep moved out of the UploadInit request path onto the
  scanner ticker via a Sweeper hook (upload.U satisfies it)
- unit tests for the package without PG/Redis; contract pinned by the
  existing handler integration tests (all green)
2026-09-14 22:00:25 +08:00
27 changed files with 2176 additions and 647 deletions
+17 -15
View File
@@ -7,14 +7,16 @@ import (
"booklib/cmd/webui/handlers" "booklib/cmd/webui/handlers"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/redispkg" "booklib/internal/ports"
"booklib/internal/scanner"
"booklib/internal/store"
) )
func NewRouter(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner.Scanner) *gin.Engine { // NewRouter 只依赖 port 接口:main.go 传具体实现(*store.Store 满足 5 个 store
// 接口,*redispkg.R 满足 RateLimiter),测试传 portsfake。
func NewRouter(cfg *config.Config, users ports.UserStore, libs ports.LibraryStore, books ports.BookStore,
progress ports.ProgressStore, bookmarks ports.BookmarkStore, rl ports.RateLimiter,
sc ports.Scanner, med ports.Media, up ports.UploadSessions) *gin.Engine {
gin.SetMode(gin.ReleaseMode) gin.SetMode(gin.ReleaseMode)
h := handlers.New(cfg, st, rdb, sc) h := handlers.New(cfg, users, libs, books, progress, bookmarks, rl, sc, med, up)
r := gin.New() r := gin.New()
if e := r.SetTrustedProxies(cfg.TrustedProxies); e != nil { if e := r.SetTrustedProxies(cfg.TrustedProxies); e != nil {
panic(e) panic(e)
@@ -27,17 +29,17 @@ func NewRouter(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner
p := g.Group("", h.AuthMw()) p := g.Group("", h.AuthMw())
p.GET("/auth/me", h.Me) p.GET("/auth/me", h.Me)
users := p.Group("/users", h.AdminOnly()) usersGrp := p.Group("/users", h.AdminOnly())
users.GET("", h.ListUsers) usersGrp.GET("", h.ListUsers)
users.POST("", h.CreateUser) usersGrp.POST("", h.CreateUser)
users.DELETE("/:id", h.DeleteUser) usersGrp.DELETE("/:id", h.DeleteUser)
libs := p.Group("/libraries") libsGrp := p.Group("/libraries")
libs.GET("", h.ListLibraries) libsGrp.GET("", h.ListLibraries)
libs.POST("", h.AdminOnly(), h.CreateLibrary) libsGrp.POST("", h.AdminOnly(), h.CreateLibrary)
libs.POST("/:id/scan", h.AdminOnly(), h.ScanLibrary) libsGrp.POST("/:id/scan", h.AdminOnly(), h.ScanLibrary)
libs.POST("/:id/upload", h.AdminOnly(), h.Upload) libsGrp.POST("/:id/upload", h.AdminOnly(), h.Upload)
libs.POST("/:id/upload/init", h.AdminOnly(), h.UploadInit) libsGrp.POST("/:id/upload/init", h.AdminOnly(), h.UploadInit)
uploads := p.Group("/uploads", h.AdminOnly()) uploads := p.Group("/uploads", h.AdminOnly())
uploads.GET("/:uid", h.UploadStatus) uploads.GET("/:uid", h.UploadStatus)
+54 -2
View File
@@ -7,7 +7,6 @@ import (
"time" "time"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/redispkg"
) )
func testCfg() *config.Config { func testCfg() *config.Config {
@@ -16,7 +15,7 @@ func testCfg() *config.Config {
} }
func TestHealthz(t *testing.T) { func TestHealthz(t *testing.T) {
r := NewRouter(testCfg(), nil, redispkg.New(""), nil) r := NewRouter(testCfg(), nil, nil, nil, nil, nil, nil, nil, nil, nil)
req := httptest.NewRequest(http.MethodGet, "/api/healthz", nil) req := httptest.NewRequest(http.MethodGet, "/api/healthz", nil)
w := httptest.NewRecorder() w := httptest.NewRecorder()
r.ServeHTTP(w, req) r.ServeHTTP(w, req)
@@ -24,3 +23,56 @@ func TestHealthz(t *testing.T) {
t.Fatalf("healthz = %d, want 200", w.Code) t.Fatalf("healthz = %d, want 200", w.Code)
} }
} }
// TestRouterContract 钉死全量路由表(Task 27):增删改任何路由都会使此测试失败,
// 必须显式更新——路由即对外契约。
func TestRouterContract(t *testing.T) {
r := NewRouter(testCfg(), nil, nil, nil, nil, nil, nil, nil, nil, nil)
want := map[string]bool{
"GET /api/healthz": true,
"POST /api/auth/login": true,
"GET /api/auth/me": true,
"GET /api/users": true,
"POST /api/users": true,
"DELETE /api/users/:id": true,
"GET /api/libraries": true,
"POST /api/libraries": true,
"POST /api/libraries/:id/scan": true,
"POST /api/libraries/:id/upload": true,
"POST /api/libraries/:id/upload/init": true,
"GET /api/uploads/:uid": true,
"POST /api/uploads/:uid/complete": true,
"PUT /api/uploads/:uid/parts/:index": true,
"GET /api/books": true,
"GET /api/books/:id": true,
"DELETE /api/books/:id": true,
"GET /api/books/:id/cover": true,
"GET /api/books/:id/file": true,
"GET /api/books/:id/pages": true,
"GET /api/books/:id/pages/:n": true,
"PUT /api/books/:id/progress": true,
"GET /api/progress": true,
"GET /api/books/:id/bookmarks": true,
"POST /api/books/:id/bookmarks": true,
"PATCH /api/bookmarks/:id": true,
"DELETE /api/bookmarks/:id": true,
}
got := map[string]bool{}
for _, rt := range r.Routes() {
key := rt.Method + " " + rt.Path
if got[key] {
t.Errorf("duplicate route %s", key)
}
got[key] = true
}
for k := range want {
if !got[k] {
t.Errorf("missing route %s", k)
}
}
for k := range got {
if !want[k] {
t.Errorf("unexpected route %s", k)
}
}
}
+3 -3
View File
@@ -20,11 +20,11 @@ func (h *H) Login(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "username and password required") err(c, http.StatusBadRequest, "bad_request", "username and password required")
return return
} }
if n := h.rdb.IncrWindow(c, "loginrl:"+c.ClientIP(), loginWindow); n > loginMax { if n := h.rl.IncrWindow(c, "loginrl:"+c.ClientIP(), loginWindow); n > loginMax {
err(c, http.StatusTooManyRequests, "rate_limited", "too many login attempts") err(c, http.StatusTooManyRequests, "rate_limited", "too many login attempts")
return return
} }
u, qerr := h.st.GetUserByName(c, req.Username) u, qerr := h.users.GetUserByName(c, req.Username)
if qerr != nil { if qerr != nil {
if !errors.Is(qerr, pgx.ErrNoRows) { if !errors.Is(qerr, pgx.ErrNoRows) {
dbErr(c, qerr) dbErr(c, qerr)
@@ -49,7 +49,7 @@ func (h *H) Login(c *gin.Context) {
func (h *H) Me(c *gin.Context) { func (h *H) Me(c *gin.Context) {
// B7: only no-rows → 401; other errors (PG down) go through dbErr → 503. // B7: only no-rows → 401; other errors (PG down) go through dbErr → 503.
u, qerr := h.st.GetUserByID(c, uid(c)) u, qerr := h.users.GetUserByID(c, uid(c))
if qerr != nil { if qerr != nil {
if errors.Is(qerr, pgx.ErrNoRows) { if errors.Is(qerr, pgx.ErrNoRows) {
err(c, http.StatusUnauthorized, "unauthorized", "no such user") err(c, http.StatusUnauthorized, "unauthorized", "no such user")
+6 -2
View File
@@ -18,9 +18,11 @@ import (
"booklib/internal/auth" "booklib/internal/auth"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/db" "booklib/internal/db"
"booklib/internal/media"
"booklib/internal/redispkg" "booklib/internal/redispkg"
"booklib/internal/scanner" "booklib/internal/scanner"
"booklib/internal/store" "booklib/internal/store"
"booklib/internal/upload"
) )
func testCfg() *config.Config { func testCfg() *config.Config {
@@ -57,8 +59,10 @@ func setupAPI(t *testing.T) (*store.Store, *scanner.Scanner, http.Handler, strin
cfg.BooksDir = booksDir cfg.BooksDir = booksDir
cfg.CacheDir = t.TempDir() cfg.CacheDir = t.TempDir()
rdb := redispkg.New(os.Getenv("REDIS_URL")) rdb := redispkg.New(os.Getenv("REDIS_URL"))
sc := scanner.New(st, cfg, rdb) med := media.New(cfg, rdb)
r := api.NewRouter(cfg, st, rdb, sc) up := upload.New(cfg.BooksDir, cfg.UploadMaxMB)
sc := scanner.New(st, cfg, rdb, up)
r := api.NewRouter(cfg, st, st, st, st, st, rdb, sc, med, up)
if u := os.Getenv("REDIS_URL"); u != "" { // 测试卫生: 共享 redis 上重置登录限流桶, 防跨测试累计 429 if u := os.Getenv("REDIS_URL"); u != "" { // 测试卫生: 共享 redis 上重置登录限流桶, 防跨测试累计 429
if opt, e := redis.ParseURL(u); e == nil { if opt, e := redis.ParseURL(u); e == nil {
rc := redis.NewClient(opt) rc := redis.NewClient(opt)
+4 -4
View File
@@ -27,7 +27,7 @@ func (h *H) ListBookmarks(c *gin.Context) {
if !ok { if !ok {
return return
} }
rows, e := h.st.ListBookmarks(c, uid(c), b.LibraryID, b.Path) rows, e := h.bookmarks.ListBookmarks(c, uid(c), b.LibraryID, b.Path)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -65,7 +65,7 @@ func (h *H) CreateBookmark(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)") err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)")
return return
} }
id, e := h.st.InsertBookmark(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent, req.Note) id, e := h.bookmarks.InsertBookmark(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent, req.Note)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -101,7 +101,7 @@ func (h *H) PatchBookmark(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)") err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)")
return return
} }
updated, e := h.st.UpdateBookmarkNote(c, uid(c), id, req.Note) updated, e := h.bookmarks.UpdateBookmarkNote(c, uid(c), id, req.Note)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -118,7 +118,7 @@ func (h *H) DeleteBookmark(c *gin.Context) {
if !ok { if !ok {
return return
} }
deleted, e := h.st.DeleteBookmark(c, uid(c), id) deleted, e := h.bookmarks.DeleteBookmark(c, uid(c), id)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
+11 -30
View File
@@ -7,7 +7,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings"
"time" "time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
@@ -18,7 +17,7 @@ import (
) )
func (h *H) getBookRow(c *gin.Context, id int64) (store.Book, bool) { func (h *H) getBookRow(c *gin.Context, id int64) (store.Book, bool) {
b, e := h.st.GetBook(c, id) b, e := h.books.GetBook(c, id)
if e != nil { if e != nil {
if errors.Is(e, pgx.ErrNoRows) { if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such book") err(c, http.StatusNotFound, "not_found", "no such book")
@@ -31,41 +30,23 @@ func (h *H) getBookRow(c *gin.Context, id int64) (store.Book, bool) {
} }
func (h *H) bookFromParam(c *gin.Context) (store.Book, bool) { func (h *H) bookFromParam(c *gin.Context) (store.Book, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64) id, ok := idParam(c)
if e != nil { if !ok {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return store.Book{}, false return store.Book{}, false
} }
return h.getBookRow(c, id) return h.getBookRow(c, id)
} }
func (h *H) getLibRow(c *gin.Context, id int64) (store.Library, bool) { // absBookPath: books.path 永远相对且不含 ..;拼接后二次前缀校验(纵深防御)。
l, e := h.st.GetLibrary(c, id) // bookfile.Contains 带 EvalSymlinks,比裸 filepath.Rel 更能拦住软链逃逸。
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return l, true
}
// absBookPath: books.path 永远相对且不含 ..;拼接后二次前缀校验(纵深防御)
func absBookPath(root string, b store.Book) (string, error) { func absBookPath(root string, b store.Book) (string, error) {
abs := filepath.Join(root, filepath.FromSlash(b.Path)) abs := filepath.Join(root, filepath.FromSlash(b.Path))
if filepath.Clean(abs) != abs || !hasPrefixDir(abs, root) { if filepath.Clean(abs) != abs || !bookfile.Contains(root, abs) {
return "", os.ErrPermission return "", os.ErrPermission
} }
return abs, nil return abs, nil
} }
func hasPrefixDir(p, dir string) bool {
rel, err := filepath.Rel(filepath.Clean(dir), filepath.Clean(p))
return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(os.PathSeparator))
}
func bookJSON(b store.Book, percent float64, libraryName string) gin.H { func bookJSON(b store.Book, percent float64, libraryName string) gin.H {
h := bookfile.Hash(b.FileSize, b.ModTS) h := bookfile.Hash(b.FileSize, b.ModTS)
j := gin.H{ j := gin.H{
@@ -89,7 +70,7 @@ func bookJSON(b store.Book, percent float64, libraryName string) gin.H {
func (h *H) ListBooks(c *gin.Context) { func (h *H) ListBooks(c *gin.Context) {
libID, _ := strconv.ParseInt(c.Query("library"), 10, 64) libID, _ := strconv.ParseInt(c.Query("library"), 10, 64)
views, e := h.st.ListBooks(c, libID, c.Query("q"), c.Query("prefix"), uid(c)) views, e := h.books.ListBooks(c, libID, c.Query("q"), c.Query("prefix"), uid(c))
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -106,12 +87,12 @@ func (h *H) GetBook(c *gin.Context) {
if !ok { if !ok {
return return
} }
p, e := h.st.GetProgress(c, uid(c), b.LibraryID, b.Path) // ErrNoRows → 零值 percent p, e := h.progress.GetProgress(c, uid(c), b.LibraryID, b.Path) // ErrNoRows → 零值 percent
if e != nil && !errors.Is(e, pgx.ErrNoRows) { if e != nil && !errors.Is(e, pgx.ErrNoRows) {
dbErr(c, e) dbErr(c, e)
return return
} }
lib, e := h.st.GetLibrary(c, b.LibraryID) lib, e := h.libs.GetLibrary(c, b.LibraryID)
if e != nil && !errors.Is(e, pgx.ErrNoRows) { // 库被并发删则留空 library 名,书仍可见 if e != nil && !errors.Is(e, pgx.ErrNoRows) { // 库被并发删则留空 library 名,书仍可见
dbErr(c, e) dbErr(c, e)
return return
@@ -124,7 +105,7 @@ func (h *H) DeleteBook(c *gin.Context) {
if !ok { if !ok {
return return
} }
lib, ok := h.getLibRow(c, b.LibraryID) lib, ok := h.getLib(c, b.LibraryID)
if !ok { if !ok {
return return
} }
@@ -144,7 +125,7 @@ func (h *H) DeleteBook(c *gin.Context) {
key := bookfile.DirKey(b.ID, bookfile.Hash(b.FileSize, b.ModTS)) key := bookfile.DirKey(b.ID, bookfile.Hash(b.FileSize, b.ModTS))
os.RemoveAll(bookfile.CoverDir(h.cfg.CacheDir, key)) os.RemoveAll(bookfile.CoverDir(h.cfg.CacheDir, key))
os.RemoveAll(bookfile.PagesDir(h.cfg.CacheDir, key)) os.RemoveAll(bookfile.PagesDir(h.cfg.CacheDir, key))
if e := h.st.DeleteBook(c, b.ID); e != nil { if e := h.books.DeleteBook(c, b.ID); e != nil {
dbErr(c, e) dbErr(c, e)
return return
} }
+69 -150
View File
@@ -1,15 +1,13 @@
package handlers package handlers
import ( import (
"fmt" "errors"
"log" "io/fs"
"net/http" "net/http"
"os" "os"
"path"
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
@@ -20,7 +18,7 @@ import (
const defaultCover = `<svg xmlns="http://www.w3.org/2000/svg" width="120" height="170"><rect width="120" height="170" rx="6" fill="#2a2a33"/><path d="M30 25h60v120H30z" fill="#3a3a45"/><path d="M30 25h60M60 25v120" stroke="#555" stroke-width="2"/></svg>` const defaultCover = `<svg xmlns="http://www.w3.org/2000/svg" width="120" height="170"><rect width="120" height="170" rx="6" fill="#2a2a33"/><path d="M30 25h60v120H30z" fill="#3a3a45"/><path d="M30 25h60M60 25v120" stroke="#555" stroke-width="2"/></svg>`
func (h *H) bookRoot(c *gin.Context, b store.Book) (string, bool) { func (h *H) bookRoot(c *gin.Context, b store.Book) (string, bool) {
lib, ok := h.getLibRow(c, b.LibraryID) lib, ok := h.getLib(c, b.LibraryID)
if !ok { if !ok {
return "", false return "", false
} }
@@ -31,6 +29,26 @@ func (h *H) immutable(c *gin.Context) {
c.Header("Cache-Control", "public, max-age=31536000, immutable") c.Header("Cache-Control", "public, max-age=31536000, immutable")
} }
// checkPath 纵深防御:path 越界 → 403,与原契约一致。
func checkPath(c *gin.Context, root string, b store.Book) (string, bool) {
abs, perr := absBookPath(root, b)
if perr != nil {
err(c, http.StatusForbidden, "forbidden", "unsafe path")
return "", false
}
return abs, true
}
// mapContentErr 把 media/bookfile 的文件级失败映射回原契约状态码:
// 文件不在盘上 → 404,其余(坏包等)由调用方决定 422/500。
func mapContentErr(c *gin.Context, e error) bool {
if errors.Is(e, fs.ErrNotExist) {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return true
}
return false
}
func (h *H) ServeCover(c *gin.Context) { func (h *H) ServeCover(c *gin.Context) {
b, ok := h.bookFromParam(c) b, ok := h.bookFromParam(c)
if !ok { if !ok {
@@ -48,36 +66,30 @@ func (h *H) ServeCover(c *gin.Context) {
} }
if b.Format == "cbz" || b.Format == "epub" { // 自愈:缓存丢了就地抽封面(重启/卷漂移/扫描器还没跑到) if b.Format == "cbz" || b.Format == "epub" { // 自愈:缓存丢了就地抽封面(重启/卷漂移/扫描器还没跑到)
if root, ok := h.bookRoot(c, b); ok { if root, ok := h.bookRoot(c, b); ok {
if f, size, ok := h.openBook(c, b, root); ok { if _, ok := checkPath(c, root, b); ok {
defer f.Close() e := h.med.EnsureCover(c, b.ID, b.Format, b.FileSize, b.ModTS, root, b.Path)
var img []byte switch {
var ext string case e == nil:
var e error if entries, re := os.ReadDir(dir); re == nil {
if b.Format == "cbz" { for _, en := range entries {
img, ext, e = bookfile.CBZCover(f, size) if !strings.Contains(en.Name(), ".tmp") {
} else { http.ServeFile(c.Writer, c.Request, filepath.Join(dir, en.Name()))
img, ext, e = bookfile.EPUBCover(f, size) return
} }
if e == nil {
dst := filepath.Join(dir, "cover"+ext)
if e := os.MkdirAll(dir, 0o755); e == nil {
tmp := fmt.Sprintf("%s.tmp-%d", dst, time.Now().UnixNano())
// B11: check all write errors; clean tmp only on failure.
if we := os.WriteFile(tmp, img, 0o644); we != nil {
log.Printf("serve: write cover tmp: %v", we)
os.Remove(tmp)
} else if re := os.Rename(tmp, dst); re != nil {
log.Printf("serve: rename cover: %v", re)
os.Remove(tmp)
} else {
http.ServeFile(c.Writer, c.Request, dst)
} }
} }
case mapContentErr(c, e):
return // 404 已回复
} }
// 抽取失败(坏包)→ 落到占位 SVG,与原契约一致
} else {
return // 403 已回复
} }
} else {
return // 404/503 已回复
} }
} }
if c.Writer.Written() { // openBook/bookRoot 已写 403/404/500,不再叠加占位图 if c.Writer.Written() {
return return
} }
c.Data(http.StatusOK, "image/svg+xml", []byte(defaultCover)) c.Data(http.StatusOK, "image/svg+xml", []byte(defaultCover))
@@ -92,55 +104,32 @@ func (h *H) ServeFile(c *gin.Context) {
if !ok { if !ok {
return return
} }
abs, perr := absBookPath(root, b) abs, ok := checkPath(c, root, b)
if perr != nil { if !ok {
err(c, http.StatusForbidden, "forbidden", "unsafe path")
return return
} }
c.Header("ETag", `"`+bookfile.Hash(b.FileSize, b.ModTS)+`"`) if _, e := os.Stat(abs); e != nil {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return
}
c.Header("ETag", `"`+h.med.CacheBuster(b.FileSize, b.ModTS)+`"`)
c.Header("Cache-Control", "private, must-revalidate") c.Header("Cache-Control", "private, must-revalidate")
http.ServeFile(c.Writer, c.Request, abs) http.ServeFile(c.Writer, c.Request, abs)
} }
func (h *H) openBook(c *gin.Context, b store.Book, root string) (*os.File, int64, bool) { // pageIndex 走 media(redis 缓存 + 索引提取);路径校验仍在 handler,保住 403 契约。
abs, perr := absBookPath(root, b) func (h *H) pageIndex(c *gin.Context, b store.Book, root string) ([]string, bool) {
if perr != nil { if _, ok := checkPath(c, root, b); !ok {
err(c, http.StatusForbidden, "forbidden", "unsafe path") return nil, false
return nil, 0, false
} }
f, perr := os.Open(abs) idx, e := h.med.PageIndex(c, b.ID, b.FileSize, b.ModTS, root, b.Path)
if perr != nil {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return nil, 0, false
}
st, perr := f.Stat()
if perr != nil {
f.Close()
err(c, http.StatusInternalServerError, "internal", "stat")
return nil, 0, false
}
return f, st.Size(), true
}
func (h *H) pageIndex(c *gin.Context, b store.Book, root string) ([]string, error) {
hash := bookfile.Hash(b.FileSize, b.ModTS)
key := fmt.Sprintf("pagesidx2:%d:%s", b.ID, hash) // 前缀换代 = 索引逻辑变更时一次性作废旧缓存
if v, ok := h.rdb.Get(c, key); ok && v != "" {
return strings.Split(v, "\n"), nil
}
f, size, ok := h.openBook(c, b, root)
if !ok {
return nil, os.ErrNotExist
}
defer f.Close()
idx, e := bookfile.PageIndex(f, size)
if e != nil { if e != nil {
return nil, e if !mapContentErr(c, e) {
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
}
return nil, false
} }
if len(idx) > 0 { // 空索引不缓存,否则 warm 命中 "" 会 Split 出幽灵页 return idx, true
h.rdb.Set(c, key, strings.Join(idx, "\n"), 7*24*time.Hour)
}
return idx, nil
} }
func (h *H) PagesCount(c *gin.Context) { func (h *H) PagesCount(c *gin.Context) {
@@ -156,56 +145,11 @@ func (h *H) PagesCount(c *gin.Context) {
if !ok { if !ok {
return return
} }
idx, e := h.pageIndex(c, b, root) idx, ok := h.pageIndex(c, b, root)
if e != nil { if !ok {
if c.Writer.Written() {
return // openBook 已写 403/404,不再叠加 422
}
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
return return
} }
c.JSON(http.StatusOK, gin.H{"count": len(idx), "chapters": chaptersOf(idx)}) c.JSON(http.StatusOK, gin.H{"count": len(idx), "chapters": h.med.ChaptersOf(idx)})
}
type cbzChapter struct {
Title string `json:"title"`
Start int `json:"start"`
}
// chaptersOf 页索引已自然序排列,按父目录分组:每个新目录开一章,标题取目录名;扁平包或只有一组返回 nil
func chaptersOf(idx []string) []cbzChapter {
type grp struct {
dir string
start int
}
var grps []grp
last := "\x00"
for i, n := range idx {
d := path.Dir(n)
if d == last {
continue
}
last = d
if d == "." { // 根目录散页不开章
continue
}
grps = append(grps, grp{d, i})
}
if len(grps) < 2 {
return nil
}
titles := make(map[string]int)
out := make([]cbzChapter, len(grps))
for i, g := range grps {
out[i] = cbzChapter{Title: path.Base(g.dir), Start: g.start}
titles[out[i].Title]++
}
for i, g := range grps { // 同名目录(不同父级)撞车 → 用全路径消歧
if titles[out[i].Title] > 1 {
out[i].Title = g.dir
}
}
return out
} }
func (h *H) Page(c *gin.Context) { func (h *H) Page(c *gin.Context) {
@@ -226,47 +170,22 @@ func (h *H) Page(c *gin.Context) {
if !ok { if !ok {
return return
} }
idx, e := h.pageIndex(c, b, root) idx, ok := h.pageIndex(c, b, root)
if e != nil { if !ok {
if c.Writer.Written() {
return // openBook 已写 403/404,不再叠加 422
}
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
return return
} }
if n >= len(idx) { if n >= len(idx) {
err(c, http.StatusNotFound, "not_found", "no such page") err(c, http.StatusNotFound, "not_found", "no such page")
return return
} }
ext := strings.ToLower(filepath.Ext(idx[n])) // miss → 解压落盘(media 内部唯一 tmp 名 + rename 原子,并发重做同页幂等)
dir := bookfile.PagesDir(h.cfg.CacheDir, bookfile.DirKey(b.ID, bookfile.Hash(b.FileSize, b.ModTS))) dst, e := h.med.EnsurePage(c, b.ID, b.FileSize, b.ModTS, root, b.Path, n, idx)
dst := filepath.Join(dir, strconv.Itoa(n)+ext) if e != nil {
if _, e := os.Stat(dst); e != nil { // miss → 解压落盘(并发重做同页幂等,唯一 tmp 名 + rename 原子) if mapContentErr(c, e) {
f, size, ok := h.openBook(c, b, root)
if !ok {
return
}
defer f.Close()
data, e := bookfile.ReadEntry(f, size, idx[n])
if e != nil {
err(c, http.StatusInternalServerError, "internal", "extract page")
return
}
if e := os.MkdirAll(dir, 0o755); e != nil {
err(c, http.StatusInternalServerError, "internal", "cache dir")
return
}
tmp := fmt.Sprintf("%s.tmp-%d", dst, time.Now().UnixNano())
if e := os.WriteFile(tmp, data, 0o644); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "write cache")
return
}
if e := os.Rename(tmp, dst); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename cache")
return return
} }
err(c, http.StatusInternalServerError, "internal", "extract page")
return
} }
h.immutable(c) h.immutable(c)
http.ServeFile(c.Writer, c.Request, dst) http.ServeFile(c.Writer, c.Request, dst)
+29 -9
View File
@@ -6,6 +6,7 @@ import (
"log" "log"
"net" "net"
"net/http" "net/http"
"strconv"
"strings" "strings"
"syscall" "syscall"
@@ -15,26 +16,45 @@ import (
"booklib/internal/auth" "booklib/internal/auth"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/redispkg" "booklib/internal/ports"
"booklib/internal/scanner"
"booklib/internal/store"
) )
// H 只依赖 ports 里的小口径接口(Task 25):具体实现由 main.go 装配,
// 测试可注入手写 fake(portsfake),无需 PG/Redis。
type H struct { type H struct {
cfg *config.Config cfg *config.Config
st *store.Store users ports.UserStore
rdb *redispkg.R libs ports.LibraryStore
sc *scanner.Scanner books ports.BookStore
progress ports.ProgressStore
bookmarks ports.BookmarkStore
rl ports.RateLimiter
sc ports.Scanner
med ports.Media
up ports.UploadSessions
} }
func New(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner.Scanner) *H { func New(cfg *config.Config, users ports.UserStore, libs ports.LibraryStore, books ports.BookStore,
return &H{cfg: cfg, st: st, rdb: rdb, sc: sc} progress ports.ProgressStore, bookmarks ports.BookmarkStore, rl ports.RateLimiter,
sc ports.Scanner, med ports.Media, up ports.UploadSessions) *H {
return &H{cfg: cfg, users: users, libs: libs, books: books, progress: progress,
bookmarks: bookmarks, rl: rl, sc: sc, med: med, up: up}
} }
func err(c *gin.Context, status int, code, msg string) { func err(c *gin.Context, status int, code, msg string) {
c.AbortWithStatusJSON(status, gin.H{"error": gin.H{"code": code, "message": msg}}) c.AbortWithStatusJSON(status, gin.H{"error": gin.H{"code": code, "message": msg}})
} }
// idParam 解析 :id 路径参数,失败已回复 400。
func idParam(c *gin.Context) (int64, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return 0, false
}
return id, true
}
// dbErr 统一处理 store 层失败:记日志;连接类错误 503(Service Unavailable),其余 500 // dbErr 统一处理 store 层失败:记日志;连接类错误 503(Service Unavailable),其余 500
// 注:brief 里的 pgxpool.ErrClosedPool 在 pgx v5 不存在,实际由 puddle 原样透出,用它替代; // 注:brief 里的 pgxpool.ErrClosedPool 在 pgx v5 不存在,实际由 puddle 原样透出,用它替代;
// PG 停机时池内连接先收到 SQLSTATE 57P01(administrator shutdown),故把 08xx/57Pxx 也归为 503 // PG 停机时池内连接先收到 SQLSTATE 57P01(administrator shutdown),故把 08xx/57Pxx 也归为 503
+32 -52
View File
@@ -16,6 +16,7 @@ import (
"booklib/internal/bookfile" "booklib/internal/bookfile"
"booklib/internal/media" "booklib/internal/media"
"booklib/internal/ports"
"booklib/internal/store" "booklib/internal/store"
) )
@@ -23,7 +24,7 @@ import (
func (h *H) libRoot(c *gin.Context, lib store.Library) (string, bool) { func (h *H) libRoot(c *gin.Context, lib store.Library) (string, bool) {
root := filepath.Clean(lib.RootPath) root := filepath.Clean(lib.RootPath)
books := filepath.Clean(h.cfg.BooksDir) books := filepath.Clean(h.cfg.BooksDir)
if !filepath.IsAbs(root) || (root != books && !strings.HasPrefix(root, books+string(os.PathSeparator))) { if !filepath.IsAbs(root) || !bookfile.Contains(books, root) {
err(c, http.StatusForbidden, "forbidden", "library root outside books dir") err(c, http.StatusForbidden, "forbidden", "library root outside books dir")
return "", false return "", false
} }
@@ -34,8 +35,22 @@ func (h *H) libRoot(c *gin.Context, lib store.Library) (string, bool) {
return root, true return root, true
} }
// getLib 查库行,404/503/500 已回复(原 getLibrary/getLibRow 合一,Task 25)
func (h *H) getLib(c *gin.Context, id int64) (store.Library, bool) {
l, e := h.libs.GetLibrary(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return l, true
}
func (h *H) ListLibraries(c *gin.Context) { func (h *H) ListLibraries(c *gin.Context) {
libs, e := h.st.ListLibraries(c) libs, e := h.libs.ListLibraries(c)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -68,9 +83,9 @@ func (h *H) CreateLibrary(c *gin.Context) {
return return
} }
root := filepath.Join(filepath.Clean(h.cfg.BooksDir), safe) root := filepath.Join(filepath.Clean(h.cfg.BooksDir), safe)
id, e := h.st.CreateLibrary(c, req.Name, root) id, e := h.libs.CreateLibrary(c, req.Name, root)
if e != nil { if e != nil {
if isUnique(e) { if ports.IsUniqueViolation(e) {
err(c, http.StatusConflict, "exists", "name taken") err(c, http.StatusConflict, "exists", "name taken")
return return
} }
@@ -80,26 +95,12 @@ func (h *H) CreateLibrary(c *gin.Context) {
c.JSON(http.StatusCreated, gin.H{"id": id, "name": req.Name, "root_path": root}) c.JSON(http.StatusCreated, gin.H{"id": id, "name": req.Name, "root_path": root})
} }
func (h *H) getLibrary(c *gin.Context) (store.Library, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return store.Library{}, false
}
lib, e := h.st.GetLibrary(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return lib, true
}
func (h *H) ScanLibrary(c *gin.Context) { func (h *H) ScanLibrary(c *gin.Context) {
lib, ok := h.getLibrary(c) id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok { if !ok {
return return
} }
@@ -111,7 +112,11 @@ func (h *H) ScanLibrary(c *gin.Context) {
} }
func (h *H) Upload(c *gin.Context) { func (h *H) Upload(c *gin.Context) {
lib, ok := h.getLibrary(c) id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok { if !ok {
return return
} }
@@ -136,12 +141,12 @@ func (h *H) Upload(c *gin.Context) {
return return
} }
// B12: retry on O_EXCL collision — concurrent uploads with the same name // B12: retry on O_EXCL collision — concurrent uploads with the same name
// can both get the same candidate from uniquePath (stat-then-create race). // can both get the same candidate from UniquePath (stat-then-create race).
var dst, tmp string var dst, tmp string
var out *os.File var out *os.File
for attempt := 0; attempt < 5; attempt++ { for attempt := 0; attempt < 5; attempt++ {
var e error var e error
dst, e = h.uniquePath(root, name) dst, e = h.up.UniquePath(root, name)
if e != nil { if e != nil {
err(c, http.StatusForbidden, "forbidden", e.Error()) err(c, http.StatusForbidden, "forbidden", e.Error())
return return
@@ -155,7 +160,7 @@ func (h *H) Upload(c *gin.Context) {
err(c, http.StatusInternalServerError, "internal", "create tmp") err(c, http.StatusInternalServerError, "internal", "create tmp")
return return
} }
// O_EXCL collision — retry with fresh uniquePath. // O_EXCL collision — retry with fresh UniquePath.
} }
if out == nil { if out == nil {
err(c, http.StatusConflict, "conflict", "too many concurrent uploads with same name") err(c, http.StatusConflict, "conflict", "too many concurrent uploads with same name")
@@ -190,28 +195,3 @@ func (h *H) Upload(c *gin.Context) {
} }
c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": strings.TrimPrefix(dst, root+string(os.PathSeparator))}) c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": strings.TrimPrefix(dst, root+string(os.PathSeparator))})
} }
// uniquePath 清洗后的 name 必须仍在 root 内;重名加 " (n)" 后缀
func (h *H) uniquePath(root, name string) (string, error) {
ext := filepath.Ext(name)
base := strings.TrimSuffix(name, ext)
for i := 0; ; i++ {
cand := base + ext
if i > 0 {
cand = base + " (" + strconv.Itoa(i) + ")" + ext
}
p := filepath.Join(root, cand)
if filepath.Clean(p) != filepath.Join(root, filepath.Clean(cand)) ||
!strings.HasPrefix(filepath.Clean(p), root+string(os.PathSeparator)) {
return "", os.ErrInvalid
}
if _, e := os.Stat(p); os.IsNotExist(e) {
return p, nil
} else if e != nil {
return "", e
}
if i > 999 {
return "", os.ErrExist
}
}
}
@@ -0,0 +1,232 @@
package handlers_test
import (
"net/http"
"strconv"
"strings"
"testing"
"time"
"booklib/internal/ports"
"booklib/internal/upload"
)
// ---------- libraries (Task 27, portsfake) ----------
func TestUnit_ListLibraries(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "comics")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/libraries", atok, nil)
httpOK(t, w, 200, "list libraries")
var libs []map[string]any
if err := jsonUnmarshal(w, &libs); err != nil {
t.Fatal(err)
}
if len(libs) != 1 || libs[0]["name"] != "comics" {
t.Fatalf("unexpected libs %v", libs)
}
}
func TestUnit_CreateLibrary_ReservedName(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
// 保留名清单以 media/reserved.go 为准(B8);大小写不敏感。
// 契约(与重构前一致): code="bad_request", message="reserved_name"
for _, name := range []string{"cache", ".uploads", ".trash", "CACHE", "Cache"} {
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": name})
if w.Code != 400 || errCode(t, w) != "bad_request" || !strings.Contains(w.Body.String(), "reserved_name") {
t.Fatalf("name=%q want 400/bad_request/reserved_name got %d %s", name, w.Code, w.Body.String())
}
}
}
func TestUnit_CreateLibrary_BadName(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
for _, name := range []string{"", "..", "///"} {
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": name})
httpOK(t, w, 400, "bad name "+name)
}
}
func TestUnit_CreateLibrary_Duplicate(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "dup")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": "dup"})
httpOK(t, w, 409, "duplicate library")
if code := errCode(t, w); code != "exists" {
t.Fatalf("error code want exists got %q", code)
}
}
func TestUnit_CreateLibrary_OK(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": "new lib"})
httpOK(t, w, 201, "create library")
body := jsonBody(t, w)
if body["name"] != "new lib" {
t.Fatalf("unexpected body %v", body)
}
}
func TestUnit_ScanLibrary_NotFound(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/999/scan", atok, nil)
httpOK(t, w, 404, "scan missing library")
}
func TestUnit_ScanLibrary_Accepted(t *testing.T) {
e := newTestEnv(t)
libID, _ := e.seedLib(t, "scannable")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(libID, 10)+"/scan", atok, nil)
httpOK(t, w, 202, "scan accepted")
if !e.sc.WaitForScan(1, time.Second) {
t.Fatalf("ScanLibraryByID never called; seen=%v", e.sc.Seen())
}
if seen := e.sc.Seen(); seen[0] != libID {
t.Fatalf("scanned lib %d want %d", seen[0], libID)
}
}
func TestUnit_ScanLibrary_MemberForbidden(t *testing.T) {
e := newTestEnv(t)
libID, _ := e.seedLib(t, "nope")
mtok := e.token(t, "member", 2)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(libID, 10)+"/scan", mtok, nil)
httpOK(t, w, 403, "member scan")
}
// ---------- uploads: sentinel → status mapping ----------
func TestUnit_UploadInit_MapsSentinels(t *testing.T) {
libIDPath := func(e *testEnv) string {
id, _ := e.seedLib(t, "up")
return "/api/libraries/" + strconv.FormatInt(id, 10) + "/upload/init"
}
cases := []struct {
name string
initErr error
wantCode int
wantErr string
}{
{"too large", ports.ErrTooLarge, 413, "too_large"},
{"bad name", upload.ErrBadName, 400, "bad_request"},
{"bad format", upload.ErrBadFormat, 400, "bad_format"},
{"bad size", upload.ErrBadSize, 400, "bad_request"},
{"bad chunk", upload.ErrBadChunk, 400, "bad_request"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
e := newTestEnv(t)
e.up.InitErr = tc.initErr
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, libIDPath(e), atok,
map[string]any{"name": "b.cbz", "size": 10, "chunkSize": 4})
httpOK(t, w, tc.wantCode, "init "+tc.name)
if code := errCode(t, w); code != tc.wantErr {
t.Fatalf("error code want %q got %q", tc.wantErr, code)
}
})
}
}
func TestUnit_UploadInit_OK(t *testing.T) {
e := newTestEnv(t)
id, _ := e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(id, 10)+"/upload/init", atok,
map[string]any{"name": "b.cbz", "size": 10, "chunkSize": 4})
httpOK(t, w, 200, "init ok")
body := jsonBody(t, w)
if body["uploadId"] != e.up.UID {
t.Fatalf("uploadId want %q got %v", e.up.UID, body["uploadId"])
}
}
func TestUnit_UploadInit_BadBody(t *testing.T) {
e := newTestEnv(t)
id, _ := e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(id, 10)+"/upload/init", atok, nil)
httpOK(t, w, 400, "init without json body")
}
func TestUnit_UploadStatus_NotFound(t *testing.T) {
e := newTestEnv(t)
e.up.StatusErr = ports.ErrNotFound
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/uploads/"+e.up.UID, atok, nil)
httpOK(t, w, 404, "status missing upload")
if code := errCode(t, w); code != "not_found" {
t.Fatalf("error code want not_found got %q", code)
}
}
func TestUnit_UploadStatus_OK(t *testing.T) {
e := newTestEnv(t)
e.up.Received = []int64{0, 2}
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/uploads/"+e.up.UID, atok, nil)
httpOK(t, w, 200, "status ok")
var body struct {
Received []int64 `json:"received"`
}
if err := jsonUnmarshal(w, &body); err != nil {
t.Fatal(err)
}
if len(body.Received) != 2 || body.Received[1] != 2 {
t.Fatalf("received want [0 2] got %v", body.Received)
}
}
func TestUnit_UploadPart_BadIndex(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPut, "/api/uploads/"+e.up.UID+"/parts/abc", atok, nil)
httpOK(t, w, 400, "part bad index")
}
func TestUnit_UploadPart_TooBig(t *testing.T) {
e := newTestEnv(t)
e.up.PutErr = upload.ErrPartTooBig
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPut, "/api/uploads/"+e.up.UID+"/parts/0", atok, nil)
httpOK(t, w, 413, "part too big")
}
func TestUnit_UploadComplete_Incomplete(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "up") // fake LibraryID=1 与 seed 的第一条对齐
e.up.CompleteErr = ports.ErrIncomplete
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
httpOK(t, w, 400, "complete incomplete")
}
func TestUnit_UploadComplete_OK(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
httpOK(t, w, 202, "complete ok")
body := jsonBody(t, w)
if body["path"] != e.up.RelPath {
t.Fatalf("path want %q got %v", e.up.RelPath, body["path"])
}
}
func TestUnit_UploadComplete_LibGone(t *testing.T) {
e := newTestEnv(t) // LibraryID=1 但库里没有 id=1 → GetLibrary ErrNoRows → 404? dbErr → 500
e.up.LibID = 42
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
// dbErr 把 pgx.ErrNoRows 归为 500(非连接类),契约与真库一致
if w.Code != 500 {
t.Fatalf("lib gone want 500 got %d body=%s", w.Code, w.Body.String())
}
}
+2 -2
View File
@@ -32,7 +32,7 @@ func (h *H) PutProgress(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "locator must be valid json") err(c, http.StatusBadRequest, "bad_request", "locator must be valid json")
return return
} }
if e := h.st.UpsertProgress(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent); e != nil { if e := h.progress.UpsertProgress(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent); e != nil {
dbErr(c, e) dbErr(c, e)
return return
} }
@@ -40,7 +40,7 @@ func (h *H) PutProgress(c *gin.Context) {
} }
func (h *H) ListProgress(c *gin.Context) { func (h *H) ListProgress(c *gin.Context) {
rows, e := h.st.ListProgress(c, uid(c)) rows, e := h.progress.ListProgress(c, uid(c))
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -0,0 +1,142 @@
package handlers_test
import (
"bytes"
"encoding/json"
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
"github.com/gin-gonic/gin"
"booklib/cmd/webui/api"
"booklib/internal/auth"
"booklib/internal/config"
"booklib/internal/ports/portsfake"
)
// testEnv 是纯 fake 装配(Task 27):无 PG/Redis。请求走 api.NewRouter 组装的
// 真实 engine,因此认证/鉴权中间件、路由匹配与 gin 的 header 落盘行为都被覆盖;
// 集成测试(auth_test.go 的 setupAPI)另走真库,两层互补。
type testEnv struct {
cfg *config.Config
router *gin.Engine
users *portsfake.Users
libs *portsfake.Libraries
books *portsfake.Books
progress *portsfake.Progress
bookmarks *portsfake.Bookmarks
rl *portsfake.RateLimiter
sc *portsfake.Scanner
med *portsfake.Media
up *portsfake.Uploads
booksDir string
}
func newTestEnv(t *testing.T) *testEnv {
t.Helper()
gin.SetMode(gin.TestMode)
resolved, err := filepath.EvalSymlinks(t.TempDir())
if err != nil {
t.Fatal(err)
}
cfg := &config.Config{Addr: ":8080", JWTSecret: []byte("s3cret"), UploadMaxMB: 1,
ScanInterval: time.Minute, BooksDir: resolved, CacheDir: t.TempDir(),
TrustedProxies: []string{"172.16.0.0/12"}}
users := portsfake.NewUsers()
libs := portsfake.NewLibraries()
books := portsfake.NewBooks()
progress := portsfake.NewProgress(libs, books)
e := &testEnv{
cfg: cfg, users: users, libs: libs, books: books,
progress: progress, bookmarks: portsfake.NewBookmarks(),
rl: portsfake.NewRateLimiter(), sc: portsfake.NewScanner(),
med: portsfake.NewMedia(), up: portsfake.NewUploads(),
booksDir: resolved,
}
e.router = api.NewRouter(cfg, users, libs, books, progress, e.bookmarks,
e.rl, e.sc, e.med, e.up)
return e
}
// token 签发一个带角色的 JWT(uid 固定 1,与 fake 里 seed 的用户对应)。
func (e *testEnv) token(t *testing.T, role string, uid int64) string {
t.Helper()
tok, err := auth.Sign(e.cfg.JWTSecret, uid, role)
if err != nil {
t.Fatal(err)
}
return tok
}
// do 以 Bearer token 走完整 engine;body 非 nil 时按 JSON 发送。
func (e *testEnv) do(t *testing.T, method, path, tok string, body any) *httptest.ResponseRecorder {
t.Helper()
var reader *bytes.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
t.Fatal(err)
}
reader = bytes.NewReader(b)
} else {
reader = bytes.NewReader(nil)
}
req := httptest.NewRequest(method, path, reader)
if tok != "" {
req.Header.Set("Authorization", "Bearer "+tok)
}
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
w := httptest.NewRecorder()
e.router.ServeHTTP(w, req)
return w
}
// jsonBody 解析响应体为 map。
func jsonBody(t *testing.T, w *httptest.ResponseRecorder) map[string]any {
t.Helper()
var out map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatalf("unmarshal %q: %v", w.Body.String(), err)
}
return out
}
// jsonUnmarshal 解析响应体到任意目标(slice/struct)。
func jsonUnmarshal(w *httptest.ResponseRecorder, dst any) error {
return json.Unmarshal(w.Body.Bytes(), dst)
}
// errCode 取 error.code 字段(错误响应契约的核心)。
func errCode(t *testing.T, w *httptest.ResponseRecorder) string {
t.Helper()
body := jsonBody(t, w)
e, _ := body["error"].(map[string]any)
if e == nil {
t.Fatalf("no error object in %q", w.Body.String())
}
return fmt.Sprint(e["code"])
}
// seedLib 建一个 root 落在 BooksDir 内的库(fake 行 + 真实目录),返回 id 与 root。
func (e *testEnv) seedLib(t *testing.T, name string) (int64, string) {
t.Helper()
root := filepath.Join(e.booksDir, name)
if err := os.MkdirAll(root, 0o755); err != nil {
t.Fatal(err)
}
return e.libs.Seed(name, root), root
}
// httpOK 断言状态码,失败时带上响应体便于定位。
func httpOK(t *testing.T, w *httptest.ResponseRecorder, want int, what string) {
t.Helper()
if w.Code != want {
t.Fatalf("%s: want %d got %d body=%q", what, want, w.Code, w.Body.String())
}
}
+70 -223
View File
@@ -1,89 +1,29 @@
package handlers package handlers
import ( import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors" "errors"
"fmt"
"io"
"net/http" "net/http"
"os" "os"
"path/filepath"
"sort"
"strconv" "strconv"
"strings"
"time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"booklib/internal/bookfile" "booklib/internal/ports"
"booklib/internal/upload"
) )
// 分片上传:init(指纹→确定性 uploadId,天然支持续传)→ PUT parts → complete 拼接原子落盘。 // 分片上传:HTTP 层只做参数绑定与错误映射,域逻辑(指纹续传/分片落盘/拼接/清扫)
// 会话即 <BooksDir>/.uploads/<uid>/(meta.json + parts/N),无独立状态存储;24h 未 complete opportunistic 清扫。 // 全部在 internal/upload;会话清扫由 scanner ticker 接管(B16),不在请求路径。
const ( // maxChunkBytes 与 upload 包内常量同值,作为 PutPart 的防御性读取上限。
maxChunkBytes = 32 << 20 const maxChunkBytes = 32 << 20
defaultChunk = 8 << 20
uploadSessTTL = 24 * time.Hour
uploadSessionIn = ".uploads"
)
type uploadMeta struct {
Name string `json:"name"`
Size int64 `json:"size"`
ChunkSize int64 `json:"chunkSize"`
LibraryID int64 `json:"libraryId"`
}
func validUploadID(s string) bool {
if len(s) != 32 {
return false
}
for _, r := range s {
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')) {
return false
}
}
return true
}
func uploadIDFor(libID int64, name string, size, chunk int64) string {
h := sha256.Sum256([]byte(fmt.Sprintf("%d|%s|%d|%d", libID, name, size, chunk)))
return hex.EncodeToString(h[:16])
}
func (h *H) uploadDir(uid string) string {
return filepath.Join(filepath.Clean(h.cfg.BooksDir), uploadSessionIn, uid)
}
func chunkRange(m uploadMeta, i int64) (int64, int64) {
lo := i * m.ChunkSize
hi := min(lo+m.ChunkSize, m.Size)
return lo, hi
}
func (h *H) numParts(m uploadMeta) int64 {
return (m.Size + m.ChunkSize - 1) / m.ChunkSize
}
// sweepUploads 删除过期会话目录;尽力而为,失败不影响主流程
func (h *H) sweepUploads() {
base := filepath.Join(filepath.Clean(h.cfg.BooksDir), uploadSessionIn)
es, e := os.ReadDir(base)
if e != nil {
return
}
for _, en := range es {
if fi, e := en.Info(); e == nil && time.Since(fi.ModTime()) > uploadSessTTL {
os.RemoveAll(filepath.Join(base, en.Name()))
}
}
}
func (h *H) UploadInit(c *gin.Context) { func (h *H) UploadInit(c *gin.Context) {
lib, ok := h.getLibrary(c) id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok { if !ok {
return return
} }
@@ -99,154 +39,47 @@ func (h *H) UploadInit(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "name, size required") err(c, http.StatusBadRequest, "bad_request", "name, size required")
return return
} }
name := bookfile.SafeName(req.Name) uid, e := h.up.Init(c, lib.ID, req.Name, req.Size, req.ChunkSize)
if name == "" { if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad name") h.mapUploadErr(c, e)
return
}
if bookfile.FormatFromExt(name) == "" {
err(c, http.StatusBadRequest, "bad_format", "extension must be cbz/pdf/epub/txt/md")
return
}
if req.Size <= 0 {
err(c, http.StatusBadRequest, "bad_request", "bad size")
return
}
if req.Size > h.cfg.UploadMaxMB<<20 {
err(c, http.StatusRequestEntityTooLarge, "too_large", "file exceeds upload limit of "+strconv.FormatInt(h.cfg.UploadMaxMB, 10)+"MB")
return
}
if req.ChunkSize == 0 {
req.ChunkSize = defaultChunk
}
if req.ChunkSize > maxChunkBytes {
err(c, http.StatusBadRequest, "bad_request", "chunkSize must be <= 33554432")
return
}
uid := uploadIDFor(lib.ID, name, req.Size, req.ChunkSize)
dir := h.uploadDir(uid)
meta := uploadMeta{Name: name, Size: req.Size, ChunkSize: req.ChunkSize, LibraryID: lib.ID}
if b, e := os.ReadFile(filepath.Join(dir, "meta.json")); e == nil {
var old uploadMeta
if json.Unmarshal(b, &old) == nil && old == meta { // 同指纹 → 复用会话(续传)
c.JSON(http.StatusOK, gin.H{"uploadId": uid})
return
}
os.RemoveAll(dir) // 指纹撞上但内容不同 → 从头再来
}
h.sweepUploads()
if e := os.MkdirAll(filepath.Join(dir, "parts"), 0o755); e != nil {
err(c, http.StatusInternalServerError, "internal", "create session")
return
}
b, _ := json.Marshal(meta)
if e := os.WriteFile(filepath.Join(dir, "meta.json"), b, 0o644); e != nil {
err(c, http.StatusInternalServerError, "internal", "write meta")
return return
} }
c.JSON(http.StatusOK, gin.H{"uploadId": uid}) c.JSON(http.StatusOK, gin.H{"uploadId": uid})
} }
// loadMeta 校验 uid 与路径,404/400 已回复
func (h *H) loadMeta(c *gin.Context) (uploadMeta, string, bool) {
uid := c.Param("uid")
if !validUploadID(uid) {
err(c, http.StatusBadRequest, "bad_request", "bad upload id")
return uploadMeta{}, "", false
}
dir := h.uploadDir(uid)
var m uploadMeta
b, e := os.ReadFile(filepath.Join(dir, "meta.json"))
if e != nil {
err(c, http.StatusNotFound, "not_found", "no such upload")
return uploadMeta{}, "", false
}
if json.Unmarshal(b, &m) != nil {
err(c, http.StatusInternalServerError, "internal", "corrupt session")
return uploadMeta{}, "", false
}
return m, dir, true
}
func (h *H) UploadStatus(c *gin.Context) { func (h *H) UploadStatus(c *gin.Context) {
_, dir, ok := h.loadMeta(c) recv, e := h.up.Status(c, c.Param("uid"))
if !ok { if e != nil {
h.mapUploadErr(c, e)
return return
} }
recv := []int64{}
es, e := os.ReadDir(filepath.Join(dir, "parts"))
if e == nil {
for _, en := range es {
if i, e := strconv.ParseInt(en.Name(), 10, 64); e == nil {
recv = append(recv, i)
}
}
}
sort.Slice(recv, func(i, j int) bool { return recv[i] < recv[j] })
c.JSON(http.StatusOK, gin.H{"received": recv}) c.JSON(http.StatusOK, gin.H{"received": recv})
} }
func (h *H) UploadPart(c *gin.Context) { func (h *H) UploadPart(c *gin.Context) {
m, dir, ok := h.loadMeta(c) uid := c.Param("uid")
if !ok {
return
}
idx, e := strconv.ParseInt(c.Param("index"), 10, 64) idx, e := strconv.ParseInt(c.Param("index"), 10, 64)
if e != nil || idx < 0 || idx >= h.numParts(m) { if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad part index") err(c, http.StatusBadRequest, "bad_request", "bad part index")
return return
} }
lo, hi := chunkRange(m, idx) // maxSize 防御性上限:分片声明大小由会话 meta 决定,这里再垫一层 32MB 全局上限
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, hi-lo) e = h.up.PutPart(c, uid, idx, c.Request.Body, maxChunkBytes)
// B4: write to .tmp then rename — prevents truncated parts from being
// reported as "received" by UploadStatus if the process crashes mid-write.
p := filepath.Join(dir, "parts", strconv.FormatInt(idx, 10))
tmp := p + ".tmp"
f, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil { if e != nil {
err(c, http.StatusInternalServerError, "internal", "create part") h.mapUploadErr(c, e)
return
}
n, e := io.Copy(f, c.Request.Body)
f.Close()
if e != nil || n != hi-lo {
os.Remove(tmp)
var mbe *http.MaxBytesError
code, msg := "too_large", "part size mismatch"
if errors.As(e, &mbe) {
msg = "part exceeds declared size"
}
err(c, http.StatusRequestEntityTooLarge, code, msg)
return
}
if e := os.Rename(tmp, p); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename part")
return return
} }
c.JSON(http.StatusAccepted, gin.H{"accepted": true}) c.JSON(http.StatusAccepted, gin.H{"accepted": true})
} }
func (h *H) UploadComplete(c *gin.Context) { func (h *H) UploadComplete(c *gin.Context) {
m, dir, ok := h.loadMeta(c) uid := c.Param("uid")
if !ok { libID, e := h.up.LibraryID(c, uid)
if e != nil {
h.mapUploadErr(c, e)
return return
} }
var total int64 lib, e := h.libs.GetLibrary(c, libID)
for i := int64(0); i < h.numParts(m); i++ {
lo, hi := chunkRange(m, i)
fi, e := os.Stat(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil || fi.Size() != hi-lo {
err(c, http.StatusBadRequest, "bad_request", "upload incomplete; missing or corrupt parts, re-upload them")
return
}
total += fi.Size()
}
if total != m.Size {
err(c, http.StatusBadRequest, "bad_request", "total size mismatch")
return
}
lib, e := h.st.GetLibrary(c, m.LibraryID)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -255,39 +88,53 @@ func (h *H) UploadComplete(c *gin.Context) {
if !ok { if !ok {
return return
} }
dst, e := h.uniquePath(root, m.Name) rel, e := h.up.Complete(c, uid, root)
if e != nil { if e != nil {
err(c, http.StatusForbidden, "forbidden", e.Error()) h.mapUploadErr(c, e)
return return
} }
tmp := filepath.Join(dir, "assembled") c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": rel})
out, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644) }
if e != nil {
err(c, http.StatusInternalServerError, "internal", "create tmp") // mapUploadErr 把 upload 包的 sentinel 错误映射为原契约的 status/code/message。
return func (h *H) mapUploadErr(c *gin.Context, e error) {
} switch {
for i := int64(0); i < h.numParts(m); i++ { case errors.Is(e, ports.ErrTooLarge):
pf, e := os.Open(filepath.Join(dir, "parts", strconv.FormatInt(i, 10))) err(c, http.StatusRequestEntityTooLarge, "too_large", "file exceeds upload limit of "+strconv.FormatInt(h.cfg.UploadMaxMB, 10)+"MB")
if e != nil { case errors.Is(e, upload.ErrBadName):
out.Close() err(c, http.StatusBadRequest, "bad_request", "bad name")
err(c, http.StatusInternalServerError, "internal", "open part") case errors.Is(e, upload.ErrBadFormat):
return err(c, http.StatusBadRequest, "bad_format", "extension must be cbz/pdf/epub/txt/md")
} case errors.Is(e, upload.ErrBadSize):
if _, e := io.Copy(out, pf); e != nil { err(c, http.StatusBadRequest, "bad_request", "bad size")
pf.Close() case errors.Is(e, upload.ErrBadChunk):
out.Close() err(c, http.StatusBadRequest, "bad_request", "chunkSize must be <= 33554432")
os.Remove(tmp) case errors.Is(e, upload.ErrBadUploadID):
err(c, http.StatusInternalServerError, "internal", "assemble") err(c, http.StatusBadRequest, "bad_request", "bad upload id")
case errors.Is(e, upload.ErrBadIndex):
err(c, http.StatusBadRequest, "bad_request", "bad part index")
case errors.Is(e, upload.ErrPartTooBig):
err(c, http.StatusRequestEntityTooLarge, "too_large", "part exceeds declared size")
case errors.Is(e, upload.ErrPartSizeMismatch):
err(c, http.StatusRequestEntityTooLarge, "too_large", "part size mismatch")
case errors.Is(e, ports.ErrNotFound):
err(c, http.StatusNotFound, "not_found", "no such upload")
case errors.Is(e, upload.ErrCorrupt):
err(c, http.StatusInternalServerError, "internal", "corrupt session")
case errors.Is(e, ports.ErrIncomplete):
err(c, http.StatusBadRequest, "bad_request", "upload incomplete; missing or corrupt parts, re-upload them")
case errors.Is(e, ports.ErrSizeMismatch):
err(c, http.StatusBadRequest, "bad_request", "total size mismatch")
case errors.Is(e, os.ErrInvalid), errors.Is(e, os.ErrExist):
err(c, http.StatusForbidden, "forbidden", e.Error())
case errors.Is(e, os.ErrPermission), errors.Is(e, os.ErrClosed):
err(c, http.StatusInternalServerError, "internal", "io error")
default:
var oe *upload.OpError
if errors.As(e, &oe) {
err(c, http.StatusInternalServerError, "internal", oe.Op)
return return
} }
pf.Close() err(c, http.StatusInternalServerError, "internal", "upload failed")
}
out.Close()
if e := os.Rename(tmp, dst); e != nil { // 原子落盘,scanner 自动收编
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename")
return
} }
os.RemoveAll(dir)
c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": strings.TrimPrefix(dst, root+string(os.PathSeparator))})
} }
+7 -11
View File
@@ -3,21 +3,18 @@ package handlers
import ( import (
"errors" "errors"
"net/http" "net/http"
"strconv"
"time" "time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5"
"booklib/internal/auth" "booklib/internal/auth"
"booklib/internal/ports"
"booklib/internal/store" "booklib/internal/store"
) )
// isUnique is a convenience alias for store.IsUniqueViolation.
func isUnique(e error) bool { return store.IsUniqueViolation(e) }
func (h *H) ListUsers(c *gin.Context) { func (h *H) ListUsers(c *gin.Context) {
users, e := h.st.ListUsers(c) users, e := h.users.ListUsers(c)
if e != nil { if e != nil {
dbErr(c, e) dbErr(c, e)
return return
@@ -49,9 +46,9 @@ func (h *H) CreateUser(c *gin.Context) {
err(c, http.StatusInternalServerError, "internal", "hash") err(c, http.StatusInternalServerError, "internal", "hash")
return return
} }
id, e := h.st.CreateUser(c, req.Username, hp, req.Role) id, e := h.users.CreateUser(c, req.Username, hp, req.Role)
if e != nil { if e != nil {
if isUnique(e) { if ports.IsUniqueViolation(e) {
err(c, http.StatusConflict, "exists", "username taken") err(c, http.StatusConflict, "exists", "username taken")
return return
} }
@@ -62,9 +59,8 @@ func (h *H) CreateUser(c *gin.Context) {
} }
func (h *H) DeleteUser(c *gin.Context) { func (h *H) DeleteUser(c *gin.Context) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64) id, ok := idParam(c)
if e != nil { if !ok {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return return
} }
if id == uid(c) { if id == uid(c) {
@@ -72,7 +68,7 @@ func (h *H) DeleteUser(c *gin.Context) {
return return
} }
// B5: transactional last-admin check eliminates TOCTOU race. // B5: transactional last-admin check eliminates TOCTOU race.
if e := h.st.DeleteUser(c, id); e != nil { if e := h.users.DeleteUser(c, id); e != nil {
if errors.Is(e, pgx.ErrNoRows) { if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such user") err(c, http.StatusNotFound, "not_found", "no such user")
return return
@@ -0,0 +1,154 @@
package handlers_test
import (
"context"
"errors"
"net/http"
"strconv"
"testing"
"github.com/jackc/pgx/v5"
"booklib/internal/auth"
)
// ---------- users: admin-only CRUD branches (Task 27, portsfake) ----------
func TestUnit_ListUsers(t *testing.T) {
e := newTestEnv(t)
hash, _ := auth.HashPassword("password1234")
e.users.Seed("admin", hash, "admin")
e.users.Seed("member1", hash, "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/users", atok, nil)
httpOK(t, w, 200, "list users")
var users []map[string]any
if err := jsonUnmarshal(w, &users); err != nil {
t.Fatal(err)
}
if len(users) != 2 {
t.Fatalf("want 2 users got %d", len(users))
}
}
func TestUnit_ListUsers_MemberForbidden(t *testing.T) {
e := newTestEnv(t)
mtok := e.token(t, "member", 2)
w := e.do(t, http.MethodGet, "/api/users", mtok, nil)
httpOK(t, w, 403, "member list users")
}
func TestUnit_CreateUser_RoleValidation(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
for _, role := range []string{"superadmin", "", "Member"} {
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "u", "Password": "password1234", "Role": role})
httpOK(t, w, 400, "create user role="+role)
}
}
func TestUnit_CreateUser_ShortPassword(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "u", "Password": "short", "Role": "member"})
httpOK(t, w, 400, "short password")
}
func TestUnit_CreateUser_DuplicateName(t *testing.T) {
e := newTestEnv(t)
hash, _ := auth.HashPassword("password1234")
e.users.Seed("existing", hash, "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "existing", "Password": "password1234", "Role": "member"})
httpOK(t, w, 409, "duplicate name")
if code := errCode(t, w); code != "exists" {
t.Fatalf("error code want 'exists' got %q", code)
}
}
func TestUnit_CreateUser_OK(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "newbie", "Password": "password1234", "Role": "member"})
httpOK(t, w, 201, "create user")
body := jsonBody(t, w)
if body["username"] != "newbie" || body["role"] != "member" {
t.Fatalf("unexpected body %v", body)
}
}
func TestUnit_DeleteUser_BadID(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/notanum", atok, nil)
httpOK(t, w, 400, "bad id")
}
func TestUnit_DeleteUser_Self(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("self", "hash", "admin")
atok := e.token(t, "admin", uid) // 自己删自己
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(uid, 10), atok, nil)
httpOK(t, w, 400, "delete self")
}
func TestUnit_DeleteUser_LastAdmin(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("lastadmin", "hash", "admin")
atok := e.token(t, "admin", 999) // 另一个(不存在的)操作者
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(uid, 10), atok, nil)
httpOK(t, w, 400, "last admin")
}
func TestUnit_DeleteUser_NotFound(t *testing.T) {
e := newTestEnv(t)
e.users.Seed("other", "hash", "admin")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/99999", atok, nil)
httpOK(t, w, 404, "missing user")
}
func TestUnit_DeleteUser_OK(t *testing.T) {
e := newTestEnv(t)
e.users.Seed("admin", "hash", "admin") // 保住 last-admin 保护不触发
target := e.users.Seed("todelete", "hash", "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(target, 10), atok, nil)
httpOK(t, w, 204, "delete user")
if _, err := e.users.GetUserByID(context.Background(), target); !errors.Is(err, pgx.ErrNoRows) {
t.Fatal("user should be gone")
}
}
func TestUnit_Me_OK(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("me", "hash", "member")
mtok := e.token(t, "member", uid)
w := e.do(t, http.MethodGet, "/api/auth/me", mtok, nil)
httpOK(t, w, 200, "me")
body := jsonBody(t, w)
if body["username"] != "me" {
t.Fatalf("username want 'me' got %v", body["username"])
}
}
func TestUnit_Me_UserGone(t *testing.T) {
e := newTestEnv(t)
mtok := e.token(t, "member", 99999)
w := e.do(t, http.MethodGet, "/api/auth/me", mtok, nil)
httpOK(t, w, 401, "me after user gone")
}
func TestUnit_Me_NoToken(t *testing.T) {
e := newTestEnv(t)
w := e.do(t, http.MethodGet, "/api/auth/me", "", nil)
httpOK(t, w, 401, "me without token")
}
+7 -2
View File
@@ -12,10 +12,12 @@ import (
"booklib/cmd/webui/api" "booklib/cmd/webui/api"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/db" "booklib/internal/db"
"booklib/internal/media"
"booklib/internal/redispkg" "booklib/internal/redispkg"
"booklib/internal/scanner" "booklib/internal/scanner"
"booklib/internal/seed" "booklib/internal/seed"
"booklib/internal/store" "booklib/internal/store"
"booklib/internal/upload"
) )
func main() { func main() {
@@ -39,11 +41,14 @@ func main() {
log.Fatalf("seed: %v", err) log.Fatalf("seed: %v", err)
} }
rdb := redispkg.New(cfg.RedisURL) rdb := redispkg.New(cfg.RedisURL)
sc := scanner.New(st, cfg, rdb) med := media.New(cfg, rdb)
up := upload.New(cfg.BooksDir, cfg.UploadMaxMB)
sc := scanner.New(st, cfg, rdb, up) // B16: sweep rides the scan ticker
go sc.Run(ctx) go sc.Run(ctx)
serveErr := make(chan error, 1) serveErr := make(chan error, 1)
srv := &http.Server{Addr: cfg.Addr, Handler: api.NewRouter(cfg, st, rdb, sc), // *store.Store 同时满足 5 个 store 接口;*redispkg.R 满足 RateLimiter。
srv := &http.Server{Addr: cfg.Addr, Handler: api.NewRouter(cfg, st, st, st, st, st, rdb, sc, med, up),
ReadHeaderTimeout: 10 * time.Second} ReadHeaderTimeout: 10 * time.Second}
go func() { go func() {
log.Printf("listening on %s", cfg.Addr) log.Printf("listening on %s", cfg.Addr)
+14 -17
View File
@@ -4,7 +4,6 @@ import (
"context" "context"
"fmt" "fmt"
"io" "io"
"log"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
@@ -14,6 +13,7 @@ import (
"booklib/internal/bookfile" "booklib/internal/bookfile"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/ports"
"booklib/internal/redispkg" "booklib/internal/redispkg"
) )
@@ -23,6 +23,9 @@ type M struct {
rdb *redispkg.R rdb *redispkg.R
} }
// compile-time proof that *M satisfies the consumer-side interface.
var _ ports.Media = (*M)(nil)
func New(cfg *config.Config, rdb *redispkg.R) *M { func New(cfg *config.Config, rdb *redispkg.R) *M {
return &M{cfg: cfg, rdb: rdb} return &M{cfg: cfg, rdb: rdb}
} }
@@ -54,7 +57,7 @@ func (m *M) EnsureCover(ctx context.Context, bookID int64, format string, size,
return err return err
} }
dir := bookfile.CoverDir(m.cfg.CacheDir, bookfile.DirKey(bookID, bookfile.Hash(size, modTS))) dir := bookfile.CoverDir(m.cfg.CacheDir, bookfile.DirKey(bookID, bookfile.Hash(size, modTS)))
return writeAtomic(dir, "cover"+ext, img) return WriteAtomic(dir, "cover"+ext, img)
} }
// PageIndex returns the page list for a CBZ, using redis cache when available. // PageIndex returns the page list for a CBZ, using redis cache when available.
@@ -80,7 +83,7 @@ func (m *M) PageIndex(ctx context.Context, bookID int64, size, modTS int64, root
} }
// ChaptersOf derives chapters from a page index by grouping pages by parent directory. // ChaptersOf derives chapters from a page index by grouping pages by parent directory.
func (m *M) ChaptersOf(idx []string) []Chapter { func (m *M) ChaptersOf(idx []string) []ports.Chapter {
type grp struct { type grp struct {
dir string dir string
start int start int
@@ -102,9 +105,9 @@ func (m *M) ChaptersOf(idx []string) []Chapter {
return nil return nil
} }
titles := make(map[string]int) titles := make(map[string]int)
out := make([]Chapter, len(grps)) out := make([]ports.Chapter, len(grps))
for i, g := range grps { for i, g := range grps {
out[i] = Chapter{Title: path.Base(g.dir), Start: g.start} out[i] = ports.Chapter{Title: path.Base(g.dir), Start: g.start}
titles[out[i].Title]++ titles[out[i].Title]++
} }
for i, g := range grps { for i, g := range grps {
@@ -115,14 +118,8 @@ func (m *M) ChaptersOf(idx []string) []Chapter {
return out return out
} }
// Chapter represents a CBZ chapter.
type Chapter struct {
Title string `json:"title"`
Start int `json:"start"`
}
// EnsurePage extracts a single page to the cache. Returns the cache file path. // EnsurePage extracts a single page to the cache. Returns the cache file path.
func (m *M) EnsurePage(bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error) { func (m *M) EnsurePage(_ context.Context, bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error) {
if n >= len(idx) { if n >= len(idx) {
return "", fmt.Errorf("page %d out of range", n) return "", fmt.Errorf("page %d out of range", n)
} }
@@ -141,26 +138,26 @@ func (m *M) EnsurePage(bookID int64, size, modTS int64, root, rel string, n int,
if err != nil { if err != nil {
return "", err return "", err
} }
if err := writeAtomic(dir, strconv.Itoa(n)+ext, data); err != nil { if err := WriteAtomic(dir, strconv.Itoa(n)+ext, data); err != nil {
return "", err return "", err
} }
return dst, nil return dst, nil
} }
// writeAtomic writes data to dir/name via tmp+rename. B11: cleans tmp only on failure. // WriteAtomic writes data to dir/name via tmp+rename. B11: cleans tmp only on
func writeAtomic(dir, name string, data []byte) error { // failure, so a crash mid-write never leaves a readable half-written file.
// It does not log — callers own the context (which book, which page).
func WriteAtomic(dir, name string, data []byte) error {
if err := os.MkdirAll(dir, 0o755); err != nil { if err := os.MkdirAll(dir, 0o755); err != nil {
return err return err
} }
tmp := filepath.Join(dir, fmt.Sprintf("%s.tmp-%d", name, time.Now().UnixNano())) tmp := filepath.Join(dir, fmt.Sprintf("%s.tmp-%d", name, time.Now().UnixNano()))
if err := os.WriteFile(tmp, data, 0o644); err != nil { if err := os.WriteFile(tmp, data, 0o644); err != nil {
log.Printf("media: write %s: %v", name, err)
os.Remove(tmp) os.Remove(tmp)
return err return err
} }
dst := filepath.Join(dir, name) dst := filepath.Join(dir, name)
if err := os.Rename(tmp, dst); err != nil { if err := os.Rename(tmp, dst); err != nil {
log.Printf("media: rename %s: %v", name, err)
os.Remove(tmp) os.Remove(tmp)
return err return err
} }
+8
View File
@@ -78,6 +78,9 @@ type Scanner interface {
type UploadSessions interface { type UploadSessions interface {
Init(ctx context.Context, libID int64, name string, size, chunkSize int64) (string, error) Init(ctx context.Context, libID int64, name string, size, chunkSize int64) (string, error)
// LibraryID returns the target library recorded in the session, so the
// handler can resolve+validate the library root before Complete.
LibraryID(ctx context.Context, uploadID string) (int64, error)
Status(ctx context.Context, uploadID string) ([]int64, error) Status(ctx context.Context, uploadID string) ([]int64, error)
PutPart(ctx context.Context, uploadID string, index int64, body io.Reader, maxSize int64) error PutPart(ctx context.Context, uploadID string, index int64, body io.Reader, maxSize int64) error
Complete(ctx context.Context, uploadID string, root string) (string, error) Complete(ctx context.Context, uploadID string, root string) (string, error)
@@ -86,7 +89,12 @@ type UploadSessions interface {
} }
type Media interface { type Media interface {
// EnsureCover extracts+caches the cover if not already cached. Returns nil
// for formats without cover support.
EnsureCover(ctx context.Context, bookID int64, format string, size, modTS int64, root, rel string) error EnsureCover(ctx context.Context, bookID int64, format string, size, modTS int64, root, rel string) error
// EnsurePage extracts page n (from a pre-fetched idx) to the cache and
// returns the cache file path.
EnsurePage(ctx context.Context, bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error)
ChaptersOf(idx []string) []Chapter ChaptersOf(idx []string) []Chapter
PageIndex(ctx context.Context, bookID int64, size, modTS int64, root, rel string) ([]string, error) PageIndex(ctx context.Context, bookID int64, size, modTS int64, root, rel string) ([]string, error)
CacheBuster(size, modTS int64) string CacheBuster(size, modTS int64) string
@@ -0,0 +1,198 @@
package portsfake
import (
"context"
"fmt"
"io"
"sync"
"time"
"booklib/internal/ports"
)
// ---------- RateLimiter ----------
// RateLimiter counts IncrWindow calls per key and returns a programmable
// window value. Default: always allow (return 1).
type RateLimiter struct {
mu sync.Mutex
calls map[string]int
Result int // value returned by IncrWindow; 0 means "call count"
Hook func(key string, n int) int // optional override
}
func NewRateLimiter() *RateLimiter { return &RateLimiter{calls: map[string]int{}} }
func (r *RateLimiter) IncrWindow(_ context.Context, key string, _ time.Duration) int {
r.mu.Lock()
r.calls[key]++
n := r.calls[key]
r.mu.Unlock()
if r.Hook != nil {
return r.Hook(key, n)
}
if r.Result > 0 {
return r.Result
}
return n
}
// Calls reports how many times IncrWindow was invoked for key.
func (r *RateLimiter) Calls(key string) int {
r.mu.Lock()
defer r.mu.Unlock()
return r.calls[key]
}
// ---------- Scanner ----------
// Scanner records ScanLibraryByID calls. The handler invokes it in a goroutine,
// so reads go through the mutex; WaitForScan blocks until at least n calls
// landed (or the timeout expires) to keep tests deterministic.
type Scanner struct {
mu sync.Mutex
seen []int64
}
func NewScanner() *Scanner { return &Scanner{} }
func (s *Scanner) ScanLibraryByID(_ context.Context, id int64) {
s.mu.Lock()
s.seen = append(s.seen, id)
s.mu.Unlock()
}
// Seen returns the library ids scanned so far, in call order.
func (s *Scanner) Seen() []int64 {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]int64, len(s.seen))
copy(out, s.seen)
return out
}
// WaitForScan blocks until len(Seen()) >= n or timeout elapses; reports success.
func (s *Scanner) WaitForScan(n int, timeout time.Duration) bool {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
if len(s.Seen()) >= n {
return true
}
time.Sleep(time.Millisecond)
}
return len(s.Seen()) >= n
}
// ---------- Media ----------
// Media is a programmable stand-in for the media service. Defaults answer
// successfully with the configured fixture data; individual hooks let a test
// force one error branch without touching the others.
type Media struct {
Pages []string // PageIndex result
Chapters []ports.Chapter // ChaptersOf result
CoverErr error // EnsureCover result
IndexErr error // PageIndex result
PagePath string // EnsurePage result path (test writes the file first)
PageErr error // EnsurePage result
EnsureCoverCalls int
EnsurePageCalls int
}
func NewMedia() *Media { return &Media{} }
func (m *Media) EnsureCover(_ context.Context, _ int64, _ string, _, _ int64, _, _ string) error {
m.EnsureCoverCalls++
return m.CoverErr
}
func (m *Media) EnsurePage(_ context.Context, _ int64, _, _ int64, _, _ string, _ int, _ []string) (string, error) {
m.EnsurePageCalls++
if m.PageErr != nil {
return "", m.PageErr
}
return m.PagePath, nil
}
func (m *Media) ChaptersOf(_ []string) []ports.Chapter { return m.Chapters }
func (m *Media) PageIndex(_ context.Context, _ int64, _, _ int64, _, _ string) ([]string, error) {
if m.IndexErr != nil {
return nil, m.IndexErr
}
return m.Pages, nil
}
func (m *Media) CacheBuster(size, modTS int64) string { return fmt.Sprintf("%d-%d", size, modTS) }
// ---------- UploadSessions ----------
// Uploads is a programmable stand-in for the chunked-upload subsystem.
// Defaults simulate a happy session (uid "fakeuid…", LibraryID 1);
// hooks force error branches.
type Uploads struct {
UID string // returned by Init
LibID int64 // returned by LibraryID
Received []int64 // returned by Status
RelPath string // returned by Complete
InitErr error
LibErr error
StatusErr error
PutErr error
CompleteErr error
PutCalls []int64 // part indices passed to PutPart
SweepCalls int
UniqueCalls int
}
func NewUploads() *Uploads {
return &Uploads{UID: "0123456789abcdef0123456789abcdef", LibID: 1, RelPath: "book.cbz"}
}
func (u *Uploads) Init(_ context.Context, _ int64, _ string, _, _ int64) (string, error) {
if u.InitErr != nil {
return "", u.InitErr
}
return u.UID, nil
}
func (u *Uploads) LibraryID(_ context.Context, _ string) (int64, error) {
if u.LibErr != nil {
return 0, u.LibErr
}
return u.LibID, nil
}
func (u *Uploads) Status(_ context.Context, _ string) ([]int64, error) {
if u.StatusErr != nil {
return nil, u.StatusErr
}
return u.Received, nil
}
func (u *Uploads) PutPart(_ context.Context, _ string, index int64, body io.Reader, _ int64) error {
u.PutCalls = append(u.PutCalls, index)
if body != nil { // drain so callers using pipes don't block
io.Copy(io.Discard, body)
}
return u.PutErr
}
func (u *Uploads) Complete(_ context.Context, _, _ string) (string, error) {
if u.CompleteErr != nil {
return "", u.CompleteErr
}
return u.RelPath, nil
}
func (u *Uploads) Sweep(_ context.Context) error {
u.SweepCalls++
return nil
}
func (u *Uploads) UniquePath(root, name string) (string, error) {
u.UniqueCalls++
return root + "/" + name, nil
}
+384
View File
@@ -0,0 +1,384 @@
// Package portsfake provides hand-written, in-memory implementations of every
// interface in internal/ports. They let handler tests run without PG or Redis,
// while reproducing the real store's error semantics exactly (pgx.ErrNoRows for
// missing rows, store.ErrLastAdmin / store.ErrUniqueViolation for the guarded
// paths), so the branches under test behave as they do against the database.
//
// Fakes are safe for single-goroutine test use only; they are not locked.
package portsfake
import (
"context"
"sort"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"booklib/internal/ports"
"booklib/internal/store"
)
// uniqueViolation 复刻真 PG 的唯一约束冲突:store.IsUniqueViolation 只认
// SQLSTATE 23505 的 *pgconn.PgError(生产链路里 pgx 原样透出),fake 必须同形。
func uniqueViolation(constraint string) error {
return &pgconn.PgError{Code: "23505", ConstraintName: constraint}
}
// compile-time proof that the fakes satisfy the consumer-side interfaces.
var (
_ ports.UserStore = (*Users)(nil)
_ ports.LibraryStore = (*Libraries)(nil)
_ ports.BookStore = (*Books)(nil)
_ ports.ProgressStore = (*Progress)(nil)
_ ports.BookmarkStore = (*Bookmarks)(nil)
_ ports.RateLimiter = (*RateLimiter)(nil)
_ ports.Scanner = (*Scanner)(nil)
_ ports.Media = (*Media)(nil)
_ ports.UploadSessions = (*Uploads)(nil)
)
// now is frozen per-fake-set so CreatedAt comparisons are stable within a test.
func now() time.Time { return time.Date(2026, 9, 14, 12, 0, 0, 0, time.UTC) }
// ---------- UserStore ----------
type Users struct {
m map[int64]store.User
next int64
}
func NewUsers() *Users { return &Users{m: map[int64]store.User{}, next: 1} }
// Seed adds a user and returns its id — test convenience, not part of the port.
func (u *Users) Seed(username, hash, role string) int64 {
id := u.next
u.next++
u.m[id] = store.User{ID: id, Username: username, PasswordHash: hash, Role: role, CreatedAt: now()}
return id
}
func (u *Users) CountUsers(_ context.Context) (int, error) { return len(u.m), nil }
func (u *Users) CreateUser(_ context.Context, username, hash, role string) (int64, error) {
for _, v := range u.m {
if v.Username == username {
return 0, uniqueViolation("users_username_key")
}
}
return u.Seed(username, hash, role), nil
}
func (u *Users) GetUserByName(_ context.Context, username string) (store.User, error) {
for _, v := range u.m {
if v.Username == username {
return v, nil
}
}
return store.User{}, pgx.ErrNoRows
}
func (u *Users) GetUserByID(_ context.Context, id int64) (store.User, error) {
if v, ok := u.m[id]; ok {
return v, nil
}
return store.User{}, pgx.ErrNoRows
}
func (u *Users) ListUsers(_ context.Context) ([]store.User, error) {
out := make([]store.User, 0, len(u.m))
for _, v := range u.m {
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (u *Users) DeleteUser(_ context.Context, id int64) error {
v, ok := u.m[id]
if !ok {
return pgx.ErrNoRows
}
if v.Role == "admin" {
n := 0
for _, x := range u.m {
if x.Role == "admin" {
n++
}
}
if n <= 1 {
return store.ErrLastAdmin
}
}
delete(u.m, id)
return nil
}
// ---------- LibraryStore ----------
type Libraries struct {
m map[int64]store.Library
next int64
}
func NewLibraries() *Libraries { return &Libraries{m: map[int64]store.Library{}, next: 1} }
// Seed adds a library and returns its id — test convenience.
func (l *Libraries) Seed(name, root string) int64 {
id := l.next
l.next++
l.m[id] = store.Library{ID: id, Name: name, RootPath: root, CreatedAt: now()}
return id
}
func (l *Libraries) CreateLibrary(_ context.Context, name, root string) (int64, error) {
for _, v := range l.m {
if v.Name == name {
return 0, uniqueViolation("libraries_name_key")
}
}
return l.Seed(name, root), nil
}
func (l *Libraries) ListLibraries(_ context.Context) ([]store.Library, error) {
out := make([]store.Library, 0, len(l.m))
for _, v := range l.m {
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (l *Libraries) GetLibrary(_ context.Context, id int64) (store.Library, error) {
if v, ok := l.m[id]; ok {
return v, nil
}
return store.Library{}, pgx.ErrNoRows
}
// ---------- BookStore ----------
type Books struct {
m map[int64]store.Book
next int64
}
func NewBooks() *Books { return &Books{m: map[int64]store.Book{}, next: 1} }
// Seed adds a book and returns its id — test convenience.
func (b *Books) Seed(libID int64, path, title, format string, size, modTS int64, pageCount int) int64 {
id := b.next
b.next++
b.m[id] = store.Book{ID: id, LibraryID: libID, Path: path, Title: title, Format: format,
FileSize: size, ModTS: modTS, PageCount: pageCount, State: "ok", AddedAt: now()}
return id
}
func (b *Books) InsertBook(_ context.Context, libID int64, path, title, format string, size, modTS int64, pageCount int) (int64, error) {
return b.Seed(libID, path, title, format, size, modTS, pageCount), nil
}
func (b *Books) GetBook(_ context.Context, id int64) (store.Book, error) {
if v, ok := b.m[id]; ok {
return v, nil
}
return store.Book{}, pgx.ErrNoRows
}
func (b *Books) ListBookMeta(_ context.Context, libID int64) (map[string]store.BookMeta, error) {
out := map[string]store.BookMeta{}
for _, v := range b.m {
if v.LibraryID == libID {
out[v.Path] = store.BookMeta{ID: v.ID, Size: v.FileSize, ModTS: v.ModTS, Format: v.Format}
}
}
return out, nil
}
func (b *Books) UpdateBookFile(_ context.Context, id, size, modTS int64, pageCount int) error {
v, ok := b.m[id]
if !ok {
return pgx.ErrNoRows
}
v.FileSize, v.ModTS, v.PageCount = size, modTS, pageCount
b.m[id] = v
return nil
}
func (b *Books) DeleteBookByPath(_ context.Context, libID int64, path string) error {
for id, v := range b.m {
if v.LibraryID == libID && v.Path == path {
delete(b.m, id)
return nil
}
}
return nil
}
func (b *Books) DeleteBook(_ context.Context, id int64) error {
if _, ok := b.m[id]; !ok {
return pgx.ErrNoRows
}
delete(b.m, id)
return nil
}
func (b *Books) SetBookState(_ context.Context, id int64, state, msg string) error {
v, ok := b.m[id]
if !ok {
return pgx.ErrNoRows
}
v.State, v.ErrMsg = state, msg
b.m[id] = v
return nil
}
// ListBooks applies the same q/prefix filter as the SQL view (q matches title or
// path, prefix matches path prefix), then joins progress + library name.
func (b *Books) ListBooks(_ context.Context, libID int64, q, prefix string, userID int64) ([]store.BookView, error) {
out := []store.BookView{}
q = strings.ToLower(q)
for _, v := range b.m {
if libID != 0 && v.LibraryID != libID {
continue
}
if prefix != "" && !strings.HasPrefix(v.Path, prefix) {
continue
}
if q != "" && !strings.Contains(strings.ToLower(v.Title), q) && !strings.Contains(strings.ToLower(v.Path), q) {
continue
}
out = append(out, store.BookView{Book: v})
}
sort.Slice(out, func(i, j int) bool { return out[i].Book.ID < out[j].Book.ID })
return out, nil
}
func (b *Books) BookHashes(_ context.Context) (map[int64][2]int64, error) {
out := map[int64][2]int64{}
for _, v := range b.m {
out[v.ID] = [2]int64{v.FileSize, v.ModTS}
}
return out, nil
}
// ---------- ProgressStore ----------
type progressKey struct {
userID, libID int64
bookPath string
}
type Progress struct {
m map[progressKey]store.Progress
libs *Libraries // for LibraryName join; may be nil
books *Books // for Title join; may be nil
}
func NewProgress(libs *Libraries, books *Books) *Progress {
return &Progress{m: map[progressKey]store.Progress{}, libs: libs, books: books}
}
func (p *Progress) UpsertProgress(_ context.Context, userID, libID int64, bookPath string, locator []byte, percent float64) error {
k := progressKey{userID, libID, bookPath}
old := p.m[k]
p.m[k] = store.Progress{LibraryID: libID, BookPath: bookPath, Locator: locator,
Percent: percent, UpdatedAt: now(), LibraryName: old.LibraryName, Title: old.Title}
return nil
}
func (p *Progress) GetProgress(_ context.Context, userID, libID int64, bookPath string) (store.Progress, error) {
if v, ok := p.m[progressKey{userID, libID, bookPath}]; ok {
return v, nil
}
return store.Progress{}, pgx.ErrNoRows
}
func (p *Progress) ListProgress(_ context.Context, userID int64) ([]store.Progress, error) {
out := []store.Progress{}
for k, v := range p.m {
if k.userID != userID {
continue
}
if p.libs != nil {
if l, e := p.libs.GetLibrary(context.Background(), v.LibraryID); e == nil {
v.LibraryName = l.Name
}
}
if p.books != nil {
for _, b := range p.books.m {
if b.LibraryID == v.LibraryID && b.Path == v.BookPath {
v.Title = b.Title
break
}
}
}
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].BookPath < out[j].BookPath })
return out, nil
}
// ---------- BookmarkStore ----------
// bookmark pairs a row with its owner id. The real store carries the owner in a
// users-scoped join; the fake keeps it alongside so ListBookmarks/Patch/Delete
// can honour owner-scoped 404 semantics without a global index.
type bookmark struct {
row store.Bookmark
userID int64
}
type Bookmarks struct {
m map[int64]bookmark
next int64
}
func NewBookmarks() *Bookmarks { return &Bookmarks{m: map[int64]bookmark{}, next: 1} }
func (bm *Bookmarks) InsertBookmark(_ context.Context, userID, libID int64, bookPath string, locator []byte, percent float64, note string) (int64, error) {
id := bm.next
bm.next++
bm.m[id] = bookmark{
row: store.Bookmark{ID: id, LibraryID: libID, BookPath: bookPath,
Locator: locator, Percent: percent, Note: note, CreatedAt: now()},
userID: userID,
}
return id, nil
}
func (bm *Bookmarks) ListBookmarks(_ context.Context, userID, libID int64, bookPath string) ([]store.Bookmark, error) {
out := []store.Bookmark{}
for _, v := range bm.m {
if v.userID == userID && v.row.LibraryID == libID && v.row.BookPath == bookPath {
out = append(out, v.row)
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Percent != out[j].Percent {
return out[i].Percent < out[j].Percent
}
return out[i].ID < out[j].ID
})
return out, nil
}
func (bm *Bookmarks) UpdateBookmarkNote(_ context.Context, userID, id int64, note string) (bool, error) {
v, ok := bm.m[id]
if !ok || v.userID != userID { // owner-scoped: foreign id is a 404, not a 403
return false, nil
}
v.row.Note = note
bm.m[id] = v
return true, nil
}
func (bm *Bookmarks) DeleteBookmark(_ context.Context, userID, id int64) (bool, error) {
v, ok := bm.m[id]
if !ok || v.userID != userID {
return false, nil
}
delete(bm.m, id)
return true, nil
}
+48 -84
View File
@@ -7,7 +7,6 @@ import (
"io" "io"
"io/fs" "io/fs"
"log" "log"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync" "sync"
@@ -15,21 +14,28 @@ import (
"booklib/internal/bookfile" "booklib/internal/bookfile"
"booklib/internal/config" "booklib/internal/config"
"booklib/internal/media"
"booklib/internal/redispkg" "booklib/internal/redispkg"
"booklib/internal/store" "booklib/internal/store"
) )
// Sweeper 是 scanner 每轮顺手调用的清理钩子;upload.U 满足它(B16)。
type Sweeper interface {
Sweep(ctx context.Context) error
}
type Scanner struct { type Scanner struct {
st *store.Store st *store.Store
cfg *config.Config cfg *config.Config
rdb *redispkg.R rdb *redispkg.R
sweepers []Sweeper
// B9-②: per-library single-flight — concurrent scan triggers for the same // B9-②: per-library single-flight — concurrent scan triggers for the same
// library are merged into one execution, even without redis. // library are merged into one execution, even without redis.
flights sync.Map // map[int64]*sync.WaitGroup flights sync.Map // map[int64]*sync.WaitGroup
} }
func New(st *store.Store, cfg *config.Config, rdb *redispkg.R) *Scanner { func New(st *store.Store, cfg *config.Config, rdb *redispkg.R, sweepers ...Sweeper) *Scanner {
return &Scanner{st: st, cfg: cfg, rdb: rdb} return &Scanner{st: st, cfg: cfg, rdb: rdb, sweepers: sweepers}
} }
func (s *Scanner) Run(ctx context.Context) { func (s *Scanner) Run(ctx context.Context) {
@@ -40,6 +46,11 @@ func (s *Scanner) Run(ctx context.Context) {
case <-ctx.Done(): case <-ctx.Done():
return return
case <-t.C: case <-t.C:
for _, sw := range s.sweepers { // B16: 上传会话清扫随扫描周期跑
if err := sw.Sweep(ctx); err != nil {
log.Printf("scan: sweep: %v", err)
}
}
libs, err := s.st.ListLibraries(ctx) libs, err := s.st.ListLibraries(ctx)
if err != nil { if err != nil {
log.Printf("scan: list libraries: %v", err) log.Printf("scan: list libraries: %v", err)
@@ -86,7 +97,7 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
defer unlock() defer unlock()
root, err := filepath.EvalSymlinks(filepath.Clean(lib.RootPath)) root, err := filepath.EvalSymlinks(filepath.Clean(lib.RootPath))
if err != nil || !inside(s.cfg.BooksDir, root) { if err != nil || !bookfile.Contains(s.cfg.BooksDir, root) {
log.Printf("scan: library %d root %q rejected", lib.ID, lib.RootPath) log.Printf("scan: library %d root %q rejected", lib.ID, lib.RootPath)
return return
} }
@@ -105,9 +116,9 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
delete(dbMeta, rel) delete(dbMeta, rel)
switch { switch {
case !exists: case !exists:
s.add(ctx, lib.ID, root, rel, ds) s.ingest(ctx, lib.ID, 0, root, rel, ds, true)
case old.Size != ds.size || old.ModTS != ds.modTS: case old.Size != ds.size || old.ModTS != ds.modTS:
s.update(ctx, lib.ID, old.ID, root, rel, ds) s.ingest(ctx, lib.ID, old.ID, root, rel, ds, false)
} }
} }
for rel := range dbMeta { // 只剩被删的文件 for rel := range dbMeta { // 只剩被删的文件
@@ -120,11 +131,6 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
type diskStat struct{ size, modTS int64 } type diskStat struct{ size, modTS int64 }
func inside(booksDir, root string) bool {
b := filepath.Clean(booksDir)
return root == b || strings.HasPrefix(root, b+string(os.PathSeparator))
}
func walk(root string) (map[string]diskStat, error) { func walk(root string) (map[string]diskStat, error) {
out := map[string]diskStat{} out := map[string]diskStat{}
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error { err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
@@ -157,7 +163,9 @@ func titleOf(rel string) string {
return strings.TrimSpace(strings.ReplaceAll(strings.TrimSuffix(base, filepath.Ext(base)), "_", " ")) return strings.TrimSpace(strings.ReplaceAll(strings.TrimSuffix(base, filepath.Ext(base)), "_", " "))
} }
func (s *Scanner) add(ctx context.Context, libID int64, root, rel string, ds diskStat) { // ingest 是 add/update 的合一实现(Task 24):isNew 决定走 Insert 还是 UpdateBookFile,
// 之后的错误处理与封面生成完全共享。bookID 仅在 isNew=false 时有意义。
func (s *Scanner) ingest(ctx context.Context, libID, bookID int64, root, rel string, ds diskStat, isNew bool) {
format := bookfile.FormatFromExt(filepath.Base(rel)) format := bookfile.FormatFromExt(filepath.Base(rel))
pageCount := 0 pageCount := 0
var idxErr error var idxErr error
@@ -169,40 +177,21 @@ func (s *Scanner) add(ctx context.Context, libID int64, root, rel string, ds dis
idxErr = errors.New("no images in archive") idxErr = errors.New("no images in archive")
} }
} }
id, err := s.st.InsertBook(ctx, libID, rel, titleOf(rel), format, ds.size, ds.modTS, pageCount) if isNew {
if err != nil { id, err := s.st.InsertBook(ctx, libID, rel, titleOf(rel), format, ds.size, ds.modTS, pageCount)
log.Printf("scan: insert %s: %v", rel, err) if err != nil {
return log.Printf("scan: insert %s: %v", rel, err)
} return
if idxErr != nil {
// B10: log SetBookState errors instead of discarding.
if e := s.st.SetBookState(ctx, id, "error", idxErr.Error()); e != nil {
log.Printf("scan: SetBookState error for %s: %v", rel, e)
} }
return bookID = id
} } else if err := s.st.UpdateBookFile(ctx, bookID, ds.size, ds.modTS, pageCount); err != nil {
s.cover(ctx, id, root, rel, format, ds)
}
func (s *Scanner) update(ctx context.Context, libID, bookID int64, root, rel string, ds diskStat) {
format := bookfile.FormatFromExt(filepath.Base(rel))
pageCount := 0
var idxErr error
if format == "cbz" {
idx, err := s.zipIndex(root, rel)
pageCount = len(idx)
idxErr = err
if idxErr == nil && pageCount == 0 {
idxErr = errors.New("no images in archive")
}
}
if err := s.st.UpdateBookFile(ctx, bookID, ds.size, ds.modTS, pageCount); err != nil {
log.Printf("scan: update %s: %v", rel, err) log.Printf("scan: update %s: %v", rel, err)
return return
} }
if idxErr != nil { if idxErr != nil {
// B10: log SetBookState errors instead of discarding.
if e := s.st.SetBookState(ctx, bookID, "error", idxErr.Error()); e != nil { if e := s.st.SetBookState(ctx, bookID, "error", idxErr.Error()); e != nil {
log.Printf("scan: SetBookState error for %s: %v", rel, e) log.Printf("scan: SetBookState %s: %v", rel, e)
} }
return return
} }
@@ -210,66 +199,41 @@ func (s *Scanner) update(ctx context.Context, libID, bookID int64, root, rel str
} }
func (s *Scanner) zipIndex(root, rel string) ([]string, error) { func (s *Scanner) zipIndex(root, rel string) ([]string, error) {
f, err := os.Open(filepath.Join(root, filepath.FromSlash(rel))) f, size, err := bookfile.OpenReaderAt(root, rel)
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer f.Close() defer f.Close()
st, err := f.Stat() return bookfile.PageIndex(f, size)
if err != nil {
return nil, err
}
return bookfile.PageIndex(f, st.Size())
} }
// cover writes the cover image to the cache dir. B11: all write errors are // cover writes the cover image to the cache dir via media.WriteAtomic.
// checked and logged; orphan .tmp files are cleaned only on failure path. // B11: write failures are logged; orphan .tmp files are cleaned only on failure.
func (s *Scanner) cover(ctx context.Context, id int64, root, rel, format string, ds diskStat) { func (s *Scanner) cover(ctx context.Context, id int64, root, rel, format string, ds diskStat) {
var img []byte var fn func(io.ReaderAt, int64) ([]byte, string, error)
var ext string
var err error
switch format { switch format {
case "cbz": case "cbz":
img, ext, err = s.readCover(root, rel, bookfile.CBZCover) fn = bookfile.CBZCover
case "epub": case "epub":
img, ext, err = s.readCover(root, rel, bookfile.EPUBCover) fn = bookfile.EPUBCover
default: default:
return // pdf/txt/md 用占位 SVG,不落盘 return // pdf/txt/md 用占位 SVG,不落盘
} }
f, size, err := bookfile.OpenReaderAt(root, rel)
if err != nil {
log.Printf("scan: cover %s: %v", rel, err)
return
}
defer f.Close()
img, ext, err := fn(f, size)
if err != nil { if err != nil {
log.Printf("scan: cover %s: %v", rel, err) log.Printf("scan: cover %s: %v", rel, err)
return return
} }
dir := bookfile.CoverDir(s.cfg.CacheDir, bookfile.DirKey(id, bookfile.Hash(ds.size, ds.modTS))) dir := bookfile.CoverDir(s.cfg.CacheDir, bookfile.DirKey(id, bookfile.Hash(ds.size, ds.modTS)))
if e := os.MkdirAll(dir, 0o755); e != nil { if e := media.WriteAtomic(dir, "cover"+ext, img); e != nil {
log.Printf("scan: coverdir %s: %v", rel, e) log.Printf("scan: cover write %s: %v", rel, e)
return
} }
tmp := filepath.Join(dir, "cover"+ext+".tmp")
dst := filepath.Join(dir, "cover"+ext)
if e := os.WriteFile(tmp, img, 0o644); e != nil {
log.Printf("scan: write cover tmp %s: %v", rel, e)
os.Remove(tmp)
return
}
if e := os.Rename(tmp, dst); e != nil {
log.Printf("scan: rename cover %s: %v", rel, e)
os.Remove(tmp)
return
}
}
func (s *Scanner) readCover(root, rel string, fn func(io.ReaderAt, int64) ([]byte, string, error)) ([]byte, string, error) {
f, err := os.Open(filepath.Join(root, filepath.FromSlash(rel)))
if err != nil {
return nil, "", err
}
defer f.Close()
st, err := f.Stat()
if err != nil {
return nil, "", err
}
return fn(f, st.Size())
} }
func (s *Scanner) sweepCache(ctx context.Context) { func (s *Scanner) sweepCache(ctx context.Context) {
+1 -5
View File
@@ -2,11 +2,8 @@ package seed
import ( import (
"context" "context"
"errors"
"log" "log"
"github.com/jackc/pgx/v5/pgconn"
"booklib/internal/auth" "booklib/internal/auth"
"booklib/internal/store" "booklib/internal/store"
) )
@@ -28,8 +25,7 @@ func Admin(ctx context.Context, s *store.Store, user, pass string) error {
return err return err
} }
if _, err := s.CreateUser(ctx, user, h, "admin"); err != nil { if _, err := s.CreateUser(ctx, user, h, "admin"); err != nil {
var pgErr *pgconn.PgError if store.IsUniqueViolation(err) {
if errors.As(err, &pgErr) && pgErr.Code == "23505" {
log.Printf("seed admin %q may already exist: %v", user, err) log.Printf("seed admin %q may already exist: %v", user, err)
return nil return nil
} }
+357
View File
@@ -0,0 +1,357 @@
// Package upload owns the resumable chunked-upload subsystem and the shared
// unique-path placement helper used by both chunked and single-file uploads.
//
// A session lives at <BooksDir>/.uploads/<uid>/ (meta.json + parts/N). The uid
// is a fingerprint of (libID, name, size, chunkSize), so re-initialising the
// same file resumes the existing session instead of restarting it. Expired
// sessions are swept by the scanner ticker (B16), not on the request path.
//
// This package is HTTP-free: it returns sentinel errors that the handlers map
// onto status/code/message tuples. The prior client-facing contract is
// preserved exactly.
package upload
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
"booklib/internal/bookfile"
"booklib/internal/ports"
)
const (
maxChunkBytes = 32 << 20
defaultChunk = 8 << 20
uploadSessTTL = 24 * time.Hour
uploadSessionIn = ".uploads"
)
// Sentinel errors returned to handlers for status/code mapping. The four shared
// with ports (ErrTooLarge/ErrNotFound/ErrIncomplete/ErrSizeMismatch) live in the
// ports package so the interface contract and the implementation agree.
var (
ErrBadName = errors.New("bad name")
ErrBadFormat = errors.New("bad format")
ErrBadSize = errors.New("bad size")
ErrBadChunk = errors.New("bad chunk size")
ErrBadUploadID = errors.New("bad upload id")
ErrBadIndex = errors.New("bad part index")
ErrPartTooBig = errors.New("part exceeds declared size")
ErrPartSizeMismatch = errors.New("part size mismatch")
ErrCorrupt = errors.New("corrupt session")
)
// OpError wraps an internal filesystem/IO failure with the short operation label
// that the client-facing 500 message uses, preserving the prior contract strings
// ("create session", "write meta", "create part", ...).
type OpError struct {
Op string
Err error
}
func (e *OpError) Error() string { return e.Op + ": " + e.Err.Error() }
func (e *OpError) Unwrap() error { return e.Err }
func opErr(op string, err error) error { return &OpError{Op: op, Err: err} }
type uploadMeta struct {
Name string `json:"name"`
Size int64 `json:"size"`
ChunkSize int64 `json:"chunkSize"`
LibraryID int64 `json:"libraryId"`
}
// U is the upload subsystem. It is stateless beyond the filesystem session dir.
type U struct {
booksDir string
uploadMaxMB int64
}
// New builds the upload subsystem. booksDir is the storage root (sessions live
// under booksDir/.uploads); uploadMaxMB caps the total declared file size.
func New(booksDir string, uploadMaxMB int64) *U {
return &U{booksDir: booksDir, uploadMaxMB: uploadMaxMB}
}
// compile-time proof that *U satisfies the consumer-side interface.
var _ ports.UploadSessions = (*U)(nil)
// ---------- pure helpers ----------
func validUploadID(s string) bool {
if len(s) != 32 {
return false
}
for _, r := range s {
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')) {
return false
}
}
return true
}
func uploadIDFor(libID int64, name string, size, chunk int64) string {
h := sha256.Sum256([]byte(fmt.Sprintf("%d|%s|%d|%d", libID, name, size, chunk)))
return hex.EncodeToString(h[:16])
}
func (u *U) uploadDir(uid string) string {
return filepath.Join(filepath.Clean(u.booksDir), uploadSessionIn, uid)
}
func chunkRange(m uploadMeta, i int64) (int64, int64) {
lo := i * m.ChunkSize
hi := min(lo+m.ChunkSize, m.Size)
return lo, hi
}
func numParts(m uploadMeta) int64 {
return (m.Size + m.ChunkSize - 1) / m.ChunkSize
}
// ---------- session meta ----------
// loadMeta validates uid + reads meta.json. Returns ErrBadUploadID,
// ports.ErrNotFound, or ErrCorrupt on failure.
func (u *U) loadMeta(uid string) (uploadMeta, string, error) {
if !validUploadID(uid) {
return uploadMeta{}, "", ErrBadUploadID
}
dir := u.uploadDir(uid)
b, e := os.ReadFile(filepath.Join(dir, "meta.json"))
if e != nil {
return uploadMeta{}, "", ports.ErrNotFound
}
var m uploadMeta
if json.Unmarshal(b, &m) != nil {
return uploadMeta{}, "", ErrCorrupt
}
return m, dir, nil
}
// LibraryID returns the target library id recorded in the session, so the
// handler can resolve+validate the library root before calling Complete.
func (u *U) LibraryID(ctx context.Context, uid string) (int64, error) {
m, _, e := u.loadMeta(uid)
if e != nil {
return 0, e
}
return m.LibraryID, nil
}
// ---------- public API (satisfies ports.UploadSessions) ----------
// Init validates the declared upload, then creates or resumes a session. The
// returned uid is deterministic for a given (libID, safeName, size, chunkSize),
// so a re-init of the same file resumes; a fingerprint collision with different
// content restarts the session.
func (u *U) Init(ctx context.Context, libID int64, name string, size, chunkSize int64) (string, error) {
safe := bookfile.SafeName(name)
if safe == "" {
return "", ErrBadName
}
if bookfile.FormatFromExt(safe) == "" {
return "", ErrBadFormat
}
if size <= 0 {
return "", ErrBadSize
}
if size > u.uploadMaxMB<<20 {
return "", ports.ErrTooLarge
}
if chunkSize == 0 {
chunkSize = defaultChunk
}
if chunkSize > maxChunkBytes {
return "", ErrBadChunk
}
uid := uploadIDFor(libID, safe, size, chunkSize)
dir := u.uploadDir(uid)
meta := uploadMeta{Name: safe, Size: size, ChunkSize: chunkSize, LibraryID: libID}
if b, e := os.ReadFile(filepath.Join(dir, "meta.json")); e == nil {
var old uploadMeta
if json.Unmarshal(b, &old) == nil && old == meta { // same fingerprint → resume
return uid, nil
}
os.RemoveAll(dir) // fingerprint collided but content differs → restart
}
if e := os.MkdirAll(filepath.Join(dir, "parts"), 0o755); e != nil {
return "", opErr("create session", e)
}
b, _ := json.Marshal(meta)
if e := os.WriteFile(filepath.Join(dir, "meta.json"), b, 0o644); e != nil {
return "", opErr("write meta", e)
}
return uid, nil
}
// Status returns the sorted indices of parts already received.
func (u *U) Status(ctx context.Context, uid string) ([]int64, error) {
_, dir, e := u.loadMeta(uid)
if e != nil {
return nil, e
}
recv := []int64{}
es, e := os.ReadDir(filepath.Join(dir, "parts"))
if e == nil {
for _, en := range es {
if i, e := strconv.ParseInt(en.Name(), 10, 64); e == nil {
recv = append(recv, i)
}
}
}
sort.Slice(recv, func(i, j int) bool { return recv[i] < recv[j] })
return recv, nil
}
// PutPart writes one part via tmp+rename (B4: a truncated part is never reported
// as received). body is read up to the declared part size; reading past it yields
// ErrPartTooBig, a short read yields ErrPartSizeMismatch. maxSize, when > 0, is a
// defensive ceiling on bytes read.
func (u *U) PutPart(ctx context.Context, uid string, index int64, body io.Reader, maxSize int64) error {
m, dir, e := u.loadMeta(uid)
if e != nil {
return e
}
if index < 0 || index >= numParts(m) {
return ErrBadIndex
}
lo, hi := chunkRange(m, index)
want := hi - lo
limit := want + 1
if maxSize > 0 && maxSize+1 < limit {
limit = maxSize + 1
}
p := filepath.Join(dir, "parts", strconv.FormatInt(index, 10))
tmp := p + ".tmp"
f, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil {
return opErr("create part", e)
}
n, copyErr := io.Copy(f, io.LimitReader(body, limit))
f.Close()
if copyErr != nil {
os.Remove(tmp)
return ErrPartSizeMismatch
}
if n > want {
os.Remove(tmp)
return ErrPartTooBig
}
if n != want {
os.Remove(tmp)
return ErrPartSizeMismatch
}
if e := os.Rename(tmp, p); e != nil {
os.Remove(tmp)
return opErr("rename part", e)
}
return nil
}
// Complete verifies every part is present and correctly sized, assembles them
// into a tmp file, then atomically renames it into root under a unique name.
// It returns the path relative to root. The session dir is removed on success.
func (u *U) Complete(ctx context.Context, uid, root string) (string, error) {
m, dir, e := u.loadMeta(uid)
if e != nil {
return "", e
}
var total int64
for i := int64(0); i < numParts(m); i++ {
lo, hi := chunkRange(m, i)
fi, e := os.Stat(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil || fi.Size() != hi-lo {
return "", ports.ErrIncomplete
}
total += fi.Size()
}
if total != m.Size {
return "", ports.ErrSizeMismatch
}
dst, e := u.UniquePath(root, m.Name)
if e != nil {
return "", e // os.ErrInvalid / os.ErrExist → handler maps to 403
}
tmp := filepath.Join(dir, "assembled")
out, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil {
return "", opErr("create tmp", e)
}
for i := int64(0); i < numParts(m); i++ {
pf, e := os.Open(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil {
out.Close()
return "", opErr("open part", e)
}
_, copyErr := io.Copy(out, pf)
pf.Close()
if copyErr != nil {
out.Close()
os.Remove(tmp)
return "", opErr("assemble", copyErr)
}
}
out.Close()
if e := os.Rename(tmp, dst); e != nil { // atomic placement; scanner picks it up
os.Remove(tmp)
return "", opErr("rename", e)
}
os.RemoveAll(dir)
return strings.TrimPrefix(dst, root+string(os.PathSeparator)), nil
}
// Sweep removes session dirs untouched for longer than the TTL. Best-effort:
// a missing/unreadable base dir is not an error. Run from the scanner ticker
// (B16) instead of the request path.
func (u *U) Sweep(ctx context.Context) error {
base := filepath.Join(filepath.Clean(u.booksDir), uploadSessionIn)
es, e := os.ReadDir(base)
if e != nil {
return nil // no sessions yet
}
for _, en := range es {
if fi, e := en.Info(); e == nil && time.Since(fi.ModTime()) > uploadSessTTL {
os.RemoveAll(filepath.Join(base, en.Name()))
}
}
return nil
}
// UniquePath returns a path under root for name that does not yet exist,
// appending " (n)" on collision. The cleaned name must stay inside root.
// Shared by single-file and chunked upload completion.
func (u *U) UniquePath(root, name string) (string, error) {
ext := filepath.Ext(name)
base := strings.TrimSuffix(name, ext)
for i := 0; ; i++ {
cand := base + ext
if i > 0 {
cand = base + " (" + strconv.Itoa(i) + ")" + ext
}
p := filepath.Join(root, cand)
if filepath.Clean(p) != filepath.Join(root, filepath.Clean(cand)) ||
!strings.HasPrefix(filepath.Clean(p), root+string(os.PathSeparator)) {
return "", os.ErrInvalid
}
if _, e := os.Stat(p); os.IsNotExist(e) {
return p, nil
} else if e != nil {
return "", e
}
if i > 999 {
return "", os.ErrExist
}
}
}
+252
View File
@@ -0,0 +1,252 @@
package upload
import (
"bytes"
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"booklib/internal/ports"
)
func newU(t *testing.T) *U {
t.Helper()
dir := t.TempDir()
resolved, err := filepath.EvalSymlinks(dir)
if err != nil {
t.Fatal(err)
}
return New(resolved, 1) // 1MB cap, mirrors handler test cfg
}
func mustInit(t *testing.T, u *U, libID int64, name string, size, chunk int64) string {
t.Helper()
uid, err := u.Init(context.Background(), libID, name, size, chunk)
if err != nil {
t.Fatalf("Init(%q): %v", name, err)
}
return uid
}
func TestUploadIDForDeterministic(t *testing.T) {
a := uploadIDFor(1, "x.cbz", 100, 50)
b := uploadIDFor(1, "x.cbz", 100, 50)
c := uploadIDFor(2, "x.cbz", 100, 50)
if a != b {
t.Fatalf("same fingerprint must yield same uid: %s vs %s", a, b)
}
if a == c {
t.Fatal("different libID must yield different uid")
}
if !validUploadID(a) || validUploadID("zzz") || validUploadID(strings.Repeat("a", 31)) {
t.Fatal("validUploadID broken")
}
}
func TestInitValidation(t *testing.T) {
u := newU(t)
ctx := context.Background()
if _, err := u.Init(ctx, 1, "", 100, 0); !errors.Is(err, ErrBadName) {
t.Fatalf("empty name: want ErrBadName got %v", err)
}
if _, err := u.Init(ctx, 1, "virus.exe", 100, 0); !errors.Is(err, ErrBadFormat) {
t.Fatalf("bad ext: want ErrBadFormat got %v", err)
}
if _, err := u.Init(ctx, 1, "ok.cbz", 0, 0); !errors.Is(err, ErrBadSize) {
t.Fatalf("zero size: want ErrBadSize got %v", err)
}
if _, err := u.Init(ctx, 1, "big.cbz", 2<<20, 0); !errors.Is(err, ports.ErrTooLarge) {
t.Fatalf("oversize: want ErrTooLarge got %v", err)
}
if _, err := u.Init(ctx, 1, "ok.cbz", 100, 40<<20); !errors.Is(err, ErrBadChunk) {
t.Fatalf("huge chunk: want ErrBadChunk got %v", err)
}
// 默认 chunk 生效且 uid 稳定
uid := mustInit(t, u, 1, "ok.cbz", 100, 0)
if uid != uploadIDFor(1, "ok.cbz", 100, defaultChunk) {
t.Fatal("chunkSize=0 must default to defaultChunk in fingerprint")
}
// SafeName 清洗:path 形式取 base
if uid2 := mustInit(t, u, 1, `C:\dir\book.cbz`, 100, 0); uid2 != uploadIDFor(1, "book.cbz", 100, defaultChunk) {
t.Fatalf("windows path must be sanitized to base name, got uid %s", uid2)
}
}
func TestInitResumeAndRestart(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 1, "r.cbz", 900000, 400000)
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("x"), 400000)), 0); err != nil {
t.Fatalf("PutPart: %v", err)
}
// 同指纹 re-init → 复用会话,分片保留
uid2, err := u.Init(ctx, 1, "r.cbz", 900000, 400000)
if err != nil || uid2 != uid {
t.Fatalf("resume: want same uid, got %s err=%v", uid2, err)
}
recv, err := u.Status(ctx, uid)
if err != nil || len(recv) != 1 || recv[0] != 0 {
t.Fatalf("resume must keep parts: %v err=%v", recv, err)
}
// 同指纹位但内容不同(size 变)→ 新会话,旧目录被换掉是安全的(uid 不同)
uid3 := mustInit(t, u, 1, "r.cbz", 800000, 400000)
if uid3 == uid {
t.Fatal("different size must yield different uid")
}
}
func TestPutPartErrors(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 1, "p.cbz", 1000, 400)
if err := u.PutPart(ctx, uid, 9, bytes.NewReader(bytes.Repeat([]byte("y"), 400)), 0); !errors.Is(err, ErrBadIndex) {
t.Fatalf("index oob: want ErrBadIndex got %v", err)
}
if err := u.PutPart(ctx, uid, -1, bytes.NewReader(nil), 0); !errors.Is(err, ErrBadIndex) {
t.Fatalf("negative index: want ErrBadIndex got %v", err)
}
// 超期望体积 → ErrPartTooBig
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("y"), 500)), 0); !errors.Is(err, ErrPartTooBig) {
t.Fatalf("oversize part: want ErrPartTooBig got %v", err)
}
// 短读 → ErrPartSizeMismatch
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("y"), 300)), 0); !errors.Is(err, ErrPartSizeMismatch) {
t.Fatalf("short part: want ErrPartSizeMismatch got %v", err)
}
// 失败不留 parts/B4: 截断分片不被 Status 报告为已接收
recv, err := u.Status(ctx, uid)
if err != nil || len(recv) != 0 {
t.Fatalf("failed parts must not be received: %v err=%v", recv, err)
}
// 未知/非法 uid
if _, err := u.Status(ctx, "deadbeefdeadbeefdeadbeefdeadbeef"); !errors.Is(err, ports.ErrNotFound) {
t.Fatalf("unknown uid: want ErrNotFound got %v", err)
}
if _, err := u.Status(ctx, "zzz"); !errors.Is(err, ErrBadUploadID) {
t.Fatalf("bad uid: want ErrBadUploadID got %v", err)
}
}
func TestCompleteHappyPathAndCleanup(t *testing.T) {
u := newU(t)
ctx := context.Background()
root := filepath.Join(u.booksDir, "lib")
if err := os.MkdirAll(root, 0o755); err != nil {
t.Fatal(err)
}
content := bytes.Repeat([]byte("調教開關第二季!"), 300) // ~6.3KB multibyte
size := int64(len(content))
const chunk = int64(2000)
uid := mustInit(t, u, 7, "調教開關:第二季.zip", size, chunk)
n := int((size + chunk - 1) / chunk)
// 乱序上传:索引顺序 1,2,...,n-1,0 —— 覆盖所有分片且非递增
for step := 1; step <= n; step++ {
idx := int64(step % n)
lo := idx * chunk
hi := min(lo+chunk, size)
if err := u.PutPart(ctx, uid, idx, bytes.NewReader(content[lo:hi]), 0); err != nil {
t.Fatalf("PutPart %d: %v", idx, err)
}
}
// 缺片 → ErrIncomplete(先删一片验证)
part0 := filepath.Join(u.uploadDir(uid), "parts", "0")
saved, _ := os.ReadFile(part0)
os.Remove(part0)
if _, err := u.Complete(ctx, uid, root); !errors.Is(err, ports.ErrIncomplete) {
t.Fatalf("missing part: want ErrIncomplete got %v", err)
}
os.WriteFile(part0, saved, 0o644)
rel, err := u.Complete(ctx, uid, root)
if err != nil {
t.Fatalf("Complete: %v", err)
}
if rel != "調教開關:第二季.zip" {
t.Fatalf("rel path: %q", rel)
}
got, err := os.ReadFile(filepath.Join(root, rel))
if err != nil || !bytes.Equal(got, content) {
t.Fatalf("assembled content wrong: err=%v", err)
}
if _, err := os.Stat(u.uploadDir(uid)); !os.IsNotExist(err) {
t.Fatal("session dir must be removed after complete")
}
}
func TestCompleteUniquePathCollision(t *testing.T) {
u := newU(t)
ctx := context.Background()
root := filepath.Join(u.booksDir, "lib")
os.MkdirAll(root, 0o755)
os.WriteFile(filepath.Join(root, "dup.cbz"), []byte("existing"), 0o644)
uid := mustInit(t, u, 1, "dup.cbz", 4, 4)
if err := u.PutPart(ctx, uid, 0, bytes.NewReader([]byte("new!")), 0); err != nil {
t.Fatal(err)
}
rel, err := u.Complete(ctx, uid, root)
if err != nil {
t.Fatalf("Complete: %v", err)
}
if rel != "dup (1).cbz" {
t.Fatalf("collision must add suffix, got %q", rel)
}
}
func TestUniquePathTraversalRejected(t *testing.T) {
u := newU(t)
root := filepath.Join(u.booksDir, "lib")
if _, err := u.UniquePath(root, "../escape.cbz"); !errors.Is(err, os.ErrInvalid) {
t.Fatalf("traversal: want os.ErrInvalid got %v", err)
}
p, err := u.UniquePath(root, "ok.cbz")
if err != nil || p != filepath.Join(root, "ok.cbz") {
t.Fatalf("valid name: %q %v", p, err)
}
}
func TestSweepRemovesOnlyExpired(t *testing.T) {
u := newU(t)
ctx := context.Background()
fresh := mustInit(t, u, 1, "fresh.cbz", 10, 10)
stale := mustInit(t, u, 1, "stale.cbz", 10, 10)
// 把 stale 会话 mtime 拨到 TTL 之前
old := time.Now().Add(-uploadSessTTL - time.Hour)
dir := u.uploadDir(stale)
if err := os.Chtimes(filepath.Join(dir, "meta.json"), old, old); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(dir, old, old); err != nil {
t.Fatal(err)
}
if err := u.Sweep(ctx); err != nil {
t.Fatalf("Sweep: %v", err)
}
if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Fatal("stale session must be swept")
}
if _, err := os.Stat(u.uploadDir(fresh)); err != nil {
t.Fatalf("fresh session must survive: %v", err)
}
// 空目录/不存在 base 都不报错
os.RemoveAll(filepath.Join(u.booksDir, uploadSessionIn))
if err := u.Sweep(ctx); err != nil {
t.Fatalf("Sweep on missing base: %v", err)
}
}
func TestLibraryID(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 42, "lib.cbz", 10, 10)
id, err := u.LibraryID(ctx, uid)
if err != nil || id != 42 {
t.Fatalf("LibraryID: %d %v", id, err)
}
if _, err := u.LibraryID(ctx, "deadbeefdeadbeefdeadbeefdeadbeef"); !errors.Is(err, ports.ErrNotFound) {
t.Fatalf("unknown uid: want ErrNotFound got %v", err)
}
}
+43 -36
View File
@@ -8,6 +8,49 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab
## [Unreleased] ## [Unreleased]
### Added / 新增
- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with a periodic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk).
- API:新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,定期清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。
- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent).
- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。
- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization.
- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。
- Tests: router contract test (`TestRouterContract`) pins the full route table — any added, removed or renamed route fails the test until the expectation is updated deliberately.
- 测试:新增路由契约测试(`TestRouterContract`),锁定全量路由表——任何路由的增删改名都会使该测试失败,必须显式更新预期。
- Tests: hand-written in-memory fakes (`internal/ports/portsfake`) cover all port interfaces, enabling handler unit tests with no PG or Redis. Error semantics mirror the real store (`pgx.ErrNoRows`, `store.ErrLastAdmin`, `*pgconn.PgError{Code:23505}`), so the fakes exercise the same 404/409/400 branches as the database.
- 测试:新增手写内存 fake(`internal/ports/portsfake`),覆盖全部 port 接口,使 handler 单测无需 PG/Redis 即可运行。错误语义与真实 store 一致(`pgx.ErrNoRows`、`store.ErrLastAdmin`、`*pgconn.PgError{Code:23505}`),因此 fake 走到的是与真库相同的 404/409/400 分支。
- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions.
- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。
### Changed / 变更
- Backend restructured around hexagonal ports: HTTP handlers now depend only on small consumer-side interfaces (`internal/ports`) instead of concrete `*store.Store` / `*redispkg.R` / `*scanner.Scanner`. Domain logic moved out of handlers into `internal/media` (cover/page extraction, page index cache, atomic cache writes) and `internal/upload` (chunked session lifecycle). `cmd/webui/main.go` is the composition root; `api.NewRouter` accepts pure interfaces.
- 后端按六边形端口重构:HTTP handler 现在只依赖 `internal/ports` 中的小口径消费端接口,不再直接持有 `*store.Store` / `*redispkg.R` / `*scanner.Scanner` 等具体类型。域逻辑从 handler 下沉到 `internal/media`(封面/页抽取、页索引缓存、缓存原子写)与 `internal/upload`(分片会话生命周期)。`cmd/webui/main.go` 作为装配根,`api.NewRouter` 只收接口。
- Upload session sweep moved off the request path onto the scanner's ticker cycle (B16), so `POST /upload/init` no longer pays for a directory walk.
- 上传会话清理从请求路径移到扫描器的定时周期(B16),`POST /upload/init` 不再顺带付出一次目录遍历的开销。
- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward.
- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。
- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader.
- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。
- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`.
- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。
- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/<sanitized name>` (no client-supplied paths, validated at creation).
- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。
- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked.
- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`.
### Fixed / 修复 ### Fixed / 修复
- Rate limiter `IncrWindow` uses atomic Lua script for INCR+EXPIRE, preventing permanent IP lockout on EXPIRE failure (B1). - Rate limiter `IncrWindow` uses atomic Lua script for INCR+EXPIRE, preventing permanent IP lockout on EXPIRE failure (B1).
@@ -36,45 +79,9 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab
- 上传处理器在 `O_EXCL` 冲突时重试,处理并发同名上传(B12)。 - 上传处理器在 `O_EXCL` 冲突时重试,处理并发同名上传(B12)。
- Bookmark methods check `err` before `RowsAffected` to avoid invalid reads on query failure (B13). - Bookmark methods check `err` before `RowsAffected` to avoid invalid reads on query failure (B13).
- 书签方法先检查 `err` 再读 `RowsAffected`,避免查询失败时的无效读取(B13)。 - 书签方法先检查 `err` 再读 `RowsAffected`,避免查询失败时的无效读取(B13)。
### Changed / 变更
- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward.
- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。
### Fixed / 修复
- Serve goroutine `log.Fatalf` replaced with channel-based shutdown to preserve graceful teardown (B14). - Serve goroutine `log.Fatalf` replaced with channel-based shutdown to preserve graceful teardown (B14).
- 服务 goroutine 中的 `log.Fatalf` 改为 channel 通知方式,确保优雅关停流程不被绕过(B14)。 - 服务 goroutine 中的 `log.Fatalf` 改为 channel 通知方式,确保优雅关停流程不被绕过(B14)。
- `DATABASE_URL` is now validated at startup (required, parseable); empty `REDIS_URL` logs a clear "redis disabled" message (B15). - `DATABASE_URL` is now validated at startup (required, parseable); empty `REDIS_URL` logs a clear "redis disabled" message (B15).
- `DATABASE_URL` 在启动时校验(必填、可解析);空 `REDIS_URL` 记录明确的 "redis disabled" 日志(B15)。 - `DATABASE_URL` 在启动时校验(必填、可解析);空 `REDIS_URL` 记录明确的 "redis disabled" 日志(B15)。
- `scripts/smoke.sh` aligned with current API contract, removed ignored `root_path` field (B17). - `scripts/smoke.sh` aligned with current API contract, removed ignored `root_path` field (B17).
- `scripts/smoke.sh` 对齐当前 API 契约,移除被忽略的 `root_path` 字段(B17)。 - `scripts/smoke.sh` 对齐当前 API 契约,移除被忽略的 `root_path` 字段(B17)。
### Added / 新增
- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions.
- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。
### Added / 新增
- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent).
- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。
- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization.
- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。
- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with 24h opportunistic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk).
- API: 新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,超 24h 顺手清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。
### Changed / 变更
- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader.
- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。
- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`.
- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。
- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/<sanitized name>` (no client-supplied paths, validated at creation).
- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。
- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked.
- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`.
+16
View File
@@ -78,6 +78,22 @@ Schema changes go through the ordered migration system in `backend/internal/db/m
Local gate before each batch merge: `go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...` with dev PG+Redis running. Local gate before each batch merge: `go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...` with dev PG+Redis running.
## Backend structure
The backend is organized around consumer-side port interfaces (hexagonal style):
- `cmd/webui` — binary entry point and composition root: `main.go` builds concrete implementations (`store.Store`, `redispkg.R`, `scanner.Scanner`, `media.M`, `upload.U`) and hands them to `api.NewRouter`, which only accepts the port interfaces.
- `cmd/webui/handlers` — HTTP layer: request binding, auth/authz, error → status mapping. No SQL, no archive/file logic.
- `internal/ports` — the small interfaces handlers depend on (`UserStore`, `LibraryStore`, `BookStore`, `ProgressStore`, `BookmarkStore`, `RateLimiter`, `Scanner`, `Media`, `UploadSessions`) plus shared sentinel errors. Interfaces live on the consumer side, implementations satisfy them.
- `internal/ports/portsfake` — hand-written in-memory fakes for every port, with error semantics mirroring the real store (`pgx.ErrNoRows`, `ErrLastAdmin`, PgError 23505). Handler unit tests run against these with no PG/Redis.
- `internal/media` — cover/page extraction, page-index cache (Redis-backed), atomic cache writes.
- `internal/upload` — chunked upload session lifecycle (init/part/status/complete/sweep).
- `internal/store` — all SQL, one place.
- `internal/scanner` — library walk, ingest (add/update/delete in one pass), sweep riding the scan ticker.
- `internal/bookfile` — shared file utilities (`SafeName`, `Contains`, `Hash`, `FormatFromExt`, cache dir layout).
Testing is two-tiered: integration tests hit a real PG+Redis via the full router (`handlers/*_test.go` with `setupAPI`), unit tests hit the same router with `portsfake` injected (`handlers/*_unit_test.go`). The route table itself is pinned by `TestRouterContract` in `cmd/webui/api`.
## CI ## CI
- Workflow: `.github/workflows/ci.yml` (standard GitHub Actions syntax, Gitea Actions compatible). - Workflow: `.github/workflows/ci.yml` (standard GitHub Actions syntax, Gitea Actions compatible).
+16
View File
@@ -78,6 +78,22 @@ Schema 变更通过 `backend/internal/db/migrations/` 中的有序迁移系统
每批合入前的本地门禁:`go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...`(需启动 dev PG+Redis)。 每批合入前的本地门禁:`go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...`(需启动 dev PG+Redis)。
## 后端结构
后端按消费端接口组织(六边形风格):
- `cmd/webui` —— 二进制入口与装配根:`main.go` 构造具体实现(`store.Store`、`redispkg.R`、`scanner.Scanner`、`media.M`、`upload.U`)并交给 `api.NewRouter`,后者只接受 port 接口。
- `cmd/webui/handlers` —— HTTP 层:参数绑定、认证/鉴权、错误→状态码映射。没有 SQL,没有压缩包/文件逻辑。
- `internal/ports` —— handlers 依赖的小口径接口(`UserStore`、`LibraryStore`、`BookStore`、`ProgressStore`、`BookmarkStore`、`RateLimiter`、`Scanner`、`Media`、`UploadSessions`)与共享 sentinel 错误。接口定义在消费端,实现方来满足它们。
- `internal/ports/portsfake` —— 全部 port 的手写内存 fake,错误语义与真实 store 一致(`pgx.ErrNoRows`、`ErrLastAdmin`、PgError 23505)。handler 单测无需 PG/Redis。
- `internal/media` —— 封面/页抽取、页索引缓存(Redis)、缓存原子写。
- `internal/upload` —— 分片上传会话生命周期(init/part/status/complete/sweep)。
- `internal/store` —— 所有 SQL,集中一处。
- `internal/scanner` —— 书库遍历、ingest(增/改/删一趟完成)、会话清扫搭扫描 ticker 顺风车。
- `internal/bookfile` —— 共享文件工具(`SafeName`、`Contains`、`Hash`、`FormatFromExt`、缓存目录布局)。
测试分两层:集成测试走真实 PG+Redis、过完整 router(`handlers/*_test.go` 的 `setupAPI`);单测注入 `portsfake`、过同一个 router(`handlers/*_unit_test.go`)。路由表本身由 `cmd/webui/api` 的 `TestRouterContract` 钉死。
## CI ## CI
- 工作流:`.github/workflows/ci.yml`(标准 GitHub Actions 语法,兼容 Gitea Actions)。 - 工作流:`.github/workflows/ci.yml`(标准 GitHub Actions 语法,兼容 Gitea Actions)。