Compare commits

...
6 Commits
Author SHA1 Message Date
XingfenD 8305af9d5c docs: changelog + README backend structure for batch C (Task 29)
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
- CHANGELOG [Unreleased]: consolidated the duplicated Added/Changed/Fixed
  groups left by earlier batches into one group each (no entry dropped);
  added batch C entries — port-based restructure, media/upload domain
  packages, sweep moved to scanner ticker (B16), router contract test,
  portsfake unit-test layer. Upload-sweep wording no longer promises the
  old 24h opportunistic request-path behaviour.
- README.md / README_zh.md: new 'Backend structure' section documenting
  the port/fake layout (cmd/webui composition root, handlers as HTTP-only,
  internal/ports + portsfake, media/upload/store/scanner/bookfile
  responsibilities) and the two-tier testing approach (real PG+Redis
  integration vs fake-injected unit, route table pinned by contract test)

Gate: gofmt clean, go vet clean, go test -p 1 all pass (0 skip),
scripts/smoke.sh ALL SMOKE TESTS PASSED against a live webui on :18080
2026-09-14 23:37:36 +08:00
XingfenD 85c61d9f24 refactor: consolidate isUnique and path validation helpers (Task 28)
- seed.Admin: hand-rolled pgconn.PgError 23505 check → store.IsUniqueViolation
  (single predicate for unique violations across the codebase)
- books.absBookPath: local hasPrefixDir (filepath.Rel-based) removed in favor
  of bookfile.Contains, which also resolves symlinks — stricter escape check
- handlers isUnique alias + libraries.go local prefix check were already
  folded into ports.IsUniqueViolation / bookfile.Contains in Task 25;
  scanner.inside was folded in Task 24 — this commit closes the last two

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 23:26:16 +08:00
XingfenD cc1470f6e4 test: portsfake + router contract test + handler unit tests (Task 27)
- internal/ports/portsfake: hand-written in-memory fakes for all 9 ports
  (Users/Libraries/Books/Progress/Bookmarks/RateLimiter/Scanner/Media/Uploads);
  error semantics mirror the real store exactly — pgx.ErrNoRows for misses,
  store.ErrLastAdmin guard, and *pgconn.PgError{Code:23505} for unique
  violations (ports.IsUniqueViolation only accepts the PgError shape, so the
  fakes must produce it to exercise the 409 branch without touching prod code)
- Media/Uploads fakes are hook-programmable: one field per error branch, so a
  test can force e.g. CompleteErr=ErrIncomplete without stubbing the rest
- Scanner fake is mutex-guarded + WaitForScan: handler fires ScanLibraryByID
  in a goroutine, tests stay deterministic
- router_test.go: TestRouterContract pins all 27 routes — any route table
  change now fails the test explicitly
- handler unit tests (~30 cases, no PG/Redis): users CRUD branches (self-delete
  400, last-admin 400, dup 409, 204 ok), library reserved names (contract:
  code=bad_request message=reserved_name, per original impl), upload sentinel
  mapping (413/400/404 per branch)
- NewRouter takes pure port interfaces; main.go distributes *store.Store
  across the 5 store ports at the composition root

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 23:23:10 +08:00
XingfenD 7baadedeb4 refactor(handlers): consume port interfaces, wire media+upload assembly (Task 25+26)
- H now holds only small port interfaces (UserStore/LibraryStore/BookStore/
  ProgressStore/BookmarkStore/RateLimiter/Scanner/Media/UploadSessions);
  NewRouter is the composition root distributing *store.Store and *redispkg.R
- ports.Media gains EnsurePage; ChaptersOf returns ports.Chapter (media's
  local duplicate dropped); *media.M now provably satisfies ports.Media;
  ports.UploadSessions gains LibraryID for root validation before Complete
- content.go: duplicated page-index cache + cover self-heal + page extract
  logic removed in favor of media service — same redis keys, same contract;
  path traversal check stays in handler (403 semantics preserved)
- getLibrary/getLibRow merged into getLib(c, id); idParam helper dedupes
  :id parsing; isUnique replaced by ports.IsUniqueViolation (Task 28 partial)
- main.go assembles media + upload and passes upload.U as scanner Sweeper

Full gate green: gofmt, vet, go test -p 1 (real PG+Redis, 0 skip)
2026-09-14 22:56:15 +08:00
XingfenD b9b7022212 refactor(scanner): merge add/update into ingest, use shared utilities
- add/update consolidated into ingest(isNew) — persist branch differs,
  error handling and cover path fully shared (Task 24)
- zipIndex/cover use bookfile.OpenReaderAt instead of hand-rolled
  open+stat pairs
- root containment check uses bookfile.Contains (local inside removed)
- cover write goes through media.WriteAtomic (now exported, log-free —
  callers own context); media no longer logs inside the atomic helper
2026-09-14 22:12:30 +08:00
XingfenD fbd5cd243d feat(upload): extract upload subsystem + move sweep to scanner ticker (B16)
- internal/upload owns chunked-upload domain logic (fingerprint resume,
  part tmp+rename, assemble, session sweep) and the shared UniquePath
  helper used by both single-file and chunked completion paths
- handlers/uploads.go is now HTTP-only: bind params, call upload.U, map
  sentinel errors to the unchanged status/code/message contract
- B16: session sweep moved out of the UploadInit request path onto the
  scanner ticker via a Sweeper hook (upload.U satisfies it)
- unit tests for the package without PG/Redis; contract pinned by the
  existing handler integration tests (all green)
2026-09-14 22:00:25 +08:00
27 changed files with 2176 additions and 647 deletions
+17 -15
View File
@@ -7,14 +7,16 @@ import (
"booklib/cmd/webui/handlers"
"booklib/internal/config"
"booklib/internal/redispkg"
"booklib/internal/scanner"
"booklib/internal/store"
"booklib/internal/ports"
)
func NewRouter(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner.Scanner) *gin.Engine {
// NewRouter 只依赖 port 接口:main.go 传具体实现(*store.Store 满足 5 个 store
// 接口,*redispkg.R 满足 RateLimiter),测试传 portsfake。
func NewRouter(cfg *config.Config, users ports.UserStore, libs ports.LibraryStore, books ports.BookStore,
progress ports.ProgressStore, bookmarks ports.BookmarkStore, rl ports.RateLimiter,
sc ports.Scanner, med ports.Media, up ports.UploadSessions) *gin.Engine {
gin.SetMode(gin.ReleaseMode)
h := handlers.New(cfg, st, rdb, sc)
h := handlers.New(cfg, users, libs, books, progress, bookmarks, rl, sc, med, up)
r := gin.New()
if e := r.SetTrustedProxies(cfg.TrustedProxies); e != nil {
panic(e)
@@ -27,17 +29,17 @@ func NewRouter(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner
p := g.Group("", h.AuthMw())
p.GET("/auth/me", h.Me)
users := p.Group("/users", h.AdminOnly())
users.GET("", h.ListUsers)
users.POST("", h.CreateUser)
users.DELETE("/:id", h.DeleteUser)
usersGrp := p.Group("/users", h.AdminOnly())
usersGrp.GET("", h.ListUsers)
usersGrp.POST("", h.CreateUser)
usersGrp.DELETE("/:id", h.DeleteUser)
libs := p.Group("/libraries")
libs.GET("", h.ListLibraries)
libs.POST("", h.AdminOnly(), h.CreateLibrary)
libs.POST("/:id/scan", h.AdminOnly(), h.ScanLibrary)
libs.POST("/:id/upload", h.AdminOnly(), h.Upload)
libs.POST("/:id/upload/init", h.AdminOnly(), h.UploadInit)
libsGrp := p.Group("/libraries")
libsGrp.GET("", h.ListLibraries)
libsGrp.POST("", h.AdminOnly(), h.CreateLibrary)
libsGrp.POST("/:id/scan", h.AdminOnly(), h.ScanLibrary)
libsGrp.POST("/:id/upload", h.AdminOnly(), h.Upload)
libsGrp.POST("/:id/upload/init", h.AdminOnly(), h.UploadInit)
uploads := p.Group("/uploads", h.AdminOnly())
uploads.GET("/:uid", h.UploadStatus)
+54 -2
View File
@@ -7,7 +7,6 @@ import (
"time"
"booklib/internal/config"
"booklib/internal/redispkg"
)
func testCfg() *config.Config {
@@ -16,7 +15,7 @@ func testCfg() *config.Config {
}
func TestHealthz(t *testing.T) {
r := NewRouter(testCfg(), nil, redispkg.New(""), nil)
r := NewRouter(testCfg(), nil, nil, nil, nil, nil, nil, nil, nil, nil)
req := httptest.NewRequest(http.MethodGet, "/api/healthz", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
@@ -24,3 +23,56 @@ func TestHealthz(t *testing.T) {
t.Fatalf("healthz = %d, want 200", w.Code)
}
}
// TestRouterContract 钉死全量路由表(Task 27):增删改任何路由都会使此测试失败,
// 必须显式更新——路由即对外契约。
func TestRouterContract(t *testing.T) {
r := NewRouter(testCfg(), nil, nil, nil, nil, nil, nil, nil, nil, nil)
want := map[string]bool{
"GET /api/healthz": true,
"POST /api/auth/login": true,
"GET /api/auth/me": true,
"GET /api/users": true,
"POST /api/users": true,
"DELETE /api/users/:id": true,
"GET /api/libraries": true,
"POST /api/libraries": true,
"POST /api/libraries/:id/scan": true,
"POST /api/libraries/:id/upload": true,
"POST /api/libraries/:id/upload/init": true,
"GET /api/uploads/:uid": true,
"POST /api/uploads/:uid/complete": true,
"PUT /api/uploads/:uid/parts/:index": true,
"GET /api/books": true,
"GET /api/books/:id": true,
"DELETE /api/books/:id": true,
"GET /api/books/:id/cover": true,
"GET /api/books/:id/file": true,
"GET /api/books/:id/pages": true,
"GET /api/books/:id/pages/:n": true,
"PUT /api/books/:id/progress": true,
"GET /api/progress": true,
"GET /api/books/:id/bookmarks": true,
"POST /api/books/:id/bookmarks": true,
"PATCH /api/bookmarks/:id": true,
"DELETE /api/bookmarks/:id": true,
}
got := map[string]bool{}
for _, rt := range r.Routes() {
key := rt.Method + " " + rt.Path
if got[key] {
t.Errorf("duplicate route %s", key)
}
got[key] = true
}
for k := range want {
if !got[k] {
t.Errorf("missing route %s", k)
}
}
for k := range got {
if !want[k] {
t.Errorf("unexpected route %s", k)
}
}
}
+3 -3
View File
@@ -20,11 +20,11 @@ func (h *H) Login(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "username and password required")
return
}
if n := h.rdb.IncrWindow(c, "loginrl:"+c.ClientIP(), loginWindow); n > loginMax {
if n := h.rl.IncrWindow(c, "loginrl:"+c.ClientIP(), loginWindow); n > loginMax {
err(c, http.StatusTooManyRequests, "rate_limited", "too many login attempts")
return
}
u, qerr := h.st.GetUserByName(c, req.Username)
u, qerr := h.users.GetUserByName(c, req.Username)
if qerr != nil {
if !errors.Is(qerr, pgx.ErrNoRows) {
dbErr(c, qerr)
@@ -49,7 +49,7 @@ func (h *H) Login(c *gin.Context) {
func (h *H) Me(c *gin.Context) {
// B7: only no-rows → 401; other errors (PG down) go through dbErr → 503.
u, qerr := h.st.GetUserByID(c, uid(c))
u, qerr := h.users.GetUserByID(c, uid(c))
if qerr != nil {
if errors.Is(qerr, pgx.ErrNoRows) {
err(c, http.StatusUnauthorized, "unauthorized", "no such user")
+6 -2
View File
@@ -18,9 +18,11 @@ import (
"booklib/internal/auth"
"booklib/internal/config"
"booklib/internal/db"
"booklib/internal/media"
"booklib/internal/redispkg"
"booklib/internal/scanner"
"booklib/internal/store"
"booklib/internal/upload"
)
func testCfg() *config.Config {
@@ -57,8 +59,10 @@ func setupAPI(t *testing.T) (*store.Store, *scanner.Scanner, http.Handler, strin
cfg.BooksDir = booksDir
cfg.CacheDir = t.TempDir()
rdb := redispkg.New(os.Getenv("REDIS_URL"))
sc := scanner.New(st, cfg, rdb)
r := api.NewRouter(cfg, st, rdb, sc)
med := media.New(cfg, rdb)
up := upload.New(cfg.BooksDir, cfg.UploadMaxMB)
sc := scanner.New(st, cfg, rdb, up)
r := api.NewRouter(cfg, st, st, st, st, st, rdb, sc, med, up)
if u := os.Getenv("REDIS_URL"); u != "" { // 测试卫生: 共享 redis 上重置登录限流桶, 防跨测试累计 429
if opt, e := redis.ParseURL(u); e == nil {
rc := redis.NewClient(opt)
+4 -4
View File
@@ -27,7 +27,7 @@ func (h *H) ListBookmarks(c *gin.Context) {
if !ok {
return
}
rows, e := h.st.ListBookmarks(c, uid(c), b.LibraryID, b.Path)
rows, e := h.bookmarks.ListBookmarks(c, uid(c), b.LibraryID, b.Path)
if e != nil {
dbErr(c, e)
return
@@ -65,7 +65,7 @@ func (h *H) CreateBookmark(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)")
return
}
id, e := h.st.InsertBookmark(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent, req.Note)
id, e := h.bookmarks.InsertBookmark(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent, req.Note)
if e != nil {
dbErr(c, e)
return
@@ -101,7 +101,7 @@ func (h *H) PatchBookmark(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "note too long (max 500 characters)")
return
}
updated, e := h.st.UpdateBookmarkNote(c, uid(c), id, req.Note)
updated, e := h.bookmarks.UpdateBookmarkNote(c, uid(c), id, req.Note)
if e != nil {
dbErr(c, e)
return
@@ -118,7 +118,7 @@ func (h *H) DeleteBookmark(c *gin.Context) {
if !ok {
return
}
deleted, e := h.st.DeleteBookmark(c, uid(c), id)
deleted, e := h.bookmarks.DeleteBookmark(c, uid(c), id)
if e != nil {
dbErr(c, e)
return
+11 -30
View File
@@ -7,7 +7,6 @@ import (
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
@@ -18,7 +17,7 @@ import (
)
func (h *H) getBookRow(c *gin.Context, id int64) (store.Book, bool) {
b, e := h.st.GetBook(c, id)
b, e := h.books.GetBook(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such book")
@@ -31,41 +30,23 @@ func (h *H) getBookRow(c *gin.Context, id int64) (store.Book, bool) {
}
func (h *H) bookFromParam(c *gin.Context) (store.Book, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
id, ok := idParam(c)
if !ok {
return store.Book{}, false
}
return h.getBookRow(c, id)
}
func (h *H) getLibRow(c *gin.Context, id int64) (store.Library, bool) {
l, e := h.st.GetLibrary(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return l, true
}
// absBookPath: books.path 永远相对且不含 ..;拼接后二次前缀校验(纵深防御)
// absBookPath: books.path 永远相对且不含 ..;拼接后二次前缀校验(纵深防御)。
// bookfile.Contains 带 EvalSymlinks,比裸 filepath.Rel 更能拦住软链逃逸。
func absBookPath(root string, b store.Book) (string, error) {
abs := filepath.Join(root, filepath.FromSlash(b.Path))
if filepath.Clean(abs) != abs || !hasPrefixDir(abs, root) {
if filepath.Clean(abs) != abs || !bookfile.Contains(root, abs) {
return "", os.ErrPermission
}
return abs, nil
}
func hasPrefixDir(p, dir string) bool {
rel, err := filepath.Rel(filepath.Clean(dir), filepath.Clean(p))
return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(os.PathSeparator))
}
func bookJSON(b store.Book, percent float64, libraryName string) gin.H {
h := bookfile.Hash(b.FileSize, b.ModTS)
j := gin.H{
@@ -89,7 +70,7 @@ func bookJSON(b store.Book, percent float64, libraryName string) gin.H {
func (h *H) ListBooks(c *gin.Context) {
libID, _ := strconv.ParseInt(c.Query("library"), 10, 64)
views, e := h.st.ListBooks(c, libID, c.Query("q"), c.Query("prefix"), uid(c))
views, e := h.books.ListBooks(c, libID, c.Query("q"), c.Query("prefix"), uid(c))
if e != nil {
dbErr(c, e)
return
@@ -106,12 +87,12 @@ func (h *H) GetBook(c *gin.Context) {
if !ok {
return
}
p, e := h.st.GetProgress(c, uid(c), b.LibraryID, b.Path) // ErrNoRows → 零值 percent
p, e := h.progress.GetProgress(c, uid(c), b.LibraryID, b.Path) // ErrNoRows → 零值 percent
if e != nil && !errors.Is(e, pgx.ErrNoRows) {
dbErr(c, e)
return
}
lib, e := h.st.GetLibrary(c, b.LibraryID)
lib, e := h.libs.GetLibrary(c, b.LibraryID)
if e != nil && !errors.Is(e, pgx.ErrNoRows) { // 库被并发删则留空 library 名,书仍可见
dbErr(c, e)
return
@@ -124,7 +105,7 @@ func (h *H) DeleteBook(c *gin.Context) {
if !ok {
return
}
lib, ok := h.getLibRow(c, b.LibraryID)
lib, ok := h.getLib(c, b.LibraryID)
if !ok {
return
}
@@ -144,7 +125,7 @@ func (h *H) DeleteBook(c *gin.Context) {
key := bookfile.DirKey(b.ID, bookfile.Hash(b.FileSize, b.ModTS))
os.RemoveAll(bookfile.CoverDir(h.cfg.CacheDir, key))
os.RemoveAll(bookfile.PagesDir(h.cfg.CacheDir, key))
if e := h.st.DeleteBook(c, b.ID); e != nil {
if e := h.books.DeleteBook(c, b.ID); e != nil {
dbErr(c, e)
return
}
+67 -148
View File
@@ -1,15 +1,13 @@
package handlers
import (
"fmt"
"log"
"errors"
"io/fs"
"net/http"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
@@ -20,7 +18,7 @@ import (
const defaultCover = `<svg xmlns="http://www.w3.org/2000/svg" width="120" height="170"><rect width="120" height="170" rx="6" fill="#2a2a33"/><path d="M30 25h60v120H30z" fill="#3a3a45"/><path d="M30 25h60M60 25v120" stroke="#555" stroke-width="2"/></svg>`
func (h *H) bookRoot(c *gin.Context, b store.Book) (string, bool) {
lib, ok := h.getLibRow(c, b.LibraryID)
lib, ok := h.getLib(c, b.LibraryID)
if !ok {
return "", false
}
@@ -31,6 +29,26 @@ func (h *H) immutable(c *gin.Context) {
c.Header("Cache-Control", "public, max-age=31536000, immutable")
}
// checkPath 纵深防御:path 越界 → 403,与原契约一致。
func checkPath(c *gin.Context, root string, b store.Book) (string, bool) {
abs, perr := absBookPath(root, b)
if perr != nil {
err(c, http.StatusForbidden, "forbidden", "unsafe path")
return "", false
}
return abs, true
}
// mapContentErr 把 media/bookfile 的文件级失败映射回原契约状态码:
// 文件不在盘上 → 404,其余(坏包等)由调用方决定 422/500。
func mapContentErr(c *gin.Context, e error) bool {
if errors.Is(e, fs.ErrNotExist) {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return true
}
return false
}
func (h *H) ServeCover(c *gin.Context) {
b, ok := h.bookFromParam(c)
if !ok {
@@ -48,36 +66,30 @@ func (h *H) ServeCover(c *gin.Context) {
}
if b.Format == "cbz" || b.Format == "epub" { // 自愈:缓存丢了就地抽封面(重启/卷漂移/扫描器还没跑到)
if root, ok := h.bookRoot(c, b); ok {
if f, size, ok := h.openBook(c, b, root); ok {
defer f.Close()
var img []byte
var ext string
var e error
if b.Format == "cbz" {
img, ext, e = bookfile.CBZCover(f, size)
if _, ok := checkPath(c, root, b); ok {
e := h.med.EnsureCover(c, b.ID, b.Format, b.FileSize, b.ModTS, root, b.Path)
switch {
case e == nil:
if entries, re := os.ReadDir(dir); re == nil {
for _, en := range entries {
if !strings.Contains(en.Name(), ".tmp") {
http.ServeFile(c.Writer, c.Request, filepath.Join(dir, en.Name()))
return
}
}
}
case mapContentErr(c, e):
return // 404 已回复
}
// 抽取失败(坏包)→ 落到占位 SVG,与原契约一致
} else {
img, ext, e = bookfile.EPUBCover(f, size)
return // 403 已回复
}
if e == nil {
dst := filepath.Join(dir, "cover"+ext)
if e := os.MkdirAll(dir, 0o755); e == nil {
tmp := fmt.Sprintf("%s.tmp-%d", dst, time.Now().UnixNano())
// B11: check all write errors; clean tmp only on failure.
if we := os.WriteFile(tmp, img, 0o644); we != nil {
log.Printf("serve: write cover tmp: %v", we)
os.Remove(tmp)
} else if re := os.Rename(tmp, dst); re != nil {
log.Printf("serve: rename cover: %v", re)
os.Remove(tmp)
} else {
http.ServeFile(c.Writer, c.Request, dst)
return // 404/503 已回复
}
}
}
}
}
}
if c.Writer.Written() { // openBook/bookRoot 已写 403/404/500,不再叠加占位图
if c.Writer.Written() {
return
}
c.Data(http.StatusOK, "image/svg+xml", []byte(defaultCover))
@@ -92,55 +104,32 @@ func (h *H) ServeFile(c *gin.Context) {
if !ok {
return
}
abs, perr := absBookPath(root, b)
if perr != nil {
err(c, http.StatusForbidden, "forbidden", "unsafe path")
abs, ok := checkPath(c, root, b)
if !ok {
return
}
c.Header("ETag", `"`+bookfile.Hash(b.FileSize, b.ModTS)+`"`)
if _, e := os.Stat(abs); e != nil {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return
}
c.Header("ETag", `"`+h.med.CacheBuster(b.FileSize, b.ModTS)+`"`)
c.Header("Cache-Control", "private, must-revalidate")
http.ServeFile(c.Writer, c.Request, abs)
}
func (h *H) openBook(c *gin.Context, b store.Book, root string) (*os.File, int64, bool) {
abs, perr := absBookPath(root, b)
if perr != nil {
err(c, http.StatusForbidden, "forbidden", "unsafe path")
return nil, 0, false
// pageIndex 走 media(redis 缓存 + 索引提取);路径校验仍在 handler,保住 403 契约。
func (h *H) pageIndex(c *gin.Context, b store.Book, root string) ([]string, bool) {
if _, ok := checkPath(c, root, b); !ok {
return nil, false
}
f, perr := os.Open(abs)
if perr != nil {
err(c, http.StatusNotFound, "not_found", "file missing on disk")
return nil, 0, false
}
st, perr := f.Stat()
if perr != nil {
f.Close()
err(c, http.StatusInternalServerError, "internal", "stat")
return nil, 0, false
}
return f, st.Size(), true
}
func (h *H) pageIndex(c *gin.Context, b store.Book, root string) ([]string, error) {
hash := bookfile.Hash(b.FileSize, b.ModTS)
key := fmt.Sprintf("pagesidx2:%d:%s", b.ID, hash) // 前缀换代 = 索引逻辑变更时一次性作废旧缓存
if v, ok := h.rdb.Get(c, key); ok && v != "" {
return strings.Split(v, "\n"), nil
}
f, size, ok := h.openBook(c, b, root)
if !ok {
return nil, os.ErrNotExist
}
defer f.Close()
idx, e := bookfile.PageIndex(f, size)
idx, e := h.med.PageIndex(c, b.ID, b.FileSize, b.ModTS, root, b.Path)
if e != nil {
return nil, e
if !mapContentErr(c, e) {
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
}
if len(idx) > 0 { // 空索引不缓存,否则 warm 命中 "" 会 Split 出幽灵页
h.rdb.Set(c, key, strings.Join(idx, "\n"), 7*24*time.Hour)
return nil, false
}
return idx, nil
return idx, true
}
func (h *H) PagesCount(c *gin.Context) {
@@ -156,56 +145,11 @@ func (h *H) PagesCount(c *gin.Context) {
if !ok {
return
}
idx, e := h.pageIndex(c, b, root)
if e != nil {
if c.Writer.Written() {
return // openBook 已写 403/404,不再叠加 422
}
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
idx, ok := h.pageIndex(c, b, root)
if !ok {
return
}
c.JSON(http.StatusOK, gin.H{"count": len(idx), "chapters": chaptersOf(idx)})
}
type cbzChapter struct {
Title string `json:"title"`
Start int `json:"start"`
}
// chaptersOf 页索引已自然序排列,按父目录分组:每个新目录开一章,标题取目录名;扁平包或只有一组返回 nil
func chaptersOf(idx []string) []cbzChapter {
type grp struct {
dir string
start int
}
var grps []grp
last := "\x00"
for i, n := range idx {
d := path.Dir(n)
if d == last {
continue
}
last = d
if d == "." { // 根目录散页不开章
continue
}
grps = append(grps, grp{d, i})
}
if len(grps) < 2 {
return nil
}
titles := make(map[string]int)
out := make([]cbzChapter, len(grps))
for i, g := range grps {
out[i] = cbzChapter{Title: path.Base(g.dir), Start: g.start}
titles[out[i].Title]++
}
for i, g := range grps { // 同名目录(不同父级)撞车 → 用全路径消歧
if titles[out[i].Title] > 1 {
out[i].Title = g.dir
}
}
return out
c.JSON(http.StatusOK, gin.H{"count": len(idx), "chapters": h.med.ChaptersOf(idx)})
}
func (h *H) Page(c *gin.Context) {
@@ -226,48 +170,23 @@ func (h *H) Page(c *gin.Context) {
if !ok {
return
}
idx, e := h.pageIndex(c, b, root)
if e != nil {
if c.Writer.Written() {
return // openBook 已写 403/404,不再叠加 422
}
err(c, http.StatusUnprocessableEntity, "broken", e.Error())
idx, ok := h.pageIndex(c, b, root)
if !ok {
return
}
if n >= len(idx) {
err(c, http.StatusNotFound, "not_found", "no such page")
return
}
ext := strings.ToLower(filepath.Ext(idx[n]))
dir := bookfile.PagesDir(h.cfg.CacheDir, bookfile.DirKey(b.ID, bookfile.Hash(b.FileSize, b.ModTS)))
dst := filepath.Join(dir, strconv.Itoa(n)+ext)
if _, e := os.Stat(dst); e != nil { // miss → 解压落盘(并发重做同页幂等,唯一 tmp 名 + rename 原子)
f, size, ok := h.openBook(c, b, root)
if !ok {
// miss → 解压落盘(media 内部唯一 tmp 名 + rename 原子,并发重做同页幂等)
dst, e := h.med.EnsurePage(c, b.ID, b.FileSize, b.ModTS, root, b.Path, n, idx)
if e != nil {
if mapContentErr(c, e) {
return
}
defer f.Close()
data, e := bookfile.ReadEntry(f, size, idx[n])
if e != nil {
err(c, http.StatusInternalServerError, "internal", "extract page")
return
}
if e := os.MkdirAll(dir, 0o755); e != nil {
err(c, http.StatusInternalServerError, "internal", "cache dir")
return
}
tmp := fmt.Sprintf("%s.tmp-%d", dst, time.Now().UnixNano())
if e := os.WriteFile(tmp, data, 0o644); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "write cache")
return
}
if e := os.Rename(tmp, dst); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename cache")
return
}
}
h.immutable(c)
http.ServeFile(c.Writer, c.Request, dst)
}
+28 -8
View File
@@ -6,6 +6,7 @@ import (
"log"
"net"
"net/http"
"strconv"
"strings"
"syscall"
@@ -15,26 +16,45 @@ import (
"booklib/internal/auth"
"booklib/internal/config"
"booklib/internal/redispkg"
"booklib/internal/scanner"
"booklib/internal/store"
"booklib/internal/ports"
)
// H 只依赖 ports 里的小口径接口(Task 25):具体实现由 main.go 装配,
// 测试可注入手写 fake(portsfake),无需 PG/Redis。
type H struct {
cfg *config.Config
st *store.Store
rdb *redispkg.R
sc *scanner.Scanner
users ports.UserStore
libs ports.LibraryStore
books ports.BookStore
progress ports.ProgressStore
bookmarks ports.BookmarkStore
rl ports.RateLimiter
sc ports.Scanner
med ports.Media
up ports.UploadSessions
}
func New(cfg *config.Config, st *store.Store, rdb *redispkg.R, sc *scanner.Scanner) *H {
return &H{cfg: cfg, st: st, rdb: rdb, sc: sc}
func New(cfg *config.Config, users ports.UserStore, libs ports.LibraryStore, books ports.BookStore,
progress ports.ProgressStore, bookmarks ports.BookmarkStore, rl ports.RateLimiter,
sc ports.Scanner, med ports.Media, up ports.UploadSessions) *H {
return &H{cfg: cfg, users: users, libs: libs, books: books, progress: progress,
bookmarks: bookmarks, rl: rl, sc: sc, med: med, up: up}
}
func err(c *gin.Context, status int, code, msg string) {
c.AbortWithStatusJSON(status, gin.H{"error": gin.H{"code": code, "message": msg}})
}
// idParam 解析 :id 路径参数,失败已回复 400。
func idParam(c *gin.Context) (int64, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return 0, false
}
return id, true
}
// dbErr 统一处理 store 层失败:记日志;连接类错误 503(Service Unavailable),其余 500
// 注:brief 里的 pgxpool.ErrClosedPool 在 pgx v5 不存在,实际由 puddle 原样透出,用它替代;
// PG 停机时池内连接先收到 SQLSTATE 57P01(administrator shutdown),故把 08xx/57Pxx 也归为 503
+32 -52
View File
@@ -16,6 +16,7 @@ import (
"booklib/internal/bookfile"
"booklib/internal/media"
"booklib/internal/ports"
"booklib/internal/store"
)
@@ -23,7 +24,7 @@ import (
func (h *H) libRoot(c *gin.Context, lib store.Library) (string, bool) {
root := filepath.Clean(lib.RootPath)
books := filepath.Clean(h.cfg.BooksDir)
if !filepath.IsAbs(root) || (root != books && !strings.HasPrefix(root, books+string(os.PathSeparator))) {
if !filepath.IsAbs(root) || !bookfile.Contains(books, root) {
err(c, http.StatusForbidden, "forbidden", "library root outside books dir")
return "", false
}
@@ -34,8 +35,22 @@ func (h *H) libRoot(c *gin.Context, lib store.Library) (string, bool) {
return root, true
}
// getLib 查库行,404/503/500 已回复(原 getLibrary/getLibRow 合一,Task 25)
func (h *H) getLib(c *gin.Context, id int64) (store.Library, bool) {
l, e := h.libs.GetLibrary(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return l, true
}
func (h *H) ListLibraries(c *gin.Context) {
libs, e := h.st.ListLibraries(c)
libs, e := h.libs.ListLibraries(c)
if e != nil {
dbErr(c, e)
return
@@ -68,9 +83,9 @@ func (h *H) CreateLibrary(c *gin.Context) {
return
}
root := filepath.Join(filepath.Clean(h.cfg.BooksDir), safe)
id, e := h.st.CreateLibrary(c, req.Name, root)
id, e := h.libs.CreateLibrary(c, req.Name, root)
if e != nil {
if isUnique(e) {
if ports.IsUniqueViolation(e) {
err(c, http.StatusConflict, "exists", "name taken")
return
}
@@ -80,26 +95,12 @@ func (h *H) CreateLibrary(c *gin.Context) {
c.JSON(http.StatusCreated, gin.H{"id": id, "name": req.Name, "root_path": root})
}
func (h *H) getLibrary(c *gin.Context) (store.Library, bool) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
return store.Library{}, false
}
lib, e := h.st.GetLibrary(c, id)
if e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such library")
return store.Library{}, false
}
dbErr(c, e)
return store.Library{}, false
}
return lib, true
}
func (h *H) ScanLibrary(c *gin.Context) {
lib, ok := h.getLibrary(c)
id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok {
return
}
@@ -111,7 +112,11 @@ func (h *H) ScanLibrary(c *gin.Context) {
}
func (h *H) Upload(c *gin.Context) {
lib, ok := h.getLibrary(c)
id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok {
return
}
@@ -136,12 +141,12 @@ func (h *H) Upload(c *gin.Context) {
return
}
// B12: retry on O_EXCL collision — concurrent uploads with the same name
// can both get the same candidate from uniquePath (stat-then-create race).
// can both get the same candidate from UniquePath (stat-then-create race).
var dst, tmp string
var out *os.File
for attempt := 0; attempt < 5; attempt++ {
var e error
dst, e = h.uniquePath(root, name)
dst, e = h.up.UniquePath(root, name)
if e != nil {
err(c, http.StatusForbidden, "forbidden", e.Error())
return
@@ -155,7 +160,7 @@ func (h *H) Upload(c *gin.Context) {
err(c, http.StatusInternalServerError, "internal", "create tmp")
return
}
// O_EXCL collision — retry with fresh uniquePath.
// O_EXCL collision — retry with fresh UniquePath.
}
if out == nil {
err(c, http.StatusConflict, "conflict", "too many concurrent uploads with same name")
@@ -190,28 +195,3 @@ func (h *H) Upload(c *gin.Context) {
}
c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": strings.TrimPrefix(dst, root+string(os.PathSeparator))})
}
// uniquePath 清洗后的 name 必须仍在 root 内;重名加 " (n)" 后缀
func (h *H) uniquePath(root, name string) (string, error) {
ext := filepath.Ext(name)
base := strings.TrimSuffix(name, ext)
for i := 0; ; i++ {
cand := base + ext
if i > 0 {
cand = base + " (" + strconv.Itoa(i) + ")" + ext
}
p := filepath.Join(root, cand)
if filepath.Clean(p) != filepath.Join(root, filepath.Clean(cand)) ||
!strings.HasPrefix(filepath.Clean(p), root+string(os.PathSeparator)) {
return "", os.ErrInvalid
}
if _, e := os.Stat(p); os.IsNotExist(e) {
return p, nil
} else if e != nil {
return "", e
}
if i > 999 {
return "", os.ErrExist
}
}
}
@@ -0,0 +1,232 @@
package handlers_test
import (
"net/http"
"strconv"
"strings"
"testing"
"time"
"booklib/internal/ports"
"booklib/internal/upload"
)
// ---------- libraries (Task 27, portsfake) ----------
func TestUnit_ListLibraries(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "comics")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/libraries", atok, nil)
httpOK(t, w, 200, "list libraries")
var libs []map[string]any
if err := jsonUnmarshal(w, &libs); err != nil {
t.Fatal(err)
}
if len(libs) != 1 || libs[0]["name"] != "comics" {
t.Fatalf("unexpected libs %v", libs)
}
}
func TestUnit_CreateLibrary_ReservedName(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
// 保留名清单以 media/reserved.go 为准(B8);大小写不敏感。
// 契约(与重构前一致): code="bad_request", message="reserved_name"
for _, name := range []string{"cache", ".uploads", ".trash", "CACHE", "Cache"} {
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": name})
if w.Code != 400 || errCode(t, w) != "bad_request" || !strings.Contains(w.Body.String(), "reserved_name") {
t.Fatalf("name=%q want 400/bad_request/reserved_name got %d %s", name, w.Code, w.Body.String())
}
}
}
func TestUnit_CreateLibrary_BadName(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
for _, name := range []string{"", "..", "///"} {
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": name})
httpOK(t, w, 400, "bad name "+name)
}
}
func TestUnit_CreateLibrary_Duplicate(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "dup")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": "dup"})
httpOK(t, w, 409, "duplicate library")
if code := errCode(t, w); code != "exists" {
t.Fatalf("error code want exists got %q", code)
}
}
func TestUnit_CreateLibrary_OK(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries", atok, map[string]string{"Name": "new lib"})
httpOK(t, w, 201, "create library")
body := jsonBody(t, w)
if body["name"] != "new lib" {
t.Fatalf("unexpected body %v", body)
}
}
func TestUnit_ScanLibrary_NotFound(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/999/scan", atok, nil)
httpOK(t, w, 404, "scan missing library")
}
func TestUnit_ScanLibrary_Accepted(t *testing.T) {
e := newTestEnv(t)
libID, _ := e.seedLib(t, "scannable")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(libID, 10)+"/scan", atok, nil)
httpOK(t, w, 202, "scan accepted")
if !e.sc.WaitForScan(1, time.Second) {
t.Fatalf("ScanLibraryByID never called; seen=%v", e.sc.Seen())
}
if seen := e.sc.Seen(); seen[0] != libID {
t.Fatalf("scanned lib %d want %d", seen[0], libID)
}
}
func TestUnit_ScanLibrary_MemberForbidden(t *testing.T) {
e := newTestEnv(t)
libID, _ := e.seedLib(t, "nope")
mtok := e.token(t, "member", 2)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(libID, 10)+"/scan", mtok, nil)
httpOK(t, w, 403, "member scan")
}
// ---------- uploads: sentinel → status mapping ----------
func TestUnit_UploadInit_MapsSentinels(t *testing.T) {
libIDPath := func(e *testEnv) string {
id, _ := e.seedLib(t, "up")
return "/api/libraries/" + strconv.FormatInt(id, 10) + "/upload/init"
}
cases := []struct {
name string
initErr error
wantCode int
wantErr string
}{
{"too large", ports.ErrTooLarge, 413, "too_large"},
{"bad name", upload.ErrBadName, 400, "bad_request"},
{"bad format", upload.ErrBadFormat, 400, "bad_format"},
{"bad size", upload.ErrBadSize, 400, "bad_request"},
{"bad chunk", upload.ErrBadChunk, 400, "bad_request"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
e := newTestEnv(t)
e.up.InitErr = tc.initErr
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, libIDPath(e), atok,
map[string]any{"name": "b.cbz", "size": 10, "chunkSize": 4})
httpOK(t, w, tc.wantCode, "init "+tc.name)
if code := errCode(t, w); code != tc.wantErr {
t.Fatalf("error code want %q got %q", tc.wantErr, code)
}
})
}
}
func TestUnit_UploadInit_OK(t *testing.T) {
e := newTestEnv(t)
id, _ := e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(id, 10)+"/upload/init", atok,
map[string]any{"name": "b.cbz", "size": 10, "chunkSize": 4})
httpOK(t, w, 200, "init ok")
body := jsonBody(t, w)
if body["uploadId"] != e.up.UID {
t.Fatalf("uploadId want %q got %v", e.up.UID, body["uploadId"])
}
}
func TestUnit_UploadInit_BadBody(t *testing.T) {
e := newTestEnv(t)
id, _ := e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/libraries/"+strconv.FormatInt(id, 10)+"/upload/init", atok, nil)
httpOK(t, w, 400, "init without json body")
}
func TestUnit_UploadStatus_NotFound(t *testing.T) {
e := newTestEnv(t)
e.up.StatusErr = ports.ErrNotFound
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/uploads/"+e.up.UID, atok, nil)
httpOK(t, w, 404, "status missing upload")
if code := errCode(t, w); code != "not_found" {
t.Fatalf("error code want not_found got %q", code)
}
}
func TestUnit_UploadStatus_OK(t *testing.T) {
e := newTestEnv(t)
e.up.Received = []int64{0, 2}
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/uploads/"+e.up.UID, atok, nil)
httpOK(t, w, 200, "status ok")
var body struct {
Received []int64 `json:"received"`
}
if err := jsonUnmarshal(w, &body); err != nil {
t.Fatal(err)
}
if len(body.Received) != 2 || body.Received[1] != 2 {
t.Fatalf("received want [0 2] got %v", body.Received)
}
}
func TestUnit_UploadPart_BadIndex(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPut, "/api/uploads/"+e.up.UID+"/parts/abc", atok, nil)
httpOK(t, w, 400, "part bad index")
}
func TestUnit_UploadPart_TooBig(t *testing.T) {
e := newTestEnv(t)
e.up.PutErr = upload.ErrPartTooBig
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPut, "/api/uploads/"+e.up.UID+"/parts/0", atok, nil)
httpOK(t, w, 413, "part too big")
}
func TestUnit_UploadComplete_Incomplete(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "up") // fake LibraryID=1 与 seed 的第一条对齐
e.up.CompleteErr = ports.ErrIncomplete
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
httpOK(t, w, 400, "complete incomplete")
}
func TestUnit_UploadComplete_OK(t *testing.T) {
e := newTestEnv(t)
e.seedLib(t, "up")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
httpOK(t, w, 202, "complete ok")
body := jsonBody(t, w)
if body["path"] != e.up.RelPath {
t.Fatalf("path want %q got %v", e.up.RelPath, body["path"])
}
}
func TestUnit_UploadComplete_LibGone(t *testing.T) {
e := newTestEnv(t) // LibraryID=1 但库里没有 id=1 → GetLibrary ErrNoRows → 404? dbErr → 500
e.up.LibID = 42
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/uploads/"+e.up.UID+"/complete", atok, nil)
// dbErr 把 pgx.ErrNoRows 归为 500(非连接类),契约与真库一致
if w.Code != 500 {
t.Fatalf("lib gone want 500 got %d body=%s", w.Code, w.Body.String())
}
}
+2 -2
View File
@@ -32,7 +32,7 @@ func (h *H) PutProgress(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "locator must be valid json")
return
}
if e := h.st.UpsertProgress(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent); e != nil {
if e := h.progress.UpsertProgress(c, uid(c), b.LibraryID, b.Path, req.Locator, req.Percent); e != nil {
dbErr(c, e)
return
}
@@ -40,7 +40,7 @@ func (h *H) PutProgress(c *gin.Context) {
}
func (h *H) ListProgress(c *gin.Context) {
rows, e := h.st.ListProgress(c, uid(c))
rows, e := h.progress.ListProgress(c, uid(c))
if e != nil {
dbErr(c, e)
return
@@ -0,0 +1,142 @@
package handlers_test
import (
"bytes"
"encoding/json"
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
"github.com/gin-gonic/gin"
"booklib/cmd/webui/api"
"booklib/internal/auth"
"booklib/internal/config"
"booklib/internal/ports/portsfake"
)
// testEnv 是纯 fake 装配(Task 27):无 PG/Redis。请求走 api.NewRouter 组装的
// 真实 engine,因此认证/鉴权中间件、路由匹配与 gin 的 header 落盘行为都被覆盖;
// 集成测试(auth_test.go 的 setupAPI)另走真库,两层互补。
type testEnv struct {
cfg *config.Config
router *gin.Engine
users *portsfake.Users
libs *portsfake.Libraries
books *portsfake.Books
progress *portsfake.Progress
bookmarks *portsfake.Bookmarks
rl *portsfake.RateLimiter
sc *portsfake.Scanner
med *portsfake.Media
up *portsfake.Uploads
booksDir string
}
func newTestEnv(t *testing.T) *testEnv {
t.Helper()
gin.SetMode(gin.TestMode)
resolved, err := filepath.EvalSymlinks(t.TempDir())
if err != nil {
t.Fatal(err)
}
cfg := &config.Config{Addr: ":8080", JWTSecret: []byte("s3cret"), UploadMaxMB: 1,
ScanInterval: time.Minute, BooksDir: resolved, CacheDir: t.TempDir(),
TrustedProxies: []string{"172.16.0.0/12"}}
users := portsfake.NewUsers()
libs := portsfake.NewLibraries()
books := portsfake.NewBooks()
progress := portsfake.NewProgress(libs, books)
e := &testEnv{
cfg: cfg, users: users, libs: libs, books: books,
progress: progress, bookmarks: portsfake.NewBookmarks(),
rl: portsfake.NewRateLimiter(), sc: portsfake.NewScanner(),
med: portsfake.NewMedia(), up: portsfake.NewUploads(),
booksDir: resolved,
}
e.router = api.NewRouter(cfg, users, libs, books, progress, e.bookmarks,
e.rl, e.sc, e.med, e.up)
return e
}
// token 签发一个带角色的 JWT(uid 固定 1,与 fake 里 seed 的用户对应)。
func (e *testEnv) token(t *testing.T, role string, uid int64) string {
t.Helper()
tok, err := auth.Sign(e.cfg.JWTSecret, uid, role)
if err != nil {
t.Fatal(err)
}
return tok
}
// do 以 Bearer token 走完整 engine;body 非 nil 时按 JSON 发送。
func (e *testEnv) do(t *testing.T, method, path, tok string, body any) *httptest.ResponseRecorder {
t.Helper()
var reader *bytes.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
t.Fatal(err)
}
reader = bytes.NewReader(b)
} else {
reader = bytes.NewReader(nil)
}
req := httptest.NewRequest(method, path, reader)
if tok != "" {
req.Header.Set("Authorization", "Bearer "+tok)
}
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
w := httptest.NewRecorder()
e.router.ServeHTTP(w, req)
return w
}
// jsonBody 解析响应体为 map。
func jsonBody(t *testing.T, w *httptest.ResponseRecorder) map[string]any {
t.Helper()
var out map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatalf("unmarshal %q: %v", w.Body.String(), err)
}
return out
}
// jsonUnmarshal 解析响应体到任意目标(slice/struct)。
func jsonUnmarshal(w *httptest.ResponseRecorder, dst any) error {
return json.Unmarshal(w.Body.Bytes(), dst)
}
// errCode 取 error.code 字段(错误响应契约的核心)。
func errCode(t *testing.T, w *httptest.ResponseRecorder) string {
t.Helper()
body := jsonBody(t, w)
e, _ := body["error"].(map[string]any)
if e == nil {
t.Fatalf("no error object in %q", w.Body.String())
}
return fmt.Sprint(e["code"])
}
// seedLib 建一个 root 落在 BooksDir 内的库(fake 行 + 真实目录),返回 id 与 root。
func (e *testEnv) seedLib(t *testing.T, name string) (int64, string) {
t.Helper()
root := filepath.Join(e.booksDir, name)
if err := os.MkdirAll(root, 0o755); err != nil {
t.Fatal(err)
}
return e.libs.Seed(name, root), root
}
// httpOK 断言状态码,失败时带上响应体便于定位。
func httpOK(t *testing.T, w *httptest.ResponseRecorder, want int, what string) {
t.Helper()
if w.Code != want {
t.Fatalf("%s: want %d got %d body=%q", what, want, w.Code, w.Body.String())
}
}
+71 -224
View File
@@ -1,89 +1,29 @@
package handlers
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
"booklib/internal/bookfile"
"booklib/internal/ports"
"booklib/internal/upload"
)
// 分片上传:init(指纹→确定性 uploadId,天然支持续传)→ PUT parts → complete 拼接原子落盘。
// 会话即 <BooksDir>/.uploads/<uid>/(meta.json + parts/N),无独立状态存储;24h 未 complete opportunistic 清扫。
// 分片上传:HTTP 层只做参数绑定与错误映射,域逻辑(指纹续传/分片落盘/拼接/清扫)
// 全部在 internal/upload;会话清扫由 scanner ticker 接管(B16),不在请求路径。
const (
maxChunkBytes = 32 << 20
defaultChunk = 8 << 20
uploadSessTTL = 24 * time.Hour
uploadSessionIn = ".uploads"
)
type uploadMeta struct {
Name string `json:"name"`
Size int64 `json:"size"`
ChunkSize int64 `json:"chunkSize"`
LibraryID int64 `json:"libraryId"`
}
func validUploadID(s string) bool {
if len(s) != 32 {
return false
}
for _, r := range s {
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')) {
return false
}
}
return true
}
func uploadIDFor(libID int64, name string, size, chunk int64) string {
h := sha256.Sum256([]byte(fmt.Sprintf("%d|%s|%d|%d", libID, name, size, chunk)))
return hex.EncodeToString(h[:16])
}
func (h *H) uploadDir(uid string) string {
return filepath.Join(filepath.Clean(h.cfg.BooksDir), uploadSessionIn, uid)
}
func chunkRange(m uploadMeta, i int64) (int64, int64) {
lo := i * m.ChunkSize
hi := min(lo+m.ChunkSize, m.Size)
return lo, hi
}
func (h *H) numParts(m uploadMeta) int64 {
return (m.Size + m.ChunkSize - 1) / m.ChunkSize
}
// sweepUploads 删除过期会话目录;尽力而为,失败不影响主流程
func (h *H) sweepUploads() {
base := filepath.Join(filepath.Clean(h.cfg.BooksDir), uploadSessionIn)
es, e := os.ReadDir(base)
if e != nil {
return
}
for _, en := range es {
if fi, e := en.Info(); e == nil && time.Since(fi.ModTime()) > uploadSessTTL {
os.RemoveAll(filepath.Join(base, en.Name()))
}
}
}
// maxChunkBytes 与 upload 包内常量同值,作为 PutPart 的防御性读取上限。
const maxChunkBytes = 32 << 20
func (h *H) UploadInit(c *gin.Context) {
lib, ok := h.getLibrary(c)
id, ok := idParam(c)
if !ok {
return
}
lib, ok := h.getLib(c, id)
if !ok {
return
}
@@ -99,154 +39,47 @@ func (h *H) UploadInit(c *gin.Context) {
err(c, http.StatusBadRequest, "bad_request", "name, size required")
return
}
name := bookfile.SafeName(req.Name)
if name == "" {
err(c, http.StatusBadRequest, "bad_request", "bad name")
return
}
if bookfile.FormatFromExt(name) == "" {
err(c, http.StatusBadRequest, "bad_format", "extension must be cbz/pdf/epub/txt/md")
return
}
if req.Size <= 0 {
err(c, http.StatusBadRequest, "bad_request", "bad size")
return
}
if req.Size > h.cfg.UploadMaxMB<<20 {
err(c, http.StatusRequestEntityTooLarge, "too_large", "file exceeds upload limit of "+strconv.FormatInt(h.cfg.UploadMaxMB, 10)+"MB")
return
}
if req.ChunkSize == 0 {
req.ChunkSize = defaultChunk
}
if req.ChunkSize > maxChunkBytes {
err(c, http.StatusBadRequest, "bad_request", "chunkSize must be <= 33554432")
return
}
uid := uploadIDFor(lib.ID, name, req.Size, req.ChunkSize)
dir := h.uploadDir(uid)
meta := uploadMeta{Name: name, Size: req.Size, ChunkSize: req.ChunkSize, LibraryID: lib.ID}
if b, e := os.ReadFile(filepath.Join(dir, "meta.json")); e == nil {
var old uploadMeta
if json.Unmarshal(b, &old) == nil && old == meta { // 同指纹 → 复用会话(续传)
c.JSON(http.StatusOK, gin.H{"uploadId": uid})
return
}
os.RemoveAll(dir) // 指纹撞上但内容不同 → 从头再来
}
h.sweepUploads()
if e := os.MkdirAll(filepath.Join(dir, "parts"), 0o755); e != nil {
err(c, http.StatusInternalServerError, "internal", "create session")
return
}
b, _ := json.Marshal(meta)
if e := os.WriteFile(filepath.Join(dir, "meta.json"), b, 0o644); e != nil {
err(c, http.StatusInternalServerError, "internal", "write meta")
return
}
c.JSON(http.StatusOK, gin.H{"uploadId": uid})
}
// loadMeta 校验 uid 与路径,404/400 已回复
func (h *H) loadMeta(c *gin.Context) (uploadMeta, string, bool) {
uid := c.Param("uid")
if !validUploadID(uid) {
err(c, http.StatusBadRequest, "bad_request", "bad upload id")
return uploadMeta{}, "", false
}
dir := h.uploadDir(uid)
var m uploadMeta
b, e := os.ReadFile(filepath.Join(dir, "meta.json"))
uid, e := h.up.Init(c, lib.ID, req.Name, req.Size, req.ChunkSize)
if e != nil {
err(c, http.StatusNotFound, "not_found", "no such upload")
return uploadMeta{}, "", false
h.mapUploadErr(c, e)
return
}
if json.Unmarshal(b, &m) != nil {
err(c, http.StatusInternalServerError, "internal", "corrupt session")
return uploadMeta{}, "", false
}
return m, dir, true
c.JSON(http.StatusOK, gin.H{"uploadId": uid})
}
func (h *H) UploadStatus(c *gin.Context) {
_, dir, ok := h.loadMeta(c)
if !ok {
recv, e := h.up.Status(c, c.Param("uid"))
if e != nil {
h.mapUploadErr(c, e)
return
}
recv := []int64{}
es, e := os.ReadDir(filepath.Join(dir, "parts"))
if e == nil {
for _, en := range es {
if i, e := strconv.ParseInt(en.Name(), 10, 64); e == nil {
recv = append(recv, i)
}
}
}
sort.Slice(recv, func(i, j int) bool { return recv[i] < recv[j] })
c.JSON(http.StatusOK, gin.H{"received": recv})
}
func (h *H) UploadPart(c *gin.Context) {
m, dir, ok := h.loadMeta(c)
if !ok {
return
}
uid := c.Param("uid")
idx, e := strconv.ParseInt(c.Param("index"), 10, 64)
if e != nil || idx < 0 || idx >= h.numParts(m) {
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad part index")
return
}
lo, hi := chunkRange(m, idx)
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, hi-lo)
// B4: write to .tmp then rename — prevents truncated parts from being
// reported as "received" by UploadStatus if the process crashes mid-write.
p := filepath.Join(dir, "parts", strconv.FormatInt(idx, 10))
tmp := p + ".tmp"
f, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
// maxSize 防御性上限:分片声明大小由会话 meta 决定,这里再垫一层 32MB 全局上限
e = h.up.PutPart(c, uid, idx, c.Request.Body, maxChunkBytes)
if e != nil {
err(c, http.StatusInternalServerError, "internal", "create part")
return
}
n, e := io.Copy(f, c.Request.Body)
f.Close()
if e != nil || n != hi-lo {
os.Remove(tmp)
var mbe *http.MaxBytesError
code, msg := "too_large", "part size mismatch"
if errors.As(e, &mbe) {
msg = "part exceeds declared size"
}
err(c, http.StatusRequestEntityTooLarge, code, msg)
return
}
if e := os.Rename(tmp, p); e != nil {
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename part")
h.mapUploadErr(c, e)
return
}
c.JSON(http.StatusAccepted, gin.H{"accepted": true})
}
func (h *H) UploadComplete(c *gin.Context) {
m, dir, ok := h.loadMeta(c)
if !ok {
uid := c.Param("uid")
libID, e := h.up.LibraryID(c, uid)
if e != nil {
h.mapUploadErr(c, e)
return
}
var total int64
for i := int64(0); i < h.numParts(m); i++ {
lo, hi := chunkRange(m, i)
fi, e := os.Stat(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil || fi.Size() != hi-lo {
err(c, http.StatusBadRequest, "bad_request", "upload incomplete; missing or corrupt parts, re-upload them")
return
}
total += fi.Size()
}
if total != m.Size {
err(c, http.StatusBadRequest, "bad_request", "total size mismatch")
return
}
lib, e := h.st.GetLibrary(c, m.LibraryID)
lib, e := h.libs.GetLibrary(c, libID)
if e != nil {
dbErr(c, e)
return
@@ -255,39 +88,53 @@ func (h *H) UploadComplete(c *gin.Context) {
if !ok {
return
}
dst, e := h.uniquePath(root, m.Name)
rel, e := h.up.Complete(c, uid, root)
if e != nil {
err(c, http.StatusForbidden, "forbidden", e.Error())
h.mapUploadErr(c, e)
return
}
tmp := filepath.Join(dir, "assembled")
out, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil {
err(c, http.StatusInternalServerError, "internal", "create tmp")
c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": rel})
}
// mapUploadErr 把 upload 包的 sentinel 错误映射为原契约的 status/code/message。
func (h *H) mapUploadErr(c *gin.Context, e error) {
switch {
case errors.Is(e, ports.ErrTooLarge):
err(c, http.StatusRequestEntityTooLarge, "too_large", "file exceeds upload limit of "+strconv.FormatInt(h.cfg.UploadMaxMB, 10)+"MB")
case errors.Is(e, upload.ErrBadName):
err(c, http.StatusBadRequest, "bad_request", "bad name")
case errors.Is(e, upload.ErrBadFormat):
err(c, http.StatusBadRequest, "bad_format", "extension must be cbz/pdf/epub/txt/md")
case errors.Is(e, upload.ErrBadSize):
err(c, http.StatusBadRequest, "bad_request", "bad size")
case errors.Is(e, upload.ErrBadChunk):
err(c, http.StatusBadRequest, "bad_request", "chunkSize must be <= 33554432")
case errors.Is(e, upload.ErrBadUploadID):
err(c, http.StatusBadRequest, "bad_request", "bad upload id")
case errors.Is(e, upload.ErrBadIndex):
err(c, http.StatusBadRequest, "bad_request", "bad part index")
case errors.Is(e, upload.ErrPartTooBig):
err(c, http.StatusRequestEntityTooLarge, "too_large", "part exceeds declared size")
case errors.Is(e, upload.ErrPartSizeMismatch):
err(c, http.StatusRequestEntityTooLarge, "too_large", "part size mismatch")
case errors.Is(e, ports.ErrNotFound):
err(c, http.StatusNotFound, "not_found", "no such upload")
case errors.Is(e, upload.ErrCorrupt):
err(c, http.StatusInternalServerError, "internal", "corrupt session")
case errors.Is(e, ports.ErrIncomplete):
err(c, http.StatusBadRequest, "bad_request", "upload incomplete; missing or corrupt parts, re-upload them")
case errors.Is(e, ports.ErrSizeMismatch):
err(c, http.StatusBadRequest, "bad_request", "total size mismatch")
case errors.Is(e, os.ErrInvalid), errors.Is(e, os.ErrExist):
err(c, http.StatusForbidden, "forbidden", e.Error())
case errors.Is(e, os.ErrPermission), errors.Is(e, os.ErrClosed):
err(c, http.StatusInternalServerError, "internal", "io error")
default:
var oe *upload.OpError
if errors.As(e, &oe) {
err(c, http.StatusInternalServerError, "internal", oe.Op)
return
}
for i := int64(0); i < h.numParts(m); i++ {
pf, e := os.Open(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil {
out.Close()
err(c, http.StatusInternalServerError, "internal", "open part")
return
}
if _, e := io.Copy(out, pf); e != nil {
pf.Close()
out.Close()
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "assemble")
return
}
pf.Close()
}
out.Close()
if e := os.Rename(tmp, dst); e != nil { // 原子落盘,scanner 自动收编
os.Remove(tmp)
err(c, http.StatusInternalServerError, "internal", "rename")
return
err(c, http.StatusInternalServerError, "internal", "upload failed")
}
os.RemoveAll(dir)
c.JSON(http.StatusAccepted, gin.H{"accepted": true, "path": strings.TrimPrefix(dst, root+string(os.PathSeparator))})
}
+7 -11
View File
@@ -3,21 +3,18 @@ package handlers
import (
"errors"
"net/http"
"strconv"
"time"
"github.com/gin-gonic/gin"
"github.com/jackc/pgx/v5"
"booklib/internal/auth"
"booklib/internal/ports"
"booklib/internal/store"
)
// isUnique is a convenience alias for store.IsUniqueViolation.
func isUnique(e error) bool { return store.IsUniqueViolation(e) }
func (h *H) ListUsers(c *gin.Context) {
users, e := h.st.ListUsers(c)
users, e := h.users.ListUsers(c)
if e != nil {
dbErr(c, e)
return
@@ -49,9 +46,9 @@ func (h *H) CreateUser(c *gin.Context) {
err(c, http.StatusInternalServerError, "internal", "hash")
return
}
id, e := h.st.CreateUser(c, req.Username, hp, req.Role)
id, e := h.users.CreateUser(c, req.Username, hp, req.Role)
if e != nil {
if isUnique(e) {
if ports.IsUniqueViolation(e) {
err(c, http.StatusConflict, "exists", "username taken")
return
}
@@ -62,9 +59,8 @@ func (h *H) CreateUser(c *gin.Context) {
}
func (h *H) DeleteUser(c *gin.Context) {
id, e := strconv.ParseInt(c.Param("id"), 10, 64)
if e != nil {
err(c, http.StatusBadRequest, "bad_request", "bad id")
id, ok := idParam(c)
if !ok {
return
}
if id == uid(c) {
@@ -72,7 +68,7 @@ func (h *H) DeleteUser(c *gin.Context) {
return
}
// B5: transactional last-admin check eliminates TOCTOU race.
if e := h.st.DeleteUser(c, id); e != nil {
if e := h.users.DeleteUser(c, id); e != nil {
if errors.Is(e, pgx.ErrNoRows) {
err(c, http.StatusNotFound, "not_found", "no such user")
return
@@ -0,0 +1,154 @@
package handlers_test
import (
"context"
"errors"
"net/http"
"strconv"
"testing"
"github.com/jackc/pgx/v5"
"booklib/internal/auth"
)
// ---------- users: admin-only CRUD branches (Task 27, portsfake) ----------
func TestUnit_ListUsers(t *testing.T) {
e := newTestEnv(t)
hash, _ := auth.HashPassword("password1234")
e.users.Seed("admin", hash, "admin")
e.users.Seed("member1", hash, "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodGet, "/api/users", atok, nil)
httpOK(t, w, 200, "list users")
var users []map[string]any
if err := jsonUnmarshal(w, &users); err != nil {
t.Fatal(err)
}
if len(users) != 2 {
t.Fatalf("want 2 users got %d", len(users))
}
}
func TestUnit_ListUsers_MemberForbidden(t *testing.T) {
e := newTestEnv(t)
mtok := e.token(t, "member", 2)
w := e.do(t, http.MethodGet, "/api/users", mtok, nil)
httpOK(t, w, 403, "member list users")
}
func TestUnit_CreateUser_RoleValidation(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
for _, role := range []string{"superadmin", "", "Member"} {
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "u", "Password": "password1234", "Role": role})
httpOK(t, w, 400, "create user role="+role)
}
}
func TestUnit_CreateUser_ShortPassword(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "u", "Password": "short", "Role": "member"})
httpOK(t, w, 400, "short password")
}
func TestUnit_CreateUser_DuplicateName(t *testing.T) {
e := newTestEnv(t)
hash, _ := auth.HashPassword("password1234")
e.users.Seed("existing", hash, "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "existing", "Password": "password1234", "Role": "member"})
httpOK(t, w, 409, "duplicate name")
if code := errCode(t, w); code != "exists" {
t.Fatalf("error code want 'exists' got %q", code)
}
}
func TestUnit_CreateUser_OK(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodPost, "/api/users", atok,
map[string]string{"Username": "newbie", "Password": "password1234", "Role": "member"})
httpOK(t, w, 201, "create user")
body := jsonBody(t, w)
if body["username"] != "newbie" || body["role"] != "member" {
t.Fatalf("unexpected body %v", body)
}
}
func TestUnit_DeleteUser_BadID(t *testing.T) {
e := newTestEnv(t)
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/notanum", atok, nil)
httpOK(t, w, 400, "bad id")
}
func TestUnit_DeleteUser_Self(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("self", "hash", "admin")
atok := e.token(t, "admin", uid) // 自己删自己
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(uid, 10), atok, nil)
httpOK(t, w, 400, "delete self")
}
func TestUnit_DeleteUser_LastAdmin(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("lastadmin", "hash", "admin")
atok := e.token(t, "admin", 999) // 另一个(不存在的)操作者
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(uid, 10), atok, nil)
httpOK(t, w, 400, "last admin")
}
func TestUnit_DeleteUser_NotFound(t *testing.T) {
e := newTestEnv(t)
e.users.Seed("other", "hash", "admin")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/99999", atok, nil)
httpOK(t, w, 404, "missing user")
}
func TestUnit_DeleteUser_OK(t *testing.T) {
e := newTestEnv(t)
e.users.Seed("admin", "hash", "admin") // 保住 last-admin 保护不触发
target := e.users.Seed("todelete", "hash", "member")
atok := e.token(t, "admin", 1)
w := e.do(t, http.MethodDelete, "/api/users/"+strconv.FormatInt(target, 10), atok, nil)
httpOK(t, w, 204, "delete user")
if _, err := e.users.GetUserByID(context.Background(), target); !errors.Is(err, pgx.ErrNoRows) {
t.Fatal("user should be gone")
}
}
func TestUnit_Me_OK(t *testing.T) {
e := newTestEnv(t)
uid := e.users.Seed("me", "hash", "member")
mtok := e.token(t, "member", uid)
w := e.do(t, http.MethodGet, "/api/auth/me", mtok, nil)
httpOK(t, w, 200, "me")
body := jsonBody(t, w)
if body["username"] != "me" {
t.Fatalf("username want 'me' got %v", body["username"])
}
}
func TestUnit_Me_UserGone(t *testing.T) {
e := newTestEnv(t)
mtok := e.token(t, "member", 99999)
w := e.do(t, http.MethodGet, "/api/auth/me", mtok, nil)
httpOK(t, w, 401, "me after user gone")
}
func TestUnit_Me_NoToken(t *testing.T) {
e := newTestEnv(t)
w := e.do(t, http.MethodGet, "/api/auth/me", "", nil)
httpOK(t, w, 401, "me without token")
}
+7 -2
View File
@@ -12,10 +12,12 @@ import (
"booklib/cmd/webui/api"
"booklib/internal/config"
"booklib/internal/db"
"booklib/internal/media"
"booklib/internal/redispkg"
"booklib/internal/scanner"
"booklib/internal/seed"
"booklib/internal/store"
"booklib/internal/upload"
)
func main() {
@@ -39,11 +41,14 @@ func main() {
log.Fatalf("seed: %v", err)
}
rdb := redispkg.New(cfg.RedisURL)
sc := scanner.New(st, cfg, rdb)
med := media.New(cfg, rdb)
up := upload.New(cfg.BooksDir, cfg.UploadMaxMB)
sc := scanner.New(st, cfg, rdb, up) // B16: sweep rides the scan ticker
go sc.Run(ctx)
serveErr := make(chan error, 1)
srv := &http.Server{Addr: cfg.Addr, Handler: api.NewRouter(cfg, st, rdb, sc),
// *store.Store 同时满足 5 个 store 接口;*redispkg.R 满足 RateLimiter。
srv := &http.Server{Addr: cfg.Addr, Handler: api.NewRouter(cfg, st, st, st, st, st, rdb, sc, med, up),
ReadHeaderTimeout: 10 * time.Second}
go func() {
log.Printf("listening on %s", cfg.Addr)
+14 -17
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"io"
"log"
"os"
"path"
"path/filepath"
@@ -14,6 +13,7 @@ import (
"booklib/internal/bookfile"
"booklib/internal/config"
"booklib/internal/ports"
"booklib/internal/redispkg"
)
@@ -23,6 +23,9 @@ type M struct {
rdb *redispkg.R
}
// compile-time proof that *M satisfies the consumer-side interface.
var _ ports.Media = (*M)(nil)
func New(cfg *config.Config, rdb *redispkg.R) *M {
return &M{cfg: cfg, rdb: rdb}
}
@@ -54,7 +57,7 @@ func (m *M) EnsureCover(ctx context.Context, bookID int64, format string, size,
return err
}
dir := bookfile.CoverDir(m.cfg.CacheDir, bookfile.DirKey(bookID, bookfile.Hash(size, modTS)))
return writeAtomic(dir, "cover"+ext, img)
return WriteAtomic(dir, "cover"+ext, img)
}
// PageIndex returns the page list for a CBZ, using redis cache when available.
@@ -80,7 +83,7 @@ func (m *M) PageIndex(ctx context.Context, bookID int64, size, modTS int64, root
}
// ChaptersOf derives chapters from a page index by grouping pages by parent directory.
func (m *M) ChaptersOf(idx []string) []Chapter {
func (m *M) ChaptersOf(idx []string) []ports.Chapter {
type grp struct {
dir string
start int
@@ -102,9 +105,9 @@ func (m *M) ChaptersOf(idx []string) []Chapter {
return nil
}
titles := make(map[string]int)
out := make([]Chapter, len(grps))
out := make([]ports.Chapter, len(grps))
for i, g := range grps {
out[i] = Chapter{Title: path.Base(g.dir), Start: g.start}
out[i] = ports.Chapter{Title: path.Base(g.dir), Start: g.start}
titles[out[i].Title]++
}
for i, g := range grps {
@@ -115,14 +118,8 @@ func (m *M) ChaptersOf(idx []string) []Chapter {
return out
}
// Chapter represents a CBZ chapter.
type Chapter struct {
Title string `json:"title"`
Start int `json:"start"`
}
// EnsurePage extracts a single page to the cache. Returns the cache file path.
func (m *M) EnsurePage(bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error) {
func (m *M) EnsurePage(_ context.Context, bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error) {
if n >= len(idx) {
return "", fmt.Errorf("page %d out of range", n)
}
@@ -141,26 +138,26 @@ func (m *M) EnsurePage(bookID int64, size, modTS int64, root, rel string, n int,
if err != nil {
return "", err
}
if err := writeAtomic(dir, strconv.Itoa(n)+ext, data); err != nil {
if err := WriteAtomic(dir, strconv.Itoa(n)+ext, data); err != nil {
return "", err
}
return dst, nil
}
// writeAtomic writes data to dir/name via tmp+rename. B11: cleans tmp only on failure.
func writeAtomic(dir, name string, data []byte) error {
// WriteAtomic writes data to dir/name via tmp+rename. B11: cleans tmp only on
// failure, so a crash mid-write never leaves a readable half-written file.
// It does not log — callers own the context (which book, which page).
func WriteAtomic(dir, name string, data []byte) error {
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
tmp := filepath.Join(dir, fmt.Sprintf("%s.tmp-%d", name, time.Now().UnixNano()))
if err := os.WriteFile(tmp, data, 0o644); err != nil {
log.Printf("media: write %s: %v", name, err)
os.Remove(tmp)
return err
}
dst := filepath.Join(dir, name)
if err := os.Rename(tmp, dst); err != nil {
log.Printf("media: rename %s: %v", name, err)
os.Remove(tmp)
return err
}
+8
View File
@@ -78,6 +78,9 @@ type Scanner interface {
type UploadSessions interface {
Init(ctx context.Context, libID int64, name string, size, chunkSize int64) (string, error)
// LibraryID returns the target library recorded in the session, so the
// handler can resolve+validate the library root before Complete.
LibraryID(ctx context.Context, uploadID string) (int64, error)
Status(ctx context.Context, uploadID string) ([]int64, error)
PutPart(ctx context.Context, uploadID string, index int64, body io.Reader, maxSize int64) error
Complete(ctx context.Context, uploadID string, root string) (string, error)
@@ -86,7 +89,12 @@ type UploadSessions interface {
}
type Media interface {
// EnsureCover extracts+caches the cover if not already cached. Returns nil
// for formats without cover support.
EnsureCover(ctx context.Context, bookID int64, format string, size, modTS int64, root, rel string) error
// EnsurePage extracts page n (from a pre-fetched idx) to the cache and
// returns the cache file path.
EnsurePage(ctx context.Context, bookID int64, size, modTS int64, root, rel string, n int, idx []string) (string, error)
ChaptersOf(idx []string) []Chapter
PageIndex(ctx context.Context, bookID int64, size, modTS int64, root, rel string) ([]string, error)
CacheBuster(size, modTS int64) string
@@ -0,0 +1,198 @@
package portsfake
import (
"context"
"fmt"
"io"
"sync"
"time"
"booklib/internal/ports"
)
// ---------- RateLimiter ----------
// RateLimiter counts IncrWindow calls per key and returns a programmable
// window value. Default: always allow (return 1).
type RateLimiter struct {
mu sync.Mutex
calls map[string]int
Result int // value returned by IncrWindow; 0 means "call count"
Hook func(key string, n int) int // optional override
}
func NewRateLimiter() *RateLimiter { return &RateLimiter{calls: map[string]int{}} }
func (r *RateLimiter) IncrWindow(_ context.Context, key string, _ time.Duration) int {
r.mu.Lock()
r.calls[key]++
n := r.calls[key]
r.mu.Unlock()
if r.Hook != nil {
return r.Hook(key, n)
}
if r.Result > 0 {
return r.Result
}
return n
}
// Calls reports how many times IncrWindow was invoked for key.
func (r *RateLimiter) Calls(key string) int {
r.mu.Lock()
defer r.mu.Unlock()
return r.calls[key]
}
// ---------- Scanner ----------
// Scanner records ScanLibraryByID calls. The handler invokes it in a goroutine,
// so reads go through the mutex; WaitForScan blocks until at least n calls
// landed (or the timeout expires) to keep tests deterministic.
type Scanner struct {
mu sync.Mutex
seen []int64
}
func NewScanner() *Scanner { return &Scanner{} }
func (s *Scanner) ScanLibraryByID(_ context.Context, id int64) {
s.mu.Lock()
s.seen = append(s.seen, id)
s.mu.Unlock()
}
// Seen returns the library ids scanned so far, in call order.
func (s *Scanner) Seen() []int64 {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]int64, len(s.seen))
copy(out, s.seen)
return out
}
// WaitForScan blocks until len(Seen()) >= n or timeout elapses; reports success.
func (s *Scanner) WaitForScan(n int, timeout time.Duration) bool {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
if len(s.Seen()) >= n {
return true
}
time.Sleep(time.Millisecond)
}
return len(s.Seen()) >= n
}
// ---------- Media ----------
// Media is a programmable stand-in for the media service. Defaults answer
// successfully with the configured fixture data; individual hooks let a test
// force one error branch without touching the others.
type Media struct {
Pages []string // PageIndex result
Chapters []ports.Chapter // ChaptersOf result
CoverErr error // EnsureCover result
IndexErr error // PageIndex result
PagePath string // EnsurePage result path (test writes the file first)
PageErr error // EnsurePage result
EnsureCoverCalls int
EnsurePageCalls int
}
func NewMedia() *Media { return &Media{} }
func (m *Media) EnsureCover(_ context.Context, _ int64, _ string, _, _ int64, _, _ string) error {
m.EnsureCoverCalls++
return m.CoverErr
}
func (m *Media) EnsurePage(_ context.Context, _ int64, _, _ int64, _, _ string, _ int, _ []string) (string, error) {
m.EnsurePageCalls++
if m.PageErr != nil {
return "", m.PageErr
}
return m.PagePath, nil
}
func (m *Media) ChaptersOf(_ []string) []ports.Chapter { return m.Chapters }
func (m *Media) PageIndex(_ context.Context, _ int64, _, _ int64, _, _ string) ([]string, error) {
if m.IndexErr != nil {
return nil, m.IndexErr
}
return m.Pages, nil
}
func (m *Media) CacheBuster(size, modTS int64) string { return fmt.Sprintf("%d-%d", size, modTS) }
// ---------- UploadSessions ----------
// Uploads is a programmable stand-in for the chunked-upload subsystem.
// Defaults simulate a happy session (uid "fakeuid…", LibraryID 1);
// hooks force error branches.
type Uploads struct {
UID string // returned by Init
LibID int64 // returned by LibraryID
Received []int64 // returned by Status
RelPath string // returned by Complete
InitErr error
LibErr error
StatusErr error
PutErr error
CompleteErr error
PutCalls []int64 // part indices passed to PutPart
SweepCalls int
UniqueCalls int
}
func NewUploads() *Uploads {
return &Uploads{UID: "0123456789abcdef0123456789abcdef", LibID: 1, RelPath: "book.cbz"}
}
func (u *Uploads) Init(_ context.Context, _ int64, _ string, _, _ int64) (string, error) {
if u.InitErr != nil {
return "", u.InitErr
}
return u.UID, nil
}
func (u *Uploads) LibraryID(_ context.Context, _ string) (int64, error) {
if u.LibErr != nil {
return 0, u.LibErr
}
return u.LibID, nil
}
func (u *Uploads) Status(_ context.Context, _ string) ([]int64, error) {
if u.StatusErr != nil {
return nil, u.StatusErr
}
return u.Received, nil
}
func (u *Uploads) PutPart(_ context.Context, _ string, index int64, body io.Reader, _ int64) error {
u.PutCalls = append(u.PutCalls, index)
if body != nil { // drain so callers using pipes don't block
io.Copy(io.Discard, body)
}
return u.PutErr
}
func (u *Uploads) Complete(_ context.Context, _, _ string) (string, error) {
if u.CompleteErr != nil {
return "", u.CompleteErr
}
return u.RelPath, nil
}
func (u *Uploads) Sweep(_ context.Context) error {
u.SweepCalls++
return nil
}
func (u *Uploads) UniquePath(root, name string) (string, error) {
u.UniqueCalls++
return root + "/" + name, nil
}
+384
View File
@@ -0,0 +1,384 @@
// Package portsfake provides hand-written, in-memory implementations of every
// interface in internal/ports. They let handler tests run without PG or Redis,
// while reproducing the real store's error semantics exactly (pgx.ErrNoRows for
// missing rows, store.ErrLastAdmin / store.ErrUniqueViolation for the guarded
// paths), so the branches under test behave as they do against the database.
//
// Fakes are safe for single-goroutine test use only; they are not locked.
package portsfake
import (
"context"
"sort"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"booklib/internal/ports"
"booklib/internal/store"
)
// uniqueViolation 复刻真 PG 的唯一约束冲突:store.IsUniqueViolation 只认
// SQLSTATE 23505 的 *pgconn.PgError(生产链路里 pgx 原样透出),fake 必须同形。
func uniqueViolation(constraint string) error {
return &pgconn.PgError{Code: "23505", ConstraintName: constraint}
}
// compile-time proof that the fakes satisfy the consumer-side interfaces.
var (
_ ports.UserStore = (*Users)(nil)
_ ports.LibraryStore = (*Libraries)(nil)
_ ports.BookStore = (*Books)(nil)
_ ports.ProgressStore = (*Progress)(nil)
_ ports.BookmarkStore = (*Bookmarks)(nil)
_ ports.RateLimiter = (*RateLimiter)(nil)
_ ports.Scanner = (*Scanner)(nil)
_ ports.Media = (*Media)(nil)
_ ports.UploadSessions = (*Uploads)(nil)
)
// now is frozen per-fake-set so CreatedAt comparisons are stable within a test.
func now() time.Time { return time.Date(2026, 9, 14, 12, 0, 0, 0, time.UTC) }
// ---------- UserStore ----------
type Users struct {
m map[int64]store.User
next int64
}
func NewUsers() *Users { return &Users{m: map[int64]store.User{}, next: 1} }
// Seed adds a user and returns its id — test convenience, not part of the port.
func (u *Users) Seed(username, hash, role string) int64 {
id := u.next
u.next++
u.m[id] = store.User{ID: id, Username: username, PasswordHash: hash, Role: role, CreatedAt: now()}
return id
}
func (u *Users) CountUsers(_ context.Context) (int, error) { return len(u.m), nil }
func (u *Users) CreateUser(_ context.Context, username, hash, role string) (int64, error) {
for _, v := range u.m {
if v.Username == username {
return 0, uniqueViolation("users_username_key")
}
}
return u.Seed(username, hash, role), nil
}
func (u *Users) GetUserByName(_ context.Context, username string) (store.User, error) {
for _, v := range u.m {
if v.Username == username {
return v, nil
}
}
return store.User{}, pgx.ErrNoRows
}
func (u *Users) GetUserByID(_ context.Context, id int64) (store.User, error) {
if v, ok := u.m[id]; ok {
return v, nil
}
return store.User{}, pgx.ErrNoRows
}
func (u *Users) ListUsers(_ context.Context) ([]store.User, error) {
out := make([]store.User, 0, len(u.m))
for _, v := range u.m {
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (u *Users) DeleteUser(_ context.Context, id int64) error {
v, ok := u.m[id]
if !ok {
return pgx.ErrNoRows
}
if v.Role == "admin" {
n := 0
for _, x := range u.m {
if x.Role == "admin" {
n++
}
}
if n <= 1 {
return store.ErrLastAdmin
}
}
delete(u.m, id)
return nil
}
// ---------- LibraryStore ----------
type Libraries struct {
m map[int64]store.Library
next int64
}
func NewLibraries() *Libraries { return &Libraries{m: map[int64]store.Library{}, next: 1} }
// Seed adds a library and returns its id — test convenience.
func (l *Libraries) Seed(name, root string) int64 {
id := l.next
l.next++
l.m[id] = store.Library{ID: id, Name: name, RootPath: root, CreatedAt: now()}
return id
}
func (l *Libraries) CreateLibrary(_ context.Context, name, root string) (int64, error) {
for _, v := range l.m {
if v.Name == name {
return 0, uniqueViolation("libraries_name_key")
}
}
return l.Seed(name, root), nil
}
func (l *Libraries) ListLibraries(_ context.Context) ([]store.Library, error) {
out := make([]store.Library, 0, len(l.m))
for _, v := range l.m {
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (l *Libraries) GetLibrary(_ context.Context, id int64) (store.Library, error) {
if v, ok := l.m[id]; ok {
return v, nil
}
return store.Library{}, pgx.ErrNoRows
}
// ---------- BookStore ----------
type Books struct {
m map[int64]store.Book
next int64
}
func NewBooks() *Books { return &Books{m: map[int64]store.Book{}, next: 1} }
// Seed adds a book and returns its id — test convenience.
func (b *Books) Seed(libID int64, path, title, format string, size, modTS int64, pageCount int) int64 {
id := b.next
b.next++
b.m[id] = store.Book{ID: id, LibraryID: libID, Path: path, Title: title, Format: format,
FileSize: size, ModTS: modTS, PageCount: pageCount, State: "ok", AddedAt: now()}
return id
}
func (b *Books) InsertBook(_ context.Context, libID int64, path, title, format string, size, modTS int64, pageCount int) (int64, error) {
return b.Seed(libID, path, title, format, size, modTS, pageCount), nil
}
func (b *Books) GetBook(_ context.Context, id int64) (store.Book, error) {
if v, ok := b.m[id]; ok {
return v, nil
}
return store.Book{}, pgx.ErrNoRows
}
func (b *Books) ListBookMeta(_ context.Context, libID int64) (map[string]store.BookMeta, error) {
out := map[string]store.BookMeta{}
for _, v := range b.m {
if v.LibraryID == libID {
out[v.Path] = store.BookMeta{ID: v.ID, Size: v.FileSize, ModTS: v.ModTS, Format: v.Format}
}
}
return out, nil
}
func (b *Books) UpdateBookFile(_ context.Context, id, size, modTS int64, pageCount int) error {
v, ok := b.m[id]
if !ok {
return pgx.ErrNoRows
}
v.FileSize, v.ModTS, v.PageCount = size, modTS, pageCount
b.m[id] = v
return nil
}
func (b *Books) DeleteBookByPath(_ context.Context, libID int64, path string) error {
for id, v := range b.m {
if v.LibraryID == libID && v.Path == path {
delete(b.m, id)
return nil
}
}
return nil
}
func (b *Books) DeleteBook(_ context.Context, id int64) error {
if _, ok := b.m[id]; !ok {
return pgx.ErrNoRows
}
delete(b.m, id)
return nil
}
func (b *Books) SetBookState(_ context.Context, id int64, state, msg string) error {
v, ok := b.m[id]
if !ok {
return pgx.ErrNoRows
}
v.State, v.ErrMsg = state, msg
b.m[id] = v
return nil
}
// ListBooks applies the same q/prefix filter as the SQL view (q matches title or
// path, prefix matches path prefix), then joins progress + library name.
func (b *Books) ListBooks(_ context.Context, libID int64, q, prefix string, userID int64) ([]store.BookView, error) {
out := []store.BookView{}
q = strings.ToLower(q)
for _, v := range b.m {
if libID != 0 && v.LibraryID != libID {
continue
}
if prefix != "" && !strings.HasPrefix(v.Path, prefix) {
continue
}
if q != "" && !strings.Contains(strings.ToLower(v.Title), q) && !strings.Contains(strings.ToLower(v.Path), q) {
continue
}
out = append(out, store.BookView{Book: v})
}
sort.Slice(out, func(i, j int) bool { return out[i].Book.ID < out[j].Book.ID })
return out, nil
}
func (b *Books) BookHashes(_ context.Context) (map[int64][2]int64, error) {
out := map[int64][2]int64{}
for _, v := range b.m {
out[v.ID] = [2]int64{v.FileSize, v.ModTS}
}
return out, nil
}
// ---------- ProgressStore ----------
type progressKey struct {
userID, libID int64
bookPath string
}
type Progress struct {
m map[progressKey]store.Progress
libs *Libraries // for LibraryName join; may be nil
books *Books // for Title join; may be nil
}
func NewProgress(libs *Libraries, books *Books) *Progress {
return &Progress{m: map[progressKey]store.Progress{}, libs: libs, books: books}
}
func (p *Progress) UpsertProgress(_ context.Context, userID, libID int64, bookPath string, locator []byte, percent float64) error {
k := progressKey{userID, libID, bookPath}
old := p.m[k]
p.m[k] = store.Progress{LibraryID: libID, BookPath: bookPath, Locator: locator,
Percent: percent, UpdatedAt: now(), LibraryName: old.LibraryName, Title: old.Title}
return nil
}
func (p *Progress) GetProgress(_ context.Context, userID, libID int64, bookPath string) (store.Progress, error) {
if v, ok := p.m[progressKey{userID, libID, bookPath}]; ok {
return v, nil
}
return store.Progress{}, pgx.ErrNoRows
}
func (p *Progress) ListProgress(_ context.Context, userID int64) ([]store.Progress, error) {
out := []store.Progress{}
for k, v := range p.m {
if k.userID != userID {
continue
}
if p.libs != nil {
if l, e := p.libs.GetLibrary(context.Background(), v.LibraryID); e == nil {
v.LibraryName = l.Name
}
}
if p.books != nil {
for _, b := range p.books.m {
if b.LibraryID == v.LibraryID && b.Path == v.BookPath {
v.Title = b.Title
break
}
}
}
out = append(out, v)
}
sort.Slice(out, func(i, j int) bool { return out[i].BookPath < out[j].BookPath })
return out, nil
}
// ---------- BookmarkStore ----------
// bookmark pairs a row with its owner id. The real store carries the owner in a
// users-scoped join; the fake keeps it alongside so ListBookmarks/Patch/Delete
// can honour owner-scoped 404 semantics without a global index.
type bookmark struct {
row store.Bookmark
userID int64
}
type Bookmarks struct {
m map[int64]bookmark
next int64
}
func NewBookmarks() *Bookmarks { return &Bookmarks{m: map[int64]bookmark{}, next: 1} }
func (bm *Bookmarks) InsertBookmark(_ context.Context, userID, libID int64, bookPath string, locator []byte, percent float64, note string) (int64, error) {
id := bm.next
bm.next++
bm.m[id] = bookmark{
row: store.Bookmark{ID: id, LibraryID: libID, BookPath: bookPath,
Locator: locator, Percent: percent, Note: note, CreatedAt: now()},
userID: userID,
}
return id, nil
}
func (bm *Bookmarks) ListBookmarks(_ context.Context, userID, libID int64, bookPath string) ([]store.Bookmark, error) {
out := []store.Bookmark{}
for _, v := range bm.m {
if v.userID == userID && v.row.LibraryID == libID && v.row.BookPath == bookPath {
out = append(out, v.row)
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Percent != out[j].Percent {
return out[i].Percent < out[j].Percent
}
return out[i].ID < out[j].ID
})
return out, nil
}
func (bm *Bookmarks) UpdateBookmarkNote(_ context.Context, userID, id int64, note string) (bool, error) {
v, ok := bm.m[id]
if !ok || v.userID != userID { // owner-scoped: foreign id is a 404, not a 403
return false, nil
}
v.row.Note = note
bm.m[id] = v
return true, nil
}
func (bm *Bookmarks) DeleteBookmark(_ context.Context, userID, id int64) (bool, error) {
v, ok := bm.m[id]
if !ok || v.userID != userID {
return false, nil
}
delete(bm.m, id)
return true, nil
}
+41 -77
View File
@@ -7,7 +7,6 @@ import (
"io"
"io/fs"
"log"
"os"
"path/filepath"
"strings"
"sync"
@@ -15,21 +14,28 @@ import (
"booklib/internal/bookfile"
"booklib/internal/config"
"booklib/internal/media"
"booklib/internal/redispkg"
"booklib/internal/store"
)
// Sweeper 是 scanner 每轮顺手调用的清理钩子;upload.U 满足它(B16)。
type Sweeper interface {
Sweep(ctx context.Context) error
}
type Scanner struct {
st *store.Store
cfg *config.Config
rdb *redispkg.R
sweepers []Sweeper
// B9-②: per-library single-flight — concurrent scan triggers for the same
// library are merged into one execution, even without redis.
flights sync.Map // map[int64]*sync.WaitGroup
}
func New(st *store.Store, cfg *config.Config, rdb *redispkg.R) *Scanner {
return &Scanner{st: st, cfg: cfg, rdb: rdb}
func New(st *store.Store, cfg *config.Config, rdb *redispkg.R, sweepers ...Sweeper) *Scanner {
return &Scanner{st: st, cfg: cfg, rdb: rdb, sweepers: sweepers}
}
func (s *Scanner) Run(ctx context.Context) {
@@ -40,6 +46,11 @@ func (s *Scanner) Run(ctx context.Context) {
case <-ctx.Done():
return
case <-t.C:
for _, sw := range s.sweepers { // B16: 上传会话清扫随扫描周期跑
if err := sw.Sweep(ctx); err != nil {
log.Printf("scan: sweep: %v", err)
}
}
libs, err := s.st.ListLibraries(ctx)
if err != nil {
log.Printf("scan: list libraries: %v", err)
@@ -86,7 +97,7 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
defer unlock()
root, err := filepath.EvalSymlinks(filepath.Clean(lib.RootPath))
if err != nil || !inside(s.cfg.BooksDir, root) {
if err != nil || !bookfile.Contains(s.cfg.BooksDir, root) {
log.Printf("scan: library %d root %q rejected", lib.ID, lib.RootPath)
return
}
@@ -105,9 +116,9 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
delete(dbMeta, rel)
switch {
case !exists:
s.add(ctx, lib.ID, root, rel, ds)
s.ingest(ctx, lib.ID, 0, root, rel, ds, true)
case old.Size != ds.size || old.ModTS != ds.modTS:
s.update(ctx, lib.ID, old.ID, root, rel, ds)
s.ingest(ctx, lib.ID, old.ID, root, rel, ds, false)
}
}
for rel := range dbMeta { // 只剩被删的文件
@@ -120,11 +131,6 @@ func (s *Scanner) ScanLibrary(ctx context.Context, lib store.Library) {
type diskStat struct{ size, modTS int64 }
func inside(booksDir, root string) bool {
b := filepath.Clean(booksDir)
return root == b || strings.HasPrefix(root, b+string(os.PathSeparator))
}
func walk(root string) (map[string]diskStat, error) {
out := map[string]diskStat{}
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
@@ -157,7 +163,9 @@ func titleOf(rel string) string {
return strings.TrimSpace(strings.ReplaceAll(strings.TrimSuffix(base, filepath.Ext(base)), "_", " "))
}
func (s *Scanner) add(ctx context.Context, libID int64, root, rel string, ds diskStat) {
// ingest 是 add/update 的合一实现(Task 24):isNew 决定走 Insert 还是 UpdateBookFile,
// 之后的错误处理与封面生成完全共享。bookID 仅在 isNew=false 时有意义。
func (s *Scanner) ingest(ctx context.Context, libID, bookID int64, root, rel string, ds diskStat, isNew bool) {
format := bookfile.FormatFromExt(filepath.Base(rel))
pageCount := 0
var idxErr error
@@ -169,40 +177,21 @@ func (s *Scanner) add(ctx context.Context, libID int64, root, rel string, ds dis
idxErr = errors.New("no images in archive")
}
}
if isNew {
id, err := s.st.InsertBook(ctx, libID, rel, titleOf(rel), format, ds.size, ds.modTS, pageCount)
if err != nil {
log.Printf("scan: insert %s: %v", rel, err)
return
}
if idxErr != nil {
// B10: log SetBookState errors instead of discarding.
if e := s.st.SetBookState(ctx, id, "error", idxErr.Error()); e != nil {
log.Printf("scan: SetBookState error for %s: %v", rel, e)
}
return
}
s.cover(ctx, id, root, rel, format, ds)
}
func (s *Scanner) update(ctx context.Context, libID, bookID int64, root, rel string, ds diskStat) {
format := bookfile.FormatFromExt(filepath.Base(rel))
pageCount := 0
var idxErr error
if format == "cbz" {
idx, err := s.zipIndex(root, rel)
pageCount = len(idx)
idxErr = err
if idxErr == nil && pageCount == 0 {
idxErr = errors.New("no images in archive")
}
}
if err := s.st.UpdateBookFile(ctx, bookID, ds.size, ds.modTS, pageCount); err != nil {
bookID = id
} else if err := s.st.UpdateBookFile(ctx, bookID, ds.size, ds.modTS, pageCount); err != nil {
log.Printf("scan: update %s: %v", rel, err)
return
}
if idxErr != nil {
// B10: log SetBookState errors instead of discarding.
if e := s.st.SetBookState(ctx, bookID, "error", idxErr.Error()); e != nil {
log.Printf("scan: SetBookState error for %s: %v", rel, e)
log.Printf("scan: SetBookState %s: %v", rel, e)
}
return
}
@@ -210,66 +199,41 @@ func (s *Scanner) update(ctx context.Context, libID, bookID int64, root, rel str
}
func (s *Scanner) zipIndex(root, rel string) ([]string, error) {
f, err := os.Open(filepath.Join(root, filepath.FromSlash(rel)))
f, size, err := bookfile.OpenReaderAt(root, rel)
if err != nil {
return nil, err
}
defer f.Close()
st, err := f.Stat()
if err != nil {
return nil, err
}
return bookfile.PageIndex(f, st.Size())
return bookfile.PageIndex(f, size)
}
// cover writes the cover image to the cache dir. B11: all write errors are
// checked and logged; orphan .tmp files are cleaned only on failure path.
// cover writes the cover image to the cache dir via media.WriteAtomic.
// B11: write failures are logged; orphan .tmp files are cleaned only on failure.
func (s *Scanner) cover(ctx context.Context, id int64, root, rel, format string, ds diskStat) {
var img []byte
var ext string
var err error
var fn func(io.ReaderAt, int64) ([]byte, string, error)
switch format {
case "cbz":
img, ext, err = s.readCover(root, rel, bookfile.CBZCover)
fn = bookfile.CBZCover
case "epub":
img, ext, err = s.readCover(root, rel, bookfile.EPUBCover)
fn = bookfile.EPUBCover
default:
return // pdf/txt/md 用占位 SVG,不落盘
}
f, size, err := bookfile.OpenReaderAt(root, rel)
if err != nil {
log.Printf("scan: cover %s: %v", rel, err)
return
}
defer f.Close()
img, ext, err := fn(f, size)
if err != nil {
log.Printf("scan: cover %s: %v", rel, err)
return
}
dir := bookfile.CoverDir(s.cfg.CacheDir, bookfile.DirKey(id, bookfile.Hash(ds.size, ds.modTS)))
if e := os.MkdirAll(dir, 0o755); e != nil {
log.Printf("scan: coverdir %s: %v", rel, e)
return
if e := media.WriteAtomic(dir, "cover"+ext, img); e != nil {
log.Printf("scan: cover write %s: %v", rel, e)
}
tmp := filepath.Join(dir, "cover"+ext+".tmp")
dst := filepath.Join(dir, "cover"+ext)
if e := os.WriteFile(tmp, img, 0o644); e != nil {
log.Printf("scan: write cover tmp %s: %v", rel, e)
os.Remove(tmp)
return
}
if e := os.Rename(tmp, dst); e != nil {
log.Printf("scan: rename cover %s: %v", rel, e)
os.Remove(tmp)
return
}
}
func (s *Scanner) readCover(root, rel string, fn func(io.ReaderAt, int64) ([]byte, string, error)) ([]byte, string, error) {
f, err := os.Open(filepath.Join(root, filepath.FromSlash(rel)))
if err != nil {
return nil, "", err
}
defer f.Close()
st, err := f.Stat()
if err != nil {
return nil, "", err
}
return fn(f, st.Size())
}
func (s *Scanner) sweepCache(ctx context.Context) {
+1 -5
View File
@@ -2,11 +2,8 @@ package seed
import (
"context"
"errors"
"log"
"github.com/jackc/pgx/v5/pgconn"
"booklib/internal/auth"
"booklib/internal/store"
)
@@ -28,8 +25,7 @@ func Admin(ctx context.Context, s *store.Store, user, pass string) error {
return err
}
if _, err := s.CreateUser(ctx, user, h, "admin"); err != nil {
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) && pgErr.Code == "23505" {
if store.IsUniqueViolation(err) {
log.Printf("seed admin %q may already exist: %v", user, err)
return nil
}
+357
View File
@@ -0,0 +1,357 @@
// Package upload owns the resumable chunked-upload subsystem and the shared
// unique-path placement helper used by both chunked and single-file uploads.
//
// A session lives at <BooksDir>/.uploads/<uid>/ (meta.json + parts/N). The uid
// is a fingerprint of (libID, name, size, chunkSize), so re-initialising the
// same file resumes the existing session instead of restarting it. Expired
// sessions are swept by the scanner ticker (B16), not on the request path.
//
// This package is HTTP-free: it returns sentinel errors that the handlers map
// onto status/code/message tuples. The prior client-facing contract is
// preserved exactly.
package upload
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
"booklib/internal/bookfile"
"booklib/internal/ports"
)
const (
maxChunkBytes = 32 << 20
defaultChunk = 8 << 20
uploadSessTTL = 24 * time.Hour
uploadSessionIn = ".uploads"
)
// Sentinel errors returned to handlers for status/code mapping. The four shared
// with ports (ErrTooLarge/ErrNotFound/ErrIncomplete/ErrSizeMismatch) live in the
// ports package so the interface contract and the implementation agree.
var (
ErrBadName = errors.New("bad name")
ErrBadFormat = errors.New("bad format")
ErrBadSize = errors.New("bad size")
ErrBadChunk = errors.New("bad chunk size")
ErrBadUploadID = errors.New("bad upload id")
ErrBadIndex = errors.New("bad part index")
ErrPartTooBig = errors.New("part exceeds declared size")
ErrPartSizeMismatch = errors.New("part size mismatch")
ErrCorrupt = errors.New("corrupt session")
)
// OpError wraps an internal filesystem/IO failure with the short operation label
// that the client-facing 500 message uses, preserving the prior contract strings
// ("create session", "write meta", "create part", ...).
type OpError struct {
Op string
Err error
}
func (e *OpError) Error() string { return e.Op + ": " + e.Err.Error() }
func (e *OpError) Unwrap() error { return e.Err }
func opErr(op string, err error) error { return &OpError{Op: op, Err: err} }
type uploadMeta struct {
Name string `json:"name"`
Size int64 `json:"size"`
ChunkSize int64 `json:"chunkSize"`
LibraryID int64 `json:"libraryId"`
}
// U is the upload subsystem. It is stateless beyond the filesystem session dir.
type U struct {
booksDir string
uploadMaxMB int64
}
// New builds the upload subsystem. booksDir is the storage root (sessions live
// under booksDir/.uploads); uploadMaxMB caps the total declared file size.
func New(booksDir string, uploadMaxMB int64) *U {
return &U{booksDir: booksDir, uploadMaxMB: uploadMaxMB}
}
// compile-time proof that *U satisfies the consumer-side interface.
var _ ports.UploadSessions = (*U)(nil)
// ---------- pure helpers ----------
func validUploadID(s string) bool {
if len(s) != 32 {
return false
}
for _, r := range s {
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')) {
return false
}
}
return true
}
func uploadIDFor(libID int64, name string, size, chunk int64) string {
h := sha256.Sum256([]byte(fmt.Sprintf("%d|%s|%d|%d", libID, name, size, chunk)))
return hex.EncodeToString(h[:16])
}
func (u *U) uploadDir(uid string) string {
return filepath.Join(filepath.Clean(u.booksDir), uploadSessionIn, uid)
}
func chunkRange(m uploadMeta, i int64) (int64, int64) {
lo := i * m.ChunkSize
hi := min(lo+m.ChunkSize, m.Size)
return lo, hi
}
func numParts(m uploadMeta) int64 {
return (m.Size + m.ChunkSize - 1) / m.ChunkSize
}
// ---------- session meta ----------
// loadMeta validates uid + reads meta.json. Returns ErrBadUploadID,
// ports.ErrNotFound, or ErrCorrupt on failure.
func (u *U) loadMeta(uid string) (uploadMeta, string, error) {
if !validUploadID(uid) {
return uploadMeta{}, "", ErrBadUploadID
}
dir := u.uploadDir(uid)
b, e := os.ReadFile(filepath.Join(dir, "meta.json"))
if e != nil {
return uploadMeta{}, "", ports.ErrNotFound
}
var m uploadMeta
if json.Unmarshal(b, &m) != nil {
return uploadMeta{}, "", ErrCorrupt
}
return m, dir, nil
}
// LibraryID returns the target library id recorded in the session, so the
// handler can resolve+validate the library root before calling Complete.
func (u *U) LibraryID(ctx context.Context, uid string) (int64, error) {
m, _, e := u.loadMeta(uid)
if e != nil {
return 0, e
}
return m.LibraryID, nil
}
// ---------- public API (satisfies ports.UploadSessions) ----------
// Init validates the declared upload, then creates or resumes a session. The
// returned uid is deterministic for a given (libID, safeName, size, chunkSize),
// so a re-init of the same file resumes; a fingerprint collision with different
// content restarts the session.
func (u *U) Init(ctx context.Context, libID int64, name string, size, chunkSize int64) (string, error) {
safe := bookfile.SafeName(name)
if safe == "" {
return "", ErrBadName
}
if bookfile.FormatFromExt(safe) == "" {
return "", ErrBadFormat
}
if size <= 0 {
return "", ErrBadSize
}
if size > u.uploadMaxMB<<20 {
return "", ports.ErrTooLarge
}
if chunkSize == 0 {
chunkSize = defaultChunk
}
if chunkSize > maxChunkBytes {
return "", ErrBadChunk
}
uid := uploadIDFor(libID, safe, size, chunkSize)
dir := u.uploadDir(uid)
meta := uploadMeta{Name: safe, Size: size, ChunkSize: chunkSize, LibraryID: libID}
if b, e := os.ReadFile(filepath.Join(dir, "meta.json")); e == nil {
var old uploadMeta
if json.Unmarshal(b, &old) == nil && old == meta { // same fingerprint → resume
return uid, nil
}
os.RemoveAll(dir) // fingerprint collided but content differs → restart
}
if e := os.MkdirAll(filepath.Join(dir, "parts"), 0o755); e != nil {
return "", opErr("create session", e)
}
b, _ := json.Marshal(meta)
if e := os.WriteFile(filepath.Join(dir, "meta.json"), b, 0o644); e != nil {
return "", opErr("write meta", e)
}
return uid, nil
}
// Status returns the sorted indices of parts already received.
func (u *U) Status(ctx context.Context, uid string) ([]int64, error) {
_, dir, e := u.loadMeta(uid)
if e != nil {
return nil, e
}
recv := []int64{}
es, e := os.ReadDir(filepath.Join(dir, "parts"))
if e == nil {
for _, en := range es {
if i, e := strconv.ParseInt(en.Name(), 10, 64); e == nil {
recv = append(recv, i)
}
}
}
sort.Slice(recv, func(i, j int) bool { return recv[i] < recv[j] })
return recv, nil
}
// PutPart writes one part via tmp+rename (B4: a truncated part is never reported
// as received). body is read up to the declared part size; reading past it yields
// ErrPartTooBig, a short read yields ErrPartSizeMismatch. maxSize, when > 0, is a
// defensive ceiling on bytes read.
func (u *U) PutPart(ctx context.Context, uid string, index int64, body io.Reader, maxSize int64) error {
m, dir, e := u.loadMeta(uid)
if e != nil {
return e
}
if index < 0 || index >= numParts(m) {
return ErrBadIndex
}
lo, hi := chunkRange(m, index)
want := hi - lo
limit := want + 1
if maxSize > 0 && maxSize+1 < limit {
limit = maxSize + 1
}
p := filepath.Join(dir, "parts", strconv.FormatInt(index, 10))
tmp := p + ".tmp"
f, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil {
return opErr("create part", e)
}
n, copyErr := io.Copy(f, io.LimitReader(body, limit))
f.Close()
if copyErr != nil {
os.Remove(tmp)
return ErrPartSizeMismatch
}
if n > want {
os.Remove(tmp)
return ErrPartTooBig
}
if n != want {
os.Remove(tmp)
return ErrPartSizeMismatch
}
if e := os.Rename(tmp, p); e != nil {
os.Remove(tmp)
return opErr("rename part", e)
}
return nil
}
// Complete verifies every part is present and correctly sized, assembles them
// into a tmp file, then atomically renames it into root under a unique name.
// It returns the path relative to root. The session dir is removed on success.
func (u *U) Complete(ctx context.Context, uid, root string) (string, error) {
m, dir, e := u.loadMeta(uid)
if e != nil {
return "", e
}
var total int64
for i := int64(0); i < numParts(m); i++ {
lo, hi := chunkRange(m, i)
fi, e := os.Stat(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil || fi.Size() != hi-lo {
return "", ports.ErrIncomplete
}
total += fi.Size()
}
if total != m.Size {
return "", ports.ErrSizeMismatch
}
dst, e := u.UniquePath(root, m.Name)
if e != nil {
return "", e // os.ErrInvalid / os.ErrExist → handler maps to 403
}
tmp := filepath.Join(dir, "assembled")
out, e := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if e != nil {
return "", opErr("create tmp", e)
}
for i := int64(0); i < numParts(m); i++ {
pf, e := os.Open(filepath.Join(dir, "parts", strconv.FormatInt(i, 10)))
if e != nil {
out.Close()
return "", opErr("open part", e)
}
_, copyErr := io.Copy(out, pf)
pf.Close()
if copyErr != nil {
out.Close()
os.Remove(tmp)
return "", opErr("assemble", copyErr)
}
}
out.Close()
if e := os.Rename(tmp, dst); e != nil { // atomic placement; scanner picks it up
os.Remove(tmp)
return "", opErr("rename", e)
}
os.RemoveAll(dir)
return strings.TrimPrefix(dst, root+string(os.PathSeparator)), nil
}
// Sweep removes session dirs untouched for longer than the TTL. Best-effort:
// a missing/unreadable base dir is not an error. Run from the scanner ticker
// (B16) instead of the request path.
func (u *U) Sweep(ctx context.Context) error {
base := filepath.Join(filepath.Clean(u.booksDir), uploadSessionIn)
es, e := os.ReadDir(base)
if e != nil {
return nil // no sessions yet
}
for _, en := range es {
if fi, e := en.Info(); e == nil && time.Since(fi.ModTime()) > uploadSessTTL {
os.RemoveAll(filepath.Join(base, en.Name()))
}
}
return nil
}
// UniquePath returns a path under root for name that does not yet exist,
// appending " (n)" on collision. The cleaned name must stay inside root.
// Shared by single-file and chunked upload completion.
func (u *U) UniquePath(root, name string) (string, error) {
ext := filepath.Ext(name)
base := strings.TrimSuffix(name, ext)
for i := 0; ; i++ {
cand := base + ext
if i > 0 {
cand = base + " (" + strconv.Itoa(i) + ")" + ext
}
p := filepath.Join(root, cand)
if filepath.Clean(p) != filepath.Join(root, filepath.Clean(cand)) ||
!strings.HasPrefix(filepath.Clean(p), root+string(os.PathSeparator)) {
return "", os.ErrInvalid
}
if _, e := os.Stat(p); os.IsNotExist(e) {
return p, nil
} else if e != nil {
return "", e
}
if i > 999 {
return "", os.ErrExist
}
}
}
+252
View File
@@ -0,0 +1,252 @@
package upload
import (
"bytes"
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"booklib/internal/ports"
)
func newU(t *testing.T) *U {
t.Helper()
dir := t.TempDir()
resolved, err := filepath.EvalSymlinks(dir)
if err != nil {
t.Fatal(err)
}
return New(resolved, 1) // 1MB cap, mirrors handler test cfg
}
func mustInit(t *testing.T, u *U, libID int64, name string, size, chunk int64) string {
t.Helper()
uid, err := u.Init(context.Background(), libID, name, size, chunk)
if err != nil {
t.Fatalf("Init(%q): %v", name, err)
}
return uid
}
func TestUploadIDForDeterministic(t *testing.T) {
a := uploadIDFor(1, "x.cbz", 100, 50)
b := uploadIDFor(1, "x.cbz", 100, 50)
c := uploadIDFor(2, "x.cbz", 100, 50)
if a != b {
t.Fatalf("same fingerprint must yield same uid: %s vs %s", a, b)
}
if a == c {
t.Fatal("different libID must yield different uid")
}
if !validUploadID(a) || validUploadID("zzz") || validUploadID(strings.Repeat("a", 31)) {
t.Fatal("validUploadID broken")
}
}
func TestInitValidation(t *testing.T) {
u := newU(t)
ctx := context.Background()
if _, err := u.Init(ctx, 1, "", 100, 0); !errors.Is(err, ErrBadName) {
t.Fatalf("empty name: want ErrBadName got %v", err)
}
if _, err := u.Init(ctx, 1, "virus.exe", 100, 0); !errors.Is(err, ErrBadFormat) {
t.Fatalf("bad ext: want ErrBadFormat got %v", err)
}
if _, err := u.Init(ctx, 1, "ok.cbz", 0, 0); !errors.Is(err, ErrBadSize) {
t.Fatalf("zero size: want ErrBadSize got %v", err)
}
if _, err := u.Init(ctx, 1, "big.cbz", 2<<20, 0); !errors.Is(err, ports.ErrTooLarge) {
t.Fatalf("oversize: want ErrTooLarge got %v", err)
}
if _, err := u.Init(ctx, 1, "ok.cbz", 100, 40<<20); !errors.Is(err, ErrBadChunk) {
t.Fatalf("huge chunk: want ErrBadChunk got %v", err)
}
// 默认 chunk 生效且 uid 稳定
uid := mustInit(t, u, 1, "ok.cbz", 100, 0)
if uid != uploadIDFor(1, "ok.cbz", 100, defaultChunk) {
t.Fatal("chunkSize=0 must default to defaultChunk in fingerprint")
}
// SafeName 清洗:path 形式取 base
if uid2 := mustInit(t, u, 1, `C:\dir\book.cbz`, 100, 0); uid2 != uploadIDFor(1, "book.cbz", 100, defaultChunk) {
t.Fatalf("windows path must be sanitized to base name, got uid %s", uid2)
}
}
func TestInitResumeAndRestart(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 1, "r.cbz", 900000, 400000)
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("x"), 400000)), 0); err != nil {
t.Fatalf("PutPart: %v", err)
}
// 同指纹 re-init → 复用会话,分片保留
uid2, err := u.Init(ctx, 1, "r.cbz", 900000, 400000)
if err != nil || uid2 != uid {
t.Fatalf("resume: want same uid, got %s err=%v", uid2, err)
}
recv, err := u.Status(ctx, uid)
if err != nil || len(recv) != 1 || recv[0] != 0 {
t.Fatalf("resume must keep parts: %v err=%v", recv, err)
}
// 同指纹位但内容不同(size 变)→ 新会话,旧目录被换掉是安全的(uid 不同)
uid3 := mustInit(t, u, 1, "r.cbz", 800000, 400000)
if uid3 == uid {
t.Fatal("different size must yield different uid")
}
}
func TestPutPartErrors(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 1, "p.cbz", 1000, 400)
if err := u.PutPart(ctx, uid, 9, bytes.NewReader(bytes.Repeat([]byte("y"), 400)), 0); !errors.Is(err, ErrBadIndex) {
t.Fatalf("index oob: want ErrBadIndex got %v", err)
}
if err := u.PutPart(ctx, uid, -1, bytes.NewReader(nil), 0); !errors.Is(err, ErrBadIndex) {
t.Fatalf("negative index: want ErrBadIndex got %v", err)
}
// 超期望体积 → ErrPartTooBig
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("y"), 500)), 0); !errors.Is(err, ErrPartTooBig) {
t.Fatalf("oversize part: want ErrPartTooBig got %v", err)
}
// 短读 → ErrPartSizeMismatch
if err := u.PutPart(ctx, uid, 0, bytes.NewReader(bytes.Repeat([]byte("y"), 300)), 0); !errors.Is(err, ErrPartSizeMismatch) {
t.Fatalf("short part: want ErrPartSizeMismatch got %v", err)
}
// 失败不留 parts/B4: 截断分片不被 Status 报告为已接收
recv, err := u.Status(ctx, uid)
if err != nil || len(recv) != 0 {
t.Fatalf("failed parts must not be received: %v err=%v", recv, err)
}
// 未知/非法 uid
if _, err := u.Status(ctx, "deadbeefdeadbeefdeadbeefdeadbeef"); !errors.Is(err, ports.ErrNotFound) {
t.Fatalf("unknown uid: want ErrNotFound got %v", err)
}
if _, err := u.Status(ctx, "zzz"); !errors.Is(err, ErrBadUploadID) {
t.Fatalf("bad uid: want ErrBadUploadID got %v", err)
}
}
func TestCompleteHappyPathAndCleanup(t *testing.T) {
u := newU(t)
ctx := context.Background()
root := filepath.Join(u.booksDir, "lib")
if err := os.MkdirAll(root, 0o755); err != nil {
t.Fatal(err)
}
content := bytes.Repeat([]byte("調教開關第二季!"), 300) // ~6.3KB multibyte
size := int64(len(content))
const chunk = int64(2000)
uid := mustInit(t, u, 7, "調教開關:第二季.zip", size, chunk)
n := int((size + chunk - 1) / chunk)
// 乱序上传:索引顺序 1,2,...,n-1,0 —— 覆盖所有分片且非递增
for step := 1; step <= n; step++ {
idx := int64(step % n)
lo := idx * chunk
hi := min(lo+chunk, size)
if err := u.PutPart(ctx, uid, idx, bytes.NewReader(content[lo:hi]), 0); err != nil {
t.Fatalf("PutPart %d: %v", idx, err)
}
}
// 缺片 → ErrIncomplete(先删一片验证)
part0 := filepath.Join(u.uploadDir(uid), "parts", "0")
saved, _ := os.ReadFile(part0)
os.Remove(part0)
if _, err := u.Complete(ctx, uid, root); !errors.Is(err, ports.ErrIncomplete) {
t.Fatalf("missing part: want ErrIncomplete got %v", err)
}
os.WriteFile(part0, saved, 0o644)
rel, err := u.Complete(ctx, uid, root)
if err != nil {
t.Fatalf("Complete: %v", err)
}
if rel != "調教開關:第二季.zip" {
t.Fatalf("rel path: %q", rel)
}
got, err := os.ReadFile(filepath.Join(root, rel))
if err != nil || !bytes.Equal(got, content) {
t.Fatalf("assembled content wrong: err=%v", err)
}
if _, err := os.Stat(u.uploadDir(uid)); !os.IsNotExist(err) {
t.Fatal("session dir must be removed after complete")
}
}
func TestCompleteUniquePathCollision(t *testing.T) {
u := newU(t)
ctx := context.Background()
root := filepath.Join(u.booksDir, "lib")
os.MkdirAll(root, 0o755)
os.WriteFile(filepath.Join(root, "dup.cbz"), []byte("existing"), 0o644)
uid := mustInit(t, u, 1, "dup.cbz", 4, 4)
if err := u.PutPart(ctx, uid, 0, bytes.NewReader([]byte("new!")), 0); err != nil {
t.Fatal(err)
}
rel, err := u.Complete(ctx, uid, root)
if err != nil {
t.Fatalf("Complete: %v", err)
}
if rel != "dup (1).cbz" {
t.Fatalf("collision must add suffix, got %q", rel)
}
}
func TestUniquePathTraversalRejected(t *testing.T) {
u := newU(t)
root := filepath.Join(u.booksDir, "lib")
if _, err := u.UniquePath(root, "../escape.cbz"); !errors.Is(err, os.ErrInvalid) {
t.Fatalf("traversal: want os.ErrInvalid got %v", err)
}
p, err := u.UniquePath(root, "ok.cbz")
if err != nil || p != filepath.Join(root, "ok.cbz") {
t.Fatalf("valid name: %q %v", p, err)
}
}
func TestSweepRemovesOnlyExpired(t *testing.T) {
u := newU(t)
ctx := context.Background()
fresh := mustInit(t, u, 1, "fresh.cbz", 10, 10)
stale := mustInit(t, u, 1, "stale.cbz", 10, 10)
// 把 stale 会话 mtime 拨到 TTL 之前
old := time.Now().Add(-uploadSessTTL - time.Hour)
dir := u.uploadDir(stale)
if err := os.Chtimes(filepath.Join(dir, "meta.json"), old, old); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(dir, old, old); err != nil {
t.Fatal(err)
}
if err := u.Sweep(ctx); err != nil {
t.Fatalf("Sweep: %v", err)
}
if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Fatal("stale session must be swept")
}
if _, err := os.Stat(u.uploadDir(fresh)); err != nil {
t.Fatalf("fresh session must survive: %v", err)
}
// 空目录/不存在 base 都不报错
os.RemoveAll(filepath.Join(u.booksDir, uploadSessionIn))
if err := u.Sweep(ctx); err != nil {
t.Fatalf("Sweep on missing base: %v", err)
}
}
func TestLibraryID(t *testing.T) {
u := newU(t)
ctx := context.Background()
uid := mustInit(t, u, 42, "lib.cbz", 10, 10)
id, err := u.LibraryID(ctx, uid)
if err != nil || id != 42 {
t.Fatalf("LibraryID: %d %v", id, err)
}
if _, err := u.LibraryID(ctx, "deadbeefdeadbeefdeadbeefdeadbeef"); !errors.Is(err, ports.ErrNotFound) {
t.Fatalf("unknown uid: want ErrNotFound got %v", err)
}
}
+43 -36
View File
@@ -8,6 +8,49 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab
## [Unreleased]
### Added / 新增
- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with a periodic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk).
- API:新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,定期清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。
- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent).
- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。
- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization.
- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。
- Tests: router contract test (`TestRouterContract`) pins the full route table — any added, removed or renamed route fails the test until the expectation is updated deliberately.
- 测试:新增路由契约测试(`TestRouterContract`),锁定全量路由表——任何路由的增删改名都会使该测试失败,必须显式更新预期。
- Tests: hand-written in-memory fakes (`internal/ports/portsfake`) cover all port interfaces, enabling handler unit tests with no PG or Redis. Error semantics mirror the real store (`pgx.ErrNoRows`, `store.ErrLastAdmin`, `*pgconn.PgError{Code:23505}`), so the fakes exercise the same 404/409/400 branches as the database.
- 测试:新增手写内存 fake(`internal/ports/portsfake`),覆盖全部 port 接口,使 handler 单测无需 PG/Redis 即可运行。错误语义与真实 store 一致(`pgx.ErrNoRows`、`store.ErrLastAdmin`、`*pgconn.PgError{Code:23505}`),因此 fake 走到的是与真库相同的 404/409/400 分支。
- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions.
- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。
### Changed / 变更
- Backend restructured around hexagonal ports: HTTP handlers now depend only on small consumer-side interfaces (`internal/ports`) instead of concrete `*store.Store` / `*redispkg.R` / `*scanner.Scanner`. Domain logic moved out of handlers into `internal/media` (cover/page extraction, page index cache, atomic cache writes) and `internal/upload` (chunked session lifecycle). `cmd/webui/main.go` is the composition root; `api.NewRouter` accepts pure interfaces.
- 后端按六边形端口重构:HTTP handler 现在只依赖 `internal/ports` 中的小口径消费端接口,不再直接持有 `*store.Store` / `*redispkg.R` / `*scanner.Scanner` 等具体类型。域逻辑从 handler 下沉到 `internal/media`(封面/页抽取、页索引缓存、缓存原子写)与 `internal/upload`(分片会话生命周期)。`cmd/webui/main.go` 作为装配根,`api.NewRouter` 只收接口。
- Upload session sweep moved off the request path onto the scanner's ticker cycle (B16), so `POST /upload/init` no longer pays for a directory walk.
- 上传会话清理从请求路径移到扫描器的定时周期(B16),`POST /upload/init` 不再顺带付出一次目录遍历的开销。
- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward.
- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。
- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader.
- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。
- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`.
- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。
- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/<sanitized name>` (no client-supplied paths, validated at creation).
- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。
- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked.
- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`.
### Fixed / 修复
- Rate limiter `IncrWindow` uses atomic Lua script for INCR+EXPIRE, preventing permanent IP lockout on EXPIRE failure (B1).
@@ -36,45 +79,9 @@ The format loosely follows Keep a Changelog and can be adapted to the team's hab
- 上传处理器在 `O_EXCL` 冲突时重试,处理并发同名上传(B12)。
- Bookmark methods check `err` before `RowsAffected` to avoid invalid reads on query failure (B13).
- 书签方法先检查 `err` 再读 `RowsAffected`,避免查询失败时的无效读取(B13)。
### Changed / 变更
- Add ordered migration system with `schema_migrations` tracking and pg advisory lock for safe multi-replica schema evolution. Existing databases are auto-baselined. To change the schema, add a new `NNNN_description.sql` file under `backend/internal/db/migrations/`; never modify an already-applied file. No down migrations — rollback via backup, fix-forward.
- 新增有序迁移系统,通过 `schema_migrations` 表和 pg advisory lock 实现安全的多副本 schema 演进,已有数据库自动基线化。修改 schema 时在 `backend/internal/db/migrations/` 下新增 `NNNN_description.sql`,已应用的文件不可修改。不支持 down 迁移——回滚靠备份,fix-forward。
### Fixed / 修复
- Serve goroutine `log.Fatalf` replaced with channel-based shutdown to preserve graceful teardown (B14).
- 服务 goroutine 中的 `log.Fatalf` 改为 channel 通知方式,确保优雅关停流程不被绕过(B14)。
- `DATABASE_URL` is now validated at startup (required, parseable); empty `REDIS_URL` logs a clear "redis disabled" message (B15).
- `DATABASE_URL` 在启动时校验(必填、可解析);空 `REDIS_URL` 记录明确的 "redis disabled" 日志(B15)。
- `scripts/smoke.sh` aligned with current API contract, removed ignored `root_path` field (B17).
- `scripts/smoke.sh` 对齐当前 API 契约,移除被忽略的 `root_path` 字段(B17)。
### Added / 新增
- CI workflow (`.github/workflows/ci.yml`) compatible with both GitHub Actions and Gitea Actions.
- CI 工作流(`.github/workflows/ci.yml`),兼容 GitHub Actions 和 Gitea Actions。
### Added / 新增
- API: per-user bookmarks — `GET/POST /api/books/:id/bookmarks` (locator+percent snapshot with optional ≤500-char note; list ordered by percent) and `PATCH/DELETE /api/bookmarks/:id`; not-yours uniformly 404. New `bookmarks` table keyed like progress, cleaned up with the user (no cascade on book delete, same precedent).
- API:新增按用户隔离的书签——`GET/POST /api/books/:id/bookmarks`(存当前 locator+percent,备注可选、≤500 字,列表按进度升序)与 `PATCH/DELETE /api/bookmarks/:id`;不属于自己的一律 404。新 `bookmarks` 表与进度同款定位键,随用户删除而清(删书不级联,沿用既有先例)。
- API: CBZ page indexing now skips macOS packaging junk (`__MACOSX/…` and `._*` AppleDouble files), which used to land in the page list as ~163-byte black "pages"; `GET /api/books/:id/pages` additionally returns `chapters:[{title,start}]` derived from the archive's folder structure (e.g. 第1話…), so per-folder comics expose their real organization.
- API:CBZ 页索引现会跳过 macOS 打包垃圾(`__MACOSX/…` 与 `._*` 资源叉文件),此前它们以 ~163 字节黑页混入页列表;`GET /api/books/:id/pages` 新增 `chapters:[{title,start}]`,按压缩包内目录结构(如 第1話…)给出真实章节。
- API: resumable chunked upload protocol for large files — `POST /api/libraries/:id/upload/init` (fingerprint-derived deterministic `uploadId`, rejects totals over `UPLOAD_MAX_MB` with `413 too_large`), `PUT /api/uploads/:uid/parts/:index` (parts ≤ 32MB), `GET /api/uploads/:uid` (received parts, for resume), `POST /api/uploads/:uid/complete` (assemble + atomic land, same path contract as single-POST upload). Sessions persist under `BOOKS_DIR/.uploads/` with 24h opportunistic sweep. `UPLOAD_MAX_MB` is now wired through both compose stacks/`.env`; `.env.example` sets 2048 and drops `NGINX_CLIENT_MAX_BODY_SIZE` to 32m (nginx only ever sees one chunk).
- API: 新增大文件可续传分片上传协议——`POST /api/libraries/:id/upload/init`(按指纹派生确定性 `uploadId`,总量超 `UPLOAD_MAX_MB` 返回 `413 too_large`)、`PUT /api/uploads/:uid/parts/:index`(单片 ≤32MB)、`GET /api/uploads/:uid`(查询已传分片以续传)、`POST /api/uploads/:uid/complete`(拼接后原子落盘,返回与单发上传一致的 `path`)。会话存于 `BOOKS_DIR/.uploads/`,超 24h 顺手清理。`UPLOAD_MAX_MB` 已接入两份 compose/`.env`;`.env.example` 调至 2048 并将 `NGINX_CLIENT_MAX_BODY_SIZE` 降为 32m(nginx 只见单个分片)。
### Changed / 变更
- Scanner: an image-list-less archive (`.zip`/`.cbz` with no page images — video packs, document dumps) is now recorded as `state=error` ("no images in archive") instead of registering as an empty CBZ with a blank reader.
- 扫描器:不含任何图片条目的 `.zip`/`.cbz`(视频包、文档包)现记录为 `state=error`("no images in archive"),不再注册成空 CBZ 留下一个白板阅读器。
- API: upload over `UPLOAD_MAX_MB` now returns `413 too_large` with the limit in the message; previously the size abort was misreported as `400 bad_request "multipart field 'file' required"`.
- API:超过 `UPLOAD_MAX_MB` 的上传现在返回 `413 too_large` 并在消息中带上限额;此前体积超限被误报为 `400 bad_request "multipart field 'file' required"`。
- API: `POST /api/libraries` now takes only `{name}`; `root_path` is generated server-side as `BOOKS_DIR/<sanitized name>` (no client-supplied paths, validated at creation).
- API:`POST /api/libraries` 只需 `{name}`;`root_path` 由服务端生成为 `BOOKS_DIR/<清洗后的库名>`(不再接受客户端指定路径,创建时即校验)。
- Repo structure conformed to `AGENTS.md`: `web/` renamed to `frontend/`; backend HTTP layer moved from `internal/api` to `cmd/webui/{api,handlers}` (`cmd/server` → `cmd/webui`); README/CHANGELOGs relocated under `docs/` (`README_zh.md` added as Chinese mirror); module-level `.gitignore`s added (`backend/`, `deploy/`); stray root `library/` removed (book files live in `deploy/api/storage/`); debug binaries untracked.
- Docs: `docs/README.md` is now the English primary; previous Chinese README mirrored to `docs/README_zh.md`.
+16
View File
@@ -78,6 +78,22 @@ Schema changes go through the ordered migration system in `backend/internal/db/m
Local gate before each batch merge: `go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...` with dev PG+Redis running.
## Backend structure
The backend is organized around consumer-side port interfaces (hexagonal style):
- `cmd/webui` — binary entry point and composition root: `main.go` builds concrete implementations (`store.Store`, `redispkg.R`, `scanner.Scanner`, `media.M`, `upload.U`) and hands them to `api.NewRouter`, which only accepts the port interfaces.
- `cmd/webui/handlers` — HTTP layer: request binding, auth/authz, error → status mapping. No SQL, no archive/file logic.
- `internal/ports` — the small interfaces handlers depend on (`UserStore`, `LibraryStore`, `BookStore`, `ProgressStore`, `BookmarkStore`, `RateLimiter`, `Scanner`, `Media`, `UploadSessions`) plus shared sentinel errors. Interfaces live on the consumer side, implementations satisfy them.
- `internal/ports/portsfake` — hand-written in-memory fakes for every port, with error semantics mirroring the real store (`pgx.ErrNoRows`, `ErrLastAdmin`, PgError 23505). Handler unit tests run against these with no PG/Redis.
- `internal/media` — cover/page extraction, page-index cache (Redis-backed), atomic cache writes.
- `internal/upload` — chunked upload session lifecycle (init/part/status/complete/sweep).
- `internal/store` — all SQL, one place.
- `internal/scanner` — library walk, ingest (add/update/delete in one pass), sweep riding the scan ticker.
- `internal/bookfile` — shared file utilities (`SafeName`, `Contains`, `Hash`, `FormatFromExt`, cache dir layout).
Testing is two-tiered: integration tests hit a real PG+Redis via the full router (`handlers/*_test.go` with `setupAPI`), unit tests hit the same router with `portsfake` injected (`handlers/*_unit_test.go`). The route table itself is pinned by `TestRouterContract` in `cmd/webui/api`.
## CI
- Workflow: `.github/workflows/ci.yml` (standard GitHub Actions syntax, Gitea Actions compatible).
+16
View File
@@ -78,6 +78,22 @@ Schema 变更通过 `backend/internal/db/migrations/` 中的有序迁移系统
每批合入前的本地门禁:`go vet ./... && gofmt -l . && go test -p 1 -count=1 ./...`(需启动 dev PG+Redis)。
## 后端结构
后端按消费端接口组织(六边形风格):
- `cmd/webui` —— 二进制入口与装配根:`main.go` 构造具体实现(`store.Store`、`redispkg.R`、`scanner.Scanner`、`media.M`、`upload.U`)并交给 `api.NewRouter`,后者只接受 port 接口。
- `cmd/webui/handlers` —— HTTP 层:参数绑定、认证/鉴权、错误→状态码映射。没有 SQL,没有压缩包/文件逻辑。
- `internal/ports` —— handlers 依赖的小口径接口(`UserStore`、`LibraryStore`、`BookStore`、`ProgressStore`、`BookmarkStore`、`RateLimiter`、`Scanner`、`Media`、`UploadSessions`)与共享 sentinel 错误。接口定义在消费端,实现方来满足它们。
- `internal/ports/portsfake` —— 全部 port 的手写内存 fake,错误语义与真实 store 一致(`pgx.ErrNoRows`、`ErrLastAdmin`、PgError 23505)。handler 单测无需 PG/Redis。
- `internal/media` —— 封面/页抽取、页索引缓存(Redis)、缓存原子写。
- `internal/upload` —— 分片上传会话生命周期(init/part/status/complete/sweep)。
- `internal/store` —— 所有 SQL,集中一处。
- `internal/scanner` —— 书库遍历、ingest(增/改/删一趟完成)、会话清扫搭扫描 ticker 顺风车。
- `internal/bookfile` —— 共享文件工具(`SafeName`、`Contains`、`Hash`、`FormatFromExt`、缓存目录布局)。
测试分两层:集成测试走真实 PG+Redis、过完整 router(`handlers/*_test.go` 的 `setupAPI`);单测注入 `portsfake`、过同一个 router(`handlers/*_unit_test.go`)。路由表本身由 `cmd/webui/api` 的 `TestRouterContract` 钉死。
## CI
- 工作流:`.github/workflows/ci.yml`(标准 GitHub Actions 语法,兼容 Gitea Actions)。