update docs
This commit is contained in:
@@ -0,0 +1,92 @@
|
||||
# AGENTS.md — backend
|
||||
|
||||
## Module & Layout
|
||||
|
||||
Go module: `github.com/XingfenD/yoresee_doc` (rooted at `backend/`).
|
||||
|
||||
```
|
||||
cmd/main.go — API server entry (Connect RPC)
|
||||
cmd/migrate/ — GORM AutoMigrate (runs first on startup)
|
||||
cmd/db_init/ — seed data (admin user, default groups, templates…)
|
||||
cmd/es_init/ — Elasticsearch index setup
|
||||
cmd/notification-worker/ — RabbitMQ consumer
|
||||
cmd/search-sync-worker/ — RabbitMQ consumer
|
||||
cmd/snapshot-worker/ — RabbitMQ consumer (Yjs snapshot persistence)
|
||||
|
||||
internal/
|
||||
bootstrap/ — chain-style Initializer (config → infra → repos → services)
|
||||
config/ — Viper TOML loader; GlobalConfig singleton
|
||||
transport/connectserver/ — HTTP mux, Connect RPC handler wiring, auth interceptor, CORS
|
||||
transport/grpcserver/ — per-service RPC implementations (calls into service layer)
|
||||
service/<domain>_service/ — business logic; package-level singletons (*Svc)
|
||||
service/interface/ — service interfaces + MQ consumer contract
|
||||
service/mq_service/ — MQ abstraction init (RabbitMQ)
|
||||
repository/<domain>_repo/ — GORM queries; constructed via repository.NewRepositories()
|
||||
model/ — GORM model definitions (16 models)
|
||||
dto/ mapper/ types/ — transfer objects, mapping helpers, domain types
|
||||
status/ — typed error codes (status.GenErrWithCustomMsg)
|
||||
|
||||
pkg/
|
||||
storage/ — infra clients: DB, KVS (Redis), MinioClient, ES, Consul (package-level globals)
|
||||
mq/ — MQ interface + RabbitMQ impl
|
||||
cache/ lock/ key/ — shared cache keys, distributed lock, cache helpers
|
||||
errs/ constant/ — shared errors and constants
|
||||
gen/ — GENERATED protobuf Go stubs (gitignored)
|
||||
```
|
||||
|
||||
## Build & Run
|
||||
|
||||
```bash
|
||||
# Full dev stack (from repo root):
|
||||
bash deploy/script/start.sh dev up
|
||||
|
||||
# Run backend only (after infra is up):
|
||||
go run ./cmd/main.go
|
||||
|
||||
# Run individual workers:
|
||||
go run ./cmd/snapshot-worker
|
||||
go run ./cmd/notification-worker
|
||||
go run ./cmd/search-sync-worker
|
||||
|
||||
# Migrate + seed (runs automatically in container):
|
||||
go run ./cmd/migrate && go run ./cmd/db_init && go run ./cmd/es_init
|
||||
```
|
||||
|
||||
No Makefile, no tests, no linter config. Verify with `go build ./...` and `go vet ./...`.
|
||||
|
||||
## Config
|
||||
|
||||
- File: `config.toml` (gitignored) — generated from `config.toml.tmpl` by `deploy/script/prepare.sh`
|
||||
- Format: TOML, loaded by Viper with `${ENV_VAR}` substitution
|
||||
- Env vars defined in `deploy/.env` (template: `deploy/.env.example`)
|
||||
- Access via `config.GlobalConfig.*`
|
||||
|
||||
## Startup Order
|
||||
|
||||
Container CMD: `migrate → db_init → es_init → main`
|
||||
Workers are separate binaries with their own bootstrap chains.
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Connect RPC** (not raw gRPC server): handlers in `connectserver/handlers.go` wrap `grpcserver/*` implementations
|
||||
- **Two HTTP listeners**: gRPC on `grpc_port` (9090), gRPC-web on `grpc_web_port` (8080); both use h2c
|
||||
- **Auth interceptor**: JWT-based; allowlist for Login, Register, Health, SystemInfo
|
||||
- **Service singletons**: `auth_service.AuthSvc`, `document_service.DocumentSvc`, etc. — set in `service.Init()`
|
||||
- **Global infra**: `storage.DB`, `storage.KVS`, `storage.MinioClient`, `storage.ES`, `storage.Consul`
|
||||
- **Migration**: GORM `AutoMigrate` (schema from model structs) + raw SQL for Postgres extensions (`ltree`) and indexes
|
||||
- **DB seeding**: `cmd/db_init/` creates admin user, default permissions, templates, user groups
|
||||
- **Elasticsearch**: optional — `InitElasticsearchAllowFail` degrades gracefully
|
||||
- **Consul**: required — `RequireConsulEnabled` fatals if disabled
|
||||
- **MQ**: RabbitMQ via `pkg/mq`; workers consume topics registered through `mq_service`
|
||||
|
||||
## Protobuf Codegen
|
||||
|
||||
Single proto: `proto/yoresee_doc/v1/yoresee_doc.proto`
|
||||
Generated Go code: `pkg/gen/` (gitignored)
|
||||
Regenerate: `bash deploy/script/gen_proto.sh` (from repo root)
|
||||
|
||||
## Health Endpoints
|
||||
|
||||
- `/health` — liveness
|
||||
- `/readyz` — readiness (checks DB + Redis)
|
||||
- `/livez` — liveness alias
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
# Yoresee Doc Backend
|
||||
|
||||
Backend service for the Yoresee Doc collaborative document management platform, built with Go and Connect RPC.
|
||||
|
||||
## Overview
|
||||
|
||||
Yoresee Doc Backend provides:
|
||||
- **Document Management**: Create, edit, version, and organize documents with attachment support
|
||||
- **Knowledge Base**: Hierarchical knowledge base organization
|
||||
- **User & Access Control**: User groups, organization structure, membership management, invitations
|
||||
- **Real-time Collaboration**: Yjs snapshot persistence and conflict resolution
|
||||
- **Full-text Search**: Elasticsearch integration for document search
|
||||
- **Notifications**: Event-driven notification system via RabbitMQ
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level Structure
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ API Layer │
|
||||
│ Connect RPC handlers (HTTP/2 + gRPC-web) │
|
||||
│ - Auth interceptor (JWT) │
|
||||
│ - CORS support │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Service Layer │
|
||||
│ Business logic with package-level singletons │
|
||||
│ - auth_service, document_service, etc. │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Repository Layer │
|
||||
│ Data access with GORM (PostgreSQL) │
|
||||
│ - Constructed via repository.NewRepositories() │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Infrastructure Layer │
|
||||
│ - PostgreSQL (primary data) │
|
||||
│ - Redis (cache, sessions, locks) │
|
||||
│ - MinIO (file storage) │
|
||||
│ - Elasticsearch (search index) │
|
||||
│ - RabbitMQ (message queue) │
|
||||
│ - Consul (service discovery/config) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Project Layout
|
||||
|
||||
```
|
||||
cmd/
|
||||
├── main.go # API server entry point
|
||||
├── migrate/ # Database migration (GORM AutoMigrate)
|
||||
├── db_init/ # Seed data (admin user, default groups, templates)
|
||||
├── es_init/ # Elasticsearch index initialization
|
||||
├── notification-worker/ # Notification consumer
|
||||
├── search-sync-worker/ # Search index sync consumer
|
||||
└── snapshot-worker/ # Yjs snapshot persistence consumer
|
||||
|
||||
internal/
|
||||
├── bootstrap/ # Chain-style initializer (config → infra → services)
|
||||
├── config/ # Viper TOML config loader
|
||||
├── transport/
|
||||
│ ├── connectserver/ # HTTP mux, RPC handlers, auth interceptor
|
||||
│ └── grpcserver/ # Per-service RPC implementations
|
||||
├── service/ # Business logic (domain services)
|
||||
│ ├── <domain>_service/ # Service implementations
|
||||
│ ├── interface/ # Service interfaces
|
||||
│ └── mq_service/ # Message queue abstraction
|
||||
├── repository/ # Data access layer
|
||||
│ ├── <domain>_repo/ # Repository implementations
|
||||
│ └── init.go # Repository construction
|
||||
├── model/ # GORM model definitions (16 models)
|
||||
├── dto/ # Data transfer objects
|
||||
├── mapper/ # Model ↔ DTO mappers
|
||||
├── types/ # Domain types
|
||||
└── status/ # Typed error codes
|
||||
|
||||
pkg/
|
||||
├── storage/ # Infrastructure clients (DB, Redis, MinIO, ES, Consul)
|
||||
├── mq/ # Message queue interface + RabbitMQ impl
|
||||
├── cache/ # Cache utilities
|
||||
├── lock/ # Distributed lock
|
||||
├── key/ # Shared cache keys
|
||||
├── errs/ # Shared errors
|
||||
├── constant/ # Shared constants
|
||||
└── gen/ # GENERATED protobuf stubs (gitignored)
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Go 1.24+**
|
||||
- **PostgreSQL 15+** with `ltree` extension support
|
||||
- **Redis 7+**
|
||||
- **MinIO** (or S3-compatible object storage)
|
||||
- **RabbitMQ 3+**
|
||||
- **Consul 1.16+**
|
||||
- **Elasticsearch 8+** (optional, for search functionality)
|
||||
|
||||
All infrastructure services can be started via the project's Docker Compose setup.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Config File
|
||||
|
||||
Configuration is loaded from `config.toml` (gitignored), generated from `config.toml.tmpl` by `deploy/script/prepare.sh`.
|
||||
|
||||
**Generation:**
|
||||
```bash
|
||||
# From repo root
|
||||
bash deploy/script/prepare.sh
|
||||
```
|
||||
|
||||
**Format:** TOML with environment variable substitution (`${ENV_VAR}`).
|
||||
|
||||
**Environment variables:** Defined in `deploy/.env` (template: `deploy/.env.example`).
|
||||
|
||||
**Access:** Via `config.GlobalConfig.*` singleton.
|
||||
|
||||
### Key Configuration Sections
|
||||
|
||||
```toml
|
||||
[server]
|
||||
grpc_port = 9090 # Native gRPC port
|
||||
grpc_web_port = 8080 # gRPC-web port (browser clients)
|
||||
|
||||
[database]
|
||||
host = "localhost"
|
||||
port = 5432
|
||||
user = "root"
|
||||
password = "your_password"
|
||||
name = "yoresee_doc_db"
|
||||
|
||||
[redis]
|
||||
host = "localhost"
|
||||
port = 6379
|
||||
password = "your_redis_password"
|
||||
db = 0
|
||||
|
||||
[consul]
|
||||
enabled = true
|
||||
address = "localhost:8500"
|
||||
scheme = "http"
|
||||
token = "yoresee_doc_root_token"
|
||||
prefix = "yoresee_doc"
|
||||
|
||||
[minio]
|
||||
endpoint = "localhost:9000"
|
||||
access_key = "minioadmin"
|
||||
secret_key = "minioadmin"
|
||||
bucket = "yoresee_doc"
|
||||
use_ssl = false
|
||||
|
||||
[elasticsearch]
|
||||
enabled = true
|
||||
addresses = ["http://localhost:9200"]
|
||||
username = "elastic"
|
||||
password = "your_password"
|
||||
index_prefix = "yoresee_doc"
|
||||
|
||||
[mq_config.rabbitmq]
|
||||
url = "amqp://guest:guest@localhost:5672/"
|
||||
|
||||
[backend]
|
||||
system_name = "yoresee_doc"
|
||||
internal_rpc_key = "yoresee_doc_internal_key"
|
||||
|
||||
[backend.jwt]
|
||||
secret = "yoresee_doc_jwt_secret_key"
|
||||
expire = 86400 # 24 hours
|
||||
refresh_expire = 604800 # 7 days
|
||||
|
||||
[backend.security]
|
||||
password_hash_cost = 12
|
||||
max_login_attempts = 5
|
||||
login_lock_duration = 30 # minutes
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
### 1. Start Infrastructure
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
# Generate config files
|
||||
bash deploy/script/configure.sh # Interactive setup
|
||||
# OR
|
||||
cp deploy/.env.example deploy/.env
|
||||
bash deploy/script/prepare.sh # Non-interactive
|
||||
|
||||
# Start all services via Docker Compose
|
||||
bash deploy/script/start.sh dev up
|
||||
```
|
||||
|
||||
This starts:
|
||||
- PostgreSQL, Redis, Consul, MinIO, RabbitMQ, Elasticsearch
|
||||
- Backend API server (with automatic migration + seeding)
|
||||
- Background workers (snapshot, notification, search-sync)
|
||||
- Frontend, collaboration services, nginx
|
||||
|
||||
### 2. Run Backend Standalone (Optional)
|
||||
|
||||
If infrastructure is already running externally:
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# Run migration + seeding
|
||||
go run ./cmd/migrate
|
||||
go run ./cmd/db_init
|
||||
go run ./cmd/es_init
|
||||
|
||||
# Start API server
|
||||
go run ./cmd/main.go
|
||||
```
|
||||
|
||||
### 3. Verify
|
||||
|
||||
Health endpoints:
|
||||
- `GET /health` — Liveness check
|
||||
- `GET /readyz` — Readiness (checks DB + Redis)
|
||||
- `GET /livez` — Liveness alias
|
||||
|
||||
## API Structure
|
||||
|
||||
### Connect RPC
|
||||
|
||||
The backend exposes two HTTP listeners using Connect RPC (not raw gRPC):
|
||||
|
||||
| Port | Purpose | Protocol |
|
||||
|------|---------|----------|
|
||||
| 9090 | Native gRPC | HTTP/2 |
|
||||
| 8080 | gRPC-web | HTTP/1.1+ (browser clients) |
|
||||
|
||||
Both listeners use h2c (HTTP/2 cleartext).
|
||||
|
||||
### Services
|
||||
|
||||
The API is organized into the following services:
|
||||
|
||||
- **AuthService**: Login, register, profile management
|
||||
- **DocumentService**: Document CRUD, versioning, attachments, templates
|
||||
- **KnowledgeBaseService**: Knowledge base management
|
||||
- **MembershipService**: User groups, org structure, memberships
|
||||
- **InvitationService**: Invitation creation and management
|
||||
- **NotificationService**: Notification CRUD, read status
|
||||
- **CommentService**: Document comments
|
||||
- **SettingService**: System settings
|
||||
- **SystemService**: Health checks, system info
|
||||
|
||||
### Authentication
|
||||
|
||||
JWT-based authentication with the following unauthenticated endpoints:
|
||||
- `AuthService.Login`
|
||||
- `AuthService.Register`
|
||||
- `SystemService.Health`
|
||||
- `SystemService.SystemInfo`
|
||||
|
||||
All other endpoints require a valid JWT in the `Authorization: Bearer <token>` header.
|
||||
|
||||
## Background Workers
|
||||
|
||||
### Snapshot Worker
|
||||
|
||||
**Purpose:** Persist Yjs document snapshots to the database.
|
||||
|
||||
**Queue:** Consumes from RabbitMQ topic for dirty document notifications.
|
||||
|
||||
**Flow:**
|
||||
1. Collab service marks document as dirty after edits
|
||||
2. Snapshot worker receives notification
|
||||
3. Fetches current Yjs state from collab service
|
||||
4. Persists snapshot to `document_yjs_snapshots` table
|
||||
|
||||
### Notification Worker
|
||||
|
||||
**Purpose:** Process and deliver notifications.
|
||||
|
||||
**Queue:** Consumes from RabbitMQ notification topics.
|
||||
|
||||
### Search Sync Worker
|
||||
|
||||
**Purpose:** Keep Elasticsearch index in sync with database.
|
||||
|
||||
**Queue:** Consumes from RabbitMQ for document/KB changes.
|
||||
|
||||
**Flow:**
|
||||
1. Document/KB created/updated/deleted
|
||||
2. Worker receives event
|
||||
3. Updates Elasticsearch index accordingly
|
||||
|
||||
## Database
|
||||
|
||||
### Schema Management
|
||||
|
||||
**Migration:** GORM `AutoMigrate` based on model struct definitions.
|
||||
|
||||
**Location:** `cmd/migrate/migration.go`
|
||||
|
||||
**Process:**
|
||||
1. Creates PostgreSQL `ltree` extension (for hierarchical org structure)
|
||||
2. Auto-migrates all 16 models
|
||||
3. Creates custom indexes (unique constraints on recent items)
|
||||
4. Applies manual migrations (e.g., dropping deprecated columns)
|
||||
|
||||
**Models:**
|
||||
- User
|
||||
- Document, DocumentVersion, DocumentYjsSnapshot, DocumentComment
|
||||
- KnowledgeBase
|
||||
- Template
|
||||
- Attachment
|
||||
- Invitation, InvitationRecord
|
||||
- Notification
|
||||
- UserGroupMeta, MembershipRelation
|
||||
- OrgNodeMeta
|
||||
- RecentDocument, RecentKnowledgeBase, RecentTemplate
|
||||
|
||||
### Seeding
|
||||
|
||||
**Location:** `cmd/db_init/`
|
||||
|
||||
**Process:**
|
||||
1. Creates admin user (username: `admin`, password from config)
|
||||
2. Creates default permissions
|
||||
3. Creates default user groups
|
||||
4. Creates default templates
|
||||
|
||||
## Development
|
||||
|
||||
### Code Structure Patterns
|
||||
|
||||
**Service Layer:**
|
||||
```go
|
||||
// Package-level singleton
|
||||
var AuthSvc *AuthService
|
||||
|
||||
// Initialization in service.Init()
|
||||
func Init(cfg *config.Config, repos *repository.Repositories) error {
|
||||
auth_service.AuthSvc = auth_service.NewAuthService(repos)
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Repository Layer:**
|
||||
```go
|
||||
// Constructed via Repositories struct
|
||||
type Repositories struct {
|
||||
DB *gorm.DB
|
||||
Redis *redis.Client
|
||||
Document *document_repo.DocumentRepository
|
||||
// ...
|
||||
}
|
||||
|
||||
func NewRepositories(db *gorm.DB, redis *redis.Client) *Repositories {
|
||||
return &Repositories{
|
||||
Document: document_repo.NewDocumentRepository(db, redis),
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Error Handling:**
|
||||
```go
|
||||
// Typed errors with status codes
|
||||
return status.GenErrWithCustomMsg(
|
||||
status.StatusServiceInternalError,
|
||||
"operation failed",
|
||||
)
|
||||
```
|
||||
|
||||
### Protobuf Codegen
|
||||
|
||||
**Source:** `proto/yoresee_doc/v1/yoresee_doc.proto`
|
||||
|
||||
**Generated Go code:** `pkg/gen/` (gitignored)
|
||||
|
||||
**Regenerate:**
|
||||
```bash
|
||||
# From repo root
|
||||
bash deploy/script/gen_proto.sh
|
||||
```
|
||||
|
||||
**Targets:**
|
||||
- `backend/pkg/gen` — Go stubs
|
||||
- `collab-go/pkg/gen` — Go stubs
|
||||
- `frontend/src/gen` — TypeScript ES modules
|
||||
- `collab/src/gen` — Node gRPC stubs
|
||||
|
||||
### Verification
|
||||
|
||||
No linter or test suite configured. Verify changes with:
|
||||
|
||||
```bash
|
||||
go build ./...
|
||||
go vet ./...
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Docker Images
|
||||
|
||||
| Dockerfile | Purpose |
|
||||
|------------|---------|
|
||||
| `Dockerfile` | Dev image (with Delve debugger) |
|
||||
| `Dockerfile.prod` | Production API server |
|
||||
| `Dockerfile.snapshot.prod` | Production snapshot worker |
|
||||
| `Dockerfile.notification.prod` | Production notification worker |
|
||||
| `Dockerfile.search-sync.prod` | Production search-sync worker |
|
||||
|
||||
### Startup Sequence
|
||||
|
||||
Container CMD for production:
|
||||
```bash
|
||||
./migrate.out && ./db_init.out && ./es_init.out && ./main.out
|
||||
```
|
||||
|
||||
Workers are separate containers with their own entrypoints.
|
||||
|
||||
### Health Checks
|
||||
|
||||
Docker Compose uses `wget` to check `/health` endpoint:
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://backend:9090/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
start_period: 30s
|
||||
```
|
||||
|
||||
## Infrastructure Ports
|
||||
|
||||
| Service | Default Port |
|
||||
|---------|--------------|
|
||||
| Backend gRPC | 9090 |
|
||||
| Backend gRPC-web | 8080 |
|
||||
| Backend Debug (Delve) | 2345 |
|
||||
| PostgreSQL | 5432 |
|
||||
| Redis | 6379 |
|
||||
| Consul | 8500 |
|
||||
| MinIO API | 9000 |
|
||||
| MinIO Console | 9001 |
|
||||
| RabbitMQ AMQP | 5672 |
|
||||
| RabbitMQ Management | 15672 |
|
||||
| Elasticsearch | 9200 |
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
- **Connect RPC** (`connectrpc.com/connect`) — Modern RPC framework
|
||||
- **GORM** (`gorm.io/gorm`) — ORM for PostgreSQL
|
||||
- **go-redis** (`github.com/redis/go-redis/v9`) — Redis client
|
||||
- **MinIO Go** (`github.com/minio/minio-go/v7`) — S3-compatible storage
|
||||
- **Viper** (`github.com/spf13/viper`) — Configuration management
|
||||
- **Logrus** (`github.com/sirupsen/logrus`) — Structured logging
|
||||
- **JWT** (`github.com/golang-jwt/jwt/v5`) — Authentication
|
||||
- **amqp091-go** (`github.com/rabbitmq/amqp091-go`) — RabbitMQ client
|
||||
- **improbable-eng/grpc-web** — gRPC-web proxy for browser clients
|
||||
- **Snowflake** (`github.com/bwmarrin/snowflake`) — Distributed ID generation
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All environment variables are defined in `deploy/.env` (see `deploy/.env.example` for template).
|
||||
|
||||
Key variables:
|
||||
- `POSTGRES_*` — Database connection
|
||||
- `REDIS_*` — Redis connection
|
||||
- `CONSUL_*` — Consul configuration
|
||||
- `MINIO_*` — MinIO/S3 configuration
|
||||
- `ELASTICSEARCH_*` — Elasticsearch configuration
|
||||
- `RABBITMQ_*` — RabbitMQ configuration
|
||||
- `BACKEND_*` — Backend service configuration
|
||||
- `JWT_SECRET` — JWT signing secret
|
||||
|
||||
## License
|
||||
|
||||
[Your license here]
|
||||
|
||||
## Contributing
|
||||
|
||||
[Your contributing guidelines here]
|
||||
@@ -0,0 +1,483 @@
|
||||
# Yoresee Doc 后端
|
||||
|
||||
Yoresee Doc 协作文档管理平台的后端服务，基于 Go 和 Connect RPC 构建。
|
||||
|
||||
## 概述
|
||||
|
||||
Yoresee Doc 后端提供：
|
||||
- **文档管理**：创建、编辑、版本控制、组织文档，支持附件
|
||||
- **知识库**：层级化知识库组织
|
||||
- **用户与权限控制**：用户组、组织结构、成员管理、邀请
|
||||
- **实时协作**：Yjs 快照持久化和冲突解决
|
||||
- **全文搜索**：Elasticsearch 集成实现文档搜索
|
||||
- **通知系统**：基于 RabbitMQ 的事件驱动通知
|
||||
|
||||
## 架构
|
||||
|
||||
### 高层结构
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ API 层 │
|
||||
│ Connect RPC 处理器 (HTTP/2 + gRPC-web) │
|
||||
│ - 认证拦截器 (JWT) │
|
||||
│ - CORS 支持 │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 服务层 │
|
||||
│ 业务逻辑，使用包级单例 │
|
||||
│ - auth_service, document_service 等 │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 仓储层 │
|
||||
│ 使用 GORM (PostgreSQL) 的数据访问 │
|
||||
│ - 通过 repository.NewRepositories() 构造 │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 基础设施层 │
|
||||
│ - PostgreSQL (主数据存储) │
|
||||
│ - Redis (缓存、会话、锁) │
|
||||
│ - MinIO (文件存储) │
|
||||
│ - Elasticsearch (搜索索引) │
|
||||
│ - RabbitMQ (消息队列) │
|
||||
│ - Consul (服务发现/配置) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 项目结构
|
||||
|
||||
```
|
||||
cmd/
|
||||
├── main.go # API 服务器入口
|
||||
├── migrate/ # 数据库迁移 (GORM AutoMigrate)
|
||||
├── db_init/ # 种子数据 (管理员用户、默认分组、模板)
|
||||
├── es_init/ # Elasticsearch 索引初始化
|
||||
├── notification-worker/ # 通知消费者
|
||||
├── search-sync-worker/ # 搜索索引同步消费者
|
||||
└── snapshot-worker/ # Yjs 快照持久化消费者
|
||||
|
||||
internal/
|
||||
├── bootstrap/ # 链式初始化器 (配置 → 基础设施 → 服务)
|
||||
├── config/ # Viper TOML 配置加载器
|
||||
├── transport/
|
||||
│ ├── connectserver/ # HTTP 多路复用、RPC 处理器、认证拦截器
|
||||
│ └── grpcserver/ # 各服务的 RPC 实现
|
||||
├── service/ # 业务逻辑 (领域服务)
|
||||
│ ├── <domain>_service/ # 服务实现
|
||||
│ ├── interface/ # 服务接口
|
||||
│ └── mq_service/ # 消息队列抽象
|
||||
├── repository/ # 数据访问层
|
||||
│ ├── <domain>_repo/ # 仓储实现
|
||||
│ └── init.go # 仓储构造
|
||||
├── model/ # GORM 模型定义 (16 个模型)
|
||||
├── dto/ # 数据传输对象
|
||||
├── mapper/ # 模型 ↔ DTO 映射器
|
||||
├── types/ # 领域类型
|
||||
└── status/ # 类型化错误码
|
||||
|
||||
pkg/
|
||||
├── storage/ # 基础设施客户端 (DB、Redis、MinIO、ES、Consul)
|
||||
├── mq/ # 消息队列接口 + RabbitMQ 实现
|
||||
├── cache/ # 缓存工具
|
||||
├── lock/ # 分布式锁
|
||||
├── key/ # 共享缓存键
|
||||
├── errs/ # 共享错误
|
||||
├── constant/ # 共享常量
|
||||
└── gen/ # 生成的 protobuf 代码 (gitignored)
|
||||
```
|
||||
|
||||
## 前置要求
|
||||
|
||||
- **Go 1.24+**
|
||||
- **PostgreSQL 15+** 支持 `ltree` 扩展
|
||||
- **Redis 7+**
|
||||
- **MinIO** (或 S3 兼容对象存储)
|
||||
- **RabbitMQ 3+**
|
||||
- **Consul 1.16+**
|
||||
- **Elasticsearch 8+** (可选，用于搜索功能)
|
||||
|
||||
所有基础设施服务可通过项目的 Docker Compose 配置启动。
|
||||
|
||||
## 配置
|
||||
|
||||
### 配置文件
|
||||
|
||||
配置从 `config.toml` (gitignored) 加载，由 `deploy/script/prepare.sh` 从 `config.toml.tmpl` 生成。
|
||||
|
||||
**生成：**
|
||||
```bash
|
||||
# 在仓库根目录
|
||||
bash deploy/script/prepare.sh
|
||||
```
|
||||
|
||||
**格式：** TOML，支持环境变量替换 (`${ENV_VAR}`)。
|
||||
|
||||
**环境变量：** 在 `deploy/.env` 中定义 (模板：`deploy/.env.example`)。
|
||||
|
||||
**访问方式：** 通过 `config.GlobalConfig.*` 单例。
|
||||
|
||||
### 关键配置部分
|
||||
|
||||
```toml
|
||||
[server]
|
||||
grpc_port = 9090 # 原生 gRPC 端口
|
||||
grpc_web_port = 8080 # gRPC-web 端口 (浏览器客户端)
|
||||
|
||||
[database]
|
||||
host = "localhost"
|
||||
port = 5432
|
||||
user = "root"
|
||||
password = "your_password"
|
||||
name = "yoresee_doc_db"
|
||||
|
||||
[redis]
|
||||
host = "localhost"
|
||||
port = 6379
|
||||
password = "your_redis_password"
|
||||
db = 0
|
||||
|
||||
[consul]
|
||||
enabled = true
|
||||
address = "localhost:8500"
|
||||
scheme = "http"
|
||||
token = "yoresee_doc_root_token"
|
||||
prefix = "yoresee_doc"
|
||||
|
||||
[minio]
|
||||
endpoint = "localhost:9000"
|
||||
access_key = "minioadmin"
|
||||
secret_key = "minioadmin"
|
||||
bucket = "yoresee_doc"
|
||||
use_ssl = false
|
||||
|
||||
[elasticsearch]
|
||||
enabled = true
|
||||
addresses = ["http://localhost:9200"]
|
||||
username = "elastic"
|
||||
password = "your_password"
|
||||
index_prefix = "yoresee_doc"
|
||||
|
||||
[mq_config.rabbitmq]
|
||||
url = "amqp://guest:guest@localhost:5672/"
|
||||
|
||||
[backend]
|
||||
system_name = "yoresee_doc"
|
||||
internal_rpc_key = "yoresee_doc_internal_key"
|
||||
|
||||
[backend.jwt]
|
||||
secret = "yoresee_doc_jwt_secret_key"
|
||||
expire = 86400 # 24 小时
|
||||
refresh_expire = 604800 # 7 天
|
||||
|
||||
[backend.security]
|
||||
password_hash_cost = 12
|
||||
max_login_attempts = 5
|
||||
login_lock_duration = 30 # 分钟
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 启动基础设施
|
||||
|
||||
在仓库根目录：
|
||||
|
||||
```bash
|
||||
# 生成配置文件
|
||||
bash deploy/script/configure.sh # 交互式设置
|
||||
# 或
|
||||
cp deploy/.env.example deploy/.env
|
||||
bash deploy/script/prepare.sh # 非交互式
|
||||
|
||||
# 通过 Docker Compose 启动所有服务
|
||||
bash deploy/script/start.sh dev up
|
||||
```
|
||||
|
||||
这将启动：
|
||||
- PostgreSQL、Redis、Consul、MinIO、RabbitMQ、Elasticsearch
|
||||
- 后端 API 服务器 (自动迁移 + 种子数据)
|
||||
- 后台工作器 (快照、通知、搜索同步)
|
||||
- 前端、协作服务、nginx
|
||||
|
||||
### 2. 独立运行后端 (可选)
|
||||
|
||||
如果基础设施已在外部运行：
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# 运行迁移 + 种子数据
|
||||
go run ./cmd/migrate
|
||||
go run ./cmd/db_init
|
||||
go run ./cmd/es_init
|
||||
|
||||
# 启动 API 服务器
|
||||
go run ./cmd/main.go
|
||||
```
|
||||
|
||||
### 3. 验证
|
||||
|
||||
健康检查端点：
|
||||
- `GET /health` — 存活检查
|
||||
- `GET /readyz` — 就绪检查 (检查 DB + Redis)
|
||||
- `GET /livez` — 存活检查别名
|
||||
|
||||
## API 结构
|
||||
|
||||
### Connect RPC
|
||||
|
||||
后端使用 Connect RPC (非原生 gRPC) 暴露两个 HTTP 监听器：
|
||||
|
||||
| 端口 | 用途 | 协议 |
|
||||
|------|------|------|
|
||||
| 9090 | 原生 gRPC | HTTP/2 |
|
||||
| 8080 | gRPC-web | HTTP/1.1+ (浏览器客户端) |
|
||||
|
||||
两个监听器都使用 h2c (HTTP/2 明文)。
|
||||
|
||||
### 服务
|
||||
|
||||
API 组织为以下服务：
|
||||
|
||||
- **AuthService**：登录、注册、个人资料管理
|
||||
- **DocumentService**：文档 CRUD、版本控制、附件、模板
|
||||
- **KnowledgeBaseService**：知识库管理
|
||||
- **MembershipService**：用户组、组织结构、成员关系
|
||||
- **InvitationService**：邀请创建和管理
|
||||
- **NotificationService**：通知 CRUD、已读状态
|
||||
- **CommentService**：文档评论
|
||||
- **SettingService**：系统设置
|
||||
- **SystemService**：健康检查、系统信息
|
||||
|
||||
### 认证
|
||||
|
||||
基于 JWT 的认证，以下端点无需认证：
|
||||
- `AuthService.Login`
|
||||
- `AuthService.Register`
|
||||
- `SystemService.Health`
|
||||
- `SystemService.SystemInfo`
|
||||
|
||||
所有其他端点需要在 `Authorization: Bearer <token>` 头中提供有效的 JWT。
|
||||
|
||||
## 后台工作器
|
||||
|
||||
### 快照工作器
|
||||
|
||||
**用途：** 将 Yjs 文档快照持久化到数据库。
|
||||
|
||||
**队列：** 消费 RabbitMQ 主题中的脏文档通知。
|
||||
|
||||
**流程：**
|
||||
1. 协作服务在编辑后将文档标记为脏
|
||||
2. 快照工作器接收通知
|
||||
3. 从协作服务获取当前 Yjs 状态
|
||||
4. 将快照持久化到 `document_yjs_snapshots` 表
|
||||
|
||||
### 通知工作器
|
||||
|
||||
**用途：** 处理和投递通知。
|
||||
|
||||
**队列：** 消费 RabbitMQ 通知主题。
|
||||
|
||||
### 搜索同步工作器
|
||||
|
||||
**用途：** 保持 Elasticsearch 索引与数据库同步。
|
||||
|
||||
**队列：** 消费 RabbitMQ 中的文档/知识库变更。
|
||||
|
||||
**流程：**
|
||||
1. 文档/知识库被创建/更新/删除
|
||||
2. 工作器接收事件
|
||||
3. 相应更新 Elasticsearch 索引
|
||||
|
||||
## 数据库
|
||||
|
||||
### 模式管理
|
||||
|
||||
**迁移：** 基于模型结构体定义的 GORM `AutoMigrate`。
|
||||
|
||||
**位置：** `cmd/migrate/migration.go`
|
||||
|
||||
**流程：**
|
||||
1. 创建 PostgreSQL `ltree` 扩展 (用于层级组织结构)
|
||||
2. 自动迁移所有 16 个模型
|
||||
3. 创建自定义索引 (最近访问项的唯一约束)
|
||||
4. 应用手动迁移 (例如，删除已弃用的列)
|
||||
|
||||
**模型：**
|
||||
- User (用户)
|
||||
- Document, DocumentVersion, DocumentYjsSnapshot, DocumentComment (文档相关)
|
||||
- KnowledgeBase (知识库)
|
||||
- Template (模板)
|
||||
- Attachment (附件)
|
||||
- Invitation, InvitationRecord (邀请相关)
|
||||
- Notification (通知)
|
||||
- UserGroupMeta, MembershipRelation (用户组和成员关系)
|
||||
- OrgNodeMeta (组织节点)
|
||||
- RecentDocument, RecentKnowledgeBase, RecentTemplate (最近访问)
|
||||
|
||||
### 种子数据
|
||||
|
||||
**位置：** `cmd/db_init/`
|
||||
|
||||
**流程：**
|
||||
1. 创建管理员用户 (用户名：`admin`，密码来自配置)
|
||||
2. 创建默认权限
|
||||
3. 创建默认用户组
|
||||
4. 创建默认模板
|
||||
|
||||
## 开发
|
||||
|
||||
### 代码结构模式
|
||||
|
||||
**服务层：**
|
||||
```go
|
||||
// 包级单例
|
||||
var AuthSvc *AuthService
|
||||
|
||||
// 在 service.Init() 中初始化
|
||||
func Init(cfg *config.Config, repos *repository.Repositories) error {
|
||||
auth_service.AuthSvc = auth_service.NewAuthService(repos)
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**仓储层：**
|
||||
```go
|
||||
// 通过 Repositories 结构体构造
|
||||
type Repositories struct {
|
||||
DB *gorm.DB
|
||||
Redis *redis.Client
|
||||
Document *document_repo.DocumentRepository
|
||||
// ...
|
||||
}
|
||||
|
||||
func NewRepositories(db *gorm.DB, redis *redis.Client) *Repositories {
|
||||
return &Repositories{
|
||||
Document: document_repo.NewDocumentRepository(db, redis),
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**错误处理：**
|
||||
```go
|
||||
// 带状态码的类型化错误
|
||||
return status.GenErrWithCustomMsg(
|
||||
status.StatusServiceInternalError,
|
||||
"operation failed",
|
||||
)
|
||||
```
|
||||
|
||||
### Protobuf 代码生成
|
||||
|
||||
**源文件：** `proto/yoresee_doc/v1/yoresee_doc.proto`
|
||||
|
||||
**生成的 Go 代码：** `pkg/gen/` (gitignored)
|
||||
|
||||
**重新生成：**
|
||||
```bash
|
||||
# 在仓库根目录
|
||||
bash deploy/script/gen_proto.sh
|
||||
```
|
||||
|
||||
**目标：**
|
||||
- `backend/pkg/gen` — Go 代码
|
||||
- `collab-go/pkg/gen` — Go 代码
|
||||
- `frontend/src/gen` — TypeScript ES 模块
|
||||
- `collab/src/gen` — Node gRPC 代码
|
||||
|
||||
### 代码验证
|
||||
|
||||
未配置 linter 或测试套件。使用以下命令验证更改：
|
||||
|
||||
```bash
|
||||
go build ./...
|
||||
go vet ./...
|
||||
```
|
||||
|
||||
## 部署
|
||||
|
||||
### Docker 镜像
|
||||
|
||||
| Dockerfile | 用途 |
|
||||
|------------|------|
|
||||
| `Dockerfile` | 开发镜像 (含 Delve 调试器) |
|
||||
| `Dockerfile.prod` | 生产 API 服务器 |
|
||||
| `Dockerfile.snapshot.prod` | 生产快照工作器 |
|
||||
| `Dockerfile.notification.prod` | 生产通知工作器 |
|
||||
| `Dockerfile.search-sync.prod` | 生产搜索同步工作器 |
|
||||
|
||||
### 启动顺序
|
||||
|
||||
生产环境容器 CMD：
|
||||
```bash
|
||||
./migrate.out && ./db_init.out && ./es_init.out && ./main.out
|
||||
```
|
||||
|
||||
工作器是独立的容器，有自己的入口点。
|
||||
|
||||
### 健康检查
|
||||
|
||||
Docker Compose 使用 `wget` 检查 `/health` 端点：
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://backend:9090/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
start_period: 30s
|
||||
```
|
||||
|
||||
## 基础设施端口
|
||||
|
||||
| 服务 | 默认端口 |
|
||||
|------|----------|
|
||||
| 后端 gRPC | 9090 |
|
||||
| 后端 gRPC-web | 8080 |
|
||||
| 后端调试 (Delve) | 2345 |
|
||||
| PostgreSQL | 5432 |
|
||||
| Redis | 6379 |
|
||||
| Consul | 8500 |
|
||||
| MinIO API | 9000 |
|
||||
| MinIO 控制台 | 9001 |
|
||||
| RabbitMQ AMQP | 5672 |
|
||||
| RabbitMQ 管理界面 | 15672 |
|
||||
| Elasticsearch | 9200 |
|
||||
|
||||
## 关键依赖
|
||||
|
||||
- **Connect RPC** (`connectrpc.com/connect`) — 现代 RPC 框架
|
||||
- **GORM** (`gorm.io/gorm`) — PostgreSQL ORM
|
||||
- **go-redis** (`github.com/redis/go-redis/v9`) — Redis 客户端
|
||||
- **MinIO Go** (`github.com/minio/minio-go/v7`) — S3 兼容存储
|
||||
- **Viper** (`github.com/spf13/viper`) — 配置管理
|
||||
- **Logrus** (`github.com/sirupsen/logrus`) — 结构化日志
|
||||
- **JWT** (`github.com/golang-jwt/jwt/v5`) — 认证
|
||||
- **amqp091-go** (`github.com/rabbitmq/amqp091-go`) — RabbitMQ 客户端
|
||||
- **improbable-eng/grpc-web** — 浏览器客户端的 gRPC-web 代理
|
||||
- **Snowflake** (`github.com/bwmarrin/snowflake`) — 分布式 ID 生成
|
||||
|
||||
## 环境变量
|
||||
|
||||
所有环境变量在 `deploy/.env` 中定义 (参见 `deploy/.env.example` 获取模板)。
|
||||
|
||||
关键变量：
|
||||
- `POSTGRES_*` — 数据库连接
|
||||
- `REDIS_*` — Redis 连接
|
||||
- `CONSUL_*` — Consul 配置
|
||||
- `MINIO_*` — MinIO/S3 配置
|
||||
- `ELASTICSEARCH_*` — Elasticsearch 配置
|
||||
- `RABBITMQ_*` — RabbitMQ 配置
|
||||
- `BACKEND_*` — 后端服务配置
|
||||
- `JWT_SECRET` — JWT 签名密钥
|
||||
|
||||
## 许可证
|
||||
|
||||
[在此填写您的许可证]
|
||||
|
||||
## 贡献
|
||||
|
||||
[在此填写您的贡献指南]
|
||||
Reference in New Issue
Block a user