# Collab — Yjs Collaboration Server

Node.js (CommonJS) real-time collaboration server that maintains Yjs documents in memory, persists updates to Redis, and notifies downstream consumers of dirty documents via Redis Pub/Sub or RabbitMQ.

## Quick Start

```bash
npm install
npm start
```

Server listens on `PORT` (default `1234`).

## Architecture

```
server.js
├── src/http/server.js      — plain Node HTTP server (no Express)
│   └── src/http/router.js  — routes: /health, /livez, /readyz, /api/active-rooms, /internal/yjs/*
├── src/ws-gateway.js       — WebSocket upgrade handler, extracts room name from URL path
├── src/ws-utils.js         — Yjs sync/awareness protocol, WSSharedDoc, connection lifecycle
├── src/doc-loader.js       — loads doc: Redis list → gRPC snapshot → gRPC content (cascade)
├── src/persistence.js      — binds Y.Doc 'update' event → Redis list append + dirty tracking
├── src/redis/              — Redis client, room tracking, document buffer lists
├── src/grpc/               — gRPC client to backend (DocumentService, SystemService)
├── src/mq.js               — dirty-doc publisher (Redis Pub/Sub and/or RabbitMQ)
├── src/config.js            — all config from process.env (no .env loader)
├── src/key.js               — Redis key patterns
├── src/bootstrap.js         — listen + init Redis/gRPC
└── src/lifecycle.js         — graceful shutdown (SIGTERM/SIGINT)
```

## Connection Flow

1. Client connects via WebSocket to `/{roomName}` (e.g. `/doc-{docId}`)
2. `ws-gateway.js` extracts room name from URL path
3. `ws-utils.js` loads or creates `WSSharedDoc` for the room
4. `doc-loader.js` attempts to load existing state:
   - Redis list buffers → gRPC `GetDocumentYjsSnapshot` → gRPC `GetDocumentContent` → empty doc
5. `persistence.js` binds to Y.Doc `update` event
6. Updates are appended to Redis list and tracked for dirty-doc notifications

## Persistence Model

- Each document update is appended to Redis list: `collab:yjs:doc:updates:{docId}`
- After `DOC_UPDATE_COMPACT_THRESHOLD` (default 1000) updates, list is compacted into single state update
- Document is marked dirty in Redis set: `collab:yjs:dirty:doc`
- After `DIRTY_DOC_NOTIFY_THRESHOLD` (default 200) updates, dirty-doc event is published via MQ

## Dirty-Document Notification

Dirty-doc events notify downstream consumers (snapshot-worker) that a document has been modified.

MQ backend controlled by `DIRTY_DOC_MQ` environment variable:
- `redis` — Redis Pub/Sub (default, no consumer groups)
- `rabbitmq` — RabbitMQ with consumer groups
- `both` — publish to both

Payload format:
```json
{
  "doc_id": "document-id",
  "ts": 1234567890
}
```

Topic: `DIRTY_DOC_TOPIC` (default `collab.dirty_docs`)

## HTTP Routes

Health probes:
- `GET /health` — overall health status
- `GET /livez` — liveness probe
- `GET /readyz` — readiness probe (fails during drain)

API:
- `GET /api/active-rooms` — list active WebSocket rooms

Internal (used by snapshot-worker):
- `GET /internal/yjs/doc-snapshot/{docId}?type={type}` — get document snapshot
- `GET /internal/yjs/doc/{docId}` — get document as binary Yjs update

## WebSocket Protocol

Room name is extracted from WebSocket URL path. For `/doc-{docId}`, the doc ID is `{docId}`.

Protocol messages:
- Sync messages (Yjs sync protocol)
- Awareness messages (cursor positions, user presence)

When last connection to a room closes, the document is removed from memory.

## Proto Codegen

`src/gen/` contains gRPC stubs generated from parent repo's `proto/yoresee_doc/v1/yoresee_doc.proto`.

This directory is **gitignored**. In production, `Dockerfile.prod` generates stubs during build. For development, run from parent repo:

```bash
bash deploy/script/gen_proto.sh
```

## Environment Variables

| Variable | Default | Description |
|---|---|---|
| `PORT` | `1234` | HTTP listen port |
| `REDIS_HOST` | `redis` | Redis hostname |
| `REDIS_PORT` | `6379` | Redis port |
| `REDIS_PASSWORD` | `''` | Redis password |
| `REDIS_DB` | `0` | Redis DB index |
| `BACKEND_ADDR` | `backend:9090` | Backend gRPC address |
| `DIRTY_DOC_MQ` | `redis` | MQ type: `redis`, `rabbitmq`, or `both` |
| `DIRTY_DOC_TOPIC` | `collab.dirty_docs` | Dirty-doc pub/sub topic |
| `RABBITMQ_URL` | `''` | RabbitMQ connection URL (empty = disabled) |
| `INTERNAL_RPC_KEY` | `''` | gRPC metadata auth key (`x-internal-key`) |
| `DIRTY_DOC_NOTIFY_THRESHOLD` | `200` | Updates before publishing dirty-doc event |
| `DOC_UPDATES_TTL` | `259200` | TTL for doc update counters (seconds) |
| `DOC_UPDATE_COMPACT_THRESHOLD` | `1000` | Updates before compacting Redis list |
| `GC` | `true` | Yjs garbage collection (`false`/`0` to disable) |
| `BACKEND_SNAPSHOT_TIMEOUT_MS` | `3000` | Timeout for backend snapshot gRPC call |

## Redis Keys

Defined in `src/key.js`:
- `collab:yjs:doc:updates:{docId}` — document update list
- `collab:room:doc-{docId}` — room metadata
- `collab:yjs:dirty:doc` — dirty document set

## Graceful Shutdown

On `SIGTERM` or `SIGINT`:
1. Server stops accepting new connections
2. Readiness probe returns `not_ready`
3. Existing WebSocket connections are closed
4. MQ connections are closed
5. Redis connections are closed
6. Process exits (10s timeout, then force exit)