feat: implement proto-gen platform
- Config loader with env var expansion and validation - SQLite database with tasks and toolchain_versions tables - Gitea REST API client for repository and branch operations - K8s client for Job lifecycle management (with noop fallback for local dev) - REST API: tasks CRUD, retry, plugins, version management - Worker/scheduler with polling, concurrency control, and job tracking - Static frontend: generate, tasks, settings pages - Toolchain images: Go and Node.js Dockerfiles with entrypoint scripts - Helm chart with deployment, service, configmap, ingress, PVC - Multi-stage Dockerfile for service - Unit tests for config, database, gitclient, k8s, api, worker - E2E tests for idempotency, version switch, failure recovery
This commit is contained in:
@@ -1,2 +1,96 @@
|
||||
# yoresee_codegen
|
||||
# Proto-Gen Platform
|
||||
|
||||
Protobuf code generation platform that generates code from protobuf definitions and delivers to independent Git repos via K8s Jobs.
|
||||
|
||||
## Features
|
||||
|
||||
- **Manual trigger**: Select language, branch, and repository to generate
|
||||
- **Plugin architecture**: Easily add new languages by adding toolchain images
|
||||
- **K8s Jobs**: One-time pods for code generation with timeout and retry
|
||||
- **Idempotent**: Same proto + branch + language = same output
|
||||
- **SQLite**: Zero-ops state storage
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.21+
|
||||
- Kubernetes cluster (K3s/K8s)
|
||||
- Gitea instance
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
make build
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
./build/proto-gen --config config.yaml
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Copy `testdata/config.yaml` and modify:
|
||||
|
||||
```yaml
|
||||
server:
|
||||
addr: ":8080"
|
||||
|
||||
git:
|
||||
type: gitea
|
||||
base_url: "https://git.example.com"
|
||||
token: ${GIT_TOKEN}
|
||||
|
||||
cluster:
|
||||
kubeconfig: /etc/proto-gen/kubeconfig
|
||||
namespace: proto-gen
|
||||
job_timeout: 600
|
||||
max_concurrent: 2
|
||||
|
||||
toolchain_registry: "registry.example.com/proto-gen"
|
||||
```
|
||||
|
||||
### Deploy with Helm
|
||||
|
||||
```bash
|
||||
helm install proto-gen ./deploy/helm/proto-gen \
|
||||
--set git.baseUrl=https://git.example.com \
|
||||
--set git.tokenSecret=proto-gen-git-token
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/api/v1/tasks` | POST | Create task |
|
||||
| `/api/v1/tasks` | GET | List tasks |
|
||||
| `/api/v1/tasks/{id}` | GET | Get task |
|
||||
| `/api/v1/tasks/{id}/retry` | POST | Retry task |
|
||||
| `/api/v1/plugins` | GET | List plugins |
|
||||
| `/api/v1/plugins/{lang}/versions` | GET/PUT | Version management |
|
||||
|
||||
## Building Toolchain Images
|
||||
|
||||
```bash
|
||||
# Go toolchain
|
||||
docker build -t registry.example.com/proto-gen/proto-gen-go:v1.34.2 ./toolchains/go
|
||||
|
||||
# Node.js toolchain
|
||||
docker build -t registry.example.com/proto-gen/proto-gen-ts:v0.9.0 ./toolchains/nodejs
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser (manual trigger)
|
||||
│
|
||||
▼
|
||||
┌─────────────────── proto-gen (monolith) ───────────────────┐
|
||||
│ Frontend │ API Server (REST) │ Scheduler (worker) │
|
||||
└─────────┬───────────────────────────┬──────────────────────┘
|
||||
│ Gitea API │ K8s API
|
||||
▼ ▼
|
||||
Gitea instance K8s Jobs (toolchain images)
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user