What you will have at the end
- A public URL serving your RAG app
- A pipeline: test → build → staging → eval gate → production
- Health checks, versioned prompts, a dashboard and a rehearsed rollback
Step by step
- ContaineriseDockerfile for the API and worker, compose.yaml for local runs with pgvector and Redis.
- Config and secretsTyped settings, per-environment values, secrets in the platform’s secret manager.
- Health checks/health and /ready endpoints wired into the platform.
- Version promptsMove prompts into prompts/<name>/<version>.yaml and log version with each response. Pin model IDs.
- CILint, tests and the CI eval subset on pull requests.
- CD with an eval gateOn main: build image tagged by SHA, deploy to staging, run evals against staging, deploy to production only if the gate passes.
- ObserveDashboard for errors, p95 latency, TTFT and cost per hour, plus one cost-spike alert.
- RollbackA manual workflow to redeploy any previous SHA. Rehearse it and record the time.
DONE WHEN
- Merging to main deploys automatically when evals pass
- A degraded prompt is blocked by the gate
- Rollback takes under 5 minutes and has been rehearsed
Stretch goals
- Canary releases at 10%
- Feature flag for the reranker
- Terraform for all infrastructure