Area 07 · Deployment · Build this

Ship your RAG app with CI/CD, an eval gate and one-step rollback

Take the RAG app from Phase 2 (with the evals from Phase 4) and run it as a real service: containerised, deployed by GitHub Actions on every merge, blocked by an eval gate, observable, and easy to roll back.

Phase 5 · Weeks 18–20

What you will have at the end

  • A public URL serving your RAG app
  • A pipeline: test → build → staging → eval gate → production
  • Health checks, versioned prompts, a dashboard and a rehearsed rollback

Step by step

  1. ContaineriseDockerfile for the API and worker, compose.yaml for local runs with pgvector and Redis.
  2. Config and secretsTyped settings, per-environment values, secrets in the platform’s secret manager.
  3. Health checks/health and /ready endpoints wired into the platform.
  4. Version promptsMove prompts into prompts/<name>/<version>.yaml and log version with each response. Pin model IDs.
  5. CILint, tests and the CI eval subset on pull requests.
  6. CD with an eval gateOn main: build image tagged by SHA, deploy to staging, run evals against staging, deploy to production only if the gate passes.
  7. ObserveDashboard for errors, p95 latency, TTFT and cost per hour, plus one cost-spike alert.
  8. RollbackA manual workflow to redeploy any previous SHA. Rehearse it and record the time.
DONE WHEN
  • Merging to main deploys automatically when evals pass
  • A degraded prompt is blocked by the gate
  • Rollback takes under 5 minutes and has been rehearsed

Stretch goals

  • Canary releases at 10%
  • Feature flag for the reranker
  • Terraform for all infrastructure