The idea
Platforms need to know if an instance should receive traffic or be restarted. A liveness check (/health) answers "is the process alive?"; failing it causes a restart. A readiness check (/ready) answers "can I serve requests right now?" by checking dependencies like the database and that models or indexes are loaded; failing it removes the instance from the load balancer without restarting it.
Keep liveness cheap and independent of external services, or a provider outage will restart all your instances in a loop. Readiness can check the database and cache; checking the model provider is usually better done by monitoring than by readiness.
Add a startup probe for slow starts like loading model weights.
Correct health checks make deployments and failures self-healing; wrong ones cause restart storms.
Words to know
- LivenessIs the process alive?
- ReadinessIs it ready to serve traffic?
Try it without code
Design two endpoints
- Write what /health and /ready should check in your app.
- For each dependency, decide which endpoint (if any) checks it.
Use it in code
Liveness and readiness endpoints in FastAPI.
from fastapi import FastAPI, Response
app = FastAPI()
STATE = {"index_loaded": False}
@app.get("/health")
def health():
return {"status": "ok"} # no external calls
@app.get("/ready")
def ready(response: Response):
checks = {"index_loaded": STATE["index_loaded"]}
try:
db.execute("SELECT 1"); checks["db"] = True
except Exception:
checks["db"] = False
if not all(checks.values()):
response.status_code = 503
return checksAdd both endpoints to your API, wire them into your platform’s probes, and confirm a stopped database makes the instance not-ready.
Common mistakes
- Liveness that calls the LLM provider.
- Readiness that always returns 200.