Area 07 · Deployment · Concept 20 of 20

Rollbacks

Go back to the last good version in one step.

11–20 · Production levelPhase 5 · Weeks 18–20≈ 30 min

The idea

Every release can go wrong. A rollback returns production to the last known-good version quickly, before you understand the cause. It should be one command or one click, rehearsed, and fast.

Make it possible: immutable, versioned artefacts (images tagged by commit SHA), keeping the previous release deployable, and database migrations that are backwards-compatible (expand first, contract later) so old code still works with the new schema. For AI, roll back prompt versions, model pins and flags as well as code.

After rolling back, fix forward calmly with a proper investigation.

Why it matters

Fast rollback turns a bad release from an incident into a blip.

Words to know

  • Known-good versionThe last release that worked in production.
  • Expand/contract migrationChange schemas in backward-compatible steps.

Try it without code

Rehearse

  1. Deploy version N, then N+1 of your app.
  2. Time how long it takes to get back to N.

Use it in code

A manual rollback workflow that redeploys any previous image tag.

yaml
name: rollback
on:
  workflow_dispatch:
    inputs:
      tag: { description: "Image tag (commit SHA) to restore", required: true }
jobs:
  rollback:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: google-github-actions/auth@v2
        with: { workload_identity_provider: "${{ vars.WIF_PROVIDER }}", service_account: "${{ vars.DEPLOY_SA }}" }
      - uses: google-github-actions/setup-gcloud@v2
      - run: |
          gcloud run deploy rag-api --region europe-west1 \
            --image europe-west1-docker.pkg.dev/${{ vars.PROJECT }}/apps/rag-api:${{ inputs.tag }}
PRACTICE · TICK IT AFTER THIS

Add a rollback workflow to your repo and rehearse it once, recording the time taken in the README.

Common mistakes

  • Destructive migrations that make old code unusable.
  • Rollback steps nobody has tried.

Go deeper