Area 08 · Backend · Concept 11 of 20

Caching with Redis

Keep hot data in memory for fast reads and fewer database or LLM calls.

11–20 · Production levelPhase 0 + 5 · Weeks 1–2, 18–20≈ 1 hour

The idea

Redis is an in-memory data store with sub-millisecond reads. As a cache, it holds results that are expensive to compute and read often: user profiles, permission lookups, embedding results, and exact-match LLM responses for identical requests.

The common pattern is cache-aside: check Redis; on a miss, compute, store with a TTL, return. Choose keys carefully (include tenant, model and prompt version in LLM cache keys) and decide how entries are invalidated when underlying data changes.

Redis also powers rate limiters, queues, sessions, locks and pub/sub, so it is usually already in an AI stack.

Why it matters

Caching cuts latency and cost at the same time, and Redis is the default tool for it.

Words to know

  • TTLTime to live: when a key expires.
  • Cache-asideThe app checks the cache and fills it on a miss.

Try it without code

Try Redis

  1. Run: docker run -p 6379:6379 redis:7
  2. Run redis-cli, then SET greeting hello EX 10, GET greeting, wait 10 seconds and GET again.

Use it in code

Cache-aside for exact-match LLM responses and embeddings.

python
import hashlib, json, redis
r = redis.Redis(decode_responses=True)

def cache_key(kind, payload):
    return f"{kind}:" + hashlib.sha256(json.dumps(payload, sort_keys=True).encode()).hexdigest()

def cached_completion(call_llm, model, prompt_version, messages, ttl=3600):
    key = cache_key("llm", {"m": model, "v": prompt_version, "msgs": messages})
    if (hit := r.get(key)) is not None:
        return json.loads(hit)
    result = call_llm(model, messages)
    r.set(key, json.dumps(result), ex=ttl)
    return result
PRACTICE · TICK IT AFTER THIS

Cache embeddings of repeated queries in your RAG app and report hit rate and latency saved over 200 requests.

Common mistakes

  • Cache keys missing the tenant or prompt version.
  • No TTL, so stale data lives forever.

Go deeper