Topic · 3 articles

System Design

October 20261

  • LLM Inference System Design: The Arithmetic Interviewers Want to See

    Design a chat-completion service and the scarce resource is not CPU or database connections - it is accelerator memory and bandwidth. KV-cache sizing, prefill versus decode, continuous batching, TTFT and TPOT, and token-based admission control, worked with numbers.

    10 min readAI CareersInterviewsSystem Design

August 20262