Topic · 3 articles

LLMs

October 20263

  • The LLM Practical Round: Concurrency, Retries, Token Budgets and a Quick Eval

    Some AI-lab loops now include a round where you work with a model through an API in a notebook. It is not a prompt-writing contest. It tests whether you can call a model a thousand times safely, cheaply and measurably. The patterns that pass, with code.

    9 min readAI CareersInterviewsLLMs
  • LLM Inference System Design: The Arithmetic Interviewers Want to See

    Design a chat-completion service and the scarce resource is not CPU or database connections - it is accelerator memory and bandwidth. KV-cache sizing, prefill versus decode, continuous batching, TTFT and TPOT, and token-based admission control, worked with numbers.

    10 min readAI CareersInterviewsSystem Design
  • Implement Attention From Scratch: The Interview Version and Its Follow-Ups

    Scaled dot-product attention fits in ten lines of NumPy. The interview is the follow-ups: why divide by the square root of d, how to make softmax stable, where the causal mask goes, what the KV cache saves, and why grouped-query attention exists. A complete walkthrough.

    10 min readAI CareersInterviewsMachine Learning