System Design: Google Borg

Teams get machines: service A owns hosts 1–40, batch jobs SSH into whatever is free, and a wiki page tracks who owns what. Every org starts here, and it fails in a specific, quantifiable way: each team over-provisions for its own peak, so fleet-wide utilization sits painfully low while everyone still claims to be out of capacity.

Static assignment also cannot handle failure (a dead machine means a human re-deploys) or bursty batch demand. The fix is to stop giving teams machines and start taking job descriptions: 'run 100 replicas, each needing 2 CPUs and 4 GB, priority production' — and let a scheduler decide where they run.

Carve the fleet into cells of ~10K machines. Each cell runs a replicated Borgmaster — the API and source of truth — with state checkpointed through a Paxos-backed store (Chubby-style) so a master failover completes in seconds without losing what is running where. Elected leader serves; replicas shadow.

On every machine sits a Borglet agent: it starts tasks in containers, enforces cgroup-style resource limits, and reports health. Crucially, the master polls Borglets rather than letting ~10K agents stampede a freshly elected master — a small design choice that is really a thundering-herd answer, and worth saying out loud.

The scheduler runs feasibility then scoring: filter machines that satisfy the task's resources and constraints (kernel version, architecture, anti-affinity), then score survivors — packing efficiency, spreading replicas across failure domains, and preferring machines that already cache the task's packages so startup is fast.

At around ~1M placements/min (~16K decisions/sec) during churny periods, the scheduler cannot re-scan the world per task. It scores against cached cell state, evaluates identical tasks in a batch once (equivalence classes), and samples random feasible machines rather than exhaustively ranking all ~10K. Decoupling the scheduler from the master's request path keeps placement latency off the API.

A full cell plus an incoming prod job is the moment the design earns its keep: every job has a priority band (monitoring > prod > batch > best-effort), and higher priority preempts lower — evicted batch tasks simply reschedule elsewhere. Prod jobs get killed for capacity approximately never; batch jobs treat eviction as routine, which is why batch fills the fleet fearlessly.

Quota keeps priorities honest: admission control checks a team's purchased quota at submit time, at that priority, so priority is not a free-for-all — you cannot bid everything to 'prod' without paying for it. Thousands of teams share a cell through this two-gate system: quota decides admission, priority decides survival.

Reserved-but-unused resources are the fleet's biggest waste, so Borg reclaims them: the Borglet measures actual usage, the master offers the headroom to batch work, and reclamation backs off when prod usage climbs. Mixing prod and batch in shared cells — rather than segregated clusters — is what pushes utilization high enough to matter at fleet economics scale; this is the headline result of the Borg paper.

Above the cell sits a thin global layer: per-cell autonomy for blast-radius containment, global tooling for cross-cell job submission, maintenance drains, and rolling updates that respect disruption budgets. Telemetry rounds it out — Borglets report every few seconds, tens of millions of metrics/min feeding dashboards and the health checks that auto-restart failed tasks. End by naming the descendants: this architecture, generalized, is Kubernetes.