Area 07 · Deployment · Concept 03 of 20

Kubernetes Basics

Pods, deployments and services: running and scaling containers on a cluster.

01–10 · The basePhase 5 · Weeks 18–20≈ 3 hours

The idea

Kubernetes runs containers across a cluster of machines and keeps them in the state you declare. A Pod is one or more containers scheduled together. A Deployment says "keep 3 replicas of this pod running with this image" and handles rolling updates. A Service gives pods a stable network name and load-balances across them.

You describe resources in YAML and apply them with kubectl. Kubernetes restarts crashed containers, replaces failed nodes, scales replicas and rolls out new versions gradually.

It is powerful and complex. Many AI teams start on simpler platforms (Cloud Run, ECS, Fly) and move to Kubernetes for GPU scheduling, many services or strict control.

Why it matters

Most large companies run on Kubernetes, and GPU model serving at scale usually means Kubernetes.

Picture it

Words to know

  • PodSmallest deployable unit: one or more containers.
  • DeploymentManages replicas and rollouts of pods.
  • ServiceStable address and load balancing for pods.

Try it without code

Interactive tutorial

  1. Open the Kubernetes Basics tutorial.
  2. Create a deployment, expose it, scale it and update it in the browser sandbox.

Use it in code

Deployment and Service for the RAG API.

yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: rag-api }
spec:
  replicas: 2
  selector: { matchLabels: { app: rag-api } }
  template:
    metadata: { labels: { app: rag-api } }
    spec:
      containers:
        - name: api
          image: ghcr.io/you/rag-api:1.4.0
          ports: [{ containerPort: 8000 }]
          envFrom: [{ secretRef: { name: rag-api-secrets } }]
          resources: { requests: { cpu: 250m, memory: 512Mi }, limits: { memory: 1Gi } }
          readinessProbe: { httpGet: { path: /ready, port: 8000 } }
          livenessProbe: { httpGet: { path: /health, port: 8000 } }
---
apiVersion: v1
kind: Service
metadata: { name: rag-api }
spec:
  selector: { app: rag-api }
  ports: [{ port: 80, targetPort: 8000 }]
$ kind create cluster && kubectl apply -f k8s.yaml && kubectl get pods
PRACTICE · TICK IT AFTER THIS

Run your API on a local kind or minikube cluster with 2 replicas, then kill a pod and watch it come back.

Common mistakes

  • No resource requests, so scheduling is unpredictable.
  • Using :latest tags, so you cannot tell what is running.

Go deeper