The idea
Closed models are reached only through an API. You get the strongest general capability, no infrastructure work and per-token pricing, but your data goes to the provider (under their terms) and you depend on their availability and model updates.
Open-weight models can be downloaded and run anywhere: your laptop, your cloud, an on-prem server. You control the data, versions and fine-tuning, and at high steady volume self-hosting can be cheaper. You also own uptime, scaling, security patches and GPU bills.
Most teams mix them: a closed frontier model for hard, low-volume work and a small open or cheap model for high-volume simple tasks. Check licences: "open weights" licences differ in commercial use rules.
This choice drives cost, privacy, compliance and how much infrastructure your team must run.
Words to know
- Open weightsThe trained parameters are published for download.
- Data residencyWhere your data is processed and stored, often a legal requirement.
Try it without code
Same prompt, two worlds
- Run a prompt from your domain in a closed model chat app.
- Run it on a local open model with Ollama.
- Score both and note speed and privacy differences.
Use it in code
One function, two backends: a hosted API and a local Ollama model.
# pip install anthropic ollama
import anthropic, ollama
def ask(prompt, backend="api"):
if backend == "api":
msg = anthropic.Anthropic().messages.create(
model="claude-haiku-5-5", max_tokens=300,
messages=[{"role": "user", "content": prompt}])
return msg.content[0].text
return ollama.chat(model="llama3.2:3b",
messages=[{"role": "user", "content": prompt}])["message"]["content"]
q = "Summarise the GDPR right to erasure in two sentences."
print("API: ", ask(q))
print("LOCAL:", ask(q, "local"))Write a one-page decision note for a real use case: which model type, why, expected monthly cost and the privacy position.
Common mistakes
- Assuming open models are free; GPUs and engineers are not.
- Ignoring licence restrictions on open weights.