The idea
B2B AI products serve many customers (tenants) from shared infrastructure. Every layer must keep them apart: documents and vectors filtered by tenant, caches keyed by tenant, memories and logs scoped by tenant, and prompts that never mix data from two customers.
Fairness matters too: one tenant’s huge batch job must not exhaust the provider rate limit for everyone. Use per-tenant quotas, queues and concurrency limits. Track cost per tenant for billing and margin.
Some customers will require stronger isolation: dedicated indexes, their own encryption keys, a specific region, or their own model deployment. Design the tenant ID into everything from day one so these options stay possible.
Data leakage between tenants is a company-ending bug, and noisy neighbours are a constant production issue.
Words to know
- TenantOne customer organisation.
- Noisy neighbourA tenant whose load hurts others.
Try it without code
Trace a tenant ID
- Follow one request through your system on paper.
- At every store and cache, write how tenant isolation is enforced.
Use it in code
Enforce tenant isolation in Postgres with row-level security.
ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON chunks
USING (tenant_id = current_setting('app.tenant_id')::uuid);
-- In each request, inside a transaction:
SET LOCAL app.tenant_id = '6f1c...';
SELECT body FROM chunks ORDER BY embedding <=> $1 LIMIT 5; -- only this tenant's rowsAdd tenant_id and RLS to your RAG tables, then write a test that proves tenant A cannot retrieve tenant B’s chunks.
Common mistakes
- Filtering by tenant in application code only.
- Shared semantic cache across tenants.