Introduction
Everyone wants privacy, but the best models often require giving up control of data. There's a real menu of options for keeping data private while embracing AI: private/self-hosted models, trusted execution environments, differential privacy, and confidential computing. The right answer depends on your data sensitivity, threat model, scale, and engineering maturity — not on which acronym your CISO read about last week.
Why this matters
- Regulated industries (health, finance, government) can't use commercial APIs without controls.
- Privacy techniques have very different cost, performance, and assurance profiles.
- Choosing the wrong technique wastes years; choosing the right one unlocks markets.
- Customers increasingly ask for technical privacy guarantees, not just policy ones.
Core concepts
Self-hosted / private models
Run open-weight models in your own VPC or on-prem. Data never leaves; you own everything. Costs: GPUs, ops complexity, model lag behind frontier.
Trusted Execution Environments (TEEs)
Hardware enclaves (Intel TDX, AMD SEV-SNP, NVIDIA Confidential Computing) that protect data in use, even from cloud operators. Strong assurance with cryptographic attestation.
Differential privacy
Mathematical guarantees that any single record's contribution to outputs is bounded. Excellent for aggregate analytics and fine-tuning; less useful for per-user generative outputs.
Confidential computing in practice
Combines TEEs, attestation, and remote-key services so a workload only runs on a verified, trusted environment. Increasingly available from major clouds.
Practical patterns
Sensitivity-tiered routing
Public data → commercial API. Internal → vendored API with DPA. Sensitive → self-hosted. Top secret → TEE-attested.
PII redaction at the edge
Detect and tokenise PII before it ever reaches the model; de-tokenise responses on the way back.
Differential privacy for fine-tuning
When you must train on user data, use DP-SGD; produces a quantifiable privacy budget.
BYO-key encryption
Customer-managed keys; even if data passes through your infra, you can't decrypt without explicit consent.
Pitfalls to avoid
- Confusing "private model" with "no leakage" — bad ops can leak from anywhere.
- Choosing TEEs when DP would do, or vice versa.
- Ignoring inference-time logging; the prompt is often the most sensitive thing.
- Promising privacy in marketing that engineering can't deliver.
Key takeaways
- 1There's a privacy spectrum; pick the point your data demands, not less, not more.
- 2TEEs and confidential computing are mainstream now — use them.
- 3Differential privacy is the right tool when aggregates matter and individuals don't.
- 4Document your privacy claims so engineering and legal agree on what's true.
Go deeper · external resources
Curated reading list to take you from primer to practitioner. All links are external and free to read.