Open vs Closed Models

Why This Matters

This is one of the highest-stakes architecture decisions in any AI product. It determines your cost structure (per-token forever vs infrastructure you amortize), your data privacy posture (prompts leaving your network or not), your vendor risk (a single deprecation email can force a re-validation of your whole product), and your compliance story. Teams that treat it as ideology ("open good" / "closed better") instead of engineering make expensive mistakes in both directions: the startup that burns three engineer-months standing up GPU serving for 50K requests a day that an API would have handled for a few hundred dollars a month, and the hospital that ships patient data to a third-party API because nobody did the sovereignty analysis until legal review week.

Everyday Analogy

Closed API models are like taking taxis: no maintenance, always a recent car, pay per ride, but you never own the vehicle, prices can change, and the service can refuse routes. Open-weights models are like buying a car: upfront effort, you do the maintenance, but it is yours, and nobody can change the terms, raise the fare, or see where you drive. The analogy extends to the crossover math everyone eventually does: occasional riders should never buy; daily long-haul commuters should never rent. Your token volume is your mileage.

Three Categories, Not Two

• Closed / API-only (GPT, Claude, Gemini): weights never leave the provider; you rent capability per token. You get frontier quality, managed infrastructure, and someone else's safety and uptime engineering, and in exchange you accept their pricing, deprecation schedule, usage policies, and data-handling terms.
• Open-weights (Llama, DeepSeek, Mistral, Qwen, gpt-oss): you can download and run the weights anywhere, but training data and code stay private, and licenses may carry restrictions. This is what most "open source AI" actually means: DeepSeek-V3/R1 ship under MIT, Qwen mostly under Apache 2.0, while Llama uses a custom community license.
• Truly open-source (AI2's OLMo, EleutherAI's Pythia): weights, training data, and training code all released, giving full reproducibility and auditability, though mostly research-oriented and behind the open-weights leaders in capability.

The distinction is not pedantry. The Open Source Initiative has explicitly argued that Llama-style licenses don't meet the open-source definition, and the EU AI Act attaches different obligations to models depending on how openly they're released. When a vendor says "open," your first question is: which of the three?

Reading the license fine print

Concrete restrictions you will actually encounter:

Read before you build. "We found the license problem in week 30" is a genuinely common and entirely avoidable failure. Three specific clauses deserve a highlighter: the commercial-scale threshold (Llama's 700M monthly active users), the output-training restriction (may you distill from this model? see AI-065 on why that matters), and the acceptable-use policy (some licenses enumerate prohibited application domains, which can include yours). A five-minute read against those three questions catches the majority of real-world license surprises. Stanford's Foundation Model Transparency Index is a useful companion when you need to compare how open competing releases actually are; it scores vendors on disclosure of data, compute, and downstream policies rather than taking "open" labels at face value.

The Trade-off Table

Notice the pattern: closed wins every speed row, open wins every control row, and the cost rows flip depending on volume, which is why cost deserves actual arithmetic, not vibes.

Worked Example: The Break-Even Math

You run a summarization feature: each request is ~2,000 input tokens and ~500 output tokens. Compare a mid-tier closed API (illustrative pricing: $3/M input, $15/M output) against self-hosting an open 8B–70B-class model on one dedicated H100-class GPU at $2/hour ($1,450/month, reserved cloud pricing).

1. Per-request API cost: 2,000 × $3/1M + 500 × $15/1M = $0.006 + $0.0075 = $0.0135.
2. Self-host capacity: with vLLM-style batching, a well-tuned single GPU can sustain on the order of 5,000–10,000 tokens/s aggregate throughput on a small model. Call it ~10 requests/s for this workload, or ~860K requests/day at full utilization. Real fleets run at 30–50% utilization; assume 300K/day serviceable.
3. Break-even volume: $1,450/month ÷ $0.0135 ≈ 107K requests/month, about 3,600/day. Below that, the API is cheaper. Above it, each GPU you saturate replaces up to ~$120K/month of API spend at the 300K/day rate (300K × 30 × $0.0135 ≈ $121K).
4. Now add honest costs to the open side: an engineer maintaining serving at even 20% time is ~$3–5K/month loaded; add monitoring, redundancy (a second GPU for failover doubles hardware), and eval work every model upgrade. Realistic break-even moves to roughly 10–30K requests/day for most teams.
5. And the quality asterisk: this math assumed the open model meets your quality bar. If only a frontier model does, the comparison is void; capability requirements trump cost math.

The reusable skill is the procedure: (tokens/request × price) × volume vs (hardware + people), with utilization and redundancy included. Run it with current prices, since both sides move every quarter.

Vendor Risk, Concretely

"Vendor risk" sounds abstract until you enumerate what a closed provider can change on you, each of which has happened repeatedly across the industry:

• Deprecation. API models are retired on the provider's schedule, typically with 6–12 months' notice. Every deprecation forces you to re-run evals on the successor, and successors are not drop-in: prompt behavior, formatting habits, and refusal boundaries shift between versions. If your product embeds hundreds of tuned prompts, a model retirement is a project, not a config change.
• Silent behavior drift. Even a "same" model endpoint can change behind the scenes as providers update safety systems and serving stacks. Without your own regression evals running on a schedule, you learn about drift from your users.
• Pricing swings in both directions. Frontier per-token prices have fallen dramatically year over year, which is great, but the relative pricing between tiers changes too, invalidating last quarter's routing economics. Budget owners get surprised by both cuts and repackaging.
• Policy and terms changes. Acceptable-use policies, data-retention defaults, and training-on-your-data terms are amendable contracts. A use case that was fine at signing can become non-compliant later.

The open-weights hedge against all four is permanence: a pinned checkpoint of Llama or DeepSeek behaves identically forever, on hardware you choose. The corresponding cost is that nobody upgrades it for you either: frozen also means no free capability gains. The practical middle path is the one this lesson keeps returning to: your own eval suite run on every change (theirs or yours), plus an abstraction layer that makes any model swappable.

The Middle Paths

The open/closed poles hide a spectrum of real deployment options, and most enterprises land between them:

1. Closed API, standard terms: fastest, least control.
2. Closed API with enterprise contract: zero data retention, regional processing, HIPAA BAAs; solves many privacy objections without self-hosting.
3. Closed model in your cloud tenancy: offerings like Claude on AWS Bedrock or Gemini on Vertex keep traffic inside your cloud provider's boundary and your existing compliance perimeter, while weights remain the vendor's.
4. Open weights on a managed host (Together, Fireworks, Bedrock's open catalog): per-token pricing and no ops, but the model is portable, so if the host misbehaves, you take the same weights elsewhere, including in-house.
5. Open weights, self-hosted: maximum control, maximum responsibility.

Sliding down this list trades convenience for control one notch at a time. Many teams that "went open" actually run option 4, which delivers most of the portability and license benefits at API-level effort, and reserve option 5 for the workloads where sovereignty or scale demands it.

A Decision Framework

Work through four gates in order:

1. Sovereignty gate. Can the data legally and contractually leave your network? If no (patient records, defense, some banking under GDPR/data-residency rules) → open-weights, self-hosted (or a closed model deployed in your cloud tenancy; hybrid offerings like Bedrock/Vertex sit between the poles). Decision made; skip the rest.
2. Capability gate. Does an open model pass your eval suite for this task? If only frontier closed models pass → closed API, and revisit quarterly as open models improve. Most extraction, summarization, classification, and routine chat passes on good open models today; frontier-hard reasoning and agentic work often doesn't (see AI-063).
3. Economics gate. Run the break-even math above at your realistic 12-month volume. Under ~10K requests/day, the API almost always wins on total cost once engineering time is priced in. Above ~100K/day with stable load, self-hosting usually wins.
4. Ops gate. Do you have (or want to build) the team to run GPU serving with on-call, upgrades, and evals? If not, note that managed open-model hosts (Together, Fireworks, Bedrock's open-model catalog) give you open-weights portability with API-style operations, often the best of both for mid-sized teams.

Default for a new product: start closed, design portable. Put an abstraction layer over the model call, keep your prompts and evals provider-agnostic, and log everything, so the switch (either direction) is a config change plus an eval run, not a rewrite.

Do both when mature: many production systems route sensitive or high-volume traffic to self-hosted open models and hard problems to a frontier API, the same routing idea from AI-063 applied to sourcing.

The Narrowing Gap

In early 2023 the gap between frontier closed models and the best open models was enormous; GPT-4 had no open peer all year. Llama 2 (July 2023) and Llama 3 (2024) compressed the lag to months. Then DeepSeek-V3 and R1 (December 2024–January 2025) showed open-weights models matching frontier-class performance in math and code at a reported $5.6M pre-training compute cost (see AI-064), and the market repriced accordingly: Nvidia fell roughly 17% ($590B of market value, the largest single-day loss in US market history) on January 27, 2025. Even OpenAI, the archetypal closed lab, released open-weights models (gpt-oss) in 2025.

Why does any gap persist at all? Structural reasons: frontier training runs cost hundreds of millions of dollars in compute, and the labs that spend it monetize through APIs, so the newest capabilities debut closed. Open releases are subsidized by different business models: Meta commoditizing its complement, Chinese labs competing for mindshare and talent, Mistral selling enterprise deployments. Those incentives fund fast-following better than they fund frontier-first releases. Meanwhile the gap narrows from below because techniques travel even when weights don't: papers, distillation (see AI-065), and synthetic-data recipes (see AI-067) let open models reproduce closed capabilities months later at a fraction of the original cost.

The honest current picture: the very frontier, meaning the best reasoning and the longest agentic workflows, remains closed at any given moment, but the lag for "very good" has shrunk from years to months, and for many production tasks the open option already passes the bar. The strategic consequence: betting your architecture on "open models will always be far behind" is no longer safe, and neither is "open has caught up so APIs are obsolete." Design for portability; re-evaluate on a schedule.

Real-World Showcase

• Meta's Llama strategy, releasing strong open weights (over a billion downloads claimed by 2025), reshaped the market, seeded the entire fine-tuning and local-inference ecosystem, and made "self-host Llama" the default enterprise privacy answer.
• DeepSeek-R1 (January 2025) triggered a global repricing of AI: an MIT-licensed reasoning model matching expensive closed ones erased ~$590B of Nvidia market value in a day and forced every AI budget owner to rerun their open-vs-closed math.
• EU enterprises in regulated sectors increasingly mandate open-weights deployments inside their own datacenters for GDPR and data-sovereignty reasons. Sovereignty, not price, is the number-one cited driver in European open-model adoption.
• Managed open-model hosts (Together AI, Fireworks, AWS Bedrock's open catalog) built a whole business layer on the middle path: open-weights portability with closed-API convenience.

Common Misconceptions

• "Open source means I can do anything with it." Most "open" models are open-weights under licenses with real terms: Llama's 700M-MAU clause, non-commercial research releases, attribution requirements. The license is a build dependency; read it first.
• "Self-hosting is obviously cheaper, since there are no per-token fees." Below roughly 10K requests/day, hardware plus engineering time usually exceeds API spend (see the worked example). Unit economics favor open only at sustained volume.
• "Self-hosting is impossibly hard." It was in 2023; vLLM, TGI, and managed hosts have cut the burden dramatically. A single engineer can now run a solid mid-size deployment, and the cost is real but bounded.
• "Closed models are always better." For many production tasks, such as extraction, summarization, and classification, good open models already pass the quality bar, and per-use-case evals, not leaderboards, should decide.
• "Open models are safer/more private by definition." Only if you actually operate them well: a misconfigured self-hosted deployment leaks worse than a well-contracted API. Sovereignty is an opportunity, not a guarantee.
• "Pick once and move on." Prices, licenses, and capabilities shift quarterly; the durable asset is a portable architecture and your own eval suite, not this quarter's model choice.

Try It Yourself

1. Pick your own (real or hypothetical) product and score both options: monthly token spend at expected scale vs the cost of a GPU server; whether your data may legally leave your network; how much model customization you need. Write a three-sentence recommendation. This memo is the actual artifact engineering leaders produce.
2. Open the actual license of one "open" model you might use (Llama's community license and DeepSeek's MIT license make a sharp pair). List every obligation or restriction you find. Time yourself; it's shorter than you fear and more surprising than you expect.
3. Rerun the break-even worked example with your provider's current real prices and your realistic request volume. Which side of the crossover are you on, and by how much?

Mini Project

Build a portability proof. Take a small task your product might need (e.g., classify 50 support messages by intent). Implement it three ways behind one function interface: a closed API model, an open model via a managed host, and a local open model via Ollama. Use identical prompts and a shared eval script that reports accuracy, median latency, and cost per 1,000 requests for each backend (compute local "cost" from your machine's power draw or a rented-GPU hourly rate). Produce a one-page comparison table and a recommendation memo. The deliverable proves the architecture pattern this lesson advocates: models as swappable backends behind your own eval harness.

Key Takeaways

• Three tiers, not two: closed API, open-weights, truly open-source. Most "open source AI" is actually open-weights with a license you must read (Llama's 700M-MAU clause vs DeepSeek's MIT is the canonical contrast).
• The real trade is control, privacy, permanence, and at-scale unit economics versus convenience, speed-to-market, and peak capability.
• Cost is arithmetic, not ideology: at ~$0.0135/request vs ~$1,450/month per GPU, break-even lands around 10–30K requests/day once engineering time is honestly priced.
• Sovereignty requirements decide first, capability evals second, economics third, ops capacity fourth: run the gates in that order.
• The capability gap has compressed from years to months (Llama, then DeepSeek-R1's ~$590B market shock); design for portability and re-evaluate quarterly rather than betting on either side winning forever.
• Mature systems mix both, routing by sensitivity, cost, and difficulty. The durable assets are your abstraction layer and eval suite, not any single model choice.