Open vs Closed Source AI: When Each Wins, with Examples

Llama is free. GPT-4 charges $30 per million tokens. So why does anyone still pay?

Because "free" isn't the same as "cheap," and "open" isn't the same as "best." The honest answer is that most operators still pay because closed models are still better at the hardest tasks — for now.

This guide is for the builder, operator, or investor who has to actually choose. We'll skip the ideological war between Yann LeCun and Sam Altman and give you a usable framework: what each side actually offers, where each one wins, and what the convergence trend means for the bet you're about to make.

What "open" actually means in 2026

Almost no frontier large language model is "open source" in the strict Free Software Foundation sense. What gets shipped is open weights — the trained model parameters, free to download, fine-tune and run on your own hardware. The training data, the training code, and most of the recipe stay private.

The leading open-weights families are Meta's Llama (Llama 3.1 reached 405 billion parameters in July 2024), Mistral (the French lab whose Mixtral 8x22B and Mistral Large 2 set the bar for European deployments), DeepSeek (the Chinese lab whose V3 and R1 models shocked the field by matching GPT-4 class quality at a fraction of the training cost), Alibaba's Qwen, and Google's Gemma — the polite open cousin of the closed Gemini.

"Closed" means the weights live on the lab's servers and you rent access through an API, billed per token (a token is roughly three-quarters of an English word). The big three are OpenAI (GPT-4o, o1, o3), Anthropic (Claude 3.5 Sonnet, Claude 3 Opus), and Google DeepMind (Gemini 1.5 Pro, Gemini 2.0). You don't see the model. You see its outputs.

The distinction matters because it determines what you're allowed to do with the thing — host it, fine-tune it, audit it, run it on a plane — and what you have to ask permission for. Everything that follows flows from that one fact.

When closed wins

Three places, predictably.

The first is frontier quality. On the hardest benchmarks — multi-step reasoning, coding agents, olympiad-level math, genuine multimodality — closed models still lead by a measurable margin. As of early 2026, the top of the LMSYS Chatbot Arena (a public leaderboard where humans blind-rank model outputs) and the GPQA reasoning benchmark are owned by closed models from OpenAI, Anthropic and Google. If your application breaks when the model is wrong 5% more often — legal review, medical triage, complex financial analysis — that gap matters.

The second is operational overhead. Calling Claude or GPT-4 is one HTTP request. Self-hosting Llama 3.1 405B is a sixteen-GPU cluster, a model server (vLLM or TensorRT-LLM, the software that turns weights into a callable endpoint), an autoscaler, a monitoring stack, and someone on call when the inference engine runs out of memory at 3 a.m. For a team without an ML platform engineer, the API is the difference between shipping next Tuesday and shipping next quarter.

The third is iteration speed. Closed labs ship new model versions roughly quarterly, and your application inherits the improvement for free. The day GPT-4o replaced GPT-4 Turbo, every chatbot built on it got faster, cheaper and smarter overnight. Open models don't update themselves; you re-deploy, re-benchmark, re-test.

The pattern is sharper than it looks: closed wins the top of the difficulty distribution and the bottom of the team-size distribution. Hard problems and small teams. Both extremes pay the API premium for the same reason — they can't afford to be wrong about the model.

When open wins

Four places, not always obvious.

Data sovereignty. If your data cannot legally leave your premises — patient records under HIPAA, banking data under GDPR's data-localization regimes, defense contracts, EU institutions under the AI Act's high-risk classification — you cannot send it to a US-hosted API. End of discussion. Mistral or Llama running on a French or German bare-metal server is the only option that keeps your lawyer calm. This is the single biggest reason Mistral exists as a business.

Cost at scale. The crossover point is real. A startup processing ten million tokens a day pays roughly $300 a day on GPT-4o. The same workload on a self-hosted Llama 3 70B costs, by some estimates, around $30 to $50 a day in GPU rental once amortized. At a hundred million tokens, the math is brutal: a self-hosted open model becomes five to ten times cheaper than the equivalent closed API. DoorDash, Shopify and Stripe have all publicly described this exact migration — prototype on closed, scale on open.

Fine-tuning control. Closed APIs offer fine-tuning, but it's a black box: you upload data, you get back a model handle, you can't inspect what changed. Open weights let you do real surgery — LoRA adapters (small parameter add-ons that adjust behavior without retraining the whole model), full fine-tunes, distillation from a larger model into a smaller one, reinforcement learning on your own preference data. Bloomberg trained BloombergGPT on forty years of financial text. You couldn't do that on top of GPT-4 even if OpenAI let you, because you don't have the weights.

Auditability and longevity. A closed model can be deprecated. OpenAI announced the retirement of GPT-3.5 and original GPT-4 in 2025; thousands of fine-tuned applications had to migrate on someone else's schedule. An open weight you downloaded last year still runs next year. For regulated industries — medical devices, trading systems, anything that ships through a multi-year certification process — that immutability is the feature.

The mirror image of closed: open wins the middle of the difficulty distribution and the top of the volume distribution. Routine work, large teams. What most people miss is that the two camps don't really compete head-to-head — they own different parts of the workload.

The convergence trend

For most of 2023, frontier closed models led open weights by twelve to eighteen months and a wide quality margin. By early 2026, the gap is roughly six to nine months and shrinking.

Two events bent the curve. Llama 3.1 405B (July 2024) was the first open weight that benchmarks confirmed was within striking distance of GPT-4 on most non-reasoning tasks. DeepSeek V3 and R1 (December 2024 and January 2025) were reportedly trained for under $6 million and matched or beat GPT-4o on coding and math while shipping fully open. DeepSeek wiped roughly $600 billion off NVIDIA's market cap in a single January 2025 trading session — not because the underlying science was new, but because the cost curve they implied was.

The structural reason for convergence is that techniques diffuse. The transformer architecture — the design that underlies every modern LLM — was open from day one (Vaswani et al., 2017). Reinforcement learning from human feedback (RLHF, the technique that made ChatGPT feel useful), mixture-of-experts (splitting one big model into specialist sub-networks that each handle part of the work), and chain-of-thought reasoning (getting the model to show its work step by step) all reached open implementations within twelve months of the closed lab introducing them. The labs lead, the open community catches up, the labs lead again. (For more on why the gains keep coming, see our guide on scaling laws.)

What does not diffuse easily is data and compute. OpenAI and Anthropic spend on the order of a billion dollars on a single training run. Meta has the compute but treats Llama as a strategic loss leader, not a profit center. The open ecosystem catches up technically but not financially — which is why the frontier stays closed even as the body of the distribution opens up.

What this means for you

If you're a builder with an idea and uncertain demand, prototype on Claude or GPT-4. The wrong question to ask early is "which model is cheapest at scale" — you don't have scale, and you might not get there. You're paying for speed of iteration and a known quality floor. When your monthly API bill starts to rival the salary of the engineer it would take to migrate — call it $15K to $30K a month — start the open migration in parallel. Not before.

If you're an investor, the bet is not "open beats closed" or vice versa. Both win, in different segments. The interesting bet is on the picks-and-shovels of the open stack: the inference providers (Together, Fireworks, Groq, Cerebras), the fine-tuning tooling, the data-curation pipelines that turn a generic open weight into a vertical-specific asset. The closed labs will keep the frontier and the consumer mindshare. The open ecosystem will eat the enterprise back-end.

If you're a citizen or a regulator, the question is whether the frontier should ever be fully closed. Most jurisdictions will answer no — the EU AI Act already mandates technical documentation and risk assessments for the largest models, open or closed. Expect that pressure to grow. The interesting outcome is not victory for one side but a settled equilibrium: closed at the frontier, open as the substrate, regulation forcing transparency from both.

Ask Gradient

If you want to push further — into the specific cost crossover for your workload, or which open model best matches your latency and accuracy budget — try our Ask Gradient retrieval tool. A useful starter prompt: "At what monthly token volume does self-hosting Llama 3.1 70B beat the GPT-4o API for a customer-support workload?" The numerical answer changes every quarter. The framework above is what stays constant.