Build vs Buy: A Practical Decision Tree for AI Products in 2026

Three engineers, a $40k OpenAI bill, and a meeting that goes nowhere

Most "build versus buy" debates in 2026 are not really debates. They are decisions made on vibes — by founders who want to "own the model," CTOs who want to control costs, or investors who want a defensible moat on the pitch deck.

This guide gives you the framework underneath the vibes. You will leave with one question that settles the call, and a clear view of which of the four layers of the AI stack you should actually be playing on.

The four layers — where value lives, and where it doesn't

Every AI product is a stack of four layers. Confusing them is the most expensive mistake you can make.

The foundation model layer — the raw model trained from scratch on trillions of tokens, the kind of thing GPT-4, Claude 3, or Llama 3 is — sits with OpenAI, Anthropic, Google DeepMind, Meta, and a handful of Chinese labs (DeepSeek, Qwen, Kimi). Training a frontier model costs $100M to $1B in compute alone, plus a research team of 50 to 300 people. For the longer version of what these models actually are, see What is an LLM.

The fine-tuning layer takes a foundation model and adapts it — through supervised post-training, RLHF (reinforcement learning from human feedback), or domain-specific data — to a narrower task. Costs range from $10k for a LoRA adapter (low-rank adaptation, a cheap technique that nudges behavior by editing only a small slice of the weights) to $10M for a full RLHF run.

The application layer is the product the user actually touches: retrieval, agents, UX, evaluation harnesses, integrations. Cursor, Harvey, Perplexity, Notion AI all live here. The model is a component, not the product.

The distribution layer is everything that gets the product into users' hands: brand, sales motion, embedded placement in existing workflows. Microsoft Copilot does not win because GPT-4 is the best model on the market. It wins because it ships inside Word.

What most people miss: value is moving down the stack. Foundation models commoditize. Distribution does not.

Building at the foundation layer: don't

This is the easiest call in the entire decision tree, and the one most often gotten wrong by ambitious founders. If you do not have $1B in committed capital and a research team that can credibly hire from OpenAI or DeepMind, you are not building a foundation model. Period.

The numbers do not bend. GPT-4 reportedly cost north of $100M to train. Llama 3.1 405B used roughly 30M H100-hours, somewhere above $60M in compute, on top of a research org of several hundred people. xAI's Memphis cluster — 100,000 H100s and counting — represents around $4B in NVIDIA hardware before electricity. These are the costs of catching up to last year's frontier, not setting this year's.

The temptation is to say "but we will fine-tune on our domain, that is different." It is — that is the next layer down, and we will get to it. Building a frontier model from scratch is a $1B+ research bet against five labs that already have a 24-month head start, $50B+ in compute commitments, and the only people on earth who have done it before.

The exceptions are sovereign-AI plays — France's Mistral, the UAE's Falcon, India's Krutrim — which exist for reasons that are not strictly economic. If you have those reasons, you already know.

Building at the fine-tuning and application layer: when domain expertise is the moat

This is where the interesting decisions happen. Build here when — and only when — your domain knowledge is something the foundation labs cannot easily replicate.

Harvey, the legal-AI company valued at around $5B in 2025, did not build a foundation model. It built an application on top of GPT-4 with deep integration into legal workflows, evaluation against actual case law, and partnerships with firms like Allen & Overy that gave it proprietary feedback at scale. The moat is not the model. It is the legal-domain data and feedback loop that no general lab will spend three years assembling.

Cursor, the AI code editor, runs on Anthropic's Claude. Its moat is the editor itself: how it indexes a codebase, how it predicts your next edit, how it handles multi-file refactors. Cursor reportedly went from $1M to $100M in annual revenue in roughly twelve months because the application layer was where the actual user experience lived.

The test is simple. Ask: what does my company know that OpenAI does not, and cannot easily learn in eighteen months? If the answer is "nothing — we just want our own GPT," buy. If the answer is "we have ten years of proprietary medical imaging labels, fifty radiologists on staff, and an FDA clearance pathway," build.

Fine-tuning specifically is worth its own note. Most teams overestimate it. For 90% of use cases, prompting plus retrieval — RAG (retrieval-augmented generation), where the model is handed the relevant documents at query time — beats fine-tuning on cost, latency, and maintenance. Fine-tune when you need behavior the base model cannot be prompted into: narrow style, structured output, low-latency edge deployment. Not because it sounds more sophisticated in a board deck.

Buying at the API layer: the default for 95% of cases

If you are reading this and shipping a product in 2026, the default is to call an API. This will feel anticlimactic. It is supposed to.

API pricing has fallen roughly 10x per year since GPT-3.5 launched in late 2022. GPT-4 input tokens cost $30 per million in March 2023; GPT-4o cost $2.50 per million by mid-2024; current frontier-class small models like Claude Haiku 4.5 and Gemini Flash sit below $1 per million on most tasks. The economic argument for "we will be cheaper if we self-host" was true in 2023 and is false in 2026 for almost everyone.

Switching costs are also collapsing. The OpenAI Chat Completions API, the Anthropic Messages API, and Google's Gemini API are close enough in shape that abstraction layers — LiteLLM, OpenRouter, Vercel's AI SDK — make swapping providers a config change rather than a rewrite. Lock-in fears were overcorrected.

The real cost of buying is not the per-token bill. It is exposure to the lab's roadmap: model deprecations, rate limits, content-policy changes, latency variance, sudden capability shifts. Plan for it. Use multiple providers. Run evaluations on every model bump. Do not pretend it is not happening.

The one place buying breaks down is regulated workloads — healthcare data, defense, EU residency, financial KYC — where data cannot leave a specific jurisdiction or an air-gapped network. Even there, the answer is usually a self-hosted Llama 3 or Mistral, not a from-scratch model.

Which collapses the entire framework into one question. Where, in your product's value chain, is the part a competitor cannot copy in eighteen months? If the answer is "the model itself," you need $1B and you do not have it — reconsider the business. If the answer is "our domain data, our feedback loop, our integration into a workflow," build at the application layer and buy the model underneath. If the answer is "we don't know yet," buy. You can always build later. You cannot un-build $50M of wasted compute.

What this means for you

If you are a builder: assume buy by default, justify build by exception. Write down the moat in one sentence before you write a line of code. If you cannot, you are about to spend a year building something a $20-per-month subscription will replace by Q3.

If you are an investor: be suspicious of anyone pitching a foundation model without a sovereign-AI thesis or a $500M+ Series B already lined up. Be interested in application-layer companies whose moat is a feedback loop — proprietary user data that improves the product faster than competitors can copy. Cursor, Harvey, Glean, and Perplexity all fit this pattern. Pure wrappers without feedback loops do not, and the market is starting to notice.

If you are an operator adding AI to an existing product: buy the API, ship in six weeks, measure usage, iterate. The companies that won the early mobile era were not the ones who built their own iOS. They were the ones who shipped a great app on iOS in 2009. The same pattern is repeating now, on a faster clock.

Ask Gradient

If you want to pressure-test your own build-versus-buy call against current model pricing, capability benchmarks, and the recent moves of the frontier labs, use Ask Gradient — our RAG-based research product trained on every Gradient guide, leaderboard snapshot, and Radar analysis. Try a question like: "Given my use case is legal contract review for a 50-person firm, should I fine-tune Claude or build on top of a vertical API like Harvey?" Ask Gradient will pull the relevant frameworks, current pricing, and comparable case studies into a single answer.