Deciphering the Allure of the Open-Weight Model

“Love, er, usage load profiles are all you need”

The Beatles, today, probably

Open-weight models. At least seven portfolio companies / clients / investors / ecosystem partners have asked me for my view on open-weight models in the past three weeks. This barrage of questions probably stems from the spectacular performance of some newly released open-weight models on recognised public benchmarks. [NB: and they are spectacular at the headline level. On the broad composite indices, frontier models are sometimes only about half a step to a step ahead of the latest open-weight contenders.¹]

Much of the recent discussion I have had on this topic revolves around one critical question: how much is using this stuff going to cost us now (and in the future)? After earlier generations of headline token price cuts, the effective cost per completed task from the frontier model providers had been holding steady or declining slightly. [These prices are constantly changing so take these observations qualitatively.] And CFOs are looking for any relief from margin erosion as they retool their business and operating models to become GenAI-enabled (especially as much of the early increase in individual productive capacity still has not translated into business model returns.) I explored the ROI question recently in Tools, Tools, Tools — But Not a Drop of ROI in Sight.

Open-weight models have recently provided a seductive alternative. Their headline prices seem to be a lot lower. The unbundling of infrastructure from model-weight IP allows you to shop around for the GPU power and storage your workflows require (including self-hosting, specialist LLM hosting, or general cloud container hosting).

Many of the questions over the past few weeks collapse three different architecture and procurement decisions into a single thread. Most of my friends were weighing up one (or more) of three strategies:

1. Consume an open-weight model (OWM) from the developer using their API or a specialist serverless host (paying by the token)
2. Rent GPU capacity (mostly by the hour) from a hyperscale or specialist cloud provider and serve the weights themselves
3. Buy the GPU racks (with all the capital and operating expense), find the power and cooling, and run the OWM themselves.

The “OWMs are cheap” crowd generally clustered around strategy 1 and the “I actually want control” crowd around strategy 3. These alternatives do not belong in a side-by-side comparison, as the underlying motivations for them are really quite different. They also put the risk in very different places:

Three routes with three risk profiles

So, to be controversial, I think that the rush to embrace open-weight models is overhyped. Open weights are very good, and getting better, but model price is the wrong unit of comparison. What matters is the cost of a successfully completed task, which depends on your workload, architecture, quality bar, utilisation and contract.

The math in its simplest form is:

cost per successful task = (price per token × tokens per attempt) ÷ probability of success

I know it can be very dangerous when a business integrator like me wades into the technical architecture and technology procurement decision space. But, stay with me?!? [And to my CTIO, CAIO, CDO, CPrO friends, please forgive me? I generally do not wade into technology architecture questions with this much detail.]

My view on open-weight versus commercial model providers swings on six beliefs, which move from price, to cost, to cost per successful task.

1. Price does not equal cost

Yes, commercial frontier model providers have headline token prices substantially higher than their open-weight alternatives. However, I believe that this price disparity narrows substantially when you factor in three things:

Enterprise clients are subsidising everyone else. Much of the inference capacity (i.e., the day-to-day use of LLMs to do work) is supporting free and seat-based users in consumer plans or legacy contracts. OpenAI is the clearest case: roughly 95% of its 900 million or so weekly users pay nothing.² Its company-wide adjusted gross margin was reportedly 33% in 2025, while its “compute margin” on paying users reportedly reached about 70%.³ The customer value economics of OpenAI (and the enterprise customers’ role in supporting it) are pretty clear.

Gross margins on inference are large. Publicly reported gross margins look thin because (again for OpenAI at least) they blend the cost of serving paid tokens with the cost of serving free ones (the research and training programme sits outside the cost of revenue altogether; OpenAI reportedly spent $6.7bn on R&D against $4.3bn of revenue in the first half of 2025).⁴ Anthropic’s figure is closer to a paid-customer margin, and its trajectory is an interesting signal: roughly −94% in 2024, about 40% in 2025 and (on recent analysts’ estimates) somewhere between the mid-forties and the low sixties this year.⁵

That is what is reported. My own very rough estimate (triangulated against those margins and against what the labs pay for capacity at scale) is that a frontier lab’s true serving cost is around a quarter of the list price. So on the document-processing workload below, it would be somewhere under $2 per million tokens against a “list price” of a little over $7. Which is to say: the frontier providers are not that much more expensive to run than OWA. They are (currently) expensive to buy from. This poses a different problem.

One caution. Anthropic’s serving cost is invisible to your procurement team so what you pay is your negotiated price. The serving cost tells you how much room there might be to negotiate now (and in the future) and how far prices can fall. Leverage also scales with commitment: a portfolio company spending low seven figures a year will not get the terms of an eight-figure commit (a credible open-weight fallback might help as I note at the end).

Total cost of ownership of self-hosted open-weight models is underestimated. The self-hosted business case I see most often is a GPU price, an electricity bill, and a throughput number from a vendor benchmark.

Four things are missing, and each is larger than the electricity:

• Utilisation. Vendor maths assumes the GPUs are busy 80–100% of the time. In my experience, real enterprise fleets can run nearer 20–40%, because you provision for the noon peak and pay for midnight.⁶ Moving from 30% to 70% utilisation more than halves the cost per token but below 30% costs can skyrocket.
• Cost of capital and useful life. A $540k eight-GPU node charged at a 10% hurdle rate over five years is roughly $140k a year before it does anything. Whether the useful life is one year or three years or six is the subject of a live argument between the neoclouds’ accountants and their short-sellers and that answer moves the capital charge by more than 40%.
• Redundancy and people. A 24×7 production endpoint needs N+1 capacity and someone on call. Two fully loaded MLOps engineers may cost around $600k a year before the first token,⁶ which makes a lightly utilised single-node deployment very hard to justify against using a good cloud API.
• Facilities. Rack-scale Blackwell systems need liquid cooling. GPU-dense colocation in the primary US markets is scarce, 18–24 months out and rising in price⁶. The grid interconnect is the constraint on anything you build yourself at large enterprise scale.
• Electricity (the line everyone argues about) is only a single-digit percentage of fully loaded cost in the configurations I have modelled.

Add those missing four things up and the picture changes. The node itself (capital, facilities, power and maintenance) may run to around $220k a year (or about $3 per GPU-hour at full utilisation). That means roughly $0.60 per million tokens serving a GLM-5.3-class model on a chat-style workload at 70% utilisation. That’s not a bad deal (if you can keep it full). But a production endpoint also needs the spare node and the people. Fully loaded, a single-node deployment costs about $1m a year or roughly $2.80 per million tokens at 70% utilisation. This cost is more than GLM’s own developer charges through its API with none of the capital deployment and none of the people. A fleet only gets below $1 per million tokens at around eight busy nodes (or roughly 240 billion tokens a month).⁶

[NB: again please take these figures as directional and not decisive. They do not represent any sort of advice – investment or otherwise.]

2. Headline “tokens as a proxy” is now a broken currency

A sometimes overlooked aspect of recent frontier model makers’ pricing announcements is that a token is not a token is not a token. Well, they still are, kinda. But the realised cost to the user of that “token” differs substantially. Reading material already in a frontier model’s prompt cache now costs only 2.5–10% of ingesting new material into the context window.⁷ Thus, the true ‘blended price’ of the tokens (input, cached, output) is hugely dependent on your workflows, usage patterns, and repetition. The open-weight developers are moving the same way, if less aggressively (GLM-5.3 prices cached input at about a fifth of fresh input).

Two example workflows illustrate the point (at list prices on 28th September 2026). A document-processing flow that reads four million tokens of new material for every million it writes costs about $7.20 per million on Opus 5.5 and $2 on GLM-5.3 through its own API (the roughly four-to-one gap the sceptics quote). A coding agent that carries a 64,000-token working context, writes 400 tokens per turn and hits the cache nine times in ten costs about four and a half cents per request on Opus 5.5 and ten cents on Fable 5.1 and two and a half cents on GLM. Change nothing but the shape of the work and the frontier premium falls from 3.6x to 1.8x on the very workload where the quality difference matters most. Anthropic’s mid-tier Sonnet 5 (whose $2/$10 introductory price became permanent in August) comes in under three cents, which is roughly level with GLM.⁷

Caching changes the self-hosting maths too. If you rent or own the GPUs, you pay flat hourly or capital charges whatever you run. So a cached prefix shows up as throughput rather than as a discount. In other words, the same node serves more requests, which is why cache-heavy agentic work is the one shape where self-hosting can shine. [NB: some serverless open-model hosts do discount cache reads, but right now less deeply.]

3. “Solution inflation” is real

While it is tempting to treat a workflow processed through a frontier model the same way as it would be processed through an open-weight model, this assumption is probably not true anymore.

The public evidence has become harder to decipher in the last few months. On the headline coding benchmark the gap has all but closed (e.g., on Vals AI’s independent SWE-bench run: Opus 5 97.0%, DeepSeek V4 Pro 96.4%, GLM-5.3 95.4%). Vals recently retired SWE-bench Verified as it became saturated.⁸ On broader composite measures, the frontier still leads by a few points.¹ Where the benchmarks still discriminate (i.e., on longer multi-step agentic work) the gap is much wider. On Vals AI’s Terminal-Bench 2.1 runs, Opus 5 scores 84.6% against GLM-5.3’s 71.5% (and the gap on SkillsBench is similar).⁸ Put crudely, the frontier model fails roughly one of those tasks in six or seven; the open model fails one in three or four. A model that needs the extra attempts with the longer reasoning traces and greater amount of human review stacks up more expense for all of it (both in rework tokens and in human time).

This is why I tell my companies that the right unit is not the token but the completed task. In its fuller form:

economic cost per successful task = (inference + infrastructure + orchestration + retry + human-review cost) ÷ probability of success + expected cost of undetected failure

That last term can dominate. For brainstorming, a failure might cost pennies. For a mis-drafted many hundred-page contract, a wrong reconciliation or a broken deployment, a single false success can easily outweigh the token savings. On the coding-agent example above (at SWE-bench success rates), GLM-5.3 would need about 1.7 times as many tokens as Opus 5.5 to finish the same job before Opus 5.5 becomes the cheaper option per completed task. If you take it at Terminal-Bench success rates, that falls to about 1.5x. You must measure the inflation factor on your own traffic before you believe any vendor’s number (including mine!).

4. Frontier model cost structures are not static

Frontier model providers are increasingly using techniques to reduce the inference processing burden and newcomers are promising even greater reductions (Subquadratic, for example, emerged from stealth in May with a sparse-attention architecture).⁹ Open-weight developers are doing the same (e.g., Kimi K3 uses a hybrid linear attention), so efficiency gains are being made by both sides. The question is how quickly they translate into price reductions. As a painful case in point: while I was finishing this piece, Anthropic released Opus 5.5 with 20% lower token prices and 60% cheaper cache reads than Opus 5.⁷ In one release and a round of LLM-assisted draft revisions on this thought piece, the break-even inflation factor on my coding-agent example (at SWE-bench success rates) fell from about 2.7x to about 1.7x.

5. You are not paying just for inferential tokens

Commercial frontier model providers also provide SOC 2 compliance, cybersecurity, uptime reliability, etc. These attributes are becoming (and may already be) as important as the pure inferential performance of the model itself, especially as new regulatory regimes arrive.

The list of these benefits is longer than you might think: contractual SLAs with credits; IP indemnification for model output; data-residency and regional serving; a published deprecation policy so that a model you built on is not withdrawn on ninety days’ notice; safety and abuse guardrails you did not have to build; consistent evaluation harnesses so that “the model got worse” is a claim you can test. (To be fair, much of this is available on the open-weight side too if you buy the OWM through a cloud hyperscaler.)

Two further factors influence your non-inference “value add” analysis. The first, version pinning, favours the static open-weight providers (a weight file never changes under you while an API model may). The second, provenance, mostly does not. Note the providers who currently lead the open-weight league table (Zhipu, DeepSeek, Moonshot, Alibaba, MiniMax). The licences are no longer uniformly permissive (DeepSeek V4 is MIT; GLM-5.3 and Kimi K3 carry custom licences).¹⁰ The US-origin OWM alternatives trail well behind⁸ and for a regulated US or UK business the export-control, data-provenance and supplier-risk questions can be difficult. And unfortunately, the cheapest serverless hosts for the leading models are frequently the least willing to sign anything. That is a compliance cost that never appears in the per-token price.

6. The hardware is a depreciating bet on a deflating market

For an equivalent level of capability, the price of a million tokens has fallen roughly ten-fold a year on the most-cited estimate, and much faster on some tasks.¹¹ The serverless open-model hosts often pass that deflation through quickly. A GPU bought today is a five-year fixed-price bet against that curve (H100 rental rates are already down roughly 70–75% from their peak).¹² For a PE-backed company, I think the practical consequence is blunt. GPU capex on a portfolio-company balance sheet is a financing decision as much as a technology one. It should be made at the cheapest available cost of capital – or not at all. Be careful what you benchmark against. The first investment-grade GPU-backed financing (CoreWeave’s in March) priced at roughly 6%, but the rating rested on a contract with an investment-grade customer (Meta).¹³ Without that kind of offtake contract, you might expect something closer to private-credit pricing (historically 12–15%).⁶

So where does this leave us?

Well, I am not advocating that the open-weight movement is a waste of time. Open-weight models likely have some meaningful role to play in your AI model strategy (especially at higher agentic burn production levels).

There are five situations where I would argue for open weights with little hesitation:

• Sovereignty and data that cannot leave. If the data cannot go to a third party, the economics are irrelevant. The only question for your inference workload model is which route: rent or own.
• Fine-tuning and distillation. The strongest open-weight play is to distil a narrow, small, fast model for the 80% of volume that is routine, with full control over how it is trained, tuned, served and retained. (Be careful using a frontier model as the teacher: provider terms typically restrict training competing models on outputs, and the newest frontier models ship with anti-distillation safeguards.⁷)
• Cache-heavy agentic scale. Hundreds of billions of tokens a month of standing-context agent traffic, at 60%-plus utilisation across a fleet, is the one workload where owned hardware should beat every API on price. It is also the shape that many AI-native product companies actually have.
• Version pinning and vendor concentration. A frozen weight file is a hedge against deprecation, silent regressions and a model supplier that has just become your competitor. It also hedges against access being interrupted from outside. As we know in June, Anthropic suspended access to Fable 5 and Mythos 5 for nearly three weeks to comply with US export controls. That access only was restored on 1st July once the controls were lifted.¹⁴
• Negotiating leverage. A credible, evaluated open-weight fallback might be worth real money in a frontier-provider negotiation even if it never serves a production token. That alone can justify the pilot and the evaluation of production preparation requirements.

But there a number of broader insights which management teams (and data engineering implementation firms and tech operating partners) need to remember:

• Technology procurement is now as important as model selection. If you remember only one thing from this long thought piece, remember that you need to manage and negotiate your Total Cost of (Model Operations) Ownership hard. The full lifecycle costs (opex and capex), the ‘swappability of models’ (which can vary a lot, especially if you operate in a regulated industry), and the surrounding commercial terms all matter. I appreciate that right now it sometimes can feel like “we just need to strike an agreement with X”. That is only somewhat true. When you start building GenAI-native capabilities, you do need to think about the ongoing run-operate production costs if the organisation will adopt them at scale. This is a Target Operating Model question as much as a procurement one: it lives in the technology infrastructure and data layers and swappability depends on the harness you build around the models (as I argued in The Model Is Not the Moat).
• Other operating and technology architecture levers also matter. Much is made of smart routing of workload into more capable / more expensive and less capable / less expensive models. That capability remains a key cost control approach in many circumstances. But cache and context window switching mid-workflow can get really expensive: a re-route that forces a fresh cache write (billed at 1.25–2x base input on some platforms) can wipe out any inference cost saving.⁷ So do not re-route extremely high-value output work for small marginal savings in model opex. The savings may be smaller than you think, and the quality penalty material.
• Measure before you build. Three numbers dominate the economics (none of them is on a rate card): the production utilisation you can sustain on a single model, the prompt-cache hit rate of your real workflows, and the solution-inflation factor of the open-weight model on your tasks. A ninety-day evaluation on live traffic will give you a good view on all three for less than the cost of one idle GPU node. Then take the decisions in order:
  1. Does the data have to stay with you? If it does, the economics only decide whether you rent or own.
  2. Does the open-weight model clear your quality bar? Compare its inflation factor on your own tasks with the break-even against the frontier price you can actually negotiate (on my coding-agent example at list prices, about 1.5–1.7x against Opus 5.5). If it does not clear, buy frontier tokens and negotiate.
  3. If it clears, start with an open-weight API, through a host that will sign your terms. You pay by the token, so utilisation is the host’s problem, which makes it the cheaper route per outcome at any volume.
  4. Move to your own GPUs only for large, steady volume. As a directional heuristic, sustained utilisation above 60% and cache hits above 80%, at hundreds of billions of tokens a month, is where self-hosting starts to beat the API.
  5. Then decide rent or own on cost of capital. Own only if you can finance close to investment-grade rates and keep the fleet busy for its useful life; otherwise rent.

  Latency and switching costs can still override the answers to the sequence above. And you should probably re-run this analysis after every frontier price cut, because the break-even will move with them. [And if you’re already super close to cost parity, you may just opt for the frontier model.]

• Negotiate the parts of the rate card nobody reads. To the degree that you can, you should try to address cache-read and cache-write prices, batch discounts, committed-use tiers, tokeniser-change protection and deprecation notice periods. These contractual elements may move the effective price more than the headline input/output rates do. An input-heavy workload that is 90% cached on Fable 5.1 costs less than the same workload uncached on a model two tiers down. The negotiation should be about the multiplier rather than the sticker ‘token price’.

So my answer to everyone who has asked me lately has been basically the same. Open weights aren’t inherently cheap and frontier APIs aren’t inherently expensive. Model economics are an emergent expression of your workflows, workloads, operating architecture, quality expectations, model utilisation and commercial contracts. Frontier providers are cheaper to run than you might think and more expensive to buy from than they need to be. Open-weight alternatives may be cheaper to buy than you think but much more expensive to run than most people will tell you.

The gap between these two alternatives is your load profile, your utilisation, your quality bar, your cost of capital and your negotiating position. All of these are factors which are yours to manage (and none of which will be fixed by a benchmark). Love, in other words, is not all you need. The load profile is.

Notes

Figures in the text are my entirely estimates rather than published data; note 6 sets out the modelling assumptions.

1. Artificial Analysis Intelligence Index v4.1.1: Claude Opus 5 (max) 63, Kimi K3 (max) 60, as reported by Codersera, “Kimi K3 Benchmarks vs Opus 5, GPT-5.6 Sol & Fable 5”, August 2026. codersera.com/blog/kimi-k3-benchmarks-comparison-2026 ↩

2. The Information, as reported by Michael Parekh, “AI: Diving into OpenAI vs Anthropic financial growth metrics”, March 2026 (non-paying users roughly 95% of about 900 million weekly actives). michaelparekh.substack.com/p/ai-diving-into-openai-vs-anthropic ↩

3. Adjusted gross margin of 33% in 2025, down from 40% in 2024: The Information, as summarised by Sacra (sacra.com/c/openai). Compute margin of about 70% by October 2025: The Information, as summarised by SaaStr (saastr.com). ↩

4. The Information and Reuters reporting on OpenAI’s first-half 2025 financials. ↩

5. The Information, “Anthropic Lowers Gross Margin Projection as Revenue Skyrockets”, 21st January 2026 (about 40% for 2025 on sales to businesses and developers; −94% in 2024). 2026 estimates vary with method: roughly 44% on compute cost per revenue dollar (Yahoo Finance, “Anthropic’s gross margin is the most important number in tech”, June 2026) to around 60% on other analyst estimates (Value Add VC, “Is Anthropic Profitable in 2026?”, August 2026). A 63% plan figure has been attributed to The Information. ↩

6. Author’s modelling assumptions and estimates are not observed market data. Enterprise fleet utilisation of 20–40% and colocation lead times of 18–24 months are the author’s estimates and not published figures. Two MLOps engineers at about $300k each, fully loaded (US estimated rates). Private GPU lending at 12–15% is a historical range (CoreWeave’s 2023 facility priced at about 15%); current terms for a portfolio company will vary. Node model: $540k eight-GPU B200 node, 10% hurdle rate, five-year life, roughly $80k a year of facilities, power and maintenance, one spare node, and about 2,000 tokens per second per GPU on a chat-style workload. ↩

7. Anthropic, “Introducing Claude Opus 5.5”, 22nd September 2026 (Opus 5.5 at $4/$20 per million tokens, cache reads $0.20, cache writes $5; Opus 5 at $5/$25, $0.50 and $6.25; the preserved-thinking anti-distillation safeguard). anthropic.com/claude-opus-5-5. Fable 5.1 at $10/$50 with $0.25 cache reads (BenchLM, September 2026). GLM-5.3 at $1.40/$4.40 with $0.26 cached input (Z.ai list prices via LLM Stats). Sonnet 5 at $2/$10, made permanent on 11th August 2026 (Anthropic, as reported by explainx.ai), with cache reads assumed at Anthropic’s standard 0.1x of input. Worked examples computed by the author. ↩

8. Vals AI independent SWE-bench run, 1st September 2026, as mirrored by BenchLM (benchlm.ai/benchmarks/valsswebench). The same run shows Nvidia Nemotron 3 Ultra at 69.0% and OpenAI gpt-oss-120B at 33.6%. Vals removed SWE-bench Verified from its index as saturated on 4th May 2026. Terminal-Bench 2.1 (Opus 5 84.6%, GLM-5.3 71.5%) and SkillsBench (60.4%, 47.5%): Vals AI results as compiled by gradually.ai, “Claude Opus 5 vs GLM-5.3”, data to late August 2026. ↩

9. SiliconANGLE, “Subquadratic launches with $29M to bring 12M-token context windows to AI”, 5th May 2026. ↩

10. GLM-5.3 License (Z.ai; model-as-a-service businesses above $10bn revenue require a security review) and Kimi K3 License (Moonshot AI; model-as-a-service businesses above $20m trailing revenue require an agreement). huggingface.co/moonshotai/Kimi-K3 ↩

11. Guido Appenzeller (a16z), “Welcome to LLMflation: LLM inference cost is going down fast”, 12th November 2024, a16z.com/llmflation-llm-inference-cost (about 10x a year for equivalent MMLU performance); Epoch AI, “LLM inference prices have fallen rapidly but unequally across tasks”, March 2025, epoch.ai/blog/llm-inference-price-trends (9x to 900x a year depending on the benchmark threshold; median 50x). ↩

12. Apex36, “H100 GPU Prices Just Reset AI Feature Unit Economics”, July 2026, citing Spheron (on-demand H100 rates of roughly $2–2.50 an hour in June 2026, against a 2023–24 peak of about $8). ↩

13. CoreWeave, DDTL 4.0 announcement, 31st March 2026 (rated A3 by Moody’s; fixed tranche about 5.9%, floating SOFR + 2.25%); Forbes, “GPU Debt Has Gone Investment Grade. Here’s Who Holds The Risk”, 9th June 2026. ↩

14. Anthropic, statement on access to Claude Fable 5 and Claude Mythos 5 (access suspended 12th June 2026 to comply with US Department of Commerce export controls; restored 1st July after the controls were lifted on 30th June). anthropic.com/news/fable-mythos-access ↩