All ten best-value models held their positions this week; combining cheaper models matched top quality on code work for a ninth of the cost.
Every week we test whether the best-value AI model for each kind of work is still the best value. A model is best value when nothing else beats it on quality, cost and speed at once. This week all ten held steady. Three newly released models were tested and none beat what was already there. The finding that matters: on code work, using cheaper models together matched the most expensive option for less than a ninth of its price.
Week 34 screened 325 newly released models and measured 3 of them against the current best-value picks. A frontier is the short list of models for one kind of work that nothing else beats on quality, cost and speed at once. No frontier moved. On code work, a combination of measured models scored 1.000 in quality, matching the best single model, and cost more than eight times less per thousand requests.
This week's frontiers
One row per kind of work. Routed pick is the model Potion currently sends that work to. Stored quality is its exam score when it was measured in full, give or take the margin. Canary is this week's small re-check: a few fresh tasks, scored the same way, to catch a model that has got worse. Held means the re-check landed inside the margin.
| kind of work | verdict | routed pick | stored quality | canary |
|---|---|---|---|---|
| agentic-tool-use | held | or-gemini-flash | 0.893 ± 0.095 | 0.900 n=4 |
| classification | held | name withheld | 0.976 ± 0.033 | 1.000 n=4 |
| code-gen | held | name withheld | 0.979 ± 0.020 | 1.000 n=4 |
| code-review | held | or-gpt-mini | 0.900 ± 0.081 | 1.000 n=4 |
| creative | held | or-sonnet | 0.907 ± 0.032 | 0.875 n=4 |
| extraction | held | name withheld | 0.960 ± 0.018 | 1.000 n=4 |
| multi-step-reasoning | held | or-deepseek-v4-flash-0731 | 0.980 ± 0.039 | 1.000 n=4 |
| rag-answer | held | name withheld | 0.920 ± 0.076 | 0.750 n=4 |
| rewrite-edit | held | or-gpt-mini | 0.872 ± 0.059 | 0.850 n=4 |
| summarization | held | or-ling-3.0-flash | 0.893 ± 0.063 | 0.925 n=4 |
A canary is a small weekly re-check of four requests to confirm a model has not collapsed. Every pick passed. The model called or-gemini-flash, used for reasoning work that involves calling tools, scored 0.900 this week against its stored score of 0.893, give or take 0.095. A margin of error is how sure we are: the true score sits inside that range nineteen times out of twenty. The model used for classification work scored 1.000 against 0.976, give or take 0.033. The model used for generating code scored 1.000 against 0.979, give or take 0.020.
Auditions
An audition is a newly released model's first exam, on the kind of work it looks suited to. It earns a place only by beating the model already doing that work on quality or price.
An audition is a first exam for a newly released model. Three models were measured this week. A model called or-ling-2-6-flash, built for work in many languages, was tested on rewriting and editing tasks and did not beat the current pick. Two models aimed at small, cheap deployments, or-l3-lunaris-8b and or-nex-n2-mini, were tested on classification work and neither beat the current pick.
- or-ling-2-6-flash · multilingual · rewrite-edit · did not beat the incumbent
- or-l3-lunaris-8b · small/cheap · classification · did not beat the incumbent
- or-nex-n2-mini · small/cheap · classification · did not beat the incumbent
Mixing
Mixing means using two or three cheaper models together in a particular way instead of one expensive one. We score combinations by replaying results we already have, so this costs nothing to explore. How a combination works is part of the product and is not published; what it achieves is.
A replay is re-scoring a combination from results we already have, without spending money on new requests. On code work, a combination of measured models was replayed over 90 requests and scored 1.000 in quality, matching the best single model. It cost more than eight times less per thousand requests. On structured output work, a combination scored 1.000 and cost between four and eight times less. On writing work, a combination scored 0.979 and cost between two and four times less.
- code work · cheaper-and-as-good · quality 1.000 · more than 8× cheaper · n=90
- structured output work · cheaper-and-as-good · quality 1.000 · between 4× and 8× cheaper · n=80
- writing work · cheaper-and-as-good · quality 0.979 · between 2× and 4× cheaper · n=14
- retrieval work · cheaper-and-as-good · quality 0.980 · between 2× and 4× cheaper · n=50
- reasoning work · cheaper-and-as-good · quality 0.993 · between 1.5× and 2× cheaper · n=14
- summarization · cheaper-and-as-good · quality 1.000 · 19% cheaper · n=14
- code work · frontier-candidate · quality 0.995 · under 1.5× cheaper · n=42
- writing work · frontier-candidate · quality 0.929 · under 1.5× cheaper · n=14
- extraction · frontier-candidate · quality 0.987 · -507% cheaper · n=74
What it means for you
If you pay for AI by the request, you pay for the gap between the best model and the cheapest one that passes your quality bar. This week that gap was measured again across ten kinds of work. It remains wide. Routing each request to the cheapest model that scored high enough is how you keep the quality and stop paying for the gap.
How the numbers are made
How these numbers are made, in plain words. We sort requests into kinds of work: sorting text into categories, pulling fields out of documents, writing code, answering from a set of documents, and so on. For each kind we keep a private exam of tasks the models have never seen. Every model sits the same exam. Code is marked by running it; other answers are marked against a reference answer. A model's quality is its average mark, and because an exam is a sample we also give a margin of error: two models whose margins overlap are called a tie. Cost is what a thousand requests would cost at the provider's public prices. A frontier is the short list of models that are the best deal at their level of quality, meaning nothing else is both better and cheaper. Each week we re-check every model on that list with a few fresh tasks to catch any that have got worse, and we give newly released models the exam for the work they look suited to.
- Every quality figure is a mean over a retrieval-hostile suite with a bootstrap 95% interval; two points whose intervals overlap are reported as tied.
- Code-generation and code-review quality is scored by executing the code; other clusters are scored by a rubric against a reference answer.
- A canary is a small weekly sample (four items) against the stored measurement; it detects collapse, not one-point movement.
- Combinations of models are replayed from stored item-level results; agreement between models is modelled conservatively, so the combination figures understate rather than overstate.
- Free-tier listings are excluded from auditions: their quality is not stable enough to measure.
Numbers
10 canaries · 10 held · 0 moved · 0 inconclusive · 40 items graded · 325 listings screened · 3 measured · $2.01 spent
Questions
- What is a routing frontier?
- The short list of models for one kind of work that nothing else beats on quality, cost and speed at the same time. A routing policy picks from it: cheapest above a quality floor, fastest above a quality floor, or best quality regardless of cost.
- Did any model get worse this week?
- No. All ten best-value picks reproduced their stored quality inside the margin of error on a four-request canary. A canary is a small weekly re-check that detects collapse, not small movements.
- Which models are the best value this week?
- or-gemini-flash for reasoning work that calls tools, scoring 0.893, give or take 0.095. or-gpt-mini for reviewing code, scoring 0.900, give or take 0.081. or-sonnet for creative writing, scoring 0.907, give or take 0.032. or-deepseek-v4-flash-0731 for multi-step reasoning, scoring 0.980, give or take 0.039. or-ling-3.0-flash for summarization, scoring 0.893, give or take 0.063. Some picks are not named; their scores are published.
Glossary. A frontier is the short list of models that are the best deal at their level of quality: nothing else is both better and cheaper. A floor is the lowest exam score you are willing to accept. A margin (or interval) is how far the true score could sit from the measured one, because an exam is a sample. A canary is a small weekly re-check. See the docs and the evidence.