On this page
Can an AI model find the cheapest mobile plan that fully fits a household's need when it has every operator's offer in front of it? We tested three models on 34 household questions in five countries, on 9 and 10 November 2026.
Results
| Model | Right | Misses |
|---|---|---|
| Claude Opus | 25 of 34 (74%) | all 9 fit but cost more (median +2.49 a month) |
| OpenAI Codex | 24 of 34 (71%) | all 10 fit but cost more (median +2.09 a month) |
| Claude Sonnet | 20 of 34 (59%) | all 14 fit but cost more (median +2.33 a month) |
Amounts in EUR or GBP, the market's own currency. Right = the named plan fits the need and costs at most 1 a month more than the cheapest plan that fits.
Without the pages
The same kind of task with no pages at all (Claude Sonnet, best guess from memory, 112 questions including simpler ones): no answer was verifiably right. In 93 of 112 the model's own stated cost was more than 1 a month above the cheapest plan that fits (median +4 a month), and 85 answers named a plan too vaguely to match any plan on sale (for example "Mobile Vikings small plan").
Method
- Markets: Belgium, the Netherlands, the United Kingdom, Ireland and Lithuania.
- A frozen market per country: one page per operator and per comparison or cashback site, listing every offer as it stood on 9 November 2026: prices, promotions, data, minutes, minimum terms, fees, eligibility, roaming terms and passes, and the small print read on the operator's page. The model reads these pages with file tools: no live web, and no Veritaset ranking shown.
- Questions: 34 household questions, each with a trip abroad plus one or two other conditions (several lines, a home-internet bundle, age, eSIM only, buying direct, a contract allowed). Generated from typical needs and kept only when the extra condition changes the right answer.
- Counting rules given to the model: average monthly cost over 12 months (promotions as they run, one-off fees spread, use beyond the bundle at the stated rates; longer terms averaged over the term; several lines in total); "unlimited" means no slow-down below 10 Mbit/s; an eligibility condition only if the customer meets it; deals via comparison or cashback sites at face value; EU roaming at the regulation's minimum when a plan states no amount.
- Answer key and scoring: Veritaset's published ranking method (mobile 0.24.0) on the same day's data.
- Models: Claude Opus, OpenAI Codex and Claude Sonnet, each run once through its standard command-line tool.
Limits
- One run per model; a small sample (34 questions); five markets.
- The pages are generated from Veritaset's verified records, not the operators' original web pages.
- Before scoring we checked failures against the key: one question whose stored need contradicted its text was left out, and an early run was discarded because roaming passes were missing from the pages (fixed and re-run).
- An earlier test on the live web, with ChatGPT, Gemini and Grok as people use them: Right Deal Index, October 2026 (35–48% right, 37–47% extra cost when the plan fitted).