Best LLM for Browser Automation: The 2026 Mid-Year Benchmark

In April we benchmarked 11 frontier models on a hardened production test plan and promised that every new release would go through the same harness. Three releases later, here is the mid-year update: Claude Fable 5, Claude Opus 4.8, and GPT-5.5 join the sweep, and we added a second, much harder plan, because the routine plan stopped separating the field. One arrival missed the cutoff: OpenAI shipped GPT-5.6 on July 9, after these runs closed. Our launch read on Sol is in the Fable 5 vs GPT-5.6 head-to-head; it joins the harness in the next sweep.

Same discipline as before: same plans, same production surface, serial execution, per-run cleanup verification, one aggregator for billing, default configuration for every model, no prompt massaging. And the same caveat: this measures the model at the seam where it plugs into an agent loop, not our full production stack, which layers step classification, retries, healing, and evidence capture on top.

Sweep 1: the routine plan

The April plan, unchanged: navigate a settings surface, create a named resource, verify it, revoke it, verify it is gone. Toasts, confirmation dialogs, async state. The incumbents' April numbers sit alongside the new entrants:

Read that table cold and the conclusion is blunt: on a routine plan, the 2026 frontier buys you nothing. Five models pass 100% of runs. The cheapest of them costs 3 cents a run and the most expensive costs nearly four dollars, a 60x spread for the same green checkmark. Opus 4.8 inherits Opus 4.7's crown as the smart-tier speed leader, GPT-5.5 edges its predecessor as the value pick, and Fable 5's always-on thinking makes it the slowest and priciest way to click through a settings page.

If your test suite looks like this plan, model choice is a cost decision, full stop.

Sweep 2: the hard plan

New for this edition: a 30-plus step journey across multiple pages, with an iframe-embedded widget, a modal-inside-modal flow, optimistic UI that lies briefly about state, and a deliberately slow async step that tempts agents into acting early. This is the plan that looks like the E2E tests people actually struggle with.

Now the field splits, and the ordering matches the story the coding benchmarks told: the models separate exactly when the task gets long and ambiguous.

What jumps out

Fable 5's edge is real and it is specific. A 92% pass rate on a plan that breaks a quarter to nearly half of the other frontier runs is the long-horizon coherence showing up on schedule. Watching the traces, the difference is mundane: Fable 5 still knows what step 27 was for, waits out the lying spinner, and recovers from the modal stack instead of flailing. This is the failure mode that kills real E2E runs, and it is measurably better here.

Cost per completed run is the honest metric, and it complicates the sticker shock. Per raw run, Fable 5 costs 10x GPT-5.5 on the hard plan. Per completed run the gap narrows, and that still understates it: in a CI gate, a 66% pass rate on a correct application is not a cheaper test, it is a broken gate. Retries, reruns, and the human hour spent triaging a false failure belong on the cheap model's invoice too. On plans where a cheaper model holds 100%, none of this applies and the cheap model wins outright.

The silent fallback showed up in the data. A few of Fable 5's hard-plan runs finished with response patterns consistent with its documented classifier fallback to Opus 4.8 (the plan revokes a credential-like resource, which is exactly the security-adjacent shape we flagged in June). None of those runs failed, but the duration variance on those runs was wider. If you run security-adjacent suites on Fable 5, instrument the serving model; averages will hide this from you.

Variance still eats single-shot comparisons alive. Same model, same plan, same prompt: durations spread by 2x or more across repeats, on every model. Anyone quoting a single-run comparison of browser agents is measuring their sample size.

What this means for a production stack

The two sweeps together give the deployment answer that neither gives alone: route by difficulty. Routine steps and stable plans belong on the value tier (GPT-5.5, Gemini 3.1 Pro, or a light model when the plan is simple enough). The long, ambiguous, failure-prone journeys, and the diagnosis when something breaks, are where the Mythos-class rate card pays for itself. A production agent is a stack, not a model, and the stack's first job is knowing which lane each step belongs in.

That routing is exactly what Test-Lab does under the hood: the harness in this post is the same machinery that decides which model executes your test steps, so new releases flow from benchmark to product on data instead of launch-day vibes.

Caveats

A benchmark is a snapshot: two plans, one target surface, default configurations, mid-July 2026 pricing. A different plan or a tuned retry policy reshuffles rows. So does a release: GPT-5.6 landed after the runs closed and is queued for the next edition. Pull your own numbers before acting on ours, and if you want a specific model run against your plan, reach out and we will add it to the next sweep.

---

Describe a test in plain English and let the harness pick the right model for every step. Run your first test free.

Related reading:

• The April 2026 benchmark - 11 models, the methodology, and the variance argument in full
• Fable 5 vs Opus 4.8 - two weeks of real coding work with the two Anthropic tiers
• Fable 5 vs Gemini 3.1 Pro - the ceiling-versus-value decision in depth