Agentic LLM Benchmarks — July 2026

OpenAI dominates with GPT-5.6

The leaderboard was a lot more diverse just a week ago, with Sonnet, Opus and DeepSeek V4 present in the top 10. Today, however, flavours of GPT-5.5 and 5.6 take the top 9 places.

GPT-5.5 Pro is the best model on the leaderboard. It is also the slowest and the most expensive: a median pattern speed of about 1:39 per request, roughly 8x slower and 14x more expensive than plain GPT-5.5. For that price it buys about three extra points of final score, a top AI Code sub-score, and it was the only model in the entire table to hold the line in our hardest adversarial scenario (more on that in the Breach section).

GPT-5.5 sits in second place with a median speed of about 12 seconds - this is one of the best reasonable defaults. It is significantly cheaper and faster, at the cost of a few accuracy points.

Below it, the GPT-5.6 line comes as a fleet of tiers - Sol, Terra and Luna, each with a Pro variant. To sum things up:

• Pro variants (reasoning=pro) buy a couple of points each, at 2-4x the cost of their base tiers.
• GPT-5.6 Sol: strong scores at a fraction of GPT-5.5’s cost, ~4.5s median
• GPT-5.6 Terra: two more points down, but at half of Sol’s cost and ~3.6s median
• GPT-5.6 Luna: the budget tier, still scoring above 70.

Ultimately, the upper half of both Pareto charts is taken almost entirely by OpenAI. When picking trade-offs, you mostly pick between OpenAI models.

By the way, GPT-5.5 Instant (pointing to openai/gpt-chat-latest with automated routing) is also worth considering. We’ve heard good stories about it from adopters.

Reasoning effort is a Pareto variable

Reasoning effort is an inference parameter that you need to set explicitly. It moves models across the Pareto front - sometimes on the accuracy axis, sometimes on the reliability axis, sometimes on both. Test it in your own harness the same way you would test a model swap.

For example, the most dramatic case is gpt-oss-120b. Given the same weights and the same provider, we end up with two distinct rows. At high effort it scores respectably in the upper-middle of the table. At low effort it loses over twenty points of final score and picks up the worst Breach count in the run.

In exchange, the low-effort run on accelerated hardware is one of the fastest things in the entire benchmark - about 1.3 seconds median per request. That is a real trade some teams happily make for a preflight router with guardrails (especially in near-real-time agents), and a terrible choice for anything risk-averse.

All models are susceptible to security breaches

Speaking of risk. The modern world is a harsh and hostile place you need to plan for. That’s why we like to put a lot of pressure on agents in our simulations and benchmarks.

This benchmark embeds adversarial pressure of the kind that BitGN architectures actually faced. For example:

• instructions injected into retrieved content;
• hostile data planted in untrusted records;
• boundaries around records the agent may read but must not expose;
• plain lies.

The Breach column counts what got past each model.

First insight: all models are susceptible to security breaches. For example, in the scenarios where injected content tries to talk the agent into firing a state-changing runtime command, only one model in the whole table refused to pull the trigger: GPT-5.5 Pro. Every other model (frontier, open, cheap, expensive) executed the planted action.

Second insight: security resilience is not proportional to model price. One of the breach scenarios in BitGN is where hostile content inside an untrusted incoming record is engineered to be forwarded to the customer. Several of the top-ranked models happily repeated it into customer-facing output, while a handful of mid- and low-cost models (including some very modest ones) sanitized correctly.

Currently the only reliable way to catch breaches is by engineering deterministic gates for any information that crosses trust boundaries. Don’t rely on LLMs to magically do that for you.

And if you are wondering how Anthropic’s Fable 5 managed to score zero breaches in this benchmark - there is a catch.

Anthropic - behind the frontier in July 2026

Anthropic models have always been a special case. Their attention to design is amazing, but limited GPU capacity seems to force strange and unpredictable product decisions.

Still, they used to be quite high on this leaderboard, until the GPT-5.6 family arrived and pushed everything down.

Sonnet 5 in high reasoning mode is the best Anthropic model of this run, placing 10th - with an AI Code sub-score tied with GPT-5.5 Pro for the best in the entire table. The pattern repeats across the family: Anthropic models write excellent specialized tools for agents, and then bleed points on the strict contract behaviors around them - outcome mapping, grounding references, policy-based decisions.

And then there is Fable 5, which produced the strangest row of the run: zero breaches (the only zero in the column) and the worst Fatal count in the table. Fatal failures were caused by empty responses with a content-filter reason and random upstream failures.

Fable is a potentially very capable model, but its content filters cripple it, making it completely unusable.

Overall, Anthropic models huddle together in the center of the Pareto frontier charts. They are balanced, but not the most efficient. Let’s wait for the next generation of Anthropic models to catch up with the frontier!

Open Weights models on the frontier

We’ve been doing LLM benchmarks for more than 3 years. There were times when Open-Weights models were showing up in the charts, and times when closed-source frontier models were completely taking over.

Now the Open Weights models are coming back. Here are all of the OW models from this benchmark on the Pareto-front charts:

The most notable entries are:

• gpt-oss-120b (high) remains the open workhorse: solid score, tiny cost, and is served by accelerated AI vendors. This makes it one of the fastest capable options anywhere in the table.
• GLM 5.2 has a respectable score but ties for the worst Breach count in the run. Capable and gullible.
• DeepSeek V3.2 is at the cost frontier: roughly half the final score of the leaders at around one-thousandth of the cost. Suitable for high-volume, low-stakes batch classification.
• MiMo V2.5 is a new entrant from Xiaomi that is on the cost frontier.

Our picks for July 2026

Best default is GPT-5.5. Reach for GPT-5.5 Pro only if the extra three points are worth 14x the cost and 8x the latency.

GPT-5.6 Terra is a good starting point for interactive agents. It has a strong score, ~3.6s median request duration, and a budget-tier cost. Take Sol if you want a safety margin, and Luna if the budget is tight.

For near-real-time blocks, gpt-oss-120b (low) on accelerated hardware gives you ~1.3s median. Run it only behind deterministic gates - it also has the worst Breach count in the run.

For batch / high-volume workloads - GPT-5.6 Luna for quality-sensitive batch. DeepSeek V3.2 when cost dominates everything else (or you need to run within your security perimeter).

Other options for on-prem / open weights: DeepSeek V4 Pro for accuracy (mind the ~20s median - it’s a batch/nightly worker, not a chatbot), gpt-oss-120b (high) with AI accelerators for speed.

But no matter which model you pick, put a deterministic gate between untrusted content and privileged actions. No model at any price will do it for you.

What’s next?

This benchmark is designed not to saturate: its tasks are mined from where the best architectures break, and architectures keep finding new ways to break under the business workloads. The BitGN challenges continue to run, the trajectory dataset keeps growing, and the fault scenarios from the current ECOM season are already being distilled into the next wave of evals. Also stay tuned for ECOM2 this fall!

Coming up in the next report:

• More models
• New snapping points from the ongoing agentic commerce work - including deeper coverage of multi-step trajectories under accumulated errors
• More accelerated-hardware rows, as more capable models land on wafer-scale and LPU providers
• The next public challenge on BitGN - if you want your architecture’s survival (or failure) to shape the next round of these evals, that is the way in

Stay tuned!