A modern AI agent is a pipeline of blocks running in a loop - a router, a policy check, a tool writer, a verifier. It is only as strong as its weakest block, because loops accumulate errors: a bad classification at step 3 is a wrong refund at step 12.
This benchmark measures which LLMs hold at the points where good agentic architectures break. The evals are extracted from the failures of hundreds of real, competing architectures on the BitGN platform, replayed against modern models, and graded deterministically. No LLM-as-a-judge.
We publish a new report every month with a focus on Agentic Commerce. EU and US companies use these reports to pick models for AI agents that have to ship: quickly, cheaply, and securely.
OpenAI takes the top nine places. Reasoning effort reshapes the frontier, open weights offer practical trade-offs, and every model remains vulnerable to attacks.
July 2026: final score versus workload cost.July 2026: final score versus pattern speed.
Higher is more accurate; further right is cheaper or faster. Teal points trace the Pareto frontier. How to read these charts →
