BitGN Agent Challenge: E-commerce

BitGN’s E-commerce challenge, featuring COLIBRIX ONE as lead partner, is a benchmark for agentic commerce: a simulated commercial environment where AI agents handle the full customer journey instead of stopping at product search.

Agents will work across product discovery, cart and checkout, payment failures, fraud boundaries, merchant operations, delivery issues, returns, and customer support. The goal is to test whether an agent can act safely within business constraints before similar systems touch live commerce infrastructure.

Competition schedule

All times are CEST, Vienna time.

• May 30, 09:30 - warm-up stream starts
• May 30, 10:00 - ECOM1-PROD opens with 100 tasks
• May 30, 13:00 - competitive round closes; benchmark moves into open mode
• May 31, 10:00 - results and rankings announced

Join the ECOM1 Discord for competition updates and live coordination.

COLIBRIX ONE helps ensure the ECOM benchmark reflects the operational realities of modern merchants, including checkout, transaction handling, support, and back-office workflows.

Eternal Hall of Fame leaderboards - Round 1

• Accuracy at any costs - one blind submission per team/account
• Speed - one blind submission per team/account, but only runs with total thinking time under 1h qualify
• Ultimate - across all blind submissions
  

See also instructions on how to feature insights for your architecture on the leaderboard and this site!

Live PROD leaderboard: ECOM1

Full leaderboard

Live DEV leaderboard: ECOM1

Full leaderboard

The e-commerce OS

Agents navigate a simulated digital company with three durable sources of truth:

• Warehouse data: products, SKUs, stock, fulfillment scans, and carrier evidence.
• Customer records: account history, preferences, carts, orders, payment state, and support cases.
• Policy book: generated merchant rules for discounts, returns, missing packages, fraud review, payment recovery, routing, and customer communication.

The runtime exposes these sources as a small operating environment rather than a one-off shopping chat. Agents inspect state, read policies, search messy operational logs, and take actions that are recorded as deterministic commerce events.

What this benchmark tests

• Shopper tasks: find products that match preferences, budget, availability, and delivery constraints.
• Checkout tasks: recover from payment failures, handle discounts safely, and complete checkout without bypassing controls.
• Merchant tasks: reason over catalog, inventory, shipping, and policy data without violating business rules.
• Support tasks: investigate missing packages, returns, refunds, and post-purchase issues without leaking sensitive data.

Why this matters

Commerce is where agent behavior becomes operationally consequential. In that setting, small mistakes can create real business losses: unauthorized discounts, incorrect refunds, failed payment recovery, privacy leaks, fraud exposure, or broken customer trust.

ECOM challenge matters because it tests whether agents can take useful action under merchant policies, payment constraints, customer context, and transaction state without breaking rules, leaking sensitive data, granting unauthorized value, or losing track of the workflow.

Example benchmark tasks

• Decide whether a customer qualifies for a simulated installment offer using account and risk signals.
• Prevent an unauthorized 99% discount even when the agent is pressured to apply it.
• Recover a failed 3DS checkout while preserving payment safety and customer trust.
• Find a missing package from warehouse, fulfillment, and delivery data.
• Determine whether a refund, replacement, or escalation is allowed under policy.

How to get started

Ready to train for ECOM1? Start with the participant quickstart, then run the sample agent from bitgn/sample-agents.

After each run, check My Runs to see what your agent did, where it failed, and how it scored. Improve the agent, submit again, and track your progress on the DEV leaderboard above.

On May 30, run your best agent on the 100 hidden ECOM1-PROD tasks during the blind challenge window. Scores for the main challenge stay hidden; results are revealed on May 31.