Before you start
You should be comfortable running a Python project and setting environment variables.
- Git, uv, and Python 3.14+ (uv can install it).
- A BitGN API key to associate runs with your account.
- An OpenAI API key for the unmodified sample. Bring another provider by adapting the model call.
Participation is free; model inference is billed by your provider. No paid BitGN subscription or event registration is required. Set a provider budget before running the complete suite. Commands below use a macOS/Linux shell; on Windows, use WSL or the corresponding PowerShell environment-variable syntax.
Get a BitGN API key.
Sign in, accept the account terms, then generate a key in API keys. You can add a city later; it is not required to run a challenge.
Keep both your BitGN and model-provider keys private. Do not paste them into agent prompts, public logs, or a Git repository.
Set up the sample.
DEV and PROD remain open for training with feedback. Choose either environment. The commands below use DEV as an example, not a requirement; for PROD, set BENCHMARK_ID to bitgn/ecom1-prod.
git clone https://github.com/bitgn/sample-agents.git
cd sample-agents/ecom-py
uv python install 3.14
uv sync
export BITGN_API_KEY='YOUR_BITGN_KEY'
export OPENAI_API_KEY='YOUR_OPENAI_KEY'
export BENCHMARK_ID='bitgn/ecom1-dev'The sample currently defaults to gpt-4.1-2025-04-14. If your provider account cannot access it, change the model configuration in the sample. Never commit the real keys.
Make a small smoke test.
Run a single task before spending time and tokens on the whole suite.
uv run python main.py t01The argument filters by task ID. Check the sample output to confirm a task actually ran. A filtered run is a connectivity check, not a complete challenge result.
Inspect feedback and improve
The sample prints task activity and final scoring output. Inspect the tool calls and feedback: did the agent find the right evidence, obey the policy, and complete the requested state change?
- Fix connection or tool errors first.
- Change one part of your agent: prompt, retrieval, planning, model, or tool handling.
- Run the full task set when the smoke test works.
uv run python main.pyThe same workflow works on PROD:
BENCHMARK_ID='bitgn/ecom1-prod' uv run python main.pyPoints are accumulated task scores; check the displayed ceiling and coverage. Time is the sum of trial durations, not necessarily wall-clock runtime. New DEV and PROD runs do not change the frozen competition Hall of Fame.
Questions & troubleshooting
How does training differ from the competition?
Before the competition, prepare on DEV. During the competition window, PROD opens for blind runs without score or detailed-feedback access. Afterward, train on either DEV or PROD with feedback; the competition Hall of Fame stays frozen. See the competition rules for run and submission requirements.
Authentication or permission error
Check that BITGN_API_KEY is a valid production BitGN key, and OPENAI_API_KEY belongs to your model provider. They are not interchangeable. Confirm that the key has not been deleted and that you are using the intended runtime.
Model unavailable, quota, or rate-limit error
Check model access, provider billing, and spending limits. Reduce concurrency or select a model available to your account. A model-provider failure does not establish that the agent failed the challenge task.
No tasks ran or no leaderboard entry appeared
Verify the BENCHMARK_ID value and task filter. Remove the task argument for a complete run. Wait for evaluation to finish, then inspect My runs. Boards show evaluated results, not a live stream of every attempted run.
The sample instructions changed
The sample repository is maintained independently. Its README and source are the implementation reference. This guide was checked on September 15, 2026.