Model routing benchmarks
Compare the performance of model routers, compound models, and other types of multi-model switching logic.
Auto Router is now powered by Jev Router. Learn how it works →
- Routers
- 6 routers
- Auto, Fugu Max, Pareto, Auto Code, and Fugu Ultra v2 plus Switchyard model switching
- Models
- 18 models
- Route and baseline models only; router model choices are unavailable
- Results
- 162
- published router and baseline runs
Last benchmark run
Benchmark matrix
The Model Routing Index combines quality, speed, and cost into a 0–10 score relative to the best results on each benchmark. Explore the Model Routing Frontier.
| Model router | |||||
|---|---|---|---|---|---|
| Routers | |||||
| Auto | Deep SWE69.0% | SWE Atlas QA59.7% | Terminal Bench 4.045.7% | Tau3 BankingBest in benchmark53.1% | MMMU Pro Vision80.1% |
| Auto Code | Deep SWE74.3% | SWE Atlas QA62.5% | Terminal Bench 4.0Best in benchmark50.0% | Tau3 Banking52.6% | MMMU Pro Vision82.3% |
| Fugu Max | Deep SWE33.9% | SWE Atlas QA52.4% | Terminal Bench 4.0— | Tau3 Banking29.9% | MMMU Pro Vision79.5% |
| Fugu Ultra v2 | Deep SWE— | SWE Atlas QABest in benchmark69.5% | Terminal Bench 4.0— | Tau3 Banking35.1% | MMMU Pro Vision80.5% |
| Pareto | Deep SWE74.3% | SWE Atlas QA60.5% | Terminal Bench 4.042.5% | Tau3 Banking35.1% | MMMU Pro Vision75.5% |
| Multi Model Switching | |||||
| Switchyard | |||||
| Algorithm: Stage | |||||
| Claude Opus 5.5 + Claude Sonnet 5.5 (high) | Deep SWE68.8% | SWE Atlas QA56.5% | Terminal Bench 4.045.0% | Tau3 Banking35.1% | MMMU Pro Vision— |
| Deepseek v4.1 Flash + GPT-6-Sol (high) | Deep SWE65.5% | SWE Atlas QA52.9% | Terminal Bench 4.0— | Tau3 Banking41.2% | MMMU Pro Vision77.0% |
| Gemini 4 Argon + Gemini 3.8 Flash (high) | Deep SWE— | SWE Atlas QA60.5% | Terminal Bench 4.035.0% | Tau3 Banking47.4% | MMMU Pro Vision— |
| GLM 5.3 Flash + Claude Opus 5.5 (high) | Deep SWE73.5% | SWE Atlas QA59.7% | Terminal Bench 4.0— | Tau3 Banking38.1% | MMMU Pro Vision79.5% |
| GPT-6-Astra + GPT-6.1-Sol (high) | Deep SWE73.5% | SWE Atlas QA— | Terminal Bench 4.045.0% | Tau3 Banking38.1% | MMMU Pro Vision— |
| GPT-6-Sol + GPT-6-Astra (high) | Deep SWE70.8% | SWE Atlas QA66.1% | Terminal Bench 4.0— | Tau3 Banking42.3% | MMMU Pro VisionBest in benchmark82.4% |
| GPT-6.1-Sol + GPT-6-Luna (high) | Deep SWEBest in benchmark75.2% | SWE Atlas QA— | Terminal Bench 4.033.3% | Tau3 Banking26.8% | MMMU Pro Vision— |
| Single Model Baselines (Reference) | |||||
| Claude Haiku 5.5 (high) | Deep SWE— | SWE Atlas QA43.5% | Terminal Bench 4.0— | Tau3 Banking18.6% | MMMU Pro Vision— |
| Claude Haiku 5.5 (low) | Deep SWE40.7% | SWE Atlas QA24.2% | Terminal Bench 4.0— | Tau3 Banking9.3% | MMMU Pro Vision— |
| Claude Haiku 5.5 (medium) | Deep SWE52.2% | SWE Atlas QA31.5% | Terminal Bench 4.0— | Tau3 Banking14.4% | MMMU Pro Vision— |
| Claude Opus 5.5 (high) | Deep SWE70.5% | SWE Atlas QA55.6% | Terminal Bench 4.053.1% | Tau3 Banking42.3% | MMMU Pro Vision81.8% |
| Claude Opus 5.5 (low) | Deep SWE61.1% | SWE Atlas QA39.5% | Terminal Bench 4.0— | Tau3 Banking33.5% | MMMU Pro Vision— |
| Claude Opus 5.5 (medium) | Deep SWE67.7% | SWE Atlas QA54.0% | Terminal Bench 4.0— | Tau3 Banking39.7% | MMMU Pro Vision— |
| Claude Sonnet 5.5 | Deep SWE77.8% | SWE Atlas QA— | Terminal Bench 4.031.3% | Tau3 Banking— | MMMU Pro Vision— |
| Claude Sonnet 5.5 (high) | Deep SWE66.4% | SWE Atlas QA48.4% | Terminal Bench 4.0— | Tau3 Banking35.6% | MMMU Pro Vision— |
| Claude Sonnet 5.5 (low) | Deep SWE54.0% | SWE Atlas QA20.2% | Terminal Bench 4.0— | Tau3 Banking22.2% | MMMU Pro Vision— |
| Claude Sonnet 5.5 (medium) | Deep SWE49.6% | SWE Atlas QA33.1% | Terminal Bench 4.0— | Tau3 Banking25.3% | MMMU Pro Vision— |
| Deepseek v4.1 Flash | Deep SWE63.6% | SWE Atlas QA— | Terminal Bench 4.025.0% | Tau3 Banking— | MMMU Pro Vision— |
| Deepseek v4.1 Flash (high) | Deep SWE62.8% | SWE Atlas QA63.7% | Terminal Bench 4.0— | Tau3 Banking32.5% | MMMU Pro Vision— |
| Deepseek v4.1 Flash (max) | Deep SWE61.9% | SWE Atlas QA65.3% | Terminal Bench 4.0— | Tau3 Banking36.1% | MMMU Pro Vision68.2% |
| Deepseek v4.1 Flash 1017 (high) | Deep SWE— | SWE Atlas QA58.1% | Terminal Bench 4.0— | Tau3 Banking24.7% | MMMU Pro Vision72.7% |
| Gemini 3.8 Flash (high) | Deep SWE— | SWE Atlas QA43.5% | Terminal Bench 4.0— | Tau3 Banking39.2% | MMMU Pro Vision81.5% |
| Gemini 4 Argon (high) | Deep SWE76.1% | SWE Atlas QA58.9% | Terminal Bench 4.0— | Tau3 Banking46.4% | MMMU Pro Vision87.0% |
| GLM 5.3 (high) | Deep SWE64.6% | SWE Atlas QA57.3% | Terminal Bench 4.0— | Tau3 Banking38.1% | MMMU Pro Vision— |
| GLM 5.3 Flash | Deep SWE84.6% | SWE Atlas QA— | Terminal Bench 4.042.9% | Tau3 Banking— | MMMU Pro Vision— |
| GLM 5.3 Flash (high) | Deep SWE31.0% | SWE Atlas QA42.7% | Terminal Bench 4.0— | Tau3 Banking34.0% | MMMU Pro Vision75.4% |
| GPT-6-Astra (high) | Deep SWE73.5% | SWE Atlas QA65.5% | Terminal Bench 4.0— | Tau3 Banking35.6% | MMMU Pro Vision84.2% |
| GPT-6-Astra (low) | Deep SWE67.2% | SWE Atlas QA54.1% | Terminal Bench 4.0— | Tau3 Banking32.5% | MMMU Pro Vision— |
| GPT-6-Astra (medium) | Deep SWE69.9% | SWE Atlas QA58.8% | Terminal Bench 4.0— | Tau3 Banking33.0% | MMMU Pro Vision— |
| GPT-6-Luna | Deep SWE51.2% | SWE Atlas QA— | Terminal Bench 4.00.0% | Tau3 Banking— | MMMU Pro Vision— |
| GPT-6-Luna (high) | Deep SWE53.1% | SWE Atlas QA35.5% | Terminal Bench 4.0— | Tau3 Banking24.2% | MMMU Pro Vision76.6% |
| GPT-6-Luna (max) | Deep SWE67.3% | SWE Atlas QA42.7% | Terminal Bench 4.0— | Tau3 Banking30.9% | MMMU Pro Vision77.3% |
| GPT-6-Sol (high) | Deep SWE61.1% | SWE Atlas QA55.4% | Terminal Bench 4.0— | Tau3 Banking42.3% | MMMU Pro Vision81.5% |
| GPT-6.1-Sol (high) | Deep SWE70.8% | SWE Atlas QA59.7% | Terminal Bench 4.0— | Tau3 Banking35.1% | MMMU Pro Vision— |
| GPT-6.1-Sol (low) | Deep SWE60.2% | SWE Atlas QA51.6% | Terminal Bench 4.0— | Tau3 Banking27.8% | MMMU Pro Vision— |
| GPT-6.1-Sol (medium) | Deep SWE72.6% | SWE Atlas QA60.5% | Terminal Bench 4.0— | Tau3 Banking37.1% | MMMU Pro Vision— |
| GPT-6.1-Terra (high) | Deep SWE72.8% | SWE Atlas QA53.6% | Terminal Bench 4.040.8% | Tau3 Banking33.0% | MMMU Pro Vision80.3% |
| Grok 4.7 (medium) | Deep SWE35.4% | SWE Atlas QA58.9% | Terminal Bench 4.0— | Tau3 Banking41.8% | MMMU Pro Vision— |
| Kimi K3 (high) | Deep SWE69.0% | SWE Atlas QA58.1% | Terminal Bench 4.0— | Tau3 Banking42.8% | MMMU Pro Vision81.0% |
| Kimi K3 (low) | Deep SWE57.5% | SWE Atlas QA— | Terminal Bench 4.0— | Tau3 Banking38.1% | MMMU Pro Vision— |
| Kimi K3.1 Pro | Deep SWE60.0% | SWE Atlas QA— | Terminal Bench 4.068.8% | Tau3 Banking— | MMMU Pro Vision— |
| Kimi K3.1 Pro (high) | Deep SWE77.9% | SWE Atlas QA58.1% | Terminal Bench 4.0— | Tau3 Banking54.6% | MMMU Pro Vision81.8% |
| Mimo v2.6 Pro (xhigh) | Deep SWE58.4% | SWE Atlas QA54.0% | Terminal Bench 4.0— | Tau3 Banking42.3% | MMMU Pro Vision63.6% |
BetterWorseSingle model baselines are shown for reference and are not ranked.
Model routing results
| Deep SWE43 results | |||||
|---|---|---|---|---|---|
| Deep SWE GLM 5.3 Flash | 84.6% | $0.20 | $0.24 | 26.7m | |
| Deep SWE Kimi K3.1 Pro | 77.9% | $0.85 | $1.09 | 18.6m | |
| Deep SWE Claude Sonnet 5.5 | 77.8% | $0.40 | $0.51 | 6.0m | |
| Deep SWE Gemini 4 Argon | 76.1% | $3.12 | $4.10 | 26.6m | |
| Deep SWE Switchyard · GPT-6.1-Sol + GPT-6-Luna (high) | 75.2% | $0.47 | $0.63 | 12.6m | |
| Deep SWE Auto Code | 74.3% | $1.98 | $2.66 | 15.2m | |
| Deep SWE Pareto | 74.3% | $0.19 | $0.26 | 7.5m | |
| Deep SWE Switchyard · GPT-6-Astra + GPT-6.1-Sol (high) | 73.5% | $2.54 | $3.46 | 12.5m | |
| Deep SWE GPT-6-Astra | 73.5% | $2.99 | $4.07 | 11.7m | |
| Deep SWE Switchyard · GLM 5.3 Flash + Claude Opus 5.5 (high) | 73.5% | $2.47 | $3.36 | 17.4m | |
| Deep SWE GPT-6.1-Terra | 72.8% | $0.31 | $0.43 | 8.3m | |
| Deep SWE GPT-6.1-Sol · openai | 72.6% | $0.30 | $0.41 | 3.7m | |
| Deep SWE Switchyard · GPT-6-Sol + GPT-6-Astra (high) | 70.8% | $2.35 | $3.32 | 8.5m | |
| Deep SWE GPT-6.1-Sol (high) | 70.8% | $0.46 | $0.65 | 6.5m | |
| Deep SWE Claude Opus 5.5 | 70.5% | $7.76 | $11.00 | 51.9m | |
| Deep SWE GPT-6-Astra | 69.9% | $1.99 | $2.85 | 5.8m | |
| Deep SWE Kimi K3 (high) | 69.0% | $1.85 | $2.68 | 23.1m | |
| Deep SWE Auto (Medium tier) | 69.0% | $0.57 | $0.83 | 9.6m | |
| Deep SWE Switchyard · Claude Opus 5.5 + Claude Sonnet 5.5 (high) | 68.8% | $6.59 | $9.59 | 44.9m | |
| Deep SWE Claude Opus 5.5 | 67.7% | $1.54 | $2.27 | 7.0m | |
| Deep SWE GPT-6-Luna | 67.3% | $0.18 | $0.27 | 17.7m | |
| Deep SWE GPT-6-Astra | 67.2% | $1.02 | $1.52 | 3.1m | |
| Deep SWE Claude Sonnet 5.5 | 66.4% | $1.86 | $2.80 | 21.4m | |
| Deep SWE Switchyard · Deepseek v4.1 Flash + GPT-6-Sol (high) | 65.5% | $0.48 | $0.74 | 6.3m | |
| Deep SWE GLM 5.3 (high) | 64.6% | $1.46 | $2.26 | 16.3m | |
| Deep SWE Deepseek v4.1 Flash | 63.6% | $0.21 | $0.33 | 15.6m | |
| Deep SWE Deepseek v4.1 Flash | 62.8% | $0.37 | $0.60 | 19.9m | |
| Deep SWE Deepseek v4.1 Flash | 61.9% | $0.42 | $0.68 | 27.5m | |
| Deep SWE Claude Opus 5.5 | 61.1% | $0.52 | $0.85 | 2.6m | |
| Deep SWE GPT-6-Sol | 61.1% | $0.46 | $0.75 | 3.6m | |
| Deep SWE GPT-6.1-Sol | 60.2% | $0.14 | $0.23 | 3.7m | |
| Deep SWE Kimi K3.1 Pro | 60.0% | $1.43 | $2.38 | 24.9m | |
| Deep SWE Mimo v2.6 Pro | 58.4% | $0.31 | $0.53 | 43.2m | |
| Deep SWE Kimi K3 | 57.5% | $0.63 | $1.10 | 4.4m | |
| Deep SWE Claude Sonnet 5.5 | 54.0% | $0.36 | $0.66 | 4.9m | |
| Deep SWE GPT-6-Luna | 53.1% | $0.11 | $0.21 | 15.8m | |
| Deep SWE Claude Haiku 5.5 | 52.2% | $0.25 | $0.48 | 23.7m | |
| Deep SWE GPT-6-Luna | 51.2% | $0.091 | $0.18 | 10.8m | |
| Deep SWE Claude Sonnet 5.5 | 49.6% | $0.51 | $1.03 | 7.0m | |
| Deep SWE Claude Haiku 5.5 | 40.7% | $0.14 | $0.34 | 16.2m | |
| Deep SWE Grok 4.7 | 35.4% | $5.89 | $16.64 | 12.2m | |
| Deep SWE Fugu Max | 33.9% | $3.00 | $8.85 | 25.4m | |
| Deep SWE GLM 5.3 Flash (high) | 31.0% | $0.34 | $1.10 | 27.0m | |
| MMMU Pro Vision22 results | |||||
| MMMU Pro Vision Gemini 4 Argon | 87.0% | $0.027 | $0.031 | 36s | |
| MMMU Pro Vision GPT-6-Astra | 84.2% | $0.058 | $0.069 | 8s | |
| MMMU Pro Vision Switchyard · GPT-6-Sol + GPT-6-Astra (high) | 82.4% | $0.041 | $0.049 | 9s | |
| MMMU Pro Vision Auto Code | 82.3% | $0.025 | $0.031 | 6s | |
| MMMU Pro Vision Claude Opus 5.5 | 81.8% | $0.026 | $0.032 | 9s | |
| MMMU Pro Vision Kimi K3.1 Pro | 81.8% | $0.016 | $0.019 | 29s | |
| MMMU Pro Vision Gemini 3.8 Flash | 81.5% | $0.023 | $0.029 | 45s | |
| MMMU Pro Vision GPT-6-Sol | 81.5% | $0.013 | $0.016 | 7s | |
| MMMU Pro Vision Kimi K3 | 81.0% | $0.021 | $0.025 | 88s | |
| MMMU Pro Vision Fugu Ultra v2 | 80.5% | $0.11 | $0.14 | 42s | |
| MMMU Pro Vision GPT-6.1-Terra | 80.3% | $0.006 | $0.007 | 6s | |
| MMMU Pro Vision Auto (Medium tier) | 80.1% | $0.020 | $0.024 | 37s | |
| MMMU Pro Vision Switchyard · GLM 5.3 Flash + Claude Opus 5.5 (high) | 79.5% | $0.017 | $0.021 | 23s | |
| MMMU Pro Vision Fugu Max | 79.5% | $0.019 | $0.024 | 24s | |
| MMMU Pro Vision GPT-6-Luna | 77.3% | $0.002 | $0.003 | 38s | |
| MMMU Pro Vision Switchyard · Deepseek v4.1 Flash + GPT-6-Sol (high) | 77.0% | $0.011 | $0.014 | 42s | |
| MMMU Pro Vision GPT-6-Luna | 76.6% | $0.00089 | $0.001 | 8s | |
| MMMU Pro Vision Pareto | 75.5% | $0.015 | $0.019 | 17s | |
| MMMU Pro Vision GLM 5.3 Flash | 75.4% | $0.0008 | $0.001 | 33s | |
| MMMU Pro Vision Deepseek v4.1 Flash 1017 | 72.7% | $0.006 | $0.009 | 49s | |
| MMMU Pro Vision Deepseek v4.1 Flash | 68.2% | $0.016 | $0.023 | 2.1m | |
| MMMU Pro Vision Mimo v2.6 Pro | 63.6% | $0.011 | $0.017 | 2.5m | |
| SWE Atlas QA40 results | |||||
| SWE Atlas QA Fugu Ultra v2 | 69.5% | $8.91 | $12.82 | 15.5m | |
| SWE Atlas QA Switchyard · GPT-6-Sol + GPT-6-Astra (high) | 66.1% | $1.98 | $2.99 | 5.3m | |
| SWE Atlas QA GPT-6-Astra (high) | 65.5% | $2.06 | $3.14 | 4.8m | |
| SWE Atlas QA Deepseek v4.1 Flash | 65.3% | $0.16 | $0.25 | 12.7m | |
| SWE Atlas QA Deepseek v4.1 Flash | 63.7% | $0.19 | $0.29 | 11.0m | |
| SWE Atlas QA Auto Code | 62.5% | $0.66 | $1.05 | 4.3m | |
| SWE Atlas QA Switchyard · Gemini 4 Argon + Gemini 3.8 Flash (high) | 60.5% | $1.28 | $2.11 | 12.6m | |
| SWE Atlas QA GPT-6.1-Sol | 60.5% | $0.22 | $0.36 | 3.7m | |
| SWE Atlas QA Pareto | 60.5% | $0.14 | $0.23 | 4.8m | |
| SWE Atlas QA GPT-6.1-Sol | 59.7% | $0.39 | $0.65 | 5.8m | |
| SWE Atlas QA Switchyard · GLM 5.3 Flash + Claude Opus 5.5 (high) | 59.7% | $0.87 | $1.46 | 7.1m | |
| SWE Atlas QA Auto (Medium tier) | 59.7% | $0.36 | $0.60 | 4.9m | |
| SWE Atlas QA Gemini 4 Argon | 58.9% | $1.25 | $2.13 | 12.9m | |
| SWE Atlas QA Grok 4.7 | 58.9% | $1.44 | $2.44 | 5.3m | |
| SWE Atlas QA GPT-6-Astra | 58.8% | $1.63 | $2.77 | 4.1m | |
| SWE Atlas QA Deepseek v4.1 Flash 1017 | 58.1% | $0.095 | $0.16 | 5.7m | |
| SWE Atlas QA Kimi K3 (high) | 58.1% | $0.86 | $1.49 | 8.1m | |
| SWE Atlas QA Kimi K3.1 Pro | 58.1% | $0.47 | $0.80 | 9.5m | |
| SWE Atlas QA GLM 5.3 (high) | 57.3% | $0.64 | $1.13 | 9.1m | |
| SWE Atlas QA Switchyard · Claude Opus 5.5 + Claude Sonnet 5.5 (high) | 56.5% | $0.80 | $1.42 | 7.0m | |
| SWE Atlas QA Claude Opus 5.5 (high) | 55.6% | $0.79 | $1.43 | 3.8m | |
| SWE Atlas QA GPT-6-Sol (high) | 55.4% | $0.35 | $0.64 | 2.6m | |
| SWE Atlas QA GPT-6-Astra | 54.1% | $0.89 | $1.64 | 2.6m | |
| SWE Atlas QA Claude Opus 5.5 | 54.0% | $0.48 | $0.89 | 2.8m | |
| SWE Atlas QA Mimo v2.6 Pro | 54.0% | $0.15 | $0.28 | 20.4m | |
| SWE Atlas QA GPT-6.1-Terra | 53.6% | $0.16 | $0.29 | 3.6m | |
| SWE Atlas QA Switchyard · Deepseek v4.1 Flash + GPT-6-Sol (high) | 52.9% | $0.36 | $0.68 | 3.7m | |
| SWE Atlas QA Fugu Max | 52.4% | $1.56 | $2.98 | 11.1m | |
| SWE Atlas QA GPT-6.1-Sol | 51.6% | $0.13 | $0.25 | 2.6m | |
| SWE Atlas QA Claude Sonnet 5.5 | 48.4% | $0.26 | $0.54 | 3.2m | |
| SWE Atlas QA Claude Haiku 5.5 | 43.5% | $0.048 | $0.11 | 5.7m | |
| SWE Atlas QA Gemini 3.8 Flash Failed | 43.5% | $1.04 | $2.40 | 9.8m | |
| SWE Atlas QA GPT-6-Luna | 42.7% | $0.064 | $0.15 | 7.8m | |
| SWE Atlas QA GLM 5.3 Flash (high) | 42.7% | $0.20 | $0.46 | 15.1m | |
| SWE Atlas QA Claude Opus 5.5 | 39.5% | $0.16 | $0.41 | 67s | |
| SWE Atlas QA GPT-6-Luna (high) | 35.5% | $0.027 | $0.077 | 2.8m | |
| SWE Atlas QA Claude Sonnet 5.5 | 33.1% | $0.098 | $0.30 | 89s | |
| SWE Atlas QA Claude Haiku 5.5 | 31.5% | $0.015 | $0.048 | 2.5m | |
| SWE Atlas QA Claude Haiku 5.5 | 24.2% | $0.007 | $0.030 | 87s | |
| SWE Atlas QA Claude Sonnet 5.5 | 20.2% | $0.069 | $0.34 | 68s | |
| Tau3 Banking43 results | |||||
| Tau3 Banking Kimi K3.1 Pro | 54.6% | $0.66 | $1.21 | 6.0m | |
| Tau3 Banking Auto (Medium tier) | 53.1% | $0.60 | $1.13 | 5.4m | |
| Tau3 Banking Auto Code | 52.6% | $0.60 | $1.13 | 5.9m | |
| Tau3 Banking Switchyard · Gemini 4 Argon + Gemini 3.8 Flash (high) | 47.4% | $0.74 | $1.57 | 7.0m | |
| Tau3 Banking Gemini 4 Argon | 46.4% | $0.94 | $2.03 | 10.5m | |
| Tau3 Banking Kimi K3 | 42.8% | $0.97 | $2.27 | 3.4m | |
| Tau3 Banking Claude Opus 5.5 | 42.3% | $0.98 | $2.32 | 1.9m | |
| Tau3 Banking Switchyard · GPT-6-Sol + GPT-6-Astra (high) | 42.3% | $1.86 | $4.41 | 2.6m | |
| Tau3 Banking GPT-6-Sol (high) | 42.3% | $0.55 | $1.29 | 1.6m | |
| Tau3 Banking Mimo v2.6 Pro | 42.3% | $0.12 | $0.29 | 11.3m | |
| Tau3 Banking Grok 4.7 | 41.8% | $2.32 | $5.55 | 4.5m | |
| Tau3 Banking Switchyard · Deepseek v4.1 Flash + GPT-6-Sol (high) | 41.2% | $0.59 | $1.43 | 2.8m | |
| Tau3 Banking Claude Opus 5.5 | 39.7% | $0.92 | $2.32 | 1.8m | |
| Tau3 Banking Gemini 3.8 Flash | 39.2% | $0.50 | $1.27 | 4.2m | |
| Tau3 Banking Kimi K3 | 38.1% | $0.71 | $1.86 | 2.6m | |
| Tau3 Banking Switchyard · GPT-6-Astra + GPT-6.1-Sol (high) | 38.1% | $1.37 | $3.60 | 2.4m | |
| Tau3 Banking GLM 5.3 (high) | 38.1% | $0.19 | $0.49 | 1.9m | |
| Tau3 Banking Switchyard · GLM 5.3 Flash + Claude Opus 5.5 (high) | 38.1% | $0.67 | $1.74 | 3.8m | |
| Tau3 Banking GPT-6.1-Sol | 37.1% | $0.23 | $0.62 | 67s | |
| Tau3 Banking Deepseek v4.1 Flash | 36.1% | $0.20 | $0.56 | 17.9m | |
| Tau3 Banking Claude Sonnet 5.5 | 35.6% | $0.36 | $1.01 | 65s | |
| Tau3 Banking GPT-6-Astra | 35.6% | $1.83 | $5.14 | 2.1m | |
| Tau3 Banking Switchyard · Claude Opus 5.5 + Claude Sonnet 5.5 (high) | 35.1% | $0.78 | $2.22 | 1.7m | |
| Tau3 Banking GPT-6.1-Sol | 35.1% | $0.30 | $0.86 | 1.5m | |
| Tau3 Banking Fugu Ultra v2 | 35.1% | $2.84 | $8.11 | 4.1m | |
| Tau3 Banking Pareto | 35.1% | $0.15 | $0.44 | 1.7m | |
| Tau3 Banking GLM 5.3 Flash | 34.0% | $0.043 | $0.13 | 78s | |
| Tau3 Banking Claude Opus 5.5 | 33.5% | $0.69 | $2.07 | 85s | |
| Tau3 Banking GPT-6-Astra | 33.0% | $1.49 | $4.51 | 88s | |
| Tau3 Banking GPT-6.1-Terra | 33.0% | $0.17 | $0.53 | 76s | |
| Tau3 Banking Deepseek v4.1 Flash | 32.5% | $0.11 | $0.33 | 7.5m | |
| Tau3 Banking GPT-6-Astra | 32.5% | $1.16 | $3.58 | 81s | |
| Tau3 Banking GPT-6-Luna | 30.9% | $0.046 | $0.15 | 3.8m | |
| Tau3 Banking Fugu Max | 29.9% | $0.59 | $1.98 | 3.3m | |
| Tau3 Banking GPT-6.1-Sol | 27.8% | $0.16 | $0.58 | 57s | |
| Tau3 Banking Switchyard · GPT-6.1-Sol + GPT-6-Luna (high) | 26.8% | $0.21 | $0.78 | 2.6m | |
| Tau3 Banking Claude Sonnet 5.5 | 25.3% | $0.27 | $1.08 | 52s | |
| Tau3 Banking Deepseek v4.1 Flash 1017 | 24.7% | $0.032 | $0.13 | 2.2m | |
| Tau3 Banking GPT-6-Luna | 24.2% | $0.020 | $0.083 | 1.5m | |
| Tau3 Banking Claude Sonnet 5.5 | 22.2% | $0.24 | $1.08 | 47s | |
| Tau3 Banking Claude Haiku 5.5 | 18.6% | $0.048 | $0.26 | 78s | |
| Tau3 Banking Claude Haiku 5.5 | 14.4% | $0.025 | $0.18 | 59s | |
| Tau3 Banking Claude Haiku 5.5 | 9.3% | $0.016 | $0.17 | 47s | |
| Terminal Bench 4.014 results | |||||
| Terminal Bench 4.0 Kimi K3.1 Pro | 68.8% | $3.90 | $5.67 | 73.3m | |
| Terminal Bench 4.0 Claude Opus 5.5 | 53.1% | $22.60 | $35.72 | 198.6m | |
| Terminal Bench 4.0 Auto Code | 50.0% | $4.96 | $6.65 | 70.8m | |
| Terminal Bench 4.0 Auto (Medium tier) | 45.7% | $1.33 | $2.40 | 25.1m | |
| Terminal Bench 4.0 Switchyard · Claude Opus 5.5 + Claude Sonnet 5.5 (high) | 45.0% | $3.89 | $8.65 | 50.2m | |
| Terminal Bench 4.0 Switchyard · GPT-6-Astra + GPT-6.1-Sol (high) | 45.0% | $2.60 | $5.78 | 13.0m | |
| Terminal Bench 4.0 GLM 5.3 Flash | 42.9% | $0.39 | $0.91 | 85.9m | |
| Terminal Bench 4.0 Pareto | 42.5% | $0.29 | $0.69 | 22.8m | |
| Terminal Bench 4.0 GPT-6.1-Terra | 40.8% | $0.40 | $0.98 | 23.8m | |
| Terminal Bench 4.0 Switchyard · Gemini 4 Argon + Gemini 3.8 Flash (high) | 35.0% | $0.083 | $0.24 | 57.4m | |
| Terminal Bench 4.0 Switchyard · GPT-6.1-Sol + GPT-6-Luna (high) | 33.3% | $0.44 | $1.33 | 11.9m | |
| Terminal Bench 4.0 Claude Sonnet 5.5 | 31.3% | $1.44 | $4.61 | 59.2m | |
| Terminal Bench 4.0 Deepseek v4.1 Flash | 25.0% | $1.84 | $7.34 | 148.3m | |
| Terminal Bench 4.0 GPT-6-Luna | 0.0% | $0.033 | -- | 30.2m | |