Harnesses Composite Index

Test model
BudgetOpenClaw

v2026.11.1 · $1.00 / solved

38.7%pass rate
$0.39/ run
Best valuePi

v1.4.0 / v1.2.0 · $1.16 / solved

44.8%pass rate
$0.52/ run
Top scoreAnomaly

v2.2.6 · $1.18 / solved

48.0%pass rate
$0.57/ run
Coding-agent harness pass rate versus cost per runPass rate (%)Cost / run (log)30%35%40%45%50%55%$0.4$0.5$0.6$0.7$0.8$0.9$1
HarnessProviderPass rateTB 2.1TB 4.0DeepSWECost / runCost / solvedPass all 3
Anomaly48.0%60.9%N/A44.1%$0.57$1.1830.5%
Zhipu48.0%77.8%20.0%N/A$0.60$1.2630.3%
Moonshot47.0%65.3%17.9%43.6%$0.64$1.36N/A
SpaceXAI46.2%63.6%17.4%43.5%$0.68$1.4829.1%
Nous Research45.6%62.6%20.0%42.5%$0.65$1.4227.1%
DeepSeek45.5%61.1%17.4%43.0%$0.62$1.3729.7%
DeepSeek45.3%69.8%20.0%N/A$0.66$1.4527.8%
DeepSeek45.1%72.1%18.6%N/A$0.61$1.3528.2%
SWE-agent45.0%64.4%15.9%41.0%$0.67$1.4828.1%
Anthropic45.0%61.7%17.9%42.5%$0.76$1.6828.2%
Pi44.8%62.9%15.6%41.4%$0.52$1.1627.3%
Maka44.6%73.0%17.4%N/A$0.61$1.3626.3%
Oh My Pi44.1%72.7%16.8%N/A$0.52$1.1725.6%
OpenCode43.8%73.1%16.2%N/A$0.59$1.3426.1%
DeepSeek43.2%69.9%16.8%N/A$0.67$1.5625.4%
GitHub43.0%72.8%15.0%N/A$0.75$1.7527.8%
OpenAI42.9%64.8%18.2%36.6%$0.55$1.2825.1%
MiniMax42.7%72.7%14.7%N/A$0.59$1.3824.3%
Google42.6%69.9%15.9%N/A$0.82$1.9324.3%
Alma42.5%78.0%11.8%N/A$0.67$1.57N/A
Vercel Labs41.9%53.4%N/A38.7%$0.65$1.5524.5%
Cline41.0%75.7%10.6%N/A$0.64$1.55N/A
Exo39.3%64.7%13.5%N/A$0.64$1.6321.7%
Alibaba39.3%52.7%N/A35.1%$0.57$1.4423.0%
OpenClaw38.7%49.4%N/A36.0%$0.39$1.0022.2%