BoardBench

Scotland Yard

The model is Mr. X. Three strong engine detectives hunt. Hidden movement, periodic reveals, ticket economy. Only Mr. X is quality-scored. The headline is how often Mr. X’s choice matches the oracle, not whether the game ended in escape. See Methodology.

  1. 100.0$2.07Claude Fable 5Medium
  2. 93.8$0.08Claude Sonnet 5Medium
  3. 86.7$0.01GPT-5.6 LunaHigh
  4. 85.3$0.06Grok 4.6Medium
  5. 85.0$0.18GPT-5.6 SolMedium
  6. 84.8$0.02Grok 4.5Medium
  7. 82.8$0.00DeepSeek V4 Flash 0731High
  8. 81.5$0.03Kimi K3Medium
  9. 66.7$0.21Claude Opus 5Medium
  10. 66.7$0.01Grok 4.5High
  11. 66.7$0.01Grok 4.5Low
  12. 60.0$0.02Grok 4.6High

Quality is exact ideal %, 0–100. Protocol boardbench-0.1.

Quality vs cost

405060708090100$0.001$0.01$0.1$1$10Claude Fable 5 Medium · 100.0% · $2.07Claude Opus 5 Medium · 66.7% · $0.21Claude Sonnet 5 Medium · 93.8% · $0.08DeepSeek V4 Flash 0731 High · 82.8% · $0.00GPT-5.6 Luna High · 86.7% · $0.01GPT-5.6 Sol Medium · 85.0% · $0.18Grok 4.5 High · 66.7% · $0.01Grok 4.5 Medium · 84.8% · $0.02Grok 4.5 Low · 66.7% · $0.01Grok 4.6 High · 60.0% · $0.02Grok 4.6 Medium · 85.3% · $0.06Kimi K3 Medium · 81.5% · $0.03Claude Fable 5 MediumClaude Sonnet 5 MediumGPT-5.6 Luna HighGrok 4.6 MediumGPT-5.6 Sol MediumGrok 4.5 MediumDeepSeek V4 Flash 0731 HighKimi K3 MediumGrok 4.5 HighGrok 4.5 LowClaude Opus 5 MediumGrok 4.6 High

Horizontal axis is logarithmic. Lines join the same model at different efforts.

ModelQualityLegalityCostn
Fable 5Medium100.0100.0$2.071 × 26
Sonnet 5Medium93.8100.0$0.083 × 32
LunaHigh86.7100.0$0.012 × 15
Grok 4.6Medium85.3100.0$0.062 × 34
SolMedium85.0100.0$0.182 × 20
Grok 4.5Medium84.8100.0$0.023 × 33
DeepSeek V4 Flash 0731High82.8100.0$0.004 × 29
Kimi K3Medium81.5100.0$0.034 × 27
Opus 5Medium66.7100.0$0.212 × 9
Grok 4.5High66.7100.0$0.011 × 3
Grok 4.5Low66.7100.0$0.011 × 3
Grok 4.6High60.0100.0$0.022 × 10

Headline is exact ideal %. n is quality matches × scored decisions.

Look