Scrabble
A board, the mover’s rack and the unseen tiles, ENABLE lexicon. Simulation scores every legal play by equity, not just points; the answer is a square and a word. See Methodology.
Quality is the mean oracle score of the moves played, × 100. The best move scores 100; an illegal move or no move scores 0. Protocol boardbench-0.2.
Quality vs cost
6 of 6 models
- Anthropic
- OpenAI
- DeepSeek
- Z.ai
- Pareto frontier: nothing plotted is both cheaper and better
- Anthropic
- OpenAI
- DeepSeek
- Z.ai
Horizontal axis is logarithmic.
Table
| Model | Quality | Legality | Cost | n |
|---|---|---|---|---|
| GLM-5.3 FlashHigh | 56.0 | 100.0 | $0.02 | 3 |
| SolHigh | 53.9 | 66.7 | $0.15 | 3 |
| LunaHigh | 36.1 | 66.7 | $0.01 | 3 |
| Haiku 5.5High | 32.2 | 66.7 | $0.01 | 3 |
| DeepSeek V4.1 FlashHigh | 22.1 | 33.3 | $0.06 | 3 |
| Opus 5.5High | 14.5 | 33.3 | $0.23 | 3 |
| Muse Spark 1.3High | 0.0 | 0.0 | — | 3 |
Headline is the mean move score × 100. n is answers: puzzles × samples.