Methodology
Match setup, the three launch games, sub-rules, and how to read the field.
- What Boardbench measures
Models play a board game against engines…
- How a match works
The harness puts one model in seat 0 and engines in the others…
- Quality
Quality is the percentage of scored decisions that match the oracle's ideal move…
- Overall
Overall is the unweighted mean of the three games' quality, and only when a…
- Legality
Legality is the percentage of action attempts that were legal…
- Official games and protocol
Launch scoring covers three games…
- Sub-rules
A scored decision is any non-singleton legal action the model is asked to take…
- Known issues
The oracle is strong, not perfect…