★VALS INDEX
40.1%
+/- 28.5
Accuracy
⏱VALS INDEX
37.1s
Latency (median)
$VALS INDEX
$0.2113
Cost / Test (1P-est)
SCORE STATUSscored
TEST SETS7
RECORDED TRIALS125
CUSTOM PASS48%
Industry benchmarks
Accuracyattempts · responded
GPQA_DIAMOND
GPQA Diamond
20.0%
50 attempts · 50 responded
IFEVAL
IFEval
60.3%
50 attempts · 50 responded
Custom 9-task suite
Mean /5n=trials
CUSTOM
json_strict
★
5.00
5 trials
CUSTOM
summarize
★
4.80
5 trials
CUSTOM
code_debug
-
3.00
5 trials
CUSTOM
reasoning_multistep
★
1.00
5 trials
CUSTOM
code_gen_long
-
0.00
5 trials
Per-task distribution
Score
All trials on this page were run through the harness. Industry suites use deterministic family-specific reference graders (no LLM judge). The custom suite is graded by consensus of minimax-m3 + claude-fable-5. See /industry/ for the full benchmark suite.