★VALS INDEX
40.2%
+/- 33.1
Accuracy
⏱VALS INDEX
60.5s
Latency (median)
$VALS INDEX
$0.2823
Cost / Test (1P-est)
SCORE STATUSscored
TEST SETS12
RECORDED TRIALS165
CUSTOM PASS60%
DEEP VALIDATION · 2026-08-20
FULL REPORT →Dual-judge family campaign
CONSENSUS MEAN4.167 / 5
BOTH-JUDGE PASS75.6%
CROSS-JUDGED45 / 45
MEDIAN WALL37.40s
GPQA_DIAMONDserialized verification sample
80.0%
23/25 responses
IFEVALserialized verification sample
80.0%
25/25 responses
MMLU_PROserialized verification sample
48.0%
24/25 responses
Industry benchmarks
Accuracyattempts · responded
GPQA_DIAMOND
GPQA Diamond
3.3%
30 attempts · 1 responded
IFEVAL
IFEval
67.2%
30 attempts · 27 responded
MMLU_PRO
MMLU-Pro
50.0%
60 attempts · 60 responded
Custom 9-task suite
Mean /5n=trials
CUSTOM
agentic_tool_use
★
5.00
5 trials
CUSTOM
code_gen_long
★
5.00
5 trials
CUSTOM
json_strict
★
5.00
5 trials
CUSTOM
reasoning_multistep
★
5.00
5 trials
CUSTOM
summarize
★
4.20
5 trials
CUSTOM
creative_write
★
2.80
5 trials
CUSTOM
refactor_existing_code
-
2.20
5 trials
CUSTOM
code_debug
-
2.00
5 trials
CUSTOM
agentic_prompt
-
1.20
5 trials
Per-task distribution
Score
The accumulated custom suite below uses minimax-m3 + claude-fable-5 consensus. The dedicated 2026-08-20 campaign above is supplemental and uses minimax-m3 + a blind grok-4.6 re-judge on every successful response. Industry suites use deterministic family-specific graders. See methodology and the campaign report.