XAI-DIRECT
grok-4.6-direct
Grok 4.6 direct OAuth (api.x.ai REST) — 500k ctx, verified 2026-08-13
★VALS INDEX
91.3%
+/- 0.9
Accuracy
⏱VALS INDEX
27.3s
Latency (median)
$VALS INDEX
$0.1281
Cost / Test (1P-est)
SCORE STATUSscored
TEST SETS11
RECORDED TRIALS95
CUSTOM PASS80%
Industry benchmarks
Accuracyattempts · responded
GPQA_DIAMOND
GPQA Diamond
92.0%
25 attempts · 25 responded
IFEVAL
IFEval
90.7%
25 attempts · 25 responded
Custom 9-task suite
Mean /5n=trials
CUSTOM
agentic_prompt
★
5.00
5 trials
CUSTOM
code_debug
★
5.00
5 trials
CUSTOM
code_gen_long
★
5.00
5 trials
CUSTOM
json_strict
★
5.00
5 trials
CUSTOM
reasoning_multistep
★
5.00
5 trials
CUSTOM
agentic_tool_use
★
4.60
5 trials
CUSTOM
summarize
★
4.20
5 trials
CUSTOM
refactor_existing_code
-
3.00
5 trials
CUSTOM
creative_write
★
2.00
5 trials
Per-task distribution
Score
All trials on this page were run through the harness. Industry suites use deterministic family-specific reference graders (no LLM judge). The custom suite is graded by consensus of minimax-m3 + claude-fable-5. See /industry/ for the full benchmark suite.