All models

Empirical cross-provider LLM benchmark. 9 custom tasks + 4 industry-standard families (GPQA Diamond, MMLU-Pro, IFEval, HumanEval+). 57/81 models have at least one industry-benchmark score.

57

WITH INDUSTRY DATA

24

CUSTOM-SUITE ONLY

SORT BY

FILTER

claude-cli/claude-fable

ACC — WALL — 1P$ —

GPQA Diamond not scored

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-fable-4

ACC — WALL — 1P$ —

MMLU-Pro not scored

claude-cli/claude-fable-5

ACC — WALL — 1P$ —

GPQA Diamond not scored

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-fable-5-t

ACC — WALL — 1P$ —

MMLU-Pro not scored

claude-cli/claude-haiku

ACC — WALL — 1P$ —

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-opus

ACC — WALL — 1P$ —

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-opus-4-8

ACC — WALL — 1P$ —

GPQA Diamond not scored

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-sonnet

ACC — WALL — 1P$ —

IFEval not scored

MMLU-Pro not scored

claude-cli/claude-sonnet-4-6

ACC — WALL — 1P$ —

GPQA Diamond not scored

IFEval not scored

CLAUDE-CLI long-context

claude-fable

ACC 73.7% WALL 18.1s 1P$ $9.68

GPQA Diamond 97%

CLAUDE-CLI fast

claude-haiku

ACC 54.3% WALL 40.4s 1P$ $22.48

GPQA Diamond 37%

CLAUDE-CLI reasoning

claude-opus

ACC 69.7% WALL 14.0s 1P$ $13.31

GPQA Diamond 93%

CLAUDE-CLI reasoning

claude-opus-5

ACC 80.6% WALL 15.9s 1P$ $1.38

GPQA Diamond 85%

CLAUDE-CLI reasoning

claude-opus-5-high

ACC 85.2% WALL 23.1s 1P$ —

GPQA Diamond 88%

CLAUDE-CLI general

claude-sonnet

ACC 70.6% WALL 9.6s 1P$ $2.95

GPQA Diamond 90%

codex-gpt-5.3-spark

ACC 75.7% WALL 7.7s 1P$ $1.81

GPQA Diamond 63%

CODEX-CLI general

codex-gpt-5.4

ACC 73.3% WALL 12.4s 1P$ $4.73

GPQA Diamond 73%

codex-gpt-5.4-mini

ACC 66.7% WALL 12.4s 1P$ $0.46

GPQA Diamond 67%

CODEX-CLI reasoning

codex-gpt-5.5

ACC 86.7% WALL 13.2s 1P$ $7.33

GPQA Diamond 87%

codex-gpt-5.6-luna

ACC 89.7% WALL 20.3s 1P$ $0.93

GPQA Diamond 83%

CODEX-CLI reasoning

codex-gpt-5.6-sol

ACC 88.7% WALL 11.9s 1P$ $7.23

GPQA Diamond 93%

CODEX-CLI general

codex-gpt-5.6-terra

ACC 86.7% WALL 10.7s 1P$ $5.22

GPQA Diamond 93%

ANTIGRAVITY-CLI reasoning

gemini-3.1-pro-high

ACC 71.1% WALL 25.5s 1P$ $2.55

GPQA Diamond 92%

ANTIGRAVITY-CLI fast

gemini-3.6-flash-high

ACC 70.7% WALL 25.7s 1P$ $1.64

GPQA Diamond 90%

XAI-DIRECT fast

grok-3-mini-direct

ACC 86.1% WALL 8.8s 1P$ $0.04

GPQA Diamond 83%

XAI-DIRECT reasoning

grok-4-1-fast-direct

ACC 87.8% WALL 5.8s 1P$ $0.03

GPQA Diamond 90%

XAI-DIRECT fast

grok-4-fast-direct

ACC 89.3% WALL 6.5s 1P$ —

GPQA Diamond 92%

XAI-DIRECT general

grok-4.20-non-reasoning-direct

ACC 80.5% WALL 2.1s 1P$ $0.47

GPQA Diamond 70%

XAI-DIRECT reasoning

grok-4.20-reasoning-direct

ACC 90.3% WALL 20.8s 1P$ $0.37

GPQA Diamond 89%

XAI-DIRECT general

grok-4.3-direct

ACC 86.1% WALL 4.8s 1P$ $0.38

GPQA Diamond 83%

XAI-DIRECT general

grok-4.3-latest-direct

ACC 90.3% WALL 8.0s 1P$ $0.69

GPQA Diamond 91%

XAI-DIRECT reasoning

grok-4.5-direct

ACC 86.7% WALL 12.2s 1P$ $4.00

GPQA Diamond 87%

HumanEval+ not scored

MMLU-Pro not scored

XAI-DIRECT reasoning

grok-4.6-direct

ACC 91.3% WALL 27.3s 1P$ $0.13

GPQA Diamond 92%

XAI-DIRECT general

grok-build-0.1

ACC 86.1% WALL 18.6s 1P$ $0.68

GPQA Diamond 80%

kimi-k3

ACC — WALL 17.6s 1P$ —

GPQA Diamond not scored

HumanEval+ not scored

IFEval not scored

MMLU-Pro not scored

SKYNET reasoning

minimax-m2.7

ACC 58.0% WALL 26.0s 1P$ $0.12

GPQA Diamond 41%

SKYNET reasoning

minimax-m2.7-highspeed

ACC 59.8% WALL 26.1s 1P$ $0.06

GPQA Diamond 46%

SKYNET reasoning

minimax-m3

ACC 59.3% WALL 16.8s 1P$ $0.24

GPQA Diamond 52%

minimax-medium

ACC — WALL — 1P$ —

GPQA Diamond not scored

IFEval not scored

mistral-medium-2508

ACC 68.0% WALL 4.2s 1P$ —

GPQA Diamond 63%

mistral/devstral

ACC 60.2% WALL 3.6s 1P$ —

GPQA Diamond 59%

SKYNET reasoning

mistral/magistral-medium

ACC 40.1% WALL 37.1s 1P$ $0.21

GPQA Diamond 20%

SKYNET reasoning

mistral/magistral-small

ACC 62.0% WALL 1.7s 1P$ $0.02

GPQA Diamond 62%

mistral/mistral-large

ACC 43.0% WALL 5.5s 1P$ $0.03

GPQA Diamond 17%

mistral/mistral-medium

ACC 66.4% WALL 3.4s 1P$ $0.32

GPQA Diamond 58%

LLAMA.CPP-LOCAL reasoning

muse-glimmer-30b-q4_k_xl-local

ACC 85.1% WALL 34.2s 1P$ —

GPQA Diamond 68%

HumanEval+ 100%

oai/grok-4-1-fast-reasoning

ACC — WALL — 1P$ —

GPQA Diamond not scored

SKYNET reasoning

ollama-cl/deepseek-v4-flash

ACC 63.0% WALL 6.5s 1P$ $0.01

GPQA Diamond 53%

SKYNET reasoning

ollama-cl/deepseek-v4-pro

ACC 40.8% WALL 16.4s 1P$ $0.14

GPQA Diamond 29%

ollama-cl/gemma4

ACC 71.8% WALL 3.8s 1P$ —

GPQA Diamond 73%

ollama-cl/gemma4:26b

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/gpt-oss-120b

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/gpt-oss:120b

ACC 43.4% WALL 10.5s 1P$ —

GPQA Diamond 52%

ollama-cl/gpt-oss:20b

ACC 52.3% WALL 25.2s 1P$ —

GPQA Diamond 43%

ollama-cl/kimi-k2.5

ACC 47.6% WALL 19.8s 1P$ $0.51

GPQA Diamond 23%

ollama-cl/kimi-k2.6

ACC 56.0% WALL 27.3s 1P$ $0.30

GPQA Diamond 40%

ollama-cl/kimi-k2.7-code

ACC 50.5% WALL 11.6s 1P$ $0.12

GPQA Diamond 31%

ollama-cl/minimax-m2.5

ACC — WALL 13.6s 1P$ —

GPQA Diamond not scored

HumanEval+ not scored

IFEval not scored

MMLU-Pro not scored

ollama-cl/mistral-large-3-675b

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/mistral-large-3:675b

ACC 23.3% WALL 4.6s 1P$ —

GPQA Diamond 23%

ollama-cl/nemotron-3-nano-30b

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/nemotron-3-nano:30b

ACC 33.3% WALL 9.4s 1P$ —

GPQA Diamond 33%

ollama-cl/nemotron-3-super

ACC 67.3% WALL 14.5s 1P$ —

GPQA Diamond 40%

ollama-cl/nemotron-3-ultra

ACC 51.8% WALL 73.1s 1P$ —

GPQA Diamond 50%

ollama-cl/qwen3-coder-next

ACC 62.2% WALL 4.4s 1P$ —

GPQA Diamond 53%

ollama-cl/qwen3-coder:480b

ACC 65.2% WALL 2.9s 1P$ —

GPQA Diamond 53%

ollama-cl/qwen3.5

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/qwen3.5-code

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/qwen3.5:27b

ACC — WALL — 1P$ —

IFEval not scored

ollama-cl/qwen3.5:397b

ACC 44.4% WALL 48.7s 1P$ —

GPQA Diamond 13%

SKYNET reasoning

xai/grok-3-mini

ACC — WALL 6.2s 1P$ $0.04

GPQA Diamond not scored

SKYNET reasoning

xai/grok-4-1-fast-reasoning

ACC — WALL 6.2s 1P$ $0.04

GPQA Diamond not scored

xai/grok-4.3-latest

ACC — WALL 6.7s 1P$ $1.04

GPQA Diamond not scored

xai/grok-4.5

ACC — WALL — 1P$ —

GPQA Diamond not scored

zai-coding/glm-4.6

ACC 46.3% WALL — 1P$ —

GPQA Diamond 3%

zai-coding/glm-4.7

ACC 21.6% WALL 64.8s 1P$ $0.50

GPQA Diamond 15%

zai-coding/glm-5

ACC 14.1% WALL 31.5s 1P$ $0.34

GPQA Diamond 2%

zai-coding/glm-5-turbo

ACC 40.2% WALL 60.5s 1P$ $0.28

GPQA Diamond 3%

zai-coding/glm-5.1

ACC 40.7% WALL 20.0s 1P$ $0.27

GPQA Diamond 23%

zai-coding/glm-5.2

ACC 30.5% WALL 25.2s 1P$ $0.29

GPQA Diamond 22%

SKYNET fast-reasoning

zai-coding/glm-5.3-flash

ACC 69.3% WALL 45.5s 1P$ —

GPQA Diamond 56%