Leaderboard

SimpleQA

simpleqa

Record

Category: knowledge
Scored models: 80

Scores

Model	Score
deepseek-ai/DeepSeek-V3.2-Exp-Base	96.8
deepseek-ai/DeepSeek-V3.1-Base	93.4
deepseek-ai/DeepSeek-V3.1-Terminus	93.4
zai-org/GLM-4-32B-0414	88.1
THUDM/GLM-4-32B-0414	88.1
zai-org/GLM-4-9B-0414	79
THUDM/GLM-4-9B-0414	79
zai-org/GLM-4-32B-Base-0414	79
THUDM/GLM-4-9B-0414	79
THUDM/GLM-4-9B-0414	79
THUDM/GLM-4-9B-0414	79
Qwen/Qwen3-4B	76.8
deepseek-ai/DeepSeek-V4-Flash	76.4
deepseek-ai/DeepSeek-V4-Pro	76.4
—	76.4
deepseek-ai/DeepSeek-V4-Flash	76.4
deepseek-ai/DeepSeek-V4-Flash	76.4
—	76.4
deepseek-ai/DeepSeek-V4-Pro	76.4
Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen/Qwen3-235B-A22B-Instruct-2507-FP8	71.1
MiniMaxAI/MiniMax-Text-01-hf	64.6

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: knowledge
id: simpleqa
name: SimpleQA
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
DeepSeek-V3.2-Exp	medium	SimpleQA	deepseek-v3-2-exp-base	deepseek-ai	1	deepseek-ai/DeepSeek-V3.2-Exp-Base	96.8
DeepSeek-V3.1-Terminus	medium	SimpleQA	deepseek-v3-1-base	deepseek-ai	2	deepseek-ai/DeepSeek-V3.1-Base	93.4
DeepSeek-V3.1-Terminus-GGUF	medium	SimpleQA	deepseek-v3-1-terminus	unsloth	3	deepseek-ai/DeepSeek-V3.1-Terminus	93.4
GLM-4-32B-0414	high	Unknown	glm-4-32b-0414	zai-org	4	zai-org/GLM-4-32B-0414	88.1
GLM-4-32B-0414-GGUF	high	Unknown	Unknown	unsloth	5	THUDM/GLM-4-32B-0414	88.1
GLM-4-9B-0414	low	Unknown	glm-4-9b-0414	zai-org	6	zai-org/GLM-4-9B-0414	79
GLM-4-9B-0414-GGUF	low	Unknown	glm-4-9b-0414	unsloth	7	THUDM/GLM-4-9B-0414	79
GLM-4-32B-Base-0414	low	Unknown	glm-4-32b-base-0414	zai-org	8	zai-org/GLM-4-32B-Base-0414	79
GLM-4-9B-0414-unsloth-bnb-4bit	low	Unknown	glm-4-9b-0414	unsloth	9	THUDM/GLM-4-9B-0414	79
GLM-4-9B-0414	low	Unknown	glm-4-9b-0414	unsloth	10	THUDM/GLM-4-9B-0414	79
GLM-4-9B-0414-bnb-4bit	low	Unknown	glm-4-9b-0414	unsloth	11	THUDM/GLM-4-9B-0414	79
II-Search-4B-GGUF	medium	OpenAI/SimpleQA	qwen3-4b-base	Mungert	12	Qwen/Qwen3-4B	76.8
DeepSeek-V4-Flash	medium	Chinese-SimpleQA (Pass@1)	deepseek-v4-flash	deepseek-ai	13	deepseek-ai/DeepSeek-V4-Flash	76.4
DeepSeek-V4-Pro	medium	Chinese-SimpleQA (Pass@1)	deepseek-v4-pro	deepseek-ai	14	deepseek-ai/DeepSeek-V4-Pro	76.4
DeepSeek-V4-Flash-DSpark	medium	Chinese-SimpleQA (Pass@1)	Unknown	deepseek-ai	15	—	76.4
DeepSeek-V4-Flash-GGUF	medium	Chinese-SimpleQA (Pass@1)	deepseek-v4-flash	unsloth	16	deepseek-ai/DeepSeek-V4-Flash	76.4
DeepSeek-V4-Flash	medium	Chinese-SimpleQA (Pass@1)	deepseek-v4-flash	unsloth	17	deepseek-ai/DeepSeek-V4-Flash	76.4
DeepSeek-V4-Pro-DSpark	medium	Chinese-SimpleQA (Pass@1)	Unknown	deepseek-ai	18	—	76.4
DeepSeek-V4-Pro	medium	Chinese-SimpleQA (Pass@1)	deepseek-v4-pro	unsloth	19	deepseek-ai/DeepSeek-V4-Pro	76.4
Qwen3-235B-A22B-Instruct-2507-FP8	medium	CSimpleQA	qwen3-235b-a22b-instruct-2507	Qwen	20	Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen3-235B-A22B-Instruct-2507	medium	CSimpleQA	qwen3-235b-a22b-instruct-2507	Qwen	21	Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen3-235B-A22B-Instruct-2507-GGUF	medium	CSimpleQA	qwen3-235b-a22b-instruct-2507	unsloth	22	Qwen/Qwen3-235B-A22B-Instruct-2507	71.1
Qwen3-235B-A22B-Instruct-2507-FP8	medium	CSimpleQA	qwen3-235b-a22b-instruct-2507-fp8	unsloth	23	Qwen/Qwen3-235B-A22B-Instruct-2507-FP8	71.1
MiniMax-Text-01-hf	medium	C-SimpleQA	minimax-text-01-hf	MiniMaxAI	24	MiniMaxAI/MiniMax-Text-01-hf	64.6
MiniMax-Text-01	medium	C-SimpleQA	minimax-text-01	MiniMaxAI	25	MiniMaxAI/MiniMax-Text-01	64.6
Ling-2.6-1T-base	medium	C-SimpleQA	ling-2-6-1t	inclusionAI	26	inclusionAI/Ling-2.6-1T	64.53
DR-Tulu-8B-GGUF	low	Unknown	Unknown	Mungert	27	rl-research/DR-Tulu-SFT-8B	52.6
Ling-2.6-flash-base	medium	C-SimpleQA	ling-2-6-flash	inclusionAI	28	inclusionAI/Ling-2.6-flash	49.43
DeepSeek-V4-Flash	medium	SimpleQA-Verified (Pass@1)	deepseek-v4-flash	deepseek-ai	29	deepseek-ai/DeepSeek-V4-Flash	46.2
DeepSeek-V4-Pro	medium	SimpleQA-Verified (Pass@1)	deepseek-v4-pro	deepseek-ai	30	deepseek-ai/DeepSeek-V4-Pro	46.2
DeepSeek-V4-Flash-DSpark	medium	SimpleQA-Verified (Pass@1)	Unknown	deepseek-ai	31	—	46.2
DeepSeek-V4-Flash-GGUF	medium	SimpleQA-Verified (Pass@1)	deepseek-v4-flash	unsloth	32	deepseek-ai/DeepSeek-V4-Flash	46.2
DeepSeek-V4-Flash	medium	SimpleQA-Verified (Pass@1)	deepseek-v4-flash	unsloth	33	deepseek-ai/DeepSeek-V4-Flash	46.2
DeepSeek-V4-Pro-DSpark	medium	SimpleQA-Verified (Pass@1)	Unknown	deepseek-ai	34	—	46.2
DeepSeek-V4-Pro	medium	SimpleQA-Verified (Pass@1)	deepseek-v4-pro	unsloth	35	deepseek-ai/DeepSeek-V4-Pro	46.2
MiniMax-Text-01-hf	medium	SimpleQA	minimax-text-01-hf	MiniMaxAI	36	MiniMaxAI/MiniMax-Text-01-hf	39
MiniMax-Text-01	medium	SimpleQA	minimax-text-01	MiniMaxAI	37	MiniMaxAI/MiniMax-Text-01	39
DeepSeek-V4-Flash	medium	Simple-QA verified (EM)	deepseek-v4-flash	deepseek-ai	38	deepseek-ai/DeepSeek-V4-Flash	28.3
DeepSeek-V4-Pro	medium	Simple-QA verified (EM)	deepseek-v4-pro	deepseek-ai	39	deepseek-ai/DeepSeek-V4-Pro	28.3
DeepSeek-V4-Flash-DSpark	medium	Simple-QA verified (EM)	Unknown	deepseek-ai	40	—	28.3
DeepSeek-V4-Flash-GGUF	medium	Simple-QA verified (EM)	deepseek-v4-flash	unsloth	41	deepseek-ai/DeepSeek-V4-Flash	28.3
DeepSeek-V4-Flash	medium	Simple-QA verified (EM)	deepseek-v4-flash	unsloth	42	deepseek-ai/DeepSeek-V4-Flash	28.3
DeepSeek-V4-Pro-DSpark	medium	Simple-QA verified (EM)	Unknown	deepseek-ai	43	—	28.3
DeepSeek-V4-Pro	medium	Simple-QA verified (EM)	deepseek-v4-pro	unsloth	44	deepseek-ai/DeepSeek-V4-Pro	28.3
Qwen3-235B-A22B-Instruct-2507-FP8	medium	SimpleQA	qwen3-235b-a22b-instruct-2507	Qwen	45	Qwen/Qwen3-235B-A22B-Instruct-2507	27.2
Qwen3-235B-A22B-Instruct-2507	medium	SimpleQA	qwen3-235b-a22b-instruct-2507	Qwen	46	Qwen/Qwen3-235B-A22B-Instruct-2507	27.2
Qwen3-235B-A22B-Instruct-2507-GGUF	medium	SimpleQA	qwen3-235b-a22b-instruct-2507	unsloth	47	Qwen/Qwen3-235B-A22B-Instruct-2507	27.2
Qwen3-235B-A22B-Instruct-2507-FP8	medium	SimpleQA	qwen3-235b-a22b-instruct-2507-fp8	unsloth	48	Qwen/Qwen3-235B-A22B-Instruct-2507-FP8	27.2
Ling-2.6-1T-base	medium	SimpleQA	ling-2-6-1t	inclusionAI	49	inclusionAI/Ling-2.6-1T	20.87
Hunyuan-7B-Instruct-GGUF	medium	Chinese SimpleQA	hunyuan-7b-pretrain	Mungert	50	tencent/Hunyuan-7B-Pretrain	12.51
Mistral-Small-3.1-24B-Instruct-2503	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	mistralai	51	mistralai/Mistral-Small-3.1-24B-Base-2503	10.43
Mistral-Small-3.2-24B-Instruct-2506	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	mistralai	52	mistralai/Mistral-Small-3.1-24B-Base-2503	10.43
Mistral-Small-3.2-24B-Instruct-2506-GGUF	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	unsloth	53	mistralai/Mistral-Small-3.2-24B-Instruct-2506	10.43
Mistral-Small-3.2-24B-Instruct-2506-bnb-4bit	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	unsloth	54	mistralai/Mistral-Small-3.2-24B-Instruct-2506	10.43
Mistral-Small-3.2-24B-Instruct-2506	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	unsloth	55	mistralai/Mistral-Small-3.2-24B-Instruct-2506	10.43
Mistral-Small-3.1-24B-Instruct-2503-GGUF	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-instruct-2503	unsloth	56	mistralai/Mistral-Small-3.1-24B-Instruct-2503	10.43
Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	unsloth	57	mistralai/Mistral-Small-3.2-24B-Instruct-2506	10.43
Mistral-Small-3.1-24B-Instruct-2503-bnb-4bit	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-instruct-2503	unsloth	58	mistralai/Mistral-Small-3.1-24B-Instruct-2503	10.43
Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bit	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-instruct-2503	unsloth	59	mistralai/Mistral-Small-3.1-24B-Instruct-2503	10.43
Mistral-Small-3.1-24B-Instruct-2503	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-instruct-2503	unsloth	60	mistralai/Mistral-Small-3.1-24B-Instruct-2503	10.43
Mistral-Small-3.1-24B-Instruct-2503-GGUF	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	Mungert	61	mistralai/Mistral-Small-3.1-24B-Base-2503	10.43
Mistral-Small-3.2-24B-Instruct-2506-GGUF	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	Mungert	62	mistralai/Mistral-Small-3.1-24B-Base-2503	10.43
Mistral-Small-3.2-24B-Instruct-2506-FP8	low	MMLU Pro (5-shot CoT) \| GPQA Main (5-shot CoT) \| GPQA Diamond (5-shot CoT )	mistral-small-3-1-24b-base-2503	unsloth	63	mistralai/Mistral-Small-3.2-24B-Instruct-2506	10.43
Ling-2.6-flash-base	medium	SimpleQA	ling-2-6-flash	inclusionAI	64	inclusionAI/Ling-2.6-flash	10.01
Hunyuan-7B-Instruct-GGUF	medium	simpleQA (5shot)	hunyuan-7b-pretrain	Mungert	65	tencent/Hunyuan-7B-Pretrain	2.38
gemma-3-270m	medium	[SimpleQA][]	gemma-3-270m	google	66	google/gemma-3-270m	2.2
gemma-3-270m-it	medium	[SimpleQA][]	gemma-3-270m	google	67	google/gemma-3-270m	2.2
gemma-3-270m-it-GGUF	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	68	google/gemma-3-270m-it	2.2
gemma-3-270m-it	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	69	google/gemma-3-270m-it	2.2
gemma-3-270m-it-unsloth-bnb-4bit	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	70	google/gemma-3-270m-it	2.2
gemma-3-270m-it-qat-GGUF	medium	[SimpleQA][]	gemma-3-270m-it-qat-q4-0-unquantized	unsloth	71	google/gemma-3-270m-it-qat-q4_0-unquantized	2.2
gemma-3-270m	medium	[SimpleQA][]	gemma-3-270m	unsloth	72	google/gemma-3-270m	2.2
gemma-3-270m-unsloth-bnb-4bit	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	73	google/gemma-3-270m-it	2.2
gemma-3-270m-it-qat-q4_0-unquantized	medium	[SimpleQA][]	gemma-3-270m-it	google	74	google/gemma-3-270m-it	2.2
gemma-3-270m-it-bnb-4bit	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	75	google/gemma-3-270m-it	2.2
gemma-3-270m-it-qat	medium	[SimpleQA][]	gemma-3-270m-it-qat-q4-0-unquantized	unsloth	76	google/gemma-3-270m-it-qat-q4_0-unquantized	2.2
gemma-3-270m-bnb-4bit	medium	[SimpleQA][]	gemma-3-270m	unsloth	77	google/gemma-3-270m	2.2
gemma-3-270m-qat-q4_0-unquantized	medium	[SimpleQA][]	gemma-3-270m	google	78	google/gemma-3-270m	2.2
gemma-3-270m-it-GGUF	medium	[SimpleQA][]	gemma-3-270m	Mungert	79	google/gemma-3-270m	2.2
gemma-3-270m-it-torchao-FP8	medium	[SimpleQA][]	gemma-3-270m-it	unsloth	80	google/gemma-3-270m-it	2.2

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/simpleqa.html
url: huggingface.co/spaces/simpleqa ↗