Leaderboard

BBH

bbh

Record

Category: reasoning
Scored models: 392

Scores

Model	Score
allenai/Olmo-3-32B-Think-DPO	88.8
allenai/Olmo-3-1125-32B	88.8
allenai/Olmo-3-32B-Think	88.8
allenai/Olmo-3-32B-Think-SFT	88.8
allenai/Olmo-3-32B-Think	88.8
allenai/Olmo-3-32B-Think-DPO	88.6
allenai/Olmo-3.1-32B-Think	88.6
meta-llama/Llama-3.1-8B	88
allenai/Llama-3.1-Tulu-3-8B-SFT	88
allenai/Llama-3.1-Tulu-3-8B-SFT	88
allenai/Llama-3.1-Tulu-3-8B-DPO	88
allenai/Llama-3.1-Tulu-3-70B-DPO	88
meta-llama/Llama-3.1-70B	88
allenai/Llama-3.1-Tulu-3-405B-DPO	88
allenai/Llama-3.1-Tulu-3-70B-SFT	88
meta-llama/Llama-3.1-405B	88
allenai/Llama-3.1-Tulu-3-405B-SFT	88
deepseek-ai/DeepSeek-V4-Flash	87.6
deepseek-ai/DeepSeek-V4-Pro	87.6
—	87.6
deepseek-ai/DeepSeek-V4-Flash	87.6
deepseek-ai/DeepSeek-V4-Flash	87.6
—	87.6
deepseek-ai/DeepSeek-V4-Pro	87.6

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: reasoning
id: bbh
name: BBH
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
Olmo-3-32B-Think	medium	BigBenchHard	olmo-3-32b-think-dpo	allenai	1	allenai/Olmo-3-32B-Think-DPO	88.8
Olmo-3-32B-Think-SFT	medium	BigBenchHard	olmo-3-1125-32b	allenai	2	allenai/Olmo-3-1125-32B	88.8
Olmo-3-32B-Think-GGUF	medium	BigBenchHard	olmo-3-32b-think-dpo	unsloth	3	allenai/Olmo-3-32B-Think	88.8
Olmo-3-32B-Think-DPO	medium	BigBenchHard	olmo-3-32b-think-sft	allenai	4	allenai/Olmo-3-32B-Think-SFT	88.8
Olmo-3-32B-Think-unsloth-bnb-4bit	medium	BigBenchHard	olmo-3-32b-think-dpo	unsloth	5	allenai/Olmo-3-32B-Think	88.8
Olmo-3.1-32B-Think	medium	BigBenchHard	olmo-3-32b-think-dpo	allenai	6	allenai/Olmo-3-32B-Think-DPO	88.6
Olmo-3.1-32B-Think-GGUF	medium	BigBenchHard	olmo-3-1-32b-think	unsloth	7	allenai/Olmo-3.1-32B-Think	88.6
Llama-3.1-Tulu-3-8B-SFT	medium	BigBenchHard (0 shot, CoT)	llama-3-1-8b	allenai	8	meta-llama/Llama-3.1-8B	88
Llama-3.1-Tulu-3-8B-DPO	medium	BigBenchHard (0 shot, CoT)	llama-3-1-tulu-3-8b-sft	allenai	9	allenai/Llama-3.1-Tulu-3-8B-SFT	88
Llama-3.1-Tulu-3-8B-RM	medium	BigBenchHard (0 shot, CoT)	llama-3-1-tulu-3-8b-sft	allenai	10	allenai/Llama-3.1-Tulu-3-8B-SFT	88
Llama-3.1-Tulu-3.1-8B	medium	BigBenchHard (0 shot, CoT)	llama-3-1-tulu-3-8b-dpo	allenai	11	allenai/Llama-3.1-Tulu-3-8B-DPO	88
Llama-3.1-Tulu-3-70B	medium	BigBenchHard (0 shot, CoT)	llama-3-1-tulu-3-70b-dpo	allenai	12	allenai/Llama-3.1-Tulu-3-70B-DPO	88
Llama-3.1-Tulu-3-70B-SFT	medium	BigBenchHard (0 shot, CoT)	llama-3-1-70b	allenai	13	meta-llama/Llama-3.1-70B	88
Llama-3.1-Tulu-3-405B	medium	BigBenchHard (0 shot, CoT)	Unknown	allenai	14	allenai/Llama-3.1-Tulu-3-405B-DPO	88
Llama-3.1-Tulu-3-70B-DPO	medium	BigBenchHard (0 shot, CoT)	llama-3-1-tulu-3-70b-sft	allenai	15	allenai/Llama-3.1-Tulu-3-70B-SFT	88
Llama-3.1-Tulu-3-405B-SFT	medium	BigBenchHard (0 shot, CoT)	llama-3-1-405b	allenai	16	meta-llama/Llama-3.1-405B	88
Llama-3.1-Tulu-3-405B-DPO	medium	BigBenchHard (0 shot, CoT)	Unknown	allenai	17	allenai/Llama-3.1-Tulu-3-405B-SFT	88
DeepSeek-V4-Flash	medium	BBH (EM)	deepseek-v4-flash	deepseek-ai	18	deepseek-ai/DeepSeek-V4-Flash	87.6
DeepSeek-V4-Pro	medium	BBH (EM)	deepseek-v4-pro	deepseek-ai	19	deepseek-ai/DeepSeek-V4-Pro	87.6
DeepSeek-V4-Flash-DSpark	medium	BBH (EM)	Unknown	deepseek-ai	20	—	87.6
DeepSeek-V4-Flash-GGUF	medium	BBH (EM)	deepseek-v4-flash	unsloth	21	deepseek-ai/DeepSeek-V4-Flash	87.6
DeepSeek-V4-Flash	medium	BBH (EM)	deepseek-v4-flash	unsloth	22	deepseek-ai/DeepSeek-V4-Flash	87.6
DeepSeek-V4-Pro-DSpark	medium	BBH (EM)	Unknown	deepseek-ai	23	—	87.6
DeepSeek-V4-Pro	medium	BBH (EM)	deepseek-v4-pro	unsloth	24	deepseek-ai/DeepSeek-V4-Pro	87.6
Ling-2.6-1T-base	medium	BBH	ling-2-6-1t	inclusionAI	25	inclusionAI/Ling-2.6-1T	86.88
Hunyuan-A13B-Instruct-GGUF	medium	BBH	hunyuan-a13b-instruct	unsloth	26	tencent/Hunyuan-A13B-Instruct	86.3
Nemotron-4-340B-Base	low	Unknown	Unknown	nvidia	27	nvidia/Nemotron-4-340B-Base	85.44
Ling-2.6-flash-base	medium	BBH	ling-2-6-flash	inclusionAI	28	inclusionAI/Ling-2.6-flash	84.82
SynLogic-32B	low	Unknown	qwen2-5-32b	MiniMaxAI	29	Qwen/Qwen2.5-32B	84.5
Phi-3-medium-4k-instruct	medium	BigBench Hard<br>3-shot	phi-3-medium-4k-instruct	microsoft	30	microsoft/Phi-3-medium-4k-instruct	81.4
GRIN-MoE	medium	BigBench-Hard	grin-moe	microsoft	31	microsoft/GRIN-MoE	81.4
Olmo-3-1125-32B	low	Unknown	olmo-3-1125-32b	allenai	32	allenai/Olmo-3-1125-32B	81.1
DeepSeek-V2-Chat	medium	BBH	deepseek-v2-chat	deepseek-ai	33	deepseek-ai/DeepSeek-V2-Chat	81
DeepSeek-V2	medium	BBH	deepseek-v2	deepseek-ai	34	deepseek-ai/DeepSeek-V2	81
DeepSeek-V2-Chat-0628	medium	BBH	deepseek-v2-chat-0628	deepseek-ai	35	deepseek-ai/DeepSeek-V2-Chat-0628	79.7
Phi-3-small-8k-instruct	medium	BigBench Hard<br>3-shot	phi-3-small-8k-instruct	microsoft	36	microsoft/Phi-3-small-8k-instruct	79.1
Qwen2-72B	medium	BBH	qwen2-72b	Qwen	37	Qwen/Qwen2-72B	78.9
Phi-3-medium-128k-instruct	medium	BigBench Hard<br>3-shot	phi-3-medium-128k-instruct	microsoft	38	microsoft/Phi-3-medium-128k-instruct	77.9
internlm2_5-20b	medium	BBH	internlm2-5-20b	internlm	39	internlm/internlm2_5-20b	77.82
Phi-3-small-128k-instruct	medium	BigBench Hard<br>3-shot	phi-3-small-128k-instruct	microsoft	40	microsoft/Phi-3-small-128k-instruct	77.6
internlm2_5-20b-chat	medium	BBH (3-shot CoT)	internlm2-5-20b-chat	internlm	41	internlm/internlm2_5-20b-chat	76.3
Ling-lite-1.5	medium	BBH(EM)	ling-lite-1-5	inclusionAI	42	inclusionAI/Ling-lite-1.5	75.75
Ling-lite	medium	BBH(EM)	ling-lite	inclusionAI	43	inclusionAI/Ling-lite	74.58
internlm2_5-7b-chat	medium	BBH (3-shot CoT)	internlm2-5-7b-chat	internlm	44	internlm/internlm2_5-7b-chat	71.6
Falcon-H1-34B-Instruct	medium	BBH	falcon-h1-34b-base	tiiuae	45	tiiuae/Falcon-H1-34B-Base	70.68
Falcon-H1-34B-Instruct-GGUF	medium	BBH	falcon-h1-34b-instruct	tiiuae	46	tiiuae/Falcon-H1-34B-Instruct	70.68
Falcon-H1-34B-Instruct-GGUF	medium	BBH	falcon-h1-34b-instruct	unsloth	47	tiiuae/Falcon-H1-34B-Instruct	70.68
Falcon-H1-34B-Instruct-GPTQ-Int4	medium	BBH	falcon-h1-34b-instruct	tiiuae	48	tiiuae/Falcon-H1-34B-Instruct	70.68
Phi-4-mini-instruct	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	microsoft	49	microsoft/Phi-4-mini-instruct	70.4
Phi-4-mini-instruct-GGUF	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	unsloth	50	microsoft/Phi-4-mini-instruct	70.4
Phi-4-mini-instruct-unsloth-bnb-4bit	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	unsloth	51	microsoft/Phi-4-mini-instruct	70.4
Phi-4-mini-instruct-bnb-4bit	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	unsloth	52	microsoft/Phi-4-mini-instruct	70.4
Phi-4-mini-instruct	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	unsloth	53	microsoft/Phi-4-mini-instruct	70.4
Phi-4-mini-instruct-GGUF	medium	BigBench Hard (0-shot, CoT)	phi-4-mini-instruct	Mungert	54	microsoft/Phi-4-mini-instruct	70.4
internlm2_5-7b	medium	BBH	internlm2-5-7b	internlm	55	internlm/internlm2_5-7b	70.1
Falcon-H1-34B-Base	medium	BBH	falcon-h1-34b-base	tiiuae	56	tiiuae/Falcon-H1-34B-Base	69.36
Olmo-3.1-32B-Instruct	medium	BigBenchHard	olmo-3-1-32b-instruct-dpo	allenai	57	allenai/Olmo-3.1-32B-Instruct-DPO	69
Olmo-3.1-32B-Instruct-SFT	medium	BigBenchHard	olmo-3-1125-32b	allenai	58	allenai/Olmo-3-1125-32B	69
Olmo-3.1-32B-Instruct-DPO	medium	BigBenchHard	olmo-3-1-32b-instruct-sft	allenai	59	allenai/Olmo-3.1-32B-Instruct-SFT	69
Olmo-3.1-32B-Instruct-GGUF	medium	BigBenchHard	olmo-3-1-32b-instruct	unsloth	60	allenai/Olmo-3.1-32B-Instruct	69
Olmo-3.1-32B-Instruct	medium	BigBenchHard	olmo-3-1-32b-instruct	unsloth	61	allenai/Olmo-3.1-32B-Instruct	69
Llama-3.1-Tulu-3-8B-SFT	medium	BigBenchHard (3 shot, CoT)	llama-3-1-8b	allenai	62	meta-llama/Llama-3.1-8B	67.9
Llama-3.1-Tulu-3-8B	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-8b-dpo	allenai	63	allenai/Llama-3.1-Tulu-3-8B-DPO	67.9
Llama-3.1-Tulu-3-8B-DPO	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-8b-sft	allenai	64	allenai/Llama-3.1-Tulu-3-8B-SFT	67.9
Llama-3.1-Tulu-3-8B-RM	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-8b-sft	allenai	65	allenai/Llama-3.1-Tulu-3-8B-SFT	67.9
Llama-3.1-Tulu-3.1-8B	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-8b-dpo	allenai	66	allenai/Llama-3.1-Tulu-3-8B-DPO	67.9
Llama-3.1-Tulu-3-70B	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-70b-dpo	allenai	67	allenai/Llama-3.1-Tulu-3-70B-DPO	67.9
Llama-3.1-Tulu-3-70B-SFT	medium	BigBenchHard (3 shot, CoT)	llama-3-1-70b	allenai	68	meta-llama/Llama-3.1-70B	67.9
Llama-3.1-Tulu-3-405B	medium	BigBenchHard (3 shot, CoT)	Unknown	allenai	69	allenai/Llama-3.1-Tulu-3-405B-DPO	67.9
Llama-3.1-Tulu-3-70B-DPO	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-70b-sft	allenai	70	allenai/Llama-3.1-Tulu-3-70B-SFT	67.9
Llama-3.1-Tulu-3-8B-GGUF	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-8b	unsloth	71	allenai/Llama-3.1-Tulu-3-8B	67.9
Llama-3.1-Tulu-3-405B-SFT	medium	BigBenchHard (3 shot, CoT)	llama-3-1-405b	allenai	72	meta-llama/Llama-3.1-405B	67.9
Llama-3.1-Tulu-3-70B-GGUF	medium	BigBenchHard (3 shot, CoT)	llama-3-1-tulu-3-70b	unsloth	73	allenai/Llama-3.1-Tulu-3-70B	67.9
Llama-3.1-Tulu-3-405B-DPO	medium	BigBenchHard (3 shot, CoT)	Unknown	allenai	74	allenai/Llama-3.1-Tulu-3-405B-SFT	67.9
SUS-Chat-34B-GGUF	high	Unknown	Unknown	TheBloke	75	SUSTech/SUS-Chat-34B	67.62
UNAversal-8x7B-v1beta-GGUF	medium	Unknown	unaversal-8x7b-v1beta	TheBloke	76	fblgit/UNAversal-8x7B-v1beta	67.52
OLMo-2-0425-1B	low	Unknown	olmo-2-0425-1b	allenai	77	allenai/OLMo-2-0425-1B	66.6
OLMo-2-0325-32B-Instruct	low	Unknown	Unknown	allenai	78	allenai/OLMo-2-0325-32B-DPO	66.6
OLMo-2-0325-32B-Instruct-GGUF	low	Unknown	olmo-2-0325-32b-instruct	unsloth	79	allenai/OLMo-2-0325-32B-Instruct	66.6
OLMo-2-0325-32B-SFT	low	Unknown	olmo-2-0325-32b	allenai	80	allenai/OLMo-2-0325-32B	66.6
OLMo-2-0325-32B-DPO	low	Unknown	Unknown	allenai	81	allenai/OLMo-2-0325-32B-SFT	66.6
OLMo-2-0325-32B-Instruct	low	Unknown	olmo-2-0325-32b-instruct	unsloth	82	allenai/OLMo-2-0325-32B-Instruct	66.6
llama-3-tulu-2-8b	low	GSM8k 8-shot cot \| BBH 3-shot cot	meta-llama-3-8b	allenai	83	meta-llama/Meta-Llama-3-8B	65.3
llama-3-tulu-2-70b	low	GSM8k 8-shot cot \| BBH 3-shot cot	meta-llama-3-70b	allenai	84	meta-llama/Meta-Llama-3-70B	65.3
internlm2-chat-7b	medium	BBH	internlm2-chat-7b	internlm	85	internlm/internlm2-chat-7b	65
internlm2-7b	medium	BBH	Unknown	internlm	86	internlm/internlm2-7b	65
internlm2-chat-20b	medium	BBH	internlm2-chat-20b	internlm	87	internlm/internlm2-chat-20b	65
internlm2-20b	medium	BBH	Unknown	internlm	88	internlm/internlm2-20b	65
internlm2-base-7b	medium	BBH	Unknown	internlm	89	internlm/internlm2-base-7b	65
internlm2-base-20b	medium	BBH	Unknown	internlm	90	internlm/internlm2-base-20b	65
internlm2-chat-7b-sft	medium	BBH	internlm2-chat-7b-sft	internlm	91	internlm/internlm2-chat-7b-sft	65
internlm2-chat-20b-sft	medium	BBH	internlm2-chat-20b-sft	internlm	92	internlm/internlm2-chat-20b-sft	65
DiscoLM-70B-GGUF	medium	BBH	discolm-70b	TheBloke	93	DiscoResearch/DiscoLM-70b	63.4
SynLogic-7B	low	Unknown	qwen2-5-7b	MiniMaxAI	94	Qwen/Qwen2.5-7B	62.7
Falcon-H1-7B-Instruct	medium	BBH	falcon-h1-7b-base	tiiuae	95	tiiuae/Falcon-H1-7B-Base	62.28
Falcon-H1-7B-Instruct-GPTQ-Int4	medium	BBH	falcon-h1-7b-instruct	tiiuae	96	tiiuae/Falcon-H1-7B-Instruct	62.28
Falcon-H1-7B-Instruct-GGUF	medium	BBH	falcon-h1-7b-instruct	tiiuae	97	tiiuae/Falcon-H1-7B-Instruct	62.28
Falcon-H1-7B-Instruct-GGUF	medium	BBH	falcon-h1-7b-instruct	unsloth	98	tiiuae/Falcon-H1-7B-Instruct	62.28
Falcon-H1-7B-Base	medium	BBH	falcon-h1-7b-base	tiiuae	99	tiiuae/Falcon-H1-7B-Base	60.61
Qwen2-7B	medium	BBH	qwen2-7b	Qwen	100	Qwen/Qwen2-7B	56.1
Olmo-3-1025-7B	low	Unknown	olmo-3-1025-7b	allenai	101	allenai/Olmo-3-1025-7B	55.6
Olmo-Hybrid-7B	low	Unknown	olmo-hybrid-7b	allenai	102	allenai/Olmo-Hybrid-7B	55.6
Olmo-3-1025-7B-GGUF	low	Unknown	olmo-3-1025-7b	Mungert	103	allenai/Olmo-3-1025-7B	55.6
Phi-mini-MoE-instruct	high	Unknown	phi-mini-moe-instruct	microsoft	104	microsoft/Phi-mini-MoE-instruct	55.27
Falcon-H1-1.5B-Deep-Instruct	medium	BBH	falcon-h1-1-5b-deep-base	tiiuae	105	tiiuae/Falcon-H1-1.5B-Deep-Base	54.43
Falcon-H1-1.5B-Deep-Instruct-GGUF	medium	BBH	falcon-h1-1-5b-deep-instruct	tiiuae	106	tiiuae/Falcon-H1-1.5B-Deep-Instruct	54.43
Falcon-H1-1.5B-Deep-Instruct-GGUF	medium	BBH	falcon-h1-1-5b-deep-instruct	unsloth	107	tiiuae/Falcon-H1-1.5B-Deep-Instruct	54.43
Falcon-H1-1.5B-Deep-Instruct	medium	BBH	falcon-h1-1-5b-deep-instruct	unsloth	108	tiiuae/Falcon-H1-1.5B-Deep-Instruct	54.43
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4	medium	BBH	falcon-h1-1-5b-deep-instruct	tiiuae	109	tiiuae/Falcon-H1-1.5B-Deep-Instruct	54.43
Falcon-H1-3B-Instruct	medium	BBH	falcon-h1-3b-base	tiiuae	110	tiiuae/Falcon-H1-3B-Base	53.69
Falcon-H1-3B-Instruct-GGUF	medium	BBH	falcon-h1-3b-base	tiiuae	111	tiiuae/Falcon-H1-3B-Base	53.69
Falcon-H1-3B-Instruct-GGUF	medium	BBH	falcon-h1-3b-instruct	unsloth	112	tiiuae/Falcon-H1-3B-Instruct	53.69
Falcon-H1-3B-Base	medium	BBH	falcon-h1-3b-base	tiiuae	113	tiiuae/Falcon-H1-3B-Base	53.17
Yi-34B-200K-GGUF	high	Unknown	yi-34b-200k	TheBloke	114	01-ai/Yi-34B-200K	52.7
Yi-34B-200K-Llamafied-GGUF	high	Unknown	yi-34b-200k-llamafied	TheBloke	115	larryvrh/Yi-34B-200K-Llamafied	52.7
Falcon-H1-1.5B-Deep-Base	medium	BBH	falcon-h1-1-5b-deep-base	tiiuae	116	tiiuae/Falcon-H1-1.5B-Deep-Base	52.37
Hermes-3-Llama-3.1-70B-lorablated	medium	BBH (3-Shot)	hermes-3-llama-3-1-70b	mlabonne	117	NousResearch/Hermes-3-Llama-3.1-70B	52.34
Yi-34B-Chat-GGUF	high	Unknown	yi-34b-chat	TheBloke	118	01-ai/Yi-34B-Chat	51.41
Yi-34B-Chat-AWQ	high	Unknown	yi-34b-chat	TheBloke	119	01-ai/Yi-34B-Chat	51.41
Nanbeige-16B-Base-GGUF	high	Unknown	Unknown	TheBloke	120	Nanbeige/Nanbeige-16B-Base	50.68
open-instruct-human-mix-65B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	121	allenai/open-instruct-human-mix-65b	50.1
open-instruct-human-mix-65b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	122	allenai/open-instruct-human-mix-65b	50.1
LFM2.5-Audio-1.5B	high	Unknown	lfm2-1-2b	LiquidAI	123	LiquidAI/LFM2-1.2B	49.54
LFM2.5-Audio-1.5B-GGUF	high	Unknown	lfm2-1-2b	Mungert	124	LiquidAI/LFM2-1.2B	49.54
openchat-3.5-1210-GGUF	high	BBH CoT	openchat-3-5-1210	TheBloke	125	openchat/openchat-3.5-1210	49.5
openchat-3.5-1210-GPTQ	high	BBH CoT	openchat-3-5-1210	TheBloke	126	openchat/openchat-3.5-1210	49.5
OpenOrcaxOpenChat-Preview2-13B-GPTQ	medium	Unknown	Unknown	TheBloke	127	Open-Orca/OpenOrcaxOpenChat-Preview2-13B	48.8
WizardLM-2-8x22B	medium	BBH (3-Shot)	Unknown	alpindale	128	—	48.58
tulu-65b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	129	allenai/tulu-65b	48.1
openchat-3.5-0106-GGUF	high	BBH CoT	openchat-3-5-0106	TheBloke	130	openchat/openchat-3.5-0106	47.6
openchat_3.5-GGUF	low	BBH CoT	Unknown	TheBloke	131	openchat/openchat_3.5	47.6
openchat_3.5-16k-GGUF	low	BBH CoT	Unknown	TheBloke	132	NurtureAI/openchat_3.5-16k	47.6
openchat_3.5-GPTQ	low	BBH CoT	Unknown	TheBloke	133	openchat/openchat_3.5	47.6
flan-ul2	low	MMLU-CoT \| BBH-CoT	Unknown	google	134	google/flan-ul2	47.5
Falcon-H1-1.5B-Base	medium	BBH	falcon-h1-1-5b-base	tiiuae	135	tiiuae/Falcon-H1-1.5B-Base	46.57
Falcon-H1-1.5B-Instruct	medium	BBH	falcon-h1-1-5b-base	tiiuae	136	tiiuae/Falcon-H1-1.5B-Base	46.47
Falcon-H1-1.5B-Instruct-GGUF	medium	BBH	falcon-h1-1-5b-instruct	tiiuae	137	tiiuae/Falcon-H1-1.5B-Instruct	46.47
Falcon-H1-1.5B-Instruct-GGUF	medium	BBH	falcon-h1-1-5b-instruct	unsloth	138	tiiuae/Falcon-H1-1.5B-Instruct	46.47
Falcon-H1-1.5B-Instruct-GPTQ-Int4	medium	BBH	falcon-h1-1-5b-instruct	tiiuae	139	tiiuae/Falcon-H1-1.5B-Instruct	46.47
Falcon-H1-1.5B-Instruct-GPTQ-Int8	medium	BBH	falcon-h1-1-5b-instruct	tiiuae	140	tiiuae/Falcon-H1-1.5B-Instruct	46.47
Hunyuan-7B-Instruct-GGUF	medium	BBH	hunyuan-7b-pretrain	Mungert	141	tencent/Hunyuan-7B-Pretrain	45.92
tulu-30B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	142	allenai/tulu-30b	45.8
tulu-30B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	143	—	45.8
tulu-30b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	144	allenai/tulu-30b	45.8
tulu-30B-GPTQ	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	145	allenai/tulu-30b	45.8
Flex-math-2x7B-1T	high	Unknown	flex-math-2x7b-1t	allenai	146	allenai/Flex-math-2x7B-1T	45.6
Phi-tiny-MoE-instruct	high	Unknown	phi-tiny-moe-instruct	microsoft	147	microsoft/Phi-tiny-MoE-instruct	45.58
Qwen2-57B-A14B	medium	BBH	qwen2-57b-a14b	Qwen	148	Qwen/Qwen2-57B-A14B	45.4
Falcon3-10B-Instruct	medium	BBH (3-Shot)	falcon3-10b-base	tiiuae	149	tiiuae/Falcon3-10B-Base	44.82
gemma-3n-E2B-it	medium	[BIG-Bench Hard][]	gemma-3n-e4b	google	150	google/gemma-3n-E4B-it	44.3
gemma-3n-E4B-it	medium	[BIG-Bench Hard][]	gemma-3n-e4b	google	151	google/gemma-3n-E4B	44.3
gemma-3n-E4B-it-litert-lm	medium	[BIG-Bench Hard][]	Unknown	google	152	google/gemma-3n-E4B-it-litert-lm	44.3
gemma-3n-E2B-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e2b-it	unsloth	153	google/gemma-3n-E2B-it	44.3
gemma-3n-E4B-it	medium	[BIG-Bench Hard][]	gemma-3n-e4b	unsloth	154	google/gemma-3n-E4B-it	44.3
gemma-3n-E2B	medium	[BIG-Bench Hard][]	gemma-3n-e4b	google	155	google/gemma-3n-E4B	44.3
gemma-3n-E2B-it-litert-lm	medium	[BIG-Bench Hard][]	Unknown	google	156	google/gemma-3n-E2B-it-litert-lm	44.3
gemma-3n-E4B-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b	unsloth	157	google/gemma-3n-E4B-it	44.3
gemma-3n-E4B	medium	[BIG-Bench Hard][]	gemma-3n-e4b	google	158	google/gemma-3n-E4B	44.3
gemma-3n-E2B-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3n-e2b-it	unsloth	159	google/gemma-3n-E2B-it	44.3
gemma-3n-E2B-it-litert-preview-GGUF	medium	[BIG-Bench Hard][]	Unknown	unsloth	160	google/gemma-3n-E2B-it-litert-preview	44.3
gemma-3n-E4B-it-litert-preview-GGUF	medium	[BIG-Bench Hard][]	Unknown	unsloth	161	google/gemma-3n-E4B-it-litert-preview	44.3
gemma-3n-E4B-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3n-e4b	unsloth	162	google/gemma-3n-E4B-it	44.3
gemma-3n-E2B-it-ONNX	medium	[BIG-Bench Hard][]	gemma-3n-e2b-it	onnx-community	163	google/gemma-3n-E2B-it	44.3
gemma-3n-E4B-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b	Mungert	164	google/gemma-3n-E4B	44.3
gemma-3n-E4B-it-absolute-heresy-MPOA-imatrix-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b-it-absolute-heresy-mpoa	MuXodious	165	MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA	44.3
gemma-3n-E4B-it-absolute-heresy-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b	MuXodious	166	google/gemma-3n-E4B-it	44.3
gemma-3n-E4B-it-absolute-heresy-MPOA-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b-it-absolute-heresy-mpoa	MuXodious	167	MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA	44.3
gemma-3n-E2B-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3n-e4b	Mungert	168	google/gemma-3n-E4b-it	44.3
Yi-34B-GGUF	low	Unknown	yi-34b	TheBloke	169	01-ai/Yi-34B	44.1
Qwen2-0.5B	medium	BBH	qwen2-0-5b	Qwen	170	Qwen/Qwen2-0.5B	43.4
Qwen2-1.5B	medium	BBH	qwen2-1-5b	Qwen	171	Qwen/Qwen2-1.5B	43.4
Falcon-H1-0.5B-Instruct	medium	BBH	falcon-h1-0-5b-base	tiiuae	172	tiiuae/Falcon-H1-0.5B-Base	42.91
Falcon-H1-0.5B-Instruct-GGUF	medium	BBH	falcon-h1-0-5b-base	tiiuae	173	tiiuae/Falcon-H1-0.5B-Base	42.91
Falcon-H1-0.5B-Instruct-GGUF	medium	BBH	falcon-h1-0-5b-instruct	unsloth	174	tiiuae/Falcon-H1-0.5B-Instruct	42.91
Yi-6B-200K-GGUF	high	Unknown	yi-6b-200k	TheBloke	175	01-ai/Yi-6B-200K	42.8
Yi-6B-GGUF	high	Unknown	yi-6b	TheBloke	176	01-ai/Yi-6B	42.8
Llama-3.1-Tulu-3-70B-SFT	medium	BBH (3-Shot)	llama-3-1-70b	allenai	177	meta-llama/Llama-3.1-70B	42.02
internlm2_5-1_8b-chat	medium	BBH (3-shot CoT)	internlm2-5-1-8b-chat	internlm	178	internlm/internlm2_5-1_8b-chat	41.9
Falcon3-10B-Base	medium	BBH (3-Shot)	falcon3-10b-base	tiiuae	179	tiiuae/Falcon3-10B-Base	41.38
tulu-13B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	180	allenai/tulu-13b	41.3
tulu-13B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	181	—	41.3
internlm2_5-1_8b	medium	BBH	Unknown	internlm	182	internlm/internlm2_5-1_8b	41.16
Falcon-H1-0.5B-Base	medium	BBH	falcon-h1-0-5b-base	tiiuae	183	tiiuae/Falcon-H1-0.5B-Base	40.22
DeepSeek-V2-Lite-Chat	medium	BBH	deepseek-v2-lite-chat	deepseek-ai	184	deepseek-ai/DeepSeek-V2-Lite-Chat	39.5
DeepSeek-V2-Lite	medium	BBH	deepseek-v2-lite	deepseek-ai	185	deepseek-ai/DeepSeek-V2-Lite	39.5
OLMoE-1B-7B-0125-Instruct	medium	BigBenchHard (CoT)	olmoe-1b-7b-0125-dpo	allenai	186	allenai/OLMoE-1B-7B-0125-DPO	38.61
OLMoE-1B-7B-0125-DPO	medium	BigBenchHard (CoT)	Unknown	allenai	187	allenai/OLMoE-1B-7B-0125-SFT	38.61
OLMoE-1B-7B-0125-SFT	medium	BigBenchHard (CoT)	olmoe-1b-7b-0125	allenai	188	allenai/OLMoE-1B-7B-0125	38.61
open-instruct-flan-v2-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	189	allenai/open-instruct-flan-v2-7b	38.6
open-instruct-cot-13b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	190	allenai/open-instruct-cot-13b	38.6
open-instruct-sni-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	191	allenai/open-instruct-sni-7b	38.4
Qwen-72B	low	Unknown	qwen-72b	Qwen	192	Qwen/Qwen-72B	38.2
Qwen-7B	low	Unknown	qwen-7b	Qwen	193	Qwen/Qwen-7B	38.2
Qwen-14B	low	Unknown	qwen-14b	Qwen	194	Qwen/Qwen-14B	38.2
tulu-7B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	195	allenai/tulu-7b	38.1
Flex-code-2x7B-1T	high	Unknown	flex-code-2x7b-1t	allenai	196	allenai/Flex-code-2x7B-1T	38.1
tulu-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	197	allenai/tulu-7b	38.1
tulu-7B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	198	—	38.1
Genz-70b-GGUF	low	Unknown	Unknown	TheBloke	199	budecosystem/genz-70b	37.76
genz-13B-v2-GGUF	low	Unknown	Unknown	TheBloke	200	budecosystem/genz-13b-v2	37.76
internlm2-chat-1_8b	medium	BBH	internlm2-chat-1-8b	internlm	201	internlm/internlm2-chat-1_8b	37.5
internlm2-1_8b	medium	BBH	Unknown	internlm	202	internlm/internlm2-1_8b	37.5
internlm2-chat-1_8b-sft	medium	BBH	internlm2-chat-1-8b-sft	internlm	203	internlm/internlm2-chat-1_8b-sft	37.5
FlexOlmo-7x7B-1T	low	Unknown	flexolmo-7x7b-1t	allenai	204	allenai/FlexOlmo-7x7B-1T	35.7
FlexOlmo-7x7B-1T-RT	low	Unknown	flexolmo-7x7b-1t-rt	allenai	205	allenai/FlexOlmo-7x7B-1T-RT	35.7
Flex-pes2o-2x7B-1T	low	Unknown	flex-pes2o-2x7b-1t	allenai	206	allenai/Flex-pes2o-2x7B-1T	35.7
Flex-creative-2x7B-1T	low	Unknown	flex-creative-2x7b-1t	allenai	207	allenai/Flex-creative-2x7B-1T	35.7
Flex-public-7B-1T	low	Unknown	flex-public-7b-1t	allenai	208	allenai/Flex-public-7B-1T	35.7
open-instruct-human-mix-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	209	allenai/open-instruct-human-mix-7b	35.6
open-instruct-cot-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	210	allenai/open-instruct-cot-7b	33.7
open-instruct-stanford-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	211	allenai/open-instruct-stanford-alpaca-7b	32.6
SmolLM2-1.7B-Instruct-GGUF	medium	BBH (3-shot)	smollm2-1-7b	Mungert	212	HuggingFaceTB/SmolLM2-1.7B	32.2
open-instruct-unnatural-instructions-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	213	allenai/open-instruct-unnatural-instructions-7b	31.4
open-instruct-pythia-6.9b-tulu	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	214	allenai/open-instruct-pythia-6.9b-tulu	31.3
Youtu-LLM-2B-GGUF	medium	BBH	youtu-llm-2b-base	Mungert	215	tencent/Youtu-LLM-2B-Base	31
Flex-news-2x7B-1T	high	Unknown	flex-news-2x7b-1t	allenai	216	allenai/Flex-news-2x7B-1T	30.9
open-instruct-baize-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	217	allenai/open-instruct-baize-7b	30.6
LFM2-Audio-1.5B	high	Unknown	lfm2-1-2b	LiquidAI	218	LiquidAI/LFM2-1.2B	30.54
Llama-2-7b-hf	low	Unknown	llama-2-7b-hf	meta-llama	219	meta-llama/Llama-2-7b-hf	30.3
Llama-2-7b-chat-hf	low	Unknown	llama-2-7b-chat-hf	meta-llama	220	meta-llama/Llama-2-7b-chat-hf	30.3
Llama-2-13b-chat-hf	low	Unknown	llama-2-13b-chat-hf	meta-llama	221	meta-llama/Llama-2-13b-chat-hf	30.3
Llama-2-7B-Chat-GGUF	low	Unknown	llama-2-7b-chat-hf	TheBloke	222	meta-llama/Llama-2-7b-chat-hf	30.3
Llama-2-7B-Chat-GPTQ	low	Unknown	llama-2-7b-chat-hf	TheBloke	223	meta-llama/Llama-2-7b-chat-hf	30.3
Llama-2-13b-hf	low	Unknown	llama-2-13b-hf	meta-llama	224	meta-llama/Llama-2-13b-hf	30.3
Llama-2-7B-GGUF	low	Unknown	llama-2-7b-hf	TheBloke	225	meta-llama/Llama-2-7b-hf	30.3
Llama-2-70b-chat-hf	low	Unknown	llama-2-70b-chat-hf	meta-llama	226	meta-llama/Llama-2-70b-chat-hf	30.3
Llama-2-70b-hf	low	Unknown	llama-2-70b-hf	meta-llama	227	meta-llama/Llama-2-70b-hf	30.3
Llama-2-13B-chat-GGUF	low	Unknown	llama-2-13b-chat-hf	TheBloke	228	meta-llama/Llama-2-13b-chat-hf	30.3
Llama-2-7B-GPTQ	low	Unknown	llama-2-7b-hf	TheBloke	229	meta-llama/Llama-2-7b-hf	30.3
Llama-2-70B-AWQ	low	Unknown	llama-2-70b-hf	TheBloke	230	meta-llama/Llama-2-70b-hf	30.3
Speechless-Llama2-Hermes-Orca-Platypus-WizardLM-13B-GGUF	low	Unknown	Unknown	TheBloke	231	uukuguy/speechless-llama2-hermes-orca-platypus-wizardlm-13b	30.3
Llama-2-7B-AWQ	low	Unknown	llama-2-7b-hf	TheBloke	232	meta-llama/Llama-2-7b-hf	30.3
Llama-2-70B-Chat-GGUF	low	Unknown	llama-2-70b-chat-hf	TheBloke	233	meta-llama/Llama-2-70b-chat-hf	30.3
Llama-2-7B-Chat-AWQ	low	Unknown	llama-2-7b-chat-hf	TheBloke	234	meta-llama/Llama-2-7b-chat-hf	30.3
Llama-2-70B-Chat-GPTQ	low	Unknown	llama-2-70b-chat-hf	TheBloke	235	meta-llama/Llama-2-70b-chat-hf	30.3
Llama-2-7B-Chat-fp16	low	Unknown	Unknown	TheBloke	236	—	30.3
Llama-2-13B-GGUF	low	Unknown	llama-2-13b-hf	TheBloke	237	meta-llama/Llama-2-13b-hf	30.3
Speechless-Llama2-13B-GGUF	low	Unknown	speechless-llama2-13b	TheBloke	238	uukuguy/speechless-llama2-13b	30.3
Llama-2-70B-GGUF	low	Unknown	llama-2-70b-hf	TheBloke	239	meta-llama/Llama-2-70b-hf	30.3
Llama-2-13B-chat-GPTQ	low	Unknown	llama-2-13b-chat-hf	TheBloke	240	meta-llama/Llama-2-13b-chat-hf	30.3
Llama-2-70B-GPTQ	low	Unknown	llama-2-70b-hf	TheBloke	241	meta-llama/Llama-2-70b-hf	30.3
Llama-2-13B-fp16	low	Unknown	Unknown	TheBloke	242	—	30.3
Llama-2-70B-Chat-AWQ	low	Unknown	llama-2-70b-chat-hf	TheBloke	243	meta-llama/Llama-2-70b-chat-hf	30.3
Llama-2-7B-Chat-GGML	low	Unknown	llama-2-7b-chat-hf	TheBloke	244	meta-llama/Llama-2-7b-chat-hf	30.3
Llama-2-13B-chat-AWQ	low	Unknown	llama-2-13b-chat-hf	TheBloke	245	meta-llama/Llama-2-13b-chat-hf	30.3
Llama-2-13B-GPTQ	low	Unknown	llama-2-13b-hf	TheBloke	246	meta-llama/Llama-2-13b-hf	30.3
Llama-2-70B-fp16	low	Unknown	Unknown	TheBloke	247	—	30.3
Llama-2-7b	low	Unknown	Unknown	meta-llama	248	meta-llama/Llama-2-7b	30.3
Llama-2-13B-chat-GGML	low	Unknown	llama-2-13b-chat-hf	TheBloke	249	meta-llama/Llama-2-13b-chat-hf	30.3
Llama-2-7B-GGML	low	Unknown	llama-2-7b-hf	TheBloke	250	meta-llama/Llama-2-7b-hf	30.3
Llama-2-7b-chat-hf-GGUF	low	Unknown	Unknown	Mungert	251	meta-llama/Llama-2-7b	30.3
Llama-2-13B-AWQ	low	Unknown	llama-2-13b-hf	TheBloke	252	meta-llama/Llama-2-13b-hf	30.3
Llama-2-7b-chat	low	Unknown	Unknown	meta-llama	253	meta-llama/Llama-2-7b-chat	30.3
Llama-2-13b	low	Unknown	Unknown	meta-llama	254	meta-llama/Llama-2-13b	30.3
Llama-2-70b	low	Unknown	Unknown	meta-llama	255	meta-llama/Llama-2-70b	30.3
Llama-2-13b-chat	low	Unknown	Unknown	meta-llama	256	meta-llama/Llama-2-13b-chat	30.3
Llama-2-70b-chat	low	Unknown	Unknown	meta-llama	257	meta-llama/Llama-2-70b-chat	30.3
OLMo-2-0425-1B-Instruct	low	Unknown	olmo-2-0425-1b-rlvr1	allenai	258	allenai/OLMo-2-0425-1B-RLVR1	29.9
OLMo-2-0425-1B-SFT	low	Unknown	olmo-2-0425-1b	allenai	259	allenai/OLMo-2-0425-1B	29.9
OLMo-2-0425-1B-DPO	low	Unknown	Unknown	allenai	260	allenai/OLMo-2-0425-1B-SFT	29.9
OLMo-2-0425-1B-Instruct-GGUF	low	Unknown	olmo-2-0425-1b-instruct	unsloth	261	allenai/OLMo-2-0425-1B-Instruct	29.9
OLMo-2-0425-1B-Instruct	low	Unknown	olmo-2-0425-1b-instruct	unsloth	262	allenai/OLMo-2-0425-1B-Instruct	29.9
OLMo-2-0425-1B-RLVR1	low	Unknown	Unknown	allenai	263	allenai/OLMo-2-0425-1B-DPO	29.9
open-instruct-self-instruct-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	264	allenai/open-instruct-self-instruct-7b	29.9
open-instruct-code-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	265	allenai/open-instruct-code-alpaca-7b	29.6
Llama-3.1-8B-ArliAI-RPMax-v1.1	medium	BBH (3-Shot)	llama-3-1-8b-arliai-rpmax-v1-1	ArliAI	266	ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1	28.79
open-instruct-gpt4-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	267	allenai/open-instruct-gpt4-alpaca-7b	28.5
ChimeraLlama-3-8B-v2	medium	BBH (3-Shot)	meta-llama-3-8b-instruct	mlabonne	268	NousResearch/Meta-Llama-3-8B-Instruct	28.48
gemma-3-1b-it	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	google	269	google/gemma-3-1b-pt	28.4
gemma-3-4b-it	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	google	270	google/gemma-3-4b-pt	28.4
gemma-3-12b-it	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	google	271	google/gemma-3-12b-pt	28.4
gemma-3-27b-it	medium	[BIG-Bench Hard][]	gemma-3-27b-pt	google	272	google/gemma-3-27b-pt	28.4
gemma-3-4b-it	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	273	google/gemma-3-4b-it	28.4
gemma-3-1b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	ggml-org	274	google/gemma-3-1b-it	28.4
gemma-3-12b-it-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	google	275	google/gemma-3-12b-it	28.4
gemma-3-1b-it	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	276	google/gemma-3-1b-it	28.4
gemma-3-12b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	277	google/gemma-3-12b-it	28.4
gemma-3-4b-pt	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	google	278	google/gemma-3-4b-pt	28.4
gemma-3-12b-it-qat-GGUF	medium	[BIG-Bench Hard][]	gemma-3-12b-it-qat-q4-0-unquantized	unsloth	279	google/gemma-3-12b-it-qat-q4_0-unquantized	28.4
gemma-3-27b-it	medium	[BIG-Bench Hard][]	gemma-3-27b-it	unsloth	280	google/gemma-3-27b-it	28.4
gemma-3-1b-pt	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	google	281	google/gemma-3-1b-pt	28.4
gemma-3-4b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	ggml-org	282	google/gemma-3-4b-it	28.4
gemma-3-4b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	283	google/gemma-3-4b-it	28.4
gemma-3-1b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	284	google/gemma-3-1b-it	28.4
gemma-3-12b-pt	medium	[BIG-Bench Hard][]	Unknown	google	285	google/gemma-3-12b	28.4
gemma-3-12b-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	286	google/gemma-3-12b-it	28.4
gemma-3-4b-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	287	google/gemma-3-4b-it	28.4
gemma-3-12b-it	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	288	google/gemma-3-12b-pt	28.4
gemma-3-27b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-27b-it	unsloth	289	google/gemma-3-27b-it	28.4
gemma-3-1b-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	290	google/gemma-3-1b-it	28.4
gemma-3-4b-it-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	291	google/gemma-3-4b-it	28.4
gemma-3-4b-pt	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	292	google/gemma-3-4b-pt	28.4
gemma-3-27b-pt	medium	[BIG-Bench Hard][]	gemma-3-27b-pt	google	293	google/gemma-3-27b-pt	28.4
gemma-3-12b-it-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	294	google/gemma-3-12b-it	28.4
gemma-3-27b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-27b-it	ggml-org	295	google/gemma-3-27b-it	28.4
gemma-3-12b-pt	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	296	google/gemma-3-12b-pt	28.4
gemma-3-27b-it-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-27b-it	unsloth	297	google/gemma-3-27b-it	28.4
gemma-3-27b-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-27b-it	unsloth	298	google/gemma-3-27b-it	28.4
gemma-3-1b-pt	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	299	google/gemma-3-1b-pt	28.4
gemma-3-12b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	ggml-org	300	google/gemma-3-12b-it	28.4
gemma-3-4b-it-qat-GGUF	medium	[BIG-Bench Hard][]	gemma-3-4b-it-qat-q4-0-unquantized	unsloth	301	google/gemma-3-4b-it-qat-q4_0-unquantized	28.4
gemma-3-12b-it-qat-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-it-qat-q4-0-unquantized	unsloth	302	google/gemma-3-12b-it-qat-q4_0-unquantized	28.4
gemma-3-12b-it-qat-int4-GGUF	medium	[BIG-Bench Hard][]	gemma-3-12b-it-qat-int4-unquantized	unsloth	303	google/gemma-3-12b-it-qat-int4-unquantized	28.4
gemma-3-4b-it-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	google	304	google/gemma-3-4b-it	28.4
gemma-3-4b-it-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	google	305	google/gemma-3-4b-it	28.4
gemma-3-4b-pt-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	306	google/gemma-3-4b-pt	28.4
gemma-3-27b-it-qat-GGUF	medium	[BIG-Bench Hard][]	gemma-3-27b-it-qat-q4-0-unquantized	unsloth	307	google/gemma-3-27b-it-qat-q4_0-unquantized	28.4
gemma-3-12b-it-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	google	308	google/gemma-3-12b-it	28.4
gemma-3-1b-it-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	309	google/gemma-3-1b-it	28.4
gemma-3-1b-it-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	google	310	google/gemma-3-1b-it	28.4
gemma-3-4b-pt-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	311	google/gemma-3-4b-pt	28.4
gemma-3-12b-it-qat	medium	[BIG-Bench Hard][]	gemma-3-12b-it-qat-q4-0-unquantized	unsloth	312	google/gemma-3-12b-it-qat-q4_0-unquantized	28.4
gemma-3-1b-pt-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	313	google/gemma-3-1b-pt	28.4
gemma-3-4b-it-FP8-Dynamic	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	unsloth	314	google/gemma-3-4b-it	28.4
gemma-3-12b-pt-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	315	google/gemma-3-12b-pt	28.4
gemma-3-12b-it-qat-int4-unquantized	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	google	316	google/gemma-3-12b-it	28.4
gemma-3-27b-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-27b-pt	Mungert	317	google/gemma-3-27b-pt	28.4
gemma-3-27b-it-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-27b-it	google	318	google/gemma-3-27b-it	28.4
gemma-3-12b-pt-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	319	google/gemma-3-12b-pt	28.4
gemma-3-27b-it-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-27b-it	google	320	google/gemma-3-27b-it	28.4
gemma-3-27b-it-qat-q4_0-GGUF	medium	[BIG-Bench Hard][]	gemma-3-27b-it	Mungert	321	google/gemma-3-27b-it	28.4
gemma-3-4b-it-qat-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-4b-it-qat-q4-0-unquantized	unsloth	322	google/gemma-3-4b-it-qat-q4_0-unquantized	28.4
gemma-3-1b-pt-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	unsloth	323	google/gemma-3-1b-pt	28.4
gemma-3-27b-pt	medium	[BIG-Bench Hard][]	gemma-3-27b-pt	unsloth	324	google/gemma-3-27b-pt	28.4
gemma-3-12b-it-FP8-Dynamic	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	unsloth	325	google/gemma-3-12b-pt	28.4
gemma-3-1b-it-qat-int4-unquantized	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	google	326	google/gemma-3-1b-it	28.4
gemma-3-27b-pt-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-27b-pt	unsloth	327	google/gemma-3-27b-pt	28.4
gemma-3-12b-it-qat-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-12b-it-qat-q4-0-unquantized	unsloth	328	google/gemma-3-12b-it-qat-q4_0-unquantized	28.4
gemma-3-12b-pt-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-12b-pt	google	329	google/gemma-3-12b-pt	28.4
gemma-3-4b-it-qat	medium	[BIG-Bench Hard][]	gemma-3-4b-it-qat-q4-0-unquantized	unsloth	330	google/gemma-3-4b-it-qat-q4_0-unquantized	28.4
gemma-3-1b-it-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-1b-pt	google	331	google/gemma-3-1b-it	28.4
gemma-3-1b-it-qat	medium	[BIG-Bench Hard][]	gemma-3-1b-it-qat-q4-0-unquantized	unsloth	332	google/gemma-3-1b-it-qat-q4_0-unquantized	28.4
gemma-3-27b-it-qat	medium	[BIG-Bench Hard][]	gemma-3-27b-it-qat-q4-0-unquantized	unsloth	333	google/gemma-3-27b-it-qat-q4_0-unquantized	28.4
gemma-3-4b-pt-qat-q4_0-gguf	medium	[BIG-Bench Hard][]	gemma-3-4b-pt	google	334	google/gemma-3-4b-pt	28.4
SmolLM2-135M-Instruct-GGUF	medium	BBH (3-shot)	smollm2-135m	Mungert	335	HuggingFaceTB/SmolLM2-135M	28.2
ChimeraLlama-3-8B-v3	medium	BBH (3-Shot)	meta-llama-3-8b-instruct	mlabonne	336	NousResearch/Meta-Llama-3-8B-Instruct	27.65
SmolLM2-360M-Instruct-GGUF	medium	BBH (3-shot)	smollm2-360m	Mungert	337	HuggingFaceTB/SmolLM2-360M	27.3
open-instruct-dolly-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	338	allenai/open-instruct-dolly-7b	27.2
Meta-Llama-3.1-8B-Instruct-abliterated	medium	BBH (3-Shot)	meta-llama-3-1-8b-instruct	mlabonne	339	meta-llama/Meta-Llama-3.1-8B-Instruct	27.13
gemma-3-270m	medium	[BIG-Bench Hard][]	gemma-3-270m	google	340	google/gemma-3-270m	26.7
gemma-3-270m-it	medium	[BIG-Bench Hard][]	gemma-3-270m	google	341	google/gemma-3-270m	26.7
gemma-3-270m-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	342	google/gemma-3-270m-it	26.7
gemma-3-270m-it	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	343	google/gemma-3-270m-it	26.7
gemma-3-270m-it-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	344	google/gemma-3-270m-it	26.7
gemma-3-270m-it-qat-GGUF	medium	[BIG-Bench Hard][]	gemma-3-270m-it-qat-q4-0-unquantized	unsloth	345	google/gemma-3-270m-it-qat-q4_0-unquantized	26.7
gemma-3-270m	medium	[BIG-Bench Hard][]	gemma-3-270m	unsloth	346	google/gemma-3-270m	26.7
gemma-3-270m-unsloth-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	347	google/gemma-3-270m-it	26.7
gemma-3-270m-it-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-270m-it	google	348	google/gemma-3-270m-it	26.7
gemma-3-270m-it-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	349	google/gemma-3-270m-it	26.7
gemma-3-270m-it-qat	medium	[BIG-Bench Hard][]	gemma-3-270m-it-qat-q4-0-unquantized	unsloth	350	google/gemma-3-270m-it-qat-q4_0-unquantized	26.7
gemma-3-270m-bnb-4bit	medium	[BIG-Bench Hard][]	gemma-3-270m	unsloth	351	google/gemma-3-270m	26.7
gemma-3-270m-qat-q4_0-unquantized	medium	[BIG-Bench Hard][]	gemma-3-270m	google	352	google/gemma-3-270m	26.7
gemma-3-270m-it-GGUF	medium	[BIG-Bench Hard][]	gemma-3-270m	Mungert	353	google/gemma-3-270m	26.7
gemma-3-270m-it-torchao-FP8	medium	[BIG-Bench Hard][]	gemma-3-270m-it	unsloth	354	google/gemma-3-270m-it	26.7
Falcon-E-3B-Instruct	high	Unknown	falcon-e-3b-instruct	tiiuae	355	tiiuae/Falcon-E-3B-Instruct	26.45
Falcon-E-3B-Instruct-GGUF	high	Unknown	falcon-e-3b-base	tiiuae	356	tiiuae/Falcon-E-3B-Base	26.45
Mistral-Nemo-12B-ArliAI-RPMax-v1.1	medium	BBH (3-Shot)	mistral-nemo-12b-arliai-rpmax-v1-1	ArliAI	357	ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.1	24.81
t5gemma-2-270m-270m	medium	[BIG-Bench Hard]	t5gemma-2-270m-270m	google	358	google/t5gemma-2-270m-270m	22.9
t5gemma-2-4b-4b	medium	[BIG-Bench Hard]	t5gemma-2-4b-4b	google	359	google/t5gemma-2-4b-4b	22.9
t5gemma-2-1b-1b	medium	[BIG-Bench Hard]	t5gemma-2-1b-1b	google	360	google/t5gemma-2-1b-1b	22.9
open-instruct-oasst1-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	361	allenai/open-instruct-oasst1-7b	20.4
falcon-mamba-7b	low	Unknown	falcon-mamba-7b	tiiuae	362	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b	medium	BBH (3-Shot)	falcon-mamba-7b	tiiuae	363	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-instruct	low	Unknown	falcon-mamba-7b	tiiuae	364	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-instruct-Q4_K_M-GGUF	low	Unknown	falcon-mamba-7b-instruct	tiiuae	365	tiiuae/falcon-mamba-7b-instruct	19.88
falcon-mamba-7b-instruct-4bit	low	Unknown	falcon-mamba-7b-instruct	tiiuae	366	tiiuae/falcon-mamba-7b-instruct	19.88
falcon-mamba-7b-Q8_0-GGUF	low	Unknown	falcon-mamba-7b	tiiuae	367	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-Q8_0-GGUF	medium	BBH (3-Shot)	falcon-mamba-7b	tiiuae	368	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-F16-GGUF	low	Unknown	falcon-mamba-7b	tiiuae	369	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-F16-GGUF	medium	BBH (3-Shot)	falcon-mamba-7b	tiiuae	370	tiiuae/falcon-mamba-7b	19.88
falcon-mamba-7b-4bit	low	Unknown	falcon-mamba-7b	tiiuae	371	tiiuae/falcon-mamba-7b	19.88
Falcon-E-3B-Base	high	Unknown	falcon-e-3b-base	tiiuae	372	tiiuae/Falcon-E-3B-Base	19.83
Falcon-E-3B-Base-prequantized	high	Unknown	falcon-e-3b-base-prequantized	tiiuae	373	tiiuae/Falcon-E-3B-Base-prequantized	19.83
Flex-reddit-2x7B-1T	high	Unknown	flex-reddit-2x7b-1t	allenai	374	allenai/Flex-reddit-2x7B-1T	19.7
Falcon-E-1B-Instruct-GGUF	high	Unknown	falcon-e-1b-base	tiiuae	375	tiiuae/Falcon-E-1B-Base	19.42
Falcon-E-1B-Instruct	high	Unknown	falcon-e-1b-instruct	tiiuae	376	tiiuae/Falcon-E-1B-Instruct	19.42
Llama-3.1-Tulu-3-8B	medium	BBH (3-Shot)	llama-3-1-tulu-3-8b-dpo	allenai	377	allenai/Llama-3.1-Tulu-3-8B-DPO	16.86
Falcon-E-1B-Base	high	Unknown	falcon-e-1b-base	tiiuae	378	tiiuae/Falcon-E-1B-Base	12.28
open-instruct-sharegpt-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	379	allenai/open-instruct-sharegpt-7b	5.2
OLMo-2-1124-7B-Instruct	low	Unknown	Unknown	allenai	380	allenai/OLMo-2-1124-7B-DPO	2.5
OLMo-2-1124-13B-Instruct	low	Unknown	olmo-2-1124-13b-instruct-rlvr2	allenai	381	allenai/OLMo-2-1124-13B-Instruct-RLVR2	2.5
OLMo-2-1124-7B-SFT	low	Unknown	olmo2-7b-1124	allenai	382	allenai/OLMo2-7B-1124	2.5
OLMo-2-1124-7B-DPO	low	Unknown	Unknown	allenai	383	allenai/OLMo-2-1124-7B-SFT	2.5
OLMo-2-1124-13B-SFT	low	Unknown	olmo2-13b-1124	allenai	384	allenai/OLMo2-13B-1124	2.5
OLMo-2-1124-13B-DPO	low	Unknown	Unknown	allenai	385	allenai/OLMo-2-1124-13B-SFT	2.5
OLMo-2-1124-7B-Instruct-preview	low	Unknown	Unknown	allenai	386	allenai/OLMo-2-1124-7B-DPO	2.5
OLMo-2-1124-7B-RM	low	Unknown	Unknown	allenai	387	allenai/OLMo-2-1124-7B-SFT	2.5
OLMo-2-1124-7B-RM-Preview	low	Unknown	Unknown	allenai	388	allenai/OLMo-2-1124-7B-SFT	2.5
OLMo-2-1124-13B-RM	low	Unknown	Unknown	allenai	389	allenai/OLMo-2-1124-13B-SFT	2.5
OLMo-2-1124-13B-Instruct-preview	low	Unknown	olmo-2-1124-13b-dpo	allenai	390	allenai/OLMo-2-1124-13B-DPO	2.5
OLMo-2-1124-7B-SFT-Preview	low	Unknown	olmo2-7b-1124	allenai	391	allenai/OLMo2-7B-1124	2.5
OLMo-2-1124-7B-DPO-Preview	low	Unknown	Unknown	allenai	392	allenai/OLMo-2-1124-7B-SFT	2.5

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/bbh.html
url: huggingface.co/spaces/bbh ↗