Leaderboard

OpenBookQA

openbookqa

Record

Category: knowledge
Scored models: 81

Scores

Model	Score
microsoft/GRIN-MoE	89.8
microsoft/Phi-3-small-8k-instruct	88
microsoft/Phi-3-small-128k-instruct	87.8
microsoft/Phi-3-medium-4k-instruct	87.4
microsoft/Phi-3-medium-128k-instruct	87.2
microsoft/Phi-4-mini-instruct	79.2
microsoft/Phi-4-mini-instruct	79.2
microsoft/Phi-4-mini-instruct	79.2
microsoft/Phi-4-mini-instruct	79.2
microsoft/Phi-4-mini-instruct	79.2
microsoft/Phi-4-mini-instruct	79.2
inclusionAI/Ming-Lite-Omni	65.93
inclusionAI/Ling-lite	61.98
mistralai/Mistral-Nemo-Base-2407	60.6
mistralai/Mistral-Nemo-Base-2407	60.6
mistralai/Mistral-Nemo-Base-2407	60.6
allenai/OLMo-1B-0724-hf	51.2
allenai/OLMo-7B-0424-hf	51.2
allenai/OLMo-7B-0724-hf	51.2
allenai/OLMo-7B-0424	51.2
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	50.2
allenai/OLMo-1B-hf	49.8
allenai/OLMo-7B	49.8
allenai/OLMo-7B-hf	49.8

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: knowledge
id: openbookqa
name: OpenBookQA
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
GRIN-MoE	medium	OpenBookQA	grin-moe	microsoft	1	microsoft/GRIN-MoE	89.8
Phi-3-small-8k-instruct	medium	OpenBookQA<br>10-shot	phi-3-small-8k-instruct	microsoft	2	microsoft/Phi-3-small-8k-instruct	88
Phi-3-small-128k-instruct	medium	OpenBookQA<br>10-shot	phi-3-small-128k-instruct	microsoft	3	microsoft/Phi-3-small-128k-instruct	87.8
Phi-3-medium-4k-instruct	medium	OpenBookQA<br>10-shot	phi-3-medium-4k-instruct	microsoft	4	microsoft/Phi-3-medium-4k-instruct	87.4
Phi-3-medium-128k-instruct	medium	OpenBookQA<br>10-shot	phi-3-medium-128k-instruct	microsoft	5	microsoft/Phi-3-medium-128k-instruct	87.2
Phi-4-mini-instruct	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	microsoft	6	microsoft/Phi-4-mini-instruct	79.2
Phi-4-mini-instruct-GGUF	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	unsloth	7	microsoft/Phi-4-mini-instruct	79.2
Phi-4-mini-instruct-unsloth-bnb-4bit	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	unsloth	8	microsoft/Phi-4-mini-instruct	79.2
Phi-4-mini-instruct-bnb-4bit	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	unsloth	9	microsoft/Phi-4-mini-instruct	79.2
Phi-4-mini-instruct	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	unsloth	10	microsoft/Phi-4-mini-instruct	79.2
Phi-4-mini-instruct-GGUF	medium	OpenBookQA (10-shot)	phi-4-mini-instruct	Mungert	11	microsoft/Phi-4-mini-instruct	79.2
Ming-Lite-Omni-1.5	low	Average(Open-ended QA)	ming-lite-omni	inclusionAI	12	inclusionAI/Ming-Lite-Omni	65.93
Ming-Lite-Omni	high	Unknown	ling-lite	inclusionAI	13	inclusionAI/Ling-lite	61.98
Mistral-Nemo-Instruct-2407	medium	OpenBookQA (0-shot)	mistral-nemo-base-2407	mistralai	14	mistralai/Mistral-Nemo-Base-2407	60.6
Mistral-Nemo-Base-2407	medium	OpenBookQA (0-shot)	mistral-nemo-base-2407	mistralai	15	mistralai/Mistral-Nemo-Base-2407	60.6
Mistral-Nemo-Instruct-FP8-2407	medium	OpenBookQA (0-shot)	mistral-nemo-base-2407	mistralai	16	mistralai/Mistral-Nemo-Base-2407	60.6
OLMo-1B-0724-hf	medium	Unknown	olmo-1b-0724-hf	allenai	17	allenai/OLMo-1B-0724-hf	51.2
OLMo-7B-0424-hf	medium	Unknown	olmo-7b-0424-hf	allenai	18	allenai/OLMo-7B-0424-hf	51.2
OLMo-7B-0724-hf	medium	Unknown	olmo-7b-0724-hf	allenai	19	allenai/OLMo-7B-0724-hf	51.2
OLMo-7B-0424	medium	Unknown	Unknown	allenai	20	allenai/OLMo-7B-0424	51.2
NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	medium	OpenBookQA	nvidia-nemotron-3-super-120b-a12b-base-bf16	nvidia	21	nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	50.2
OLMo-1B-hf	medium	Unknown	olmo-1b-hf	allenai	22	allenai/OLMo-1B-hf	49.8
OLMo-7B	medium	Unknown	olmo-7b	allenai	23	allenai/OLMo-7B	49.8
OLMo-7B-hf	medium	Unknown	olmo-7b-hf	allenai	24	allenai/OLMo-7B-hf	49.8
OLMo-1B	medium	Unknown	olmo-1b	allenai	25	allenai/OLMo-1B	49.8
OLMo-7B-Twin-2T-hf	medium	Unknown	olmo-7b-twin-2t-hf	allenai	26	allenai/OLMo-7B-Twin-2T-hf	49.8
bagel-dpo-7B-v0.1-GGUF	low	Unknown	bagel-dpo-7b-v0-1	TheBloke	27	jondurbin/bagel-dpo-7b-v0.1	49.8
OLMo-7B-Twin-2T	medium	Unknown	olmo-7b-twin-2t	allenai	28	allenai/OLMo-7B-Twin-2T	49.8
NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16	medium	OpenBookQA	nvidia-nemotron-3-ultra-550b-a55b-base-bf16	nvidia	29	nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16	48.6
gemma-1.1-2b-it	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	gemma-1-1-2b-it	google	30	google/gemma-1.1-2b-it	47.8
gemma-2b	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	31	google/gemma-2b-it-pytorch	47.8
gemma-2b-it	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	32	google/gemma-2b-it-pytorch	47.8
gemma-7b	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	33	google/gemma-7b-it-quant-pytorch	47.8
gemma-7b-it	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	gemma-7b	google	34	google/gemma-7b	47.8
gemma-1.1-7b-it	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	gemma-1-1-7b-it	google	35	google/gemma-1.1-7b-it	47.8
gemma-2b-AWQ	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	36	google/gemma-2b-it-pytorch	47.8
gemma-7b	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	alpindale	37	google/gemma-7b-it-quant-pytorch	47.8
gemma-2b	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	alpindale	38	google/gemma-2b-it-pytorch	47.8
gemma-7b-AWQ	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	39	google/gemma-7b-it-quant-pytorch	47.8
gemma-7b-it	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	alpindale	40	google/gemma-7b-it-quant-pytorch	47.8
gemma-7b-GGUF	medium	[OpenBookQA](https://arxiv.org/abs/1809.02789)	Unknown	google	41	google/gemma-7b-it-quant-pytorch	47.8
NVIDIA-Nemotron-Nano-9B-v2-Base	medium	OpenBookQA	nvidia-nemotron-nano-9b-v2-base	nvidia	42	nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base	46
NVIDIA-Nemotron-Nano-12B-v2-Base	medium	OpenBookQA	nvidia-nemotron-nano-12b-v2-base	nvidia	43	nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base	46
NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16	medium	OpenBookQA (0-shot, acc_norm)	nvidia-nemotron-3-nano-30b-a3b-base-bf16	nvidia	44	nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16	44.8
SmolLM2-1.7B-Instruct-GGUF	medium	OpenBookQA	smollm2-1-7b	Mungert	45	HuggingFaceTB/SmolLM2-1.7B	42.2
Nous-Hermes-Llama2-70B-GGUF	medium	Unknown	Unknown	TheBloke	46	NousResearch/Nous-Hermes-Llama2-70b	38.8
Beyonder-4x7B-v2-GGUF	medium	Unknown	beyonder-4x7b-v2	TheBloke	47	mlabonne/Beyonder-4x7B-v2	37.8
Beyonder-4x7B-v2	medium	Unknown	marcoro14-7b-slerp	mlabonne	48	mlabonne/Marcoro14-7B-slerp	37.8
SmolLM2-360M-Instruct-GGUF	medium	OpenBookQA	smollm2-360m	Mungert	49	HuggingFaceTB/SmolLM2-360M	37.4
Marcoro14-7B-slerp	medium	Unknown	Unknown	mlabonne	50	AIDC-ai-business/Marcoroni-7B-v3	36.4
Nous-Hermes-13B-GGUF	medium	Unknown	Unknown	TheBloke	51	NousResearch/Nous-Hermes-13b	35.6
Tulpar-7B-v0-GGUF	medium	Unknown	Unknown	TheBloke	52	HyperbeeAI/Tulpar-7b-v0	35.6
Redmond-Puffin-13B-GGUF	medium	Unknown	Unknown	TheBloke	53	NousResearch/Redmond-Puffin-13B	35.6
Nous-Hermes-13B-SuperHOT-8K-fp16	medium	Unknown	Unknown	TheBloke	54	—	35.6
Redmond-Puffin-13B-GPTQ	medium	Unknown	Unknown	TheBloke	55	NousResearch/Redmond-Puffin-13B	35.6
Nous-Hermes-13B-SuperHOT-8K-GPTQ	medium	Unknown	Unknown	TheBloke	56	—	35.6
Nous-Hermes-2-Yi-34B-GGUF	medium	Unknown	nous-hermes-2-yi-34b	TheBloke	57	NousResearch/Nous-Hermes-2-Yi-34B	35.2
Nous-Hermes-2-Yi-34B-AWQ	medium	Unknown	nous-hermes-2-yi-34b	TheBloke	58	NousResearch/Nous-Hermes-2-Yi-34B	35.2
laser-dolphin-mixtral-2x7b-dpo-GGUF	medium	Unknown	laser-dolphin-mixtral-2x7b-dpo	TheBloke	59	macadeliccc/laser-dolphin-mixtral-2x7b-dpo	35
bitnet-b1.58-2B-4T-gguf	medium	OpenbookQA	bitnet-b1-58-2b-4t-bf16	microsoft	60	microsoft/bitnet-b1.58-2B-4T-bf16	34.8
bitnet-b1.58-2B-4T	medium	OpenbookQA	bitnet-b1-58-2b-4t	microsoft	61	microsoft/bitnet-b1.58-2B-4T	34.8
bitnet-b1.58-2B-4T-bf16	medium	OpenbookQA	bitnet-b1-58-2b-4t-bf16	microsoft	62	microsoft/bitnet-b1.58-2B-4T-bf16	34.8
OpenHermes-2-Mistral-7B-GGUF	medium	Unknown	Unknown	TheBloke	63	teknium/OpenHermes-2-Mistral-7B	34.8
dolly-v2-7b-olive-optimized	low	Unknown	Unknown	microsoft	64	microsoft/dolly-v2-7b-olive-optimized	34.8
OpenHermes-2-Mistral-7B-AWQ	medium	Unknown	Unknown	TheBloke	65	teknium/OpenHermes-2-Mistral-7B	34.8
OpenHermes-2.5-Mistral-7B-AWQ	medium	Unknown	openhermes-2-5-mistral-7b	TheBloke	66	teknium/OpenHermes-2.5-Mistral-7B	34.6
OpenHermes-2.5-Mistral-7B-GGUF	medium	Unknown	openhermes-2-5-mistral-7b	TheBloke	67	teknium/OpenHermes-2.5-Mistral-7B	34.6
OpenHermes-2.5-Mistral-7B-16k-GGUF	medium	Unknown	Unknown	TheBloke	68	NurtureAI/OpenHermes-2.5-Mistral-7B-16k	34.6
OpenHermes-2.5-Mistral-7B-GPTQ	medium	Unknown	openhermes-2-5-mistral-7b	TheBloke	69	teknium/OpenHermes-2.5-Mistral-7B	34.6
SmolLM2-135M-Instruct-GGUF	medium	OpenBookQA	smollm2-135m	Mungert	70	HuggingFaceTB/SmolLM2-135M	34.6
Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF	medium	Unknown	nous-hermes-2-mixtral-8x7b-dpo	TheBloke	71	NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO	34.4
DPOpenHermes-7B-GGUF	medium	Unknown	dpopenhermes-7b	TheBloke	72	openaccess-ai-collective/DPOpenHermes-7B	34.4
NeuralHermes-2.5-Mistral-7B-laser	medium	Unknown	openhermes-2-5-mistral-7b	mlabonne	73	teknium/OpenHermes-2.5-Mistral-7B	34.4
Nous-Hermes-2-Mixtral-8x7B-DPO-GPTQ	medium	Unknown	nous-hermes-2-mixtral-8x7b-dpo	TheBloke	74	NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO	34.4
Nous-Hermes-2-SOLAR-10.7B-GGUF	medium	Unknown	nous-hermes-2-solar-10-7b	TheBloke	75	NousResearch/Nous-Hermes-2-SOLAR-10.7B	34.2
minotaur-13B-fixed-GGUF	medium	Unknown	Unknown	TheBloke	76	openaccess-ai-collective/minotaur-13b-fixed	34.2
Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF	medium	Unknown	nous-hermes-2-mixtral-8x7b-sft	TheBloke	77	NousResearch/Nous-Hermes-2-Mixtral-8x7B-SFT	34
CollectiveCognition-v1-Mistral-7B-GGUF	medium	Unknown	Unknown	TheBloke	78	teknium/CollectiveCognition-v1-Mistral-7B	33.8
Llama2-7b-WhoIsHarryPotter	low	Unknown	Unknown	microsoft	79	microsoft/Llama2-7b-WhoIsHarryPotter	33.8
CollectiveCognition-v1.1-Mistral-7B-GGUF	medium	Unknown	Unknown	TheBloke	80	teknium/CollectiveCognition-v1.1-Mistral-7B	32.4
Redmond-Hermes-Coder-GPTQ	medium	Unknown	Unknown	TheBloke	81	—	19.6

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/openbookqa.html
url: huggingface.co/spaces/openbookqa ↗