Leaderboard

PopQA

popqa

Record

Category: knowledge
Scored models: 63

Scores

Model	Score
meta-llama/Llama-3.1-8B	55.7
allenai/Llama-3.1-Tulu-3-8B-SFT	55.7
allenai/Llama-3.1-Tulu-3-8B-SFT	55.7
allenai/Llama-3.1-Tulu-3-8B-DPO	55.7
allenai/Llama-3.1-Tulu-3-70B-DPO	55.7
meta-llama/Llama-3.1-70B	55.7
allenai/Llama-3.1-Tulu-3-405B-DPO	55.7
allenai/Llama-3.1-Tulu-3-70B-SFT	55.7
meta-llama/Llama-3.1-405B	55.7
allenai/Llama-3.1-Tulu-3-405B-SFT	55.7
Qwen/Qwen2.5-7B-Instruct	51.8
allenai/OLMo-2-0325-32B-DPO	45
allenai/OLMo-2-0325-32B-Instruct	45
allenai/OLMo-2-0325-32B	45
allenai/OLMo-2-0325-32B-SFT	45
allenai/OLMo-2-0325-32B-Instruct	45
allenai/Olmo-3-32B-Think-DPO	33.1
allenai/Olmo-3-1125-32B	33.1
allenai/Olmo-3-32B-Think	33.1
allenai/Olmo-3-32B-Think-SFT	33.1
allenai/Olmo-3-32B-Think	33.1
allenai/Olmo-3-32B-Think-DPO	30.9
allenai/Olmo-3.1-32B-Think	30.9
meta-llama/Llama-3.1-8B	29.3

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: knowledge
id: popqa
name: PopQA
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
Llama-3.1-Tulu-3-8B-SFT	medium	PopQA (3 shot)	llama-3-1-8b	allenai	1	meta-llama/Llama-3.1-8B	55.7
Llama-3.1-Tulu-3-8B-DPO	medium	PopQA (3 shot)	llama-3-1-tulu-3-8b-sft	allenai	2	allenai/Llama-3.1-Tulu-3-8B-SFT	55.7
Llama-3.1-Tulu-3-8B-RM	medium	PopQA (3 shot)	llama-3-1-tulu-3-8b-sft	allenai	3	allenai/Llama-3.1-Tulu-3-8B-SFT	55.7
Llama-3.1-Tulu-3.1-8B	medium	PopQA (3 shot)	llama-3-1-tulu-3-8b-dpo	allenai	4	allenai/Llama-3.1-Tulu-3-8B-DPO	55.7
Llama-3.1-Tulu-3-70B	medium	PopQA (3 shot)	llama-3-1-tulu-3-70b-dpo	allenai	5	allenai/Llama-3.1-Tulu-3-70B-DPO	55.7
Llama-3.1-Tulu-3-70B-SFT	medium	PopQA (3 shot)	llama-3-1-70b	allenai	6	meta-llama/Llama-3.1-70B	55.7
Llama-3.1-Tulu-3-405B	medium	PopQA (3 shot)	llama-3-1-405b	allenai	7	allenai/Llama-3.1-Tulu-3-405B-DPO	55.7
Llama-3.1-Tulu-3-70B-DPO	medium	PopQA (3 shot)	llama-3-1-tulu-3-70b-sft	allenai	8	allenai/Llama-3.1-Tulu-3-70B-SFT	55.7
Llama-3.1-Tulu-3-405B-SFT	medium	PopQA (3 shot)	llama-3-1-405b	allenai	9	meta-llama/Llama-3.1-405B	55.7
Llama-3.1-Tulu-3-405B-DPO	medium	PopQA (3 shot)	llama-3-1-405b	allenai	10	allenai/Llama-3.1-Tulu-3-405B-SFT	55.7
pokee_research_7b-GGUF	low	POPQA	qwen2-5-7b-instruct	Mungert	11	Qwen/Qwen2.5-7B-Instruct	51.8
OLMo-2-0325-32B-Instruct	low	PopQA	olmo-2-0325-32b	allenai	12	allenai/OLMo-2-0325-32B-DPO	45
OLMo-2-0325-32B-Instruct-GGUF	low	PopQA	olmo-2-0325-32b-instruct	unsloth	13	allenai/OLMo-2-0325-32B-Instruct	45
OLMo-2-0325-32B-SFT	low	PopQA	olmo-2-0325-32b	allenai	14	allenai/OLMo-2-0325-32B	45
OLMo-2-0325-32B-DPO	low	PopQA	olmo-2-0325-32b	allenai	15	allenai/OLMo-2-0325-32B-SFT	45
OLMo-2-0325-32B-Instruct	low	PopQA	olmo-2-0325-32b-instruct	unsloth	16	allenai/OLMo-2-0325-32B-Instruct	45
Olmo-3-32B-Think	medium	PopQA	olmo-3-32b-think-dpo	allenai	17	allenai/Olmo-3-32B-Think-DPO	33.1
Olmo-3-32B-Think-SFT	medium	PopQA	olmo-3-1125-32b	allenai	18	allenai/Olmo-3-1125-32B	33.1
Olmo-3-32B-Think-GGUF	medium	PopQA	olmo-3-32b-think-dpo	unsloth	19	allenai/Olmo-3-32B-Think	33.1
Olmo-3-32B-Think-DPO	medium	PopQA	olmo-3-32b-think-sft	allenai	20	allenai/Olmo-3-32B-Think-SFT	33.1
Olmo-3-32B-Think-unsloth-bnb-4bit	medium	PopQA	olmo-3-32b-think-dpo	unsloth	21	allenai/Olmo-3-32B-Think	33.1
Olmo-3.1-32B-Think	medium	PopQA	olmo-3-32b-think-dpo	allenai	22	allenai/Olmo-3-32B-Think-DPO	30.9
Olmo-3.1-32B-Think-GGUF	medium	PopQA	olmo-3-1-32b-think	unsloth	23	allenai/Olmo-3.1-32B-Think	30.9
Llama-3.1-Tulu-3-8B-SFT	medium	PopQA (15 shot)	llama-3-1-8b	allenai	24	meta-llama/Llama-3.1-8B	29.3
Llama-3.1-Tulu-3-8B	medium	PopQA (15 shot)	llama-3-1-tulu-3-8b-dpo	allenai	25	allenai/Llama-3.1-Tulu-3-8B-DPO	29.3
Llama-3.1-Tulu-3-8B-DPO	medium	PopQA (15 shot)	llama-3-1-tulu-3-8b-sft	allenai	26	allenai/Llama-3.1-Tulu-3-8B-SFT	29.3
Llama-3.1-Tulu-3-8B-RM	medium	PopQA (15 shot)	llama-3-1-tulu-3-8b-sft	allenai	27	allenai/Llama-3.1-Tulu-3-8B-SFT	29.3
Llama-3.1-Tulu-3.1-8B	medium	PopQA (15 shot)	llama-3-1-tulu-3-8b-dpo	allenai	28	allenai/Llama-3.1-Tulu-3-8B-DPO	29.3
Llama-3.1-Tulu-3-70B	medium	PopQA (15 shot)	llama-3-1-tulu-3-70b-dpo	allenai	29	allenai/Llama-3.1-Tulu-3-70B-DPO	29.3
Llama-3.1-Tulu-3-70B-SFT	medium	PopQA (15 shot)	llama-3-1-70b	allenai	30	meta-llama/Llama-3.1-70B	29.3
Llama-3.1-Tulu-3-405B	medium	PopQA (15 shot)	llama-3-1-405b	allenai	31	allenai/Llama-3.1-Tulu-3-405B-DPO	29.3
Llama-3.1-Tulu-3-70B-DPO	medium	PopQA (15 shot)	llama-3-1-tulu-3-70b-sft	allenai	32	allenai/Llama-3.1-Tulu-3-70B-SFT	29.3
Llama-3.1-Tulu-3-8B-GGUF	medium	PopQA (15 shot)	llama-3-1-tulu-3-8b	unsloth	33	allenai/Llama-3.1-Tulu-3-8B	29.3
Llama-3.1-Tulu-3-405B-SFT	medium	PopQA (15 shot)	llama-3-1-405b	allenai	34	meta-llama/Llama-3.1-405B	29.3
Llama-3.1-Tulu-3-70B-GGUF	medium	PopQA (15 shot)	llama-3-1-tulu-3-70b	unsloth	35	allenai/Llama-3.1-Tulu-3-70B	29.3
Llama-3.1-Tulu-3-405B-DPO	medium	PopQA (15 shot)	llama-3-1-405b	allenai	36	allenai/Llama-3.1-Tulu-3-405B-SFT	29.3
OLMo-2-1124-7B-Instruct	low	PopQA	Unknown	allenai	37	allenai/OLMo-2-1124-7B-DPO	28.3
OLMo-2-1124-13B-Instruct	low	PopQA	olmo-2-1124-13b-instruct-rlvr2	allenai	38	allenai/OLMo-2-1124-13B-Instruct-RLVR2	28.3
OLMo-2-1124-7B-SFT	low	PopQA	olmo2-7b-1124	allenai	39	allenai/OLMo2-7B-1124	28.3
OLMo-2-1124-7B-DPO	low	PopQA	Unknown	allenai	40	allenai/OLMo-2-1124-7B-SFT	28.3
OLMo-2-1124-13B-SFT	low	PopQA	olmo2-13b-1124	allenai	41	allenai/OLMo2-13B-1124	28.3
OLMo-2-1124-13B-DPO	low	PopQA	Unknown	allenai	42	allenai/OLMo-2-1124-13B-SFT	28.3
OLMo-2-1124-7B-Instruct-preview	low	PopQA	Unknown	allenai	43	allenai/OLMo-2-1124-7B-DPO	28.3
OLMo-2-1124-7B-RM	low	PopQA	Unknown	allenai	44	allenai/OLMo-2-1124-7B-SFT	28.3
OLMo-2-1124-7B-RM-Preview	low	PopQA	Unknown	allenai	45	allenai/OLMo-2-1124-7B-SFT	28.3
OLMo-2-1124-13B-RM	low	PopQA	Unknown	allenai	46	allenai/OLMo-2-1124-13B-SFT	28.3
OLMo-2-1124-13B-Instruct-preview	low	PopQA	olmo-2-1124-13b-dpo	allenai	47	allenai/OLMo-2-1124-13B-DPO	28.3
OLMo-2-1124-7B-SFT-Preview	low	PopQA	olmo2-7b-1124	allenai	48	allenai/OLMo2-7B-1124	28.3
OLMo-2-1124-7B-DPO-Preview	low	PopQA	Unknown	allenai	49	allenai/OLMo-2-1124-7B-SFT	28.3
Olmo-3.1-32B-Instruct	medium	PopQA	olmo-3-1-32b-instruct-dpo	allenai	50	allenai/Olmo-3.1-32B-Instruct-DPO	23.7
Olmo-3.1-32B-Instruct-SFT	medium	PopQA	olmo-3-1125-32b	allenai	51	allenai/Olmo-3-1125-32B	23.7
Olmo-3.1-32B-Instruct-DPO	medium	PopQA	olmo-3-1-32b-instruct-sft	allenai	52	allenai/Olmo-3.1-32B-Instruct-SFT	23.7
Olmo-3.1-32B-Instruct-GGUF	medium	PopQA	olmo-3-1-32b-instruct	unsloth	53	allenai/Olmo-3.1-32B-Instruct	23.7
Olmo-3.1-32B-Instruct	medium	PopQA	olmo-3-1-32b-instruct	unsloth	54	allenai/Olmo-3.1-32B-Instruct	23.7
OLMoE-1B-7B-0125-Instruct	medium	PopQA	olmoe-1b-7b-0125-dpo	allenai	55	allenai/OLMoE-1B-7B-0125-DPO	19.75
OLMoE-1B-7B-0125-DPO	medium	PopQA	olmoe-1b-7b-0125	allenai	56	allenai/OLMoE-1B-7B-0125-SFT	19.75
OLMoE-1B-7B-0125-SFT	medium	PopQA	olmoe-1b-7b-0125	allenai	57	allenai/OLMoE-1B-7B-0125	19.75
OLMo-2-0425-1B-Instruct	low	PopQA	olmo-2-0425-1b-rlvr1	allenai	58	allenai/OLMo-2-0425-1B-RLVR1	12.1
OLMo-2-0425-1B-SFT	low	PopQA	olmo-2-0425-1b	allenai	59	allenai/OLMo-2-0425-1B	12.1
OLMo-2-0425-1B-DPO	low	PopQA	olmo-2-0425-1b	allenai	60	allenai/OLMo-2-0425-1B-SFT	12.1
OLMo-2-0425-1B-Instruct-GGUF	low	PopQA	olmo-2-0425-1b-instruct	unsloth	61	allenai/OLMo-2-0425-1B-Instruct	12.1
OLMo-2-0425-1B-Instruct	low	PopQA	olmo-2-0425-1b-instruct	unsloth	62	allenai/OLMo-2-0425-1B-Instruct	12.1
OLMo-2-0425-1B-RLVR1	low	PopQA	olmo-2-0425-1b	allenai	63	allenai/OLMo-2-0425-1B-DPO	12.1

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/popqa.html, docs/notes/hf-benchmark-contexts.json
url: huggingface.co/spaces/popqa ↗