Leaderboard

ARC-Challenge

arc

Record

Category: knowledge
Scored models: 98

Scores

Model	Score
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16	97.35
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	96.08
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16	94.45
nvidia/Nemotron-4-340B-Base	94.28
nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base	93.26
nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base	93.26
nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16	91.72
microsoft/Phi-3-medium-4k-instruct	91.6
microsoft/Phi-3-medium-128k-instruct	91
microsoft/Phi-3-small-128k-instruct	90.8
microsoft/Phi-3-small-8k-instruct	90.7
microsoft/Phi-4-mini-instruct	83.7
microsoft/Phi-4-mini-instruct	83.7
microsoft/Phi-4-mini-instruct	83.7
microsoft/Phi-4-mini-instruct	83.7
microsoft/Phi-4-mini-instruct	83.7
microsoft/Phi-4-mini-instruct	83.7
psmathur/model_007	71.08
psmathur/orca_mini_v3_70b	70.98
psmathur/orca_mini_v3_70b	70.98
migtissera/Synthia-70B-v1.1	70.05
rwitz/go-bruins-v2	69.8
migtissera/Synthia-70B	69.45
rwitz/go-bruins	69.11

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: knowledge
id: arc
name: ARC-Challenge
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16	medium	ARC-Challenge	nvidia-nemotron-3-ultra-550b-a55b-base-bf16	nvidia	1	nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16	97.35
NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	medium	ARC-Challenge	nvidia-nemotron-3-super-120b-a12b-base-bf16	nvidia	2	nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16	96.08
NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16	medium	ARC-Challenge (25-shot, acc_norm)	nvidia-nemotron-3-nano-30b-a3b-base-bf16	nvidia	3	nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16	94.45
Nemotron-4-340B-Base	low	Unknown	Unknown	nvidia	4	nvidia/Nemotron-4-340B-Base	94.28
NVIDIA-Nemotron-Nano-9B-v2-Base	medium	ARC Challenge	nvidia-nemotron-nano-9b-v2-base	nvidia	5	nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base	93.26
NVIDIA-Nemotron-Nano-12B-v2-Base	medium	ARC Challenge	nvidia-nemotron-nano-12b-v2-base	nvidia	6	nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base	93.26
Nemotron-Labs-TwoTower-30B-A3B-Base-BF16	medium	ARC-Challenge (25-shot, acc_norm)	nemotron-labs-twotower-30b-a3b-base-bf16	nvidia	7	nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16	91.72
Phi-3-medium-4k-instruct	medium	ARC Challenge<br>10-shot	phi-3-medium-4k-instruct	microsoft	8	microsoft/Phi-3-medium-4k-instruct	91.6
Phi-3-medium-128k-instruct	medium	ARC Challenge<br>10-shot	phi-3-medium-128k-instruct	microsoft	9	microsoft/Phi-3-medium-128k-instruct	91
Phi-3-small-128k-instruct	medium	ARC Challenge<br>10-shot	phi-3-small-128k-instruct	microsoft	10	microsoft/Phi-3-small-128k-instruct	90.8
Phi-3-small-8k-instruct	medium	ARC Challenge<br>10-shot	phi-3-small-8k-instruct	microsoft	11	microsoft/Phi-3-small-8k-instruct	90.7
Phi-4-mini-instruct	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	microsoft	12	microsoft/Phi-4-mini-instruct	83.7
Phi-4-mini-instruct-GGUF	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	unsloth	13	microsoft/Phi-4-mini-instruct	83.7
Phi-4-mini-instruct-unsloth-bnb-4bit	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	unsloth	14	microsoft/Phi-4-mini-instruct	83.7
Phi-4-mini-instruct-bnb-4bit	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	unsloth	15	microsoft/Phi-4-mini-instruct	83.7
Phi-4-mini-instruct	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	unsloth	16	microsoft/Phi-4-mini-instruct	83.7
Phi-4-mini-instruct-GGUF	medium	ARC Challenge (10-shot)	phi-4-mini-instruct	Mungert	17	microsoft/Phi-4-mini-instruct	83.7
model_007-70B-GGUF	medium	_challenge	Unknown	TheBloke	18	psmathur/model_007	71.08
orca_mini_v3_70B-GGUF	medium	_challenge	orca-mini-v3-70b	TheBloke	19	psmathur/orca_mini_v3_70b	70.98
orca_mini_v3_70B-GPTQ	medium	_challenge	orca-mini-v3-70b	TheBloke	20	psmathur/orca_mini_v3_70b	70.98
Synthia-70B-v1.1-GGUF	medium	_challenge	Unknown	TheBloke	21	migtissera/Synthia-70B-v1.1	70.05
go-bruins-v2-GGUF	low	Unknown	Unknown	TheBloke	22	rwitz/go-bruins-v2	69.8
Synthia-70B-GGUF	medium	_challenge	Unknown	TheBloke	23	migtissera/Synthia-70B	69.45
go-bruins-GGUF	low	Unknown	Unknown	TheBloke	24	rwitz/go-bruins	69.11
NexoNimbus-7B-GGUF	medium	_challenge	nexonimbus-7b	TheBloke	25	abideen/NexoNimbus-7B	68.25
BruinsV2-OpHermesNeu-11B-GGUF	medium	_challenge	Unknown	TheBloke	26	Ba2han/BruinsV2-OpHermesNeu-11B	65.27
bagel-dpo-7B-v0.1-GGUF	low	Unknown	bagel-dpo-7b-v0-1	TheBloke	27	jondurbin/bagel-dpo-7b-v0.1	64.76
juanako-7B-UNA-GGUF	medium	_challenge	juanako-7b-una	TheBloke	28	fblgit/juanako-7b-UNA	64.76
Marcoro14-7B-slerp	medium	_challenge	Unknown	mlabonne	29	AIDC-ai-business/Marcoroni-7B-v3	63.91
orca_mini_v3_13B-GGUF	medium	_challenge	orca-mini-v3-13b	TheBloke	30	psmathur/orca_mini_v3_13b	63.14
orca_mini_v3_13B-GPTQ	medium	_challenge	orca-mini-v3-13b	TheBloke	31	psmathur/orca_mini_v3_13b	63.14
DPOpenHermes-7B-v2-GGUF	medium	_challenge	dpopenhermes-7b-v2	TheBloke	32	openaccess-ai-collective/DPOpenHermes-7B-v2	62.71
Beyonder-4x7B-v2-GGUF	medium	_challenge	beyonder-4x7b-v2	TheBloke	33	mlabonne/Beyonder-4x7B-v2	61.86
Beyonder-4x7B-v2	medium	_challenge	marcoro14-7b-slerp	mlabonne	34	mlabonne/Marcoro14-7B-slerp	61.86
orangetin-OpenHermes-Mixtral-8x7B-GGUF	medium	_challenge	Unknown	TheBloke	35	orangetin/OpenHermes-Mixtral-8x7B	60.75
Fennec-Mixtral-8x7B-GGUF	medium	_challenge	Unknown	TheBloke	36	orangetin/Fennec-Mixtral-8x7B	60.75
Nous-Hermes-2-Yi-34B-GGUF	medium	_challenge	nous-hermes-2-yi-34b	TheBloke	37	NousResearch/Nous-Hermes-2-Yi-34B	60.67
Nous-Hermes-2-Yi-34B-AWQ	medium	_challenge	nous-hermes-2-yi-34b	TheBloke	38	NousResearch/Nous-Hermes-2-Yi-34B	60.67
Synthia-13B-GGUF	medium	_challenge	Unknown	TheBloke	39	migtissera/Synthia-13B	59.98
Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF	medium	_challenge	nous-hermes-2-mixtral-8x7b-dpo	TheBloke	40	NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO	59.9
Nous-Hermes-2-Mixtral-8x7B-DPO-GPTQ	medium	_challenge	nous-hermes-2-mixtral-8x7b-dpo	TheBloke	41	NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO	59.9
falcon-11B	medium	ARC-Challenge-25shots	falcon-11b	tiiuae	42	tiiuae/falcon-11B	59.73
Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF	medium	_challenge	nous-hermes-2-mixtral-8x7b-sft	TheBloke	43	NousResearch/Nous-Hermes-2-Mixtral-8x7B-SFT	59.04
DPOpenHermes-7B-GGUF	medium	_challenge	dpopenhermes-7b	TheBloke	44	openaccess-ai-collective/DPOpenHermes-7B	58.62
NeuralHermes-2.5-Mistral-7B-laser	medium	_challenge	openhermes-2-5-mistral-7b	mlabonne	45	teknium/OpenHermes-2.5-Mistral-7B	57.76
Nous-Hermes-2-SOLAR-10.7B-GGUF	medium	_challenge	nous-hermes-2-solar-10-7b	TheBloke	46	NousResearch/Nous-Hermes-2-SOLAR-10.7B	57.68
Nous-Hermes-Llama2-70B-GGUF	medium	_challenge	Unknown	TheBloke	47	NousResearch/Nous-Hermes-Llama2-70b	57.34
orca_mini_v3_7B-GGUF	medium	_challenge	orca-mini-v3-7b	TheBloke	48	psmathur/orca_mini_v3_7b	57.17
orca_mini_v3_7B-GPTQ	medium	_challenge	orca-mini-v3-7b	TheBloke	49	psmathur/orca_mini_v3_7b	57.17
juanako-7B-v1-GGUF	medium	_challenge	juanako-7b-v1	TheBloke	50	fblgit/juanako-7b-v1	56.91
OpenHermes-2.5-Mistral-7B-AWQ	medium	_challenge	openhermes-2-5-mistral-7b	TheBloke	51	teknium/OpenHermes-2.5-Mistral-7B	56.23
OpenHermes-2.5-Mistral-7B-GGUF	medium	_challenge	openhermes-2-5-mistral-7b	TheBloke	52	teknium/OpenHermes-2.5-Mistral-7B	56.23
OpenHermes-2.5-Mistral-7B-16k-GGUF	medium	_challenge	Unknown	TheBloke	53	NurtureAI/OpenHermes-2.5-Mistral-7B-16k	56.23
OpenHermes-2.5-Mistral-7B-GPTQ	medium	_challenge	openhermes-2-5-mistral-7b	TheBloke	54	teknium/OpenHermes-2.5-Mistral-7B	56.23
Tulpar-7B-v0-GGUF	medium	_challenge	Unknown	TheBloke	55	HyperbeeAI/Tulpar-7b-v0	56.14
Mistral-11B-OmniMix-GGUF	medium	_challenge	mistral-11b-omnimix-bf16	TheBloke	56	NeverSleep/Mistral-11B-OmniMix-bf16	55.8
Luna-AI-Llama2-Uncensored-GGUF	medium	_challenge	Unknown	TheBloke	57	Tap-M/Luna-AI-Llama2-Uncensored	55.12
Luna-AI-Llama2-Uncensored-GPTQ	medium	_challenge	Unknown	TheBloke	58	Tap-M/Luna-AI-Llama2-Uncensored	55.12
OpenHermes-2-Mistral-7B-GGUF	medium	_challenge	Unknown	TheBloke	59	teknium/OpenHermes-2-Mistral-7B	54.52
OpenHermes-2-Mistral-7B-AWQ	medium	_challenge	Unknown	TheBloke	60	teknium/OpenHermes-2-Mistral-7B	54.52
Nemotron-H-4B-Base-8K	low	Unknown	nemotron-h-4b-base-8k	nvidia	61	nvidia/Nemotron-H-4B-Base-8K	54.4
Synthia-7B-v1.2-GGUF	medium	_challenge	Unknown	TheBloke	62	migtissera/Synthia-7B-v1.2	54.35
CollectiveCognition-v1-Mistral-7B-GGUF	medium	_challenge	Unknown	TheBloke	63	teknium/CollectiveCognition-v1-Mistral-7B	54.01
Trurl-2-13B-GGUF	low	Unknown	Unknown	TheBloke	64	Voicelab/trurl-2-13b	52.9
Trurl-2-7B-GGUF	low	Unknown	Unknown	TheBloke	65	Voicelab/trurl-2-7b	52.9
CollectiveCognition-v1.1-Mistral-7B-GGUF	medium	_challenge	Unknown	TheBloke	66	teknium/CollectiveCognition-v1.1-Mistral-7B	50.85
orca_mini_v2_7B-GPTQ	medium	_challenge	Unknown	TheBloke	67	—	50.77
falcon-11B	medium	ARC-Challenge-0shot	falcon-11b	tiiuae	68	tiiuae/falcon-11B	50.17
Redmond-Puffin-13B-GGUF	medium	_challenge	Unknown	TheBloke	69	NousResearch/Redmond-Puffin-13B	49.83
Redmond-Puffin-13B-GPTQ	medium	_challenge	Unknown	TheBloke	70	NousResearch/Redmond-Puffin-13B	49.83
Nous-Hermes-13B-GGUF	medium	_challenge	Unknown	TheBloke	71	NousResearch/Nous-Hermes-13b	49.15
Nous-Hermes-13B-SuperHOT-8K-fp16	medium	_challenge	Unknown	TheBloke	72	—	49.15
Nous-Hermes-13B-SuperHOT-8K-GPTQ	medium	_challenge	Unknown	TheBloke	73	—	49.15
minotaur-13B-fixed-GGUF	medium	_challenge	Unknown	TheBloke	74	openaccess-ai-collective/minotaur-13b-fixed	47.78
OLMo-1B-hf	medium	_challenge	olmo-1b-hf	allenai	75	allenai/OLMo-1B-hf	44.5
OLMo-7B	medium	_challenge	olmo-7b	allenai	76	allenai/OLMo-7B	44.5
OLMo-7B-hf	medium	_challenge	olmo-7b-hf	allenai	77	allenai/OLMo-7B-hf	44.5
OLMo-1B	medium	_challenge	olmo-1b	allenai	78	allenai/OLMo-1B	44.5
OLMo-7B-Twin-2T-hf	medium	_challenge	olmo-7b-twin-2t-hf	allenai	79	allenai/OLMo-7B-Twin-2T-hf	44.5
OLMo-7B-Twin-2T	medium	_challenge	olmo-7b-twin-2t	allenai	80	allenai/OLMo-7B-Twin-2T	44.5
nemo-megatron-gpt-5B	low	Unknown	Unknown	nvidia	81	nvidia/nemo-megatron-gpt-5B	39.76
bitnet-b1.58-2B-4T-gguf	medium	ARC-Challenge	bitnet-b1-58-2b-4t-bf16	microsoft	82	microsoft/bitnet-b1.58-2B-4T-bf16	37.8
bitnet-b1.58-2B-4T	medium	ARC-Challenge	bitnet-b1-58-2b-4t	microsoft	83	microsoft/bitnet-b1.58-2B-4T	37.8
bitnet-b1.58-2B-4T-bf16	medium	ARC-Challenge	bitnet-b1-58-2b-4t-bf16	microsoft	84	microsoft/bitnet-b1.58-2B-4T-bf16	37.8
Redmond-Hermes-Coder-GPTQ	medium	_challenge	Unknown	TheBloke	85	—	28.58
MobileLLM-125M	low	Unknown	Unknown	facebook	86	facebook/MobileLLM-125M	25.2
Nemotron-H-8B-Base-8K	medium	Unknown	nemotron-h-8b-base-8k	nvidia	87	nvidia/Nemotron-H-8B-Base-8K	25
Nemotron-H-56B-Base-8K	medium	Unknown	nemotron-h-56b-base-8k	nvidia	88	nvidia/Nemotron-H-56B-Base-8K	25
OLMo-1B-0724-hf	medium	_challenge	olmo-1b-0724-hf	allenai	89	allenai/OLMo-1B-0724-hf	25
OLMo-7B-0424-hf	medium	_challenge	olmo-7b-0424-hf	allenai	90	allenai/OLMo-7B-0424-hf	25
OLMo-7B-0724-hf	medium	_challenge	olmo-7b-0724-hf	allenai	91	allenai/OLMo-7B-0724-hf	25
LUNA-SOLARkrautLM-Instruct-GGUF	medium	_challenge	luna-solarkrautlm-instruct	TheBloke	92	fblgit/LUNA-SOLARkrautLM-Instruct	25
UNA-TheBeagle-7B-v1-GGUF	medium	_challenge	una-thebeagle-7b-v1	TheBloke	93	fblgit/UNA-TheBeagle-7b-v1	25
UNA-ThePitbull-21.4B-v2-GGUF	medium	_challenge	Unknown	bartowski	94	—	25
Nemotron-H-47B-Base-8K	medium	Unknown	nemotron-h-56b-base-8k	nvidia	95	nvidia/Nemotron-H-56B-Base-8K	25
UNAversal-8x7B-v1beta-GGUF	medium	_challenge	unaversal-8x7b-v1beta	TheBloke	96	fblgit/UNAversal-8x7B-v1beta	25
OLMo-7B-0424	medium	_challenge	Unknown	allenai	97	allenai/OLMo-7B-0424	25
Iambe-RP-v3-20B-GGUF	medium	_challenge	Unknown	TheBloke	98	athirdpath/Iambe-RP-v3-20b	25

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/arc.html
url: huggingface.co/spaces/arc ↗