Local AI Registry

Leaderboard

HumanEval

humaneval
JSON APIBack to leaderboard search

Record

Category
coding
Scored models
429

Scores

ModelScore
google/diffusiongemma-26B-A4B-it94.09
inclusionAI/LLaDA2.0-flash93.29
deepseek-ai/DeepSeek-R1-Distill-Qwen-14B92.6
allenai/Olmo-3-32B-Think-DPO91.5
allenai/Olmo-3.1-32B-Think91.5
nvidia/NVLM-D-72B90.2
MiniMaxAI/MiniMax-Text-01-hf90.2
MiniMaxAI/MiniMax-Text-0190.2
allenai/Olmo-3-32B-Think-DPO90
allenai/Olmo-3-1125-32B90
allenai/Olmo-3-32B-Think90
allenai/Olmo-3-32B-Think-SFT90
allenai/Olmo-3-32B-Think90
inclusionAI/Ling-lite89.02
mistralai/Mistral-Small-3.1-24B-Base-250388.99
mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
mistralai/Mistral-Small-3.1-24B-Base-250388.99
mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
mistralai/Mistral-Small-3.1-24B-Base-250388.41
mistralai/Mistral-Small-3.1-24B-Instruct-250388.41
mistralai/Mistral-Small-3.1-24B-Instruct-250388.41

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category
coding
id
humaneval
name
HumanEval
schema version
local-ai-registry/v1

rows

variantconfcontextmodel idorgrankrootscore
diffusiongemma-26B-A4B-it-NVFP4mediumHumanEvaldiffusiongemma-26b-a4b-itnvidia1google/diffusiongemma-26B-A4B-it94.09
LLaDA2.0-flashmediumHumanEvalllada2-0-flashinclusionAI2inclusionAI/LLaDA2.0-flash93.29
DeepCoder-14B-Preview-GGUFlowUnknowndeepseek-r1-distill-qwen-14bMungert3deepseek-ai/DeepSeek-R1-Distill-Qwen-14B92.6
Olmo-3.1-32B-ThinkmediumHumanEvalPlusolmo-3-32b-think-dpoallenai4allenai/Olmo-3-32B-Think-DPO91.5
Olmo-3.1-32B-Think-GGUFmediumHumanEvalPlusolmo-3-1-32b-thinkunsloth5allenai/Olmo-3.1-32B-Think91.5
NVLM-D-72BlowUnknownnvlm-d-72bnvidia6nvidia/NVLM-D-72B90.2
MiniMax-Text-01-hfmediumHumanEvalminimax-text-01-hfMiniMaxAI7MiniMaxAI/MiniMax-Text-01-hf90.2
MiniMax-Text-01mediumHumanEvalminimax-text-01MiniMaxAI8MiniMaxAI/MiniMax-Text-0190.2
Olmo-3-32B-ThinkmediumHumanEvalPlusolmo-3-32b-think-dpoallenai9allenai/Olmo-3-32B-Think-DPO90
Olmo-3-32B-Think-SFTmediumHumanEvalPlusolmo-3-1125-32ballenai10allenai/Olmo-3-1125-32B90
Olmo-3-32B-Think-GGUFmediumHumanEvalPlusolmo-3-32b-think-dpounsloth11allenai/Olmo-3-32B-Think90
Olmo-3-32B-Think-DPOmediumHumanEvalPlusolmo-3-32b-think-sftallenai12allenai/Olmo-3-32B-Think-SFT90
Olmo-3-32B-Think-unsloth-bnb-4bitmediumHumanEvalPlusolmo-3-32b-think-dpounsloth13allenai/Olmo-3-32B-Think90
Ling-litemediumHumanEval(Pass@1)ling-liteinclusionAI14inclusionAI/Ling-lite89.02
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai15mistralai/Mistral-Small-3.1-24B-Base-250388.99
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth16mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
Mistral-Small-3.2-24B-Instruct-2506-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth17mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth18mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth19mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert20mistralai/Mistral-Small-3.1-24B-Base-250388.99
Mistral-Small-3.2-24B-Instruct-2506-FP8lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth21mistralai/Mistral-Small-3.2-24B-Instruct-250688.99
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai22mistralai/Mistral-Small-3.1-24B-Base-250388.41
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth23mistralai/Mistral-Small-3.1-24B-Instruct-250388.41
Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth24mistralai/Mistral-Small-3.1-24B-Instruct-250388.41
Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth25mistralai/Mistral-Small-3.1-24B-Instruct-250388.41
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth26mistralai/Mistral-Small-3.1-24B-Instruct-250388.41
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert27mistralai/Mistral-Small-3.1-24B-Base-250388.41
Llama-3_1-Nemotron-Ultra-253B-CPT-v1mediumHumanEvalUnknownnvidia28nvidia/Llama-3_1-Nemotron-Ultra-253B-CPT-v188.41
Ling-Coder-lite-GPTQ-Int8mediumHumanEval-PythonUnknowninclusionAI29inclusionAI/Ling-Coder-lite88.41
Qwen3-8B-DMS-8xmediumHumanEvalqwen3-8b-basenvidia30Qwen/Qwen3-8B87.8
Ling-lite-1.5mediumHumanEval(Pass@1)ling-lite-1-5inclusionAI31inclusionAI/Ling-lite-1.587.27
Falcon-H1-34B-InstructmediumHumanEvalfalcon-h1-34b-basetiiuae32tiiuae/Falcon-H1-34B-Base87.2
Falcon-H1-34B-Instruct-GGUFmediumHumanEvalfalcon-h1-34b-instructtiiuae33tiiuae/Falcon-H1-34B-Instruct87.2
Falcon-H1-34B-Instruct-GGUFmediumHumanEvalfalcon-h1-34b-instructunsloth34tiiuae/Falcon-H1-34B-Instruct87.2
Falcon-H1-34B-Instruct-GPTQ-Int4mediumHumanEvalfalcon-h1-34b-instructtiiuae35tiiuae/Falcon-H1-34B-Instruct87.2
Falcon-H1-7B-InstructmediumHumanEvalfalcon-h1-7b-basetiiuae36tiiuae/Falcon-H1-7B-Base86.59
Falcon-H1-7B-Instruct-GPTQ-Int4mediumHumanEvalfalcon-h1-7b-instructtiiuae37tiiuae/Falcon-H1-7B-Instruct86.59
Falcon-H1-7B-Instruct-GGUFmediumHumanEvalfalcon-h1-7b-instructtiiuae38tiiuae/Falcon-H1-7B-Instruct86.59
Falcon-H1-7B-Instruct-GGUFmediumHumanEvalfalcon-h1-7b-instructunsloth39tiiuae/Falcon-H1-7B-Instruct86.59
Llama-3.1-Tulu-3-8B-SFTmedium**HumanEval (pass@10)**llama-3-1-8ballenai40meta-llama/Llama-3.1-8B86.2
Llama-3.1-Tulu-3-8Bmedium**HumanEval (pass@10)**llama-3-1-tulu-3-8b-dpoallenai41allenai/Llama-3.1-Tulu-3-8B-DPO86.2
Llama-3.1-Tulu-3-8B-DPOmedium**HumanEval (pass@10)**llama-3-1-tulu-3-8b-sftallenai42allenai/Llama-3.1-Tulu-3-8B-SFT86.2
Llama-3.1-Tulu-3-8B-RMmedium**HumanEval (pass@10)**llama-3-1-tulu-3-8b-sftallenai43allenai/Llama-3.1-Tulu-3-8B-SFT86.2
Llama-3.1-Tulu-3.1-8Bmedium**HumanEval (pass@10)**llama-3-1-tulu-3-8b-dpoallenai44allenai/Llama-3.1-Tulu-3-8B-DPO86.2
Llama-3.1-Tulu-3-70Bmedium**HumanEval (pass@10)**llama-3-1-tulu-3-70b-dpoallenai45allenai/Llama-3.1-Tulu-3-70B-DPO86.2
Llama-3.1-Tulu-3-70B-SFTmedium**HumanEval (pass@10)**llama-3-1-70ballenai46meta-llama/Llama-3.1-70B86.2
Llama-3.1-Tulu-3-405Bmedium**HumanEval (pass@10)**Unknownallenai47allenai/Llama-3.1-Tulu-3-405B-DPO86.2
Llama-3.1-Tulu-3-70B-DPOmedium**HumanEval (pass@10)**llama-3-1-tulu-3-70b-sftallenai48allenai/Llama-3.1-Tulu-3-70B-SFT86.2
Llama-3.1-Tulu-3-8B-GGUFmedium**HumanEval (pass@10)**llama-3-1-tulu-3-8bunsloth49allenai/Llama-3.1-Tulu-3-8B86.2
Llama-3.1-Tulu-3-405B-SFTmedium**HumanEval (pass@10)**llama-3-1-405ballenai50meta-llama/Llama-3.1-405B86.2
Llama-3.1-Tulu-3-70B-GGUFmedium**HumanEval (pass@10)**llama-3-1-tulu-3-70bunsloth51allenai/Llama-3.1-Tulu-3-70B86.2
Llama-3.1-Tulu-3-405B-DPOmedium**HumanEval (pass@10)**Unknownallenai52allenai/Llama-3.1-Tulu-3-405B-SFT86.2
LLaDA2.0-mini-previewmediumUnknownllada2-0-mini-previewinclusionAI53inclusionAI/LLaDA2.0-mini-preview85.98
LLaDA2.0-flash-previewmediumUnknownllada2-0-flash-previewinclusionAI54inclusionAI/LLaDA2.0-flash-preview85.98
WizardCoder-33B-V1.1-GGUFlowUnknownUnknownTheBloke55WizardLM/WizardCoder-33B-V1.185.4
wavecoder-ultra-6.7blowUnknownUnknownmicrosoft56microsoft/wavecoder-ultra-6.7b85.4
wavecoder-ds-6.7blowUnknownUnknownmicrosoft57microsoft/wavecoder-ds-6.7b85.4
wavecoder-pro-6.7blowUnknownUnknownmicrosoft58microsoft/wavecoder-pro-6.7b85.4
WizardCoder-33B-V1.1-GPTQlowUnknownUnknownTheBloke59WizardLM/WizardCoder-33B-V1.185.4
Ling-Coder-lite-GPTQ-Int8mediumEvalPlus-HumanEval+UnknowninclusionAI60inclusionAI/Ling-Coder-lite85.37
Mistral-Small-24B-Instruct-2501medium_instruct_pass@1mistral-small-24b-base-2501mistralai61mistralai/Mistral-Small-24B-Base-250184.8
Mistral-Small-24B-Instruct-2501-bnb-4bitmedium_instruct_pass@1mistral-small-24b-instruct-2501unsloth62mistralai/Mistral-Small-24B-Instruct-250184.8
Mistral-Small-24B-Instruct-2501-GGUFmedium_instruct_pass@1mistral-small-24b-instruct-2501unsloth63mistralai/Mistral-Small-24B-Instruct-250184.8
Seed-Coder-8B-Instruct-GGUFhighUnknownseed-coder-8b-instructunsloth64ByteDance-Seed/Seed-Coder-8B-Instruct84.8
Mistral-Small-24B-Instruct-2501-unsloth-bnb-4bitmedium_instruct_pass@1mistral-small-24b-instruct-2501unsloth65mistralai/Mistral-Small-24B-Instruct-250184.8
Mistral-Small-24B-Instruct-2501medium_instruct_pass@1mistral-small-24b-instruct-2501unsloth66mistralai/Mistral-Small-24B-Instruct-250184.8
Mistral-Small-24B-Base-2501-unsloth-bnb-4bitmedium_instruct_pass@1mistral-small-24b-base-2501unsloth67mistralai/Mistral-Small-24B-Base-250184.8
Seed-Coder-8B-Instruct-unsloth-bnb-4bithighUnknownseed-coder-8b-instructunsloth68ByteDance-Seed/Seed-Coder-8B-Instruct84.8
LLaDA2.0-minimediumHumanEvalllada2-0-miniinclusionAI69inclusionAI/LLaDA2.0-mini84.76
DeepSeek-V2.5mediumHumanEval pythondeepseek-v2-5deepseek-ai70deepseek-ai/DeepSeek-V2.584.5
NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16mediumHumanEvalnvidia-nemotron-3-ultra-550b-a55b-base-bf16nvidia71nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF1683.84
Ling-2.6-flash-basemediumHumanEval-Plusling-2-6-flashinclusionAI72inclusionAI/Ling-2.6-flash83.54
Ling-2.6-1T-basemediumHumanEval-Plusling-2-6-1tinclusionAI73inclusionAI/Ling-2.6-1T83.54
Phi-4-reasoningmediumHumanEvalPlusphi-4microsoft74microsoft/phi-483.5
Phi-4-reasoning-plusmediumHumanEvalPlusphi-4microsoft75microsoft/phi-483.5
Phi-4-reasoning-plus-GGUFmediumHumanEvalPlusphi-4-reasoning-plusunsloth76microsoft/Phi-4-reasoning-plus83.5
Phi-4-reasoning-GGUFmediumHumanEvalPlusphi-4-reasoningunsloth77microsoft/Phi-4-reasoning83.5
phi-4-reasoningmediumHumanEvalPlusphi-4-reasoningunsloth78microsoft/phi-4-reasoning83.5
phi-4-reasoning-unsloth-bnb-4bitmediumHumanEvalPlusphi-4-reasoningunsloth79microsoft/phi-4-reasoning83.5
phi-4-reasoning-bnb-4bitmediumHumanEvalPlusphi-4-reasoningunsloth80microsoft/phi-4-reasoning83.5
Phi-4-reasoning-plus-GGUFmediumHumanEvalPlusphi-4Mungert81microsoft/phi-483.5
Phi-4-reasoning-plusmediumHumanEvalPlusphi-4-reasoning-plusunsloth82microsoft/Phi-4-reasoning-plus83.5
Phi-4-reasoning-GGUFmediumHumanEvalPlusphi-4Mungert83microsoft/phi-483.5
SmallThinker-21BA3B-Instruct-GGUFlowUnknownUnknownMungert84—82.93
codegeex4-all-9bhighUnknownUnknownzai-org85zai-org/codegeex4-all-9b82.3
codegeex4-all-9b-GGUFhighUnknownUnknownzai-org86zai-org/codegeex4-all-9b82.3
Falcon-H1-34B-InstructmediumHumanEval+falcon-h1-34b-basetiiuae87tiiuae/Falcon-H1-34B-Base81.71
Falcon-H1-34B-Instruct-GGUFmediumHumanEval+falcon-h1-34b-instructtiiuae88tiiuae/Falcon-H1-34B-Instruct81.71
Falcon-H1-34B-Instruct-GGUFmediumHumanEval+falcon-h1-34b-instructunsloth89tiiuae/Falcon-H1-34B-Instruct81.71
Falcon-H1-34B-Instruct-GPTQ-Int4mediumHumanEval+falcon-h1-34b-instructtiiuae90tiiuae/Falcon-H1-34B-Instruct81.71
Qwen2-72B-InstructmediumHumanEvalqwen2-72bQwen91Qwen/Qwen2-72B81.7
Llama-3.1-Tulu-3-8B-SFTmedium**HumanEval+ (pass@10)**llama-3-1-8ballenai92meta-llama/Llama-3.1-8B81.4
Llama-3.1-Tulu-3-8Bmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-8b-dpoallenai93allenai/Llama-3.1-Tulu-3-8B-DPO81.4
Llama-3.1-Tulu-3-8B-DPOmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-8b-sftallenai94allenai/Llama-3.1-Tulu-3-8B-SFT81.4
Llama-3.1-Tulu-3-8B-RMmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-8b-sftallenai95allenai/Llama-3.1-Tulu-3-8B-SFT81.4
Llama-3.1-Tulu-3.1-8Bmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-8b-dpoallenai96allenai/Llama-3.1-Tulu-3-8B-DPO81.4
Llama-3.1-Tulu-3-70Bmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-70b-dpoallenai97allenai/Llama-3.1-Tulu-3-70B-DPO81.4
Llama-3.1-Tulu-3-70B-SFTmedium**HumanEval+ (pass@10)**llama-3-1-70ballenai98meta-llama/Llama-3.1-70B81.4
Llama-3.1-Tulu-3-405Bmedium**HumanEval+ (pass@10)**Unknownallenai99allenai/Llama-3.1-Tulu-3-405B-DPO81.4
Llama-3.1-Tulu-3-70B-DPOmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-70b-sftallenai100allenai/Llama-3.1-Tulu-3-70B-SFT81.4
Llama-3.1-Tulu-3-8B-GGUFmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-8bunsloth101allenai/Llama-3.1-Tulu-3-8B81.4
Llama-3.1-Tulu-3-405B-SFTmedium**HumanEval+ (pass@10)**llama-3-1-405ballenai102meta-llama/Llama-3.1-405B81.4
Llama-3.1-Tulu-3-70B-GGUFmedium**HumanEval+ (pass@10)**llama-3-1-tulu-3-70bunsloth103allenai/Llama-3.1-Tulu-3-70B81.4
Llama-3.1-Tulu-3-405B-DPOmedium**HumanEval+ (pass@10)**Unknownallenai104allenai/Llama-3.1-Tulu-3-405B-SFT81.4
Falcon-H1-7B-InstructmediumHumanEval+falcon-h1-7b-basetiiuae105tiiuae/Falcon-H1-7B-Base81.1
Falcon-H1-7B-Instruct-GPTQ-Int4mediumHumanEval+falcon-h1-7b-instructtiiuae106tiiuae/Falcon-H1-7B-Instruct81.1
DeepSeek-V2-Chat-0628medium**HumanEval**deepseek-v2-chat-0628deepseek-ai107deepseek-ai/DeepSeek-V2-Chat-062881.1
Falcon-H1-7B-Instruct-GGUFmediumHumanEval+falcon-h1-7b-instructtiiuae108tiiuae/Falcon-H1-7B-Instruct81.1
Falcon-H1-7B-Instruct-GGUFmediumHumanEval+falcon-h1-7b-instructunsloth109tiiuae/Falcon-H1-7B-Instruct81.1
Olmo-3.1-32B-InstructmediumHumanEvalPlusolmo-3-1-32b-instruct-dpoallenai110allenai/Olmo-3.1-32B-Instruct-DPO80.8
Olmo-3.1-32B-Instruct-SFTmediumHumanEvalPlusolmo-3-1125-32ballenai111allenai/Olmo-3-1125-32B80.8
Olmo-3.1-32B-Instruct-DPOmediumHumanEvalPlusolmo-3-1-32b-instruct-sftallenai112allenai/Olmo-3.1-32B-Instruct-SFT80.8
Olmo-3.1-32B-Instruct-GGUFmediumHumanEvalPlusolmo-3-1-32b-instructunsloth113allenai/Olmo-3.1-32B-Instruct80.8
Olmo-3.1-32B-InstructmediumHumanEvalPlusolmo-3-1-32b-instructunsloth114allenai/Olmo-3.1-32B-Instruct80.8
NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16mediumHumanEvalnvidia-nemotron-3-super-120b-a12b-base-bf16nvidia115nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF1679.4
Nemotron-Labs-TwoTower-30B-A3B-Base-BF16mediumHumanEval (0-shot)nemotron-labs-twotower-30b-a3b-base-bf16nvidia116nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF1679.27
Qwen2.5-Omni-3BmediumHumanEvalqwen2-5-omni-3bQwen117Qwen/Qwen2.5-Omni-3B78.7
Qwen2.5-Omni-7BmediumHumanEvalqwen2-5-omni-7bQwen118Qwen/Qwen2.5-Omni-7B78.7
Qwen2.5-Omni-7B-GGUFmediumHumanEvalqwen2-5-omni-7bunsloth119Qwen/Qwen2.5-Omni-7B78.7
Qwen2.5-Omni-3B-GGUFmediumHumanEvalqwen2-5-omni-3bunsloth120Qwen/Qwen2.5-Omni-3B78.7
Qwen2.5-Omni-3B-GGUFmediumHumanEvalqwen2-5-omni-3bMungert121Qwen/Qwen2.5-Omni-3B78.7
Qwen2.5-Omni-7BmediumHumanEvalqwen2-5-omni-7bunsloth122Qwen/Qwen2.5-Omni-7B78.7
Qwen2.5-Omni-7B-GGUFmediumHumanEvalqwen2-5-omni-7bMungert123Qwen/Qwen2.5-Omni-7B78.7
Qwen2.5-Omni-3BmediumHumanEvalqwen2-5-omni-3bunsloth124Qwen/Qwen2.5-Omni-3B78.7
Nemotron-Research-GooseReason-4B-InstructlowUnknownqwen3-4b-instruct-2507nvidia125Qwen/Qwen3-4B-Instruct-250777.56
Falcon-H1-3B-InstructmediumHumanEvalfalcon-h1-3b-basetiiuae126tiiuae/Falcon-H1-3B-Base76.83
Falcon-H1-3B-Instruct-GGUFmediumHumanEvalfalcon-h1-3b-basetiiuae127tiiuae/Falcon-H1-3B-Base76.83
Falcon-H1-3B-Instruct-GGUFmediumHumanEvalfalcon-h1-3b-instructunsloth128tiiuae/Falcon-H1-3B-Instruct76.83
Nemotron-H-4B-Instruct-128KlowUnknownnemotron-h-4b-base-8knvidia129nvidia/Nemotron-H-4B-Base-8K76.2
speechless-codellama-34b-v2.0-GGUFmedium-pythonUnknownTheBloke130uukuguy/speechless-codellama-34b-v2.075.61
Ziya-Coding-34B-v1.0-GGUFlowUnknownUnknownTheBloke131IDEA-CCNL/Ziya-Coding-34B-v1.075.5
GRIN-MoEmediumHumanEvalgrin-moemicrosoft132microsoft/GRIN-MoE74.4
XwinCoder-34B-GGUFhighUnknownUnknownTheBloke133Xwin-LM/XwinCoder-34B74.2
DeepSeek-V2.5mediumHumanEval Multideepseek-v2-5deepseek-ai134deepseek-ai/DeepSeek-V2.573.8
Falcon-H1-1.5B-Deep-InstructmediumHumanEvalfalcon-h1-1-5b-deep-basetiiuae135tiiuae/Falcon-H1-1.5B-Deep-Base73.78
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumHumanEvalfalcon-h1-1-5b-deep-instructtiiuae136tiiuae/Falcon-H1-1.5B-Deep-Instruct73.78
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumHumanEvalfalcon-h1-1-5b-deep-instructunsloth137tiiuae/Falcon-H1-1.5B-Deep-Instruct73.78
Falcon-H1-1.5B-Deep-InstructmediumHumanEvalfalcon-h1-1-5b-deep-instructunsloth138tiiuae/Falcon-H1-1.5B-Deep-Instruct73.78
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumHumanEvalfalcon-h1-1-5b-deep-instructtiiuae139tiiuae/Falcon-H1-1.5B-Deep-Instruct73.78
WizardCoder-Python-34B-V1.0-GGUFhighUnknownUnknownTheBloke140WizardLM/WizardCoder-Python-34B-V1.073.2
NextCoder-7BlowUnknownqwen2-5-coder-7bmicrosoft141Qwen/Qwen2.5-Coder-7B-Instruct73.2
WizardLM-70B-V1.0-GPTQlowUnknownUnknownTheBloke142WizardLM/WizardLM-70B-V1.073.2
WizardLM-13B-V1.2-GGUFlowUnknownUnknownTheBloke143WizardLM/WizardLM-13B-V1.273.2
WizardMath-13B-V1.0-GGUFlowUnknownUnknownTheBloke144WizardLM/WizardMath-13B-V1.073.2
WizardLM-70B-V1.0-GGUFlowUnknownUnknownTheBloke145WizardLM/WizardLM-70B-V1.073.2
WizardMath-7B-V1.0-GGUFlowUnknownUnknownTheBloke146WizardLM/WizardMath-7b-V1.073.2
WizardLM-13B-V1.1-GGUFlowUnknownUnknownTheBloke147WizardLM/WizardLM-13B-V1.173.2
wizardLM-7B-GGUFlowUnknownUnknownTheBloke148WizardLM/WizardLM-7B-V1.073.2
NextCoder-14B-GGUFlowUnknownUnknownMungert149Qwen/Qwen2.5-Coder-14B-Instruct73.2
WizardMath-70B-V1.0-GGUFlowUnknownUnknownTheBloke150WizardLM/WizardMath-70B-V1.073.2
WizardLM-30B-GGUFlowUnknownUnknownTheBloke151WizardLM/WizardLM-30B-V1.073.2
NextCoder-14BlowUnknownUnknownmicrosoft152Qwen/Qwen2.5-Coder-14B-Instruct73.2
NextCoder-32BlowUnknownUnknownmicrosoft153Qwen/Qwen2.5-Coder-32B-Instruct73.2
NextCoder-7B-GGUFlowUnknownqwen2-5-coder-7bMungert154Qwen/Qwen2.5-Coder-7B-Instruct73.2
WizardLM-13B-V1.2-GPTQlowUnknownUnknownTheBloke155WizardLM/WizardLM-13B-V1.273.2
WizardLM-13B-V1.1-GPTQlowUnknownUnknownTheBloke156WizardLM/WizardLM-13B-V1.173.2
WizardCoder-Python-34B-V1.0-GPTQhighUnknownUnknownTheBloke157WizardLM/WizardCoder-Python-34B-V1.073.2
DeepCoder-1.5B-Preview-GGUFlowUnknowndeepseek-r1-distill-qwen-1-5bMungert158deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B73
internlm2_5-20bmediumHUMANEVALinternlm2-5-20binternlm159internlm/internlm2_5-20b71.95
glm-4-9b-chathighUnknownglm-4-9b-chatzai-org160zai-org/glm-4-9b-chat71.8
glm-4-9b-chat-hfhighUnknownglm-4-9b-chatzai-org161THUDM/glm-4-9b-chat71.8
NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16mediumHumanEval (0-shot)nvidia-nemotron-3-nano-30b-a3b-base-bf16nvidia162nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF1670.73
Falcon-H1-3B-InstructmediumHumanEval+falcon-h1-3b-basetiiuae163tiiuae/Falcon-H1-3B-Base70.73
Falcon-H1-3B-Instruct-GGUFmediumHumanEval+falcon-h1-3b-basetiiuae164tiiuae/Falcon-H1-3B-Base70.73
Falcon-H1-3B-Instruct-GGUFmediumHumanEval+falcon-h1-3b-instructunsloth165tiiuae/Falcon-H1-3B-Instruct70.73
Falcon-H1-34B-BasemediumHumanEvalfalcon-h1-34b-basetiiuae166tiiuae/Falcon-H1-34B-Base70.12
glm-4-9bhighUnknownglm-4-9bzai-org167zai-org/glm-4-9b70.1
glm-4-9b-hfhighUnknownglm-4-9b-hfzai-org168zai-org/glm-4-9b-hf70.1
LFM2-8B-A1B-GGUFlowUnknownlfm2-8b-a1bunsloth169LiquidAI/LFM2-8B-A1B69.51
LFM2-8B-A1BlowUnknownlfm2-8b-a1bunsloth170LiquidAI/LFM2-8B-A1B69.51
Falcon-H1-1.5B-Deep-InstructmediumHumanEval+falcon-h1-1-5b-deep-basetiiuae171tiiuae/Falcon-H1-1.5B-Deep-Base68.9
openchat-3.5-1210-GGUFhighBBH CoTopenchat-3-5-1210TheBloke172openchat/openchat-3.5-121068.9
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumHumanEval+falcon-h1-1-5b-deep-instructtiiuae173tiiuae/Falcon-H1-1.5B-Deep-Instruct68.9
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumHumanEval+falcon-h1-1-5b-deep-instructunsloth174tiiuae/Falcon-H1-1.5B-Deep-Instruct68.9
openchat-3.5-1210-GPTQhighBBH CoTopenchat-3-5-1210TheBloke175openchat/openchat-3.5-121068.9
Falcon-H1-1.5B-Deep-InstructmediumHumanEval+falcon-h1-1-5b-deep-instructunsloth176tiiuae/Falcon-H1-1.5B-Deep-Instruct68.9
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumHumanEval+falcon-h1-1-5b-deep-instructtiiuae177tiiuae/Falcon-H1-1.5B-Deep-Instruct68.9
XwinCoder-13B-GGUFhighUnknownUnknownTheBloke178Xwin-LM/XwinCoder-13B68.8
Falcon-H1-1.5B-InstructmediumHumanEvalfalcon-h1-1-5b-basetiiuae179tiiuae/Falcon-H1-1.5B-Base68.29
Falcon-H1-1.5B-Instruct-GGUFmediumHumanEvalfalcon-h1-1-5b-instructtiiuae180tiiuae/Falcon-H1-1.5B-Instruct68.29
Falcon-H1-1.5B-Instruct-GGUFmediumHumanEvalfalcon-h1-1-5b-instructunsloth181tiiuae/Falcon-H1-1.5B-Instruct68.29
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumHumanEvalfalcon-h1-1-5b-instructtiiuae182tiiuae/Falcon-H1-1.5B-Instruct68.29
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumHumanEvalfalcon-h1-1-5b-instructtiiuae183tiiuae/Falcon-H1-1.5B-Instruct68.29
Falcon-H1-7B-BasemediumHumanEvalfalcon-h1-7b-basetiiuae184tiiuae/Falcon-H1-7B-Base67.68
gemma-3n-E2B-itmedium[HumanEval][]gemma-3n-e4bgoogle185google/gemma-3n-E4B-it66.5
gemma-3n-E4B-itmedium[HumanEval][]gemma-3n-e4bgoogle186google/gemma-3n-E4B66.5
gemma-3n-E4B-it-litert-lmmedium[HumanEval][]Unknowngoogle187google/gemma-3n-E4B-it-litert-lm66.5
gemma-3n-E2B-it-GGUFmedium[HumanEval][]gemma-3n-e2b-itunsloth188google/gemma-3n-E2B-it66.5
gemma-3n-E4B-itmedium[HumanEval][]gemma-3n-e4bunsloth189google/gemma-3n-E4B-it66.5
gemma-3n-E2Bmedium[HumanEval][]gemma-3n-e4bgoogle190google/gemma-3n-E4B66.5
gemma-3n-E2B-it-litert-lmmedium[HumanEval][]Unknowngoogle191google/gemma-3n-E2B-it-litert-lm66.5
gemma-3n-E4B-it-GGUFmedium[HumanEval][]gemma-3n-e4bunsloth192google/gemma-3n-E4B-it66.5
gemma-3n-E4Bmedium[HumanEval][]gemma-3n-e4bgoogle193google/gemma-3n-E4B66.5
gemma-3n-E2B-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3n-e2b-itunsloth194google/gemma-3n-E2B-it66.5
gemma-3n-E2B-it-litert-preview-GGUFmedium[HumanEval][]Unknownunsloth195google/gemma-3n-E2B-it-litert-preview66.5
gemma-3n-E4B-it-litert-preview-GGUFmedium[HumanEval][]Unknownunsloth196google/gemma-3n-E4B-it-litert-preview66.5
gemma-3n-E4B-unsloth-bnb-4bitmedium[HumanEval][]gemma-3n-e4bunsloth197google/gemma-3n-E4B-it66.5
gemma-3n-E2B-it-ONNXmedium[HumanEval][]gemma-3n-e2b-itonnx-community198google/gemma-3n-E2B-it66.5
gemma-3n-E4B-it-GGUFmedium[HumanEval][]gemma-3n-e4bMungert199google/gemma-3n-E4B66.5
gemma-3n-E4B-it-absolute-heresy-MPOA-imatrix-GGUFmedium[HumanEval][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious200MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA66.5
gemma-3n-E4B-it-absolute-heresy-GGUFmedium[HumanEval][]gemma-3n-e4bMuXodious201google/gemma-3n-E4B-it66.5
gemma-3n-E4B-it-absolute-heresy-MPOA-GGUFmedium[HumanEval][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious202MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA66.5
gemma-3n-E2B-it-GGUFmedium[HumanEval][]gemma-3n-e4bMungert203google/gemma-3n-E4b-it66.5
llama-3-tulu-2-8blowGSM8k 8-shot cot | BBH 3-shot cotmeta-llama-3-8ballenai204meta-llama/Meta-Llama-3-8B66.4
llama-3-tulu-2-70blowGSM8k 8-shot cot | BBH 3-shot cotmeta-llama-3-70ballenai205meta-llama/Meta-Llama-3-70B66.4
Olmo-3-1125-32BlowUnknownolmo-3-1125-32ballenai206allenai/Olmo-3-1125-32B65.6
Falcon-H1-34B-BasemediumHumanEval+falcon-h1-34b-basetiiuae207tiiuae/Falcon-H1-34B-Base64.63
openchat-3.5-1210-GGUFlowUnknownopenchat-3-5-1210TheBloke208openchat/openchat-3.5-121064.6
Qwen-72B-ChatlowUnknownqwen-72bQwen209Qwen/Qwen-72B64.6
openchat-3.5-1210-GPTQlowUnknownopenchat-3-5-1210TheBloke210openchat/openchat-3.5-121064.6
WizardCoder-Python-13B-V1.0-GGUFhighUnknownUnknownTheBloke211WizardLM/WizardCoder-Python-13B-V1.064
Youtu-LLM-2B-GGUFmediumHumanEvalyoutu-llm-2b-baseMungert212tencent/Youtu-LLM-2B-Base64
WizardCoder-Python-13B-V1.0-GPTQhighUnknownUnknownTheBloke213WizardLM/WizardCoder-Python-13B-V1.064
Falcon-H1-7B-BasemediumHumanEval+falcon-h1-7b-basetiiuae214tiiuae/Falcon-H1-7B-Base63.41
DeepSeek-V4-FlashmediumHumanEval (Pass@1)deepseek-v4-flashdeepseek-ai215deepseek-ai/DeepSeek-V4-Flash62.8
DeepSeek-V4-PromediumHumanEval (Pass@1)deepseek-v4-prodeepseek-ai216deepseek-ai/DeepSeek-V4-Pro62.8
DeepSeek-V4-Flash-DSparkmediumHumanEval (Pass@1)Unknowndeepseek-ai217—62.8
DeepSeek-V4-Flash-GGUFmediumHumanEval (Pass@1)deepseek-v4-flashunsloth218deepseek-ai/DeepSeek-V4-Flash62.8
DeepSeek-V4-FlashmediumHumanEval (Pass@1)deepseek-v4-flashunsloth219deepseek-ai/DeepSeek-V4-Flash62.8
DeepSeek-V4-Pro-DSparkmediumHumanEval (Pass@1)Unknowndeepseek-ai220—62.8
DeepSeek-V4-PromediumHumanEval (Pass@1)deepseek-v4-prounsloth221deepseek-ai/DeepSeek-V4-Pro62.8
OLMoE-1B-7B-0125-Instructmedium**HumanEval**olmoe-1b-7b-0125-dpoallenai222allenai/OLMoE-1B-7B-0125-DPO62.3
OLMoE-1B-7B-0125-DPOmedium**HumanEval**Unknownallenai223allenai/OLMoE-1B-7B-0125-SFT62.3
OLMoE-1B-7B-0125-SFTmedium**HumanEval**olmoe-1b-7b-0125allenai224allenai/OLMoE-1B-7B-012562.3
Qwen2-7B-InstructmediumHumanevalqwen2-7bQwen225Qwen/Qwen2-7B62.2
Phi-3-medium-4k-instructmediumHumanEval<br>0-shotphi-3-medium-4k-instructmicrosoft226microsoft/Phi-3-medium-4k-instruct62.2
Qwen-72B-Chat-Int8highUnknownqwen-72bQwen227Qwen/Qwen-72B62.2
AceInstruct-7BmediumHumanEvalaceinstruct-7bnvidia228nvidia/AceInstruct-7B61.6
AceInstruct-1.5BmediumHumanEvalaceinstruct-1-5bnvidia229nvidia/AceInstruct-1.5B61.6
Qwen-72B-Chat-Int4highUnknownqwen-72bQwen230Qwen/Qwen-72B61.6
AceInstruct-72BmediumHumanEvalaceinstruct-72bnvidia231nvidia/AceInstruct-72B61.6
AceInstruct-1.5B-GGUFmediumHumanEvalaceinstruct-1-5bMungert232nvidia/AceInstruct-1.5B61.6
Falcon-H1-1.5B-InstructmediumHumanEval+falcon-h1-1-5b-basetiiuae233tiiuae/Falcon-H1-1.5B-Base61.59
Falcon-H1-1.5B-Instruct-GGUFmediumHumanEval+falcon-h1-1-5b-instructtiiuae234tiiuae/Falcon-H1-1.5B-Instruct61.59
Falcon-H1-1.5B-Instruct-GGUFmediumHumanEval+falcon-h1-1-5b-instructunsloth235tiiuae/Falcon-H1-1.5B-Instruct61.59
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumHumanEval+falcon-h1-1-5b-instructtiiuae236tiiuae/Falcon-H1-1.5B-Instruct61.59
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumHumanEval+falcon-h1-1-5b-instructtiiuae237tiiuae/Falcon-H1-1.5B-Instruct61.59
NVIDIA-Nemotron-Nano-9B-v2-BasemediumHumanEval+ Pass@1nvidia-nemotron-nano-9b-v2-basenvidia238nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base61.03
NVIDIA-Nemotron-Nano-12B-v2-BasemediumHumanEval+ Pass@1nvidia-nemotron-nano-12b-v2-basenvidia239nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base61.03
Mamba-Codestral-7B-v0.1lowUnknownUnknownmistralai240mistralai/Mamba-Codestral-7B-v0.161
Phi-3-small-8k-instructmediumHumanEval<br>0-shotphi-3-small-8k-instructmicrosoft241microsoft/Phi-3-small-8k-instruct61
Nemotron-H-47B-Base-8KlowUnknownnemotron-h-56b-base-8knvidia242nvidia/Nemotron-H-56B-Base-8K61
Nemotron-H-56B-Base-8KlowUnknownnemotron-h-56b-base-8knvidia243nvidia/Nemotron-H-56B-Base-8K60.37
MobileLLM-Pro-basemediumHumanEvalmobilellm-pro-basefacebook244facebook/MobileLLM-Pro-base59.8
Nemotron-H-4B-Base-8KlowUnknownnemotron-h-4b-base-8knvidia245nvidia/Nemotron-H-4B-Base-8K59.8
MobileLLM-PromediumHumanEvalmobilellm-pro-basefacebook246facebook/MobileLLM-Pro-base59.8
Youtu-LLM-2B-GGUFmediumHumanEval+youtu-llm-2b-baseMungert247tencent/Youtu-LLM-2B-Base59.5
Falcon-H1-3B-BasemediumHumanEvalfalcon-h1-3b-basetiiuae248tiiuae/Falcon-H1-3B-Base59.15
Phi-3-small-128k-instructmediumHumanEval<br>0-shotphi-3-small-128k-instructmicrosoft249microsoft/Phi-3-small-128k-instruct59.1
Nemotron-H-8B-Base-8KlowUnknownnemotron-h-8b-base-8knvidia250nvidia/Nemotron-H-8B-Base-8K58.54
Phi-3-medium-128k-instructmediumHumanEval<br>0-shotphi-3-medium-128k-instructmicrosoft251microsoft/Phi-3-medium-128k-instruct58.5
WizardCoder-Python-7B-V1.0-GGUFhighUnknownUnknownTheBloke252WizardLM/WizardCoder-Python-7b-V1.055.5
openchat-3.5-0106-GGUFhighBBH CoTopenchat-3-5-0106TheBloke253openchat/openchat-3.5-010655.5
openchat-3.5-0106-GGUFhighUnknownopenchat-3-5-0106TheBloke254openchat/openchat-3.5-010655.5
openchat_3.5-GGUFlowBBH CoTUnknownTheBloke255openchat/openchat_3.555.5
openchat_3.5-16k-GGUFlowUnknownUnknownTheBloke256NurtureAI/openchat_3.5-16k55.5
openchat_3.5-16k-GGUFlowBBH CoTUnknownTheBloke257NurtureAI/openchat_3.5-16k55.5
openchat_3.5-GPTQlowBBH CoTUnknownTheBloke258openchat/openchat_3.555.5
OLMoE-1B-7B-0125-Instructmedium**HumanEval+**olmoe-1b-7b-0125-dpoallenai259allenai/OLMoE-1B-7B-0125-DPO54.37
OLMoE-1B-7B-0125-DPOmedium**HumanEval+**Unknownallenai260allenai/OLMoE-1B-7B-0125-SFT54.37
OLMoE-1B-7B-0125-SFTmedium**HumanEval+**olmoe-1b-7b-0125allenai261allenai/OLMoE-1B-7B-012554.37
Falcon-H1-3B-BasemediumHumanEval+falcon-h1-3b-basetiiuae262tiiuae/Falcon-H1-3B-Base53.66
Falcon-H1-1.5B-Deep-BasemediumHumanEvalfalcon-h1-1-5b-deep-basetiiuae263tiiuae/Falcon-H1-1.5B-Deep-Base52.44
Falcon-H1-0.5B-InstructmediumHumanEvalfalcon-h1-0-5b-basetiiuae264tiiuae/Falcon-H1-0.5B-Base51.83
speechless-tora-code-7B-v1.0-GGUFmedium-pythonUnknownTheBloke265uukuguy/speechless-tora-code-7b-v1.051.83
Falcon-H1-0.5B-Instruct-GGUFmediumHumanEvalfalcon-h1-0-5b-basetiiuae266tiiuae/Falcon-H1-0.5B-Base51.83
Falcon-H1-0.5B-Instruct-GGUFmediumHumanEvalfalcon-h1-0-5b-instructunsloth267tiiuae/Falcon-H1-0.5B-Instruct51.83
Falcon-H1-1.5B-BasemediumHumanEvalfalcon-h1-1-5b-basetiiuae268tiiuae/Falcon-H1-1.5B-Base50
CodeFuse-CodeLlama-34B-GGUFhighUnknownUnknownTheBloke269codefuse-ai/CodeFuse-CodeLlama-34B48.8
DeepSeek-V2-Chatmedium**HumanEval**deepseek-v2-chatdeepseek-ai270deepseek-ai/DeepSeek-V2-Chat48.2
DeepSeek-V2medium**HumanEval**deepseek-v2deepseek-ai271deepseek-ai/DeepSeek-V248.2
Qwen-14B-Chat-Int8highUnknownqwen-14bQwen272Qwen/Qwen-14B48.2
Qwen2-0.5BmediumHumanEvalqwen2-0-5bQwen273Qwen/Qwen2-0.5B47.6
Qwen2-1.5BmediumHumanEvalqwen2-1-5bQwen274Qwen/Qwen2-1.5B47.6
Falcon-H1-1.5B-Deep-BasemediumHumanEval+falcon-h1-1-5b-deep-basetiiuae275tiiuae/Falcon-H1-1.5B-Deep-Base46.34
Qwen2-72BmediumHumanEvalqwen2-72bQwen276Qwen/Qwen2-72B45.7
Qwen-14B-Chat-Int4highUnknownqwen-14bQwen277Qwen/Qwen-14B45.7
Falcon-H1-0.5B-InstructmediumHumanEval+falcon-h1-0-5b-basetiiuae278tiiuae/Falcon-H1-0.5B-Base45.12
Falcon-H1-0.5B-Instruct-GGUFmediumHumanEval+falcon-h1-0-5b-basetiiuae279tiiuae/Falcon-H1-0.5B-Base45.12
Falcon-H1-0.5B-Instruct-GGUFmediumHumanEval+falcon-h1-0-5b-instructunsloth280tiiuae/Falcon-H1-0.5B-Instruct45.12
Qwen2-57B-A14B-InstructmediumHumanEvalqwen2-57b-a14bQwen281Qwen/Qwen2-57B-A14B45.1
Qwen-14B-ChatlowUnknownqwen-14b-chatQwen282Qwen/Qwen-14B-Chat43.9
Qwen-14B-Chat-GPTQlowUnknownqwen-14b-chatTheBloke283Qwen/Qwen-14B-Chat43.9
Qwen-14B-Chat-AWQlowUnknownqwen-14b-chatTheBloke284Qwen/Qwen-14B-Chat43.9
internlm2-chat-7bmediumHumanEvalinternlm2-chat-7binternlm285internlm/internlm2-chat-7b43.3
internlm2-7bmediumHumanEvalUnknowninternlm286internlm/internlm2-7b43.3
internlm2-chat-20bmediumHumanEvalinternlm2-chat-20binternlm287internlm/internlm2-chat-20b43.3
internlm2-20bmediumHumanEvalUnknowninternlm288internlm/internlm2-20b43.3
internlm2-base-7bmediumHumanEvalUnknowninternlm289internlm/internlm2-base-7b43.3
internlm2-base-20bmediumHumanEvalUnknowninternlm290internlm/internlm2-base-20b43.3
internlm2-chat-7b-sftmediumHumanEvalinternlm2-chat-7b-sftinternlm291internlm/internlm2-chat-7b-sft43.3
internlm2-chat-20b-sftmediumHumanEvalinternlm2-chat-20b-sftinternlm292internlm/internlm2-chat-20b-sft43.3
Falcon-H1-1.5B-BasemediumHumanEval+falcon-h1-1-5b-basetiiuae293tiiuae/Falcon-H1-1.5B-Base42.68
gemma-3-270mmedium[HumanEval][]gemma-3-270mgoogle294google/gemma-3-270m41.5
gemma-3-270m-itmedium[HumanEval][]gemma-3-270mgoogle295google/gemma-3-270m41.5
gemma-3-270m-it-GGUFmedium[HumanEval][]gemma-3-270m-itunsloth296google/gemma-3-270m-it41.5
gemma-3-270m-itmedium[HumanEval][]gemma-3-270m-itunsloth297google/gemma-3-270m-it41.5
gemma-3-270m-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-270m-itunsloth298google/gemma-3-270m-it41.5
gemma-3-270m-it-qat-GGUFmedium[HumanEval][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth299google/gemma-3-270m-it-qat-q4_0-unquantized41.5
gemma-3-270mmedium[HumanEval][]gemma-3-270munsloth300google/gemma-3-270m41.5
gemma-3-270m-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-270m-itunsloth301google/gemma-3-270m-it41.5
gemma-3-270m-it-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-270m-itgoogle302google/gemma-3-270m-it41.5
gemma-3-270m-it-bnb-4bitmedium[HumanEval][]gemma-3-270m-itunsloth303google/gemma-3-270m-it41.5
gemma-3-270m-it-qatmedium[HumanEval][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth304google/gemma-3-270m-it-qat-q4_0-unquantized41.5
gemma-3-270m-bnb-4bitmedium[HumanEval][]gemma-3-270munsloth305google/gemma-3-270m41.5
gemma-3-270m-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-270mgoogle306google/gemma-3-270m41.5
gemma-3-270m-it-GGUFmedium[HumanEval][]gemma-3-270mMungert307google/gemma-3-270m41.5
gemma-3-270m-it-torchao-FP8medium[HumanEval][]gemma-3-270m-itunsloth308google/gemma-3-270m-it41.5
gemma-2-9b-itmedium[HumanEval][]gemma-2-9bgoogle309google/gemma-2-9b40.2
gemma-2-27b-itmedium[HumanEval][]gemma-2-27bgoogle310google/gemma-2-27b40.2
gemma-2-9bmedium[HumanEval][]gemma-2-9bgoogle311google/gemma-2-9b40.2
gemma-2-27bmedium[HumanEval][]gemma-2-27bgoogle312google/gemma-2-27b40.2
Qwen-7B-ChatlowUnknownqwen-7b-chatQwen313Qwen/Qwen-7B-Chat37.2
Qwen-7B-Chat-GPTQlowUnknownqwen-7b-chatTheBloke314Qwen/Qwen-7B-Chat37.2
Qwen-7B-Chat-AWQlowUnknownqwen-7b-chatTheBloke315Qwen/Qwen-7B-Chat37.2
gemma-3-1b-itmedium[HumanEval][]gemma-3-1b-ptgoogle316google/gemma-3-1b-pt36
gemma-3-4b-itmedium[HumanEval][]gemma-3-4b-ptgoogle317google/gemma-3-4b-pt36
gemma-3-12b-itmedium[HumanEval][]gemma-3-12b-ptgoogle318google/gemma-3-12b-pt36
gemma-3-27b-itmedium[HumanEval][]gemma-3-27b-ptgoogle319google/gemma-3-27b-pt36
gemma-3-4b-itmedium[HumanEval][]gemma-3-4b-ptunsloth320google/gemma-3-4b-it36
gemma-3-1b-it-GGUFmedium[HumanEval][]gemma-3-1b-ptggml-org321google/gemma-3-1b-it36
gemma-3-12b-it-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-12b-ptgoogle322google/gemma-3-12b-it36
gemma-3-1b-itmedium[HumanEval][]gemma-3-1b-ptunsloth323google/gemma-3-1b-it36
gemma-3-12b-it-GGUFmedium[HumanEval][]gemma-3-12b-ptunsloth324google/gemma-3-12b-it36
gemma-3-4b-ptmedium[HumanEval][]gemma-3-4b-ptgoogle325google/gemma-3-4b-pt36
gemma-3-12b-it-qat-GGUFmedium[HumanEval][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth326google/gemma-3-12b-it-qat-q4_0-unquantized36
gemma-3-27b-itmedium[HumanEval][]gemma-3-27b-itunsloth327google/gemma-3-27b-it36
gemma-3-1b-ptmedium[HumanEval][]gemma-3-1b-ptgoogle328google/gemma-3-1b-pt36
gemma-3-4b-it-GGUFmedium[HumanEval][]gemma-3-4b-ptggml-org329google/gemma-3-4b-it36
gemma-3-4b-it-GGUFmedium[HumanEval][]gemma-3-4b-ptunsloth330google/gemma-3-4b-it36
gemma-3-1b-it-GGUFmedium[HumanEval][]gemma-3-1b-ptunsloth331google/gemma-3-1b-it36
gemma-3-12b-ptmedium[HumanEval][]Unknowngoogle332google/gemma-3-12b36
gemma-3-12b-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-12b-ptunsloth333google/gemma-3-12b-it36
gemma-3-4b-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-4b-ptunsloth334google/gemma-3-4b-it36
gemma-3-12b-itmedium[HumanEval][]gemma-3-12b-ptunsloth335google/gemma-3-12b-pt36
gemma-3-27b-it-GGUFmedium[HumanEval][]gemma-3-27b-itunsloth336google/gemma-3-27b-it36
gemma-3-1b-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-1b-ptunsloth337google/gemma-3-1b-it36
gemma-3-4b-it-bnb-4bitmedium[HumanEval][]gemma-3-4b-ptunsloth338google/gemma-3-4b-it36
gemma-3-4b-ptmedium[HumanEval][]gemma-3-4b-ptunsloth339google/gemma-3-4b-pt36
gemma-3-27b-ptmedium[HumanEval][]gemma-3-27b-ptgoogle340google/gemma-3-27b-pt36
gemma-3-12b-it-bnb-4bitmedium[HumanEval][]gemma-3-12b-ptunsloth341google/gemma-3-12b-it36
gemma-3-27b-it-GGUFmedium[HumanEval][]gemma-3-27b-itggml-org342google/gemma-3-27b-it36
gemma-3-12b-ptmedium[HumanEval][]gemma-3-12b-ptunsloth343google/gemma-3-12b-pt36
gemma-3-27b-it-bnb-4bitmedium[HumanEval][]gemma-3-27b-itunsloth344google/gemma-3-27b-it36
gemma-3-27b-it-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-27b-itunsloth345google/gemma-3-27b-it36
gemma-3-1b-ptmedium[HumanEval][]gemma-3-1b-ptunsloth346google/gemma-3-1b-pt36
gemma-3-12b-it-GGUFmedium[HumanEval][]gemma-3-12b-ptggml-org347google/gemma-3-12b-it36
gemma-3-4b-it-qat-GGUFmedium[HumanEval][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth348google/gemma-3-4b-it-qat-q4_0-unquantized36
gemma-3-12b-it-qat-bnb-4bitmedium[HumanEval][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth349google/gemma-3-12b-it-qat-q4_0-unquantized36
gemma-3-12b-it-qat-int4-GGUFmedium[HumanEval][]gemma-3-12b-it-qat-int4-unquantizedunsloth350google/gemma-3-12b-it-qat-int4-unquantized36
gemma-3-4b-it-qat-q4_0-ggufmedium[HumanEval][]gemma-3-4b-ptgoogle351google/gemma-3-4b-it36
gemma-3-4b-it-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-4b-ptgoogle352google/gemma-3-4b-it36
gemma-3-4b-pt-bnb-4bitmedium[HumanEval][]gemma-3-4b-ptunsloth353google/gemma-3-4b-pt36
gemma-3-27b-it-qat-GGUFmedium[HumanEval][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth354google/gemma-3-27b-it-qat-q4_0-unquantized36
gemma-3-12b-it-qat-q4_0-ggufmedium[HumanEval][]gemma-3-12b-ptgoogle355google/gemma-3-12b-it36
gemma-3-1b-it-bnb-4bitmedium[HumanEval][]gemma-3-1b-ptunsloth356google/gemma-3-1b-it36
gemma-3-1b-it-qat-q4_0-ggufmedium[HumanEval][]gemma-3-1b-ptgoogle357google/gemma-3-1b-it36
gemma-3-4b-pt-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-4b-ptunsloth358google/gemma-3-4b-pt36
gemma-3-12b-it-qatmedium[HumanEval][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth359google/gemma-3-12b-it-qat-q4_0-unquantized36
gemma-3-1b-pt-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-1b-ptunsloth360google/gemma-3-1b-pt36
gemma-3-4b-it-FP8-Dynamicmedium[HumanEval][]gemma-3-4b-ptunsloth361google/gemma-3-4b-it36
gemma-3-12b-pt-bnb-4bitmedium[HumanEval][]gemma-3-12b-ptunsloth362google/gemma-3-12b-pt36
gemma-3-12b-it-qat-int4-unquantizedmedium[HumanEval][]gemma-3-12b-ptgoogle363google/gemma-3-12b-it36
gemma-3-27b-it-GGUFmedium[HumanEval][]gemma-3-27b-ptMungert364google/gemma-3-27b-pt36
gemma-3-27b-it-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-27b-itgoogle365google/gemma-3-27b-it36
gemma-3-12b-pt-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-12b-ptunsloth366google/gemma-3-12b-pt36
gemma-3-27b-it-qat-q4_0-ggufmedium[HumanEval][]gemma-3-27b-itgoogle367google/gemma-3-27b-it36
gemma-3-27b-it-qat-q4_0-GGUFmedium[HumanEval][]gemma-3-27b-itMungert368google/gemma-3-27b-it36
gemma-3-4b-it-qat-bnb-4bitmedium[HumanEval][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth369google/gemma-3-4b-it-qat-q4_0-unquantized36
gemma-3-1b-pt-bnb-4bitmedium[HumanEval][]gemma-3-1b-ptunsloth370google/gemma-3-1b-pt36
gemma-3-27b-ptmedium[HumanEval][]gemma-3-27b-ptunsloth371google/gemma-3-27b-pt36
gemma-3-12b-it-FP8-Dynamicmedium[HumanEval][]gemma-3-12b-ptunsloth372google/gemma-3-12b-pt36
gemma-3-1b-it-qat-int4-unquantizedmedium[HumanEval][]gemma-3-1b-ptgoogle373google/gemma-3-1b-it36
gemma-3-27b-pt-bnb-4bitmedium[HumanEval][]gemma-3-27b-ptunsloth374google/gemma-3-27b-pt36
gemma-3-12b-it-qat-unsloth-bnb-4bitmedium[HumanEval][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth375google/gemma-3-12b-it-qat-q4_0-unquantized36
gemma-3-12b-pt-qat-q4_0-ggufmedium[HumanEval][]gemma-3-12b-ptgoogle376google/gemma-3-12b-pt36
gemma-3-4b-it-qatmedium[HumanEval][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth377google/gemma-3-4b-it-qat-q4_0-unquantized36
gemma-3-1b-it-qat-q4_0-unquantizedmedium[HumanEval][]gemma-3-1b-ptgoogle378google/gemma-3-1b-it36
gemma-3-1b-it-qatmedium[HumanEval][]gemma-3-1b-it-qat-q4-0-unquantizedunsloth379google/gemma-3-1b-it-qat-q4_0-unquantized36
gemma-3-27b-it-qatmedium[HumanEval][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth380google/gemma-3-27b-it-qat-q4_0-unquantized36
gemma-3-4b-pt-qat-q4_0-ggufmedium[HumanEval][]gemma-3-4b-ptgoogle381google/gemma-3-4b-pt36
Falcon-H1-0.5B-BasemediumHumanEvalfalcon-h1-0-5b-basetiiuae382tiiuae/Falcon-H1-0.5B-Base35.98
internlm2_5-1_8bmediumHUMANEVALUnknowninternlm383internlm/internlm2_5-1_8b35.98
Qwen-7B-Chat-Int8highUnknownqwen-7bQwen384Qwen/Qwen-7B34.8
speechless-mistral-dolphin-orca-platypus-samantha-7B-GGUFmedium-pythonspeechless-mistral-dolphin-orca-platypus-samantha-7bTheBloke385uukuguy/speechless-mistral-dolphin-orca-platypus-samantha-7b34.15
Olmo-3-1025-7BlowUnknownolmo-3-1025-7ballenai386allenai/Olmo-3-1025-7B31.6
Olmo-Hybrid-7BlowUnknownolmo-hybrid-7ballenai387allenai/Olmo-Hybrid-7B31.6
Olmo-3-1025-7B-GGUFlowUnknownolmo-3-1025-7bMungert388allenai/Olmo-3-1025-7B31.6
Falcon-H1-0.5B-BasemediumHumanEval+falcon-h1-0-5b-basetiiuae389tiiuae/Falcon-H1-0.5B-Base31.1
bitnet-b1.58-2B-4T-ggufmediumHumanEval+bitnet-b1-58-2b-4t-bf16microsoft390microsoft/bitnet-b1.58-2B-4T-bf1631.1
bitnet-b1.58-2B-4TmediumHumanEval+bitnet-b1-58-2b-4tmicrosoft391microsoft/bitnet-b1.58-2B-4T31.1
bitnet-b1.58-2B-4T-bf16mediumHumanEval+bitnet-b1-58-2b-4t-bf16microsoft392microsoft/bitnet-b1.58-2B-4T-bf1631.1
Qwen-7B-Chat-Int4highUnknownqwen-7bQwen393Qwen/Qwen-7B29.9
Qwen2-7BmediumHumanEvalqwen2-7bQwen394Qwen/Qwen2-7B29.3
Qwen2-57B-A14BmediumHumanEvalqwen2-57b-a14bQwen395Qwen/Qwen2-57B-A14B29.3
Qwen-1_8B-Chat-Int8highUnknownqwen-1-8bQwen396Qwen/Qwen-1_8B27.4
Ministral-8B-Instruct-2410lowUnknownministral-8b-instruct-2410mistralai397mistralai/Ministral-8B-Instruct-241026.8
DeepSeek-V2-Lite-Chatmedium**HumanEval**deepseek-v2-lite-chatdeepseek-ai398deepseek-ai/DeepSeek-V2-Lite-Chat26.2
DeepSeek-V2-Litemedium**HumanEval**deepseek-v2-litedeepseek-ai399deepseek-ai/DeepSeek-V2-Lite26.2
internlm2-chat-1_8bmediumHumanEvalinternlm2-chat-1-8binternlm400internlm/internlm2-chat-1_8b25
internlm2-1_8bmediumHumanEvalUnknowninternlm401internlm/internlm2-1_8b25
Qwen-1_8B-Chat-Int4highUnknownqwen-1-8bQwen402Qwen/Qwen-1_8B25
internlm2-chat-1_8b-sftmediumHumanEvalinternlm2-chat-1-8b-sftinternlm403internlm/internlm2-chat-1_8b-sft25
Nanbeige-16B-Base-GGUFhighUnknownUnknownTheBloke404Nanbeige/Nanbeige-16B-Base24.56
gemma-1.1-2b-itmedium[HumanEval](https://arxiv.org/abs/2107.03374)gemma-1-1-2b-itgoogle405google/gemma-1.1-2b-it22
gemma-2bmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle406google/gemma-2b-it-pytorch22
gemma-2b-itmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle407google/gemma-2b-it-pytorch22
gemma-7bmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle408google/gemma-7b-it-quant-pytorch22
gemma-7b-itmedium[HumanEval](https://arxiv.org/abs/2107.03374)gemma-7bgoogle409google/gemma-7b22
gemma-1.1-7b-itmedium[HumanEval](https://arxiv.org/abs/2107.03374)gemma-1-1-7b-itgoogle410google/gemma-1.1-7b-it22
gemma-2b-AWQmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle411google/gemma-2b-it-pytorch22
gemma-7bmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknownalpindale412google/gemma-7b-it-quant-pytorch22
gemma-2bmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknownalpindale413google/gemma-2b-it-pytorch22
gemma-7b-AWQmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle414google/gemma-7b-it-quant-pytorch22
gemma-7b-itmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknownalpindale415google/gemma-7b-it-quant-pytorch22
gemma-7b-GGUFmedium[HumanEval](https://arxiv.org/abs/2107.03374)Unknowngoogle416google/gemma-7b-it-quant-pytorch22
WizardCoder-15B-1.0-GPTQlowUnknownUnknownTheBloke417—18.3
gemma-2-2b-itmedium[HumanEval][]gemma-2-2bgoogle418google/gemma-2-2b17.7
gemma-2-2bmedium[HumanEval][]gemma-2-2bgoogle419google/gemma-2-2b17.7
internlm-chat-7bmediumHumanEvalUnknowninternlm420internlm/internlm-chat-7b14
internlm-7bmediumHumanEvalUnknowninternlm421internlm/internlm-7b14
Qwen-72BlowUnknownqwen-72bQwen422Qwen/Qwen-72B12.8
Qwen-7BlowUnknownqwen-7bQwen423Qwen/Qwen-7B12.8
Qwen-14BlowUnknownqwen-14bQwen424Qwen/Qwen-14B12.8
Qwen2-1.5B-InstructmediumHumanEvalqwen2-1-5b-instructQwen425Qwen/Qwen2-1.5B-Instruct9.1
Qwen2-0.5B-InstructmediumHumanEvalqwen2-0-5bQwen426Qwen/Qwen2-0.5B9.1
t5gemma-2-270m-270mmedium[HumanEval]t5gemma-2-270m-270mgoogle427google/t5gemma-2-270m-270m4.3
t5gemma-2-4b-4bmedium[HumanEval]t5gemma-2-4b-4bgoogle428google/t5gemma-2-4b-4b4.3
t5gemma-2-1b-1bmedium[HumanEval]t5gemma-2-1b-1bgoogle429google/t5gemma-2-1b-1b4.3
Provenance & metadata (1)

source

kind
leaderboard-scrape
paths
benchmarks/humaneval.html