Local AI Registry

Leaderboard

MBPP

mbpp
JSON APIBack to leaderboard search

Record

Category
coding
Scored models
320

Scores

ModelScore
nvidia/Nemotron-H-47B-Reasoning-128K90.2
nvidia/Nemotron-H-47B-Reasoning-128K90.2
inclusionAI/LLaDA2.0-flash86.65
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF1685.97
ByteDance-Seed/Seed-Coder-8B-Instruct85.2
ByteDance-Seed/Seed-Coder-8B-Instruct85.2
inclusionAI/LLaDA2.0-flash-preview85.01
nvidia/Nemotron-H-8B-Reasoning-128K84.1
nvidia/Nemotron-H-8B-Reasoning-128K84.1
inclusionAI/LLaDA2.0-mini-preview84.07
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
tiiuae/Falcon-H1-34B-Base83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Base83.33
WizardLM/WizardCoder-33B-V1.183
WizardLM/WizardCoder-33B-V1.183
Qwen/Qwen2-72B82.3
tiiuae/Falcon-H1-7B-Base80.69

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category
coding
id
mbpp
name
MBPP
schema version
local-ai-registry/v1

rows

variantconfcontextmodel idorgrankrootscore
Nemotron-H-47B-Reasoning-128KmediumMBPPnemotron-h-47b-reasoning-128knvidia1nvidia/Nemotron-H-47B-Reasoning-128K90.2
Nemotron-H-47B-Reasoning-128K-FP8mediumMBPPnemotron-h-47b-reasoning-128knvidia2nvidia/Nemotron-H-47B-Reasoning-128K90.2
LLaDA2.0-flashmediumMBPPllada2-0-flashinclusionAI3inclusionAI/LLaDA2.0-flash86.65
NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16mediumMBPP-Sanitizednvidia-nemotron-3-ultra-550b-a55b-base-bf16nvidia4nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF1685.97
Seed-Coder-8B-Instruct-GGUFhighMBPPseed-coder-8b-instructunsloth5ByteDance-Seed/Seed-Coder-8B-Instruct85.2
Seed-Coder-8B-Instruct-unsloth-bnb-4bithighMBPPseed-coder-8b-instructunsloth6ByteDance-Seed/Seed-Coder-8B-Instruct85.2
LLaDA2.0-flash-previewmediummbppllada2-0-flash-previewinclusionAI7inclusionAI/LLaDA2.0-flash-preview85.01
Nemotron-H-8B-Reasoning-128KmediumMBPPnemotron-h-8b-reasoning-128knvidia8nvidia/Nemotron-H-8B-Reasoning-128K84.1
Nemotron-H-8B-Reasoning-128K-FP8mediumMBPPnemotron-h-8b-reasoning-128knvidia9nvidia/Nemotron-H-8B-Reasoning-128K84.1
LLaDA2.0-mini-previewmediummbppllada2-0-mini-previewinclusionAI10inclusionAI/LLaDA2.0-mini-preview84.07
Qwen2.5-VL-32B-InstructhighMBPPqwen2-5-vl-32b-instructQwen11Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-GGUFhighMBPPqwen2-5-vl-32b-instructunsloth12Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-bnb-4bithighMBPPqwen2-5-vl-32b-instructunsloth13Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-GGUFhighMBPPqwen2-5-vl-32b-instructMungert14Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bithighMBPPqwen2-5-vl-32b-instructunsloth15Qwen/Qwen2.5-VL-32B-Instruct84
Falcon-H1-34B-InstructmediumMBPPfalcon-h1-34b-basetiiuae16tiiuae/Falcon-H1-34B-Base83.86
Falcon-H1-34B-Instruct-GGUFmediumMBPPfalcon-h1-34b-instructtiiuae17tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-Instruct-GGUFmediumMBPPfalcon-h1-34b-instructunsloth18tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-34b-instructtiiuae19tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-BasemediumMBPPfalcon-h1-34b-basetiiuae20tiiuae/Falcon-H1-34B-Base83.33
WizardCoder-33B-V1.1-GGUFlowMBPPwizardcoder-33b-v1-1TheBloke21WizardLM/WizardCoder-33B-V1.183
WizardCoder-33B-V1.1-GPTQlowMBPPwizardcoder-33b-v1-1TheBloke22WizardLM/WizardCoder-33B-V1.183
Qwen2-72B-InstructmediumMBPPqwen2-72bQwen23Qwen/Qwen2-72B82.3
Falcon-H1-7B-InstructmediumMBPPfalcon-h1-7b-basetiiuae24tiiuae/Falcon-H1-7B-Base80.69
Falcon-H1-7B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-7b-instructtiiuae25tiiuae/Falcon-H1-7B-Instruct80.69
Falcon-H1-7B-Instruct-GGUFmediumMBPPfalcon-h1-7b-instructtiiuae26tiiuae/Falcon-H1-7B-Instruct80.69
Falcon-H1-7B-Instruct-GGUFmediumMBPPfalcon-h1-7b-instructunsloth27tiiuae/Falcon-H1-7B-Instruct80.69
GRIN-MoEmediumMBPPgrin-moemicrosoft28microsoft/GRIN-MoE80.3
Mistral-Large-Instruct-2407mediumMBPP Basemistral-large-instruct-2407mistralai29mistralai/Mistral-Large-Instruct-240780
Falcon-H1-3B-InstructmediumMBPPfalcon-h1-3b-basetiiuae30tiiuae/Falcon-H1-3B-Base79.63
Falcon-H1-3B-Instruct-GGUFmediumMBPPfalcon-h1-3b-basetiiuae31tiiuae/Falcon-H1-3B-Base79.63
Falcon-H1-3B-Instruct-GGUFmediumMBPPfalcon-h1-3b-instructunsloth32tiiuae/Falcon-H1-3B-Instruct79.63
LLaDA2.0-minimediumMBPPllada2-0-miniinclusionAI33inclusionAI/LLaDA2.0-mini78.92
Nemotron-H-4B-Instruct-128KlowMBPP 0-shotnemotron-h-4b-base-8knvidia34nvidia/Nemotron-H-4B-Base-8K78.6
Falcon-H1-7B-BasemediumMBPPfalcon-h1-7b-basetiiuae35tiiuae/Falcon-H1-7B-Base78.57
NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16mediumMBPP-Sanitizednvidia-nemotron-3-super-120b-a12b-base-bf16nvidia36nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF1678.38
Nemotron-H-56B-Base-8KlowMBPP (sanitized) 3-shotnemotron-h-56b-base-8knvidia37nvidia/Nemotron-H-56B-Base-8K77.82
MiniMax-Text-01-hfmediumMBPP +minimax-text-01-hfMiniMaxAI38MiniMaxAI/MiniMax-Text-01-hf76.2
MiniMax-Text-01mediumMBPP +minimax-text-01MiniMaxAI39MiniMaxAI/MiniMax-Text-0176.2
Nemotron-H-47B-Base-8KlowMBPP(sanitized) 3-shotnemotron-h-56b-base-8knvidia40nvidia/Nemotron-H-56B-Base-8K75.9
codegeex4-all-9bhigh**MBPP**Unknownzai-org41zai-org/codegeex4-all-9b75.7
codegeex4-all-9b-GGUFhigh**MBPP**Unknownzai-org42zai-org/codegeex4-all-9b75.7
Phi-3-medium-4k-instructmediumMBPP<br>3-shotphi-3-medium-4k-instructmicrosoft43microsoft/Phi-3-medium-4k-instruct75.2
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai44mistralai/Mistral-Small-3.1-24B-Base-250374.71
Nemotron-Labs-TwoTower-30B-A3B-Base-BF16mediumMBPP-Sanitized (3-shot)nemotron-labs-twotower-30b-a3b-base-bf16nvidia45nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF1674.71
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth46mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth47mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth48mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth49mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert50mistralai/Mistral-Small-3.1-24B-Base-250374.71
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai51mistralai/Mistral-Small-3.1-24B-Base-250374.63
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth52mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth53mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth54mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth55mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert56mistralai/Mistral-Small-3.1-24B-Base-250374.63
Mistral-Small-3.2-24B-Instruct-2506-FP8lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth57mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Qwen2-72BmediumMBPPqwen2-72bQwen58Qwen/Qwen2-72B73.9
Phi-3-medium-128k-instructmediumMBPP<br>3-shotphi-3-medium-128k-instructmicrosoft59microsoft/Phi-3-medium-128k-instruct73.8
Ling-Coder-lite-GPTQ-Int8mediumMBPP-Pythonling-coder-liteinclusionAI60inclusionAI/Ling-Coder-lite73.28
Ling-Coder-lite-GPTQ-Int8mediumEvalPlus-MBPP+ling-coder-liteinclusionAI61inclusionAI/Ling-Coder-lite73.28
Qwen2.5-Omni-3BmediumMBPPqwen2-5-omni-3bQwen62Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-7BmediumMBPPqwen2-5-omni-7bQwen63Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-7B-GGUFmediumMBPPqwen2-5-omni-7bunsloth64Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-3B-GGUFmediumMBPPqwen2-5-omni-3bunsloth65Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-3B-GGUFmediumMBPPqwen2-5-omni-3bMungert66Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-7BmediumMBPPqwen2-5-omni-7bunsloth67Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-7B-GGUFmediumMBPPqwen2-5-omni-7bMungert68Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-3BmediumMBPPqwen2-5-omni-3bunsloth69Qwen/Qwen2.5-Omni-3B73.2
NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16mediumMBPP-Sanitized (3-shot)nvidia-nemotron-3-nano-30b-a3b-base-bf16nvidia70nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF1673.15
Hunyuan-A13B-Instruct-GGUFmediumMBPPhunyuan-a13b-instructunsloth71tencent/Hunyuan-A13B-Instruct72.6
Phi-3-small-8k-instructmediumMBPP<br>3-shotphi-3-small-8k-instructmicrosoft72microsoft/Phi-3-small-8k-instruct71.7
Falcon-H1-34B-InstructmediumMBPP+falcon-h1-34b-basetiiuae73tiiuae/Falcon-H1-34B-Base71.43
Falcon-H1-34B-Instruct-GGUFmediumMBPP+falcon-h1-34b-instructtiiuae74tiiuae/Falcon-H1-34B-Instruct71.43
Falcon-H1-3B-BasemediumMBPPfalcon-h1-3b-basetiiuae75tiiuae/Falcon-H1-3B-Base71.43
Falcon-H1-34B-Instruct-GGUFmediumMBPP+falcon-h1-34b-instructunsloth76tiiuae/Falcon-H1-34B-Instruct71.43
Falcon-H1-34B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-34b-instructtiiuae77tiiuae/Falcon-H1-34B-Instruct71.43
Nemotron-H-8B-Reasoning-128KmediumMBPP\_PLUSnemotron-h-8b-reasoning-128knvidia78nvidia/Nemotron-H-8B-Reasoning-128K71.4
Nemotron-H-8B-Reasoning-128K-FP8mediumMBPP\_PLUSnemotron-h-8b-reasoning-128knvidia79nvidia/Nemotron-H-8B-Reasoning-128K71.4
Falcon-H1-1.5B-Deep-BasemediumMBPPfalcon-h1-1-5b-deep-basetiiuae80tiiuae/Falcon-H1-1.5B-Deep-Base70.9
Falcon-H1-34B-BasemediumMBPP+falcon-h1-34b-basetiiuae81tiiuae/Falcon-H1-34B-Base70.37
Phi-3-small-128k-instructmediumMBPP<br>3-shotphi-3-small-128k-instructmicrosoft82microsoft/Phi-3-small-128k-instruct70.3
Olmo-3-1125-32BlowMBPPolmo-3-1125-32ballenai83allenai/Olmo-3-1125-32B69.8
Mistral-Small-24B-Base-2501medium[MBPP][]mistral-small-24b-base-2501mistralai84mistralai/Mistral-Small-24B-Base-250169.64
Mistral-Large-Instruct-2407mediumMBPP Plusmistral-large-instruct-2407mistralai85mistralai/Mistral-Large-Instruct-240769
Falcon-H1-7B-InstructmediumMBPP+falcon-h1-7b-basetiiuae86tiiuae/Falcon-H1-7B-Base68.78
Falcon-H1-7B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-7b-instructtiiuae87tiiuae/Falcon-H1-7B-Instruct68.78
Falcon-H1-7B-Instruct-GGUFmediumMBPP+falcon-h1-7b-instructtiiuae88tiiuae/Falcon-H1-7B-Instruct68.78
Falcon-H1-7B-Instruct-GGUFmediumMBPP+falcon-h1-7b-instructunsloth89tiiuae/Falcon-H1-7B-Instruct68.78
DeepSeek-V2-Chatmedium**MBPP**deepseek-v2-chatdeepseek-ai90deepseek-ai/DeepSeek-V2-Chat68.6
DeepSeek-V2medium**MBPP**deepseek-v2deepseek-ai91deepseek-ai/DeepSeek-V268.6
Olmo-3.1-32B-ThinkmediumMBPP+olmo-3-32b-think-dpoallenai92allenai/Olmo-3-32B-Think-DPO68.3
Olmo-3.1-32B-Think-GGUFmediumMBPP+olmo-3-1-32b-thinkunsloth93allenai/Olmo-3.1-32B-Think68.3
Falcon-H1-1.5B-Deep-InstructmediumMBPPfalcon-h1-1-5b-deep-basetiiuae94tiiuae/Falcon-H1-1.5B-Deep-Base68.25
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-deep-instructtiiuae95tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-deep-instructunsloth96tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-InstructmediumMBPPfalcon-h1-1-5b-deep-instructunsloth97tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-1-5b-deep-instructtiiuae98tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Qwen2-7B-InstructmediumMBPPqwen2-7bQwen99Qwen/Qwen2-7B67.9
Mamba-Codestral-7B-v0.1lowMBPPmamba-codestral-7b-v0-1mistralai100mistralai/Mamba-Codestral-7B-v0.167.7
Falcon-H1-3B-InstructmediumMBPP+falcon-h1-3b-basetiiuae101tiiuae/Falcon-H1-3B-Base67.46
Falcon-H1-3B-Instruct-GGUFmediumMBPP+falcon-h1-3b-basetiiuae102tiiuae/Falcon-H1-3B-Base67.46
Falcon-H1-3B-Instruct-GGUFmediumMBPP+falcon-h1-3b-instructunsloth103tiiuae/Falcon-H1-3B-Instruct67.46
Falcon-H1-7B-BasemediumMBPP+falcon-h1-7b-basetiiuae104tiiuae/Falcon-H1-7B-Base67.2
Olmo-3-32B-ThinkmediumMBPP+olmo-3-32b-think-dpoallenai105allenai/Olmo-3-32B-Think-DPO66.7
Olmo-3-32B-Think-SFTmediumMBPP+olmo-3-1125-32ballenai106allenai/Olmo-3-1125-32B66.7
Olmo-3-32B-Think-GGUFmediumMBPP+olmo-3-32b-think-dpounsloth107allenai/Olmo-3-32B-Think66.7
Olmo-3-32B-Think-DPOmediumMBPP+olmo-3-32b-think-sftallenai108allenai/Olmo-3-32B-Think-SFT66.7
Olmo-3-32B-Think-unsloth-bnb-4bitmediumMBPP+olmo-3-32b-think-dpounsloth109allenai/Olmo-3-32B-Think66.7
Nemotron-H-8B-Base-8KlowMBPP (sanitized) 3-shotnemotron-h-8b-base-8knvidia110nvidia/Nemotron-H-8B-Base-8K65.37
Falcon-H1-1.5B-BasemediumMBPPfalcon-h1-1-5b-basetiiuae111tiiuae/Falcon-H1-1.5B-Base65.08
Nemotron-H-4B-Base-8KlowMBPP(sanitized) 3-shotnemotron-h-4b-base-8knvidia112nvidia/Nemotron-H-4B-Base-8K65
Falcon-H1-1.5B-InstructmediumMBPPfalcon-h1-1-5b-basetiiuae113tiiuae/Falcon-H1-1.5B-Base64.81
Falcon-H1-1.5B-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-instructtiiuae114tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-instructunsloth115tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-1-5b-instructtiiuae116tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumMBPPfalcon-h1-1-5b-instructtiiuae117tiiuae/Falcon-H1-1.5B-Instruct64.81
XwinCoder-34B-GGUFhighMBPP pass@1UnknownTheBloke118Xwin-LM/XwinCoder-34B64.8
Qwen2-57B-A14BmediumMBPPqwen2-57b-a14bQwen119Qwen/Qwen2-57B-A14B63.9
AceInstruct-7BmediumMBPPaceinstruct-7bnvidia120nvidia/AceInstruct-7B63.2
AceInstruct-1.5BmediumMBPPaceinstruct-1-5bnvidia121nvidia/AceInstruct-1.5B63.2
AceInstruct-72BmediumMBPPaceinstruct-72bnvidia122nvidia/AceInstruct-72B63.2
AceInstruct-1.5B-GGUFmediumMBPPaceinstruct-1-5bMungert123nvidia/AceInstruct-1.5B63.2
wavecoder-ultra-6.7blowMBPP(500)Unknownmicrosoft124microsoft/wavecoder-ultra-6.7b63
wavecoder-ds-6.7blowMBPP(500)Unknownmicrosoft125microsoft/wavecoder-ds-6.7b63
wavecoder-pro-6.7blowMBPP(500)Unknownmicrosoft126microsoft/wavecoder-pro-6.7b63
NVIDIA-Nemotron-Nano-9B-v2-BasemediumMBPP+ Pass@1nvidia-nemotron-nano-9b-v2-basenvidia127nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base61.55
NVIDIA-Nemotron-Nano-12B-v2-BasemediumMBPP+ Pass@1nvidia-nemotron-nano-12b-v2-basenvidia128nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base61.55
Olmo-3.1-32B-InstructmediumMBPP+olmo-3-1-32b-instruct-dpoallenai129allenai/Olmo-3.1-32B-Instruct-DPO61.5
Olmo-3.1-32B-Instruct-SFTmediumMBPP+olmo-3-1125-32ballenai130allenai/Olmo-3-1125-32B61.5
Olmo-3.1-32B-Instruct-DPOmediumMBPP+olmo-3-1-32b-instruct-sftallenai131allenai/Olmo-3.1-32B-Instruct-SFT61.5
Olmo-3.1-32B-Instruct-GGUFmediumMBPP+olmo-3-1-32b-instructunsloth132allenai/Olmo-3.1-32B-Instruct61.5
Olmo-3.1-32B-InstructmediumMBPP+olmo-3-1-32b-instructunsloth133allenai/Olmo-3.1-32B-Instruct61.5
WizardCoder-Python-34B-V1.0-GGUFhighMBPPUnknownTheBloke134WizardLM/WizardCoder-Python-34B-V1.061.2
WizardLM-70B-V1.0-GPTQlowMBPPUnknownTheBloke135WizardLM/WizardLM-70B-V1.061.2
WizardLM-13B-V1.2-GGUFlowMBPPUnknownTheBloke136WizardLM/WizardLM-13B-V1.261.2
WizardMath-13B-V1.0-GGUFlowMBPPUnknownTheBloke137WizardLM/WizardMath-13B-V1.061.2
WizardLM-70B-V1.0-GGUFlowMBPPUnknownTheBloke138WizardLM/WizardLM-70B-V1.061.2
WizardMath-7B-V1.0-GGUFlowMBPPUnknownTheBloke139WizardLM/WizardMath-7b-V1.061.2
WizardLM-13B-V1.1-GGUFlowMBPPUnknownTheBloke140WizardLM/WizardLM-13B-V1.161.2
wizardLM-7B-GGUFlowMBPPUnknownTheBloke141WizardLM/WizardLM-7B-V1.061.2
WizardMath-70B-V1.0-GGUFlowMBPPUnknownTheBloke142WizardLM/WizardMath-70B-V1.061.2
WizardLM-30B-GGUFlowMBPPUnknownTheBloke143WizardLM/WizardLM-30B-V1.061.2
WizardLM-13B-V1.2-GPTQlowMBPPUnknownTheBloke144WizardLM/WizardLM-13B-V1.261.2
WizardLM-13B-V1.1-GPTQlowMBPPUnknownTheBloke145WizardLM/WizardLM-13B-V1.161.2
WizardCoder-Python-34B-V1.0-GPTQhighMBPPUnknownTheBloke146WizardLM/WizardCoder-Python-34B-V1.061.2
Falcon-H1-1.5B-Deep-BasemediumMBPP+falcon-h1-1-5b-deep-basetiiuae147tiiuae/Falcon-H1-1.5B-Deep-Base60.32
XwinCoder-13B-GGUFhighMBPP pass@1UnknownTheBloke148Xwin-LM/XwinCoder-13B60.1
Qwen2-57B-A14B-InstructmediumMBPPqwen2-57b-a14bQwen149Qwen/Qwen2-57B-A14B59.5
Falcon-H1-3B-BasemediumMBPP+falcon-h1-3b-basetiiuae150tiiuae/Falcon-H1-3B-Base57.94
Falcon-H1-1.5B-Deep-InstructmediumMBPP+falcon-h1-1-5b-deep-basetiiuae151tiiuae/Falcon-H1-1.5B-Deep-Base56.61
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-deep-instructtiiuae152tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-deep-instructunsloth153tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-InstructmediumMBPP+falcon-h1-1-5b-deep-instructunsloth154tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-1-5b-deep-instructtiiuae155tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
gemma-3n-E2B-itmedium[MBPP][]gemma-3n-e4bgoogle156google/gemma-3n-E4B-it56.6
gemma-3n-E4B-itmedium[MBPP][]gemma-3n-e4bgoogle157google/gemma-3n-E4B56.6
gemma-3n-E4B-it-litert-lmmedium[MBPP][]gemma-3n-e4bgoogle158google/gemma-3n-E4B-it-litert-lm56.6
gemma-3n-E2B-it-GGUFmedium[MBPP][]gemma-3n-e2b-itunsloth159google/gemma-3n-E2B-it56.6
gemma-3n-E4B-itmedium[MBPP][]gemma-3n-e4bunsloth160google/gemma-3n-E4B-it56.6
gemma-3n-E2Bmedium[MBPP][]gemma-3n-e4bgoogle161google/gemma-3n-E4B56.6
gemma-3n-E2B-it-litert-lmmedium[MBPP][]gemma-3n-e2b-itgoogle162google/gemma-3n-E2B-it-litert-lm56.6
gemma-3n-E4B-it-GGUFmedium[MBPP][]gemma-3n-e4bunsloth163google/gemma-3n-E4B-it56.6
gemma-3n-E4Bmedium[MBPP][]gemma-3n-e4bgoogle164google/gemma-3n-E4B56.6
gemma-3n-E2B-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3n-e2b-itunsloth165google/gemma-3n-E2B-it56.6
gemma-3n-E2B-it-litert-preview-GGUFmedium[MBPP][]gemma-3n-e2b-itunsloth166google/gemma-3n-E2B-it-litert-preview56.6
gemma-3n-E4B-it-litert-preview-GGUFmedium[MBPP][]gemma-3n-e4bunsloth167google/gemma-3n-E4B-it-litert-preview56.6
gemma-3n-E4B-unsloth-bnb-4bitmedium[MBPP][]gemma-3n-e4bunsloth168google/gemma-3n-E4B-it56.6
gemma-3n-E2B-it-ONNXmedium[MBPP][]gemma-3n-e2b-itonnx-community169google/gemma-3n-E2B-it56.6
gemma-3n-E4B-it-GGUFmedium[MBPP][]gemma-3n-e4bMungert170google/gemma-3n-E4B56.6
gemma-3n-E4B-it-absolute-heresy-MPOA-imatrix-GGUFmedium[MBPP][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious171MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA56.6
gemma-3n-E4B-it-absolute-heresy-GGUFmedium[MBPP][]gemma-3n-e4bMuXodious172google/gemma-3n-E4B-it56.6
gemma-3n-E4B-it-absolute-heresy-MPOA-GGUFmedium[MBPP][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious173MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA56.6
gemma-3n-E2B-it-GGUFmedium[MBPP][]gemma-3n-e4bMungert174google/gemma-3n-E4b-it56.6
Falcon-H1-1.5B-InstructmediumMBPP+falcon-h1-1-5b-basetiiuae175tiiuae/Falcon-H1-1.5B-Base56.35
Falcon-H1-1.5B-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-instructtiiuae176tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-instructunsloth177tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-1-5b-instructtiiuae178tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumMBPP+falcon-h1-1-5b-instructtiiuae179tiiuae/Falcon-H1-1.5B-Instruct56.35
WizardCoder-Python-13B-V1.0-GGUFhighMBPPUnknownTheBloke180WizardLM/WizardCoder-Python-13B-V1.055.6
WizardCoder-Python-13B-V1.0-GPTQhighMBPPUnknownTheBloke181WizardLM/WizardCoder-Python-13B-V1.055.6
Falcon-H1-1.5B-BasemediumMBPP+falcon-h1-1-5b-basetiiuae182tiiuae/Falcon-H1-1.5B-Base55.03
Qwen2-0.5BmediumMBPPqwen2-0-5bQwen183Qwen/Qwen2-0.5B55
Qwen2-1.5BmediumMBPPqwen2-1-5bQwen184Qwen/Qwen2-1.5B55
gemma-2-9b-itmedium[MBPP][]gemma-2-9bgoogle185google/gemma-2-9b52.4
gemma-2-27b-itmedium[MBPP][]gemma-2-27bgoogle186google/gemma-2-27b52.4
gemma-2-9bmedium[MBPP][]gemma-2-9bgoogle187google/gemma-2-9b52.4
gemma-2-27bmedium[MBPP][]gemma-2-27bgoogle188google/gemma-2-27b52.4
Falcon-H1-0.5B-BasemediumMBPPfalcon-h1-0-5b-basetiiuae189tiiuae/Falcon-H1-0.5B-Base52.12
internlm2-chat-7bmediumMBPP(Sanitized)internlm2-chat-7binternlm190internlm/internlm2-chat-7b51.8
internlm2-7bmediumMBPP(Sanitized)Unknowninternlm191internlm/internlm2-7b51.8
internlm2-chat-20bmediumMBPP(Sanitized)internlm2-chat-20binternlm192internlm/internlm2-chat-20b51.8
internlm2-20bmediumMBPP(Sanitized)Unknowninternlm193internlm/internlm2-20b51.8
internlm2-base-7bmediumMBPP(Sanitized)Unknowninternlm194internlm/internlm2-base-7b51.8
internlm2-base-20bmediumMBPP(Sanitized)Unknowninternlm195internlm/internlm2-base-20b51.8
internlm2-chat-7b-sftmediumMBPP(Sanitized)internlm2-chat-7b-sftinternlm196internlm/internlm2-chat-7b-sft51.8
internlm2-chat-20b-sftmediumMBPP(Sanitized)internlm2-chat-20b-sftinternlm197internlm/internlm2-chat-20b-sft51.8
WizardCoder-Python-7B-V1.0-GGUFhighMBPPUnknownTheBloke198WizardLM/WizardCoder-Python-7b-V1.051.6
Youtu-LLM-2B-GGUFmediumMBPPyoutu-llm-2b-baseMungert199tencent/Youtu-LLM-2B-Base51.5
Qwen2-7BmediumMBPPqwen2-7bQwen200Qwen/Qwen2-7B51.1
Ministral-8B-Instruct-2410lowMBPP pass@1ministral-8b-instruct-2410mistralai201mistralai/Ministral-8B-Instruct-241050.2
MobileLLM-Pro-basemediumMBPPmobilellm-pro-basefacebook202facebook/MobileLLM-Pro-base46.8
MobileLLM-PromediumMBPPmobilellm-pro-basefacebook203facebook/MobileLLM-Pro-base46.8
gemma-3-1b-itmedium[MBPP][]gemma-3-1b-itgoogle204google/gemma-3-1b-pt46
gemma-3-4b-itmedium[MBPP][]gemma-3-4b-itgoogle205google/gemma-3-4b-pt46
gemma-3-12b-itmedium[MBPP][]gemma-3-12b-ptgoogle206google/gemma-3-12b-pt46
gemma-3-27b-itmedium[MBPP][]gemma-3-27b-ptgoogle207google/gemma-3-27b-pt46
gemma-3-4b-itmedium[MBPP][]gemma-3-4b-itunsloth208google/gemma-3-4b-it46
gemma-3-1b-it-GGUFmedium[MBPP][]gemma-3-1b-itggml-org209google/gemma-3-1b-it46
gemma-3-12b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-12b-itgoogle210google/gemma-3-12b-it46
gemma-3-1b-itmedium[MBPP][]gemma-3-1b-itunsloth211google/gemma-3-1b-it46
gemma-3-12b-it-GGUFmedium[MBPP][]gemma-3-12b-itunsloth212google/gemma-3-12b-it46
gemma-3-4b-ptmedium[MBPP][]gemma-3-4b-ptgoogle213google/gemma-3-4b-pt46
gemma-3-12b-it-qat-GGUFmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth214google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-27b-itmedium[MBPP][]gemma-3-27b-itunsloth215google/gemma-3-27b-it46
gemma-3-1b-ptmedium[MBPP][]gemma-3-1b-ptgoogle216google/gemma-3-1b-pt46
gemma-3-4b-it-GGUFmedium[MBPP][]gemma-3-4b-itggml-org217google/gemma-3-4b-it46
gemma-3-4b-it-GGUFmedium[MBPP][]gemma-3-4b-itunsloth218google/gemma-3-4b-it46
gemma-3-1b-it-GGUFmedium[MBPP][]gemma-3-1b-itunsloth219google/gemma-3-1b-it46
gemma-3-12b-ptmedium[MBPP][]gemma-3-12b-ptgoogle220google/gemma-3-12b46
gemma-3-12b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-itunsloth221google/gemma-3-12b-it46
gemma-3-4b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-4b-itunsloth222google/gemma-3-4b-it46
gemma-3-12b-itmedium[MBPP][]gemma-3-12b-ptunsloth223google/gemma-3-12b-pt46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-itunsloth224google/gemma-3-27b-it46
gemma-3-1b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-1b-itunsloth225google/gemma-3-1b-it46
gemma-3-4b-it-bnb-4bitmedium[MBPP][]gemma-3-4b-itunsloth226google/gemma-3-4b-it46
gemma-3-4b-ptmedium[MBPP][]gemma-3-4b-ptunsloth227google/gemma-3-4b-pt46
gemma-3-27b-ptmedium[MBPP][]gemma-3-27b-ptgoogle228google/gemma-3-27b-pt46
gemma-3-12b-it-bnb-4bitmedium[MBPP][]gemma-3-12b-itunsloth229google/gemma-3-12b-it46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-itggml-org230google/gemma-3-27b-it46
gemma-3-12b-ptmedium[MBPP][]gemma-3-12b-ptunsloth231google/gemma-3-12b-pt46
gemma-3-27b-it-bnb-4bitmedium[MBPP][]gemma-3-27b-itunsloth232google/gemma-3-27b-it46
gemma-3-27b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-27b-itunsloth233google/gemma-3-27b-it46
gemma-3-1b-ptmedium[MBPP][]gemma-3-1b-ptunsloth234google/gemma-3-1b-pt46
gemma-3-12b-it-GGUFmedium[MBPP][]gemma-3-12b-itggml-org235google/gemma-3-12b-it46
gemma-3-4b-it-qat-GGUFmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth236google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-bnb-4bitmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth237google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-int4-GGUFmedium[MBPP][]gemma-3-12b-it-qat-int4-unquantizedunsloth238google/gemma-3-12b-it-qat-int4-unquantized46
gemma-3-4b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-4b-itgoogle239google/gemma-3-4b-it46
gemma-3-4b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-4b-itgoogle240google/gemma-3-4b-it46
gemma-3-4b-pt-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth241google/gemma-3-4b-pt46
gemma-3-27b-it-qat-GGUFmedium[MBPP][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth242google/gemma-3-27b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-12b-itgoogle243google/gemma-3-12b-it46
gemma-3-1b-it-bnb-4bitmedium[MBPP][]gemma-3-1b-itunsloth244google/gemma-3-1b-it46
gemma-3-1b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-1b-itgoogle245google/gemma-3-1b-it46
gemma-3-4b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth246google/gemma-3-4b-pt46
gemma-3-12b-it-qatmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth247google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-1b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth248google/gemma-3-1b-pt46
gemma-3-4b-it-FP8-Dynamicmedium[MBPP][]gemma-3-4b-itunsloth249google/gemma-3-4b-it46
gemma-3-12b-pt-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth250google/gemma-3-12b-pt46
gemma-3-12b-it-qat-int4-unquantizedmedium[MBPP][]gemma-3-12b-itgoogle251google/gemma-3-12b-it46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-ptMungert252google/gemma-3-27b-pt46
gemma-3-27b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-27b-itgoogle253google/gemma-3-27b-it46
gemma-3-12b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth254google/gemma-3-12b-pt46
gemma-3-27b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-27b-itgoogle255google/gemma-3-27b-it46
gemma-3-27b-it-qat-q4_0-GGUFmedium[MBPP][]gemma-3-27b-itMungert256google/gemma-3-27b-it46
gemma-3-4b-it-qat-bnb-4bitmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth257google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-1b-pt-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth258google/gemma-3-1b-pt46
gemma-3-27b-ptmedium[MBPP][]gemma-3-27b-ptunsloth259google/gemma-3-27b-pt46
gemma-3-12b-it-FP8-Dynamicmedium[MBPP][]gemma-3-12b-ptunsloth260google/gemma-3-12b-pt46
gemma-3-1b-it-qat-int4-unquantizedmedium[MBPP][]gemma-3-1b-itgoogle261google/gemma-3-1b-it46
gemma-3-27b-pt-bnb-4bitmedium[MBPP][]gemma-3-27b-ptunsloth262google/gemma-3-27b-pt46
gemma-3-12b-it-qat-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth263google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-12b-pt-qat-q4_0-ggufmedium[MBPP][]gemma-3-12b-ptgoogle264google/gemma-3-12b-pt46
gemma-3-4b-it-qatmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth265google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-1b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-1b-itgoogle266google/gemma-3-1b-it46
gemma-3-1b-it-qatmedium[MBPP][]gemma-3-1b-it-qat-q4-0-unquantizedunsloth267google/gemma-3-1b-it-qat-q4_0-unquantized46
gemma-3-27b-it-qatmedium[MBPP][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth268google/gemma-3-27b-it-qat-q4_0-unquantized46
gemma-3-4b-pt-qat-q4_0-ggufmedium[MBPP][]gemma-3-4b-ptgoogle269google/gemma-3-4b-pt46
Youtu-LLM-2B-GGUFmediumMBPP+youtu-llm-2b-baseMungert270tencent/Youtu-LLM-2B-Base44.2
Falcon-H1-0.5B-BasemediumMBPP+falcon-h1-0-5b-basetiiuae271tiiuae/Falcon-H1-0.5B-Base43.39
Hunyuan-7B-Instruct-GGUFmediumMBPPhunyuan-7b-pretrainMungert272tencent/Hunyuan-7B-Pretrain43.38
Falcon-H1-0.5B-InstructmediumMBPPfalcon-h1-0-5b-basetiiuae273tiiuae/Falcon-H1-0.5B-Base42.59
Falcon-H1-0.5B-Instruct-GGUFmediumMBPPfalcon-h1-0-5b-basetiiuae274tiiuae/Falcon-H1-0.5B-Base42.59
Falcon-H1-0.5B-Instruct-GGUFmediumMBPPfalcon-h1-0-5b-instructunsloth275tiiuae/Falcon-H1-0.5B-Instruct42.59
DeepSeek-V2-Lite-Chatmedium**MBPP**deepseek-v2-lite-chatdeepseek-ai276deepseek-ai/DeepSeek-V2-Lite-Chat39
DeepSeek-V2-Litemedium**MBPP**deepseek-v2-litedeepseek-ai277deepseek-ai/DeepSeek-V2-Lite39
Nanbeige-16B-Base-GGUFhighMBPPUnknownTheBloke278Nanbeige/Nanbeige-16B-Base36.4
gemma-3-270mmedium[MBPP][]gemma-3-270mgoogle279google/gemma-3-270m35.2
gemma-3-270m-itmedium[MBPP][]gemma-3-270mgoogle280google/gemma-3-270m35.2
gemma-3-270m-it-GGUFmedium[MBPP][]gemma-3-270m-itunsloth281google/gemma-3-270m-it35.2
gemma-3-270m-itmedium[MBPP][]gemma-3-270m-itunsloth282google/gemma-3-270m-it35.2
gemma-3-270m-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth283google/gemma-3-270m-it35.2
gemma-3-270m-it-qat-GGUFmedium[MBPP][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth284google/gemma-3-270m-it-qat-q4_0-unquantized35.2
gemma-3-270mmedium[MBPP][]gemma-3-270munsloth285google/gemma-3-270m35.2
gemma-3-270m-unsloth-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth286google/gemma-3-270m-it35.2
gemma-3-270m-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-270m-itgoogle287google/gemma-3-270m-it35.2
gemma-3-270m-it-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth288google/gemma-3-270m-it35.2
gemma-3-270m-it-qatmedium[MBPP][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth289google/gemma-3-270m-it-qat-q4_0-unquantized35.2
gemma-3-270m-bnb-4bitmedium[MBPP][]gemma-3-270munsloth290google/gemma-3-270m35.2
gemma-3-270m-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-270mgoogle291google/gemma-3-270m35.2
gemma-3-270m-it-GGUFmedium[MBPP][]gemma-3-270mMungert292google/gemma-3-270m35.2
gemma-3-270m-it-torchao-FP8medium[MBPP][]gemma-3-270m-itunsloth293google/gemma-3-270m-it35.2
Falcon-H1-0.5B-InstructmediumMBPP+falcon-h1-0-5b-basetiiuae294tiiuae/Falcon-H1-0.5B-Base33.07
Falcon-H1-0.5B-Instruct-GGUFmediumMBPP+falcon-h1-0-5b-basetiiuae295tiiuae/Falcon-H1-0.5B-Base33.07
Falcon-H1-0.5B-Instruct-GGUFmediumMBPP+falcon-h1-0-5b-instructunsloth296tiiuae/Falcon-H1-0.5B-Instruct33.07
gemma-2-2b-itmedium[MBPP][]gemma-2-2bgoogle297google/gemma-2-2b29.6
gemma-2-2bmedium[MBPP][]gemma-2-2bgoogle298google/gemma-2-2b29.6
gemma-1.1-2b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-1-1-2b-itgoogle299google/gemma-1.1-2b-it29.2
gemma-2bmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-2b-itgoogle300google/gemma-2b-it-pytorch29.2
gemma-2b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-2b-itgoogle301google/gemma-2b-it-pytorch29.2
gemma-7bmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7b-itgoogle302google/gemma-7b-it-quant-pytorch29.2
gemma-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7bgoogle303google/gemma-7b29.2
gemma-1.1-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-1-1-7b-itgoogle304google/gemma-1.1-7b-it29.2
gemma-2b-AWQmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-2b-itgoogle305google/gemma-2b-it-pytorch29.2
gemma-7bmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7b-italpindale306google/gemma-7b-it-quant-pytorch29.2
gemma-2bmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-2b-italpindale307google/gemma-2b-it-pytorch29.2
gemma-7b-AWQmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7b-itgoogle308google/gemma-7b-it-quant-pytorch29.2
gemma-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7b-italpindale309google/gemma-7b-it-quant-pytorch29.2
gemma-7b-GGUFmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7b-itgoogle310google/gemma-7b-it-quant-pytorch29.2
internlm2-chat-1_8bmediumMBPP(Sanitized)internlm2-chat-1-8binternlm311internlm/internlm2-chat-1_8b22.2
internlm2-1_8bmediumMBPP(Sanitized)Unknowninternlm312internlm/internlm2-1_8b22.2
internlm2-chat-1_8b-sftmediumMBPP(Sanitized)internlm2-chat-1-8b-sftinternlm313internlm/internlm2-chat-1_8b-sft22.2
WizardCoder-15B-1.0-GPTQlowMBPP Pass@1UnknownTheBloke314—20.9
Qwen-72BlowMBPPqwen-72bQwen315Qwen/Qwen-72B20.8
Qwen-7BlowMBPPqwen-7bQwen316Qwen/Qwen-7B20.8
Qwen-14BlowMBPPqwen-14bQwen317Qwen/Qwen-14B20.8
t5gemma-2-270m-270mmedium[MBPP]t5gemma-2-270m-270mgoogle318google/t5gemma-2-270m-270m5.6
t5gemma-2-4b-4bmedium[MBPP]t5gemma-2-4b-4bgoogle319google/t5gemma-2-4b-4b5.6
t5gemma-2-1b-1bmedium[MBPP]t5gemma-2-1b-1bgoogle320google/t5gemma-2-1b-1b5.6
Provenance & metadata (1)

source

kind
leaderboard-scrape
paths
benchmarks/mbpp.html, docs/notes/hf-benchmark-contexts.json