Local AI Registry

Leaderboard

MBPP

mbpp
JSON APIBack to leaderboard search

Record

Category
coding
Scored models
323

Scores

ModelScore
nvidia/Nemotron-H-47B-Reasoning-128K90.2
nvidia/Nemotron-H-47B-Reasoning-128K90.2
inclusionAI/LLaDA2.0-flash86.65
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF1685.97
ByteDance-Seed/Seed-Coder-8B-Instruct85.2
ByteDance-Seed/Seed-Coder-8B-Instruct85.2
inclusionAI/LLaDA2.0-flash-preview85.01
nvidia/Nemotron-H-8B-Reasoning-128K84.1
nvidia/Nemotron-H-8B-Reasoning-128K84.1
inclusionAI/LLaDA2.0-mini-preview84.07
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
Qwen/Qwen2.5-VL-32B-Instruct84
tiiuae/Falcon-H1-34B-Base83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Instruct83.86
tiiuae/Falcon-H1-34B-Base83.33
WizardLM/WizardCoder-33B-V1.183
WizardLM/WizardCoder-33B-V1.183
Qwen/Qwen2-72B82.3
tiiuae/Falcon-H1-7B-Base80.69

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category
coding
id
mbpp
name
MBPP
schema version
local-ai-registry/v1

rows

variantconfcontextmodel idorgrankrootscore
Nemotron-H-47B-Reasoning-128KmediumMBPPnemotron-h-47b-reasoning-128knvidia1nvidia/Nemotron-H-47B-Reasoning-128K90.2
Nemotron-H-47B-Reasoning-128K-FP8mediumMBPPnemotron-h-47b-reasoning-128knvidia2nvidia/Nemotron-H-47B-Reasoning-128K90.2
LLaDA2.0-flashmediumMBPPllada2-0-flashinclusionAI3inclusionAI/LLaDA2.0-flash86.65
NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16mediumMBPP-Sanitizednvidia-nemotron-3-ultra-550b-a55b-base-bf16nvidia4nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF1685.97
Seed-Coder-8B-Instruct-GGUFhighUnknownseed-coder-8b-instructunsloth5ByteDance-Seed/Seed-Coder-8B-Instruct85.2
Seed-Coder-8B-Instruct-unsloth-bnb-4bithighUnknownseed-coder-8b-instructunsloth6ByteDance-Seed/Seed-Coder-8B-Instruct85.2
LLaDA2.0-flash-previewmediumUnknownllada2-0-flash-previewinclusionAI7inclusionAI/LLaDA2.0-flash-preview85.01
Nemotron-H-8B-Reasoning-128KmediumMBPPnemotron-h-8b-reasoning-128knvidia8nvidia/Nemotron-H-8B-Reasoning-128K84.1
Nemotron-H-8B-Reasoning-128K-FP8mediumMBPPnemotron-h-8b-reasoning-128knvidia9nvidia/Nemotron-H-8B-Reasoning-128K84.1
LLaDA2.0-mini-previewmediumUnknownllada2-0-mini-previewinclusionAI10inclusionAI/LLaDA2.0-mini-preview84.07
Qwen2.5-VL-32B-InstructhighUnknownqwen2-5-vl-32b-instructQwen11Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-GGUFhighUnknownqwen2-5-vl-32b-instructunsloth12Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-bnb-4bithighUnknownqwen2-5-vl-32b-instructunsloth13Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-GGUFhighUnknownqwen2-5-vl-32b-instructMungert14Qwen/Qwen2.5-VL-32B-Instruct84
Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bithighUnknownqwen2-5-vl-32b-instructunsloth15Qwen/Qwen2.5-VL-32B-Instruct84
Falcon-H1-34B-InstructmediumMBPPfalcon-h1-34b-basetiiuae16tiiuae/Falcon-H1-34B-Base83.86
Falcon-H1-34B-Instruct-GGUFmediumMBPPfalcon-h1-34b-instructtiiuae17tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-Instruct-GGUFmediumMBPPfalcon-h1-34b-instructunsloth18tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-34b-instructtiiuae19tiiuae/Falcon-H1-34B-Instruct83.86
Falcon-H1-34B-BasemediumMBPPfalcon-h1-34b-basetiiuae20tiiuae/Falcon-H1-34B-Base83.33
WizardCoder-33B-V1.1-GGUFlowUnknownUnknownTheBloke21WizardLM/WizardCoder-33B-V1.183
WizardCoder-33B-V1.1-GPTQlowUnknownUnknownTheBloke22WizardLM/WizardCoder-33B-V1.183
Qwen2-72B-InstructmediumMBPPqwen2-72bQwen23Qwen/Qwen2-72B82.3
Falcon-H1-7B-InstructmediumMBPPfalcon-h1-7b-basetiiuae24tiiuae/Falcon-H1-7B-Base80.69
Falcon-H1-7B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-7b-instructtiiuae25tiiuae/Falcon-H1-7B-Instruct80.69
Falcon-H1-7B-Instruct-GGUFmediumMBPPfalcon-h1-7b-instructtiiuae26tiiuae/Falcon-H1-7B-Instruct80.69
Falcon-H1-7B-Instruct-GGUFmediumMBPPfalcon-h1-7b-instructunsloth27tiiuae/Falcon-H1-7B-Instruct80.69
GRIN-MoEmediumMBPPgrin-moemicrosoft28microsoft/GRIN-MoE80.3
Mistral-Large-Instruct-2407mediumMBPP Basemistral-large-instruct-2407mistralai29mistralai/Mistral-Large-Instruct-240780
Falcon-H1-3B-InstructmediumMBPPfalcon-h1-3b-basetiiuae30tiiuae/Falcon-H1-3B-Base79.63
Falcon-H1-3B-Instruct-GGUFmediumMBPPfalcon-h1-3b-basetiiuae31tiiuae/Falcon-H1-3B-Base79.63
Falcon-H1-3B-Instruct-GGUFmediumMBPPfalcon-h1-3b-instructunsloth32tiiuae/Falcon-H1-3B-Instruct79.63
LLaDA2.0-minimediumMBPPllada2-0-miniinclusionAI33inclusionAI/LLaDA2.0-mini78.92
Nemotron-H-4B-Instruct-128KlowUnknownnemotron-h-4b-base-8knvidia34nvidia/Nemotron-H-4B-Base-8K78.6
Falcon-H1-7B-BasemediumMBPPfalcon-h1-7b-basetiiuae35tiiuae/Falcon-H1-7B-Base78.57
NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16mediumMBPP-Sanitizednvidia-nemotron-3-super-120b-a12b-base-bf16nvidia36nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF1678.38
Nemotron-H-56B-Base-8KlowUnknownnemotron-h-56b-base-8knvidia37nvidia/Nemotron-H-56B-Base-8K77.82
MiniMax-Text-01-hfmediumMBPP +minimax-text-01-hfMiniMaxAI38MiniMaxAI/MiniMax-Text-01-hf76.2
MiniMax-Text-01mediumMBPP +minimax-text-01MiniMaxAI39MiniMaxAI/MiniMax-Text-0176.2
Nemotron-H-47B-Base-8KlowUnknownnemotron-h-56b-base-8knvidia40nvidia/Nemotron-H-56B-Base-8K75.9
codegeex4-all-9bhighUnknownUnknownzai-org41zai-org/codegeex4-all-9b75.7
codegeex4-all-9b-GGUFhighUnknownUnknownzai-org42zai-org/codegeex4-all-9b75.7
Phi-3-medium-4k-instructmediumMBPP<br>3-shotphi-3-medium-4k-instructmicrosoft43microsoft/Phi-3-medium-4k-instruct75.2
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai44mistralai/Mistral-Small-3.1-24B-Base-250374.71
Nemotron-Labs-TwoTower-30B-A3B-Base-BF16mediumMBPP-Sanitized (3-shot)nemotron-labs-twotower-30b-a3b-base-bf16nvidia45nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF1674.71
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth46mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth47mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth48mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-instruct-2503unsloth49mistralai/Mistral-Small-3.1-24B-Instruct-250374.71
Mistral-Small-3.1-24B-Instruct-2503-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert50mistralai/Mistral-Small-3.1-24B-Base-250374.71
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503mistralai51mistralai/Mistral-Small-3.1-24B-Base-250374.63
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth52mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth53mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth54mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bitlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth55mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Mistral-Small-3.2-24B-Instruct-2506-GGUFlowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503Mungert56mistralai/Mistral-Small-3.1-24B-Base-250374.63
Mistral-Small-3.2-24B-Instruct-2506-FP8lowMMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )mistral-small-3-1-24b-base-2503unsloth57mistralai/Mistral-Small-3.2-24B-Instruct-250674.63
Qwen2-72BmediumMBPPqwen2-72bQwen58Qwen/Qwen2-72B73.9
Phi-3-medium-128k-instructmediumMBPP<br>3-shotphi-3-medium-128k-instructmicrosoft59microsoft/Phi-3-medium-128k-instruct73.8
Ling-Coder-lite-GPTQ-Int8mediumMBPP-PythonUnknowninclusionAI60inclusionAI/Ling-Coder-lite73.28
Ling-Coder-lite-GPTQ-Int8mediumEvalPlus-MBPP+UnknowninclusionAI61inclusionAI/Ling-Coder-lite73.28
Qwen2.5-Omni-3BmediumMBPPqwen2-5-omni-3bQwen62Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-7BmediumMBPPqwen2-5-omni-7bQwen63Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-7B-GGUFmediumMBPPqwen2-5-omni-7bunsloth64Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-3B-GGUFmediumMBPPqwen2-5-omni-3bunsloth65Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-3B-GGUFmediumMBPPqwen2-5-omni-3bMungert66Qwen/Qwen2.5-Omni-3B73.2
Qwen2.5-Omni-7BmediumMBPPqwen2-5-omni-7bunsloth67Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-7B-GGUFmediumMBPPqwen2-5-omni-7bMungert68Qwen/Qwen2.5-Omni-7B73.2
Qwen2.5-Omni-3BmediumMBPPqwen2-5-omni-3bunsloth69Qwen/Qwen2.5-Omni-3B73.2
NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16mediumMBPP-Sanitized (3-shot)nvidia-nemotron-3-nano-30b-a3b-base-bf16nvidia70nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF1673.15
Hunyuan-A13B-Instruct-GGUFmediumMBPPhunyuan-a13b-instructunsloth71tencent/Hunyuan-A13B-Instruct72.6
Phi-3-small-8k-instructmediumMBPP<br>3-shotphi-3-small-8k-instructmicrosoft72microsoft/Phi-3-small-8k-instruct71.7
Falcon-H1-34B-InstructmediumMBPP+falcon-h1-34b-basetiiuae73tiiuae/Falcon-H1-34B-Base71.43
Falcon-H1-34B-Instruct-GGUFmediumMBPP+falcon-h1-34b-instructtiiuae74tiiuae/Falcon-H1-34B-Instruct71.43
Falcon-H1-3B-BasemediumMBPPfalcon-h1-3b-basetiiuae75tiiuae/Falcon-H1-3B-Base71.43
Falcon-H1-34B-Instruct-GGUFmediumMBPP+falcon-h1-34b-instructunsloth76tiiuae/Falcon-H1-34B-Instruct71.43
Falcon-H1-34B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-34b-instructtiiuae77tiiuae/Falcon-H1-34B-Instruct71.43
Nemotron-H-8B-Reasoning-128KmediumMBPP\_PLUSnemotron-h-8b-reasoning-128knvidia78nvidia/Nemotron-H-8B-Reasoning-128K71.4
Nemotron-H-8B-Reasoning-128K-FP8mediumMBPP\_PLUSnemotron-h-8b-reasoning-128knvidia79nvidia/Nemotron-H-8B-Reasoning-128K71.4
Falcon-H1-1.5B-Deep-BasemediumMBPPfalcon-h1-1-5b-deep-basetiiuae80tiiuae/Falcon-H1-1.5B-Deep-Base70.9
Falcon-H1-34B-BasemediumMBPP+falcon-h1-34b-basetiiuae81tiiuae/Falcon-H1-34B-Base70.37
Phi-3-small-128k-instructmediumMBPP<br>3-shotphi-3-small-128k-instructmicrosoft82microsoft/Phi-3-small-128k-instruct70.3
Olmo-3-1125-32BlowUnknownolmo-3-1125-32ballenai83allenai/Olmo-3-1125-32B69.8
Mistral-Small-24B-Base-2501medium[MBPP][]mistral-small-24b-base-2501mistralai84mistralai/Mistral-Small-24B-Base-250169.64
Mistral-Large-Instruct-2407mediumMBPP Plusmistral-large-instruct-2407mistralai85mistralai/Mistral-Large-Instruct-240769
Falcon-H1-7B-InstructmediumMBPP+falcon-h1-7b-basetiiuae86tiiuae/Falcon-H1-7B-Base68.78
Falcon-H1-7B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-7b-instructtiiuae87tiiuae/Falcon-H1-7B-Instruct68.78
Falcon-H1-7B-Instruct-GGUFmediumMBPP+falcon-h1-7b-instructtiiuae88tiiuae/Falcon-H1-7B-Instruct68.78
Falcon-H1-7B-Instruct-GGUFmediumMBPP+falcon-h1-7b-instructunsloth89tiiuae/Falcon-H1-7B-Instruct68.78
DeepSeek-V2-Chatmedium**MBPP**deepseek-v2-chatdeepseek-ai90deepseek-ai/DeepSeek-V2-Chat68.6
DeepSeek-V2medium**MBPP**deepseek-v2deepseek-ai91deepseek-ai/DeepSeek-V268.6
Olmo-3.1-32B-ThinkmediumMBPP+olmo-3-32b-think-dpoallenai92allenai/Olmo-3-32B-Think-DPO68.3
Olmo-3.1-32B-Think-GGUFmediumMBPP+olmo-3-1-32b-thinkunsloth93allenai/Olmo-3.1-32B-Think68.3
Falcon-H1-1.5B-Deep-InstructmediumMBPPfalcon-h1-1-5b-deep-basetiiuae94tiiuae/Falcon-H1-1.5B-Deep-Base68.25
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-deep-instructtiiuae95tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-deep-instructunsloth96tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-InstructmediumMBPPfalcon-h1-1-5b-deep-instructunsloth97tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-1-5b-deep-instructtiiuae98tiiuae/Falcon-H1-1.5B-Deep-Instruct68.25
Qwen2-7B-InstructmediumMBPPqwen2-7bQwen99Qwen/Qwen2-7B67.9
Mamba-Codestral-7B-v0.1lowUnknownUnknownmistralai100mistralai/Mamba-Codestral-7B-v0.167.7
Falcon-H1-3B-InstructmediumMBPP+falcon-h1-3b-basetiiuae101tiiuae/Falcon-H1-3B-Base67.46
Falcon-H1-3B-Instruct-GGUFmediumMBPP+falcon-h1-3b-basetiiuae102tiiuae/Falcon-H1-3B-Base67.46
Falcon-H1-3B-Instruct-GGUFmediumMBPP+falcon-h1-3b-instructunsloth103tiiuae/Falcon-H1-3B-Instruct67.46
Falcon-H1-7B-BasemediumMBPP+falcon-h1-7b-basetiiuae104tiiuae/Falcon-H1-7B-Base67.2
Olmo-3-32B-ThinkmediumMBPP+olmo-3-32b-think-dpoallenai105allenai/Olmo-3-32B-Think-DPO66.7
Olmo-3-32B-Think-SFTmediumMBPP+olmo-3-1125-32ballenai106allenai/Olmo-3-1125-32B66.7
Olmo-3-32B-Think-GGUFmediumMBPP+olmo-3-32b-think-dpounsloth107allenai/Olmo-3-32B-Think66.7
Olmo-3-32B-Think-DPOmediumMBPP+olmo-3-32b-think-sftallenai108allenai/Olmo-3-32B-Think-SFT66.7
Olmo-3-32B-Think-unsloth-bnb-4bitmediumMBPP+olmo-3-32b-think-dpounsloth109allenai/Olmo-3-32B-Think66.7
Nemotron-H-8B-Base-8KlowUnknownnemotron-h-8b-base-8knvidia110nvidia/Nemotron-H-8B-Base-8K65.37
Falcon-H1-1.5B-BasemediumMBPPfalcon-h1-1-5b-basetiiuae111tiiuae/Falcon-H1-1.5B-Base65.08
Nemotron-H-4B-Base-8KlowUnknownnemotron-h-4b-base-8knvidia112nvidia/Nemotron-H-4B-Base-8K65
Falcon-H1-1.5B-InstructmediumMBPPfalcon-h1-1-5b-basetiiuae113tiiuae/Falcon-H1-1.5B-Base64.81
Falcon-H1-1.5B-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-instructtiiuae114tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GGUFmediumMBPPfalcon-h1-1-5b-instructunsloth115tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumMBPPfalcon-h1-1-5b-instructtiiuae116tiiuae/Falcon-H1-1.5B-Instruct64.81
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumMBPPfalcon-h1-1-5b-instructtiiuae117tiiuae/Falcon-H1-1.5B-Instruct64.81
XwinCoder-34B-GGUFhighUnknownUnknownTheBloke118Xwin-LM/XwinCoder-34B64.8
Qwen2-57B-A14BmediumMBPPqwen2-57b-a14bQwen119Qwen/Qwen2-57B-A14B63.9
AceInstruct-7BmediumMBPPaceinstruct-7bnvidia120nvidia/AceInstruct-7B63.2
AceInstruct-1.5BmediumMBPPaceinstruct-1-5bnvidia121nvidia/AceInstruct-1.5B63.2
AceInstruct-72BmediumMBPPaceinstruct-72bnvidia122nvidia/AceInstruct-72B63.2
AceInstruct-1.5B-GGUFmediumMBPPaceinstruct-1-5bMungert123nvidia/AceInstruct-1.5B63.2
wavecoder-ultra-6.7blowUnknownUnknownmicrosoft124microsoft/wavecoder-ultra-6.7b63
wavecoder-ds-6.7blowUnknownUnknownmicrosoft125microsoft/wavecoder-ds-6.7b63
wavecoder-pro-6.7blowUnknownUnknownmicrosoft126microsoft/wavecoder-pro-6.7b63
NVIDIA-Nemotron-Nano-9B-v2-BasemediumMBPP+ Pass@1nvidia-nemotron-nano-9b-v2-basenvidia127nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base61.55
NVIDIA-Nemotron-Nano-12B-v2-BasemediumMBPP+ Pass@1nvidia-nemotron-nano-12b-v2-basenvidia128nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base61.55
Olmo-3.1-32B-InstructmediumMBPP+olmo-3-1-32b-instruct-dpoallenai129allenai/Olmo-3.1-32B-Instruct-DPO61.5
Olmo-3.1-32B-Instruct-SFTmediumMBPP+olmo-3-1125-32ballenai130allenai/Olmo-3-1125-32B61.5
Olmo-3.1-32B-Instruct-DPOmediumMBPP+olmo-3-1-32b-instruct-sftallenai131allenai/Olmo-3.1-32B-Instruct-SFT61.5
Olmo-3.1-32B-Instruct-GGUFmediumMBPP+olmo-3-1-32b-instructunsloth132allenai/Olmo-3.1-32B-Instruct61.5
Olmo-3.1-32B-InstructmediumMBPP+olmo-3-1-32b-instructunsloth133allenai/Olmo-3.1-32B-Instruct61.5
WizardCoder-Python-34B-V1.0-GGUFhighUnknownUnknownTheBloke134WizardLM/WizardCoder-Python-34B-V1.061.2
WizardLM-70B-V1.0-GPTQlowUnknownUnknownTheBloke135WizardLM/WizardLM-70B-V1.061.2
WizardLM-13B-V1.2-GGUFlowUnknownUnknownTheBloke136WizardLM/WizardLM-13B-V1.261.2
WizardMath-13B-V1.0-GGUFlowUnknownUnknownTheBloke137WizardLM/WizardMath-13B-V1.061.2
WizardLM-70B-V1.0-GGUFlowUnknownUnknownTheBloke138WizardLM/WizardLM-70B-V1.061.2
WizardMath-7B-V1.0-GGUFlowUnknownUnknownTheBloke139WizardLM/WizardMath-7b-V1.061.2
WizardLM-13B-V1.1-GGUFlowUnknownUnknownTheBloke140WizardLM/WizardLM-13B-V1.161.2
wizardLM-7B-GGUFlowUnknownUnknownTheBloke141WizardLM/WizardLM-7B-V1.061.2
WizardMath-70B-V1.0-GGUFlowUnknownUnknownTheBloke142WizardLM/WizardMath-70B-V1.061.2
WizardLM-30B-GGUFlowUnknownUnknownTheBloke143WizardLM/WizardLM-30B-V1.061.2
WizardLM-13B-V1.2-GPTQlowUnknownUnknownTheBloke144WizardLM/WizardLM-13B-V1.261.2
WizardLM-13B-V1.1-GPTQlowUnknownUnknownTheBloke145WizardLM/WizardLM-13B-V1.161.2
WizardCoder-Python-34B-V1.0-GPTQhighUnknownUnknownTheBloke146WizardLM/WizardCoder-Python-34B-V1.061.2
Falcon-H1-1.5B-Deep-BasemediumMBPP+falcon-h1-1-5b-deep-basetiiuae147tiiuae/Falcon-H1-1.5B-Deep-Base60.32
XwinCoder-13B-GGUFhighUnknownUnknownTheBloke148Xwin-LM/XwinCoder-13B60.1
Qwen2-57B-A14B-InstructmediumMBPPqwen2-57b-a14bQwen149Qwen/Qwen2-57B-A14B59.5
Falcon-H1-3B-BasemediumMBPP+falcon-h1-3b-basetiiuae150tiiuae/Falcon-H1-3B-Base57.94
Falcon-H1-1.5B-Deep-InstructmediumMBPP+falcon-h1-1-5b-deep-basetiiuae151tiiuae/Falcon-H1-1.5B-Deep-Base56.61
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-deep-instructtiiuae152tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-deep-instructunsloth153tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-InstructmediumMBPP+falcon-h1-1-5b-deep-instructunsloth154tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
Falcon-H1-1.5B-Deep-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-1-5b-deep-instructtiiuae155tiiuae/Falcon-H1-1.5B-Deep-Instruct56.61
gemma-3n-E2B-itmedium[MBPP][]gemma-3n-e4bgoogle156google/gemma-3n-E4B-it56.6
gemma-3n-E4B-itmedium[MBPP][]gemma-3n-e4bgoogle157google/gemma-3n-E4B56.6
gemma-3n-E4B-it-litert-lmmedium[MBPP][]Unknowngoogle158google/gemma-3n-E4B-it-litert-lm56.6
gemma-3n-E2B-it-GGUFmedium[MBPP][]gemma-3n-e2b-itunsloth159google/gemma-3n-E2B-it56.6
gemma-3n-E4B-itmedium[MBPP][]gemma-3n-e4bunsloth160google/gemma-3n-E4B-it56.6
gemma-3n-E2Bmedium[MBPP][]gemma-3n-e4bgoogle161google/gemma-3n-E4B56.6
gemma-3n-E2B-it-litert-lmmedium[MBPP][]Unknowngoogle162google/gemma-3n-E2B-it-litert-lm56.6
gemma-3n-E4B-it-GGUFmedium[MBPP][]gemma-3n-e4bunsloth163google/gemma-3n-E4B-it56.6
gemma-3n-E4Bmedium[MBPP][]gemma-3n-e4bgoogle164google/gemma-3n-E4B56.6
gemma-3n-E2B-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3n-e2b-itunsloth165google/gemma-3n-E2B-it56.6
gemma-3n-E2B-it-litert-preview-GGUFmedium[MBPP][]Unknownunsloth166google/gemma-3n-E2B-it-litert-preview56.6
gemma-3n-E4B-it-litert-preview-GGUFmedium[MBPP][]Unknownunsloth167google/gemma-3n-E4B-it-litert-preview56.6
gemma-3n-E4B-unsloth-bnb-4bitmedium[MBPP][]gemma-3n-e4bunsloth168google/gemma-3n-E4B-it56.6
gemma-3n-E2B-it-ONNXmedium[MBPP][]gemma-3n-e2b-itonnx-community169google/gemma-3n-E2B-it56.6
gemma-3n-E4B-it-GGUFmedium[MBPP][]gemma-3n-e4bMungert170google/gemma-3n-E4B56.6
gemma-3n-E4B-it-absolute-heresy-MPOA-imatrix-GGUFmedium[MBPP][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious171MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA56.6
gemma-3n-E4B-it-absolute-heresy-GGUFmedium[MBPP][]gemma-3n-e4bMuXodious172google/gemma-3n-E4B-it56.6
gemma-3n-E4B-it-absolute-heresy-MPOA-GGUFmedium[MBPP][]gemma-3n-e4b-it-absolute-heresy-mpoaMuXodious173MuXodious/gemma-3n-E4B-it-absolute-heresy-MPOA56.6
gemma-3n-E2B-it-GGUFmedium[MBPP][]gemma-3n-e4bMungert174google/gemma-3n-E4b-it56.6
Falcon-H1-1.5B-InstructmediumMBPP+falcon-h1-1-5b-basetiiuae175tiiuae/Falcon-H1-1.5B-Base56.35
Falcon-H1-1.5B-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-instructtiiuae176tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GGUFmediumMBPP+falcon-h1-1-5b-instructunsloth177tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GPTQ-Int4mediumMBPP+falcon-h1-1-5b-instructtiiuae178tiiuae/Falcon-H1-1.5B-Instruct56.35
Falcon-H1-1.5B-Instruct-GPTQ-Int8mediumMBPP+falcon-h1-1-5b-instructtiiuae179tiiuae/Falcon-H1-1.5B-Instruct56.35
WizardCoder-Python-13B-V1.0-GGUFhighUnknownUnknownTheBloke180WizardLM/WizardCoder-Python-13B-V1.055.6
WizardCoder-Python-13B-V1.0-GPTQhighUnknownUnknownTheBloke181WizardLM/WizardCoder-Python-13B-V1.055.6
Falcon-H1-1.5B-BasemediumMBPP+falcon-h1-1-5b-basetiiuae182tiiuae/Falcon-H1-1.5B-Base55.03
Qwen2-0.5BmediumMBPPqwen2-0-5bQwen183Qwen/Qwen2-0.5B55
Qwen2-1.5BmediumMBPPqwen2-1-5bQwen184Qwen/Qwen2-1.5B55
gemma-2-9b-itmedium[MBPP][]gemma-2-9bgoogle185google/gemma-2-9b52.4
gemma-2-27b-itmedium[MBPP][]gemma-2-27bgoogle186google/gemma-2-27b52.4
gemma-2-9bmedium[MBPP][]gemma-2-9bgoogle187google/gemma-2-9b52.4
gemma-2-27bmedium[MBPP][]gemma-2-27bgoogle188google/gemma-2-27b52.4
Falcon-H1-0.5B-BasemediumMBPPfalcon-h1-0-5b-basetiiuae189tiiuae/Falcon-H1-0.5B-Base52.12
internlm2-chat-7bmediumMBPP(Sanitized)internlm2-chat-7binternlm190internlm/internlm2-chat-7b51.8
internlm2-7bmediumMBPP(Sanitized)Unknowninternlm191internlm/internlm2-7b51.8
internlm2-chat-20bmediumMBPP(Sanitized)internlm2-chat-20binternlm192internlm/internlm2-chat-20b51.8
internlm2-20bmediumMBPP(Sanitized)Unknowninternlm193internlm/internlm2-20b51.8
internlm2-base-7bmediumMBPP(Sanitized)Unknowninternlm194internlm/internlm2-base-7b51.8
internlm2-base-20bmediumMBPP(Sanitized)Unknowninternlm195internlm/internlm2-base-20b51.8
internlm2-chat-7b-sftmediumMBPP(Sanitized)internlm2-chat-7b-sftinternlm196internlm/internlm2-chat-7b-sft51.8
internlm2-chat-20b-sftmediumMBPP(Sanitized)internlm2-chat-20b-sftinternlm197internlm/internlm2-chat-20b-sft51.8
WizardCoder-Python-7B-V1.0-GGUFhighUnknownUnknownTheBloke198WizardLM/WizardCoder-Python-7b-V1.051.6
Youtu-LLM-2B-GGUFmediumMBPPyoutu-llm-2b-baseMungert199tencent/Youtu-LLM-2B-Base51.5
Qwen2-7BmediumMBPPqwen2-7bQwen200Qwen/Qwen2-7B51.1
Ministral-8B-Instruct-2410lowUnknownministral-8b-instruct-2410mistralai201mistralai/Ministral-8B-Instruct-241050.2
MobileLLM-Pro-basemediumMBPPmobilellm-pro-basefacebook202facebook/MobileLLM-Pro-base46.8
MobileLLM-PromediumMBPPmobilellm-pro-basefacebook203facebook/MobileLLM-Pro-base46.8
gemma-3-1b-itmedium[MBPP][]gemma-3-1b-ptgoogle204google/gemma-3-1b-pt46
gemma-3-4b-itmedium[MBPP][]gemma-3-4b-ptgoogle205google/gemma-3-4b-pt46
gemma-3-12b-itmedium[MBPP][]gemma-3-12b-ptgoogle206google/gemma-3-12b-pt46
gemma-3-27b-itmedium[MBPP][]gemma-3-27b-ptgoogle207google/gemma-3-27b-pt46
gemma-3-4b-itmedium[MBPP][]gemma-3-4b-ptunsloth208google/gemma-3-4b-it46
gemma-3-1b-it-GGUFmedium[MBPP][]gemma-3-1b-ptggml-org209google/gemma-3-1b-it46
gemma-3-12b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-12b-ptgoogle210google/gemma-3-12b-it46
gemma-3-1b-itmedium[MBPP][]gemma-3-1b-ptunsloth211google/gemma-3-1b-it46
gemma-3-12b-it-GGUFmedium[MBPP][]gemma-3-12b-ptunsloth212google/gemma-3-12b-it46
gemma-3-4b-ptmedium[MBPP][]gemma-3-4b-ptgoogle213google/gemma-3-4b-pt46
gemma-3-12b-it-qat-GGUFmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth214google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-27b-itmedium[MBPP][]gemma-3-27b-itunsloth215google/gemma-3-27b-it46
gemma-3-1b-ptmedium[MBPP][]gemma-3-1b-ptgoogle216google/gemma-3-1b-pt46
gemma-3-4b-it-GGUFmedium[MBPP][]gemma-3-4b-ptggml-org217google/gemma-3-4b-it46
gemma-3-4b-it-GGUFmedium[MBPP][]gemma-3-4b-ptunsloth218google/gemma-3-4b-it46
gemma-3-1b-it-GGUFmedium[MBPP][]gemma-3-1b-ptunsloth219google/gemma-3-1b-it46
gemma-3-12b-ptmedium[MBPP][]Unknowngoogle220google/gemma-3-12b46
gemma-3-12b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth221google/gemma-3-12b-it46
gemma-3-4b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth222google/gemma-3-4b-it46
gemma-3-12b-itmedium[MBPP][]gemma-3-12b-ptunsloth223google/gemma-3-12b-pt46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-itunsloth224google/gemma-3-27b-it46
gemma-3-1b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth225google/gemma-3-1b-it46
gemma-3-4b-it-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth226google/gemma-3-4b-it46
gemma-3-4b-ptmedium[MBPP][]gemma-3-4b-ptunsloth227google/gemma-3-4b-pt46
gemma-3-27b-ptmedium[MBPP][]gemma-3-27b-ptgoogle228google/gemma-3-27b-pt46
gemma-3-12b-it-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth229google/gemma-3-12b-it46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-itggml-org230google/gemma-3-27b-it46
gemma-3-12b-ptmedium[MBPP][]gemma-3-12b-ptunsloth231google/gemma-3-12b-pt46
gemma-3-27b-it-bnb-4bitmedium[MBPP][]gemma-3-27b-itunsloth232google/gemma-3-27b-it46
gemma-3-27b-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-27b-itunsloth233google/gemma-3-27b-it46
gemma-3-1b-ptmedium[MBPP][]gemma-3-1b-ptunsloth234google/gemma-3-1b-pt46
gemma-3-12b-it-GGUFmedium[MBPP][]gemma-3-12b-ptggml-org235google/gemma-3-12b-it46
gemma-3-4b-it-qat-GGUFmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth236google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-bnb-4bitmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth237google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-int4-GGUFmedium[MBPP][]gemma-3-12b-it-qat-int4-unquantizedunsloth238google/gemma-3-12b-it-qat-int4-unquantized46
gemma-3-4b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-4b-ptgoogle239google/gemma-3-4b-it46
gemma-3-4b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-4b-ptgoogle240google/gemma-3-4b-it46
gemma-3-4b-pt-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth241google/gemma-3-4b-pt46
gemma-3-27b-it-qat-GGUFmedium[MBPP][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth242google/gemma-3-27b-it-qat-q4_0-unquantized46
gemma-3-12b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-12b-ptgoogle243google/gemma-3-12b-it46
gemma-3-1b-it-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth244google/gemma-3-1b-it46
gemma-3-1b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-1b-ptgoogle245google/gemma-3-1b-it46
gemma-3-4b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-4b-ptunsloth246google/gemma-3-4b-pt46
gemma-3-12b-it-qatmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth247google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-1b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth248google/gemma-3-1b-pt46
gemma-3-4b-it-FP8-Dynamicmedium[MBPP][]gemma-3-4b-ptunsloth249google/gemma-3-4b-it46
gemma-3-12b-pt-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth250google/gemma-3-12b-pt46
gemma-3-12b-it-qat-int4-unquantizedmedium[MBPP][]gemma-3-12b-ptgoogle251google/gemma-3-12b-it46
gemma-3-27b-it-GGUFmedium[MBPP][]gemma-3-27b-ptMungert252google/gemma-3-27b-pt46
gemma-3-27b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-27b-itgoogle253google/gemma-3-27b-it46
gemma-3-12b-pt-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-ptunsloth254google/gemma-3-12b-pt46
gemma-3-27b-it-qat-q4_0-ggufmedium[MBPP][]gemma-3-27b-itgoogle255google/gemma-3-27b-it46
gemma-3-27b-it-qat-q4_0-GGUFmedium[MBPP][]gemma-3-27b-itMungert256google/gemma-3-27b-it46
gemma-3-4b-it-qat-bnb-4bitmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth257google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-1b-pt-bnb-4bitmedium[MBPP][]gemma-3-1b-ptunsloth258google/gemma-3-1b-pt46
gemma-3-27b-ptmedium[MBPP][]gemma-3-27b-ptunsloth259google/gemma-3-27b-pt46
gemma-3-12b-it-FP8-Dynamicmedium[MBPP][]gemma-3-12b-ptunsloth260google/gemma-3-12b-pt46
gemma-3-1b-it-qat-int4-unquantizedmedium[MBPP][]gemma-3-1b-ptgoogle261google/gemma-3-1b-it46
gemma-3-27b-pt-bnb-4bitmedium[MBPP][]gemma-3-27b-ptunsloth262google/gemma-3-27b-pt46
gemma-3-12b-it-qat-unsloth-bnb-4bitmedium[MBPP][]gemma-3-12b-it-qat-q4-0-unquantizedunsloth263google/gemma-3-12b-it-qat-q4_0-unquantized46
gemma-3-12b-pt-qat-q4_0-ggufmedium[MBPP][]gemma-3-12b-ptgoogle264google/gemma-3-12b-pt46
gemma-3-4b-it-qatmedium[MBPP][]gemma-3-4b-it-qat-q4-0-unquantizedunsloth265google/gemma-3-4b-it-qat-q4_0-unquantized46
gemma-3-1b-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-1b-ptgoogle266google/gemma-3-1b-it46
gemma-3-1b-it-qatmedium[MBPP][]gemma-3-1b-it-qat-q4-0-unquantizedunsloth267google/gemma-3-1b-it-qat-q4_0-unquantized46
gemma-3-27b-it-qatmedium[MBPP][]gemma-3-27b-it-qat-q4-0-unquantizedunsloth268google/gemma-3-27b-it-qat-q4_0-unquantized46
gemma-3-4b-pt-qat-q4_0-ggufmedium[MBPP][]gemma-3-4b-ptgoogle269google/gemma-3-4b-pt46
Youtu-LLM-2B-GGUFmediumMBPP+youtu-llm-2b-baseMungert270tencent/Youtu-LLM-2B-Base44.2
Falcon-H1-0.5B-BasemediumMBPP+falcon-h1-0-5b-basetiiuae271tiiuae/Falcon-H1-0.5B-Base43.39
Hunyuan-7B-Instruct-GGUFmediumMBPPhunyuan-7b-pretrainMungert272tencent/Hunyuan-7B-Pretrain43.38
Falcon-H1-0.5B-InstructmediumMBPPfalcon-h1-0-5b-basetiiuae273tiiuae/Falcon-H1-0.5B-Base42.59
Falcon-H1-0.5B-Instruct-GGUFmediumMBPPfalcon-h1-0-5b-basetiiuae274tiiuae/Falcon-H1-0.5B-Base42.59
Falcon-H1-0.5B-Instruct-GGUFmediumMBPPfalcon-h1-0-5b-instructunsloth275tiiuae/Falcon-H1-0.5B-Instruct42.59
DeepSeek-V2-Lite-Chatmedium**MBPP**deepseek-v2-lite-chatdeepseek-ai276deepseek-ai/DeepSeek-V2-Lite-Chat39
DeepSeek-V2-Litemedium**MBPP**deepseek-v2-litedeepseek-ai277deepseek-ai/DeepSeek-V2-Lite39
Olmo-3-1025-7BlowUnknownolmo-3-1025-7ballenai278allenai/Olmo-3-1025-7B36.5
Olmo-Hybrid-7BlowUnknownolmo-hybrid-7ballenai279allenai/Olmo-Hybrid-7B36.5
Olmo-3-1025-7B-GGUFlowUnknownolmo-3-1025-7bMungert280allenai/Olmo-3-1025-7B36.5
Nanbeige-16B-Base-GGUFhighUnknownUnknownTheBloke281Nanbeige/Nanbeige-16B-Base36.4
gemma-3-270mmedium[MBPP][]gemma-3-270mgoogle282google/gemma-3-270m35.2
gemma-3-270m-itmedium[MBPP][]gemma-3-270mgoogle283google/gemma-3-270m35.2
gemma-3-270m-it-GGUFmedium[MBPP][]gemma-3-270m-itunsloth284google/gemma-3-270m-it35.2
gemma-3-270m-itmedium[MBPP][]gemma-3-270m-itunsloth285google/gemma-3-270m-it35.2
gemma-3-270m-it-unsloth-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth286google/gemma-3-270m-it35.2
gemma-3-270m-it-qat-GGUFmedium[MBPP][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth287google/gemma-3-270m-it-qat-q4_0-unquantized35.2
gemma-3-270mmedium[MBPP][]gemma-3-270munsloth288google/gemma-3-270m35.2
gemma-3-270m-unsloth-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth289google/gemma-3-270m-it35.2
gemma-3-270m-it-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-270m-itgoogle290google/gemma-3-270m-it35.2
gemma-3-270m-it-bnb-4bitmedium[MBPP][]gemma-3-270m-itunsloth291google/gemma-3-270m-it35.2
gemma-3-270m-it-qatmedium[MBPP][]gemma-3-270m-it-qat-q4-0-unquantizedunsloth292google/gemma-3-270m-it-qat-q4_0-unquantized35.2
gemma-3-270m-bnb-4bitmedium[MBPP][]gemma-3-270munsloth293google/gemma-3-270m35.2
gemma-3-270m-qat-q4_0-unquantizedmedium[MBPP][]gemma-3-270mgoogle294google/gemma-3-270m35.2
gemma-3-270m-it-GGUFmedium[MBPP][]gemma-3-270mMungert295google/gemma-3-270m35.2
gemma-3-270m-it-torchao-FP8medium[MBPP][]gemma-3-270m-itunsloth296google/gemma-3-270m-it35.2
Falcon-H1-0.5B-InstructmediumMBPP+falcon-h1-0-5b-basetiiuae297tiiuae/Falcon-H1-0.5B-Base33.07
Falcon-H1-0.5B-Instruct-GGUFmediumMBPP+falcon-h1-0-5b-basetiiuae298tiiuae/Falcon-H1-0.5B-Base33.07
Falcon-H1-0.5B-Instruct-GGUFmediumMBPP+falcon-h1-0-5b-instructunsloth299tiiuae/Falcon-H1-0.5B-Instruct33.07
gemma-2-2b-itmedium[MBPP][]gemma-2-2bgoogle300google/gemma-2-2b29.6
gemma-2-2bmedium[MBPP][]gemma-2-2bgoogle301google/gemma-2-2b29.6
gemma-1.1-2b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-1-1-2b-itgoogle302google/gemma-1.1-2b-it29.2
gemma-2bmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle303google/gemma-2b-it-pytorch29.2
gemma-2b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle304google/gemma-2b-it-pytorch29.2
gemma-7bmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle305google/gemma-7b-it-quant-pytorch29.2
gemma-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-7bgoogle306google/gemma-7b29.2
gemma-1.1-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)gemma-1-1-7b-itgoogle307google/gemma-1.1-7b-it29.2
gemma-2b-AWQmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle308google/gemma-2b-it-pytorch29.2
gemma-7bmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknownalpindale309google/gemma-7b-it-quant-pytorch29.2
gemma-2bmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknownalpindale310google/gemma-2b-it-pytorch29.2
gemma-7b-AWQmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle311google/gemma-7b-it-quant-pytorch29.2
gemma-7b-itmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknownalpindale312google/gemma-7b-it-quant-pytorch29.2
gemma-7b-GGUFmedium[MBPP](https://arxiv.org/abs/2108.07732)Unknowngoogle313google/gemma-7b-it-quant-pytorch29.2
internlm2-chat-1_8bmediumMBPP(Sanitized)internlm2-chat-1-8binternlm314internlm/internlm2-chat-1_8b22.2
internlm2-1_8bmediumMBPP(Sanitized)Unknowninternlm315internlm/internlm2-1_8b22.2
internlm2-chat-1_8b-sftmediumMBPP(Sanitized)internlm2-chat-1-8b-sftinternlm316internlm/internlm2-chat-1_8b-sft22.2
WizardCoder-15B-1.0-GPTQlowUnknownUnknownTheBloke317—20.9
Qwen-72BlowUnknownqwen-72bQwen318Qwen/Qwen-72B20.8
Qwen-7BlowUnknownqwen-7bQwen319Qwen/Qwen-7B20.8
Qwen-14BlowUnknownqwen-14bQwen320Qwen/Qwen-14B20.8
t5gemma-2-270m-270mmedium[MBPP]t5gemma-2-270m-270mgoogle321google/t5gemma-2-270m-270m5.6
t5gemma-2-4b-4bmedium[MBPP]t5gemma-2-4b-4bgoogle322google/t5gemma-2-4b-4b5.6
t5gemma-2-1b-1bmedium[MBPP]t5gemma-2-1b-1bgoogle323google/t5gemma-2-1b-1b5.6
Provenance & metadata (1)

source

kind
leaderboard-scrape
paths
benchmarks/mbpp.html