Leaderboard
MMMU
mmmuRecord
- Category
- multimodal
- Scored models
- 324
Scores
| Model | Score |
|---|---|
| deepseek-ai/DeepSeek-V3-0324 | 82 |
| google/gemma-4-26B-A4B | 76.9 |
| google/gemma-4-31B | 76.9 |
| google/gemma-4-E4B | 76.9 |
| google/gemma-4-E2B | 76.9 |
| google/gemma-4-12B | 76.9 |
| google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-12B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-31B-it-assistant | 76.9 |
| google/gemma-4-26B-A4B-it | 76.9 |
| google/gemma-4-26B-A4B | 76.9 |
| google/gemma-4-31B | 76.9 |
| google/gemma-4-12B-it | 76.9 |
| google/gemma-4-12B-it | 76.9 |
| google/gemma-4-31B-it | 76.9 |
| google/gemma-4-E4B-it | 76.9 |
| google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-26B-A4B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-E4B-it | 76.9 |
| google/gemma-4-E2B-it | 76.9 |
| google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-31B-it | 76.9 |
| google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 |
| google/gemma-4-12B | 76.9 |
Remaining fields
Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.
- category
- multimodal
- id
- mmmu
- name
- MMMU
- schema version
- local-ai-registry/v1
rows
| variant | conf | context | model id | org | rank | root | score |
|---|---|---|---|---|---|---|---|
| DeepSeek-V3-0324-NVFP4 | medium | MMMU Pro | deepseek-v3-0324 | nvidia | 1 | deepseek-ai/DeepSeek-V3-0324 | 82 |
| gemma-4-26B-A4B-it | medium | MMMU Pro | gemma-4-26b-a4b | 2 | google/gemma-4-26B-A4B | 76.9 | |
| gemma-4-31B-it | medium | MMMU Pro | gemma-4-31b | 3 | google/gemma-4-31B | 76.9 | |
| gemma-4-E4B-it | medium | MMMU Pro | gemma-4-e4b | 4 | google/gemma-4-E4B | 76.9 | |
| gemma-4-E2B-it | medium | MMMU Pro | gemma-4-e2b | 5 | google/gemma-4-E2B | 76.9 | |
| gemma-4-12B-it | medium | MMMU Pro | gemma-4-12b | 6 | google/gemma-4-12B | 76.9 | |
| gemma-4-31B-it-qat-w4a16-ct | medium | MMMU Pro | gemma-4-31b-it-qat-q4-0-unquantized | 7 | google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-12B-it-qat-w4a16-ct | medium | MMMU Pro | gemma-4-12b-it-qat-q4-0-unquantized | 8 | google/gemma-4-12B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-31B-it-assistant | medium | MMMU Pro | gemma-4-31b-it-assistant | 9 | google/gemma-4-31B-it-assistant | 76.9 | |
| gemma-4-26B-A4B-it-GGUF | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 10 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-26B-A4B | medium | MMMU Pro | gemma-4-26b-a4b | 11 | google/gemma-4-26B-A4B | 76.9 | |
| gemma-4-31B | medium | MMMU Pro | gemma-4-31b | 12 | google/gemma-4-31B | 76.9 | |
| gemma-4-12b-it-GGUF | medium | MMMU Pro | gemma-4-12b-it | unsloth | 13 | google/gemma-4-12B-it | 76.9 |
| gemma-4-12B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-12b-it | 14 | google/gemma-4-12B-it | 76.9 | |
| gemma-4-31B-it-GGUF | medium | MMMU Pro | gemma-4-31b-it | unsloth | 15 | google/gemma-4-31B-it | 76.9 |
| gemma-4-E4B-it-GGUF | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 16 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-E2B-it-qat-w4a16-ct | medium | MMMU Pro | gemma-4-e2b-it-qat-q4-0-unquantized | 17 | google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-26B-A4B-it-qat-GGUF | medium | MMMU Pro | gemma-4-26b-a4b-it-qat-q4-0-unquantized | unsloth | 18 | google/gemma-4-26B-A4B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-E4B-it-unsloth-bnb-4bit | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 19 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-E2B-it-GGUF | medium | MMMU Pro | gemma-4-e2b | unsloth | 20 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-E4B-it-qat-w4a16-ct | medium | MMMU Pro | gemma-4-e4b-it-qat-q4-0-unquantized | 21 | google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-31B-it-unsloth-bnb-4bit | medium | MMMU Pro | gemma-4-31b-it | unsloth | 22 | google/gemma-4-31B-it | 76.9 |
| gemma-4-31B-it-qat-GGUF | medium | MMMU Pro | gemma-4-31b-it-qat-q4-0-unquantized | unsloth | 23 | google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-12B | medium | MMMU Pro | gemma-4-12b | 24 | google/gemma-4-12B | 76.9 | |
| gemma-4-E4B-it-qat-GGUF | medium | MMMU Pro | gemma-4-e4b-it-qat-q4-0-unquantized | unsloth | 25 | google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-26B-A4B-it-assistant | medium | MMMU Pro | gemma-4-26b-a4b-it-assistant | 26 | google/gemma-4-26B-A4B-it-assistant | 76.9 | |
| gemma-4-12B-it-qat-GGUF | medium | MMMU Pro | gemma-4-12b-it-qat-q4-0-unquantized | unsloth | 27 | google/gemma-4-12B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-12B-it-qat-q4_0-gguf | medium | MMMU Pro | gemma-4-12b-it-qat-q4-0-unquantized | 28 | google/gemma-4-12B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-E4B | medium | MMMU Pro | gemma-4-e4b | 29 | google/gemma-4-E4B | 76.9 | |
| gemma-4-E4B-it-qat-q4_0-gguf | medium | MMMU Pro | gemma-4-e4b-it-qat-q4-0-unquantized | 30 | google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-E2B-it-qat-q4_0-gguf | medium | MMMU Pro | gemma-4-e2b-it-qat-q4-0-unquantized | 31 | google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-12B-it-assistant | medium | MMMU Pro | gemma-4-12b-it-assistant | 32 | google/gemma-4-12B-it-assistant | 76.9 | |
| gemma-4-E4B-it | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 33 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-12b-it | medium | MMMU Pro | gemma-4-12b-it | unsloth | 34 | google/gemma-4-12B-it | 76.9 |
| gemma-4-E2B-it-qat-GGUF | medium | MMMU Pro | gemma-4-e2b-it-qat-q4-0-unquantized | unsloth | 35 | google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-12B-it-qat-q4_0-unquantized-assistant | medium | MMMU Pro | gemma-4-12b-it-assistant | 36 | google/gemma-4-12B-it-assistant | 76.9 | |
| gemma-4-26B-A4B-it-qat-q4_0-gguf | medium | MMMU Pro | gemma-4-26b-a4b-it-qat-q4-0-unquantized | 37 | google/gemma-4-26B-A4B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-E2B | medium | MMMU Pro | gemma-4-e2b | 38 | google/gemma-4-E2B | 76.9 | |
| gemma-4-26B-A4B-it-NVFP4 | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 39 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-E2B-it-unsloth-bnb-4bit | medium | MMMU Pro | gemma-4-e2b | unsloth | 40 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-31B-it-qat-q4_0-gguf | medium | MMMU Pro | gemma-4-31b-it-qat-q4-0-unquantized | 41 | google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 | |
| gemma-4-E2B-it | medium | MMMU Pro | gemma-4-e2b | unsloth | 42 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-31B-it-qat-w4a16 | medium | MMMU Pro | gemma-4-31b-it-qat-q4-0-unquantized | unsloth | 43 | google/gemma-4-31B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-E2B-it-assistant | medium | MMMU Pro | gemma-4-e2b-it-assistant | 44 | google/gemma-4-E2B-it-assistant | 76.9 | |
| gemma-4-E4B-it-assistant | medium | MMMU Pro | gemma-4-e4b-it-assistant | 45 | google/gemma-4-E4B-it-assistant | 76.9 | |
| gemma-4-26B-A4B-it | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 46 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-E2B-it-qat-mobile-ct | medium | MMMU Pro | gemma-4-e2b-it-qat-mobile-transformers | 47 | google/gemma-4-E2B-it-qat-mobile-transformers | 76.9 | |
| gemma-4-E2B-it-qat-mobile-GGUF | medium | MMMU Pro | gemma-4-e2b-it-qat-mobile-transformers | unsloth | 48 | google/gemma-4-E2B-it-qat-mobile-transformers | 76.9 |
| gemma-4-26B-A4B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-26b-a4b-it | 49 | google/gemma-4-26B-A4B-it | 76.9 | |
| gemma-4-E2B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-e2b | 50 | google/gemma-4-E2B-it | 76.9 | |
| gemma-4-E4B-it-qat-mobile-ct | medium | MMMU Pro | gemma-4-e4b-it-qat-mobile-transformers | 51 | google/gemma-4-E4B-it-qat-mobile-transformers | 76.9 | |
| gemma-4-12b-it-NVFP4 | medium | MMMU Pro | gemma-4-12b-it | unsloth | 52 | google/gemma-4-12B-it | 76.9 |
| gemma-4-31B-it | medium | MMMU Pro | gemma-4-31b-it | unsloth | 53 | google/gemma-4-31B-it | 76.9 |
| gemma-4-E4B-it-qat-mobile-GGUF | medium | MMMU Pro | gemma-4-e4b-it-qat-mobile-transformers | unsloth | 54 | google/gemma-4-E4B-it-qat-mobile-transformers | 76.9 |
| gemma-4-31B-it-NVFP4 | medium | MMMU Pro | gemma-4-31b-it | unsloth | 55 | google/gemma-4-31B-it | 76.9 |
| gemma-4-12b | medium | MMMU Pro | gemma-4-12b | unsloth | 56 | google/gemma-4-12B | 76.9 |
| gemma-4-E2B-it-qat-mobile-transformers | medium | MMMU Pro | gemma-4-e2b | 57 | google/gemma-4-E2B-it | 76.9 | |
| gemma-4-E4B-it-qat-mobile-transformers | medium | MMMU Pro | gemma-4-e4b-it | 58 | google/gemma-4-E4B-it | 76.9 | |
| gemma-4-31B-it-qat-q4_0-unquantized-assistant | medium | MMMU Pro | gemma-4-31b-it-assistant | 59 | google/gemma-4-31B-it-assistant | 76.9 | |
| gemma-4-E4B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-e4b-it | 60 | google/gemma-4-E4B-it | 76.9 | |
| gemma-4-26B-A4B-it-qat-q4_0-unquantized-assistant | medium | MMMU Pro | gemma-4-26b-a4b-it-assistant | 61 | google/gemma-4-26B-A4B-it-assistant | 76.9 | |
| gemma-4-31B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-31b-it | 62 | google/gemma-4-31B-it | 76.9 | |
| gemma-4-E4B-it-NVFP4 | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 63 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-E2B-it-qat-q4_0-unquantized-assistant | medium | MMMU Pro | gemma-4-e2b-it-assistant | 64 | google/gemma-4-E2B-it-assistant | 76.9 | |
| gemma-4-E4B-it-qat-q4_0-unquantized-assistant | medium | MMMU Pro | gemma-4-e4b-it-assistant | 65 | google/gemma-4-E4B-it-assistant | 76.9 | |
| gemma-4-12B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-12b-it | unsloth | 66 | google/gemma-4-12B-it | 76.9 |
| gemma-4-E2B | medium | MMMU Pro | gemma-4-e2b | unsloth | 67 | google/gemma-4-E2B | 76.9 |
| gemma-4-E2B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-e2b | unsloth | 68 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-E2B-it-NVFP4 | medium | MMMU Pro | gemma-4-e2b | unsloth | 69 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-12B-it-qat-w4a16 | medium | MMMU Pro | gemma-4-12b-it-qat-q4-0-unquantized | unsloth | 70 | google/gemma-4-12B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-26B-A4B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 71 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-12B-it-GGUF | medium | MMMU Pro | gemma-4-12b | Mungert | 72 | google/gemma-4-12B | 76.9 |
| gemma-4-E4B-it-qat-w4a16 | medium | MMMU Pro | gemma-4-e4b-it-qat-q4-0-unquantized | unsloth | 73 | google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-26b-a4b-it-UD-MLX-4bit | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 74 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-26B-A4B | medium | MMMU Pro | gemma-4-26b-a4b | unsloth | 75 | google/gemma-4-26B-A4B | 76.9 |
| gemma-4-31B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-31b-it | unsloth | 76 | google/gemma-4-31B-it | 76.9 |
| gemma-4-E2B-it-qat-w4a16 | medium | MMMU Pro | gemma-4-e2b-it-qat-q4-0-unquantized | unsloth | 77 | google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-E4B | medium | MMMU Pro | gemma-4-e4b | unsloth | 78 | google/gemma-4-E4B | 76.9 |
| gemma-4-E4B-it-qat-q4_0-unquantized | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 79 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-31B | medium | MMMU Pro | gemma-4-31b | unsloth | 80 | google/gemma-4-31B | 76.9 |
| gemma-4-E4B-it-UD-MLX-4bit | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 81 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-E2B-it-UD-MLX-4bit | medium | MMMU Pro | gemma-4-e2b | unsloth | 82 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-31b-it-MLX-8bit | medium | MMMU Pro | gemma-4-31b-it | unsloth | 83 | google/gemma-4-31B-it | 76.9 |
| gemma-4-26b-a4b-it-MLX-8bit | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 84 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-31B-it-GGUF | medium | MMMU Pro | gemma-4-31b | Mungert | 85 | google/gemma-4-31B | 76.9 |
| gemma-4-E2B-it-qat-mobile | medium | MMMU Pro | gemma-4-e2b-it-qat-q4-0-unquantized | unsloth | 86 | google/gemma-4-E2B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-E4B-it-MLX-8bit | medium | MMMU Pro | gemma-4-e4b-it | unsloth | 87 | google/gemma-4-E4B-it | 76.9 |
| gemma-4-31b-it-UD-MLX-4bit | medium | MMMU Pro | gemma-4-31b-it | unsloth | 88 | google/gemma-4-31B-it | 76.9 |
| gemma-4-E4B-it-GGUF | medium | MMMU Pro | gemma-4-e4b | Mungert | 89 | google/gemma-4-E4B | 76.9 |
| gemma-4-E2B-it-GGUF | medium | MMMU Pro | gemma-4-e2b | Mungert | 90 | google/gemma-4-E2B | 76.9 |
| gemma-4-E2B-it-MLX-8bit | medium | MMMU Pro | gemma-4-e2b | unsloth | 91 | google/gemma-4-E2B-it | 76.9 |
| gemma-4-26b-a4b-it-UD-MLX-3bit | medium | MMMU Pro | gemma-4-26b-a4b-it | unsloth | 92 | google/gemma-4-26B-A4B-it | 76.9 |
| gemma-4-E4B-it-qat-mobile | medium | MMMU Pro | gemma-4-e4b-it-qat-q4-0-unquantized | unsloth | 93 | google/gemma-4-E4B-it-qat-q4_0-unquantized | 76.9 |
| gemma-4-31b-it-UD-MLX-3bit | medium | MMMU Pro | gemma-4-31b-it | unsloth | 94 | google/gemma-4-31B-it | 76.9 |
| Qwen3.5-122B-A10B-NVFP4 | high | Unknown | qwen3-5-122b-a10b | nvidia | 95 | Qwen/Qwen3.5-122B-A10B | 75.55 |
| MiniMax-M3-NVFP4 | high | Unknown | minimax-m3 | nvidia | 96 | MiniMaxAI/MiniMax-M3 | 71.01 |
| Qwen2.5-VL-72B-Instruct | medium | MMMU<sub>val</sub> | qwen2-5-vl-72b-instruct | Qwen | 97 | Qwen/Qwen2.5-VL-72B-Instruct | 70.3 |
| Qwen2.5-VL-72B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-72b-instruct | unsloth | 98 | Qwen/Qwen2.5-VL-72B-Instruct | 70.3 |
| QVQ-72B-Preview | medium | MMMU(val) | Unknown | Qwen | 99 | Qwen/Qwen2-VL-72B | 70.3 |
| Qwen2.5-VL-72B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-72b-instruct | Mungert | 100 | Qwen/Qwen2.5-VL-72B-Instruct | 70.3 |
| Qwen2.5-VL-72B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-72b-instruct | unsloth | 101 | Qwen/Qwen2.5-VL-72B-Instruct | 70.3 |
| Qwen2.5-VL-72B-Instruct-unsloth-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-72b-instruct | unsloth | 102 | Qwen/Qwen2.5-VL-72B-Instruct | 70.3 |
| Qwen2.5-VL-32B-Instruct | medium | MMMU | qwen2-5-vl-32b-instruct | Qwen | 103 | Qwen/Qwen2.5-VL-32B-Instruct | 70.2 |
| Qwen2.5-VL-32B-Instruct-GGUF | medium | MMMU | qwen2-5-vl-32b-instruct | unsloth | 104 | Qwen/Qwen2.5-VL-32B-Instruct | 70.2 |
| Qwen2.5-VL-32B-Instruct-bnb-4bit | medium | MMMU | qwen2-5-vl-32b-instruct | unsloth | 105 | Qwen/Qwen2.5-VL-32B-Instruct | 70.2 |
| Qwen2.5-VL-32B-Instruct-GGUF | medium | MMMU | qwen2-5-vl-32b-instruct | Mungert | 106 | Qwen/Qwen2.5-VL-32B-Instruct | 70.2 |
| Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bit | medium | MMMU | qwen2-5-vl-32b-instruct | unsloth | 107 | Qwen/Qwen2.5-VL-32B-Instruct | 70.2 |
| Qwen2.5-VL-32B-Instruct-AWQ | high | Unknown | qwen2-5-vl-32b-instruct | Qwen | 108 | Qwen/Qwen2.5-VL-32B-Instruct | 70 |
| glm-4v-9b | low | Unknown | Unknown | zai-org | 109 | zai-org/glm-4v-9b | 69.2 |
| Kimi-VL-A3B-Thinking | medium | MMMU (val) (Pass@1) | kimi-vl-a3b-instruct | moonshotai | 110 | moonshotai/Kimi-VL-A3B-Instruct | 69.1 |
| Eagle2.5-8B | medium | MMMU<sub>val</sub> | qwen2-5-7b-instruct | nvidia | 111 | Qwen/Qwen2.5-7B-Instruct | 69.1 |
| Kimi-VL-A3B-Thinking-2506 | medium | MMMU (val, Pass@1) | kimi-vl-a3b-instruct | moonshotai | 112 | moonshotai/Kimi-VL-A3B-Instruct | 69.1 |
| NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 | medium | MMMU* | Unknown | nvidia | 113 | nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 | 68 |
| Ming-Lite-Uni | high | Unknown | Unknown | inclusionAI | 114 | inclusionAI/Ming-Lite-Uni | 67.9 |
| Qwen2-VL-72B-Instruct-AWQ | high | Unknown | qwen2-vl-72b-instruct | Qwen | 115 | Qwen/Qwen2-VL-72B-Instruct | 65.44 |
| Qwen2-VL-72B-Instruct-GPTQ-Int4 | high | Unknown | qwen2-vl-72b-instruct | Qwen | 116 | Qwen/Qwen2-VL-72B-Instruct | 65.44 |
| Qwen2-VL-72B-Instruct-GPTQ-Int8 | high | Unknown | qwen2-vl-72b-instruct | Qwen | 117 | Qwen/Qwen2-VL-72B-Instruct | 65.44 |
| Mistral-Small-3.1-24B-Instruct-2503 | low | Unknown | mistral-small-3-1-24b-base-2503 | mistralai | 118 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506 | low | Unknown | mistral-small-3-1-24b-base-2503 | mistralai | 119 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506-GGUF | low | Unknown | mistral-small-3-1-24b-base-2503 | unsloth | 120 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506-bnb-4bit | low | Unknown | mistral-small-3-1-24b-base-2503 | unsloth | 121 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506 | low | Unknown | mistral-small-3-1-24b-base-2503 | unsloth | 122 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 64 |
| Mistral-Small-3.1-24B-Instruct-2503-GGUF | low | Unknown | mistral-small-3-1-24b-instruct-2503 | unsloth | 123 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit | low | Unknown | mistral-small-3-1-24b-base-2503 | unsloth | 124 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 64 |
| Mistral-Small-3.1-24B-Instruct-2503-bnb-4bit | low | Unknown | mistral-small-3-1-24b-instruct-2503 | unsloth | 125 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 64 |
| Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bit | low | Unknown | mistral-small-3-1-24b-instruct-2503 | unsloth | 126 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 64 |
| Mistral-Small-3.1-24B-Instruct-2503 | low | Unknown | mistral-small-3-1-24b-instruct-2503 | unsloth | 127 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 64 |
| Mistral-Small-3.1-24B-Instruct-2503-GGUF | low | Unknown | mistral-small-3-1-24b-base-2503 | Mungert | 128 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506-GGUF | low | Unknown | mistral-small-3-1-24b-base-2503 | Mungert | 129 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 64 |
| Mistral-Small-3.2-24B-Instruct-2506-FP8 | low | Unknown | mistral-small-3-1-24b-base-2503 | unsloth | 130 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 64 |
| Pixtral-Large-Instruct-2411 | low | MathVista (CoT) | MMMU (CoT) | ChartQA (CoT) | Unknown | mistralai | 131 | mistralai/Pixtral-Large-Instruct-2411 | 64 |
| MiniMax-VL-01 | medium | MMMU<sup>*</sup> | minimax-vl-01 | MiniMaxAI | 132 | MiniMaxAI/MiniMax-VL-01 | 63.5 |
| Mistral-Medium-3.5-128B-NVFP4 | high | Unknown | mistral-medium-3-5-128b | nvidia | 133 | mistralai/Mistral-Medium-3.5-128B | 62.79 |
| Mistral-Small-3.1-24B-Instruct-2503 | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-base-2503 | mistralai | 134 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 59.27 |
| Mistral-Small-3.1-24B-Instruct-2503-GGUF | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-instruct-2503 | unsloth | 135 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 59.27 |
| Mistral-Small-3.1-24B-Base-2503 | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-base-2503 | mistralai | 136 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 59.27 |
| Mistral-Small-3.1-24B-Instruct-2503-bnb-4bit | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-instruct-2503 | unsloth | 137 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 59.27 |
| Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bit | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-instruct-2503 | unsloth | 138 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 59.27 |
| Mistral-Small-3.1-24B-Instruct-2503 | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-instruct-2503 | unsloth | 139 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 59.27 |
| Mistral-Small-3.1-24B-Instruct-2503-GGUF | low | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | mistral-small-3-1-24b-base-2503 | Mungert | 140 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 59.27 |
| Qwen2.5-Omni-3B | medium | MMMU<sub>val</sub> | qwen2-5-omni-3b | Qwen | 141 | Qwen/Qwen2.5-Omni-3B | 59.2 |
| Qwen2.5-Omni-7B | medium | MMMU<sub>val</sub> | qwen2-5-omni-7b | Qwen | 142 | Qwen/Qwen2.5-Omni-7B | 59.2 |
| Qwen2.5-Omni-7B-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-omni-7b | unsloth | 143 | Qwen/Qwen2.5-Omni-7B | 59.2 |
| Qwen2.5-Omni-3B-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-omni-3b | unsloth | 144 | Qwen/Qwen2.5-Omni-3B | 59.2 |
| Qwen2.5-Omni-3B-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-omni-3b | Mungert | 145 | Qwen/Qwen2.5-Omni-3B | 59.2 |
| Qwen2.5-Omni-7B | medium | MMMU<sub>val</sub> | qwen2-5-omni-7b | unsloth | 146 | Qwen/Qwen2.5-Omni-7B | 59.2 |
| Qwen2.5-Omni-7B-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-omni-7b | Mungert | 147 | Qwen/Qwen2.5-Omni-7B | 59.2 |
| Qwen2.5-Omni-3B | medium | MMMU<sub>val</sub> | qwen2-5-omni-3b | unsloth | 148 | Qwen/Qwen2.5-Omni-3B | 59.2 |
| Ming-Lite-Omni | medium | MMMU | ling-lite | inclusionAI | 149 | inclusionAI/Ling-lite | 56.3 |
| Qwen2.5-VL-7B-Instruct | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | Qwen | 150 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Qwen2.5-VL-7B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 151 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Qwen2.5-VL-7B-Instruct | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 152 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Qwen2.5-VL-7B-Instruct-unsloth-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 153 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Qwen2.5-VL-7B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 154 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Qwen2.5-VL-7B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-7b-instruct | Mungert | 155 | Qwen/Qwen2.5-VL-7B-Instruct | 56 |
| Phi-4-multimodal-instruct | medium | MMMU | phi-4-multimodal-instruct | microsoft | 156 | microsoft/Phi-4-multimodal-instruct | 55.1 |
| Phi-4-multimodal-instruct | medium | MMMU | phi-4-multimodal-instruct | Lexius | 157 | microsoft/Phi-4-multimodal-instruct | 55.1 |
| MiniMax-VL-01 | medium | MMMU-Pro<sup>*</sup> | minimax-vl-01 | MiniMaxAI | 158 | MiniMaxAI/MiniMax-VL-01 | 54.5 |
| diffusiongemma-26B-A4B-it | medium | MMMU Pro | diffusiongemma-26b-a4b-it | 159 | google/diffusiongemma-26B-A4B-it | 54.3 | |
| diffusiongemma-26B-A4B-it-GGUF | medium | MMMU Pro | diffusiongemma-26b-a4b-it | unsloth | 160 | google/diffusiongemma-26B-A4B-it | 54.3 |
| diffusiongemma-26B-A4B-it | medium | MMMU Pro | diffusiongemma-26b-a4b-it | unsloth | 161 | google/diffusiongemma-26B-A4B-it | 54.3 |
| Qwen2-VL-7B-Instruct-AWQ | high | Unknown | qwen2-vl-7b-instruct | Qwen | 162 | Qwen/Qwen2-VL-7B-Instruct | 53.77 |
| Qwen2-VL-7B-Instruct-GPTQ-Int8 | high | Unknown | qwen2-vl-7b-instruct | Qwen | 163 | Qwen/Qwen2-VL-7B-Instruct | 53.77 |
| Qwen2-VL-7B-Instruct-GPTQ-Int4 | high | Unknown | qwen2-vl-7b-instruct | Qwen | 164 | Qwen/Qwen2-VL-7B-Instruct | 53.77 |
| Qwen2.5-VL-3B-Instruct | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | Qwen | 165 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2.5-VL-3B-Instruct-unsloth-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 166 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2.5-VL-3B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 167 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2.5-VL-3B-Instruct-GGUF | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | Mungert | 168 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2.5-VL-3B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 169 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2.5-VL-3B-Instruct | medium | MMMU<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 170 | Qwen/Qwen2.5-VL-3B-Instruct | 52.3 |
| Qwen2-VL-7B-Instruct | medium | MMMU<sub>val</sub> | qwen2-vl-7b | Qwen | 171 | Qwen/Qwen2-VL-7B | 51.8 |
| Qwen2-VL-7B-Instruct-unsloth-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-vl-7b-instruct | unsloth | 172 | Qwen/Qwen2-VL-7B-Instruct | 51.8 |
| Qwen2-VL-7B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-vl-7b-instruct | unsloth | 173 | Qwen/Qwen2-VL-7B-Instruct | 51.8 |
| Qwen2-VL-7B-Instruct | medium | MMMU<sub>val</sub> | qwen2-vl-7b-instruct | unsloth | 174 | Qwen/Qwen2-VL-7B-Instruct | 51.8 |
| Kimi-VL-A3B-Thinking-2506 | medium | MMMU-Pro (Pass@1) | kimi-vl-a3b-instruct | moonshotai | 175 | moonshotai/Kimi-VL-A3B-Instruct | 51.7 |
| Qwen2.5-VL-32B-Instruct | medium | MMMU Pro | qwen2-5-vl-32b-instruct | Qwen | 176 | Qwen/Qwen2.5-VL-32B-Instruct | 51.1 |
| Qwen2.5-VL-32B-Instruct-GGUF | medium | MMMU Pro | qwen2-5-vl-32b-instruct | unsloth | 177 | Qwen/Qwen2.5-VL-32B-Instruct | 51.1 |
| Qwen2.5-VL-32B-Instruct-bnb-4bit | medium | MMMU Pro | qwen2-5-vl-32b-instruct | unsloth | 178 | Qwen/Qwen2.5-VL-32B-Instruct | 51.1 |
| Qwen2.5-VL-32B-Instruct-GGUF | medium | MMMU Pro | qwen2-5-vl-32b-instruct | Mungert | 179 | Qwen/Qwen2.5-VL-32B-Instruct | 51.1 |
| Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bit | medium | MMMU Pro | qwen2-5-vl-32b-instruct | unsloth | 180 | Qwen/Qwen2.5-VL-32B-Instruct | 51.1 |
| gemma-3-270m | medium | [MMMU][] (val) | gemma-3-270m | 181 | google/gemma-3-270m | 48.8 | |
| gemma-3-270m-it | medium | [MMMU][] (val) | gemma-3-270m | 182 | google/gemma-3-270m | 48.8 | |
| gemma-3-270m-it-GGUF | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 183 | google/gemma-3-270m-it | 48.8 |
| gemma-3-270m-it | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 184 | google/gemma-3-270m-it | 48.8 |
| gemma-3-270m-it-unsloth-bnb-4bit | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 185 | google/gemma-3-270m-it | 48.8 |
| gemma-3-270m-it-qat-GGUF | medium | [MMMU][] (val) | gemma-3-270m-it-qat-q4-0-unquantized | unsloth | 186 | google/gemma-3-270m-it-qat-q4_0-unquantized | 48.8 |
| gemma-3-270m | medium | [MMMU][] (val) | gemma-3-270m | unsloth | 187 | google/gemma-3-270m | 48.8 |
| gemma-3-270m-unsloth-bnb-4bit | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 188 | google/gemma-3-270m-it | 48.8 |
| gemma-3-270m-it-qat-q4_0-unquantized | medium | [MMMU][] (val) | gemma-3-270m-it | 189 | google/gemma-3-270m-it | 48.8 | |
| gemma-3-270m-it-bnb-4bit | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 190 | google/gemma-3-270m-it | 48.8 |
| gemma-3-270m-it-qat | medium | [MMMU][] (val) | gemma-3-270m-it-qat-q4-0-unquantized | unsloth | 191 | google/gemma-3-270m-it-qat-q4_0-unquantized | 48.8 |
| gemma-3-270m-bnb-4bit | medium | [MMMU][] (val) | gemma-3-270m | unsloth | 192 | google/gemma-3-270m | 48.8 |
| gemma-3-270m-qat-q4_0-unquantized | medium | [MMMU][] (val) | gemma-3-270m | 193 | google/gemma-3-270m | 48.8 | |
| gemma-3-270m-it-GGUF | medium | [MMMU][] (val) | gemma-3-270m | Mungert | 194 | google/gemma-3-270m | 48.8 |
| gemma-3-270m-it-torchao-FP8 | medium | [MMMU][] (val) | gemma-3-270m-it | unsloth | 195 | google/gemma-3-270m-it | 48.8 |
| Llama-3.1-Nemotron-Nano-VL-8B-V1 | medium | MMMU Val with chatGPT as a judge | Unknown | nvidia | 196 | nvidia/Llama-3.1-Nemotron-Nano-VL-8B-V1 | 48.2 |
| Pixtral-12B-2409 | medium | **MMMU** *(CoT)* | Unknown | mistralai | 197 | mistralai/Pixtral-12B-Base-2409 | 47.6 |
| Ming-Lite-Omni | medium | Video-MMMU | ling-lite | inclusionAI | 198 | inclusionAI/Ling-lite | 46.3 |
| LFM2.5-VL-1.6B-GGUF | low | Unknown | Unknown | unsloth | 199 | LiquidAI/LFM2.5-VL-1.6B | 40.56 |
| Phi-3-vision-128k-instruct | medium | MMMU | Unknown | microsoft | 200 | microsoft/Phi-3-vision-128k-instruct | 40.4 |
| gemma-3-1b-it | medium | [MMMU][] (pt) | gemma-3-1b-pt | 201 | google/gemma-3-1b-pt | 39.2 | |
| gemma-3-4b-it | medium | [MMMU][] (pt) | gemma-3-4b-pt | 202 | google/gemma-3-4b-pt | 39.2 | |
| gemma-3-270m | medium | [MMMU][] (pt) | gemma-3-270m | 203 | google/gemma-3-270m | 39.2 | |
| gemma-3-12b-it | medium | [MMMU][] (pt) | gemma-3-12b-pt | 204 | google/gemma-3-12b-pt | 39.2 | |
| gemma-3-27b-it | medium | [MMMU][] (pt) | gemma-3-27b-pt | 205 | google/gemma-3-27b-pt | 39.2 | |
| gemma-3-270m-it | medium | [MMMU][] (pt) | gemma-3-270m | 206 | google/gemma-3-270m | 39.2 | |
| gemma-3-4b-it | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 207 | google/gemma-3-4b-it | 39.2 |
| gemma-3-1b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-1b-pt | ggml-org | 208 | google/gemma-3-1b-it | 39.2 |
| gemma-3-12b-it-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-12b-pt | 209 | google/gemma-3-12b-it | 39.2 | |
| gemma-3-1b-it | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 210 | google/gemma-3-1b-it | 39.2 |
| gemma-3-270m-it-GGUF | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 211 | google/gemma-3-270m-it | 39.2 |
| gemma-3-12b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 212 | google/gemma-3-12b-it | 39.2 |
| gemma-3-4b-pt | medium | [MMMU][] (pt) | gemma-3-4b-pt | 213 | google/gemma-3-4b-pt | 39.2 | |
| gemma-3-12b-it-qat-GGUF | medium | [MMMU][] (pt) | gemma-3-12b-it-qat-q4-0-unquantized | unsloth | 214 | google/gemma-3-12b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-27b-it | medium | [MMMU][] (pt) | gemma-3-27b-it | unsloth | 215 | google/gemma-3-27b-it | 39.2 |
| gemma-3-1b-pt | medium | [MMMU][] (pt) | gemma-3-1b-pt | 216 | google/gemma-3-1b-pt | 39.2 | |
| gemma-3-4b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-4b-pt | ggml-org | 217 | google/gemma-3-4b-it | 39.2 |
| gemma-3-4b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 218 | google/gemma-3-4b-it | 39.2 |
| gemma-3-1b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 219 | google/gemma-3-1b-it | 39.2 |
| gemma-3-12b-pt | medium | [MMMU][] (pt) | Unknown | 220 | google/gemma-3-12b | 39.2 | |
| gemma-3-12b-it-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 221 | google/gemma-3-12b-it | 39.2 |
| gemma-3-4b-it-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 222 | google/gemma-3-4b-it | 39.2 |
| gemma-3-12b-it | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 223 | google/gemma-3-12b-pt | 39.2 |
| gemma-3-270m-it | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 224 | google/gemma-3-270m-it | 39.2 |
| gemma-3-27b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-27b-it | unsloth | 225 | google/gemma-3-27b-it | 39.2 |
| gemma-3-1b-it-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 226 | google/gemma-3-1b-it | 39.2 |
| gemma-3-4b-it-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 227 | google/gemma-3-4b-it | 39.2 |
| gemma-3-4b-pt | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 228 | google/gemma-3-4b-pt | 39.2 |
| gemma-3-27b-pt | medium | [MMMU][] (pt) | gemma-3-27b-pt | 229 | google/gemma-3-27b-pt | 39.2 | |
| gemma-3-12b-it-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 230 | google/gemma-3-12b-it | 39.2 |
| gemma-3-270m-it-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 231 | google/gemma-3-270m-it | 39.2 |
| gemma-3-27b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-27b-it | ggml-org | 232 | google/gemma-3-27b-it | 39.2 |
| gemma-3-12b-pt | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 233 | google/gemma-3-12b-pt | 39.2 |
| gemma-3-27b-it-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-27b-it | unsloth | 234 | google/gemma-3-27b-it | 39.2 |
| gemma-3-27b-it-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-27b-it | unsloth | 235 | google/gemma-3-27b-it | 39.2 |
| gemma-3-1b-pt | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 236 | google/gemma-3-1b-pt | 39.2 |
| gemma-3-12b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-12b-pt | ggml-org | 237 | google/gemma-3-12b-it | 39.2 |
| gemma-3-4b-it-qat-GGUF | medium | [MMMU][] (pt) | gemma-3-4b-it-qat-q4-0-unquantized | unsloth | 238 | google/gemma-3-4b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-270m-it-qat-GGUF | medium | [MMMU][] (pt) | gemma-3-270m-it-qat-q4-0-unquantized | unsloth | 239 | google/gemma-3-270m-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-12b-it-qat-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-it-qat-q4-0-unquantized | unsloth | 240 | google/gemma-3-12b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-270m | medium | [MMMU][] (pt) | gemma-3-270m | unsloth | 241 | google/gemma-3-270m | 39.2 |
| gemma-3-12b-it-qat-int4-GGUF | medium | [MMMU][] (pt) | gemma-3-12b-it-qat-int4-unquantized | unsloth | 242 | google/gemma-3-12b-it-qat-int4-unquantized | 39.2 |
| gemma-3-4b-it-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-4b-pt | 243 | google/gemma-3-4b-it | 39.2 | |
| gemma-3-4b-it-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-4b-pt | 244 | google/gemma-3-4b-it | 39.2 | |
| gemma-3-4b-pt-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 245 | google/gemma-3-4b-pt | 39.2 |
| gemma-3-27b-it-qat-GGUF | medium | [MMMU][] (pt) | gemma-3-27b-it-qat-q4-0-unquantized | unsloth | 246 | google/gemma-3-27b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-12b-it-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-12b-pt | 247 | google/gemma-3-12b-it | 39.2 | |
| gemma-3-1b-it-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 248 | google/gemma-3-1b-it | 39.2 |
| gemma-3-1b-it-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-1b-pt | 249 | google/gemma-3-1b-it | 39.2 | |
| gemma-3-4b-pt-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 250 | google/gemma-3-4b-pt | 39.2 |
| gemma-3-270m-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 251 | google/gemma-3-270m-it | 39.2 |
| gemma-3-12b-it-qat | medium | [MMMU][] (pt) | gemma-3-12b-it-qat-q4-0-unquantized | unsloth | 252 | google/gemma-3-12b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-1b-pt-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 253 | google/gemma-3-1b-pt | 39.2 |
| gemma-3-4b-it-FP8-Dynamic | medium | [MMMU][] (pt) | gemma-3-4b-pt | unsloth | 254 | google/gemma-3-4b-it | 39.2 |
| gemma-3-270m-it-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-270m-it | 255 | google/gemma-3-270m-it | 39.2 | |
| gemma-3-12b-pt-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 256 | google/gemma-3-12b-pt | 39.2 |
| gemma-3-12b-it-qat-int4-unquantized | medium | [MMMU][] (pt) | gemma-3-12b-pt | 257 | google/gemma-3-12b-it | 39.2 | |
| gemma-3-27b-it-GGUF | medium | [MMMU][] (pt) | gemma-3-27b-pt | Mungert | 258 | google/gemma-3-27b-pt | 39.2 |
| gemma-3-27b-it-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-27b-it | 259 | google/gemma-3-27b-it | 39.2 | |
| gemma-3-12b-pt-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 260 | google/gemma-3-12b-pt | 39.2 |
| gemma-3-270m-it-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 261 | google/gemma-3-270m-it | 39.2 |
| gemma-3-27b-it-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-27b-it | 262 | google/gemma-3-27b-it | 39.2 | |
| gemma-3-27b-it-qat-q4_0-GGUF | medium | [MMMU][] (pt) | gemma-3-27b-it | Mungert | 263 | google/gemma-3-27b-it | 39.2 |
| gemma-3-270m-it-qat | medium | [MMMU][] (pt) | gemma-3-270m-it-qat-q4-0-unquantized | unsloth | 264 | google/gemma-3-270m-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-4b-it-qat-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-4b-it-qat-q4-0-unquantized | unsloth | 265 | google/gemma-3-4b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-1b-pt-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-1b-pt | unsloth | 266 | google/gemma-3-1b-pt | 39.2 |
| gemma-3-27b-pt | medium | [MMMU][] (pt) | gemma-3-27b-pt | unsloth | 267 | google/gemma-3-27b-pt | 39.2 |
| gemma-3-12b-it-FP8-Dynamic | medium | [MMMU][] (pt) | gemma-3-12b-pt | unsloth | 268 | google/gemma-3-12b-pt | 39.2 |
| gemma-3-270m-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-270m | unsloth | 269 | google/gemma-3-270m | 39.2 |
| gemma-3-1b-it-qat-int4-unquantized | medium | [MMMU][] (pt) | gemma-3-1b-pt | 270 | google/gemma-3-1b-it | 39.2 | |
| gemma-3-27b-pt-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-27b-pt | unsloth | 271 | google/gemma-3-27b-pt | 39.2 |
| gemma-3-12b-it-qat-unsloth-bnb-4bit | medium | [MMMU][] (pt) | gemma-3-12b-it-qat-q4-0-unquantized | unsloth | 272 | google/gemma-3-12b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-270m-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-270m | 273 | google/gemma-3-270m | 39.2 | |
| gemma-3-12b-pt-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-12b-pt | 274 | google/gemma-3-12b-pt | 39.2 | |
| gemma-3-270m-it-GGUF | medium | [MMMU][] (pt) | gemma-3-270m | Mungert | 275 | google/gemma-3-270m | 39.2 |
| gemma-3-4b-it-qat | medium | [MMMU][] (pt) | gemma-3-4b-it-qat-q4-0-unquantized | unsloth | 276 | google/gemma-3-4b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-270m-it-torchao-FP8 | medium | [MMMU][] (pt) | gemma-3-270m-it | unsloth | 277 | google/gemma-3-270m-it | 39.2 |
| gemma-3-1b-it-qat-q4_0-unquantized | medium | [MMMU][] (pt) | gemma-3-1b-pt | 278 | google/gemma-3-1b-it | 39.2 | |
| gemma-3-1b-it-qat | medium | [MMMU][] (pt) | gemma-3-1b-it-qat-q4-0-unquantized | unsloth | 279 | google/gemma-3-1b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-27b-it-qat | medium | [MMMU][] (pt) | gemma-3-27b-it-qat-q4-0-unquantized | unsloth | 280 | google/gemma-3-27b-it-qat-q4_0-unquantized | 39.2 |
| gemma-3-4b-pt-qat-q4_0-gguf | medium | [MMMU][] (pt) | gemma-3-4b-pt | 281 | google/gemma-3-4b-pt | 39.2 | |
| SmolVLM-Instruct-GGUF | high | Unknown | Unknown | Mungert | 282 | HuggingFaceTB/SmolLM2-1.7B-Instruct | 38.8 |
| Phi-4-multimodal-instruct | medium | MMMU-Pro (std/vision) | phi-4-multimodal-instruct | microsoft | 283 | microsoft/Phi-4-multimodal-instruct | 38.5 |
| Phi-4-multimodal-instruct | medium | MMMU-Pro (std/vision) | phi-4-multimodal-instruct | Lexius | 284 | microsoft/Phi-4-multimodal-instruct | 38.5 |
| cogvlm2-llama3-chat-19B | low | Open Source | cogvlm2-llama3-chat-19b | zai-org | 285 | zai-org/cogvlm2-llama3-chat-19B | 37.3 |
| cogvlm2-llama3-chinese-chat-19B | low | Open Source | Unknown | zai-org | 286 | zai-org/cogvlm2-llama3-chinese-chat-19B | 37.3 |
| cogvlm2-llama3-chat-19B-int4 | low | Open Source | cogvlm2-llama3-chat-19b | zai-org | 287 | zai-org/cogvlm2-llama3-chat-19B | 37.3 |
| Qwen2.5-Omni-3B | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-3b | Qwen | 288 | Qwen/Qwen2.5-Omni-3B | 36.6 |
| Qwen2.5-Omni-7B | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-7b | Qwen | 289 | Qwen/Qwen2.5-Omni-7B | 36.6 |
| Qwen2.5-Omni-7B-GGUF | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-7b | unsloth | 290 | Qwen/Qwen2.5-Omni-7B | 36.6 |
| Qwen2.5-Omni-3B-GGUF | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-3b | unsloth | 291 | Qwen/Qwen2.5-Omni-3B | 36.6 |
| Qwen2.5-Omni-3B-GGUF | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-3b | Mungert | 292 | Qwen/Qwen2.5-Omni-3B | 36.6 |
| Qwen2.5-Omni-7B | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-7b | unsloth | 293 | Qwen/Qwen2.5-Omni-7B | 36.6 |
| Qwen2.5-Omni-7B-GGUF | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-7b | Mungert | 294 | Qwen/Qwen2.5-Omni-7B | 36.6 |
| Qwen2.5-Omni-3B | medium | MMMU-Pro<sub>overall</sub> | qwen2-5-omni-3b | unsloth | 295 | Qwen/Qwen2.5-Omni-3B | 36.6 |
| Qwen2-VL-2B-Instruct | medium | MMMU<sub>val</sub> | Unknown | Qwen | 296 | Qwen/Qwen2-VL-2B | 36.3 |
| Qwen2-VL-2B-Instruct-unsloth-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-vl-2b-instruct | unsloth | 297 | Qwen/Qwen2-VL-2B-Instruct | 36.3 |
| Qwen2-VL-2B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-vl-2b-instruct | unsloth | 298 | Qwen/Qwen2-VL-2B-Instruct | 36.3 |
| Qwen2-VL-2B-Instruct | medium | MMMU<sub>val</sub> | qwen2-vl-2b-instruct | unsloth | 299 | Qwen/Qwen2-VL-2B-Instruct | 36.3 |
| Qwen2-VL-72B-Instruct-bnb-4bit | medium | MMMU<sub>val</sub> | qwen2-vl-72b-instruct | unsloth | 300 | Qwen/Qwen2-VL-72B-Instruct | 36.3 |
| Qwen2-VL-72B-Instruct | medium | MMMU<sub>val</sub> | qwen2-vl-72b-instruct | unsloth | 301 | Qwen/Qwen2-VL-72B-Instruct | 36.3 |
| Perception-LM-1B | low | Unknown | Unknown | 302 | facebook/Perception-LM-1B | 34.8 | |
| Perception-LM-8B | low | Unknown | Unknown | 303 | facebook/Perception-LM-8B | 34.8 | |
| Qwen2.5-VL-7B-Instruct | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | Qwen | 304 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-7B-Instruct-GGUF | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 305 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-7B-Instruct | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 306 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-7B-Instruct-unsloth-bnb-4bit | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 307 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-7B-Instruct-bnb-4bit | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | unsloth | 308 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-7B-Instruct-GGUF | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-7b-instruct | Mungert | 309 | Qwen/Qwen2.5-VL-7B-Instruct | 34.3 |
| Qwen2.5-VL-3B-Instruct | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | Qwen | 310 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| Qwen2.5-VL-3B-Instruct-unsloth-bnb-4bit | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 311 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| Qwen2.5-VL-3B-Instruct-GGUF | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 312 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| Qwen2.5-VL-3B-Instruct-GGUF | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | Mungert | 313 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| Qwen2.5-VL-3B-Instruct-bnb-4bit | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 314 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| Qwen2.5-VL-3B-Instruct | medium | MMMU-Pro<sub>val</sub> | qwen2-5-vl-3b-instruct | unsloth | 315 | Qwen/Qwen2.5-VL-3B-Instruct | 32.7 |
| granite-vision-3.2-2b | medium | MMMU | Unknown | unsloth | 316 | ibm-granite/granite-vision-3.2-2b | 32 |
| Eagle2-1B | medium | MMMU<sub>val</sub> | paligemma-3b-mix-448 | nvidia | 317 | google/paligemma-3b-mix-448 | 31.4 |
| Eagle2-1B-GGUF | medium | MMMU<sub>val</sub> | paligemma-3b-mix-448 | Mungert | 318 | google/paligemma-3b-mix-448 | 31.4 |
| SmolVLM-256M-Instruct-GGUF | high | Unknown | Unknown | Mungert | 319 | HuggingFaceTB/SmolLM2-135M-Instruct | 28.3 |
| t5gemma-2-270m-270m | medium | [MMMU] (pt) | t5gemma-2-270m-270m | 320 | google/t5gemma-2-270m-270m | 22.7 | |
| t5gemma-2-4b-4b | medium | [MMMU] (pt) | t5gemma-2-4b-4b | 321 | google/t5gemma-2-4b-4b | 22.7 | |
| t5gemma-2-1b-1b | medium | [MMMU] (pt) | t5gemma-2-1b-1b | 322 | google/t5gemma-2-1b-1b | 22.7 | |
| Nemotron-3.5-Content-Safety | medium | MMMU | Unknown | nvidia | 323 | nvidia/Nemotron-3.5-Content-Safety | 3 |
| Nemotron-3-Content-Safety | medium | MMMU | gemma-3-4b-pt | nvidia | 324 | google/gemma-3-4b-it | 2.3 |
Provenance & metadata (1)
source
- kind
- leaderboard-scrape
- paths
- benchmarks/mmmu.html