COLLECTION / COMPARE
35 devices · 34 measuredCompare hardware
Every accelerator side by side: memory, vendor-published throughput, the lowest live US listing, and how many launch recipes the registry holds. Click a column to rank by it; * marks structured-sparsity figures where the vendor publishes no dense number.
Published specifications
Vendor-published compute figures. Missing precision columns remain blank instead of being estimated from a different datatype or sparse mode.
| Hardware | VRAM | Bandwidth | FP16 TFLOPS | FP8 TFLOPS | FP4 TFLOPS | INT8 TOPS | Lowest new (US) | ▾ Recipes |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 3060 | 12 GB | 360 GB/s | — | — | — | — | $500 | 178 |
| GeForce RTX 3090 | 24 GB | 936 GB/s | 142 | — | — | 284 | — | 159 |
| RTX PRO 6000 Blackwell | 96 GB | 1,792 GB/s | 503.8 | 1,007.6 | 2,015.2 | 1,007.6 | $14,400 | 158 |
| GeForce RTX 5090 | 32 GB | 1,792 GB/s | 419 | 838 | 1,676 | 838 | $2,500 | 140 |
| NVIDIA DGX Spark GB10 | 128 GB | 273 GB/s | — | — | 1,000 * | — | $4,700 | 82 |
| GeForce RTX 4070 | 12 GB | 504 GB/s | — | — | — | — | $1,087 | 33 |
| GeForce RTX 5060 Ti | 16 GB | 448 GB/s | — | — | — | — | $380 | 29 |
| GeForce RTX 4090 | 24 GB | 1,008 GB/s | 330.3 | 660.6 | — | 660.6 | $3,898 | 21 |
| GeForce RTX 3080 | 10 GB | 760 GB/s | 119 | — | — | 238 | — | 19 |
| GeForce RTX 5070 | 12 GB | 672 GB/s | 123.5 | 246.9 | 493.9 | 246.9 | $610 | 19 |
| GeForce RTX 3060 Ti | 8 GB | 448 GB/s | — | — | — | — | — | 18 |
| GeForce RTX 3080 | 12 GB | 912 GB/s | 119 | — | — | 238 | — | 18 |
| GeForce RTX 5080 | 16 GB | 960 GB/s | 225.1 | 450.2 | 900.4 | 450.2 | $1,580 | 14 |
| GeForce RTX 5070 Ti | 16 GB | 896 GB/s | 175.8 | 351.5 | 703 | 351.5 | $1,000 | 11 |
| GeForce RTX 3090 Ti | 24 GB | 1,008 GB/s | 160 | — | — | 320 | $2,195 | 10 |
| GeForce RTX 3080 Ti | 12 GB | 912 GB/s | 136.4 | — | — | 272.8 | — | 8 |
| GeForce RTX 4060 Ti | 16 GB | 288 GB/s | — | — | — | — | $499 | 6 |
| GeForce RTX 3070 | 8 GB | 448 GB/s | 81.3 | — | — | 162.6 | — | 5 |
| NVIDIA RTX 2000 Ada Generation | 16 GB | 224 GB/s | — | 191.9 * | — | — | $849 | 5 |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | 672 GB/s | — | — | 1,178 * | — | $3,100 | 5 |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | 896 GB/s | 406 | 811 | 1,617 * | — | $5,100 | 5 |
| GeForce RTX 3070 Ti | 8 GB | 608 GB/s | — | — | — | — | $600 | 3 |
| NVIDIA RTX 4000 Ada Generation | 20 GB | 360 GB/s | — | — | — | — | — | 3 |
| GeForce RTX 4060 | 8 GB | 272 GB/s | — | — | — | — | $499 | 2 |
| GeForce RTX 4070 SUPER | 12 GB | 504 GB/s | — | — | — | — | $740 | 2 |
| GeForce RTX 4070 Ti | 12 GB | 504 GB/s | — | — | — | — | — | 2 |
| GeForce RTX 4080 | 16 GB | 717 GB/s | 194.9 | 389.9 | — | 389.9 | $1,799 | 2 |
| GeForce RTX 4080 SUPER | 16 GB | 736 GB/s | — | — | — | — | $1,522 | 2 |
| RTX A6000 | 48 GB | 768 GB/s | 154.8 | — | — | 309.7 | $6,599 | 2 |
| GeForce RTX 4060 Ti | 8 GB | 288 GB/s | — | — | — | — | $499 | 1 |
| GeForce RTX 4070 Ti SUPER | 16 GB | 672 GB/s | — | — | — | — | $1,266 | 1 |
| GeForce RTX 5060 | 8 GB | 448 GB/s | — | — | — | — | $460 | 1 |
| GeForce RTX 5060 Ti | 8 GB | 448 GB/s | — | — | — | — | $380 | 1 |
| NVIDIA B200 | 180 GB | 7,700 GB/s | 4,500 | — | 9,000 | — | — | 1 |
| RTX 6000 Ada Generation | 48 GB | 960 GB/s | 364.2 | 1,457 * | — | 728.5 | $7,270 | 1 |
Measured local inference
Best observed values across the registry's cited speed sweeps. The Model variants column counts distinct model-instance records. Model, quantization, engine, context, and concurrency differ between runs; use these as an evidence index, not an equal-workload ranking.
| Hardware | Sweeps | Points | Model variants | Peak prefill | Peak decode | Best TTFT | Max observed context | Peak observed VRAM |
|---|---|---|---|---|---|---|---|---|
| RTX PRO 6000 Blackwell | 145 | 235 | 126 | 1,499,853.6 tok/s | 2,527.7 tok/s | 14.5 ms | 1,000,000 tokens | 748.9 GB |
| GeForce RTX 3090 | 159 | 193 | 145 | 328,304.8 tok/s | 666.7 tok/s | 23 ms | 1,048,576 tokens | 184.4 GB |
| GeForce RTX 3060 | 178 | 178 | 163 | 20,862.1 tok/s | 401.4 tok/s | 1 ms | 262,144 tokens | 22.9 GB |
| GeForce RTX 5090 | 139 | 169 | 130 | 1,495,084.3 tok/s | 582.5 tok/s | 0.5 ms | 262,144 tokens | 124 GB |
| NVIDIA DGX Spark GB10 | 75 | 115 | 67 | 29,476 tok/s | 807.3 tok/s | 132.2 ms | 1,048,576 tokens | 334.4 GB |
| GeForce RTX 4070 | 33 | 33 | 33 | 88,024.2 tok/s | 131.9 tok/s | 1 ms | 262,144 tokens | 11.9 GB |
| GeForce RTX 4090 | 21 | 30 | 20 | 583,241.9 tok/s | 213.9 tok/s | 76.3 ms | 172,032 tokens | 23,691 GB |
| GeForce RTX 5060 Ti | 29 | 29 | 29 | 3,469.9 tok/s | 280.9 tok/s | 21.6 ms | 262,144 tokens | 16 GB |
| GeForce RTX 3080 | 19 | 19 | 18 | 10,731.8 tok/s | 328.4 tok/s | 260.5 ms | 262,144 tokens | 9.6 GB |
| GeForce RTX 5070 | 19 | 19 | 19 | 7,615.8 tok/s | 265.5 tok/s | 46.7 ms | 131,072 tokens | 10.2 GB |
| GeForce RTX 3060 Ti | 18 | 18 | 18 | 15,047.6 tok/s | 370.3 tok/s | 54.3 ms | 131,072 tokens | 7.5 GB |
| GeForce RTX 3080 | 17 | 17 | 15 | 5,183.9 tok/s | 149 tok/s | — | 2,048 tokens | 9.8 GB |
| GeForce RTX 5080 | 13 | 13 | 13 | 35,896.7 tok/s | 448.1 tok/s | 14.3 ms | 262,144 tokens | 14.2 GB |
| NVIDIA RTX PRO 4000 Blackwell | 4 | 13 | 3 | 436,348 tok/s | 58.4 tok/s | 283.6 ms | 131,072 tokens | — |
| NVIDIA RTX PRO 4500 Blackwell | 4 | 13 | 4 | 997,269.2 tok/s | 45 tok/s | 86.4 ms | 262,144 tokens | — |
| GeForce RTX 5070 Ti | 11 | 11 | 11 | 28,408.8 tok/s | 402.3 tok/s | 16 ms | 262,144 tokens | — |
| GeForce RTX 3090 Ti | 10 | 10 | 10 | — | 155.9 tok/s | 67.6 ms | 131,072 tokens | 22 GB |
| GeForce RTX 3080 Ti | 8 | 8 | 7 | — | 101.5 tok/s | 352.8 ms | 250,000 tokens | — |
| NVIDIA RTX 2000 Ada Generation | 2 | 7 | 2 | 184,386.1 tok/s | — | 693.4 ms | 130,560 tokens | — |
| GeForce RTX 4060 Ti | 6 | 6 | 6 | 971.8 tok/s | 60.2 tok/s | 217.5 ms | 262,144 tokens | — |
| NVIDIA RTX 4000 Ada Generation | 2 | 5 | 2 | 357,665.3 tok/s | 60.2 tok/s | 257.4 ms | 262,144 tokens | — |
| GeForce RTX 3070 | 4 | 4 | 4 | — | 67.2 tok/s | 150.8 ms | 131,072 tokens | — |
| GeForce RTX 3070 Ti | 3 | 3 | 3 | — | 276.3 tok/s | 305.5 ms | 262,144 tokens | 7,949 GB |
| GeForce RTX 4060 | 2 | 2 | 2 | 494.3 tok/s | 42.9 tok/s | 793.3 ms | 131,072 tokens | 7.3 GB |
| GeForce RTX 4070 SUPER | 2 | 2 | 2 | — | 82 tok/s | 164 ms | 131,072 tokens | 7 GB |
| GeForce RTX 4070 Ti | 2 | 2 | 2 | — | 81.7 tok/s | 486.9 ms | 131,072 tokens | — |
| GeForce RTX 4080 | 2 | 2 | 2 | — | 102.1 tok/s | 376.6 ms | 262,144 tokens | — |
| GeForce RTX 4080 SUPER | 2 | 2 | 2 | — | 106.7 tok/s | 407.3 ms | 262,144 tokens | — |
| RTX A6000 | 2 | 2 | 2 | — | 165.6 tok/s | 127.1 ms | 262,144 tokens | 82.8 GB |
| GeForce RTX 4060 Ti | 1 | 1 | 1 | — | 47.9 tok/s | 172.6 ms | 131,072 tokens | — |
| GeForce RTX 4070 Ti SUPER | 1 | 1 | 1 | — | 106 tok/s | 355.8 ms | 262,144 tokens | — |
| GeForce RTX 5060 | 1 | 1 | 1 | — | 60.3 tok/s | 930.5 ms | 131,072 tokens | — |
| GeForce RTX 5060 Ti | 1 | 1 | 1 | — | 63.7 tok/s | 2,427.8 ms | 131,072 tokens | — |
| RTX 6000 Ada Generation | 1 | 1 | 1 | — | 49.5 tok/s | 524.6 ms | 262,144 tokens | — |