LFM2.5-2.6BBF16AMD Instinct MI300X1 × 192 GBsglangdocker128Kcontext122.2 tok/smeasuredvalidatedsglangdockermeasured
Ornith-1.5-35B-A3BBF16AMD Instinct MI300X1 × 192 GBsglangdocker128Kcontext96.8 tok/smeasuredvalidatedsglangdockermeasured
Qwen3.8-27BFP8AMD Instinct MI300X1 × 192 GBsglangdocker128Kcontext25.3 tok/smeasuredvalidatedsglangdockermeasured
Gemma-4-12B-it4 bpwGeForce RTX 30901 × 24 GBtabbyapidocker128Kcontext26.6 tok/smeasuredvalidatedtabbyapidockermeasured
LFM2.5-2.6BBF16GeForce RTX 30901 × 24 GBsglangdocker128Kcontext89.9 tok/smeasuredvalidatedsglangdockermeasured
Ornith-1.5-35B-A3B4 bpwGeForce RTX 30901 × 24 GBtabbyapidocker128Kcontext29 tok/smeasuredvalidatedtabbyapidockermeasured
Qwen3.6-35B-A3BINT4 W4A16GeForce RTX 30901 × 24 GBvllmdocker128Kcontext216.2 tok/smeasuredvalidatedvllmdockermeasured
Qwen3.6-35B-A3BINT4 W4A16GeForce RTX 30902 × 24 GBvllmdocker128Kcontext460.4 tok/smeasuredvalidatedvllmdockermeasured