Local AI Registry

Recipe

glm-5-3-flash-exl3-3-0bpw-rtx-pro-6000-blackwell-96gb-vllm-tp2

glm-5-3-flash-exl3-3-0bpw-rtx-pro-6000-blackwell-96gb-vllm-tp2
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.1.dev20111+g7f1e92bec.d20260827
Accelerators
2
Tensor parallel
2
Context tokens
8,192
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/zai-org/GLM-5.3-Flash
Repository
zai-org/GLM-5.3-Flash
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmteqhpnj003an001xz0us0uj

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. python
  2. -m
  3. vllm.entrypoints.openai.api_server
  4. --model
  5. /home/rarri/ai/spikes/001-selective-exl3-loader/model-view
  6. --served-model-name
  7. GLM-5.3-Flash-EXL3-3.0bpw
  8. --host
  9. 0.0.0.0
  10. --port
  11. 8012
  12. --language-model-only
  13. --tensor-parallel-size
  14. 2
  15. --dtype
  16. bfloat16
  17. --load-format
  18. safetensors
  19. --moe-backend
  20. b12x
  21. --attention-backend
  22. FLASHINFER_MLA_SPARSE_SM120
  23. --kv-cache-dtype
  24. fp8_ds_mla
  25. --max-model-len
  26. 8192
  27. --max-num-batched-tokens
  28. 1024
  29. --max-num-seqs
  30. 1
  31. --gpu-memory-utilization
  32. 0.90
  33. --enable-chunked-prefill
  34. --reasoning-parser
  35. glm45
  36. --enable-auto-tool-choice
  37. --tool-call-parser
  38. glm47
  39. --disable-custom-all-reduce
  40. --speculative-config
  41. {"method":"dflash","model":"/draft","num_speculative_tokens":7,"draft_tensor_parallel_size":2,"draft_sample_method":"probabilistic","kv_cache_dtype":"auto"}
  42. --no-enable-prefix-caching
FlagValue
-mvllm.entrypoints.openai.api_server
--model/home/rarri/ai/spikes/001-selective-exl3-loader/model-view
--served-model-nameGLM-5.3-Flash-EXL3-3.0bpw
--host0.0.0.0
--port8012
--tensor-parallel-size2
--dtypebfloat16
--load-formatsafetensors
--moe-backendb12x
--attention-backendFLASHINFER_MLA_SPARSE_SM120
--kv-cache-dtypefp8_ds_mla
--max-model-len8192
--max-num-batched-tokens1024
--max-num-seqs1
--gpu-memory-utilization0.90
--reasoning-parserglm45
--tool-call-parserglm47
--speculative-config{"method":"dflash","model":"/draft","num_speculative_tokens":7,"draft_tensor_parallel_size":2,"draft_sample_method":"probabilistic","kv_cache_dtype":"auto"}

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
18,1923,712.3193.8177.1observedglm-5-3-flash-exl3-3-0bpw-rtx-pro-6000-blackwell-96gb-vllm-tp2-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
2
hardware id
rtx-pro-6000-blackwell-96gb
id
glm-5-3-flash-exl3-3-0bpw-rtx-pro-6000-blackwell-96gb-vllm-tp2
model instance id
zai-org-glm-5-3-flash--exl3-3-0bpw
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
glm-5-3-flash-exl3-3-0bpw-rtx-pro-6000-blackwell-96gb-vllm-tp2-sweep
status
candidate

capabilities

draft launch

accelerator backend
nvidia
arguments
--model, zai-org/GLM-5.3-Flash, --tensor-parallel-size, 2, --host, 0.0.0.0, --port, 8000, --max-model-len, 8192
container port
8,000
host port
8,000
image
vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
ipc
host
kind
docker
shm size
16g
synthesized · generated at
2026-08-31T22:12:17Z
synthesized · image provenance
deepseek-fp8-rtx-pro-6000-blackwell-96gb-vllm-tp1
synthesized · template
vllm-openai-v1

mounts

read onlytarget
No/root/.cache/huggingface

engine

name
vllm
version
0.1.dev20111+g7f1e92bec.d20260827

serving

max concurrency
1
max context tokens
8,192
tensor parallel
2
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown

metadata

localmaxxing · backend
cuda
localmaxxing · hardware label
RTX PRO 6000 Blackwell
localmaxxing · notes
Automated 3-run llama-benchy 0.4.0 measurement of the custom selective EXL3 3.0bpw artifact. Fixed 512-token prompt and 1024-token output, batch 1, cold prefix cache, exact output length, temperature=0. Output 193.80 ± 12.60 tok/s; prefill 3,712.33 ± 207.24 tok/s; TTFT 177.14 ± 8.03 ms; peak one-second output 281 ± 47 tok/s. Logical checkpoint TP4 is folded onto two physical TP workers (two serialized 512-wide K3 slices per GPU). DFlash2 TP2 speculative decoding uses seven draft tokens and CUDA graphs. GPU power values are means during samples at >=50% utilization. Peak VRAM is the sum of per-card peaks. GPU 0 also drives the Hyprland desktop. Experimental custom loader; source checkpoint revision 8b099bf276507a17faea920deff3f62d5597fb52 preserved unchanged.
localmaxxing · observed command
python -m vllm.entrypoints.openai.api_server --model /home/rarri/ai/spikes/001-selective-exl3-loader/model-view --served-model-name GLM-5.3-Flash-EXL3-3.0bpw --host 0.0.0.0 --port 8012 --language-model-only --tensor-parallel-size 2 --dtype bfloat16 --load-format safetensors --moe-backend b12x --attention-backend FLASHINFER_MLA_SPARSE_SM120 --kv-cache-dtype fp8_ds_mla --max-model-len 8192 --max-num-batched-tokens 1024 --max-num-seqs 1 --gpu-memory-utilization 0.90 --enable-chunked-prefill --reasoning-parser glm45 --enable-auto-tool-choice --tool-call-parser glm47 --disable-custom-all-reduce --speculative-config '{"method":"dflash","model":"/draft","num_speculative_tokens":7,"draft_tensor_parallel_size":2,"draft_sample_method":"probabilistic","kv_cache_dtype":"auto"}' --no-enable-prefix-caching
localmaxxing · run id
cmteqhpnj003an001xz0us0uj
localmaxxing · tokenized · arguments
python, -m, vllm.entrypoints.openai.api_server, --model, /home/rarri/ai/spikes/001-selective-exl3-loader/model-view, --served-model-name, GLM-5.3-Flash-EXL3-3.0bpw, --host, 0.0.0.0, --port, 8012, --language-model-only, --tensor-parallel-size, 2, --dtype, bfloat16, --load-format, safetensors, --moe-backend, b12x, --attention-backend, FLASHINFER_MLA_SPARSE_SM120, --kv-cache-dtype, fp8_ds_mla, --max-model-len, 8192, --max-num-batched-tokens, 1024, --max-num-seqs, 1, --gpu-memory-utilization, 0.90, --enable-chunked-prefill, --reasoning-parser, glm45, --enable-auto-tool-choice, --tool-call-parser, glm47, --disable-custom-all-reduce, --speculative-config, {"method":"dflash","model":"/draft","num_speculative_tokens":7,"draft_tensor_parallel_size":2,"draft_sample_method":"probabilistic","kv_cache_dtype":"auto"}, --no-enable-prefix-caching
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmteqhpnj003an001xz0us0uj ↗