deepseek-v4-flash-0731-mxfp4-rtx-pro-6000-blackwell-96gb-vllm-tp4

facts

metadata

localmaxxing · backend

cuda

localmaxxing · hardware label

RTX PRO 6000 Blackwell

localmaxxing · observed command

docker compose up -d # image riverhouse/glm52-hybrid:v1.4-cfe43b0 (vLLM 0.11.2.dev279+eldritch.enlightenment...b12x7bfc945.cu132, b12x/sparkinfer 0.30.0) vllm serve /model --served-model-name DeepSeek-V4-Flash-0731 --trust-remote-code --tensor-parallel-size 4 --kv-cache-dtype fp8_ds_mla --moe-backend b12x --load-format safetensors -cc.pass_config.fuse_allreduce_rms=True --gpu-memory-utilization 0.90 --max-model-len 262144 --max-num-seqs 8 --max-num-batched-tokens 8192 --max-cudagraph-capture-size 48 --async-scheduling --enable-chunked-prefill --enable-auto-tool-choice --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4 --speculative-config '{"method":"dspark","num_speculative_tokens":5,"moe_backend":"b12x","draft_sample_method":"probabilistic"}' # env: CUTE_DSL_ARCH=sm_120a VLLM_USE_V2_MODEL_RUNNER=1 VLLM_USE_B12X_MOE=1 VLLM_USE_B12X_SPARSE_INDEXER=1 VLLM_USE_B12X_FP8_GEMM=0 B12X_DENSE_SPLITK_TURBO=1 B12X_MLA_SM120_PREFILL_F16_ROPE=1 NCCL_P2P_DISABLE=1 NCCL_IB_DISABLE=1 NCCL_P2P_LEVEL=SYS NCCL_PROTO=LL,LL128,Simple VLLM_ENABLE_PCIE_ALLREDUCE=0 # Notes: Threadripper PRO topology deadlocks on PCIe P2P, so P2P is disabled. b12x MoE is used; b12x MLA (B12X_MLA_SPARSE) is NOT usable for DeepSeek-V4 in b12x 0.30.0 (DSV4 topk-128 MG prefill dispatcher passes scale_format= to io_issue_gather_dsv4_nope(), which rejects it). VLLM_USE_B12X_FP8_GEMM must be 0 with DSpark or deep_gemm fp8_einsum hits a layout assertion during drafter warmup. DSpark requires decode-context-parallel-size 1. DeepseekV4 ds_mla accepts fp8_ds_mla only, not nvfp4_ds_mla.

localmaxxing · run id

cmsap6owb00jupm011x6canpd

localmaxxing · tokenized · arguments

vllm, serve, /model, --served-model-name, DeepSeek-V4-Flash-0731, --trust-remote-code, --tensor-parallel-size, 4, --kv-cache-dtype, fp8_ds_mla, --moe-backend, b12x, --load-format, safetensors, -cc.pass_config.fuse_allreduce_rms=True, --gpu-memory-utilization, 0.90, --max-model-len, 262144, --max-num-seqs, 8, --max-num-batched-tokens, 8192, --max-cudagraph-capture-size, 48, --async-scheduling, --enable-chunked-prefill, --enable-auto-tool-choice, --tokenizer-mode, deepseek_v4, --tool-call-parser, deepseek_v4, --reasoning-parser, deepseek_v4, --speculative-config, {"method":"dspark","num_speculative_tokens":5,"moe_backend":"b12x","draft_sample_method":"probabilistic"}

localmaxxing · tokenized · environment · B12X DENSE SPLITK TURBO

1

localmaxxing · tokenized · environment · B12X MLA SM120 PREFILL F16 ROPE

1

localmaxxing · tokenized · environment · CUTE DSL ARCH

sm_120a

localmaxxing · tokenized · environment · NCCL IB DISABLE

1

localmaxxing · tokenized · environment · NCCL P2P DISABLE

1

localmaxxing · tokenized · environment · NCCL P2P LEVEL

SYS

localmaxxing · tokenized · environment · NCCL PROTO

LL,LL128,Simple

localmaxxing · tokenized · environment · VLLM ENABLE PCIE ALLREDUCE

0

localmaxxing · tokenized · environment · VLLM USE B12X FP8 GEMM

0

localmaxxing · tokenized · environment · VLLM USE B12X MOE

1

localmaxxing · tokenized · environment · VLLM USE B12X SPARSE INDEXER

1

localmaxxing · tokenized · environment · VLLM USE V2 MODEL RUNNER

1

localmaxxing · tokenized · fidelity

faithful

localmaxxing · tokenized · notes

Notes: Threadripper PRO topology deadlocks on PCIe P2P, so P2P is disabled. b12x MoE is used; b12x MLA (B12X_MLA_SPARSE) is NOT usable for DeepSeek-V4 in b12x 0.30.0 (DSV4 topk-128 MG prefill dispatcher passes scale_format= to io_issue_gather_dsv4_nope(), which rejects it). VLLM_USE_B12X_FP8_GEMM must be 0 with DSpark or deep_gemm fp8_einsum hits a layout assertion during drafter warmup. DSpark requires decode-context-parallel-size 1. DeepseekV4 ds_mla accepts fp8_ds_mla only, not nvfp4_ds_mla.

provenance

captured at

2026-08-30T09:10:02Z