qwen3-6-27b-nvfp4-dgx-spark-gb10-128gb-vllm-tp1-v5lfqrqu

facts

metadata

localmaxxing · backend

cuda

localmaxxing · batch size

1

localmaxxing · hardware label

GB10 Grace Blackwell

localmaxxing · notes

Final corrected submission (replaces cmoglz0ux0004lb04zo1dhhhb and cmogm80ri0001lb04hwhu9uzx). Target weights: sakamakismile/Qwen3.6-27B-NVFP4 (community NVFP4 quant of Qwen/Qwen3.6-27B). Drafter: z-lab/Qwen3.6-27B-DFlash (DFlash spec decoding, num_speculative_tokens=15, thinking-ON). Hardware: DGX Spark GB10 (128 GB unified, peak 117 GiB measured). n=30 trials @ pp=128, tg=128, depth=0, concurrency=1, warm-pass (cold-start sample dropped): tokSOut median=32.83 (mean=40.10, std=15.63), ttfr median=268ms (mean=274). Median is the honest headline; DFlash decode std is ~40% of mean due to acceptance fluctuating with prompt content. Mean acceptance length tau=4.11 (per-position [0.75, 0.52, 0.38, ...]). Speedup vs Qwen3.6-27B-FP8 autoregressive baseline on same hardware: 4.18x (32.83 / 7.85). First DGX Spark + spec-decoding entry, first NVFP4 entry leaderboard-wide. REQUIRES the DFlash layer-tap off-by-one vLLM source patch (without it acceptance collapses ~80% -> ~3%). llama-benchy 0.3.6 (PR #11 SSE-counting fix is upstream as of 0.3.6). Full Docker reproduction: https://github.com/my-other-github-account/qwen36-dflash-spark-repro By @banana_baeee.

localmaxxing · observed command

# Reproduction — see https://github.com/my-other-github-account/qwen36-dflash-spark-repro # Download models on host: hf download sakamakismile/Qwen3.6-27B-NVFP4 --local-dir ~/models/Qwen3.6-27B-NVFP4 hf download z-lab/Qwen3.6-27B-DFlash --local-dir ~/models/Qwen3.6-27B-DFlash # Build & run: docker build -t qwen36-dflash-spark . docker run -d --name qwen36-dflash --runtime=nvidia --gpus all --network=host \ -v ~/models:/models:ro \ -e THINK_KWARGS='{"enable_thinking": true}' \ qwen36-dflash-spark # Bench (pp=128 / tg=128 / depth=0 / c=1 / n=30) — produces the headline number: docker run --rm --network=host -v ~/models:/models:ro -v $(pwd):/out \ --entrypoint bash qwen36-dflash-spark \ -c 'OUT=/out/result.json bash /repro/scripts/bench.sh' # Underlying vllm serve invocation (from launch_server.sh): vllm serve /models/Qwen3.6-27B-NVFP4 \ --served-model-name qwen36-27b \ --host 0.0.0.0 --port 8000 \ --tensor-parallel-size 1 \ --gpu-memory-utilization 0.92 \ --max-model-len 262144 \ --max-num-batched-tokens 4096 \ --max-num-seqs 1 \ --trust-remote-code \ --load-format fastsafetensors \ --attention-backend flash_attn \ --enable-prefix-caching \ --speculative-config '{"method":"dflash","num_speculative_tokens":15,"model":"/models/Qwen3.6-27B-DFlash"}' \ --default-chat-template-kwargs '{"enable_thinking": true}' \ --seed 0 # Required vLLM source patch: patches/apply_dflash_off_by_one.sh (DFlash layer-tap +1).

localmaxxing · run id

cmogn5c7w0003l704v5lfqrqu

localmaxxing · tokenized · fidelity

lossy

provenance

captured at

2026-08-30T09:26:09Z