Recipe
qwen3-6-27b-nvfp4-rtx-5090-32gb-vllm-tp1-puvumomq
qwen3-6-27b-nvfp4-rtx-5090-32gb-vllm-tp1-puvumomqObserved LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.
Record
- Status
- candidate
- Source
- localmaxxing
- Engine
- vllm
- Engine version
- 0.25.1
- Accelerators
- 1
- Tensor parallel
- 1
- Context tokens
- 131,072
- Max concurrency
- 4
- chat
- unknown
- reasoning
- unknown
- tools
- unknown
- vision
- unknown
Hugging Face model card
Identity
https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4- Repository
- unsloth/Qwen3.6-27B-NVFP4
- Status
- known
- Link type
- Exact Hub repository
Public Hugging Face repository confirmed by the Hub API.
Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.
Observed configuration
vllm
Evidence only · candidate · reference
Candidate evidence — not a Run contract
Observed source tokens
Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.
vllmserve--modelunsloth/Qwen3.6-27B-NVFP4--served-model-nameqwen3.6-27b--max-model-len131072--gpu-memory-utilization0.94--max-num-seqs4--kv-cache-dtypefp8--enable-prefix-caching--speculative-config{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}--reasoning-parserqwen3--enable-auto-tool-choice--tool-call-parserqwen3_coder
| Flag | Value |
|---|---|
--model | unsloth/Qwen3.6-27B-NVFP4 |
--served-model-name | qwen3.6-27b |
--max-model-len | 131072 |
--gpu-memory-utilization | 0.94 |
--max-num-seqs | 4 |
--kv-cache-dtype | fp8 |
--speculative-config | {"method": "qwen3_5_mtp", "num_speculative_tokens": 3} |
--reasoning-parser | qwen3 |
--tool-call-parser | qwen3_coder |
Measured speed
| Concurrency | Context | Prefill | Decode | TTFT ms | Status | Sweep |
|---|---|---|---|---|---|---|
| — | 131,072 | — | 120.9 | 82 | observed | qwen3-6-27b-nvfp4-rtx-5090-32gb-vllm-tp1-puvumomq-sweep |
Remaining fields
Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.
- hardware count
- 1
- hardware id
- rtx-5090-32gb
- id
- qwen3-6-27b-nvfp4-rtx-5090-32gb-vllm-tp1-puvumomq
- model instance id
- unsloth-qwen3-6-27b-nvfp4--nvfp4
- recipe source
- localmaxxing
- schema version
- local-ai-registry/v1
- speed sweep ids
- qwen3-6-27b-nvfp4-rtx-5090-32gb-vllm-tp1-puvumomq-sweep
- status
- candidate
capabilities
draft launch
- accelerator backend
- nvidia
- arguments
- serve, --model, unsloth/Qwen3.6-27B-NVFP4, --tensor-parallel-size, 1, --host, 0.0.0.0, --port, 8000, --max-model-len, 131072
- container port
- 8,000
- entrypoint
- vllm
- host port
- 8,000
- image
- vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
- ipc
- host
- kind
- docker
- shm size
- 16g
- synthesized · generated at
- 2026-08-31T22:12:17Z
- synthesized · image provenance
- deepseek-fp8-rtx-pro-6000-blackwell-96gb-vllm-tp1
- synthesized · template
- vllm-openai-v1
mounts
| read only | target |
|---|---|
| No | /root/.cache/huggingface |
engine
- name
- vllm
- version
- 0.25.1
serving
- max concurrency
- 4
- max context tokens
- 131,072
- tensor parallel
- 1
Provenance & metadata (3)
facts
- capabilities.chat · provenance · captured at
- 2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
- capabilities.reasoning · provenance · captured at
- 2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
- capabilities.tools · provenance · captured at
- 2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
- capabilities.vision · provenance · captured at
- 2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
- draft launch.entrypoint · provenance · captured at
- 2026-09-01T01:41:26Z
draft launch.entrypoint · reason linux-amd64-container-config-entrypoint
draft launch.entrypoint · state known
- engine.graph mode · provenance · captured at
- 2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
- serving.kv cache tokens · provenance · captured at
- 2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
- serving.max concurrency · provenance · captured at
- 2026-08-31T23:03:15Z
serving.max concurrency · reason server-capacity-derived-from-source-evidence
serving.max concurrency · state known
metadata
- localmaxxing · backend
- cuda
- localmaxxing · batch size
- 1
- localmaxxing · hardware label
- RTX 5090
- localmaxxing · notes
- Method: 8 single-turn requests, streaming, batch size 1, temperature 0, engine confirmed idle first (vllm:num_requests_running = 0). 3093 generated tokens in 25.59 s wall. TTFT is the median of the 8 (range 64-112 ms). Peak VRAM and power are peaks from nvidia-smi sampled every 1.5 s during the run. Prefill tok/s left blank on purpose: prompts averaged only 29 tokens, so TTFT is dominated by request overhead, not prefill throughput. Any prefill number derived from it would be meaningless. Speculative decoding is ON and this matters for comparison: MTP (qwen3_5_mtp draft) with num_speculative_tokens=3, measured 76.2% draft acceptance / 2.29 accepted tokens per round. That is why output tok/s exceeds the ~81 tok/s naive ceiling implied by this card's memory bandwidth for a 27B NVFP4 weight read per token. Also enabled: --kv-cache-dtype fp8 and --enable-prefix-caching (prefix caching contributes nothing here - every prompt is distinct). Runs in an unprivileged Proxmox VE 9.2 LXC with GPU passthrough, gpu-memory-utilization 0.94, max context 131072, max 4 concurrent sequences (this run used 1). The card is currently link-trained at PCIe x8 instead of x16 (unresolved hardware issue); that affects model load time, not decode.
- localmaxxing · observed command
- vllm serve --model unsloth/Qwen3.6-27B-NVFP4 --served-model-name qwen3.6-27b --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 4 --kv-cache-dtype fp8 --enable-prefix-caching --speculative-config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}' --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder
- localmaxxing · run id
- cmsng2oee00dno001puvumomq
- localmaxxing · tokenized · arguments
- vllm, serve, --model, unsloth/Qwen3.6-27B-NVFP4, --served-model-name, qwen3.6-27b, --max-model-len, 131072, --gpu-memory-utilization, 0.94, --max-num-seqs, 4, --kv-cache-dtype, fp8, --enable-prefix-caching, --speculative-config, {"method": "qwen3_5_mtp", "num_speculative_tokens": 3}, --reasoning-parser, qwen3, --enable-auto-tool-choice, --tool-call-parser, qwen3_coder
- localmaxxing · tokenized · fidelity
- faithful
provenance
- captured at
- 2026-08-30T09:10:02Z
sources
| captured at | kind | url |
|---|---|---|
| 2026-08-30T09:10:02Z | normalized-recipe | www.localmaxxing.com/en/runs/cmsng2oee00dno001puvumomq ↗ |