Recipe

qwen3-6-35b-a3b-nvfp4-rtx-5090-32gb-vllm-tp1

qwen3-6-35b-a3b-nvfp4-rtx-5090-32gb-vllm-tp1

JSON API Find compatible recipes

vllm recipe for Qwen3.6-35B-A3B on GeForce RTX 5090 with tensor parallelism 1.

Record

Status: candidate
Source: localmaxxing
Engine: vllm
Engine version: 0.20.2rc1.dev189+gfb1ac806c
Accelerators: 1
Tensor parallel: 1
Context tokens: 262,144
Max concurrency: 32
chat: unknown
reasoning: unknown
tools: unknown
vision: unknown

Hugging Face model card

Identity

https://huggingface.co/Qwen/Qwen3.6-35B-A3B

Repository: Qwen/Qwen3.6-35B-A3B
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source: https://www.localmaxxing.com/en/runs/cmp0jl1u600palo015bcqciog

No tokenized launch fields. This is measured or documented compatibility, not a Docker launch.

Measured speed

Concurrency	Context	Prefill	Decode	TTFT ms	Status	Sweep
—	262,144	10,836.4	225.6	47	historical	qwen3-6-35b-a3b-nvfp4-rtx-5090-32gb-vllm-tp1-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-5090-32gb
id: qwen3-6-35b-a3b-nvfp4-rtx-5090-32gb-vllm-tp1
model instance id: qwen-qwen3-6-35b-a3b--nvfp4
recipe source: localmaxxing
schema version: local-ai-registry/v1
speed sweep ids: qwen3-6-35b-a3b-nvfp4-rtx-5090-32gb-vllm-tp1-sweep
status: candidate

capabilities

draft launch

accelerator backend: nvidia
arguments: serve, --model, Qwen/Qwen3.6-35B-A3B, --tensor-parallel-size, 1, --host, 0.0.0.0, --port, 8000, --max-model-len, 262144
container port: 8,000
entrypoint: vllm
host port: 8,000
image: vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
ipc: host
kind: docker
shm size: 16g
synthesized · generated at: 2026-08-31T22:12:17Z
synthesized · image provenance: deepseek-fp8-rtx-pro-6000-blackwell-96gb-vllm-tp1
synthesized · template: vllm-openai-v1

mounts

read only	target
No	/root/.cache/huggingface

engine

name: vllm
version: 0.20.2rc1.dev189+gfb1ac806c

serving

max concurrency: 32
max context tokens: 262,144
tensor parallel: 1

Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at: 2026-08-27T06:04:13.773Z

capabilities.chat · reason capability-not-verified

capabilities.chat · state unknown

capabilities.reasoning · provenance · captured at: 2026-08-27T06:04:13.773Z

capabilities.reasoning · reason capability-not-verified

capabilities.reasoning · state unknown

capabilities.tools · provenance · captured at: 2026-08-27T06:04:13.773Z

capabilities.tools · reason capability-not-verified

capabilities.tools · state unknown

capabilities.vision · provenance · captured at: 2026-08-27T06:04:13.773Z

capabilities.vision · reason capability-not-verified

capabilities.vision · state unknown

description · provenance · captured at: 2026-08-31T22:39:04Z

description · reason derived-from-recipe-identifiers

description · state known

draft launch.entrypoint · provenance · captured at: 2026-09-01T01:41:26Z

draft launch.entrypoint · reason linux-amd64-container-config-entrypoint

draft launch.entrypoint · state known

engine.graph mode · provenance · captured at: 2026-08-27T06:04:13.773Z

engine.graph mode · reason runtime-detail-not-published

engine.graph mode · state unknown

metadata.localmaxxing.engine flags.attentionBackend · provenance · captured at: 2026-08-27T06:04:13.773Z

metadata.localmaxxing.engine flags.attentionBackend · reason not-observed

metadata.localmaxxing.engine flags.attentionBackend · state unknown

metadata.localmaxxing.engine flags.flashAttn · provenance · captured at: 2026-08-27T06:04:13.773Z

metadata.localmaxxing.engine flags.flashAttn · reason not-observed

metadata.localmaxxing.engine flags.flashAttn · state unknown

metadata.localmaxxing.engine flags.gpuLayers · provenance · captured at: 2026-08-27T06:04:13.773Z

metadata.localmaxxing.engine flags.gpuLayers · reason not-observed

metadata.localmaxxing.engine flags.gpuLayers · state unknown

metadata.localmaxxing.engine flags.kvCacheDtype · provenance · captured at: 2026-08-27T06:04:13.773Z

metadata.localmaxxing.engine flags.kvCacheDtype · reason not-observed

metadata.localmaxxing.engine flags.kvCacheDtype · state unknown

serving.kv cache tokens · provenance · captured at: 2026-08-27T06:04:13.773Z

serving.kv cache tokens · reason kv-cache-capacity-not-published

serving.kv cache tokens · state unknown

serving.max concurrency · provenance · captured at: 2026-09-01T01:34:02Z

serving.max concurrency · reason server-capacity-derived-from-source-evidence

serving.max concurrency · state known

serving.max context tokens · provenance · captured at: 2026-09-01T01:30:07Z

serving.max context tokens · reason exact-live-run-context-length

serving.max context tokens · state known

metadata

localmaxxing · base lineage: derived
localmaxxing · engine flags · mtpEnabled: No
localmaxxing · engine flags · specDecoding: No
localmaxxing · engine flags · tensorParallel: 1
localmaxxing · hardware label: 1x RTX 5090 32GB
localmaxxing · notes: RTX 5090, WSL2, vLLM nightly (0.20.2rc1.dev189+gfb1ac806c), NVFP4 quantization, gpu-memory-utilization=0.94, chunked prefill + prefix caching enabled, cudagraph capture size 32, max-num-seqs 32, max-num-batched-tokens 2048. Speculative decoding: MTP (Multi-Token Prediction) with 2 spec tokens using self-speculation (same model as draft + target). Tradeoff: +33.9% throughput (225.6 tokSOut) vs +124% TTFT (47ms vs 21ms baseline). Gen time for 512 tokens: 2.269s vs 3.040s baseline (-25.4%).
localmaxxing · revision unpinned: Yes
localmaxxing · run id: cmp0jl1u600palo015bcqciog

provenance

captured at: 2026-08-27T06:04:13.773Z

sources

captured at	kind	url
2026-08-27T06:04:13.773Z	normalized-recipe	www.localmaxxing.com/en/runs/cmp0jl1u600palo015bcqciog ↗