Local AI Registry

Recipe

qwen3-6-35b-a3b-quark-w8a8-int8-intel-arc-pro-b70-32gb-vllm-tp4

qwen3-6-35b-a3b-quark-w8a8-int8-intel-arc-pro-b70-32gb-vllm-tp4
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.20.2rc1.dev2+gc51df4300.d20260523
Graph
piecewise
Accelerators
4
Tensor parallel
4
Context tokens
32,768
Max concurrency
48
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/Qwen/Qwen3.6-35B-A3B
Repository
Qwen/Qwen3.6-35B-A3B
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmq9ifq0500b0r8012f27j1xl

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. vllm
  2. serve
  3. /mnt/fast-ai/llm-cache/hf/models--nameistoken--Qwen3.6-35B-A3B-Quark-W8A8-INT8/snapshots/cced56592e8c8935f8220836b4baa04dfd389118
  4. --host
  5. 127.0.0.1
  6. --port
  7. 18080
  8. --trust-remote-code
  9. --served-model-name
  10. qwen36-35b-a3b-fp8
  11. --dtype
  12. auto
  13. --quantization
  14. quark
  15. --tensor-parallel-size
  16. 4
  17. --pipeline-parallel-size
  18. 1
  19. --distributed-executor-backend
  20. mp
  21. --max-model-len
  22. 32768
  23. --max-num-batched-tokens
  24. 8192
  25. --max-num-seqs
  26. 48
  27. --gpu-memory-utilization
  28. 0.95
  29. --kv-cache-dtype
  30. auto
  31. --no-enable-prefix-caching
  32. --language-model-only
  33. --compilation-config
  34. {"cudagraph_mode":"PIECEWISE"}
  35. --generation-config
  36. vllm
FlagValue
--host127.0.0.1
--port18080
--served-model-nameqwen36-35b-a3b-fp8
--dtypeauto
--quantizationquark
--tensor-parallel-size4
--pipeline-parallel-size1
--distributed-executor-backendmp
--max-model-len32768
--max-num-batched-tokens8192
--max-num-seqs48
--gpu-memory-utilization0.95
--kv-cache-dtypeauto
--compilation-config{"cudagraph_mode":"PIECEWISE"}
--generation-configvllm

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
—32,768—99.876.5observedqwen3-6-35b-a3b-quark-w8a8-int8-intel-arc-pro-b70-32gb-vllm-tp4-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
4
hardware id
intel-arc-pro-b70-32gb
id
qwen3-6-35b-a3b-quark-w8a8-int8-intel-arc-pro-b70-32gb-vllm-tp4
model instance id
qwen-qwen3-6-35b-a3b--quark-w8a8-int8
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-6-35b-a3b-quark-w8a8-int8-intel-arc-pro-b70-32gb-vllm-tp4-sweep
status
candidate

capabilities

engine

graph mode
piecewise
name
vllm
version
0.20.2rc1.dev2+gc51df4300.d20260523

serving

max concurrency
48
max context tokens
32,768
tensor parallel
4
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-09-01T01:32:29Z
engine.graph mode · reason explicit-cudagraph-mode-in-observed-command
engine.graph mode · state known
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-09-01T10:20:39Z
serving.max concurrency · reason explicit-source-server-capacity
serving.max concurrency · state known
serving.max context tokens · provenance · captured at
2026-09-01T10:20:39Z
serving.max context tokens · reason explicit-source-context-limit
serving.max context tokens · state known

metadata

localmaxxing · batch size
1
localmaxxing · hardware label
Intel Arc Pro B70
localmaxxing · notes
Qwen3.6-35B-A3B Quark W8A8 INT8 on 4x Intel Arc Pro B70 via vLLM XPU. Accepted production-candidate frontdoor run, not diagnostic route-capture. Mean of 4 measured streaming /v1/completions runs after warmup: corrected after-first output tok/s=99.77, e2e output tok/s=98.27, TTFT=76.53 ms. Prompt/output 512/512, batch/concurrency 1, temperature 0, top_k 1, top_p 1, prefix caching disabled. Quality gates for accepted runtime passed earlier same day: exact, repeat canary, structured/math/code prompts, and 8192-token needle. Peak VRAM is total xpu-smi allocation across 4 cards; observed about 31.89 GiB per B70 because gpu_memory_utilization=0.95 reserves nearly all visible memory.
localmaxxing · observed command
vllm serve /mnt/fast-ai/llm-cache/hf/models--nameistoken--Qwen3.6-35B-A3B-Quark-W8A8-INT8/snapshots/cced56592e8c8935f8220836b4baa04dfd389118 --host 127.0.0.1 --port 18080 --trust-remote-code --served-model-name qwen36-35b-a3b-fp8 --dtype auto --quantization quark --tensor-parallel-size 4 --pipeline-parallel-size 1 --distributed-executor-backend mp --max-model-len 32768 --max-num-batched-tokens 8192 --max-num-seqs 48 --gpu-memory-utilization 0.95 --kv-cache-dtype auto --no-enable-prefix-caching --language-model-only --compilation-config '{"cudagraph_mode":"PIECEWISE"}' --generation-config vllm
localmaxxing · run id
cmq9ifq0500b0r8012f27j1xl
localmaxxing · tokenized · arguments
vllm, serve, /mnt/fast-ai/llm-cache/hf/models--nameistoken--Qwen3.6-35B-A3B-Quark-W8A8-INT8/snapshots/cced56592e8c8935f8220836b4baa04dfd389118, --host, 127.0.0.1, --port, 18080, --trust-remote-code, --served-model-name, qwen36-35b-a3b-fp8, --dtype, auto, --quantization, quark, --tensor-parallel-size, 4, --pipeline-parallel-size, 1, --distributed-executor-backend, mp, --max-model-len, 32768, --max-num-batched-tokens, 8192, --max-num-seqs, 48, --gpu-memory-utilization, 0.95, --kv-cache-dtype, auto, --no-enable-prefix-caching, --language-model-only, --compilation-config, {"cudagraph_mode":"PIECEWISE"}, --generation-config, vllm
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmq9ifq0500b0r8012f27j1xl ↗