Local AI Registry

Recipe

qwen3-6-27b-autoround-int4-w4a16-runtime-int8-lm-head-intel-arc-pro-b70-32gb-vllm-tp1-8408i1pn

qwen3-6-27b-autoround-int4-w4a16-runtime-int8-lm-head-intel-arc-pro-b70-32gb-vllm-tp1-8408i1pn
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.20.2rc1.dev13+g9557d9108.d20260620 local XPU patch stack
Graph
piecewise
Accelerators
1
Tensor parallel
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/Intel/Qwen3.6-27B-int4-AutoRound
Repository
Intel/Qwen3.6-27B-int4-AutoRound
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmr4zkcxb003yq9018408i1pn

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. experiments/qwen36-27b-autoround-int4-b70/scripts/serve-vllm.sh

Environment

VariableValue
COMPILATION_CONFIG{"cudagraph_mode":"PIECEWISE","max_cudagraph_capture_size":8}
GPU_INDEX2
MAX_MODEL_LEN2048
MAX_NUM_BATCHED_TOKENS1024
NUM_SPECULATIVE_TOKENS3
PORT19412
QWEN36_27B_ENABLE_MTP1
QWEN36_27B_ENABLE_XPU_GRAPH1
VLLM_XPU_GDN_NONSPEC_POSTPROCESS_ACCEPTED_STATE0
VLLM_XPU_GDN_PROMOTE_ACCEPTED_SPEC_STATE1
VLLM_XPU_LM_HEAD_INT81

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
—2,048—62.6606.6observedqwen3-6-27b-autoround-int4-w4a16-runtime-int8-lm-head-intel-arc-pro-b70-32gb-vllm-tp1-8408i1pn-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
1
hardware id
intel-arc-pro-b70-32gb
id
qwen3-6-27b-autoround-int4-w4a16-runtime-int8-lm-head-intel-arc-pro-b70-32gb-vllm-tp1-8408i1pn
model instance id
intel-qwen3-6-27b-int4-autoround--autoround-int4-w4a16-runtime-int8-lm-head
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-6-27b-autoround-int4-w4a16-runtime-int8-lm-head-intel-arc-pro-b70-32gb-vllm-tp1-8408i1pn-sweep
status
candidate

capabilities

engine

graph mode
piecewise
name
vllm
version
0.20.2rc1.dev13+g9557d9108.d20260620 local XPU patch stack

serving

tensor parallel
1
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-09-01T01:32:29Z
engine.graph mode · reason explicit-cudagraph-mode-in-observed-command
engine.graph mode · state known
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-08-31T23:03:15Z
serving.max concurrency · reason server-capacity-not-evidenced
serving.max concurrency · state unknown
serving.max context tokens · provenance · captured at
2026-09-01T10:20:39Z
serving.max context tokens · reason context-limit-not-evidenced
serving.max context tokens · state unknown

metadata

localmaxxing · backend
xpu
localmaxxing · batch size
1
localmaxxing · hardware label
Intel Arc Pro B70
localmaxxing · notes
Qwen3.6 27B AutoRound INT4 one-B70 vLLM/XPU strict fresh-response result with default-off runtime INT8 lm_head projection enabled (VLLM_XPU_LM_HEAD_INT8=1). This is labeled separately from the original BF16-LM-head AutoRound quantization: the BF16 lm_head remains resident, but the projection uses a transient per-output-channel INT8 weight copy plus per-token INT8 activation quantization. Primary metric is median generated-token throughput for tokens 1-100 after TTFT across the fixed realistic prompt suite. Each prompt was sent once; cached_tokens=0 for every request; no prompt/KV cache reuse, context checkpoints, response reuse, n-gram/history acceleration, or warmed repeated prompts. qwen3_next_mtp accepted tokens are verified by the target model. Full quality gate passed and matched baseline, including 32-repeat stability and the 1K long-context needle. Same-window BF16-lm-head control was 53.33 tok/s; INT8 repeat was 62.28 tok/s.
localmaxxing · observed command
GPU_INDEX=2 PORT=19412 MAX_MODEL_LEN=2048 MAX_NUM_BATCHED_TOKENS=1024 QWEN36_27B_ENABLE_MTP=1 NUM_SPECULATIVE_TOKENS=3 QWEN36_27B_ENABLE_XPU_GRAPH=1 VLLM_XPU_GDN_PROMOTE_ACCEPTED_SPEC_STATE=1 VLLM_XPU_GDN_NONSPEC_POSTPROCESS_ACCEPTED_STATE=0 VLLM_XPU_LM_HEAD_INT8=1 COMPILATION_CONFIG='{"cudagraph_mode":"PIECEWISE","max_cudagraph_capture_size":8}' experiments/qwen36-27b-autoround-int4-b70/scripts/serve-vllm.sh
localmaxxing · run id
cmr4zkcxb003yq9018408i1pn
localmaxxing · tokenized · arguments
experiments/qwen36-27b-autoround-int4-b70/scripts/serve-vllm.sh
localmaxxing · tokenized · environment · COMPILATION CONFIG
{"cudagraph_mode":"PIECEWISE","max_cudagraph_capture_size":8}
localmaxxing · tokenized · environment · GPU INDEX
2
localmaxxing · tokenized · environment · MAX MODEL LEN
2048
localmaxxing · tokenized · environment · MAX NUM BATCHED TOKENS
1024
localmaxxing · tokenized · environment · NUM SPECULATIVE TOKENS
3
localmaxxing · tokenized · environment · PORT
19412
localmaxxing · tokenized · environment · QWEN36 27B ENABLE MTP
1
localmaxxing · tokenized · environment · QWEN36 27B ENABLE XPU GRAPH
1
localmaxxing · tokenized · environment · VLLM XPU GDN NONSPEC POSTPROCESS ACCEPTED STATE
0
localmaxxing · tokenized · environment · VLLM XPU GDN PROMOTE ACCEPTED SPEC STATE
1
localmaxxing · tokenized · environment · VLLM XPU LM HEAD INT8
1
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmr4zkcxb003yq9018408i1pn ↗