Local AI Registry

Recipe

qwen3-8-27b-uncensored-int4-autoround-gptq-int4-intel-arc-pro-b70-32gb-vllm-tp2

qwen3-8-27b-uncensored-int4-autoround-gptq-int4-intel-arc-pro-b70-32gb-vllm-tp2
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.27.2rc1.dev78+g8e6d8e4f6 XPU (custom-patched: host-staged TP2 collectives + INT8 W8A8 lm_head + MTP wiring)
Accelerators
2
Tensor parallel
2
Context tokens
2,048
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound
Repository
johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmt0vu76q0fvtms017exhssx9

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. vllm
  2. serve
  3. johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound
  4. --host
  5. 0.0.0.0
  6. --port
  7. 19622
  8. --served-model-name
  9. Qwen3.8-27B-UNC-G128-AR
  10. --tensor-parallel-size
  11. 2
  12. --dtype
  13. float16
  14. --max-model-len
  15. 262144
  16. --max-num-seqs
  17. 1
  18. --max-num-batched-tokens
  19. 8192
  20. --gpu-memory-utilization
  21. 0.95
  22. --quantization
  23. gptq
  24. --speculative-config
  25. {"method":"qwen3_5_mtp","num_speculative_tokens":3}
FlagValue
--host0.0.0.0
--port19622
--served-model-nameQwen3.8-27B-UNC-G128-AR
--tensor-parallel-size2
--dtypefloat16
--max-model-len262144
--max-num-seqs1
--max-num-batched-tokens8192
--gpu-memory-utilization0.95
--quantizationgptq
--speculative-config{"method":"qwen3_5_mtp","num_speculative_tokens":3}

Environment

VariableValue
CCL_TOPO_P2P_ACCESS0
VLLM_XPU_HOST_STAGED_COLLECTIVES1
VLLM_XPU_LM_HEAD_INT81

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
—2,048755136.4841observedqwen3-8-27b-uncensored-int4-autoround-gptq-int4-intel-arc-pro-b70-32gb-vllm-tp2-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
2
hardware id
intel-arc-pro-b70-32gb
id
qwen3-8-27b-uncensored-int4-autoround-gptq-int4-intel-arc-pro-b70-32gb-vllm-tp2
model instance id
johannrplaster-qwen3-8-27b-uncensored-int4-autoround--gptq-int4
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-8-27b-uncensored-int4-autoround-gptq-int4-intel-arc-pro-b70-32gb-vllm-tp2-sweep
status
candidate

capabilities

engine

name
vllm
version
0.27.2rc1.dev78+g8e6d8e4f6 XPU (custom-patched: host-staged TP2 collectives + INT8 W8A8 lm_head + MTP wiring)

serving

max concurrency
1
max context tokens
2,048
tensor parallel
2
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-08-31T23:03:15Z
serving.max concurrency · reason server-capacity-derived-from-source-evidence
serving.max concurrency · state known

metadata

localmaxxing · backend
xpu
localmaxxing · batch size
1
localmaxxing · hardware label
Intel Arc Pro B70
localmaxxing · notes
johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound GPTQ-Int4 served via vLLM Marlin after auto-round->gptq config flip; GDN linear-attention in_proj_a/b kept FP16 (quality-preserving); load-time INT8 W8A8 lm_head (per-channel RTN). 2x Intel Arc Pro B70 TP2 (host-staged collectives, no peer IPC), MTP3 speculative decode, FP16 KV, greedy. Strict cold suite 94.58 tok/s, quality gates pass.
localmaxxing · observed command
VLLM_XPU_LM_HEAD_INT8=1 VLLM_XPU_HOST_STAGED_COLLECTIVES=1 CCL_TOPO_P2P_ACCESS=0 vllm serve johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound --host 0.0.0.0 --port 19622 --served-model-name Qwen3.8-27B-UNC-G128-AR --tensor-parallel-size 2 --dtype float16 --max-model-len 262144 --max-num-seqs 1 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.95 --quantization gptq --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
localmaxxing · run id
cmt0vu76q0fvtms017exhssx9
localmaxxing · tokenized · arguments
vllm, serve, johannrplaster/Qwen3.8-27B-Uncensored-int4-AutoRound, --host, 0.0.0.0, --port, 19622, --served-model-name, Qwen3.8-27B-UNC-G128-AR, --tensor-parallel-size, 2, --dtype, float16, --max-model-len, 262144, --max-num-seqs, 1, --max-num-batched-tokens, 8192, --gpu-memory-utilization, 0.95, --quantization, gptq, --speculative-config, {"method":"qwen3_5_mtp","num_speculative_tokens":3}
localmaxxing · tokenized · environment · CCL TOPO P2P ACCESS
0
localmaxxing · tokenized · environment · VLLM XPU HOST STAGED COLLECTIVES
1
localmaxxing · tokenized · environment · VLLM XPU LM HEAD INT8
1
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmt0vu76q0fvtms017exhssx9 ↗