Local AI Registry

Recipe

qwen3-8-27b-fp8-intel-arc-pro-b70-32gb-vllm-tp2

qwen3-8-27b-fp8-intel-arc-pro-b70-32gb-vllm-tp2
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.26.1rc1.dev771+g8e6d8e4f6 XPU (editable tree, custom-patched: host-staged collectives + mamba_utils int64 fix + INT8 lm_head backport)
Accelerators
2
Tensor parallel
2
Context tokens
262,144
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/JonathanColetti/Qwen3.8-27B-Uncensored
Repository
JonathanColetti/Qwen3.8-27B-Uncensored
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmt2a84b70cp3mv0161xvsnfh

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. vllm
  2. serve
  3. /data/models/Qwen3.8-27B-Uncensored-bf16-full
  4. --host
  5. 0.0.0.0
  6. --port
  7. 19622
  8. --served-model-name
  9. Qwen3.8-27B-UNC-FP8
  10. --tensor-parallel-size
  11. 2
  12. --dtype
  13. float16
  14. --max-model-len
  15. 262144
  16. --max-num-seqs
  17. 1
  18. --max-num-batched-tokens
  19. 8192
  20. --gpu-memory-utilization
  21. 0.95
  22. --kv-cache-dtype
  23. fp8
  24. --enable-prompt-tokens-details
  25. --enable-auto-tool-choice
  26. --tool-call-parser
  27. qwen3_coder
  28. --reasoning-parser
  29. qwen3
  30. --default-chat-template-kwargs
  31. {"enable_thinking": false, "reasoning_effort": "low"}
  32. --speculative-config
  33. {"method":"qwen3_5_mtp","num_speculative_tokens":3}
  34. --quantization
  35. fp8
  36. --chat-template
  37. /data/models/Qwen3.8-27B-Uncensored-g128-AutoRound/chat_template.jinja
FlagValue
--host0.0.0.0
--port19622
--served-model-nameQwen3.8-27B-UNC-FP8
--tensor-parallel-size2
--dtypefloat16
--max-model-len262144
--max-num-seqs1
--max-num-batched-tokens8192
--gpu-memory-utilization0.95
--kv-cache-dtypefp8
--tool-call-parserqwen3_coder
--reasoning-parserqwen3
--default-chat-template-kwargs{"enable_thinking": false, "reasoning_effort": "low"}
--speculative-config{"method":"qwen3_5_mtp","num_speculative_tokens":3}
--quantizationfp8
--chat-template/data/models/Qwen3.8-27B-Uncensored-g128-AutoRound/chat_template.jinja

Environment

VariableValue
USE_PINNED_ONECCL1
VLLM_XPU_HOST_STAGED_COLLECTIVES1

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
—2,048596.868.7316.7observedqwen3-8-27b-fp8-intel-arc-pro-b70-32gb-vllm-tp2-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
2
hardware id
intel-arc-pro-b70-32gb
id
qwen3-8-27b-fp8-intel-arc-pro-b70-32gb-vllm-tp2
model instance id
jonathancoletti-qwen3-8-27b-uncensored--fp8
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-8-27b-fp8-intel-arc-pro-b70-32gb-vllm-tp2-sweep
status
candidate

capabilities

engine

name
vllm
version
0.26.1rc1.dev771+g8e6d8e4f6 XPU (editable tree, custom-patched: host-staged collectives + mamba_utils int64 fix + INT8 lm_head backport)

serving

max concurrency
1
max context tokens
262,144
tensor parallel
2
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-09-01T10:20:39Z
serving.max concurrency · reason explicit-source-server-capacity
serving.max concurrency · state known
serving.max context tokens · provenance · captured at
2026-09-01T10:20:39Z
serving.max context tokens · reason explicit-source-context-limit
serving.max context tokens · state known

metadata

localmaxxing · backend
xpu
localmaxxing · batch size
1
localmaxxing · hardware label
Intel Arc Pro B70
localmaxxing · notes
JonathanColetti/Qwen3.8-27B-Uncensored FP8 (load-time W8A8 e4m3, Fp8PerTensorOnlineLinearMethod) served via vLLM on 2x Intel Arc Pro B70 TP2 (non-P2P host-staged collectives), MTP3 speculative decode, INT8 lm_head, FP8 KV, 262144 ctx, greedy. Prompt shape prewarmed before measuring; iterations 67.3-71.0. Strict cold-suite 66.3 tok/s (tokens 1-100 after TTFT, cached_tokens=0), quality gate passed.
localmaxxing · observed command
VLLM_XPU_HOST_STAGED_COLLECTIVES=1 USE_PINNED_ONECCL=1 vllm serve /data/models/Qwen3.8-27B-Uncensored-bf16-full --host 0.0.0.0 --port 19622 --served-model-name Qwen3.8-27B-UNC-FP8 --tensor-parallel-size 2 --dtype float16 --max-model-len 262144 --max-num-seqs 1 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.95 --kv-cache-dtype fp8 --enable-prompt-tokens-details --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --default-chat-template-kwargs '{"enable_thinking": false, "reasoning_effort": "low"}' --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' --quantization fp8 --chat-template /data/models/Qwen3.8-27B-Uncensored-g128-AutoRound/chat_template.jinja
localmaxxing · run id
cmt2a84b70cp3mv0161xvsnfh
localmaxxing · tokenized · arguments
vllm, serve, /data/models/Qwen3.8-27B-Uncensored-bf16-full, --host, 0.0.0.0, --port, 19622, --served-model-name, Qwen3.8-27B-UNC-FP8, --tensor-parallel-size, 2, --dtype, float16, --max-model-len, 262144, --max-num-seqs, 1, --max-num-batched-tokens, 8192, --gpu-memory-utilization, 0.95, --kv-cache-dtype, fp8, --enable-prompt-tokens-details, --enable-auto-tool-choice, --tool-call-parser, qwen3_coder, --reasoning-parser, qwen3, --default-chat-template-kwargs, {"enable_thinking": false, "reasoning_effort": "low"}, --speculative-config, {"method":"qwen3_5_mtp","num_speculative_tokens":3}, --quantization, fp8, --chat-template, /data/models/Qwen3.8-27B-Uncensored-g128-AutoRound/chat_template.jinja
localmaxxing · tokenized · environment · USE PINNED ONECCL
1
localmaxxing · tokenized · environment · VLLM XPU HOST STAGED COLLECTIVES
1
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmt2a84b70cp3mv0161xvsnfh ↗