Local AI Registry

Recipe

qwen3-8-27b-w4a16-autoround-rtx-3090-24gb-vllm-tp2

qwen3-8-27b-w4a16-autoround-rtx-3090-24gb-vllm-tp2
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.27.1+syv-ai.00210159
Accelerators
2
Tensor parallel
2
Context tokens
262,144
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/dbirks/Qwen3.8-27B-W4A16-AutoRound
Repository
dbirks/Qwen3.8-27B-W4A16-AutoRound
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmt8rd616009ali01qnnrasi7

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. vllm
  2. serve
  3. /app/models/Qwen3.8-27B-W4A16-AutoRound
  4. --served-model-name
  5. qwen3.8-27b
  6. --tensor-parallel-size
  7. 2
  8. --gpu-memory-utilization
  9. 0.95
  10. --max-model-len
  11. 262144
  12. --max-num-seqs
  13. 4
  14. --max-num-batched-tokens
  15. 2048
  16. --attention-backend
  17. FLASH_ATTN
  18. --kv-cache-dtype
  19. bfloat16
  20. --mamba-ssm-cache-dtype
  21. float16
  22. --language-model-only
  23. --speculative-config
  24. {"method":"dflash","model":"/app/models/Qwen3.8-27B-DFlash2-W4A16","num_speculative_tokens":4}
FlagValue
--served-model-nameqwen3.8-27b
--tensor-parallel-size2
--gpu-memory-utilization0.95
--max-model-len262144
--max-num-seqs4
--max-num-batched-tokens2048
--attention-backendFLASH_ATTN
--kv-cache-dtypebfloat16
--mamba-ssm-cache-dtypefloat16
--speculative-config{"method":"dflash","model":"/app/models/Qwen3.8-27B-DFlash2-W4A16","num_speculative_tokens":4}

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
1262,1441,343.8173.1825.3observedqwen3-8-27b-w4a16-autoround-rtx-3090-24gb-vllm-tp2-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
2
hardware id
rtx-3090-24gb
id
qwen3-8-27b-w4a16-autoround-rtx-3090-24gb-vllm-tp2
model instance id
dbirks-qwen3-8-27b-w4a16-autoround--w4a16-autoround
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-8-27b-w4a16-autoround-rtx-3090-24gb-vllm-tp2-sweep
status
candidate

capabilities

draft launch

accelerator backend
nvidia
arguments
--model, dbirks/Qwen3.8-27B-W4A16-AutoRound, --tensor-parallel-size, 2, --host, 0.0.0.0, --port, 8000, --max-model-len, 262144
container port
8,000
host port
8,000
image
vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
ipc
host
kind
docker
shm size
16g
synthesized · generated at
2026-08-31T22:12:17Z
synthesized · image provenance
deepseek-fp8-rtx-pro-6000-blackwell-96gb-vllm-tp1
synthesized · template
vllm-openai-v1

mounts

read onlytarget
No/root/.cache/huggingface

engine

name
vllm
version
0.27.1+syv-ai.00210159

serving

max concurrency
1
max context tokens
262,144
tensor parallel
2
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown

metadata

localmaxxing · backend
cuda
localmaxxing · hardware label
RTX 3090
localmaxxing · notes
C1 remote LocalMaxxing CLI run: 1 untimed warmup plus 3 timed iterations; medians submitted. Requested a nominal 1,024-token synthesized prompt, but endpoint usage reported 1,109 prompt tokens; endpoint count is authoritative and prefill throughput uses 1,109 / TTFT. Greedy temperature 0, 512 generated tokens, streaming inter-token decode timing. Target is dbirks AutoRound W4A16 group-128, prepared in place with int8 embeddings, LM head, and MTP auxiliary weights; FAST_VARIANT=0. Drafter is syvai DFlash2 W4A16 with 4 speculative tokens. BF16 KV, FP16 recurrent state, FlashAttention 2, chunked prefill, prefix caching disabled, TP2. Cards are PHB with no NVLink/P2P, so vLLM used PYNCCL host-mediated all-reduce. Peak telemetry: 22,713/22,711 MiB VRAM and 328.57/318.36 W; configured card power limits are 390/350 W. Server max context 262,144, max sequences 4, GPU utilization 0.95.
localmaxxing · observed command
vllm serve /app/models/Qwen3.8-27B-W4A16-AutoRound --served-model-name qwen3.8-27b --tensor-parallel-size 2 --gpu-memory-utilization 0.95 --max-model-len 262144 --max-num-seqs 4 --max-num-batched-tokens 2048 --attention-backend FLASH_ATTN --kv-cache-dtype bfloat16 --mamba-ssm-cache-dtype float16 --language-model-only --speculative-config '{"method":"dflash","model":"/app/models/Qwen3.8-27B-DFlash2-W4A16","num_speculative_tokens":4}'
localmaxxing · run id
cmt8rd616009ali01qnnrasi7
localmaxxing · tokenized · arguments
vllm, serve, /app/models/Qwen3.8-27B-W4A16-AutoRound, --served-model-name, qwen3.8-27b, --tensor-parallel-size, 2, --gpu-memory-utilization, 0.95, --max-model-len, 262144, --max-num-seqs, 4, --max-num-batched-tokens, 2048, --attention-backend, FLASH_ATTN, --kv-cache-dtype, bfloat16, --mamba-ssm-cache-dtype, float16, --language-model-only, --speculative-config, {"method":"dflash","model":"/app/models/Qwen3.8-27B-DFlash2-W4A16","num_speculative_tokens":4}
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmt8rd616009ali01qnnrasi7 ↗