Recipe

qwen3-6-35b-a3b-q4-k-m-rtx-5090-32gb-llama-cpp-tp1

qwen3-6-35b-a3b-q4-k-m-rtx-5090-32gb-llama-cpp-tp1

JSON API Find compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status: candidate
Source: localmaxxing
Engine: llama.cpp
Engine version: b6217 (6217b49)
Accelerators: 1
Tensor parallel: 1
Context tokens: 4,096
Max concurrency: 1
chat: unknown
reasoning: unknown
tools: unknown
vision: unknown

Hugging Face model card

Identity

https://huggingface.co/Qwen/Qwen3.6-35B-A3B

Repository: Qwen/Qwen3.6-35B-A3B
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

llama.cpp

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source: https://www.localmaxxing.com/en/runs/cmp4xieqn0002pg01fz0ir9ry

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

/home/steven/dev/llama.cpp/build/bin/llama-server
-m
Qwen3.6-35B-A3B-Q4_K_M.gguf
--alias
qwen3.6-35b-a3b-q4km
-fa
1
-ngl
999
-c
4096
-ctk
q8_0
-ctv
q8_0
--no-context-shift
--parallel
1
-b
2048
-ub
1024
--metrics
--jinja
--reasoning
off
--temp
1.0
--top-p
0.95
--top-k
20
--min-p
0.0
--host
0.0.0.0
--port
8000

Flag	Value
`-m`	`Qwen3.6-35B-A3B-Q4_K_M.gguf`
`--alias`	`qwen3.6-35b-a3b-q4km`
`-fa`	`1`
`-ngl`	`999`
`-c`	`4096`
`-ctk`	`q8_0`
`-ctv`	`q8_0`
`--parallel`	`1`
`-b`	`2048`
`-ub`	`1024`
`--reasoning`	`off`
`--temp`	`1.0`
`--top-p`	`0.95`
`--top-k`	`20`
`--min-p`	`0.0`
`--host`	`0.0.0.0`
`--port`	`8000`

Measured speed

Concurrency	Context	Prefill	Decode	TTFT ms	Status	Sweep
1	4,096	—	230.7	115.2	observed	qwen3-6-35b-a3b-q4-k-m-rtx-5090-32gb-llama-cpp-tp1-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-5090-32gb
id: qwen3-6-35b-a3b-q4-k-m-rtx-5090-32gb-llama-cpp-tp1
model instance id: qwen-qwen3-6-35b-a3b--q4-k-m
recipe source: localmaxxing
schema version: local-ai-registry/v1
speed sweep ids: qwen3-6-35b-a3b-q4-k-m-rtx-5090-32gb-llama-cpp-tp1-sweep
status: candidate

capabilities

engine

name: llama.cpp
version: b6217 (6217b49)

serving

max concurrency: 1
max context tokens: 4,096
tensor parallel: 1

Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.chat · reason capability-not-verified

capabilities.chat · state unknown

capabilities.reasoning · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.reasoning · reason capability-not-verified

capabilities.reasoning · state unknown

capabilities.tools · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.tools · reason capability-not-verified

capabilities.tools · state unknown

capabilities.vision · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.vision · reason capability-not-verified

capabilities.vision · state unknown

engine.graph mode · provenance · captured at: 2026-08-30T09:10:02Z

engine.graph mode · reason runtime-detail-not-published

engine.graph mode · state unknown

serving.kv cache tokens · provenance · captured at: 2026-08-30T09:10:02Z

serving.kv cache tokens · reason kv-cache-capacity-not-published

serving.kv cache tokens · state unknown

serving.max concurrency · provenance · captured at: 2026-08-31T23:03:15Z

serving.max concurrency · reason server-capacity-derived-from-source-evidence

serving.max concurrency · state known

metadata

localmaxxing · backend: cuda
localmaxxing · batch size: 1
localmaxxing · hardware label: RTX 5090
localmaxxing · notes: Single RTX 5090 ASUS ROG Astral OC (Blackwell GB202, sm_120a, 32 GB GDDR7), Ryzen 9 9950X + 96 GB DDR5-6000 CL28, Ubuntu 24.04 kernel 6.17, driver 580.159.03 / CUDA 13.0. Q4_K_M weights fully resident on GPU (-ngl 999), Q8 KV cache, 4K context, flash attention. Methodology: vllm bench serve random 512in/512out, 20 prompts, max-concurrency 1. Peak output token throughput 249.00 t/s during sustained decode; mean TPOT 4.12 ms.
localmaxxing · observed command: /home/steven/dev/llama.cpp/build/bin/llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf --alias qwen3.6-35b-a3b-q4km -fa 1 -ngl 999 -c 4096 -ctk q8_0 -ctv q8_0 --no-context-shift --parallel 1 -b 2048 -ub 1024 --metrics --jinja --reasoning off --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --host 0.0.0.0 --port 8000
localmaxxing · run id: cmp4xieqn0002pg01fz0ir9ry
localmaxxing · tokenized · arguments: /home/steven/dev/llama.cpp/build/bin/llama-server, -m, Qwen3.6-35B-A3B-Q4_K_M.gguf, --alias, qwen3.6-35b-a3b-q4km, -fa, 1, -ngl, 999, -c, 4096, -ctk, q8_0, -ctv, q8_0, --no-context-shift, --parallel, 1, -b, 2048, -ub, 1024, --metrics, --jinja, --reasoning, off, --temp, 1.0, --top-p, 0.95, --top-k, 20, --min-p, 0.0, --host, 0.0.0.0, --port, 8000
localmaxxing · tokenized · fidelity: faithful

provenance

captured at: 2026-08-30T09:10:02Z

sources

captured at	kind	url
2026-08-30T09:10:02Z	normalized-recipe	www.localmaxxing.com/en/runs/cmp4xieqn0002pg01fz0ir9ry ↗