Local AI Registry

Recipe

qwen3-8-27b-q6-k-xl-rtx-5090-32gb-llama-cpp-tp1

qwen3-8-27b-q6-k-xl-rtx-5090-32gb-llama-cpp-tp1
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
llama.cpp
Engine version
10437, commit 16d222fc5
Accelerators
1
Tensor parallel
1
Context tokens
131,000
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
Repository
unsloth/Qwen3.8-27B-GGUF
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

llama.cpp

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmsy5pk6w0cqtms01rbwkcn5t

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. llama-server.exe
  2. --host
  3. 127.0.0.1
  4. --port
  5. 8099
  6. -m
  7. C:\models--unsloth--Qwen3.8-27B-GGUF\\snapshots\\4604b899a826000505a834e623272db5b7fd62f6\\Qwen3.8-27B-UD-Q6_K_XL.gguf
  8. --no-mmproj
  9. --fit-target
  10. 512
  11. --load-mode
  12. dio
  13. --spec-type
  14. draft-mtp,ngram-mod,ngram-map-k4v
  15. --spec-draft-n-max
  16. 3
  17. --spec-draft-p-min
  18. 0.0
  19. --spec-ngram-mod-n-match
  20. 24
  21. --spec-ngram-mod-n-min
  22. 48
  23. --spec-ngram-mod-n-max
  24. 64
  25. --spec-ngram-map-k4v-size-n
  26. 16
  27. --spec-ngram-map-k4v-size-m
  28. 96
  29. --spec-ngram-map-k4v-min-hits
  30. 1
  31. -ngl
  32. 999
  33. -c
  34. 131000
  35. -b
  36. 512
  37. -ub
  38. 512
  39. -np
  40. 1
  41. -fa
  42. on
  43. -ctk
  44. q8_0
  45. -ctv
  46. q8_0
  47. --temp
  48. 0.0
  49. --top-k
  50. 1
  51. --jinja
  52. --chat-template-file
  53. C:\qwen3.8_chat_template_froggeric6level.jinja
FlagValue
--host127.0.0.1
--port8099
-mC:\models--unsloth--Qwen3.8-27B-GGUF\\snapshots\\4604b899a826000505a834e623272db5b7fd62f6\\Qwen3.8-27B-UD-Q6_K_XL.gguf
--fit-target512
--load-modedio
--spec-typedraft-mtp,ngram-mod,ngram-map-k4v
--spec-draft-n-max3
--spec-draft-p-min0.0
--spec-ngram-mod-n-match24
--spec-ngram-mod-n-min48
--spec-ngram-mod-n-max64
--spec-ngram-map-k4v-size-n16
--spec-ngram-map-k4v-size-m96
--spec-ngram-map-k4v-min-hits1
-ngl999
-c131000
-b512
-ub512
-np1
-faon
-ctkq8_0
-ctvq8_0
--temp0.0
--top-k1
--chat-template-fileC:\qwen3.8_chat_template_froggeric6level.jinja

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
1131,0002,032.193.6279.2observedqwen3-8-27b-q6-k-xl-rtx-5090-32gb-llama-cpp-tp1-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
1
hardware id
rtx-5090-32gb
id
qwen3-8-27b-q6-k-xl-rtx-5090-32gb-llama-cpp-tp1
model instance id
unsloth-qwen3-8-27b-gguf--q6-k-xl
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
qwen3-8-27b-q6-k-xl-rtx-5090-32gb-llama-cpp-tp1-sweep
status
candidate

capabilities

draft launch

accelerator backend
nvidia
arguments
-hf, unsloth/Qwen3.8-27B-GGUF:Q6_K_XL, --n-gpu-layers, 999, --host, 0.0.0.0, --port, 8080, -c, 131000
container port
8,080
environment · LLAMA CACHE
/root/.cache/huggingface
host port
8,080
image
ghcr.io/ggml-org/llama.cpp:server-cuda12-b10481@sha256:b2497f8834f5ecb4e38530f6bf2734b8e0be107ff48e4720145911c86930f2ce
ipc
host
kind
docker
shm size
16g
synthesized · generated at
2026-08-31T22:12:17Z
synthesized · image provenance
gemma-4-12b-q4-k-m-rtx-3060-12gb-llama-cpp-tp1
synthesized · template
llama-cpp-server-v1

mounts

read onlytarget
No/root/.cache/huggingface

engine

name
llama.cpp
version
10437, commit 16d222fc5

serving

max concurrency
1
max context tokens
131,000
tensor parallel
1
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
draft launch.entrypoint · provenance · captured at
2026-09-01T01:41:26Z
draft launch.entrypoint · reason linux-amd64-container-config-entrypoint
draft launch.entrypoint · state known
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-08-31T23:03:15Z
serving.max concurrency · reason server-capacity-derived-from-source-evidence
serving.max concurrency · state known

metadata

localmaxxing · backend
cuda
localmaxxing · batch size
1
localmaxxing · hardware label
RTX 5090
localmaxxing · notes
x.com/Rabbit_Hole_Aias-served%20llama-swap%20serving%20flags%20EXCEPT%20sampling%20forced%20greedy%20(temp%200%20top-k%201;%20production%20serves%20temp%201.0);%20median%20of%203%20single-stream%20runs%20on%203%20distinct%20~420-token%20prompts,%20512%20tok%20out,%20each%20run%20first%20sight%20of%20its%20prompt%20(cache_prompt=false,%20no%20prompt%20reuse,%20fresh%20server);%20spec%20draft%20acceptance%20per%20run:%200.407/0.494/0.445%20(greedy%20sampling%20raises%20acceptance%20vs%20production%20temp%201.0%20-%20no-spec%20tg128%20baseline%20is%20the%20floor);%20no-spec%20llama-bench%20baseline:%20pp512=3501.659567%20tg128=56.878886;%20context%20allocated%20131000,%20throughput%20measured%20at%20~420-token%20depth;%20peak%20VRAM%20=%20whole-card%20nvidia-smi%20incl%20605%20MiB%20desktop%20baseline;%20raw%20/completion%20(chat%20template%20not%20applied);%20GPU%20power%20=%20max%200.5s%20nvidia-smi%20sample%20during%20generation;%20Qwen3.8-27B%20(stock%20Unsloth%20UD-Q6_K_XL,%20control) ↗
localmaxxing · observed command
llama-server.exe --host 127.0.0.1 --port 8099 -m C:\models--unsloth--Qwen3.8-27B-GGUF\\snapshots\\4604b899a826000505a834e623272db5b7fd62f6\\Qwen3.8-27B-UD-Q6_K_XL.gguf --no-mmproj --fit-target 512 --load-mode dio --spec-type draft-mtp,ngram-mod,ngram-map-k4v --spec-draft-n-max 3 --spec-draft-p-min 0.0 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 --spec-ngram-map-k4v-size-n 16 --spec-ngram-map-k4v-size-m 96 --spec-ngram-map-k4v-min-hits 1 -ngl 999 -c 131000 -b 512 -ub 512 -np 1 -fa on -ctk q8_0 -ctv q8_0 --temp 0.0 --top-k 1 --jinja --chat-template-file C:\qwen3.8_chat_template_froggeric6level.jinja
localmaxxing · run id
cmsy5pk6w0cqtms01rbwkcn5t
localmaxxing · tokenized · arguments
llama-server.exe, --host, 127.0.0.1, --port, 8099, -m, C:\models--unsloth--Qwen3.8-27B-GGUF\\snapshots\\4604b899a826000505a834e623272db5b7fd62f6\\Qwen3.8-27B-UD-Q6_K_XL.gguf, --no-mmproj, --fit-target, 512, --load-mode, dio, --spec-type, draft-mtp,ngram-mod,ngram-map-k4v, --spec-draft-n-max, 3, --spec-draft-p-min, 0.0, --spec-ngram-mod-n-match, 24, --spec-ngram-mod-n-min, 48, --spec-ngram-mod-n-max, 64, --spec-ngram-map-k4v-size-n, 16, --spec-ngram-map-k4v-size-m, 96, --spec-ngram-map-k4v-min-hits, 1, -ngl, 999, -c, 131000, -b, 512, -ub, 512, -np, 1, -fa, on, -ctk, q8_0, -ctv, q8_0, --temp, 0.0, --top-k, 1, --jinja, --chat-template-file, C:\qwen3.8_chat_template_froggeric6level.jinja
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmsy5pk6w0cqtms01rbwkcn5t ↗