Recipe

qwen3-5-0-8b-bf16-rtx-3060-12gb-llama-cpp-tp1

qwen3-5-0-8b-bf16-rtx-3060-12gb-llama-cpp-tp1

JSON API Find compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status: candidate
Source: localmaxxing
Engine: llama.cpp
Engine version: b9775 / be4a6a63e
Accelerators: 1
Tensor parallel: 1
Context tokens: 4,608
chat: unknown
reasoning: unknown
tools: unknown
vision: unknown

Hugging Face model card

Identity

https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF

Repository: unsloth/Qwen3.5-0.8B-GGUF
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

llama.cpp

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source: https://www.localmaxxing.com/en/runs/cmrz00iuo05hwo40137peiepl

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

/home/dario/llama.cpp/build/bin/llama-bench
-m
/mnt/hdd/lmstudio_models/unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-BF16.gguf
-p
4096
-n
512
-b
4096
-ub
512
-t
32
-ngl
99
-fa
on
-r
2
-o
json
--no-warmup

Flag	Value
`-m`	`/mnt/hdd/lmstudio_models/unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-BF16.gguf`
`-p`	`4096`
`-n`	`512`
`-b`	`4096`
`-ub`	`512`
`-t`	`32`
`-ngl`	`99`
`-fa`	`on`
`-r`	`2`
`-o`	`json`

Measured speed

Concurrency	Context	Prefill	Decode	TTFT ms	Status	Sweep
1	4,608	7,020.8	143.3	590.4	observed	qwen3-5-0-8b-bf16-rtx-3060-12gb-llama-cpp-tp1-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-3060-12gb
id: qwen3-5-0-8b-bf16-rtx-3060-12gb-llama-cpp-tp1
model instance id: unsloth-qwen3-5-0-8b-gguf--bf16
recipe source: localmaxxing
schema version: local-ai-registry/v1
speed sweep ids: qwen3-5-0-8b-bf16-rtx-3060-12gb-llama-cpp-tp1-sweep
status: candidate

capabilities

draft launch

accelerator backend: nvidia
arguments: -hf, unsloth/Qwen3.5-0.8B-GGUF:BF16, --n-gpu-layers, 999, --host, 0.0.0.0, --port, 8080, -c, 4608
container port: 8,080
entrypoint: /app/llama-server
environment · LLAMA CACHE: /root/.cache/huggingface
host port: 8,080
image: ghcr.io/ggml-org/llama.cpp:server-cuda12-b10481@sha256:b2497f8834f5ecb4e38530f6bf2734b8e0be107ff48e4720145911c86930f2ce
ipc: host
kind: docker
shm size: 16g
synthesized · generated at: 2026-08-31T22:12:17Z
synthesized · image provenance: gemma-4-12b-q4-k-m-rtx-3060-12gb-llama-cpp-tp1
synthesized · template: llama-cpp-server-v1

mounts

read only	target
No	/root/.cache/huggingface

engine

name: llama.cpp
version: b9775 / be4a6a63e

serving

max context tokens: 4,608
tensor parallel: 1

Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.chat · reason capability-not-verified

capabilities.chat · state unknown

capabilities.reasoning · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.reasoning · reason capability-not-verified

capabilities.reasoning · state unknown

capabilities.tools · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.tools · reason capability-not-verified

capabilities.tools · state unknown

capabilities.vision · provenance · captured at: 2026-08-30T09:10:02Z

capabilities.vision · reason capability-not-verified

capabilities.vision · state unknown

draft launch.entrypoint · provenance · captured at: 2026-09-01T01:41:26Z

draft launch.entrypoint · reason linux-amd64-container-config-entrypoint

draft launch.entrypoint · state known

engine.graph mode · provenance · captured at: 2026-08-30T09:10:02Z

engine.graph mode · reason runtime-detail-not-published

engine.graph mode · state unknown

serving.kv cache tokens · provenance · captured at: 2026-08-30T09:10:02Z

serving.kv cache tokens · reason kv-cache-capacity-not-published

serving.kv cache tokens · state unknown

serving.max concurrency · provenance · captured at: 2026-08-31T23:03:15Z

serving.max concurrency · reason server-capacity-not-evidenced

serving.max concurrency · state unknown

metadata

localmaxxing · backend: cuda
localmaxxing · batch size: 1
localmaxxing · hardware label: RTX 3060
localmaxxing · notes: 4096 prompt / 512 generation. Two throwaway llama-bench runs preceded the recorded sweep. Winner selected by minimum measured TTFT across (b2048/ub512/t16), (b4096/ub1024/t16), and (b4096/ub512/t32), with exact command preserved. Local file: /mnt/hdd/lmstudio_models/unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-BF16.gguf. Temperature 0. Peak VRAM and maximum sampled sustained GPU power were measured during the winner run.
localmaxxing · observed command: /home/dario/llama.cpp/build/bin/llama-bench -m /mnt/hdd/lmstudio_models/unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-BF16.gguf -p 4096 -n 512 -b 4096 -ub 512 -t 32 -ngl 99 -fa on -r 2 -o json --no-warmup
localmaxxing · run id: cmrz00iuo05hwo40137peiepl
localmaxxing · tokenized · arguments: /home/dario/llama.cpp/build/bin/llama-bench, -m, /mnt/hdd/lmstudio_models/unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-BF16.gguf, -p, 4096, -n, 512, -b, 4096, -ub, 512, -t, 32, -ngl, 99, -fa, on, -r, 2, -o, json, --no-warmup
localmaxxing · tokenized · fidelity: faithful

provenance

captured at: 2026-08-30T09:10:02Z

sources

captured at	kind	url
2026-08-30T09:10:02Z	normalized-recipe	www.localmaxxing.com/en/runs/cmrz00iuo05hwo40137peiepl ↗