Local AI Registry

Recipe

gemma-4-31b-it-q4-k-m-rtx-4090-24gb-llama-cpp-tp1

gemma-4-31b-it-q4-k-m-rtx-4090-24gb-llama-cpp-tp1
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
llama.cpp
Engine version
Atomic TurboQuant fork 514e600c8
Accelerators
1
Tensor parallel
1
Context tokens
70,080
Max concurrency
1
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/LiconStudio/Gemma-4-31B-it-abliterated-GGUF
Repository
LiconStudio/Gemma-4-31B-it-abliterated-GGUF
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

llama.cpp

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmp45iwqw0073pc01sxqx90h6

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

    1. llama-server
    2. -m
    3. LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
    4. -ngl
    5. 999
    6. -c
    7. 70080
    8. -b
    9. 2048
    10. -ub
    11. 512
    12. --flash-attn
    13. on
    14. --cache-type-k
    15. turbo4
    16. --cache-type-v
    1. turbo4
    2. -np
    3. 1
    4. --jinja
    5. --reasoning
    6. off
    7. --metrics
    8. --poll
    9. 100
    10. --poll-batch
    11. 1
    12. --prio
    13. 2
    14. --prio-batch
    15. 2
    16. --threads-http
    17. 1
    18. -ngld
    19. all
    20. --override-tensor-draft
    1. token_embd.weight=CUDA0
    2. --spec-type
    3. mtp
    4. --mtp-head
    5. google/gemma-4-31B-it-assistant:Q4_K_S
    6. --draft-block-size
    7. 4
    8. --cache-ram
    9. 0
FlagValue
-mLiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
-ngl999
-c70080
-b2048
-ub512
--flash-attnon
--cache-type-kturbo4

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
170,080171.187.7—observedgemma-4-31b-it-q4-k-m-rtx-4090-24gb-llama-cpp-tp1-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
1
hardware id
rtx-4090-24gb
id
gemma-4-31b-it-q4-k-m-rtx-4090-24gb-llama-cpp-tp1
model instance id
liconstudio-gemma-4-31b-it-abliterated-gguf--q4-k-m
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
gemma-4-31b-it-q4-k-m-rtx-4090-24gb-llama-cpp-tp1-sweep
status
candidate

capabilities

draft launch

accelerator backend
nvidia
arguments
-hf, LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M, --n-gpu-layers, 999, --host, 0.0.0.0, --port, 8080, -c, 70080
container port
8,080
environment · LLAMA CACHE
/root/.cache/huggingface
host port
8,080
image
ghcr.io/ggml-org/llama.cpp:server-cuda12-b10481@sha256:b2497f8834f5ecb4e38530f6bf2734b8e0be107f0f0857e259672d1cb85b71c2
ipc
host
kind
docker
shm size
16g
synthesized · generated at
2026-08-31T22:12:17Z
synthesized · image provenance
gemma-4-12b-q4-k-m-rtx-3060-12gb-llama-cpp-tp1
synthesized · template
llama-cpp-server-v1

mounts

read onlytarget
No/root/.cache/huggingface

engine

name
llama.cpp
version
Atomic TurboQuant fork 514e600c8

serving

max concurrency
1
max context tokens
70,080
tensor parallel
1
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
engine.graph mode · provenance · captured at
2026-08-30T09:10:02Z
engine.graph mode · reason runtime-detail-not-published
engine.graph mode · state unknown
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown

metadata

localmaxxing · backend
cuda
localmaxxing · hardware label
RTX 4090
localmaxxing · notes
Fresh 3-run 512-token validation: min output 72.74 tok/s, avg output 87.67 tok/s. Runtime used llama.cpp Atomic TurboQuant fork 514e600c8 with turbo4 K/V cache, Gemma 4 MTP assistant Q4_K_S, draft block size 4, and --override-tensor-draft token_embd.weight=CUDA0. Peak observed VRAM was 23321 MiB/24564 MiB.
localmaxxing · observed command
llama-server -m LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M -ngl 999 -c 70080 -b 2048 -ub 512 --flash-attn on --cache-type-k turbo4 --cache-type-v turbo4 -np 1 --jinja --reasoning off --metrics --poll 100 --poll-batch 1 --prio 2 --prio-batch 2 --threads-http 1 -ngld all --override-tensor-draft token_embd.weight=CUDA0 --spec-type mtp --mtp-head google/gemma-4-31B-it-assistant:Q4_K_S --draft-block-size 4 --cache-ram 0
localmaxxing · run id
cmp45iwqw0073pc01sxqx90h6
localmaxxing · tokenized · arguments
llama-server, -m, LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M, -ngl, 999, -c, 70080, -b, 2048, -ub, 512, --flash-attn, on, --cache-type-k, turbo4, --cache-type-v
localmaxxing · tokenized · fidelity
faithful

localmaxxing · tokenized · steps

01234567
llama-server-mLiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M-ngl999-c70080-b
turbo4-np1--jinja--reasoningoff--metrics--poll
token_embd.weight=CUDA0--spec-typemtp--mtp-headgoogle/gemma-4-31B-it-assistant:Q4_K_S--draft-block-size4--cache-ram

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmp45iwqw0073pc01sxqx90h6 ↗