Recipe

gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1

gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1

Candidate: Gemma-4-12B-it-EXL3-4bpw on one rtx-pro-4000-blackwell-24gb via TabbyAPI/ExLlamaV3, 131072 tokens, Q6 cache, bridge networking. Pending acceptance on the card.

Record

Status: validated
Source: 0xsero
Engine: tabbyapi
Engine version: 0.0.1+e632af41
Graph: piecewise
Accelerators: 1
Tensor parallel: 1
Context tokens: 131,072
Max concurrency: 2
KV cache tokens: 131,072
chat: yes
reasoning: unknown
tools: unknown
vision: no

Hugging Face model card

Identity

https://huggingface.co/turboderp/gemma-4-12B-it-exl3

Repository: turboderp/gemma-4-12B-it-exl3
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Validated: pinned artifact, pinned runtime, and accepted evidence. This is a launch contract.

Docker configuration

tabbyapi

Container · validated · docker

Validated launch contract

Image: ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448
Digest: sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448
Port: 5000

Launch arguments

main.py
--config
/app/config.yml

Environment

Variable	Value
`NVIDIA_VISIBLE_DEVICES`	`all`

Mounts

Source	Target
`${MODEL_ROOT}/gemma-4-12b-it-exl3-4bpw`	`/workspace/models` (read-only)
`asset/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config.yml`	`/app/config.yml` (read-only)

Launch

Exact materialization of this validated launch contract: digest-pinned image, pinned model revision, and the audited arguments. Self-contained — required assets are fetched from this registry and verified against their recorded sha256 before mounting. Also available as local-ai run gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1.

Pulls the exact container image by sha256 digest — the bytes that were validated, not a floating tag.
Fetches any required engine assets from this registry and verifies each against its recorded sha256; the audited launch script verifies them again inside the container before use.
Downloads the pinned model revision into your Hugging Face cache on first run (reused afterwards).
Serves an OpenAI-compatible API on localhost:5000 — point any client at it.

ASSETS="${TMPDIR:-/tmp}/local-ai-assets" && mkdir -p "$ASSETS" && \
curl -fsSL 'https://local-ai-registry.vercel.app/api/v1/asset/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config/file' -o "$ASSETS/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config.yml" && \
{ printf '%s  %s\n' '7091652b41b8518f042bbf6e3dc7b2e96fbc9e47920aff3a1e4af6c78f53b59a' "$ASSETS/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config.yml" | sha256sum -c - 2>/dev/null || printf '%s  %s\n' '7091652b41b8518f042bbf6e3dc7b2e96fbc9e47920aff3a1e4af6c78f53b59a' "$ASSETS/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config.yml" | shasum -a 256 -c -; } && \
docker run --rm \
  --gpus all \
  --shm-size 8g \
  -p 5000:5000 \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -v '${MODEL_ROOT}/gemma-4-12b-it-exl3-4bpw:/workspace/models:ro' \
  -v "$ASSETS/gemma412bit-exl3-4bpw-128k-q6-tabbyapi-config.yml":/app/config.yml:ro \
  --entrypoint /opt/venv/bin/python3 \
  ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448 \
  main.py \
  --config \
  /app/config.yml

Measured speed

Concurrency	Context	Prefill	Decode	TTFT ms	Status	Sweep
1	131,072	—	58.4	290.5	accepted	gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1-acceptance

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-pro-4000-blackwell-24gb
id: gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1
model instance id: turboderp-gemma-4-12b-it-exl3--4-bpw
recipe source: 0xsero
recommended: Yes
schema version: local-ai-registry/v1
speed sweep ids: gemma412b-exl3-4bpw-rtxpro4000-tabbyapi-tp1-acceptance
status: validated

capabilities

chat: Yes
vision: No

engine

graph mode: piecewise
name: tabbyapi
version: 0.0.1+e632af41

serving

kv cache tokens: 131,072
max concurrency: 2
max context tokens: 131,072
tensor parallel: 1

Provenance & metadata (3)

facts

metadata

acceptance · accepted at: 2026-09-02T23:36:27Z
acceptance · harness: validate_rented.py on vast RTX PRO 4000 (weights and config materialized in-container in place of the bind mounts)
acceptance · served model id: Gemma-4-12B-it-EXL3-4bpw
weights subdir: Gemma-4-12B-it-EXL3-4bpw

provenance

captured at: 2026-09-02T22:33:59Z

sources

captured at	kind	url
2026-09-02T22:33:59Z	normalized-recipe	github.com/0xSero/local-ai-registry ↗