Recipe

qwen359b-exl3-4bpw-rtx3080-tabbyapi-tp1

qwen359b-exl3-4bpw-rtx3080-tabbyapi-tp1

Candidate: Qwen3.5-9B-EXL3-4bpw on one rtx-3080-12gb via TabbyAPI/ExLlamaV3, 131072 tokens, Q4 cache, bridge networking. Pending acceptance on the card.

Record

Status: candidate
Source: 0xsero
Engine: tabbyapi
Engine version: 0.0.1+e632af41
Graph: piecewise
Accelerators: 1
Tensor parallel: 1
Context tokens: 131,072
Max concurrency: 2
KV cache tokens: 131,072
chat: yes
reasoning: unknown
tools: unknown
vision: no

Hugging Face model card

Identity

https://huggingface.co/turboderp/Qwen3.5-9B-exl3

Repository: turboderp/Qwen3.5-9B-exl3
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

tabbyapi

Evidence only · candidate · reference

Candidate evidence — not a Run contract

No tokenized launch fields. This is measured or documented compatibility, not a Docker launch.

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-3080-12gb
id: qwen359b-exl3-4bpw-rtx3080-tabbyapi-tp1
model instance id: turboderp-qwen3-5-9b-exl3--4-bpw
recipe source: 0xsero
schema version: local-ai-registry/v1
status: candidate

capabilities

chat: Yes
vision: No

draft launch

accelerator backend: nvidia
arguments: main.py, --config, /app/config.yml
container port: 5,000
entrypoint: /opt/venv/bin/python3
environment · NVIDIA VISIBLE DEVICES: all
host port: 5,000
image: ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448
kind: docker
shm size: 8g
synthesized · generated at: 2026-09-02T22:33:56Z
synthesized · image provenance: gemma-4-12b-it-exl3-4bpw-rtx3090-tabbyapi-tp1
synthesized · template: tabbyapi-exl3-bridge-v1

mounts

read only	target
Yes	/workspace/models
Yes	/app/config.yml

engine

graph mode: piecewise
name: tabbyapi
version: 0.0.1+e632af41

serving

kv cache tokens: 131,072
max concurrency: 2
max context tokens: 131,072
tensor parallel: 1

Provenance & metadata (3)

facts

metadata

image provenance · attestation: gh attestation verify oci://ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448 -o 0xSero
image provenance · dockerfile: github.com/0xSero/local-ai-images/blob/main/tabbyapi-exl3/Dockerfile ↗
image provenance · kind: self-built-attested
image provenance · workflow: github.com/0xSero/local-ai-images/actions/runs/33690503239 ↗
weights subdir: Qwen3.5-9B-EXL3-4bpw

image provenance · source github.com/0xSero/local-ai-images ↗

provenance

captured at: 2026-09-02T22:33:56Z

sources

captured at	kind	url
2026-09-02T22:33:56Z	normalized-recipe	github.com/0xSero/local-ai-registry ↗