Recipe

qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1

qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1

Candidate: Qwen3.5-9B-EXL3-4bpw on one rtx-4080-16gb via TabbyAPI/ExLlamaV3, 262144 tokens, Q4 cache, bridge networking. Pending acceptance on the card.

Record

Status: validated
Source: 0xsero
Engine: tabbyapi
Engine version: 0.0.1+e632af41
Graph: piecewise
Accelerators: 1
Tensor parallel: 1
Context tokens: 262,144
Max concurrency: 2
KV cache tokens: 262,144
chat: yes
reasoning: unknown
tools: unknown
vision: no

Hugging Face model card

Identity

https://huggingface.co/turboderp/Qwen3.5-9B-exl3

Repository: turboderp/Qwen3.5-9B-exl3
Status: known
Link type: Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Validated: pinned artifact, pinned runtime, and accepted evidence. This is a launch contract.

Docker configuration

tabbyapi

Container · validated · docker

Validated launch contract

Image: ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448
Digest: sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448
Port: 5000

Launch arguments

main.py
--config
/app/config.yml

Environment

Variable	Value
`NVIDIA_VISIBLE_DEVICES`	`all`

Mounts

Source	Target
`${MODEL_ROOT}/qwen3-5-9b-exl3-4bpw`	`/workspace/models` (read-only)
`asset/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config.yml`	`/app/config.yml` (read-only)

Launch

Exact materialization of this validated launch contract: digest-pinned image, pinned model revision, and the audited arguments. Self-contained — required assets are fetched from this registry and verified against their recorded sha256 before mounting. Also available as local-ai run qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1.

Pulls the exact container image by sha256 digest — the bytes that were validated, not a floating tag.
Fetches any required engine assets from this registry and verifies each against its recorded sha256; the audited launch script verifies them again inside the container before use.
Downloads the pinned model revision into your Hugging Face cache on first run (reused afterwards).
Serves an OpenAI-compatible API on localhost:5000 — point any client at it.

ASSETS="${TMPDIR:-/tmp}/local-ai-assets" && mkdir -p "$ASSETS" && \
curl -fsSL 'https://local-ai-registry.vercel.app/api/v1/asset/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config/file' -o "$ASSETS/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config.yml" && \
{ printf '%s  %s\n' 'a1fdefdb835233bf8c2f983cba4545b11f9d1d8627a244c31f99f94733626227' "$ASSETS/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config.yml" | sha256sum -c - 2>/dev/null || printf '%s  %s\n' 'a1fdefdb835233bf8c2f983cba4545b11f9d1d8627a244c31f99f94733626227' "$ASSETS/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config.yml" | shasum -a 256 -c -; } && \
docker run --rm \
  --gpus all \
  --shm-size 8g \
  -p 5000:5000 \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -v '${MODEL_ROOT}/qwen3-5-9b-exl3-4bpw:/workspace/models:ro' \
  -v "$ASSETS/qwen359b-exl3-4bpw-256k-q4-tabbyapi-config.yml":/app/config.yml:ro \
  --entrypoint /opt/venv/bin/python3 \
  ghcr.io/0xsero/tabbyapi-exl3@sha256:3d35e4979f5de7fd3b1621e4acd70035fa24ab78e941f4b569ab30fe1d1e7448 \
  main.py \
  --config \
  /app/config.yml

Measured speed

Concurrency	Context	Prefill	Decode	TTFT ms	Status	Sweep
1	262,144	—	96.9	376.6	accepted	qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1-acceptance

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count: 1
hardware id: rtx-4080-16gb
id: qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1
model instance id: turboderp-qwen3-5-9b-exl3--4-bpw
recipe source: 0xsero
recommended: Yes
schema version: local-ai-registry/v1
speed sweep ids: qwen359b-exl3-4bpw-rtx4080-tabbyapi-tp1-acceptance
status: validated

capabilities

chat: Yes
vision: No

engine

graph mode: piecewise
name: tabbyapi
version: 0.0.1+e632af41

serving

kv cache tokens: 262,144
max concurrency: 2
max context tokens: 262,144
tensor parallel: 1

Provenance & metadata (3)

facts

metadata

acceptance · accepted at: 2026-09-03T10:18:43Z
acceptance · apis: chat, messages, responses
acceptance · harness: validate_rented.py on vast RTX 4080 (weights and config materialized in-container in place of the bind mounts)
acceptance · served model id: Qwen3.5-9B-EXL3-4bpw
weights subdir: Qwen3.5-9B-EXL3-4bpw

provenance

captured at: 2026-09-02T22:33:52Z

sources

captured at	kind	url
2026-09-02T22:33:52Z	normalized-recipe	github.com/0xSero/local-ai-registry ↗