gemma-4-12b-it-nvfp4-rtxpro6000-sglang-tp1

Launch

Exact materialization of this validated launch contract: digest-pinned image, pinned model revision, and the audited arguments. Self-contained — required assets are fetched from this registry and verified against their recorded sha256 before mounting. Also available as local-ai run gemma-4-12b-it-nvfp4-rtxpro6000-sglang-tp1.

1. Pulls the exact container image by sha256 digest — the bytes that were validated, not a floating tag.
2. Fetches any required engine assets from this registry and verifies each against its recorded sha256; the audited launch script verifies them again inside the container before use.
3. Downloads the pinned model revision into your Hugging Face cache on first run (reused afterwards).
4. Serves an OpenAI-compatible API on localhost:30000 — point any client at it.

ASSETS="${TMPDIR:-/tmp}/local-ai-assets" && mkdir -p "$ASSETS" && \
curl -fsSL 'https://local-ai-registry.vercel.app/api/v1/asset/gemma4-nvfp4-sglang-engine-patch/file' -o "$ASSETS/gemma4-nvfp4-sglang-engine-patch.diff" && \
{ printf '%s  %s\n' '39b94c941aa223ff96e6b9f93d4266bced8498be2e3d814207c770e54ad3af98' "$ASSETS/gemma4-nvfp4-sglang-engine-patch.diff" | sha256sum -c - 2>/dev/null || printf '%s  %s\n' '39b94c941aa223ff96e6b9f93d4266bced8498be2e3d814207c770e54ad3af98' "$ASSETS/gemma4-nvfp4-sglang-engine-patch.diff" | shasum -a 256 -c -; } && \
docker run --rm \
  --gpus all \
  --ipc host \
  --shm-size 16g \
  -p 30000:30000 \
  -e HF_HOME=/root/.cache/huggingface \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v "$ASSETS/gemma4-nvfp4-sglang-engine-patch.diff":/tmp/gemma4-engine-patch.diff:ro \
  --entrypoint /bin/bash \
  lmsysorg/sglang:dev-cu13@sha256:6cd4635214f279e0a43019f88e3120d407567640a58aa7dcc0085e3d91402cc4 \
  -lc \
  'set -euo pipefail; echo '\''dede8848dbcbfcfc6507da963a65d120add93101b6ff5fb54a03f19e86233d11  /tmp/gemma4-engine-patch.diff'\'' | sha256sum -c -; echo '\''d5498b253f35e83ab0aaf219a2c2bf2f42c6bbd3f95ffce02760e78b2e38a4e9  /sgl-workspace/sglang/python/sglang/srt/models/gemma4_unified.py'\'' | sha256sum -c -; (cd /sgl-workspace/sglang && patch -p0 < /tmp/gemma4-engine-patch.diff); echo '\''b0614b99a0d7fe654ed102fb5db04c578e3be6042896f73f9418965e6672c737  /sgl-workspace/sglang/python/sglang/srt/models/gemma4_unified.py'\'' | sha256sum -c -; exec /opt/sglang/bin/python -m sglang.launch_server --model-path unsloth/gemma-4-12b-it-NVFP4 --revision b1f649734b34aa5575b03d186abd1b9be3d0d5c4 --tp 1 --host 0.0.0.0 --port 30000 --context-length 131072 --mem-fraction-static 0.94 --attention-backend triton --kv-cache-dtype fp8_e4m3 --max-running-requests 4 --cuda-graph-max-bs-decode 4 --enable-cache-report --trust-remote-code --reasoning-parser gemma4 --tool-call-parser gemma4'

Measured speed