{"data":{"capabilities":{"chat":true,"reasoning":true,"tools":true,"vision":false},"description":"DeepSeek-V4-Flash-0731 (vendor fp8 dense + fp4 experts, served stock) on two RTX PRO 6000 Blackwell Max-Q (sm_120) via the upstream SGLang dspark image, TP-2, ctx 131,072, thinking mode. No patches. Weights 74.9 GB per rank; MLA KV pool 1,722,368 tokens. The one trap: SGLang's DeepSeek-V4 template defaults to CHAT mode (zero reasoning tokens) — send chat_template_kwargs {\"thinking\": true} per request to match the vendor API's posture; enable_thinking is ignored and reasoning_effort only adds a prompt prefix.","engine":{"graph_mode":"full decode capture bs<=4; piecewise CUDA graph disabled","name":"sglang","version":"lmsysorg/sglang:dev-cu13-inkling-dspark@sha256:fbea1a4e25b26660dbc2384a27ead8817e9b7670f257b5c3143e0450d14524d7 (stock; SGLang commit b7252cc6b0c78b25ecea7ee5efa91a6ae37d0f19); no patches"},"facts":{},"hardware_count":2,"hardware_id":"rtx-pro-6000-blackwell-96gb","id":"deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2","launch":{"accelerator_backend":"nvidia","arguments":["python3","-m","sglang.launch_server","--model-path","deepseek-ai/DeepSeek-V4-Flash-0731","--served-model-name","deepseek-v4-flash","--tp-size=2","--mem-fraction-static=0.93","--context-length=131072","--max-running-requests=4","--chunked-prefill-size=8192","--attention-backend=triton","--reasoning-parser=deepseek-v4","--tool-call-parser=deepseekv4","--disable-piecewise-cuda-graph","--cuda-graph-max-bs=4","--disable-custom-all-reduce","--trust-remote-code","--host","0.0.0.0","--port","8011"],"container":{"captured_at":"2026-08-27T06:04:13.773Z","compose_file":null,"digest":"sha256:fbea1a4e25b26660dbc2384a27ead8817e9b7670f257b5c3143e0450d14524d7","image":"lmsysorg/sglang:dev-cu13-inkling-dspark@sha256:fbea1a4e25b26660dbc2384a27ead8817e9b7670f257b5c3143e0450d14524d7","reason":"image-reference-in-launch","runtime":"docker","source":[{"captured_at":"2026-08-27T06:04:13.773Z","kind":"recipe-launch","url":"https://github.com/ppickle1989/sm120-sglang-recipes"}],"state":"digest-pinned"},"container_port":8011,"environment":{"CUDA_DEVICE_ORDER":"PCI_BUS_ID","CUDA_VISIBLE_DEVICES":"0,1","HF_HOME":"/data/huggingface","HF_HUB_OFFLINE":"1","NCCL_IB_DISABLE":"1","NCCL_P2P_DISABLE":"1","NCCL_P2P_LEVEL":"SYS","NCCL_PROTO":"LL,LL128,Simple","NVIDIA_VISIBLE_DEVICES":"${GPUS}","PYTORCH_CUDA_ALLOC_CONF":"expandable_segments:True","TORCH_CUDA_ARCH_LIST":"12.0a"},"host_port":8011,"image":"lmsysorg/sglang:dev-cu13-inkling-dspark@sha256:fbea1a4e25b26660dbc2384a27ead8817e9b7670f257b5c3143e0450d14524d7","ipc":"host","kind":"docker","launch_command":"deepseek-v4-flash-0731/run.sh (docker run, foreground)","mounts":[{"read_only":true,"source":"${HF_HOME}","target":"/data/huggingface"},{"read_only":false,"source":"dsv4-sgl-cache (named volume)","target":"/root/.cache"}],"shm_size":"32g","source":{"commit":"769f4704362af24e355f4117b60dd6f544ea0580","paths":["deepseek-v4-flash-0731/run.sh","manifest.json","benchmarks/deepseek-v4-flash-0731-tp2-tps-20260826.json"],"repository":"https://github.com/ppickle1989/sm120-sglang-recipes"}},"metadata":{"acceptance":{"completion":true,"tools":true,"vision":false},"context_note":"a request that produced a 128,000-token completion (finish=length) was served at the configured 131,072 context, i.e. the full window is usable; requests exceeding it return HTTP 400","hardware_variant":"RTX PRO 6000 Blackwell Max-Q Workstation Edition (300 W)","interconnect":"PCIe 5.0, x8 + x4 as reported by nvidia-smi under load (AM5 consumer lane split); host-mediated multi-GPU, no NVLink; NCCL_P2P_DISABLE=1 required on this board","traces":[{"date":"2026-08-25","detail":"Upstream dspark image serves the model with no patches: weights load 46 s, 74.92 GB/rank; health in ~2 min when kernels are cached, up to ~10 min on a cold first start.","event":"startup_success_stock"},{"date":"2026-08-25","detail":"First probe launched without --reasoning-parser/--tool-call-parser: tool calls could not parse. Relaunched with --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 (names from the in-image ReasoningParser.DetectorMap / ToolCallParserEnum); positive control then PASS incl. parsed tool call.","event":"gotcha_parser_flags"},{"date":"2026-08-25","detail":"SGLang dsv4 defaults to chat mode: 0 reasoning tokens unless chat_template_kwargs {\"thinking\": true} is sent per request (enable_thinking is ignored; reasoning_effort only adds a prompt prefix). Vendor API serves in thinking mode, so parity needs the kwarg.","event":"gotcha_chat_mode_default"},{"date":"2026-08-25","detail":"21-prompt parity check against the vendor API with thinking:true: 21/21 locally = 21/21 on the vendor API.","event":"canary_parity"},{"date":"2026-08-26T12:54-05:00","detail":"ctx 65536 -> 131072 (mem-fraction-static 0.93): max_total_num_tokens=1,722,368, available_gpu_mem 6.0 GB after pools. Long-generation probes (primes to 4000, 600-digit binary, 40x40 table, 800 squares at a 32k budget) all 0 errors; a 12k budget was exhausted inside reasoning on the enumeration task (budget, not drift).","event":"context_raise"},{"date":"2026-08-26T12:56-05:00","detail":"Idle-server tok/s bench (thinking:true): 56-58 tok/s effective on every prompt class, decode 57.8, TTFT 0.08 s; 4 concurrent 155.2 tok/s aggregate (46.5-48.2/stream, TTFT 0.35-0.42 s). Flat across output length.","event":"benchmark"},{"date":"2026-08-26","detail":"Sustained load window 2026-08-26 (17 h, GPUs >= 50 % busy for 16.7 h, longest contiguous span 9.6 h): GPU peaks 88 C / 277-283 W (300 W limit), CPU package 85 C, VRM 57 C, NVMe 59 C; no throttling.","event":"thermal"}]},"model_instance_id":"deepseek-ai-deepseek-v4-flash-0731--fp8","provenance":{"captured_at":"2026-08-27T06:04:13.773Z","sources":[{"captured_at":"2026-08-27T06:04:13.773Z","kind":"normalized-recipe","url":"https://github.com/ppickle1989/sm120-sglang-recipes"}]},"recipe_source":"ppickle1989","schema_version":"local-ai-registry/v1","serving":{"api":"openai/v1","kv_cache_dtype":"fp8_e4m3 (auto)","kv_cache_tokens":1722368,"max_concurrency":4,"max_context_tokens":131072,"tensor_parallel":2},"speed_sweep_ids":["deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep"],"status":"candidate","huggingface":{"link_type":"repository","provenance":{"captured_at":"2026-08-27T06:04:13.773Z","sources":[{"captured_at":"2026-08-27T06:04:13.773Z","kind":"huggingface-api","url":"https://huggingface.co/api/models/deepseek-ai/DeepSeek-V4-Flash-0731"}]},"reason":"hf-api-confirmed-public","repository":"deepseek-ai/DeepSeek-V4-Flash-0731","status":"known","url":"https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731"},"registry":{"launchable":false,"speed_evidence":{"available":true,"count":1,"speed_sweep_ids":["deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep"],"detail_urls":["/api/v1/speed-sweep/deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep"]},"runtime":"docker"},"relationships":{"hardware":{"api":"/api/v1/hardware/rtx-pro-6000-blackwell-96gb","href":"/hardware/rtx-pro-6000-blackwell-96gb","id":"rtx-pro-6000-blackwell-96gb","name":"RTX PRO 6000 Blackwell"},"model":{"api":"/api/v1/models/deepseek-v4-flash-0731","href":"/models/deepseek-v4-flash-0731","id":"deepseek-v4-flash-0731","name":"DeepSeek-V4-Flash-0731"},"model_instance":{"api":"/api/v1/model-instances/deepseek-ai-deepseek-v4-flash-0731--fp8","href":"/model-instances/deepseek-ai-deepseek-v4-flash-0731--fp8","id":"deepseek-ai-deepseek-v4-flash-0731--fp8","name":"deepseek-ai/DeepSeek-V4-Flash-0731"},"speed_sweep":[{"api":"/api/v1/speed-sweep/deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep","href":"/speed-sweep/deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep","id":"deepseek-v4-flash-0731-fp8-fp4-rtxpro6000-sglang-tp2-sweep"}]}},"meta":{"source":"registry"}}