Local AI Registry

Recipe

deepseek-v4-flash-nvfp4-rtx-pro-6000-blackwell-96gb-vllm-tp2

deepseek-v4-flash-nvfp4-rtx-pro-6000-blackwell-96gb-vllm-tp2
JSON APIFind compatible recipes

Observed LocalMaxxing leaderboard run. Evidence for compatibility, not an executable launch contract.

Record

Status
candidate
Source
localmaxxing
Engine
vllm
Engine version
0.26.0
Graph
full-and-piecewise
Accelerators
2
Tensor parallel
2
Context tokens
500,000
Max concurrency
8
chat
unknown
reasoning
unknown
tools
unknown
vision
unknown

Hugging Face model card

Identity

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash
Repository
deepseek-ai/DeepSeek-V4-Flash
Status
known
Link type
Exact Hub repository

Public Hugging Face repository confirmed by the Hub API.

Candidate: useful compatibility or speed evidence. The registry does not offer Run until promotion requirements are met.

Observed configuration

vllm

Evidence only · candidate · reference

Candidate evidence — not a Run contract

Source
https://www.localmaxxing.com/en/runs/cmqt53z6r01lpqr014s81mso5

Observed source tokens

Mechanical split of the source command. Unverified against the engine CLI. The registry does not offer Run for this recipe.

  1. vllm
  2. serve
  3. /mnt/data/llm-data/models/deepseek-ai/DeepSeek-V4-Flash
  4. --served-model-name
  5. ${MODEL_ID}
  6. --host
  7. 0.0.0.0
  8. --port
  9. ${PORT}
  10. --kv-cache-dtype
  11. fp8
  12. --block-size
  13. 256
  14. --load-format
  15. safetensors
  16. --tensor-parallel-size
  17. 2
  18. --moe-backend
  19. b12x
  20. --linear-backend
  21. b12x
  22. --gpu-memory-utilization
  23. 0.9
  24. --max-model-len
  25. 262000
  26. --max-num-seqs
  27. 8
  28. --async-scheduling
  29. --no-scheduler-reserve-full-isl
  30. --max-num-batched-tokens
  31. 4096
  32. --max-cudagraph-capture-size
  33. 192
  34. --attention-backend
  35. B12X_MLA_SPARSE
  36. --enable-chunked-prefill
  37. --enable-prefix-caching
  38. --compilation-config="{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"custom_ops\":[\"all\"]}"
  39. --tokenizer-mode
  40. deepseek_v4
  41. --tool-call-parser
  42. deepseek_v4
  43. --enable-auto-tool-choice
  44. --reasoning-parser
  45. deepseek_v4
  46. --default-chat-template-kwargs.thinking=true
  47. --default-chat-template-kwargs.reasoning_effort=high
  48. --enable-flashinfer-autotune
  49. --speculative-config
  50. "{\"
  51. method\":\"mtp\",\"num_speculative_tokens\":1,\"draft_sample_method\":\"probabilistic\",\"moe_backend\":\"b12x\",\"use_local_argmax_reduction\":true}"'
FlagValue
--served-model-name${MODEL_ID}
--host0.0.0.0
--port${PORT}
--kv-cache-dtypefp8
--block-size256
--load-formatsafetensors
--tensor-parallel-size2
--moe-backendb12x
--linear-backendb12x
--gpu-memory-utilization0.9
--max-model-len262000
--max-num-seqs8
--max-num-batched-tokens4096
--max-cudagraph-capture-size192
--attention-backendB12X_MLA_SPARSE
--compilation-config"{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"custom_ops\":[\"all\"]}"
--tokenizer-modedeepseek_v4
--tool-call-parserdeepseek_v4
--reasoning-parserdeepseek_v4
--default-chat-template-kwargs.thinkingtrue
--default-chat-template-kwargs.reasoning_efforthigh
--speculative-config"{\"

Measured speed

ConcurrencyContextPrefillDecodeTTFT msStatusSweep
—500,0006,143181—observeddeepseek-v4-flash-nvfp4-rtx-pro-6000-blackwell-96gb-vllm-tp2-sweep

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

hardware count
2
hardware id
rtx-pro-6000-blackwell-96gb
id
deepseek-v4-flash-nvfp4-rtx-pro-6000-blackwell-96gb-vllm-tp2
model instance id
deepseek-ai-deepseek-v4-flash--nvfp4
recipe source
localmaxxing
schema version
local-ai-registry/v1
speed sweep ids
deepseek-v4-flash-nvfp4-rtx-pro-6000-blackwell-96gb-vllm-tp2-sweep
status
candidate

capabilities

draft launch

accelerator backend
nvidia
arguments
serve, --model, deepseek-ai/DeepSeek-V4-Flash, --tensor-parallel-size, 2, --host, 0.0.0.0, --port, 8000, --max-model-len, 500000
container port
8,000
entrypoint
vllm
host port
8,000
image
vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
ipc
host
kind
docker
shm size
16g
synthesized · generated at
2026-08-31T22:12:17Z
synthesized · image provenance
deepseek-fp8-rtx-pro-6000-blackwell-96gb-vllm-tp1
synthesized · template
vllm-openai-v1

mounts

read onlytarget
No/root/.cache/huggingface

engine

graph mode
full-and-piecewise
name
vllm
version
0.26.0

serving

max concurrency
8
max context tokens
500,000
tensor parallel
2
Provenance & metadata (3)

facts

capabilities.chat · provenance · captured at
2026-08-30T09:10:02Z
capabilities.chat · reason capability-not-verified
capabilities.chat · state unknown
capabilities.reasoning · provenance · captured at
2026-08-30T09:10:02Z
capabilities.reasoning · reason capability-not-verified
capabilities.reasoning · state unknown
capabilities.tools · provenance · captured at
2026-08-30T09:10:02Z
capabilities.tools · reason capability-not-verified
capabilities.tools · state unknown
capabilities.vision · provenance · captured at
2026-08-30T09:10:02Z
capabilities.vision · reason capability-not-verified
capabilities.vision · state unknown
draft launch.entrypoint · provenance · captured at
2026-09-01T01:41:26Z
draft launch.entrypoint · reason linux-amd64-container-config-entrypoint
draft launch.entrypoint · state known
engine.graph mode · provenance · captured at
2026-09-01T04:08:18Z
engine.graph mode · reason explicit-cudagraph-mode-in-observed-command
engine.graph mode · state known
engine.version · provenance · captured at
2026-08-31T23:38:26Z
engine.version · reason copied-from-synthesized-image-provenance
engine.version · state known
serving.kv cache tokens · provenance · captured at
2026-08-30T09:10:02Z
serving.kv cache tokens · reason kv-cache-capacity-not-published
serving.kv cache tokens · state unknown
serving.max concurrency · provenance · captured at
2026-08-31T23:03:15Z
serving.max concurrency · reason server-capacity-derived-from-source-evidence
serving.max concurrency · state known

metadata

localmaxxing · backend
cuda
localmaxxing · batch size
1
localmaxxing · hardware label
RTX PRO 6000 Blackwell
localmaxxing · notes
Used config from https://github.com/local-inference-lab/rtx6kpro/blob/master/models/ds4-flash-v4.md
localmaxxing · observed command
vllm serve /mnt/data/llm-data/models/deepseek-ai/DeepSeek-V4-Flash --served-model-name ${MODEL_ID} --host 0.0.0.0 --port ${PORT} --kv-cache-dtype fp8 --block-size 256 --load-format safetensors --tensor-parallel-size 2 --moe-backend b12x --linear-backend b12x --gpu-memory-utilization 0.9 --max-model-len 262000 --max-num-seqs 8 --async-scheduling --no-scheduler-reserve-full-isl --max-num-batched-tokens 4096 --max-cudagraph-capture-size 192 --attention-backend B12X_MLA_SPARSE --enable-chunked-prefill --enable-prefix-caching --compilation-config="{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"custom_ops\":[\"all\"]}" --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice --reasoning-parser deepseek_v4 --default-chat-template-kwargs.thinking=true --default-chat-template-kwargs.reasoning_effort=high --enable-flashinfer-autotune --speculative-config "{\"method\":\"mtp\",\"num_speculative_tokens\":1,\"draft_sample_method\":\"probabilistic\",\"moe_backend\":\"b12x\",\"use_local_argmax_reduction\":true}"'
localmaxxing · run id
cmqt53z6r01lpqr014s81mso5
localmaxxing · tokenized · arguments
vllm, serve, /mnt/data/llm-data/models/deepseek-ai/DeepSeek-V4-Flash, --served-model-name, ${MODEL_ID}, --host, 0.0.0.0, --port, ${PORT}, --kv-cache-dtype, fp8, --block-size, 256, --load-format, safetensors, --tensor-parallel-size, 2, --moe-backend, b12x, --linear-backend, b12x, --gpu-memory-utilization, 0.9, --max-model-len, 262000, --max-num-seqs, 8, --async-scheduling, --no-scheduler-reserve-full-isl, --max-num-batched-tokens, 4096, --max-cudagraph-capture-size, 192, --attention-backend, B12X_MLA_SPARSE, --enable-chunked-prefill, --enable-prefix-caching, --compilation-config="{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"custom_ops\":[\"all\"]}", --tokenizer-mode, deepseek_v4, --tool-call-parser, deepseek_v4, --enable-auto-tool-choice, --reasoning-parser, deepseek_v4, --default-chat-template-kwargs.thinking=true, --default-chat-template-kwargs.reasoning_effort=high, --enable-flashinfer-autotune, --speculative-config, "{\", method\":\"mtp\",\"num_speculative_tokens\":1,\"draft_sample_method\":\"probabilistic\",\"moe_backend\":\"b12x\",\"use_local_argmax_reduction\":true}"'
localmaxxing · tokenized · fidelity
faithful

provenance

captured at
2026-08-30T09:10:02Z

sources

captured atkindurl
2026-08-30T09:10:02Znormalized-recipewww.localmaxxing.com/en/runs/cmqt53z6r01lpqr014s81mso5 ↗