BBuf/vllm-deepseek-v4-optimization
skills/model-optimization/vllm/vllm-deepseek-v4-optimization/SKILL.md
PR-backed optimization manual for DeepSeek V4 in vLLM. Use when an engineer needs to audit, debug, extend, or document DeepSeek V4 current-main support in vLLM, including the model module, MTP path, tokenizer/renderer, DSML tool parser, expert-dtype handling, and BF16 persistent-topk follow-up.
$ install --global
skillsauthnpx skillsauth add BBuf/AI-Infra-Auto-Driven-SKILLS vllm-deepseek-v4-optimizationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 2, 2026, 2:38 AM159.9s2 files scanned
SKILL.md
- name:
- vllm-deepseek-v4-optimization
- description:
- PR-backed optimization manual for DeepSeek V4 in vLLM. Use when an engineer needs to audit, debug, extend, or document DeepSeek V4 current-main support in vLLM, including the model module, MTP path, tokenizer/renderer, DSML tool parser, expert-dtype handling, and BF16 persistent-topk follow-up.
vLLM DeepSeek V4 Optimization
Overview
This skill tracks DeepSeek V4 as a current-main vLLM model family. The old open bring-up PR was superseded by the rebased mainline landing, and the runtime now includes the registry aliases, model/MTP files, tokenizer/renderer, tool parser, and follow-up handling for FP4-vs-FP8 expert checkpoints.
Evidence snapshot:
- vLLM
origin/main:fd74c90d9on2026-04-27 - Support status: landed on current mainline after
#40860 - Diff-reviewed PRs:
#40760,#40860,#40806,#41006,#40811 - Canonical PR notes:
references/pr-history.md - History mirrors:
model-pr-optimization-history/vllm/deepseek-v4/README.zh.mdandREADME.en.md
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the bar.
DeepSeek V4 support claims must be tied to current-main files and diff-reviewed
PR cards, not only PR titles.
Runtime Surfaces
- Registry evidence:
vllm/vllm/model_executor/models/registry.py - Model path:
vllm/vllm/model_executor/models/deepseek_v4.py - Draft/MTP path:
vllm/vllm/model_executor/models/deepseek_v4_mtp.py - Attention/compressor paths:
vllm/vllm/model_executor/layers/deepseek_v4_attention.py,vllm/vllm/model_executor/layers/deepseek_compressor.py - Tokenizer/render paths:
vllm/vllm/tokenizers/deepseek_v4.py,vllm/vllm/renderers/deepseek_v4.py - Tool parser path:
vllm/vllm/tool_parsers/deepseekv4_tool_parser.py - Kernel follow-up in open PR:
vllm/csrc/persistent_topk.cuh,vllm/csrc/topk.cu
Current Main Summary
#40860merged the rebased DeepSeek V4 bring-up: model class, MTP model, MLA attention, tokenizer/renderer, registry aliases, and DSML parser.#40760is now a closed predecessor, useful only as historical context for the initial bring-up shape.#41006added expert-dtype-aware dispatch so FP4 expert checkpoints keep the MXFP4 path while Flash-Base / FP8 expert checkpoints use the FP8 MoE path and suffix mapping.#40806fixed DSML marker leakage for DSV3.2/DSV4 streaming parser paths.- Open PR
#40811extends persistent top-k from FP32-only assumptions to BF16 input support, which matters for the DeepSeek V4 sparse indexer path.
Open Optimization Items
- #40860
[Feat] DeepSeek V4 Rebased: mainline support landing. - #41006
Support DSV4 base: FP4/FP8 expert-dtype handling and mapper split. - #40806
[Bugfix] Fix the DSML token leakage in DSV4/3.2: merged parser leak fix. - #40811
[Perf][Kernel] BF16 input support for persistent topK - DeepSeekV4: still open follow-up. - #40760
[New Model] Support DeepseekV4: closed predecessor superseded by#40860.
Validation Lanes
- Re-check mainline
registry.pybefore claiming a new architecture alias. - Validate tokenizer/renderer parity, DSML tool calls, base generation, and MTP speculative decoding after edits to model, tokenizer, renderer, or parser code.
- For base / Flash-Base checkpoints, verify
expert_dtype="fp8"selects FP8 MoE scales and does not route through MXFP4. - If
#40811merges, rerun kernel tests intests/kernels/test_top_k_per_row.py, especially BF16 decode and long-context cases. - Rerun streaming tool-parser tests to ensure DSML sentinels never leak into user-visible content.
References
references/pr-history.md: diff-reviewed DeepSeek V4 cards.
Related Skills
BBuf/vllm-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed vLLM SOTA performance loop for one LLM model: first perform the fixed fair vLLM/SGLang/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches vLLM code, optionally uses ncu-report-skill for kernel evidence, and revalidates until vLLM matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 27, 2026
BBuf/llm-pipeline-analysis
devops
VerifiedTrustedCommunity
Inspect LLM torch profiler traces at forward-pass, layer, and kernel level. Use when you need layer timings, anchor-kernel boundaries, representative kernel flows, or Perfetto time ranges.
423SKILL.mdUpdated May 21, 2026
BBuf/sglang-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed SGLang SOTA performance loop for one LLM model: first perform the fixed fair SGLang/vLLM/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches SGLang code, optionally uses ncu-report-skill for kernel evidence, and revalidates until SGLang matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 17, 2026
BBuf/model-pr-history-knowledge
documentation
VerifiedTrustedCommunity
Use when an SGLang, vLLM, or TensorRT-LLM serving/model optimization task needs prior model-family PR evidence. Query and read the PR-driven history docs under model-pr-optimization-history before choosing source paths, fast paths, kernel/fusion ideas, regression risks, or validation lanes.
423SKILL.mdUpdated May 17, 2026