BBuf/sglang-qwen35-optimization
skills/model-optimization/sglang/sglang-qwen35-optimization/SKILL.md
PR-backed and current-main optimization manual for Qwen3.5 in SGLang. Use when an engineer needs to recover, extend, or audit Qwen3.5 dense/MoE, Qwen3.5 FP8/NVFP4/MXFP4, MTP, GDN projection, PP, EPLB, AMD/NPU/Blackwell deployments, FP8 KV caution paths, or Qwen3.5 cookbook recipes.
$ install --global
skillsauthnpx skillsauth add BBuf/AI-Infra-Auto-Driven-SKILLS sglang-qwen35-optimizationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 2, 2026, 2:35 AM55.1s3 files scanned
SKILL.md
- name:
- sglang-qwen35-optimization
- description:
- PR-backed and current-main optimization manual for Qwen3.5 in SGLang. Use when an engineer needs to recover, extend, or audit Qwen3.5 dense/MoE, Qwen3.5 FP8/NVFP4/MXFP4, MTP, GDN projection, PP, EPLB, AMD/NPU/Blackwell deployments, FP8 KV caution paths, or Qwen3.5 cookbook recipes.
SGLang Qwen3.5 Optimization
Overview
Qwen3.5 is a separate optimization lane because SGLang carries dedicated model files, MTP files, quantized checkpoint support, PP fixes, GDN projection fusion, and platform-specific tests.
Current evidence snapshot:
- SGLang
origin/main:b3e6cf60aon2026-04-22 - sgl-cookbook
origin/main:816bad5on2026-04-21 - Runtime files:
python/sglang/srt/models/qwen3_5.py,python/sglang/srt/models/qwen3_5_mtp.py - Config:
python/sglang/srt/configs/qwen3_5.py - Cookbook/docs:
docs/basic_usage/qwen3_5.md,docs_new/cookbook/autoregressive/Qwen/Qwen3.5.mdx
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the production bar.
Every PR cited for this family must be based on diff reading, not only PR titles.
Before You Change Anything
Capture:
- checkpoint: dense, MoE, FP8, NVFP4, MXFP4, or compressed-tensors variant
- whether MTP is enabled
- TP/EP/PP/DP shape and whether EPLB is active
- KV cache dtype and any FP8 KV caution note
- all-reduce backend, FlashInfer all-reduce fusion, and MoE backend
- hardware lane: H200, B200/GB200/GB300, AMD MI300/MI325/MI355, NPU, CPU/XPU
Core Principle
Qwen3.5 is a high-risk loader and topology model.
- Most regressions come from weight-loader mapping, PP layer splitting, tied embeddings, quantization guards, or MTP cache behavior.
- Performance work is concentrated around GDN projection fusion, shared-expert fusion, MoE configs, all-reduce, and backend-specific quant kernels.
- Cookbook recipes are active and should be kept in sync with SGLang docs when server defaults change.
Main Runtime Surfaces
python/sglang/srt/models/qwen3_5.pypython/sglang/srt/models/qwen3_5_mtp.pypython/sglang/srt/configs/qwen3_5.pytest/registered/4-gpu-models/test_qwen35_fp4_mtp_v2.pytest/registered/4-gpu-models/test_qwen35_fp4_triton.pytest/registered/8-gpu-models/test_qwen35.pytest/registered/gb300/test_qwen35_fp8.pytest/registered/gb300/test_qwen35_nvfp4.pydocs_new/src/snippets/autoregressive/qwen35-deployment.jsx
Optimization Order
- Prove BF16 or reference checkpoint correctness.
- Confirm weight-loader mapping, tied embeddings, and PP split rules.
- Validate quantized loading under the exact target exporter.
- Add MTP and EPLB compatibility tests.
- Tune GDN projections and shared experts.
- Update registered tests and cookbook configs for each hardware lane.
PR Dossier Rule
Before adding or updating any Qwen3.5 PR entry, open the PR diff/source and then write a complete card in references/pr-history.md:
- motivation/root cause
- key implementation idea
- concrete code snippet copied from the reviewed diff
- reviewed files
- validation implication
Do not add one-line open PR lists to this skill. If a PR is still open, keep it in a clearly marked open optimization section only after reading its diff, as done for #23474.
Validation Lanes
- 4-GPU FP4/MTP and Triton tests.
- 8-GPU Qwen3.5 test.
- GB300 FP8 and NVFP4 tests.
- AMD MI300/MI325/MI355 accuracy and perf tests.
- Cookbook command generation for H200/B200/AMD.
Related Skills
BBuf/sglang-humanize-review
development
VerifiedTrustedCommunity
Perform SGLang code review in the style of human maintainers by consulting the full non-agent PR review episode corpus from project start through the latest refresh (June 2026), including inline review threads, top-level PR comments, review submissions, original multilingual text, and multi-round discussions. Use when reviewing SGLang PRs, diffs, patches, or local changes for correctness, tests, performance, GPU/runtime risks, API compatibility, and maintainability.
531SKILL.mdUpdated May 21, 2026
BBuf/model-pr-history-knowledge
documentation
VerifiedTrustedCommunity
Use when an SGLang, vLLM, or TensorRT-LLM serving/model optimization task needs prior model-family PR evidence. Query and read the PR-driven history docs under model-pr-optimization-history before choosing source paths, fast paths, kernel/fusion ideas, regression risks, or validation lanes.
531SKILL.mdUpdated May 17, 2026
BBuf/vllm-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed vLLM SOTA performance loop for one LLM model: first perform the fixed fair vLLM/SGLang/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches vLLM code, optionally uses ncu-report-skill for kernel evidence, and revalidates until vLLM matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 27, 2026
BBuf/llm-pipeline-analysis
devops
VerifiedTrustedCommunity
Inspect LLM torch profiler traces at forward-pass, layer, and kernel level. Use when you need layer timings, anchor-kernel boundaries, representative kernel flows, or Perfetto time ranges.
423SKILL.mdUpdated May 21, 2026