BBuf/sglang-qwen3-coder-optimization
skills/model-optimization/sglang/sglang-qwen3-coder-optimization/SKILL.md
PR-backed and current-main optimization manual for Qwen3-Coder and Qwen3-Coder-Next in SGLang. Use when an engineer needs to recover, extend, or audit Qwen3-Coder-480B-A35B, Qwen3-Coder-Next, tool-call parser behavior, incremental streaming tool arguments, NVFP4/FP8 loading, MoE fused configs, AMD/NPU/Blackwell recipes, or coding-agent deployment docs.
$ install --global
skillsauthnpx skillsauth add BBuf/AI-Infra-Auto-Driven-SKILLS sglang-qwen3-coder-optimizationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 2, 2026, 2:35 AM100.8s4 files scanned
SKILL.md
- name:
- sglang-qwen3-coder-optimization
- description:
- PR-backed and current-main optimization manual for Qwen3-Coder and Qwen3-Coder-Next in SGLang. Use when an engineer needs to recover, extend, or audit Qwen3-Coder-480B-A35B, Qwen3-Coder-Next, tool-call parser behavior, incremental streaming tool arguments, NVFP4/FP8 loading, MoE fused configs, AMD/NPU/Blackwell recipes, or coding-agent deployment docs.
SGLang Qwen3-Coder Optimization
Overview
Qwen3-Coder needs its own skill because its optimization surface spans both model execution and parser behavior. The parser is used by Qwen3.6 docs as well, so changes here can affect multiple Qwen deployments.
Current evidence snapshot:
- SGLang
origin/main:b3e6cf60aon2026-04-22 - sgl-cookbook
origin/main:816bad5on2026-04-21 - Parser file:
python/sglang/srt/function_call/qwen3_coder_detector.py - Docs/snippets:
docs_new/cookbook/autoregressive/Qwen/Qwen3-Coder.mdx,Qwen3-Coder-Next.mdx, and their deployment snippets - Adjacent runtime: Qwen3 MoE and Qwen3-Next model files, depending on checkpoint
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the production bar.
Every PR cited for this family must be based on diff reading, not only PR titles.
Before You Change Anything
Capture:
- checkpoint: Qwen3-Coder-480B-A35B or Qwen3-Coder-Next
- whether tool parser is enabled
- streaming or non-streaming tool-call mode
- quantization: BF16, FP8, NVFP4, or ModelOpt export
- hardware: H200/B200/GB200, AMD MI300/MI325/MI355, NPU
- whether MTP, EAGLE, or Qwen3-Next hybrid path is active
Core Principle
Separate parser correctness from model performance.
- Tool-call streaming bugs should be reproduced with small local responses before changing model runtime.
- Qwen3-Coder-Next uses the hybrid Qwen3-Next lane; model fixes should cite that skill when they are not parser-specific.
- Cookbook command changes need parser, quantization, and hardware coverage.
- Every cited PR must be handled as a diff dossier: read the full patch, write the motivation, key implementation idea, core code snippet, reviewed files, and validation impact. Do not add title-only PR lists.
Main Runtime Surfaces
python/sglang/srt/function_call/qwen3_coder_detector.pydocs_new/cookbook/autoregressive/Qwen/Qwen3-Coder.mdxdocs_new/cookbook/autoregressive/Qwen/Qwen3-Coder-Next.mdxdocs_new/src/snippets/autoregressive/qwen3-coder-deployment.jsxdocs_new/src/snippets/autoregressive/qwen3-coder-next-deployment.jsxpython/sglang/srt/models/qwen3_next.pypython/sglang/srt/models/qwen3_moe.py
Optimization Order
- Reproduce parser behavior with deterministic streaming chunks.
- Verify base model correctness without parser flags.
- Enable tool parser and test complete JSON plus incremental deltas.
- Add quantized loading tests for Coder/Next checkpoint.
- Tune MoE or GDN kernels in the corresponding Qwen3/Qwen3-Next lane.
- Update cookbook hardware recipes.
PR Dossier Rule
Use references/pr-history.md as the authoritative ledger. It includes manually reviewed merged PRs and manually reviewed open design PRs. If you cite another Qwen3-Coder PR, first open its metadata and full diff, then add a dossier entry before using it as evidence.
Open PRs that are already diff-reviewed in the ledger:
- #13411: schema-aware Qwen3-Coder parameter conversion.
- #13979: Qwen3-Coder-480B nightly performance tests.
- #21829: incremental streaming for tool-call arguments.
Validation Lanes
- Parser unit tests with multiple tools, nested JSON, and incremental streaming deltas.
- Model smoke test with
--tool-call-parser qwen3_coder. - Qwen3-Coder-Next quantized loading on target hardware.
- Cookbook command parity for NVIDIA and AMD recipes.
Related Skills
BBuf/sglang-humanize-review
development
VerifiedTrustedCommunity
Perform SGLang code review in the style of human maintainers by consulting the full non-agent PR review episode corpus from project start through the latest refresh (June 2026), including inline review threads, top-level PR comments, review submissions, original multilingual text, and multi-round discussions. Use when reviewing SGLang PRs, diffs, patches, or local changes for correctness, tests, performance, GPU/runtime risks, API compatibility, and maintainability.
531SKILL.mdUpdated May 21, 2026
BBuf/model-pr-history-knowledge
documentation
VerifiedTrustedCommunity
Use when an SGLang, vLLM, or TensorRT-LLM serving/model optimization task needs prior model-family PR evidence. Query and read the PR-driven history docs under model-pr-optimization-history before choosing source paths, fast paths, kernel/fusion ideas, regression risks, or validation lanes.
531SKILL.mdUpdated May 17, 2026
BBuf/vllm-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed vLLM SOTA performance loop for one LLM model: first perform the fixed fair vLLM/SGLang/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches vLLM code, optionally uses ncu-report-skill for kernel evidence, and revalidates until vLLM matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 27, 2026
BBuf/llm-pipeline-analysis
devops
VerifiedTrustedCommunity
Inspect LLM torch profiler traces at forward-pass, layer, and kernel level. Use when you need layer timings, anchor-kernel boundaries, representative kernel flows, or Perfetto time ranges.
423SKILL.mdUpdated May 21, 2026