BBuf/sglang-glm5-glm51-optimization
skills/model-optimization/sglang/sglang-glm5-glm51-optimization/SKILL.md
PR-backed and current-main optimization manual for GLM-5 and GLM-5.1 in SGLang. Use when an engineer needs to recover, extend, or audit GLM-5 DSA/NSA/NSA indexer paths, GLM-5.1 FP8/MXFP4/NVFP4, NextN/MTP, dense-attention threshold, NSA TileLang/AITER, tool templates, EAGLE, PCG, AMD/Blackwell/NPU validation, or GLM-5 cookbook recipes.
$ install --global
skillsauthnpx skillsauth add BBuf/AI-Infra-Auto-Driven-SKILLS sglang-glm5-glm51-optimizationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 2, 2026, 2:33 AM103.3s4 files scanned
SKILL.md
- name:
- sglang-glm5-glm51-optimization
- description:
- PR-backed and current-main optimization manual for GLM-5 and GLM-5.1 in SGLang. Use when an engineer needs to recover, extend, or audit GLM-5 DSA/NSA/NSA indexer paths, GLM-5.1 FP8/MXFP4/NVFP4, NextN/MTP, dense-attention threshold, NSA TileLang/AITER, tool templates, EAGLE, PCG, AMD/Blackwell/NPU validation, or GLM-5 cookbook recipes.
SGLang GLM-5/5.1 Optimization
Overview
GLM-5/5.1 is a separate lane because it moves from GLM4 MoE into GLM MoE DSA/NSA-adjacent behavior, NextN/MTP, FP8/MXFP4/NVFP4, NSA indexer work, and GLM-5.1 tool-template details.
Current evidence snapshot:
- SGLang
origin/main:bca3dd958on2026-04-24 - sgl-cookbook
origin/main:816bad5on2026-04-21 - Runtime files:
glm4_moe.py,glm4_moe_nextn.py,deepseek_nextn.pywhen shared MTP infrastructure is touched - Docs/snippets:
GLM-5.mdx,GLM-5.1.mdx,glm-5-deployment.jsx,glm-51-deployment.jsx - Tests:
test/registered/8-gpu-models/test_glm_51_fp8.py,test/registered/gb300/test_glm5_fp8.py,test/registered/gb300/test_glm5_nvfp4.py
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the production bar.
Every PR cited for this family must be based on diff reading, not only PR titles.
Before You Change Anything
Capture:
- checkpoint: GLM-5, GLM-5-FP8, GLM-5-MXFP4, GLM-5.1-FP8, GLM-5.1-MXFP4/NVFP4
- whether DSA/NSA or dense-attention fallback is active
- MTP/NextN enabled or disabled
- KV cache dtype, NSA backend, and dense-attention threshold envs
- quantization exporter: FP8, MXFP4, NVFP4, Quark/ModelOpt/compressed-tensors
- EAGLE, PCG, DP attention, PD, or CUDA graph enabled
- backend: AMD MI300/MI325/MI355, B200/GB300, NPU
Core Principle
GLM-5/5.1 should be debugged like a GLM-specific DSA/MTP system, not like plain GLM4.
- NSA indexer and dense-attention threshold changes may be shared with DeepSeek V3.2.
- MTP/NextN fixes can touch
deepseek_nextn.pyeven when the user-facing model is GLM. - GLM-5.1 template/tool-result behavior is user-visible and must be tested separately.
Main Runtime Surfaces
python/sglang/srt/models/glm4_moe.pypython/sglang/srt/models/glm4_moe_nextn.pypython/sglang/srt/models/deepseek_nextn.pypython/sglang/srt/layers/attention/nsa/docs_new/cookbook/autoregressive/GLM/GLM-5.mdxdocs_new/cookbook/autoregressive/GLM/GLM-5.1.mdxdocs_new/src/snippets/autoregressive/glm-5-deployment.jsxdocs_new/src/snippets/autoregressive/glm-51-deployment.jsx
Optimization Order
- Verify BF16/FP8 baseline with dense-attention threshold recorded.
- Validate NSA indexer and backend selection.
- Validate quantized loading and KV path.
- Enable MTP/NextN and verify draft quant config.
- Validate tool template and tool-result normalization.
- Tune NSA/AMD/Blackwell kernels.
- Add PCG/EAGLE only after base DSA/MTP is stable.
PR Dossier Rule
Before adding GLM-5/5.1 PR evidence, open the PR diff/source and write a full card in references/pr-history.md with motivation, key implementation, code excerpt, reviewed files, and validation implications.
Do not add one-line open PR lists. Open PRs can be recorded only after their diffs are reviewed and they are clearly separated from merged history.
Validation Lanes
- GLM-5 FP8/NVFP4 GB300 tests.
- GLM-5.1 FP8 8-GPU test.
- AMD GLM-5/5.1 FP8/MXFP4 accuracy and perf.
- MTP/NextN with draft quant config.
- Tool-template and tool-result normalization.
Related Skills
BBuf/sglang-humanize-review
development
VerifiedTrustedCommunity
Perform SGLang code review in the style of human maintainers by consulting the full non-agent PR review episode corpus from project start through the latest refresh (June 2026), including inline review threads, top-level PR comments, review submissions, original multilingual text, and multi-round discussions. Use when reviewing SGLang PRs, diffs, patches, or local changes for correctness, tests, performance, GPU/runtime risks, API compatibility, and maintainability.
531SKILL.mdUpdated May 21, 2026
BBuf/model-pr-history-knowledge
documentation
VerifiedTrustedCommunity
Use when an SGLang, vLLM, or TensorRT-LLM serving/model optimization task needs prior model-family PR evidence. Query and read the PR-driven history docs under model-pr-optimization-history before choosing source paths, fast paths, kernel/fusion ideas, regression risks, or validation lanes.
531SKILL.mdUpdated May 17, 2026
BBuf/vllm-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed vLLM SOTA performance loop for one LLM model: first perform the fixed fair vLLM/SGLang/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches vLLM code, optionally uses ncu-report-skill for kernel evidence, and revalidates until vLLM matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 27, 2026
BBuf/llm-pipeline-analysis
devops
VerifiedTrustedCommunity
Inspect LLM torch profiler traces at forward-pass, layer, and kernel level. Use when you need layer timings, anchor-kernel boundaries, representative kernel flows, or Perfetto time ranges.
423SKILL.mdUpdated May 21, 2026