BBuf/sglang-ernie45-optimization
skills/model-optimization/sglang/sglang-ernie45-optimization/SKILL.md
PR-backed optimization manual for Ernie4.5 / Ernie4.5-VL in SGLang. Use when an engineer needs to audit, debug, extend, or document the SGLang Ernie4.5 multimodal runtime, especially the initial VL landing, fused Triton rotary path, and later cos/sin cache rewrite for Ernie4.5-VL.
$ install --global
skillsauthnpx skillsauth add BBuf/AI-Infra-Auto-Driven-SKILLS sglang-ernie45-optimizationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 2, 2026, 2:33 AM166.7s2 files scanned
SKILL.md
- name:
- sglang-ernie45-optimization
- description:
- PR-backed optimization manual for Ernie4.5 / Ernie4.5-VL in SGLang. Use when an engineer needs to audit, debug, extend, or document the SGLang Ernie4.5 multimodal runtime, especially the initial VL landing, fused Triton rotary path, and later cos/sin cache rewrite for Ernie4.5-VL.
SGLang Ernie4.5 / Ernie4.5-VL Optimization
Overview
Ernie4.5 is already supported on the checked SGLang mainline. The high-signal changes for this family are the initial VL bring-up, then two successive rotary embedding optimizations for the vision tower.
Evidence snapshot:
- SGLang mainline checked around
c122d343adb969cd9bbd1af2ca86727a11be3845 - sgl-cookbook checked around
e88b0fd8ac5b1caa6eb42766035029220053369b - Canonical PR notes:
references/pr-history.md - History mirrors:
model-pr-optimization-history/sglang/ernie45/README.zh.mdandREADME.en.md
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the bar.
For Ernie4.5-VL, name the exact rotary path being changed:
original rotary embedding, fused Triton rope, or cached cos/sin route.
Runtime Surfaces
sglang/python/sglang/srt/configs/model_config.pysglang/python/sglang/srt/models/ernie45_vl.pysglang/python/sglang/srt/models/ernie45_moe_vl.pysglang/python/sglang/srt/multimodal/processors/ernie45_vl.pysglang/python/sglang/srt/layers/rotary_embedding.py
Current Main Summary
#15679lands the Ernie4.5-VL and Ernie4.5-MoE-VL runtime plus the multimodal processor and model-config registration.#18856adds a fused Triton Q/K rope kernel tailored to Ernie4.5-VL's(h, w, t)layout.#19743rewrites the vision tower to useget_rope(...).get_cos_sin(...)instead of recomputing rotary embeddings for every call.
Key Landed PRs
- #15679
Add Ernie4.5 VL model support - #18856
Optimize Ernie4.5-VL rotary embedding with fused triton kernel - #19743
Support cos sin cache for Ernie4.5-VL
Validation Lanes
- Ernie4.5-VL startup and processor/token expansion.
- Vision-tower rotary correctness under both the fused kernel and cached cos/sin path.
- Long-input and multi-image / video cases that stress
grid_thwand rotary indexing.
References
references/pr-history.md: diff-reviewed Ernie4.5 cards.
Non-Negotiable Evidence Rule
Use skills/model-optimization/model-pr-diff-dossier/SKILL.md as the production bar.
Every PR cited for this family must be based on diff reading, not only PR titles.
Related Skills
BBuf/sglang-humanize-review
development
VerifiedTrustedCommunity
Perform SGLang code review in the style of human maintainers by consulting the full non-agent PR review episode corpus from project start through the latest refresh (June 2026), including inline review threads, top-level PR comments, review submissions, original multilingual text, and multi-round discussions. Use when reviewing SGLang PRs, diffs, patches, or local changes for correctness, tests, performance, GPU/runtime risks, API compatibility, and maintainability.
531SKILL.mdUpdated May 21, 2026
BBuf/model-pr-history-knowledge
documentation
VerifiedTrustedCommunity
Use when an SGLang, vLLM, or TensorRT-LLM serving/model optimization task needs prior model-family PR evidence. Query and read the PR-driven history docs under model-pr-optimization-history before choosing source paths, fast paths, kernel/fusion ideas, regression risks, or validation lanes.
531SKILL.mdUpdated May 17, 2026
BBuf/vllm-sota-humanize-loop
development
VerifiedTrustedCommunity
Run an autonomous Humanize-governed vLLM SOTA performance loop for one LLM model: first perform the fixed fair vLLM/SGLang/TensorRT-LLM deployment search and benchmark, then start one RLCR loop that repeatedly decides the gap, profiles the current bottleneck, runs layer/kernel pipeline analysis, patches vLLM code, optionally uses ncu-report-skill for kernel evidence, and revalidates until vLLM matches or beats the best observed framework under the same workload and SLA.
423SKILL.mdUpdated May 27, 2026
BBuf/llm-pipeline-analysis
devops
VerifiedTrustedCommunity
Inspect LLM torch profiler traces at forward-pass, layer, and kernel level. Use when you need layer timings, anchor-kernel boundaries, representative kernel flows, or Perfetto time ranges.
423SKILL.mdUpdated May 21, 2026