latestaiagents/chunking-strategies
plugins/rag-architect/skills/chunking-strategies/SKILL.md
Expert guidance on document chunking strategies for RAG systems. Use this skill when designing how to split documents for vector embeddings. Activate when: chunking, chunk size, text splitting, document segmentation, overlap, semantic chunking, recursive splitting.
$ install --global
skillsauthnpx skillsauth add latestaiagents/agent-skills chunking-strategiesInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 24, 2026, 2:55 AM11.9s1 file scanned
SKILL.md
- name:
- chunking-strategies
- description:
- |
- Activate when:
- chunking, chunk size, text splitting, document segmentation, overlap, semantic chunking, recursive splitting.
Chunking Strategies for RAG
Optimize document splitting for retrieval accuracy and context preservation.
When to Use
- Designing a new RAG pipeline
- Retrieval quality is poor due to chunk boundaries
- Documents have mixed content types (code, tables, prose)
- Need to balance context window limits with retrieval precision
Chunking Methods
1. Fixed-Size Chunking
from langchain.text_splitter import CharacterTextSplitter
splitter = CharacterTextSplitter(
chunk_size=1000,
chunk_overlap=200,
separator="\n"
)
chunks = splitter.split_text(document)
Best for: Homogeneous content, quick prototyping Avoid when: Documents have natural boundaries (sections, paragraphs)
2. Recursive Character Splitting
from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=1000,
chunk_overlap=200,
separators=["\n\n", "\n", ".", " ", ""]
)
chunks = splitter.split_documents(docs)
Best for: General-purpose text, maintains paragraph integrity Hierarchy: Tries larger separators first, falls back to smaller
3. Semantic Chunking
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings
splitter = SemanticChunker(
embeddings=OpenAIEmbeddings(),
breakpoint_threshold_type="percentile",
breakpoint_threshold_amount=95
)
chunks = splitter.split_text(document)
Best for: When meaning matters more than size Trade-off: Slower, requires embedding calls
4. Document-Specific Chunking
Markdown
from langchain.text_splitter import MarkdownHeaderTextSplitter
headers = [
("#", "h1"),
("##", "h2"),
("###", "h3"),
]
splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers)
chunks = splitter.split_text(markdown_doc)
Code
from langchain.text_splitter import Language, RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter.from_language(
language=Language.PYTHON,
chunk_size=2000,
chunk_overlap=200
)
chunks = splitter.split_documents(code_docs)
HTML
from langchain.text_splitter import HTMLHeaderTextSplitter
splitter = HTMLHeaderTextSplitter(
headers_to_split_on=[("h1", "h1"), ("h2", "h2"), ("h3", "h3")]
)
chunks = splitter.split_text(html_doc)
Chunk Size Guidelines
| Content Type | Recommended Size | Overlap | |--------------|------------------|---------| | Dense technical docs | 500-1000 tokens | 10-20% | | Conversational/FAQ | 200-500 tokens | 5-10% | | Legal/contracts | 1000-1500 tokens | 15-20% | | Code | 1500-2000 tokens | 10-15% | | Mixed content | 800-1200 tokens | 15% |
Advanced: Parent-Child Chunking
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
# Small chunks for retrieval, large chunks for context
child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)
store = InMemoryStore()
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=store,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
)
Why: Small chunks = precise retrieval, large chunks = better context
Metadata Enrichment
Always attach metadata to chunks:
for i, chunk in enumerate(chunks):
chunk.metadata.update({
"source": doc.metadata["source"],
"chunk_index": i,
"total_chunks": len(chunks),
"doc_type": detect_doc_type(chunk.page_content),
"has_code": bool(re.search(r'```', chunk.page_content)),
"timestamp": datetime.now().isoformat()
})
Evaluation Checklist
- [ ] Chunks don't break mid-sentence
- [ ] Code blocks stay intact
- [ ] Tables aren't split across chunks
- [ ] Headers stay with their content
- [ ] Overlap preserves context continuity
- [ ] Metadata enables filtering
Best Practices
- Start with recursive splitting - works for 80% of cases
- Test retrieval quality - not just chunk count
- Use overlap - 10-20% prevents context loss at boundaries
- Match chunk size to model - consider embedding model's optimal input
- Preserve structure - use document-aware splitters when possible
Related Skills
latestaiagents/skill-testing
development
VerifiedTrustedCommunity
Test skills for correct activation, content quality, and regression — both automated checks (frontmatter validity, lint) and manual verification (query-suite activation testing). Covers CI integration and how to catch skill regressions before users do. Use this skill when adding skills to a repo, setting up CI for a skill library, or debugging "the skill exists but doesn't work". Activate when: test skills, validate skills, skill CI, skill linting, skill activation test, skill regression.
2SKILL.mdUpdated Apr 23, 2026
latestaiagents/skill-frontmatter
documentation
VerifiedTrustedCommunity
Write the YAML frontmatter for a SKILL.md file so it activates reliably — name, description, and activation keywords that the model matches against. Covers length, tone, and the most common frontmatter mistakes. Use this skill when authoring a new skill, fixing a skill that isn't auto-activating, or reviewing skills for publication. Activate when: SKILL.md frontmatter, skill description, skill activation, skill YAML, write a skill, author a skill.
2SKILL.mdUpdated Apr 23, 2026
latestaiagents/skill-activation-patterns
development
VerifiedTrustedCommunity
Design skills that fire at the right moment — neither over-eager (noise) nor under-eager (silent). Covers activation specificity, trigger phrases, disambiguation between overlapping skills, and debugging activation. Use this skill when multiple skills could fire on the same query, a skill never fires, or a skill fires too often. Activate when: skill won't activate, skill over-activates, overlapping skills, skill triggers, skill selection, skill disambiguation.
2SKILL.mdUpdated Apr 23, 2026
latestaiagents/progressive-disclosure
development
VerifiedTrustedCommunity
Structure SKILL.md content so the model reads just enough — concise summary up front, progressively deeper detail, examples on demand. Covers section ordering, length budgets, when to split into multiple skills. Use this skill when writing or refactoring a skill body, one skill has grown too long, or a skill is wordy but not useful. Activate when: SKILL.md structure, skill content, skill too long, split skill, progressive disclosure, skill body.
2SKILL.mdUpdated Apr 23, 2026