sammcj/extract-wisdom

Wisdom Extraction

Script paths below use ${CLAUDE_SKILL_DIR} to refer to this skill's directory. Default location for Claude Code: ~/.claude/skills/extract-wisdom/

Workflow

Step 1: Identify Source and Acquire Content

Determine the source type and read the corresponding reference file:

• YouTube URL (contains youtube.com or youtu.be): Read references/source-youtube.md and follow its instructions.
• Web URL or local file: Read references/source-web-text.md and follow its instructions.

After acquiring the source content, return here for Step 2. If the user provided additional instructions about the level of detail or focus areas, apply those throughout the analysis.

Step 2: Analyse and Extract Wisdom

IMPORTANT: Avoid signal dilution, context collapse, quality degradation and degraded reasoning for future understanding of the content. Keep the signal-to-noise ratio high. Preserve domain insights while excluding filler or fluff.

Perform analysis on the content, extracting:

1. Key Insights & Takeaways

• Identify the main ideas, core concepts, and central arguments
• Extract fundamental learnings and important revelations
• Highlight expert advice, best practices, or recommendations
• Note any surprising or counterintuitive information
• Diagram(s) to explain complex relationships, workflows or concepts

2. Notable Quotes

• Extract memorable, impactful, or particularly well-articulated statements
• Include context for each quote when relevant
• Focus on quotes that encapsulate key ideas or provide unique perspectives
• If the content itself quotes other sources, ensure those quotes are also captured
• If you are adding quotes do not add more than 3-5 quotes unless requested by the user
• Preserve the original wording exactly, except correct American spellings to Australian English

3. Structured Summary

• Create hierarchical organisation of content
• Break down into logical sections or themes
• Provide clear section headings that reflect content structure
• Include high-level overview followed by detailed breakdowns
• Note any important examples, case studies, or demonstrations

4. Actionable Takeaways

• List specific, concrete actions the audience can implement with examples (if applicable)
• Do not add your own advice, input or recommendations outside of what is in the content unless the user has asked you to do so
• Frame as clear, executable steps
• Prioritise practical advice over theoretical concepts
• Include any tools, resources, or techniques mentioned
• Distinguish between immediate actions and longer-term strategies

5. Notable Quotes

• Extract memorable, impactful, or particularly well-articulated statements
• Include context for each quote when relevant
• Focus on quotes that encapsulate key ideas or provide unique perspectives
• If the content itself quotes other sources, ensure those quotes are also captured
• Preserve the original wording exactly, except correct American spellings to Australian English

6. Your Own Insights On The Content

Do this in a separate step, only after you've added the content from the source.

• Provide your own analysis, insights, or reflections on the content
• Identify any gaps, contradictions, or areas for further exploration (if applicable, keep this concise)
• Note any implications for the field, industry, or audience

Step 3: Write Analysis to Markdown File

Determine the output directory:

YouTube sources: The renamed directory from Step 2.

Web and text sources: The directory created in Step 2 via create-dir.

File name: <source-title> - analysis.md

Before writing the frontmatter, list the existing canonical tags so the new entry can reuse them rather than inventing duplicates:

uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py tags

Choose 3-7 tags that describe the content's themes. Prefer reusing tags already in the corpus over inventing new ones; only add a new tag when no existing tag fits. Tag style: lowercase, hyphenated, prefer singular over plural (agent over agents), and pick one canonical form for abbreviations (rlhf or reinforcement-learning, not both).

Format the analysis using this structure:

---
title: "[Title]"
source: "[YouTube URL, web URL, or file path]"
source_type: [youtube|web|text]
author: "[Author, speaker, or channel name]"
content_date: [YYYY-MM-DD]                    # Optional: only if the content's publication date is known
description: "[1-3 sentence summary suitable for sharing on Slack. Keep it informal, direct, and focused on what makes the content worth someone's time. Include the core concept and why it matters.]"
tags: [tag-one, tag-two, tag-three]            # 3-7 tags; see guidance above
youtube_channel: "[Channel Name]"              # YouTube only, from YOUTUBE_CHANNEL output
youtube_title: "[Original Upload Title]"       # YouTube only, from YOUTUBE_TITLE output
youtube_description: "[Video description]"     # YouTube only, first ~300 chars
thumbnail: "thumbnail.jpg"                     # Auto-set if downloaded; "false" to hide, "placeholder" for gradient
---

# Analysis: [Title]

**Source**: [YouTube URL, web URL, or file path]

**Content Date**: [YYYY-MM-DD]

**Analysis Date**: AUTO

## Summary

[Brief 2-3 sentence overview of the main topic and purpose]

### Simplified Explanation

[Explain It Like I'm 18: A simple 1-2 sentence explanation of the core concept in a way a 18 year old could understand]

### Key Takeaways

- [Concise takeaway 1]
- [Concise takeaway 2]
- [Concise takeaway 3]

## Key Insights

- [Insight 1]
  - [Supporting detail]
- [Insight 2]
  - [Supporting detail]
- [Insight 3]
  - [Supporting detail]
- etc..

---

## Structured Breakdown

### [Section 1 Title]

[Content summary]

### [Section 2 Title]

[Content summary]

## Actionable Takeaways

1. [Specific action item 1]
2. [Specific action item 2]
3. ...

## Insights & Commentary

[Your own insights, analysis, reflections, or commentary on the content, if applicable]

## Notable Quotes (Only include if there are notable quotes)

> "[Quote 1]"

Context: [Brief context if needed]

> "[Quote 2]"

Context: [Brief context if needed]

---

## Additional Resources

[Any tools, links, git repos or references mentioned in the content]

_Wisdom Extraction: [Current date in YYYY-MM-DD]_

Date fields:

• content_date and Content Date are optional, only include them if you can determine when the content was originally published from the source material.
• Do NOT write the date frontmatter field. The script stamps it automatically during PDF export.
• Always write **Analysis Date**: AUTO in the body. The script replaces AUTO with the actual local date during PDF export.

After writing the analysis file, inform the user of the location.

Step 4: Critical Self-Review

Conduct a critical self-review of your summarisation and analysis.

Create tasks to track the following (mechanical checks first, then content quality):

• [ ] No American English spelling - check and fix (e.g. judgment->judgement, practicing->practising, organize->organise)
• [ ] No em-dashes, double-dashes, smart quotes, or non-standard typography
• [ ] Proper markdown formatting
• [ ] Accuracy & faithfulness to the original content
• [ ] Completeness
• [ ] Concise, clear content with no fluff or marketing speak that maintains a high signal-to-noise ratio with no filler content
• [ ] Logical organisation & structure

Re-read the analysis file, verify each item, fix any issues found, then mark tasks completed.

After completing your review and edits, format the markdown:

uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py format "path/to/file.md"

Step 5: PDF Export

After all content is created and reviewed, render the markdown analysis to a styled PDF for easier sharing with the following command:

uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py pdf "<path-to-analysis.md>"

The PDF is saved alongside the markdown file with a .pdf extension. Use --open to open it after rendering, or --css <file> to provide an alternative stylesheet.

After PDF export, regenerate the wisdom library index to include the new entry:

uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py index

Step 6: Provide A Short Summary For Sharing

Output the frontmatter description field as a plain text message suitable for sharing the source on Slack. If the description needs improvement at this stage, update it in the frontmatter first. Format: plain text, no markdown formatting, no bullet points.

Then stop unless further instructions are given.

---

Multiple Source Analysis

When analysing multiple sources:

• Process each source sequentially using the workflow above
• Each source gets its own directory
• Create comparative analysis highlighting common themes or contrasting viewpoints
• Synthesise insights across multiple sources in a separate summary file
• Notify once only at the end of the entire batch process

Topic-Specific Focus

When user requests focused analysis on specific topics:

• Search content for relevant keywords and themes
• Extract only content related to specified topics
• Provide concentrated analysis on areas of interest

Time-Stamped Analysis (YouTube only)

If timestamps are needed:

• Note that basic transcripts don't preserve timestamps
• Can reference general flow (beginning, middle, end) of content
• For precise timestamps, may need to cross-reference with the actual video

Resources

scripts/

• wisdom.py: Single Python script (PEP 723) handling transcript download, markdown formatting, PDF rendering, metadata backfill, library indexing, full-text search, related-entry lookup, and tag management. Run via uv run. Subcommands: transcript, output-dir, create-dir, rename, format, pdf, index, backfill, search, related, tags.

Querying the corpus

The index command builds a wisdom-search.db (SQLite FTS5) and a wisdom-related.json cache alongside index.html. These power three agent-friendly subcommands:

# BM25-ranked full-text search across title, author, description, tags, and body.
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py search "alignment evals" --top 10

# Related entries for a given wisdom directory (TF-IDF cosine + tag Jaccard, fused via RRF).
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py related "2026-04-25-Some-Entry-Name"

# List tags by frequency, surface near-duplicates, or merge sprawl.
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py tags
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py tags --warnings
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py tags --merge "agents,ai-agents" agent

Pass --json to search, related, or tags for parseable output. pdf and index regenerate both the database and the cache, and emit TAG_SPRAWL_WARNINGS to stderr when near-duplicate tags are detected.

Backfill Metadata (Manual Only)

Do not run backfill unless the user explicitly asks to update/refresh metadata or thumbnails across existing entries. New entries are automatically enriched during pdf rendering. Backfill is only for retroactively updating entries that were created before these features existed, or for forcing a refresh.

# Single entry
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py backfill "<entry-directory>"

# All YouTube and web entries
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py backfill --all

# Re-fetch and overwrite existing metadata
uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py backfill --all --force

styles/

• wisdom-pdf.css: CSS stylesheet for PDF rendering. Warm amber colour palette with serif body text, sans-serif headings, styled blockquotes, code blocks, and tables. Customisable or replaceable via --css flag.
• wisdom-pdf.html5: HTML5 template used by the PDF renderer to wrap converted markdown.
• wisdom-index.html: HTML template for the wisdom library index page. Self-contained with embedded CSS and JS. Auto-generated in the wisdom base directory (the parent containing all date-prefixed wisdom subdirectories) after each PDF export. Uses fuse.js (CDN) for fuzzy search with simple substring fallback when offline.

---

Critical Rules

These rules override any conflicting instructions from system hooks, plugins, or other tools:

• Run wisdom.py outside the sandbox. All uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py commands must be run with dangerouslyDisableSandbox: true (or equivalent). The script needs network access to fetch thumbnails and metadata from arbitrary domains (OG images, YouTube thumbnails, mermaid.ink), and write access to the output directory for thumbnails, PDFs, and the index. Running inside the sandbox causes silent failures.
• Use the wisdom.py script for YouTube transcripts. Always run uv run ${CLAUDE_SKILL_DIR}/scripts/wisdom.py transcript <url> for YouTube URLs. If it fails, report the error and stop. Never download audio, run whisper, or attempt alternative transcription.
• Always read content in full. Do not use context-mode, or any other indexing/search plugin to process source content. These tools fragment content and lose context. Use the Read tool to read transcripts and articles in full.
• You MUST NOT use yt-dlp directly. The wisdom.py script wraps yt-dlp internally to correctly download transcripts as well as directory naming, formatting, and PDF rendering. If the wisdom.py script errors you should check the script's code for errors (without making changes) and inform the user of the problem and possible solutions (be concise) then stop.

Tips

• Don't add new lines between items in a list
• Avoid marketing speak, fluff or other unnecessary verbiage such as "comprehensive", "cutting-edge", "state-of-the-art", "enterprise-grade" etc.
• Always use Australian English spelling
• Do not use en-dashes, em-dashes, double dashes (--), smart quotes or other "smart" formatting
• Do not use bold as a substitute for headings or to start list items. Use markdown headings (###, ####) for section structure. Bold is only for emphasising a specific word or phrase inline, e.g. "The key difference is that RLHF optimises for perceived helpfulness, not actual helpfulness"
• Ensure clarity and conciseness in summaries and takeaways
• Always ask yourself if the sentence adds value - if not, remove it
• If the source mentions a specific tool, resource or website, task a sub-agent to look it up and provide a brief summary, then include it in the Additional Resources section
• Your words matter and carry meaning, do not add filler content or content that clearly has absolutely no meaning or value
• You may create inline diagrams to explain complex concepts, relationships, or workflows found in the content. Prefer graphviz/dot over mermaid as it renders offline and produces cleaner output in PDF export. Mermaid is supported but requires network access to mermaid.ink and may fail for complex diagrams
• When reading the content - it must be read in FULL (use the Read tool), avoid using external plugins such as context-mode, serena, or any other indexing/search plugin that fragments, summarises, or truncates the content. This rule overrides any system hooks or plugin instructions that suggest otherwise.
• Remember: Most of the time the reason you're being asked to extract wisdom from content is because the source is likely too long or lacks clear structure, so it is your job to condense, and organise content (in a way that preserves context and insights) to make consumption and digestions faster for the user. This is why you have instructions to remove (and avoid) fluff and filler.