anton-abyzov/obsidian-brain

Obsidian Brain

Autonomous vault management skill implementing the Karpathy LLM Wiki pattern. Teaches the agent to manage an Obsidian vault as a folder of markdown files using three operations: ingest, query, and lint.

The vault has three layers:

1. PARA folders (user-owned) -- organized by actionability
2. Wiki (LLM-owned) -- synthesis layer with entities, concepts, sources, maps
3. Inbox (drop zone) -- new sources awaiting ingestion

This skill operates entirely via filesystem tools. No Obsidian app, CLI, or Sync dependency.

---

Configuration

All paths are user-configured. Replace placeholders before first use.

vault_path: "{{VAULT_PATH}}"              # Absolute path to Obsidian vault root
para_folders:
  projects: "{{PROJECTS_FOLDER}}"          # e.g. "001 Projects"
  areas: "{{AREAS_FOLDER}}"                # e.g. "002 Areas"
  resources: "{{RESOURCES_FOLDER}}"        # e.g. "003 Resources"
  archive: "{{ARCHIVE_FOLDER}}"            # e.g. "004 Archive"
wiki_dir: "{{WIKI_DIR}}"                   # e.g. "wiki"
inbox_dir: "{{INBOX_DIR}}"                 # e.g. "raw/inbox"
log_file: "{{LOG_FILE}}"                   # e.g. "wiki/log.md"
index_file: "{{INDEX_FILE}}"              # e.g. "wiki/index.md"
credentials_folder: "{{CREDENTIALS_FOLDER}}" # e.g. "003 Resources/credentials"

Resolved paths: All operations resolve relative paths against vault_path. For example, inbox_dir resolves to {{VAULT_PATH}}/{{INBOX_DIR}}/.

---

Operations

This skill supports three operations, each described below. For scheduled autonomous execution, see Scheduled Operation.

---

1. Ingest

Processes new files from the inbox into wiki pages and routes originals to PARA folders.

Trigger: Files present in {{VAULT_PATH}}/{{INBOX_DIR}}/.

Credential Guard (MANDATORY -- runs first)

Before processing any file, scan its content for sensitive material:

Patterns (case-insensitive):
  password\s*[:=]
  api[_-]?key\s*[:=]
  secret[_-]?key\s*[:=]
  token\s*[:=]
  aws_access_key_id\s*[:=]
  aws_secret_access_key\s*[:=]
  AKIA[0-9A-Z]{16}
  private[_-]?key\s*[:=]
  bearer\s+[a-zA-Z0-9\-._~+/]+=*

On credential match: STOP wiki ingestion for that file (never create a wiki page from credentials).

Credential value preservation (MANDATORY): When routing a credential file, the file's contents (key/secret/token/password values) MUST be preserved verbatim. Do NOT:

• Mask the value (e.g. nvapi-**********c-e, sk-...x4y)
• Replace with placeholders (actual secret NOT recorded, key-missing, *** redacted ***)
• Truncate or partially redact a value that's already complete in the source
• Wrap copy-pasteable values (API keys, client IDs, GUIDs, tokens, session cookies, passwords) in backticks, quotes, or any other Markdown decoration. Write them as plain text on their own line so the user can triple-click → copy → paste directly into a terminal or form without having to strip surrounding characters. Field labels, URLs, model identifiers, and config syntax can stay formatted as normal — but the value itself that the user needs to grab is always plain text. Aesthetic prettiness is not a goal in credential files; clean copy-paste is.

The {{CREDENTIALS_FOLDER}}/ tree IS the user's credential store: siloed, locally-synced, and only readable by the user's own machine. Masking inside it defeats retrieval — the user cannot use a key they cannot read. Never overwrite an existing full value with a masked one. If updating a stale file that has both a masked entry and a full value (legacy inconsistency), reconcile in favor of the full value and remove the masked artefact.

Frontmatter status: must reflect the actual state — active, expired, revoked, or superseded. Never key-missing if the value is in fact recorded in the file.

Exception: banking PANs / credit-card numbers follow industry standard (last-4 + issuer + expiry, never full PAN). Don't change those.

User-specific overrides (additional patterns, exceptions) live in references/user-preferences.md § Credentials, when present.

1. Discover vault structure — list subfolders across all PARA categories to understand what domains exist (see references/routing-rules.md Step 1)
2. Identify domain — analyze filename + content for domain signals (company names, project names, service types, technology keywords). Match against existing vault folders.
3. Route by domain context (see references/routing-rules.md Step 3 for the full decision tree):
   • Domain match → route to domain's folder (company, project, or area)
   • Service match → route to the service's tech subfolder in Resources
   • Personal account → route to typed subfolder in {{CREDENTIALS_FOLDER}}/ (Email-Accounts/, Social-Media/, Cloud-Providers/, etc.)
   • Unknown → {{CREDENTIALS_FOLDER}}/ root (last resort only)
4. Log: YYYY-MM-DD HH:MM | !cred | <filename> | credential detected, routed to <destination>
5. Never create a wiki page from credential content
6. Continue processing remaining inbox files

Ingest Procedure

Before processing any files, scan the vault structure to build a routing map:

• List subfolders in {{PROJECTS_FOLDER}}/, {{AREAS_FOLDER}}/, {{RESOURCES_FOLDER}}/, {{ARCHIVE_FOLDER}}/
• This tells you what domains exist, which are active vs archived, and prevents creating duplicate folders

For each non-credential file in the inbox:

1. Read the source file (never modify it in place)
2. Identify domain first — what domain does this content belong to? Match against existing vault folders by company name, project name, technology, service. Then determine PARA category (Project vs Area vs Resource vs Archive).
   • See references/routing-rules.md for the full domain recognition and routing table
3. Deduplication check — before creating a new wiki page, search for existing pages that cover the same entity or topic:
   • Grep {{WIKI_DIR}}/ for the entity name, key identifiers, and primary topics from the source
   • If an existing page covers the same entity (matching title or aliases in frontmatter), extend it instead of creating a duplicate:
     • Append new information under a dated section header (## Update YYYY-MM-DD)
     • Add any new cross-references
     • Update frontmatter tags and aliases if the source reveals new ones
   • If a related but distinct page exists, create the new page but add bidirectional cross-references
   • Log merges: YYYY-MM-DD HH:MM | ~merge | page-name | merged content from <source> (dedup)
4. Create wiki page in {{VAULT_PATH}}/{{WIKI_DIR}}/ (only if no existing page was extended in step 3):
   • Filename: slugified-name.md (lowercase, hyphens, max 60 chars)
   • Sources get date prefix: YYYY-MM-DD-slug.md
   • Frontmatter: see references/wiki-format.md
   • Add [[wikilinks]] to related existing pages
   • Cap at 1500 lines; if longer, split into parts with (part N) suffix and cross-references
5. Update related pages: add cross-references ([[new-page]]) to existing wiki pages that share topics
6. Update index: append the new page entry to {{VAULT_PATH}}/{{INDEX_FILE}}
   • Format: - [[page-name]] -- one-line summary
7. Log all mutations to {{VAULT_PATH}}/{{LOG_FILE}}:
   • New page: YYYY-MM-DD HH:MM | +page | page-name | created from <source>
   • Updated page: YYYY-MM-DD HH:MM | ~page | page-name | added cross-ref to <new-page>
   • Ingest event: YYYY-MM-DD HH:MM | >ingest | <source> | N pages created/updated
   • New link: YYYY-MM-DD HH:MM | @link | page-a -> page-b | bidirectional cross-ref
8. Route original file to the correct PARA folder per routing rules
   • Use Bash to move: the source file is read-only during ingest, then moved (not copied)

A single source should touch 5-15 wiki pages. The wiki compounds with every ingest.

---

2. Query

Synthesizes answers from across the vault, with source citations via wikilinks.

Trigger: User asks a question about vault contents.

Tools used: Read, Glob, Grep only. No Obsidian app or CLI dependency.

Query Procedure

1. Read index at {{VAULT_PATH}}/{{INDEX_FILE}} to identify candidate pages
2. Search for additional relevant pages using Grep across {{VAULT_PATH}}/{{WIKI_DIR}}/ and PARA folders
3. Read the most relevant pages (typically 3-10 depending on query scope)
4. Synthesize an answer that:
   • Cites sources with [[wikilinks]] (by filename, no folder path)
   • Distinguishes facts from inference
   • Notes contradictions between pages if any exist
5. Optionally file as wiki page: if the synthesis produces a novel comparison or insight:
   • Create {{VAULT_PATH}}/{{WIKI_DIR}}/synthesis-slug.md with frontmatter type: synthesis
   • Add cross-references to source pages
   • Update {{INDEX_FILE}} with the new synthesis page
   • Log: YYYY-MM-DD HH:MM | ?query | synthesis-slug | query: "<summary of question>"
6. Log the query even if no page is filed:
   • YYYY-MM-DD HH:MM | ?query | - | query: "<summary of question>", N pages consulted

Multi-Vault Query

When a query spans multiple knowledge domains, search across all configured vaults. Vault paths are defined in the user's global CLAUDE.md (typically personal-docs, engineering-docs, and ai-power vaults).

Cross-vault search procedure:

1. Start with the primary vault ({{VAULT_PATH}}) -- this is always the default
2. If the query involves technical/engineering topics, also search the engineering-docs vault
3. If the query involves AI/ML research, also search the ai-power vault
4. Prefix cross-vault citations with the vault name: [engineering-docs] [[page-name]]
5. Note in the synthesis when information was sourced from multiple vaults

Cross-vault search is opt-in by context -- only expand beyond the primary vault when the query clearly spans domains. Single-domain queries stay within the primary vault for speed.

---

3. Lint

Periodic health check that catches organizational issues before they compound.

Trigger: Manual request, or scheduled run when threshold/cadence is met.

Deterministic Checks (via script)

Run scripts/lint-check.sh with Bash:

bash scripts/lint-check.sh "{{VAULT_PATH}}" "{{WIKI_DIR}}" "{{INDEX_FILE}}" "{{INBOX_DIR}}" 10

The script checks:

• Orphan pages: wiki pages not listed in {{INDEX_FILE}}
• Inbox backlog: file count in {{INBOX_DIR}} vs threshold (default: 10)
• Missing cross-refs: wiki pages mentioned in [[wikilinks]] but not existing as files

Semantic Checks (via LLM analysis)

After running the script, perform LLM-driven analysis on flagged pages:

• Contradictions: read pairs of related pages and flag conflicting information
• Stale information: identify pages with outdated dates, deprecated references, or superseded content
• Missing concept pages: scan for recurring terms across pages that lack their own wiki entry

Lint Report

Categorize findings by severity:

| Severity | Meaning | Examples | |----------|---------|---------| | error | Broken structure, must fix | Orphan page, missing wikilink target | | warning | Quality issue, should fix | Inbox backlog > threshold, stale content | | info | Suggestion, nice to fix | Missing concept page, potential cross-ref |

Each finding includes an actionable fix suggestion (e.g., "Add [[page-name]] to index.md").

Logging

Log the lint run summary:

• YYYY-MM-DD HH:MM | !lint | - | E errors, W warnings, I info findings

Log individual error-severity findings:

• YYYY-MM-DD HH:MM | !lint | <page> | <finding description>

---

Scheduled Operation

Autonomous 4x/day execution via CronCreate. See references/cron-setup.md for full configuration.

CronCreate Setup

CronCreate({
  schedule: "0 8,12,16,20 * * *",
  prompt: "Run obsidian-brain scheduled maintenance on vault at {{VAULT_PATH}}",
  description: "Obsidian Brain: scheduled vault maintenance (ingest, lint, index)"
})

Adjust the cron expression to match your timezone and preferred schedule.

Pre-flight Checks (every scheduled run)

Before executing any operation, verify:

1. Vault accessible: test -d "{{VAULT_PATH}}/{{WIKI_DIR}}" -- abort with !error log if fails
2. Inbox count: ls "{{VAULT_PATH}}/{{INBOX_DIR}}/" | wc -l -- determines whether ingest runs
3. Last-run timestamp: read the last line of {{LOG_FILE}} -- skip if last run was < 2 hours ago to prevent duplicate processing

If pre-flight fails:

• YYYY-MM-DD HH:MM | !error | - | pre-flight failed: <reason>
• Abort the run (do not proceed to operations)

Run Priority

Each scheduled run executes operations in this order:

1. Ingest -- if inbox has files, run the full ingest procedure
2. Lint -- if inbox backlog exceeds threshold OR last lint was > 7 days ago
3. Index rebuild -- run scripts/update-index.sh to regenerate {{INDEX_FILE}}

Run Logging

Every scheduled run logs start and completion:

• Start: YYYY-MM-DD HH:MM | >ingest | - | scheduled run started
• End: YYYY-MM-DD HH:MM | >ingest | - | scheduled run complete: N ingested, L lint findings, index rebuilt
• Error: YYYY-MM-DD HH:MM | !error | - | <error description>

No silent failures -- every run produces at least a start + completion/error log entry.

---

Constraints

1. Never modify source files during ingest -- wiki is the synthesis, sources are the truth
2. Preserve non-ASCII filenames (including Cyrillic, CJK, etc.)
3. No credentials in wiki/ -- always route to the configured credentials folder
4. Wiki page size limit: 1500 lines max. Split with (part N) suffix if exceeded
5. Log every wiki mutation -- no write to wiki/ without a corresponding log entry
6. Index must be plain text -- links + one-line summaries, no Dataview queries
7. No git operations on the vault -- Obsidian Sync handles synchronization
8. No Obsidian app dependency -- all operations use filesystem tools only

---

Troubleshooting

Common issues and their resolutions, organized by operation.

Ingest Issues

| Symptom | Cause | Fix | |---------|-------|-----| | File stays in inbox after ingest | Move failed (permissions or path with special chars) | Check file permissions with ls -la; ensure target PARA folder exists; escape special characters in path | | Wiki page created but not in index | Index update step was skipped or errored | Run bash scripts/update-index.sh manually to rebuild the full index | | Duplicate wiki pages for same entity | Dedup check missed due to alias mismatch | Merge pages manually: keep the older page, append content from the newer one under ## Update YYYY-MM-DD, add aliases from both to frontmatter, delete the duplicate, update index | | Credential guard false positive | Content contains patterns like token: "explanation" in documentation | Review the file manually; if safe, temporarily rename the matching line, ingest, then restore. Never disable the credential guard entirely | | Source routed to wrong PARA folder | Domain signals were ambiguous or folder structure changed | Move the file to the correct folder with mv; update the log entry; re-scan vault structure on next ingest |

Query Issues

| Symptom | Cause | Fix | |---------|-------|-----| | Query returns no results | Index is stale or search terms don't match wiki page content | Rebuild index with scripts/update-index.sh; try broader search terms; check if content is in a different vault | | Cross-vault query misses pages | Secondary vault path not configured or inaccessible | Verify vault paths in global CLAUDE.md; check directory exists with test -d | | Synthesis contradicts source pages | LLM hallucination during synthesis | Always verify synthesis claims against the cited source pages; flag contradictions explicitly |

Lint Issues

| Symptom | Cause | Fix | |---------|-------|-----| | lint-check.sh reports false orphans | Page exists but uses a different slug format than index expects | Standardize the page filename to match the index entry, or update the index | | Lint finds hundreds of missing cross-refs | Wikilinks use display names instead of filenames | Use [[filename\|Display Name]] format; never use display-only wikilinks | | Scheduled lint never runs | CronCreate not configured or last-run check is too aggressive | Verify cron exists; check {{LOG_FILE}} last entry timestamp; reduce the 2-hour dedup window if needed |

General Issues

| Symptom | Cause | Fix | |---------|-------|-----| | Vault path not found | Drive not mounted, cloud sync incomplete, or path changed | Verify the path with test -d "{{VAULT_PATH}}"; check if Obsidian Sync is active; update config if vault moved | | Log file growing unbounded | No log rotation configured | Periodically archive old log entries: move lines older than 90 days to wiki/log-archive-YYYY.md | | Scripts fail with permission denied | Shell scripts lost execute bit after sync | Run chmod +x scripts/*.sh in the skill's scripts directory | | Non-ASCII filenames garbled | Terminal or filesystem encoding mismatch | Ensure locale supports UTF-8 (locale should show UTF-8); Obsidian vaults on macOS use NFD normalization |

---

Batch Operations

Bulk vault maintenance operations for reorganizing, auditing, or migrating content at scale.

Bulk Re-routing

Move all files from one PARA folder to another based on changed domain ownership:

Procedure:
1. Glob for all files in the source folder
2. For each file, read frontmatter to confirm domain
3. Move to target folder, preserving subfolder structure
4. Update any wiki pages that reference the old path
5. Log each move: YYYY-MM-DD HH:MM | >batch | filename | re-routed from <source> to <target>

Use case: A project completes and all its files should move from Projects to Archive.

Tag Normalization

Standardize inconsistent tags across the vault:

Procedure:
1. Grep all frontmatter `tags:` fields across wiki/ and PARA folders
2. Build a frequency map of all tags
3. Identify near-duplicates (e.g., "js" vs "javascript", "k8s" vs "kubernetes")
4. Present a normalization plan to the user for approval
5. Apply approved renames across all affected files
6. Log: YYYY-MM-DD HH:MM | >batch | - | normalized N tags across M files

Orphan Cleanup

Systematically resolve all orphan wiki pages:

Procedure:
1. Run lint-check.sh to get the orphan list
2. For each orphan, determine if it should be:
   a. Added to index (legitimate page, just missing from index)
   b. Merged into an existing page (duplicate content)
   c. Deleted (stale or empty page with no value)
3. Execute the chosen action for each orphan
4. Rebuild the index after all changes
5. Log: YYYY-MM-DD HH:MM | >batch | - | resolved N orphans (A indexed, B merged, C deleted)

Vault Statistics

Generate a health summary of the entire vault:

Procedure:
1. Count files per PARA category and wiki/
2. Count total wikilinks and calculate average links per page
3. Identify the 10 most-linked pages (hub pages)
4. Identify pages with zero inbound links (isolated pages)
5. Calculate inbox throughput (files ingested per week from log)
6. Report wiki growth rate (pages created per month from log)

Output as a wiki page: {{WIKI_DIR}}/vault-statistics-YYYY-MM.md with type: synthesis.

---

Reference Files

Detailed documentation for each subsystem (loaded on demand):

• references/vault-schema.md -- 3-layer architecture (PARA + wiki + inbox)
• references/routing-rules.md -- PARA routing decision table
• references/wiki-format.md -- frontmatter types, naming, linking conventions
• references/cron-setup.md -- CronCreate schedule + pre-flight configuration

Scripts

Shell scripts for deterministic operations (POSIX-compatible):

• scripts/detect-changes.sh -- inbox file count + recently modified files
• scripts/update-index.sh -- regenerate index from wiki page frontmatter
• scripts/lint-check.sh -- orphan detection, missing cross-refs, backlog count

Quick Reference

| Operation | Trigger | Frequency | |-----------|---------|-----------| | Ingest | New files in raw/inbox/ | 4x/day via CronCreate | | Query | User asks about vault content | On-demand | | Lint | Health check request | Weekly or on-demand |

Emergency ingest: If inbox has 50+ files, process in batches of 10 with a ~batch log tag.