richfrem/memory-management

Memory Management

Tiered memory system that makes an AI agent a continuous collaborator across sessions.

Architecture

The memory system has six tiers, configurable per project:

Tier 1: HOT CACHE (always loaded at boot -- ~200 lines target)
+-- <primer_file>             Role, identity, constraints
+-- <boot_digest_file>        Tactical status, active tasks
+-- <boot_contract_file>      Immutable constraints
+-- <snapshot_file>           Cognitive Hologram (1 sentence per file)

Tier 2: RLM SUMMARY LEDGER (fast keyword lookup -- loaded on demand)
+-- <summary_cache_file>      Pre-generated text summaries: docs, protocols, research
                              Plugin: rlm-factory | Skill: rlm-search | Profile: project
+-- <tool_cache_file>         Pre-generated text summaries: plugins, skills, scripts
                              Plugin: rlm-factory | Skill: rlm-search | Profile: tools

Tier 3: VECTOR STORE (semantic embedding search -- loaded on demand)
+-- <vector_db_backend>       ChromaDB via vector-db plugin / vector-db-agent skill
                              Profile: knowledge | Port: configured in vector_profiles.json

Tier 4: DEEP STORAGE (filesystem -- authoritative source, loaded on demand)
+-- <domain_data_dir>/        Research topics: {topic}/analysis.md
+-- <design_docs_dir>/        ADRs, RFCs
+-- <governance_dir>/         Protocols, playbooks

Tier 5: VAULT (Obsidian -- linked knowledge graph, loaded on demand)
+-- <vault_dir>/              Plugin: obsidian-wiki-engine
                              Skills: obsidian-vault-crud, obsidian-canvas-architect,
                                      obsidian-graph-traversal, obsidian-bases-manager,
                                      obsidian-wiki-builder, obsidian-query-agent,
                                      obsidian-rlm-distiller, obsidian-wiki-linter
                              Env: VAULT_PATH or OBSIDIAN_VAULT_PATH

Tier 6: SOUL (external persistence -- optional, synced at session seal)
+-- <traces_file>             Plugin: project-specific (e.g. huggingface-utils)
                              e.g. lineage/, data/, soul_traces.jsonl on HF Hub

Projects define their own file paths for each slot. Tiers may be omitted or added based on project complexity.

Lookup Flow (3-Phase Search Protocol)

When searching for information, ALWAYS escalate in order. Never skip ahead.

Query arrives ->
1. HOT CACHE                     Instant. Boot files cover ~90% of context needs.
2. DEEP STORAGE (topic/decision) Load specific domain dir or design doc by subject.
3. RLM SUMMARY LEDGER (Phase 1)  Keyword search via rlm-factory:rlm-search skill.
4. VECTOR STORE (Phase 2)        Semantic search via vector-db:vector-db-search skill.
5. GREP / EXACT SEARCH (Phase 3) rg/grep scoped to paths from Steps 3 or 4.
6. Ask user                      Unknown? Learn it and persist it.

Phase 1 -- RLM Summary Scan (Table of Contents)

RLM is amortized prework: each file read ONCE, summarized ONCE, cached as plain text JSON. Searching summaries is O(1) keyword lookup -- no embeddings, no inference.

Trigger the rlm-factory:rlm-search skill, providing the profile and search term.

Use Phase 1 when: You need to understand what a file does, find which file owns a feature, or navigate the codebase without reading individual files.

Escalate to Phase 2 when: The summary is insufficient or no match found.

Phase 2 -- Vector Store Semantic Search (Back-of-Book Index)

Embedding-based nearest-neighbor search across all indexed chunks. Returns ranked parent chunks with RLM Super-RAG context pre-injected.

Trigger the vector-db:vector-db-search skill, providing the query, profile, and limit.

Use Phase 2 when: You need specific code snippets, patterns, or implementations.

Escalate to Phase 3 when: You have a file path (from Phase 1 or 2) and need an exact line.

Phase 3 -- Grep / Exact Search (Ctrl+F)

Precise keyword or regex match. Always scope to paths discovered in earlier phases.

# Scoped to a specific path (use paths from Phase 1/2)
grep_search "VectorDBOperations" ../../skills/

# Ripgrep for regex
rg "def query" ../../ --type py

Anti-patterns: Never run a full-repo grep without scoping. Never skip Phase 1.

Dependencies

rlm-factory -- RLM Summary Ledger (Tier 2)

| Component | Value | |:----------|:------| | Plugin | .agents/skills/ (rlm-curator, rlm-search, rlm-init, rlm-distill-agent) | | Skill (write) | skills/rlm-curator/ -- distill, inject, audit, cleanup | | Skill (read) | skills/rlm-search/ -- query the ledger | | Skill (Phase 1 search) | rlm-factory:rlm-search | | Skill (write/inject) | rlm-factory:rlm-curator | | Skill (audit coverage) | rlm-factory:rlm-curator | | Skill (shared config) | rlm-factory:rlm-curator | | Cache files | .agent/learning/rlm_summary_cache.json (docs), .agent/learning/rlm_tool_cache.json (tools) |

vector-db -- Vector Store (Tier 3)

| Component | Value | |:----------|:------| | Plugin | .agents/skills/ (vector-db-search, vector-db-ingest, vector-db-launch, vector-db-init) | | Skill | skills/vector-db-agent/ -- ingest, query, operations | | Skill (Phase 2 search) | vector-db:vector-db-search | | Skill (ingest files) | vector-db:vector-db-ingest | | Skill (operations) | vector-db:vector-db-search | | Skill (config) | vector-db:vector-db-search | | Backend | ChromaDB (chromadb.HttpClient with PersistentClient fallback) |

obsidian-wiki-engine -- Linked Vault + LLM Wiki (Tier 5)

| Component | Value | |:----------|:------| | Plugin | .agents/skills/ (obsidian-vault-crud, obsidian-init, obsidian-wiki-builder, obsidian-query-agent, obsidian-rlm-distiller, obsidian-wiki-linter, obsidian-canvas-architect, obsidian-graph-traversal) | | Skill: vault setup | obsidian-wiki-engine:obsidian-init -- prerequisites, .obsidian/ config, exclusion filters | | Skill: read/write notes | obsidian-wiki-engine:obsidian-vault-crud -- atomic create/read/update/append | | Skill: build wiki nodes | obsidian-wiki-engine:obsidian-wiki-builder -- Karpathy-style concept nodes | | Skill: query wiki | obsidian-wiki-engine:obsidian-query-agent -- 3-phase RLM+vector+grep search | | Skill: distill RLM | obsidian-wiki-engine:obsidian-rlm-distiller -- generates RLM summary layers | | Skill: semantic lint | obsidian-wiki-engine:obsidian-wiki-linter -- inconsistencies, gaps, stale articles | | Config | .agent/learning/rlm_wiki_raw_sources_manifest.json (sources), .agent/learning/vector_profiles.json (wiki profile) | | Requires | pip:ruamel.yaml (lossless YAML frontmatter), Obsidian Desktop | | Env | VAULT_PATH -- absolute path to the vault root |

Promotion / Demotion Rules

Promote to Hot Cache when:

• Knowledge is referenced in 3+ consecutive sessions
• It's critical for active work (current spec, active protocol)
• It's a constraint or identity anchor

Demote to Deep Storage when:

• Spec/feature is completed and merged
• Governing document is superseded by newer version
• Topic research is concluded
• Technical decision is ratified (move from draft to archive)

What Goes Where

| Type | Hot Cache | On-Demand Tier | |------|-----------|----------------| | Active tasks | Boot digest | -- | | Identity/role | Primer file | -- | | Constraints | Boot contract | -- | | Session state | Snapshot file | Tier 6 Soul (traces) | | Research topics | Summary in snapshot | Tier 4: domain_data_dir/{name}/ | | Design decisions | Referenced by ID | Tier 4: design_docs_dir/{id}_{name}.md | | Governing docs | Referenced by ID | Tier 4: governance_dir/{id}_{name}.md | | Plugins/scripts/tools | -- | Tier 2: RLM Summary Ledger (tool cache) | | Docs/protocols/research | -- | Tier 2: RLM Summary Ledger (summary cache) | | System docs | -- | Tier 2 RLM + Tier 3 Vector Store | | Linked notes, canvases | -- | Tier 5: Vault (Obsidian) | | External persistence | -- | Tier 6: Soul (HuggingFace or equivalent) |

Session Memory Workflow

At Session Start (Boot)

1. Load hot cache files in order (primer -> contract -> digest -> snapshot)
2. Integrity check validates snapshot is current
3. If snapshot stale -> flag for refresh at session end

During Session

• New learning -> Write to <domain_data_dir>/{topic}/
• New decision -> Create design document draft
• New tool -> Register in tool inventory
• Correction -> Update relevant file + note in disputes log if contradicting

At Session End (Seal)

1. Update snapshot file with new content learned this session
2. Seal validates no drift since last audit
3. Persist traces to external storage (if configured)

Conventions

• Hot cache target: ~200 lines total across all boot files
• Snapshot: 1 sentence per file, machine-readable
• Topic folders: lowercase-hyphens/
• Document numbering: 3-digit, sequential
• Always capture corrections and contradictions in a disputes log

Configuration

Projects configure the memory system by setting file paths in their project-specific plugin:

| Variable | Purpose | |:---------|:--------| | MEMORY_PRIMER_FILE | Path to cognitive primer / role definition | | MEMORY_BOOT_DIGEST | Path to tactical boot digest | | MEMORY_BOOT_CONTRACT | Path to immutable constraints | | MEMORY_SNAPSHOT_FILE | Path to learning snapshot (hologram) | | MEMORY_DOMAIN_DIR | Directory for domain research | | MEMORY_DESIGN_DIR | Directory for design docs (e.g. ADRs) | | MEMORY_GOVERNANCE_DIR | Directory for governing docs (e.g. Protocols) |

Architecture Diagrams

| Diagram | What It Shows | |:--------|:--------------| | memory_architecture.mmd | Full 4-tier memory system with exact plugin/skill/script names per tier | | memory_lookup_flow.mmd | 3-phase search sequence: Hot Cache -> RLM Ledger -> Vector Store -> Grep | | memory_session_lifecycle.mmd | Session Boot -> Active -> Seal lifecycle with all event types |