a-organvm/source-evaluation-bibliography

Source Evaluation & Bibliography

Evaluate, document, and maintain curated collections of references and sources.

Source Evaluation Framework

The CRAAP Test

| Criterion | Questions | Weight | |-----------|-----------|--------| | Currency | When was it published/updated? Is the information still valid? | Medium | | Relevance | Does it address the specific question? Who is the intended audience? | High | | Authority | Who is the author? What credentials/affiliations? | High | | Accuracy | Is it evidence-based? Can claims be verified? Peer-reviewed? | Critical | | Purpose | Is it informative, persuasive, or commercial? Any bias? | Medium |

Source Tier Classification

| Tier | Source Type | Reliability | Example | |------|-----------|-------------|---------| | S | Primary research, official specs | Highest | Peer-reviewed papers, RFC documents, official API docs | | A | Authoritative secondary | High | Textbooks, reputable technical blogs, conference proceedings | | B | Community knowledge | Moderate | Stack Overflow answers (high-vote), well-maintained wikis | | C | Informal | Variable | Blog posts, tutorials, forum discussions | | D | Unverified | Low | Social media, anonymous posts, undated content |

Evaluation Record

source:
  title: "Designing Data-Intensive Applications"
  author: "Martin Kleppmann"
  type: book
  year: 2017
  url: "https://dataintensive.net/"
  tier: A
  evaluation:
    currency: Good (concepts still current despite 2017 publication)
    relevance: High (directly addresses distributed systems patterns)
    authority: Strong (researcher at Cambridge, industry experience)
    accuracy: Peer-reviewed content, extensively cited
    purpose: Educational, no commercial bias
  notes: "Definitive reference for distributed data systems. Chapter 5-9 most relevant."
  tags: [distributed-systems, databases, architecture]
  last_verified: 2026-03-20

Annotated Bibliography Format

Entry Structure

## {Author Last}, {First}. "{Title}." *{Publication}*, {Year}. {URL}

**Tier:** {S/A/B/C/D} | **Relevance:** {High/Medium/Low} | **Last verified:** {Date}

**Summary:** {2-3 sentences describing the content and main argument}

**Key contributions:**
- {Specific idea, framework, or finding #1}
- {Specific idea, framework, or finding #2}

**Limitations:** {Any caveats, biases, or gaps}

**Connection:** {How this source relates to other sources or the project}

Example Entry

## Kleppmann, Martin. "Designing Data-Intensive Applications." O'Reilly, 2017.

**Tier:** A | **Relevance:** High | **Last verified:** 2026-03-20

**Summary:** Comprehensive guide to distributed data systems covering replication,
partitioning, transactions, and stream processing. Bridges theoretical CS concepts
with practical engineering tradeoffs.

**Key contributions:**
- Clear taxonomy of consistency models (linearizability, causal, eventual)
- Practical comparison of batch vs. stream processing architectures
- The "unbundling the database" framing for microservice data patterns

**Limitations:** Pre-dates serverless and edge computing patterns. Some implementation
details are framework-specific and may be dated.

**Connection:** Foundation for resilience-patterns and data-pipeline-architect skills.
Complements the CAP theorem discussion in redis-patterns.

Bibliography Management

Collection Structure

references/
├── bibliography.yaml          # Machine-readable catalog
├── by-topic/
│   ├── distributed-systems.md # Topic-organized annotations
│   ├── security.md
│   └── ai-agents.md
├── by-tier/
│   ├── tier-s.md              # Primary sources only
│   └── tier-a.md              # Authoritative secondary
└── reading-log.md             # Chronological reading notes

Machine-Readable Catalog

bibliography:
  - id: kleppmann2017
    title: "Designing Data-Intensive Applications"
    author: "Martin Kleppmann"
    year: 2017
    type: book
    tier: A
    topics: [distributed-systems, databases]
    cited_by: [resilience-patterns, data-pipeline-architect]

  - id: fowler2002
    title: "Patterns of Enterprise Application Architecture"
    author: "Martin Fowler"
    year: 2002
    type: book
    tier: A
    topics: [architecture, patterns]
    cited_by: [backend-implementation-patterns]

Citation in Documents

The circuit breaker pattern [kleppmann2017, ch.8] provides fault isolation
between services. This aligns with the bulkhead pattern described in
[nygard2018, ch.5], which isolates failure domains at the resource level.

Curation Workflow

Adding New Sources

1. Discover — Find source through research, recommendation, or citation chain
2. Evaluate — Apply CRAAP test and assign tier
3. Annotate — Write summary, key contributions, limitations
4. Connect — Link to existing sources and project skills
5. Catalog — Add to bibliography.yaml and topic files

Periodic Review

## Quarterly Review Checklist

- [ ] Verify URLs still resolve (automated: link-checker script)
- [ ] Check for updated editions or superseding publications
- [ ] Re-evaluate tier for sources whose domains have evolved
- [ ] Remove or demote sources that are no longer current
- [ ] Add newly discovered sources from recent research

Anti-Patterns

• Uncritical acceptance — Every source needs evaluation, even authoritative ones
• Recency bias — Older sources can be foundational; evaluate on merit, not date alone
• Single-source reliance — Triangulate claims across multiple independent sources
• No annotation — A bibliography without annotations is just a link list
• Stale references — Sources need periodic re-verification
• Collecting without connecting — Always link sources to the questions they help answer