curiositech/caching-strategies

Caching Strategies

Caching is the most commonly misapplied performance technique. The failure mode is not "cache too little" — it is "cache without an invalidation strategy and then discover the problem in production six months later when users complain about stale data that you cannot explain."

When to Use

✅ Use for:

• Choosing which caching pattern fits a use case (cache-aside, write-through, write-behind)
• Designing TTL values for different data freshness requirements
• Implementing Redis caching patterns: sorted sets, pub/sub invalidation, Lua scripts
• Configuring Cache-Control headers, ETags, and CDN behavior
• Preventing cache stampedes via locking, probabilistic early expiry, or background refresh
• Cache warming strategies for cold-start scenarios
• Multi-tier cache design (in-memory L1, Redis L2, CDN L3)

❌ NOT for:

• Database-internal query plan caching (handled by the database)
• Python functools.lru_cache / JavaScript memoize utilities (pure function memoization)
• CPU branch prediction or hardware cache tuning
• Session storage (use dedicated session skill)

---

Which Caching Pattern?

flowchart TD
    Q1{Who writes to cache?} --> WA[Application writes]
    Q1 --> WC[Cache writes automatically]
    WA --> Q2{When does the cache get populated?}
    Q2 -->|On read miss| CA[Cache-Aside\n'Lazy loading']
    Q2 -->|On every write| WT[Write-Through\n'Eager write']
    WC --> Q3{Sync or async write-back?}
    Q3 -->|Sync — write completes when cache updates| WT
    Q3 -->|Async — write returns fast, flush later| WB[Write-Behind\n'Write-back']
    CA --> N1{Is stale data OK\nfor a short period?}
    N1 -->|Yes| CA_USE[Use cache-aside\nwith TTL expiry]
    N1 -->|No| INVAL[Add explicit invalidation\nor use write-through]
    WT --> NOTE2[Good for read-heavy data\nthat changes infrequently]
    WB --> NOTE3[Good for write-heavy workloads\nRisk: data loss on crash]

---

Multi-Tier Cache Architecture

flowchart LR
    USER[User Request] --> CDN{CDN / Edge Cache\nL3 — 100ms+ saved}
    CDN -->|Cache hit| RESP[Response]
    CDN -->|Cache miss| LB[Load Balancer]
    LB --> APP[App Server]
    APP --> L1{In-Process Cache\nL1 — ~0ms}
    L1 -->|Hit| APP
    L1 -->|Miss| REDIS{Redis\nL2 — 1-5ms}
    REDIS -->|Hit| APP
    REDIS -->|Miss| DB[(Database\n10-100ms)]
    DB --> REDIS
    REDIS --> APP
    APP --> L1
    APP --> CDN
    APP --> RESP

| Tier | Technology | Latency | Capacity | Shared? | |---|---|---|---|---| | L1: In-process | Node.js Map, Python dict, LRU-cache | ~0ms | Small (MB) | No — per instance | | L2: Distributed | Redis, Memcached | 1-5ms | Large (GB) | Yes — all instances | | L3: Edge/CDN | Cloudflare, Fastly, CloudFront | 10-100ms | Massive | Yes — globally |

Rule: Data mutates in one place first. Invalidation flows outward: DB → Redis → CDN. Never skip tiers in invalidation.

---

Cache-Aside Pattern (Most Common)

Application manages cache explicitly. On read: check cache, if miss fetch from DB, populate cache, return. On write: update DB, delete cache entry.

class UserCache {
  private redis: Redis;
  private readonly TTL_SECONDS = 300; // 5 minutes

  async getUser(userId: string): Promise<User> {
    const key = `user:${userId}`;

    // 1. Check cache
    const cached = await this.redis.get(key);
    if (cached) return JSON.parse(cached);

    // 2. Cache miss — fetch from source
    const user = await db.users.findById(userId);
    if (!user) throw new NotFoundError('User', userId);

    // 3. Populate cache
    await this.redis.setex(key, this.TTL_SECONDS, JSON.stringify(user));

    return user;
  }

  async updateUser(userId: string, data: Partial<User>): Promise<User> {
    const user = await db.users.update(userId, data);

    // 4. Invalidate — delete, don't update
    // Updating in cache risks race conditions; let the next read repopulate
    await this.redis.del(`user:${userId}`);

    return user;
  }
}

When invalidation deletes vs overwrites: Delete is almost always correct. Overwriting in cache after a write creates a race: another request may have fetched the old value between your DB write and your cache write. Delete forces the next reader to fetch fresh.

---

Write-Through Pattern

Every write goes to cache and DB synchronously. Cache is always populated. Good for data that is written once and read many times.

async function createProduct(data: CreateProductInput): Promise<Product> {
  // Write to DB first (source of truth)
  const product = await db.products.create(data);

  // Immediately populate cache — no future cache miss for this product
  const key = `product:${product.id}`;
  await redis.setex(key, 3600, JSON.stringify(product));

  // Also invalidate list caches that include this product
  await redis.del('products:list:*'); // pattern delete via SCAN, see redis-patterns.md

  return product;
}

Trade-off: Higher write latency (two writes per operation). Wasted cache space for items that are never read again after creation. Best for data with high read:write ratio.

---

TTL Design

TTL is not a cache invalidation strategy — it is a staleness budget. Design TTLs based on data volatility and acceptable staleness:

| Data Type | TTL | Rationale | |---|---|---| | User session token | Match session expiry | Security requirement | | User profile (name, avatar) | 5-15 minutes | Changes rarely; short enough for responsiveness | | Product catalog | 1-4 hours | Changes occasionally; acceptable lag | | Inventory counts | 30 seconds | Changes frequently; short but not zero | | Exchange rates | 60 seconds | Regulatory; must not be too stale | | Static config / feature flags | 60 seconds + pub/sub invalidation | Needs push invalidation on change | | Computed aggregates (daily stats) | Until next computation | Explicit invalidation on recalculate |

TTL jitter: When many keys have the same TTL, they expire simultaneously, causing a thundering herd. Add random jitter:

const jitter = Math.floor(Math.random() * 60); // 0-60 seconds
await redis.setex(key, baseTtl + jitter, value);

---

Cache Stampede Prevention

A stampede (also: dog-pile, thundering herd) occurs when many requests simultaneously miss an expired cache key and all rush to compute or fetch the value.

Strategy 1: Probabilistic Early Expiry (XFetch)

Re-fetch before expiry with probability proportional to how close the key is to expiring:

async function getWithEarlyExpiry<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttlSeconds: number,
  beta = 1.0
): Promise<T> {
  const entry = await redis.get(key + ':meta');
  if (entry) {
    const { value, expiresAt, fetchDurationMs } = JSON.parse(entry);
    const now = Date.now();
    const ttlRemaining = expiresAt - now;
    // Fetch early if within probabilistic window
    const shouldRefetch = ttlRemaining < beta * fetchDurationMs * Math.log(Math.random());
    if (!shouldRefetch) return value;
  }

  // Fetch and cache
  const start = Date.now();
  const value = await fetcher();
  const fetchDurationMs = Date.now() - start;
  const expiresAt = Date.now() + ttlSeconds * 1000;

  await redis.setex(key + ':meta', ttlSeconds, JSON.stringify({ value, expiresAt, fetchDurationMs }));
  return value;
}

Strategy 2: Mutex Lock on Miss

Only one worker recomputes the value; others wait on the lock or return stale data:

async function getWithLock<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttl: number
): Promise<T> {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const lockKey = `lock:${key}`;
  const lockAcquired = await redis.set(lockKey, '1', 'NX', 'PX', 5000); // 5s TTL

  if (!lockAcquired) {
    // Another worker is computing — poll briefly then return stale or throw
    await sleep(100);
    const retried = await redis.get(key);
    if (retried) return JSON.parse(retried);
    throw new Error('Cache unavailable');
  }

  try {
    const value = await fetcher();
    await redis.setex(key, ttl, JSON.stringify(value));
    return value;
  } finally {
    await redis.del(lockKey);
  }
}

Consult references/redis-patterns.md for the Lua-atomic version of this lock (prevents lock release by wrong client).

---

Anti-Patterns

Anti-Pattern: Cache Everything Forever

Novice: "Caching makes things fast. Set TTL to 0 (no expiry) or a year to maximize cache hit rate."

Expert: Unbounded caches are memory leaks with extra steps. They also guarantee stale data — users see prices, permissions, and content from months ago. Production incidents traced to "why is this user seeing the old plan limit" are almost always cache-forever bugs.

// Wrong — no expiry means the cache grows forever
await redis.set(`user:${id}`, JSON.stringify(user)); // no TTL

// Right — every cache entry has a maximum lifetime
await redis.setex(`user:${id}`, 300, JSON.stringify(user)); // 5 minutes

Python equivalent:

# Wrong
redis.set(f"user:{id}", json.dumps(user))

# Right
redis.setex(f"user:{id}", 300, json.dumps(user))

Detection: redis.set(key, value) without EX/PX/EXAT options. Redis TTL key returning -1 for cache keys. Memory growth over time with no plateau.

Timeline: This has always been wrong, but the Redis default of no-expiry makes it easy to do accidentally. Redis 7.0 (2022) introduced key eviction policies as default, reducing severity — but you still get stale data.

---

Anti-Pattern: No Invalidation Strategy

Novice: "I'll set a short TTL and the stale data problem solves itself."

Expert: TTL-only invalidation means every change to data has a propagation delay equal to the TTL. For some data (user roles, permissions, prices after a sale ends) that lag is unacceptable. Worse: this creates an implicit contract that is never documented, and teams later increase the TTL for performance without realizing they just made the staleness window much larger.

// Problem: user loses admin role, but can still access admin routes for 5 minutes
await redis.setex(`user:permissions:${id}`, 300, JSON.stringify(permissions));

// Right: invalidate explicitly on change
async function revokeAdminRole(userId: string) {
  await db.userRoles.delete(userId, 'admin');
  await redis.del(`user:permissions:${userId}`); // immediate invalidation
  // Also publish to notify other app instances to clear L1 caches
  await redis.publish('permissions:invalidated', userId);
}

LLM mistake: LLMs frequently omit invalidation logic in code generation because it is invisible in simple cache-aside examples. Every tutorial shows "set on write," few show "delete on update."

Detection: Cache sets with no corresponding deletes in write paths. TTL as the only eviction mechanism for user-controlled data (roles, permissions, settings). No DEL, UNLINK, or pub/sub events in the codebase's update handlers.

---

References

• references/redis-patterns.md — Consult for Redis-specific patterns: sorted sets for leaderboards, Lua atomic operations, pub/sub cache invalidation, SCAN-based key deletion, pipeline batching
• references/http-caching.md — Consult for browser caching: Cache-Control directives, ETags, Vary headers, CDN configuration, service worker caching strategies