klod68/caching-strategy

---

name: caching-strategy description: "Use when deciding between IMemoryCache, IDistributedCache, Response Caching, or Output Caching, or when caching is applied inconsistently across a codebase and a decision framework is needed. Covers a decision matrix by data volatility and deployment topology, cache key design conventions, invalidation strategies (TTL, event-based, versioned, tag-based), and stampede prevention with GetOrCreateAsync. Domain: Performance, Architecture. Level: Intermediate. Tags: caching, IMemoryCache, IDistributedCache, Redis, performance, output-caching."

Caching Strategy

Problem

Teams choose caching strategies ad-hoc, leading to inconsistencies: some code uses ConcurrentDictionary, others use IMemoryCache, some use IDistributedCache, and cache invalidation is an afterthought. Without a clear decision framework, caches either leak memory, serve stale data, or don't exist where they should.

Solution: Decision Matrix + Consistent Patterns

Select the caching layer based on data volatility, consistency requirements, and deployment topology.

Decision Matrix

| Factor | IMemoryCache | IDistributedCache | Response Caching | Output Caching (.NET 7+) | |--------|-------------|-------------------|------------------|-------------------------| | Scope | Single process | Cross-process / cross-server | HTTP client/proxy | Server-side HTTP | | Speed | Fastest (in-process) | Network round-trip | No server hit | Cache at endpoint level | | Consistency | Per-instance (diverges in scale-out) | Shared across instances | Stale until expiry | Stale until expiry | | Best for | Reference data, config, computations | Session state, shared lookups | Static content, CDN | API responses, pages | | Invalidation | Explicit or time-based | Explicit or time-based | Cache-Control headers | Tag-based or time-based |

When to Use Each

| Scenario | Cache Type | Why | |----------|-----------|-----| | Lookup table rarely changes (countries, currencies) | IMemoryCache | In-process, no network hop, stale data acceptable | | User session across load-balanced servers | IDistributedCache (Redis) | Must be shared across instances | | Expensive computation result (per request) | IMemoryCache | Short TTL, per-instance is fine | | API response that changes daily | Output Caching | Server-side, tag-based invalidation | | Static assets (images, CSS, JS) | Response Caching | Let browsers and CDNs cache | | Feature flag state | IMemoryCache with short TTL | Per-instance, eventual consistency acceptable | | Search results with pagination | IDistributedCache | Shared across instances, keyed by query + page |

Implementation Patterns

IMemoryCache

public class CountryService
{
    private readonly IMemoryCache _cache;
    private static readonly TimeSpan CacheDuration = TimeSpan.FromHours(1);

    public async Task<IReadOnlyList<Country>> GetCountriesAsync(CancellationToken ct)
    {
        return await _cache.GetOrCreateAsync("countries:all", async entry =>
        {
            entry.AbsoluteExpirationRelativeToNow = CacheDuration;
            entry.Size = 1; // Required if SizeLimit is configured
            return await _repository.GetAllCountriesAsync(ct);
        }) ?? [];
    }
}

IDistributedCache

public async Task<UserProfile?> GetProfileAsync(int userId, CancellationToken ct)
{
    var key = $"profile:{userId}";
    var cached = await _distributedCache.GetStringAsync(key, ct);

    if (cached is not null)
        return JsonSerializer.Deserialize<UserProfile>(cached);

    var profile = await _repository.GetProfileAsync(userId, ct);

    if (profile is not null)
    {
        await _distributedCache.SetStringAsync(key,
            JsonSerializer.Serialize(profile),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(15),
                SlidingExpiration = TimeSpan.FromMinutes(5)
            }, ct);
    }

    return profile;
}

Cache Key Design

• Format: {entity}:{discriminator}:{id} — e.g., orders:status:active, user:42:profile
• Include all query parameters that affect the result
• Use constants for key prefixes — never string interpolation with magic strings
• Document key format per cached entity

Invalidation Strategies

| Strategy | When to Use | Example | |----------|------------|---------| | Time-based (TTL) | Data that changes predictably | Refresh config every 5 minutes | | Event-based | Data changed by known operations | Invalidate on OrderUpdated event | | Versioned keys | Data with infrequent bulk changes | countries:v3 — bump version on update | | Tag-based | Groups of related cached items | Invalidate all orders:* on bulk update |

When to Use

• Any application with read-heavy workloads or expensive computations
• APIs serving reference data or slowly changing datasets
• Scale-out deployments needing shared state across instances

When NOT to Use

• Write-heavy workflows where data changes every request
• Data with real-time consistency requirements (use database directly)
• Small datasets that fit in a simple Dictionary with no TTL needs

Gotchas

1. Always set a size limit on IMemoryCache (SizeLimit option) — unbounded caches leak memory.
2. Always set expiration — both AbsoluteExpiration (max lifetime) and SlidingExpiration (idle timeout).
3. Never cache mutable references — serialize/deserialize or use immutable types.
4. Cache stampede prevention: use SemaphoreSlim or GetOrCreateAsync — not manual check-then-set.
5. Log cache hits/misses at Debug level for diagnostics.
6. Don't cache errors — failed lookups should not poison the cache.

Anti-Patterns

| Pattern | Problem | Correction | |---------|---------|-----------| | ConcurrentDictionary as unbounded cache | Memory leak — entries never expire | Use IMemoryCache with expiration | | Check-then-set without lock | Cache stampede under concurrency | Use GetOrCreateAsync or SemaphoreSlim | | Caching mutable objects | Shared reference corruption | Cache serialized copy or use immutable types | | Same TTL for all cached data | Over-caching volatile data | Match TTL to data volatility | | Distributed cache for in-process-only data | Unnecessary network round-trip | Use IMemoryCache for process-local data | | No expiration on cached items | Stale data served indefinitely | Always set AbsoluteExpirationRelativeToNow |

Related Skills

• Retry with Exponential Backoff (retry cache misses that hit a failing backend)
• Health Check Endpoint (cache health check for Redis connectivity)