curiositech/redis-patterns-expert

Redis Patterns Expert

Redis is a data-structure server. The interesting part isn't the commands — it's the patterns. Most of the trouble comes from treating Redis as a database when you should treat it as a coordination primitive, or vice versa.

When to use

• Adding a cache layer in front of a slow data source.
• Need a distributed lock (single-leader election, mutual exclusion).
• Rate limiting an API by user/IP/key.
• Real-time leaderboards or sorted feeds.
• Event streams (XADD/XREAD with consumer groups) for multi-consumer durable queues.
• Pub/sub for notifications (best-effort, single delivery attempt).

Core capabilities

Caching strategies

Cache-aside (most common):

async function getUser(id: string) {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);
  const fresh = await db.users.findById(id);
  // SET with EX in one command — never SET then EXPIRE.
  await redis.set(`user:${id}`, JSON.stringify(fresh), 'EX', 300 + Math.floor(Math.random() * 60));
  return fresh;
}

The + random() jitter prevents the dogpile when many keys expire together. Without it, every cache key set during a deploy expires within the same minute 5 minutes later → thundering herd.

Stale-while-revalidate with a soft + hard TTL:

const SOFT = 300, HARD = 600;
const entry = await redis.get(key);
if (entry) {
  const { value, mtime } = JSON.parse(entry);
  if (Date.now() - mtime < SOFT * 1000) return value;
  // Stale but valid — refresh in background, return cached.
  void refresh(key);
  return value;
}
// No entry — block on origin.

Single-flight — collapse concurrent regeneration to one origin call:

async function getUserSingleflight(id: string) {
  const cacheKey = `user:${id}`;
  const lockKey  = `user:${id}:lock`;
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  const lockAcquired = await redis.set(lockKey, '1', 'NX', 'EX', 5);
  if (!lockAcquired) {
    // Someone else is fetching. Sleep + retry the cache.
    await sleep(50);
    return getUserSingleflight(id);
  }
  try {
    const fresh = await db.users.findById(id);
    await redis.set(cacheKey, JSON.stringify(fresh), 'EX', 300);
    return fresh;
  } finally {
    await redis.del(lockKey);
  }
}

Distributed locks

Single-instance Redis lock with safe release:

-- release.lua
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
else
  return 0
end

const token = crypto.randomUUID();
const acquired = await redis.set(lockKey, token, 'NX', 'PX', leaseMs);
if (!acquired) throw new Error('lock not acquired');
try {
  // ... critical section, must complete before leaseMs ...
} finally {
  await redis.eval(releaseScript, 1, lockKey, token);
}

The token prevents you from releasing someone else's lock if your work overran the lease. Don't use GET + DEL separately — that's a TOCTOU race.

Redlock (multi-instance) is contentious; for most applications a single Redis primary with a finite lease is fine. If you need stronger guarantees, use a real consensus system (etcd, Consul).

Rate limiting — sliding window with Lua

-- sliding-window.lua
-- KEYS[1] = bucket key  ARGV[1] = window seconds  ARGV[2] = limit  ARGV[3] = now-ms
local now = tonumber(ARGV[3])
local window = tonumber(ARGV[1]) * 1000
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)
local count = redis.call('ZCARD', KEYS[1])
if count >= tonumber(ARGV[2]) then
  return 0
end
redis.call('ZADD', KEYS[1], now, now)
redis.call('PEXPIRE', KEYS[1], window)
return 1

Token bucket is cheaper if you can tolerate burstiness; sliding window is fairer.

Sorted sets — leaderboards and time-series

// Top scorers
await redis.zincrby('game:scores', points, userId);
await redis.zrevrange('game:scores', 0, 9, 'WITHSCORES');

// Time-series with score = epoch
await redis.zadd('user:42:events', Date.now(), JSON.stringify(event));
await redis.zrangebyscore('user:42:events', Date.now() - 3600_000, '+inf');

// Cap to last 1000 events
await redis.zremrangebyrank('user:42:events', 0, -1001);

Streams + consumer groups

Durable queue with multiple consumers and at-least-once delivery:

// Producer
await redis.xadd('jobs', '*', 'type', 'send_email', 'to', user.email);

// One-time setup
await redis.xgroup('CREATE', 'jobs', 'workers', '$', 'MKSTREAM');

// Consumer
const messages = await redis.xreadgroup(
  'GROUP', 'workers', `worker-${pid}`,
  'COUNT', 10, 'BLOCK', 5000,
  'STREAMS', 'jobs', '>',
);
for (const [stream, entries] of messages ?? []) {
  for (const [id, fields] of entries) {
    try {
      await handle(fields);
      await redis.xack('jobs', 'workers', id);
    } catch {
      // No XACK → message stays in the PEL (pending entries list).
    }
  }
}

// Reclaim work from a crashed peer
const pending = await redis.xpending('jobs', 'workers', 'IDLE', 60000, '-', '+', 100);
for (const { id } of pending) {
  await redis.xclaim('jobs', 'workers', `worker-${pid}`, 60000, id);
}

// Cap stream growth
await redis.xadd('jobs', 'MAXLEN', '~', 100000, '*', ...fields);

The ~ makes MAXLEN approximate (cheaper). PEL is your story for poison messages — set a max-deliveries threshold and dead-letter beyond it.

Pub/sub

const sub = redis.duplicate(); // SUBSCRIBE blocks the connection
await sub.subscribe('user-events');
sub.on('message', (channel, payload) => handle(JSON.parse(payload)));

await redis.publish('user-events', JSON.stringify({ userId, kind: 'login' }));

At-most-once delivery; subscribers offline at publish time miss the message. For durable jobs, use Streams.

Lua scripts

EVAL ships the script every time; EVALSHA after SCRIPT LOAD is faster. Atomic — the whole script runs without other clients interleaving. KEYS are passed for cluster routing; non-key arguments go in ARGV. Cluster requires all KEYS to live on the same slot — use hash tags {user:42} for related keys.

Eviction policies

maxmemory                100mb
maxmemory-policy         allkeys-lru

| Policy | When to use | |--------|-------------| | noeviction | Redis as durable store (you should be using a real DB). | | allkeys-lru | Pure cache. | | allkeys-lfu | Cache where some keys are hot for a long time. | | volatile-lru | Mixed: some keys must persist (no TTL), evict only TTL'd ones. | | volatile-ttl | Evict shortest-TTL first; useful for staged data. |

Anti-patterns

KEYS in production

Symptom: Latency spikes correlate with admin commands. Diagnosis: KEYS * blocks the server while it scans every key. Fix: SCAN cursor-based iteration. For Cluster, scan each shard.

DEL on a 10M-item structure

Symptom: Single command takes seconds; replicas lag; clients time out. Diagnosis: DEL is synchronous in the main thread. Fix: UNLINK (non-blocking, async free). For sorted sets, batch via ZSCAN + ZREM chunks.

SET then EXPIRE

Symptom: Occasional keys with no TTL. Diagnosis: Crash between SET and EXPIRE leaves the key forever. Fix: SET key val EX seconds in one command.

Big-key OOM

Symptom: Single hash >1M fields; replication breaks; OOM on restart. Diagnosis: Unbounded growth — usually session data or user-item maps. Fix: Shard by user prefix (session:abc:user:42), or move to a sorted-set + TTL pattern.

Pub/sub for durable jobs

Symptom: Workers restart, miss the work, customer support tickets. Diagnosis: Pub/sub is at-most-once. Subscribers offline at publish lose the message. Fix: Use Streams + consumer groups. Always.

Hot-key on a single cluster slot

Symptom: One node CPU at 100%, others idle. Diagnosis: Cache key collision → all reads on one slot. Fix: Add a per-key salt or shard the key (user:42:cache:0, …:1) and randomize reads.

Quality gates

• [ ] No KEYS in production code (verify by linter or grep).
• [ ] Every cache key has TTL with jitter (5-15% randomization).
• [ ] Distributed locks use ownership tokens; release is via Lua script.
• [ ] Stream consumers handle XPENDING reclaim from crashed peers.
• [ ] Streams capped with MAXLEN ~ N.
• [ ] maxmemory and maxmemory-policy set explicitly per-environment.
• [ ] Big collections sharded by some key prefix; no unbounded growth.
• [ ] Pub/sub used only for ephemeral notifications, never durable jobs.
• [ ] UNLINK over DEL for any structure with thousands of items.

NOT for

• Redis Cluster admin/operations — failover, slot migration, persistence tuning is a separate skill (no dedicated skill yet).
• Webhook receiver dedup keyed in Redis — Redis is fragile for idempotency. → webhook-receiver-design (use a DB unique constraint instead).
• Embedded KV stores (SQLite, leveldb, RocksDB) — different consistency model.
• In-process LRU (lru-cache npm) — single-node, no shared state.
• Memcached — fewer data structures, simpler semantics.
• DragonflyDB / KeyDB — Redis-compatible but with different operational characteristics.