kienbui1995/dynamodb-design

Amazon DynamoDB Design

When to Use

• Designing a NoSQL schema for a new DynamoDB table
• Choosing partition key and sort key to support required access patterns
• Planning Global Secondary Indexes (GSI) and Local Secondary Indexes (LSI)
• Selecting between Provisioned and On-Demand capacity modes
• Implementing event-driven reactions with DynamoDB Streams + Lambda
• Caching hot read patterns with DAX (DynamoDB Accelerator)
• Preparing for AWS DEA-C01 or DVA-C02 exams

Core Jobs

1. Partition Key Design

• Partition key = hash key — determines physical storage partition (1 partition = 10GB, 3000 RCU, 1000 WCU)
• High cardinality keys → even data distribution across partitions
• Avoid hot partition keys (e.g., status field with only 3 values, date field with millions of writes per day)

Good partition key examples:

• user_id (UUID) — high cardinality, random distribution
• order_id (UUID) — unique per order
• device_id — high cardinality for IoT

Hot partition fixes:

1. Write sharding: append random suffix user_id#1, user_id#2 … user_id#N; scatter reads across shards then aggregate
2. Composite key: combine high-cardinality attribute with time bucket (user_id#2024-01)
3. Partition key + sort key: use composite primary key to spread items under same "logical" entity

2. GSI vs LSI

| Feature | Global Secondary Index (GSI) | Local Secondary Index (LSI) | |---------|-----------------------------|-----------------------------| | Partition key | Different from table partition key | SAME as table partition key | | Sort key | Any attribute (optional) | Different from table sort key | | Capacity | Separate RCU/WCU from base table | Shares capacity with base table | | Consistency | Eventually consistent ONLY | Supports strongly consistent reads | | Size limit | No constraint per partition | 10GB per partition key value | | Creation time | Anytime (even after table creation) | ONLY at table creation time | | Max per table | 20 GSIs | 5 LSIs | | Projection | Can project any attributes | Can project any attributes |

GSI use case: Query orders by customer email (different partition key than order_id):

Table: PK=order_id, SK=timestamp
GSI: PK=customer_email, SK=order_date

LSI use case: Query user's posts sorted by likes (same user_id partition, different sort):

Table: PK=user_id, SK=post_date
LSI: PK=user_id, SK=like_count

3. Capacity Mode Selection

| Mode | Pricing | Best For | Scaling | |------|---------|---------|---------| | Provisioned | Per RCU/WCU-hour | Predictable, sustained workloads | Manual or Auto Scaling | | On-Demand | Per request (Read Request Unit / Write Request Unit) | New, unpredictable, spiky workloads | Automatic, instant |

Cost comparison: On-Demand = ~2–2.5x more expensive per RCU/WCU than right-sized Provisioned + Auto Scaling.

Auto Scaling (Provisioned): Set min/max RCU/WCU; DynamoDB adjusts within bounds using CloudWatch alarms. Scale-up is fast; scale-down slower (4 decreases/day limit historically, now managed automatically).

4. Read Consistency Models

| Type | Freshness | Cost | Use Case | |------|----------|------|---------| | Eventually consistent | May be slightly stale | 0.5 RCU per 4KB | Default; fine for most reads | | Strongly consistent | Always latest data | 1 RCU per 4KB | Financial balances, inventory counts | | Transactional | ACID across items | 2 RCU per 4KB | Multi-step operations needing atomicity |

GSI reads are always eventually consistent — never strongly consistent.

5. DynamoDB Streams

• Captures a time-ordered sequence of item-level changes (INSERT, MODIFY, REMOVE)
• Retention: 24 hours (non-configurable)
• StreamViewType options:
  • KEYS_ONLY — only PK/SK of changed item
  • NEW_IMAGE — full item after change
  • OLD_IMAGE — full item before change
  • NEW_AND_OLD_IMAGES — both before and after

Lambda integration:

• Lambda polls stream and invokes function in batches (up to 10,000 records)
• Exactly-once processing: bisect on error to identify bad record; send to DLQ
• Parallel streams: up to 2 Lambda functions per shard concurrently

6. DAX (DynamoDB Accelerator)

• In-memory cache for DynamoDB; write-through caching
• Microsecond read latency (vs single-digit millisecond from DynamoDB)
• Transparent API compatibility — same DynamoDB SDK calls, just point to DAX endpoint
• Cache types:
  • Item cache — caches individual item GetItem/TransactGetItems results
  • Query cache — caches Query/Scan results (based on query parameters as key)
• DAX does NOT help write-heavy workloads; helps read-heavy patterns with hot items
• Multi-AZ cluster: deploy at least 3 nodes across AZs for HA

7. DynamoDB Transactions

• TransactWriteItems: up to 100 items, across multiple tables, all-or-nothing ACID
• TransactGetItems: up to 100 items, atomic consistent read
• Use cases: inventory deduction + order creation; bank transfer (debit + credit)
• Cost: 2x the normal RCU/WCU (transactional reads/writes)
• Cannot mix transactional and non-transactional operations in same call

Key Concepts

• RCU (Read Capacity Unit) — 1 strongly consistent read of 4KB/s; 2 eventually consistent reads of 4KB/s
• WCU (Write Capacity Unit) — 1 write of 1KB/s
• Hot partition — single partition receiving disproportionate traffic (bottleneck at 3000 RCU or 1000 WCU)
• Adaptive capacity — DynamoDB automatically boosts hot partition capacity (short-term relief; not a long-term fix)
• Conditional writes — ConditionExpression on PutItem/UpdateItem/DeleteItem; optimistic locking pattern
• TTL (Time to Live) — mark items for automatic deletion after timestamp; no WCU consumed; deletion within 48h
• Global tables — multi-region, multi-active replication; last-writer-wins conflict resolution
• Export to S3 — point-in-time snapshot of table to S3 in DynamoDB JSON or ION format (no RCU consumed)

Checklist

• [ ] Partition key has high cardinality (no hot partitions)?
• [ ] All required access patterns covered by table keys or indexes (GSI/LSI)?
• [ ] LSI created at table creation time (cannot add later)?
• [ ] GSI WCU provisioned separately from base table?
• [ ] Capacity mode chosen: On-Demand for new/spiky, Provisioned + Auto Scaling for steady?
• [ ] TTL enabled for ephemeral data (sessions, cache, events)?
• [ ] DynamoDB Streams + Lambda for event-driven reactions?
• [ ] DAX cluster deployed for read-heavy hot-item workloads?

Output Format

• 🔴 Critical — hot partition key with low cardinality (status, boolean, region with few values); LSI added after table creation (impossible — must recreate table)
• 🟡 Warning — On-Demand mode used for sustained high-throughput (2x cost vs Provisioned + Auto Scaling); missing index for required access pattern (application doing full table scans)
• 🟢 Suggestion — TTL for session/cache data; DAX for read-heavy hot items; write sharding for uneven write distribution

Exam Tips

• Hot partition = too many reads/writes to same partition key → use write sharding (add random suffix #0 to #N)
• GSI = eventually consistent reads only; LSI supports strongly consistent reads (same partition key as table)
• DynamoDB Streams retention = 24 hours only; process promptly; use Lambda bisect-on-error + DLQ for failures
• DAX = read-heavy workloads with hot items; does NOT help write-heavy; microsecond latency vs DynamoDB's milliseconds
• On-Demand = ~2x more expensive at sustained load vs right-sized Provisioned + Auto Scaling; use for unpredictable/new workloads only
• Conditional writes = optimistic locking — prevent lost updates in concurrent write scenarios (e.g., check version attribute before updating)
• LSI must be created at table creation time — cannot be added later; plan access patterns before creating table
• Global Tables use last-writer-wins conflict resolution; applications must handle potential conflicts in multi-region active-active writes