igbuend/transparent-encrypted-storage-pattern

Transparent Encrypted Storage Security Pattern

Storage infrastructure automatically encrypts all data before writing to disk and decrypts when reading. Application is unaware of encryption—it happens transparently at the storage layer.

Problem Addressed

Leak data at rest: All stored data could be exposed through physical theft, backup compromise, or unauthorized storage access.

Core Components

| Role | Type | Responsibility | |------|------|----------------| | Application | Entity | Reads/writes data normally | | Storage Manager | Entity | Intercepts I/O, manages encryption | | Cryptographer | Cryptographic Primitive | Performs encryption/decryption | | Physical Storage | Storage | Stores encrypted data | | Key Manager | Entity | Manages encryption keys |

Data Elements

• data: Plaintext data from application
• {data}_k: Encrypted data on disk
• key: Encryption key (managed by infrastructure)

Pattern Flow

Write Operation

Application → [write(data)] → Storage Manager
Storage Manager → [encrypt(data)] → Cryptographer
Cryptographer → [{data}_k] → Storage Manager
Storage Manager → [write({data}_k)] → Physical Storage

Read Operation

Application → [read()] → Storage Manager
Storage Manager → [read()] → Physical Storage
Physical Storage → [{data}_k] → Storage Manager
Storage Manager → [decrypt({data}_k)] → Cryptographer
Cryptographer → [data] → Storage Manager
Storage Manager → [data] → Application

Key Characteristics

Transparent to Application

• No application code changes
• Encryption happens at storage layer
• Application sees plaintext

All-or-Nothing

• Everything in storage is encrypted
• No selective encryption
• Simpler model, broader protection

Infrastructure Managed

• Database or filesystem handles encryption
• Key management often integrated
• Operational, not developmental

Implementation Options

Database-Level TDE

Most databases support Transparent Data Encryption:

• SQL Server TDE: Encrypts data files, logs, backups
• Oracle TDE: Tablespace or column encryption
• PostgreSQL: pg_tde extension
• MySQL: InnoDB tablespace encryption
• MongoDB: Encrypted storage engine

Filesystem/Disk Encryption

• LUKS (Linux)
• BitLocker (Windows)
• FileVault (macOS)
• dm-crypt (Linux)
• Cloud provider disk encryption (AWS EBS, Azure Disk, GCP)

Cloud Provider Options

• AWS RDS encryption
• Azure SQL TDE
• GCP Cloud SQL encryption
• All major cloud storage services offer encryption at rest

Security Considerations

What TDE Protects Against

✓ Physical disk theft ✓ Decommissioned disk exposure ✓ Backup media theft ✓ Unauthorized file system access

What TDE Does NOT Protect Against

✗ Authorized database access (data decrypted for queries) ✗ SQL injection (attacker queries through application) ✗ Application vulnerabilities ✗ Compromised database credentials ✗ Memory attacks (data decrypted in memory)

Important: TDE is ONE layer of defense, not complete protection.

Key Management

Critical: Key security determines encryption security

Considerations:

• Key stored separately from encrypted data
• Use Hardware Security Module (HSM) where possible
• Key backup and recovery procedures
• Key rotation strategy
• Access controls on key material

Cloud providers offer:

• AWS KMS
• Azure Key Vault
• GCP Cloud KMS

Performance Impact

• Modern CPUs have AES-NI hardware acceleration
• Typically 2-10% overhead
• Test with realistic workloads
• Consider SSD vs HDD differences

Backup Encryption

TDE typically encrypts backups automatically:

• Verify backup encryption is enabled
• Test backup restore procedures
• Key availability for restores

Comparison with Selective Encryption

| Aspect | Transparent | Selective | |--------|-------------|-----------| | Application changes | None | Required | | Granularity | All data | Specific fields | | Protection scope | Storage layer | End-to-end possible | | Query on encrypted | Yes (decrypted for query) | No (unless special techniques) | | Key management | Infrastructure | Application |

Recommendation: Use both when appropriate

• TDE for baseline storage protection
• Selective encryption for highly sensitive fields

Implementation Checklist

• [ ] TDE enabled on database/storage
• [ ] Key stored in HSM/KMS
• [ ] Key backup procedures tested
• [ ] Key rotation schedule defined
• [ ] Backup encryption verified
• [ ] Performance impact measured
• [ ] Recovery procedures tested
• [ ] Access to key material audited
• [ ] Understood: TDE ≠ complete protection

When to Use

Use Transparent Encryption When:

• Need broad storage protection
• Cannot modify application
• Compliance requires encryption at rest
• Protecting against physical threats

Combine with Selective Encryption When:

• Some data needs additional protection
• Data must be protected from DBAs
• End-to-end encryption required

Related Patterns

• Selective encrypted storage (alternative: field-level)
• Encryption (underlying operations)
• Cryptographic key management (key handling)
• Cryptography as a service (KMS usage)

References

• Source: https://securitypatterns.distrinet-research.be/patterns/07_01_002__transparent_encrypted_storage/
• OWASP Cryptographic Storage Cheat Sheet
• Database vendor TDE documentation