Mastering Caching in System Design
A comprehensive guide to building scalable and efficient systems
Understanding Caching
Caching is a fundamental technique in system design that temporarily stores frequently accessed data in a faster storage layer. This optimization significantly improves system performance and reduces load on backend services.
Caching Strategies
Write-Through Cache
Data is written to both cache and database simultaneously. Ensures consistency but higher write latency.
Write-Back Cache
Data is written to cache first, then asynchronously to database. Faster but risk of data loss.
Read-Through Cache
Cache automatically loads missing items from database. Consistent but initial request latency.
Popular Caching Solutions
| Product | Best For | Features | Performance |
|---|---|---|---|
| Redis | Complex data structures | Data types, persistence, replication | ~100k ops/sec |
| Memcached | Simple key-value data | Distributed memory caching | ~80k ops/sec |
| Varnish | HTTP caching | HTTP acceleration, ESI | ~150k req/sec |
Real-World Use Cases
E-Commerce Platform
Product catalog caching during Black Friday sales
Social Media Feed
News feed caching for millions of users
Gaming Leaderboard
Real-time score updates and rankings
Advanced Caching Patterns
Cache Coherence Patterns
Cache-Aside (Lazy Loading)
The application first checks the cache for data. On a cache miss, it fetches from the database and updates the cache. This pattern is ideal for read-heavy workloads with eventual consistency requirements.
def get_user_data(user_id):
# Try cache first
user_data = cache.get(user_id)
if user_data is None:
# Cache miss - get from database
user_data = db.query_user(user_id)
# Update cache with TTL
cache.set(user_id, user_data, ttl=3600)
return user_data
Cache-as-SoR (Source of Record)
The cache becomes the primary source of truth, with the database acting as a backup. This pattern is used in high-throughput systems where consistency can be relaxed.
Cache Invalidation Strategies
Time-Based Invalidation
cache.set(key, value, TTL=3600) # Expires in 1 hour cache.set(key, value, TTL=86400) # Expires in 1 day
Event-Based Invalidation
# When user updates profile
def update_profile(user_id, data):
db.update_user(user_id, data)
cache.delete(f"user:{user_id}")
cache.delete(f"user_friends:{user_id}")
Common Challenges & Solutions
Cache Stampede
Multiple requests trying to regenerate the same cached item simultaneously when it expires.
Solution: Cache Warming
def get_with_probabilistic_early_recomputation(key):
value, expire_time = cache.get_with_expire_time(key)
if value is None:
return compute_and_cache(key)
# Start recomputing before expiry
if time.now() > expire_time - 300: # 5 min before
if random.random() < 0.1: # 10% chance
async_recompute(key)
return value
Choosing the Right Caching Solution
Decision Factors
Simple key-value vs complex structures
Single node vs distributed system
Strong vs eventual consistency
Performance Optimization Tips
Compression
Use compression for large values to reduce memory usage and network transfer time.
import zlib
def cache_compressed(key, value):
compressed = zlib.compress(json.dumps(value).encode())
cache.set(key, compressed)
def get_compressed(key):
compressed = cache.get(key)
if compressed:
return json.loads(zlib.decompress(compressed))
Batch Operations
Use multi-get operations to reduce network roundtrips.
# Instead of multiple gets
keys = [f"user:{id}" for id in user_ids]
users = cache.mget(keys) # Single network call