Hot Keys

Identifying Hot Keys

Hot keys are cache entries that receive disproportionate traffic, potentially causing performance bottlenecks.

What Makes a Key "Hot"?

Hot Key Characteristics:

1. High Request Rate:
   - Normal key: 100 QPS
   - Hot key: 10,000+ QPS

2. Skewed Distribution (Zipfian):
   - 1% of keys handle 50%+ of traffic
   - Example: Trending product, viral post

3. Single Node Overload:
   - All requests hit same cache node
   - Network/CPU bottleneck

Detection Methods

# 1. Key Access Counter
class HotKeyDetector:
    def __init__(self, threshold=1000, window_seconds=60):
        self.threshold = threshold
        self.window = window_seconds
        self.counters = {}
    
    def track_access(self, key):
        current_window = int(time.time() / self.window)
        counter_key = f"{key}:{current_window}"
        
        count = self.cache.incr(counter_key)
        if count == 1:
            self.cache.expire(counter_key, self.window * 2)
        
        return count >= self.threshold

# 2. Redis MONITOR (Development only)
# redis-cli MONITOR | awk '{print $4}' | sort | uniq -c | sort -rn | head -20

# 3. Access Log Analysis
import collections

def analyze_hot_keys(logs):
    key_counts = collections.Counter()
    for log in logs:
        key = extract_cache_key(log)
        key_counts[key] += 1
    
    # Return top 1% of keys
    total = sum(key_counts.values())
    threshold = total * 0.01
    
    return [(k, v) for k, v in key_counts.most_common() if v > threshold]

Monitoring Hot Keys

# Redis slowlog for hot key detection
redis.slowlog_get(100)  # Check slow queries

# Redis memory analysis
redis-cli memory usage <key>

# Application metrics
metrics.histogram('cache.key.access_rate', key_access_rate)
metrics.gauge('cache.hot_keys.count', len(hot_keys))

Sharding Hot Keys

Sharding distributes a single hot key across multiple cache entries to spread the load.

Key Sharding Strategy

Original Hot Key:
"product:featured" → 100,000 QPS on single node

Sharded (10 shards):
"product:featured:0" → 10,000 QPS
"product:featured:1" → 10,000 QPS
...
"product:featured:9" → 10,000 QPS

Total: Same 100,000 QPS, distributed across 10 keys!

Implementation

class ShardedHotKeyCache:
    def __init__(self, cache_client, num_shards=10):
        self.cache = cache_client
        self.num_shards = num_shards
    
    def _get_shard(self, key, seed=None):
        """Determine shard for a key"""
        if seed is None:
            seed = hash(key)
        shard = seed % self.num_shards
        return f"{key}:shard:{shard}"
    
    def set(self, key, value, ttl=3600):
        """Set value across all shards"""
        # Store in all shards for read distribution
        for i in range(self.num_shards):
            shard_key = f"{key}:shard:{i}"
            self.cache.set(shard_key, value, ttl)
    
    def get(self, key):
        """Get from random shard"""
        shard = random.randint(0, self.num_shards - 1)
        shard_key = f"{key}:shard:{shard}"
        return self.cache.get(shard_key)
    
    def delete(self, key):
        """Delete all shards"""
        for i in range(self.num_shards):
            self.cache.delete(f"{key}:shard:{i}")

# Usage
hot_cache = ShardedHotKeyCache(redis, num_shards=10)
hot_cache.set("product:featured", product_data)  # Writes to 10 shards
value = hot_cache.get("product:featured")  # Reads from random shard

Sharding Strategies

Strategy	Description	Pros	Cons
Random	Random shard selection	Simple, even distribution	No locality
Consistent Hash	Hash-based shard	Deterministic	More complex
Round-Robin	Sequential shard	Even distribution	Predictable
Weighted	Weighted by capacity	Respects node capacity	Complex

Local Caching for Hot Keys

Local (in-process) caching eliminates network round-trips for extremely hot keys.

Multi-Level Cache Architecture

Request Flow:

App → L1 (Local/Memory) → L2 (Redis) → L3 (DB)
         ↓ Hit (ns)         ↓ Hit (ms)    ↓ (ms)

L1: In-process, ~1ns access
L2: Network cache, ~1ms access
L3: Database, ~10-100ms access

Implementation

import cachetools
import threading

class MultiLevelCache:
    def __init__(self, redis_client, l1_size=1000, l1_ttl=60):
        self.redis = redis_client
        self.l1_cache = cachetools.TTLCache(maxsize=l1_size, ttl=l1_ttl)
        self.lock = threading.Lock()
    
    def get(self, key):
        # L1: Local memory (fastest)
        with self.lock:
            if key in self.l1_cache:
                return self.l1_cache[key]
        
        # L2: Redis
        value = self.redis.get(key)
        if value is not None:
            with self.lock:
                self.l1_cache[key] = value
            return value
        
        return None
    
    def set(self, key, value, ttl=3600):
        # Update both levels
        with self.lock:
            self.l1_cache[key] = value
        self.redis.setex(key, ttl, value)
    
    def invalidate(self, key):
        with self.lock:
            self.l1_cache.pop(key, None)
        self.redis.delete(key)

# Usage for hot keys
hot_key_cache = MultiLevelCache(redis, l1_size=100, l1_ttl=30)

# Hot product with local caching
def get_featured_product():
    return hot_key_cache.get("product:featured")

Cache Warming

class CacheWarmer:
    def __init__(self, cache, db):
        self.cache = cache
        self.db = db
    
    def warm_hot_keys(self, key_list):
        """Pre-populate cache with hot keys"""
        for key in key_list:
            data = self.db.get(key)
            if data:
                self.cache.set(key, data, ttl=3600)
    
    def warm_from_analytics(self):
        """Warm cache based on access patterns"""
        # Get top accessed keys from analytics
        hot_keys = analytics.get_top_keys(limit=1000)
        self.warm_hot_keys(hot_keys)

# Warm cache on startup
warmer = CacheWarmer(cache, db)
warmer.warm_from_analytics()

When to Use Local Caching

Scenario	Recommendation
Hot keys with 10K+ QPS	Local cache + sharding
Read-heavy, low consistency	Local cache acceptable
Real-time data needed	Short TTL local cache
Write-heavy workload	Avoid local cache
Strong consistency required	Skip local cache