Identifying Bottlenecks

Why Bottleneck Identification Matters at Amazon

Amazon interviewers expect you to identify bottlenecks before they ask. This is the difference between an SDE-1 who waits for direction and one who thinks like an engineer. When you proactively call out bottlenecks, you demonstrate:

• System thinking — You understand the whole system, not just your component
• Production experience — You've seen these problems in real systems
• Risk awareness — You anticipate failures before they happen

The 6 Most Common Bottlenecks (Memorize These)

Bottleneck	Frequency	Impact	Detection Signal
Database	Most common (70% of interviews)	High	Slow queries, high latency, connection errors
Network	Very common (40%)	High	Cross-region latency, timeouts, packet loss
Single Point of Failure	Common (30%)	Critical	No redundancy, one server handles everything
Memory	Moderate (20%)	Medium	OOM errors, cache eviction, slow garbage collection
CPU	Moderate (20%)	Medium	High utilization, slow computations, encoding/decoding
Disk I/O	Less common (10%)	Medium	Slow writes, log bottlenecks, disk full

Database Bottlenecks (The #1 Interview Topic)

Every system design interview at Amazon will touch on database performance. Here's what to watch for:

Read Bottlenecks:

• Complex JOIN operations across large tables
• Full table scans without proper indexes
• Too many read requests hitting a single database instance
• No read replicas to distribute load

Write Bottlenecks:

• High write throughput on a single database
• Write contention on hot rows (e.g., counter updates, inventory)
• Transaction locks blocking concurrent writes
• Sequential writes to a single partition

Connection Bottlenecks:

• Database max connection limit reached (default: 100-200 for PostgreSQL)
• Connection pool exhaustion under load
• Long-running queries holding connections

Example — E-commerce Inventory:

Problem: 10,000 concurrent users checking inventory

Single DB instance handles:
- 10,000 SELECT queries/second
- 500 UPDATE queries/second (purchases)
- Connection limit: 200

Result: Connection timeout errors for 90% of users

Network Bottlenecks

Cross-Region Calls:

• US-East to EU-West: ~80ms latency
• Each hop adds 5-20ms
• Chatty protocols (many small requests) amplify this

Large Payloads:

• Uploading 100MB files over single connection
• Large JSON responses (megabytes of data)
• No compression on API responses

Chatty Protocols:

• 10 sequential API calls to render one page
• Each call waits for previous to complete
• Total latency = sum of all call latencies

Single Points of Failure (SPOF)

A SPOF is any component that, if it fails, takes down the entire system:

• Single server — One machine handles all traffic
• Single database — One database instance with no replicas
• Single cache node — One Redis instance, no cluster
• Single load balancer — One ALB with no failover
• Single region — All infrastructure in one AWS region

How to detect SPOFs: Ask yourself "What happens if this component fails?" If the answer is "the system goes down," it's a SPOF.

Memory Bottlenecks

• Cache size limits — Redis runs out of memory, evicts important data
• Large object allocations — Loading entire datasets into memory
• Memory leaks — Gradual degradation over time
• Garbage collection pauses — Java/Go apps pausing during GC

CPU Bottlenecks

• Complex computations — Real-time analytics, ML inference
• Encryption/decryption — TLS termination on every request
• JSON serialization — Large payloads, many fields
• Image/video processing — Thumbnail generation, transcoding

Amazon's Expectation: Proactive vs Reactive

Reactive (bad): Wait for interviewer to ask "What about scalability?"

Proactive (good): "I'm noticing a potential bottleneck here — our single database instance will become a problem at scale. Let me address that."

Amazon interviewers specifically evaluate:

1. Do you identify bottlenecks before being asked?
2. Can you quantify the impact (latency, throughput, availability)?
3. Do you prioritize bottlenecks by severity?
4. Can you propose solutions with trade-offs?

The 4-Step Bottleneck Analysis Framework

Use this framework every time you discuss bottlenecks in an interview:

┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│  1. IDENTIFY │───►│  2. MEASURE  │───►│  3. PRIORITIZE│───►│  4. SOLVE   │
│  What is the │    │  How bad is  │    │  What matters │    │  How to fix │
│  bottleneck? │    │  it?         │    │  most?        │    │  it?        │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘

Step 1: Identify the Bottleneck

Questions to ask yourself:

• Where does data flow through a single component?
• Which component has the highest utilization?
• What fails first under load?
• Are there any single points of failure?

Identification techniques:

1. Follow the data flow — Trace a request from client to database. The slowest component is the bottleneck.
2. Check for fan-in/fan-out — Many requests hitting one component = bottleneck
3. Look for single instances — Anything with no redundancy is a potential SPOF
4. Check connection limits — Databases, caches, and message queues have hard limits
5. Consider scale — What works at 1K users fails at 1M users

Example identification:

System: Social media feed

Data flow:
Client → API Gateway → Feed Service → User DB → Response
                     → Post DB → Response
                     → Cache → Response

Bottleneck candidates:
- Feed Service: Single instance? → SPOF
- Post DB: Millions of posts, high read volume → DB bottleneck
- Cache: What if cache misses? → DB gets hammered

Step 2: Measure the Impact

Don't just say "it's slow" — quantify it:

Metric	What It Tells You	How to Estimate
Latency	How long a request takes	P50, P95, P99 response times
Throughput	How many requests per second	QPS (queries per second)
Error Rate	What percentage fails	5xx errors, timeouts
Availability	Uptime percentage	99.9% = 8.76 hours downtime/year
Capacity	Maximum load before failure	Peak QPS before degradation

Example measurement:

Current system: Single PostgreSQL instance

Measurement:
- Reads: 5,000 QPS (hitting read limit)
- Writes: 1,000 QPS (approaching write limit)
- Latency P99: 200ms (target: <100ms)
- Connection pool: 180/200 (90% utilized)
- Availability: 99.9% (single region, no failover)

Impact: At 2x traffic, system will fail

Step 3: Prioritize by Severity

Not all bottlenecks matter equally. Use this priority matrix:

Priority	Criteria	Example
P0 - Critical	System fails completely, no workaround	Single database with no replica
P1 - High	Significant degradation, partial outage	Cache failure causing DB overload
P2 - Medium	Performance issues, not yet failing	Slow queries at current scale
P3 - Low	Future concern, not urgent now	Memory growth over months

Prioritization framework:

1. Severity — What's the impact if this fails?
2. Likelihood — How likely is this to fail?
3. Urgency — When will this become a problem?

Example prioritization:

Bottlenecks identified:
1. Single database (P0) — System fails if DB goes down
2. Cache eviction (P1) — Slowdown during cache miss storms
3. Cross-region latency (P2) — 80ms added for EU users
4. Memory growth (P3) — Won't be a problem for 6 months

Priority order: Fix database first, then cache, then latency

Step 4: Propose Solutions with Trade-offs

For each solution, explain:

• What it solves
• What trade-offs it introduces
• What complexity it adds

Solution categories:

Solution	Bottleneck Addressed	Trade-off
Read replicas	DB read throughput	Eventual consistency
Sharding	DB write throughput	Cross-shard queries
Caching	Latency, DB load	Cache invalidation complexity
Async processing	Latency, throughput	Eventual consistency
Connection pooling	Connection limits	Pool management overhead
Load balancing	Single server SPOF	Session affinity complexity
CDNs	Network latency	Stale content risk
Rate limiting	Traffic spikes	User experience degradation

Putting It All Together: Example Analysis

System: Real-time chat application (WhatsApp-like)

Step 1 - Identify:
- Message Service handles all chat messages
- Single PostgreSQL database for all messages
- WebSocket connections from 1M concurrent users

Step 2 - Measure:
- Messages: 10,000/sec write, 50,000/sec read
- Database: 100GB data, growing 10GB/day
- Connections: 1M WebSocket, 200 DB connections
- Latency P99: 150ms (target: <50ms)

Step 3 - Prioritize:
- P0: Single database (SPOF + write bottleneck)
- P1: Connection limits (200 DB connections for 1M users)
- P2: Cross-region latency (EU users get 80ms extra)

Step 4 - Solve:
1. Shard DB by chatId → distributes writes, solves SPOF
2. Add connection pooler (PgBouncer) → solves connection limits
3. Add read replicas → distributes read load
4. Add Redis cache for recent messages → reduces DB reads
5. Add CDN for static assets → reduces latency

Trade-offs:
- Sharding: Cross-shard queries need application-level joins
- Read replicas: Eventual consistency (slave lag)
- Cache: Cache invalidation complexity on new messages

Solution 1: Caching

When to use:

• Read-heavy workloads (80%+ reads)
• Data that's expensive to compute or fetch
• Data that can tolerate staleness
• Repeated access to the same data

Cache strategies:

Strategy	How It Works	Trade-off
Cache-Aside	App checks cache first, falls back to DB	Cache miss = extra DB call
Read-Through	Cache fetches from DB on miss	First access is slow
Write-Through	Writes go to cache and DB simultaneously	Write latency increases
Write-Behind	Writes go to cache, async to DB	Risk of data loss

Cache invalidation approaches:

• TTL (Time-to-Live) — Data expires after N seconds
• Event-based — Invalidate on write events
• Version-based — Cache key includes version number

Amazon cache solutions:

• ElastiCache (Redis/Memcached) — In-memory caching
• DAX (DynamoDB Accelerator) — DynamoDB-specific caching
• CloudFront — CDN caching for static assets

Cache pitfalls to mention:

• Cache stampede — Many requests hit DB simultaneously when cache expires
• Hot key problem — One cache key gets disproportionate traffic
• Inconsistency — Stale data served from cache

Interview example:

Problem: Product catalog has 1M items, 100K QPS reads

Solution: Redis cache with Cache-Aside pattern
- Cache top 10% of products (covers 90% of traffic)
- TTL = 5 minutes
- On miss: fetch from DB, populate cache

Result: 90% cache hit rate, DB load reduced 10x
Trade-off: 5% of requests see stale data (acceptable for catalog)

Solution 2: Read Replicas

When to use:

• Read-heavy workloads
• Single database can't handle read throughput
• Data can tolerate slight staleness (seconds)

Architecture:

Write Path:  App ──► Primary DB ──► Replicated to replicas
Read Path:   App ──► Read Replica 1
             App ──► Read Replica 2
             App ──► Read Replica 3

Amazon read replica options:

• RDS Read Replicas (MySQL, PostgreSQL, MariaDB)
• DynamoDB Global Tables (multi-region, multi-active)
• Aurora Serverless v2 (auto-scaling replicas)

Consistency considerations:

• Replication lag: typically 10-100ms
• Read-your-writes consistency: Use primary for recent writes
• Eventual consistency: Acceptable for most read workloads

Replica routing strategies:

• Round-robin across replicas
• Load-based (route to least loaded)
• Consistent hashing (same user → same replica)

Solution 3: Database Sharding

When to use:

• Write-heavy workloads
• Single database can't handle write throughput
• Data too large for single instance (TB+ range)

Sharding strategies:

Strategy	How It Works	Best For
Hash-based	hash(userId) % numShards	Even distribution
Range-based	Shard by ID range (A-F, G-M, etc.)	Range queries
Directory-based	Lookup table maps keys to shards	Flexible rebalancing

Amazon sharding approaches:

• DynamoDB (automatic partitioning by partition key)
• Aurora Global Database (manual sharding by shard key)
• Application-level sharding with multiple RDS instances

Sharding challenges to mention:

• Cross-shard queries — Joins across shards require application logic
• Hot shards — Uneven data distribution
• Rebalancing — Adding/removing shards requires data migration
• Distributed transactions — ACID across shards is complex

Interview example:

Problem: Chat app with 1B messages/day, single DB overwhelmed

Solution: Shard by chatId
- 64 shards, each handling ~15M messages/day
- hash(chatId) % 64 determines shard
- Each shard: 1 primary + 2 read replicas

Result: 64x write capacity, linear scaling
Trade-off: Cross-chat queries require scatter-gather

Solution 4: Async Processing

When to use:

• Operations that don't need immediate response
• Long-running tasks (image processing, email sending)
• Decoupling services
• Buffering traffic spikes

Amazon async services:

• SQS (Simple Queue Service) — FIFO or standard queues
• SNS (Simple Notification Service) — Pub/sub messaging
• EventBridge — Event-driven architecture
• Step Functions — Workflow orchestration

Async patterns:

Pattern	Use Case	Example
Task Queue	Background jobs	Image processing, email sending
Pub/Sub	Event distribution	Order events → Inventory, Shipping, Billing
Event Sourcing	Audit trail	Every state change stored as event
CQRS	Read/write separation	Separate read model from write model

Interview example:

Problem: User registers, needs welcome email, profile setup, analytics

Synchronous (bad): Registration takes 3 seconds (all 3 tasks block)

Async (good):
1. User registers → immediate response (<100ms)
2. Publish 'user.created' event to SNS
3. Email service subscribes → sends welcome email
4. Profile service subscribes → creates default profile
5. Analytics service subscribes → records signup event

Result: Registration is fast, tasks complete in background

Solution 5: Connection Pooling

When to use:

• Database connection limit reached
• Many short-lived connections
• Connection overhead is significant

How it works:

Without pooling: Each request opens new DB connection (expensive)
With pooling: Shared pool of N connections, reused across requests

App → Connection Pool (200 connections) → Database

Amazon connection pooling options:

• RDS Proxy — AWS-managed connection pooler
• PgBouncer — PostgreSQL connection pooler
• HikariCP — Java connection pool (application-level)

Pool sizing guidelines:

• Too few: Requests wait for connections
• Too many: Database runs out of connections
• Rule of thumb: (cores * 2) + effective_spindle_count

Solution 6: Load Balancing

When to use:

• Single server SPOF
• Need horizontal scaling
• Need health checking and failover

Amazon load balancer types:

Type	Use Case	Protocol
ALB (Application)	HTTP/HTTPS, WebSockets	L7
NLB (Network)	TCP/UDP, extreme performance	L4
CLB (Classic)	Legacy applications	L4/L7

Load balancing algorithms:

• Round-robin — Equal distribution
• Least connections — Route to least busy server
• IP hash — Sticky sessions (same client → same server)
• Weighted — Route more traffic to more powerful servers

Solution 7: Rate Limiting

When to use:

• Protect against traffic spikes
• Prevent abuse
• Ensure fair resource usage

Rate limiting strategies:

Strategy	How It Works	Best For
Token Bucket	Tokens refill at fixed rate	Burst-friendly
Leaky Bucket	Requests process at fixed rate	Smooth output
Fixed Window	Count requests in time window	Simple implementation
Sliding Window	Count in rolling window	More accurate

Amazon rate limiting:

• API Gateway throttling (per client, per method)
• WAF rate-based rules
• Application-level with Redis counters

How to Discuss Bottlenecks in an Interview

The Amazon-approved approach:

1. Be proactive — Identify bottlenecks before being asked
2. Quantify impact — Use numbers (latency, QPS, error rates)
3. Prioritize — Address critical issues first
4. Show trade-offs — Every solution has costs
5. Reference real tools — Mention specific AWS services

Example dialogue:

Interviewer: "Tell me about your design."

You: "Let me walk through the data flow. The client sends a request
       to the API Gateway, which routes to the Order Service...

       I'm already seeing a potential bottleneck: our single
       PostgreSQL instance will struggle at 10K QPS writes.
       Let me address that.

       I'll add read replicas for the 80% read traffic, reducing
       load on the primary by 8x. For writes, I'll implement
       connection pooling with RDS Proxy to handle the connection
       limit. The trade-off is eventual consistency for reads,
       which is acceptable for order status.

       At higher scale, I'd shard the orders table by customerId
       to distribute writes across multiple database instances."

Common Bottleneck Solutions Cheatsheet

Bottleneck	Quick Fix	Better Fix	Best Fix
DB reads slow	Add indexes	Read replicas	Cache + replicas
DB writes slow	Optimize queries	Connection pooling	Sharding
Single server	Add load balancer	Auto-scaling group	Multi-region
Network latency	Compress responses	CDN	Edge computing
Cache miss storm	Increase TTL	Warm cache	Request coalescing
Connection limit	Increase limit	Connection pooling	RDS Proxy