Load Balancer

Different algorithms distribute traffic based on different criteria.

Algorithm Overview

Load Balancing Algorithms
├── Static (no runtime info)
│   ├── Round Robin
│   ├── Weighted Round Robin
│   └── IP Hash
└── Dynamic (uses runtime info)
    ├── Least Connections
    ├── Least Response Time
    └── Resource-Based

Round Robin

Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A (back to start)
Request 5 → Server B

Pros: Simple, even distribution
Cons: Doesn't account for server load or capacity

Weighted Round Robin

Server A: Weight 5 (powerful)
Server B: Weight 3 (medium)
Server C: Weight 2 (small)

Request distribution:
A, A, A, A, A, B, B, B, C, C

Pros: Accounts for different capacities
Cons: Weights are static, don't reflect real-time load

Least Connections

Server A: 5 active connections
Server B: 2 active connections  ← Next request
Server C: 8 active connections

New request → Server B (fewest connections)

Pros: Accounts for real-time load
Cons: Requires tracking connections

Least Response Time

Server A: 200ms avg response, 5 connections
Server B: 50ms avg response, 8 connections
Server C: 100ms avg response, 3 connections

New request → Server B (fastest response)

Pros: Optimizes for user experience
Cons: Requires response time tracking

IP Hash

Hash(Client IP) % Num Servers = Server Index

Client 192.168.1.1 → Hash → Server A
Client 192.168.1.2 → Hash → Server B
Client 192.168.1.1 → Hash → Server A (same)

Pros: Session affinity (same client → same server)
Cons: Uneven distribution if IPs cluster

Algorithm Selection

Algorithm	Best For	Use Case
Round Robin	Equal-capacity servers	Simple workloads
Weighted RR	Mixed-capacity servers	Heterogeneous fleet
Least Connections	Long-lived connections	WebSocket, databases
Least Response Time	Latency-sensitive	API servers
IP Hash	Session affinity	Stateful applications

Health checks ensure traffic only goes to healthy servers.

Health Check Types

1. Passive Health Checks
   - Monitor actual requests
   - Mark server down after failures
   - No extra traffic

2. Active Health Checks
   - Periodic probe requests
   - Detect issues before requests
   - Additional traffic overhead

3. Hybrid
   - Active checks + passive monitoring
   - Most reliable

Health Check Configuration

HealthCheck:
  Protocol: HTTP
  Path: /health
  Port: 8080
  Interval: 10 seconds        # How often to check
  Timeout: 5 seconds          # Max wait for response
  UnhealthyThreshold: 3       # Failures before marking down
  HealthyThreshold: 2         # Successes before marking up

Health Check Response

// Healthy
GET /health
200 OK
{
  "status": "healthy",
  "database": "connected",
  "cache": "connected",
  "uptime": 86400
}

// Unhealthy
GET /health
503 Service Unavailable
{
  "status": "unhealthy",
  "database": "disconnected"
}

Health Check Best Practices

1. Separate health endpoint: Don't use main app endpoint
2. Check dependencies: Database, cache, external services
3. Return meaningful status: Include component health
4. Set appropriate thresholds: Avoid flapping
5. Log health changes: Monitor for issues

Health Check States

Server States:

Unknown → Initial state
    │
    ▼
Healthy ←──────┐
    │           │
    │ (fails)   │ (recovers)
    ▼           │
Unhealthy ─────┘

Flapping prevention:
- Require N failures before marking unhealthy
- Require N successes before marking healthy

SSL termination offloads encryption/decryption to the load balancer.

How SSL Termination Works

Without SSL Termination:
Client ←──HTTPS──→ Server (handles SSL)

With SSL Termination:
Client ←──HTTPS──→ Load Balancer ←──HTTP──→ Server

Load balancer handles:
- SSL handshake
- Certificate management
- Encryption/Decryption

Backend servers receive plain HTTP

SSL Termination Benefits

1. Performance
   - Dedicated SSL hardware
   - Connection pooling
   - Reduce backend CPU usage

2. Certificate Management
   - Single point for certificates
   - Easier renewal
   - Centralized control

3. Security
   - Backend not exposed to SSL attacks
   - Centralized TLS policy

4. Flexibility
   - Backend can be HTTP
   - Easier debugging

SSL Configuration

server {
    listen 443 ssl;
    server_name example.com;

    ssl_certificate /etc/ssl/certs/example.com.pem;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # HSTS
    add_header Strict-Transport-Security "max-age=31536000" always;

    location / {
        proxy_pass http://backend;
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    return 301 https://$host$request_uri;
}

SSL Passthrough vs Termination

Aspect	SSL Termination	SSL Passthrough
SSL handling	At load balancer	At backend server
Backend protocol	HTTP	HTTPS
Performance	Better (offloaded)	Worse (backend handles)
Certificate mgmt	Centralized	Distributed
End-to-end encryption	No	Yes

When to Use Each

SSL Termination (default):
- Most web applications
- APIs with internal trust
- Performance-critical

SSL Passthrough:
- Compliance requirements (end-to-end encryption)
- Untrusted internal network
- Financial/medical systems