Timeouts

Setting Timeouts

Timeouts prevent indefinite waiting for responses that may never come.

Why Timeouts?

Without Timeout:

Client → Request → Server (down)
Client waits forever...

With Timeout:

Client → Request → Server (down)
Client waits 5s → Timeout error
Client continues...

Setting Timeout Values

# HTTP timeout
import requests

# Connection timeout + Read timeout
response = requests.get(
    url,
    timeout=(3.0, 10.0)  # (connect, read)
)

# gRPC timeout
channel = grpc.insecure_channel(
    'api:50051',
    options=[('grpc.keepalive_time_ms', 10000)]
)

# Database timeout
db = Database(
    host='db-host',
    connect_timeout=5,
    read_timeout=30
)

Timeout Guidelines

Service Type	Recommended Timeout
Internal API	1-5 seconds
External API	5-30 seconds
Database query	1-10 seconds
File upload	30-300 seconds
Batch job	300-3600 seconds

Timeout Hierarchy

Client timeout > Server timeout

Client: 10s
  → API Gateway: 8s
    → Service A: 5s
      → Service B: 3s
      → Database: 2s

Each layer shorter than parent

Configuration

# Application config
timeouts:
  http:
    connect: 3s
    read: 10s
    write: 10s
  grpc:
    deadline: 5s
  database:
    connect: 5s
    query: 30s

Cascading Failures

The Problem

Cascading Failure:

Service A → Service B (slow) → Service C (down)
    ↓            ↓                ↓
Timeout    Timeout           Timeout
    ↓            ↓                ↓
Threads    Threads            Threads
exhausted  exhausted          exhausted
    ↓
Service A crashes!

One slow service takes down everything!

Prevention

class TimeoutManager:
    def __init__(self):
        self.timeouts = {}
    
    def get_timeout(self, service, operation):
        """Get appropriate timeout"""
        key = f"{service}:{operation}"
        return self.timeouts.get(key, self.default_timeout)
    
    def set_timeout(self, service, operation, timeout):
        key = f"{service}:{operation}"
        self.timeouts[key] = timeout

# Use timeouts consistently
def call_service(service, operation, data):
    timeout = timeout_manager.get_timeout(service, operation)
    try:
        return service.call(operation, data, timeout=timeout)
    except TimeoutError:
        # Handle timeout gracefully
        return fallback_response()

Timeout + Retry + Circuit Breaker

Protection Layers:

1. Timeout: Prevent indefinite wait
2. Retry: Handle transient failures
3. Circuit Breaker: Stop calling failing service

Together they prevent cascading failures

Monitoring

def monitor_timeouts():
    metrics = {
        'timeout_rate': get_timeout_percentage(),
        'timeout_latency': get_timeout_latency(),
        'cascading_risk': calculate_cascading_risk()
    }
    
    if metrics['timeout_rate'] > 0.1:  # 10%
        alert('High timeout rate - potential cascading failure')

Best Practices

1. Set timeouts on all external calls
2. Use timeout hierarchy (client > server)
3. Monitor timeout rates
4. Combine with circuit breaker
5. Have fallback responses

Timeout Patterns

Simple Timeout

import signal

class TimeoutError(Exception):
    pass

def timeout_handler(signum, frame):
    raise TimeoutError('Operation timed out')

def with_timeout(func, timeout_seconds):
    signal.signal(signal.SIGALRM, timeout_handler)
    signal.alarm(timeout_seconds)
    try:
        result = func()
    finally:
        signal.alarm(0)
    return result

Deadline Propagation

class DeadlinePropagator:
    def __init__(self):
        self.deadline = None
    
    def set_deadline(self, deadline):
        self.deadline = deadline
    
    def get_remaining_time(self):
        if self.deadline is None:
            return None
        return max(0, self.deadline - time.time())
    
    def propagate(self, context):
        """Propagate deadline to downstream calls"""
        remaining = self.get_remaining_time()
        if remaining is not None:
            context.deadline = remaining

Adaptive Timeouts

class AdaptiveTimeout:
    def __init__(self, initial_timeout=5.0):
        self.timeout = initial_timeout
        self.history = []
    
    def record(self, latency, success):
        self.history.append((latency, success))
        if len(self.history) > 100:
            self.history.pop(0)
        
        # Adjust timeout based on history
        if len(self.history) >= 10:
            latencies = [h[0] for h in self.history[-10:]]
            p95 = sorted(latencies)[int(len(latencies) * 0.95)]
            self.timeout = p95 * 1.5  # 1.5x P95 latency
    
    def get_timeout(self):
        return self.timeout

Timeout Patterns Summary

Pattern	Description	Use Case
Fixed	Constant timeout	Stable services
Adaptive	Adjusts based on history	Variable latency
Deadline	Absolute time limit	Multi-hop calls
Cascading	Decreasing timeouts	Nested calls

Best Practices

1. Use deadline propagation for nested calls
2. Monitor p95/p99 latency to set timeouts
3. Use adaptive timeouts for variable workloads
4. Have timeout budget for call chains
5. Test timeout behavior under failure