Logging & Structured Logs

Levels, Loggers, and Propagation

The logging module uses a numeric severity scale — DEBUG 10, INFO 20, WARNING 30, ERROR 40, CRITICAL 50 — and a dot-separated logger hierarchy: app.api is a child of app, which is a child of the root logger.

import logging
log = logging.getLogger(__name__)   # "package.module", not the root

def handle(req):
    log.debug("payload: %s", req)   # %s stays lazy — no work if DEBUG is off
    if req.get("bad"):
        log.error("rejected %s", req["id"])

A logger with NOTSET level inherits from its nearest ancestor. Messages also propagate up to ancestor handlers unless propagate = False. Effective level comes from the root logger by default, which is WARNING until you configure it.

Handlers, Formatters, and Rotation

Loggers decide whether to record; handlers decide where it goes:

import logging
from logging.handlers import RotatingFileHandler

root = logging.getLogger()
root.setLevel(logging.INFO)

fmt = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s")

console = logging.StreamHandler()
console.setFormatter(fmt)

fileh = RotatingFileHandler("app.log", maxBytes=5_000_000, backupCount=3,
                            encoding="utf-8")
fileh.setFormatter(fmt)
root.addHandler(fileh)

maxBytes + backupCount bound disk usage: the active file rolls to app.log.1, and the oldest backup is deleted. Libraries should only call logging.getLogger(...) and never install handlers — configuration belongs to the application entry point via basicConfig or dictConfig.

Structured Logs and Common Mistakes

Human lines are fine for local dev; production wants machine-parseable events. Emit one JSON object per record with python-json-logger or structlog, adding context fields through extra=:

log.info("checkout.done", extra={"order_id": oid, "ms": elapsed})

Common failure modes:

• print() bypasses levels, formatting, and streams — it disappears when stdout is redirected
• logging.basicConfig() is a no-op if handlers already exist, so "my INFO lines vanished" often means the root logger already had a WARNING handler
• f-strings in log calls format eagerly even when the level is disabled — prefer %s args
• Missing encoding="utf-8" on FileHandler mangles non-ASCII under some locales