Common Probability Distributions

The Simplest Distribution

A Bernoulli random variable models a single binary trial: success (1) with probability p, failure (0) with probability 1-p. It is the building block for every binary metric in product analytics.

P(X = 1) = p
P(X = 0) = 1 - p
E[X] = p
Var(X) = p(1 - p)

The variance p(1-p) is maximized at p = 0.5 and minimized near p = 0 or p = 1. This has a direct practical consequence: metrics with extreme rates (very high or very low) have lower variance and are easier to estimate than metrics near 50%.

Binary Metrics in Product Analytics

Every conversion event is a Bernoulli trial:

• Did the user sign up? (yes/no)
• Did the user click the button? (yes/no)
• Did the user churn this month? (yes/no)
• Did the session exceed 30 seconds? (yes/no)

When you compute a conversion rate, you are estimating the parameter p of a Bernoulli distribution. The sample proportion p̂ = (number of successes) / (number of trials) is the maximum likelihood estimator.

The Variance of a Conversion Rate

The variance of the sample proportion is:

Var(p̂) = p(1 - p) / n

This is the foundation of conversion rate confidence intervals. For p = 0.05 and n = 1,000:

SE = sqrt(0.05 * 0.95 / 1000) = sqrt(0.0000475) = 0.0069

The 95% confidence interval is approximately 0.05 ± 2 * 0.0069 = [0.036, 0.064]. This tells you the precision of your estimate and whether observed differences between variants are statistically meaningful.

import numpy as np
from scipy import stats

p = 0.05
n = 1000
se = np.sqrt(p * (1 - p) / n)
ci = stats.norm.interval(0.95, loc=p, scale=se)
print(f'95% CI: [{ci[0]:.4f}, {ci[1]:.4f}]')

When the Bernoulli Model Breaks

The Bernoulli model assumes each trial is independent with the same probability p. This assumption fails when:

1. Users influence each other. Viral features create correlated conversions. One user's conversion makes another user's conversion more likely. The effective sample size is smaller than the raw count.

2. The probability changes over time. A holiday sale increases conversion probability during the sale period. Pooling sale and non-sale data violates the constant-p assumption.

3. Trials are not identical. Users from different channels have different conversion probabilities. The overall rate is a mixture of channel-specific rates.

When independence fails, the true variance exceeds p(1-p)/n, and your confidence intervals are too narrow. This is why experiments with network effects require specialized analysis (cluster randomization, switchback designs) rather than simple Bernoulli models.

From Single Trials to Counts

If you perform n independent Bernoulli trials, the number of successes follows a binomial distribution:

P(X = k) = C(n, k) * p^k * (1-p)^(n-k)
E[X] = n*p
Var(X) = n*p*(1-p)

The binomial distribution answers: out of n users, how many will convert? It models counts, not just rates.

Binomial Metrics in Product Analytics

• Out of 1,000 new users, how many will activate within 7 days?
• Out of 500 sessions, how many will result in a purchase?
• Out of 200 trial users, how many will convert to paid?

These are all binomial questions. The count of successes in n independent trials with probability p.

Computing Binomial Probabilities

from scipy import stats

n = 1000
p = 0.05

# Probability of exactly 50 conversions
prob_50 = stats.binom.pmf(50, n, p)

# Probability of 60 or more conversions
prob_60_plus = 1 - stats.binom.cdf(59, n, p)

# 95% interval for the number of conversions
ci = stats.binom.interval(0.95, n, p)
print(f'P(X = 50) = {prob_50:.4f}')
print(f'P(X >= 60) = {prob_60_plus:.4f}')
print(f'95% interval: [{ci[0]:.0f}, {ci[1]:.0f}]')

The pmf (probability mass function) gives the probability of exactly k successes. The cdf (cumulative distribution function) gives the probability of k or fewer successes. The interval function gives a central interval containing 95% of the probability mass.

The Normal Approximation to the Binomial

When n is large and p is not too close to 0 or 1, the binomial distribution is well-approximated by a normal distribution:

Binomial(n, p) ≈ Normal(μ = np, σ² = np(1-p))

This is the de Moivre-Laplace theorem. It allows you to use the simpler normal distribution for calculations when the binomial is computationally inconvenient.

The approximation works well when np ≥ 10 and n(1-p) ≥ 10. For p = 0.05, you need n ≥ 200. For p = 0.5, you need n ≥ 20.

from scipy import stats
import numpy as np

n, p = 1000, 0.05
mu = n * p
sigma = np.sqrt(n * p * (1 - p))

# Exact binomial: P(X >= 60)
exact = 1 - stats.binom.cdf(59, n, p)

# Normal approximation with continuity correction
approx = 1 - stats.norm.cdf(59.5, loc=mu, scale=sigma)

print(f'Exact: {exact:.4f}, Approx: {approx:.4f}')

The continuity correction (using 59.5 instead of 60) improves the approximation by accounting for the discrete nature of the binomial.

When the Normal Approximation Fails

The approximation fails when:

1. p is very small. For p = 0.001 and n = 100, np = 0.1. The binomial is heavily skewed; the normal is symmetric. Use the Poisson approximation instead.

2. n is small. With n = 10, the binomial has only 11 possible values. A continuous approximation is crude.

3. You need tail probabilities. The normal approximation is least accurate in the tails, which is exactly where anomaly detection operates.

The rule of thumb: check np and n(1-p). If both exceed 10, the normal approximation is reasonable for central probabilities. For tail probabilities, require both to exceed 20 or use the exact binomial.

Why the Normal Distribution Appears Everywhere

The normal (Gaussian) distribution is the most important distribution in statistics because of the Central Limit Theorem: the sum of many independent random variables, regardless of their individual distributions, converges to a normal distribution.

This is why sample means are normally distributed even when the underlying data is not. Session durations are right-skewed, but the mean session duration across 1,000 sessions is approximately normal. Revenue per user is heavy-tailed, but the average revenue across 10,000 users is approximately normal.

f(x) = (1 / (σ * sqrt(2π))) * exp(-(x - μ)² / (2σ²))

The distribution is characterized by two parameters: μ (mean) and σ (standard deviation). It is symmetric, bell-shaped, and fully described by its first two moments.

The 68-95-99.7 Rule

For a normal distribution:

• 68% of values fall within μ ± σ
• 95% of values fall within μ ± 2σ
• 99.7% of values fall within μ ± 3σ

This rule lets you reason about probabilities without computation. If daily active users have a mean of 50,000 and a standard deviation of 5,000, then:

• 68% of days have between 45,000 and 55,000 DAU
• 95% of days have between 40,000 and 60,000 DAU
• 99.7% of days have between 35,000 and 65,000 DAU

A day with 70,000 DAU is a 4-sigma event — it should occur less than 0.01% of the time under normal conditions. If it happens, something structural changed.

Using the Normal Distribution for Alert Thresholds

Control charts use the normal distribution to set alert bounds. A metric with mean μ and standard deviation σ triggers an alert when it exceeds μ ± 3σ:

import pandas as pd
import numpy as np

df = pd.read_csv('daily_dau.csv')
mean = df['dau'].mean()
std = df['dau'].std()

df['alert'] = (df['dau'] > mean + 3 * std) | (df['dau'] < mean - 3 * std)
print(f'Alert threshold: [{mean - 3*std:.0f}, {mean + 3*std:.0f}]')
print(f'Alerts: {df["alert"].sum()}')

The 3-sigma threshold produces a false positive rate of 0.3% under the normal assumption. If the metric is not normally distributed (e.g., DAU is right-skewed), the actual false positive rate may be higher in the upper tail.

Confidence Intervals for Normally Distributed Metrics

For a sample mean with known standard deviation σ:

CI = x̄ ± z * σ / sqrt(n)

For a 95% confidence interval, z = 1.96. This is the interval you report when you say "conversion rate is 5.2% ± 0.3%."

from scipy import stats
import numpy as np

samples = np.random.normal(5.2, 1.5, size=1000)  # simulated metric
mean = samples.mean()
se = samples.std(ddof=1) / np.sqrt(len(samples))
ci = stats.t.interval(0.95, len(samples) - 1, loc=mean, scale=se)

print(f'Mean: {mean:.2f}, 95% CI: [{ci[0]:.2f}, {ci[1]:.2f}]')

The t-distribution is used instead of the normal when the standard deviation is estimated from the sample. For large samples, the t-distribution converges to the normal.

When the Normal Assumption Fails

The normal assumption fails for:

1. Heavy-tailed metrics. Revenue per user, session duration, and time-to-purchase have long right tails. The normal distribution underestimates the probability of extreme values.

2. Bounded metrics. Conversion rates are bounded in [0, 1]. The normal distribution assigns probability to impossible values (negative rates, rates above 100%).

3. Zero-inflated metrics. Many users have zero revenue; a few have large revenue. The distribution is a spike at zero plus a continuous tail — not normal.

For these metrics, use the exact distribution (binomial for rates, log-normal for multiplicative metrics) or non-parametric methods (bootstrap confidence intervals) that do not assume normality.

The Poisson Distribution: Events in Time

The Poisson distribution models the number of events occurring in a fixed interval of time or space, given that events occur independently at a constant average rate λ:

P(X = k) = λ^k * e^(-λ) / k!
E[X] = λ
Var(X) = λ

The mean equals the variance — this is a defining property of the Poisson distribution. If the variance of your count data exceeds the mean (overdispersion), the Poisson model is inadequate.

Poisson Metrics in Product Analytics

• Number of support tickets per day
• Number of app crashes per 1,000 sessions
• Number of purchases per hour
• Number of new signups per week

All of these are counts of rare events over a fixed interval. If events occur independently at a constant rate, the count is Poisson-distributed.

from scipy import stats

lambda_ = 3.5  # average tickets per day

# Probability of exactly 5 tickets
prob_5 = stats.poisson.pmf(5, lambda_)

# Probability of 8 or more tickets (unusually high)
prob_8_plus = 1 - stats.poisson.cdf(7, lambda_)

print(f'P(X = 5) = {prob_5:.4f}')
print(f'P(X >= 8) = {prob_8_plus:.4f}')

The Poisson Process and Its Implications

The Poisson distribution arises from the Poisson process, which assumes:

1. Independence. Events do not influence each other.
2. Constant rate. The expected number of events per unit time is constant.
3. No simultaneous events. Two events cannot occur at exactly the same instant.

When these assumptions hold, the number of events in any interval is Poisson-distributed with mean λt, where λ is the rate and t is the interval length.

The assumptions break when events cluster. A server crash causes a burst of error events — they are not independent. A marketing campaign causes a surge in signups — the rate is not constant. In these cases, use a more flexible model (negative binomial for overdispersed counts, or a time-varying rate model).

The Exponential Distribution: Time Between Events

If events follow a Poisson process with rate λ, the time between consecutive events follows an exponential distribution:

f(t) = λ * e^(-λt)  for t >= 0
E[T] = 1/λ
Var(T) = 1/λ²

The exponential distribution models wait times: time until the next purchase, time until the next crash, time until the next support ticket.

Exponential Metrics in Product Analytics

• Time between a user's first and second session
• Time from signup to first purchase
• Time between consecutive orders
• Time until a user churns (survival analysis)

from scipy import stats
import numpy as np

lambda_ = 0.2  # average 5 days between purchases (1/0.2)

# Probability that a user purchases within 3 days
prob_within_3 = 1 - np.exp(-lambda_ * 3)

# Median time to purchase
median = np.log(2) / lambda_

print(f'P(purchase within 3 days) = {prob_within_3:.4f}')
print(f'Median time to purchase: {median:.1f} days')

The Memoryless Property

The exponential distribution is memoryless: the probability of an event occurring in the next t time units does not depend on how long you have already waited.

P(T > s + t | T > s) = P(T > t)

For product analytics, this means: if a user has not purchased in 30 days, the probability they purchase in the next 7 days is the same as for a user who signed up 7 days ago. This is rarely true — users who have been active for 30 days are typically more engaged than new users. When the memoryless property fails, use a Weibull distribution, which generalizes the exponential with a shape parameter that captures increasing or decreasing hazard rates.

Choosing the Right Distribution

The decision framework:

1. Binary outcome (yes/no)? → Bernoulli
2. Count of successes in n trials? → Binomial
3. Count of rare events in fixed interval? → Poisson
4. Continuous, symmetric, unbounded? → Normal
5. Time between independent events? → Exponential
6. None of the above fit? → Use empirical distributions or bootstrap methods

The choice is not aesthetic — it determines the validity of your confidence intervals, alert thresholds, and sample size calculations. A wrong distributional assumption produces wrong conclusions even with perfect data.