Skip to content
Machine Learning & Data Science

Statistics interview questions and answers

75 questions, 75 with code. About 3 h 9 min to read them all.

By TestLab · Updated

Plan for an interview
75 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

What is the difference between descriptive and inferential statistics?

Descriptive statistics summarize and present the data you have: measures of centre (mean, median), spread (standard deviation, IQR), shape (skewness), frequency tables and charts. They make no claims beyond the observed data. Inferential statistics use a sample to draw conclusions about a larger population, accounting for sampling uncertainty: estimating parameters with confidence intervals, testing hypotheses, and building models for prediction. Descriptive work usually comes first, to understand the data and check assumptions, and inference builds on it.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, SciPy 1.13+

Comparison

Aspect Descriptive Inferential
Goal Summarize observed data Generalize to a population
Tools Mean, median, SD, IQR, percentiles, histograms, box plots Confidence intervals, hypothesis tests, regression, Bayesian inference
Uncertainty Not quantified Central (standard errors, p-values)
Requires sampling assumptions No Yes (random, representative samples)
Example “Average order in our sample was 42” “Average order of all customers is 40-44 with 95% confidence”

Code Example

import numpy as np
import pandas as pd
from scipy import stats

rng = np.random.default_rng(42)
df = pd.DataFrame({
    "store": np.repeat(["north", "south"], 150),
    "basket": np.r_[rng.gamma(4, 10, 150), rng.gamma(4, 11.5, 150)],
})

# Descriptive: summarize what was observed
print(df.groupby("store")["basket"].describe().round(2))

# Inferential: generalize beyond the sample
north = df.loc[df.store == "north", "basket"]
south = df.loc[df.store == "south", "basket"]
ci = stats.t.interval(0.95, len(north) - 1, loc=north.mean(), scale=stats.sem(north))
print(f"95% CI for the north store's mean basket: [{ci[0]:.2f}, {ci[1]:.2f}]")
res = stats.ttest_ind(south, north, equal_var=False)
print(f"Welch t-test, south vs north: t = {res.statistic:.2f}, p = {res.pvalue:.4f}")

The describe table is purely descriptive; the confidence interval and the test make claims about all baskets at each store, assuming the sampled baskets are representative.

Common Pitfalls

  • Drawing population conclusions from descriptive statistics of a biased sample.

  • Running inferential tests on data that are already the whole population.

  • Skipping descriptive checks before inference (outliers, skew, data errors).

2

What is the difference between a population and a sample?

A population is the entire group you want to draw conclusions about: all customers, all voters in a country, every product made on a line. Its numerical characteristics, such as the mean μ or proportion p, are parameters, which are usually unknown. A sample is the subset you actually observe, and its summaries, such as x̄ or p̂, are statistics used to estimate the parameters. Because samples vary, estimates come with sampling error, quantified by standard errors and confidence intervals, and they are only trustworthy if the sample is representative, ideally obtained by random sampling.

Read full answer

Applies toPython 3.12, NumPy 2.x

Comparison

Aspect Population Sample
Definition Every unit of interest Observed subset
Measures Parameters (μ, σ, p) Statistics (x̄, s, p̂)
Usually known? No Yes
Variance formula Σ(x − μ)² / N Σ(x − x̄)² / (n − 1)
Error None (if fully measured) Sampling error + possible bias

Sampling Methods

Method Description
Simple random Every unit equally likely
Stratified Random sampling within groups (regions, age bands)
Cluster Randomly choose groups, then measure everyone in them
Systematic Every k-th unit
Convenience Whoever is easy to reach (high bias risk)

Code Example

import numpy as np

rng = np.random.default_rng(42)
population = rng.gamma(2, 20, 1_000_000)          # e.g. all customers' monthly spend
print(f"population mean (parameter) μ = {population.mean():.2f}")

for n in (30, 300, 3000):
    estimates = [rng.choice(population, n, replace=False).mean() for _ in range(500)]
    print(f"n = {n:>4}: average estimate {np.mean(estimates):.2f}, spread (SE) {np.std(estimates):.2f}")

# Biased sampling: only surveying customers who spend more than 20
biased = rng.choice(population[population > 20], 3000, replace=False)
print(f"biased sample mean {biased.mean():.2f} (far from μ even with n = 3000)")

Random samples are centred on the true mean, with spread shrinking as n grows; a biased sample stays wrong however large it is.

Common Pitfalls

  • Generalizing from a convenience or self-selected sample.

  • Believing a very large sample fixes bias.

  • Using N instead of n − 1 when estimating the population variance from a sample.

3

What is a distribution in statistics, and what are common examples?

A distribution describes how the values of a variable are spread: which values occur and how often (for data), or how probability is assigned to possible values (for a random variable). An empirical distribution summarizes observed data (histograms, frequency tables); a probability distribution is a mathematical model with parameters, given by a PMF for discrete variables or a PDF for continuous ones, plus a CDF. Choosing the right distribution lets you compute probabilities, simulate data, build likelihoods and choose appropriate tests.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Common Distributions

Distribution Type Models Example
Bernoulli Discrete One yes/no trial Did a user click?
Binomial Discrete Successes in n trials Conversions out of 1,000 visitors
Poisson Discrete Counts per interval Calls per hour
Geometric Discrete Trials until first success Attempts until a sale
Uniform Continuous Equally likely values Random number in [0, 1]
Normal Continuous Sums/averages, measurement error Heights, test scores
Exponential Continuous Waiting times Time between arrivals
Log-normal Continuous Positive, right-skewed Incomes, file sizes
t, chi-square, F Continuous Sampling distributions of test statistics t-tests, ANOVA
Beta Continuous Probabilities and proportions Bayesian prior for a rate

Code Example: Fitting and Comparing Candidates

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
data = rng.lognormal(mean=3, sigma=0.5, size=2000)          # e.g. order values

candidates = {"normal": stats.norm, "lognormal": stats.lognorm, "gamma": stats.gamma}
for name, dist in candidates.items():
    params = dist.fit(data) if name == "normal" else dist.fit(data, floc=0)
    loglik = np.sum(dist.logpdf(data, *params))
    aic = 2 * len(params) - 2 * loglik
    ks_p = stats.kstest(data, dist.cdf, args=params).pvalue
    print(f"{name:>9}: AIC {aic:9.1f}, KS p-value {ks_p:.3f}")

print("empirical P(value > 40):", round((data > 40).mean(), 4))
shape, loc, scale = stats.lognorm.fit(data, floc=0)
print("lognormal model P(value > 40):", round(stats.lognorm.sf(40, shape, loc, scale), 4))

The lognormal fit has the lowest AIC and a reasonable KS p-value, while the normal fit fails. (KS p-values are optimistic when parameters are estimated from the same data, so use them as a rough guide.)

Common Pitfalls

  • Assuming normality by default for skewed or bounded data.

  • Confusing the distribution of the data with the sampling distribution of a statistic.

  • Choosing a distribution only by a goodness-of-fit p-value; with large samples every model gets rejected.

4

What is the Central Limit Theorem, and why is it important?

The Central Limit Theorem (CLT) states that the distribution of the sample mean of independent, identically distributed observations with finite variance approaches a normal distribution as the sample size grows, whatever the shape of the original distribution. The sampling distribution is centred on the population mean μ with standard deviation σ/√n (the standard error). It is important because it justifies normal-based confidence intervals and tests (z-tests, t-tests, A/B tests on conversion rates) even for skewed data, as long as samples are reasonably large.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Key Points

Point Detail
Applies to Sample means and sums (and many other estimators)
Centre Population mean μ
Spread σ/√n, shrinking with sample size
Sample size needed Often 30+ is enough; strongly skewed or heavy-tailed data need more
Requirements Independence and finite variance
Does not say That the raw data become normal

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
population = rng.exponential(scale=10, size=1_000_000)       # strongly right-skewed
mu, sigma = population.mean(), population.std()
print(f"population skewness: {stats.skew(population):.2f}")

for n in (2, 10, 30, 200):
    means = rng.choice(population, size=(20_000, n)).mean(axis=1)
    print(f"n={n:>3}: mean of sample means {means.mean():6.2f} (μ = {mu:.2f}), "
          f"SD {means.std():5.2f} (σ/√n = {sigma / np.sqrt(n):5.2f}), skew {stats.skew(means):5.2f}")

# Consequence: a normal-based 95% CI for the mean works with n = 50 despite the skew
covered = 0
for _ in range(5000):
    s = rng.exponential(10, 50)
    half = 1.96 * s.std(ddof=1) / np.sqrt(50)
    covered += s.mean() - half <= 10 <= s.mean() + half
print("coverage of 95% CI with n = 50:", covered / 5000)

As n increases, the skewness of the sample means falls toward 0 and their spread matches σ/√n. The coverage check shows normal-based intervals work reasonably well (slightly below 95% with this skewed data).

Common Pitfalls

  • Believing the CLT makes the data themselves normal.

  • Applying it to heavy-tailed distributions without finite variance (e.g. Cauchy).

  • Assuming n = 30 is always enough; strong skew needs larger samples or the bootstrap.

5

What is a p-value, and what does it say about significance?

A p-value is the probability of obtaining a result at least as extreme as the one observed, assuming the null hypothesis is true. A small p-value means the data would be surprising under H₀, which counts as evidence against it; if p ≤ α (commonly 0.05), the result is called statistically significant and H₀ is rejected. A p-value is not the probability that H₀ is true, not the probability the result happened by chance, and says nothing about the size or importance of an effect, which is why it should be reported with an effect size and confidence interval.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

What a p-value Is and Is Not

Correct Incorrect
P(data this extreme | H₀ true) P(H₀ true | data)
A measure of incompatibility with H₀ The probability the result is due to chance
Depends on effect size and sample size A measure of effect size
Uniformly distributed when H₀ is true Evidence for H₀ when large

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# Permutation view: how often does shuffling produce a difference this large?
a = rng.normal(10.6, 2, 40)
b = rng.normal(10.0, 2, 40)
observed = a.mean() - b.mean()
pooled = np.r_[a, b]
diffs = []
for _ in range(10_000):
    perm = rng.permutation(pooled)
    diffs.append(perm[:40].mean() - perm[40:].mean())
p_perm = np.mean(np.abs(diffs) >= abs(observed))
print(f"observed difference {observed:.3f}; permutation p = {p_perm:.4f}; t-test p = {stats.ttest_ind(a, b).pvalue:.4f}")

# Same tiny effect, huge sample: very small p-value, negligible effect
big_a, big_b = rng.normal(10.05, 2, 200_000), rng.normal(10.0, 2, 200_000)
res = stats.ttest_ind(big_a, big_b)
print(f"n = 200,000 per group: difference {big_a.mean() - big_b.mean():.3f}, p = {res.pvalue:.2g}")

# Under H0, p-values are uniform: about 5% fall below 0.05
null_p = [stats.ttest_ind(rng.normal(0, 1, 30), rng.normal(0, 1, 30)).pvalue for _ in range(5000)]
print("share of null p-values below 0.05:", np.mean(np.array(null_p) < 0.05))

The permutation test shows the definition directly: the p-value is the share of “no-difference” worlds producing a difference as large as observed. The large-sample case shows a tiny p-value can accompany a trivial effect.

Common Pitfalls

  • Treating p = 0.049 and p = 0.051 as fundamentally different.

  • Reporting “p < 0.05” without effect sizes.

  • p-hacking: trying many analyses and reporting the one that crosses 0.05.

6

What does statistical power mean?

Statistical power is the probability that a test correctly rejects the null hypothesis when a real effect of a given size exists: power = 1 − β, where β is the Type II (false negative) error rate. It depends on four things: the effect size (bigger effects are easier to detect), the sample size, the significance level α, and the variability of the data. Studies are usually designed for 80% or 90% power through a power analysis before data collection. Low power wastes resources, misses real effects, and makes the significant results it does find likely to exaggerate the true effect.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

What Increases Power

Factor Change Why
Sample size Increase Smaller standard error
Effect size Larger true effect Further from the null
Variability Reduce (better measurement, paired designs, covariates) Less noise
α Increase (e.g. 0.10) Easier to reject, but more false positives
One-sided test When justified Concentrates α in one tail

Code Example

import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower

analysis = TTestIndPower()

# Sample size for 80% power to detect d = 0.4
n = analysis.solve_power(effect_size=0.4, alpha=0.05, power=0.8)
print("per-group n for 80% power, d = 0.4:", int(np.ceil(n)))

# Power at different sample sizes
for n in (20, 50, 100, 200):
    print(f"n = {n:>3} per group: power = {analysis.power(effect_size=0.4, nobs1=n, alpha=0.05):.3f}")

# Smallest detectable effect with 100 per group
mde = analysis.solve_power(nobs1=100, alpha=0.05, power=0.8)
print("minimum detectable effect with n = 100:", round(mde, 3))

# Winner's curse: underpowered significant results overestimate the effect
rng = np.random.default_rng(42)
estimates = []
for _ in range(5000):
    a, b = rng.normal(0.2, 1, 20), rng.normal(0, 1, 20)
    if stats.ttest_ind(a, b).pvalue < 0.05:
        estimates.append(a.mean() - b.mean())
print(f"true effect 0.2; average significant estimate {np.mean(estimates):.2f} ({len(estimates)} of 5000 significant)")

With 20 per group and a true effect of 0.2, few studies reach significance, and those that do overestimate the effect several times over.

Common Pitfalls

  • “Post-hoc power” computed from the observed effect; it adds nothing beyond the p-value.

  • Choosing an unrealistically large effect size to justify a small sample.

  • Ignoring power when interpreting non-significant results.

7

What are Type I and Type II errors in hypothesis testing?

A Type I error (false positive) is rejecting the null hypothesis when it is actually true, like concluding a drug works when it does not; its probability is the significance level α. A Type II error (false negative) is failing to reject the null hypothesis when it is actually false, like missing a drug that really works; its probability is β, and power = 1 − β. For a fixed sample size, lowering α increases β, so the trade-off is set by the costs of each mistake; increasing the sample size reduces β without raising α.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

The Decision Table

H₀ true H₀ false
Reject H₀ Type I error (α) Correct (power = 1 − β)
Fail to reject H₀ Correct (1 − α) Type II error (β)

Which Is Worse?

Context Costlier error Implication
Criminal trial (H₀: innocent) Type I: convicting an innocent person Very small α
Cancer screening (H₀: healthy) Type II: missing cancer High power; follow-up tests handle false positives
Shipping a product change Type I wastes effort; Type II misses gains Balance with business cost

Code Example: Simulating Both Error Rates

import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower

rng = np.random.default_rng(42)
alpha, n, sims = 0.05, 30, 5000

# Type I error rate: no real difference
type1 = np.mean([stats.ttest_ind(rng.normal(0, 1, n), rng.normal(0, 1, n)).pvalue < alpha for _ in range(sims)])

# Type II error rate: real difference of 0.5 SD
type2 = np.mean([stats.ttest_ind(rng.normal(0.5, 1, n), rng.normal(0, 1, n)).pvalue >= alpha for _ in range(sims)])

theory_power = TTestIndPower().power(effect_size=0.5, nobs1=n, alpha=alpha)
print(f"Type I rate {type1:.3f} (α = {alpha})")
print(f"Type II rate {type2:.3f} → power {1 - type2:.3f} (theory {theory_power:.3f})")

for new_alpha in (0.10, 0.05, 0.01):
    p = TTestIndPower().power(effect_size=0.5, nobs1=n, alpha=new_alpha)
    print(f"α = {new_alpha:.2f}: β = {1 - p:.3f}")

The simulated Type I rate is close to α, and lowering α raises β for the same sample, showing the trade-off directly.

Common Pitfalls

  • Thinking α is the probability that a significant result is a false positive (that depends on prior plausibility and power).

  • Running underpowered studies, which have high β and exaggerate effects when they are significant.

  • Multiple testing, which raises the overall Type I error rate.

8

What is the significance level in a hypothesis test, and how is it chosen?

The significance level (α) is the threshold for rejecting the null hypothesis, set before the analysis: if the p-value is at or below α, the result is called significant. It equals the Type I error rate, the probability of rejecting H₀ when it is actually true, so α = 0.05 means accepting a 5% false-positive rate when there is no effect. It is chosen by weighing the cost of false positives against false negatives: lower α (0.01, 0.001, or 5σ in particle physics) when false positives are expensive, higher α (0.10) in exploratory work when missing an effect is worse, adjusted for multiple tests.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Typical Choices

α Typical context
0.10 Exploratory analysis, pilot studies
0.05 Default in many fields, A/B tests
0.01 Medical and regulatory decisions, strong claims
0.005 Proposed for new discoveries in some sciences
≈ 3 × 10⁻⁷ (5σ) Particle physics discoveries
5 × 10⁻⁸ Genome-wide association studies (many tests)

Factors in Choosing α

  • Cost of a false positive vs a false negative.

  • Number of tests (family-wise error or false discovery rate control).

  • Sample size and power: lower α needs more data for the same power.

  • Field conventions and regulatory requirements.

Code Example: α, Power and Multiple Testing

import numpy as np
from statsmodels.stats.multitest import multipletests
from statsmodels.stats.power import TTestIndPower

for alpha in (0.10, 0.05, 0.01, 0.001):
    n = TTestIndPower().solve_power(effect_size=0.3, alpha=alpha, power=0.8)
    print(f"α = {alpha:<5}: n per group for 80% power at d = 0.3 → {int(np.ceil(n))}")

# 20 independent tests at α = 0.05: chance of at least one false positive
print("family-wise error rate for 20 tests:", round(1 - 0.95 ** 20, 3))

rng = np.random.default_rng(42)
p_values = np.r_[rng.uniform(0, 1, 18), [0.001, 0.004]]       # 18 nulls, 2 real effects
for method in ("bonferroni", "fdr_bh"):
    reject, adjusted, _, _ = multipletests(p_values, alpha=0.05, method=method)
    print(f"{method:>10}: {reject.sum()} rejections")
print("unadjusted:", (p_values < 0.05).sum(), "rejections")

Stricter α demands bigger samples, and running many tests at α = 0.05 makes at least one false positive likely (about 64% for 20 tests), which is what corrections such as Bonferroni and Benjamini-Hochberg address.

Common Pitfalls

  • Choosing or changing α after seeing the p-value.

  • Treating 0.05 as a law of nature rather than a convention.

  • Ignoring multiplicity across metrics, segments and time points.

9

What is a confidence interval, and why is it important?

A confidence interval (CI) is a range of values, computed from sample data, that is likely to contain an unknown population parameter such as a mean or a difference in proportions. A 95% CI comes from a procedure that, over many repeated samples, captures the true parameter 95% of the time; any single interval either contains it or does not. CIs matter because they show both the estimate and its precision: a narrow interval means a precise estimate, a wide one means uncertainty, and an interval for a difference that excludes zero corresponds to a significant result at the matching α, while also showing whether the effect is practically meaningful.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

What Controls the Width

Factor Effect on width
Larger sample size Narrower (∝ 1/√n)
More variability in the data Wider
Higher confidence level (99% vs 95%) Wider

Formula for a mean: x̄ ± t* × s/√n.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
true_mean = 50

def mean_ci(sample, level=0.95):
    se = stats.sem(sample)
    return stats.t.interval(level, len(sample) - 1, loc=sample.mean(), scale=se)

sample = rng.normal(true_mean, 10, 40)
low, high = mean_ci(sample)
print(f"sample mean {sample.mean():.2f}, 95% CI [{low:.2f}, {high:.2f}]")

# Coverage: repeat the study many times
covered = 0
for _ in range(10_000):
    s = rng.normal(true_mean, 10, 40)
    lo, hi = mean_ci(s)
    covered += lo <= true_mean <= hi
print("share of 95% intervals containing the true mean:", covered / 10_000)

for n in (10, 100, 1000):
    lo, hi = mean_ci(rng.normal(true_mean, 10, n))
    print(f"n={n:>4}: width {hi - lo:.2f}")

# Bootstrap CI for a median (no formula needed)
boot = stats.bootstrap((sample,), np.median, confidence_level=0.95, random_state=42)
print("bootstrap 95% CI for the median:", np.round(boot.confidence_interval, 2))

About 95% of the intervals contain the true mean, which is exactly what “95% confidence” promises, and widths shrink roughly tenfold when n grows a hundredfold.

Common Pitfalls

  • Saying “there is a 95% probability the true mean is in this interval” (that is a Bayesian credible interval statement).

  • Thinking 95% of the data fall inside the CI; that is a prediction or reference interval.

  • Judging overlap of two CIs as a test of their difference; compute the CI of the difference.

10

What are the null hypothesis and the alternative hypothesis?

The null hypothesis (H₀) is the default claim of “no effect” or “no difference” that a test tries to find evidence against, such as “the new drug has the same mean effect as placebo” or “the coin is fair”. The alternative hypothesis (H₁ or Hₐ) is what you would conclude if the evidence against H₀ is strong enough: “the drug’s effect differs from placebo” (two-sided) or “the drug lowers blood pressure more” (one-sided). The test calculates how surprising the data would be if H₀ were true; you either reject H₀ or fail to reject it, and you never “accept” or prove H₀.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Examples

Question H₀ H₁ Sidedness
Is the coin fair? p = 0.5 p ≠ 0.5 Two-sided
Does the new page convert better? p_B ≤ p_A p_B > p_A One-sided
Do three diets differ? μ₁ = μ₂ = μ₃ At least one mean differs ANOVA
Are gender and preference related? Independent Associated Chi-square

Rules for Writing Hypotheses

  • They are statements about population parameters, not sample statistics.

  • They are mutually exclusive and decided before looking at the data.

  • H₀ contains the equality (=, ≤ or ≥).

  • Choose one-sided only when the other direction is irrelevant for the decision.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
flips = rng.binomial(1, 0.58, 200)          # a slightly biased coin
heads = int(flips.sum())

two_sided = stats.binomtest(heads, 200, p=0.5, alternative="two-sided")
one_sided = stats.binomtest(heads, 200, p=0.5, alternative="greater")
print(f"{heads} heads in 200 flips")
print(f"H0: p = 0.5 vs H1: p ≠ 0.5 → p-value {two_sided.pvalue:.4f}")
print(f"H0: p ≤ 0.5 vs H1: p > 0.5 → p-value {one_sided.pvalue:.4f}")

alpha = 0.05
print("decision (two-sided):", "reject H0" if two_sided.pvalue < alpha else "fail to reject H0")

The one-sided p-value is about half the two-sided one when the data go in the hypothesized direction, which is why the direction must be chosen in advance.

Common Pitfalls

  • Saying “we accept the null hypothesis”; absence of evidence is not evidence of absence.

  • Writing hypotheses about the sample (“the sample mean is 5”).

  • Switching to a one-sided test after seeing which way the result went.

11

What is Bayes' theorem, and how is it used in statistics?

Bayes’ theorem describes how to update the probability of a hypothesis given new evidence: P(H | E) = P(E | H) × P(H) / P(E), where P(H) is the prior, P(E | H) the likelihood of the evidence if the hypothesis is true, P(E) the overall probability of the evidence, and P(H | E) the posterior. In statistics it is used to interpret diagnostic tests (accounting for base rates), in spam filtering (naive Bayes), and as the foundation of Bayesian inference, where priors over parameters are updated with data to give posterior distributions.

Read full answer

Applies toPython 3.12, NumPy 2.x

Classic Example: Medical Test

A disease affects 1% of people. A test detects 95% of cases (sensitivity) and has a 5% false-positive rate. If you test positive:

P(disease | positive) = 0.95 × 0.01 / (0.95 × 0.01 + 0.05 × 0.99) ≈ 0.16

Only about 16%, because false positives among the many healthy people outnumber the true positives. Ignoring the prior is the base-rate fallacy.

Uses

Area How Bayes’ theorem is used
Diagnostic testing Positive and negative predictive values
Spam filtering P(spam | words) with naive Bayes
Bayesian inference Posterior distributions of parameters
A/B testing P(variant B is better | data)
Machine learning Generative classifiers, Bayesian networks

Code Example

import numpy as np

def posterior(prior, sensitivity, false_positive_rate):
    evidence = sensitivity * prior + false_positive_rate * (1 - prior)
    return sensitivity * prior / evidence

for prior in (0.001, 0.01, 0.10, 0.50):
    print(f"prevalence {prior:>6.1%}: P(disease | positive) = {posterior(prior, 0.95, 0.05):.3f}")

# Two positive tests in a row (independent): the first posterior becomes the new prior
p1 = posterior(0.01, 0.95, 0.05)
p2 = posterior(p1, 0.95, 0.05)
print(f"after one positive {p1:.3f}, after two positives {p2:.3f}")

# Simulation check
rng = np.random.default_rng(42)
n = 1_000_000
sick = rng.random(n) < 0.01
positive = np.where(sick, rng.random(n) < 0.95, rng.random(n) < 0.05)
print("simulated P(disease | positive):", round(sick[positive].mean(), 3))

The posterior depends heavily on the prevalence, and a second independent positive result raises the probability sharply. The simulation confirms the formula.

Common Pitfalls

  • Confusing P(E | H) with P(H | E).

  • Ignoring base rates (prevalence).

  • Treating repeated tests as independent when their errors are correlated.

12

How do discrete and continuous probability distributions differ?

Discrete distributions describe variables that take countable values (0, 1, 2, … or a list of categories), such as the number of defects or the roll of a die. Each value has a positive probability given by a probability mass function (PMF), and the probabilities sum to 1. Continuous distributions describe variables that can take any value in an interval, such as height or time. They are described by a probability density function (PDF): the probability of any exact value is 0, probabilities come from the area under the curve over an interval, and the total area is 1. Both have a cumulative distribution function (CDF).

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Aspect Discrete Continuous
Values Countable Any value in an interval
Described by PMF: P(X = x) PDF: f(x), a density
P(X = exact value) Can be positive Always 0
Probabilities Sum of PMF values Integral (area) of the PDF
CDF Step function Smooth curve
Examples Bernoulli, binomial, Poisson, geometric Normal, exponential, uniform, beta, gamma

A PDF value can exceed 1 (it is a density, not a probability).

Code Example

import numpy as np
from scipy import stats, integrate

# Discrete: number of website sign-ups per hour ~ Poisson(3)
pois = stats.poisson(3)
print("P(X = 2):", round(pois.pmf(2), 4))
print("P(X ≤ 2) as a sum of PMF values:", round(sum(pois.pmf(k) for k in range(3)), 4), "=", round(pois.cdf(2), 4))
print("PMF sums to 1:", round(sum(pois.pmf(k) for k in range(100)), 6))

# Continuous: delivery time ~ Normal(30, 5) minutes
norm = stats.norm(30, 5)
area, _ = integrate.quad(norm.pdf, 25, 35)
print("P(25 < T < 35) as an area:", round(area, 4), "=", round(norm.cdf(35) - norm.cdf(25), 4))
print("P(T = 30 exactly): 0, but the density there is", round(norm.pdf(30), 4))

narrow = stats.norm(0, 0.1)
print("a density can exceed 1:", round(narrow.pdf(0), 3))

rng = np.random.default_rng(42)
print("sample values:", rng.poisson(3, 5), rng.normal(30, 5, 3).round(2))

Discrete probabilities are added, continuous ones integrated. In SciPy, discrete distributions have pmf and continuous ones pdf.

Common Pitfalls

  • Reading a PDF value as a probability.

  • Modelling counts with a continuous distribution when counts are small.

  • Forgetting continuity corrections when approximating a discrete distribution with a continuous one.

13

What are the properties of a normal distribution?

The normal (Gaussian) distribution is a continuous, symmetric, bell-shaped distribution fully described by its mean μ and standard deviation σ. Its mean, median and mode are equal; it extends over all real numbers; and about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations of the mean. Sums and linear combinations of independent normal variables are normal, and by the Central Limit Theorem averages of many independent variables are approximately normal, which is why it is central to confidence intervals, hypothesis tests and linear regression.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Properties

Property Detail
Parameters Mean μ (location), SD σ (spread)
Shape Symmetric bell; skewness 0, excess kurtosis 0
Centre Mean = median = mode
Empirical rule 68% within ±1σ, 95% within ±2σ (exactly ±1.96σ), 99.7% within ±3σ
Inflection points At μ ± σ
Standardization z = (x − μ)/σ gives the standard normal N(0, 1)
Closure aX + b and sums of independent normals are normal
Support All real numbers (tails never reach zero)

Code Example

import numpy as np
from scipy import stats

mu, sigma = 170, 8                       # adult height in cm
dist = stats.norm(mu, sigma)

for k in (1, 2, 3):
    print(f"P(within ±{k}σ) = {dist.cdf(mu + k * sigma) - dist.cdf(mu - k * sigma):.4f}")
print("z-score of 186 cm:", (186 - mu) / sigma, "| P(height > 186):", round(dist.sf(186), 4))
print("middle 95% range:", np.round(dist.interval(0.95), 1))

rng = np.random.default_rng(42)
x = rng.normal(mu, sigma, 100_000)
print(f"mean {x.mean():.2f}, median {np.median(x):.2f}, skew {stats.skew(x):.3f}, excess kurtosis {stats.kurtosis(x):.3f}")

# Sum of independent normals is normal with added variances
y = rng.normal(10, 6, 100_000)
total = x + y
print(f"sum: mean {total.mean():.2f} (theory {mu + 10}), SD {total.std():.2f} (theory {np.sqrt(sigma**2 + 6**2):.2f})")

Standardizing to z-scores lets you use one table (or norm.cdf) for any normal distribution.

Common Pitfalls

  • Assuming real data are normal without checking (Q-Q plots); many are skewed or heavy-tailed.

  • Applying the 68-95-99.7 rule to non-normal data.

  • Ignoring that the normal assigns non-zero probability to impossible values (negative heights), usually negligible but sometimes not.

14

What is the Law of Large Numbers, and how does it relate to statistics?

The Law of Large Numbers (LLN) states that as the number of independent, identically distributed observations grows, the sample mean converges to the true population mean (the expected value). The weak law says it converges in probability; the strong law says it converges almost surely. In statistics it is the reason sample estimates become reliable with more data, it justifies Monte Carlo simulation and frequency-based probabilities, and it underpins insurance and casinos, whose average outcomes over many policies or bets are predictable even though each one is not.

Read full answer

Applies toPython 3.12, NumPy 2.x

LLN vs Central Limit Theorem

Aspect Law of Large Numbers Central Limit Theorem
Says The sample mean approaches the true mean The sample mean’s distribution becomes approximately normal
Answers Where the mean ends up How much it varies around the true mean (SE = σ/√n)
Used for Consistency of estimators, Monte Carlo Confidence intervals, hypothesis tests

Code Example

import numpy as np

rng = np.random.default_rng(42)

# Die rolls: expected value 3.5
rolls = rng.integers(1, 7, 100_000)
running_mean = np.cumsum(rolls) / np.arange(1, len(rolls) + 1)
for n in (10, 100, 1_000, 10_000, 100_000):
    print(f"after {n:>7,} rolls: mean = {running_mean[n - 1]:.4f} (error {abs(running_mean[n - 1] - 3.5):.4f})")

# Insurance: individual claims are unpredictable, the average per policy is not
claims = rng.binomial(1, 0.02, 1_000_000) * rng.gamma(2, 2500, 1_000_000)
print("expected claim per policy:", 0.02 * 2 * 2500, "| observed average:", round(claims.mean(), 1))

# Heavy tails: the Cauchy distribution has no mean, so the LLN does not apply
cauchy = rng.standard_cauchy(100_000)
print("Cauchy running means:", [round(cauchy[:n].mean(), 2) for n in (100, 10_000, 100_000)])

The die’s running mean settles at 3.5 and the error shrinks roughly like 1/√n. The Cauchy example shows the LLN requires a finite mean: its running mean never settles.

Common Pitfalls

  • Gambler’s fallacy: the LLN does not mean short-run outcomes “balance out”; earlier deviations are diluted, not corrected.

  • Assuming it applies to dependent or non-identically distributed data without checking.

  • Forgetting that convergence can be slow for heavy-tailed data.

15

What role does the binomial distribution play in statistics?

The binomial distribution gives the probability of getting exactly k successes in n independent trials, each with the same success probability p: P(X = k) = C(n, k) pᵏ (1 − p)ⁿ⁻ᵏ, with mean np and variance np(1 − p). It is the basic model for yes/no outcomes: conversions out of visitors, defective items in a batch, patients who respond to treatment. It underlies proportion estimates and their confidence intervals, the binomial test and two-proportion tests used in A/B testing, logistic regression’s likelihood, and quality-control sampling plans.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Requirements

Condition Example check
Fixed number of trials n 500 visitors
Two outcomes per trial Converted or not
Same probability p each trial Similar visitors, no time trend
Independent trials One visitor’s choice doesn’t affect another’s

For large n with small p, it approaches the Poisson (λ = np); for large np and n(1 − p), the normal.

Code Example

import numpy as np
from scipy import stats

n, p = 20, 0.3
dist = stats.binom(n, p)
print("P(X = 6):", round(dist.pmf(6), 4), "| P(X ≤ 3):", round(dist.cdf(3), 4))
print("mean", dist.mean(), "| variance", dist.var())

# Binomial test: 62 conversions out of 1000 vs a historical rate of 5%
res = stats.binomtest(62, n=1000, p=0.05, alternative="greater")
ci = res.proportion_ci(confidence_level=0.95, method="wilson")
print(f"binomial test p = {res.pvalue:.4f}; 95% Wilson CI [{ci.low:.4f}, {ci.high:.4f}]")

# Quality control: accept a batch if ≤ 2 defects in a sample of 50
for defect_rate in (0.01, 0.05, 0.10):
    print(f"defect rate {defect_rate:.0%}: P(accept) = {stats.binom.cdf(2, 50, defect_rate):.3f}")

rng = np.random.default_rng(42)
sims = rng.binomial(n, p, 100_000)
print("simulated P(X = 6):", round((sims == 6).mean(), 4))

The binomial test’s p-value is the probability of 62 or more conversions out of 1000 if the true rate were still 5%. The acceptance probabilities form the sampling plan’s operating characteristic.

Common Pitfalls

  • Using it when trials are dependent (repeat visits by the same user).

  • Using the simple Wald interval for small n or extreme p; prefer Wilson.

  • Treating varying probabilities across trials as one p (overdispersion; use beta-binomial).

16

What is the difference between joint, marginal and conditional probability?

Joint probability P(A and B) is the probability that two events happen together. Marginal probability P(A) is the probability of one event regardless of the other, obtained by summing (or integrating) the joint probabilities over all values of the other variable. Conditional probability P(A | B) is the probability of A given that B has happened: P(A | B) = P(A and B) / P(B). Two events are independent when P(A | B) = P(A), equivalently P(A and B) = P(A)P(B). These three ideas are the building blocks of Bayes’ theorem, contingency tables and probabilistic models.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+

Summary

Type Notation Question From a contingency table
Joint P(A ∩ B) Both happen? Cell count / grand total
Marginal P(A) A happens, ignoring B? Row or column total / grand total
Conditional P(A | B) A happens, given B? Cell count / column (or row) total

Note that P(A | B) and P(B | A) are usually different: P(positive test | disease) is not P(disease | positive test).

Code Example

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 10_000
smoker = rng.random(n) < 0.2
disease = rng.random(n) < np.where(smoker, 0.15, 0.05)

df = pd.DataFrame({"smoker": np.where(smoker, "smoker", "non-smoker"),
                   "disease": np.where(disease, "disease", "healthy")})
joint = pd.crosstab(df["smoker"], df["disease"], normalize="all")
print("joint probabilities:\n", joint.round(4))

marginal_disease = joint.sum(axis=0)
marginal_smoker = joint.sum(axis=1)
print("P(disease):", round(marginal_disease["disease"], 4))

p_disease_given_smoker = joint.loc["smoker", "disease"] / marginal_smoker["smoker"]
p_smoker_given_disease = joint.loc["smoker", "disease"] / marginal_disease["disease"]
print("P(disease | smoker):", round(p_disease_given_smoker, 4))
print("P(smoker | disease):", round(p_smoker_given_disease, 4))

independent = np.isclose(joint.loc["smoker", "disease"],
                         marginal_smoker["smoker"] * marginal_disease["disease"], atol=0.002)
print("looks independent:", independent)

P(disease | smoker) is about 0.15 while P(smoker | disease) is much higher: the two conditionals answer different questions. Because the joint probability differs from the product of the marginals, smoking and disease are not independent here.

Common Pitfalls

  • Confusing P(A | B) with P(B | A) (the “prosecutor’s fallacy”).

  • Assuming independence without checking, e.g. multiplying probabilities of correlated failures.

  • Forgetting that marginals come from summing the joint distribution.

17

How does the Poisson distribution differ from the normal distribution?

The Poisson distribution is a discrete distribution for counts of events in a fixed interval when events occur independently at a constant average rate λ: calls per hour, defects per batch, goals per match. It takes only non-negative integers, its mean and variance are both λ, and it is right-skewed when λ is small. The normal distribution is continuous, symmetric and defined over all real numbers, with a mean and variance that are independent parameters. For large λ (roughly above 20), the Poisson is well approximated by a normal with mean λ and variance λ.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Property Poisson(λ) Normal(μ, σ²)
Type Discrete counts 0, 1, 2, … Continuous, all real numbers
Parameters λ (rate) μ (mean), σ (SD)
Mean / variance λ / λ μ / σ² (independent)
Shape Right-skewed for small λ Symmetric bell
Typical data Event counts Measurements, sums and averages
Regression Poisson / negative binomial GLM Linear regression

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

for lam in (2, 50):
    counts = rng.poisson(lam, 100_000)
    print(f"λ={lam}: mean {counts.mean():.2f}, variance {counts.var():.2f}, skew {stats.skew(counts):.2f}")

# Probability of exactly 0 and at most 3 events when λ = 2
print("P(X=0):", round(stats.poisson.pmf(0, 2), 4), "| P(X≤3):", round(stats.poisson.cdf(3, 2), 4))

# Normal approximation (with continuity correction) works for large λ, not small
for lam, k in ((2, 3), (50, 55)):
    exact = stats.poisson.cdf(k, lam)
    approx = stats.norm.cdf(k + 0.5, loc=lam, scale=np.sqrt(lam))
    print(f"λ={lam}: P(X≤{k}) exact {exact:.4f}, normal approx {approx:.4f}")

# Overdispersion check for real count data: variance much larger than the mean suggests negative binomial
calls = rng.negative_binomial(5, 5 / (5 + 10), 1000)
print("dispersion index (variance/mean):", round(calls.var() / calls.mean(), 2))

With λ = 2 the normal approximation is poor; with λ = 50 it is close. A dispersion index well above 1 indicates the Poisson assumption of equal mean and variance fails.

Common Pitfalls

  • Modelling counts with linear regression (negative predictions, wrong variance).

  • Ignoring overdispersion, which makes Poisson standard errors too small.

  • Applying Poisson when events are not independent (e.g. clustered failures).

18

What is a cumulative distribution function (CDF)?

The cumulative distribution function (CDF) of a random variable X gives the probability that X takes a value less than or equal to x: F(x) = P(X ≤ x). It exists for every distribution, discrete or continuous, rises from 0 to 1, and never decreases. For continuous variables it is the integral of the probability density function (PDF), and interval probabilities follow directly: P(a < X ≤ b) = F(b) − F(a). Its inverse, the quantile function, gives percentiles, and the empirical CDF of a sample is used to compare distributions (Kolmogorov-Smirnov test).

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, Matplotlib 3.9+

Key Properties

Property Meaning
0 ≤ F(x) ≤ 1 It is a probability
Non-decreasing Accumulates probability
F(−∞) = 0, F(∞) = 1 All probability is eventually included
Discrete variables Step function with jumps at possible values
Continuous variables Smooth; derivative is the PDF
Quantile function F⁻¹(p) The value below which a share p of the distribution lies

Code Example

import matplotlib.pyplot as plt
import numpy as np
from scipy import stats

dist = stats.norm(loc=100, scale=15)                 # IQ-like scores
print("P(X ≤ 115):", round(dist.cdf(115), 4))
print("P(85 < X ≤ 115):", round(dist.cdf(115) - dist.cdf(85), 4))
print("90th percentile (inverse CDF):", round(dist.ppf(0.90), 2))

dice = stats.randint(1, 7)                           # discrete: a fair die
print("P(die ≤ 4):", dice.cdf(4))

rng = np.random.default_rng(42)
sample = rng.normal(100, 15, 200)
x_sorted = np.sort(sample)
ecdf = np.arange(1, len(sample) + 1) / len(sample)
print("KS test vs N(100, 15):", round(stats.kstest(sample, dist.cdf).pvalue, 3))

xs = np.linspace(40, 160, 300)
plt.step(x_sorted, ecdf, where="post", label="empirical CDF (n = 200)")
plt.plot(xs, dist.cdf(xs), label="theoretical CDF")
plt.xlabel("score")
plt.ylabel("P(X ≤ x)")
plt.legend()
plt.show()

The empirical CDF closely tracks the theoretical curve; the KS test measures the largest vertical gap between them.

Common Pitfalls

  • Confusing the CDF with the PDF (the PDF value is a density, not a probability).

  • Forgetting P(X ≤ x) and P(X < x) differ for discrete variables.

  • Using ppf and cdf with the wrong scale parameterization.

19

What are the use cases of the exponential and uniform distributions?

The exponential distribution models the waiting time until the next event when events happen independently at a constant average rate λ (a Poisson process): time between customer arrivals, time until a component fails (with a constant hazard), or time between website requests. It has mean 1/λ and is memoryless: the chance of waiting another minute does not depend on how long you have already waited. The uniform distribution gives equal probability to every value in an interval [a, b]. It models complete uncertainty within known limits, rounding errors and random arrival within a window, and is the basis of random number generation and simulation.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Property Exponential(λ) Uniform(a, b)
Support x ≥ 0 a ≤ x ≤ b
PDF λe^(−λx) 1/(b − a)
Mean 1/λ (a + b)/2
Variance 1/λ² (b − a)²/12
Key property Memoryless All values equally likely
Uses Waiting times, lifetimes, queueing Simulation, random sampling, priors with bounds, rounding error

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# Customers arrive on average 12 per hour → mean gap 5 minutes
rate_per_min = 12 / 60
gaps = rng.exponential(scale=1 / rate_per_min, size=100_000)
print("mean gap (min):", round(gaps.mean(), 2), "| P(gap > 10 min):", round((gaps > 10).mean(), 4),
      "| theory:", round(np.exp(-rate_per_min * 10), 4))

# Memorylessness: P(gap > 15 | gap > 5) equals P(gap > 10)
print("P(>15 | >5):", round((gaps > 15).sum() / (gaps > 5).sum(), 4))

# Uniform: a bus arrives at a random moment within a 20-minute window
wait = rng.uniform(0, 20, 100_000)
print("uniform mean wait:", round(wait.mean(), 2), "| variance:", round(wait.var(), 2), "| theory:", round(20 ** 2 / 12, 2))

# Inverse transform sampling: uniform draws become exponential draws
u = rng.uniform(size=100_000)
from_uniform = -np.log(1 - u) / rate_per_min
print("KS test vs exponential, p:", round(stats.kstest(from_uniform, "expon", args=(0, 1 / rate_per_min)).pvalue, 3))

The last step shows a key link: uniform random numbers can be transformed into any distribution through its inverse CDF.

Common Pitfalls

  • Using the exponential for lifetimes that wear out (increasing hazard); use Weibull or gamma.

  • Confusing the rate λ with the scale 1/λ (NumPy and SciPy use the scale).

  • Assuming “uniform” when you simply lack data about the shape.

20

How is covariance different from correlation?

Covariance measures whether two variables move together: positive when they tend to be above (or below) their means at the same time, negative when one is above while the other is below. Its size depends on the units, so a covariance of 500 cannot be judged as strong or weak. Correlation is covariance standardized by both standard deviations: r = cov(X, Y) / (σₓ σᵧ). It is unitless, always between −1 and +1, and so comparable across variable pairs. Both measure only linear association, and neither implies causation.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Aspect Covariance Correlation
Formula Σ(x − x̄)(y − ȳ) / (n − 1) cov / (sₓ sᵧ)
Range −∞ to +∞ −1 to +1
Units Product of the variables’ units None
Affected by rescaling Yes No (except sign changes)
Main use Covariance matrices (PCA, portfolio risk, multivariate normal) Comparing strength of relationships

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
height_m = rng.normal(1.70, 0.08, 500)
weight_kg = 20 + 28 * height_m + rng.normal(0, 6, 500)

cov_m = np.cov(height_m, weight_kg)[0, 1]
cov_cm = np.cov(height_m * 100, weight_kg)[0, 1]
r = np.corrcoef(height_m, weight_kg)[0, 1]
r_cm = np.corrcoef(height_m * 100, weight_kg)[0, 1]
print(f"covariance (m, kg): {cov_m:.3f} | (cm, kg): {cov_cm:.3f}")       # changes 100×
print(f"correlation (m, kg): {r:.3f} | (cm, kg): {r_cm:.3f}")             # unchanged

manual_r = cov_m / (height_m.std(ddof=1) * weight_kg.std(ddof=1))
print("manual correlation:", round(manual_r, 3), "| Pearson test p-value:", f"{stats.pearsonr(height_m, weight_kg).pvalue:.2g}")

# A strong non-linear relationship can have near-zero covariance and correlation
x = rng.uniform(-3, 3, 500)
print("corr(x, x²):", round(np.corrcoef(x, x ** 2)[0, 1], 3))

Changing height from metres to centimetres multiplies the covariance by 100 but leaves the correlation unchanged. The x² example shows that zero correlation does not mean no relationship.

Common Pitfalls

  • Comparing covariances between variable pairs measured in different units.

  • Concluding “no relationship” from a correlation near zero without plotting.

  • Letting outliers drive the correlation; consider Spearman’s rank correlation.

21

What are measures of central tendency, and why do they matter?

Measures of central tendency summarize a dataset with a single “typical” value. The main ones are the mean (arithmetic average), the median (middle value when sorted) and the mode (most frequent value); others include the trimmed mean, the geometric mean (for growth rates and ratios) and the harmonic mean (for rates such as speeds). They matter because they let you summarize and compare groups quickly, are the basis of many statistical tests, and serve as baselines for predictions, but they must be chosen to fit the data’s shape and scale and reported with a measure of spread.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Which Measure to Use

Measure Best for Weakness
Mean Symmetric numeric data Sensitive to outliers
Median Skewed data, ordinal data Ignores magnitudes
Mode Categorical data Can be non-unique or unstable
Trimmed mean Data with some outliers Discards information
Geometric mean Growth rates, ratios Positive values only
Harmonic mean Rates (speed, price/earnings) Dominated by small values

Code Example

from collections import Counter

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
order_values = np.r_[rng.gamma(2, 25, 500), [1500, 2200]]
print(f"mean {order_values.mean():.2f}, median {np.median(order_values):.2f}, "
      f"10% trimmed mean {stats.trim_mean(order_values, 0.1):.2f}")

payment_methods = ["card", "card", "wallet", "card", "cash", "wallet", "card"]
print("mode of payment method:", Counter(payment_methods).most_common(1)[0][0])

yearly_growth = np.array([1.10, 0.95, 1.20, 1.05])           # +10%, −5%, +20%, +5%
print(f"average growth: arithmetic {yearly_growth.mean() - 1:.2%}, geometric {stats.gmean(yearly_growth) - 1:.2%}")

speeds = np.array([60, 40])                                  # km/h over two equal distances
print("average speed over the trip:", round(stats.hmean(speeds), 1), "km/h")

The geometric mean gives the true compound growth rate, and the harmonic mean the true average speed over equal distances; the arithmetic mean would overstate both.

Common Pitfalls

  • Using the arithmetic mean of growth rates or ratios.

  • Computing a mean of ordinal codes (e.g. 1-5 satisfaction) without saying so.

  • Reporting a centre without the spread (SD or IQR).

22

What are range, IQR, variance and standard deviation?

Measures of dispersion describe how spread out data are. The range is max − min: simple but determined entirely by the two most extreme values. The interquartile range (IQR) is Q3 − Q1, the spread of the middle 50%, robust to outliers. The variance is the average squared deviation from the mean (divided by n − 1 for a sample, to make it unbiased), and the standard deviation (SD) is its square root, in the same units as the data. Use SD with the mean for roughly symmetric data, and IQR with the median for skewed data or data with outliers.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Measure Formula Units Robust to outliers? Pair with
Range max − min Data units No Quick check
IQR Q3 − Q1 Data units Yes Median
Variance Σ(x − x̄)² / (n − 1) Squared units No Used in models and tests
Standard deviation √variance Data units No Mean
MAD median(|x − median|) Data units Yes Median

For normal data, about 68% of values lie within 1 SD of the mean and 95% within 2 SD.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
clean = rng.normal(50, 5, 200)
with_outlier = np.r_[clean, [150]]

for name, x in (("clean", clean), ("with one outlier", with_outlier)):
    q1, q3 = np.percentile(x, [25, 75])
    print(f"{name:>16}: range {np.ptp(x):6.2f}, IQR {q3 - q1:5.2f}, "
          f"variance {x.var(ddof=1):7.2f}, SD {x.std(ddof=1):5.2f}, "
          f"MAD {stats.median_abs_deviation(x):5.2f}")

# Population (ddof=0) vs sample (ddof=1) variance
small = np.array([4, 8, 6, 5, 7])
print("population variance:", small.var(ddof=0), "| sample variance:", small.var(ddof=1))

A single outlier greatly inflates the range, variance and SD, while the IQR and MAD barely change. NumPy’s default ddof=0 gives the population formula, so pass ddof=1 for sample estimates (pandas defaults to ddof=1).

Common Pitfalls

  • Mixing up ddof defaults between NumPy and pandas.

  • Interpreting variance directly; it is in squared units.

  • Applying the 68-95-99.7 rule to skewed data.

23

What is the difference between mean and median, and when to use each?

The mean is the arithmetic average (sum ÷ count); it uses every value, so it is efficient for symmetric data but is pulled toward outliers and long tails. The median is the middle value of the sorted data; it depends only on the order of values, so it is robust to outliers and skew. Use the mean for roughly symmetric data, or when totals matter (total revenue = mean × count). Use the median for skewed data such as incomes, house prices, response times or waiting times, and whenever outliers could distort the typical value.

Read full answer

Applies toPython 3.12, NumPy 2.x

Comparison

Aspect Mean Median
Uses All values and their sizes Only the middle position
Outliers Sensitive Robust (50% breakdown point)
Skewed data Pulled toward the tail Represents the typical value
Totals and expected values Directly related Not related
Statistical efficiency (normal data) Higher About 64% of the mean’s

Code Example

import numpy as np

rng = np.random.default_rng(42)

salaries = np.r_[rng.normal(60_000, 8_000, 49), [2_000_000]]     # one CEO
print(f"mean salary {salaries.mean():,.0f} vs median {np.median(salaries):,.0f}")

# Sampling variability on normal data: the mean is more precise
means, medians = [], []
for _ in range(5000):
    s = rng.normal(100, 15, 50)
    means.append(s.mean())
    medians.append(np.median(s))
print(f"normal data, SD of estimate: mean {np.std(means):.2f}, median {np.std(medians):.2f}")

# Heavy-tailed data: the median is more stable
means, medians = [], []
for _ in range(5000):
    s = rng.standard_t(2, 50)
    means.append(s.mean())
    medians.append(np.median(s))
print(f"heavy-tailed data, SD of estimate: mean {np.std(means):.2f}, median {np.std(medians):.2f}")

One extreme salary moves the mean by tens of thousands but barely moves the median. With normal data the mean is the more precise estimator; with heavy tails the median wins.

Common Pitfalls

  • Reporting “average income” as a mean when most people earn far less.

  • Using the median when you need totals (budgeting, expected revenue).

  • Reporting either without a measure of spread.

24

How do you describe skewness and kurtosis in a dataset?

Skewness measures the asymmetry of a distribution: positive (right) skew has a long tail toward high values, so the mean is pulled above the median (incomes, response times); negative (left) skew has a long tail toward low values (exam scores near a ceiling); zero means roughly symmetric. Kurtosis measures how heavy the tails are relative to a normal distribution; excess kurtosis (kurtosis − 3) is 0 for a normal distribution, positive for heavy-tailed data with more extreme outliers (returns in finance), and negative for light-tailed, flatter data (uniform). Both guide the choice of summaries, transforms and tests.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Interpretation

Measure Value Meaning Example
Skewness > 0 Long right tail; mean > median Income, house prices
Skewness < 0 Long left tail; mean < median Age at death in rich countries
Skewness ≈ 0 Symmetric Heights
Excess kurtosis > 0 Heavy tails, more outliers Stock returns (t distribution)
Excess kurtosis < 0 Light tails Uniform distribution

Rough guide: |skewness| above 1 is strong; between 0.5 and 1 moderate.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
samples = {
    "normal": rng.normal(0, 1, 10_000),
    "lognormal (right skew)": rng.lognormal(0, 0.6, 10_000),
    "ceiling scores (left skew)": 100 - rng.gamma(2, 5, 10_000),
    "t, 4 df (heavy tails)": rng.standard_t(4, 10_000),
    "uniform (light tails)": rng.uniform(-1, 1, 10_000),
}
for name, x in samples.items():
    print(f"{name:>27}: skew {stats.skew(x):6.2f}, excess kurtosis {stats.kurtosis(x):6.2f}, "
          f"mean − median {x.mean() - np.median(x):6.3f}")

x = samples["lognormal (right skew)"]
print("skew after log transform:", round(stats.skew(np.log(x)), 3))
print("skewness test p-value (normal sample):", round(stats.skewtest(samples["normal"]).pvalue, 3))

scipy.stats.kurtosis returns excess kurtosis by default (fisher=True). The log transform removes most of the right skew of lognormal data.

Common Pitfalls

  • Saying high kurtosis means a “peaked” distribution; it is about the tails.

  • Trusting skewness and kurtosis from small samples; they are very sensitive to single outliers.

  • Reporting the mean for strongly skewed data without the median.

25

What is the five-number summary in descriptive statistics?

The five-number summary describes a dataset with five values: the minimum, the first quartile (Q1, 25th percentile), the median (Q2), the third quartile (Q3, 75th percentile) and the maximum. Together they show the centre (median), the spread of the middle half (the interquartile range, IQR = Q3 − Q1), the overall range and the skew. It is robust to outliers in its middle three values, is the basis of the box plot, and supports the common outlier rule: values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR are flagged.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, Matplotlib 3.9+

What Each Value Tells You

Value Meaning
Minimum Smallest observation
Q1 25% of values are at or below it
Median Middle value; half below, half above
Q3 75% of values are at or below it
Maximum Largest observation
IQR = Q3 − Q1 Spread of the middle 50%

If the median is closer to Q1 than to Q3, and the upper whisker is longer, the data are right-skewed.

Code Example

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
delivery_minutes = np.r_[rng.gamma(4, 8, 200), [150, 180]]     # right-skewed with two late deliveries

q1, median, q3 = np.percentile(delivery_minutes, [25, 50, 75])
summary = {"min": delivery_minutes.min(), "Q1": q1, "median": median, "Q3": q3, "max": delivery_minutes.max()}
print({k: round(v, 1) for k, v in summary.items()})

iqr = q3 - q1
low, high = q1 - 1.5 * iqr, q3 + 1.5 * iqr
outliers = delivery_minutes[(delivery_minutes < low) | (delivery_minutes > high)]
print(f"IQR = {iqr:.1f}; outlier fences [{low:.1f}, {high:.1f}]; outliers: {np.sort(outliers).round(1)}")

print(pd.Series(delivery_minutes).describe().round(1))        # pandas gives the same quartiles plus mean/std

plt.boxplot(delivery_minutes, vert=False)
plt.xlabel("delivery time (minutes)")
plt.title("Box plot of the five-number summary")
plt.show()

The box spans Q1 to Q3 with a line at the median; whiskers extend to the most extreme points within the fences, and points beyond them are drawn individually.

Common Pitfalls

  • Different software uses slightly different quartile methods, so small differences are normal.

  • Treating every 1.5 × IQR point as an error; in skewed data many are genuine.

  • Hiding multimodal distributions, which a box plot cannot show (add a histogram).

26

What are the steps in conducting a hypothesis test?

A hypothesis test follows a fixed sequence: (1) state the null hypothesis H₀ (no effect) and the alternative H₁; (2) choose the significance level α (often 0.05) and, ideally, plan the sample size with a power analysis; (3) pick a test whose assumptions fit the data; (4) collect data and compute the test statistic; (5) compute the p-value, the probability of a result at least this extreme if H₀ were true; (6) decide: reject H₀ if p ≤ α, otherwise fail to reject it; and (7) report the effect size with a confidence interval and interpret it in context.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

The Steps

Step Example: does a new checkout reduce time to pay?
1. Hypotheses H₀: μ_new = μ_old; H₁: μ_new < μ_old (one-sided, decided in advance)
2. α and power α = 0.05, power 0.8 for a 5-second difference
3. Test Welch’s t-test (independent groups, unequal variances)
4. Statistic t computed from the samples
5. p-value From the t distribution
6. Decision Reject H₀ if p ≤ 0.05
7. Report Mean difference, 95% CI, practical importance

Code Example

import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower

rng = np.random.default_rng(42)

# Step 2: sample size for a 5-second difference with SD 15
n = int(np.ceil(TTestIndPower().solve_power(effect_size=5 / 15, alpha=0.05, power=0.8, alternative="larger")))
print("planned n per group:", n)

# Step 4-5: collect data and test (one-sided: old > new)
old = rng.normal(60, 15, n)
new = rng.normal(54, 15, n)
res = stats.ttest_ind(old, new, equal_var=False, alternative="greater")
print(f"t = {res.statistic:.2f}, p = {res.pvalue:.4f}")

# Step 6-7: decide and report
diff = old.mean() - new.mean()
ci = stats.ttest_ind(old, new, equal_var=False).confidence_interval(0.95)
print(f"mean reduction {diff:.2f} s, 95% CI [{ci.low:.2f}, {ci.high:.2f}]")
print("reject H0" if res.pvalue <= 0.05 else "fail to reject H0")

“Fail to reject” does not prove H₀; it means the data are not strong enough evidence against it.

Common Pitfalls

  • Choosing a one-sided test after seeing the direction of the data.

  • Changing α, the test or the outcome after looking at results.

  • Equating statistical significance with practical importance.

27

How is a t-test performed, and when should you use one?

A t-test compares means when the population standard deviation is unknown and estimated from the sample. It computes t = (observed difference − hypothesized difference) / standard error and compares it with a t distribution with the appropriate degrees of freedom to get a p-value. There are three main forms: one-sample (a mean vs a known value), independent two-sample (two separate groups; Welch’s version by default), and paired (the same subjects measured twice). It is appropriate for continuous outcomes, independent observations, and roughly normal data or moderately large samples.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Types

Test Question Statistic df
One-sample Is the mean 50? (x̄ − μ₀) / (s/√n) n − 1
Welch two-sample Do groups A and B differ? (x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂) Welch-Satterthwaite
Paired Did scores change after training? d̄ / (s_d/√n) n − 1

When to Use

  • Continuous outcome, comparing one or two means.

  • Data roughly normal, or n large enough (often 30+ per group) for the CLT.

  • Without these: Mann-Whitney U or Wilcoxon; for 3+ groups: ANOVA.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# One-sample: do bottles contain 500 ml on average?
volumes = rng.normal(498.5, 4, 40)
r1 = stats.ttest_1samp(volumes, popmean=500)
print(f"one-sample: mean {volumes.mean():.2f}, t = {r1.statistic:.2f}, p = {r1.pvalue:.4f}")

# Manual calculation matches
t_manual = (volumes.mean() - 500) / (volumes.std(ddof=1) / np.sqrt(len(volumes)))
print("manual t:", round(t_manual, 2))

# Paired: scores before and after training for the same 25 people
before = rng.normal(70, 8, 25)
after = before + rng.normal(3, 4, 25)
r2 = stats.ttest_rel(after, before)
print(f"paired: mean change {np.mean(after - before):.2f}, t = {r2.statistic:.2f}, p = {r2.pvalue:.4f}")

# Independent groups (Welch)
a, b = rng.normal(70, 8, 30), rng.normal(75, 12, 35)
r3 = stats.ttest_ind(a, b, equal_var=False)
print(f"Welch: t = {r3.statistic:.2f}, df = {r3.df:.1f}, p = {r3.pvalue:.4f}")

A small p-value in the one-sample test means average volumes this far from 500 ml would be unlikely if the true mean were 500. The paired test uses within-person differences, which removes person-to-person variation and gives more power.

Common Pitfalls

  • Running an independent t-test on paired data.

  • Multiple t-tests across many groups instead of ANOVA.

  • Heavy outliers in small samples distorting the mean.

28

What is ANOVA, and when is it used?

ANOVA (analysis of variance) tests whether the means of three or more groups are all equal, by comparing variation between group means with variation within groups. The F statistic is the ratio of these two variances; if group means differ more than within-group noise would explain, F is large and the p-value small. Use ANOVA instead of many pairwise t-tests, which would inflate the false-positive rate, then follow a significant result with a post-hoc test (Tukey’s HSD) to find which groups differ. Variants handle several factors (two-way ANOVA), repeated measures and covariates (ANCOVA).

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, SciPy 1.13+, statsmodels 0.14+

Types

Type Use
One-way One factor with 3+ levels (e.g. three diets)
Two-way Two factors and their interaction (diet × exercise)
Repeated measures Same subjects measured under several conditions
ANCOVA Adjusts for a continuous covariate (baseline score)
Welch’s ANOVA Unequal variances
Kruskal-Wallis Nonparametric alternative

Assumptions

Independent observations, approximately normal residuals, and similar group variances (Levene’s test).

Code Example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from scipy import stats
from statsmodels.stats.anova import anova_lm
from statsmodels.stats.multicomp import pairwise_tukeyhsd

rng = np.random.default_rng(42)
df = pd.DataFrame({
    "diet": np.repeat(["A", "B", "C"], 40),
    "weight_loss": np.r_[rng.normal(3, 2, 40), rng.normal(4, 2, 40), rng.normal(5.5, 2, 40)],
})

groups = [g["weight_loss"].to_numpy() for _, g in df.groupby("diet")]
f, p = stats.f_oneway(*groups)
print(f"one-way ANOVA: F = {f:.2f}, p = {p:.2e}")
print("Levene p (equal variances):", round(stats.levene(*groups).pvalue, 3))

model = smf.ols("weight_loss ~ C(diet)", data=df).fit()
table = anova_lm(model, typ=2)
eta_sq = table.loc["C(diet)", "sum_sq"] / table["sum_sq"].sum()
print("eta squared:", round(eta_sq, 3))

print(pairwise_tukeyhsd(df["weight_loss"], df["diet"], alpha=0.05))

The small p-value says at least one diet’s mean differs; Tukey’s table shows which pairs differ with adjusted p-values, and η² gives the share of variance explained by diet.

Common Pitfalls

  • Concluding all groups differ from a significant F; it only says at least one does.

  • Running ANOVA on repeated measurements as if they were independent.

  • Skipping effect sizes.

29

What are effect size and Cohen's d?

Effect size measures how large a difference or relationship is, independent of sample size, which a p-value does not tell you: with huge samples, trivial differences become “significant”. Cohen’s d is the standardized difference between two means: d = (mean₁ − mean₂) / pooled standard deviation, so d = 0.5 means the means differ by half a standard deviation. Cohen’s rough benchmarks are 0.2 small, 0.5 medium and 0.8 large, but the practical meaning depends on the field. Effect sizes are needed for power analysis, for comparing studies and for meta-analysis.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Common Effect Size Measures

Measure Used for
Cohen’s d / Hedges’ g Difference between two means (g corrects small-sample bias)
Pearson r Correlation
η² / partial η² ANOVA (share of variance explained)
Odds ratio, risk ratio Binary outcomes
Cramér’s V Chi-square tests
Absolute difference with units Often the most meaningful for decisions

Code Example

import numpy as np
from scipy import stats

def cohens_d(a, b):
    a, b = np.asarray(a, float), np.asarray(b, float)
    na, nb = len(a), len(b)
    pooled_var = ((na - 1) * a.var(ddof=1) + (nb - 1) * b.var(ddof=1)) / (na + nb - 2)
    return (a.mean() - b.mean()) / np.sqrt(pooled_var)

def hedges_g(a, b):
    n = len(a) + len(b)
    return cohens_d(a, b) * (1 - 3 / (4 * n - 9))

rng = np.random.default_rng(42)
for n in (30, 300, 30_000):
    a = rng.normal(100.5, 15, n)          # true d = 0.5 / 15 ≈ 0.03 (tiny)
    b = rng.normal(100.0, 15, n)
    p = stats.ttest_ind(a, b).pvalue
    print(f"n={n:>6} per group: d = {cohens_d(a, b):6.3f}, p = {p:.4f}")

a = rng.normal(108, 15, 25)
b = rng.normal(100, 15, 25)
print(f"small study: d = {cohens_d(a, b):.3f}, Hedges' g = {hedges_g(a, b):.3f}")

With 30,000 per group a negligible effect of about 0.03 SD can reach p < 0.05, while the effect size stays tiny, which is why both should be reported.

Common Pitfalls

  • Treating Cohen’s benchmarks as universal; a 0.1 effect may matter in medicine or at scale.

  • Using the wrong standard deviation for paired designs (use the SD of differences, or say which one).

  • Reporting effect sizes without confidence intervals.

30

How do you perform a chi-squared test, and what does it show?

A chi-squared test compares observed counts with the counts expected under a null hypothesis. The goodness-of-fit version checks whether one categorical variable follows a specified distribution (e.g. a fair die); the test of independence checks whether two categorical variables are associated, using a contingency table. The statistic χ² = Σ (O − E)² / E measures the total discrepancy; a small p-value means the observed counts would be unlikely if the null hypothesis were true. It tells you whether a difference or association exists, not how strong it is, so report an effect size such as Cramér’s V.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Steps

  1. State H₀ (e.g. the die is fair; or gender and product choice are independent).

  2. Tabulate the observed counts.

  3. Compute expected counts (from the hypothesized proportions, or row total × column total / grand total).

  4. Compute χ² and the degrees of freedom (k − 1 for goodness of fit; (r − 1)(c − 1) for independence).

  5. Get the p-value and check that expected counts are mostly ≥ 5.

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# Goodness of fit: is a die fair?
rolls = rng.choice(6, size=600, p=[0.15, 0.15, 0.15, 0.15, 0.15, 0.25])
observed = np.bincount(rolls, minlength=6)
chi2, p = stats.chisquare(observed)                  # expected: 100 per face
print("observed counts:", observed, f"| chi2 = {chi2:.2f}, p = {p:.4f}")

# Independence: plan type vs region
table = np.array([[120, 90, 40],
                  [80, 110, 60]])
chi2, p, dof, expected = stats.chi2_contingency(table)
cramers_v = np.sqrt(chi2 / (table.sum() * (min(table.shape) - 1)))
print(f"independence: chi2 = {chi2:.2f}, dof = {dof}, p = {p:.4f}, Cramér's V = {cramers_v:.3f}")
print("expected counts:\n", expected.round(1))

The loaded die produces too many sixes, so the goodness-of-fit test rejects fairness; the independence test shows whether plan choice differs by region, and Cramér’s V indicates the association’s strength.

Common Pitfalls

  • Using percentages instead of counts.

  • Small expected counts (use Fisher’s exact test or combine categories).

  • Counting the same person in several cells (observations must be independent).

31

What is a nonparametric test, and why would you use one?

A nonparametric test makes few or no assumptions about the shape of the population distribution, such as normality. Most work with ranks or signs rather than raw values, which makes them robust to outliers and skewed data and suitable for ordinal data (ratings, rankings). You use them when samples are small and clearly non-normal, data are ordinal, or outliers would distort means. The trade-off is somewhat lower power than parametric tests when the parametric assumptions actually hold, and their hypotheses concern distributions or medians rather than means.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Parametric Tests and Their Nonparametric Alternatives

Situation Parametric Nonparametric
Two independent groups Welch t-test Mann-Whitney U
Paired measurements Paired t-test Wilcoxon signed-rank
Three or more groups One-way ANOVA Kruskal-Wallis
Repeated measures, 3+ conditions Repeated-measures ANOVA Friedman
Correlation Pearson Spearman, Kendall
Distribution comparison – Kolmogorov-Smirnov

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
# Skewed response times (seconds) with outliers
old_design = rng.lognormal(1.2, 0.6, 30)
new_design = rng.lognormal(1.0, 0.6, 30)
old_design[:2] = [60, 75]                          # a couple of extreme outliers

print("medians:", round(np.median(old_design), 2), round(np.median(new_design), 2))
print("Shapiro p (old):", round(stats.shapiro(old_design).pvalue, 4))

t = stats.ttest_ind(old_design, new_design, equal_var=False)
u = stats.mannwhitneyu(old_design, new_design, alternative="two-sided")
print(f"Welch t-test p = {t.pvalue:.4f}")
print(f"Mann-Whitney U = {u.statistic:.0f}, p = {u.pvalue:.4f}")

# Effect size for Mann-Whitney: probability a random old value exceeds a random new value
print("P(old > new):", round(u.statistic / (len(old_design) * len(new_design)), 3))

The outliers inflate the variance and weaken the t-test, while the rank-based Mann-Whitney test is unaffected by how extreme they are. A small Mann-Whitney p-value means values in one group tend to be larger than in the other.

Common Pitfalls

  • Saying Mann-Whitney compares medians; it compares distributions, and equals a median test only when shapes are the same.

  • Switching to a nonparametric test after the parametric one “failed” (p-hacking).

  • Using them by default with large samples, where the CLT often makes t-tests fine.

32

What is linear regression, and when is it used?

Linear regression models a continuous outcome as a linear combination of one or more predictors plus random error: y = β₀ + β₁x₁ + … + βₚxₚ + ε. The coefficients are estimated by ordinary least squares, minimizing the sum of squared residuals; each coefficient is the expected change in y for a one-unit change in that predictor, holding the others fixed. It is used to explain relationships (with confidence intervals and tests for each coefficient), to predict numeric outcomes, and as a strong, interpretable baseline before trying complex models.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Typical Uses

Use Example
Explanation How price and advertising relate to sales
Prediction House prices from size, location, age
Trend estimation Growth rate over time
Adjusting for confounders Effect of a treatment controlling for age and sex
Baseline model Benchmark for tree or neural models

When It Is Appropriate

  • Outcome is continuous and roughly linear in the predictors (or can be after transforms).

  • Errors are independent with roughly constant variance.

  • You value interpretability and inference.

For binary outcomes use logistic regression; for counts, Poisson or negative binomial regression.

Code Example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(42)
n = 400
df = pd.DataFrame({
    "size": rng.normal(120, 30, n),                 # square metres
    "age": rng.uniform(0, 60, n),
    "city": rng.choice(["A", "B"], n),
})
df["price"] = (50 + 2.5 * df["size"] - 0.8 * df["age"] + 40 * (df["city"] == "B")
               + rng.normal(0, 25, n))

fit = smf.ols("price ~ size + age + C(city)", data=df).fit()
print(fit.summary().tables[1])
print("R²:", round(fit.rsquared, 3))

new_house = pd.DataFrame({"size": [100], "age": [10], "city": ["B"]})
pred = fit.get_prediction(new_house).summary_frame(alpha=0.05)
print(pred[["mean", "obs_ci_lower", "obs_ci_upper"]].round(1))

The size coefficient (about 2.5) means each extra square metre adds about 2.5 price units with age and city held fixed. The prediction interval (obs_ci_*) is wider than the confidence interval for the mean because it includes individual noise.

Common Pitfalls

  • Extrapolating outside the range of the data.

  • Reading coefficients causally when confounders are missing.

  • Ignoring outliers and influential points (check Cook’s distance).

33

How do you interpret R-squared and adjusted R-squared?

R-squared (R²) is the proportion of variance in the outcome explained by the model: R² = 1 − SS_residual / SS_total. An R² of 0.70 means the model’s predictions explain 70% of the variation around the mean. R² never decreases when you add predictors, even useless ones, so adjusted R² penalizes the number of predictors: 1 − (1 − R²)(n − 1)/(n − p − 1). Adjusted R² rises only when a new variable improves the fit more than chance would; use it to compare models with different numbers of predictors, and remember that neither measures causal validity or out-of-sample accuracy.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Interpretation Guide

Situation Meaning
R² near 1 Model explains most variation (check for leakage or overfitting)
R² near 0 Predictors explain little of the variation
Low R² but significant coefficients Real effects in noisy data (common in social science)
R² rises, adjusted R² falls New predictor adds noise, not information
Large gap between in-sample and test R² Overfitting

What counts as a “good” R² depends on the field: physical measurements may reach 0.99, while human behaviour models with 0.2 can still be useful.

Code Example

import numpy as np
import statsmodels.api as sm

rng = np.random.default_rng(42)
n = 60
x = rng.normal(size=n)
y = 2 + 1.5 * x + rng.normal(0, 2, n)

base = sm.OLS(y, sm.add_constant(x)).fit()
print(f"1 predictor:   R² = {base.rsquared:.3f}, adjusted R² = {base.rsquared_adj:.3f}")

noise = rng.normal(size=(n, 15))                       # 15 useless predictors
bloated = sm.OLS(y, sm.add_constant(np.c_[x, noise])).fit()
print(f"16 predictors: R² = {bloated.rsquared:.3f}, adjusted R² = {bloated.rsquared_adj:.3f}")

# Manual calculation
ss_res = np.sum(base.resid ** 2)
ss_tot = np.sum((y - y.mean()) ** 2)
r2 = 1 - ss_res / ss_tot
p = 1
print("manual R²:", round(r2, 3), "| manual adjusted R²:", round(1 - (1 - r2) * (n - 1) / (n - p - 1), 3))

Adding 15 noise variables raises R² but lowers adjusted R², showing why plain R² cannot be used to choose between models of different size.

Common Pitfalls

  • Treating a high R² as proof that the model is correct or causal.

  • Comparing R² across models with different outcome variables (e.g. y vs log y).

  • Using in-sample R² to judge predictive performance; use cross-validation.

34

What are the assumptions of linear regression?

Linear regression relies on a few assumptions for its estimates and inference to be valid: linearity (the mean of y is a linear function of the predictors), independence of errors, homoscedasticity (constant error variance), normality of errors (mainly for small-sample tests and intervals), and no perfect multicollinearity, plus the key condition that errors are uncorrelated with the predictors (exogeneity, i.e. no omitted confounders). Violations affect different things: some bias the coefficients, others only make standard errors and p-values wrong, so each needs its own check and fix.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

Assumptions, Checks and Fixes

Assumption If violated Check Fix
Linearity Biased predictions and coefficients Residuals vs fitted; component plots Transform variables, splines, polynomial terms
Independent errors SEs too small Durbin-Watson; study design Clustered SEs, time series models
Homoscedasticity Wrong SEs and p-values Breusch-Pagan, funnel-shaped residuals Robust (HC3) SEs, log transform, WLS
Normal errors Unreliable small-sample inference Q-Q plot, Shapiro-Wilk Larger n (CLT), transforms, bootstrap
No perfect multicollinearity Coefficients undefined/unstable VIF Drop/combine predictors, ridge
Exogeneity Biased coefficients Domain reasoning Add confounders, instrumental variables

Normality of the predictors or of y itself is not required, only of the errors.

Code Example: Diagnostics

import numpy as np
import statsmodels.api as sm
from scipy import stats
from statsmodels.stats.diagnostic import het_breuschpagan
from statsmodels.stats.stattools import durbin_watson

rng = np.random.default_rng(42)
n = 300
x1 = rng.uniform(0, 10, n)
x2 = rng.normal(5, 2, n)
y = 1 + 2 * x1 + 0.3 * x1 ** 2 - 1.5 * x2 + rng.normal(0, 2, n)   # true relationship is curved in x1

X = sm.add_constant(np.c_[x1, x2])
fit = sm.OLS(y, X).fit()
resid = fit.resid

print("Breusch-Pagan p:", round(het_breuschpagan(resid, X)[1], 4))
print("Durbin-Watson (≈2 means no autocorrelation):", round(durbin_watson(resid), 2))
print("Shapiro-Wilk p on residuals:", round(stats.shapiro(resid).pvalue, 4))

# Linearity check: residuals correlate with x1² if the curve was missed
print("corr(residuals, x1²):", round(np.corrcoef(resid, x1 ** 2)[0, 1], 3))
fixed = sm.OLS(y, sm.add_constant(np.c_[x1, x1 ** 2, x2])).fit()
print("R² linear:", round(fit.rsquared, 3), "| R² with x1²:", round(fixed.rsquared, 3))

The misspecified model leaves curvature in its residuals; adding the squared term fixes the linearity violation (the model is still “linear” because it is linear in its coefficients).

Common Pitfalls

  • Testing normality of y instead of the residuals.

  • Relying only on formal tests; plots often reveal more.

  • Believing assumptions only matter for p-values: omitted confounders bias the coefficients themselves.

35

What is multicollinearity, and why is it a problem in regression?

Multicollinearity occurs when two or more predictors in a regression are highly correlated, so one can be nearly predicted from the others. It does not bias predictions much, but it makes individual coefficients unstable: their standard errors inflate, signs can flip between samples, and variables can look insignificant even when they jointly matter. Detect it with the variance inflation factor (VIF) (values above about 5-10 are a warning) and the correlation matrix; fix it by removing or combining redundant variables, using PCA, or applying ridge regression.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+, scikit-learn 1.5+

Effects and Remedies

Effect Remedy
Inflated standard errors Drop or combine redundant predictors
Unstable or counter-intuitive signs Ridge regression (shrinks correlated coefficients together)
Hard to attribute effects Principal component regression, domain-driven composite indices
Little effect on prediction accuracy If you only need predictions, it may be acceptable

VIFⱼ = 1 / (1 − R²ⱼ), where R²ⱼ comes from regressing predictor j on the others.

Code Example

import numpy as np
import pandas as pd
import statsmodels.api as sm
from sklearn.linear_model import Ridge
from statsmodels.stats.outliers_influence import variance_inflation_factor

rng = np.random.default_rng(42)
n = 200
size_sqft = rng.normal(1500, 300, n)
rooms = size_sqft / 300 + rng.normal(0, 0.3, n)          # almost a copy of size
age = rng.uniform(0, 50, n)
price = 50 + 0.1 * size_sqft + 5 * rooms - 0.5 * age + rng.normal(0, 15, n)

X = pd.DataFrame({"size_sqft": size_sqft, "rooms": rooms, "age": age})
Xc = sm.add_constant(X)
vif = {col: round(variance_inflation_factor(Xc.values, i), 1) for i, col in enumerate(Xc.columns) if col != "const"}
print("VIF:", vif)

fit = sm.OLS(price, Xc).fit()
print(fit.params.round(3).to_dict())
print("standard errors:", fit.bse.round(3).to_dict())

Xs = (X - X.mean()) / X.std()
ridge = Ridge(alpha=10).fit(Xs, price)
print("ridge coefficients (standardized):", dict(zip(X.columns, ridge.coef_.round(2))))

Size and rooms have very high VIFs, and their OLS coefficients have large standard errors (refitting on another sample could change them a lot), while age is unaffected. Ridge spreads the shared effect more stably between the correlated predictors.

Common Pitfalls

  • Dropping a variable only because its p-value is large when it is collinear with another.

  • Including dummy variables for every category plus an intercept (perfect collinearity, the “dummy variable trap”).

36

What is the difference between correlation and causation?

Correlation means two variables move together: when one is higher, the other tends to be higher (or lower). Causation means changing one variable produces a change in the other. Correlation can arise without causation through confounding (a third variable drives both, as hot weather drives both ice cream sales and drownings), reverse causation (the effect actually causes the “cause”), selection bias, or pure chance, especially when many variables are compared. Establishing causation requires randomized experiments or careful causal designs such as instrumental variables, difference-in-differences or regression discontinuity.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Why Correlation Is Not Causation

Explanation Example
Confounding Ice cream sales and drownings both rise with temperature
Reverse causation Hospitals have sicker people, but hospitals do not cause illness
Selection bias Studying only successful startups
Chance / multiple comparisons Testing hundreds of variables finds spurious correlations
Simpson’s paradox A trend reverses when groups are combined

Ways to Establish Causation

  • Randomized controlled trials (A/B tests).

  • Natural experiments, instrumental variables, regression discontinuity, difference-in-differences.

  • Adjusting for confounders (regression, matching) when they are measured, guided by a causal diagram (DAG).

Code Example: A Confounder Creates Correlation

import numpy as np
import statsmodels.api as sm

rng = np.random.default_rng(42)
n = 2000
temperature = rng.normal(25, 5, n)
ice_cream = 20 + 4 * temperature + rng.normal(0, 10, n)
drownings = 1 + 0.3 * temperature + rng.normal(0, 2, n)     # no direct link to ice cream

print("correlation(ice cream, drownings):", round(np.corrcoef(ice_cream, drownings)[0, 1], 3))

naive = sm.OLS(drownings, sm.add_constant(ice_cream)).fit()
adjusted = sm.OLS(drownings, sm.add_constant(np.c_[ice_cream, temperature])).fit()
print("ice cream coefficient, naive:   ", round(naive.params[1], 4), "p =", f"{naive.pvalues[1]:.2g}")
print("ice cream coefficient, adjusted:", round(adjusted.params[1], 4), "p =", f"{adjusted.pvalues[1]:.2g}")

The naive regression finds a strongly “significant” link; once temperature is included, the ice cream coefficient falls to about zero. Adjustment only works for confounders you measure, which is why randomization is the gold standard.

Common Pitfalls

  • Adjusting for variables caused by the treatment (mediators or colliders), which can create bias.

  • Claiming causation from a high R² or a small p-value.

37

How do you detect and fix heteroscedasticity in regression?

Heteroscedasticity means the variance of regression residuals is not constant: for example, errors grow with income or firm size. Ordinary least squares coefficients stay unbiased, but their standard errors are wrong, so p-values and confidence intervals are unreliable, and OLS is no longer the most efficient estimator. Detect it with a residuals vs fitted plot (a funnel shape) and formal tests (Breusch-Pagan, White). Remedies: heteroscedasticity-robust standard errors (HC3), transforming the outcome (log), weighted least squares when the variance structure is known, or modelling the variance directly.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Detection and Remedies

Step Tool
Visual Residuals vs fitted values; scale-location plot
Test Breusch-Pagan (het_breuschpagan), White (het_white)
Quick fix for inference Robust (HC3) standard errors
Transform Log or Box-Cox of y when variance grows with the mean
Weighting WLS with weights ∝ 1/variance
Different model GLM (Poisson, Gamma) whose variance depends on the mean

Code Example

import numpy as np
import statsmodels.api as sm
from statsmodels.stats.diagnostic import het_breuschpagan

rng = np.random.default_rng(42)
n = 500
x = rng.uniform(1, 10, n)
y = 2 + 3 * x + rng.normal(0, 0.5 * x)          # error SD grows with x

X = sm.add_constant(x)
ols = sm.OLS(y, X).fit()
lm_stat, lm_p, _, _ = het_breuschpagan(ols.resid, X)
print(f"Breusch-Pagan p-value: {lm_p:.2e}")      # small → heteroscedasticity

robust = sm.OLS(y, X).fit(cov_type="HC3")
print("slope SE: classic", round(ols.bse[1], 4), "| robust HC3", round(robust.bse[1], 4))

wls = sm.WLS(y, X, weights=1 / x ** 2).fit()     # variance ∝ x², so weight by 1/x²
print("WLS slope:", round(wls.params[1], 3), "SE", round(wls.bse[1], 4))
print("WLS residual Breusch-Pagan p:",
      round(het_breuschpagan(wls.resid * np.sqrt(1 / x ** 2), X)[1], 3))

The Breusch-Pagan test’s null is constant variance, so a tiny p-value confirms the problem. Robust standard errors fix inference without changing coefficients; WLS gives more efficient estimates when the weights are right.

Common Pitfalls

  • Trusting classic OLS p-values without checking residual plots.

  • Guessing WLS weights badly; robust standard errors are the safer default.

  • Confusing heteroscedasticity with non-linearity (a curved residual pattern needs a different model).

38

What is logistic regression, and how does it differ from linear regression?

Logistic regression models the probability of a binary outcome (yes/no, churn/stay) by passing a linear combination of predictors through the sigmoid function, so predictions lie between 0 and 1; equivalently, it models the log-odds as linear in the predictors. It is fitted by maximum likelihood, and each coefficient is a change in log-odds, so exp(coefficient) is an odds ratio. Linear regression models a continuous outcome directly as a linear function, is fitted by least squares, and its coefficients are changes in the expected value; using it for binary outcomes can produce probabilities below 0 or above 1.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Comparison

Aspect Linear regression Logistic regression
Outcome Continuous Binary (multinomial/ordinal extensions)
Model E[y] = β₀ + β₁x log(p / (1 − p)) = β₀ + β₁x
Fitting Least squares Maximum likelihood
Coefficient meaning Change in y per unit x Change in log-odds; exp(β) = odds ratio
Error distribution Normal Bernoulli
Fit measures R², RMSE Log-loss, AUC, pseudo-R², calibration
Predictions Any real number Probabilities in (0, 1)

Code Example

import numpy as np
import statsmodels.api as sm

rng = np.random.default_rng(42)
n = 1000
hours = rng.uniform(0, 10, n)
p_pass = 1 / (1 + np.exp(-(-4 + 0.8 * hours)))
passed = rng.binomial(1, p_pass)

X = sm.add_constant(hours)
logit = sm.Logit(passed, X).fit(disp=False)
print(logit.params.round(3))
print("odds ratio per extra hour:", round(np.exp(logit.params[1]), 3))
print("predicted P(pass) at 2, 5, 8 hours:", logit.predict(sm.add_constant(np.array([2.0, 5.0, 8.0]))).round(3))

# Linear probability model for comparison
lpm = sm.OLS(passed, X).fit()
print("linear model predictions at 0 and 12 hours:", lpm.predict(sm.add_constant(np.array([0.0, 12.0]))).round(3))

An odds ratio above 1 means each extra study hour multiplies the odds of passing by that factor. The linear model’s predictions can fall outside [0, 1] when extrapolated, which logistic regression avoids.

Common Pitfalls

  • Interpreting logistic coefficients as changes in probability (the effect on probability depends on the starting point).

  • Ignoring class imbalance and calibration when using predicted probabilities.

  • Perfect separation, which makes coefficients diverge; use regularization.

39

What is a time series, and how does it differ from other data?

A time series is a sequence of observations of the same variable recorded in time order, usually at regular intervals: daily stock prices, hourly temperature, monthly sales. It differs from ordinary (cross-sectional) data because the order matters and observations are not independent: today’s value is correlated with yesterday’s (autocorrelation), and the series can contain trend, seasonality, cycles and changing variance. This changes how you model it (ARIMA, exponential smoothing, state-space and sequence models) and how you validate it: always on later periods, never with random splits.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Differences from Cross-Sectional Data

Aspect Cross-sectional data Time series
Ordering Irrelevant Essential
Independence Often assumed Violated (autocorrelation)
Structure Relationships between variables Trend, seasonality, cycles, noise
Validation Random k-fold Time-ordered (walk-forward) splits
Goal Explain or predict across units Forecast future values, detect changes

Components

Component Example
Trend Long-term growth in users
Seasonality Higher retail sales every December
Cycle Multi-year business cycles (irregular length)
Irregular Random noise, one-off events

Code Example: Decomposition and Autocorrelation

import numpy as np
import pandas as pd
from statsmodels.tsa.seasonal import seasonal_decompose

rng = np.random.default_rng(42)
idx = pd.date_range("2020-01-01", periods=48, freq="MS")
t = np.arange(48)
sales = pd.Series(200 + 3 * t + 25 * np.sin(2 * np.pi * t / 12) + rng.normal(0, 8, 48), index=idx)

parts = seasonal_decompose(sales, model="additive", period=12)
print("trend (last 3):", parts.trend.dropna().tail(3).round(1).tolist())
print("seasonal effect by month:", parts.seasonal.iloc[:12].round(1).tolist())

print("lag-1 autocorrelation:", round(sales.autocorr(lag=1), 3))
print("lag-12 autocorrelation:", round(sales.autocorr(lag=12), 3))

shuffled = pd.Series(rng.permutation(sales.to_numpy()))
print("lag-1 autocorrelation after shuffling:", round(shuffled.autocorr(lag=1), 3))

Shuffling destroys the autocorrelation: the same values in a different order are a different time series, which is exactly why order matters.

Common Pitfalls

  • Random train/test splits, which leak the future into training.

  • Ignoring autocorrelation, which makes standard errors too small.

  • Mixing up seasonality (fixed period) with cycles (variable length).

40

What are autocorrelation and partial autocorrelation in time series?

Autocorrelation (ACF) at lag k is the correlation between a series and itself shifted by k periods; it includes both direct effects and indirect effects carried through the intermediate lags. Partial autocorrelation (PACF) at lag k is the correlation between yₜ and yₜ₋ₖ after removing the effects of lags 1 to k − 1, so it shows only the direct relationship. Together they identify ARIMA orders: an AR(p) process has a PACF that cuts off after lag p and an ACF that decays; an MA(q) process has an ACF that cuts off after lag q and a PACF that decays.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Identification Patterns

Process ACF PACF
White noise No significant lags No significant lags
AR(p) Decays gradually Cuts off after lag p
MA(q) Cuts off after lag q Decays gradually
ARMA(p, q) Decays after lag q Decays after lag p
Random walk Very slow decay Large spike at lag 1
Seasonal (period m) Spikes at m, 2m, … Spike at m

Approximate 95% significance bounds are ±1.96/√n.

Code Example

import numpy as np
from statsmodels.tsa.stattools import acf, pacf

rng = np.random.default_rng(42)
n = 1000
e = rng.normal(size=n)

ar2 = np.zeros(n)
for t in range(2, n):
    ar2[t] = 0.5 * ar2[t - 1] + 0.3 * ar2[t - 2] + e[t]
ma1 = e + 0.8 * np.r_[0, e[:-1]]

bound = 1.96 / np.sqrt(n)
for name, series in (("AR(2)", ar2), ("MA(1)", ma1)):
    a = acf(series, nlags=5)[1:]
    p = pacf(series, nlags=5)[1:]
    print(name)
    print("  ACF  lags 1-5:", a.round(2))
    print("  PACF lags 1-5:", p.round(2), f"(significance bound ±{bound:.3f})")

For AR(2), the PACF has two large values and then falls inside the bounds; for MA(1), the ACF has one large value at lag 1 and then near zero. plot_acf and plot_pacf in statsmodels.graphics.tsaplots draw the same with shaded bounds.

Common Pitfalls

  • Reading ACF/PACF of a non-stationary series; difference first.

  • Treating one lag barely outside the bounds as meaningful (about 1 in 20 will be by chance).

  • Relying only on plots; confirm orders with AIC and residual checks.

41

What is stationarity in a time series, and why does it matter?

A time series is (weakly) stationary when its mean and variance are constant over time and the covariance between two values depends only on the gap (lag) between them, not on when they occur. Stationarity matters because most classical time series models (AR, MA, ARMA) and their statistical results assume it: with a stable process, past patterns are informative about the future, parameter estimates are consistent, and relationships between series are not spurious. Trends, seasonality and changing variance break stationarity and must be removed or modelled.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Stationary vs Non-Stationary

Series Stationary? Why
White noise Yes Constant mean and variance, no autocorrelation
AR(1) with |φ| < 1 Yes Reverts to its mean
Random walk No Variance grows with time (unit root)
Series with trend No Mean changes over time
Monthly sales with seasonality No Mean depends on the month

Why It Matters

  • ARMA models and their forecasts assume stationarity.

  • Spurious regression: regressing one random walk on another often gives a “significant” relationship that does not exist.

  • Sample statistics (mean, autocorrelation) only estimate something meaningful when they are stable.

Code Example: Testing Stationarity

import numpy as np
from statsmodels.tsa.stattools import adfuller, kpss

rng = np.random.default_rng(42)
n = 500
noise = rng.normal(size=n)
ar1 = np.zeros(n)
for t in range(1, n):
    ar1[t] = 0.7 * ar1[t - 1] + noise[t]
random_walk = np.cumsum(noise)

for name, series in (("AR(1)", ar1), ("random walk", random_walk), ("differenced walk", np.diff(random_walk))):
    adf_p = adfuller(series)[1]
    kpss_p = kpss(series, regression="c", nlags="auto")[1]
    print(f"{name:>16}: ADF p = {adf_p:.4f} (small → stationary), KPSS p = {kpss_p:.3f} (small → non-stationary)")

# Spurious regression between two independent random walks
other_walk = np.cumsum(rng.normal(size=n))
print("correlation of two independent random walks:", round(np.corrcoef(random_walk, other_walk)[0, 1], 3))

The ADF test’s null is a unit root; KPSS’s null is stationarity. Using both gives a more reliable verdict. Independent random walks can show large correlations by chance, which is the spurious regression problem.

Common Pitfalls

  • Confusing “stationary” with “constant”; stationary series still fluctuate.

  • Relying on a single test with low power on short series.

42

What methods can make a non-stationary time series stationary?

Non-stationary series are made stationary by removing whatever changes over time: differencing removes stochastic trends (unit roots), seasonal differencing removes repeating seasonal patterns, log or Box-Cox transforms stabilize variance that grows with the level, and detrending by regression removes deterministic trends. Decomposition (STL) can separate trend and seasonality explicitly. After each step, check with ADF/KPSS tests and ACF plots, and keep track of the transformations so forecasts can be converted back to the original scale.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Methods

Method Removes Example
First difference Stochastic trend y′ₜ = yₜ − yₜ₋₁
Seasonal difference Seasonality y′ₜ = yₜ − yₜ₋₁₂ (monthly)
Log transform Variance growing with level log(sales)
Box-Cox General variance stabilization λ chosen by maximum likelihood
Detrending Deterministic trend Residuals from a regression on time
STL decomposition Trend and seasonality Model the remainder

Code Example

import numpy as np
import pandas as pd
from scipy import stats
from statsmodels.tsa.seasonal import STL
from statsmodels.tsa.stattools import adfuller

rng = np.random.default_rng(42)
t = np.arange(120)
series = pd.Series(
    np.exp(3 + 0.015 * t + 0.15 * np.sin(2 * np.pi * t / 12) + rng.normal(0, 0.03, 120)),
    index=pd.date_range("2016-01-01", periods=120, freq="MS"),
)

def adf_p(s):
    return round(adfuller(s.dropna())[1], 4)

print("original:", adf_p(series))
logged = np.log(series)
print("log:", adf_p(logged))
seasonal_diff = logged.diff(12)
print("log + seasonal difference:", adf_p(seasonal_diff))

transformed, lam = stats.boxcox(series)
print("Box-Cox lambda:", round(lam, 3))

detrended = logged - np.polyval(np.polyfit(t, logged, 1), t)
print("log + linear detrend:", adf_p(detrended))

remainder = STL(logged, period=12, robust=True).fit().resid
print("STL remainder:", adf_p(remainder))

Here the trend is deterministic, so detrending (or seasonal differencing) is enough; for a random-walk trend, ordinary differencing is the right tool.

Common Pitfalls

  • Differencing a series with a deterministic trend more than needed.

  • Applying log or Box-Cox to data with zeros or negative values.

  • Forgetting to reverse transforms (and adjust for the log bias) when reporting forecasts.

43

What is ARIMA, and how is it used to forecast time series?

ARIMA (AutoRegressive Integrated Moving Average) models a univariate time series with three parts: AR(p), which regresses the value on its own p past values; I(d), which differences the series d times to make it stationary; and MA(q), which models the value as a combination of the q most recent forecast errors. Forecasting follows the Box-Jenkins cycle: make the series stationary, identify p and q from ACF/PACF plots or information criteria (AIC), fit, check that residuals look like white noise, then forecast with prediction intervals. SARIMA adds seasonal terms (P, D, Q)ₘ.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Components

Part Parameter Role Identified by
AR p Dependence on past values PACF cuts off after lag p
I d Number of differences ADF/KPSS tests
MA q Dependence on past errors ACF cuts off after lag q
Seasonal (P, D, Q)ₘ Same at the seasonal lag m Spikes at lags m, 2m

Code Example

import numpy as np
import pandas as pd
from statsmodels.stats.diagnostic import acorr_ljungbox
from statsmodels.tsa.arima.model import ARIMA

rng = np.random.default_rng(42)
n = 200
e = rng.normal(0, 1, n)
x = np.zeros(n)
for t in range(1, n):
    x[t] = 0.6 * x[t - 1] + e[t] + 0.3 * e[t - 1]       # ARMA(1,1) increments
y = pd.Series(50 + np.cumsum(x), index=pd.date_range("2024-01-01", periods=n, freq="D"))

train, test = y[:-20], y[-20:]
candidates = {}
for p in range(3):
    for q in range(3):
        candidates[(p, 1, q)] = ARIMA(train, order=(p, 1, q)).fit().aic
best = min(candidates, key=candidates.get)
print("best order by AIC:", best)

fit = ARIMA(train, order=best).fit()
print("Ljung-Box p-value on residuals:", round(acorr_ljungbox(fit.resid, lags=[10])["lb_pvalue"].iloc[0], 3))
fc = fit.get_forecast(steps=20)
interval = fc.conf_int(alpha=0.05)
mae = np.abs(fc.predicted_mean - test).mean()
naive_mae = np.abs(train.iloc[-1] - test).mean()
print(f"forecast MAE {mae:.3f} vs naive (last value) MAE {naive_mae:.3f}")
print("share of test points inside 95% interval:",
      ((test >= interval.iloc[:, 0]) & (test <= interval.iloc[:, 1])).mean())

A large Ljung-Box p-value means no remaining autocorrelation in residuals, so the model captured the structure. Always compare with a naive baseline.

Common Pitfalls

  • Fitting ARIMA to a series with strong seasonality without seasonal terms.

  • Over-differencing (d too high).

  • Expecting ARIMA to capture effects of external drivers; use ARIMAX/SARIMAX with exogenous variables.

  • Long-horizon forecasts: intervals widen quickly, and point forecasts revert to the trend or mean.

44

What is the purpose of dimensionality reduction in data analysis?

Dimensionality reduction represents data with fewer variables while keeping the information that matters. Its purposes are to fight the curse of dimensionality (sparse data, unreliable distances), reduce overfitting and model complexity, remove redundancy from correlated features, speed up training and storage, reduce noise, and visualize high-dimensional data in 2D or 3D. There are two families: feature selection keeps a subset of original variables, and feature extraction (PCA, autoencoders, t-SNE, UMAP) creates new combined variables.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

Methods

Method Type Linear? Main use
Variance / correlation filters Selection n/a Remove useless or duplicate features
L1 regularization (Lasso) Selection Yes Sparse predictive models
PCA Extraction Yes Compression, decorrelation, preprocessing
LDA Extraction (supervised) Yes Class separation
t-SNE, UMAP Extraction No Visualization of clusters
Autoencoders Extraction No Complex data (images, signals)

Code Example

import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_digits(return_X_y=True)        # 64 pixel features
print("original features:", X.shape[1])

pipelines = {
    "all 64 features": make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000)),
    "PCA 20 components": make_pipeline(StandardScaler(), PCA(n_components=20, random_state=42),
                                       LogisticRegression(max_iter=2000)),
    "best 20 pixels": make_pipeline(StandardScaler(), SelectKBest(f_classif, k=20),
                                    LogisticRegression(max_iter=2000)),
}
for name, pipe in pipelines.items():
    print(f"{name:>18}: CV accuracy {cross_val_score(pipe, X, y, cv=5).mean():.3f}")

pca = PCA(random_state=42).fit(StandardScaler().fit_transform(X))
print("components for 90% variance:", int(np.searchsorted(np.cumsum(pca.explained_variance_ratio_), 0.9) + 1))

Comparing accuracy with fewer inputs shows how much information the reduced representation keeps. The reducers sit inside pipelines so they are fitted only on training folds.

Common Pitfalls

  • Fitting the reduction on all data before splitting (leakage).

  • Using t-SNE/UMAP output as features for a model, or reading distances between t-SNE clusters literally.

  • Losing interpretability when stakeholders need to know which original variables matter.

45

What is Principal Component Analysis, and what are its applications?

Principal Component Analysis (PCA) is a linear dimensionality-reduction method that finds new orthogonal axes, the principal components, ordered by how much of the data’s variance they capture. Each component is a weighted combination of the original (usually standardized) features; projecting onto the first few keeps most of the information in fewer dimensions. PCA is used for visualization, noise reduction, compression, removing multicollinearity before regression, and speeding up downstream models.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

How It Works

  1. Standardize features (otherwise large-scale features dominate).

  2. Compute the covariance matrix, or the SVD of the centered data.

  3. Eigenvectors are the components; eigenvalues are the variance each explains.

  4. Keep k components (e.g. 90-95% cumulative explained variance, or a scree-plot elbow).

  5. Project the data onto them.

Applications

Application Example
Visualization Plot high-dimensional data in 2D
Compression Images, gene expression, sensor arrays
Noise reduction Drop low-variance components
Multicollinearity Principal component regression
Feature engineering Inputs for clustering or classifiers
Finance Yield-curve factors (level, slope, curvature)

Code Example

import numpy as np
from sklearn.datasets import load_wine
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_wine(return_X_y=True)                     # 13 features
pca = make_pipeline(StandardScaler(), PCA(random_state=42)).fit(X)
ratios = pca[-1].explained_variance_ratio_
print("explained variance ratios:", ratios.round(3)[:5])
print("components for 90%:", int(np.searchsorted(np.cumsum(ratios), 0.90) + 1))

loadings = pca[-1].components_[0]
print("largest |loadings| on PC1 (feature index):", np.argsort(-np.abs(loadings))[:3])

for k in (2, 5, 13):
    model = make_pipeline(StandardScaler(), PCA(n_components=k, random_state=42),
                          LogisticRegression(max_iter=1000))
    print(f"{k:>2} components: CV accuracy {cross_val_score(model, X, y, cv=5).mean():.3f}")

Comparing accuracy at different k shows how much predictive information a few components retain. PCA sits inside the pipeline, so it is refit on each training fold.

Limitations

  • Linear only; t-SNE, UMAP or kernel PCA handle non-linear structure.

  • Components maximize variance, which may not be the variance that matters for the target.

  • Components can be hard to interpret.

46

How does factor analysis differ from PCA?

PCA and factor analysis both reduce many correlated variables to fewer dimensions, but they have different goals and models. PCA is a descriptive transformation: components are linear combinations of the observed variables chosen to explain as much total variance as possible, with no error model. Factor analysis (FA) is a statistical model: each observed variable is a combination of a few unobserved latent factors plus a variable-specific error, and it explains only the shared (common) variance, the correlations between variables. FA suits measuring underlying constructs (e.g. personality traits from questionnaire items); PCA suits compression and preprocessing.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

Comparison

Aspect PCA Factor analysis
Model Components = weighted sums of variables Variables = loadings × factors + unique error
Variance explained Total (common + unique) Common only; unique variance modelled separately
Goal Data reduction, compression Discover latent constructs
Result uniqueness Unique (up to sign) Rotations (varimax, promax) give equivalent solutions
Typical fields ML preprocessing, imaging Psychology, social science, survey design

Code Example

import numpy as np
from sklearn.decomposition import PCA, FactorAnalysis
from sklearn.preprocessing import StandardScaler

rng = np.random.default_rng(42)
n = 1000
verbal, math = rng.normal(size=n), rng.normal(size=n)     # two latent abilities
noise = lambda sd: rng.normal(0, sd, n)
X = np.c_[
    verbal + noise(0.5), verbal + noise(0.6), verbal + noise(0.7),    # reading tests
    math + noise(0.5), math + noise(0.6), math + noise(0.7),          # math tests
]
X = StandardScaler().fit_transform(X)

fa = FactorAnalysis(n_components=2, rotation="varimax", random_state=42).fit(X)
print("factor loadings (rows = factors):\n", fa.components_.round(2))
print("unique variances:", fa.noise_variance_.round(2))

pca = PCA(n_components=2, random_state=42).fit(X)
print("PCA components:\n", pca.components_.round(2))
print("PCA explained variance ratio:", pca.explained_variance_ratio_.round(3))

After varimax rotation, each factor loads mainly on one group of tests, recovering the “verbal” and “math” abilities, and FA estimates each test’s unique noise variance, which PCA does not model.

Common Pitfalls

  • Treating PCA components as latent constructs.

  • Running FA on too few observations or weakly correlated variables (check the Kaiser-Meyer-Olkin measure and Bartlett’s test).

  • Choosing the number of factors only by eigenvalues > 1; use parallel analysis or theory.

47

What is the curse of dimensionality?

The curse of dimensionality describes problems that appear as the number of features grows: the volume of the space grows exponentially, so data become sparse; the amount of data needed to cover the space grows exponentially; and distances between points become nearly equal, so nearest-neighbour and distance-based methods lose meaning. Models also have more parameters to fit, raising the risk of overfitting. Remedies include feature selection, dimensionality reduction, regularization and models that exploit structure in the data.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Effects

Effect Consequence
Sparsity Few neighbours in any region; density estimation fails
Distance concentration Nearest and farthest neighbours almost equally far
Exponential data needs Covering a grid with 10 bins per axis needs 10ᵈ cells
Overfitting Many features relative to samples fit noise
Volume at the edges Most of a hypercube’s volume lies near its surface

Code Example: Distances Concentrate

import numpy as np
from scipy.spatial.distance import cdist

rng = np.random.default_rng(42)
for d in (2, 10, 100, 1000):
    X = rng.random((500, d))
    query = rng.random((1, d))
    dist = cdist(query, X)[0]
    contrast = (dist.max() - dist.min()) / dist.min()
    print(f"d={d:>4}: (max - min) / min distance = {contrast:.3f}")

# Fraction of a hypercube within 10% of its boundary
for d in (2, 10, 50):
    print(f"d={d:>2}: share of volume near the surface = {1 - 0.8 ** d:.3f}")

As d grows, the relative contrast between the nearest and farthest points shrinks toward zero, which is why k-NN and clustering by Euclidean distance degrade on raw high-dimensional data.

Remedies

  • Feature selection (filter, L1 regularization, model-based importance).

  • Dimensionality reduction (PCA, autoencoders, embeddings).

  • More data, and models with strong inductive biases (CNNs for images).

  • Regularization and cross-validation to control overfitting.

Common Pitfalls

  • Adding features “just in case” with a small sample.

  • Selecting features on the full dataset before cross-validation (leakage).

48

What is SVD, and how is it used in machine learning?

Singular Value Decomposition (SVD) factorizes any m × n matrix as X = U Σ Vᵀ, where U and V have orthonormal columns (left and right singular vectors) and Σ is diagonal with non-negative singular values in decreasing order. Keeping only the top k singular values gives the best rank-k approximation of X in the least-squares sense (Eckart-Young theorem). In machine learning SVD underlies PCA, latent semantic analysis for text, matrix-factorization recommenders, compression and denoising, and numerically stable least squares and pseudo-inverses.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

Uses

Application How SVD is used
PCA SVD of the centered data; right singular vectors are principal axes
Latent semantic analysis Truncated SVD of a TF-IDF term-document matrix finds topics
Recommenders Low-rank factorization of the user-item matrix
Compression / denoising Keep the top k components of an image or signal
Least squares Pseudo-inverse V Σ⁺ Uᵀ handles rank-deficient problems

Code Example: Low-Rank Approximation and Truncated SVD

import numpy as np
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer

rng = np.random.default_rng(42)
# A rank-3 matrix plus noise
X = rng.normal(size=(100, 3)) @ rng.normal(size=(3, 40)) + 0.1 * rng.normal(size=(100, 40))

U, S, Vt = np.linalg.svd(X, full_matrices=False)
print("top singular values:", S[:6].round(2))      # three large, then small noise values

for k in (1, 3, 10):
    X_k = U[:, :k] @ np.diag(S[:k]) @ Vt[:k]
    err = np.linalg.norm(X - X_k) / np.linalg.norm(X)
    print(f"rank {k}: relative reconstruction error {err:.4f}")

docs = ["cats purr and meow", "dogs bark and wag", "kittens meow softly",
        "puppies bark loudly", "stocks rose today", "markets fell on rates"]
tfidf = TfidfVectorizer().fit_transform(docs)
lsa = TruncatedSVD(n_components=2, random_state=42)
print("document coordinates in 2 latent dimensions:\n", lsa.fit_transform(tfidf).round(2))

The sharp drop after the third singular value reveals the true rank, and rank 3 already reconstructs the matrix almost perfectly. TruncatedSVD works directly on sparse matrices, unlike PCA.

Common Pitfalls

  • Forgetting to center data when you want PCA rather than plain SVD.

  • Computing a full SVD of a huge matrix; use truncated or randomized SVD.

  • Using classic SVD on a ratings matrix with missing values; recommenders use factorization methods that fit only observed entries.

49

What is A/B testing, and why is it important in statistics?

A/B testing is a randomized controlled experiment that compares two versions of something, A (control) and B (treatment), by randomly assigning users to each and measuring a predefined metric such as conversion rate. Statistical hypothesis testing then decides whether the observed difference is larger than random variation would explain. It is important because randomization makes it one of the few reliable ways to establish causation in business settings, replacing opinions and misleading before/after comparisons with evidence.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Core Concepts

Concept Meaning
Randomization Removes selection bias and confounding
Null hypothesis No difference between A and B
p-value Probability of a difference at least this extreme if the null is true
Significance level (α) Accepted false-positive rate, e.g. 0.05
Power Probability of detecting a true effect of a given size
Confidence interval Plausible range for the true difference
Practical significance Whether the effect is large enough to matter

Why It Matters

  • Measures causal impact of product, pricing and marketing changes.

  • Quantifies uncertainty instead of relying on intuition.

  • Protects against regressions through guardrail metrics.

  • Builds a culture of incremental, evidence-based decisions.

Code Example: Simulated Test and False-Positive Check

import numpy as np
from statsmodels.stats.proportion import proportions_ztest

rng = np.random.default_rng(42)
n = 10_000

conv_a = rng.binomial(n, 0.10)
conv_b = rng.binomial(n, 0.115)
z, p = proportions_ztest([conv_b, conv_a], [n, n])
print(f"A {conv_a / n:.3f}, B {conv_b / n:.3f}, z = {z:.2f}, p = {p:.4f}")

# A/A tests: when there is no real difference, about 5% are "significant" at alpha 0.05
false_pos = 0
for _ in range(1000):
    a, b = rng.binomial(n, 0.10), rng.binomial(n, 0.10)
    false_pos += proportions_ztest([a, b], [n, n])[1] < 0.05
print("false-positive rate in A/A tests:", false_pos / 1000)

The A/A simulation shows what α means in practice: even with no difference, about 1 in 20 tests looks significant, which is why peeking or testing many metrics inflates false discoveries.

Limitations

  • Needs enough traffic for useful power.

  • Measures short-term effects; long-term effects may differ.

  • Network effects (marketplaces, social apps) violate independence between groups.

50

How do you design an A/B test and determine its sample size?

Designing an A/B test means choosing a hypothesis, a single primary metric, the randomization unit, guardrail metrics, and the minimum detectable effect (MDE), the smallest change worth detecting. Sample size then follows from four inputs: the baseline rate or variance, the MDE, the significance level α (false-positive rate, usually 0.05) and the power 1 − β (probability of detecting a true effect of the MDE, usually 0.8). Divide the required users by daily traffic to get the duration, and round up to whole weeks.

Read full answer

Applies toPython 3.12, NumPy 2.x, statsmodels 0.14+

Design Checklist

Item Example
Hypothesis A shorter checkout increases purchase rate
Primary metric Purchase conversion per user
Unit User (hashed user ID)
Guardrails Refund rate, page load time
Baseline 4.0% conversion
MDE +0.4 percentage points (10% relative)
α / power 0.05 two-sided / 0.8
Duration Required users ÷ daily traffic, whole weeks

Sample Size Formula (Two Proportions)

n per group ≈ (z₁₋α/₂ + z₁₋β)² × [p₁(1 − p₁) + p₂(1 − p₂)] / (p₂ − p₁)²

Halving the MDE roughly quadruples the sample size.

Code Example

import numpy as np
from scipy import stats
from statsmodels.stats.power import NormalIndPower, TTestIndPower
from statsmodels.stats.proportion import proportion_effectsize

p1, p2, alpha, power = 0.040, 0.044, 0.05, 0.80

# Formula
z_a, z_b = stats.norm.ppf(1 - alpha / 2), stats.norm.ppf(power)
n_formula = (z_a + z_b) ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) / (p2 - p1) ** 2
print("per group (formula):", int(np.ceil(n_formula)))

# statsmodels (Cohen's h effect size)
h = proportion_effectsize(p2, p1)
n_sm = NormalIndPower().solve_power(effect_size=h, alpha=alpha, power=power)
print("per group (statsmodels):", int(np.ceil(n_sm)))

daily_users = 8_000
days = 2 * n_sm / daily_users
print("days needed:", int(np.ceil(days)), "→ run", int(np.ceil(days / 7)) * 7, "days")

# Continuous metric: detect a 0.5 change in mean with SD 8
n_cont = TTestIndPower().solve_power(effect_size=0.5 / 8, alpha=alpha, power=power)
print("per group (revenue metric):", int(np.ceil(n_cont)))

for mde in (0.002, 0.004, 0.008):
    n = NormalIndPower().solve_power(proportion_effectsize(p1 + mde, p1), alpha=alpha, power=power)
    print(f"MDE +{mde:.3f}: {int(np.ceil(n)):,} per group")

The loop shows the trade-off: smaller effects need far larger samples, so the MDE must be chosen from business value, not hope.

Common Pitfalls

  • Choosing an MDE so small the test would run for months.

  • Stopping early when results look good.

  • Forgetting variance reduction (e.g. CUPED with pre-experiment data), which can cut the required sample.

51

What are control and treatment groups in an experiment?

The treatment group receives the intervention being tested (a new drug, website design or teaching method), while the control group does not; it gets a placebo, the current standard, or nothing. Because subjects are randomly assigned to the groups, the two groups are similar on average in every other respect, measured and unmeasured, so any difference in outcomes can be attributed to the treatment. The control group provides the counterfactual: what would have happened without the intervention.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Types of Control

Control type Example Purpose
Placebo Sugar pill Separates psychological from real effects
Active / standard of care Existing drug or current website Measures improvement over what is used now
No treatment Nothing Measures absolute effect
Holdout Users who see no ads Measures incremental impact of marketing

Why Randomization Matters

Without random assignment, groups can differ systematically: motivated customers opt into a loyalty program and would have spent more anyway. Randomization balances such confounders, and blinding prevents expectations from changing behaviour or measurement.

Code Example: Randomization Balances a Hidden Confounder

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
n = 2000
motivation = rng.normal(0, 1, n)                   # unobserved confounder

# Self-selection: motivated people choose the program
self_selected = rng.random(n) < 1 / (1 + np.exp(-2 * motivation))
# Randomized: a coin flip decides
randomized = rng.random(n) < 0.5

true_effect = 5.0
def outcome(treated):
    return 50 + 10 * motivation + true_effect * treated + rng.normal(0, 5, n)

for name, treated in (("self-selected", self_selected), ("randomized", randomized)):
    y = outcome(treated)
    diff = y[treated].mean() - y[~treated].mean()
    balance = motivation[treated].mean() - motivation[~treated].mean()
    p = stats.ttest_ind(y[treated], y[~treated], equal_var=False).pvalue
    print(f"{name:>13}: estimated effect {diff:5.2f} (true {true_effect}), "
          f"motivation gap {balance:.2f}, p = {p:.2g}")

The self-selected comparison greatly overstates the effect because motivated people are over-represented in treatment; the randomized comparison recovers roughly the true effect.

Common Pitfalls

  • Letting people or staff choose who gets the treatment.

  • Comparing to a historical control period affected by other changes.

  • Contamination: control subjects getting the treatment (e.g. sharing a promo code).

52

How do you use hypothesis testing to analyze A/B test results?

Analysing an A/B test with hypothesis testing means stating the null hypothesis (no difference between variants in the primary metric) and alternative, choosing the significance level (usually 0.05) and test before the experiment, then computing the test statistic and p-value, the probability of a difference at least as extreme as observed if the null is true. For conversion rates use a two-proportion z-test or chi-square test; for continuous metrics like revenue use Welch’s t-test (or bootstrap for heavy tails). Report the effect size with a confidence interval and judge practical significance as well as statistical significance.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

Steps

Step Example
Hypotheses H₀: p_B = p_A; H₁: p_B ≠ p_A
Alpha, power, sample size α = 0.05, power 0.8, set before launch
Sanity checks Sample ratio mismatch, guardrail metrics
Test Two-proportion z-test (conversion), Welch t-test (revenue per user)
Effect size Absolute and relative lift with 95% CI
Decision Significant and above the minimum worthwhile effect → ship
Multiple metrics Benjamini-Hochberg for secondary metrics

Code Example

import numpy as np
from scipy import stats
from statsmodels.stats.multitest import multipletests
from statsmodels.stats.proportion import confint_proportions_2indep, proportions_ztest

rng = np.random.default_rng(42)
n_a = n_b = 20_000
conv_a = rng.binomial(n_a, 0.050)
conv_b = rng.binomial(n_b, 0.056)

z, p = proportions_ztest([conv_b, conv_a], [n_b, n_a])
low, high = confint_proportions_2indep(conv_b, n_b, conv_a, n_a)
print(f"conversion A {conv_a / n_a:.4f}, B {conv_b / n_b:.4f}; z = {z:.2f}, p = {p:.4f}")
print(f"95% CI for B − A: [{low:.4f}, {high:.4f}]")

# Revenue per user: skewed, many zeros → Welch t-test (large n)
rev_a = rng.binomial(1, 0.05, n_a) * rng.lognormal(3.5, 1.0, n_a)
rev_b = rng.binomial(1, 0.056, n_b) * rng.lognormal(3.5, 1.0, n_b)
t = stats.ttest_ind(rev_b, rev_a, equal_var=False)
print(f"revenue per user A {rev_a.mean():.3f}, B {rev_b.mean():.3f}; p = {t.pvalue:.4f}")

# Secondary metrics: control the false discovery rate
secondary_p = [0.012, 0.21, 0.049, 0.003]
reject, adjusted, _, _ = multipletests(secondary_p, alpha=0.05, method="fdr_bh")
print("BH-adjusted p-values:", adjusted.round(3), "| significant:", reject)

If p is below 0.05 and the interval lies above the minimum effect worth shipping, the result is both statistically and practically significant. “Not significant” means the data do not clearly show a difference, not that there is none.

Common Pitfalls

  • Peeking and stopping as soon as p < 0.05.

  • Analysing per-session data when randomization was per user (non-independent observations).

  • Celebrating statistically significant but tiny lifts.

53

How can you avoid bias in experiments and A/B tests?

Avoiding bias in A/B tests starts with proper randomization at the right unit (user, session, store), a pre-registered plan (primary metric, sample size, duration, analysis method) and checks that the randomization worked, such as a sample ratio mismatch (SRM) test. During the test, avoid peeking and stopping early, run full weekly cycles to cover day-of-week effects, and keep treatment and control isolated from each other. In analysis, use intention-to-treat, correct for multiple comparisons, and be careful with novelty effects and interference between users.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Common Biases and Fixes

Bias Cause Fix
Selection bias Users choose or are chosen for treatment Random assignment by a stable hash of the user ID
Sample ratio mismatch Bugs, bots, redirects drop users unevenly Chi-square SRM check; investigate before analysing
Peeking / optional stopping Stopping when p < 0.05 Fixed horizon, or sequential tests with alpha spending
Novelty and primacy effects Users react to change itself Run longer; analyse new users separately
Day-of-week / seasonality Short tests Run whole weeks
Interference (network effects) Treated users affect control users Cluster or geo randomization
Multiple testing Many metrics and segments Pre-specify primary metric; Bonferroni or Benjamini-Hochberg
Survivorship Analysing only users who completed a step Analyse everyone assigned

Code Example: Sample Ratio Mismatch Check

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# Planned 50/50 split; a redirect bug dropped some treatment users
control_users = 50_400
treatment_users = 49_100

observed = [control_users, treatment_users]
expected = [sum(observed) / 2] * 2
chi2, p = stats.chisquare(observed, expected)
print(f"chi2 = {chi2:.2f}, p = {p:.2e}")
print("SRM detected - do not trust the results" if p < 0.001 else "split looks fine")

# Deterministic assignment from a hashed id keeps users in one group
def assign(user_id: int, salt: int = 42) -> str:
    return "treatment" if hash((salt, user_id)) % 2 else "control"
print([assign(u) for u in range(5)])

A tiny p-value means a split this uneven would be very unlikely under correct 50/50 randomization, so something in the pipeline is biased. (In production use a stable hash such as MD5 or MurmurHash of the ID, because Python’s hash of strings changes between runs.)

Common Pitfalls

  • Analysing only users who saw the feature rather than everyone assigned.

  • Changing the metric after seeing results.

  • Running overlapping experiments that interact without accounting for it.

54

How does Bayesian statistics differ from frequentist statistics?

Bayesian statistics treats unknown parameters as random variables with probability distributions that represent uncertainty, combines a prior with the data through Bayes’ theorem, and reports a posterior distribution, so statements like “there is a 95% probability the parameter lies in this interval” are valid. Frequentist statistics treats parameters as fixed but unknown, and defines probability as long-run frequency over repeated samples, so its tools (p-values, confidence intervals) describe how procedures behave across hypothetical repetitions rather than the probability of a hypothesis.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Comparison

Aspect Frequentist Bayesian
Probability means Long-run frequency Degree of belief / uncertainty
Parameters Fixed, unknown Random variables with distributions
Prior information Not formally used Explicit prior
Output Point estimate, CI, p-value Posterior distribution
Interval 95% CI: the procedure covers the true value 95% of the time 95% credible interval: 95% probability given model and prior
Hypotheses Reject or fail to reject H₀ P(hypothesis | data), Bayes factors
Computation Usually closed form, fast Often MCMC or variational inference
Sequential looks Need corrections Posterior is valid at any time (decision rules still need care)

Code Example: Same Data, Both Views

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
conversions_a, n_a = rng.binomial(1000, 0.10), 1000
conversions_b, n_b = rng.binomial(1000, 0.13), 1000

# Frequentist: two-proportion test via chi-square
table = [[conversions_a, n_a - conversions_a], [conversions_b, n_b - conversions_b]]
chi2, p, _, _ = stats.chi2_contingency(table, correction=False)
print(f"frequentist: rates {conversions_a / n_a:.3f} vs {conversions_b / n_b:.3f}, p = {p:.4f}")

# Bayesian: Beta(1, 1) priors, posterior draws
post_a = stats.beta(1 + conversions_a, 1 + n_a - conversions_a).rvs(50_000, random_state=42)
post_b = stats.beta(1 + conversions_b, 1 + n_b - conversions_b).rvs(50_000, random_state=43)
lift = post_b - post_a
print(f"bayesian: P(B > A) = {(lift > 0).mean():.3f}, "
      f"95% credible interval for the difference {np.percentile(lift, [2.5, 97.5]).round(4)}")

With flat priors and plenty of data the two approaches give similar intervals, but they answer different questions: the p-value is about the data assuming no difference; P(B > A) is about the hypothesis given the data.

Common Pitfalls

  • Interpreting a p-value as P(H₀ is true).

  • Interpreting a confidence interval as having 95% probability of containing the parameter.

  • Believing Bayesian methods are “subjective” and therefore unusable; priors should be stated and tested for sensitivity.

55

What are the prior, likelihood and posterior in Bayesian inference?

In Bayesian inference, the prior p(θ) expresses what you believe about a parameter before seeing the data; the likelihood p(data | θ) measures how probable the observed data are for each possible parameter value; and the posterior p(θ | data) is the updated belief after seeing the data. Bayes’ theorem links them: posterior = likelihood × prior / evidence, where the evidence p(data) is a normalizing constant. The posterior is a full distribution, so you can report its mean, credible intervals or the probability of any hypothesis about θ.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

The Three Pieces

Term Notation Meaning Example (coin bias θ)
Prior p(θ) Belief before data Beta(2, 2): probably near fair
Likelihood p(data | θ) Fit of the data for each θ 7 heads in 10 flips: θ⁷(1 − θ)³
Evidence p(data) Average likelihood over the prior Constant that makes the posterior integrate to 1
Posterior p(θ | data) Updated belief Beta(9, 5)

The likelihood is not a probability distribution over θ; it is a function of θ for fixed data and need not integrate to 1.

Code Example: Grid Approximation

import numpy as np
from scipy import stats

theta = np.linspace(0, 1, 1001)
prior = stats.beta.pdf(theta, 2, 2)
heads, flips = 7, 10
likelihood = stats.binom.pmf(heads, flips, theta)

unnormalized = likelihood * prior
posterior = unnormalized / np.trapezoid(unnormalized, theta)     # divide by the evidence

post_mean = np.trapezoid(theta * posterior, theta)
cdf = np.cumsum(posterior) * (theta[1] - theta[0])
low, high = theta[np.searchsorted(cdf, 0.025)], theta[np.searchsorted(cdf, 0.975)]
print(f"posterior mean {post_mean:.3f}, 95% credible interval [{low:.3f}, {high:.3f}]")
print(f"exact Beta(9, 5) mean {9 / 14:.3f}, interval {np.round(stats.beta(9, 5).interval(0.95), 3)}")
print(f"P(theta > 0.5 | data) = {posterior[theta > 0.5].sum() * (theta[1] - theta[0]):.3f}")

The posterior mean (about 0.64) sits between the prior mean (0.5) and the observed rate (0.7): with more data, the likelihood dominates and the prior matters less.

Common Pitfalls

  • Calling the likelihood “the probability of θ”.

  • Choosing a prior after seeing the data to get a desired result.

  • Reporting only the posterior mean and ignoring its spread.

56

Describe a scenario where Bayesian statistics would be advantageous.

Bayesian statistics is advantageous when data are scarce, prior knowledge is real, or decisions need direct probabilities. A strong scenario is estimating failure or conversion rates across many small groups, such as click-through rates for hundreds of products, many with only a few views. A frequentist estimate for a product with 2 clicks in 3 views says 67%, which is absurd; a hierarchical Bayesian model shrinks each product’s estimate toward the overall rate in proportion to how little data it has, giving sensible rankings and credible intervals, and answers questions like “what is the probability product A beats product B?”.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Other Good Scenarios

Scenario Bayesian advantage
Rare disease diagnosis Combines test accuracy with prevalence (prior)
Small clinical trials, rare diseases Uses historical data as an informative prior
A/B testing P(B better than A), expected loss, natural sequential monitoring
Sequential decisions (bandits) Thompson sampling uses posterior draws
Complex hierarchical data (schools, hospitals, regions) Partial pooling across groups
Uncertainty in forecasts Full predictive distributions

Code Example: Shrinking Small-Sample Rates

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
n_products = 200
true_ctr = rng.beta(4, 96, n_products)                   # most products near 4%
views = rng.integers(3, 2000, n_products)
clicks = rng.binomial(views, true_ctr)

raw = clicks / views

# Empirical Bayes: fit a Beta prior to the products, then update each one
mean, var = raw.mean(), raw.var()
common = mean * (1 - mean) / var - 1
a0, b0 = mean * common, (1 - mean) * common
posterior_mean = (a0 + clicks) / (a0 + b0 + views)

print(f"fitted prior: Beta({a0:.1f}, {b0:.1f})")
print("mean abs error, raw rates:      ", round(np.abs(raw - true_ctr).mean(), 4))
print("mean abs error, Bayesian rates: ", round(np.abs(posterior_mean - true_ctr).mean(), 4))

# Probability one product beats another
i, j = 0, 1
draws_i = stats.beta(a0 + clicks[i], b0 + views[i] - clicks[i]).rvs(20000, random_state=42)
draws_j = stats.beta(a0 + clicks[j], b0 + views[j] - clicks[j]).rvs(20000, random_state=43)
print(f"P(product {i} has higher CTR than product {j}) = {(draws_i > draws_j).mean():.3f}")

The shrunken estimates should be closer to the true rates on average, especially for products with few views. A full hierarchical model (PyMC, Stan) also accounts for uncertainty in the prior itself.

Common Pitfalls

  • Choosing a prior to get the desired answer; justify it and run a sensitivity analysis.

  • Ignoring computational cost and convergence checks for complex models.

57

What is Markov Chain Monte Carlo, and where is it used in statistics?

Markov Chain Monte Carlo (MCMC) is a family of algorithms that draw samples from a probability distribution, usually a Bayesian posterior, that is too complex to compute directly. It builds a Markov chain whose long-run (stationary) distribution is the target, so after a burn-in period the chain’s states behave like dependent samples from it. Averages over those samples estimate posterior means, credible intervals and predictions. MCMC is used for Bayesian regression and hierarchical models, missing-data problems, phylogenetics, epidemiology and physics.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+ (PyMC 5 or Stan for real models)

Main Algorithms

Algorithm Idea Notes
Metropolis-Hastings Propose a move; accept with a probability based on density ratios Simple; slow in many dimensions
Gibbs sampling Sample each parameter from its conditional distribution Needs tractable conditionals
Hamiltonian Monte Carlo (HMC) Uses gradients to make long, efficient moves Default in Stan, PyMC
NUTS HMC that tunes its path length automatically The standard choice today

Checking Convergence

  • Trace plots of several chains should overlap and look like noise.

  • R-hat close to 1 (e.g. below 1.01) means chains agree.

  • Effective sample size accounts for autocorrelation between draws.

  • Divergent transitions (HMC) signal geometry problems.

Code Example: Gibbs Sampler for a Normal Mean and Variance

import numpy as np

rng = np.random.default_rng(42)
data = rng.normal(5.0, 2.0, 100)
n, x_bar = len(data), data.mean()

mu, tau = 0.0, 1.0                     # tau = precision = 1 / variance
samples = []
for i in range(6000):
    # mu | tau, data  (flat prior on mu)
    mu = rng.normal(x_bar, 1 / np.sqrt(n * tau))
    # tau | mu, data  (Gamma(0.001, 0.001) prior on tau)
    shape = 0.001 + n / 2
    rate = 0.001 + 0.5 * np.sum((data - mu) ** 2)
    tau = rng.gamma(shape, 1 / rate)
    samples.append((mu, 1 / np.sqrt(tau)))

draws = np.array(samples[1000:])        # discard burn-in
print("posterior mean of mu:", draws[:, 0].mean().round(3), "| 95% interval:", np.percentile(draws[:, 0], [2.5, 97.5]).round(3))
print("posterior mean of sigma:", draws[:, 1].mean().round(3))

The posterior for μ should centre near the sample mean and the σ posterior near 2, the values used to simulate the data.

Common Pitfalls

  • Reporting results from a single chain without convergence diagnostics.

  • Thinking draws are independent; use effective sample size.

  • Poorly scaled or highly correlated parameters slowing mixing (reparameterize, use NUTS).

58

How do you update a Bayesian model with new data?

Updating a Bayesian model applies Bayes’ theorem: posterior ∝ likelihood × prior. Today’s posterior becomes tomorrow’s prior, so when new data arrive you combine them with the current posterior to get an updated one; for independent data this gives exactly the same result as analysing all data at once. With conjugate priors the update is a simple formula (Beta-Binomial, Normal-Normal, Gamma-Poisson); for other models you rerun MCMC or variational inference with all data, or use sequential Monte Carlo for streaming updates.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Conjugate Updates

Data Prior Posterior after data
Successes k in n trials Beta(α, β) Beta(α + k, β + n − k)
Counts with total exposure t Gamma(α, β) Gamma(α + Σcounts, β + t)
Normal mean, known σ Normal(μ₀, τ₀²) Precision-weighted average of prior mean and data mean

Code Example: Conversion Rate Updated Week by Week

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
true_rate = 0.12
alpha, beta = 2, 18                     # prior: mean 0.10, fairly weak

for week in range(1, 5):
    visitors = 500
    conversions = rng.binomial(visitors, true_rate)
    alpha, beta = alpha + conversions, beta + visitors - conversions    # posterior becomes the new prior
    post = stats.beta(alpha, beta)
    low, high = post.interval(0.95)
    print(f"week {week}: {conversions}/{visitors} → posterior mean {post.mean():.4f}, "
          f"95% credible interval [{low:.4f}, {high:.4f}], P(rate > 0.10) = {post.sf(0.10):.3f}")

# Batch update gives the same posterior as sequential updates
print("final posterior: Beta(", alpha, ",", beta, ")")

Each week the credible interval narrows as evidence accumulates, and the prior’s influence fades. P(rate > 0.10) answers a direct business question that a p-value cannot.

When the Data Change Over Time

If the underlying rate drifts, discount old evidence (for example, multiply α and β by a forgetting factor below 1 before each update) or use a dynamic model.

Common Pitfalls

  • Using the same data twice (once to choose the prior and again in the likelihood).

  • An overly confident prior that new data can barely move.

  • Treating sequential updating as different from batch analysis; for exchangeable data they agree.

59

How do you calculate mean, median and mode from a list in Python?

Mean, median and mode are the three main measures of central tendency. The mean is the sum divided by the count and is sensitive to outliers; the median is the middle value of the sorted data (the average of the two middle values for an even count) and is robust to outliers and skew; the mode is the most frequent value, and there may be several. Python’s built-in statistics module computes all three, and writing them from scratch shows the definitions, including how to handle ties and empty input.

Read full answer

Applies toPython 3.12 (standard library), NumPy 2.x

When to Use Which

Measure Best for Weakness
Mean Symmetric data; totals matter Pulled by outliers
Median Skewed data (income, house prices, response times) Ignores the size of extreme values
Mode Categorical data, most common value May be several or not meaningful for continuous data

Code Example

from collections import Counter
import statistics

import numpy as np

def mean(values):
    if not values:
        raise ValueError("mean of empty data")
    return sum(values) / len(values)

def median(values):
    if not values:
        raise ValueError("median of empty data")
    s = sorted(values)
    mid = len(s) // 2
    return s[mid] if len(s) % 2 else (s[mid - 1] + s[mid]) / 2

def modes(values):
    if not values:
        raise ValueError("mode of empty data")
    counts = Counter(values)
    top = max(counts.values())
    return sorted(v for v, c in counts.items() if c == top)   # all values tied for most frequent

data = [3, 7, 7, 2, 9, 4, 3, 7, 3, 120]                  # 120 is an outlier
print("mean:", mean(data), "| median:", median(data), "| modes:", modes(data))
print("statistics module:", statistics.mean(data), statistics.median(data), statistics.multimode(data))

rng = np.random.default_rng(42)
incomes = rng.lognormal(10.5, 0.8, 10_000)               # right-skewed data
print(f"skewed incomes: mean {incomes.mean():,.0f} vs median {np.median(incomes):,.0f}")

The outlier 120 drags the mean well above most values while the median stays at the centre, and the skewed income sample shows the mean above the median, which is why the median is preferred for incomes.

Common Pitfalls

  • Reporting a single mode when the data are multimodal (statistics.mode returns only the first).

  • Using the mean for heavily skewed data.

  • Computing the median without sorting.

60

How do you generate and plot 1,000 normally distributed random points?

Generating normal data in Python is done with NumPy’s random generator: rng.normal(loc=mean, scale=sd, size=1000). To visualize it, plot a histogram with density scaling and overlay the theoretical normal curve, and add a Q-Q plot, which shows points close to a straight line when the data are normal. Checking the sample mean and standard deviation against the true values, and the share within ±1, 2 and 3 SD (about 68%, 95% and 99.7%), confirms the sample behaves as expected.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, Matplotlib 3.9+

Code Example

import matplotlib.pyplot as plt
import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
mu, sigma = 50, 10
data = rng.normal(loc=mu, scale=sigma, size=1000)

print(f"sample mean {data.mean():.2f} (true {mu}), sample SD {data.std(ddof=1):.2f} (true {sigma})")
for k in (1, 2, 3):
    share = np.mean(np.abs(data - mu) <= k * sigma)
    print(f"within ±{k} SD: {share:.3f}")
print("Shapiro-Wilk p-value:", round(stats.shapiro(data).pvalue, 3))

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
ax1.hist(data, bins=30, density=True, alpha=0.6, edgecolor="white", label="sample")
xs = np.linspace(data.min(), data.max(), 300)
ax1.plot(xs, stats.norm.pdf(xs, mu, sigma), linewidth=2, label="N(50, 10²) density")
ax1.set_xlabel("value")
ax1.set_ylabel("density")
ax1.legend()
ax1.set_title("Histogram with normal curve")

stats.probplot(data, dist="norm", plot=ax2)
ax2.set_title("Normal Q-Q plot")
fig.tight_layout()
plt.show()

The Shapiro-Wilk p-value should usually be large here (the data really are normal); with real data and large samples, it can reject normality for tiny, unimportant deviations, so look at the Q-Q plot too.

Common Pitfalls

  • Using the legacy np.random.seed / np.random.normal global state instead of a Generator.

  • Histograms with counts on the y-axis compared with a density curve (use density=True).

  • Too few or too many bins hiding the shape.

61

How do you implement simple linear regression from scratch in Python?

Simple linear regression fits a line y = β₀ + β₁x by ordinary least squares, minimizing the sum of squared residuals. The closed-form estimates are β₁ = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)² and β₀ = ȳ − β₁x̄. A complete from-scratch implementation also computes R², the residual standard error, the standard error of the slope, a t-test for “slope = 0” and a confidence interval, and it should match a library such as SciPy’s linregress.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Formulas

Quantity Formula
Slope β₁ = Sxy / Sxx
Intercept β₀ = ȳ − β₁x̄
Residual variance s² = Σ residual² / (n − 2)
SE of slope √(s² / Sxx)
t statistic β₁ / SE(β₁), with n − 2 degrees of freedom
R² 1 − SS_res / SS_tot

Code Example

import numpy as np
from scipy import stats

class SimpleLinearRegression:
    def fit(self, x, y):
        x, y = np.asarray(x, float), np.asarray(y, float)
        n = len(x)
        x_bar, y_bar = x.mean(), y.mean()
        sxx = ((x - x_bar) ** 2).sum()
        self.slope = ((x - x_bar) * (y - y_bar)).sum() / sxx
        self.intercept = y_bar - self.slope * x_bar
        residuals = y - self.predict(x)
        self.r2 = 1 - (residuals ** 2).sum() / ((y - y_bar) ** 2).sum()
        s2 = (residuals ** 2).sum() / (n - 2)
        self.se_slope = np.sqrt(s2 / sxx)
        t = self.slope / self.se_slope
        self.p_value = 2 * stats.t.sf(abs(t), df=n - 2)
        t_crit = stats.t.ppf(0.975, df=n - 2)
        self.slope_ci = (self.slope - t_crit * self.se_slope, self.slope + t_crit * self.se_slope)
        return self

    def predict(self, x):
        return self.intercept + self.slope * np.asarray(x, float)

rng = np.random.default_rng(42)
x = rng.uniform(0, 10, 80)
y = 3.0 + 2.0 * x + rng.normal(0, 2, 80)

model = SimpleLinearRegression().fit(x, y)
print(f"y = {model.intercept:.3f} + {model.slope:.3f}x | R² = {model.r2:.3f}")
print(f"slope SE = {model.se_slope:.4f}, p = {model.p_value:.3g}, 95% CI = "
      f"[{model.slope_ci[0]:.3f}, {model.slope_ci[1]:.3f}]")

ref = stats.linregress(x, y)
print("matches scipy:", np.isclose(model.slope, ref.slope), np.isclose(model.se_slope, ref.stderr))

The slope interval should contain the true value 2.0 used in the simulation, and the tiny p-value shows the slope is clearly non-zero.

Common Pitfalls

  • Skipping residual plots (non-linearity, changing variance, outliers).

  • Extrapolating far beyond the observed x range.

  • Reading R² as proof of causation.

62

How do you simulate and analyze the Monty Hall problem in Python?

The Monty Hall problem: a prize is behind one of three doors; you pick a door, the host, who knows where the prize is, opens another door with no prize, and offers you the chance to switch. Switching wins with probability 2/3, staying wins with 1/3, because your first pick is right only one time in three and the host’s choice gives information. Simulating many games confirms this, and a confidence interval or binomial test shows the observed win rates agree with theory.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Why Switching Wins

Your first pick Probability Stay Switch
Prize door 1/3 Win Lose
Empty door 2/3 Lose Win (host must reveal the other empty door)

Code Example

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
n_games = 100_000

prize = rng.integers(0, 3, n_games)
choice = rng.integers(0, 3, n_games)

# Host opens a door that is neither the player's choice nor the prize
host = np.empty(n_games, dtype=int)
for i in range(n_games):
    options = [d for d in range(3) if d != choice[i] and d != prize[i]]
    host[i] = rng.choice(options)

switch_to = 3 - choice - host          # the remaining door (doors are 0, 1, 2)
stay_wins = (choice == prize).sum()
switch_wins = (switch_to == prize).sum()

for name, wins, theory in (("stay", stay_wins, 1 / 3), ("switch", switch_wins, 2 / 3)):
    rate = wins / n_games
    ci = stats.binomtest(int(wins), n_games).proportion_ci(confidence_level=0.95)
    p = stats.binomtest(int(wins), n_games, p=theory).pvalue
    print(f"{name:>6}: win rate {rate:.4f}, 95% CI [{ci.low:.4f}, {ci.high:.4f}], "
          f"test vs {theory:.3f}: p = {p:.3f}")

The intervals should contain 1/3 and 2/3, and the tests against the theoretical values should not reject them. The intuition: switching wins exactly when your first pick was wrong, which happens two times in three.

Common Pitfalls

  • Simulating a host who opens a random door (which might reveal the prize); that is a different problem with a 1/2 answer when the prize is not revealed.

  • Treating the two remaining doors as equally likely because “there are two left”.

63

How do you write a Python function to perform a two-sample t-test?

A two-sample t-test checks whether two independent groups have different population means. A good function computes the group summaries, uses Welch’s t-test by default (it does not assume equal variances and loses little when variances are equal), returns the t statistic, degrees of freedom, p-value, a confidence interval for the mean difference and an effect size (Cohen’s d), and supports a paired option when the same subjects are measured twice. The p-value is the probability of a difference at least this extreme if the true means were equal.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Which t-test?

Situation Test SciPy
Two independent groups Welch’s t-test ttest_ind(a, b, equal_var=False)
Independent groups, equal variances assumed Student’s t-test ttest_ind(a, b)
Same subjects, before/after Paired t-test ttest_rel(a, b)
One group vs a known value One-sample t-test ttest_1samp(a, mu)
Non-normal, small samples Mann-Whitney U / Wilcoxon mannwhitneyu, wilcoxon

Code Example

import numpy as np
from scipy import stats

def t_test(a, b, paired=False, alpha=0.05):
    a, b = np.asarray(a, float), np.asarray(b, float)
    if paired:
        res = stats.ttest_rel(a, b)
        diff = a - b
        d = diff.mean() / diff.std(ddof=1)
    else:
        res = stats.ttest_ind(a, b, equal_var=False)
        pooled_sd = np.sqrt((a.var(ddof=1) + b.var(ddof=1)) / 2)
        d = (a.mean() - b.mean()) / pooled_sd
    ci = res.confidence_interval(confidence_level=1 - alpha)
    return {"t": res.statistic, "df": res.df, "p": res.pvalue,
            "mean_diff": a.mean() - b.mean(), "ci": (ci.low, ci.high), "cohens_d": d,
            "reject_null": res.pvalue < alpha}

rng = np.random.default_rng(42)
group_a = rng.normal(72, 10, 40)        # e.g. test scores with a new teaching method
group_b = rng.normal(66, 12, 45)

result = t_test(group_a, group_b)
for key, value in result.items():
    print(f"{key:>11}: {np.round(value, 4)}")

If reject_null is true, a mean difference this large would be unlikely if the methods had equal means; the confidence interval shows the plausible size of the difference, and Cohen’s d puts it on a standardized scale (about 0.2 small, 0.5 medium, 0.8 large).

Common Pitfalls

  • Using an independent test on paired data (or the reverse).

  • Assuming equal variances by default.

  • Treating p > 0.05 as proof the means are equal.

64

How do you compute and plot a correlation matrix in Python?

A correlation matrix shows the pairwise correlation coefficient between every pair of numeric variables, from −1 (perfect negative) through 0 (no linear relationship) to +1 (perfect positive). In Python, DataFrame.corr() computes it with Pearson (linear relationships, sensitive to outliers) or Spearman (monotonic relationships, rank-based) correlation, and a heatmap displays it. A good script also tests which correlations are significant, keeping in mind that many pairs means many tests, and that correlation does not imply causation.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, SciPy 1.13+, Matplotlib 3.9+

Pearson vs Spearman

Method Measures Use when
Pearson Linear association Continuous, roughly normal, no big outliers
Spearman Monotonic association (ranks) Skewed data, outliers, ordinal variables
Kendall Concordance of pairs Small samples, many ties

Code Example

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats

rng = np.random.default_rng(42)
n = 300
hours = rng.normal(5, 2, n)
df = pd.DataFrame({
    "study_hours": hours,
    "sleep_hours": rng.normal(7, 1, n),
    "score": 50 + 6 * hours + rng.normal(0, 8, n),
    "screen_time": 6 - 0.5 * hours + rng.normal(0, 1.5, n),
})

corr = df.corr(method="pearson")
print(corr.round(2))

# p-values for each pair
cols = df.columns
pvals = pd.DataFrame(np.ones((len(cols), len(cols))), index=cols, columns=cols)
for i, a in enumerate(cols):
    for b in cols[i + 1:]:
        pvals.loc[a, b] = pvals.loc[b, a] = stats.pearsonr(df[a], df[b]).pvalue
print(pvals.round(4))

fig, ax = plt.subplots(figsize=(6, 5))
im = ax.imshow(corr, cmap="RdBu_r", vmin=-1, vmax=1)
ax.set_xticks(range(len(cols)), cols, rotation=45, ha="right")
ax.set_yticks(range(len(cols)), cols)
for i in range(len(cols)):
    for j in range(len(cols)):
        ax.text(j, i, f"{corr.iloc[i, j]:.2f}", ha="center", va="center", fontsize=9)
fig.colorbar(im, ax=ax, label="Pearson r")
ax.set_title("Correlation matrix")
fig.tight_layout()
plt.show()

A diverging colour map centred at 0 with fixed limits −1 to 1 makes positive and negative correlations easy to compare. Seaborn’s heatmap(corr, annot=True) is a one-line alternative.

Common Pitfalls

  • Reading correlation as causation.

  • Missing non-linear relationships that have near-zero Pearson correlation (always plot the data).

  • Testing dozens of pairs without a multiple-testing correction.

65

How do you implement Metropolis-Hastings for simple Bayesian inference?

Metropolis-Hastings (MH) is a Markov chain Monte Carlo algorithm that draws samples from a posterior distribution known only up to a constant. At each step it proposes a new parameter value (e.g. current value plus Gaussian noise), computes the acceptance ratio of posterior densities (times the proposal ratio, which cancels for symmetric proposals), and accepts the proposal with probability min(1, ratio), otherwise keeping the current value. After discarding a burn-in period, the chain’s samples approximate the posterior, so you can compute means and credible intervals; working in log space avoids numerical underflow.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+

Algorithm

  1. Start at θ₀.

  2. Propose θ* ~ q(θ* | θₜ), for example Normal(θₜ, σ²).

  3. Compute log α = log p(θ) + log L(θ) − log p(θₜ) − log L(θₜ).

  4. Draw u ~ Uniform(0, 1); accept if log u < log α.

  5. Repeat; tune σ for an acceptance rate around 20-50%.

Code Example: Coin Bias with a Beta Prior

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
data = rng.binomial(1, 0.7, 50)                  # 50 coin flips
heads, n = data.sum(), len(data)
a_prior, b_prior = 2, 2

def log_posterior(theta):
    if not 0 < theta < 1:
        return -np.inf
    return stats.beta.logpdf(theta, a_prior, b_prior) + stats.binom.logpmf(heads, n, theta)

def metropolis_hastings(n_samples=20000, step=0.1, start=0.5):
    samples = np.empty(n_samples)
    current, current_lp = start, log_posterior(start)
    accepted = 0
    for i in range(n_samples):
        proposal = current + rng.normal(0, step)          # symmetric proposal
        proposal_lp = log_posterior(proposal)
        if np.log(rng.random()) < proposal_lp - current_lp:
            current, current_lp = proposal, proposal_lp
            accepted += 1
        samples[i] = current
    return samples, accepted / n_samples

samples, rate = metropolis_hastings()
posterior = samples[2000:]                               # drop burn-in
print("acceptance rate:", round(rate, 3))
print("posterior mean:", round(posterior.mean(), 3), "| 95% credible interval:", np.percentile(posterior, [2.5, 97.5]).round(3))

# Conjugate check: the exact posterior is Beta(a + heads, b + tails)
exact = stats.beta(a_prior + heads, b_prior + n - heads)
print("exact mean:", round(exact.mean(), 3), "| exact interval:", np.array(exact.interval(0.95)).round(3))

The MH estimates should closely match the exact conjugate posterior, which validates the sampler before using it on models without closed forms.

Common Pitfalls

  • Multiplying raw probabilities instead of adding log densities.

  • A step size so small that the chain barely moves, or so large that almost nothing is accepted.

  • Skipping convergence checks (trace plots, multiple chains, R-hat, effective sample size).

  • For asymmetric proposals, forgetting the Hastings correction q(θₜ | θ) / q(θ | θₜ).

66

How do you estimate Pi with a Monte Carlo simulation in Python?

Estimating π by Monte Carlo uses random sampling: draw points uniformly in the unit square, and count how many fall inside the quarter circle x² + y² ≤ 1. The quarter circle’s area is π/4, so the fraction inside estimates π/4, and multiplying by 4 estimates π. The estimate’s standard error shrinks like 1/√n, so each extra decimal digit of accuracy needs about 100 times more points, which is the key statistical lesson of the exercise.

Read full answer

Applies toPython 3.12, NumPy 2.x

How It Works

Quantity Value
Probability a point lands inside p = π/4 ≈ 0.785
Estimator π̂ = 4 × (inside / n)
Standard error 4 × √(p(1 − p) / n)
95% confidence interval π̂ ± 1.96 × SE

Code Example

import numpy as np

def estimate_pi(n, rng):
    x, y = rng.random(n), rng.random(n)
    inside = (x ** 2 + y ** 2 <= 1.0)
    p_hat = inside.mean()
    pi_hat = 4 * p_hat
    se = 4 * np.sqrt(p_hat * (1 - p_hat) / n)
    return pi_hat, se

rng = np.random.default_rng(42)
for n in (1_000, 100_000, 10_000_000):
    pi_hat, se = estimate_pi(n, rng)
    print(f"n={n:>10,}  estimate={pi_hat:.5f}  SE={se:.5f}  "
          f"95% CI=[{pi_hat - 1.96 * se:.5f}, {pi_hat + 1.96 * se:.5f}]  error={abs(pi_hat - np.pi):.5f}")

As n grows by 100×, the standard error drops by about 10×, and the interval should contain the true π about 95% of the time.

Where Monte Carlo Is Used

  • Integrals in many dimensions, where grid methods are too expensive.

  • Bayesian computation (MCMC), option pricing, risk simulation and bootstrap resampling.

Common Pitfalls

  • Looping in pure Python over millions of points instead of vectorizing with NumPy.

  • Reporting an estimate without its standard error.

  • Using rng.random() once per coordinate in a loop with a reseeded generator, which repeats samples.

67

How do you perform a chi-squared test of independence in Python?

A chi-squared test of independence checks whether two categorical variables are associated, using a contingency table of observed counts. Under the null hypothesis of independence, expected counts are row total × column total / grand total; the statistic sums (observed − expected)² / expected and is compared with a chi-squared distribution with (rows − 1)(columns − 1) degrees of freedom. In Python, scipy.stats.chi2_contingency does the whole calculation; check that expected counts are at least about 5, and report an effect size such as Cramér’s V.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, SciPy 1.13+

Code Example

import numpy as np
import pandas as pd
from scipy.stats import chi2_contingency, fisher_exact

rng = np.random.default_rng(42)
n = 600
device = rng.choice(["mobile", "desktop", "tablet"], n, p=[0.5, 0.4, 0.1])
p_buy = np.select([device == "mobile", device == "desktop"], [0.10, 0.18], default=0.12)
bought = np.where(rng.random(n) < p_buy, "yes", "no")

table = pd.crosstab(device, bought)
print(table)

chi2, p, dof, expected = chi2_contingency(table)
print(f"chi2 = {chi2:.2f}, dof = {dof}, p = {p:.4f}")
print("minimum expected count:", round(expected.min(), 1))

cramers_v = np.sqrt(chi2 / (table.to_numpy().sum() * (min(table.shape) - 1)))
print("Cramér's V:", round(cramers_v, 3))

# For a 2×2 table with small expected counts, use Fisher's exact test instead
small = np.array([[3, 7], [9, 2]])
print("Fisher exact p-value:", round(fisher_exact(small)[1], 4))

If p is below 0.05, purchase rates differ by device more than chance would explain under independence; Cramér’s V tells you how strong the association is (values near 0.1 are weak).

Assumptions

Assumption Note
Independent observations Each person counted once
Counts, not percentages Use raw frequencies
Expected counts ≥ 5 (most cells) Otherwise merge categories or use Fisher’s exact test

Common Pitfalls

  • Passing proportions instead of counts.

  • Reading a significant result on a huge sample as a strong association; check the effect size.

  • Forgetting chi2_contingency applies Yates’ continuity correction to 2×2 tables by default (correction=True).

68

How do you make a non-stationary time series stationary in Python?

Making a time series stationary means removing trend, changing variance and seasonality so that its mean, variance and autocorrelation stay constant over time. A practical function first stabilizes variance with a log (or Box-Cox) transform for positive data, then applies differencing, seasonal first if needed, then ordinary, and checks after each step with the Augmented Dickey-Fuller (ADF) test (null: unit root, non-stationary) and optionally KPSS (null: stationary). It should return the transformed series and the steps taken so forecasts can be inverted.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Techniques

Problem Fix
Variance grows with level Log or Box-Cox transform
Stochastic trend (unit root) First difference
Seasonality Seasonal difference at the period (12 for monthly)
Deterministic trend Detrend by regression

Code Example

import numpy as np
import pandas as pd
from statsmodels.tsa.stattools import adfuller, kpss

def make_stationary(series, seasonal_period=None, alpha=0.05, max_diff=2):
    s = series.astype(float).dropna()
    steps = []
    if (s > 0).all():
        s = np.log(s)
        steps.append("log")
    if seasonal_period:
        s = s.diff(seasonal_period).dropna()
        steps.append(f"seasonal diff {seasonal_period}")
    for _ in range(max_diff):
        if adfuller(s, autolag="AIC")[1] < alpha:
            break
        s = s.diff().dropna()
        steps.append("diff 1")
    adf_p = adfuller(s, autolag="AIC")[1]
    kpss_p = kpss(s, regression="c", nlags="auto")[1]   # p-value is capped between 0.01 and 0.1
    return s, steps, adf_p, kpss_p

rng = np.random.default_rng(42)
t = np.arange(144)
monthly = pd.Series(np.exp(0.01 * t + 0.1 * np.sin(2 * np.pi * t / 12) + np.cumsum(rng.normal(0, 0.02, 144))),
                    index=pd.date_range("2014-01-01", periods=144, freq="MS"))

print("ADF p-value before:", round(adfuller(monthly)[1], 4))
stationary, steps, adf_p, kpss_p = make_stationary(monthly, seasonal_period=12)
print("steps:", steps, "| ADF p-value after:", round(adf_p, 4), "| KPSS p-value:", round(kpss_p, 3))

A small ADF p-value rejects the unit root; a KPSS p-value above 0.05 fails to reject stationarity, so both tests support a stationary result.

Common Pitfalls

  • Over-differencing, which adds negative autocorrelation and noise.

  • Forgetting to invert transforms when converting forecasts back.

  • Applying a log to series containing zeros or negatives.

69

How do you conduct a one-way ANOVA test in R?

A one-way ANOVA tests whether the means of three or more groups are equal (H₀: all means equal; H₁: at least one differs), by comparing between-group variance with within-group variance through the F statistic. In R, fit it with aov(outcome ~ group, data = df) and read summary(), check the assumptions (independent observations, roughly normal residuals, equal variances), use Welch’s ANOVA (oneway.test) when variances differ or Kruskal-Wallis when normality fails, and follow a significant result with Tukey’s HSD to see which pairs differ.

Read full answer

Applies toR 4.4+ (base R `stats` package)

Assumptions and Alternatives

Assumption Check If violated
Independence Study design Mixed models / repeated-measures ANOVA
Normal residuals Q-Q plot, shapiro.test(residuals) kruskal.test
Equal variances bartlett.test, Levene’s test (car package) oneway.test(var.equal = FALSE)

Code Example

set.seed(42)
df <- data.frame(
  fertilizer = factor(rep(c("A", "B", "C"), each = 30)),
  yield = c(rnorm(30, mean = 20, sd = 3),
            rnorm(30, mean = 22, sd = 3),
            rnorm(30, mean = 25, sd = 3))
)

# Descriptive statistics per group
aggregate(yield ~ fertilizer, data = df, FUN = function(x) c(mean = mean(x), sd = sd(x)))

# One-way ANOVA
fit <- aov(yield ~ fertilizer, data = df)
summary(fit)

# Assumption checks
shapiro.test(residuals(fit))
bartlett.test(yield ~ fertilizer, data = df)

# Robust alternatives
oneway.test(yield ~ fertilizer, data = df, var.equal = FALSE)   # Welch's ANOVA
kruskal.test(yield ~ fertilizer, data = df)

# Which pairs differ? (family-wise error controlled)
TukeyHSD(fit)

# Effect size: eta squared = SS_between / SS_total
ss <- summary(fit)[[1]][["Sum Sq"]]
cat("eta squared:", round(ss[1] / sum(ss), 3), "\n")

Read the Pr(>F) value: a small p-value means that, if all fertilizers had the same mean yield, differences this large would rarely occur, so at least one mean differs. Tukey’s table then shows which pairs, with adjusted p-values and confidence intervals.

Common Pitfalls

  • Using a numeric group code instead of a factor (R would fit a regression slope).

  • Running many t-tests instead of ANOVA plus a post-hoc test.

  • Reporting only the p-value without an effect size.

70

How do you implement PCA for dimensionality reduction in Python?

Principal Component Analysis (PCA) reduces dimensionality by projecting data onto the orthogonal directions of maximum variance. Implementing it means centering (and usually standardizing) the features, computing the covariance matrix’s eigenvectors or, more stably, the singular value decomposition of the centered data, sorting components by explained variance, and keeping enough components to explain a target share (e.g. 95%). In practice, use scikit-learn’s PCA inside a pipeline; a from-scratch NumPy version shows the mechanics and should match it.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

Steps

  1. Standardize features (PCA is scale-sensitive).

  2. Compute the SVD of the centered matrix: X = U S Vᵀ.

  3. Rows of Vᵀ are principal directions; explained variance is S² / (n − 1).

  4. Project: Z = X V_k for the first k components.

  5. Choose k from the cumulative explained variance ratio.

Code Example

import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

X, _ = load_digits(return_X_y=True)                   # 1797 samples × 64 features
X_std = StandardScaler().fit_transform(X)

def pca_svd(X, n_components):
    Xc = X - X.mean(axis=0)
    U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
    explained_var = S ** 2 / (len(X) - 1)
    ratio = explained_var / explained_var.sum()
    return Xc @ Vt[:n_components].T, ratio

Z_manual, ratio = pca_svd(X_std, 10)
k95 = int(np.searchsorted(np.cumsum(ratio), 0.95) + 1)
print("components for 95% variance:", k95, "of", X.shape[1])

pca = PCA(n_components=10, random_state=42).fit(X_std)
Z_sklearn = pca.transform(X_std)
# Component signs are arbitrary, so compare absolute values
print("matches scikit-learn:", np.allclose(np.abs(Z_manual), np.abs(Z_sklearn), atol=1e-6))
print("first 3 explained variance ratios:", pca.explained_variance_ratio_[:3].round(3))

PCA(n_components=0.95) picks the number of components automatically.

Common Pitfalls

  • Fitting PCA on all data before a train/test split (leakage); put it in a Pipeline.

  • Forgetting to standardize features measured in different units.

  • Interpreting components as real-world factors without checking loadings.

71

How would you find which factors contribute most to supermarket sales?

Finding the factors behind supermarket sales combines exploratory analysis, a regression model of sales on candidate drivers (price, promotions, store size, footfall, season, holidays, location), and a careful reading of effect sizes. Standardized coefficients or partial effects show the relative importance of each factor, and permutation importance from a tree model gives a model-agnostic cross-check. Because the data are observational, these are associations; causal claims about levers like promotions need experiments or quasi-experimental designs.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+, scikit-learn 1.5+

Approach

Step Method
Explore Sales by store, week, category; correlation matrix; outliers
Model Multiple regression (log sales often works better), fixed effects for store and week
Importance Standardized coefficients, partial R², permutation importance
Diagnostics Residual plots, multicollinearity (VIF), heteroscedasticity-robust standard errors
Validation Hold out later weeks; check stability across regions
Causality Randomized promotions or price tests for decisions

Code Example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from sklearn.ensemble import RandomForestRegressor
from sklearn.inspection import permutation_importance

rng = np.random.default_rng(42)
n = 3000
df = pd.DataFrame({
    "price_index": rng.normal(1.0, 0.05, n),
    "promo": rng.binomial(1, 0.3, n),
    "store_size": rng.normal(2000, 400, n),
    "footfall": rng.normal(5000, 1000, n),
    "holiday": rng.binomial(1, 0.08, n),
})
df["sales"] = (20000 - 30000 * (df.price_index - 1) + 3000 * df.promo + 2 * df.store_size
               + 1.5 * df.footfall + 4000 * df.holiday + rng.normal(0, 2000, n))

# Standardize predictors so coefficients are comparable (effect of +1 SD)
cols = ["price_index", "promo", "store_size", "footfall", "holiday"]
z = df.copy()
z[cols] = (df[cols] - df[cols].mean()) / df[cols].std()
fit = smf.ols("sales ~ " + " + ".join(cols), data=z).fit(cov_type="HC3")
print(fit.params.drop("Intercept").abs().sort_values(ascending=False).round(0))
print("R²:", round(fit.rsquared, 3))

rf = RandomForestRegressor(n_estimators=200, random_state=42).fit(df[cols], df["sales"])
perm = permutation_importance(rf, df[cols], df["sales"], n_repeats=5, random_state=42)
print(pd.Series(perm.importances_mean, index=cols).sort_values(ascending=False).round(3))

Both methods should agree on the ranking here; when they disagree on real data, look for non-linear effects or interactions.

Common Pitfalls

  • Ranking raw coefficients measured in different units.

  • Correlated drivers (footfall and store size) splitting importance unpredictably.

  • Confounding: promotions may be run in weeks that would sell well anyway.

72

How do you determine whether a new drug is effective from trial data?

Determining drug effectiveness relies on a randomized controlled trial with a pre-registered primary endpoint, sample size from a power calculation, and a pre-specified analysis plan. You compare the treatment and control (placebo or standard care) groups on the primary endpoint using the appropriate test, with an intention-to-treat analysis, report the effect size with a confidence interval, and judge both statistical significance and clinical relevance. Secondary endpoints and subgroups need multiple-testing control, and safety is analysed alongside efficacy.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

Key Elements

Element Purpose
Randomization (often stratified) Balances known and unknown confounders
Blinding Prevents bias in treatment and assessment
Primary endpoint One pre-specified outcome to avoid p-hacking
Power analysis Enough patients to detect the minimal clinically important difference
Intention-to-treat Analyse patients in their randomized group, preserving randomization
Effect size + CI e.g. mean difference, risk ratio, hazard ratio
Multiplicity Hierarchical testing or corrections for secondary endpoints
Interim analyses Pre-planned with alpha spending (O’Brien-Fleming)

Choosing the Test

Endpoint Test
Continuous, roughly normal Welch two-sample t-test or ANCOVA adjusting for baseline
Continuous, skewed Mann-Whitney U
Binary (cured yes/no) Chi-square or Fisher’s exact; risk difference CI
Time to event Log-rank test, Cox model

Code Example: Blood Pressure Reduction

import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower

rng = np.random.default_rng(42)

# Power: detect a 5 mmHg difference with SD 12, 90% power, alpha 0.05
n = TTestIndPower().solve_power(effect_size=5 / 12, alpha=0.05, power=0.9)
print("patients per arm:", int(np.ceil(n)))

drug = rng.normal(-12, 12, 130)       # change in systolic BP (mmHg)
placebo = rng.normal(-6, 12, 130)
t, p = stats.ttest_ind(drug, placebo, equal_var=False)    # Welch's t-test
diff = drug.mean() - placebo.mean()
se = np.sqrt(drug.var(ddof=1) / len(drug) + placebo.var(ddof=1) / len(placebo))
print(f"mean difference {diff:.2f} mmHg, 95% CI [{diff - 1.96 * se:.2f}, {diff + 1.96 * se:.2f}]")
print(f"t = {t:.2f}, p = {p:.2g}")

If the CI excludes zero and its range is clinically meaningful (e.g. the whole interval is beyond a 3 mmHg minimal important difference), the drug is both statistically and clinically effective on this endpoint.

Common Pitfalls

  • Per-protocol analysis only, which breaks randomization.

  • Switching endpoints after seeing data.

  • Claiming efficacy in a subgroup after a non-significant main result.

  • Confusing “not significant” with “no effect” in an underpowered trial.

73

How would you statistically evaluate an online advertising campaign's success?

Evaluating an ad campaign means measuring its incremental effect, the conversions or revenue that would not have happened without the ads, not just counting conversions among people who saw them. The strongest design is a randomized holdout (lift test): randomly withhold ads from a control group, then compare conversion rates with a two-proportion test and report the lift with a confidence interval. Without randomization, use geo experiments, difference-in-differences or other quasi-experimental methods, and translate the lift into incremental cost per acquisition and return on ad spend.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, statsmodels 0.14+

Plan

Step Detail
Goal metric Conversions, revenue per user; decided before launch
Design Randomized holdout (users or regions)
Sample size Power analysis for the smallest lift worth paying for
Analysis Two-proportion z-test or chi-square; CI for absolute and relative lift
Business view Incremental conversions, incremental CPA, incremental ROAS
Robustness Check balance between groups; segment results only if pre-planned

Why Not Compare Exposed vs Unexposed Users?

Ad platforms target people who were already likely to buy, so exposed users convert more even if ads do nothing (selection bias). Randomization removes that bias.

Code Example: Lift Test

import numpy as np
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import confint_proportions_2indep, proportion_effectsize, proportions_ztest

rng = np.random.default_rng(42)

# Sample size to detect 2.0% → 2.3% conversion with 80% power at alpha 0.05
effect = proportion_effectsize(0.023, 0.020)
n_per_group = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print("users needed per group:", int(np.ceil(n_per_group)))

# Simulated results
n_test, n_control = 50_000, 50_000
conv_test = rng.binomial(n_test, 0.023)
conv_control = rng.binomial(n_control, 0.020)

stat, p = proportions_ztest([conv_test, conv_control], [n_test, n_control])
low, high = confint_proportions_2indep(conv_test, n_test, conv_control, n_control, method="wald")
print(f"test {conv_test / n_test:.4f} vs control {conv_control / n_control:.4f}")
print(f"z = {stat:.2f}, p = {p:.4f}, 95% CI for the difference: [{low:.4f}, {high:.4f}]")

spend = 20_000
incremental = conv_test - conv_control * n_test / n_control
print("incremental conversions:", round(incremental), "| incremental CPA:",
      round(spend / incremental, 2) if incremental > 0 else "n/a")

If the interval excludes zero, the campaign produced a real lift; whether it was worth it depends on the incremental CPA compared with the value of a customer.

Common Pitfalls

  • Using last-click attribution as proof of impact.

  • Peeking daily and stopping when p < 0.05, which inflates false positives.

  • Testing many segments and reporting only the significant ones without correction.

74

How would you use time series analysis to forecast stock prices?

Forecasting stock prices with time series analysis usually means modelling returns rather than prices, because prices are non-stationary (close to a random walk) while log returns are closer to stationary. The honest approach is to test for stationarity, fit simple models (random-walk baseline, ARIMA for the mean, GARCH for volatility), and evaluate with walk-forward validation against the naive “tomorrow equals today” forecast. In efficient markets, return forecasts rarely beat that baseline, but volatility is genuinely predictable, which matters for risk management.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, statsmodels 0.14+

Approach

Step Method
Transform Log returns r_t = log(P_t / P_{t-1})
Stationarity Augmented Dickey-Fuller test on prices and returns
Autocorrelation ACF/PACF plots, Ljung-Box test
Mean model ARIMA / AR on returns; compare to zero-return baseline
Volatility GARCH(1,1) (the arch package)
Validation Walk-forward (rolling origin), never random splits
Metrics RMSE/MAE vs naive baseline; directional accuracy; trading costs if used for strategy

Code Example

import numpy as np
import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller

rng = np.random.default_rng(42)
returns = rng.normal(0.0003, 0.01, 1000)                 # simulated daily log returns
prices = pd.Series(100 * np.exp(np.cumsum(returns)))
log_ret = np.log(prices).diff().dropna()

print("ADF p-value, prices :", round(adfuller(prices)[1], 4))      # large: non-stationary
print("ADF p-value, returns:", round(adfuller(log_ret)[1], 4))     # small: stationary

# Walk-forward one-step forecasts of returns: AR(1) vs naive zero-return baseline
errors_ar, errors_naive = [], []
for t in range(900, 999):
    fit = ARIMA(log_ret.iloc[:t].to_numpy(), order=(1, 0, 0)).fit()
    forecast = fit.forecast(1)[0]
    actual = log_ret.iloc[t]
    errors_ar.append((actual - forecast) ** 2)
    errors_naive.append(actual ** 2)
print("RMSE AR(1):", round(np.sqrt(np.mean(errors_ar)), 5),
      "| RMSE zero-return baseline:", round(np.sqrt(np.mean(errors_naive)), 5))

On these random-walk returns the AR model cannot beat the baseline in any meaningful way, which is exactly what you should expect, and check for, with real stock data.

Common Pitfalls

  • Fitting models to prices and reporting a high R² (yesterday’s price explains today’s).

  • Look-ahead bias: using information not available at forecast time.

  • Ignoring transaction costs and regime changes when claiming a profitable strategy.

75

How would you design a statistical study of subscription churn?

A churn study starts by defining churn exactly (cancellation, or no renewal within a grace period) and the question you want answered: describing who churns, predicting who will churn, or estimating what causes churn. Descriptive work uses churn rates by cohort and segment with confidence intervals; time-to-churn is best handled with survival analysis (Kaplan-Meier curves, Cox regression) because many customers have not churned yet (censoring); and causal questions, such as whether a price change or onboarding email reduces churn, need a randomized experiment or careful quasi-experimental design.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.2+, SciPy 1.13+, lifelines 0.29+

Study Design

Step Decisions
Definition What counts as churn; observation window; voluntary vs involuntary (failed payment)
Population All active subscribers, or a sign-up cohort
Data Tenure, plan, price, usage, support contacts, acquisition channel, payment failures
Descriptive Monthly churn rate per cohort and segment, with confidence intervals
Time to churn Kaplan-Meier survival curves; log-rank test between groups
Drivers Cox proportional hazards or logistic regression (association, not causation)
Causal tests A/B test of interventions with a power analysis beforehand
Reporting Effect sizes with intervals, not just p-values

Code Example: Survival Curves and a Log-Rank Test

import numpy as np
import pandas as pd
from lifelines import KaplanMeierFitter
from lifelines.statistics import logrank_test

rng = np.random.default_rng(42)
n = 2000
plan = rng.choice(["monthly", "annual"], n, p=[0.7, 0.3])
# Monthly subscribers churn faster (shorter expected lifetime)
lifetime = np.where(plan == "monthly", rng.exponential(10, n), rng.exponential(25, n))
observed_for = rng.uniform(1, 24, n)                      # months since sign-up
df = pd.DataFrame({
    "plan": plan,
    "duration": np.minimum(lifetime, observed_for),
    "churned": (lifetime <= observed_for).astype(int),    # 0 = still active (censored)
})

kmf = KaplanMeierFitter()
for name, group in df.groupby("plan"):
    kmf.fit(group["duration"], group["churned"], label=name)
    print(name, "| median months to churn:", kmf.median_survival_time_,
          "| retained at 12 months:", round(float(kmf.predict(12)), 3))

m, a = df[df.plan == "monthly"], df[df.plan == "annual"]
result = logrank_test(m["duration"], a["duration"], m["churned"], a["churned"])
print("log-rank statistic:", round(result.test_statistic, 2), "| p-value:", result.p_value)

A small p-value means that if the two plans truly had the same survival curve, a difference this large would be very unlikely, so retention differs by plan. It does not show the plan causes the difference: annual buyers may simply be more committed customers.

Common Pitfalls

  • Computing churn only among customers who already left, ignoring censored active customers.

  • Treating regression coefficients as causal drivers.

  • Mixing voluntary and payment-failure churn, which have different fixes.

  • Simpson’s paradox: overall churn can move opposite to every segment when the customer mix changes.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 75 Statistics answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All Machine Learning & Data Science subjects