What is the difference between descriptive and inferential statistics?
KnownRevise
Descriptive statistics summarize and present the data you have: measures of centre (mean, median), spread (standard deviation, IQR), shape (skewness), frequency tables and charts. They make no claims beyond the observed data. Inferential statistics use a sample to draw conclusions about a larger population, accounting for sampling uncertainty: estimating parameters with confidence intervals, testing hypotheses, and building models for prediction. Descriptive work usually comes first, to understand the data and check assumptions, and inference builds on it.
“Average order of all customers is 40-44 with 95% confidence”
Code Example
import numpy as np
import pandas as pd
from scipy import stats
rng = np.random.default_rng(42)
df = pd.DataFrame({
"store": np.repeat(["north", "south"], 150),
"basket": np.r_[rng.gamma(4, 10, 150), rng.gamma(4, 11.5, 150)],
})
# Descriptive: summarize what was observed
print(df.groupby("store")["basket"].describe().round(2))
# Inferential: generalize beyond the sample
north = df.loc[df.store == "north", "basket"]
south = df.loc[df.store == "south", "basket"]
ci = stats.t.interval(0.95, len(north) - 1, loc=north.mean(), scale=stats.sem(north))
print(f"95% CI for the north store's mean basket: [{ci[0]:.2f}, {ci[1]:.2f}]")
res = stats.ttest_ind(south, north, equal_var=False)
print(f"Welch t-test, south vs north: t = {res.statistic:.2f}, p = {res.pvalue:.4f}")
The describe table is purely descriptive; the confidence interval and the test make claims about all baskets at each store, assuming the sampled baskets are representative.
Common Pitfalls
Drawing population conclusions from descriptive statistics of a biased sample.
Running inferential tests on data that are already the whole population.
Skipping descriptive checks before inference (outliers, skew, data errors).
2
What is the difference between a population and a sample?
KnownRevise
A population is the entire group you want to draw conclusions about: all customers, all voters in a country, every product made on a line. Its numerical characteristics, such as the mean μ or proportion p, are parameters, which are usually unknown. A sample is the subset you actually observe, and its summaries, such as x̄ or p̂, are statistics used to estimate the parameters. Because samples vary, estimates come with sampling error, quantified by standard errors and confidence intervals, and they are only trustworthy if the sample is representative, ideally obtained by random sampling.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Comparison
Aspect
Population
Sample
Definition
Every unit of interest
Observed subset
Measures
Parameters (μ, σ, p)
Statistics (x̄, s, p̂)
Usually known?
No
Yes
Variance formula
Σ(x − μ)² / N
Σ(x − x̄)² / (n − 1)
Error
None (if fully measured)
Sampling error + possible bias
Sampling Methods
Method
Description
Simple random
Every unit equally likely
Stratified
Random sampling within groups (regions, age bands)
Cluster
Randomly choose groups, then measure everyone in them
Systematic
Every k-th unit
Convenience
Whoever is easy to reach (high bias risk)
Code Example
import numpy as np
rng = np.random.default_rng(42)
population = rng.gamma(2, 20, 1_000_000) # e.g. all customers' monthly spend
print(f"population mean (parameter) μ = {population.mean():.2f}")
for n in (30, 300, 3000):
estimates = [rng.choice(population, n, replace=False).mean() for _ in range(500)]
print(f"n = {n:>4}: average estimate {np.mean(estimates):.2f}, spread (SE) {np.std(estimates):.2f}")
# Biased sampling: only surveying customers who spend more than 20
biased = rng.choice(population[population > 20], 3000, replace=False)
print(f"biased sample mean {biased.mean():.2f} (far from μ even with n = 3000)")
Random samples are centred on the true mean, with spread shrinking as n grows; a biased sample stays wrong however large it is.
Common Pitfalls
Generalizing from a convenience or self-selected sample.
Believing a very large sample fixes bias.
Using N instead of n − 1 when estimating the population variance from a sample.
3
What is a distribution in statistics, and what are common examples?
KnownRevise
A distribution describes how the values of a variable are spread: which values occur and how often (for data), or how probability is assigned to possible values (for a random variable). An empirical distribution summarizes observed data (histograms, frequency tables); a probability distribution is a mathematical model with parameters, given by a PMF for discrete variables or a PDF for continuous ones, plus a CDF. Choosing the right distribution lets you compute probabilities, simulate data, build likelihoods and choose appropriate tests.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Common Distributions
Distribution
Type
Models
Example
Bernoulli
Discrete
One yes/no trial
Did a user click?
Binomial
Discrete
Successes in n trials
Conversions out of 1,000 visitors
Poisson
Discrete
Counts per interval
Calls per hour
Geometric
Discrete
Trials until first success
Attempts until a sale
Uniform
Continuous
Equally likely values
Random number in [0, 1]
Normal
Continuous
Sums/averages, measurement error
Heights, test scores
Exponential
Continuous
Waiting times
Time between arrivals
Log-normal
Continuous
Positive, right-skewed
Incomes, file sizes
t, chi-square, F
Continuous
Sampling distributions of test statistics
t-tests, ANOVA
Beta
Continuous
Probabilities and proportions
Bayesian prior for a rate
Code Example: Fitting and Comparing Candidates
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
data = rng.lognormal(mean=3, sigma=0.5, size=2000) # e.g. order values
candidates = {"normal": stats.norm, "lognormal": stats.lognorm, "gamma": stats.gamma}
for name, dist in candidates.items():
params = dist.fit(data) if name == "normal" else dist.fit(data, floc=0)
loglik = np.sum(dist.logpdf(data, *params))
aic = 2 * len(params) - 2 * loglik
ks_p = stats.kstest(data, dist.cdf, args=params).pvalue
print(f"{name:>9}: AIC {aic:9.1f}, KS p-value {ks_p:.3f}")
print("empirical P(value > 40):", round((data > 40).mean(), 4))
shape, loc, scale = stats.lognorm.fit(data, floc=0)
print("lognormal model P(value > 40):", round(stats.lognorm.sf(40, shape, loc, scale), 4))
The lognormal fit has the lowest AIC and a reasonable KS p-value, while the normal fit fails. (KS p-values are optimistic when parameters are estimated from the same data, so use them as a rough guide.)
Common Pitfalls
Assuming normality by default for skewed or bounded data.
Confusing the distribution of the data with the sampling distribution of a statistic.
Choosing a distribution only by a goodness-of-fit p-value; with large samples every model gets rejected.
4
What is the Central Limit Theorem, and why is it important?
KnownRevise
The Central Limit Theorem (CLT) states that the distribution of the sample mean of independent, identically distributed observations with finite variance approaches a normal distribution as the sample size grows, whatever the shape of the original distribution. The sampling distribution is centred on the population mean μ with standard deviation σ/√n (the standard error). It is important because it justifies normal-based confidence intervals and tests (z-tests, t-tests, A/B tests on conversion rates) even for skewed data, as long as samples are reasonably large.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Key Points
Point
Detail
Applies to
Sample means and sums (and many other estimators)
Centre
Population mean μ
Spread
σ/√n, shrinking with sample size
Sample size needed
Often 30+ is enough; strongly skewed or heavy-tailed data need more
Requirements
Independence and finite variance
Does not say
That the raw data become normal
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
population = rng.exponential(scale=10, size=1_000_000) # strongly right-skewed
mu, sigma = population.mean(), population.std()
print(f"population skewness: {stats.skew(population):.2f}")
for n in (2, 10, 30, 200):
means = rng.choice(population, size=(20_000, n)).mean(axis=1)
print(f"n={n:>3}: mean of sample means {means.mean():6.2f} (μ = {mu:.2f}), "
f"SD {means.std():5.2f} (σ/√n = {sigma / np.sqrt(n):5.2f}), skew {stats.skew(means):5.2f}")
# Consequence: a normal-based 95% CI for the mean works with n = 50 despite the skew
covered = 0
for _ in range(5000):
s = rng.exponential(10, 50)
half = 1.96 * s.std(ddof=1) / np.sqrt(50)
covered += s.mean() - half <= 10 <= s.mean() + half
print("coverage of 95% CI with n = 50:", covered / 5000)
As n increases, the skewness of the sample means falls toward 0 and their spread matches σ/√n. The coverage check shows normal-based intervals work reasonably well (slightly below 95% with this skewed data).
Common Pitfalls
Believing the CLT makes the data themselves normal.
Applying it to heavy-tailed distributions without finite variance (e.g. Cauchy).
Assuming n = 30 is always enough; strong skew needs larger samples or the bootstrap.
5
What is a p-value, and what does it say about significance?
KnownRevise
A p-value is the probability of obtaining a result at least as extreme as the one observed, assuming the null hypothesis is true. A small p-value means the data would be surprising under H₀, which counts as evidence against it; if p ≤ α (commonly 0.05), the result is called statistically significant and H₀ is rejected. A p-value is not the probability that H₀ is true, not the probability the result happened by chance, and says nothing about the size or importance of an effect, which is why it should be reported with an effect size and confidence interval.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
What a p-value Is and Is Not
Correct
Incorrect
P(data this extreme | H₀ true)
P(H₀ true | data)
A measure of incompatibility with H₀
The probability the result is due to chance
Depends on effect size and sample size
A measure of effect size
Uniformly distributed when H₀ is true
Evidence for H₀ when large
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# Permutation view: how often does shuffling produce a difference this large?
a = rng.normal(10.6, 2, 40)
b = rng.normal(10.0, 2, 40)
observed = a.mean() - b.mean()
pooled = np.r_[a, b]
diffs = []
for _ in range(10_000):
perm = rng.permutation(pooled)
diffs.append(perm[:40].mean() - perm[40:].mean())
p_perm = np.mean(np.abs(diffs) >= abs(observed))
print(f"observed difference {observed:.3f}; permutation p = {p_perm:.4f}; t-test p = {stats.ttest_ind(a, b).pvalue:.4f}")
# Same tiny effect, huge sample: very small p-value, negligible effect
big_a, big_b = rng.normal(10.05, 2, 200_000), rng.normal(10.0, 2, 200_000)
res = stats.ttest_ind(big_a, big_b)
print(f"n = 200,000 per group: difference {big_a.mean() - big_b.mean():.3f}, p = {res.pvalue:.2g}")
# Under H0, p-values are uniform: about 5% fall below 0.05
null_p = [stats.ttest_ind(rng.normal(0, 1, 30), rng.normal(0, 1, 30)).pvalue for _ in range(5000)]
print("share of null p-values below 0.05:", np.mean(np.array(null_p) < 0.05))
The permutation test shows the definition directly: the p-value is the share of “no-difference” worlds producing a difference as large as observed. The large-sample case shows a tiny p-value can accompany a trivial effect.
Common Pitfalls
Treating p = 0.049 and p = 0.051 as fundamentally different.
Reporting “p < 0.05” without effect sizes.
p-hacking: trying many analyses and reporting the one that crosses 0.05.
6
What does statistical power mean?
KnownRevise
Statistical power is the probability that a test correctly rejects the null hypothesis when a real effect of a given size exists: power = 1 − β, where β is the Type II (false negative) error rate. It depends on four things: the effect size (bigger effects are easier to detect), the sample size, the significance level α, and the variability of the data. Studies are usually designed for 80% or 90% power through a power analysis before data collection. Low power wastes resources, misses real effects, and makes the significant results it does find likely to exaggerate the true effect.
import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()
# Sample size for 80% power to detect d = 0.4
n = analysis.solve_power(effect_size=0.4, alpha=0.05, power=0.8)
print("per-group n for 80% power, d = 0.4:", int(np.ceil(n)))
# Power at different sample sizes
for n in (20, 50, 100, 200):
print(f"n = {n:>3} per group: power = {analysis.power(effect_size=0.4, nobs1=n, alpha=0.05):.3f}")
# Smallest detectable effect with 100 per group
mde = analysis.solve_power(nobs1=100, alpha=0.05, power=0.8)
print("minimum detectable effect with n = 100:", round(mde, 3))
# Winner's curse: underpowered significant results overestimate the effect
rng = np.random.default_rng(42)
estimates = []
for _ in range(5000):
a, b = rng.normal(0.2, 1, 20), rng.normal(0, 1, 20)
if stats.ttest_ind(a, b).pvalue < 0.05:
estimates.append(a.mean() - b.mean())
print(f"true effect 0.2; average significant estimate {np.mean(estimates):.2f} ({len(estimates)} of 5000 significant)")
With 20 per group and a true effect of 0.2, few studies reach significance, and those that do overestimate the effect several times over.
Common Pitfalls
“Post-hoc power” computed from the observed effect; it adds nothing beyond the p-value.
Choosing an unrealistically large effect size to justify a small sample.
Ignoring power when interpreting non-significant results.
7
What are Type I and Type II errors in hypothesis testing?
KnownRevise
A Type I error (false positive) is rejecting the null hypothesis when it is actually true, like concluding a drug works when it does not; its probability is the significance level α. A Type II error (false negative) is failing to reject the null hypothesis when it is actually false, like missing a drug that really works; its probability is β, and power = 1 − β. For a fixed sample size, lowering α increases β, so the trade-off is set by the costs of each mistake; increasing the sample size reduces β without raising α.
High power; follow-up tests handle false positives
Shipping a product change
Type I wastes effort; Type II misses gains
Balance with business cost
Code Example: Simulating Both Error Rates
import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower
rng = np.random.default_rng(42)
alpha, n, sims = 0.05, 30, 5000
# Type I error rate: no real difference
type1 = np.mean([stats.ttest_ind(rng.normal(0, 1, n), rng.normal(0, 1, n)).pvalue < alpha for _ in range(sims)])
# Type II error rate: real difference of 0.5 SD
type2 = np.mean([stats.ttest_ind(rng.normal(0.5, 1, n), rng.normal(0, 1, n)).pvalue >= alpha for _ in range(sims)])
theory_power = TTestIndPower().power(effect_size=0.5, nobs1=n, alpha=alpha)
print(f"Type I rate {type1:.3f} (α = {alpha})")
print(f"Type II rate {type2:.3f} → power {1 - type2:.3f} (theory {theory_power:.3f})")
for new_alpha in (0.10, 0.05, 0.01):
p = TTestIndPower().power(effect_size=0.5, nobs1=n, alpha=new_alpha)
print(f"α = {new_alpha:.2f}: β = {1 - p:.3f}")
The simulated Type I rate is close to α, and lowering α raises β for the same sample, showing the trade-off directly.
Common Pitfalls
Thinking α is the probability that a significant result is a false positive (that depends on prior plausibility and power).
Running underpowered studies, which have high β and exaggerate effects when they are significant.
Multiple testing, which raises the overall Type I error rate.
8
What is the significance level in a hypothesis test, and how is it chosen?
KnownRevise
The significance level (α) is the threshold for rejecting the null hypothesis, set before the analysis: if the p-value is at or below α, the result is called significant. It equals the Type I error rate, the probability of rejecting H₀ when it is actually true, so α = 0.05 means accepting a 5% false-positive rate when there is no effect. It is chosen by weighing the cost of false positives against false negatives: lower α (0.01, 0.001, or 5σ in particle physics) when false positives are expensive, higher α (0.10) in exploratory work when missing an effect is worse, adjusted for multiple tests.
Number of tests (family-wise error or false discovery rate control).
Sample size and power: lower α needs more data for the same power.
Field conventions and regulatory requirements.
Code Example: α, Power and Multiple Testing
import numpy as np
from statsmodels.stats.multitest import multipletests
from statsmodels.stats.power import TTestIndPower
for alpha in (0.10, 0.05, 0.01, 0.001):
n = TTestIndPower().solve_power(effect_size=0.3, alpha=alpha, power=0.8)
print(f"α = {alpha:<5}: n per group for 80% power at d = 0.3 → {int(np.ceil(n))}")
# 20 independent tests at α = 0.05: chance of at least one false positive
print("family-wise error rate for 20 tests:", round(1 - 0.95 ** 20, 3))
rng = np.random.default_rng(42)
p_values = np.r_[rng.uniform(0, 1, 18), [0.001, 0.004]] # 18 nulls, 2 real effects
for method in ("bonferroni", "fdr_bh"):
reject, adjusted, _, _ = multipletests(p_values, alpha=0.05, method=method)
print(f"{method:>10}: {reject.sum()} rejections")
print("unadjusted:", (p_values < 0.05).sum(), "rejections")
Stricter α demands bigger samples, and running many tests at α = 0.05 makes at least one false positive likely (about 64% for 20 tests), which is what corrections such as Bonferroni and Benjamini-Hochberg address.
Common Pitfalls
Choosing or changing α after seeing the p-value.
Treating 0.05 as a law of nature rather than a convention.
Ignoring multiplicity across metrics, segments and time points.
9
What is a confidence interval, and why is it important?
KnownRevise
A confidence interval (CI) is a range of values, computed from sample data, that is likely to contain an unknown population parameter such as a mean or a difference in proportions. A 95% CI comes from a procedure that, over many repeated samples, captures the true parameter 95% of the time; any single interval either contains it or does not. CIs matter because they show both the estimate and its precision: a narrow interval means a precise estimate, a wide one means uncertainty, and an interval for a difference that excludes zero corresponds to a significant result at the matching α, while also showing whether the effect is practically meaningful.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
What Controls the Width
Factor
Effect on width
Larger sample size
Narrower (∝ 1/√n)
More variability in the data
Wider
Higher confidence level (99% vs 95%)
Wider
Formula for a mean: x̄ ± t* × s/√n.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
true_mean = 50
def mean_ci(sample, level=0.95):
se = stats.sem(sample)
return stats.t.interval(level, len(sample) - 1, loc=sample.mean(), scale=se)
sample = rng.normal(true_mean, 10, 40)
low, high = mean_ci(sample)
print(f"sample mean {sample.mean():.2f}, 95% CI [{low:.2f}, {high:.2f}]")
# Coverage: repeat the study many times
covered = 0
for _ in range(10_000):
s = rng.normal(true_mean, 10, 40)
lo, hi = mean_ci(s)
covered += lo <= true_mean <= hi
print("share of 95% intervals containing the true mean:", covered / 10_000)
for n in (10, 100, 1000):
lo, hi = mean_ci(rng.normal(true_mean, 10, n))
print(f"n={n:>4}: width {hi - lo:.2f}")
# Bootstrap CI for a median (no formula needed)
boot = stats.bootstrap((sample,), np.median, confidence_level=0.95, random_state=42)
print("bootstrap 95% CI for the median:", np.round(boot.confidence_interval, 2))
About 95% of the intervals contain the true mean, which is exactly what “95% confidence” promises, and widths shrink roughly tenfold when n grows a hundredfold.
Common Pitfalls
Saying “there is a 95% probability the true mean is in this interval” (that is a Bayesian credible interval statement).
Thinking 95% of the data fall inside the CI; that is a prediction or reference interval.
Judging overlap of two CIs as a test of their difference; compute the CI of the difference.
10
What are the null hypothesis and the alternative hypothesis?
KnownRevise
The null hypothesis (H₀) is the default claim of “no effect” or “no difference” that a test tries to find evidence against, such as “the new drug has the same mean effect as placebo” or “the coin is fair”. The alternative hypothesis (H₁ or Hₐ) is what you would conclude if the evidence against H₀ is strong enough: “the drug’s effect differs from placebo” (two-sided) or “the drug lowers blood pressure more” (one-sided). The test calculates how surprising the data would be if H₀ were true; you either reject H₀ or fail to reject it, and you never “accept” or prove H₀.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Examples
Question
H₀
H₁
Sidedness
Is the coin fair?
p = 0.5
p ≠ 0.5
Two-sided
Does the new page convert better?
p_B ≤ p_A
p_B > p_A
One-sided
Do three diets differ?
μ₁ = μ₂ = μ₃
At least one mean differs
ANOVA
Are gender and preference related?
Independent
Associated
Chi-square
Rules for Writing Hypotheses
They are statements about population parameters, not sample statistics.
They are mutually exclusive and decided before looking at the data.
H₀ contains the equality (=, ≤ or ≥).
Choose one-sided only when the other direction is irrelevant for the decision.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
flips = rng.binomial(1, 0.58, 200) # a slightly biased coin
heads = int(flips.sum())
two_sided = stats.binomtest(heads, 200, p=0.5, alternative="two-sided")
one_sided = stats.binomtest(heads, 200, p=0.5, alternative="greater")
print(f"{heads} heads in 200 flips")
print(f"H0: p = 0.5 vs H1: p ≠ 0.5 → p-value {two_sided.pvalue:.4f}")
print(f"H0: p ≤ 0.5 vs H1: p > 0.5 → p-value {one_sided.pvalue:.4f}")
alpha = 0.05
print("decision (two-sided):", "reject H0" if two_sided.pvalue < alpha else "fail to reject H0")
The one-sided p-value is about half the two-sided one when the data go in the hypothesized direction, which is why the direction must be chosen in advance.
Common Pitfalls
Saying “we accept the null hypothesis”; absence of evidence is not evidence of absence.
Writing hypotheses about the sample (“the sample mean is 5”).
Switching to a one-sided test after seeing which way the result went.
11
What is Bayes' theorem, and how is it used in statistics?
KnownRevise
Bayes’ theorem describes how to update the probability of a hypothesis given new evidence: P(H | E) = P(E | H) × P(H) / P(E), where P(H) is the prior, P(E | H) the likelihood of the evidence if the hypothesis is true, P(E) the overall probability of the evidence, and P(H | E) the posterior. In statistics it is used to interpret diagnostic tests (accounting for base rates), in spam filtering (naive Bayes), and as the foundation of Bayesian inference, where priors over parameters are updated with data to give posterior distributions.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Classic Example: Medical Test
A disease affects 1% of people. A test detects 95% of cases (sensitivity) and has a 5% false-positive rate. If you test positive:
Only about 16%, because false positives among the many healthy people outnumber the true positives. Ignoring the prior is the base-rate fallacy.
Uses
Area
How Bayes’ theorem is used
Diagnostic testing
Positive and negative predictive values
Spam filtering
P(spam | words) with naive Bayes
Bayesian inference
Posterior distributions of parameters
A/B testing
P(variant B is better | data)
Machine learning
Generative classifiers, Bayesian networks
Code Example
import numpy as np
def posterior(prior, sensitivity, false_positive_rate):
evidence = sensitivity * prior + false_positive_rate * (1 - prior)
return sensitivity * prior / evidence
for prior in (0.001, 0.01, 0.10, 0.50):
print(f"prevalence {prior:>6.1%}: P(disease | positive) = {posterior(prior, 0.95, 0.05):.3f}")
# Two positive tests in a row (independent): the first posterior becomes the new prior
p1 = posterior(0.01, 0.95, 0.05)
p2 = posterior(p1, 0.95, 0.05)
print(f"after one positive {p1:.3f}, after two positives {p2:.3f}")
# Simulation check
rng = np.random.default_rng(42)
n = 1_000_000
sick = rng.random(n) < 0.01
positive = np.where(sick, rng.random(n) < 0.95, rng.random(n) < 0.05)
print("simulated P(disease | positive):", round(sick[positive].mean(), 3))
The posterior depends heavily on the prevalence, and a second independent positive result raises the probability sharply. The simulation confirms the formula.
Common Pitfalls
Confusing P(E | H) with P(H | E).
Ignoring base rates (prevalence).
Treating repeated tests as independent when their errors are correlated.
12
How do discrete and continuous probability distributions differ?
KnownRevise
Discrete distributions describe variables that take countable values (0, 1, 2, … or a list of categories), such as the number of defects or the roll of a die. Each value has a positive probability given by a probability mass function (PMF), and the probabilities sum to 1. Continuous distributions describe variables that can take any value in an interval, such as height or time. They are described by a probability density function (PDF): the probability of any exact value is 0, probabilities come from the area under the curve over an interval, and the total area is 1. Both have a cumulative distribution function (CDF).
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Comparison
Aspect
Discrete
Continuous
Values
Countable
Any value in an interval
Described by
PMF: P(X = x)
PDF: f(x), a density
P(X = exact value)
Can be positive
Always 0
Probabilities
Sum of PMF values
Integral (area) of the PDF
CDF
Step function
Smooth curve
Examples
Bernoulli, binomial, Poisson, geometric
Normal, exponential, uniform, beta, gamma
A PDF value can exceed 1 (it is a density, not a probability).
Code Example
import numpy as np
from scipy import stats, integrate
# Discrete: number of website sign-ups per hour ~ Poisson(3)
pois = stats.poisson(3)
print("P(X = 2):", round(pois.pmf(2), 4))
print("P(X ≤ 2) as a sum of PMF values:", round(sum(pois.pmf(k) for k in range(3)), 4), "=", round(pois.cdf(2), 4))
print("PMF sums to 1:", round(sum(pois.pmf(k) for k in range(100)), 6))
# Continuous: delivery time ~ Normal(30, 5) minutes
norm = stats.norm(30, 5)
area, _ = integrate.quad(norm.pdf, 25, 35)
print("P(25 < T < 35) as an area:", round(area, 4), "=", round(norm.cdf(35) - norm.cdf(25), 4))
print("P(T = 30 exactly): 0, but the density there is", round(norm.pdf(30), 4))
narrow = stats.norm(0, 0.1)
print("a density can exceed 1:", round(narrow.pdf(0), 3))
rng = np.random.default_rng(42)
print("sample values:", rng.poisson(3, 5), rng.normal(30, 5, 3).round(2))
Discrete probabilities are added, continuous ones integrated. In SciPy, discrete distributions have pmf and continuous ones pdf.
Common Pitfalls
Reading a PDF value as a probability.
Modelling counts with a continuous distribution when counts are small.
Forgetting continuity corrections when approximating a discrete distribution with a continuous one.
13
What are the properties of a normal distribution?
KnownRevise
The normal (Gaussian) distribution is a continuous, symmetric, bell-shaped distribution fully described by its mean μ and standard deviation σ. Its mean, median and mode are equal; it extends over all real numbers; and about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations of the mean. Sums and linear combinations of independent normal variables are normal, and by the Central Limit Theorem averages of many independent variables are approximately normal, which is why it is central to confidence intervals, hypothesis tests and linear regression.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Properties
Property
Detail
Parameters
Mean μ (location), SD σ (spread)
Shape
Symmetric bell; skewness 0, excess kurtosis 0
Centre
Mean = median = mode
Empirical rule
68% within ±1σ, 95% within ±2σ (exactly ±1.96σ), 99.7% within ±3σ
Inflection points
At μ ± σ
Standardization
z = (x − μ)/σ gives the standard normal N(0, 1)
Closure
aX + b and sums of independent normals are normal
Support
All real numbers (tails never reach zero)
Code Example
import numpy as np
from scipy import stats
mu, sigma = 170, 8 # adult height in cm
dist = stats.norm(mu, sigma)
for k in (1, 2, 3):
print(f"P(within ±{k}σ) = {dist.cdf(mu + k * sigma) - dist.cdf(mu - k * sigma):.4f}")
print("z-score of 186 cm:", (186 - mu) / sigma, "| P(height > 186):", round(dist.sf(186), 4))
print("middle 95% range:", np.round(dist.interval(0.95), 1))
rng = np.random.default_rng(42)
x = rng.normal(mu, sigma, 100_000)
print(f"mean {x.mean():.2f}, median {np.median(x):.2f}, skew {stats.skew(x):.3f}, excess kurtosis {stats.kurtosis(x):.3f}")
# Sum of independent normals is normal with added variances
y = rng.normal(10, 6, 100_000)
total = x + y
print(f"sum: mean {total.mean():.2f} (theory {mu + 10}), SD {total.std():.2f} (theory {np.sqrt(sigma**2 + 6**2):.2f})")
Standardizing to z-scores lets you use one table (or norm.cdf) for any normal distribution.
Common Pitfalls
Assuming real data are normal without checking (Q-Q plots); many are skewed or heavy-tailed.
Applying the 68-95-99.7 rule to non-normal data.
Ignoring that the normal assigns non-zero probability to impossible values (negative heights), usually negligible but sometimes not.
14
What is the Law of Large Numbers, and how does it relate to statistics?
KnownRevise
The Law of Large Numbers (LLN) states that as the number of independent, identically distributed observations grows, the sample mean converges to the true population mean (the expected value). The weak law says it converges in probability; the strong law says it converges almost surely. In statistics it is the reason sample estimates become reliable with more data, it justifies Monte Carlo simulation and frequency-based probabilities, and it underpins insurance and casinos, whose average outcomes over many policies or bets are predictable even though each one is not.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
LLN vs Central Limit Theorem
Aspect
Law of Large Numbers
Central Limit Theorem
Says
The sample mean approaches the true mean
The sample mean’s distribution becomes approximately normal
Answers
Where the mean ends up
How much it varies around the true mean (SE = σ/√n)
Used for
Consistency of estimators, Monte Carlo
Confidence intervals, hypothesis tests
Code Example
import numpy as np
rng = np.random.default_rng(42)
# Die rolls: expected value 3.5
rolls = rng.integers(1, 7, 100_000)
running_mean = np.cumsum(rolls) / np.arange(1, len(rolls) + 1)
for n in (10, 100, 1_000, 10_000, 100_000):
print(f"after {n:>7,} rolls: mean = {running_mean[n - 1]:.4f} (error {abs(running_mean[n - 1] - 3.5):.4f})")
# Insurance: individual claims are unpredictable, the average per policy is not
claims = rng.binomial(1, 0.02, 1_000_000) * rng.gamma(2, 2500, 1_000_000)
print("expected claim per policy:", 0.02 * 2 * 2500, "| observed average:", round(claims.mean(), 1))
# Heavy tails: the Cauchy distribution has no mean, so the LLN does not apply
cauchy = rng.standard_cauchy(100_000)
print("Cauchy running means:", [round(cauchy[:n].mean(), 2) for n in (100, 10_000, 100_000)])
The die’s running mean settles at 3.5 and the error shrinks roughly like 1/√n. The Cauchy example shows the LLN requires a finite mean: its running mean never settles.
Common Pitfalls
Gambler’s fallacy: the LLN does not mean short-run outcomes “balance out”; earlier deviations are diluted, not corrected.
Assuming it applies to dependent or non-identically distributed data without checking.
Forgetting that convergence can be slow for heavy-tailed data.
15
What role does the binomial distribution play in statistics?
KnownRevise
The binomial distribution gives the probability of getting exactly k successes in n independent trials, each with the same success probability p: P(X = k) = C(n, k) pᵏ (1 − p)ⁿ⁻ᵏ, with mean np and variance np(1 − p). It is the basic model for yes/no outcomes: conversions out of visitors, defective items in a batch, patients who respond to treatment. It underlies proportion estimates and their confidence intervals, the binomial test and two-proportion tests used in A/B testing, logistic regression’s likelihood, and quality-control sampling plans.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Requirements
Condition
Example check
Fixed number of trials n
500 visitors
Two outcomes per trial
Converted or not
Same probability p each trial
Similar visitors, no time trend
Independent trials
One visitor’s choice doesn’t affect another’s
For large n with small p, it approaches the Poisson (λ = np); for large np and n(1 − p), the normal.
Code Example
import numpy as np
from scipy import stats
n, p = 20, 0.3
dist = stats.binom(n, p)
print("P(X = 6):", round(dist.pmf(6), 4), "| P(X ≤ 3):", round(dist.cdf(3), 4))
print("mean", dist.mean(), "| variance", dist.var())
# Binomial test: 62 conversions out of 1000 vs a historical rate of 5%
res = stats.binomtest(62, n=1000, p=0.05, alternative="greater")
ci = res.proportion_ci(confidence_level=0.95, method="wilson")
print(f"binomial test p = {res.pvalue:.4f}; 95% Wilson CI [{ci.low:.4f}, {ci.high:.4f}]")
# Quality control: accept a batch if ≤ 2 defects in a sample of 50
for defect_rate in (0.01, 0.05, 0.10):
print(f"defect rate {defect_rate:.0%}: P(accept) = {stats.binom.cdf(2, 50, defect_rate):.3f}")
rng = np.random.default_rng(42)
sims = rng.binomial(n, p, 100_000)
print("simulated P(X = 6):", round((sims == 6).mean(), 4))
The binomial test’s p-value is the probability of 62 or more conversions out of 1000 if the true rate were still 5%. The acceptance probabilities form the sampling plan’s operating characteristic.
Common Pitfalls
Using it when trials are dependent (repeat visits by the same user).
Using the simple Wald interval for small n or extreme p; prefer Wilson.
Treating varying probabilities across trials as one p (overdispersion; use beta-binomial).
16
What is the difference between joint, marginal and conditional probability?
KnownRevise
Joint probability P(A and B) is the probability that two events happen together. Marginal probability P(A) is the probability of one event regardless of the other, obtained by summing (or integrating) the joint probabilities over all values of the other variable. Conditional probability P(A | B) is the probability of A given that B has happened: P(A | B) = P(A and B) / P(B). Two events are independent when P(A | B) = P(A), equivalently P(A and B) = P(A)P(B). These three ideas are the building blocks of Bayes’ theorem, contingency tables and probabilistic models.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, pandas 2.2+
Summary
Type
Notation
Question
From a contingency table
Joint
P(A ∩ B)
Both happen?
Cell count / grand total
Marginal
P(A)
A happens, ignoring B?
Row or column total / grand total
Conditional
P(A | B)
A happens, given B?
Cell count / column (or row) total
Note that P(A | B) and P(B | A) are usually different: P(positive test | disease) is not P(disease | positive test).
P(disease | smoker) is about 0.15 while P(smoker | disease) is much higher: the two conditionals answer different questions. Because the joint probability differs from the product of the marginals, smoking and disease are not independent here.
Common Pitfalls
Confusing P(A | B) with P(B | A) (the “prosecutor’s fallacy”).
Assuming independence without checking, e.g. multiplying probabilities of correlated failures.
Forgetting that marginals come from summing the joint distribution.
17
How does the Poisson distribution differ from the normal distribution?
KnownRevise
The Poisson distribution is a discrete distribution for counts of events in a fixed interval when events occur independently at a constant average rate λ: calls per hour, defects per batch, goals per match. It takes only non-negative integers, its mean and variance are both λ, and it is right-skewed when λ is small. The normal distribution is continuous, symmetric and defined over all real numbers, with a mean and variance that are independent parameters. For large λ (roughly above 20), the Poisson is well approximated by a normal with mean λ and variance λ.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Comparison
Property
Poisson(λ)
Normal(μ, σ²)
Type
Discrete counts 0, 1, 2, …
Continuous, all real numbers
Parameters
λ (rate)
μ (mean), σ (SD)
Mean / variance
λ / λ
μ / σ² (independent)
Shape
Right-skewed for small λ
Symmetric bell
Typical data
Event counts
Measurements, sums and averages
Regression
Poisson / negative binomial GLM
Linear regression
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
for lam in (2, 50):
counts = rng.poisson(lam, 100_000)
print(f"λ={lam}: mean {counts.mean():.2f}, variance {counts.var():.2f}, skew {stats.skew(counts):.2f}")
# Probability of exactly 0 and at most 3 events when λ = 2
print("P(X=0):", round(stats.poisson.pmf(0, 2), 4), "| P(X≤3):", round(stats.poisson.cdf(3, 2), 4))
# Normal approximation (with continuity correction) works for large λ, not small
for lam, k in ((2, 3), (50, 55)):
exact = stats.poisson.cdf(k, lam)
approx = stats.norm.cdf(k + 0.5, loc=lam, scale=np.sqrt(lam))
print(f"λ={lam}: P(X≤{k}) exact {exact:.4f}, normal approx {approx:.4f}")
# Overdispersion check for real count data: variance much larger than the mean suggests negative binomial
calls = rng.negative_binomial(5, 5 / (5 + 10), 1000)
print("dispersion index (variance/mean):", round(calls.var() / calls.mean(), 2))
With λ = 2 the normal approximation is poor; with λ = 50 it is close. A dispersion index well above 1 indicates the Poisson assumption of equal mean and variance fails.
Common Pitfalls
Modelling counts with linear regression (negative predictions, wrong variance).
Ignoring overdispersion, which makes Poisson standard errors too small.
Applying Poisson when events are not independent (e.g. clustered failures).
18
What is a cumulative distribution function (CDF)?
KnownRevise
The cumulative distribution function (CDF) of a random variable X gives the probability that X takes a value less than or equal to x: F(x) = P(X ≤ x). It exists for every distribution, discrete or continuous, rises from 0 to 1, and never decreases. For continuous variables it is the integral of the probability density function (PDF), and interval probabilities follow directly: P(a < X ≤ b) = F(b) − F(a). Its inverse, the quantile function, gives percentiles, and the empirical CDF of a sample is used to compare distributions (Kolmogorov-Smirnov test).
The empirical CDF closely tracks the theoretical curve; the KS test measures the largest vertical gap between them.
Common Pitfalls
Confusing the CDF with the PDF (the PDF value is a density, not a probability).
Forgetting P(X ≤ x) and P(X < x) differ for discrete variables.
Using ppf and cdf with the wrong scale parameterization.
19
What are the use cases of the exponential and uniform distributions?
KnownRevise
The exponential distribution models the waiting time until the next event when events happen independently at a constant average rate λ (a Poisson process): time between customer arrivals, time until a component fails (with a constant hazard), or time between website requests. It has mean 1/λ and is memoryless: the chance of waiting another minute does not depend on how long you have already waited. The uniform distribution gives equal probability to every value in an interval [a, b]. It models complete uncertainty within known limits, rounding errors and random arrival within a window, and is the basis of random number generation and simulation.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Comparison
Property
Exponential(λ)
Uniform(a, b)
Support
x ≥ 0
a ≤ x ≤ b
PDF
λe^(−λx)
1/(b − a)
Mean
1/λ
(a + b)/2
Variance
1/λ²
(b − a)²/12
Key property
Memoryless
All values equally likely
Uses
Waiting times, lifetimes, queueing
Simulation, random sampling, priors with bounds, rounding error
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# Customers arrive on average 12 per hour → mean gap 5 minutes
rate_per_min = 12 / 60
gaps = rng.exponential(scale=1 / rate_per_min, size=100_000)
print("mean gap (min):", round(gaps.mean(), 2), "| P(gap > 10 min):", round((gaps > 10).mean(), 4),
"| theory:", round(np.exp(-rate_per_min * 10), 4))
# Memorylessness: P(gap > 15 | gap > 5) equals P(gap > 10)
print("P(>15 | >5):", round((gaps > 15).sum() / (gaps > 5).sum(), 4))
# Uniform: a bus arrives at a random moment within a 20-minute window
wait = rng.uniform(0, 20, 100_000)
print("uniform mean wait:", round(wait.mean(), 2), "| variance:", round(wait.var(), 2), "| theory:", round(20 ** 2 / 12, 2))
# Inverse transform sampling: uniform draws become exponential draws
u = rng.uniform(size=100_000)
from_uniform = -np.log(1 - u) / rate_per_min
print("KS test vs exponential, p:", round(stats.kstest(from_uniform, "expon", args=(0, 1 / rate_per_min)).pvalue, 3))
The last step shows a key link: uniform random numbers can be transformed into any distribution through its inverse CDF.
Common Pitfalls
Using the exponential for lifetimes that wear out (increasing hazard); use Weibull or gamma.
Confusing the rate λ with the scale 1/λ (NumPy and SciPy use the scale).
Assuming “uniform” when you simply lack data about the shape.
20
How is covariance different from correlation?
KnownRevise
Covariance measures whether two variables move together: positive when they tend to be above (or below) their means at the same time, negative when one is above while the other is below. Its size depends on the units, so a covariance of 500 cannot be judged as strong or weak. Correlation is covariance standardized by both standard deviations: r = cov(X, Y) / (σₓ σᵧ). It is unitless, always between −1 and +1, and so comparable across variable pairs. Both measure only linear association, and neither implies causation.
Changing height from metres to centimetres multiplies the covariance by 100 but leaves the correlation unchanged. The x² example shows that zero correlation does not mean no relationship.
Common Pitfalls
Comparing covariances between variable pairs measured in different units.
Concluding “no relationship” from a correlation near zero without plotting.
Letting outliers drive the correlation; consider Spearman’s rank correlation.
21
What are measures of central tendency, and why do they matter?
KnownRevise
Measures of central tendency summarize a dataset with a single “typical” value. The main ones are the mean (arithmetic average), the median (middle value when sorted) and the mode (most frequent value); others include the trimmed mean, the geometric mean (for growth rates and ratios) and the harmonic mean (for rates such as speeds). They matter because they let you summarize and compare groups quickly, are the basis of many statistical tests, and serve as baselines for predictions, but they must be chosen to fit the data’s shape and scale and reported with a measure of spread.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Which Measure to Use
Measure
Best for
Weakness
Mean
Symmetric numeric data
Sensitive to outliers
Median
Skewed data, ordinal data
Ignores magnitudes
Mode
Categorical data
Can be non-unique or unstable
Trimmed mean
Data with some outliers
Discards information
Geometric mean
Growth rates, ratios
Positive values only
Harmonic mean
Rates (speed, price/earnings)
Dominated by small values
Code Example
from collections import Counter
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
order_values = np.r_[rng.gamma(2, 25, 500), [1500, 2200]]
print(f"mean {order_values.mean():.2f}, median {np.median(order_values):.2f}, "
f"10% trimmed mean {stats.trim_mean(order_values, 0.1):.2f}")
payment_methods = ["card", "card", "wallet", "card", "cash", "wallet", "card"]
print("mode of payment method:", Counter(payment_methods).most_common(1)[0][0])
yearly_growth = np.array([1.10, 0.95, 1.20, 1.05]) # +10%, −5%, +20%, +5%
print(f"average growth: arithmetic {yearly_growth.mean() - 1:.2%}, geometric {stats.gmean(yearly_growth) - 1:.2%}")
speeds = np.array([60, 40]) # km/h over two equal distances
print("average speed over the trip:", round(stats.hmean(speeds), 1), "km/h")
The geometric mean gives the true compound growth rate, and the harmonic mean the true average speed over equal distances; the arithmetic mean would overstate both.
Common Pitfalls
Using the arithmetic mean of growth rates or ratios.
Computing a mean of ordinal codes (e.g. 1-5 satisfaction) without saying so.
Reporting a centre without the spread (SD or IQR).
22
What are range, IQR, variance and standard deviation?
KnownRevise
Measures of dispersion describe how spread out data are. The range is max − min: simple but determined entirely by the two most extreme values. The interquartile range (IQR) is Q3 − Q1, the spread of the middle 50%, robust to outliers. The variance is the average squared deviation from the mean (divided by n − 1 for a sample, to make it unbiased), and the standard deviation (SD) is its square root, in the same units as the data. Use SD with the mean for roughly symmetric data, and IQR with the median for skewed data or data with outliers.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Comparison
Measure
Formula
Units
Robust to outliers?
Pair with
Range
max − min
Data units
No
Quick check
IQR
Q3 − Q1
Data units
Yes
Median
Variance
Σ(x − x̄)² / (n − 1)
Squared units
No
Used in models and tests
Standard deviation
√variance
Data units
No
Mean
MAD
median(|x − median|)
Data units
Yes
Median
For normal data, about 68% of values lie within 1 SD of the mean and 95% within 2 SD.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
clean = rng.normal(50, 5, 200)
with_outlier = np.r_[clean, [150]]
for name, x in (("clean", clean), ("with one outlier", with_outlier)):
q1, q3 = np.percentile(x, [25, 75])
print(f"{name:>16}: range {np.ptp(x):6.2f}, IQR {q3 - q1:5.2f}, "
f"variance {x.var(ddof=1):7.2f}, SD {x.std(ddof=1):5.2f}, "
f"MAD {stats.median_abs_deviation(x):5.2f}")
# Population (ddof=0) vs sample (ddof=1) variance
small = np.array([4, 8, 6, 5, 7])
print("population variance:", small.var(ddof=0), "| sample variance:", small.var(ddof=1))
A single outlier greatly inflates the range, variance and SD, while the IQR and MAD barely change. NumPy’s default ddof=0 gives the population formula, so pass ddof=1 for sample estimates (pandas defaults to ddof=1).
Common Pitfalls
Mixing up ddof defaults between NumPy and pandas.
Interpreting variance directly; it is in squared units.
Applying the 68-95-99.7 rule to skewed data.
23
What is the difference between mean and median, and when to use each?
KnownRevise
The mean is the arithmetic average (sum ÷ count); it uses every value, so it is efficient for symmetric data but is pulled toward outliers and long tails. The median is the middle value of the sorted data; it depends only on the order of values, so it is robust to outliers and skew. Use the mean for roughly symmetric data, or when totals matter (total revenue = mean × count). Use the median for skewed data such as incomes, house prices, response times or waiting times, and whenever outliers could distort the typical value.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Comparison
Aspect
Mean
Median
Uses
All values and their sizes
Only the middle position
Outliers
Sensitive
Robust (50% breakdown point)
Skewed data
Pulled toward the tail
Represents the typical value
Totals and expected values
Directly related
Not related
Statistical efficiency (normal data)
Higher
About 64% of the mean’s
Code Example
import numpy as np
rng = np.random.default_rng(42)
salaries = np.r_[rng.normal(60_000, 8_000, 49), [2_000_000]] # one CEO
print(f"mean salary {salaries.mean():,.0f} vs median {np.median(salaries):,.0f}")
# Sampling variability on normal data: the mean is more precise
means, medians = [], []
for _ in range(5000):
s = rng.normal(100, 15, 50)
means.append(s.mean())
medians.append(np.median(s))
print(f"normal data, SD of estimate: mean {np.std(means):.2f}, median {np.std(medians):.2f}")
# Heavy-tailed data: the median is more stable
means, medians = [], []
for _ in range(5000):
s = rng.standard_t(2, 50)
means.append(s.mean())
medians.append(np.median(s))
print(f"heavy-tailed data, SD of estimate: mean {np.std(means):.2f}, median {np.std(medians):.2f}")
One extreme salary moves the mean by tens of thousands but barely moves the median. With normal data the mean is the more precise estimator; with heavy tails the median wins.
Common Pitfalls
Reporting “average income” as a mean when most people earn far less.
Using the median when you need totals (budgeting, expected revenue).
Reporting either without a measure of spread.
24
How do you describe skewness and kurtosis in a dataset?
KnownRevise
Skewness measures the asymmetry of a distribution: positive (right) skew has a long tail toward high values, so the mean is pulled above the median (incomes, response times); negative (left) skew has a long tail toward low values (exam scores near a ceiling); zero means roughly symmetric. Kurtosis measures how heavy the tails are relative to a normal distribution; excess kurtosis (kurtosis − 3) is 0 for a normal distribution, positive for heavy-tailed data with more extreme outliers (returns in finance), and negative for light-tailed, flatter data (uniform). Both guide the choice of summaries, transforms and tests.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Interpretation
Measure
Value
Meaning
Example
Skewness
> 0
Long right tail; mean > median
Income, house prices
Skewness
< 0
Long left tail; mean < median
Age at death in rich countries
Skewness
≈ 0
Symmetric
Heights
Excess kurtosis
> 0
Heavy tails, more outliers
Stock returns (t distribution)
Excess kurtosis
< 0
Light tails
Uniform distribution
Rough guide: |skewness| above 1 is strong; between 0.5 and 1 moderate.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
samples = {
"normal": rng.normal(0, 1, 10_000),
"lognormal (right skew)": rng.lognormal(0, 0.6, 10_000),
"ceiling scores (left skew)": 100 - rng.gamma(2, 5, 10_000),
"t, 4 df (heavy tails)": rng.standard_t(4, 10_000),
"uniform (light tails)": rng.uniform(-1, 1, 10_000),
}
for name, x in samples.items():
print(f"{name:>27}: skew {stats.skew(x):6.2f}, excess kurtosis {stats.kurtosis(x):6.2f}, "
f"mean − median {x.mean() - np.median(x):6.3f}")
x = samples["lognormal (right skew)"]
print("skew after log transform:", round(stats.skew(np.log(x)), 3))
print("skewness test p-value (normal sample):", round(stats.skewtest(samples["normal"]).pvalue, 3))
scipy.stats.kurtosis returns excess kurtosis by default (fisher=True). The log transform removes most of the right skew of lognormal data.
Common Pitfalls
Saying high kurtosis means a “peaked” distribution; it is about the tails.
Trusting skewness and kurtosis from small samples; they are very sensitive to single outliers.
Reporting the mean for strongly skewed data without the median.
25
What is the five-number summary in descriptive statistics?
KnownRevise
The five-number summary describes a dataset with five values: the minimum, the first quartile (Q1, 25th percentile), the median (Q2), the third quartile (Q3, 75th percentile) and the maximum. Together they show the centre (median), the spread of the middle half (the interquartile range, IQR = Q3 − Q1), the overall range and the skew. It is robust to outliers in its middle three values, is the basis of the box plot, and supports the common outlier rule: values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR are flagged.
If the median is closer to Q1 than to Q3, and the upper whisker is longer, the data are right-skewed.
Code Example
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(42)
delivery_minutes = np.r_[rng.gamma(4, 8, 200), [150, 180]] # right-skewed with two late deliveries
q1, median, q3 = np.percentile(delivery_minutes, [25, 50, 75])
summary = {"min": delivery_minutes.min(), "Q1": q1, "median": median, "Q3": q3, "max": delivery_minutes.max()}
print({k: round(v, 1) for k, v in summary.items()})
iqr = q3 - q1
low, high = q1 - 1.5 * iqr, q3 + 1.5 * iqr
outliers = delivery_minutes[(delivery_minutes < low) | (delivery_minutes > high)]
print(f"IQR = {iqr:.1f}; outlier fences [{low:.1f}, {high:.1f}]; outliers: {np.sort(outliers).round(1)}")
print(pd.Series(delivery_minutes).describe().round(1)) # pandas gives the same quartiles plus mean/std
plt.boxplot(delivery_minutes, vert=False)
plt.xlabel("delivery time (minutes)")
plt.title("Box plot of the five-number summary")
plt.show()
The box spans Q1 to Q3 with a line at the median; whiskers extend to the most extreme points within the fences, and points beyond them are drawn individually.
Common Pitfalls
Different software uses slightly different quartile methods, so small differences are normal.
Treating every 1.5 × IQR point as an error; in skewed data many are genuine.
Hiding multimodal distributions, which a box plot cannot show (add a histogram).
26
What are the steps in conducting a hypothesis test?
KnownRevise
A hypothesis test follows a fixed sequence: (1) state the null hypothesis H₀ (no effect) and the alternative H₁; (2) choose the significance level α (often 0.05) and, ideally, plan the sample size with a power analysis; (3) pick a test whose assumptions fit the data; (4) collect data and compute the test statistic; (5) compute the p-value, the probability of a result at least this extreme if H₀ were true; (6) decide: reject H₀ if p ≤ α, otherwise fail to reject it; and (7) report the effect size with a confidence interval and interpret it in context.
import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower
rng = np.random.default_rng(42)
# Step 2: sample size for a 5-second difference with SD 15
n = int(np.ceil(TTestIndPower().solve_power(effect_size=5 / 15, alpha=0.05, power=0.8, alternative="larger")))
print("planned n per group:", n)
# Step 4-5: collect data and test (one-sided: old > new)
old = rng.normal(60, 15, n)
new = rng.normal(54, 15, n)
res = stats.ttest_ind(old, new, equal_var=False, alternative="greater")
print(f"t = {res.statistic:.2f}, p = {res.pvalue:.4f}")
# Step 6-7: decide and report
diff = old.mean() - new.mean()
ci = stats.ttest_ind(old, new, equal_var=False).confidence_interval(0.95)
print(f"mean reduction {diff:.2f} s, 95% CI [{ci.low:.2f}, {ci.high:.2f}]")
print("reject H0" if res.pvalue <= 0.05 else "fail to reject H0")
“Fail to reject” does not prove H₀; it means the data are not strong enough evidence against it.
Common Pitfalls
Choosing a one-sided test after seeing the direction of the data.
Changing α, the test or the outcome after looking at results.
Equating statistical significance with practical importance.
27
How is a t-test performed, and when should you use one?
KnownRevise
A t-test compares means when the population standard deviation is unknown and estimated from the sample. It computes t = (observed difference − hypothesized difference) / standard error and compares it with a t distribution with the appropriate degrees of freedom to get a p-value. There are three main forms: one-sample (a mean vs a known value), independent two-sample (two separate groups; Welch’s version by default), and paired (the same subjects measured twice). It is appropriate for continuous outcomes, independent observations, and roughly normal data or moderately large samples.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Types
Test
Question
Statistic
df
One-sample
Is the mean 50?
(x̄ − μ₀) / (s/√n)
n − 1
Welch two-sample
Do groups A and B differ?
(x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂)
Welch-Satterthwaite
Paired
Did scores change after training?
d̄ / (s_d/√n)
n − 1
When to Use
Continuous outcome, comparing one or two means.
Data roughly normal, or n large enough (often 30+ per group) for the CLT.
Without these: Mann-Whitney U or Wilcoxon; for 3+ groups: ANOVA.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# One-sample: do bottles contain 500 ml on average?
volumes = rng.normal(498.5, 4, 40)
r1 = stats.ttest_1samp(volumes, popmean=500)
print(f"one-sample: mean {volumes.mean():.2f}, t = {r1.statistic:.2f}, p = {r1.pvalue:.4f}")
# Manual calculation matches
t_manual = (volumes.mean() - 500) / (volumes.std(ddof=1) / np.sqrt(len(volumes)))
print("manual t:", round(t_manual, 2))
# Paired: scores before and after training for the same 25 people
before = rng.normal(70, 8, 25)
after = before + rng.normal(3, 4, 25)
r2 = stats.ttest_rel(after, before)
print(f"paired: mean change {np.mean(after - before):.2f}, t = {r2.statistic:.2f}, p = {r2.pvalue:.4f}")
# Independent groups (Welch)
a, b = rng.normal(70, 8, 30), rng.normal(75, 12, 35)
r3 = stats.ttest_ind(a, b, equal_var=False)
print(f"Welch: t = {r3.statistic:.2f}, df = {r3.df:.1f}, p = {r3.pvalue:.4f}")
A small p-value in the one-sample test means average volumes this far from 500 ml would be unlikely if the true mean were 500. The paired test uses within-person differences, which removes person-to-person variation and gives more power.
Common Pitfalls
Running an independent t-test on paired data.
Multiple t-tests across many groups instead of ANOVA.
Heavy outliers in small samples distorting the mean.
28
What is ANOVA, and when is it used?
KnownRevise
ANOVA (analysis of variance) tests whether the means of three or more groups are all equal, by comparing variation between group means with variation within groups. The F statistic is the ratio of these two variances; if group means differ more than within-group noise would explain, F is large and the p-value small. Use ANOVA instead of many pairwise t-tests, which would inflate the false-positive rate, then follow a significant result with a post-hoc test (Tukey’s HSD) to find which groups differ. Variants handle several factors (two-way ANOVA), repeated measures and covariates (ANCOVA).
Two factors and their interaction (diet × exercise)
Repeated measures
Same subjects measured under several conditions
ANCOVA
Adjusts for a continuous covariate (baseline score)
Welch’s ANOVA
Unequal variances
Kruskal-Wallis
Nonparametric alternative
Assumptions
Independent observations, approximately normal residuals, and similar group variances (Levene’s test).
Code Example
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from scipy import stats
from statsmodels.stats.anova import anova_lm
from statsmodels.stats.multicomp import pairwise_tukeyhsd
rng = np.random.default_rng(42)
df = pd.DataFrame({
"diet": np.repeat(["A", "B", "C"], 40),
"weight_loss": np.r_[rng.normal(3, 2, 40), rng.normal(4, 2, 40), rng.normal(5.5, 2, 40)],
})
groups = [g["weight_loss"].to_numpy() for _, g in df.groupby("diet")]
f, p = stats.f_oneway(*groups)
print(f"one-way ANOVA: F = {f:.2f}, p = {p:.2e}")
print("Levene p (equal variances):", round(stats.levene(*groups).pvalue, 3))
model = smf.ols("weight_loss ~ C(diet)", data=df).fit()
table = anova_lm(model, typ=2)
eta_sq = table.loc["C(diet)", "sum_sq"] / table["sum_sq"].sum()
print("eta squared:", round(eta_sq, 3))
print(pairwise_tukeyhsd(df["weight_loss"], df["diet"], alpha=0.05))
The small p-value says at least one diet’s mean differs; Tukey’s table shows which pairs differ with adjusted p-values, and η² gives the share of variance explained by diet.
Common Pitfalls
Concluding all groups differ from a significant F; it only says at least one does.
Running ANOVA on repeated measurements as if they were independent.
Skipping effect sizes.
29
What are effect size and Cohen's d?
KnownRevise
Effect size measures how large a difference or relationship is, independent of sample size, which a p-value does not tell you: with huge samples, trivial differences become “significant”. Cohen’s d is the standardized difference between two means: d = (mean₁ − mean₂) / pooled standard deviation, so d = 0.5 means the means differ by half a standard deviation. Cohen’s rough benchmarks are 0.2 small, 0.5 medium and 0.8 large, but the practical meaning depends on the field. Effect sizes are needed for power analysis, for comparing studies and for meta-analysis.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Common Effect Size Measures
Measure
Used for
Cohen’s d / Hedges’ g
Difference between two means (g corrects small-sample bias)
Pearson r
Correlation
η² / partial η²
ANOVA (share of variance explained)
Odds ratio, risk ratio
Binary outcomes
Cramér’s V
Chi-square tests
Absolute difference with units
Often the most meaningful for decisions
Code Example
import numpy as np
from scipy import stats
def cohens_d(a, b):
a, b = np.asarray(a, float), np.asarray(b, float)
na, nb = len(a), len(b)
pooled_var = ((na - 1) * a.var(ddof=1) + (nb - 1) * b.var(ddof=1)) / (na + nb - 2)
return (a.mean() - b.mean()) / np.sqrt(pooled_var)
def hedges_g(a, b):
n = len(a) + len(b)
return cohens_d(a, b) * (1 - 3 / (4 * n - 9))
rng = np.random.default_rng(42)
for n in (30, 300, 30_000):
a = rng.normal(100.5, 15, n) # true d = 0.5 / 15 ≈ 0.03 (tiny)
b = rng.normal(100.0, 15, n)
p = stats.ttest_ind(a, b).pvalue
print(f"n={n:>6} per group: d = {cohens_d(a, b):6.3f}, p = {p:.4f}")
a = rng.normal(108, 15, 25)
b = rng.normal(100, 15, 25)
print(f"small study: d = {cohens_d(a, b):.3f}, Hedges' g = {hedges_g(a, b):.3f}")
With 30,000 per group a negligible effect of about 0.03 SD can reach p < 0.05, while the effect size stays tiny, which is why both should be reported.
Common Pitfalls
Treating Cohen’s benchmarks as universal; a 0.1 effect may matter in medicine or at scale.
Using the wrong standard deviation for paired designs (use the SD of differences, or say which one).
Reporting effect sizes without confidence intervals.
30
How do you perform a chi-squared test, and what does it show?
KnownRevise
A chi-squared test compares observed counts with the counts expected under a null hypothesis. The goodness-of-fit version checks whether one categorical variable follows a specified distribution (e.g. a fair die); the test of independence checks whether two categorical variables are associated, using a contingency table. The statistic χ² = Σ (O − E)² / E measures the total discrepancy; a small p-value means the observed counts would be unlikely if the null hypothesis were true. It tells you whether a difference or association exists, not how strong it is, so report an effect size such as Cramér’s V.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Steps
State H₀ (e.g. the die is fair; or gender and product choice are independent).
Tabulate the observed counts.
Compute expected counts (from the hypothesized proportions, or row total × column total / grand total).
Compute χ² and the degrees of freedom (k − 1 for goodness of fit; (r − 1)(c − 1) for independence).
Get the p-value and check that expected counts are mostly ≥ 5.
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# Goodness of fit: is a die fair?
rolls = rng.choice(6, size=600, p=[0.15, 0.15, 0.15, 0.15, 0.15, 0.25])
observed = np.bincount(rolls, minlength=6)
chi2, p = stats.chisquare(observed) # expected: 100 per face
print("observed counts:", observed, f"| chi2 = {chi2:.2f}, p = {p:.4f}")
# Independence: plan type vs region
table = np.array([[120, 90, 40],
[80, 110, 60]])
chi2, p, dof, expected = stats.chi2_contingency(table)
cramers_v = np.sqrt(chi2 / (table.sum() * (min(table.shape) - 1)))
print(f"independence: chi2 = {chi2:.2f}, dof = {dof}, p = {p:.4f}, Cramér's V = {cramers_v:.3f}")
print("expected counts:\n", expected.round(1))
The loaded die produces too many sixes, so the goodness-of-fit test rejects fairness; the independence test shows whether plan choice differs by region, and Cramér’s V indicates the association’s strength.
Common Pitfalls
Using percentages instead of counts.
Small expected counts (use Fisher’s exact test or combine categories).
Counting the same person in several cells (observations must be independent).
31
What is a nonparametric test, and why would you use one?
KnownRevise
A nonparametric test makes few or no assumptions about the shape of the population distribution, such as normality. Most work with ranks or signs rather than raw values, which makes them robust to outliers and skewed data and suitable for ordinal data (ratings, rankings). You use them when samples are small and clearly non-normal, data are ordinal, or outliers would distort means. The trade-off is somewhat lower power than parametric tests when the parametric assumptions actually hold, and their hypotheses concern distributions or medians rather than means.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Parametric Tests and Their Nonparametric Alternatives
Situation
Parametric
Nonparametric
Two independent groups
Welch t-test
Mann-Whitney U
Paired measurements
Paired t-test
Wilcoxon signed-rank
Three or more groups
One-way ANOVA
Kruskal-Wallis
Repeated measures, 3+ conditions
Repeated-measures ANOVA
Friedman
Correlation
Pearson
Spearman, Kendall
Distribution comparison
–
Kolmogorov-Smirnov
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# Skewed response times (seconds) with outliers
old_design = rng.lognormal(1.2, 0.6, 30)
new_design = rng.lognormal(1.0, 0.6, 30)
old_design[:2] = [60, 75] # a couple of extreme outliers
print("medians:", round(np.median(old_design), 2), round(np.median(new_design), 2))
print("Shapiro p (old):", round(stats.shapiro(old_design).pvalue, 4))
t = stats.ttest_ind(old_design, new_design, equal_var=False)
u = stats.mannwhitneyu(old_design, new_design, alternative="two-sided")
print(f"Welch t-test p = {t.pvalue:.4f}")
print(f"Mann-Whitney U = {u.statistic:.0f}, p = {u.pvalue:.4f}")
# Effect size for Mann-Whitney: probability a random old value exceeds a random new value
print("P(old > new):", round(u.statistic / (len(old_design) * len(new_design)), 3))
The outliers inflate the variance and weaken the t-test, while the rank-based Mann-Whitney test is unaffected by how extreme they are. A small Mann-Whitney p-value means values in one group tend to be larger than in the other.
Common Pitfalls
Saying Mann-Whitney compares medians; it compares distributions, and equals a median test only when shapes are the same.
Switching to a nonparametric test after the parametric one “failed” (p-hacking).
Using them by default with large samples, where the CLT often makes t-tests fine.
32
What is linear regression, and when is it used?
KnownRevise
Linear regression models a continuous outcome as a linear combination of one or more predictors plus random error: y = β₀ + β₁x₁ + … + βₚxₚ + ε. The coefficients are estimated by ordinary least squares, minimizing the sum of squared residuals; each coefficient is the expected change in y for a one-unit change in that predictor, holding the others fixed. It is used to explain relationships (with confidence intervals and tests for each coefficient), to predict numeric outcomes, and as a strong, interpretable baseline before trying complex models.
The size coefficient (about 2.5) means each extra square metre adds about 2.5 price units with age and city held fixed. The prediction interval (obs_ci_*) is wider than the confidence interval for the mean because it includes individual noise.
Common Pitfalls
Extrapolating outside the range of the data.
Reading coefficients causally when confounders are missing.
Ignoring outliers and influential points (check Cook’s distance).
33
How do you interpret R-squared and adjusted R-squared?
KnownRevise
R-squared (R²) is the proportion of variance in the outcome explained by the model: R² = 1 − SS_residual / SS_total. An R² of 0.70 means the model’s predictions explain 70% of the variation around the mean. R² never decreases when you add predictors, even useless ones, so adjusted R² penalizes the number of predictors: 1 − (1 − R²)(n − 1)/(n − p − 1). Adjusted R² rises only when a new variable improves the fit more than chance would; use it to compare models with different numbers of predictors, and remember that neither measures causal validity or out-of-sample accuracy.
Adding 15 noise variables raises R² but lowers adjusted R², showing why plain R² cannot be used to choose between models of different size.
Common Pitfalls
Treating a high R² as proof that the model is correct or causal.
Comparing R² across models with different outcome variables (e.g. y vs log y).
Using in-sample R² to judge predictive performance; use cross-validation.
34
What are the assumptions of linear regression?
KnownRevise
Linear regression relies on a few assumptions for its estimates and inference to be valid: linearity (the mean of y is a linear function of the predictors), independence of errors, homoscedasticity (constant error variance), normality of errors (mainly for small-sample tests and intervals), and no perfect multicollinearity, plus the key condition that errors are uncorrelated with the predictors (exogeneity, i.e. no omitted confounders). Violations affect different things: some bias the coefficients, others only make standard errors and p-values wrong, so each needs its own check and fix.
Normality of the predictors or of y itself is not required, only of the errors.
Code Example: Diagnostics
import numpy as np
import statsmodels.api as sm
from scipy import stats
from statsmodels.stats.diagnostic import het_breuschpagan
from statsmodels.stats.stattools import durbin_watson
rng = np.random.default_rng(42)
n = 300
x1 = rng.uniform(0, 10, n)
x2 = rng.normal(5, 2, n)
y = 1 + 2 * x1 + 0.3 * x1 ** 2 - 1.5 * x2 + rng.normal(0, 2, n) # true relationship is curved in x1
X = sm.add_constant(np.c_[x1, x2])
fit = sm.OLS(y, X).fit()
resid = fit.resid
print("Breusch-Pagan p:", round(het_breuschpagan(resid, X)[1], 4))
print("Durbin-Watson (≈2 means no autocorrelation):", round(durbin_watson(resid), 2))
print("Shapiro-Wilk p on residuals:", round(stats.shapiro(resid).pvalue, 4))
# Linearity check: residuals correlate with x1² if the curve was missed
print("corr(residuals, x1²):", round(np.corrcoef(resid, x1 ** 2)[0, 1], 3))
fixed = sm.OLS(y, sm.add_constant(np.c_[x1, x1 ** 2, x2])).fit()
print("R² linear:", round(fit.rsquared, 3), "| R² with x1²:", round(fixed.rsquared, 3))
The misspecified model leaves curvature in its residuals; adding the squared term fixes the linearity violation (the model is still “linear” because it is linear in its coefficients).
Common Pitfalls
Testing normality of y instead of the residuals.
Relying only on formal tests; plots often reveal more.
Believing assumptions only matter for p-values: omitted confounders bias the coefficients themselves.
35
What is multicollinearity, and why is it a problem in regression?
KnownRevise
Multicollinearity occurs when two or more predictors in a regression are highly correlated, so one can be nearly predicted from the others. It does not bias predictions much, but it makes individual coefficients unstable: their standard errors inflate, signs can flip between samples, and variables can look insignificant even when they jointly matter. Detect it with the variance inflation factor (VIF) (values above about 5-10 are a warning) and the correlation matrix; fix it by removing or combining redundant variables, using PCA, or applying ridge regression.
Principal component regression, domain-driven composite indices
Little effect on prediction accuracy
If you only need predictions, it may be acceptable
VIFⱼ = 1 / (1 − R²ⱼ), where R²ⱼ comes from regressing predictor j on the others.
Code Example
import numpy as np
import pandas as pd
import statsmodels.api as sm
from sklearn.linear_model import Ridge
from statsmodels.stats.outliers_influence import variance_inflation_factor
rng = np.random.default_rng(42)
n = 200
size_sqft = rng.normal(1500, 300, n)
rooms = size_sqft / 300 + rng.normal(0, 0.3, n) # almost a copy of size
age = rng.uniform(0, 50, n)
price = 50 + 0.1 * size_sqft + 5 * rooms - 0.5 * age + rng.normal(0, 15, n)
X = pd.DataFrame({"size_sqft": size_sqft, "rooms": rooms, "age": age})
Xc = sm.add_constant(X)
vif = {col: round(variance_inflation_factor(Xc.values, i), 1) for i, col in enumerate(Xc.columns) if col != "const"}
print("VIF:", vif)
fit = sm.OLS(price, Xc).fit()
print(fit.params.round(3).to_dict())
print("standard errors:", fit.bse.round(3).to_dict())
Xs = (X - X.mean()) / X.std()
ridge = Ridge(alpha=10).fit(Xs, price)
print("ridge coefficients (standardized):", dict(zip(X.columns, ridge.coef_.round(2))))
Size and rooms have very high VIFs, and their OLS coefficients have large standard errors (refitting on another sample could change them a lot), while age is unaffected. Ridge spreads the shared effect more stably between the correlated predictors.
Common Pitfalls
Dropping a variable only because its p-value is large when it is collinear with another.
Including dummy variables for every category plus an intercept (perfect collinearity, the “dummy variable trap”).
36
What is the difference between correlation and causation?
KnownRevise
Correlation means two variables move together: when one is higher, the other tends to be higher (or lower). Causation means changing one variable produces a change in the other. Correlation can arise without causation through confounding (a third variable drives both, as hot weather drives both ice cream sales and drownings), reverse causation (the effect actually causes the “cause”), selection bias, or pure chance, especially when many variables are compared. Establishing causation requires randomized experiments or careful causal designs such as instrumental variables, difference-in-differences or regression discontinuity.
Adjusting for confounders (regression, matching) when they are measured, guided by a causal diagram (DAG).
Code Example: A Confounder Creates Correlation
import numpy as np
import statsmodels.api as sm
rng = np.random.default_rng(42)
n = 2000
temperature = rng.normal(25, 5, n)
ice_cream = 20 + 4 * temperature + rng.normal(0, 10, n)
drownings = 1 + 0.3 * temperature + rng.normal(0, 2, n) # no direct link to ice cream
print("correlation(ice cream, drownings):", round(np.corrcoef(ice_cream, drownings)[0, 1], 3))
naive = sm.OLS(drownings, sm.add_constant(ice_cream)).fit()
adjusted = sm.OLS(drownings, sm.add_constant(np.c_[ice_cream, temperature])).fit()
print("ice cream coefficient, naive: ", round(naive.params[1], 4), "p =", f"{naive.pvalues[1]:.2g}")
print("ice cream coefficient, adjusted:", round(adjusted.params[1], 4), "p =", f"{adjusted.pvalues[1]:.2g}")
The naive regression finds a strongly “significant” link; once temperature is included, the ice cream coefficient falls to about zero. Adjustment only works for confounders you measure, which is why randomization is the gold standard.
Common Pitfalls
Adjusting for variables caused by the treatment (mediators or colliders), which can create bias.
Claiming causation from a high R² or a small p-value.
37
How do you detect and fix heteroscedasticity in regression?
KnownRevise
Heteroscedasticity means the variance of regression residuals is not constant: for example, errors grow with income or firm size. Ordinary least squares coefficients stay unbiased, but their standard errors are wrong, so p-values and confidence intervals are unreliable, and OLS is no longer the most efficient estimator. Detect it with a residuals vs fitted plot (a funnel shape) and formal tests (Breusch-Pagan, White). Remedies: heteroscedasticity-robust standard errors (HC3), transforming the outcome (log), weighted least squares when the variance structure is known, or modelling the variance directly.
Breusch-Pagan (het_breuschpagan), White (het_white)
Quick fix for inference
Robust (HC3) standard errors
Transform
Log or Box-Cox of y when variance grows with the mean
Weighting
WLS with weights ∝ 1/variance
Different model
GLM (Poisson, Gamma) whose variance depends on the mean
Code Example
import numpy as np
import statsmodels.api as sm
from statsmodels.stats.diagnostic import het_breuschpagan
rng = np.random.default_rng(42)
n = 500
x = rng.uniform(1, 10, n)
y = 2 + 3 * x + rng.normal(0, 0.5 * x) # error SD grows with x
X = sm.add_constant(x)
ols = sm.OLS(y, X).fit()
lm_stat, lm_p, _, _ = het_breuschpagan(ols.resid, X)
print(f"Breusch-Pagan p-value: {lm_p:.2e}") # small → heteroscedasticity
robust = sm.OLS(y, X).fit(cov_type="HC3")
print("slope SE: classic", round(ols.bse[1], 4), "| robust HC3", round(robust.bse[1], 4))
wls = sm.WLS(y, X, weights=1 / x ** 2).fit() # variance ∝ x², so weight by 1/x²
print("WLS slope:", round(wls.params[1], 3), "SE", round(wls.bse[1], 4))
print("WLS residual Breusch-Pagan p:",
round(het_breuschpagan(wls.resid * np.sqrt(1 / x ** 2), X)[1], 3))
The Breusch-Pagan test’s null is constant variance, so a tiny p-value confirms the problem. Robust standard errors fix inference without changing coefficients; WLS gives more efficient estimates when the weights are right.
Common Pitfalls
Trusting classic OLS p-values without checking residual plots.
Guessing WLS weights badly; robust standard errors are the safer default.
Confusing heteroscedasticity with non-linearity (a curved residual pattern needs a different model).
38
What is logistic regression, and how does it differ from linear regression?
KnownRevise
Logistic regression models the probability of a binary outcome (yes/no, churn/stay) by passing a linear combination of predictors through the sigmoid function, so predictions lie between 0 and 1; equivalently, it models the log-odds as linear in the predictors. It is fitted by maximum likelihood, and each coefficient is a change in log-odds, so exp(coefficient) is an odds ratio. Linear regression models a continuous outcome directly as a linear function, is fitted by least squares, and its coefficients are changes in the expected value; using it for binary outcomes can produce probabilities below 0 or above 1.
import numpy as np
import statsmodels.api as sm
rng = np.random.default_rng(42)
n = 1000
hours = rng.uniform(0, 10, n)
p_pass = 1 / (1 + np.exp(-(-4 + 0.8 * hours)))
passed = rng.binomial(1, p_pass)
X = sm.add_constant(hours)
logit = sm.Logit(passed, X).fit(disp=False)
print(logit.params.round(3))
print("odds ratio per extra hour:", round(np.exp(logit.params[1]), 3))
print("predicted P(pass) at 2, 5, 8 hours:", logit.predict(sm.add_constant(np.array([2.0, 5.0, 8.0]))).round(3))
# Linear probability model for comparison
lpm = sm.OLS(passed, X).fit()
print("linear model predictions at 0 and 12 hours:", lpm.predict(sm.add_constant(np.array([0.0, 12.0]))).round(3))
An odds ratio above 1 means each extra study hour multiplies the odds of passing by that factor. The linear model’s predictions can fall outside [0, 1] when extrapolated, which logistic regression avoids.
Common Pitfalls
Interpreting logistic coefficients as changes in probability (the effect on probability depends on the starting point).
Ignoring class imbalance and calibration when using predicted probabilities.
Perfect separation, which makes coefficients diverge; use regularization.
39
What is a time series, and how does it differ from other data?
KnownRevise
A time series is a sequence of observations of the same variable recorded in time order, usually at regular intervals: daily stock prices, hourly temperature, monthly sales. It differs from ordinary (cross-sectional) data because the order matters and observations are not independent: today’s value is correlated with yesterday’s (autocorrelation), and the series can contain trend, seasonality, cycles and changing variance. This changes how you model it (ARIMA, exponential smoothing, state-space and sequence models) and how you validate it: always on later periods, never with random splits.
import numpy as np
import pandas as pd
from statsmodels.tsa.seasonal import seasonal_decompose
rng = np.random.default_rng(42)
idx = pd.date_range("2020-01-01", periods=48, freq="MS")
t = np.arange(48)
sales = pd.Series(200 + 3 * t + 25 * np.sin(2 * np.pi * t / 12) + rng.normal(0, 8, 48), index=idx)
parts = seasonal_decompose(sales, model="additive", period=12)
print("trend (last 3):", parts.trend.dropna().tail(3).round(1).tolist())
print("seasonal effect by month:", parts.seasonal.iloc[:12].round(1).tolist())
print("lag-1 autocorrelation:", round(sales.autocorr(lag=1), 3))
print("lag-12 autocorrelation:", round(sales.autocorr(lag=12), 3))
shuffled = pd.Series(rng.permutation(sales.to_numpy()))
print("lag-1 autocorrelation after shuffling:", round(shuffled.autocorr(lag=1), 3))
Shuffling destroys the autocorrelation: the same values in a different order are a different time series, which is exactly why order matters.
Common Pitfalls
Random train/test splits, which leak the future into training.
Ignoring autocorrelation, which makes standard errors too small.
Mixing up seasonality (fixed period) with cycles (variable length).
40
What are autocorrelation and partial autocorrelation in time series?
KnownRevise
Autocorrelation (ACF) at lag k is the correlation between a series and itself shifted by k periods; it includes both direct effects and indirect effects carried through the intermediate lags. Partial autocorrelation (PACF) at lag k is the correlation between yₜ and yₜ₋ₖ after removing the effects of lags 1 to k − 1, so it shows only the direct relationship. Together they identify ARIMA orders: an AR(p) process has a PACF that cuts off after lag p and an ACF that decays; an MA(q) process has an ACF that cuts off after lag q and a PACF that decays.
import numpy as np
from statsmodels.tsa.stattools import acf, pacf
rng = np.random.default_rng(42)
n = 1000
e = rng.normal(size=n)
ar2 = np.zeros(n)
for t in range(2, n):
ar2[t] = 0.5 * ar2[t - 1] + 0.3 * ar2[t - 2] + e[t]
ma1 = e + 0.8 * np.r_[0, e[:-1]]
bound = 1.96 / np.sqrt(n)
for name, series in (("AR(2)", ar2), ("MA(1)", ma1)):
a = acf(series, nlags=5)[1:]
p = pacf(series, nlags=5)[1:]
print(name)
print(" ACF lags 1-5:", a.round(2))
print(" PACF lags 1-5:", p.round(2), f"(significance bound ±{bound:.3f})")
For AR(2), the PACF has two large values and then falls inside the bounds; for MA(1), the ACF has one large value at lag 1 and then near zero. plot_acf and plot_pacf in statsmodels.graphics.tsaplots draw the same with shaded bounds.
Common Pitfalls
Reading ACF/PACF of a non-stationary series; difference first.
Treating one lag barely outside the bounds as meaningful (about 1 in 20 will be by chance).
Relying only on plots; confirm orders with AIC and residual checks.
41
What is stationarity in a time series, and why does it matter?
KnownRevise
A time series is (weakly) stationary when its mean and variance are constant over time and the covariance between two values depends only on the gap (lag) between them, not on when they occur. Stationarity matters because most classical time series models (AR, MA, ARMA) and their statistical results assume it: with a stable process, past patterns are informative about the future, parameter estimates are consistent, and relationships between series are not spurious. Trends, seasonality and changing variance break stationarity and must be removed or modelled.
ARMA models and their forecasts assume stationarity.
Spurious regression: regressing one random walk on another often gives a “significant” relationship that does not exist.
Sample statistics (mean, autocorrelation) only estimate something meaningful when they are stable.
Code Example: Testing Stationarity
import numpy as np
from statsmodels.tsa.stattools import adfuller, kpss
rng = np.random.default_rng(42)
n = 500
noise = rng.normal(size=n)
ar1 = np.zeros(n)
for t in range(1, n):
ar1[t] = 0.7 * ar1[t - 1] + noise[t]
random_walk = np.cumsum(noise)
for name, series in (("AR(1)", ar1), ("random walk", random_walk), ("differenced walk", np.diff(random_walk))):
adf_p = adfuller(series)[1]
kpss_p = kpss(series, regression="c", nlags="auto")[1]
print(f"{name:>16}: ADF p = {adf_p:.4f} (small → stationary), KPSS p = {kpss_p:.3f} (small → non-stationary)")
# Spurious regression between two independent random walks
other_walk = np.cumsum(rng.normal(size=n))
print("correlation of two independent random walks:", round(np.corrcoef(random_walk, other_walk)[0, 1], 3))
The ADF test’s null is a unit root; KPSS’s null is stationarity. Using both gives a more reliable verdict. Independent random walks can show large correlations by chance, which is the spurious regression problem.
Common Pitfalls
Confusing “stationary” with “constant”; stationary series still fluctuate.
Relying on a single test with low power on short series.
42
What methods can make a non-stationary time series stationary?
KnownRevise
Non-stationary series are made stationary by removing whatever changes over time: differencing removes stochastic trends (unit roots), seasonal differencing removes repeating seasonal patterns, log or Box-Cox transforms stabilize variance that grows with the level, and detrending by regression removes deterministic trends. Decomposition (STL) can separate trend and seasonality explicitly. After each step, check with ADF/KPSS tests and ACF plots, and keep track of the transformations so forecasts can be converted back to the original scale.
Here the trend is deterministic, so detrending (or seasonal differencing) is enough; for a random-walk trend, ordinary differencing is the right tool.
Common Pitfalls
Differencing a series with a deterministic trend more than needed.
Applying log or Box-Cox to data with zeros or negative values.
Forgetting to reverse transforms (and adjust for the log bias) when reporting forecasts.
43
What is ARIMA, and how is it used to forecast time series?
KnownRevise
ARIMA (AutoRegressive Integrated Moving Average) models a univariate time series with three parts: AR(p), which regresses the value on its own p past values; I(d), which differences the series d times to make it stationary; and MA(q), which models the value as a combination of the q most recent forecast errors. Forecasting follows the Box-Jenkins cycle: make the series stationary, identify p and q from ACF/PACF plots or information criteria (AIC), fit, check that residuals look like white noise, then forecast with prediction intervals. SARIMA adds seasonal terms (P, D, Q)ₘ.
import numpy as np
import pandas as pd
from statsmodels.stats.diagnostic import acorr_ljungbox
from statsmodels.tsa.arima.model import ARIMA
rng = np.random.default_rng(42)
n = 200
e = rng.normal(0, 1, n)
x = np.zeros(n)
for t in range(1, n):
x[t] = 0.6 * x[t - 1] + e[t] + 0.3 * e[t - 1] # ARMA(1,1) increments
y = pd.Series(50 + np.cumsum(x), index=pd.date_range("2024-01-01", periods=n, freq="D"))
train, test = y[:-20], y[-20:]
candidates = {}
for p in range(3):
for q in range(3):
candidates[(p, 1, q)] = ARIMA(train, order=(p, 1, q)).fit().aic
best = min(candidates, key=candidates.get)
print("best order by AIC:", best)
fit = ARIMA(train, order=best).fit()
print("Ljung-Box p-value on residuals:", round(acorr_ljungbox(fit.resid, lags=[10])["lb_pvalue"].iloc[0], 3))
fc = fit.get_forecast(steps=20)
interval = fc.conf_int(alpha=0.05)
mae = np.abs(fc.predicted_mean - test).mean()
naive_mae = np.abs(train.iloc[-1] - test).mean()
print(f"forecast MAE {mae:.3f} vs naive (last value) MAE {naive_mae:.3f}")
print("share of test points inside 95% interval:",
((test >= interval.iloc[:, 0]) & (test <= interval.iloc[:, 1])).mean())
A large Ljung-Box p-value means no remaining autocorrelation in residuals, so the model captured the structure. Always compare with a naive baseline.
Common Pitfalls
Fitting ARIMA to a series with strong seasonality without seasonal terms.
Over-differencing (d too high).
Expecting ARIMA to capture effects of external drivers; use ARIMAX/SARIMAX with exogenous variables.
Long-horizon forecasts: intervals widen quickly, and point forecasts revert to the trend or mean.
44
What is the purpose of dimensionality reduction in data analysis?
KnownRevise
Dimensionality reduction represents data with fewer variables while keeping the information that matters. Its purposes are to fight the curse of dimensionality (sparse data, unreliable distances), reduce overfitting and model complexity, remove redundancy from correlated features, speed up training and storage, reduce noise, and visualize high-dimensional data in 2D or 3D. There are two families: feature selection keeps a subset of original variables, and feature extraction (PCA, autoencoders, t-SNE, UMAP) creates new combined variables.
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_digits(return_X_y=True) # 64 pixel features
print("original features:", X.shape[1])
pipelines = {
"all 64 features": make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000)),
"PCA 20 components": make_pipeline(StandardScaler(), PCA(n_components=20, random_state=42),
LogisticRegression(max_iter=2000)),
"best 20 pixels": make_pipeline(StandardScaler(), SelectKBest(f_classif, k=20),
LogisticRegression(max_iter=2000)),
}
for name, pipe in pipelines.items():
print(f"{name:>18}: CV accuracy {cross_val_score(pipe, X, y, cv=5).mean():.3f}")
pca = PCA(random_state=42).fit(StandardScaler().fit_transform(X))
print("components for 90% variance:", int(np.searchsorted(np.cumsum(pca.explained_variance_ratio_), 0.9) + 1))
Comparing accuracy with fewer inputs shows how much information the reduced representation keeps. The reducers sit inside pipelines so they are fitted only on training folds.
Common Pitfalls
Fitting the reduction on all data before splitting (leakage).
Using t-SNE/UMAP output as features for a model, or reading distances between t-SNE clusters literally.
Losing interpretability when stakeholders need to know which original variables matter.
45
What is Principal Component Analysis, and what are its applications?
KnownRevise
Principal Component Analysis (PCA) is a linear dimensionality-reduction method that finds new orthogonal axes, the principal components, ordered by how much of the data’s variance they capture. Each component is a weighted combination of the original (usually standardized) features; projecting onto the first few keeps most of the information in fewer dimensions. PCA is used for visualization, noise reduction, compression, removing multicollinearity before regression, and speeding up downstream models.
Standardize features (otherwise large-scale features dominate).
Compute the covariance matrix, or the SVD of the centered data.
Eigenvectors are the components; eigenvalues are the variance each explains.
Keep k components (e.g. 90-95% cumulative explained variance, or a scree-plot elbow).
Project the data onto them.
Applications
Application
Example
Visualization
Plot high-dimensional data in 2D
Compression
Images, gene expression, sensor arrays
Noise reduction
Drop low-variance components
Multicollinearity
Principal component regression
Feature engineering
Inputs for clustering or classifiers
Finance
Yield-curve factors (level, slope, curvature)
Code Example
import numpy as np
from sklearn.datasets import load_wine
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_wine(return_X_y=True) # 13 features
pca = make_pipeline(StandardScaler(), PCA(random_state=42)).fit(X)
ratios = pca[-1].explained_variance_ratio_
print("explained variance ratios:", ratios.round(3)[:5])
print("components for 90%:", int(np.searchsorted(np.cumsum(ratios), 0.90) + 1))
loadings = pca[-1].components_[0]
print("largest |loadings| on PC1 (feature index):", np.argsort(-np.abs(loadings))[:3])
for k in (2, 5, 13):
model = make_pipeline(StandardScaler(), PCA(n_components=k, random_state=42),
LogisticRegression(max_iter=1000))
print(f"{k:>2} components: CV accuracy {cross_val_score(model, X, y, cv=5).mean():.3f}")
Comparing accuracy at different k shows how much predictive information a few components retain. PCA sits inside the pipeline, so it is refit on each training fold.
Limitations
Linear only; t-SNE, UMAP or kernel PCA handle non-linear structure.
Components maximize variance, which may not be the variance that matters for the target.
Components can be hard to interpret.
46
How does factor analysis differ from PCA?
KnownRevise
PCA and factor analysis both reduce many correlated variables to fewer dimensions, but they have different goals and models.PCA is a descriptive transformation: components are linear combinations of the observed variables chosen to explain as much total variance as possible, with no error model. Factor analysis (FA) is a statistical model: each observed variable is a combination of a few unobserved latent factors plus a variable-specific error, and it explains only the shared (common) variance, the correlations between variables. FA suits measuring underlying constructs (e.g. personality traits from questionnaire items); PCA suits compression and preprocessing.
Rotations (varimax, promax) give equivalent solutions
Typical fields
ML preprocessing, imaging
Psychology, social science, survey design
Code Example
import numpy as np
from sklearn.decomposition import PCA, FactorAnalysis
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(42)
n = 1000
verbal, math = rng.normal(size=n), rng.normal(size=n) # two latent abilities
noise = lambda sd: rng.normal(0, sd, n)
X = np.c_[
verbal + noise(0.5), verbal + noise(0.6), verbal + noise(0.7), # reading tests
math + noise(0.5), math + noise(0.6), math + noise(0.7), # math tests
]
X = StandardScaler().fit_transform(X)
fa = FactorAnalysis(n_components=2, rotation="varimax", random_state=42).fit(X)
print("factor loadings (rows = factors):\n", fa.components_.round(2))
print("unique variances:", fa.noise_variance_.round(2))
pca = PCA(n_components=2, random_state=42).fit(X)
print("PCA components:\n", pca.components_.round(2))
print("PCA explained variance ratio:", pca.explained_variance_ratio_.round(3))
After varimax rotation, each factor loads mainly on one group of tests, recovering the “verbal” and “math” abilities, and FA estimates each test’s unique noise variance, which PCA does not model.
Common Pitfalls
Treating PCA components as latent constructs.
Running FA on too few observations or weakly correlated variables (check the Kaiser-Meyer-Olkin measure and Bartlett’s test).
Choosing the number of factors only by eigenvalues > 1; use parallel analysis or theory.
47
What is the curse of dimensionality?
KnownRevise
The curse of dimensionality describes problems that appear as the number of features grows: the volume of the space grows exponentially, so data become sparse; the amount of data needed to cover the space grows exponentially; and distances between points become nearly equal, so nearest-neighbour and distance-based methods lose meaning. Models also have more parameters to fit, raising the risk of overfitting. Remedies include feature selection, dimensionality reduction, regularization and models that exploit structure in the data.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Effects
Effect
Consequence
Sparsity
Few neighbours in any region; density estimation fails
Distance concentration
Nearest and farthest neighbours almost equally far
Exponential data needs
Covering a grid with 10 bins per axis needs 10ᵈ cells
Overfitting
Many features relative to samples fit noise
Volume at the edges
Most of a hypercube’s volume lies near its surface
Code Example: Distances Concentrate
import numpy as np
from scipy.spatial.distance import cdist
rng = np.random.default_rng(42)
for d in (2, 10, 100, 1000):
X = rng.random((500, d))
query = rng.random((1, d))
dist = cdist(query, X)[0]
contrast = (dist.max() - dist.min()) / dist.min()
print(f"d={d:>4}: (max - min) / min distance = {contrast:.3f}")
# Fraction of a hypercube within 10% of its boundary
for d in (2, 10, 50):
print(f"d={d:>2}: share of volume near the surface = {1 - 0.8 ** d:.3f}")
As d grows, the relative contrast between the nearest and farthest points shrinks toward zero, which is why k-NN and clustering by Euclidean distance degrade on raw high-dimensional data.
More data, and models with strong inductive biases (CNNs for images).
Regularization and cross-validation to control overfitting.
Common Pitfalls
Adding features “just in case” with a small sample.
Selecting features on the full dataset before cross-validation (leakage).
48
What is SVD, and how is it used in machine learning?
KnownRevise
Singular Value Decomposition (SVD) factorizes any m × n matrix as X = U Σ Vᵀ, where U and V have orthonormal columns (left and right singular vectors) and Σ is diagonal with non-negative singular values in decreasing order. Keeping only the top k singular values gives the best rank-k approximation of X in the least-squares sense (Eckart-Young theorem). In machine learning SVD underlies PCA, latent semantic analysis for text, matrix-factorization recommenders, compression and denoising, and numerically stable least squares and pseudo-inverses.
SVD of the centered data; right singular vectors are principal axes
Latent semantic analysis
Truncated SVD of a TF-IDF term-document matrix finds topics
Recommenders
Low-rank factorization of the user-item matrix
Compression / denoising
Keep the top k components of an image or signal
Least squares
Pseudo-inverse V Σ⁺ Uᵀ handles rank-deficient problems
Code Example: Low-Rank Approximation and Truncated SVD
import numpy as np
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer
rng = np.random.default_rng(42)
# A rank-3 matrix plus noise
X = rng.normal(size=(100, 3)) @ rng.normal(size=(3, 40)) + 0.1 * rng.normal(size=(100, 40))
U, S, Vt = np.linalg.svd(X, full_matrices=False)
print("top singular values:", S[:6].round(2)) # three large, then small noise values
for k in (1, 3, 10):
X_k = U[:, :k] @ np.diag(S[:k]) @ Vt[:k]
err = np.linalg.norm(X - X_k) / np.linalg.norm(X)
print(f"rank {k}: relative reconstruction error {err:.4f}")
docs = ["cats purr and meow", "dogs bark and wag", "kittens meow softly",
"puppies bark loudly", "stocks rose today", "markets fell on rates"]
tfidf = TfidfVectorizer().fit_transform(docs)
lsa = TruncatedSVD(n_components=2, random_state=42)
print("document coordinates in 2 latent dimensions:\n", lsa.fit_transform(tfidf).round(2))
The sharp drop after the third singular value reveals the true rank, and rank 3 already reconstructs the matrix almost perfectly. TruncatedSVD works directly on sparse matrices, unlike PCA.
Common Pitfalls
Forgetting to center data when you want PCA rather than plain SVD.
Computing a full SVD of a huge matrix; use truncated or randomized SVD.
Using classic SVD on a ratings matrix with missing values; recommenders use factorization methods that fit only observed entries.
49
What is A/B testing, and why is it important in statistics?
KnownRevise
A/B testing is a randomized controlled experiment that compares two versions of something, A (control) and B (treatment), by randomly assigning users to each and measuring a predefined metric such as conversion rate. Statistical hypothesis testing then decides whether the observed difference is larger than random variation would explain. It is important because randomization makes it one of the few reliable ways to establish causation in business settings, replacing opinions and misleading before/after comparisons with evidence.
Probability of a difference at least this extreme if the null is true
Significance level (α)
Accepted false-positive rate, e.g. 0.05
Power
Probability of detecting a true effect of a given size
Confidence interval
Plausible range for the true difference
Practical significance
Whether the effect is large enough to matter
Why It Matters
Measures causal impact of product, pricing and marketing changes.
Quantifies uncertainty instead of relying on intuition.
Protects against regressions through guardrail metrics.
Builds a culture of incremental, evidence-based decisions.
Code Example: Simulated Test and False-Positive Check
import numpy as np
from statsmodels.stats.proportion import proportions_ztest
rng = np.random.default_rng(42)
n = 10_000
conv_a = rng.binomial(n, 0.10)
conv_b = rng.binomial(n, 0.115)
z, p = proportions_ztest([conv_b, conv_a], [n, n])
print(f"A {conv_a / n:.3f}, B {conv_b / n:.3f}, z = {z:.2f}, p = {p:.4f}")
# A/A tests: when there is no real difference, about 5% are "significant" at alpha 0.05
false_pos = 0
for _ in range(1000):
a, b = rng.binomial(n, 0.10), rng.binomial(n, 0.10)
false_pos += proportions_ztest([a, b], [n, n])[1] < 0.05
print("false-positive rate in A/A tests:", false_pos / 1000)
The A/A simulation shows what α means in practice: even with no difference, about 1 in 20 tests looks significant, which is why peeking or testing many metrics inflates false discoveries.
Limitations
Needs enough traffic for useful power.
Measures short-term effects; long-term effects may differ.
Network effects (marketplaces, social apps) violate independence between groups.
50
How do you design an A/B test and determine its sample size?
KnownRevise
Designing an A/B test means choosing a hypothesis, a single primary metric, the randomization unit, guardrail metrics, and the minimum detectable effect (MDE), the smallest change worth detecting. Sample size then follows from four inputs: the baseline rate or variance, the MDE, the significance level α (false-positive rate, usually 0.05) and the power 1 − β (probability of detecting a true effect of the MDE, usually 0.8). Divide the required users by daily traffic to get the duration, and round up to whole weeks.
n per group ≈ (z₁₋α/₂ + z₁₋β)² × [p₁(1 − p₁) + p₂(1 − p₂)] / (p₂ − p₁)²
Halving the MDE roughly quadruples the sample size.
Code Example
import numpy as np
from scipy import stats
from statsmodels.stats.power import NormalIndPower, TTestIndPower
from statsmodels.stats.proportion import proportion_effectsize
p1, p2, alpha, power = 0.040, 0.044, 0.05, 0.80
# Formula
z_a, z_b = stats.norm.ppf(1 - alpha / 2), stats.norm.ppf(power)
n_formula = (z_a + z_b) ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) / (p2 - p1) ** 2
print("per group (formula):", int(np.ceil(n_formula)))
# statsmodels (Cohen's h effect size)
h = proportion_effectsize(p2, p1)
n_sm = NormalIndPower().solve_power(effect_size=h, alpha=alpha, power=power)
print("per group (statsmodels):", int(np.ceil(n_sm)))
daily_users = 8_000
days = 2 * n_sm / daily_users
print("days needed:", int(np.ceil(days)), "→ run", int(np.ceil(days / 7)) * 7, "days")
# Continuous metric: detect a 0.5 change in mean with SD 8
n_cont = TTestIndPower().solve_power(effect_size=0.5 / 8, alpha=alpha, power=power)
print("per group (revenue metric):", int(np.ceil(n_cont)))
for mde in (0.002, 0.004, 0.008):
n = NormalIndPower().solve_power(proportion_effectsize(p1 + mde, p1), alpha=alpha, power=power)
print(f"MDE +{mde:.3f}: {int(np.ceil(n)):,} per group")
The loop shows the trade-off: smaller effects need far larger samples, so the MDE must be chosen from business value, not hope.
Common Pitfalls
Choosing an MDE so small the test would run for months.
Stopping early when results look good.
Forgetting variance reduction (e.g. CUPED with pre-experiment data), which can cut the required sample.
51
What are control and treatment groups in an experiment?
KnownRevise
The treatment group receives the intervention being tested (a new drug, website design or teaching method), while the control group does not; it gets a placebo, the current standard, or nothing. Because subjects are randomly assigned to the groups, the two groups are similar on average in every other respect, measured and unmeasured, so any difference in outcomes can be attributed to the treatment. The control group provides the counterfactual: what would have happened without the intervention.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Types of Control
Control type
Example
Purpose
Placebo
Sugar pill
Separates psychological from real effects
Active / standard of care
Existing drug or current website
Measures improvement over what is used now
No treatment
Nothing
Measures absolute effect
Holdout
Users who see no ads
Measures incremental impact of marketing
Why Randomization Matters
Without random assignment, groups can differ systematically: motivated customers opt into a loyalty program and would have spent more anyway. Randomization balances such confounders, and blinding prevents expectations from changing behaviour or measurement.
Code Example: Randomization Balances a Hidden Confounder
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
n = 2000
motivation = rng.normal(0, 1, n) # unobserved confounder
# Self-selection: motivated people choose the program
self_selected = rng.random(n) < 1 / (1 + np.exp(-2 * motivation))
# Randomized: a coin flip decides
randomized = rng.random(n) < 0.5
true_effect = 5.0
def outcome(treated):
return 50 + 10 * motivation + true_effect * treated + rng.normal(0, 5, n)
for name, treated in (("self-selected", self_selected), ("randomized", randomized)):
y = outcome(treated)
diff = y[treated].mean() - y[~treated].mean()
balance = motivation[treated].mean() - motivation[~treated].mean()
p = stats.ttest_ind(y[treated], y[~treated], equal_var=False).pvalue
print(f"{name:>13}: estimated effect {diff:5.2f} (true {true_effect}), "
f"motivation gap {balance:.2f}, p = {p:.2g}")
The self-selected comparison greatly overstates the effect because motivated people are over-represented in treatment; the randomized comparison recovers roughly the true effect.
Common Pitfalls
Letting people or staff choose who gets the treatment.
Comparing to a historical control period affected by other changes.
Contamination: control subjects getting the treatment (e.g. sharing a promo code).
52
How do you use hypothesis testing to analyze A/B test results?
KnownRevise
Analysing an A/B test with hypothesis testing means stating the null hypothesis (no difference between variants in the primary metric) and alternative, choosing the significance level (usually 0.05) and test before the experiment, then computing the test statistic and p-value, the probability of a difference at least as extreme as observed if the null is true. For conversion rates use a two-proportion z-test or chi-square test; for continuous metrics like revenue use Welch’s t-test (or bootstrap for heavy tails). Report the effect size with a confidence interval and judge practical significance as well as statistical significance.
Two-proportion z-test (conversion), Welch t-test (revenue per user)
Effect size
Absolute and relative lift with 95% CI
Decision
Significant and above the minimum worthwhile effect → ship
Multiple metrics
Benjamini-Hochberg for secondary metrics
Code Example
import numpy as np
from scipy import stats
from statsmodels.stats.multitest import multipletests
from statsmodels.stats.proportion import confint_proportions_2indep, proportions_ztest
rng = np.random.default_rng(42)
n_a = n_b = 20_000
conv_a = rng.binomial(n_a, 0.050)
conv_b = rng.binomial(n_b, 0.056)
z, p = proportions_ztest([conv_b, conv_a], [n_b, n_a])
low, high = confint_proportions_2indep(conv_b, n_b, conv_a, n_a)
print(f"conversion A {conv_a / n_a:.4f}, B {conv_b / n_b:.4f}; z = {z:.2f}, p = {p:.4f}")
print(f"95% CI for B − A: [{low:.4f}, {high:.4f}]")
# Revenue per user: skewed, many zeros → Welch t-test (large n)
rev_a = rng.binomial(1, 0.05, n_a) * rng.lognormal(3.5, 1.0, n_a)
rev_b = rng.binomial(1, 0.056, n_b) * rng.lognormal(3.5, 1.0, n_b)
t = stats.ttest_ind(rev_b, rev_a, equal_var=False)
print(f"revenue per user A {rev_a.mean():.3f}, B {rev_b.mean():.3f}; p = {t.pvalue:.4f}")
# Secondary metrics: control the false discovery rate
secondary_p = [0.012, 0.21, 0.049, 0.003]
reject, adjusted, _, _ = multipletests(secondary_p, alpha=0.05, method="fdr_bh")
print("BH-adjusted p-values:", adjusted.round(3), "| significant:", reject)
If p is below 0.05 and the interval lies above the minimum effect worth shipping, the result is both statistically and practically significant. “Not significant” means the data do not clearly show a difference, not that there is none.
Common Pitfalls
Peeking and stopping as soon as p < 0.05.
Analysing per-session data when randomization was per user (non-independent observations).
Celebrating statistically significant but tiny lifts.
53
How can you avoid bias in experiments and A/B tests?
KnownRevise
Avoiding bias in A/B tests starts with proper randomization at the right unit (user, session, store), a pre-registered plan (primary metric, sample size, duration, analysis method) and checks that the randomization worked, such as a sample ratio mismatch (SRM) test. During the test, avoid peeking and stopping early, run full weekly cycles to cover day-of-week effects, and keep treatment and control isolated from each other. In analysis, use intention-to-treat, correct for multiple comparisons, and be careful with novelty effects and interference between users.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Common Biases and Fixes
Bias
Cause
Fix
Selection bias
Users choose or are chosen for treatment
Random assignment by a stable hash of the user ID
Sample ratio mismatch
Bugs, bots, redirects drop users unevenly
Chi-square SRM check; investigate before analysing
Peeking / optional stopping
Stopping when p < 0.05
Fixed horizon, or sequential tests with alpha spending
Novelty and primacy effects
Users react to change itself
Run longer; analyse new users separately
Day-of-week / seasonality
Short tests
Run whole weeks
Interference (network effects)
Treated users affect control users
Cluster or geo randomization
Multiple testing
Many metrics and segments
Pre-specify primary metric; Bonferroni or Benjamini-Hochberg
Survivorship
Analysing only users who completed a step
Analyse everyone assigned
Code Example: Sample Ratio Mismatch Check
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
# Planned 50/50 split; a redirect bug dropped some treatment users
control_users = 50_400
treatment_users = 49_100
observed = [control_users, treatment_users]
expected = [sum(observed) / 2] * 2
chi2, p = stats.chisquare(observed, expected)
print(f"chi2 = {chi2:.2f}, p = {p:.2e}")
print("SRM detected - do not trust the results" if p < 0.001 else "split looks fine")
# Deterministic assignment from a hashed id keeps users in one group
def assign(user_id: int, salt: int = 42) -> str:
return "treatment" if hash((salt, user_id)) % 2 else "control"
print([assign(u) for u in range(5)])
A tiny p-value means a split this uneven would be very unlikely under correct 50/50 randomization, so something in the pipeline is biased. (In production use a stable hash such as MD5 or MurmurHash of the ID, because Python’s hash of strings changes between runs.)
Common Pitfalls
Analysing only users who saw the feature rather than everyone assigned.
Changing the metric after seeing results.
Running overlapping experiments that interact without accounting for it.
54
How does Bayesian statistics differ from frequentist statistics?
KnownRevise
Bayesian statistics treats unknown parameters as random variables with probability distributions that represent uncertainty, combines a prior with the data through Bayes’ theorem, and reports a posterior distribution, so statements like “there is a 95% probability the parameter lies in this interval” are valid. Frequentist statistics treats parameters as fixed but unknown, and defines probability as long-run frequency over repeated samples, so its tools (p-values, confidence intervals) describe how procedures behave across hypothetical repetitions rather than the probability of a hypothesis.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Comparison
Aspect
Frequentist
Bayesian
Probability means
Long-run frequency
Degree of belief / uncertainty
Parameters
Fixed, unknown
Random variables with distributions
Prior information
Not formally used
Explicit prior
Output
Point estimate, CI, p-value
Posterior distribution
Interval
95% CI: the procedure covers the true value 95% of the time
95% credible interval: 95% probability given model and prior
Hypotheses
Reject or fail to reject H₀
P(hypothesis | data), Bayes factors
Computation
Usually closed form, fast
Often MCMC or variational inference
Sequential looks
Need corrections
Posterior is valid at any time (decision rules still need care)
With flat priors and plenty of data the two approaches give similar intervals, but they answer different questions: the p-value is about the data assuming no difference; P(B > A) is about the hypothesis given the data.
Common Pitfalls
Interpreting a p-value as P(H₀ is true).
Interpreting a confidence interval as having 95% probability of containing the parameter.
Believing Bayesian methods are “subjective” and therefore unusable; priors should be stated and tested for sensitivity.
55
What are the prior, likelihood and posterior in Bayesian inference?
KnownRevise
In Bayesian inference, the prior p(θ) expresses what you believe about a parameter before seeing the data; the likelihood p(data | θ) measures how probable the observed data are for each possible parameter value; and the posterior p(θ | data) is the updated belief after seeing the data. Bayes’ theorem links them: posterior = likelihood × prior / evidence, where the evidence p(data) is a normalizing constant. The posterior is a full distribution, so you can report its mean, credible intervals or the probability of any hypothesis about θ.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
The Three Pieces
Term
Notation
Meaning
Example (coin bias θ)
Prior
p(θ)
Belief before data
Beta(2, 2): probably near fair
Likelihood
p(data | θ)
Fit of the data for each θ
7 heads in 10 flips: θ⁷(1 − θ)³
Evidence
p(data)
Average likelihood over the prior
Constant that makes the posterior integrate to 1
Posterior
p(θ | data)
Updated belief
Beta(9, 5)
The likelihood is not a probability distribution over θ; it is a function of θ for fixed data and need not integrate to 1.
The posterior mean (about 0.64) sits between the prior mean (0.5) and the observed rate (0.7): with more data, the likelihood dominates and the prior matters less.
Common Pitfalls
Calling the likelihood “the probability of θ”.
Choosing a prior after seeing the data to get a desired result.
Reporting only the posterior mean and ignoring its spread.
56
Describe a scenario where Bayesian statistics would be advantageous.
KnownRevise
Bayesian statistics is advantageous when data are scarce, prior knowledge is real, or decisions need direct probabilities. A strong scenario is estimating failure or conversion rates across many small groups, such as click-through rates for hundreds of products, many with only a few views. A frequentist estimate for a product with 2 clicks in 3 views says 67%, which is absurd; a hierarchical Bayesian model shrinks each product’s estimate toward the overall rate in proportion to how little data it has, giving sensible rankings and credible intervals, and answers questions like “what is the probability product A beats product B?”.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Other Good Scenarios
Scenario
Bayesian advantage
Rare disease diagnosis
Combines test accuracy with prevalence (prior)
Small clinical trials, rare diseases
Uses historical data as an informative prior
A/B testing
P(B better than A), expected loss, natural sequential monitoring
Sequential decisions (bandits)
Thompson sampling uses posterior draws
Complex hierarchical data (schools, hospitals, regions)
Partial pooling across groups
Uncertainty in forecasts
Full predictive distributions
Code Example: Shrinking Small-Sample Rates
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
n_products = 200
true_ctr = rng.beta(4, 96, n_products) # most products near 4%
views = rng.integers(3, 2000, n_products)
clicks = rng.binomial(views, true_ctr)
raw = clicks / views
# Empirical Bayes: fit a Beta prior to the products, then update each one
mean, var = raw.mean(), raw.var()
common = mean * (1 - mean) / var - 1
a0, b0 = mean * common, (1 - mean) * common
posterior_mean = (a0 + clicks) / (a0 + b0 + views)
print(f"fitted prior: Beta({a0:.1f}, {b0:.1f})")
print("mean abs error, raw rates: ", round(np.abs(raw - true_ctr).mean(), 4))
print("mean abs error, Bayesian rates: ", round(np.abs(posterior_mean - true_ctr).mean(), 4))
# Probability one product beats another
i, j = 0, 1
draws_i = stats.beta(a0 + clicks[i], b0 + views[i] - clicks[i]).rvs(20000, random_state=42)
draws_j = stats.beta(a0 + clicks[j], b0 + views[j] - clicks[j]).rvs(20000, random_state=43)
print(f"P(product {i} has higher CTR than product {j}) = {(draws_i > draws_j).mean():.3f}")
The shrunken estimates should be closer to the true rates on average, especially for products with few views. A full hierarchical model (PyMC, Stan) also accounts for uncertainty in the prior itself.
Common Pitfalls
Choosing a prior to get the desired answer; justify it and run a sensitivity analysis.
Ignoring computational cost and convergence checks for complex models.
57
What is Markov Chain Monte Carlo, and where is it used in statistics?
KnownRevise
Markov Chain Monte Carlo (MCMC) is a family of algorithms that draw samples from a probability distribution, usually a Bayesian posterior, that is too complex to compute directly. It builds a Markov chain whose long-run (stationary) distribution is the target, so after a burn-in period the chain’s states behave like dependent samples from it. Averages over those samples estimate posterior means, credible intervals and predictions. MCMC is used for Bayesian regression and hierarchical models, missing-data problems, phylogenetics, epidemiology and physics.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+ (PyMC 5 or Stan for real models)
Main Algorithms
Algorithm
Idea
Notes
Metropolis-Hastings
Propose a move; accept with a probability based on density ratios
Simple; slow in many dimensions
Gibbs sampling
Sample each parameter from its conditional distribution
Needs tractable conditionals
Hamiltonian Monte Carlo (HMC)
Uses gradients to make long, efficient moves
Default in Stan, PyMC
NUTS
HMC that tunes its path length automatically
The standard choice today
Checking Convergence
Trace plots of several chains should overlap and look like noise.
R-hat close to 1 (e.g. below 1.01) means chains agree.
Effective sample size accounts for autocorrelation between draws.
Divergent transitions (HMC) signal geometry problems.
Code Example: Gibbs Sampler for a Normal Mean and Variance
import numpy as np
rng = np.random.default_rng(42)
data = rng.normal(5.0, 2.0, 100)
n, x_bar = len(data), data.mean()
mu, tau = 0.0, 1.0 # tau = precision = 1 / variance
samples = []
for i in range(6000):
# mu | tau, data (flat prior on mu)
mu = rng.normal(x_bar, 1 / np.sqrt(n * tau))
# tau | mu, data (Gamma(0.001, 0.001) prior on tau)
shape = 0.001 + n / 2
rate = 0.001 + 0.5 * np.sum((data - mu) ** 2)
tau = rng.gamma(shape, 1 / rate)
samples.append((mu, 1 / np.sqrt(tau)))
draws = np.array(samples[1000:]) # discard burn-in
print("posterior mean of mu:", draws[:, 0].mean().round(3), "| 95% interval:", np.percentile(draws[:, 0], [2.5, 97.5]).round(3))
print("posterior mean of sigma:", draws[:, 1].mean().round(3))
The posterior for μ should centre near the sample mean and the σ posterior near 2, the values used to simulate the data.
Common Pitfalls
Reporting results from a single chain without convergence diagnostics.
Thinking draws are independent; use effective sample size.
Poorly scaled or highly correlated parameters slowing mixing (reparameterize, use NUTS).
58
How do you update a Bayesian model with new data?
KnownRevise
Updating a Bayesian model applies Bayes’ theorem: posterior ∝ likelihood × prior. Today’s posterior becomes tomorrow’s prior, so when new data arrive you combine them with the current posterior to get an updated one; for independent data this gives exactly the same result as analysing all data at once. With conjugate priors the update is a simple formula (Beta-Binomial, Normal-Normal, Gamma-Poisson); for other models you rerun MCMC or variational inference with all data, or use sequential Monte Carlo for streaming updates.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Conjugate Updates
Data
Prior
Posterior after data
Successes k in n trials
Beta(α, β)
Beta(α + k, β + n − k)
Counts with total exposure t
Gamma(α, β)
Gamma(α + Σcounts, β + t)
Normal mean, known σ
Normal(μ₀, τ₀²)
Precision-weighted average of prior mean and data mean
Code Example: Conversion Rate Updated Week by Week
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
true_rate = 0.12
alpha, beta = 2, 18 # prior: mean 0.10, fairly weak
for week in range(1, 5):
visitors = 500
conversions = rng.binomial(visitors, true_rate)
alpha, beta = alpha + conversions, beta + visitors - conversions # posterior becomes the new prior
post = stats.beta(alpha, beta)
low, high = post.interval(0.95)
print(f"week {week}: {conversions}/{visitors} → posterior mean {post.mean():.4f}, "
f"95% credible interval [{low:.4f}, {high:.4f}], P(rate > 0.10) = {post.sf(0.10):.3f}")
# Batch update gives the same posterior as sequential updates
print("final posterior: Beta(", alpha, ",", beta, ")")
Each week the credible interval narrows as evidence accumulates, and the prior’s influence fades. P(rate > 0.10) answers a direct business question that a p-value cannot.
When the Data Change Over Time
If the underlying rate drifts, discount old evidence (for example, multiply α and β by a forgetting factor below 1 before each update) or use a dynamic model.
Common Pitfalls
Using the same data twice (once to choose the prior and again in the likelihood).
An overly confident prior that new data can barely move.
Treating sequential updating as different from batch analysis; for exchangeable data they agree.
59
How do you calculate mean, median and mode from a list in Python?
KnownRevise
Mean, median and mode are the three main measures of central tendency. The mean is the sum divided by the count and is sensitive to outliers; the median is the middle value of the sorted data (the average of the two middle values for an even count) and is robust to outliers and skew; the mode is the most frequent value, and there may be several. Python’s built-in statistics module computes all three, and writing them from scratch shows the definitions, including how to handle ties and empty input.
Skewed data (income, house prices, response times)
Ignores the size of extreme values
Mode
Categorical data, most common value
May be several or not meaningful for continuous data
Code Example
from collections import Counter
import statistics
import numpy as np
def mean(values):
if not values:
raise ValueError("mean of empty data")
return sum(values) / len(values)
def median(values):
if not values:
raise ValueError("median of empty data")
s = sorted(values)
mid = len(s) // 2
return s[mid] if len(s) % 2 else (s[mid - 1] + s[mid]) / 2
def modes(values):
if not values:
raise ValueError("mode of empty data")
counts = Counter(values)
top = max(counts.values())
return sorted(v for v, c in counts.items() if c == top) # all values tied for most frequent
data = [3, 7, 7, 2, 9, 4, 3, 7, 3, 120] # 120 is an outlier
print("mean:", mean(data), "| median:", median(data), "| modes:", modes(data))
print("statistics module:", statistics.mean(data), statistics.median(data), statistics.multimode(data))
rng = np.random.default_rng(42)
incomes = rng.lognormal(10.5, 0.8, 10_000) # right-skewed data
print(f"skewed incomes: mean {incomes.mean():,.0f} vs median {np.median(incomes):,.0f}")
The outlier 120 drags the mean well above most values while the median stays at the centre, and the skewed income sample shows the mean above the median, which is why the median is preferred for incomes.
Common Pitfalls
Reporting a single mode when the data are multimodal (statistics.mode returns only the first).
Using the mean for heavily skewed data.
Computing the median without sorting.
60
How do you generate and plot 1,000 normally distributed random points?
KnownRevise
Generating normal data in Python is done with NumPy’s random generator: rng.normal(loc=mean, scale=sd, size=1000). To visualize it, plot a histogram with density scaling and overlay the theoretical normal curve, and add a Q-Q plot, which shows points close to a straight line when the data are normal. Checking the sample mean and standard deviation against the true values, and the share within ±1, 2 and 3 SD (about 68%, 95% and 99.7%), confirms the sample behaves as expected.
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
mu, sigma = 50, 10
data = rng.normal(loc=mu, scale=sigma, size=1000)
print(f"sample mean {data.mean():.2f} (true {mu}), sample SD {data.std(ddof=1):.2f} (true {sigma})")
for k in (1, 2, 3):
share = np.mean(np.abs(data - mu) <= k * sigma)
print(f"within ±{k} SD: {share:.3f}")
print("Shapiro-Wilk p-value:", round(stats.shapiro(data).pvalue, 3))
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
ax1.hist(data, bins=30, density=True, alpha=0.6, edgecolor="white", label="sample")
xs = np.linspace(data.min(), data.max(), 300)
ax1.plot(xs, stats.norm.pdf(xs, mu, sigma), linewidth=2, label="N(50, 10²) density")
ax1.set_xlabel("value")
ax1.set_ylabel("density")
ax1.legend()
ax1.set_title("Histogram with normal curve")
stats.probplot(data, dist="norm", plot=ax2)
ax2.set_title("Normal Q-Q plot")
fig.tight_layout()
plt.show()
The Shapiro-Wilk p-value should usually be large here (the data really are normal); with real data and large samples, it can reject normality for tiny, unimportant deviations, so look at the Q-Q plot too.
Common Pitfalls
Using the legacy np.random.seed / np.random.normal global state instead of a Generator.
Histograms with counts on the y-axis compared with a density curve (use density=True).
Too few or too many bins hiding the shape.
61
How do you implement simple linear regression from scratch in Python?
KnownRevise
Simple linear regression fits a line y = β₀ + β₁x by ordinary least squares, minimizing the sum of squared residuals. The closed-form estimates are β₁ = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)² and β₀ = ȳ − β₁x̄. A complete from-scratch implementation also computes R², the residual standard error, the standard error of the slope, a t-test for “slope = 0” and a confidence interval, and it should match a library such as SciPy’s linregress.
How do you simulate and analyze the Monty Hall problem in Python?
KnownRevise
The Monty Hall problem: a prize is behind one of three doors; you pick a door, the host, who knows where the prize is, opens another door with no prize, and offers you the chance to switch. Switching wins with probability 2/3, staying wins with 1/3, because your first pick is right only one time in three and the host’s choice gives information. Simulating many games confirms this, and a confidence interval or binomial test shows the observed win rates agree with theory.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Why Switching Wins
Your first pick
Probability
Stay
Switch
Prize door
1/3
Win
Lose
Empty door
2/3
Lose
Win (host must reveal the other empty door)
Code Example
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
n_games = 100_000
prize = rng.integers(0, 3, n_games)
choice = rng.integers(0, 3, n_games)
# Host opens a door that is neither the player's choice nor the prize
host = np.empty(n_games, dtype=int)
for i in range(n_games):
options = [d for d in range(3) if d != choice[i] and d != prize[i]]
host[i] = rng.choice(options)
switch_to = 3 - choice - host # the remaining door (doors are 0, 1, 2)
stay_wins = (choice == prize).sum()
switch_wins = (switch_to == prize).sum()
for name, wins, theory in (("stay", stay_wins, 1 / 3), ("switch", switch_wins, 2 / 3)):
rate = wins / n_games
ci = stats.binomtest(int(wins), n_games).proportion_ci(confidence_level=0.95)
p = stats.binomtest(int(wins), n_games, p=theory).pvalue
print(f"{name:>6}: win rate {rate:.4f}, 95% CI [{ci.low:.4f}, {ci.high:.4f}], "
f"test vs {theory:.3f}: p = {p:.3f}")
The intervals should contain 1/3 and 2/3, and the tests against the theoretical values should not reject them. The intuition: switching wins exactly when your first pick was wrong, which happens two times in three.
Common Pitfalls
Simulating a host who opens a random door (which might reveal the prize); that is a different problem with a 1/2 answer when the prize is not revealed.
Treating the two remaining doors as equally likely because “there are two left”.
63
How do you write a Python function to perform a two-sample t-test?
KnownRevise
A two-sample t-test checks whether two independent groups have different population means. A good function computes the group summaries, uses Welch’s t-test by default (it does not assume equal variances and loses little when variances are equal), returns the t statistic, degrees of freedom, p-value, a confidence interval for the mean difference and an effect size (Cohen’s d), and supports a paired option when the same subjects are measured twice. The p-value is the probability of a difference at least this extreme if the true means were equal.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Which t-test?
Situation
Test
SciPy
Two independent groups
Welch’s t-test
ttest_ind(a, b, equal_var=False)
Independent groups, equal variances assumed
Student’s t-test
ttest_ind(a, b)
Same subjects, before/after
Paired t-test
ttest_rel(a, b)
One group vs a known value
One-sample t-test
ttest_1samp(a, mu)
Non-normal, small samples
Mann-Whitney U / Wilcoxon
mannwhitneyu, wilcoxon
Code Example
import numpy as np
from scipy import stats
def t_test(a, b, paired=False, alpha=0.05):
a, b = np.asarray(a, float), np.asarray(b, float)
if paired:
res = stats.ttest_rel(a, b)
diff = a - b
d = diff.mean() / diff.std(ddof=1)
else:
res = stats.ttest_ind(a, b, equal_var=False)
pooled_sd = np.sqrt((a.var(ddof=1) + b.var(ddof=1)) / 2)
d = (a.mean() - b.mean()) / pooled_sd
ci = res.confidence_interval(confidence_level=1 - alpha)
return {"t": res.statistic, "df": res.df, "p": res.pvalue,
"mean_diff": a.mean() - b.mean(), "ci": (ci.low, ci.high), "cohens_d": d,
"reject_null": res.pvalue < alpha}
rng = np.random.default_rng(42)
group_a = rng.normal(72, 10, 40) # e.g. test scores with a new teaching method
group_b = rng.normal(66, 12, 45)
result = t_test(group_a, group_b)
for key, value in result.items():
print(f"{key:>11}: {np.round(value, 4)}")
If reject_null is true, a mean difference this large would be unlikely if the methods had equal means; the confidence interval shows the plausible size of the difference, and Cohen’s d puts it on a standardized scale (about 0.2 small, 0.5 medium, 0.8 large).
Common Pitfalls
Using an independent test on paired data (or the reverse).
Assuming equal variances by default.
Treating p > 0.05 as proof the means are equal.
64
How do you compute and plot a correlation matrix in Python?
KnownRevise
A correlation matrix shows the pairwise correlation coefficient between every pair of numeric variables, from −1 (perfect negative) through 0 (no linear relationship) to +1 (perfect positive). In Python, DataFrame.corr() computes it with Pearson (linear relationships, sensitive to outliers) or Spearman (monotonic relationships, rank-based) correlation, and a heatmap displays it. A good script also tests which correlations are significant, keeping in mind that many pairs means many tests, and that correlation does not imply causation.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats
rng = np.random.default_rng(42)
n = 300
hours = rng.normal(5, 2, n)
df = pd.DataFrame({
"study_hours": hours,
"sleep_hours": rng.normal(7, 1, n),
"score": 50 + 6 * hours + rng.normal(0, 8, n),
"screen_time": 6 - 0.5 * hours + rng.normal(0, 1.5, n),
})
corr = df.corr(method="pearson")
print(corr.round(2))
# p-values for each pair
cols = df.columns
pvals = pd.DataFrame(np.ones((len(cols), len(cols))), index=cols, columns=cols)
for i, a in enumerate(cols):
for b in cols[i + 1:]:
pvals.loc[a, b] = pvals.loc[b, a] = stats.pearsonr(df[a], df[b]).pvalue
print(pvals.round(4))
fig, ax = plt.subplots(figsize=(6, 5))
im = ax.imshow(corr, cmap="RdBu_r", vmin=-1, vmax=1)
ax.set_xticks(range(len(cols)), cols, rotation=45, ha="right")
ax.set_yticks(range(len(cols)), cols)
for i in range(len(cols)):
for j in range(len(cols)):
ax.text(j, i, f"{corr.iloc[i, j]:.2f}", ha="center", va="center", fontsize=9)
fig.colorbar(im, ax=ax, label="Pearson r")
ax.set_title("Correlation matrix")
fig.tight_layout()
plt.show()
A diverging colour map centred at 0 with fixed limits −1 to 1 makes positive and negative correlations easy to compare. Seaborn’s heatmap(corr, annot=True) is a one-line alternative.
Common Pitfalls
Reading correlation as causation.
Missing non-linear relationships that have near-zero Pearson correlation (always plot the data).
Testing dozens of pairs without a multiple-testing correction.
65
How do you implement Metropolis-Hastings for simple Bayesian inference?
KnownRevise
Metropolis-Hastings (MH) is a Markov chain Monte Carlo algorithm that draws samples from a posterior distribution known only up to a constant. At each step it proposes a new parameter value (e.g. current value plus Gaussian noise), computes the acceptance ratio of posterior densities (times the proposal ratio, which cancels for symmetric proposals), and accepts the proposal with probability min(1, ratio), otherwise keeping the current value. After discarding a burn-in period, the chain’s samples approximate the posterior, so you can compute means and credible intervals; working in log space avoids numerical underflow.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, SciPy 1.13+
Algorithm
Start at θ₀.
Propose θ* ~ q(θ* | θₜ), for example Normal(θₜ, σ²).
For asymmetric proposals, forgetting the Hastings correction q(θₜ | θ) / q(θ | θₜ).
66
How do you estimate Pi with a Monte Carlo simulation in Python?
KnownRevise
Estimating π by Monte Carlo uses random sampling: draw points uniformly in the unit square, and count how many fall inside the quarter circle x² + y² ≤ 1. The quarter circle’s area is π/4, so the fraction inside estimates π/4, and multiplying by 4 estimates π. The estimate’s standard error shrinks like 1/√n, so each extra decimal digit of accuracy needs about 100 times more points, which is the key statistical lesson of the exercise.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
How It Works
Quantity
Value
Probability a point lands inside
p = π/4 ≈ 0.785
Estimator
π̂ = 4 × (inside / n)
Standard error
4 × √(p(1 − p) / n)
95% confidence interval
π̂ ± 1.96 × SE
Code Example
import numpy as np
def estimate_pi(n, rng):
x, y = rng.random(n), rng.random(n)
inside = (x ** 2 + y ** 2 <= 1.0)
p_hat = inside.mean()
pi_hat = 4 * p_hat
se = 4 * np.sqrt(p_hat * (1 - p_hat) / n)
return pi_hat, se
rng = np.random.default_rng(42)
for n in (1_000, 100_000, 10_000_000):
pi_hat, se = estimate_pi(n, rng)
print(f"n={n:>10,} estimate={pi_hat:.5f} SE={se:.5f} "
f"95% CI=[{pi_hat - 1.96 * se:.5f}, {pi_hat + 1.96 * se:.5f}] error={abs(pi_hat - np.pi):.5f}")
As n grows by 100×, the standard error drops by about 10×, and the interval should contain the true π about 95% of the time.
Where Monte Carlo Is Used
Integrals in many dimensions, where grid methods are too expensive.
Bayesian computation (MCMC), option pricing, risk simulation and bootstrap resampling.
Common Pitfalls
Looping in pure Python over millions of points instead of vectorizing with NumPy.
Reporting an estimate without its standard error.
Using rng.random() once per coordinate in a loop with a reseeded generator, which repeats samples.
67
How do you perform a chi-squared test of independence in Python?
KnownRevise
A chi-squared test of independence checks whether two categorical variables are associated, using a contingency table of observed counts. Under the null hypothesis of independence, expected counts are row total × column total / grand total; the statistic sums (observed − expected)² / expected and is compared with a chi-squared distribution with (rows − 1)(columns − 1) degrees of freedom. In Python, scipy.stats.chi2_contingency does the whole calculation; check that expected counts are at least about 5, and report an effect size such as Cramér’s V.
import numpy as np
import pandas as pd
from scipy.stats import chi2_contingency, fisher_exact
rng = np.random.default_rng(42)
n = 600
device = rng.choice(["mobile", "desktop", "tablet"], n, p=[0.5, 0.4, 0.1])
p_buy = np.select([device == "mobile", device == "desktop"], [0.10, 0.18], default=0.12)
bought = np.where(rng.random(n) < p_buy, "yes", "no")
table = pd.crosstab(device, bought)
print(table)
chi2, p, dof, expected = chi2_contingency(table)
print(f"chi2 = {chi2:.2f}, dof = {dof}, p = {p:.4f}")
print("minimum expected count:", round(expected.min(), 1))
cramers_v = np.sqrt(chi2 / (table.to_numpy().sum() * (min(table.shape) - 1)))
print("Cramér's V:", round(cramers_v, 3))
# For a 2×2 table with small expected counts, use Fisher's exact test instead
small = np.array([[3, 7], [9, 2]])
print("Fisher exact p-value:", round(fisher_exact(small)[1], 4))
If p is below 0.05, purchase rates differ by device more than chance would explain under independence; Cramér’s V tells you how strong the association is (values near 0.1 are weak).
Assumptions
Assumption
Note
Independent observations
Each person counted once
Counts, not percentages
Use raw frequencies
Expected counts ≥ 5 (most cells)
Otherwise merge categories or use Fisher’s exact test
Common Pitfalls
Passing proportions instead of counts.
Reading a significant result on a huge sample as a strong association; check the effect size.
Forgetting chi2_contingency applies Yates’ continuity correction to 2×2 tables by default (correction=True).
68
How do you make a non-stationary time series stationary in Python?
KnownRevise
Making a time series stationary means removing trend, changing variance and seasonality so that its mean, variance and autocorrelation stay constant over time. A practical function first stabilizes variance with a log (or Box-Cox) transform for positive data, then applies differencing, seasonal first if needed, then ordinary, and checks after each step with the Augmented Dickey-Fuller (ADF) test (null: unit root, non-stationary) and optionally KPSS (null: stationary). It should return the transformed series and the steps taken so forecasts can be inverted.
Seasonal difference at the period (12 for monthly)
Deterministic trend
Detrend by regression
Code Example
import numpy as np
import pandas as pd
from statsmodels.tsa.stattools import adfuller, kpss
def make_stationary(series, seasonal_period=None, alpha=0.05, max_diff=2):
s = series.astype(float).dropna()
steps = []
if (s > 0).all():
s = np.log(s)
steps.append("log")
if seasonal_period:
s = s.diff(seasonal_period).dropna()
steps.append(f"seasonal diff {seasonal_period}")
for _ in range(max_diff):
if adfuller(s, autolag="AIC")[1] < alpha:
break
s = s.diff().dropna()
steps.append("diff 1")
adf_p = adfuller(s, autolag="AIC")[1]
kpss_p = kpss(s, regression="c", nlags="auto")[1] # p-value is capped between 0.01 and 0.1
return s, steps, adf_p, kpss_p
rng = np.random.default_rng(42)
t = np.arange(144)
monthly = pd.Series(np.exp(0.01 * t + 0.1 * np.sin(2 * np.pi * t / 12) + np.cumsum(rng.normal(0, 0.02, 144))),
index=pd.date_range("2014-01-01", periods=144, freq="MS"))
print("ADF p-value before:", round(adfuller(monthly)[1], 4))
stationary, steps, adf_p, kpss_p = make_stationary(monthly, seasonal_period=12)
print("steps:", steps, "| ADF p-value after:", round(adf_p, 4), "| KPSS p-value:", round(kpss_p, 3))
A small ADF p-value rejects the unit root; a KPSS p-value above 0.05 fails to reject stationarity, so both tests support a stationary result.
Common Pitfalls
Over-differencing, which adds negative autocorrelation and noise.
Forgetting to invert transforms when converting forecasts back.
Applying a log to series containing zeros or negatives.
69
How do you conduct a one-way ANOVA test in R?
KnownRevise
A one-way ANOVA tests whether the means of three or more groups are equal (H₀: all means equal; H₁: at least one differs), by comparing between-group variance with within-group variance through the F statistic. In R, fit it with aov(outcome ~ group, data = df) and read summary(), check the assumptions (independent observations, roughly normal residuals, equal variances), use Welch’s ANOVA (oneway.test) when variances differ or Kruskal-Wallis when normality fails, and follow a significant result with Tukey’s HSD to see which pairs differ.
Read full answerHide full answer
Applies toR 4.4+ (base R `stats` package)
Assumptions and Alternatives
Assumption
Check
If violated
Independence
Study design
Mixed models / repeated-measures ANOVA
Normal residuals
Q-Q plot, shapiro.test(residuals)
kruskal.test
Equal variances
bartlett.test, Levene’s test (car package)
oneway.test(var.equal = FALSE)
Code Example
set.seed(42)
df <- data.frame(
fertilizer = factor(rep(c("A", "B", "C"), each = 30)),
yield = c(rnorm(30, mean = 20, sd = 3),
rnorm(30, mean = 22, sd = 3),
rnorm(30, mean = 25, sd = 3))
)
# Descriptive statistics per group
aggregate(yield ~ fertilizer, data = df, FUN = function(x) c(mean = mean(x), sd = sd(x)))
# One-way ANOVA
fit <- aov(yield ~ fertilizer, data = df)
summary(fit)
# Assumption checks
shapiro.test(residuals(fit))
bartlett.test(yield ~ fertilizer, data = df)
# Robust alternatives
oneway.test(yield ~ fertilizer, data = df, var.equal = FALSE) # Welch's ANOVA
kruskal.test(yield ~ fertilizer, data = df)
# Which pairs differ? (family-wise error controlled)
TukeyHSD(fit)
# Effect size: eta squared = SS_between / SS_total
ss <- summary(fit)[[1]][["Sum Sq"]]
cat("eta squared:", round(ss[1] / sum(ss), 3), "\n")
Read the Pr(>F) value: a small p-value means that, if all fertilizers had the same mean yield, differences this large would rarely occur, so at least one mean differs. Tukey’s table then shows which pairs, with adjusted p-values and confidence intervals.
Common Pitfalls
Using a numeric group code instead of a factor (R would fit a regression slope).
Running many t-tests instead of ANOVA plus a post-hoc test.
Reporting only the p-value without an effect size.
70
How do you implement PCA for dimensionality reduction in Python?
KnownRevise
Principal Component Analysis (PCA) reduces dimensionality by projecting data onto the orthogonal directions of maximum variance. Implementing it means centering (and usually standardizing) the features, computing the covariance matrix’s eigenvectors or, more stably, the singular value decomposition of the centered data, sorting components by explained variance, and keeping enough components to explain a target share (e.g. 95%). In practice, use scikit-learn’s PCA inside a pipeline; a from-scratch NumPy version shows the mechanics and should match it.
Compute the SVD of the centered matrix: X = U S Vᵀ.
Rows of Vᵀ are principal directions; explained variance is S² / (n − 1).
Project: Z = X V_k for the first k components.
Choose k from the cumulative explained variance ratio.
Code Example
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
X, _ = load_digits(return_X_y=True) # 1797 samples × 64 features
X_std = StandardScaler().fit_transform(X)
def pca_svd(X, n_components):
Xc = X - X.mean(axis=0)
U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
explained_var = S ** 2 / (len(X) - 1)
ratio = explained_var / explained_var.sum()
return Xc @ Vt[:n_components].T, ratio
Z_manual, ratio = pca_svd(X_std, 10)
k95 = int(np.searchsorted(np.cumsum(ratio), 0.95) + 1)
print("components for 95% variance:", k95, "of", X.shape[1])
pca = PCA(n_components=10, random_state=42).fit(X_std)
Z_sklearn = pca.transform(X_std)
# Component signs are arbitrary, so compare absolute values
print("matches scikit-learn:", np.allclose(np.abs(Z_manual), np.abs(Z_sklearn), atol=1e-6))
print("first 3 explained variance ratios:", pca.explained_variance_ratio_[:3].round(3))
PCA(n_components=0.95) picks the number of components automatically.
Common Pitfalls
Fitting PCA on all data before a train/test split (leakage); put it in a Pipeline.
Forgetting to standardize features measured in different units.
Interpreting components as real-world factors without checking loadings.
71
How would you find which factors contribute most to supermarket sales?
KnownRevise
Finding the factors behind supermarket sales combines exploratory analysis, a regression model of sales on candidate drivers (price, promotions, store size, footfall, season, holidays, location), and a careful reading of effect sizes. Standardized coefficients or partial effects show the relative importance of each factor, and permutation importance from a tree model gives a model-agnostic cross-check. Because the data are observational, these are associations; causal claims about levers like promotions need experiments or quasi-experimental designs.
Both methods should agree on the ranking here; when they disagree on real data, look for non-linear effects or interactions.
Common Pitfalls
Ranking raw coefficients measured in different units.
Correlated drivers (footfall and store size) splitting importance unpredictably.
Confounding: promotions may be run in weeks that would sell well anyway.
72
How do you determine whether a new drug is effective from trial data?
KnownRevise
Determining drug effectiveness relies on a randomized controlled trial with a pre-registered primary endpoint, sample size from a power calculation, and a pre-specified analysis plan. You compare the treatment and control (placebo or standard care) groups on the primary endpoint using the appropriate test, with an intention-to-treat analysis, report the effect size with a confidence interval, and judge both statistical significance and clinical relevance. Secondary endpoints and subgroups need multiple-testing control, and safety is analysed alongside efficacy.
Enough patients to detect the minimal clinically important difference
Intention-to-treat
Analyse patients in their randomized group, preserving randomization
Effect size + CI
e.g. mean difference, risk ratio, hazard ratio
Multiplicity
Hierarchical testing or corrections for secondary endpoints
Interim analyses
Pre-planned with alpha spending (O’Brien-Fleming)
Choosing the Test
Endpoint
Test
Continuous, roughly normal
Welch two-sample t-test or ANCOVA adjusting for baseline
Continuous, skewed
Mann-Whitney U
Binary (cured yes/no)
Chi-square or Fisher’s exact; risk difference CI
Time to event
Log-rank test, Cox model
Code Example: Blood Pressure Reduction
import numpy as np
from scipy import stats
from statsmodels.stats.power import TTestIndPower
rng = np.random.default_rng(42)
# Power: detect a 5 mmHg difference with SD 12, 90% power, alpha 0.05
n = TTestIndPower().solve_power(effect_size=5 / 12, alpha=0.05, power=0.9)
print("patients per arm:", int(np.ceil(n)))
drug = rng.normal(-12, 12, 130) # change in systolic BP (mmHg)
placebo = rng.normal(-6, 12, 130)
t, p = stats.ttest_ind(drug, placebo, equal_var=False) # Welch's t-test
diff = drug.mean() - placebo.mean()
se = np.sqrt(drug.var(ddof=1) / len(drug) + placebo.var(ddof=1) / len(placebo))
print(f"mean difference {diff:.2f} mmHg, 95% CI [{diff - 1.96 * se:.2f}, {diff + 1.96 * se:.2f}]")
print(f"t = {t:.2f}, p = {p:.2g}")
If the CI excludes zero and its range is clinically meaningful (e.g. the whole interval is beyond a 3 mmHg minimal important difference), the drug is both statistically and clinically effective on this endpoint.
Common Pitfalls
Per-protocol analysis only, which breaks randomization.
Switching endpoints after seeing data.
Claiming efficacy in a subgroup after a non-significant main result.
Confusing “not significant” with “no effect” in an underpowered trial.
73
How would you statistically evaluate an online advertising campaign's success?
KnownRevise
Evaluating an ad campaign means measuring its incremental effect, the conversions or revenue that would not have happened without the ads, not just counting conversions among people who saw them. The strongest design is a randomized holdout (lift test): randomly withhold ads from a control group, then compare conversion rates with a two-proportion test and report the lift with a confidence interval. Without randomization, use geo experiments, difference-in-differences or other quasi-experimental methods, and translate the lift into incremental cost per acquisition and return on ad spend.
Check balance between groups; segment results only if pre-planned
Why Not Compare Exposed vs Unexposed Users?
Ad platforms target people who were already likely to buy, so exposed users convert more even if ads do nothing (selection bias). Randomization removes that bias.
Code Example: Lift Test
import numpy as np
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import confint_proportions_2indep, proportion_effectsize, proportions_ztest
rng = np.random.default_rng(42)
# Sample size to detect 2.0% → 2.3% conversion with 80% power at alpha 0.05
effect = proportion_effectsize(0.023, 0.020)
n_per_group = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print("users needed per group:", int(np.ceil(n_per_group)))
# Simulated results
n_test, n_control = 50_000, 50_000
conv_test = rng.binomial(n_test, 0.023)
conv_control = rng.binomial(n_control, 0.020)
stat, p = proportions_ztest([conv_test, conv_control], [n_test, n_control])
low, high = confint_proportions_2indep(conv_test, n_test, conv_control, n_control, method="wald")
print(f"test {conv_test / n_test:.4f} vs control {conv_control / n_control:.4f}")
print(f"z = {stat:.2f}, p = {p:.4f}, 95% CI for the difference: [{low:.4f}, {high:.4f}]")
spend = 20_000
incremental = conv_test - conv_control * n_test / n_control
print("incremental conversions:", round(incremental), "| incremental CPA:",
round(spend / incremental, 2) if incremental > 0 else "n/a")
If the interval excludes zero, the campaign produced a real lift; whether it was worth it depends on the incremental CPA compared with the value of a customer.
Common Pitfalls
Using last-click attribution as proof of impact.
Peeking daily and stopping when p < 0.05, which inflates false positives.
Testing many segments and reporting only the significant ones without correction.
74
How would you use time series analysis to forecast stock prices?
KnownRevise
Forecasting stock prices with time series analysis usually means modelling returns rather than prices, because prices are non-stationary (close to a random walk) while log returns are closer to stationary. The honest approach is to test for stationarity, fit simple models (random-walk baseline, ARIMA for the mean, GARCH for volatility), and evaluate with walk-forward validation against the naive “tomorrow equals today” forecast. In efficient markets, return forecasts rarely beat that baseline, but volatility is genuinely predictable, which matters for risk management.
Augmented Dickey-Fuller test on prices and returns
Autocorrelation
ACF/PACF plots, Ljung-Box test
Mean model
ARIMA / AR on returns; compare to zero-return baseline
Volatility
GARCH(1,1) (the arch package)
Validation
Walk-forward (rolling origin), never random splits
Metrics
RMSE/MAE vs naive baseline; directional accuracy; trading costs if used for strategy
Code Example
import numpy as np
import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
rng = np.random.default_rng(42)
returns = rng.normal(0.0003, 0.01, 1000) # simulated daily log returns
prices = pd.Series(100 * np.exp(np.cumsum(returns)))
log_ret = np.log(prices).diff().dropna()
print("ADF p-value, prices :", round(adfuller(prices)[1], 4)) # large: non-stationary
print("ADF p-value, returns:", round(adfuller(log_ret)[1], 4)) # small: stationary
# Walk-forward one-step forecasts of returns: AR(1) vs naive zero-return baseline
errors_ar, errors_naive = [], []
for t in range(900, 999):
fit = ARIMA(log_ret.iloc[:t].to_numpy(), order=(1, 0, 0)).fit()
forecast = fit.forecast(1)[0]
actual = log_ret.iloc[t]
errors_ar.append((actual - forecast) ** 2)
errors_naive.append(actual ** 2)
print("RMSE AR(1):", round(np.sqrt(np.mean(errors_ar)), 5),
"| RMSE zero-return baseline:", round(np.sqrt(np.mean(errors_naive)), 5))
On these random-walk returns the AR model cannot beat the baseline in any meaningful way, which is exactly what you should expect, and check for, with real stock data.
Common Pitfalls
Fitting models to prices and reporting a high R² (yesterday’s price explains today’s).
Look-ahead bias: using information not available at forecast time.
Ignoring transaction costs and regime changes when claiming a profitable strategy.
75
How would you design a statistical study of subscription churn?
KnownRevise
A churn study starts by defining churn exactly (cancellation, or no renewal within a grace period) and the question you want answered: describing who churns, predicting who will churn, or estimating what causes churn. Descriptive work uses churn rates by cohort and segment with confidence intervals; time-to-churn is best handled with survival analysis (Kaplan-Meier curves, Cox regression) because many customers have not churned yet (censoring); and causal questions, such as whether a price change or onboarding email reduces churn, need a randomized experiment or careful quasi-experimental design.
What counts as churn; observation window; voluntary vs involuntary (failed payment)
Population
All active subscribers, or a sign-up cohort
Data
Tenure, plan, price, usage, support contacts, acquisition channel, payment failures
Descriptive
Monthly churn rate per cohort and segment, with confidence intervals
Time to churn
Kaplan-Meier survival curves; log-rank test between groups
Drivers
Cox proportional hazards or logistic regression (association, not causation)
Causal tests
A/B test of interventions with a power analysis beforehand
Reporting
Effect sizes with intervals, not just p-values
Code Example: Survival Curves and a Log-Rank Test
import numpy as np
import pandas as pd
from lifelines import KaplanMeierFitter
from lifelines.statistics import logrank_test
rng = np.random.default_rng(42)
n = 2000
plan = rng.choice(["monthly", "annual"], n, p=[0.7, 0.3])
# Monthly subscribers churn faster (shorter expected lifetime)
lifetime = np.where(plan == "monthly", rng.exponential(10, n), rng.exponential(25, n))
observed_for = rng.uniform(1, 24, n) # months since sign-up
df = pd.DataFrame({
"plan": plan,
"duration": np.minimum(lifetime, observed_for),
"churned": (lifetime <= observed_for).astype(int), # 0 = still active (censored)
})
kmf = KaplanMeierFitter()
for name, group in df.groupby("plan"):
kmf.fit(group["duration"], group["churned"], label=name)
print(name, "| median months to churn:", kmf.median_survival_time_,
"| retained at 12 months:", round(float(kmf.predict(12)), 3))
m, a = df[df.plan == "monthly"], df[df.plan == "annual"]
result = logrank_test(m["duration"], a["duration"], m["churned"], a["churned"])
print("log-rank statistic:", round(result.test_statistic, 2), "| p-value:", result.p_value)
A small p-value means that if the two plans truly had the same survival curve, a difference this large would be very unlikely, so retention differs by plan. It does not show the plan causes the difference: annual buyers may simply be more committed customers.
Common Pitfalls
Computing churn only among customers who already left, ignoring censored active customers.
Treating regression coefficients as causal drivers.
Mixing voluntary and payment-failure churn, which have different fixes.
Simpson’s paradox: overall churn can move opposite to every segment when the customer mix changes.