What are the differences between Python 2 and Python 3?
KnownRevise
Python 3 is the current language and Python 2 reached end of life on 1 January 2020, so new code, and every major ML library (NumPy, pandas, scikit-learn, PyTorch, TensorFlow), is Python 3 only; the key differences are that print became a function, strings are Unicode text by default with a separate bytes type, / always performs true division, many built-ins (range, map, zip, dict.items) return lazy iterables instead of lists, and integer types were unified. Python 3 also added features ML code relies on daily: f-strings, type hints, async/await, dataclasses, the @ matrix-multiplication operator, and pathlib.
Read full answerHide full answer
Applies toPython 2.7 vs Python 3.12
Key Differences
Feature
Python 2
Python 3
Print
print "x" statement
print("x") function
Strings
str = bytes, unicode separate
str = Unicode text, bytes separate
Division
3 / 2 == 1 (for ints)
3 / 2 == 1.5; // is floor division
range, map, zip, filter
Return lists
Return lazy iterators/views
Integers
int and long
Single arbitrary-precision int
Exceptions
except E, e:
except E as e:
Input
raw_input()
input()
Formatting
% and .format
Also f-strings: f"{x:.2f}"
Matrix multiply
np.dot(a, b)
a @ b (PEP 465)
Support
Ended 2020
Active; yearly releases
Code Example
from pathlib import Path
import numpy as np
print(7 / 2, 7 // 2) # 3.5 3
text = "naïve café"
print(len(text), len(text.encode("utf-8"))) # 10 characters, 12 bytes
squares = map(lambda n: n * n, range(4))
print(squares.__class__.__name__, list(squares)) # map [0, 1, 4, 9]
a = np.array([[1, 2], [3, 4]])
print(a @ a) # [[ 7 10] [15 22]]
accuracy = 0.91234
print(f"accuracy = {accuracy:.2%}") # accuracy = 91.23%
print(Path("data") / "train.csv") # data/train.csv
print(2 ** 100) # big integers work without overflow
Unicode strings make text processing (NLP) far less error-prone.
Common Pitfalls
Copying Python 2 snippets from old tutorials.
Expecting zip or map results to be reusable lists (they are one-pass iterators).
Interview Tips
Lead with print, Unicode strings, division and lazy iterators.
Mention that Python 2 is end-of-life and libraries are Python 3 only.
2
How does Python manage memory?
KnownRevise
CPython (the standard interpreter) manages memory automatically: every value is an object on a private heap, and each object carries a reference count that is increased when a name, container or argument refers to it and decreased when that reference goes away; when the count reaches zero the object is freed immediately. A cyclic garbage collector (the gc module) periodically finds groups of objects that reference each other but are unreachable, which reference counting alone can’t free. Small objects are allocated from pools managed by the pymalloc allocator, and large data such as NumPy arrays and PyTorch tensors keep their buffers in separately allocated memory (C memory or GPU memory) owned by a Python object.
Read full answerHide full answer
Applies toCPython 3.12
Layers
Layer
Role
Names and references
Variables are references to objects, not boxes holding values
Reference counting
Immediate freeing when count hits 0
Cyclic GC (gc)
Generational collector for reference cycles
pymalloc
Fast allocator for small objects (≤ 512 bytes)
Extension buffers
NumPy arrays, tensors: large raw buffers outside pymalloc
Code Example
import gc
import sys
import numpy as np
a = [1, 2, 3]
b = a # second reference, same object
print(a is b, sys.getrefcount(a)) # True, a small count (includes the temporary argument reference)
b.append(4)
print(a) # [1, 2, 3, 4]: both names see the change
class Node:
def __init__(self):
self.other = None
x, y = Node(), Node()
x.other, y.other = y, x # reference cycle
del x, y # counts don't reach zero
print("collected:", gc.collect() >= 2) # True: the cycle collector frees them
arr = np.zeros((1000, 1000)) # 8 MB buffer owned by the array object
view = arr[:10] # a view keeps the whole buffer alive
print(arr.nbytes / 1e6, view.base is arr) # 8.0 True
del arr
print(view.base.nbytes / 1e6) # 8.0: the buffer survives while the view exists
Memory Tips for ML Code
Delete large intermediate objects (del df_tmp) and avoid keeping references in lists or notebooks.
Views (slices) keep large arrays alive; use .copy() when you need only a small piece.
Use smaller dtypes (float32, category) for big tables.
Measure with tracemalloc, memory_profiler, or df.memory_usage(deep=True).
In PyTorch, store loss.item() rather than the loss tensor so graphs are freed.
Common Pitfalls
Thinking b = a copies a list.
Hidden references in notebook output history (Out[...]) keeping large objects alive.
Expecting freed memory to return to the operating system immediately (allocators may keep it).
Interview Tips
Explain reference counting plus the cyclic collector.
Connect to practical ML memory issues: views, dtypes, lingering references.
3
What is PEP 8 and why is it important?
KnownRevise
PEP 8 is Python’s official style guide for code: it sets conventions for indentation (4 spaces), line length (79 characters in the guide, though many projects use 88–120), naming (snake_case for functions and variables, PascalCase for classes, UPPER_CASE for constants), whitespace, imports (one per line, grouped standard library / third-party / local) and comments. It matters because consistent style makes code easier to read, review and maintain across a team; in ML projects, where research code often becomes production code, it reduces friction, and tools such as Ruff and Black apply it automatically so people don’t argue about formatting.
Read full answerHide full answer
Applies toPython 3.12; Ruff 0.6+, Black 24+
Key Rules
Area
Convention
Indentation
4 spaces, no tabs
Line length
79 (guide); 88 (Black/Ruff default)
Names
train_model, learning_rate, class DataLoader, MAX_EPOCHS
Imports
Top of file; stdlib, third-party, local groups; no wildcard imports
Whitespace
x = 1, f(a, b), no spaces inside brackets
Comparisons
if x is None, if not items
Blank lines
2 around top-level definitions, 1 between methods
Docstrings
PEP 257 triple-quoted summaries
Code Example
"""Train a simple model (PEP 8 style example)."""
import random
from dataclasses import dataclass
import numpy as np
RANDOM_SEED = 42
@dataclass
class TrainConfig:
learning_rate: float = 0.1
n_epochs: int = 50
def fit_mean_predictor(targets: np.ndarray, config: TrainConfig) -> float:
"""Learn a constant prediction by gradient descent on squared error."""
prediction = 0.0
for _ in range(config.n_epochs):
gradient = 2 * (prediction - targets).mean()
prediction -= config.learning_rate * gradient
return prediction
if __name__ == "__main__":
random.seed(RANDOM_SEED)
rng = np.random.default_rng(RANDOM_SEED)
y = rng.normal(loc=5.0, scale=1.0, size=1000)
fitted = fit_mean_predictor(y, TrainConfig())
print(f"fitted {fitted:.3f} vs sample mean {y.mean():.3f}") # nearly equal
Tooling
ruff check . (linting) and ruff format . or black . (formatting).
pre-commit hooks to run them on every commit.
nbqa or Ruff’s notebook support for Jupyter notebooks.
Common Pitfalls
Treating PEP 8 as absolute; the guide itself says consistency within a project matters more.
Hand-formatting instead of using tools.
Single-letter names everywhere in ML code (X, y are fine conventions; a1, t2 are not).
Interview Tips
Give a few concrete rules and name the tools that enforce them.
Explain readability and team consistency as the reason.
4
What are the differences between a list, a tuple and a set in Python?
KnownRevise
A list is an ordered, mutable sequence that allows duplicates ([1, 2, 2]); a tuple is an ordered, immutable sequence that also allows duplicates and can be hashed if its items are hashable ((1, 2)), so it works as a dictionary key; a set is an unordered, mutable collection of unique, hashable items ({1, 2}) with fast O(1) average membership tests and set algebra (union, intersection, difference). In ML code, lists hold growing collections (metrics per epoch), tuples hold fixed records (shapes, (X, y) pairs, dictionary keys), and sets handle uniqueness (vocabularies, de-duplicating IDs, comparing feature lists).
(5) is just the integer 5; a one-item tuple needs a comma: (5,).
{} creates an empty dict, not an empty set.
Relying on set order (iteration order is not guaranteed).
Using x in big_list inside loops instead of converting to a set.
Interview Tips
Compare ordering, mutability, duplicates and hashability.
Give an ML use for each, such as detecting train/test overlap with sets.
5
How does a dictionary work in Python?
KnownRevise
A Python dictionary (dict) maps unique keys to values and is implemented as a hash table: the key’s hash decides where the entry is stored, so looking up, inserting or deleting by key takes O(1) time on average; keys must be hashable (immutable types such as strings, numbers or tuples of them), while values can be anything, and since Python 3.7 dictionaries keep insertion order. In ML code dictionaries hold configurations and hyperparameters, label-to-index mappings, metric results, feature maps and model state_dicts.
Read full answerHide full answer
Applies toPython 3.12
Operations
Operation
Syntax
Time (average)
Create
{"lr": 0.01}, dict(lr=0.01)
Read
d["lr"] (KeyError if missing), d.get("lr", default)
O(1)
Write / update
d["lr"] = 0.1, d.update(other), d | other
O(1) per key
Delete
del d["lr"], d.pop("lr", None)
O(1)
Membership
"lr" in d
O(1)
Iterate
for k, v in d.items()
O(n)
Code Example
from collections import Counter, defaultdict
config = {"model": "resnet18", "lr": 1e-3, "epochs": 10}
config["batch_size"] = 64
print(config.get("dropout", 0.0), "lr" in config) # 0.0 True
overrides = {"lr": 3e-4}
final = config | overrides # merge (Python 3.9+); right side wins
print(final["lr"], list(final)[:2]) # 0.0003 ['model', 'lr']
labels = ["cat", "dog", "cat", "bird", "dog", "cat"]
label_to_idx = {label: i for i, label in enumerate(sorted(set(labels)))}
print(label_to_idx) # {'bird': 0, 'cat': 1, 'dog': 2}
print([label_to_idx[x] for x in labels]) # [1, 2, 1, 0, 2, 1]
counts = Counter(labels)
print(counts.most_common(1)) # [('cat', 3)]
by_split = defaultdict(list)
for sample_id, split in [(1, "train"), (2, "test"), (3, "train")]:
by_split[split].append(sample_id)
print(dict(by_split)) # {'train': [1, 3], 'test': [2]}
try:
{["a"]: 1}
except TypeError as err:
print("lists can't be keys:", err)
How Hashing Works (Briefly)
hash(key) gives an integer; the dict uses it to pick a slot.
Equal keys must have equal hashes; collisions are resolved by probing other slots.
The table resizes as it fills, keeping lookups fast.
Common Pitfalls
Mutable keys (lists, dicts) raise TypeError.
d[key] on a missing key raises KeyError; use get, setdefault or defaultdict.
Modifying a dictionary’s size while iterating over it.
Interview Tips
Explain hash tables and average O(1) access.
Show get, comprehensions, Counter and defaultdict.
6
What is a list comprehension in Python? Give an example.
KnownRevise
A list comprehension builds a new list from an iterable in a single expression, [expression for item in iterable if condition], which is shorter and usually faster than an equivalent for loop with append. The same syntax creates dictionaries ({k: v for ...}), sets ({x for ...}) and lazy generators ((x for ...)). In ML code they are handy for small transformations such as building file lists or label maps, but large numeric work should use vectorized NumPy or pandas operations instead.
Read full answerHide full answer
Applies toPython 3.12
Forms
Form
Example
Result
List
[x ** 2 for x in range(5)]
[0, 1, 4, 9, 16]
With filter
[x for x in range(10) if x % 3 == 0]
[0, 3, 6, 9]
With if/else
["pos" if x > 0 else "neg" for x in [3, -1]]
['pos', 'neg']
Nested
[(i, j) for i in range(2) for j in range(2)]
[(0, 0), (0, 1), (1, 0), (1, 1)]
Dict
{w: len(w) for w in ["ml", "data"]}
{'ml': 2, 'data': 4}
Set
{w.lower() for w in ["A", "a", "B"]}
{'a', 'b'}
Generator
sum(x * x for x in range(1_000_000))
Computed lazily
Code Example
import numpy as np
files = ["img_01.png", "notes.txt", "img_02.png", "img_03.jpg"]
images = [f for f in files if f.endswith((".png", ".jpg"))]
print(images) # ['img_01.png', 'img_02.png', 'img_03.jpg']
sentences = ["Deep learning rocks", "Python is great"]
tokens = [word.lower() for s in sentences for word in s.split()]
vocab = {word: i for i, word in enumerate(sorted(set(tokens)))}
print(tokens[:3], vocab["python"]) # ['deep', 'learning', 'rocks'] 4
scores = [0.2, 0.8, 0.55, 0.4]
labels = [1 if s >= 0.5 else 0 for s in scores]
print(labels) # [0, 1, 1, 0]
rng = np.random.default_rng(42)
big = rng.random(1_000_000)
loop_style = [1 if s >= 0.5 else 0 for s in big] # works, but slow for large arrays
vectorized = (big >= 0.5).astype(int) # much faster: runs in C
print(sum(loop_style) == vectorized.sum()) # True
Common Pitfalls
Deeply nested comprehensions that are hard to read; use a loop or helper function.
Using a list comprehension only for side effects ([print(x) for x in xs]).
Building huge lists when a generator expression would stream the values.
Interview Tips
Write the general form and a filtered example.
Mention dict/set comprehensions and when vectorization is better.
7
What are generators in Python and how do they differ from list comprehensions?
KnownRevise
A generator produces values lazily, one at a time, only when asked (by next() or a for loop), instead of building the whole collection in memory; you create one with a function that uses yield or with a generator expression in parentheses, (x * x for x in data). A list comprehension builds the complete list immediately, so it uses memory proportional to the number of items and can be indexed and reused, while a generator uses constant memory, can represent infinite or streaming sequences, and can be iterated only once. In ML, generators stream batches from files too large for memory, read logs line by line, and power data pipelines.
Read full answerHide full answer
Applies toPython 3.12
Comparison
Aspect
List comprehension
Generator
Syntax
[f(x) for x in it]
(f(x) for x in it) or def ...: yield
Evaluation
Eager: all at once
Lazy: on demand
Memory
O(n)
O(1) per item
Reusable
Yes
No: exhausted after one pass
Indexing / len
Yes
No
Infinite sequences
No
Yes
Code Example
import sys
from itertools import islice
import numpy as np
squares_list = [x * x for x in range(1_000_000)]
squares_gen = (x * x for x in range(1_000_000))
print(sys.getsizeof(squares_list) > 1_000_000, sys.getsizeof(squares_gen) < 1_000) # True True
def batch_generator(X, y, batch_size, seed=42):
"""Yield shuffled mini-batches forever, like a training data stream."""
rng = np.random.default_rng(seed)
n = len(X)
while True:
order = rng.permutation(n)
for start in range(0, n, batch_size):
idx = order[start:start + batch_size]
yield X[idx], y[idx]
rng = np.random.default_rng(42)
X, y = rng.normal(size=(10, 3)), np.arange(10)
batches = batch_generator(X, y, batch_size=4)
for xb, yb in islice(batches, 3): # take only 3 batches from the infinite stream
print(xb.shape, len(yb)) # (4, 3) 4, then (4, 3) 4, then (2, 3) 2
def read_lines(path):
with open(path, encoding="utf-8") as f:
for line in f: # files are already lazy iterators
yield line.rstrip("\n")
with open("log.txt", "w", encoding="utf-8") as f:
f.write("epoch 1 loss 0.9\nepoch 2 loss 0.6\n")
print(max(float(line.split()[-1]) for line in read_lines("log.txt"))) # 0.9
gen = (x for x in range(3))
print(list(gen), list(gen)) # [0, 1, 2] []: exhausted after one pass
Common Pitfalls
Iterating a generator twice and getting nothing the second time.
Calling len() on a generator.
Using a generator when you need random access; convert with list() if it fits in memory.
Interview Tips
Explain lazy evaluation and memory use.
Give a batch-streaming example for ML.
8
How do you use *args and **kwargs in Python function definitions?
KnownRevise
In a function definition, *args collects any extra positional arguments into a tuple and **kwargs collects any extra keyword arguments into a dictionary, so a function can accept a variable number of inputs; in a call, * and ** do the reverse, unpacking a sequence into positional arguments and a dictionary into keyword arguments. In ML code they are used to forward options to another function (for example passing hyperparameters through a wrapper to a scikit-learn or PyTorch constructor), to write decorators that wrap any function, and to build models from configuration dictionaries with Model(**config).
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
def describe(*args, **kwargs):
print("args:", args, "| kwargs:", kwargs)
describe(1, 2, lr=0.01, epochs=5) # args: (1, 2) | kwargs: {'lr': 0.01, 'epochs': 5}
def evaluate(model_cls, X, y, *, cv=5, **model_params):
"""Forward any hyperparameters to the model constructor."""
model = model_cls(**model_params)
return cross_val_score(model, X, y, cv=cv).mean()
X, y = make_classification(n_samples=300, n_features=10, random_state=42)
config = {"n_estimators": 100, "max_depth": 5, "random_state": 42}
score = evaluate(RandomForestClassifier, X, y, cv=3, **config) # unpack the dict
print(f"mean CV accuracy: {score:.3f}")
def weighted_mean(*values, weights=None):
weights = weights or [1] * len(values)
return sum(v * w for v, w in zip(values, weights)) / sum(weights)
scores = [0.8, 0.9, 0.7]
print(round(weighted_mean(*scores), 3)) # 0.8
print(round(weighted_mean(*scores, weights=[1, 2, 1]), 3)) # (0.8 + 1.8 + 0.7) / 4 = 0.825
Common Pitfalls
Accepting **kwargs and silently ignoring misspelled options; validate or forward them to something that checks.
Confusing definition (*args collects) with call (*seq unpacks).
Overusing them, which hides a function’s real interface; type hints and explicit parameters are clearer.
Interview Tips
Explain packing in definitions and unpacking in calls.
Show forwarding hyperparameters with Model(**params).
9
How does Python's garbage collection work?
KnownRevise
CPython frees most objects through reference counting: each object tracks how many references point to it and is deallocated the moment that count drops to zero; on top of that, a generational cyclic garbage collector (the gc module) finds and frees groups of container objects that reference each other but are no longer reachable from the program, which reference counting alone can’t free. The cyclic collector groups objects into generations (new objects start young and survivors move to older generations, which are scanned less often), runs automatically when allocation thresholds are crossed, and can be triggered with gc.collect() or tuned with gc.set_threshold.
Read full answerHide full answer
Applies toCPython 3.12
Two Mechanisms
Mechanism
Frees
When
Reference counting
Objects with no references
Immediately
Cyclic GC
Unreachable reference cycles (lists, dicts, class instances)
Periodically, by generation thresholds
How the Cyclic Collector Finds Garbage
Look only at container objects that can form cycles (lists, dicts, instances).
For each object in the generation being scanned, subtract references that come from other objects in the same set.
Objects still referenced from outside are reachable; everything reachable from them is kept.
What remains is unreachable cyclic garbage and is freed (finalizers run first).
Code Example
import gc
import weakref
class Model:
def __init__(self, name):
self.name = name
self.peer = None
def __del__(self):
print(f"freed {self.name}")
m = Model("solo")
del m # prints "freed solo" right away: refcount hit 0
a, b = Model("a"), Model("b")
a.peer, b.peer = b, a # a cycle
watch = weakref.ref(a) # a weak reference doesn't keep a alive
del a, b
print("still alive before collect:", watch() is not None) # True
gc.collect() # prints "freed" for a and b (order may vary)
print("alive after collect:", watch() is not None) # False
print(gc.get_threshold()) # generation thresholds, e.g. (700, 10, 10) in 3.12
Practical Notes for ML Code
Most large ML objects (arrays, tensors, DataFrames) are freed by reference counting as soon as the last reference goes.
Lingering references, not the GC, are the usual cause of memory growth: lists of tensors, notebook Out[...] history, global caches, exception tracebacks.
gc.collect() plus torch.cuda.empty_cache() can help reclaim GPU memory in notebooks after deleting models.
Weak references (weakref) let caches refer to objects without keeping them alive.
Common Pitfalls
Calling gc.collect() in tight loops (slow, rarely needed).
Assuming del x frees memory when other references remain.
Disabling GC without understanding the cycles in your code.
Interview Tips
Explain reference counting first, then why cycles need a separate collector.
Mention generations and practical memory debugging.
10
What are decorators in Python and when would you use one?
KnownRevise
A decorator is a function that takes another function (or class) and returns a modified version of it, applied with the @decorator syntax above a definition; it lets you add behaviour such as timing, logging, caching, retries, input validation or registration without changing the original function’s code. Python ships useful ones (@functools.lru_cache, @property, @staticmethod, @dataclass), and ML code commonly uses them for timing training steps, caching expensive feature computations, retrying flaky data downloads, registering models by name, and framework features such as @torch.no_grad() and @torch.compile.
Read full answerHide full answer
Applies toPython 3.12
Code Example
import functools
import time
def timed(func):
"""Print how long each call takes."""
@functools.wraps(func) # keep the original name and docstring
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
print(f"{func.__name__} took {time.perf_counter() - start:.4f}s")
return result
return wrapper
def retry(times=3, exceptions=(ConnectionError,)):
"""A decorator factory: takes arguments and returns a decorator."""
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(1, times + 1):
try:
return func(*args, **kwargs)
except exceptions:
if attempt == times:
raise
print(f"attempt {attempt} failed, retrying")
return wrapper
return decorator
MODELS = {}
def register(name):
def decorator(cls):
MODELS[name] = cls
return cls
return decorator
@register("baseline")
class MeanModel:
def fit(self, y):
self.mean_ = sum(y) / len(y)
return self
@timed
@functools.lru_cache(maxsize=None) # cache results of an expensive pure function
def expensive_feature(n):
return sum(i * i for i in range(n))
calls = {"count": 0}
@retry(times=3)
def flaky_download():
calls["count"] += 1
if calls["count"] < 3:
raise ConnectionError("network blip")
return "data.csv"
print(expensive_feature(2_000_000)) # slow the first time
print(expensive_feature(2_000_000)) # near-instant: served from the cache
print(flaky_download()) # two retries, then 'data.csv'
print(MODELS["baseline"]().fit([1, 2, 3]).mean_) # 2.0
Forgetting functools.wraps, which hides the wrapped function’s name and docstring.
Caching functions that take unhashable arguments (lists, arrays) or have side effects.
Stacking decorators without realizing they apply bottom-up.
Interview Tips
Explain that @d above def f means f = d(f).
Write a timing decorator with wraps, and mention decorator factories.
11
Which Python libraries are most used in machine learning and what are they for?
KnownRevise
The core Python ML stack is NumPy (fast n-dimensional arrays and linear algebra), pandas (tabular data loading, cleaning and transformation), Matplotlib and seaborn (plotting), SciPy (scientific routines: statistics, optimization, sparse matrices), scikit-learn (classical ML models, preprocessing, pipelines and evaluation), XGBoost / LightGBM / CatBoost (gradient-boosted trees for tabular data), statsmodels (statistical models and tests), and PyTorch or TensorFlow/Keras (deep learning), with Jupyter notebooks for interactive work. Around them sit Hugging Face Transformers for pre-trained NLP and vision models, Optuna for hyperparameter tuning, MLflow for experiment tracking, and Polars or Dask for larger data.
Reaching for deep learning on small tabular datasets, where boosted trees usually win.
Unpinned library versions, which break notebooks months later.
Interview Tips
Group libraries by stage: data, visualization, classical ML, boosting, deep learning, tooling.
Say which you’d pick for tabular vs image or text problems.
12
What is NumPy and how is it useful in machine learning?
KnownRevise
NumPy is the foundational Python library for numerical computing: it provides the ndarray, a fixed-type, contiguous n-dimensional array, along with fast vectorized operations, broadcasting, linear algebra, random number generation and statistics, all running in compiled C instead of Python loops. In machine learning, datasets, feature matrices, images and model weights are arrays, and pandas, scikit-learn, SciPy and Matplotlib are built on NumPy; even PyTorch and TensorFlow tensors convert to and from NumPy arrays.
Read full answerHide full answer
Applies toNumPy 2.x, Python 3.12
Why It Matters for ML
Feature
Benefit
Vectorization
10–100× faster than Python loops
Broadcasting
Operate on arrays of different shapes without copies (e.g. subtract feature means)
Linear algebra (@, np.linalg)
Matrix products, solving, decompositions
Random generators
Reproducible sampling (np.random.default_rng(42))
Fixed dtypes
Compact memory (float32, int8)
Indexing / masking
Fast filtering and selection
Code Example
import time
import numpy as np
rng = np.random.default_rng(42)
X = rng.normal(loc=10, scale=3, size=(100_000, 5)) # 100k samples, 5 features
X_std = (X - X.mean(axis=0)) / X.std(axis=0) # broadcasting: standardize each column
print(X_std.mean(axis=0).round(6), X_std.std(axis=0).round(6)) # zeros and ones
start = time.perf_counter()
loop_sum = sum(x * x for x in X[:, 0])
loop_t = time.perf_counter() - start
start = time.perf_counter()
vec_sum = np.dot(X[:, 0], X[:, 0])
vec_t = time.perf_counter() - start
print(np.isclose(loop_sum, vec_sum), f"vectorized is ~{loop_t / vec_t:.0f}x faster")
# Linear regression with the normal equation: w = (X^T X)^-1 X^T y, solved stably with lstsq
true_w = np.array([2.0, -1.0, 0.5, 0.0, 3.0])
y = X_std @ true_w + 4 + rng.normal(scale=0.1, size=len(X_std))
X_b = np.column_stack([np.ones(len(X_std)), X_std])
w, *_ = np.linalg.lstsq(X_b, y, rcond=None)
print(w.round(2)) # close to [4, 2, -1, 0.5, 0, 3]
mask = y > 10
print(f"{mask.mean():.1%} of samples have y > 10") # boolean masking
Common Pitfalls
Python loops over array elements instead of vectorized operations.
Shape mistakes: (n,) vs (n, 1) broadcasting into (n, n).
Views vs copies: slices share memory, so modifying them changes the original.
Using the legacy global np.random.seed; prefer a Generator from default_rng.
Interview Tips
Explain vectorization and broadcasting with an example.
Mention that most ML libraries build on NumPy arrays.
13
What is pandas and why is it important for data manipulation?
KnownRevise
pandas is the standard Python library for working with tabular data: its DataFrame (a table of named, typed columns with an index) and Series (one column) let you load data from CSV, Excel, SQL, Parquet and JSON, then clean, filter, transform, join, reshape, group and aggregate it with concise, vectorized operations. In ML it is where most data preparation happens (handling missing values, encoding categories, creating features, splitting by time or group) before converting the result to NumPy arrays or passing DataFrames directly to scikit-learn.
Vectorized arithmetic, .str and .dt accessors, assign
Aggregate
groupby().agg(), pivot_table
Combine
merge, concat, join
Reshape
melt, pivot, stack
Time series
DatetimeIndex, resample, rolling
Code Example
import numpy as np
import pandas as pd
rng = np.random.default_rng(42)
orders = pd.DataFrame({
"order_id": range(1, 9),
"customer": ["a", "b", "a", "c", "b", "a", "c", "d"],
"amount": [120.0, 80.0, np.nan, 300.0, 45.0, 60.0, 210.0, 95.0],
"date": pd.date_range("2026-01-01", periods=8, freq="W"),
})
customers = pd.DataFrame({"customer": ["a", "b", "c", "d"], "segment": ["gold", "silver", "gold", "new"]})
orders["amount"] = orders["amount"].fillna(orders["amount"].median()) # median of the 7 values = 95
orders["month"] = orders["date"].dt.month
df = orders.merge(customers, on="customer", how="left")
features = (df.groupby("customer")
.agg(n_orders=("order_id", "count"), total=("amount", "sum"), avg=("amount", "mean"))
.reset_index()
.round(2))
print(features)
# customer n_orders total avg
# 0 a 3 275.0 91.67 (120 + 95 + 60)
# 1 b 2 125.0 62.50
# 2 c 2 510.0 255.00
# 3 d 1 95.0 95.00
print(df.pivot_table(index="segment", columns="month", values="amount", aggfunc="sum", fill_value=0))
X = features[["n_orders", "total", "avg"]].to_numpy() # ready for a model
print(X.shape) # (4, 3)
Common Pitfalls
Row-by-row loops with iterrows instead of vectorized operations.
Chained assignment (df[df.x > 0]["y"] = 1), which may not modify df; use df.loc[mask, "y"] = 1.
object dtype for strings and categories wasting memory; use category or the string dtype.
Silent row explosions from merges on non-unique keys (use validate="m:1").
Interview Tips
Name the main operations: load, clean, groupby, merge, reshape.
Show vectorized feature engineering with groupby().agg().
14
How does scikit-learn fit into the machine learning workflow?
KnownRevise
scikit-learn is the standard Python library for classical machine learning, and it covers most of the workflow with one consistent API: preprocessing (scalers, encoders, imputers), feature selection and dimensionality reduction, models for classification, regression and clustering, Pipeline and ColumnTransformer to chain steps without leakage, model selection (train/test splits, cross-validation, grid and random search) and evaluation metrics. Every estimator follows fit / predict (or transform), so models are interchangeable, and fitted pipelines can be saved with joblib for deployment.
Not designed for deep learning or GPU training (use PyTorch/TensorFlow).
Datasets must usually fit in memory (use partial_fit models, Dask-ML or Spark for more).
Common Pitfalls
Fitting scalers or encoders on the full dataset before splitting (data leakage); use pipelines.
Ignoring random_state, so results aren’t reproducible.
Evaluating with accuracy on imbalanced data.
Interview Tips
Describe the fit/predict/transform API and pipelines.
Walk through split → pipeline → CV tuning → test evaluation → save.
15
What are Matplotlib and seaborn used for in data visualization?
KnownRevise
Matplotlib is Python’s foundational plotting library: it gives full control over figures, axes, lines, labels and styles and can produce almost any static chart, at the cost of more code; seaborn is built on Matplotlib and adds a high-level, statistics-oriented API that works directly with pandas DataFrames to draw distributions, relationships, categorical comparisons and correlation heatmaps in one call, with good default styling. In ML work you use seaborn for fast exploratory analysis (feature distributions, pair plots, class balance) and Matplotlib for custom result plots (learning curves, ROC curves, publication figures) or fine-tuning seaborn output.
Read full answerHide full answer
Applies toMatplotlib 3.9+, seaborn 0.13+
Comparison
Aspect
Matplotlib
seaborn
Level
Low-level, flexible
High-level, opinionated
Data input
Arrays, lists
pandas DataFrames with column names
Statistical plots
Manual
Built in (KDE, regression fits, confidence intervals)
Using the implicit plt.* state machine in complex scripts; the fig, ax = plt.subplots() style is clearer.
Unlabelled axes or missing units.
Overplotting large datasets (use transparency, hexbin or sampling).
Rainbow colormaps for ordered data (use perceptually uniform ones such as viridis).
Interview Tips
Say seaborn for quick statistical EDA, Matplotlib for control.
Name the plots you use for distributions, relationships and results.
16
What are the differences between SciPy and NumPy?
KnownRevise
NumPy provides the core data structure, the n-dimensional array, together with basic operations on it: vectorized math, broadcasting, indexing, basic linear algebra, random numbers and simple statistics; SciPy is built on NumPy arrays and adds higher-level scientific algorithms: optimization, statistical distributions and tests, sparse matrices, integration, interpolation, signal and image processing, spatial algorithms and more complete linear algebra. In ML, NumPy is the data container and math layer, while SciPy supplies tools such as scipy.optimize.minimize for fitting custom models, scipy.stats for hypothesis tests, and scipy.sparse for large sparse feature matrices like bag-of-words, which scikit-learn uses directly.
Read full answerHide full answer
Applies toNumPy 2.x, SciPy 1.13+
Comparison
Area
NumPy
SciPy
Role
Array container + core numerics
Scientific algorithms on arrays
Linear algebra
np.linalg basics
scipy.linalg: more decompositions, BLAS/LAPACK access
import numpy as np
from scipy import optimize, sparse, stats
from scipy.spatial.distance import cdist
rng = np.random.default_rng(42)
a = rng.normal(0.0, 1.0, 200) # NumPy: arrays and basic stats
b = rng.normal(0.3, 1.0, 200)
print(round(float(a.mean()), 3), round(float(b.mean()), 3))
t, p = stats.ttest_ind(a, b) # SciPy: hypothesis test
print(f"t-test p-value: {p:.4f}")
x = np.linspace(0, 5, 50) # SciPy: fit a custom model
y = 3.0 * np.exp(-0.7 * x) + rng.normal(0, 0.05, 50)
params, _ = optimize.curve_fit(lambda x, a, k: a * np.exp(-k * x), x, y, p0=[1, 1])
print("fitted a, k:", params.round(2)) # close to [3.0, 0.7]
dense = np.zeros((1000, 5000))
dense[rng.integers(0, 1000, 3000), rng.integers(0, 5000, 3000)] = 1
sp = sparse.csr_matrix(dense) # SciPy: sparse storage
print(f"dense {dense.nbytes / 1e6:.0f} MB vs sparse {(sp.data.nbytes + sp.indices.nbytes + sp.indptr.nbytes) / 1e6:.2f} MB")
points = rng.random((4, 2))
print(cdist(points, points).round(2).shape) # (4, 4) pairwise distance matrix
Common Pitfalls
Converting sparse matrices to dense (.toarray()) on large data and running out of memory.
Using NumPy for problems SciPy already solves robustly (optimization, distributions).
Interview Tips
Describe NumPy as the foundation and SciPy as the algorithm toolkit.
Give ML examples: sparse text features, statistical tests, custom optimization.
17
What are the benefits of Jupyter notebooks for machine learning projects?
KnownRevise
Jupyter notebooks combine code, its output (tables, plots), and explanatory Markdown in one document that you run cell by cell, which makes them ideal for exploratory data analysis, quick experiments, visualizing results and sharing reproducible reports or tutorials. Their weaknesses are the other side of the same coin: cells run out of order create hidden state, notebooks are hard to diff and review in git, and they encourage untested copy-pasted code, so mature ML projects move reusable code into Python modules and keep notebooks for exploration and reporting.
Read full answerHide full answer
Applies toJupyterLab 4.x, Jupyter Notebook 7.x, VS Code notebooks
Benefits
Benefit
Why it helps in ML
Interactive execution
Load data once, iterate on features and models quickly
Inline visualization
See plots and tables next to the code that made them
Narrative
Markdown explains decisions, useful for reviews and stakeholders
Rich outputs
DataFrames render as tables; widgets for interactive exploration
nbstripout, Jupytext (pair with .py files), nbdime
Poor code reuse
Move functions to a src/ package and import them
No tests
Test modules with pytest; keep notebooks thin
Long training jobs
Run scripts or jobs; use notebooks to analyze results
Code Example: A Notebook-Friendly Pattern
# Cell 1: configuration in one place
from dataclasses import dataclass
import numpy as np
import pandas as pd
@dataclass
class Config:
seed: int = 42
n_samples: int = 1000
cfg = Config()
rng = np.random.default_rng(cfg.seed)
# Cell 2: logic in functions (later moved to src/features.py)
def make_features(df: pd.DataFrame) -> pd.DataFrame:
return df.assign(ratio=df["a"] / (df["b"] + 1e-9), log_a=np.log1p(df["a"]))
# Cell 3: exploration
df = pd.DataFrame({"a": rng.exponential(2, cfg.n_samples), "b": rng.exponential(1, cfg.n_samples)})
features = make_features(df)
print(features.describe().round(2).loc[["mean", "std"]])
Useful notebook commands: %timeit for micro-benchmarks, %load_ext autoreload with %autoreload 2 to pick up edits in imported modules.
Common Pitfalls
Relying on variables defined in deleted or re-ordered cells.
Committing notebooks with large outputs or secrets in them.
Using a notebook as the production pipeline.
Interview Tips
Give the benefits (interactivity, visualization, narrative) and the risks (hidden state, versioning).
Explain how you move code from notebooks into tested modules.
18
What are TensorFlow and Keras and how are they related?
KnownRevise
TensorFlow is Google’s open-source deep learning platform: a tensor computation engine with automatic differentiation, GPU/TPU support and a deployment ecosystem (TF Serving, TensorFlow Lite / LiteRT, TensorFlow.js); Keras is a high-level neural network API for defining, training and evaluating models with layers, compile and fit. Keras became TensorFlow’s official high-level API as tf.keras in TensorFlow 2.0, and since Keras 3 (2023) it is again a multi-backend library that runs on TensorFlow, JAX or PyTorch, while TensorFlow 2.16+ uses Keras 3 by default. In short: Keras is how you usually write models, and TensorFlow is one engine that runs them.
import numpy as np
import keras
import tensorflow as tf
keras.utils.set_random_seed(42)
rng = np.random.default_rng(42)
X = rng.normal(size=(1000, 10)).astype("float32")
y = (X[:, 0] + X[:, 1] > 0).astype("int32")
model = keras.Sequential([ # Keras: high-level API
keras.Input(shape=(10,)),
keras.layers.Dense(32, activation="relu"),
keras.layers.Dropout(0.2),
keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
history = model.fit(X, y, epochs=5, batch_size=32, validation_split=0.2, verbose=0)
print("val accuracy:", round(history.history["val_accuracy"][-1], 3))
w = tf.Variable(3.0) # TensorFlow: low-level autodiff
with tf.GradientTape() as tape:
loss = w ** 2 + 2 * w
print("d(loss)/dw at w=3:", tape.gradient(loss, w).numpy()) # 2*3 + 2 = 8.0
print("Keras backend:", keras.backend.backend()) # tensorflow
Common Pitfalls
Mixing standalone keras and tf.keras imports from different versions in one project.
Following TensorFlow 1.x tutorials (sessions, placeholders).
Assuming Keras is TensorFlow-only; Keras 3 runs on JAX and PyTorch too.
Interview Tips
Explain Keras as the API and TensorFlow as an engine and ecosystem.
Mention compile/fit vs GradientTape for custom training.
19
What is data cleaning and why is it important in machine learning?
KnownRevise
Data cleaning is the process of finding and fixing problems in raw data before modelling: missing values, duplicates, inconsistent formats and category spellings, wrong data types, impossible or outlier values, mislabelled targets, and leakage columns that wouldn’t be available at prediction time. It matters because models learn whatever is in the data: errors become learned patterns (“garbage in, garbage out”), many algorithms fail outright on missing or non-numeric values, and leakage produces models that look excellent in testing and fail in production. In practice it often takes more time than modelling and gives bigger accuracy gains.
In a real project, compute imputation values on the training split only (for example with scikit-learn’s SimpleImputer inside a pipeline) so that test data doesn’t leak into them.
Common Pitfalls
Dropping every row with any missing value, which can discard most of the data and bias it.
Cleaning with statistics from the whole dataset before splitting.
Removing “outliers” that are real and important (fraud, rare failures).
No record of the cleaning steps, so production data gets treated differently.
Interview Tips
List the problem types and a fix for each.
Stress leakage and fitting cleaning statistics on training data only.
20
What are the common data preprocessing steps for a machine learning model?
KnownRevise
Common preprocessing steps are: split the data first (train/validation/test, by time or group when needed), clean it (duplicates, types, impossible values), impute missing values, encode categorical features (one-hot, ordinal or target encoding), scale or normalize numeric features for models that need it, transform skewed features (log, Box-Cox), engineer new features, handle class imbalance if necessary, and optionally select features or reduce dimensions. Every step that learns from data (imputation values, scaler means, encoders) must be fitted on the training set only and applied to the rest, which scikit-learn’s Pipeline and ColumnTransformer enforce.
Distance- or gradient-based models (kNN, SVM, linear, neural nets)
StandardScaler, MinMaxScaler, RobustScaler
Skew transforms
Heavy-tailed features
FunctionTransformer(np.log1p), PowerTransformer
Feature engineering
Domain knowledge
Custom transformers, PolynomialFeatures
Imbalance
Rare classes
class_weight, resampling (imbalanced-learn)
Selection / reduction
Many features
SelectKBest, PCA
Code Example
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer, OneHotEncoder, StandardScaler
rng = np.random.default_rng(42)
n = 1000
df = pd.DataFrame({
"income": rng.lognormal(10, 0.8, n), # skewed
"age": rng.integers(18, 80, n).astype(float),
"region": rng.choice(["north", "south", "east", "west"], n),
})
df.loc[rng.choice(n, 50, replace=False), "age"] = np.nan
y = ((np.log(df["income"]) > 10.2) & (df["region"] != "south")).astype(int)
X_train, X_test, y_train, y_test = train_test_split(df, y, test_size=0.2, stratify=y, random_state=42)
preprocess = ColumnTransformer([
("income", Pipeline([("log", FunctionTransformer(np.log1p)), ("scale", StandardScaler())]), ["income"]),
("age", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), ["age"]),
("region", OneHotEncoder(handle_unknown="ignore", drop="if_binary"), ["region"]),
])
model = Pipeline([("prep", preprocess), ("clf", LogisticRegression(max_iter=1000))])
model.fit(X_train, y_train) # all statistics learned from training data only
print("test accuracy:", round(model.score(X_test, y_test), 3))
print(model.named_steps["prep"].get_feature_names_out())
# ['income__income' 'age__age' 'region__region_east' 'region__region_north' 'region__region_south' 'region__region_west']
Model-Dependent Choices
Tree-based models (random forests, gradient boosting) don’t need scaling and handle skew well.
Linear models, SVMs, kNN and neural networks need scaled features.
High-cardinality categories: target encoding or embeddings instead of one-hot.
Common Pitfalls
Preprocessing before the split (leakage).
Different preprocessing code paths in training and production; ship the fitted pipeline.
One-hot encoding high-cardinality IDs into thousands of sparse columns without need.
Interview Tips
List the steps in order, starting with the split.
Show a ColumnTransformer pipeline and explain why it prevents leakage.
21
How do you handle missing or corrupted data in a dataset using Python?
KnownRevise
First detect and understand the problem: count missing values per column (df.isna().mean()), find corrupted entries by trying to parse them (pd.to_numeric(..., errors="coerce"), pd.to_datetime(..., errors="coerce")) and by checking domain rules (negative ages, future dates), and ask why values are missing, since data missing for a reason carries information. Then treat it: drop rows or columns only when the loss is small or a column is mostly empty, convert corrupted values to NaN, impute (median or mode, KNN or iterative model-based imputation, forward-fill for time series), add missing-indicator features, or use models that accept missing values natively (HistGradientBoosting, XGBoost, LightGBM), fitting every imputer on the training split only.
Read full answerHide full answer
Applies topandas 2.x, scikit-learn 1.5+
Strategies
Situation
Strategy
Few rows missing at random
Drop rows (dropna)
Column mostly empty (> 60–80%)
Drop the column or keep only an indicator
Numeric, skewed
Median imputation + missing indicator
Categorical
Most frequent value or a new “missing” category
Related features available
KNNImputer, IterativeImputer
Time series
Forward-fill, interpolation
Tree boosting models
Leave NaN; models learn a default direction
Corrupted values
Coerce to NaN, then treat as missing
Code Example
import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.impute import KNNImputer, SimpleImputer
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(42)
n = 500
df = pd.DataFrame({
"temp": rng.normal(25, 5, n).round(1).astype(object),
"humidity": rng.uniform(30, 90, n).round(1),
"sensor": rng.choice(["A", "B", "C"], n).astype(object),
})
df["load"] = 2 * df["temp"].astype(float) + 0.5 * df["humidity"] + rng.normal(0, 2, n)
df.loc[rng.choice(n, 20, replace=False), "temp"] = "ERR" # corrupted readings
df.loc[rng.choice(n, 40, replace=False), "humidity"] = np.nan
df.loc[rng.choice(n, 15, replace=False), "sensor"] = None
df["temp"] = pd.to_numeric(df["temp"], errors="coerce") # "ERR" -> NaN
df.loc[~df["humidity"].between(0, 100), "humidity"] = np.nan # domain rule check
print(df.isna().sum().to_dict()) # {'temp': 20, 'humidity': 40, 'sensor': 15, 'load': 0}
X, y = df.drop(columns="load"), df["load"]
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
X_train, X_test = X_train.copy(), X_test.copy()
for frame in (X_train, X_test):
frame["humidity_missing"] = frame["humidity"].isna().astype(int) # keep the signal
cat_imp = SimpleImputer(strategy="most_frequent").fit(X_train[["sensor"]])
num_imp = KNNImputer(n_neighbors=5).fit(X_train[["temp", "humidity"]]) # fitted on train only
for frame in (X_train, X_test):
frame[["sensor"]] = cat_imp.transform(frame[["sensor"]])
frame[["temp", "humidity"]] = num_imp.transform(frame[["temp", "humidity"]])
print(X_train.isna().sum().sum(), X_test.isna().sum().sum()) # 0 0
# Alternative: a model that handles NaN natively (no imputation step)
native = HistGradientBoostingRegressor(random_state=42)
native.fit(X.loc[y_train.index, ["temp", "humidity"]], y_train)
print("R^2 with native NaN handling:", round(native.score(X.loc[y_test.index, ["temp", "humidity"]], y_test), 3))
Common Pitfalls
Imputing with statistics from the full dataset (leakage).
Filling with 0 when 0 is a meaningful value.
Dropping rows without checking whether missingness relates to the target.
Treating sentinel values (−999, “N/A”, “unknown”) as real data; convert them first.
Interview Tips
Separate detection, understanding why, and treatment.
Mention missing indicators and models with native NaN support.
22
What is feature scaling and why is it necessary?
KnownRevise
Feature scaling transforms numeric features onto comparable ranges, most commonly by standardization (subtract the mean and divide by the standard deviation, giving mean 0 and standard deviation 1) or min-max normalization (map to [0, 1]); robust scaling with the median and interquartile range is used when outliers are present. It is necessary for algorithms that depend on distances or gradient magnitudes: without it, a feature measured in thousands (income) dominates one measured in units (number of children) in kNN, k-means, SVMs and PCA, and gradient descent for linear models and neural networks converges slowly; regularization penalties also assume comparable scales. Tree-based models don’t need it, and the scaler must be fitted on training data only.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Scalers
Scaler
Formula
Use when
StandardScaler
(x − mean) / std
Default for linear models, SVMs, neural nets, PCA
MinMaxScaler
(x − min) / (max − min)
Bounded inputs (images, some neural nets)
RobustScaler
(x − median) / IQR
Outliers present
MaxAbsScaler
x / max(|x|)
Sparse data (keeps zeros)
PowerTransformer / log
Reduce skew, then standardize
Heavily skewed features
Which Models Need It
Needs scaling
Doesn’t
kNN, k-means, SVM, PCA
Decision trees, random forests
Linear/logistic regression with regularization
Gradient boosting (XGBoost, LightGBM)
Neural networks
Naive Bayes (mostly)
Code Example
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=600, n_features=5, n_informative=3, random_state=42)
X[:, 0] *= 1000 # one informative-or-not feature on a huge scale
X[:, 3] *= 0.01 # and one on a tiny scale
raw = cross_val_score(KNeighborsClassifier(), X, y, cv=5).mean()
scaled = cross_val_score(make_pipeline(StandardScaler(), KNeighborsClassifier()), X, y, cv=5).mean()
print(f"kNN accuracy without scaling: {raw:.3f}, with scaling: {scaled:.3f}")
# Scaling usually helps a lot here, because unscaled distances are dominated by feature 0
scaler = StandardScaler().fit(X[:480]) # fit on the training part only
X_test_scaled = scaler.transform(X[480:])
print(X_test_scaled.mean(axis=0).round(2)) # near 0, not exactly: test data uses training statistics
Common Pitfalls
Fitting the scaler on all data before splitting (leakage).
Scaling the target unintentionally, or forgetting to invert target scaling for predictions.
Min-max scaling with outliers, which squashes most values into a tiny range.
Scaling one-hot or binary columns without reason.
Interview Tips
Name standardization vs normalization and which models need them.
Mention fitting on training data inside a pipeline.
23
What is the difference between label encoding and one-hot encoding?
KnownRevise
Label (ordinal) encoding replaces each category with an integer (red → 0, green → 1, blue → 2) in a single column, while one-hot encoding creates one binary column per category with a 1 marking the row’s category. Label encoding is compact but implies an order and spacing between categories, so it suits truly ordinal features (low < medium < high) and tree-based models, which can split on the integers; one-hot encoding makes no order assumption and is the safe choice for nominal categories in linear models, SVMs and neural networks, at the cost of many columns for high-cardinality features. In scikit-learn, use OrdinalEncoder for features and LabelEncoder only for target labels.
Read full answerHide full answer
Applies toscikit-learn 1.5+, pandas 2.x
Comparison
Aspect
Label / ordinal encoding
One-hot encoding
Output
One integer column
k binary columns (k categories)
Implies order
Yes
No
Dimensionality
Low
Grows with categories
Good for
Ordinal features; tree models
Nominal features; linear models, neural nets
scikit-learn
OrdinalEncoder (features), LabelEncoder (target)
OneHotEncoder
High cardinality
Fine for trees
Many sparse columns; consider target encoding or embeddings
Label-encoding nominal features for linear models or kNN, which invents an order.
Using pd.get_dummies separately on train and test, giving mismatched columns; use a fitted OneHotEncoder.
The dummy-variable trap in unregularized linear regression (use drop="first" when needed).
Exploding dimensionality with IDs or zip codes.
Interview Tips
Explain the implied-order problem with an example.
Say which models tolerate integer codes and which need one-hot.
24
Why do we split data into training, validation and test sets?
KnownRevise
Splitting data separates the jobs of learning, choosing and judging: the training set is used to fit model parameters, the validation set to compare models and tune hyperparameters, and the test set, used once at the end, to give an unbiased estimate of performance on new data. If you tuned on the test set, its score would be optimistically biased because you’d have chosen the model that happens to fit it best. Typical splits are 60/20/20 or 70/15/15 (or train/test plus k-fold cross-validation on the training part), and the split must match how the model will be used: stratified for imbalanced classes, chronological for time series, and grouped when several rows belong to the same user or patient.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Roles
Set
Used for
Touched
Training
Fitting weights / trees
Every training run
Validation
Hyperparameters, early stopping, feature choices, model selection
Many times
Test
Final unbiased estimate
Once
Choosing the Split Type
Data
Split
i.i.d. rows, balanced
Random
Imbalanced classes
Stratified (stratify=y)
Time series
Chronological: train on the past, test on the future
Repeated entities (users, patients)
Group split (GroupShuffleSplit, GroupKFold)
Small datasets
Train/test + k-fold CV on train
Code Example
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, weights=[0.9, 0.1], random_state=42)
X_temp, X_test, y_temp, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.25, stratify=y_temp, random_state=42)
print(len(X_train), len(X_val), len(X_test)) # 600 200 200
print(round(y_train.mean(), 3), round(y_val.mean(), 3), round(y_test.mean(), 3)) # similar class ratios
best_C, best_score = None, -1.0
for C in [0.01, 0.1, 1, 10]: # tune on validation only
score = LogisticRegression(C=C, max_iter=1000).fit(X_train, y_train).score(X_val, y_val)
if score > best_score:
best_C, best_score = C, score
final = LogisticRegression(C=best_C, max_iter=1000).fit(X_temp, y_temp) # refit on train + validation
print(f"best C = {best_C}, test accuracy = {final.score(X_test, y_test):.3f}") # reported once
Common Pitfalls
Preprocessing (scaling, imputation, feature selection) fitted before splitting.
Random splits for time series or grouped data, which leak information.
Repeatedly checking the test set and adjusting, which turns it into a validation set.
Duplicate or near-duplicate rows across splits.
Interview Tips
Explain each set’s role and why the test set must stay untouched.
Mention stratified, time-based and group splits.
25
How do scikit-learn pipelines streamline preprocessing?
KnownRevise
A scikit-learn Pipeline chains preprocessing steps and a final model into one estimator, so a single fit learns every step in order on the training data and a single predict applies the same fitted transformations to new data; ColumnTransformer routes different columns to different preprocessing (scale numeric columns, one-hot encode categorical ones). Pipelines prevent data leakage in cross-validation (each fold re-fits the preprocessing on its own training part), make hyperparameter search over preprocessing and model settings possible with step__param names, and package everything into one object you can save and deploy.
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder, StandardScaler
rng = np.random.default_rng(42)
n = 600
df = pd.DataFrame({
"tenure": rng.integers(0, 72, n).astype(float),
"monthly": rng.normal(70, 25, n),
"contract": rng.choice(["monthly", "yearly", "two-year"], n),
"payment": rng.choice(["card", "bank", "cash"], n),
})
df.loc[rng.choice(n, 30, replace=False), "monthly"] = np.nan
y = ((df["contract"] == "monthly") & (df["tenure"] < 24)).astype(int)
numeric = Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())])
categorical = Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
("onehot", OneHotEncoder(handle_unknown="ignore"))])
prep = ColumnTransformer([
("num", numeric, make_column_selector(dtype_include="number")),
("cat", categorical, make_column_selector(dtype_include=object)),
])
pipe = Pipeline([("prep", prep), ("model", LogisticRegression(max_iter=1000))])
X_train, X_test, y_train, y_test = train_test_split(df, y, stratify=y, random_state=42)
grid = GridSearchCV(pipe, {
"prep__num__scale": [StandardScaler(), MinMaxScaler()], # tune preprocessing too
"model__C": [0.1, 1, 10],
}, cv=5)
grid.fit(X_train, y_train)
print(grid.best_params_)
print("test accuracy:", round(grid.score(X_test, y_test), 3))
print(grid.best_estimator_.predict(X_test.head(3))) # raw DataFrame in, predictions out
Common Pitfalls
Fitting a scaler outside the pipeline and then cross-validating (leaky scores).
Wrong parameter names in grids; check pipe.get_params().keys().
Custom transformers that don’t implement fit/transform correctly (inherit from BaseEstimator, TransformerMixin).
Interview Tips
Lead with leakage prevention in cross-validation.
Show ColumnTransformer and tuning with step__param names.
26
How do you handle categorical data in machine learning models?
KnownRevise
Most models need numbers, so categorical features are encoded according to their type, cardinality and the model: one-hot encoding for nominal features with few categories (especially for linear models and neural networks), ordinal encoding for ordered categories or for tree models, target (mean) encoding with cross-fitting or frequency encoding for high-cardinality features, and learned embeddings in neural networks. Some libraries handle categories natively: CatBoost, LightGBM, and scikit-learn’s HistGradientBoosting with categorical_features. Also plan for categories that are rare (group them as “other”) or unseen at prediction time (handle_unknown="ignore").
Group into “other” (min_frequency, max_categories)
Code Example
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, TargetEncoder
rng = np.random.default_rng(42)
n = 2000
cities = [f"city_{i}" for i in range(200)]
city_effect = dict(zip(cities, rng.normal(0, 1, len(cities))))
df = pd.DataFrame({
"city": rng.choice(cities, n), # high cardinality
"plan": rng.choice(["basic", "plus", "pro"], n), # few nominal levels
"rating": rng.choice(["low", "medium", "high"], n), # ordinal
})
logit = df["city"].map(city_effect) + (df["plan"] == "pro") * 1.0 + df["rating"].map({"low": -1, "medium": 0, "high": 1})
y = (logit + rng.normal(0, 0.5, n) > 0).astype(int)
prep = ColumnTransformer([
("city", TargetEncoder(random_state=42), ["city"]), # cross-fitted to avoid leakage
("plan", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
("rating", OrdinalEncoder(categories=[["low", "medium", "high"]]), ["rating"]),
])
linear = make_pipeline(prep, LogisticRegression(max_iter=1000))
print("logistic + mixed encodings:", round(cross_val_score(linear, df, y, cv=5).mean(), 3))
df_cat = df.astype("category")
native = HistGradientBoostingClassifier(categorical_features="from_dtype", random_state=42)
print("native categorical boosting:", round(cross_val_score(native, df_cat, y, cv=5).mean(), 3))
rare = OneHotEncoder(min_frequency=15, handle_unknown="infrequent_if_exist").fit(df[["city"]])
print("one-hot columns for city with rare grouping:", len(rare.get_feature_names_out()))
Common Pitfalls
Target encoding without cross-fitting, which leaks the target and overfits.
One-hot encoding thousands of IDs for a linear model without regularization.
Unseen categories in production crashing the encoder.
Inconsistent spellings creating fake categories.
Interview Tips
Choose encodings by cardinality, ordering and model type.
Mention target encoding leakage and native categorical support.
27
What is the process of building a machine learning model in Python?
KnownRevise
Building a model in Python follows a repeatable workflow: define the problem and success metric; collect and explore the data (pandas, seaborn); split it into train/validation/test sets before anything else; preprocess and engineer features inside a scikit-learn pipeline; train a simple baseline, then stronger models; tune hyperparameters with cross-validation; evaluate once on the test set with the chosen metric and error analysis; then save the pipeline (joblib or the framework’s format) and deploy and monitor it. The code is easiest to maintain when each stage is a function or module with a fixed random seed and configuration.
Read full answerHide full answer
Applies toPython 3.12, scikit-learn 1.5+
Workflow
Step
Python tools
1. Define problem, metric
Business goal → e.g. F1 or RMSE
2. Load and explore
pandas, seaborn, df.describe()
3. Split
train_test_split (stratified/time/group)
4. Preprocess + features
Pipeline, ColumnTransformer
5. Baseline
DummyClassifier, logistic regression
6. Better models
Random forest, gradient boosting
7. Tune
GridSearchCV, RandomizedSearchCV, Optuna
8. Evaluate
Test metrics, confusion matrix, error analysis
9. Save + deploy
joblib, FastAPI, batch jobs
10. Monitor
Drift and performance tracking, retraining
Code Example
import joblib
from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, f1_score
from sklearn.model_selection import RandomizedSearchCV, cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
SEED = 42
def load_data():
data = load_breast_cancer(as_frame=True)
return data.frame.drop(columns="target"), data.frame["target"]
def compare_models(X_train, y_train):
models = {
"baseline": DummyClassifier(strategy="most_frequent"),
"logistic": make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)),
"boosting": HistGradientBoostingClassifier(random_state=SEED),
}
for name, model in models.items():
score = cross_val_score(model, X_train, y_train, cv=5, scoring="f1").mean()
print(f"{name:9s} CV F1 = {score:.3f}")
def tune(X_train, y_train):
search = RandomizedSearchCV(
HistGradientBoostingClassifier(random_state=SEED),
{"learning_rate": [0.03, 0.1, 0.3], "max_depth": [3, 5, None], "l2_regularization": [0.0, 1.0]},
n_iter=8, cv=5, scoring="f1", random_state=SEED,
)
return search.fit(X_train, y_train)
X, y = load_data()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=SEED)
compare_models(X_train, y_train)
search = tune(X_train, y_train)
best = search.best_estimator_
print("best params:", search.best_params_)
print(f"test F1 = {f1_score(y_test, best.predict(X_test)):.3f}")
print(classification_report(y_test, best.predict(X_test), digits=3))
joblib.dump(best, "model.joblib")
Common Pitfalls
Skipping the baseline, so you can’t tell whether the model adds value.
Leakage from preprocessing before the split.
Choosing a metric that doesn’t match the business goal.
Notebook-only code that can’t be re-run.
Interview Tips
Walk through the steps in order, naming a tool for each.
Emphasize baselines, leakage prevention and a single final test evaluation.
28
How do you make sure a model is not overfitting?
KnownRevise
You detect overfitting by comparing training performance with performance on data the model hasn’t seen (a validation set, cross-validation folds or a final test set): a large gap, such as 99% training accuracy and 80% validation accuracy, or a validation loss that rises while training loss keeps falling, signals that the model has memorized noise. To prevent it, use more or augmented data, simpler models or fewer features, regularization (L1/L2, dropout, weight decay), tree limits (max depth, min samples per leaf), early stopping, ensembling, and proper validation that respects time order and groups so the estimate is honest.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Python 3.12
Detection
Check
Overfitting sign
Train vs validation score
Large gap
Learning curve (score vs training size)
Validation score far below training, gap shrinks slowly with data
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_validate, validation_curve
from sklearn.tree import DecisionTreeClassifier
X, y = make_classification(n_samples=600, n_features=20, n_informative=5, flip_y=0.1, random_state=42)
for depth in [None, 4]:
res = cross_validate(DecisionTreeClassifier(max_depth=depth, random_state=42), X, y,
cv=5, return_train_score=True)
print(f"max_depth={depth}: train {res['train_score'].mean():.3f}, validation {res['test_score'].mean():.3f}")
# The unlimited tree scores 1.000 on training data with a big validation gap; depth 4 narrows the gap
depths = np.arange(1, 16)
train_scores, val_scores = validation_curve(
DecisionTreeClassifier(random_state=42), X, y, param_name="max_depth", param_range=depths, cv=5)
best = int(depths[val_scores.mean(axis=1).argmax()])
print("depth with the best validation score:", best)
Common Pitfalls
Judging only on training accuracy.
Tuning against the test set until it overfits too.
Leakage that hides overfitting by making validation look good.
Interview Tips
Explain the train-validation gap and learning/validation curves.
List several remedies and match them to the model type.
29
What is cross-validation and where does it fit in model training?
KnownRevise
Cross-validation estimates how well a model generalizes by repeatedly splitting the training data into training and validation parts: in k-fold cross-validation the data is divided into k folds, the model is trained on k − 1 of them and scored on the remaining one, and this repeats k times so every row is used for validation exactly once; the mean and spread of the k scores give a more reliable estimate than a single split. It sits between data splitting and final testing: you hold out a test set first, then use cross-validation on the training data to compare models, select features and tune hyperparameters (as GridSearchCV does), and finally evaluate the chosen model once on the test set.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Variants
Variant
Use when
KFold (k = 5 or 10)
General regression / balanced data
StratifiedKFold
Classification, especially imbalanced
GroupKFold
Several rows per user/patient; keep groups together
TimeSeriesSplit
Ordered data: train on the past, validate on the future
RepeatedStratifiedKFold
Small datasets; reduce variance of the estimate
Leave-one-out
Very small datasets (expensive, high variance)
Nested CV
Unbiased estimate when tuning hyperparameters
Code Example
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV, StratifiedKFold, cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)) # scaler re-fit in every fold
scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc")
print("fold AUCs:", np.round(scores, 3), f"mean {scores.mean():.3f} ± {scores.std():.3f}")
search = GridSearchCV(pipe, {"logisticregression__C": [0.01, 0.1, 1, 10]}, cv=cv, scoring="roc_auc")
search.fit(X_train, y_train) # CV used for tuning
print("best C:", search.best_params_["logisticregression__C"])
print(f"test AUC (once): {search.score(X_test, y_test):.3f}")
outer = cross_val_score(search, X_train, y_train, cv=StratifiedKFold(5, shuffle=True, random_state=0),
scoring="roc_auc") # nested CV: tuning inside each outer fold
print(f"nested CV AUC: {outer.mean():.3f}")
Common Pitfalls
Preprocessing the whole dataset before cross-validation (leakage); put it in a pipeline.
Shuffled k-fold on time series or grouped data.
Reporting the best CV score from a large search as the expected performance (optimistic); use nested CV or the test set.
Interview Tips
Explain k-fold with a diagram-like description and where it sits in the workflow.
Mention stratified, group and time-series variants.
30
What is the bias-variance tradeoff in machine learning?
KnownRevise
The bias-variance tradeoff describes two sources of prediction error that move in opposite directions as model complexity changes: bias is error from overly simple assumptions (the model can’t capture the true pattern, so it underfits), and variance is error from sensitivity to the particular training sample (the model fits noise, so it overfits and changes a lot with different data). For squared error, expected error = bias² + variance + irreducible noise; increasing complexity lowers bias but raises variance, so the best model sits at the complexity that minimizes their sum, found in practice with validation data or cross-validation, and regularization, more data or ensembling shift the balance.
Read full answerHide full answer
Applies togeneral concepts; example with scikit-learn 1.5+
Comparison
Aspect
High bias
High variance
Symptom
Underfitting: poor train and validation scores
Overfitting: great train, poor validation
Example
Linear model on a curved pattern
Deep unpruned tree, high-degree polynomial
Fixes
More features, more complex model, less regularization
More data, regularization, simpler model, bagging
Code Example: Measuring Both
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
rng = np.random.default_rng(42)
true_f = np.sin
x_test = np.linspace(0, 3, 50)
for degree in [1, 4, 15]:
preds = []
for _ in range(200): # many training sets from the same process
x = rng.uniform(0, 3, 30)
y = true_f(x) + rng.normal(0, 0.3, 30)
model = make_pipeline(PolynomialFeatures(degree), LinearRegression()).fit(x[:, None], y)
preds.append(model.predict(x_test[:, None]))
preds = np.array(preds)
bias_sq = float(((preds.mean(axis=0) - true_f(x_test)) ** 2).mean())
variance = float(preds.var(axis=0).mean())
print(f"degree {degree:2d}: bias^2 = {bias_sq:.4f}, variance = {variance:.4f}")
# Degree 1: high bias, low variance. Degree 15: low bias, much higher variance. Degree 4 balances them.
Ways to Manage It
Regularization (L1/L2, dropout) adds a little bias to cut variance.
Bagging and random forests average many high-variance models to reduce variance.
Boosting combines weak, high-bias learners to reduce bias.
More data reduces variance without adding bias.
Modern Note
Very large neural networks can show “double descent”: test error falls again past the point where they fit the training data perfectly, so the classic U-shaped curve isn’t the whole story for overparameterized models.
Common Pitfalls
Treating bias and variance as properties you can read from one training run (they’re defined over many possible training sets).
Assuming more complexity always means overfitting.
Interview Tips
Give the decomposition and the underfit/overfit symptoms.
Pair each side with remedies.
31
What steps do you take to improve a model's accuracy?
KnownRevise
Improve a model systematically, starting from evidence rather than trying random changes: confirm the metric and validation setup are right (no leakage, a split that mirrors production), compare with a baseline, diagnose whether the model is underfitting or overfitting with learning curves, then act on the biggest weakness, usually data first (more or cleaner data, fixed labels, better features), then model choice and hyperparameter tuning, then ensembling and threshold tuning, and finally error analysis on the examples it gets wrong. Each change should be measured with the same cross-validation so improvements are real and not noise.
Read full answerHide full answer
Applies toPython 3.12, scikit-learn 1.5+
A Step-by-Step Approach
Step
Action
1. Check evaluation
Right metric, leakage-free validation, enough validation data
2. Baseline
Dummy model, simple linear model
3. Diagnose
Train vs validation gap; learning curve
4. Data quality
Clean errors, fix labels, handle missing values and outliers
5. Features
Domain features, interactions, aggregates, encodings, text/date features
6. Model choice
Try gradient boosting for tabular, pre-trained networks for images/text
7. Tuning
Random/Bayesian search with CV
8. Imbalance / threshold
Class weights, tuned decision threshold
9. Ensembles
Averaging or stacking diverse models
10. Error analysis
Inspect worst errors by segment; feed insights back to steps 4–5
Code Example: Measuring Each Change
import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import Ridge
from sklearn.model_selection import KFold, cross_val_score
rng = np.random.default_rng(42)
n = 2000
df = pd.DataFrame({"length": rng.uniform(1, 10, n), "width": rng.uniform(1, 10, n),
"material": rng.integers(0, 3, n)})
price = 5 * df["length"] * df["width"] * (1 + 0.5 * df["material"]) + rng.normal(0, 10, n)
cv = KFold(5, shuffle=True, random_state=42)
def score(model, X):
return -cross_val_score(model, X, price, cv=cv, scoring="neg_root_mean_squared_error").mean()
base = df[["length", "width", "material"]]
print(f"1) ridge, raw features: RMSE {score(Ridge(), base):6.1f}")
featured = base.assign(area=df["length"] * df["width"])
featured = featured.assign(area_x_material=featured["area"] * featured["material"])
print(f"2) ridge, + domain features: RMSE {score(Ridge(), featured):6.1f}")
print(f"3) boosting, raw features: RMSE {score(HistGradientBoostingRegressor(random_state=42), base):6.1f}")
print(f"4) boosting, + domain features: RMSE {score(HistGradientBoostingRegressor(random_state=42), featured):6.1f}")
# The domain features (area and its interaction with material) give the biggest single improvement
Common Pitfalls
Tuning hyperparameters before fixing data and features, where most gains usually are.
Accepting improvements smaller than the cross-validation noise.
Optimizing accuracy when the business cares about recall, cost or calibration.
Overfitting the validation set by trying hundreds of variants.
Interview Tips
Present a diagnosis-first process, not a list of tricks.
Stress data and feature work, then tuning and ensembles, measured consistently.
32
What are hyperparameters and how do you tune them?
KnownRevise
Hyperparameters are settings chosen before training that control how a model learns or how complex it can be, such as the learning rate, the number and depth of trees, the regularization strength, or the number of neighbours in kNN; unlike parameters (weights, split thresholds), they are not learned from the data by the training algorithm itself. You tune them by searching over candidate values and scoring each with cross-validation on the training data: grid search for small spaces, random search for larger ones (it finds good regions with far fewer trials), and Bayesian optimization (Optuna, scikit-optimize) or successive halving to focus the budget on promising settings; the final model is then checked once on the test set.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Optuna 3.x/4.x
Parameters vs Hyperparameters
Parameters
Hyperparameters
Set by
Training algorithm
You (or a search)
Examples
Linear weights, tree splits, neural network weights
learning_rate, max_depth, C, n_estimators, alpha
Selected with
Training data
Validation / cross-validation
Search Methods
Method
How
Best for
Grid search
All combinations
Few hyperparameters with few values
Random search
Random samples from distributions
Larger spaces, limited budget
Successive halving
Many configs on little data, keep the best
Expensive models
Bayesian (TPE, Gaussian processes)
Model the score, sample promising regions
Expensive evaluations
Early stopping
Stop iterations when validation stops improving
Boosting, neural networks
Code Example
import optuna
from scipy.stats import randint
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RandomizedSearchCV, cross_val_score, train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
random_search = RandomizedSearchCV(
RandomForestClassifier(random_state=42),
{"n_estimators": randint(100, 500), "max_depth": randint(2, 20),
"min_samples_leaf": randint(1, 10), "max_features": ["sqrt", "log2", None]},
n_iter=20, cv=5, scoring="roc_auc", random_state=42, n_jobs=-1,
)
random_search.fit(X_train, y_train)
print("random search:", random_search.best_params_, round(random_search.best_score_, 4))
def objective(trial):
model = RandomForestClassifier(
n_estimators=trial.suggest_int("n_estimators", 100, 500),
max_depth=trial.suggest_int("max_depth", 2, 20),
min_samples_leaf=trial.suggest_int("min_samples_leaf", 1, 10),
max_features=trial.suggest_categorical("max_features", ["sqrt", "log2"]),
random_state=42, n_jobs=-1,
)
return cross_val_score(model, X_train, y_train, cv=5, scoring="roc_auc").mean()
optuna.logging.set_verbosity(optuna.logging.WARNING)
study = optuna.create_study(direction="maximize", sampler=optuna.samplers.TPESampler(seed=42))
study.optimize(objective, n_trials=20)
print("optuna:", study.best_params, round(study.best_value, 4))
print("test AUC of the random-search model:", round(random_search.score(X_test, y_test), 4))
# For learning rates or C, sample on a log scale: scipy.stats.loguniform(1e-4, 1e-1)
Common Pitfalls
Tuning on the test set.
Linear ranges for scale-like hyperparameters (use log scales for learning rate, C, alpha).
Searching too many hyperparameters at once with a tiny budget.
Ignoring the variance across CV folds when comparing close scores.
Interview Tips
Define the difference from parameters with examples.
Explain grid vs random vs Bayesian search and why random beats grid in big spaces.
33
How do ensemble methods work? Give an example where they are useful.
KnownRevise
Ensemble methods combine the predictions of several models so their errors partly cancel out, producing a model that is more accurate and more stable than any single member; this works best when the members are individually decent and make different mistakes. The main families are bagging (train many high-variance models such as deep trees on bootstrap samples and average them; random forests add random feature selection), boosting (train models sequentially, each correcting the previous ones’ errors; gradient boosting, XGBoost, LightGBM), and stacking or voting (combine different model types through a vote, an average or a meta-model). A typical use is tabular credit-risk or churn prediction, where gradient-boosted trees, often blended with a linear model, are hard to beat.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Families
Method
How it combines
Main effect
Examples
Bagging
Average models trained on bootstrap samples
Reduces variance
BaggingClassifier, RandomForestClassifier
Boosting
Sequential models fit to previous errors
Reduces bias (and variance)
HistGradientBoostingClassifier, XGBoost, LightGBM
Voting
Majority vote or averaged probabilities
Combines diverse strengths
VotingClassifier
Stacking
Meta-model learns how to weight base models
Often best accuracy
StackingClassifier
Code Example: Churn-Style Classification
from sklearn.datasets import make_classification
from sklearn.ensemble import (HistGradientBoostingClassifier, RandomForestClassifier, StackingClassifier,
VotingClassifier)
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.tree import DecisionTreeClassifier
X, y = make_classification(n_samples=2000, n_features=20, n_informative=8, weights=[0.8, 0.2],
flip_y=0.05, random_state=42)
logreg = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
models = {
"single tree": DecisionTreeClassifier(random_state=42),
"logistic": logreg,
"random forest (bagging)": RandomForestClassifier(n_estimators=300, random_state=42, n_jobs=-1),
"gradient boosting": HistGradientBoostingClassifier(random_state=42),
"soft voting": VotingClassifier([("lr", logreg), ("rf", RandomForestClassifier(n_estimators=300, random_state=42)),
("gb", HistGradientBoostingClassifier(random_state=42))], voting="soft"),
"stacking": StackingClassifier([("rf", RandomForestClassifier(n_estimators=300, random_state=42)),
("gb", HistGradientBoostingClassifier(random_state=42))],
final_estimator=LogisticRegression(), cv=5),
}
for name, model in models.items():
auc = cross_val_score(model, X, y, cv=5, scoring="roc_auc").mean()
print(f"{name:24s} AUC {auc:.3f}")
# The ensembles typically beat the single tree by a wide margin and the logistic model by a smaller one
When Ensembles Are Useful
Tabular business problems (churn, fraud, credit risk, pricing).
Competitions, where small accuracy gains matter.
Noisy data, where averaging reduces variance.
Less useful when interpretability, latency or memory limits dominate.
Common Pitfalls
Combining near-identical models (no diversity, little gain).
Stacking without out-of-fold predictions, which leaks training labels into the meta-model.
Ignoring the added complexity and serving cost.
Interview Tips
Explain bagging (variance) vs boosting (bias) and stacking.
Give a concrete use case and mention diversity as the key ingredient.
34
How do you implement gradient descent in Python?
KnownRevise
Gradient descent minimizes a loss by repeatedly moving the parameters a small step in the direction opposite to the loss’s gradient: θ ← θ − η·∇L(θ), where η is the learning rate. For linear regression with mean squared error, the gradient is ∇L = (2/n)·Xᵀ(Xθ − y), which NumPy computes in one vectorized line; the loop runs until the loss stops improving or a step limit is reached, and the result can be checked against the closed-form least-squares solution.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Code Example
import numpy as np
rng = np.random.default_rng(42)
n = 500
X = rng.normal(size=(n, 3))
true_theta = np.array([3.0, -2.0, 0.5])
y = X @ true_theta + 1.0 + rng.normal(0, 0.1, n)
def gradient_descent(X, y, lr=0.1, n_iters=1000, tol=1e-10):
X_b = np.column_stack([np.ones(len(X)), X]) # add the intercept column
theta = np.zeros(X_b.shape[1])
history = []
for i in range(n_iters):
residual = X_b @ theta - y
loss = float(np.mean(residual ** 2))
history.append(loss)
grad = (2 / len(y)) * X_b.T @ residual # vectorized gradient
theta -= lr * grad
if i > 0 and abs(history[-2] - history[-1]) < tol:
break
return theta, history
theta, history = gradient_descent(X, y)
print("gradient descent:", theta.round(3)) # close to [1.0, 3.0, -2.0, 0.5]
print("iterations:", len(history), "final MSE:", round(history[-1], 4)) # MSE near the noise variance 0.01
X_b = np.column_stack([np.ones(n), X])
closed_form, *_ = np.linalg.lstsq(X_b, y, rcond=None)
print("closed form: ", closed_form.round(3))
print("match:", np.allclose(theta, closed_form, atol=1e-4)) # True
for lr in [0.001, 0.1, 1.1]: # learning rate matters
_, h = gradient_descent(X, y, lr=lr, n_iters=200)
print(f"lr={lr}: loss after {len(h)} steps = {h[-1]:.3g}")
# 0.001 is still far from converged, 0.1 converges, 1.1 diverges (loss explodes)
Variants
Variant
Gradient computed on
Trade-off
Batch GD
All samples
Stable, slow per step on big data
Stochastic GD
One sample
Noisy, cheap steps
Mini-batch GD
32–512 samples
Standard in deep learning
Momentum / Adam
Mini-batches + running averages
Faster, less tuning
Common Pitfalls
Unscaled features, which make the loss surface elongated and convergence slow.
A learning rate that is too large (divergence) or too small (no progress).
Forgetting the intercept column.
Python loops over samples instead of vectorized gradients.
Interview Tips
Write the update rule and the MSE gradient.
Verify against the closed-form solution and discuss the learning rate.
35
How do you assess a model's performance? Name at least three metrics.
KnownRevise
Assess a model on data it hasn’t trained on (cross-validation and a held-out test set) with metrics that match the task and the business goal: for classification, accuracy, precision, recall, F1, ROC-AUC, PR-AUC and log loss; for regression, MAE, RMSE, R² and MAPE; and compare every number with a simple baseline. Beyond single numbers, check the confusion matrix, calibration of probabilities, performance per segment (fairness and robustness), and the cost of errors in the real application, for example missing a fraud case versus flagging a legitimate payment.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Classification Metrics
Metric
Meaning
Use when
Accuracy
Share of correct predictions
Balanced classes, equal error costs
Precision
Of predicted positives, share truly positive
False positives are costly
Recall
Of actual positives, share found
Missing positives is costly
F1
Harmonic mean of precision and recall
Imbalanced data, one balanced score
ROC-AUC
Ranking quality across thresholds
Comparing models independent of threshold
PR-AUC
Precision-recall trade-off
Rare positive class
Log loss / Brier
Quality of probabilities
Calibrated probabilities needed
Regression Metrics
Metric
Meaning
MAE
Average absolute error, in target units; robust to outliers
RMSE
Penalizes large errors more
R²
Share of variance explained (1 = perfect, 0 = predicting the mean)
MAPE
Percentage error; unstable near zero targets
Code Example
from sklearn.datasets import make_classification, make_regression
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.metrics import (accuracy_score, average_precision_score, f1_score, mean_absolute_error,
precision_score, r2_score, recall_score, roc_auc_score, root_mean_squared_error)
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, weights=[0.9, 0.1], random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
for name, model in [("baseline", DummyClassifier(strategy="most_frequent")),
("forest", RandomForestClassifier(random_state=42))]:
model.fit(X_tr, y_tr)
pred, proba = model.predict(X_te), model.predict_proba(X_te)[:, 1]
print(f"{name:8s} acc {accuracy_score(y_te, pred):.3f} precision {precision_score(y_te, pred, zero_division=0):.3f} "
f"recall {recall_score(y_te, pred):.3f} F1 {f1_score(y_te, pred):.3f} "
f"ROC-AUC {roc_auc_score(y_te, proba):.3f} PR-AUC {average_precision_score(y_te, proba):.3f}")
# The baseline gets about 0.90 accuracy with zero recall: accuracy alone is misleading here
Xr, yr = make_regression(n_samples=1000, n_features=5, noise=15, random_state=42)
Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, random_state=42)
pr = RandomForestRegressor(random_state=42).fit(Xr_tr, yr_tr).predict(Xr_te)
print(f"MAE {mean_absolute_error(yr_te, pr):.1f} RMSE {root_mean_squared_error(yr_te, pr):.1f} R2 {r2_score(yr_te, pr):.3f}")
Common Pitfalls
Reporting accuracy on imbalanced data.
Evaluating on training data.
Choosing the metric after seeing results.
Ignoring the decision threshold (precision and recall depend on it).
Interview Tips
Name metrics for both classification and regression and when each fits.
Always compare against a baseline and connect the metric to error costs.
36
What is a confusion matrix and how do you interpret it?
KnownRevise
A confusion matrix is a table that counts a classifier’s predictions against the true labels: for binary classification it has four cells: true positives (TP, positives correctly flagged), false positives (FP, negatives wrongly flagged), false negatives (FN, positives missed) and true negatives (TN). Reading it shows which kinds of mistakes the model makes, not just how many, and every common metric comes from it: accuracy = (TP + TN) / total, precision = TP / (TP + FP), recall = TP / (TP + FN), specificity = TN / (TN + FP). For multi-class problems it is a k × k table whose off-diagonal cells show which classes get confused with each other.
Large FN: the model misses positives; lower the threshold or improve recall.
Large FP: too many false alarms; raise the threshold or improve precision.
Off-diagonal clusters in multi-class: classes the model can’t separate; look for better features or merge labels.
Common Pitfalls
Mixing up the axis convention (some textbooks put predictions on rows).
Judging an imbalanced problem by accuracy instead of the FN and FP cells.
Forgetting that the matrix depends on the decision threshold.
Interview Tips
Draw the 2 × 2 table and derive precision and recall from it.
Tie FP and FN to real costs in the application.
37
What are precision and recall in classification?
KnownRevise
Precision is the share of the model’s positive predictions that are actually positive, TP / (TP + FP), so it answers “when the model says yes, how often is it right?”; recall (also called sensitivity or true positive rate) is the share of actual positives the model finds, TP / (TP + FN), answering “of all the real positives, how many did it catch?” They trade off through the decision threshold: lowering it catches more positives (higher recall) but adds false alarms (lower precision). Which one matters more depends on error costs: spam filters favour precision, while cancer screening and fraud detection favour recall; F1, their harmonic mean, summarizes both.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Definitions
Metric
Formula
Question it answers
Prioritize when
Precision
TP / (TP + FP)
How trustworthy are positive predictions?
False positives are costly (spam, legal flags)
Recall
TP / (TP + FN)
How many positives are found?
Missing positives is costly (disease, fraud)
F1
2·P·R / (P + R)
Balance of both
Imbalanced data, single summary
Fβ
(1 + β²)·P·R / (β²·P + R)
Weighted balance (β > 1 favours recall)
Unequal costs
Hand Example
A screening model flags 80 patients; 60 truly have the condition, out of 100 who have it.
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, precision_recall_curve, precision_score, recall_score
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=3000, weights=[0.9, 0.1], class_sep=0.8, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
proba = LogisticRegression(max_iter=1000).fit(X_tr, y_tr).predict_proba(X_te)[:, 1]
for threshold in [0.2, 0.5, 0.8]:
pred = (proba >= threshold).astype(int)
print(f"threshold {threshold}: precision {precision_score(y_te, pred, zero_division=0):.3f}, "
f"recall {recall_score(y_te, pred):.3f}, F1 {f1_score(y_te, pred):.3f}")
# Raising the threshold increases precision and lowers recall
precision, recall, thresholds = precision_recall_curve(y_te, proba)
ok = recall[:-1] >= 0.8 # pick the most precise threshold with recall >= 0.8
best = int(np.argmax(np.where(ok, precision[:-1], -1)))
print(f"threshold for recall >= 0.8: {thresholds[best]:.3f} (precision {precision[best]:.3f})")
Common Pitfalls
Reporting precision without recall (or the reverse); either can be made perfect alone.
Using the default 0.5 threshold without checking the trade-off.
Averaging per-class scores without saying macro, micro or weighted.
Interview Tips
Give formulas, plain-language meanings and an application for each.
Explain the threshold trade-off and F1.
38
What is the ROC curve and what does AUC measure?
KnownRevise
The ROC (receiver operating characteristic) curve plots a classifier’s true positive rate (recall) against its false positive rate (FP / (FP + TN)) as the decision threshold sweeps from high to low, showing every trade-off between catching positives and raising false alarms; the area under it (ROC-AUC) summarizes the curve in one number between 0 and 1. AUC equals the probability that a randomly chosen positive example receives a higher score than a randomly chosen negative one: 0.5 is random guessing, 1.0 is perfect ranking. It is threshold-independent and insensitive to class balance, which makes it good for comparing models, but on rare-positive problems the precision-recall curve and PR-AUC are often more informative.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Reading the Curve
Point / shape
Meaning
(0, 0)
Threshold so high nothing is flagged
(1, 1)
Threshold so low everything is flagged
Diagonal
Random classifier (AUC 0.5)
Curve hugging the top-left
Strong separation
Below the diagonal
Worse than random (labels or scores inverted)
Code Example
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import RocCurveDisplay, roc_auc_score, roc_curve
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, n_informative=5, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
fig, ax = plt.subplots(figsize=(5, 5))
for model in [LogisticRegression(max_iter=1000), RandomForestClassifier(random_state=42)]:
model.fit(X_tr, y_tr)
RocCurveDisplay.from_estimator(model, X_te, y_te, ax=ax)
ax.plot([0, 1], [0, 1], "k--", label="random")
ax.legend()
fig.savefig("roc.png", dpi=120)
# AUC as a ranking probability, checked by hand on a tiny example
scores = np.array([0.9, 0.8, 0.6, 0.55, 0.4, 0.2])
labels = np.array([1, 1, 0, 1, 0, 0])
pairs = [(p, n) for p in scores[labels == 1] for n in scores[labels == 0]]
manual_auc = np.mean([p > n for p, n in pairs])
print(round(float(manual_auc), 4), round(roc_auc_score(labels, scores), 4)) # 0.8889 0.8889 (8 of 9 pairs)
fpr, tpr, thr = roc_curve(labels, scores)
print(np.c_[fpr, tpr].round(3).tolist())
The tiny example has 3 positives and 3 negatives, so 9 positive-negative pairs; only the pair (0.55 positive, 0.6 negative) is ranked wrongly, giving AUC = 8/9 ≈ 0.889.
ROC-AUC vs PR-AUC
ROC-AUC
PR-AUC
Axes
TPR vs FPR
Precision vs recall
Random baseline
0.5
Positive class rate
Rare positives
Can look optimistic
More sensitive to false positives
Common Pitfalls
Computing AUC from hard 0/1 predictions instead of scores.
Treating a high AUC as proof of good calibration (it measures ranking only).
Relying on ROC-AUC alone for 1%-positive problems.
Interview Tips
Define the axes, the threshold sweep and the ranking interpretation of AUC.
Mention PR-AUC for imbalanced data.
39
How does evaluation differ between supervised and unsupervised learning?
KnownRevise
In supervised learning, evaluation is direct: you compare predictions against known labels on held-out data with metrics such as accuracy, F1, ROC-AUC, RMSE or R². In unsupervised learning there are no true labels, so evaluation relies on internal measures of structure (silhouette score, Davies-Bouldin index, inertia for clustering; reconstruction error or explained variance for dimensionality reduction and autoencoders), external labels when some exist (adjusted Rand index, normalized mutual information), stability across samples or seeds, and above all whether the result is useful for the downstream task or makes sense to domain experts.
Stability (bootstrap, different seeds), downstream performance
Judged by
Prediction error on new data
Structure quality, usefulness, interpretability
Code Example
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, adjusted_rand_score, davies_bouldin_score, silhouette_score)
from sklearn.model_selection import train_test_split
X, y = make_blobs(n_samples=600, centers=4, n_features=6, cluster_std=1.5, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42) # supervised
clf = LogisticRegression(max_iter=1000).fit(X_tr, y_tr)
print("supervised accuracy:", round(accuracy_score(y_te, clf.predict(X_te)), 3))
for k in [2, 3, 4, 5, 6]: # unsupervised
labels = KMeans(n_clusters=k, n_init=10, random_state=42).fit_predict(X)
print(f"k={k}: silhouette {silhouette_score(X, labels):.3f}, Davies-Bouldin {davies_bouldin_score(X, labels):.3f}")
# The best silhouette (highest) and Davies-Bouldin (lowest) usually point to k = 4 here
labels4 = KMeans(n_clusters=4, n_init=10, random_state=42).fit_predict(X)
print("ARI against hidden labels:", round(adjusted_rand_score(y, labels4), 3)) # only possible when labels exist
pca = PCA().fit(X)
print("variance explained by 2 components:", round(float(pca.explained_variance_ratio_[:2].sum()), 3))
Common Pitfalls
Treating a good silhouette score as proof the clusters are meaningful for the business.
Comparing inertia across different numbers of clusters (it always decreases with k).
Evaluating clustering with accuracy (cluster IDs are arbitrary; use ARI or NMI).
Interview Tips
Contrast label-based metrics with internal and external clustering measures.
Stress downstream usefulness and stability for unsupervised results.
40
How do you use a learning curve to diagnose model performance?
KnownRevise
A learning curve plots training and validation scores against the number of training examples; its shape shows whether more data would help and whether the model underfits or overfits: if both curves converge at a poor score, the model has high bias (underfitting) and needs more capacity or better features, not more data; if the training score is high and the validation score is much lower but still rising, the model has high variance (overfitting) and more data or regularization should help. In scikit-learn, learning_curve (or LearningCurveDisplay) computes these scores with cross-validation.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Reading the Curves
Pattern
Diagnosis
Action
Both low, close together, flat
High bias (underfit)
More complex model, better features, less regularization
Train high, validation much lower, gap shrinking
High variance (overfit)
More data, regularization, simpler model
Both high, close together
Good fit
Ship or fine-tune
Validation still rising at the largest size
Data-limited
Collecting more data is worthwhile
Validation flat for the last sizes
Data-saturated
More data won’t help much
Code Example
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import learning_curve
from sklearn.tree import DecisionTreeClassifier
X, y = make_classification(n_samples=3000, n_features=20, n_informative=10, flip_y=0.05, random_state=42)
sizes = np.linspace(0.1, 1.0, 8)
fig, axes = plt.subplots(1, 2, figsize=(11, 4), sharey=True)
for ax, (name, model) in zip(axes, [("deep tree (high variance)", DecisionTreeClassifier(random_state=42)),
("logistic regression", LogisticRegression(max_iter=1000))]):
n, train_scores, val_scores = learning_curve(model, X, y, train_sizes=sizes, cv=5, n_jobs=-1)
ax.plot(n, train_scores.mean(axis=1), "o-", label="train")
ax.plot(n, val_scores.mean(axis=1), "o-", label="validation")
ax.fill_between(n, val_scores.mean(1) - val_scores.std(1), val_scores.mean(1) + val_scores.std(1), alpha=0.2)
ax.set(title=name, xlabel="training examples", ylabel="accuracy")
ax.legend()
gap = train_scores.mean(axis=1)[-1] - val_scores.mean(axis=1)[-1]
print(f"{name}: final train-validation gap {gap:.3f}")
fig.tight_layout()
fig.savefig("learning_curves.png", dpi=120)
# The deep tree shows a large gap (train accuracy 1.0); logistic regression shows two close curves
Learning Curve vs Validation Curve
Learning curve: score vs training set size; answers “will more data help?”
Validation curve: score vs a hyperparameter (e.g. tree depth); answers “what complexity is right?”
Common Pitfalls
Using a single split instead of cross-validation (noisy curves).
Reading curves on accuracy when the real metric is F1 or AUC.
Forgetting that a time-ordered dataset needs time-aware splits here too.
Interview Tips
Describe the two classic shapes and the action for each.
Mention that a still-rising validation curve justifies collecting more data.
41
What validation strategies exist, such as k-fold cross-validation?
KnownRevise
Validation strategies differ in how they carve training data into fit and evaluation parts: a simple hold-out split (fast, but noisy on small data), k-fold cross-validation (k rotations so every row is validated once), stratified k-fold (keeps class proportions in each fold), group k-fold (keeps all rows of a user, patient or session in the same fold), time-series splits (always validate on data after the training window), repeated k-fold (averages several shuffles), leave-one-out (k = n, for tiny datasets), and nested cross-validation (an outer loop for evaluation around an inner loop for tuning). The right choice mirrors how the model will meet new data in production; a mismatch, such as random folds on time-ordered data, leaks information and inflates scores.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Strategies
Strategy
scikit-learn
Use when
Hold-out
train_test_split
Large datasets, quick checks
k-fold
KFold(n_splits=5, shuffle=True)
General regression
Stratified k-fold
StratifiedKFold
Classification, imbalanced classes
Group k-fold
GroupKFold, StratifiedGroupKFold
Repeated entities
Time series
TimeSeriesSplit(gap=...)
Forecasting, ordered events
Repeated
RepeatedStratifiedKFold
Small data, stable estimates
Leave-one-out
LeaveOneOut
Very small data
Nested CV
cross_val_score(GridSearchCV(...), ...)
Unbiased estimate with tuning
Code Example
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GroupKFold, KFold, StratifiedKFold, TimeSeriesSplit, cross_val_score
rng = np.random.default_rng(42)
n_users, rows_per_user = 60, 10
groups = np.repeat(np.arange(n_users), rows_per_user)
user_trait = rng.normal(size=n_users)[groups] # each user has a stable trait
X = np.column_stack([user_trait + rng.normal(0, 0.1, len(groups)), rng.normal(size=(len(groups), 3))])
y = (rng.random(n_users) > 0.5).astype(int)[groups] # label depends only on the user
model = RandomForestClassifier(n_estimators=200, random_state=42)
random_cv = cross_val_score(model, X, y, cv=KFold(5, shuffle=True, random_state=42)).mean()
group_cv = cross_val_score(model, X, y, cv=GroupKFold(5), groups=groups).mean()
print(f"random k-fold: {random_cv:.3f} group k-fold: {group_cv:.3f}")
# Random folds put the same user in train and validation, so the model memorizes users and scores near 1.0;
# group folds show the honest result for new users, close to chance here
for fold, (tr, va) in enumerate(StratifiedKFold(3, shuffle=True, random_state=42).split(X, y)):
print(f"stratified fold {fold}: positive rate train {y[tr].mean():.2f}, val {y[va].mean():.2f}")
for tr, va in TimeSeriesSplit(n_splits=3, gap=5).split(np.arange(40)):
print(f"time split: train 0..{tr[-1]}, validate {va[0]}..{va[-1]}") # validation always later
Common Pitfalls
Random folds on grouped or temporal data.
Preprocessing outside the cross-validation loop.
Too few folds on small data (noisy) or too many on large data (slow, little gain).
Reporting the tuned CV score as the final estimate without nested CV or a test set.
Interview Tips
Match the split to the deployment scenario.
Give the group-leakage example and mention nested CV.
42
How do you normalize an array to the range [0, 1] in Python?
KnownRevise
Min-max normalization maps each value x to (x − min) / (max − min), so the smallest value becomes 0 and the largest 1; a robust function vectorizes this with NumPy, works per column for 2-D feature matrices, handles constant columns (where max − min = 0) without dividing by zero, ignores NaN values when finding the range, and can reuse the training minimum and maximum to transform new data. In real pipelines scikit-learn’s MinMaxScaler does the same, fitted on training data only.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Code Example
import numpy as np
from sklearn.preprocessing import MinMaxScaler
def min_max_normalize(x, axis=0, data_min=None, data_max=None):
"""Scale values to [0, 1] along `axis`.
Pass data_min/data_max from training data to transform new data consistently.
Constant columns become 0. NaN values are ignored when computing the range and stay NaN.
"""
x = np.asarray(x, dtype=float)
data_min = np.nanmin(x, axis=axis, keepdims=True) if data_min is None else data_min
data_max = np.nanmax(x, axis=axis, keepdims=True) if data_max is None else data_max
span = data_max - data_min
safe_span = np.where(span == 0, 1.0, span) # avoid division by zero
return (x - data_min) / safe_span, data_min, data_max
print(min_max_normalize([2, 4, 6, 10])[0]) # [0. 0.25 0.5 1. ]
X_train = np.array([[1.0, 200.0, 5.0],
[2.0, 400.0, 5.0],
[3.0, 300.0, 5.0]])
X_scaled, mins, maxs = min_max_normalize(X_train)
print(X_scaled)
# [[0. 0. 0. ]
# [0.5 1. 0. ]
# [1. 0.5 0. ]] the constant third column becomes 0
X_new = np.array([[4.0, 250.0, 5.0]])
print(min_max_normalize(X_new, data_min=mins, data_max=maxs)[0]) # [[1.5 0.25 0. ]]: outside [0, 1] is possible
print(min_max_normalize([1.0, np.nan, 3.0])[0]) # [0. nan 1. ]
sk = MinMaxScaler().fit(X_train)
print(np.allclose(sk.transform(X_train), X_scaled)) # True
Design Notes
Choice
Why
Vectorized NumPy
Fast, no Python loops
keepdims=True
Broadcasting works per column
Return min and max
Reuse on validation/test data
Safe span
Constant features don’t produce NaN or inf
Common Pitfalls
Computing min and max on the full dataset, including test data.
Expecting new data to stay in [0, 1]; clip if the model requires it.
Outliers squashing all other values toward 0; consider robust scaling.
Interview Tips
Write the formula, handle the zero-range case, and show reuse on new data.
Mention MinMaxScaler and fitting on training data only.
43
How do you build a Python class for a simple perceptron?
KnownRevise
A perceptron is the simplest linear classifier: it computes a weighted sum of inputs plus a bias, predicts class 1 if the sum is at least 0 and class 0 otherwise, and learns by the perceptron rule, adjusting weights only when it misclassifies an example: w ← w + η·(y − ŷ)·x and b ← b + η·(y − ŷ). A clean class structure stores the hyperparameters in __init__, learns in fit, exposes net_input and predict, and records the number of mistakes per epoch; it converges when the classes are linearly separable and never fully converges (it keeps oscillating) otherwise.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Code Example
import numpy as np
class Perceptron:
"""Binary perceptron classifier with labels 0 and 1."""
def __init__(self, learning_rate=0.1, n_epochs=50, random_state=42):
self.learning_rate = learning_rate
self.n_epochs = n_epochs
self.random_state = random_state
def fit(self, X, y):
X, y = np.asarray(X, dtype=float), np.asarray(y)
rng = np.random.default_rng(self.random_state)
self.weights_ = rng.normal(0, 0.01, X.shape[1])
self.bias_ = 0.0
self.errors_ = []
for _ in range(self.n_epochs):
mistakes = 0
for i in rng.permutation(len(X)): # shuffle each epoch
update = self.learning_rate * (y[i] - self.predict(X[i]))
if update != 0:
self.weights_ += update * X[i]
self.bias_ += update
mistakes += 1
self.errors_.append(mistakes)
if mistakes == 0: # converged on separable data
break
return self
def net_input(self, X):
return np.asarray(X, dtype=float) @ self.weights_ + self.bias_
def predict(self, X):
return np.where(self.net_input(X) >= 0.0, 1, 0)
def score(self, X, y):
return float(np.mean(self.predict(X) == np.asarray(y)))
X_and = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
model = Perceptron().fit(X_and, [0, 0, 0, 1]) # AND is linearly separable
print(model.predict(X_and).tolist(), model.errors_) # [0, 0, 0, 1], mistakes fall to 0
xor = Perceptron(n_epochs=20).fit(X_and, [0, 1, 1, 0]) # XOR is not
print("XOR accuracy:", xor.score(X_and, [0, 1, 1, 0]), "| mistakes in last epoch:", xor.errors_[-1])
# XOR never reaches 0 mistakes: a single perceptron can't separate it
rng = np.random.default_rng(42)
X = np.vstack([rng.normal(-2, 1, (50, 2)), rng.normal(2, 1, (50, 2))])
y = np.r_[np.zeros(50, int), np.ones(50, int)]
print("blobs accuracy:", Perceptron().fit(X, y).score(X, y)) # 1.0 or very close
Design Notes
Element
Purpose
Trailing underscore attributes (weights_)
scikit-learn convention for learned values
fit returns self
Allows Perceptron().fit(X, y).predict(...)
errors_
Monitors convergence
Early stop when mistakes = 0
Saves time on separable data
Common Pitfalls
Unscaled features with very different ranges, which slow convergence.
Expecting probabilities (a perceptron gives hard labels; use logistic regression for probabilities).
Using it on non-separable data without a fixed epoch limit.
Interview Tips
Write the update rule and explain why updates happen only on mistakes.
Mention the XOR limitation that motivated multi-layer networks.
44
How do you implement k-means clustering using only the Python standard library?
KnownRevise
K-means groups points into k clusters by alternating two steps until assignments stop changing: assign each point to its nearest centroid (by squared Euclidean distance), then move each centroid to the mean of the points assigned to it. With only the standard library you represent points as tuples, use random.Random(seed) for reproducible initialization (k-means++ seeding, which picks spread-out starting centroids, gives better results than purely random picks), compute distances with math.dist or a sum of squares, handle empty clusters by re-seeding them, and stop when centroids move less than a tolerance or a maximum number of iterations is reached.
Read full answerHide full answer
Applies toPython 3.12 (standard library only)
Code Example
import math
import random
def squared_distance(a, b):
return sum((x - y) ** 2 for x, y in zip(a, b))
def mean_point(points):
n = len(points)
return tuple(sum(coords) / n for coords in zip(*points))
def kmeans_plus_plus(points, k, rng):
centroids = [rng.choice(points)]
while len(centroids) < k:
d2 = [min(squared_distance(p, c) for c in centroids) for p in points]
centroids.append(rng.choices(points, weights=d2, k=1)[0]) # far points are more likely
return centroids
def kmeans(points, k, max_iter=100, tol=1e-6, seed=42):
rng = random.Random(seed)
centroids = kmeans_plus_plus(points, k, rng)
for iteration in range(1, max_iter + 1):
clusters = [[] for _ in range(k)]
for p in points:
nearest = min(range(k), key=lambda j: squared_distance(p, centroids[j]))
clusters[nearest].append(p)
new_centroids = [mean_point(c) if c else rng.choice(points) for c in clusters] # re-seed empty clusters
shift = max(math.dist(a, b) for a, b in zip(centroids, new_centroids))
centroids = new_centroids
if shift < tol:
break
labels = [min(range(k), key=lambda j: squared_distance(p, centroids[j])) for p in points]
inertia = sum(squared_distance(p, centroids[label]) for p, label in zip(points, labels))
return centroids, labels, inertia, iteration
rng = random.Random(42)
true_centers = [(0, 0), (6, 6), (0, 8)]
data = [(rng.gauss(cx, 1), rng.gauss(cy, 1)) for cx, cy in true_centers for _ in range(100)]
centroids, labels, inertia, n_iter = kmeans(data, k=3)
print("centroids:", sorted((round(x, 1), round(y, 1)) for x, y in centroids)) # near the true centers
print("cluster sizes:", sorted(labels.count(j) for j in range(3))) # about [100, 100, 100]
print("inertia:", round(inertia, 1), "| iterations:", n_iter)
for k in range(1, 6): # elbow method: inertia drops sharply until k = 3
print(k, round(kmeans(data, k)[2], 1))
Complexity and Notes
Each iteration costs O(n · k · d); NumPy or scikit-learn’s KMeans are far faster for real data.
The result depends on initialization; run several seeds and keep the lowest inertia.
K-means assumes roughly spherical, similar-size clusters; scale features first.
Common Pitfalls
Empty clusters crashing the mean computation.
Unscaled features dominating the distance.
Comparing inertia across k as if lower is always better (it always decreases with k).
Interview Tips
Describe the assign-update loop and the stopping rule.
Mention k-means++ initialization and the elbow or silhouette method for choosing k.
45
How do you perform linear regression on a dataset using NumPy?
KnownRevise
With NumPy you fit linear regression by adding a column of ones for the intercept to the feature matrix X and solving the least-squares problem min ‖Xβ − y‖², either with np.linalg.lstsq (numerically stable, the recommended way) or with the normal equation β = (XᵀX)⁻¹Xᵀy (solve it with np.linalg.solve rather than computing an inverse). Then predict with Xβ and evaluate with R² and RMSE on held-out data; the script below also computes coefficient standard errors to show how uncertain each estimate is.
Collinear features making coefficients unstable (check the condition number, consider ridge).
Interpreting coefficients causally.
Interview Tips
Write the normal equation and say why lstsq is preferred.
Show prediction, R² and a held-out evaluation.
46
How do you write a function that minimizes a cost function with gradient descent?
KnownRevise
A general gradient descent optimizer takes the cost function, its gradient (or a numerical approximation), a starting point and a learning rate, then repeats x ← x − η·∇f(x) until the gradient norm or the change in cost falls below a tolerance or an iteration limit is reached, recording the history so you can check convergence. Making it generic (functions passed in as arguments), adding optional momentum and a backtracking line search, and testing it on a function with a known minimum shows both the algorithm and good engineering.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Code Example
import numpy as np
def numerical_gradient(f, x, h=1e-6):
"""Central differences, useful when no analytic gradient is available."""
grad = np.zeros_like(x)
for i in range(len(x)):
step = np.zeros_like(x)
step[i] = h
grad[i] = (f(x + step) - f(x - step)) / (2 * h)
return grad
def gradient_descent(f, x0, grad=None, lr=0.1, momentum=0.0, tol=1e-8, max_iter=10_000):
"""Minimize f starting at x0. Returns (x, history of f values)."""
x = np.asarray(x0, dtype=float).copy()
velocity = np.zeros_like(x)
grad = grad or (lambda z: numerical_gradient(f, z))
history = [f(x)]
for _ in range(max_iter):
g = grad(x)
if np.linalg.norm(g) < tol:
break
velocity = momentum * velocity - lr * g
x = x + velocity
history.append(f(x))
if not np.isfinite(history[-1]):
raise FloatingPointError("diverged: lower the learning rate")
return x, history
# 1) A quadratic bowl with minimum at (3, -1)
def bowl(x):
return (x[0] - 3) ** 2 + 10 * (x[1] + 1) ** 2
def bowl_grad(x):
return np.array([2 * (x[0] - 3), 20 * (x[1] + 1)])
x_min, hist = gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.05)
print(x_min.round(4), len(hist)) # [ 3. -1.], converged
x_mom, hist_mom = gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.05, momentum=0.8)
print(x_mom.round(4), len(hist_mom)) # same minimum
# 2) Rosenbrock function (minimum at (1, 1)) with a numerical gradient
def rosenbrock(x):
return (1 - x[0]) ** 2 + 100 * (x[1] - x[0] ** 2) ** 2
x_r, hist_r = gradient_descent(rosenbrock, [-1.0, 1.0], lr=1e-3, momentum=0.9, max_iter=50_000)
print(x_r.round(3), f"final cost {hist_r[-1]:.2e}") # close to [1, 1]
# 3) Divergence is detected
try:
gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.2) # 0.2 * 20 > 2: unstable along x[1]
except FloatingPointError as err:
print(err)
Why the Last Call Diverges
Along x[1] the gradient is 20·(x[1] + 1), so each step multiplies the error by (1 − 20·lr). With lr = 0.2 that factor is −3, so the error triples in size every step; stability needs |1 − 20·lr| < 1, that is lr < 0.1.
Common Pitfalls
A single learning rate for badly scaled problems (use feature scaling, momentum or adaptive methods).
Stopping on a fixed number of steps without checking convergence.
Numerical gradients with too small or too large h (1e-6 to 1e-5 works for doubles).
Interview Tips
Write the generic loop with a stopping criterion.
Explain the learning-rate stability condition on a quadratic.
47
How do you read and clean a CSV file with pandas?
KnownRevise
A clean pandas workflow reads the CSV with explicit options (dtype, parse_dates, na_values for codes like “N/A” or “-”), inspects it (info, describe, isna().sum(), value_counts), standardizes column names, removes duplicates, fixes types with pd.to_numeric(..., errors="coerce") and pd.to_datetime, cleans text categories, handles missing values and impossible values, derives useful columns, and finally saves the result in a typed format such as Parquet. Writing these steps as a function (or a chain of .pipe calls) makes the cleaning repeatable on new files.
What happens to the sample rows: the duplicate 1002 row is removed, “N/A” and “-” become NaN, the −4 units become NaN and are filled with the median of the valid units (3, 2, 1 → 2), names and regions are title-cased, and order 1004 is dropped because its date can’t be parsed.
Letting pandas guess types, so numbers with stray text stay as strings.
Chained assignment instead of .loc.
Filling missing values before splitting data for modelling (fit imputation on training data).
Saving cleaned data back to CSV and losing dtypes.
Interview Tips
Present cleaning as a repeatable function.
Mention na_values, errors="coerce" and saving to Parquet.
48
How do you implement a decision tree from scratch in Python?
KnownRevise
A decision tree classifier from scratch recursively splits the data: at each node it tries every feature and candidate threshold, picks the split that most reduces impurity (Gini impurity 1 − Σp² or entropy), and recurses on the left and right subsets until a stopping rule applies (maximum depth, minimum samples, or a pure node), where it stores a leaf predicting the majority class. Prediction walks each sample from the root down to a leaf. A compact implementation uses a small Node dataclass, NumPy for the impurity calculations, and checks its accuracy against scikit-learn’s DecisionTreeClassifier.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Code Example
from dataclasses import dataclass
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
@dataclass
class Node:
feature: int | None = None
threshold: float | None = None
left: "Node | None" = None
right: "Node | None" = None
value: int | None = None # class label for leaves
def gini(y):
_, counts = np.unique(y, return_counts=True)
p = counts / counts.sum()
return 1.0 - np.sum(p ** 2)
class DecisionTree:
def __init__(self, max_depth=5, min_samples_split=2):
self.max_depth, self.min_samples_split = max_depth, min_samples_split
def fit(self, X, y):
self.root_ = self._grow(np.asarray(X, float), np.asarray(y), depth=0)
return self
def _best_split(self, X, y):
best_gain, best = 0.0, None
parent = gini(y)
for f in range(X.shape[1]):
values = np.unique(X[:, f])
for t in (values[:-1] + values[1:]) / 2: # midpoints between sorted values
left = X[:, f] <= t
n_left = left.sum()
child = (n_left * gini(y[left]) + (len(y) - n_left) * gini(y[~left])) / len(y)
if parent - child > best_gain:
best_gain, best = parent - child, (f, t)
return best
def _grow(self, X, y, depth):
majority = int(np.bincount(y).argmax())
if depth >= self.max_depth or len(y) < self.min_samples_split or len(np.unique(y)) == 1:
return Node(value=majority)
split = self._best_split(X, y)
if split is None:
return Node(value=majority)
f, t = split
mask = X[:, f] <= t
return Node(feature=f, threshold=t,
left=self._grow(X[mask], y[mask], depth + 1),
right=self._grow(X[~mask], y[~mask], depth + 1))
def _predict_one(self, x, node):
while node.value is None:
node = node.left if x[node.feature] <= node.threshold else node.right
return node.value
def predict(self, X):
return np.array([self._predict_one(x, self.root_) for x in np.asarray(X, float)])
print(round(gini(np.array([0, 0, 1, 1])), 3), round(gini(np.array([0, 0, 0, 0])), 3)) # 0.5 0.0
X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=42)
ours = DecisionTree(max_depth=3).fit(X_tr, y_tr)
sk = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_tr, y_tr)
print("from scratch accuracy:", round(float(np.mean(ours.predict(X_te) == y_te)), 3))
print("scikit-learn accuracy:", round(sk.score(X_te, y_te), 3)) # very similar
print("root split: feature", ours.root_.feature, "<=", round(ours.root_.threshold, 2))
Complexity and Extensions
Naive split search is O(features · thresholds · n) per node; sorting once per feature makes it much faster.
Regression trees replace Gini with variance (MSE) and leaves predict the mean.
Pruning (cost-complexity) and min_samples_leaf reduce overfitting.
Common Pitfalls
No stopping rule, giving one leaf per sample (overfitting).
Splitting on every unique value instead of midpoints.
Forgetting the case where no split improves impurity.
Interview Tips
Explain Gini/entropy, the greedy split search and stopping rules.
Compare against scikit-learn to show correctness.
49
How do you write a train/test split function in Python?
KnownRevise
A train/test split function shuffles row indices with a seeded random generator, takes the first fraction as the test set and the rest as training, and returns the matching feature and label subsets; useful options are stratification (preserving class proportions by splitting within each class), and a no-shuffle mode for time-ordered data. Using index arrays instead of shuffling the data itself works for both NumPy arrays and pandas DataFrames, and the result can be compared with scikit-learn’s train_test_split.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, pandas 2.x
Code Example
import numpy as np
import pandas as pd
def split_train_test(X, y, test_size=0.2, shuffle=True, stratify=None, seed=42):
"""Split X and y (arrays or DataFrames) into train and test parts.
test_size: fraction in (0, 1) or an integer number of rows.
stratify: labels to keep proportional across splits (usually y).
shuffle=False keeps the original order (time series: the test set is the last rows).
"""
n = len(X)
if len(y) != n:
raise ValueError("X and y must have the same length")
n_test = int(round(test_size * n)) if isinstance(test_size, float) else int(test_size)
if not 0 < n_test < n:
raise ValueError("test_size must leave rows in both splits")
rng = np.random.default_rng(seed)
if not shuffle:
idx = np.arange(n)
train_idx, test_idx = idx[:n - n_test], idx[n - n_test:]
elif stratify is None:
idx = rng.permutation(n)
test_idx, train_idx = idx[:n_test], idx[n_test:]
else:
labels = np.asarray(stratify)
test_parts = []
for cls in np.unique(labels):
cls_idx = rng.permutation(np.flatnonzero(labels == cls))
test_parts.append(cls_idx[:int(round(len(cls_idx) * n_test / n))])
test_idx = np.concatenate(test_parts)
train_idx = np.setdiff1d(np.arange(n), test_idx)
rng.shuffle(train_idx)
def take(data, rows):
return data.iloc[rows] if isinstance(data, (pd.DataFrame, pd.Series)) else np.asarray(data)[rows]
return take(X, train_idx), take(X, test_idx), take(y, train_idx), take(y, test_idx)
rng = np.random.default_rng(42)
X = rng.normal(size=(1000, 4))
y = (rng.random(1000) < 0.1).astype(int) # about 10% positives
X_tr, X_te, y_tr, y_te = split_train_test(X, y, test_size=0.2)
print(X_tr.shape, X_te.shape) # (800, 4) (200, 4)
_, _, ys_tr, ys_te = split_train_test(X, y, test_size=0.2, stratify=y)
print(round(y.mean(), 3), round(ys_tr.mean(), 3), round(ys_te.mean(), 3)) # nearly identical rates
df = pd.DataFrame(X, columns=list("abcd")).assign(day=np.arange(1000))
d_tr, d_te, _, _ = split_train_test(df, pd.Series(y), test_size=0.2, shuffle=False)
print(d_tr["day"].max(), d_te["day"].min()) # 799 800: test is strictly later
Common Pitfalls
Shuffling time-series data.
Splitting X and y with different random orders, which misaligns labels.
Rows from the same user or patient on both sides (use a group split).
Not seeding, so splits change every run.
Interview Tips
Shuffle indices with a seed, then index both X and y.
Mention stratification, time order and group splits.
50
How do you write a Python script that automates hyperparameter tuning with grid search?
KnownRevise
An automated grid search script defines the model as a pipeline (so preprocessing is re-fitted inside every fold), a parameter grid using step__param names, a cross-validation strategy and a scoring metric that matches the goal, then runs GridSearchCV in parallel (n_jobs=-1), reports the top configurations with their mean and standard deviation, evaluates the refitted best model once on a held-out test set, and saves both the model and the results. Command-line arguments or a config file make it reusable, and a fixed random seed makes it reproducible.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Python 3.12
Code Example (tune.py)
import argparse
import json
import joblib
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV, StratifiedKFold, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
PARAM_GRID = [
{"model__kernel": ["linear"], "model__C": [0.1, 1, 10]},
{"model__kernel": ["rbf"], "model__C": [0.1, 1, 10, 100], "model__gamma": ["scale", 0.01, 0.001]},
]
def parse_args():
parser = argparse.ArgumentParser(description="Grid search for an SVM classifier")
parser.add_argument("--scoring", default="f1")
parser.add_argument("--folds", type=int, default=5)
parser.add_argument("--seed", type=int, default=42)
parser.add_argument("--out", default="best_model.joblib")
return parser.parse_args()
def main():
args = parse_args()
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=args.seed)
pipe = Pipeline([("scale", StandardScaler()), ("model", SVC())])
search = GridSearchCV(
pipe, PARAM_GRID, scoring=args.scoring, n_jobs=-1, refit=True,
cv=StratifiedKFold(args.folds, shuffle=True, random_state=args.seed),
)
search.fit(X_train, y_train)
results = pd.DataFrame(search.cv_results_)
top = results.sort_values("rank_test_score")[["params", "mean_test_score", "std_test_score"]].head(5)
print(top.to_string(index=False))
print("best:", search.best_params_, f"CV {args.scoring} = {search.best_score_:.4f}")
y_pred = search.predict(X_test) # test set used once, at the end
print(classification_report(y_test, y_pred, digits=3))
joblib.dump(search.best_estimator_, args.out)
results.to_csv("grid_results.csv", index=False)
with open("best_params.json", "w") as f:
json.dump({k: v for k, v in search.best_params_.items()}, f, indent=2, default=str)
if __name__ == "__main__":
main()
Run with python tune.py --scoring roc_auc --folds 5. The grid has 3 + 12 = 15 combinations, so with 5 folds it trains 75 models plus one final refit.
When Grid Search Isn’t Enough
Situation
Alternative
Many hyperparameters
RandomizedSearchCV
Expensive models
HalvingGridSearchCV, Optuna with pruning
Continuous ranges
Random or Bayesian search with log-uniform distributions
Common Pitfalls
Scaling outside the pipeline (leakage into CV scores).
Huge grids that take hours for tiny gains.
Picking the best mean score while ignoring a large standard deviation.
Evaluating on the test set during the search.
Interview Tips
Show a pipeline, step__param grid names and the final single test evaluation.
Mention random and Bayesian search for larger spaces.
51
What if a model does well in training but poorly on new data?
KnownRevise
Good training performance with poor results on new data means the model is not generalizing, either because it overfits (memorizes noise) or because the new data differs from the training data; first confirm which: compare training, validation and production metrics, check for leakage that made validation look better than reality, and compare feature distributions between training and new data (data drift). For overfitting, simplify or regularize the model, get more or augmented data, reduce features, use early stopping and proper cross-validation; for drift or a mismatched split, retrain on recent, representative data, use validation splits that mirror deployment (time-based or group-based), and make sure preprocessing is identical in training and serving.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Python 3.12
Diagnosis
Check
Points to
Train ≫ validation score
Overfitting
Validation good, production bad
Leakage, drift, or training-serving skew
Feature distributions differ (KS test, PSI)
Data drift
Same user/time period in train and validation
Leaky split
Different preprocessing in production
Training-serving skew
Remedies
Cause
Fix
Overfitting
Regularization, simpler model, max_depth/min_samples_leaf, dropout, early stopping
Too little data
More data, augmentation, transfer learning
Too many features
Feature selection, dimensionality reduction
Leakage
Remove leaky features; fit preprocessing on training folds only
Drift
Retrain on recent data, monitor, time-based validation
Skew
Ship the fitted pipeline; one feature code path
Code Example
import numpy as np
from scipy.stats import ks_2samp
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_validate
X, y = make_classification(n_samples=800, n_features=30, n_informative=5, flip_y=0.1, random_state=42)
for params in [{}, {"max_depth": 6, "min_samples_leaf": 5, "max_features": 0.3}]:
res = cross_validate(RandomForestClassifier(random_state=42, **params), X, y, cv=5, return_train_score=True)
print(params or "default", f"train {res['train_score'].mean():.3f} validation {res['test_score'].mean():.3f}")
# The constrained forest gives up some training accuracy and narrows the gap
rng = np.random.default_rng(42)
train_feature = rng.normal(0, 1, 2000)
new_feature = rng.normal(0.4, 1.2, 2000) # production data has shifted
stat, p = ks_2samp(train_feature, new_feature)
print(f"KS statistic {stat:.3f}, p = {p:.1e} -> drift detected" if p < 0.01 else "no drift")
Common Pitfalls
Assuming it’s always overfitting and only adding regularization.
Re-tuning on the test set until it looks good.
Ignoring that the world changed (seasonality, new users, new products).
Interview Tips
Separate overfitting from distribution shift and leakage before fixing.
Mention monitoring and validation that mirrors production.
52
How would you approach feature selection in a large dataset?
KnownRevise
Approach feature selection in stages, cheapest first: remove useless features (constant or near-constant, mostly missing, duplicates, ID-like columns and anything that leaks the target), drop one of each highly correlated pair, then rank the rest with fast filter methods (mutual information, ANOVA F-test, chi-squared), use embedded methods that select while training (L1/Lasso, tree-based or permutation importance), and finish with wrapper methods (recursive feature elimination or sequential selection) on the shortlist if compute allows. Do selection inside cross-validation so it doesn’t leak into the evaluation, and judge each step by the validation score and by domain sense, not by importance numbers alone.
mutual_info_classif, f_classif, chi2 with SelectKBest
Low
Ignores feature interactions
Embedded
Lasso/L1 logistic, tree importances, SelectFromModel
Medium
Model-specific
Permutation importance
sklearn.inspection.permutation_importance
Medium
On validation data; less biased than impurity importance
Wrapper
RFECV, SequentialFeatureSelector
High
Best on a shortlist
Dimensionality reduction
PCA, embeddings
Medium
Creates new features, less interpretable
Code Example
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.feature_selection import RFECV, SelectFromModel, VarianceThreshold
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=1500, n_features=200, n_informative=10, n_redundant=10, random_state=42)
X = pd.DataFrame(X, columns=[f"f{i}" for i in range(200)])
X["constant"] = 1.0
X["dup_f0"] = X["f0"]
X1 = X.loc[:, VarianceThreshold(1e-8).fit(X).get_support()] # drop constant columns
corr = X1.corr().abs()
upper = corr.where(np.triu(np.ones(corr.shape, dtype=bool), k=1))
X2 = X1.drop(columns=[c for c in upper.columns if (upper[c] > 0.95).any()]) # drop near-duplicates
print(X.shape[1], "->", X1.shape[1], "->", X2.shape[1]) # 202 -> 201 -> about 200 (dup_f0 removed)
cv = StratifiedKFold(5, shuffle=True, random_state=42)
full = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
l1_select = make_pipeline(
StandardScaler(),
SelectFromModel(LogisticRegression(penalty="l1", C=0.05, solver="liblinear")), # selection inside CV
LogisticRegression(max_iter=2000),
)
print("all features AUC:", round(cross_val_score(full, X2, y, cv=cv, scoring="roc_auc").mean(), 3))
print("L1-selected AUC: ", round(cross_val_score(l1_select, X2, y, cv=cv, scoring="roc_auc").mean(), 3))
l1_select.fit(X2, y)
kept = X2.columns[l1_select.named_steps["selectfrommodel"].get_support()]
print("features kept by L1:", len(kept))
rfe = RFECV(LogisticRegression(max_iter=2000), step=5, cv=cv, scoring="roc_auc", min_features_to_select=5)
rfe.fit(StandardScaler().fit_transform(X2[kept]), y) # wrapper method on the shortlist
print("RFECV keeps:", rfe.n_features_)
Common Pitfalls
Selecting features on the full dataset and then cross-validating (optimistic scores).
Trusting impurity-based importances, which favour high-cardinality features.
Dropping a feature that looks weak alone but matters in interactions.
Removing features a regulator or the business needs to see.
Interview Tips
Describe a staged funnel from cheap filters to wrappers.
Stress doing selection inside cross-validation.
53
What strategies help with imbalanced datasets?
KnownRevise
For imbalanced datasets (for example 1% fraud), start by choosing the right evaluation (precision, recall, F1, PR-AUC, cost-based metrics, never plain accuracy) and stratified splits; then rebalance the learning with class weights (class_weight="balanced"), resampling (random undersampling of the majority, oversampling or SMOTE for the minority, applied only to training folds), or algorithms that cope well, such as gradient boosting with scale_pos_weight; and finally tune the decision threshold to the business trade-off. Often the cheapest large gains come from threshold tuning and better features rather than resampling, and probability calibration may be needed after reweighting.
PR-AUC, recall at fixed precision, F1, cost matrix
Accuracy misleads
Splitting
Stratified splits and folds
Keeps minority examples in every fold
Algorithm
class_weight="balanced", scale_pos_weight
No data duplication
Data
Undersampling, oversampling, SMOTE
Only on training data, inside CV
Decision
Threshold tuning
Matches the cost of FP vs FN
Problem framing
Anomaly detection, more minority data
For extreme imbalance
Code Example
import numpy as np
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import make_pipeline as make_imb_pipeline
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, f1_score, precision_recall_curve, recall_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=20_000, n_features=15, weights=[0.98, 0.02], class_sep=0.8, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, stratify=y, random_state=42)
models = {
"plain": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
"class weights": make_pipeline(StandardScaler(), LogisticRegression(class_weight="balanced", max_iter=1000)),
"SMOTE": make_imb_pipeline(StandardScaler(), SMOTE(random_state=42), LogisticRegression(max_iter=1000)),
}
for name, model in models.items():
model.fit(X_tr, y_tr) # SMOTE resamples only the training data
pred, proba = model.predict(X_te), model.predict_proba(X_te)[:, 1]
print(f"{name:13s} recall {recall_score(y_te, pred):.3f} F1 {f1_score(y_te, pred):.3f} "
f"PR-AUC {average_precision_score(y_te, proba):.3f}")
proba = models["plain"].predict_proba(X_te)[:, 1] # threshold tuning on the plain model
precision, recall, thresholds = precision_recall_curve(y_te, proba)
f1 = 2 * precision[:-1] * recall[:-1] / np.clip(precision[:-1] + recall[:-1], 1e-12, None)
best = int(np.argmax(f1))
print(f"best F1 threshold {thresholds[best]:.3f} -> F1 {f1[best]:.3f}")
# In practice choose the threshold on a validation set, not the test set
Common Pitfalls
Oversampling before splitting, which puts copies of the same minority rows in train and test.
Reporting accuracy.
Using reweighted probabilities as if they were calibrated.
Choosing the threshold on the test set.
Interview Tips
Cover metrics, algorithm-level, data-level and threshold strategies.
Warn about resampling leakage.
54
How does regularization work in linear models? Give a Python example.
KnownRevise
Regularization adds a penalty on the size of the coefficients to the linear model’s loss so the model can’t fit noise with extreme weights: ridge (L2) adds α·Σw², shrinking all coefficients smoothly toward zero and stabilizing correlated features; lasso (L1) adds α·Σ|w|, which drives some coefficients exactly to zero and so performs feature selection; elastic net mixes both. The strength α (or C = 1/α in scikit-learn’s logistic regression) trades a little bias for lower variance and is chosen by cross-validation, and features must be standardized first because the penalty treats all coefficients on the same scale.
Read full answerHide full answer
Applies toscikit-learn 1.5+
Comparison
Model
Penalty
Effect
Use when
Ridge
α·‖w‖₂²
Shrinks all weights; handles collinearity
Many correlated useful features
Lasso
α·‖w‖₁
Sets some weights to 0
Sparse signal, feature selection
Elastic net
α·(ρ‖w‖₁ + (1 − ρ)/2·‖w‖₂²)
Sparse and stable with correlated groups
Many correlated features, some irrelevant
Logistic regression
penalty="l2" / "l1", strength C (smaller = stronger)
Same ideas for classification
Code Example
import numpy as np
from sklearn.linear_model import ElasticNetCV, LassoCV, LinearRegression, RidgeCV
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(42)
n, p = 120, 60 # few samples, many features: easy to overfit
X = rng.normal(size=(n, p))
X[:, 1] = X[:, 0] + rng.normal(0, 0.05, n) # two highly correlated features
true_w = np.zeros(p)
true_w[[0, 2, 5]] = [3.0, -2.0, 1.5] # only 3 features matter
y = X @ true_w + rng.normal(0, 1.0, n)
alphas = np.logspace(-3, 2, 30)
models = {
"OLS": make_pipeline(StandardScaler(), LinearRegression()),
"Ridge": make_pipeline(StandardScaler(), RidgeCV(alphas=alphas)),
"Lasso": make_pipeline(StandardScaler(), LassoCV(alphas=alphas, cv=5, random_state=42, max_iter=50_000)),
"ElasticNet": make_pipeline(StandardScaler(), ElasticNetCV(l1_ratio=0.5, alphas=alphas, cv=5,
random_state=42, max_iter=50_000)),
}
for name, model in models.items():
r2 = cross_val_score(model, X, y, cv=5, scoring="r2").mean()
model.fit(X, y)
coef = model[-1].coef_
print(f"{name:10s} CV R^2 {r2:6.3f} non-zero coefs {int(np.sum(np.abs(coef) > 1e-6)):2d} "
f"largest |coef| {np.abs(coef).max():.2f}")
# OLS overfits with all 60 coefficients; Lasso keeps only a handful, close to the true 3
lasso = models["Lasso"][-1]
print("Lasso alpha:", round(float(lasso.alpha_), 4), "| top features:", np.argsort(-np.abs(lasso.coef_))[:4].tolist())
Choosing α
Use RidgeCV, LassoCV, ElasticNetCV or GridSearchCV with a log-spaced range.
Plot the regularization path (coefficients vs α) to see which features survive.
Common Pitfalls
Regularizing unscaled features, which penalizes features unevenly.
Reading lasso’s choice among correlated features as “the” important one (it picks somewhat arbitrarily).
Forgetting that in LogisticRegression a smaller C means stronger regularization.
Interview Tips
Write both penalties and explain sparsity vs shrinkage.
Mention scaling and choosing α by cross-validation.
55
What are the advantages of SGD over standard gradient descent?
KnownRevise
Stochastic gradient descent (SGD) updates the parameters using the gradient from one example or a small mini-batch instead of the full dataset, so each step is far cheaper, the model starts improving after seeing only a fraction of the data, memory use doesn’t grow with dataset size, it supports online and streaming learning, and its gradient noise can help escape shallow local minima and saddle points. Full-batch gradient descent gives exact, smooth gradients and stable convergence, but each step requires a pass over all the data, which is impractical for large datasets and deep learning; the price of SGD is noisy convergence, which is handled with learning rate schedules, momentum and mini-batches.
Read full answerHide full answer
Applies togeneral concepts; examples in Python 3.12 with NumPy 2.x and scikit-learn 1.5+
Comparison
Aspect
Batch gradient descent
SGD / mini-batch SGD
Gradient per step
All n samples
1 sample or a batch (32–512)
Cost per step
O(n)
O(batch)
Progress per epoch
1 update
n / batch updates
Memory
Whole dataset
One batch
Convergence path
Smooth
Noisy (needs decaying LR)
Online learning
No
Yes (partial_fit)
Escaping saddle points
Harder
Noise helps
Hardware use
—
Mini-batches use GPUs efficiently
Code Example
import time
import numpy as np
from sklearn.linear_model import SGDRegressor
rng = np.random.default_rng(42)
n, d = 200_000, 10
X = rng.normal(size=(n, d))
w_true = rng.normal(size=d)
y = X @ w_true + rng.normal(0, 0.1, n)
def batch_gd(epochs=20, lr=0.1):
w = np.zeros(d)
for _ in range(epochs): # one update per pass over all data
w -= lr * (2 / n) * X.T @ (X @ w - y)
return w
def minibatch_sgd(epochs=1, lr=0.05, batch=256):
w = np.zeros(d)
for _ in range(epochs):
order = rng.permutation(n)
for start in range(0, n, batch): # ~780 updates in one pass
idx = order[start:start + batch]
w -= lr * (2 / len(idx)) * X[idx].T @ (X[idx] @ w - y[idx])
return w
for name, fn in [("batch GD, 20 passes", batch_gd), ("mini-batch SGD, 1 pass", minibatch_sgd)]:
start = time.perf_counter()
w = fn()
print(f"{name:22s} error {np.linalg.norm(w - w_true):.4f} time {time.perf_counter() - start:.2f}s")
# One SGD pass usually gets closer to the true weights than 20 full-batch steps
online = SGDRegressor(learning_rate="invscaling", eta0=0.01, random_state=42)
for start in range(0, n, 10_000): # streaming chunks, like data arriving over time
online.partial_fit(X[start:start + 10_000], y[start:start + 10_000])
print("online SGD error:", round(float(np.linalg.norm(online.coef_ - w_true)), 4))
Common Pitfalls
A constant learning rate that keeps the solution bouncing around the minimum; decay it.
Not shuffling data between epochs.
Unscaled features, which make SGD unstable.
Interview Tips
Contrast cost per step, updates per epoch and noise.
Mention mini-batches as the practical middle ground and partial_fit for streaming.
56
How can you parallelize computations in Python for machine learning?
KnownRevise
Python ML code parallelizes at several levels: vectorized NumPy and pandas operations already run in optimized, often multi-threaded C code; libraries expose n_jobs=-1 (scikit-learn, XGBoost, LightGBM) to use all CPU cores for cross-validation, grid search and tree building; for your own CPU-bound functions you use processes (multiprocessing, concurrent.futures.ProcessPoolExecutor, joblib.Parallel) because the GIL stops threads from running Python bytecode in parallel; threads suit I/O-bound work such as downloads; Numba compiles numeric loops with prange; GPUs accelerate deep learning and RAPIDS; and Dask, Ray or Spark spread work across machines.
import time
from concurrent.futures import ThreadPoolExecutor
import numpy as np
from joblib import Parallel, delayed
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
def slow_feature(seed):
"""A CPU-bound pure-Python task, e.g. a simulation per customer."""
rng = np.random.default_rng(seed)
total = 0.0
for _ in range(200_000):
total += rng.random() ** 2
return total
def timed(label, fn):
start = time.perf_counter()
result = fn()
print(f"{label:28s} {time.perf_counter() - start:.2f}s")
return result
if __name__ == "__main__": # required for process pools on macOS/Windows
seeds = range(8)
serial = timed("serial", lambda: [slow_feature(s) for s in seeds])
procs = timed("joblib processes (n_jobs=-1)", lambda: Parallel(n_jobs=-1)(delayed(slow_feature)(s) for s in seeds))
with ThreadPoolExecutor(max_workers=8) as pool:
threads = timed("threads (GIL-bound)", lambda: list(pool.map(slow_feature, seeds)))
print("same results:", np.allclose(serial, procs) and np.allclose(serial, threads))
X, y = make_classification(n_samples=5000, n_features=30, random_state=42)
rf = RandomForestClassifier(n_estimators=200, random_state=42)
timed("CV n_jobs=1", lambda: cross_val_score(rf, X, y, cv=5, n_jobs=1))
timed("CV n_jobs=-1", lambda: cross_val_score(rf, X, y, cv=5, n_jobs=-1))
# Processes and n_jobs=-1 speed up CPU-bound work on multi-core machines; threads barely help here
Common Pitfalls
Nested parallelism (parallel CV around a model that also uses all cores) oversubscribing the CPU; parallelize one level.
Large data copied to every process; use shared memory, memory-mapped arrays (joblib does this for big NumPy arrays) or Dask.
Parallelizing tiny tasks, where process start-up costs more than the work.
Forgetting the if __name__ == "__main__": guard.
Interview Tips
Explain the GIL and why processes beat threads for CPU-bound Python.
Start with vectorization and n_jobs, then joblib, Numba, GPUs and Dask/Ray.
57
Why does model persistence matter and how do you save and load models?
KnownRevise
Model persistence means saving a trained model to disk so it can be reused without retraining: it is what lets you deploy a model to a server, reproduce results, compare versions, roll back to an earlier model, and share models across teams. For scikit-learn, save the whole fitted pipeline (preprocessing plus model) with joblib.dump and load it with joblib.load in the same library versions; for PyTorch, save the state_dict with torch.save and load it into the same model class; for Keras, use model.save("model.keras"); and for cross-language or runtime-independent deployment, export to ONNX. Store metadata (library versions, training data version, metrics, feature names) next to the file, and never load pickle-based files from untrusted sources.
import json
import platform
from datetime import date
import joblib
import numpy as np
import sklearn
from sklearn.datasets import load_wine
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_wine(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
pipe = make_pipeline(StandardScaler(), RandomForestClassifier(random_state=42)).fit(X_tr, y_tr)
joblib.dump(pipe, "wine_model.joblib", compress=3) # preprocessing and model together
metadata = {
"trained_on": str(date.today()),
"sklearn_version": sklearn.__version__,
"python_version": platform.python_version(),
"test_accuracy": round(pipe.score(X_te, y_te), 4),
"n_features": X.shape[1],
}
with open("wine_model.json", "w") as f:
json.dump(metadata, f, indent=2)
loaded = joblib.load("wine_model.joblib") # only load files you trust
with open("wine_model.json") as f:
meta = json.load(f)
if meta["sklearn_version"] != sklearn.__version__:
print("warning: model was saved with a different scikit-learn version")
print("identical predictions:", np.array_equal(loaded.predict(X_te), pipe.predict(X_te))) # True
print(meta["test_accuracy"])
Common Pitfalls
Saving only the model and not the fitted preprocessing, so production inputs are transformed differently.
Loading a joblib/pickle file with a different library version (warnings or wrong behaviour).
Loading pickles from untrusted sources (arbitrary code execution).
No versioning, so you can’t tell which model is in production.
Show saving the full pipeline plus metadata, and mention ONNX and security.
58
How do you interpret the coefficients of a logistic regression model?
KnownRevise
A logistic regression coefficient is the change in the log-odds of the positive class for a one-unit increase in that feature, holding the other features constant; exponentiating it gives an odds ratio, so a coefficient of 0.7 means the odds are multiplied by e^0.7 ≈ 2.01 per unit, a negative coefficient lowers the odds, and zero means no effect. The intercept is the log-odds when all features are zero (or at their means if features are centred). To compare features, standardize them first so coefficients are per standard deviation, and remember that coefficients describe associations in the data, not causes, and become unreliable when features are strongly correlated or heavily regularized.
Read full answerHide full answer
Applies toscikit-learn 1.5+, statsmodels 0.14+
Interpretation Guide
Quantity
Meaning
β (coefficient)
Change in log-odds per unit of x
e^β (odds ratio)
Multiplicative change in odds per unit of x
e^β > 1
Feature raises the probability
e^β < 1
Feature lowers the probability
Standardized β
Effect per standard deviation; comparable across features
Intercept
Log-odds when all features are 0
Worked Example
With β_age = 0.05 per year: e^0.05 ≈ 1.051, so each extra year multiplies the odds by about 1.05 (+5%); ten years multiply them by e^0.5 ≈ 1.65. The effect on probability is not constant: it is largest near p = 0.5 and small near 0 or 1.
Code Example
import numpy as np
import pandas as pd
import statsmodels.api as sm
from sklearn.linear_model import LogisticRegression
rng = np.random.default_rng(42)
n = 5000
df = pd.DataFrame({
"age": rng.uniform(20, 70, n),
"smoker": rng.integers(0, 2, n),
"exercise_hours": rng.uniform(0, 10, n),
})
log_odds = -6 + 0.08 * df["age"] + 1.2 * df["smoker"] - 0.15 * df["exercise_hours"]
df["disease"] = (rng.random(n) < 1 / (1 + np.exp(-log_odds))).astype(int)
X = df[["age", "smoker", "exercise_hours"]]
model = LogisticRegression(penalty=None, max_iter=1000).fit(X, df["disease"]) # no regularization
coefs = pd.Series(model.coef_[0], index=X.columns)
print(pd.DataFrame({"coef": coefs.round(3), "odds_ratio": np.exp(coefs).round(3)}))
# age close to 0.08 (odds x1.08 per year), smoker close to 1.2 (odds x3.3), exercise close to -0.15 (x0.86 per hour)
sm_model = sm.Logit(df["disease"], sm.add_constant(X)).fit(disp=False) # p-values and intervals
ci = np.exp(sm_model.conf_int())
ci.columns = ["odds_ratio_2.5%", "odds_ratio_97.5%"]
print(ci.round(3))
X_std = (X - X.mean()) / X.std()
std_coefs = LogisticRegression(penalty=None, max_iter=1000).fit(X_std, df["disease"]).coef_[0]
print("per-SD effects:", dict(zip(X.columns, std_coefs.round(3)))) # comparable sizes
Common Pitfalls
Reading coefficients as changes in probability (they are log-odds).
Comparing raw coefficients across features with different units.
Interpreting regularized coefficients (shrunk toward zero) as unbiased effect sizes.
Multicollinearity making signs and sizes unstable.
Claiming causality from observational data.
Interview Tips
Convert a coefficient to an odds ratio out loud.
Mention standardization, regularization and correlation caveats.
59
What is dimensionality reduction and when would you use it?
KnownRevise
Dimensionality reduction transforms data with many features into fewer features that keep most of the useful information, either by feature extraction (creating new combined features: PCA, truncated SVD, autoencoders, or t-SNE and UMAP for visualization) or by feature selection (keeping a subset of the original columns). You use it to visualize high-dimensional data in 2-D or 3-D, to speed up training and reduce memory, to remove noise and multicollinearity, to fight the curse of dimensionality in distance-based methods like kNN and clustering, and to compress inputs such as images or text embeddings, while accepting some loss of information and interpretability.
Read full answerHide full answer
Applies toscikit-learn 1.5+; umap-learn optional
Methods
Method
Type
Preserves
Use for
PCA
Linear
Global variance
Compression, decorrelation, preprocessing
Truncated SVD
Linear
Variance (works on sparse)
Text TF-IDF (LSA)
LDA (linear discriminant)
Linear, supervised
Class separation
Classification preprocessing
t-SNE
Non-linear
Local neighbourhoods
2-D visualization only
UMAP
Non-linear
Local + some global structure
Visualization, clustering preprocessing
Autoencoders
Non-linear (neural)
Learned structure
Images, complex data
Feature selection
Subset
Original features
Interpretability
Code Example
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.manifold import TSNE
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_digits(return_X_y=True) # 1797 images, 64 pixel features
pca = PCA().fit(StandardScaler().fit_transform(X))
n95 = int(np.searchsorted(np.cumsum(pca.explained_variance_ratio_), 0.95) + 1)
print(f"components for 95% of variance: {n95} of 64")
for k in [64, n95, 10]:
model = make_pipeline(StandardScaler(), PCA(n_components=k), LogisticRegression(max_iter=3000))
print(f"{k:2d} components: CV accuracy {cross_val_score(model, X, y, cv=5).mean():.3f}")
# Accuracy stays close to the full model with far fewer features
emb = TSNE(n_components=2, random_state=42, init="pca").fit_transform(X) # visualization only
plt.figure(figsize=(6, 5))
plt.scatter(emb[:, 0], emb[:, 1], c=y, cmap="tab10", s=5)
plt.colorbar(label="digit")
plt.title("t-SNE of handwritten digits")
plt.savefig("digits_tsne.png", dpi=120)
When Not to Use It
When interpretability of individual original features is required (prefer feature selection).
With tree-based models on moderate feature counts, which handle many features well.
Using t-SNE/UMAP coordinates as model features without care (distances between clusters aren’t meaningful).
Common Pitfalls
Running PCA on unscaled features (large-scale features dominate).
Fitting PCA on all data before the train/test split.
Reading t-SNE cluster sizes or distances literally.
Interview Tips
Separate extraction from selection and linear from non-linear.
Give use cases: visualization, speed, noise removal, curse of dimensionality.
60
What is a neural network and how would you implement one in Python?
KnownRevise
A neural network is a model built from layers of simple units (“neurons”): each computes a weighted sum of its inputs plus a bias and passes it through a non-linear activation such as ReLU, so stacked layers can learn complex, non-linear functions; training adjusts the weights by backpropagation (computing gradients of a loss with the chain rule) and gradient descent. In Python you can write a small network from scratch with NumPy to understand the mechanics, use scikit-learn’s MLPClassifier for quick tabular baselines, and use PyTorch or Keras for real deep learning with GPUs, automatic differentiation and modern layers.
import numpy as np
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
rng = np.random.default_rng(42)
X, y = make_moons(n_samples=1000, noise=0.2, random_state=42) # not linearly separable
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)
def init(n_in, n_hidden, n_out=1):
return {"W1": rng.normal(0, np.sqrt(2 / n_in), (n_in, n_hidden)), "b1": np.zeros(n_hidden),
"W2": rng.normal(0, np.sqrt(1 / n_hidden), (n_hidden, n_out)), "b2": np.zeros(n_out)}
def forward(p, X):
z1 = X @ p["W1"] + p["b1"]
a1 = np.maximum(0, z1) # ReLU
z2 = a1 @ p["W2"] + p["b2"]
return z1, a1, 1 / (1 + np.exp(-z2)) # sigmoid output probability
def train(X, y, n_hidden=16, lr=0.5, epochs=2000):
p = init(X.shape[1], n_hidden)
y = y.reshape(-1, 1)
for epoch in range(epochs):
z1, a1, out = forward(p, X)
loss = -np.mean(y * np.log(out + 1e-12) + (1 - y) * np.log(1 - out + 1e-12))
d_z2 = (out - y) / len(X) # gradient of BCE w.r.t. pre-sigmoid output
grads = {"W2": a1.T @ d_z2, "b2": d_z2.sum(axis=0)}
d_z1 = (d_z2 @ p["W2"].T) * (z1 > 0) # backprop through ReLU
grads.update({"W1": X.T @ d_z1, "b1": d_z1.sum(axis=0)})
for k in p:
p[k] -= lr * grads[k]
if epoch % 500 == 0:
print(f"epoch {epoch}: loss {loss:.4f}")
return p
params = train(X_tr, y_tr)
acc = np.mean((forward(params, X_te)[2].ravel() > 0.5) == y_te)
print(f"from-scratch network test accuracy: {acc:.3f}") # well above a linear model's accuracy
mlp = MLPClassifier(hidden_layer_sizes=(16,), max_iter=2000, random_state=42).fit(X_tr, y_tr)
print(f"scikit-learn MLP test accuracy: {mlp.score(X_te, y_te):.3f}")
For larger networks, the same model in PyTorch is nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 1)) trained with nn.BCEWithLogitsLoss and an optimizer, with autograd computing the gradients written by hand above.
Common Pitfalls
Unscaled inputs, which slow or break training.
Poor initialization (all zeros makes every neuron identical).
Learning rates that are too high (loss explodes) or too low (no progress).
Using neural networks on small tabular datasets where gradient boosting is usually better.
Interview Tips
Explain layers, activations, loss, backprop and gradient descent in order.
Show you can write the forward and backward pass, then say you’d use PyTorch or Keras in practice.
61
What is reinforcement learning and what are its implementation challenges?
KnownRevise
Reinforcement learning (RL) trains an agent to make a sequence of decisions by interacting with an environment: at each step the agent observes a state, takes an action, and receives a reward and the next state, and it learns a policy that maximizes the expected cumulative (discounted) reward, using methods such as Q-learning and DQN (value-based), policy gradients and PPO (policy-based), or actor-critic algorithms like SAC. The main implementation challenges are designing a reward that produces the intended behaviour, sample inefficiency (millions of interactions), the exploration-exploitation trade-off, unstable training and high variance across random seeds, delayed credit assignment, and the gap between simulation and the real world; in Python, Gymnasium supplies environments and Stable-Baselines3 or TorchRL supply tested algorithms.
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
env = gym.make("CartPole-v1")
model = PPO("MlpPolicy", env, seed=42, verbose=0)
before, _ = evaluate_policy(model, env, n_eval_episodes=10)
model.learn(total_timesteps=50_000)
after, std = evaluate_policy(model, env, n_eval_episodes=20)
print(f"mean episode reward: before {before:.1f}, after {after:.1f} +/- {std:.1f}") # 500 is the maximum
obs, info = env.reset(seed=42) # running the trained policy
total = 0.0
for _ in range(500):
action, _ = model.predict(obs, deterministic=True)
obs, reward, terminated, truncated, info = env.step(action)
total += reward
if terminated or truncated:
break
print("one greedy episode:", total)
Common Pitfalls
Writing algorithms from scratch for production instead of using tested libraries.
Judging from a single seed or a single episode.
Environment bugs (wrong termination, leaking information) mistaken for algorithm problems.
Interview Tips
Define the agent-environment loop and the main algorithm families.
Name reward design, sample efficiency and exploration as the key challenges.
62
What are generative adversarial networks and what are they used for?
KnownRevise
A generative adversarial network (GAN) consists of two neural networks trained against each other: a generator that turns random noise into synthetic samples and a discriminator that tries to tell real samples from generated ones; the generator improves by fooling the discriminator, and at equilibrium it produces samples that resemble the training data. GANs have been used for realistic image synthesis (StyleGAN faces), image-to-image translation (pix2pix, CycleGAN), super-resolution, data augmentation and synthetic tabular data (CTGAN), and anomaly detection. They are known to be hard to train (instability, mode collapse), and diffusion models have overtaken them for most image generation, though GANs remain popular where single-step, fast generation matters.
Read full answerHide full answer
Applies togeneral concepts; example in PyTorch 2.x
How Training Works
Sample real data and generate fake data from noise.
Train the discriminator to output 1 for real and 0 for fake.
Train the generator so the discriminator outputs 1 for its fakes.
Repeat; neither network should overpower the other.
Use Cases
Use case
Example
Image synthesis
StyleGAN faces, artwork
Image-to-image
Sketches to photos, day to night, map to satellite (pix2pix, CycleGAN)
Super-resolution
SRGAN, ESRGAN
Data augmentation
Extra training images for rare classes
Synthetic tabular data
CTGAN for privacy-preserving data sharing
Anomaly detection
Poor reconstruction marks unusual samples
Code Example (Compact)
import torch
import torch.nn as nn
torch.manual_seed(42)
real_data = lambda n: torch.randn(n, 1) * 0.5 + 3.0 # target distribution: N(3, 0.5)
G = nn.Sequential(nn.Linear(4, 32), nn.ReLU(), nn.Linear(32, 1))
D = nn.Sequential(nn.Linear(1, 32), nn.LeakyReLU(0.2), nn.Linear(32, 1))
opt_g = torch.optim.Adam(G.parameters(), lr=1e-3)
opt_d = torch.optim.Adam(D.parameters(), lr=1e-3)
bce = nn.BCEWithLogitsLoss()
for step in range(3000):
real, fake = real_data(128), G(torch.randn(128, 4))
d_loss = bce(D(real), torch.ones(128, 1)) + bce(D(fake.detach()), torch.zeros(128, 1))
opt_d.zero_grad()
d_loss.backward()
opt_d.step()
g_loss = bce(D(G(torch.randn(128, 4))), torch.ones(128, 1))
opt_g.zero_grad()
g_loss.backward()
opt_g.step()
with torch.no_grad():
samples = G(torch.randn(5000, 4))
print(f"generated mean {samples.mean():.2f}, std {samples.std():.2f}") # should approach 3.0 and 0.5
Challenges
Mode collapse: the generator produces a few similar outputs.
Instability: oscillating losses; use spectral normalization, WGAN-GP, careful learning rates.
Evaluation: no single loss to trust; use FID, precision/recall and visual inspection.
Ethics: deepfakes and misuse; provenance and watermarking matter.
Interview Tips
Explain the generator-discriminator game and the alternating updates.
Name use cases and training challenges, and compare briefly with diffusion models.
63
What is transfer learning and how do you implement it with Python libraries?
KnownRevise
Transfer learning reuses a model trained on a large source dataset as the starting point for a related target task, so you need much less labelled data and compute: the pre-trained network’s early layers already capture general features (edges and textures for images, grammar and meaning for text), and you adapt it by replacing the output layer and either training only the new head (feature extraction) or fine-tuning some or all layers with a low learning rate. In Python, torchvision and timm provide pre-trained vision models for PyTorch, Keras Applications provide them for TensorFlow, and Hugging Face Transformers provides pre-trained language and vision models through from_pretrained.
Read full answerHide full answer
Applies toPyTorch 2.x / torchvision 0.17+, Keras 3, Hugging Face Transformers 4.40+
Approaches
Approach
Trainable layers
Data needed
Feature extraction
New head only
Small
Partial fine-tuning
Top layers + head
Medium
Full fine-tuning
All layers, low LR
Larger, or a different domain
Parameter-efficient (LoRA, adapters)
Small added modules
Large models, limited GPU
Code Example: Keras (Feature Extraction, Then Fine-Tuning)
import keras
import numpy as np
keras.utils.set_random_seed(42)
base = keras.applications.MobileNetV2(input_shape=(160, 160, 3), include_top=False, weights="imagenet")
base.trainable = False # stage 1: freeze the pre-trained backbone
inputs = keras.Input(shape=(160, 160, 3))
x = keras.applications.mobilenet_v2.preprocess_input(inputs)
x = base(x, training=False) # keep BatchNorm in inference mode
x = keras.layers.GlobalAveragePooling2D()(x)
x = keras.layers.Dropout(0.2)(x)
outputs = keras.layers.Dense(3, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.compile(optimizer=keras.optimizers.Adam(1e-3), loss="sparse_categorical_crossentropy", metrics=["accuracy"])
rng = np.random.default_rng(42)
images = rng.uniform(0, 255, (32, 160, 160, 3)).astype("float32") # stands in for a real image dataset
labels = rng.integers(0, 3, 32)
model.fit(images, labels, epochs=1, batch_size=8, verbose=0)
base.trainable = True # stage 2: fine-tune the top layers
for layer in base.layers[:-20]:
layer.trainable = False
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(images, labels, epochs=1, batch_size=8, verbose=0)
print("trainable weights after unfreezing:", len(model.trainable_weights))
Hugging Face Text Example
from transformers import AutoModelForSequenceClassification, AutoTokenizer
name = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name, num_labels=2) # new classification head
batch = tokenizer(["great product", "terrible support"], padding=True, return_tensors="pt")
print(model(**batch).logits.shape) # torch.Size([2, 2]); fine-tune with Trainer
Common Pitfalls
Wrong preprocessing for the pre-trained model (each family has its own).
A high learning rate during fine-tuning, which destroys learned features.
Forgetting to recompile (Keras) or rebuild the optimizer (PyTorch) after unfreezing.
Interview Tips
Explain feature extraction vs fine-tuning and when to use each.
Name the libraries for each framework.
64
What is the difference between batch learning and online learning?
KnownRevise
Batch (offline) learning trains a model on the whole available dataset at once and then deploys it unchanged until it is retrained from scratch on a new snapshot; online (incremental) learning updates the model continuously as new examples or mini-batches arrive, without revisiting all past data. Batch learning is simpler, easier to validate and the default for most systems; online learning suits data streams too large to store, fast-changing environments (news recommendations, fraud, ad click prediction) and memory-limited devices, but it needs safeguards against noisy or malicious data and drift in quality, plus careful choice of learning rate (how fast it forgets).
Read full answerHide full answer
Applies toscikit-learn 1.5+ (`partial_fit`), River 0.21+
Comparison
Aspect
Batch learning
Online learning
Data
Full dataset in one go
Streamed one example or chunk at a time
Update
Periodic full retraining
Continuous incremental updates
Memory
Must handle the whole dataset
Constant
Adapts to change
At the next retrain
Quickly
Validation
Standard train/test, CV
Prequential (test-then-train) evaluation
Risks
Stale models
Bad data immediately affects the model
Tools
Any fit
partial_fit (SGD, Naive Bayes, MiniBatchKMeans), River
Code Example
import numpy as np
from sklearn.linear_model import SGDClassifier
from sklearn.metrics import accuracy_score
rng = np.random.default_rng(42)
def make_chunk(n, shift):
X = rng.normal(size=(n, 2))
y = (X[:, 0] + shift * X[:, 1] > 0).astype(int) # the decision rule drifts over time
return X, y
chunks = [make_chunk(500, shift) for shift in np.linspace(-1, 1, 10)]
batch_model = SGDClassifier(loss="log_loss", random_state=42).fit(*chunks[0]) # trained once at the start
online_model = SGDClassifier(loss="log_loss", random_state=42)
for i, (X, y) in enumerate(chunks):
if i > 0: # test on new data before training on it
batch_acc = accuracy_score(y, batch_model.predict(X))
online_acc = accuracy_score(y, online_model.predict(X))
print(f"chunk {i}: batch {batch_acc:.2f} online {online_acc:.2f}")
online_model.partial_fit(X, y, classes=[0, 1]) # incremental update
# As the rule drifts, the batch model's accuracy falls while the online model keeps up
Hybrid Practice
Many production systems retrain in batch on a schedule (daily or weekly) and add online updates or fast fine-tuning between retrains.
Keep a holdout stream and rollback ability for online models.
Common Pitfalls
Online learning without monitoring: one bad data source can quickly degrade the model.
Evaluating an online model with a static test set instead of prequential evaluation.
Forgetting to pass classes on the first partial_fit call.
Interview Tips
Contrast data handling, adaptation and risk.
Give an example of where each fits and mention partial_fit.
65
What role do attention mechanisms play in NLP models?
KnownRevise
Attention lets a model decide, for each word (token) it processes, how much to draw on every other token in the sequence: each token’s query vector is compared with every token’s key vector to produce weights (scaled dot products passed through softmax), and the output is the weighted sum of the value vectors. This solved the bottleneck of RNN encoder-decoders, which squeezed a whole sentence into one fixed vector, and it captures long-range dependencies (linking a pronoun to its noun many words back) in a single step. Self-attention, stacked with multiple heads in the Transformer architecture, is the core of modern NLP models such as BERT and GPT, which are trained in parallel on large text corpora and power translation, summarization, question answering and chatbots.
Read full answerHide full answer
Applies togeneral concepts; examples in Python 3.12 with NumPy 2.x and Hugging Face Transformers
Scaled Dot-Product Attention
Attention(Q, K, V) = softmax(QKᵀ / √d_k) · V
Part
Role
Query (Q)
What this token is looking for
Key (K)
What each token offers
Value (V)
The information passed along
√d_k scaling
Keeps softmax from saturating
Masking
Hides padding or future tokens (in GPT-style decoders)
Multi-head
Several attention patterns in parallel (syntax, coreference, position)
Code Example
import numpy as np
def softmax(x, axis=-1):
e = np.exp(x - x.max(axis=axis, keepdims=True))
return e / e.sum(axis=axis, keepdims=True)
def attention(Q, K, V, causal=False):
scores = Q @ K.T / np.sqrt(K.shape[-1])
if causal: # a token can't look at later tokens
scores = np.where(np.triu(np.ones_like(scores), k=1) == 1, -1e9, scores)
weights = softmax(scores)
return weights @ V, weights
rng = np.random.default_rng(42)
tokens = ["the", "cat", "sat", "down"]
d = 8
X = rng.normal(size=(len(tokens), d)) # token embeddings
Wq, Wk, Wv = (rng.normal(size=(d, d)) / np.sqrt(d) for _ in range(3))
out, w = attention(X @ Wq, X @ Wk, X @ Wv)
print(out.shape, w.sum(axis=1).round(6)) # (4, 8) [1. 1. 1. 1.]: each row of weights sums to 1
_, w_causal = attention(X @ Wq, X @ Wk, X @ Wv, causal=True)
print(np.round(w_causal, 2)) # zeros above the diagonal
With Hugging Face you can inspect real attention weights: AutoModel.from_pretrained("bert-base-uncased", output_attentions=True) returns one attention tensor per layer, shaped (batch, heads, tokens, tokens).
Impact on NLP
Before attention
With attention / Transformers
RNNs process tokens sequentially
All tokens processed in parallel (fast on GPUs)
Long-range dependencies fade
Direct links between any two tokens
Fixed context vector
Dynamic, token-specific context
Task-specific models
Pre-trained models fine-tuned for many tasks
Common Pitfalls
Treating attention weights as a full explanation of model decisions (they are only one signal).
Ignoring the quadratic cost in sequence length (efficient attention variants address it).
Interview Tips
Write the attention formula and explain Q, K, V.
Connect attention to Transformers, BERT and GPT.
66
How do you implement a recommendation system in Python?
KnownRevise
A recommendation system in Python usually starts with the interaction data (user, item, rating or implicit signal like clicks) and one of three approaches: content-based filtering (recommend items similar to what the user liked, using item features such as TF-IDF of descriptions), collaborative filtering (learn from patterns across users, via user/item similarity or matrix factorization such as SVD or ALS), or a hybrid of both. You evaluate with a time-based split and ranking metrics such as precision@k, recall@k and NDCG, compare against a popularity baseline, and handle cold-start users and items with content features or popular items. Libraries include scikit-learn and SciPy for building blocks, Surprise for explicit ratings, implicit for implicit-feedback ALS, and LightFM or TensorFlow Recommenders for hybrid and deep models.
Code Example: Matrix Factorization with Truncated SVD
import numpy as np
from scipy.sparse import csr_matrix
from scipy.sparse.linalg import svds
rng = np.random.default_rng(42)
n_users, n_items, k_true = 300, 200, 5
U_true, V_true = rng.normal(size=(n_users, k_true)), rng.normal(size=(n_items, k_true))
scores = U_true @ V_true.T
rows, cols, vals = [], [], []
for u in range(n_users): # each user interacts with their top items plus noise
liked = np.argsort(-scores[u] + rng.normal(0, 2, n_items))[:15]
rows += [u] * len(liked)
cols += liked.tolist()
vals += [1.0] * len(liked)
R = csr_matrix((vals, (rows, cols)), shape=(n_users, n_items))
test = {u: int(R[u].indices[rng.integers(R[u].nnz)]) for u in range(n_users)} # hide one item per user
R_train = R.tolil()
for u, i in test.items():
R_train[u, i] = 0
R_train = R_train.tocsr()
R_train.eliminate_zeros() # drop the hidden entries completely
U, s, Vt = svds(R_train, k=10) # low-rank factorization
pred = U @ np.diag(s) @ Vt
def hit_rate(pred, k=10):
hits = 0
for u, held_out in test.items():
p = pred[u].copy()
p[R_train[u].indices] = -np.inf # don't recommend already-seen items
hits += held_out in np.argsort(-p)[:k]
return hits / len(test)
popularity = np.tile(np.asarray(R_train.sum(axis=0)).ravel(), (n_users, 1))
print(f"hit rate@10 SVD: {hit_rate(pred):.3f} popularity: {hit_rate(popularity):.3f}")
# The personalized factorization should beat the popularity baseline
Production Considerations
Candidate generation (fast retrieval of hundreds of items) followed by a ranking model.
Freshness: retrain or update embeddings regularly.
Diversity and business rules on top of scores.
Online A/B tests, since offline metrics don’t fully predict engagement.
Common Pitfalls
Random train/test splits that leak future interactions.
Recommending items the user has already bought.
No popularity baseline.
Interview Tips
Compare content-based, collaborative and hybrid methods.
Mention ranking metrics, cold start and time-based evaluation.
67
How would you build a spam detection system in Python?
KnownRevise
A spam detector is a text classification pipeline: collect labelled messages (spam vs not spam), clean and tokenize the text, turn it into features (TF-IDF of words and character n-grams, plus metadata such as the number of links, capital-letter ratio or sender reputation), train a classifier (multinomial Naive Bayes or logistic regression as strong, fast baselines; fine-tuned transformers when accuracy matters most), and evaluate with precision and recall, because flagging a real email as spam is usually worse than letting one spam through. In production, the trained pipeline is saved, served behind an API or mail filter, monitored, and retrained regularly, since spammers change tactics.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Python 3.12
Design
Stage
Choice
Data
Labelled emails/SMS; time-based split
Text features
TF-IDF word 1–2-grams, character 3–5-grams (catch obfuscations like “fr€e”)
Multinomial NB, logistic regression, linear SVM; transformers for the best accuracy
Metric
Precision on the spam class (few false positives), recall, PR-AUC
Threshold
Tuned to a maximum false positive rate
Deployment
Saved pipeline, API, feedback loop from “report spam” buttons
Code Example
import joblib
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.pipeline import FeatureUnion, Pipeline
rng = np.random.default_rng(42)
spam_templates = ["WIN a FREE {} now!!! click {}", "Cheap {} offer, limited time, visit {}",
"You have been selected for a {} prize, claim at {}", "URGENT: your {} account, verify at {}"]
ham_templates = ["Can we move the {} meeting to {}?", "Here are the notes from {} about {}",
"Lunch at {} tomorrow? Bring the {} report", "Thanks for the {} update, see you {}"]
words = ["iphone", "project", "loan", "monday", "budget", "team", "gift card", "friday", "vacation", "invoice"]
links = ["http://bit.ly/x9", "www.win-now.biz", "http://claim.example"]
def make(templates, n, is_spam):
out = []
for _ in range(n):
t = templates[rng.integers(len(templates))]
second = links[rng.integers(len(links))] if is_spam else words[rng.integers(len(words))]
out.append(t.format(words[rng.integers(len(words))], second))
return out
texts = make(spam_templates, 300, True) + make(ham_templates, 700, False)
labels = np.r_[np.ones(300, int), np.zeros(700, int)]
X_tr, X_te, y_tr, y_te = train_test_split(texts, labels, test_size=0.25, stratify=labels, random_state=42)
features = FeatureUnion([
("words", TfidfVectorizer(ngram_range=(1, 2), lowercase=True, min_df=2)),
("chars", TfidfVectorizer(analyzer="char_wb", ngram_range=(3, 5), min_df=2)),
])
spam_filter = Pipeline([("features", features), ("clf", LogisticRegression(max_iter=1000, C=5))])
spam_filter.fit(X_tr, y_tr)
print(classification_report(y_te, spam_filter.predict(X_te), target_names=["ham", "spam"], digits=3))
new = ["Claim your FREE gift card now at http://claim.example", "Can we review the budget on friday?"]
print(spam_filter.predict_proba(new)[:, 1].round(3)) # high for the first, low for the second
joblib.dump(spam_filter, "spam_filter.joblib")
Synthetic templates make the task easy here; real data is messier, so expect lower scores and more feature work.
Common Pitfalls
Random splits when messages are near-duplicates (template spam), which inflates scores; group or time-split.
Optimizing accuracy instead of the false-positive rate.
Never retraining as spam campaigns change.
Interview Tips
Walk through data, features, model, metric, threshold and deployment.
Emphasize precision and adversarial drift.
68
How would you design a Python system that predicts house prices?
KnownRevise
Design it as a regression pipeline: gather data on past sales with features such as size, rooms, location, age, condition and sale date; clean it and engineer features (price per area by neighbourhood, age at sale, distance to the city centre, log of size); split by time so the model is tested on later sales; train a baseline (median price by area, then linear regression on log price) and stronger models such as gradient boosting; evaluate with MAE or RMSE in currency and MAPE for relative error; then package the fitted pipeline behind an API or batch job, monitor errors as the market changes, and retrain regularly. Predicting the log of price usually works better because prices are right-skewed and errors scale with price.
Read full answerHide full answer
Applies toscikit-learn 1.5+, pandas 2.x
System Components
Component
Choice
Data
Sales records, property attributes, location data
Target
log(price), converted back for reporting
Split
Time-based (train on earlier sales)
Features
Area, rooms, age, location encodings, neighbourhood aggregates (from training data only)
Error on new sales, feature drift, retraining schedule
Code Example
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer, TransformedTargetRegressor
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import Ridge
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_error
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
rng = np.random.default_rng(42)
n = 3000
df = pd.DataFrame({
"area_m2": rng.uniform(40, 250, n),
"rooms": rng.integers(1, 6, n),
"age_years": rng.uniform(0, 60, n),
"neighbourhood": rng.choice(["central", "north", "south", "suburb"], n, p=[0.2, 0.3, 0.3, 0.2]),
"sale_month": rng.integers(0, 36, n), # three years of sales
})
base = {"central": 9000, "north": 6000, "south": 5000, "suburb": 3500}
df["price"] = (df["neighbourhood"].map(base) * df["area_m2"] * (1 - 0.004 * df["age_years"])
* (1 + 0.005 * df["sale_month"]) * np.exp(rng.normal(0, 0.1, n)))
train, test = df[df["sale_month"] < 30], df[df["sale_month"] >= 30] # time-based split
features = ["area_m2", "rooms", "age_years", "neighbourhood", "sale_month"]
prep = ColumnTransformer([
("num", StandardScaler(), ["area_m2", "rooms", "age_years", "sale_month"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["neighbourhood"]),
])
models = {
"ridge on log price": TransformedTargetRegressor(make_pipeline(prep, Ridge()), func=np.log, inverse_func=np.exp),
"boosting on log price": TransformedTargetRegressor(
make_pipeline(prep, HistGradientBoostingRegressor(random_state=42)), func=np.log, inverse_func=np.exp),
}
median_by_area = train.groupby("neighbourhood")["price"].median()
baseline_pred = test["neighbourhood"].map(median_by_area)
print(f"baseline (median by neighbourhood): MAE {mean_absolute_error(test['price'], baseline_pred):,.0f}")
for name, model in models.items():
model.fit(train[features], train["price"])
pred = model.predict(test[features])
print(f"{name:22s}: MAE {mean_absolute_error(test['price'], pred):,.0f}, "
f"MAPE {mean_absolute_percentage_error(test['price'], pred):.1%}")
Common Pitfalls
Random splits that let the model see future market prices.
Neighbourhood averages computed with test sales (target leakage).
Reporting only RMSE, which is dominated by expensive houses.
Extrapolating to property types missing from training data.
Interview Tips
Cover data, target transform, time split, baselines, models, metrics and monitoring.
Mention log price and leakage in location features.
69
How would you build a sentiment analysis model in Python?
KnownRevise
Build sentiment analysis as text classification: get labelled text (reviews or posts tagged positive, negative and possibly neutral), split it, and start with a fast baseline of TF-IDF features and logistic regression, which handles many cases well; then, for higher accuracy on nuanced language (negation, sarcasm, context), fine-tune a pre-trained transformer such as DistilBERT or RoBERTa with Hugging Face, or use an existing sentiment model if the domain matches. Evaluate with macro F1 and a confusion matrix, inspect errors (negations like “not bad”, domain-specific words), and deploy the chosen pipeline behind an API with monitoring.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Hugging Face Transformers 4.40+
Options
Approach
Accuracy
Speed
Data needed
Lexicon (VADER)
Moderate
Very fast
None
TF-IDF + logistic regression
Good
Fast
Thousands of labels
Fine-tuned transformer
Best
Slower, GPU helpful
Hundreds to thousands
Zero-shot / LLM prompting
Good
Slow per item
None
Code Example: Baseline
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(42)
pos = ["great", "excellent", "loved", "fantastic", "not bad", "works perfectly", "highly recommend"]
neg = ["terrible", "awful", "hated", "broke", "not good", "waste of money", "never again"]
items = ["phone", "laptop", "headphones", "service", "delivery"]
def review(words):
return f"The {items[rng.integers(len(items))]} was {words[rng.integers(len(words))]}, " \
f"{words[rng.integers(len(words))]} overall."
texts = [review(pos) for _ in range(500)] + [review(neg) for _ in range(500)]
labels = np.r_[np.ones(500, int), np.zeros(500, int)]
X_tr, X_te, y_tr, y_te = train_test_split(texts, labels, test_size=0.2, stratify=labels, random_state=42)
model = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)), LogisticRegression(max_iter=1000)) # bigrams catch "not bad"
model.fit(X_tr, y_tr)
print(classification_report(y_te, model.predict(X_te), target_names=["negative", "positive"], digits=3))
print(model.predict(["The delivery was not good", "The laptop was not bad at all"])) # expected [0 1]
vocab = model[0].get_feature_names_out()
coefs = model[-1].coef_[0]
print("most positive terms:", vocab[np.argsort(-coefs)[:5]].tolist())
Transformer Option
from transformers import pipeline
classifier = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
print(classifier(["The battery life is fantastic", "Support never answered my emails"]))
# [{'label': 'POSITIVE', ...}, {'label': 'NEGATIVE', ...}]; fine-tune on your own labels for domain text
Common Pitfalls
Unigram-only features that miss negation.
A pre-trained model from another domain (movie reviews) applied to, say, finance text without checking.
Ignoring neutral or mixed sentiment.
Interview Tips
Present a baseline first, then a transformer for accuracy.
Mention error analysis for negation and domain shift.
70
How would you build and deploy a model that predicts customer churn?
KnownRevise
Start by defining churn precisely (for example, no purchase or cancelled subscription within the next 30 days) and fixing the prediction point, then build features from data available before that point (tenure, usage trends, support tickets, payment failures, plan changes), split by time, train a baseline and a gradient-boosting model, and evaluate with PR-AUC, recall at the retention team’s capacity, and calibration, since the scores drive who gets contacted. Deploy the saved pipeline as a scheduled batch job (or an API) that scores customers and pushes the riskiest to a CRM, then monitor score drift and actual churn, and measure business value with a holdout or A/B test of the retention offers.
Only data before the snapshot: usage trends, tenure, tickets, billing
Split
Train on older snapshots, test on the latest
Models
Logistic regression baseline, gradient boosting
Metrics
PR-AUC, precision/recall at top-k, calibration
Explain
SHAP or permutation importance for retention teams
Deploy
Weekly batch scoring job or API; CRM integration
Monitor
Drift, realized churn, campaign uplift
Code Example: Training and Scoring
import joblib
import numpy as np
import pandas as pd
from sklearn.calibration import CalibratedClassifierCV
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder
rng = np.random.default_rng(42)
n = 6000
df = pd.DataFrame({
"snapshot": rng.integers(0, 12, n), # month of the snapshot
"tenure_months": rng.integers(1, 72, n),
"logins_last_30d": rng.poisson(12, n),
"logins_trend": rng.normal(0, 1, n), # change vs previous month
"support_tickets": rng.poisson(1, n),
"plan": rng.choice(["basic", "plus", "pro"], n),
})
risk = (-2 - 0.03 * df["tenure_months"] - 0.08 * df["logins_last_30d"] - 0.8 * df["logins_trend"]
+ 0.5 * df["support_tickets"] + (df["plan"] == "basic") * 0.7)
df["churned_next_30d"] = (rng.random(n) < 1 / (1 + np.exp(-(risk + 2)))).astype(int)
features = ["tenure_months", "logins_last_30d", "logins_trend", "support_tickets", "plan"]
train, test = df[df["snapshot"] < 10], df[df["snapshot"] >= 10]
prep = ColumnTransformer([("plan", OneHotEncoder(handle_unknown="ignore"), ["plan"])], remainder="passthrough")
model = CalibratedClassifierCV(make_pipeline(prep, HistGradientBoostingClassifier(random_state=42)), cv=3)
model.fit(train[features], train["churned_next_30d"])
proba = model.predict_proba(test[features])[:, 1]
print(f"base churn rate {test['churned_next_30d'].mean():.3f}, PR-AUC {average_precision_score(test['churned_next_30d'], proba):.3f}")
top = test.assign(score=proba).nlargest(int(0.1 * len(test)), "score") # retention team can call 10%
print(f"churn rate in the top 10% scored: {top['churned_next_30d'].mean():.3f}") # well above the base rate
joblib.dump(model, "churn_model.joblib")
Serving Sketch
A weekly job loads churn_model.joblib, builds features for every active customer from the warehouse, writes customer_id, score, top_reasons to a table the CRM reads, and logs the score distribution for drift monitoring. For real-time use, the same model sits behind a FastAPI endpoint.
Common Pitfalls
Leakage: features computed after the snapshot (such as “cancellation reason”).
Random splits across snapshot dates.
Targeting customers who would have stayed anyway; measure uplift with a control group.
Uncalibrated scores used as probabilities in cost calculations.
Interview Tips
Start with the churn definition and prediction point.
Cover time splits, business-aligned metrics, deployment and impact measurement.
71
How would you develop an image classification system in Python?
KnownRevise
An image classification system in Python goes from data to deployment in clear stages: collect and label images into class folders, split them into train/validation/test sets (by source or patient when images are related), build an input pipeline with resizing, normalization and augmentation, start from a pre-trained convolutional network or vision transformer (torchvision, timm or Keras Applications) and fine-tune it, evaluate with per-class precision, recall and a confusion matrix, and export the model (TorchScript, torch.export or ONNX) behind an API, then monitor its predictions. Transfer learning is the default because it reaches high accuracy with a few hundred to a few thousand images per class.
Read full answerHide full answer
Applies toPyTorch 2.x, torchvision 0.17+
Pipeline
Stage
Tools / choices
Data
ImageFolder layout: data/train/<class>/*.jpg
Split
Stratified; grouped if many photos per object
Preprocess
Resize, normalize with pre-trained stats (weights.transforms())
Augment
Random crop, flip, color jitter (training only)
Model
Pre-trained ResNet / EfficientNet / ViT, new head
Train
AdamW, cosine schedule, mixed precision, early stopping
Confidence distribution, drift, human review of low-confidence cases
Code Example
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset
from torchvision.models import ResNet18_Weights, resnet18
from torchvision.transforms import v2
torch.manual_seed(42)
device = "cuda" if torch.cuda.is_available() else "cpu"
num_classes = 4
weights = ResNet18_Weights.DEFAULT
model = resnet18(weights=weights)
model.fc = nn.Linear(model.fc.in_features, num_classes)
model = model.to(device)
train_tf = v2.Compose([v2.RandomResizedCrop(224, scale=(0.7, 1.0), antialias=True), v2.RandomHorizontalFlip(),
v2.ToDtype(torch.float32, scale=True),
v2.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
# Real data: datasets.ImageFolder("data/train", transform=train_tf). Here: random stand-in images.
images = torch.randint(0, 256, (64, 3, 256, 256), dtype=torch.uint8)
labels = torch.randint(0, num_classes, (64,))
loader = DataLoader(TensorDataset(images, labels), batch_size=16, shuffle=True)
optimizer = torch.optim.AdamW([
{"params": [p for n, p in model.named_parameters() if not n.startswith("fc")], "lr": 1e-4},
{"params": model.fc.parameters(), "lr": 1e-3},
], weight_decay=0.01)
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)
model.train()
for xb, yb in loader:
xb, yb = train_tf(xb).to(device), yb.to(device) # v2 transforms work on batched tensors
optimizer.zero_grad()
loss = loss_fn(model(xb), yb)
loss.backward()
optimizer.step()
print("last batch loss:", round(loss.item(), 3))
model.eval()
example = torch.randn(1, 3, 224, 224, device=device)
with torch.no_grad():
probs = torch.softmax(model(example), dim=1)
print("predicted class:", int(probs.argmax()), "confidence:", round(float(probs.max()), 3))
torch.onnx.export(model.cpu(), (example.cpu(),), "classifier.onnx", dynamo=True) # for serving
Common Pitfalls
Near-duplicate photos across train and test, inflating accuracy.
Mismatched preprocessing between training and serving.
Accuracy only, hiding poor performance on rare classes.
Training from scratch with little data.
Interview Tips
Walk through data, transfer learning, evaluation and deployment.
Mention leakage from duplicates and per-class metrics.
72
How would you detect fraudulent transactions with machine learning in Python?
KnownRevise
Fraud detection is an extremely imbalanced classification problem (often well under 1% fraud) where data arrives as a stream and fraudsters adapt, so a solid approach combines feature engineering on transaction behaviour (amount relative to the customer’s usual spend, transaction velocity in the last hour, new device or merchant, distance from the last location, time of day), a time-based split, a gradient-boosting classifier with class weighting, and evaluation with PR-AUC and recall at a fixed alert budget or false-positive rate, plus an unsupervised anomaly detector such as Isolation Forest to catch new patterns. In production, scores feed rules and human review queues in real time, confirmed cases feed back as labels, and the model is retrained often.
Read full answerHide full answer
Applies toscikit-learn 1.5+, pandas 2.x
Approach
Component
Choice
Labels
Confirmed fraud (chargebacks, investigations), which arrive with delay
Features
Per-customer aggregates over time windows, velocity, novelty flags
Split
By time; never random
Supervised model
Gradient boosting with class weights / scale_pos_weight
Unsupervised
Isolation Forest, autoencoders for unknown patterns
Metrics
PR-AUC, recall at k alerts per day, cost of fraud missed vs review cost
Serving
Low-latency API, rules + model, analyst queue
Feedback
Analyst decisions become new labels
Code Example
import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier, IsolationForest
from sklearn.metrics import average_precision_score
rng = np.random.default_rng(42)
n = 50_000
df = pd.DataFrame({
"customer": rng.integers(0, 2000, n),
"time": np.sort(rng.uniform(0, 90, n)), # days
"amount": rng.lognormal(3.5, 1.0, n),
"new_device": rng.random(n) < 0.05,
"night": rng.random(n) < 0.15,
})
fraud_logit = -7 + 1.5 * np.log(df["amount"] / 30).clip(0) + 2.0 * df["new_device"] + 1.0 * df["night"]
df["fraud"] = (rng.random(n) < 1 / (1 + np.exp(-fraud_logit))).astype(int)
df = df.sort_values(["customer", "time"]) # behavioural features from past transactions only
g = df.groupby("customer")
df["cust_mean_amount"] = g["amount"].transform(lambda s: s.shift().expanding().mean()).fillna(df["amount"].median())
df["amount_ratio"] = df["amount"] / df["cust_mean_amount"]
df["txn_count_before"] = g.cumcount()
df = df.sort_values("time")
features = ["amount", "amount_ratio", "txn_count_before", "new_device", "night"]
train, test = df[df["time"] < 70], df[df["time"] >= 70] # train on the past, test on the future
print(f"fraud rate: {df['fraud'].mean():.3%}")
clf = HistGradientBoostingClassifier(class_weight="balanced", random_state=42).fit(train[features], train["fraud"])
proba = clf.predict_proba(test[features])[:, 1]
print(f"PR-AUC (supervised): {average_precision_score(test['fraud'], proba):.3f}")
alerts_per_day = 50
k = int(alerts_per_day * (test["time"].max() - 70))
top_k = np.argsort(-proba)[:k]
caught = test["fraud"].to_numpy()[top_k].sum()
print(f"recall with {alerts_per_day} alerts/day: {caught / test['fraud'].sum():.2%}")
iso = IsolationForest(contamination=0.01, random_state=42).fit(train[features])
anomaly = -iso.score_samples(test[features]) # higher = more unusual
print(f"PR-AUC (Isolation Forest, no labels): {average_precision_score(test['fraud'], anomaly):.3f}")
Common Pitfalls
Random splits, which leak future customer behaviour into training.
Aggregates computed with the current or future transactions.
Measuring accuracy (99% by predicting “not fraud”).
Ignoring label delay: recent transactions may not be labelled yet.
Interview Tips
Stress imbalance, time-aware validation and behavioural features.
Tie the metric to alert capacity and fraud cost.
73
How do local, nonlocal and global scopes affect variable access in Python?
KnownRevise
Python looks names up with the LEGB rule: first the local scope of the current function, then enclosing functions’ scopes, then the module’s global scope, then built-ins; assigning to a name inside a function makes it local unless you declare it global (to rebind a module-level name) or nonlocal (to rebind a variable of an enclosing function). In ML code this matters for configuration and state: functions that silently read global variables (a global model, df or learning rate) are hard to test and break when notebook cells run in a different order, closures with nonlocal are handy for counters and callbacks such as tracking the best validation score, and passing values as arguments or keeping state in objects is the clearest option.
Read full answerHide full answer
Applies toPython 3.12
Scopes
Scope
Where
Rebind with
Local
Inside the current function
Plain assignment
Enclosing
Outer function of a nested function
nonlocal name
Global
Module level
global name
Built-in
len, print, sum…
Don’t shadow them
Code Example
LEARNING_RATE = 0.1 # module-level (global) constant
step_count = 0
def train_step(weight, grad):
global step_count # rebinding a global needs a declaration
step_count += 1
return weight - LEARNING_RATE * grad # reading a global works without one
def broken_counter():
try:
step_count += 1 # noqa: F823, F841 (assignment makes it local, so reading it first fails)
except UnboundLocalError as err:
print("UnboundLocalError:", err)
def make_early_stopper(patience=2):
best = float("inf")
bad_epochs = 0
def should_stop(val_loss):
nonlocal best, bad_epochs # rebind variables of the enclosing function
if val_loss < best:
best, bad_epochs = val_loss, 0
else:
bad_epochs += 1
return bad_epochs >= patience
return should_stop
w = 1.0
for g in [0.5, 0.4, 0.3]:
w = train_step(w, g)
print(round(w, 3), step_count) # 0.88 3
broken_counter()
stop = make_early_stopper(patience=2)
for epoch, loss in enumerate([0.9, 0.7, 0.72, 0.71, 0.69], start=1):
if stop(loss):
print("stop at epoch", epoch) # stop at epoch 4
break
def evaluate(model_predict, X, y): # better: pass dependencies explicitly
return sum(model_predict(x) == t for x, t in zip(X, y)) / len(y)
print(evaluate(lambda x: x > 0, [-1, 2, 3], [False, True, False])) # 0.666...
Common Pitfalls
UnboundLocalError from assigning to a name that also exists globally.
Functions that depend on notebook globals, which give different results after cells are re-run.
Mutable globals shared by threads or processes.
Shadowing built-ins (list = [...], sum = 0).
Interview Tips
Explain LEGB and when global and nonlocal are needed.
Recommend explicit arguments or classes for ML state.
74
How does the GIL affect Python concurrency in machine learning applications?
KnownRevise
CPython’s Global Interpreter Lock (GIL) allows only one thread at a time to execute Python bytecode in a process, so threads cannot speed up CPU-bound pure-Python code such as feature loops or custom simulations; they still help for I/O-bound work (downloading data, calling APIs, reading files) because the GIL is released while waiting. The GIL matters less for most ML workloads than it seems: NumPy, pandas, scikit-learn, PyTorch and TensorFlow do their heavy computation in C, C++ or CUDA code that releases the GIL, and multi-core parallelism is available through processes (multiprocessing, joblib), library n_jobs settings, Numba’s nogil mode and GPUs. Python 3.13 added an experimental free-threaded build without the GIL, which libraries are gradually supporting.
Read full answerHide full answer
Applies toCPython 3.12 (free-threaded builds from 3.13 are experimental)
Impact by Workload
Workload
Threads help?
Better option
Pure-Python CPU loops
No
Processes, vectorization, Numba
NumPy / BLAS math
Yes (GIL released; BLAS is already multi-threaded)
n_jobs=-1 (processes or GIL-free threads internally)
Code Example
import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
import numpy as np
def python_cpu_task(n=3_000_000):
total = 0
for i in range(n): # pure Python: holds the GIL
total += i * i
return total
def numpy_task(seed):
rng = np.random.default_rng(seed)
a = rng.random((600, 600))
return float(np.linalg.eigvalsh(a @ a.T).max()) # C/LAPACK code releases the GIL
def io_task(seconds=0.5):
time.sleep(seconds) # stands in for network or disk waits; releases the GIL
return seconds
def timed(label, fn):
start = time.perf_counter()
fn()
print(f"{label:34s} {time.perf_counter() - start:5.2f}s")
if __name__ == "__main__":
timed("CPU task x4, serial", lambda: [python_cpu_task() for _ in range(4)])
with ThreadPoolExecutor(4) as pool:
timed("CPU task x4, threads (GIL-bound)", lambda: list(pool.map(lambda _: python_cpu_task(), range(4))))
with ProcessPoolExecutor(4) as pool:
timed("CPU task x4, processes", lambda: list(pool.map(python_cpu_task, [3_000_000] * 4)))
with ThreadPoolExecutor(8) as pool:
timed("I/O task x8, threads", lambda: list(pool.map(io_task, [0.5] * 8))) # about 0.5s, not 4s
with ThreadPoolExecutor(4) as pool:
timed("NumPy task x4, threads", lambda: list(pool.map(numpy_task, range(4))))
Practical Guidance
Vectorize first; it removes most Python-level loops.
Use processes for CPU-bound Python work; beware of copying large data between them.
Use threads or asyncio for I/O such as downloading datasets or calling model APIs.
In serving, scale with multiple worker processes rather than threads.
Common Pitfalls
Expecting threads to speed up pandas apply with Python functions.
Spawning many processes that each load a large model, exhausting memory.
Oversubscription: processes × BLAS threads exceeding the number of cores.
Interview Tips
Define the GIL and separate CPU-bound from I/O-bound work.
Explain why NumPy and deep learning frameworks are mostly unaffected.
75
How do you use context managers in Python? Give an ML example.
KnownRevise
A context manager sets something up when a with block starts and reliably cleans it up when the block ends, even if an exception occurs; it is defined by __enter__ and __exit__ methods or, more simply, with the @contextlib.contextmanager decorator around a generator that yields once. Familiar examples are with open(...) for files, with torch.no_grad(): for inference, and with mlflow.start_run(): for experiment tracking; in your own ML code they are useful for timing training stages, temporarily changing settings (random seeds, NumPy error handling, model eval mode), managing database connections, and making sure resources such as GPU memory or temporary files are released.
Read full answerHide full answer
Applies toPython 3.12
Code Example
import os
import tempfile
import time
from contextlib import contextmanager
import numpy as np
class Timer:
"""Class-based context manager that records elapsed time."""
def __init__(self, label):
self.label = label
def __enter__(self):
self.start = time.perf_counter()
return self
def __exit__(self, exc_type, exc, tb):
self.elapsed = time.perf_counter() - self.start
print(f"{self.label}: {self.elapsed:.3f}s")
return False # don't swallow exceptions
@contextmanager
def temporary_seed(seed):
"""Run a block with a fixed NumPy global seed, then restore the previous state."""
state = np.random.get_state()
np.random.seed(seed)
try:
yield
finally:
np.random.set_state(state)
@contextmanager
def checkpoint_dir():
"""A scratch directory for checkpoints that is always removed afterwards."""
with tempfile.TemporaryDirectory() as path:
yield path
with Timer("matrix training stand-in"):
X = np.random.default_rng(42).normal(size=(1500, 1500))
_ = X @ X.T
with temporary_seed(42):
a = np.random.rand(3)
with temporary_seed(42):
b = np.random.rand(3)
print(np.array_equal(a, b)) # True: the same seed inside both blocks
with checkpoint_dir() as d:
path = os.path.join(d, "model.npy")
np.save(path, np.arange(5))
print(os.path.exists(path)) # True
print(os.path.exists(path)) # False: cleaned up automatically
with np.errstate(divide="ignore", invalid="ignore"): # built-in NumPy context manager
ratio = np.array([1.0, 0.0]) / np.array([0.0, 0.0])
print(ratio) # [inf nan] without warnings
Common ML Context Managers
Context manager
Purpose
open(...)
Files closed automatically
torch.no_grad() / torch.inference_mode()
No gradient tracking
torch.autocast(...)
Mixed precision
mlflow.start_run()
Experiment run lifecycle
np.errstate(...)
Temporary floating-point error settings
tempfile.TemporaryDirectory()
Scratch space
threadpoolctl.threadpool_limits(...)
Limit BLAS threads temporarily
Common Pitfalls
Forgetting try/finally in a generator-based manager, so cleanup is skipped on errors.
Returning True from __exit__, which silently swallows exceptions.
Using a value outside the with block after its resource is closed.
Interview Tips
Explain __enter__/__exit__ and @contextmanager.
Give ML examples such as torch.no_grad() and a timing or seed manager.
76
What are __slots__ in Python classes and how can they help in ML code?
KnownRevise
Defining __slots__ in a class tells Python to reserve fixed attribute slots instead of creating a per-instance __dict__, which cuts memory per object substantially and makes attribute access slightly faster, while preventing new attributes from being added by mistake (a typo like self.lable raises an error instead of silently creating a new attribute). In ML applications this helps when you create millions of small Python objects, such as records for samples, tokens, graph nodes, tree nodes in a from-scratch model, or events in a streaming pipeline; for large numeric data, NumPy arrays or DataFrames are still far more efficient than any per-object design. Dataclasses support it with @dataclass(slots=True).
Read full answerHide full answer
Applies toPython 3.12
Code Example
import sys
import tracemalloc
from dataclasses import dataclass
class SampleDict:
def __init__(self, idx, label, weight):
self.idx, self.label, self.weight = idx, label, weight
class SampleSlots:
__slots__ = ("idx", "label", "weight")
def __init__(self, idx, label, weight):
self.idx, self.label, self.weight = idx, label, weight
@dataclass(slots=True)
class Token:
text: str
position: int
embedding_id: int
def memory_for(cls, n=200_000):
tracemalloc.start()
objs = [cls(i, i % 2, 1.0) for i in range(n)]
current, _ = tracemalloc.get_traced_memory()
tracemalloc.stop()
return current / 1e6, objs
dict_mb, _ = memory_for(SampleDict)
slots_mb, _ = memory_for(SampleSlots)
print(f"200k objects: with __dict__ {dict_mb:.1f} MB, with __slots__ {slots_mb:.1f} MB")
# __slots__ typically uses noticeably less memory
s = SampleSlots(1, 0, 1.0)
try:
s.lable = 1 # typo
except AttributeError as err:
print("AttributeError:", err)
t = Token("hello", 0, 42)
print(t, hasattr(t, "__dict__")) # Token(text='hello', position=0, embedding_id=42) False
print(sys.getsizeof(SampleDict(1, 0, 1.0).__dict__) > 0) # the per-instance dict that slots avoid
Trade-offs
Benefit
Cost
Less memory per instance
No dynamic attributes
Faster attribute access
Inheritance needs care (each subclass defines its own __slots__)
Catches attribute typos
Some libraries that rely on __dict__ (certain pickling or monkey-patching) need adjustments
When to Use
Millions of small, fixed-shape objects.
Long-running services where memory matters.
Not for a handful of objects, and not as a substitute for arrays of numbers.
Common Pitfalls
Expecting large savings for few objects.
Forgetting "__weakref__" in __slots__ if you need weak references.
Using lots of objects when a NumPy array or DataFrame would be better.
Interview Tips
Explain the missing __dict__ and the memory saving.
Mention @dataclass(slots=True) and when arrays are better.
77
How does the collections module help manage data structures for ML?
KnownRevise
The collections module adds specialized containers that make common data-handling tasks shorter and faster: Counter counts items (class balance, word frequencies, vocabulary building), defaultdict groups values without key checks (samples by label, features by user), deque keeps a fixed-size sliding window with O(1) appends and pops at both ends (moving averages of loss, replay buffers in reinforcement learning, streaming features), namedtuple creates lightweight, readable records, OrderedDict offers order-aware operations such as move_to_end for LRU caches, and ChainMap layers configuration dictionaries (defaults, file, command-line overrides).
Read full answerHide full answer
Applies toPython 3.12
Containers
Container
Use in ML code
Counter
Class counts, token frequencies, top-k
defaultdict(list)
Group samples or predictions by key
deque(maxlen=n)
Rolling windows, replay buffers
namedtuple
Immutable records (experiment results)
OrderedDict
LRU caches (move_to_end, popitem(last=False))
ChainMap
Layered configs
Code Example
from collections import ChainMap, Counter, OrderedDict, defaultdict, deque, namedtuple
import numpy as np
labels = ["cat", "dog", "cat", "bird", "cat", "dog"]
counts = Counter(labels)
print(counts.most_common(2), round(counts["cat"] / len(labels), 3)) # [('cat', 3), ('dog', 2)] 0.5
text = "the model learns the data and the model improves"
vocab = {w: i for i, (w, _) in enumerate(Counter(text.split()).most_common())}
print(list(vocab)[:3]) # ['the', 'model', 'learns']
by_label = defaultdict(list)
for idx, label in enumerate(labels):
by_label[label].append(idx)
print(dict(by_label)) # {'cat': [0, 2, 4], 'dog': [1, 5], 'bird': [3]}
window = deque(maxlen=3) # moving average of the last 3 losses
for loss in [1.0, 0.8, 0.6, 0.5, 0.45]:
window.append(loss)
print(round(sum(window) / len(window), 3)) # (0.6 + 0.5 + 0.45) / 3 = 0.517
rng = np.random.default_rng(42)
replay = deque(maxlen=1000) # RL replay buffer: oldest transitions drop off
for t in range(1500):
replay.append((t, int(rng.integers(2)), float(rng.random())))
print(len(replay), replay[0][0]) # 1000 500
Result = namedtuple("Result", ["model", "accuracy", "seconds"])
results = [Result("logreg", 0.91, 1.2), Result("forest", 0.94, 8.5)]
best = max(results, key=lambda r: r.accuracy)
print(best.model, best.accuracy) # forest 0.94
defaults = {"lr": 1e-3, "epochs": 10, "batch_size": 32}
file_cfg = {"epochs": 20}
cli = {"lr": 3e-4}
config = ChainMap(cli, file_cfg, defaults) # first mapping wins
print(config["lr"], config["epochs"], config["batch_size"]) # 0.0003 20 32
cache = OrderedDict()
for key in ["a", "b", "c", "a", "d"]: # tiny LRU cache with capacity 3
cache[key] = True
cache.move_to_end(key)
if len(cache) > 3:
cache.popitem(last=False)
print(list(cache)) # ['c', 'a', 'd']
Common Pitfalls
defaultdict silently creating keys when you only meant to read (d[k] on a missing key inserts it).
Using lists as queues (list.pop(0) is O(n)); use deque.
Counting with manual dictionaries instead of Counter.
Interview Tips
Name each container with an ML use.
Show deque(maxlen=...) for windows and replay buffers.
78
What is a microservices architecture for deploying ML models?
KnownRevise
In a microservices architecture, an application is split into small, independently deployable services that talk over the network (HTTP/REST, gRPC or message queues); for machine learning, the model runs as its own inference service with a clear API, separate from services such as feature computation, user management or the web front end. This lets the model team deploy, scale and roll back models without redeploying everything, run the model on different hardware (GPUs), use different language stacks, and test new versions with canary releases or A/B tests; the costs are network latency, more operational complexity (monitoring, service discovery, versioned APIs), and the need for consistent features between training and serving, often solved with a feature store.
Read full answerHide full answer
Applies togeneral architecture; examples with FastAPI 0.110+ and Docker
Typical Services
Service
Responsibility
API gateway
Authentication, routing, rate limiting
Feature service / feature store
Same features for training and serving
Model inference service
Loads model, validates input, predicts
Model registry
Versions and metadata (MLflow)
Training pipeline
Scheduled retraining jobs (Airflow, Kubeflow)
Monitoring
Latency, errors, drift, prediction logs
Message queue
Async or batch requests (Kafka, RabbitMQ)
Benefits and Costs
Benefit
Cost
Independent deploys and rollbacks
Network latency between services
Scale the model service separately (GPU nodes)
More infrastructure to operate
Different tech stacks per service
Distributed debugging and tracing
Clear ownership
API versioning and backward compatibility
Safe experiments (canary, A/B)
Data consistency across services
Code Example: A Minimal Inference Service
from fastapi import FastAPI
from pydantic import BaseModel
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
MODEL = LogisticRegression(max_iter=1000).fit(X, y) # in production: load from a model registry
MODEL_VERSION = "iris-logreg-1.0.0"
app = FastAPI(title="iris-inference")
class Features(BaseModel):
sepal_length: float
sepal_width: float
petal_length: float
petal_width: float
@app.get("/health")
def health():
return {"status": "ok", "model_version": MODEL_VERSION}
@app.post("/v1/predict")
def predict(f: Features):
row = [[f.sepal_length, f.sepal_width, f.petal_length, f.petal_width]]
proba = MODEL.predict_proba(row)[0]
return {"class_id": int(proba.argmax()), "probability": round(float(proba.max()), 4),
"model_version": MODEL_VERSION}
# Run: uvicorn service:app --port 8080. Package with a Dockerfile; deploy behind a gateway,
# scale replicas on Kubernetes, and route a small share of traffic to a new version first.
When Not to Use Microservices
A small team with one model and low traffic: a single application or batch job is simpler.
Very low-latency paths where an extra network hop is too slow: embed the model in the calling service.
Common Pitfalls
Computing features differently in the training pipeline and the inference service.
No API versioning, so model changes break clients.
Too many tiny services for a small system.
Interview Tips
Describe the inference service and its neighbours (features, registry, monitoring).
Weigh the benefits against latency and operational overhead.
79
What should you consider when scaling a Python machine learning application?
KnownRevise
Scaling an ML application means handling more data, more training compute and more prediction traffic while keeping latency, cost and reliability under control: for data, move from in-memory pandas to columnar formats (Parquet), chunked processing, Polars, Dask or Spark; for training, use vectorized code, all CPU cores (n_jobs), GPUs, distributed training and efficient hyperparameter search; for serving, run stateless model services with multiple worker processes behind a load balancer, autoscale replicas, batch requests, cache frequent predictions and optimize models (quantization, ONNX Runtime). Around all of this you need monitoring (latency, throughput, errors, drift), reproducible deployments (containers, pinned versions), and cost tracking.
Read full answerHide full answer
Applies toPython 3.12 ecosystem
Considerations
Area
Scaling techniques
Data volume
Parquet, chunking, Polars, Dask, Spark, sampling for exploration
Feature computation
Precompute in batch, feature store, incremental updates
Training
Vectorization, n_jobs, GPUs, distributed (DDP, Ray Train), early stopping
Model optimization, caching, keeping models in memory, async I/O
Python limits
GIL (use processes), memory per worker, startup time
Reliability
Health checks, timeouts, graceful degradation, rollbacks
Cost
Right-size instances, spot instances for training, batch vs real-time choice
Observability
Metrics, logs, tracing, data and prediction drift
Code Example: Scaling Data Processing with Chunks
import numpy as np
import pandas as pd
rng = np.random.default_rng(42)
pd.DataFrame({
"user_id": rng.integers(0, 10_000, 1_000_000),
"amount": rng.lognormal(3, 1, 1_000_000).round(2),
}).to_csv("transactions.csv", index=False) # stands in for a file too big for memory
totals, counts = pd.Series(dtype=float), pd.Series(dtype=float)
for chunk in pd.read_csv("transactions.csv", chunksize=200_000, dtype={"user_id": "int32", "amount": "float32"}):
grouped = chunk.groupby("user_id")["amount"]
totals = totals.add(grouped.sum(), fill_value=0) # combine partial aggregates
counts = counts.add(grouped.count(), fill_value=0)
features = pd.DataFrame({"total": totals, "mean": totals / counts})
print(features.shape, features["total"].sum().round(0) > 0) # (10000, 2) True
features.to_parquet("user_features.parquet") # columnar and typed for the next stage
For larger data, the same aggregation in Polars (pl.scan_csv(...).group_by("user_id").agg(...)) or Dask runs lazily and in parallel.
Serving at Scale
Run several Uvicorn/Gunicorn worker processes per container; load the model once per worker.
Scale containers horizontally on Kubernetes or a managed platform with autoscaling on CPU or request rate.
Use batch scoring for predictions that don’t need to be real time.
Common Pitfalls
Scaling infrastructure before profiling the code.
Loading the full dataset into pandas when only a few columns are needed (usecols).
Each worker loading a huge model and exhausting memory.
No load testing before launch.
Interview Tips
Separate data, training and serving scaling.
Mention profiling first, then horizontal scaling and monitoring.
80
What are the options for deploying a Python machine learning model?
KnownRevise
The main deployment options are batch scoring (a scheduled job that writes predictions to a database or file), a real-time REST or gRPC API (FastAPI, Flask, or model servers like TorchServe, NVIDIA Triton, BentoML, MLflow serving), streaming inference attached to a message queue (Kafka consumers), serverless functions (AWS Lambda, Google Cloud Functions) for spiky, light traffic, managed ML platforms (SageMaker, Vertex AI, Azure ML) that handle hosting and scaling, and edge or in-browser deployment (ONNX Runtime, TensorFlow Lite / LiteRT, Core ML, ONNX Runtime Web) for on-device predictions. The choice depends on latency needs, traffic patterns, model size, team skills and cost, and most options package the model and its dependencies in a Docker container.
Read full answerHide full answer
Applies toPython 3.12 ecosystem
Options
Option
Latency
Best for
Tools
Batch scoring
Minutes to hours
Daily churn scores, reports
Airflow, cron, Spark
REST/gRPC API
Milliseconds
User-facing predictions
FastAPI, Flask, BentoML
Model server
Milliseconds, high throughput
GPU models, many models
TorchServe, Triton, TF Serving
Streaming
Seconds
Event-driven fraud, IoT
Kafka, Flink, Faust
Serverless
Cold starts possible
Spiky, small models
Lambda, Cloud Run, Cloud Functions
Managed platform
Configurable
Teams wanting less ops
SageMaker, Vertex AI, Azure ML
Edge / mobile / browser
Local, offline
Privacy, no network
ONNX Runtime, LiteRT, Core ML
Embedded in an app
In-process
Simple apps, notebooks to dashboards
Streamlit, Gradio
Code Example: Same Model, Two Deployment Styles
import joblib
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
X, y = make_classification(n_samples=1000, n_features=5, random_state=42)
joblib.dump(RandomForestClassifier(random_state=42).fit(X, y), "model.joblib")
# 1) Batch scoring job (run nightly by a scheduler)
def batch_score(input_path, output_path, model_path="model.joblib"):
model = joblib.load(model_path)
df = pd.read_parquet(input_path)
df["score"] = model.predict_proba(df[[f"f{i}" for i in range(5)]])[:, 1]
df[["customer_id", "score"]].to_parquet(output_path, index=False)
return len(df)
rng = np.random.default_rng(42)
pd.DataFrame(rng.normal(size=(200, 5)), columns=[f"f{i}" for i in range(5)]).assign(
customer_id=range(200)).to_parquet("customers.parquet")
print("scored rows:", batch_score("customers.parquet", "scores.parquet"))
# 2) Real-time API: the same model behind FastAPI (run with `uvicorn app:app`)
API_CODE = '''
from fastapi import FastAPI
import joblib
app = FastAPI()
model = joblib.load("model.joblib")
@app.post("/predict")
def predict(features: list[float]):
return {"score": float(model.predict_proba([features])[0, 1])}
'''
print(API_CODE.strip().splitlines()[0])
Choosing
Predictions needed instantly by users → API or model server.
Predictions needed for many records at once → batch.
Data arrives as events → streaming.
No connectivity or strict privacy → edge.
Common Pitfalls
Building a real-time service when daily batch predictions would do.
Not containerizing, so dependencies differ between environments.
No monitoring or rollback plan.
Interview Tips
List batch, API, streaming, serverless, managed and edge with a use case for each.
Explain how latency and traffic drive the choice.
81
How does containerization with Docker benefit ML applications?
KnownRevise
Docker packages an ML application with everything it needs (a specific Python version, pinned libraries such as NumPy, scikit-learn or PyTorch, system libraries like CUDA runtimes, the model file and the serving code) into an image that runs the same way on a laptop, a CI server, a cloud VM or a Kubernetes cluster. That removes “works on my machine” problems, makes training and inference environments reproducible, allows rollback by redeploying an older image tag, isolates dependencies between projects, and makes scaling straightforward because orchestrators start more identical containers; GPU workloads use NVIDIA’s container toolkit with CUDA base images.
Read full answerHide full answer
Applies toDocker 24+, Python 3.12
Benefits
Benefit
Why it matters for ML
Reproducibility
Same library versions for training and serving
Portability
Run anywhere Docker runs: local, cloud, on-prem
Isolation
Conflicting dependencies (e.g. two CUDA versions) on one host
Versioning and rollback
Image tags per model version
Scaling
Kubernetes or ECS start more replicas
CI/CD
Build, test and deploy the same artifact
GPU support
CUDA base images with --gpus all
Example Dockerfile for a Model API
FROM python:3.12-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt # pinned: fastapi==..., scikit-learn==..., joblib==...
COPY app/ ./app/
COPY models/churn_model.joblib ./models/
RUN useradd --create-home appuser
USER appuser
EXPOSE 8000
HEALTHCHECK CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]
Build and run: docker build -t churn-api:1.3.0 . then docker run -p 8000:8000 churn-api:1.3.0. For GPUs, start from an image such as pytorch/pytorch or nvidia/cuda and run with docker run --gpus all ....
Code Example: The App Inside the Container
from pathlib import Path
import joblib
from fastapi import FastAPI
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
MODEL_PATH = Path("models/churn_model.joblib")
if not MODEL_PATH.exists(): # for local testing; the image ships a trained model
MODEL_PATH.parent.mkdir(parents=True, exist_ok=True)
X, y = make_classification(n_samples=500, n_features=4, random_state=42)
joblib.dump(LogisticRegression().fit(X, y), MODEL_PATH)
model = joblib.load(MODEL_PATH)
app = FastAPI()
@app.get("/health")
def health():
return {"status": "ok"}
@app.post("/predict")
def predict(features: list[float]):
return {"churn_probability": round(float(model.predict_proba([features])[0, 1]), 4)}
Best Practices
Pin dependency versions and use small base images (-slim).
Copy requirements.txt before the code so the dependency layer is cached.
Don’t bake secrets into images; pass them as environment variables or secrets.
Run as a non-root user and add health checks.
Store large models in a registry or object storage if they change often.
Common Pitfalls
Huge images from including datasets, caches or full CUDA toolkits when not needed.
latest tags in production, which make rollbacks unclear.
Training and serving images with different library versions.
Interview Tips
Lead with reproducibility and portability.
Describe a Dockerfile and how images fit CI/CD and orchestration.
82
What is model versioning and how do you manage it in practice?
KnownRevise
Model versioning means giving every trained model a unique, traceable identity linked to everything needed to reproduce and audit it: the code commit, training data version, hyperparameters, environment (library versions), evaluation metrics, and the model artifact itself, plus its lifecycle stage (staging, production, archived). It lets you compare versions, roll back quickly when a new model misbehaves, run A/B tests, and answer “which model made this prediction?” In practice teams use a model registry (MLflow Model Registry, Weights & Biases, SageMaker or Vertex AI registries), data versioning (DVC, lakeFS, or dated snapshots), git for code, semantic or incrementing version numbers, and they log the model version with every prediction.
Read full answerHide full answer
Applies toMLflow 2.x, Python 3.12
What to Version
Item
How
Code
Git commit hash
Data
DVC, lakeFS, snapshot paths with hashes
Parameters
Logged config (YAML/JSON)
Environment
requirements.txt / lock file, Docker image tag
Artifact
Model file in a registry or object storage
Metrics
Validation and test results
Stage / alias
staging, production, champion / challenger
Lineage
Which data and run produced it
Code Example: MLflow Tracking and Registry
import mlflow
import mlflow.sklearn
from mlflow.models import infer_signature
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split
mlflow.set_tracking_uri("sqlite:///mlflow.db") # local registry for the example
mlflow.set_experiment("cancer-classifier")
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
params = {"n_estimators": 300, "max_depth": 6, "random_state": 42}
with mlflow.start_run() as run:
model = RandomForestClassifier(**params).fit(X_tr, y_tr)
auc = roc_auc_score(y_te, model.predict_proba(X_te)[:, 1])
mlflow.log_params(params)
mlflow.log_metric("test_auc", auc)
mlflow.set_tag("data_version", "breast_cancer_sklearn_v1")
mlflow.sklearn.log_model(model, artifact_path="model", signature=infer_signature(X_tr, model.predict(X_tr)),
registered_model_name="cancer-classifier")
client = mlflow.MlflowClient()
latest = max(client.search_model_versions("name='cancer-classifier'"), key=lambda v: int(v.version))
client.set_registered_model_alias("cancer-classifier", "champion", latest.version)
print(f"registered version {latest.version} as champion (AUC {auc:.3f})")
loaded = mlflow.sklearn.load_model("models:/cancer-classifier@champion") # serving code loads by alias
print(loaded.predict(X_te.head(3)))
Rolling back is then a matter of pointing the champion alias at the previous version.
Practices
Never overwrite a model file in place; every retrain creates a new version.
Log the model version in prediction logs and API responses.
Promote through stages with checks (tests, metric thresholds, approval).
Keep older versions available for rollback and audits.
Common Pitfalls
Versioning the model but not the data, so results can’t be reproduced.
Manual file names like model_final_v2_new.pkl.
No record of which version served which predictions.
Interview Tips
List what must be versioned and why (reproducibility, rollback, audit).
Name a registry and the promotion/rollback flow.
83
How would you investigate a machine learning model failure using Python?
KnownRevise
Take a concrete scenario: a churn model that scored well offline suddenly flags far fewer customers, and actual churn rises. Investigate systematically: confirm the symptom with monitoring data, check the inputs first (schema changes, missing values, new categories, unit changes, upstream pipeline bugs), compare feature distributions between training data and recent production data for drift, compare the prediction distribution over time, evaluate the model on recently labelled data by segment, verify that the serving pipeline transforms features exactly as training did, and only then consider retraining. Python tools for this are pandas for slicing data, SciPy statistics or Evidently for drift, the model’s own predictions for slice analysis, and SHAP or permutation importance to see which features changed the model’s behaviour.
A column becomes all null after an upstream change
Missing-value rates, schema checks
Unit or encoding change
Amounts sent in cents instead of rupees
Distribution shift in one feature
New categories
A new plan type unseen in training
Unknown-category counts
Concept drift
Customer behaviour changes after a competitor launches
Performance drops on fresh labels
Training-serving skew
Different preprocessing code in production
Compare features for the same records
Feedback loops
Model decisions change future data
Policy and outcome analysis
Code Example: Diagnosing Drift and a Broken Feature
import numpy as np
import pandas as pd
from scipy.stats import ks_2samp
from sklearn.ensemble import HistGradientBoostingClassifier
rng = np.random.default_rng(42)
def make(n, amount_scale=1.0, null_rate=0.0):
df = pd.DataFrame({"tenure": rng.integers(1, 60, n), "monthly_amount": rng.normal(700, 150, n) * amount_scale,
"tickets": rng.poisson(1, n)})
df.loc[rng.random(n) < null_rate, "tenure"] = np.nan
logit = -1 - 0.05 * df["tenure"].fillna(30) + 0.004 * (df["monthly_amount"] / amount_scale - 700) + 0.6 * df["tickets"]
df["churn"] = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
return df
train = make(5000)
prod = make(2000, amount_scale=100, null_rate=0.3) # amounts now in cents; tenure often missing
features = ["tenure", "monthly_amount", "tickets"]
model = HistGradientBoostingClassifier(random_state=42).fit(train[features], train["churn"])
report = []
for col in features:
stat, p = ks_2samp(train[col].dropna(), prod[col].dropna())
report.append({"feature": col, "train_mean": train[col].mean(), "prod_mean": prod[col].mean(),
"prod_null_rate": prod[col].isna().mean(), "ks_stat": stat})
print(pd.DataFrame(report).round(3).to_string(index=False))
# monthly_amount: mean jumps about 100x (a unit change); tenure: 30% nulls (a pipeline problem)
print(f"mean predicted churn: train {model.predict_proba(train[features])[:, 1].mean():.3f}, "
f"prod {model.predict_proba(prod[features])[:, 1].mean():.3f}")
print(f"actual churn: train {train['churn'].mean():.3f}, prod {prod['churn'].mean():.3f}")
fixed = prod.assign(monthly_amount=prod["monthly_amount"] / 100) # after fixing the unit bug
print(f"mean predicted churn after the fix: {model.predict_proba(fixed[features])[:, 1].mean():.3f}")
Investigation Checklist
Reproduce: same model version, same inputs, same outputs?
Inputs: schema, nulls, ranges, categories, units.
Drift: KS tests or PSI per feature; prediction distribution.
Performance by segment on recent labels.
Pipeline parity: recompute features offline for a sample of production requests.
Fix the root cause, add a data validation check (Great Expectations, Pandera), then retrain if the world changed.
Common Pitfalls
Retraining immediately, which can bake a data bug into the next model.
Looking only at aggregate metrics and missing a broken segment.
No input validation in the serving path.
Interview Tips
Use a concrete scenario and a step-by-step investigation.
Check data and pipeline issues before blaming the model.
84
What are good strategies for logging and monitoring ML applications?
KnownRevise
Effective ML monitoring covers four layers: system health (latency, throughput, error rates, CPU/GPU and memory), data quality (schema violations, missing values, out-of-range inputs), model behaviour (prediction distributions, confidence, feature and prediction drift against the training baseline), and business outcomes (accuracy on delayed labels, conversion or fraud loss). Logging should be structured (JSON with request ID, model version, latency, key inputs or their hashes and the prediction), sampled where volume is high, free of personal data or secrets, and shipped to a central store; metrics go to tools like Prometheus and Grafana, with alerts on thresholds and drift, and dedicated tools such as Evidently, WhyLabs or Arize help with model-specific monitoring.
Log model version and request ID with every prediction.
Store a baseline of training feature distributions to compare against.
Join predictions with delayed ground truth to track real performance.
Alert on trends, not single noisy points; route alerts to an owner.
Redact personal data; follow retention policies.
Common Pitfalls
Monitoring only uptime while the model silently degrades.
Unstructured text logs that can’t be queried.
Logging full raw inputs with personal data.
Interview Tips
Name the four layers (system, data, model, outcomes) with a metric each.
Mention structured logs, drift checks and delayed labels.
85
What are Python's profiling tools and how do they help optimize ML code?
KnownRevise
Profilers measure where a program actually spends time or memory, so you optimize the real bottleneck instead of guessing: timeit times small snippets, cProfile (with pstats or the SnakeViz viewer) reports time per function, line_profiler shows time per line inside chosen functions, py-spy samples a running process with almost no overhead (useful in production), tracemalloc and memory_profiler track memory, Scalene reports CPU, memory and GPU together, and framework profilers such as torch.profiler break down operator and GPU time. In ML code they usually reveal Python loops that should be vectorized, slow data loading or preprocessing, repeated work that can be cached, and memory spikes from unnecessary copies.
Read full answerHide full answer
Applies toPython 3.12
Tools
Tool
Measures
When
timeit / %timeit
Snippet run time
Comparing two implementations
cProfile + pstats
Time per function
First look at a whole script
line_profiler (@profile, kernprof)
Time per line
Inside a hot function
py-spy
Sampling profiler, flame graphs
Running or production processes
Scalene
CPU, memory, GPU per line
Mixed Python/native code
tracemalloc, memory_profiler
Memory allocations
Memory growth, OOM errors
torch.profiler
Operator, CUDA and data-loading time
Deep learning training loops
Code Example
import cProfile
import io
import pstats
import timeit
import tracemalloc
import numpy as np
rng = np.random.default_rng(42)
X = rng.normal(size=(20_000, 20))
def zscore_loop(X):
out = np.empty_like(X)
for j in range(X.shape[1]):
col = X[:, j]
mean, std = col.mean(), col.std()
for i in range(X.shape[0]): # slow Python loop
out[i, j] = (X[i, j] - mean) / std
return out
def zscore_vectorized(X):
return (X - X.mean(axis=0)) / X.std(axis=0)
def pipeline():
a = zscore_loop(X)
b = zscore_vectorized(X)
return np.allclose(a, b)
profiler = cProfile.Profile()
profiler.enable()
same = pipeline()
profiler.disable()
stream = io.StringIO()
pstats.Stats(profiler, stream=stream).sort_stats("cumulative").print_stats(5)
print(same)
print("\n".join(stream.getvalue().splitlines()[:15])) # zscore_loop dominates the time
loop_t = timeit.timeit(lambda: zscore_loop(X[:2000]), number=3) / 3
vec_t = timeit.timeit(lambda: zscore_vectorized(X[:2000]), number=3) / 3
print(f"loop {loop_t * 1000:.1f} ms vs vectorized {vec_t * 1000:.2f} ms")
tracemalloc.start()
_ = [X.copy() for _ in range(5)] # an unnecessary 5 copies
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(f"peak memory during copies: {peak / 1e6:.1f} MB") # about 5 x 3.2 MB
Run py-spy top --pid <PID> or py-spy record -o profile.svg -- python train.py to profile a script without changing it.
Common Pitfalls
Optimizing before profiling.
Profiling tiny inputs where the bottleneck differs from real data sizes.
Timing GPU code without synchronization (torch.cuda.synchronize()).
Profiler overhead distorting very fast functions (use sampling profilers).
Interview Tips
Name a time profiler, a line profiler, a sampling profiler and a memory profiler.
Give a typical finding, such as a Python loop that vectorization fixes.
86
How do unit and integration tests keep ML code correct?
KnownRevise
Unit tests check small pieces of ML code in isolation with known inputs and outputs: a feature function returns the right values and shapes, a preprocessing step handles missing values and unseen categories, a custom loss matches a hand-computed result, and a model can overfit a tiny dataset; integration tests check that the pieces work together end to end: raw data goes through loading, feature engineering, training and prediction, the saved model reloads and gives the same predictions, and the serving API returns valid responses. Because ML outputs are numeric and partly random, tests use fixed seeds, tolerances (np.allclose, pytest.approx), and property checks such as probabilities summing to 1, alongside data validation tests and minimum-performance tests that guard against silent regressions.
Read full answerHide full answer
Applies topytest 8.x, Python 3.12
What to Test
Test type
Examples
Unit: features
Known input → known output, edge cases (empty, NaN, zero division)
Unit: shapes and dtypes
Model output shape, probability range
Unit: math
Custom metric/loss vs hand calculation
Unit: learning sanity
Overfits 10 samples; loss decreases
Data validation
Schema, ranges, no duplicates, label balance
Integration: pipeline
Train → save → load → predict on sample data
Integration: API
HTTP request → valid JSON, error handling
Regression / performance
Metric above a threshold on a fixed validation set
Code Example (test_pipeline.py, run with pytest -q)
import joblib
import numpy as np
import pandas as pd
import pytest
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
def add_ratio_feature(df: pd.DataFrame) -> pd.DataFrame:
"""Feature under test: spend per visit, 0 when there are no visits."""
visits = df["visits"].replace(0, np.nan)
return df.assign(spend_per_visit=(df["spend"] / visits).fillna(0.0))
def build_pipeline() -> Pipeline:
prep = ColumnTransformer([
("num", SimpleImputer(strategy="median"), ["spend", "visits", "spend_per_visit"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
])
return Pipeline([("prep", prep), ("model", LogisticRegression(max_iter=1000))])
@pytest.fixture
def sample_data():
rng = np.random.default_rng(42)
df = pd.DataFrame({"spend": rng.uniform(0, 100, 200), "visits": rng.integers(0, 10, 200),
"plan": rng.choice(["a", "b"], 200)})
y = (df["spend"] > 50).astype(int)
return add_ratio_feature(df), y
def test_ratio_feature_values_and_zero_division():
df = pd.DataFrame({"spend": [10.0, 5.0], "visits": [2, 0]})
assert add_ratio_feature(df)["spend_per_visit"].tolist() == [5.0, 0.0]
def test_predict_proba_is_valid(sample_data):
X, y = sample_data
proba = build_pipeline().fit(X, y).predict_proba(X)
assert proba.shape == (len(X), 2)
assert np.allclose(proba.sum(axis=1), 1.0)
def test_handles_unseen_category_and_missing_values(sample_data):
X, y = sample_data
pipe = build_pipeline().fit(X, y)
new = add_ratio_feature(pd.DataFrame({"spend": [np.nan], "visits": [3], "plan": ["zzz"]}))
assert pipe.predict(new).shape == (1,)
def test_save_load_round_trip(sample_data, tmp_path): # integration: train -> save -> load -> predict
X, y = sample_data
pipe = build_pipeline().fit(X, y)
path = tmp_path / "model.joblib"
joblib.dump(pipe, path)
assert np.array_equal(joblib.load(path).predict(X), pipe.predict(X))
def test_minimum_quality(sample_data):
X, y = sample_data
assert build_pipeline().fit(X, y).score(X, y) >= 0.9 # guards against silent regressions
Common Pitfalls
Testing only that code runs, not that results are correct.
Exact float comparisons; use tolerances.
Tests that depend on random seeds not being fixed, causing flaky failures.
Slow tests on full datasets; use small fixtures and mark slow tests separately.
Interview Tips
Separate unit (functions, math, shapes) from integration (pipeline, persistence, API).
Mention data validation and performance thresholds in CI.
87
How do you handle exceptions and errors when deploying ML models in Python?
KnownRevise
In a deployed model, error handling should validate inputs at the boundary (types, ranges, required fields, with Pydantic schemas) and reject bad requests with clear 4xx responses, catch only the specific exceptions you can handle (for example a missing feature store value or a timeout from a dependency), fall back safely when the model or a dependency fails (a cached prediction, a simple rule or a default decision), apply timeouts and retries with backoff to external calls, and log every failure with context (request ID, model version, stack trace) without leaking internals or personal data to the client. The model must also fail fast at start-up if it can’t load, and health checks should report readiness so the orchestrator doesn’t send traffic to a broken instance.
import logging
import time
from contextlib import asynccontextmanager
import numpy as np
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
logger = logging.getLogger("model-api")
state = {}
class FeatureStoreTimeout(Exception):
pass
@asynccontextmanager
async def lifespan(app: FastAPI):
X, y = make_classification(n_samples=500, n_features=3, random_state=42)
state["model"] = LogisticRegression().fit(X, y) # a load failure here stops start-up (fail fast)
yield
app = FastAPI(lifespan=lifespan)
class Request3(BaseModel):
customer_id: int = Field(gt=0)
amount: float = Field(ge=0, le=1_000_000)
tenure_months: int = Field(ge=0, le=600)
def fetch_history_feature(customer_id, retries=2, timeout=0.2):
for attempt in range(retries + 1):
try:
if customer_id % 13 == 0: # simulate a flaky dependency
raise FeatureStoreTimeout(f"feature store timed out after {timeout}s")
return float(customer_id % 7)
except FeatureStoreTimeout:
if attempt == retries:
raise
time.sleep(0.05 * 2 ** attempt) # exponential backoff
@app.get("/ready")
def ready():
return {"ready": "model" in state}
@app.post("/predict")
def predict(req: Request3): # Pydantic returns 422 for invalid input
try:
history = fetch_history_feature(req.customer_id)
fallback = False
except FeatureStoreTimeout:
logger.warning("feature store unavailable, using fallback", extra={"customer_id": req.customer_id})
history, fallback = 0.0, True # safe default
x = np.array([[req.amount / 1000, req.tenure_months / 12, history]])
if not np.isfinite(x).all():
raise HTTPException(status_code=422, detail="features must be finite")
score = float(state["model"].predict_proba(x)[0, 1])
return {"score": round(score, 4), "fallback_used": fallback}
@app.exception_handler(Exception)
async def unhandled(request: Request, exc: Exception):
logger.exception("unhandled error on %s", request.url.path) # full trace goes to the logs only
return JSONResponse(status_code=500, content={"error": "internal error"})
Common Pitfalls
Bare except: that hides bugs and swallows KeyboardInterrupt.
Returning stack traces or model internals to clients.
Silently returning a default prediction without logging or flagging it.
No timeouts on external calls, so one slow dependency blocks all workers.
Interview Tips
Separate validation errors, dependency failures and internal errors.
Mention fallbacks, retries with backoff, logging and health checks.
88
How do you write a Python generator that yields batches from a large dataset?
KnownRevise
A batch generator reads the data lazily and yields one batch at a time, so memory stays constant no matter how large the dataset is: for a big CSV, read it in chunks with pd.read_csv(chunksize=...) and re-slice the chunks into fixed-size batches; for a large NumPy array on disk, use np.load(..., mmap_mode="r") and slice shuffled index ranges; and keep leftover rows so no data is lost between chunks. Options such as shuffling with a seed, a drop_last flag and an infinite mode for training loops make it a practical replacement for loading everything into memory.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, pandas 2.x
Code Example
from collections.abc import Iterator
import numpy as np
import pandas as pd
def csv_batches(path: str, batch_size: int, chunk_size: int = 10_000, target: str = "label",
drop_last: bool = False) -> Iterator[tuple[np.ndarray, np.ndarray]]:
"""Stream (X, y) batches from a CSV that may not fit in memory."""
buffer = None
for chunk in pd.read_csv(path, chunksize=chunk_size):
buffer = chunk if buffer is None else pd.concat([buffer, chunk], ignore_index=True)
while len(buffer) >= batch_size:
batch, buffer = buffer.iloc[:batch_size], buffer.iloc[batch_size:]
yield batch.drop(columns=target).to_numpy(np.float32), batch[target].to_numpy()
if buffer is not None and len(buffer) and not drop_last:
yield buffer.drop(columns=target).to_numpy(np.float32), buffer[target].to_numpy() # final partial batch
def memmap_batches(X_path: str, y_path: str, batch_size: int, shuffle: bool = True, seed: int = 42,
epochs: int | None = 1) -> Iterator[tuple[np.ndarray, np.ndarray]]:
"""Batches from .npy files opened with memory mapping; epochs=None repeats forever."""
X = np.load(X_path, mmap_mode="r")
y = np.load(y_path, mmap_mode="r")
rng = np.random.default_rng(seed)
epoch = 0
while epochs is None or epoch < epochs:
order = rng.permutation(len(X)) if shuffle else np.arange(len(X))
for start in range(0, len(X), batch_size):
idx = np.sort(order[start:start + batch_size]) # sorted reads are faster on disk
yield np.asarray(X[idx]), np.asarray(y[idx])
epoch += 1
rng = np.random.default_rng(42)
data = pd.DataFrame(rng.normal(size=(25_000, 4)), columns=list("abcd"))
data["label"] = (data["a"] > 0).astype(int)
data.to_csv("big.csv", index=False)
np.save("X.npy", data[list("abcd")].to_numpy(np.float32))
np.save("y.npy", data["label"].to_numpy())
sizes = [len(yb) for _, yb in csv_batches("big.csv", batch_size=4096)]
print(len(sizes), sizes[-1], sum(sizes)) # 7 424 25000
gen = memmap_batches("X.npy", "y.npy", batch_size=1000, epochs=None)
for step, (xb, yb) in zip(range(3), gen): # take 3 batches from the infinite stream
print(step, xb.shape, xb.dtype, round(float(yb.mean()), 2))
The CSV has 25,000 rows: six full batches of 4,096 use 24,576 rows, leaving a final partial batch of 25,000 − 24,576 = 424 rows (dropped when drop_last=True).
Common Pitfalls
Losing the rows left over at the end of each chunk.
Converting the whole file to a list before batching, which defeats the purpose.
Shuffling only within chunks when data is sorted (for example by date or label); shuffle files beforehand or use a shuffle buffer.
Generators are single-use; create a new one per epoch or loop forever deliberately.
Interview Tips
Show yield, chunked reading and leftover handling.
Mention memory mapping and how frameworks (PyTorch IterableDataset, tf.data) build on the same idea.
89
How do you implement a convolutional neural network in PyTorch or TensorFlow?
KnownRevise
A CNN stacks convolution layers, which slide small learnable filters over the image to detect local patterns, with non-linear activations and pooling layers that shrink the spatial size, and ends with a classifier head; in Keras you define it with Conv2D, MaxPooling2D, Flatten/GlobalAveragePooling2D and Dense layers and train with compile and fit, and in PyTorch you subclass nn.Module with nn.Conv2d layers and write the training loop. On MNIST, even a small CNN like the one below typically reaches about 99% test accuracy within a few epochs, because convolutions share weights across positions and exploit the spatial structure of images.
import torch
import torch.nn as nn
torch.manual_seed(42)
class SmallCNN(nn.Module):
def __init__(self, n_classes=10):
super().__init__()
self.features = nn.Sequential(
nn.Conv2d(1, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
)
self.head = nn.Sequential(nn.Flatten(), nn.Dropout(0.3), nn.Linear(64 * 7 * 7, n_classes))
def forward(self, x): # x: (batch, 1, 28, 28), channels first
return self.head(self.features(x))
model = SmallCNN()
x = torch.randn(8, 1, 28, 28)
print(model(x).shape) # torch.Size([8, 10])
print(sum(p.numel() for p in model.parameters())) # 320 + 18,496 + 31,370 = 50,186
Layer-by-Layer Parameters (PyTorch version)
Conv 1: 32 filters × (3·3·1) + 32 biases = 320.
Conv 2: 64 × (3·3·32) + 64 = 18,496.
Dense: 64·7·7 = 3,136 inputs × 10 + 10 = 31,370.
Framework Differences
Aspect
Keras
PyTorch
Input layout
Channels last (N, H, W, C)
Channels first (N, C, H, W)
Training
compile + fit
Explicit loop
Output
Softmax in the model
Logits + CrossEntropyLoss
Common Pitfalls
Wrong channel order between frameworks.
Forgetting to scale pixel values to [0, 1] or normalize them.
Applying softmax before PyTorch’s CrossEntropyLoss.
Interview Tips
Explain convolution, pooling and weight sharing.
Compute output shapes and parameter counts for a layer.
90
How do you write a genetic algorithm in Python to optimize a simple problem?
KnownRevise
A genetic algorithm searches for good solutions by evolving a population of candidates: it scores each candidate with a fitness function, selects parents that favour fitter candidates (for example by tournament selection), creates children by crossover (mixing two parents) and mutation (small random changes), keeps the best few unchanged (elitism), and repeats for many generations. It needs no gradients, so it suits discrete, noisy or non-differentiable problems such as feature selection, scheduling or hyperparameter search; the example below solves a 0/1 knapsack problem and shows the best fitness improving over generations.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Components
Component
Knapsack choice
Representation
Bit string: 1 = item packed
Fitness
Total value, or 0 if over the weight limit
Selection
Tournament of 3
Crossover
Single-point, probability 0.9
Mutation
Flip each bit with probability 1/n_items
Elitism
Keep the best 2 each generation
Stopping
Fixed number of generations
Code Example
import numpy as np
rng = np.random.default_rng(42)
n_items = 30
weights = rng.integers(5, 40, n_items)
values = rng.integers(10, 100, n_items)
capacity = int(weights.sum() * 0.4)
def fitness(pop):
total_w = pop @ weights
total_v = pop @ values
return np.where(total_w <= capacity, total_v, 0) # infeasible solutions score 0
def tournament(pop, fit, k=3):
contestants = rng.integers(0, len(pop), k)
return pop[contestants[np.argmax(fit[contestants])]]
def genetic_algorithm(pop_size=100, generations=150, crossover_rate=0.9, elite=2):
pop = (rng.random((pop_size, n_items)) < 0.3).astype(int)
history = []
for _ in range(generations):
fit = fitness(pop)
history.append(int(fit.max()))
children = [pop[i].copy() for i in np.argsort(-fit)[:elite]] # elitism
while len(children) < pop_size:
p1, p2 = tournament(pop, fit), tournament(pop, fit)
if rng.random() < crossover_rate:
cut = rng.integers(1, n_items)
child = np.concatenate([p1[:cut], p2[cut:]])
else:
child = p1.copy()
flip = rng.random(n_items) < 1 / n_items # mutation
child[flip] = 1 - child[flip]
children.append(child)
pop = np.array(children)
fit = fitness(pop)
best = pop[np.argmax(fit)]
return best, int(fit.max()), history
best, best_value, history = genetic_algorithm()
print("best value by generation:", history[::30]) # non-decreasing thanks to elitism
print("best value:", best_value, "| weight used:", int(best @ weights), "of", capacity)
def greedy(): # baseline: best value-per-weight first
chosen, total = np.zeros(n_items, int), 0
for i in np.argsort(-values / weights):
if total + weights[i] <= capacity:
chosen[i], total = 1, total + weights[i]
return int(chosen @ values)
print("greedy baseline:", greedy()) # the GA usually matches or beats it
Tuning Tips
Larger populations explore more but cost more per generation.
Too much mutation turns the search random; too little stalls in local optima.
Penalty or repair functions handle constraints better than discarding infeasible solutions in harder problems.
Common Pitfalls
No elitism, so the best solution can be lost.
A fitness function that gives every infeasible solution the same low score, leaving no gradient toward feasibility in hard problems.
Comparing against nothing; always include a simple baseline.
Interview Tips
Describe selection, crossover, mutation and elitism.
Say when GAs fit (no gradients, discrete search) and when gradient methods are better.
91
How do you simulate and compare optimization techniques on a fixed dataset?
KnownRevise
To compare optimizers fairly, fix the dataset, the model, the loss, the starting weights and the random seed, then run each optimizer (for example batch gradient descent, mini-batch SGD, SGD with momentum, RMSprop and Adam) for the same budget of gradient evaluations or epochs, record the loss curve, and compare speed of convergence, final loss and stability, ideally over several seeds. Implementing the update rules yourself in NumPy for a logistic regression makes the differences visible: momentum smooths and accelerates steps, RMSprop scales each parameter’s step by its recent gradient size, and Adam combines both with bias correction.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x, Matplotlib 3.9+
Update Rules
Optimizer
Update
GD / SGD
w ← w − η·g
Momentum
v ← βv + g; w ← w − η·v
RMSprop
s ← ρs + (1 − ρ)g²; w ← w − η·g / (√s + ε)
Adam
m ← β₁m + (1 − β₁)g; s ← β₂s + (1 − β₂)g²; bias-correct; w ← w − η·m̂ / (√ŝ + ε)
Code Example
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=2000, n_features=10, n_informative=6, random_state=42)
X = np.column_stack([np.ones(len(X)), StandardScaler().fit_transform(X)])
def loss_and_grad(w, Xb, yb):
p = 1 / (1 + np.exp(-(Xb @ w)))
loss = -np.mean(yb * np.log(p + 1e-12) + (1 - yb) * np.log(1 - p + 1e-12))
return loss, Xb.T @ (p - yb) / len(yb)
def run(optimizer, lr, epochs=30, batch_size=64, seed=42):
rng = np.random.default_rng(seed)
w = np.zeros(X.shape[1])
m, s, t = np.zeros_like(w), np.zeros_like(w), 0
history = []
for _ in range(epochs):
order = rng.permutation(len(X)) if optimizer != "gd" else np.arange(len(X))
size = len(X) if optimizer == "gd" else batch_size
for start in range(0, len(X), size):
idx = order[start:start + size]
_, g = loss_and_grad(w, X[idx], y[idx])
t += 1
if optimizer in ("gd", "sgd"):
w -= lr * g
elif optimizer == "momentum":
m = 0.9 * m + g
w -= lr * m
elif optimizer == "rmsprop":
s = 0.9 * s + 0.1 * g ** 2
w -= lr * g / (np.sqrt(s) + 1e-8)
elif optimizer == "adam":
m = 0.9 * m + 0.1 * g
s = 0.999 * s + 0.001 * g ** 2
m_hat, s_hat = m / (1 - 0.9 ** t), s / (1 - 0.999 ** t)
w -= lr * m_hat / (np.sqrt(s_hat) + 1e-8)
history.append(loss_and_grad(w, X, y)[0]) # full-data loss after each epoch
return history
configs = {"gd": 0.5, "sgd": 0.1, "momentum": 0.01, "rmsprop": 0.01, "adam": 0.01}
results = {name: run(name, lr) for name, lr in configs.items()}
for name, hist in results.items():
print(f"{name:8s} loss after 1 epoch {hist[0]:.4f}, after 30 epochs {hist[-1]:.4f}")
# Full-batch GD makes one step per epoch, so it lags far behind the mini-batch methods early on
plt.figure(figsize=(7, 4))
for name, hist in results.items():
plt.plot(range(1, 31), hist, label=name)
plt.yscale("log")
plt.xlabel("epoch")
plt.ylabel("training log loss")
plt.legend()
plt.savefig("optimizer_comparison.png", dpi=120)
Fair Comparison Checklist
Same initialization, data order seeds and budget (epochs or gradient evaluations).
Tune each optimizer’s learning rate separately; a bad learning rate can make any optimizer look poor.
Repeat over several seeds and report mean and spread.
Compare validation loss too, not only training loss.
Common Pitfalls
Using one learning rate for all optimizers.
Comparing by iterations when per-iteration cost differs (full batch vs mini-batch).
Drawing conclusions from one run.
Interview Tips
Write the update rules and explain what momentum and adaptive scaling add.
Describe how you’d make the comparison fair.
92
How do you visualize a classifier's decision boundaries in Python?
KnownRevise
To visualize decision boundaries for a two-feature classifier, create a fine grid covering the feature space with np.meshgrid, predict the class (or probability) for every grid point, reshape the predictions to the grid’s shape, draw them with plt.contourf, and overlay the training points with plt.scatter; scikit-learn’s DecisionBoundaryDisplay.from_estimator does the grid work for you. Comparing several models side by side shows how they generalize: linear models draw straight lines, k-nearest neighbours and deep trees draw jagged regions that can overfit, and RBF SVMs and ensembles draw smooth curves. For data with more than two features, plot boundaries on two selected features or on a 2-D projection (PCA), knowing the picture is only a slice.
Read full answerHide full answer
Applies toscikit-learn 1.5+, Matplotlib 3.9+
Code Example
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_moons
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import DecisionBoundaryDisplay
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = make_moons(n_samples=400, noise=0.25, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=42)
models = {
"Logistic regression": make_pipeline(StandardScaler(), LogisticRegression()),
"k-NN (k=1)": KNeighborsClassifier(n_neighbors=1),
"RBF SVM": make_pipeline(StandardScaler(), SVC(gamma=2, C=1, probability=True, random_state=42)),
"Random forest": RandomForestClassifier(n_estimators=200, random_state=42),
}
fig, axes = plt.subplots(1, 4, figsize=(18, 4.2))
for ax, (name, model) in zip(axes, models.items()):
model.fit(X_tr, y_tr)
DecisionBoundaryDisplay.from_estimator(model, X, response_method="predict_proba", cmap="RdBu",
alpha=0.6, ax=ax, grid_resolution=200)
ax.scatter(X_tr[:, 0], X_tr[:, 1], c=y_tr, cmap="RdBu", edgecolors="k", s=18)
ax.set_title(f"{name}\ntest acc {model.score(X_te, y_te):.2f}")
fig.tight_layout()
fig.savefig("decision_boundaries.png", dpi=120)
# The same idea by hand with meshgrid + contourf
model = models["RBF SVM"]
xx, yy = np.meshgrid(np.linspace(X[:, 0].min() - 0.5, X[:, 0].max() + 0.5, 300),
np.linspace(X[:, 1].min() - 0.5, X[:, 1].max() + 0.5, 300))
grid = np.c_[xx.ravel(), yy.ravel()]
zz = model.predict_proba(grid)[:, 1].reshape(xx.shape)
fig, ax = plt.subplots(figsize=(5, 4))
contour = ax.contourf(xx, yy, zz, levels=20, cmap="RdBu", alpha=0.7)
ax.contour(xx, yy, zz, levels=[0.5], colors="k") # the decision boundary itself
ax.scatter(X[:, 0], X[:, 1], c=y, cmap="RdBu", edgecolors="k", s=15)
fig.colorbar(contour, label="P(class 1)")
fig.savefig("svm_boundary.png", dpi=120)
print("grid points predicted:", grid.shape[0]) # 90000
Reading the Plots
Pattern
Meaning
Straight line
Linear model; underfits curved data
Islands around single points
Overfitting (k = 1, deep trees)
Smooth curve following the data
Good fit
Wide uncertain band (probability ≈ 0.5)
Region where the model is unsure
Common Pitfalls
Grids too coarse, making boundaries look blocky.
Forgetting to apply the same scaling to the grid as to the data (pipelines handle it).
Treating a 2-D slice of a high-dimensional model as the full picture.
Interview Tips
Explain meshgrid → predict → reshape → contourf.
Use the plots to discuss underfitting and overfitting.
93
How do you implement the A* search algorithm for grid pathfinding in Python?
KnownRevise
A* finds a shortest path from a start cell to a goal by always expanding the cell with the lowest f(n) = g(n) + h(n), where g is the exact cost from the start and h is a heuristic estimate of the remaining cost; with an admissible heuristic (one that never overestimates, such as Manhattan distance on a 4-connected grid) the path it returns is optimal. A clean Python implementation uses heapq as the priority queue, a dictionary of best known g-costs, a came_from map to rebuild the path, and skips stale queue entries; with h = 0 it becomes Dijkstra’s algorithm, which explores many more cells.
Read full answerHide full answer
Applies toPython 3.12 (standard library)
Code Example
import heapq
GRID = [
"S...#......",
".##.#.####.",
".#..#....#.",
".#.###.#.#.",
".#.....#...",
".#####.###.",
".......#..G",
]
def parse(grid):
walls, start, goal = set(), None, None
for r, row in enumerate(grid):
for c, ch in enumerate(row):
if ch == "#":
walls.add((r, c))
elif ch == "S":
start = (r, c)
elif ch == "G":
goal = (r, c)
return walls, start, goal, len(grid), len(grid[0])
def manhattan(a, b):
return abs(a[0] - b[0]) + abs(a[1] - b[1])
def a_star(grid, heuristic=manhattan):
walls, start, goal, rows, cols = parse(grid)
open_heap = [(heuristic(start, goal), 0, start)] # (f, g, cell)
g_cost = {start: 0}
came_from = {}
expanded = 0
while open_heap:
f, g, cell = heapq.heappop(open_heap)
if g > g_cost.get(cell, float("inf")):
continue # stale entry
expanded += 1
if cell == goal:
path = [cell]
while cell in came_from:
cell = came_from[cell]
path.append(cell)
return path[::-1], expanded
r, c = cell
for nr, nc in ((r - 1, c), (r + 1, c), (r, c - 1), (r, c + 1)):
nxt = (nr, nc)
if not (0 <= nr < rows and 0 <= nc < cols) or nxt in walls:
continue
new_g = g + 1
if new_g < g_cost.get(nxt, float("inf")):
g_cost[nxt] = new_g
came_from[nxt] = cell
heapq.heappush(open_heap, (new_g + heuristic(nxt, goal), new_g, nxt))
return None, expanded # no path
path, expanded = a_star(GRID)
_, expanded_dijkstra = a_star(GRID, heuristic=lambda a, b: 0)
print("path length (moves):", len(path) - 1)
print("cells expanded: A*", expanded, "| Dijkstra", expanded_dijkstra) # A* expands fewer cells
canvas = [list(row) for row in GRID]
for r, c in path[1:-1]:
canvas[r][c] = "*"
print("\n".join("".join(row) for row in canvas))
Both searches return a path of the same (optimal) length, because the Manhattan distance never overestimates the true remaining cost on a 4-connected grid; A* simply reaches it after expanding fewer cells.
Heuristics
Movement
Admissible heuristic
4-connected, cost 1
Manhattan distance
8-connected, diagonal cost √2
Octile distance
Any direction
Euclidean distance
Weighted terrain
Heuristic × minimum step cost
Common Pitfalls
Inadmissible heuristics (e.g. Euclidean × 2), which can return non-optimal paths.
Not skipping stale heap entries, which wastes time.
Ties broken arbitrarily; adding g to the tuple (as above) or a counter keeps ordering stable.
Interview Tips
Explain f = g + h and admissibility.
Show the priority queue and path reconstruction, and compare with Dijkstra.
94
How do you implement a simple reinforcement learning agent for a basic game?
KnownRevise
A simple RL agent for a small game uses tabular Q-learning: define the game’s states, actions and rewards, keep a Q-table of estimated returns for each state-action pair, choose actions ε-greedily (mostly the best known action, sometimes a random one), and after every step update Q(s, a) ← Q(s, a) + α·(r + γ·max Q(s′, ·) − Q(s, a)). The example below teaches an agent a “catch the falling fruit” game on a small grid and compares its catch rate with a random player, showing learning without any neural network.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x
Game Design
Element
Definition
Board
5 columns, fruit falls from row 0 to row 4
State
(fruit column, fruit row, basket column) = 5 × 5 × 5 = 125 states
Actions
Move left, stay, move right
Reward
+1 catch, −1 miss (at the bottom row), 0 otherwise
Episode
One fruit drop (4 steps)
Code Example
import numpy as np
class CatchGame:
def __init__(self, width=5, height=5, seed=42):
self.width, self.height = width, height
self.rng = np.random.default_rng(seed)
def reset(self):
self.fruit_col = int(self.rng.integers(self.width))
self.fruit_row = 0
self.basket = int(self.rng.integers(self.width))
return self.state()
def state(self):
return (self.fruit_col, self.fruit_row, self.basket)
def step(self, action): # 0 = left, 1 = stay, 2 = right
self.basket = int(np.clip(self.basket + action - 1, 0, self.width - 1))
self.fruit_row += 1
if self.fruit_row == self.height - 1:
reward = 1.0 if self.basket == self.fruit_col else -1.0
return self.state(), reward, True
return self.state(), 0.0, False
def train(episodes=5000, alpha=0.2, gamma=0.95, seed=42):
env, rng = CatchGame(seed=seed), np.random.default_rng(seed)
Q = np.zeros((env.width, env.height, env.width, 3))
for ep in range(episodes):
s, done = env.reset(), False
epsilon = max(0.05, 1 - ep / (0.6 * episodes))
while not done:
a = int(rng.integers(3)) if rng.random() < epsilon else int(np.argmax(Q[s]))
s2, r, done = env.step(a)
target = r if done else r + gamma * Q[s2].max()
Q[s + (a,)] += alpha * (target - Q[s + (a,)])
s = s2
return Q
def catch_rate(policy, episodes=2000, seed=7):
env, rng = CatchGame(seed=seed), np.random.default_rng(seed)
caught = 0
for _ in range(episodes):
s, done = env.reset(), False
while not done:
s, r, done = env.step(policy(s, rng))
caught += r > 0
return caught / episodes
Q = train()
print(f"random player catch rate: {catch_rate(lambda s, rng: int(rng.integers(3))):.2f}")
print(f"trained agent catch rate: {catch_rate(lambda s, rng: int(np.argmax(Q[s]))):.2f}")
# The fruit takes 4 steps to fall and the basket can move up to 4 columns, so a perfect player
# always catches it; the trained agent gets close to 1.0
Why It Works
Rewards only arrive at the end, but the discounted update propagates value back to earlier states.
Exploration early on visits all states; decaying ε lets the agent exploit what it learned.
125 states × 3 actions is small enough for a table; bigger games need function approximation (DQN).
Common Pitfalls
Bootstrapping from terminal states.
Evaluating with exploration on.
A state that leaves out needed information (e.g. omitting the basket position).
Interview Tips
Define state, actions and reward before coding.
Write the Q-update and compare with a random baseline.
95
How do you forecast stock market time series with a Python library?
KnownRevise
For stock data, forecast with time-series tools while respecting what prices are: daily closing prices behave close to a random walk, so forecast returns (or log returns) rather than raw prices, use libraries such as statsmodels (ARIMA, exponential smoothing, GARCH via the arch package for volatility) or forecasting frameworks like Prophet, sktime, Darts or gradient boosting on lagged features, validate with walk-forward (rolling-origin) backtests, and always compare against the naive forecast (“tomorrow equals today”), which is very hard to beat for prices. Volatility is far more predictable than direction, and any model must be judged on out-of-sample errors after costs, never on in-sample fit.
Read full answerHide full answer
Applies tostatsmodels 0.14+, pandas 2.x (yfinance for real data)
Approach
Step
Detail
Data
Daily OHLCV (for real data: yfinance.download("AAPL", start=...))
Transform
Log returns: ln(Pₜ / Pₜ₋₁)
Baselines
Naive (last price), zero return, historical mean
Models
ARIMA on returns, ETS, GARCH for volatility, ML on lagged features
Validation
Walk-forward: train on [0, t], predict t + 1, roll forward
Metrics
MAE/RMSE of returns, directional accuracy, trading P&L after costs
Code Example
import warnings
import numpy as np
import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
warnings.filterwarnings("ignore") # ARIMA convergence messages on noisy data
rng = np.random.default_rng(42)
dates = pd.bdate_range("2024-01-01", periods=600)
returns = 0.0003 + 0.1 * np.r_[0, rng.normal(0, 0.012, 599)] # small AR structure + noise
for t in range(1, 600):
returns[t] += 0.1 * returns[t - 1]
prices = pd.Series(100 * np.exp(np.cumsum(returns)), index=dates, name="close")
# Real data instead: prices = yfinance.download("AAPL", start="2024-01-01")["Close"].squeeze()
log_ret = np.log(prices).diff().dropna()
train_end = 500
naive_err, model_err, direction_hits = [], [], []
history = list(log_ret.iloc[:train_end])
for t in range(train_end, len(log_ret)): # walk-forward evaluation
fit = ARIMA(history, order=(1, 0, 0)).fit()
forecast = float(fit.forecast(1)[0])
actual = float(log_ret.iloc[t])
model_err.append(abs(actual - forecast))
naive_err.append(abs(actual - 0.0)) # "no change" forecast for returns
direction_hits.append(np.sign(forecast) == np.sign(actual))
history.append(actual)
print(f"MAE of returns: ARIMA {np.mean(model_err):.5f} vs naive {np.mean(naive_err):.5f}")
print(f"directional accuracy: {np.mean(direction_hits):.3f}") # near 0.5 is typical for real prices
next_price = prices.iloc[-1] * np.exp(ARIMA(log_ret, order=(1, 0, 0)).fit().forecast(1).iloc[0])
print(f"next-day price forecast: {next_price:.2f} (last close {prices.iloc[-1]:.2f})")
The synthetic series has only weak structure, which is realistic: improvements over the naive forecast, if any, are small.
Common Pitfalls
Random train/test splits, which leak the future.
Forecasting prices and reporting a low error that just copies yesterday’s price.
Look-ahead bias in features (using today’s close to predict today).
Ignoring transaction costs and regime changes; past patterns often vanish.
Interview Tips
Mention returns vs prices, walk-forward validation and the naive baseline.
Be realistic about predictability; volatility models are more reliable than direction forecasts.
96
What does quantum computing mean for machine learning, from a Python view?
KnownRevise
Quantum machine learning (QML) explores using quantum computers, which process information in qubits that can be in superpositions and become entangled, to speed up or improve learning tasks, for example through variational quantum circuits trained like neural networks, quantum kernels for classification, and quantum algorithms for sampling and optimization. Today it is a research field: available hardware is small and noisy (the “NISQ” era), proven speed-ups for practical ML problems are not established, and classical methods remain far better for real workloads; in Python you can experiment with simulators and cloud hardware through Qiskit (IBM), PennyLane (Xanadu), Cirq (Google) and Amazon Braket, which integrate with NumPy, PyTorch and JAX.
Correlations between qubits with no classical equivalent
Variational circuit
Parameterized quantum circuit trained with a classical optimizer
Quantum kernel
Data mapped into a quantum state space; similarity used by an SVM
Barren plateaus
Gradients vanish in large random circuits, making training hard
Noise / decoherence
Limits circuit depth on current hardware
Implications (Realistic View)
Area
Status
Speed-ups for standard ML (classification, regression)
Not demonstrated on practical problems
Data loading
Encoding large classical datasets into quantum states is a bottleneck
Optimization / sampling
Active research (QAOA, quantum annealing)
Quantum data (chemistry, physics simulations)
The most promising application area
Cryptography
Future large quantum computers threaten some public-key schemes; post-quantum cryptography is being adopted
Code Example: A Variational Classifier with PennyLane (Simulator)
import pennylane as qml
from pennylane import numpy as pnp
from sklearn.datasets import make_moons
from sklearn.preprocessing import MinMaxScaler
X, y = make_moons(n_samples=60, noise=0.1, random_state=42)
X = MinMaxScaler((0, pnp.pi)).fit_transform(X) # angles for encoding
y_signed = 2 * y - 1 # labels -1 / +1
dev = qml.device("default.qubit", wires=2) # classical simulator
@qml.qnode(dev)
def circuit(weights, x):
qml.AngleEmbedding(x, wires=[0, 1]) # encode the 2 features
qml.StronglyEntanglingLayers(weights, wires=[0, 1]) # trainable entangling layers
return qml.expval(qml.PauliZ(0)) # output in [-1, 1]
def cost(weights):
preds = pnp.stack([circuit(weights, x) for x in X])
return pnp.mean((preds - y_signed) ** 2)
pnp.random.seed(42)
weights = pnp.random.uniform(0, 2 * pnp.pi, size=qml.StronglyEntanglingLayers.shape(n_layers=3, n_wires=2),
requires_grad=True)
opt = qml.AdamOptimizer(0.1)
for step in range(40):
weights, loss = opt.step_and_cost(cost, weights)
accuracy = pnp.mean(pnp.sign(pnp.stack([circuit(weights, x) for x in X])) == y_signed)
print(f"final loss {float(loss):.3f}, training accuracy {float(accuracy):.2f}")
This runs on a classical simulator; a logistic regression or small neural network solves the same task faster and better, which reflects the current state of the field.
Common Pitfalls in Answers
Claiming quantum computers already speed up mainstream ML.
Ignoring noise, qubit counts and data-loading costs.
Confusing quantum-inspired classical algorithms with quantum hardware results.
Interview Tips
Explain qubits and variational circuits briefly, then give a realistic assessment.
Name the Python libraries and say you’d experiment on simulators.
97
What is the role of Explainable AI and which Python libraries support it?
KnownRevise
Explainable AI (XAI) provides methods that make a model’s predictions understandable to people: global explanations show which features drive the model overall, local explanations show why it made one specific prediction, and both help debug models, detect bias and leakage, build trust with users, and meet regulatory requirements (such as explanations for credit decisions). In Python, SHAP computes Shapley-value attributions for any model (fast exact versions for trees), LIME fits simple local surrogate models, scikit-learn provides permutation importance and partial dependence plots, InterpretML offers inherently interpretable boosting models (EBMs), Captum explains PyTorch networks (integrated gradients, saliency), and Fairlearn and AIF360 assess fairness across groups.
Read full answerHide full answer
Applies toscikit-learn 1.5+, shap 0.45+
Methods
Method
Scope
Library
Interpretable models (linear, small trees, EBMs)
Global + local
scikit-learn, InterpretML
Permutation importance
Global
sklearn.inspection
Partial dependence / ICE plots
Global / per sample
sklearn.inspection
SHAP values
Local, aggregated to global
shap
LIME
Local
lime
Integrated gradients, saliency
Local (deep learning)
Captum, tf-explain
Counterfactuals
Local (“what would change the outcome”)
DiCE, Alibi
Fairness metrics
Group level
Fairlearn, AIF360
Code Example
import matplotlib
matplotlib.use("Agg")
import numpy as np
import pandas as pd
import shap
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.inspection import PartialDependenceDisplay, permutation_importance
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(42)
n = 3000
X = pd.DataFrame({
"income": rng.normal(60, 20, n),
"debt_ratio": rng.uniform(0, 1, n),
"late_payments": rng.poisson(1, n),
"noise": rng.normal(size=n),
})
logit = -0.04 * (X["income"] - 60) + 3 * (X["debt_ratio"] - 0.5) + 0.8 * X["late_payments"] - 1
y = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int) # 1 = default
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)
model = HistGradientBoostingClassifier(random_state=42).fit(X_tr, y_tr)
perm = permutation_importance(model, X_te, y_te, scoring="roc_auc", n_repeats=10, random_state=42)
print(pd.Series(perm.importances_mean, index=X.columns).sort_values(ascending=False).round(4))
# late_payments, debt_ratio and income matter; noise is close to 0
PartialDependenceDisplay.from_estimator(model, X_te, ["debt_ratio", "income"]).figure_.savefig("pdp.png")
explainer = shap.Explainer(model, X_tr) # global and local attributions
shap_values = explainer(X_te.iloc[:200])
applicant = 0
contrib = pd.Series(shap_values.values[applicant], index=X.columns).sort_values(key=abs, ascending=False)
print("why applicant 0 got their score (log-odds contributions):")
print(contrib.round(3))
Practical Advice
Explain on held-out data, and check explanations against domain knowledge.
Correlated features split or swap attributions; group them or interpret together.
Explanations describe the model, not causal effects in the real world.
For high-stakes decisions, consider inherently interpretable models first.
Common Pitfalls
Trusting impurity-based tree importances, which are biased toward high-cardinality features.
Presenting SHAP values to non-experts without translation into plain language.
Using explanations as proof a model is fair.
Interview Tips
Separate global from local explanations and name a method for each.
Mention SHAP, LIME, permutation importance and fairness tools.
98
How have deep learning advances changed NLP work in Python?
KnownRevise
Deep learning shifted NLP in Python from hand-built pipelines (tokenization rules, bag-of-words or TF-IDF features, and separate models per task) to pre-trained neural models that learn language representations from huge text corpora: word embeddings (word2vec, GloVe), then RNNs and LSTMs, then attention and Transformers, which led to BERT-style encoders for understanding tasks and GPT-style decoders, now large language models (LLMs), for generation. In practice this means most Python NLP work today starts from Hugging Face Transformers models and fine-tunes them, or uses an LLM through an API or an open-weights model with prompting, retrieval-augmented generation (RAG) and parameter-efficient fine-tuning (LoRA), while classic tools (spaCy, NLTK, scikit-learn) remain useful for preprocessing, fast baselines and resource-limited settings.
Read full answerHide full answer
Applies toHugging Face Transformers 4.40+, PyTorch 2.x, Python 3.12
from transformers import pipeline
ner = pipeline("ner", model="dslim/bert-base-NER", aggregation_strategy="simple")
print([(e["word"], e["entity_group"]) for e in ner("Ada Lovelace worked with Charles Babbage in London.")])
# e.g. [('Ada Lovelace', 'PER'), ('Charles Babbage', 'PER'), ('London', 'LOC')]
zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
result = zero_shot("The battery drains in two hours and support never replied.",
candidate_labels=["battery", "customer service", "price", "screen"], multi_label=True)
print({label: round(score, 2) for label, score in zip(result["labels"], result["scores"])})
summarizer = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6")
text = ("Transformers process all tokens in parallel using self-attention, which lets them capture "
"long-range dependencies. Pre-training on large corpora followed by fine-tuning on small labelled "
"datasets became the standard approach for NLP tasks such as classification and question answering.")
print(summarizer(text, max_length=40, min_length=10)[0]["summary_text"])
Remaining Challenges
Hallucinations and factuality in generative models.
Compute cost and latency; distillation and quantization help.
Bias, safety and evaluation of open-ended outputs.
Domain adaptation for specialized text (legal, medical).
Common Pitfalls
Using an LLM where a small fine-tuned classifier is cheaper and more reliable.
Skipping baselines; TF-IDF + logistic regression is still a strong check.
Evaluating generation only by eye.
Interview Tips
Trace the progression from TF-IDF to embeddings, RNNs, Transformers and LLMs.
Mention Hugging Face, fine-tuning vs prompting, and RAG.
99
How do big data technologies integrate with Python in ML projects?
KnownRevise
When data outgrows a single machine’s memory, Python connects to big data systems that distribute storage and computation: PySpark runs DataFrame transformations and Spark MLlib models across a cluster, Dask scales pandas- and NumPy-style code with familiar APIs, Ray parallelizes Python functions, training and hyperparameter tuning, Polars and DuckDB process tens to hundreds of gigabytes quickly on one machine, and cloud warehouses (BigQuery, Snowflake, Redshift) run SQL at scale with Python clients. A common pattern is to do heavy filtering, joins and feature aggregation in Spark, Dask or the warehouse, store results in columnar formats (Parquet, Delta Lake, Iceberg), then train with scikit-learn, XGBoost or PyTorch on the reduced feature set, or with distributed training when the data is still too big.
Fast multi-threaded single-machine DataFrames, lazy queries
Up to hundreds of GB
DuckDB
In-process SQL on Parquet/CSV
Fast analytics without a cluster
Warehouses
Managed SQL at scale
Feature extraction close to the data
Kafka + Spark Streaming / Flink
Streams
Real-time features
Parquet / Delta Lake / Iceberg
Columnar, versioned storage
Shared data layer
Code Example: Same Aggregation, Three Engines
import duckdb
import numpy as np
import pandas as pd
import polars as pl
rng = np.random.default_rng(42)
n = 2_000_000
pd.DataFrame({
"user_id": rng.integers(0, 50_000, n),
"event": rng.choice(["view", "click", "buy"], n, p=[0.8, 0.17, 0.03]),
"amount": rng.lognormal(3, 1, n).round(2),
}).to_parquet("events.parquet") # stands in for a partitioned data lake table
polars_features = (pl.scan_parquet("events.parquet") # lazy: only needed columns are read
.group_by("user_id")
.agg(pl.len().alias("events"),
(pl.col("event") == "buy").sum().alias("purchases"),
pl.col("amount").filter(pl.col("event") == "buy").sum().alias("spend"))
.collect())
duck_features = duckdb.sql("""
SELECT user_id, COUNT(*) AS events,
SUM(CASE WHEN event = 'buy' THEN 1 ELSE 0 END) AS purchases,
SUM(CASE WHEN event = 'buy' THEN amount ELSE 0 END) AS spend
FROM 'events.parquet' GROUP BY user_id
""").df()
print(polars_features.shape, duck_features.shape) # (about 50000, 4) for both
print(int(polars_features["purchases"].sum()) == int(duck_features["purchases"].sum())) # True
# The same logic in PySpark on a cluster:
# spark.read.parquet("s3://bucket/events/").groupBy("user_id").agg(
# F.count("*").alias("events"),
# F.sum(F.when(F.col("event") == "buy", 1).otherwise(0)).alias("purchases"))
Integration Patterns
Push computation to the data (SQL in the warehouse, Spark on the lake) and bring back features.
Sample for exploration in pandas, run full jobs at scale.
Convert between engines with Apache Arrow for zero-copy transfers.
Orchestrate pipelines with Airflow, Dagster or Prefect.
Common Pitfalls
Calling .toPandas() or .compute() on huge results and running out of memory.
Reaching for a cluster when Polars or DuckDB on one machine is enough.
Small-file problems in data lakes (millions of tiny Parquet files).
Interview Tips
Match tools to data size: pandas → Polars/DuckDB → Dask/Spark/Ray.
Describe the aggregate-at-scale, train-on-features pattern.
100
What is federated learning and how can you implement it in Python?
KnownRevise
Federated learning trains a shared model across many devices or organizations (phones, hospitals, banks) without moving their raw data to a central server: each client trains the current global model on its local data for a few steps and sends only the model updates back, and the server aggregates them, typically by Federated Averaging (FedAvg, a weighted average of client weights by number of samples), then sends out the improved model for the next round. It protects privacy and helps with data residency rules, but brings challenges: non-identically distributed client data, unreliable or slow clients, communication cost, and updates that can still leak information, which is why it is combined with secure aggregation and differential privacy. In Python, Flower is the most widely used framework, alongside TensorFlow Federated, PySyft and NVIDIA FLARE.
Read full answerHide full answer
Applies toPython 3.12, NumPy 2.x; Flower 1.x for production setups
How a Round Works
Server sends global weights w to a sample of clients.
Each client k trains locally on its n_k samples and returns w_k.
Server computes w ← Σ (n_k / n) · w_k.
Repeat for many rounds.
Challenges and Mitigations
Challenge
Mitigation
Non-IID data across clients
FedProx, personalization layers, more rounds
Stragglers / dropouts
Client sampling, asynchronous aggregation
Communication cost
Fewer, larger local steps; compression, quantized updates
import numpy as np
from sklearn.datasets import make_classification
rng = np.random.default_rng(42)
X, y = make_classification(n_samples=6000, n_features=10, n_informative=6, random_state=42)
X = np.column_stack([np.ones(len(X)), X])
order = np.argsort(X[:, 1]) # non-IID split: clients see different feature ranges
client_sizes = [3000, 1500, 1000, 500]
splits = np.split(order, np.cumsum(client_sizes)[:-1])
clients = [(X[idx], y[idx]) for idx in splits]
def sigmoid(z):
return 1 / (1 + np.exp(-z))
def local_train(w, X_k, y_k, epochs=5, lr=0.1, batch=64):
w = w.copy()
for _ in range(epochs):
perm = rng.permutation(len(X_k))
for s in range(0, len(X_k), batch):
idx = perm[s:s + batch]
w -= lr * X_k[idx].T @ (sigmoid(X_k[idx] @ w) - y_k[idx]) / len(idx)
return w
def accuracy(w):
return float(np.mean((sigmoid(X @ w) > 0.5) == y))
w_global = np.zeros(X.shape[1])
for round_ in range(1, 11):
updates, sizes = [], []
for X_k, y_k in clients: # in reality this runs on each device
updates.append(local_train(w_global, X_k, y_k))
sizes.append(len(y_k))
w_global = np.average(updates, axis=0, weights=sizes) # FedAvg: weighted by sample count
if round_ in (1, 5, 10):
print(f"round {round_}: global accuracy {accuracy(w_global):.3f}")
w_central = local_train(np.zeros(X.shape[1]), X, y, epochs=10) # reference: data pooled centrally
print(f"centralized training accuracy: {accuracy(w_central):.3f}")
local_only = local_train(np.zeros(X.shape[1]), *clients[3], epochs=50)
print(f"smallest client alone: {accuracy(local_only):.3f}") # usually worse on the full population
In a real deployment, Flower provides the client and server roles: a NumPyClient implements fit and evaluate on local data, and the server runs a FedAvg strategy over the network.
Common Pitfalls
Assuming federated learning is automatically private (updates can leak data without secure aggregation or differential privacy).
Testing only with IID splits, which hides the hard part.
Ignoring client availability and bandwidth.
Interview Tips
Explain the FedAvg round and why raw data never leaves clients.
Name non-IID data and privacy leakage as the key challenges, and Flower as the Python tool.