Skip to content
Machine Learning & Data Science

PythonML interview questions and answers

100 questions, 100 with code. About 5 h 38 min to read them all.

By TestLab · Updated

Plan for an interview
100 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

What are the differences between Python 2 and Python 3?

Python 3 is the current language and Python 2 reached end of life on 1 January 2020, so new code, and every major ML library (NumPy, pandas, scikit-learn, PyTorch, TensorFlow), is Python 3 only; the key differences are that print became a function, strings are Unicode text by default with a separate bytes type, / always performs true division, many built-ins (range, map, zip, dict.items) return lazy iterables instead of lists, and integer types were unified. Python 3 also added features ML code relies on daily: f-strings, type hints, async/await, dataclasses, the @ matrix-multiplication operator, and pathlib.

Read full answer

Applies toPython 2.7 vs Python 3.12

Key Differences

Feature Python 2 Python 3
Print print "x" statement print("x") function
Strings str = bytes, unicode separate str = Unicode text, bytes separate
Division 3 / 2 == 1 (for ints) 3 / 2 == 1.5; // is floor division
range, map, zip, filter Return lists Return lazy iterators/views
Integers int and long Single arbitrary-precision int
Exceptions except E, e: except E as e:
Input raw_input() input()
Formatting % and .format Also f-strings: f"{x:.2f}"
Matrix multiply np.dot(a, b) a @ b (PEP 465)
Support Ended 2020 Active; yearly releases

Code Example

from pathlib import Path

import numpy as np

print(7 / 2, 7 // 2)                   # 3.5 3
text = "naïve café"
print(len(text), len(text.encode("utf-8")))   # 10 characters, 12 bytes
squares = map(lambda n: n * n, range(4))
print(squares.__class__.__name__, list(squares))   # map [0, 1, 4, 9]

a = np.array([[1, 2], [3, 4]])
print(a @ a)                           # [[ 7 10] [15 22]]

accuracy = 0.91234
print(f"accuracy = {accuracy:.2%}")    # accuracy = 91.23%
print(Path("data") / "train.csv")      # data/train.csv
print(2 ** 100)                        # big integers work without overflow

Why It Matters for ML

  • All modern ML libraries dropped Python 2.

  • Integer division bugs (accuracy = correct / total returning 0) disappeared.

  • Unicode strings make text processing (NLP) far less error-prone.

Common Pitfalls

  • Copying Python 2 snippets from old tutorials.

  • Expecting zip or map results to be reusable lists (they are one-pass iterators).

Interview Tips

  • Lead with print, Unicode strings, division and lazy iterators.

  • Mention that Python 2 is end-of-life and libraries are Python 3 only.

2

How does Python manage memory?

CPython (the standard interpreter) manages memory automatically: every value is an object on a private heap, and each object carries a reference count that is increased when a name, container or argument refers to it and decreased when that reference goes away; when the count reaches zero the object is freed immediately. A cyclic garbage collector (the gc module) periodically finds groups of objects that reference each other but are unreachable, which reference counting alone can’t free. Small objects are allocated from pools managed by the pymalloc allocator, and large data such as NumPy arrays and PyTorch tensors keep their buffers in separately allocated memory (C memory or GPU memory) owned by a Python object.

Read full answer

Applies toCPython 3.12

Layers

Layer Role
Names and references Variables are references to objects, not boxes holding values
Reference counting Immediate freeing when count hits 0
Cyclic GC (gc) Generational collector for reference cycles
pymalloc Fast allocator for small objects (≤ 512 bytes)
Extension buffers NumPy arrays, tensors: large raw buffers outside pymalloc

Code Example

import gc
import sys

import numpy as np

a = [1, 2, 3]
b = a                                  # second reference, same object
print(a is b, sys.getrefcount(a))      # True, a small count (includes the temporary argument reference)

b.append(4)
print(a)                               # [1, 2, 3, 4]: both names see the change

class Node:
    def __init__(self):
        self.other = None

x, y = Node(), Node()
x.other, y.other = y, x                # reference cycle
del x, y                               # counts don't reach zero
print("collected:", gc.collect() >= 2) # True: the cycle collector frees them

arr = np.zeros((1000, 1000))           # 8 MB buffer owned by the array object
view = arr[:10]                        # a view keeps the whole buffer alive
print(arr.nbytes / 1e6, view.base is arr)   # 8.0 True
del arr
print(view.base.nbytes / 1e6)          # 8.0: the buffer survives while the view exists

Memory Tips for ML Code

  • Delete large intermediate objects (del df_tmp) and avoid keeping references in lists or notebooks.

  • Views (slices) keep large arrays alive; use .copy() when you need only a small piece.

  • Use smaller dtypes (float32, category) for big tables.

  • Measure with tracemalloc, memory_profiler, or df.memory_usage(deep=True).

  • In PyTorch, store loss.item() rather than the loss tensor so graphs are freed.

Common Pitfalls

  • Thinking b = a copies a list.

  • Hidden references in notebook output history (Out[...]) keeping large objects alive.

  • Expecting freed memory to return to the operating system immediately (allocators may keep it).

Interview Tips

  • Explain reference counting plus the cyclic collector.

  • Connect to practical ML memory issues: views, dtypes, lingering references.

3

What is PEP 8 and why is it important?

PEP 8 is Python’s official style guide for code: it sets conventions for indentation (4 spaces), line length (79 characters in the guide, though many projects use 88–120), naming (snake_case for functions and variables, PascalCase for classes, UPPER_CASE for constants), whitespace, imports (one per line, grouped standard library / third-party / local) and comments. It matters because consistent style makes code easier to read, review and maintain across a team; in ML projects, where research code often becomes production code, it reduces friction, and tools such as Ruff and Black apply it automatically so people don’t argue about formatting.

Read full answer

Applies toPython 3.12; Ruff 0.6+, Black 24+

Key Rules

Area Convention
Indentation 4 spaces, no tabs
Line length 79 (guide); 88 (Black/Ruff default)
Names train_model, learning_rate, class DataLoader, MAX_EPOCHS
Imports Top of file; stdlib, third-party, local groups; no wildcard imports
Whitespace x = 1, f(a, b), no spaces inside brackets
Comparisons if x is None, if not items
Blank lines 2 around top-level definitions, 1 between methods
Docstrings PEP 257 triple-quoted summaries

Code Example

"""Train a simple model (PEP 8 style example)."""

import random
from dataclasses import dataclass

import numpy as np

RANDOM_SEED = 42


@dataclass
class TrainConfig:
    learning_rate: float = 0.1
    n_epochs: int = 50


def fit_mean_predictor(targets: np.ndarray, config: TrainConfig) -> float:
    """Learn a constant prediction by gradient descent on squared error."""
    prediction = 0.0
    for _ in range(config.n_epochs):
        gradient = 2 * (prediction - targets).mean()
        prediction -= config.learning_rate * gradient
    return prediction


if __name__ == "__main__":
    random.seed(RANDOM_SEED)
    rng = np.random.default_rng(RANDOM_SEED)
    y = rng.normal(loc=5.0, scale=1.0, size=1000)
    fitted = fit_mean_predictor(y, TrainConfig())
    print(f"fitted {fitted:.3f} vs sample mean {y.mean():.3f}")   # nearly equal

Tooling

  • ruff check . (linting) and ruff format . or black . (formatting).

  • pre-commit hooks to run them on every commit.

  • nbqa or Ruff’s notebook support for Jupyter notebooks.

Common Pitfalls

  • Treating PEP 8 as absolute; the guide itself says consistency within a project matters more.

  • Hand-formatting instead of using tools.

  • Single-letter names everywhere in ML code (X, y are fine conventions; a1, t2 are not).

Interview Tips

  • Give a few concrete rules and name the tools that enforce them.

  • Explain readability and team consistency as the reason.

4

What are the differences between a list, a tuple and a set in Python?

A list is an ordered, mutable sequence that allows duplicates ([1, 2, 2]); a tuple is an ordered, immutable sequence that also allows duplicates and can be hashed if its items are hashable ((1, 2)), so it works as a dictionary key; a set is an unordered, mutable collection of unique, hashable items ({1, 2}) with fast O(1) average membership tests and set algebra (union, intersection, difference). In ML code, lists hold growing collections (metrics per epoch), tuples hold fixed records (shapes, (X, y) pairs, dictionary keys), and sets handle uniqueness (vocabularies, de-duplicating IDs, comparing feature lists).

Read full answer

Applies toPython 3.12

Comparison

Property List Tuple Set
Syntax [1, 2] (1, 2) {1, 2} (empty: set())
Ordered Yes Yes No
Mutable Yes No Yes (frozenset is immutable)
Duplicates Allowed Allowed Removed
Indexing x[0] x[0] Not supported
Hashable (dict key) No Yes, if items are No (frozenset yes)
Membership in O(n) O(n) O(1) average
Typical ML use Loss history, batches Shapes, records, keys Vocabulary, unique labels

Code Example

losses = [0.9, 0.7, 0.7]
losses.append(0.5)
print(losses, losses[-1])                        # [0.9, 0.7, 0.7, 0.5] 0.5

shape = (32, 3, 224, 224)                        # fixed record
batch, channels, height, width = shape           # unpacking
cache = {(0.01, 64): 0.83, (0.1, 32): 0.79}      # tuples as dict keys (lr, batch_size)
print(channels, cache[(0.01, 64)])               # 3 0.83

train_ids = {101, 102, 103, 104}
test_ids = {104, 105}
print(train_ids & test_ids)                      # {104}: leakage between splits
print(sorted(train_ids | test_ids))              # [101, 102, 103, 104, 105]

tokens = "the cat sat on the mat".split()
vocab = sorted(set(tokens))
print(len(tokens), len(vocab), vocab)            # 6 5 ['cat', 'mat', 'on', 'sat', 'the']

try:
    shape[0] = 64
except TypeError as err:
    print("tuples are immutable:", err)

Common Pitfalls

  • (5) is just the integer 5; a one-item tuple needs a comma: (5,).

  • {} creates an empty dict, not an empty set.

  • Relying on set order (iteration order is not guaranteed).

  • Using x in big_list inside loops instead of converting to a set.

Interview Tips

  • Compare ordering, mutability, duplicates and hashability.

  • Give an ML use for each, such as detecting train/test overlap with sets.

5

How does a dictionary work in Python?

A Python dictionary (dict) maps unique keys to values and is implemented as a hash table: the key’s hash decides where the entry is stored, so looking up, inserting or deleting by key takes O(1) time on average; keys must be hashable (immutable types such as strings, numbers or tuples of them), while values can be anything, and since Python 3.7 dictionaries keep insertion order. In ML code dictionaries hold configurations and hyperparameters, label-to-index mappings, metric results, feature maps and model state_dicts.

Read full answer

Applies toPython 3.12

Operations

Operation Syntax Time (average)
Create {"lr": 0.01}, dict(lr=0.01)
Read d["lr"] (KeyError if missing), d.get("lr", default) O(1)
Write / update d["lr"] = 0.1, d.update(other), d | other O(1) per key
Delete del d["lr"], d.pop("lr", None) O(1)
Membership "lr" in d O(1)
Iterate for k, v in d.items() O(n)

Code Example

from collections import Counter, defaultdict

config = {"model": "resnet18", "lr": 1e-3, "epochs": 10}
config["batch_size"] = 64
print(config.get("dropout", 0.0), "lr" in config)    # 0.0 True

overrides = {"lr": 3e-4}
final = config | overrides                           # merge (Python 3.9+); right side wins
print(final["lr"], list(final)[:2])                  # 0.0003 ['model', 'lr']

labels = ["cat", "dog", "cat", "bird", "dog", "cat"]
label_to_idx = {label: i for i, label in enumerate(sorted(set(labels)))}
print(label_to_idx)                                  # {'bird': 0, 'cat': 1, 'dog': 2}
print([label_to_idx[x] for x in labels])             # [1, 2, 1, 0, 2, 1]

counts = Counter(labels)
print(counts.most_common(1))                         # [('cat', 3)]

by_split = defaultdict(list)
for sample_id, split in [(1, "train"), (2, "test"), (3, "train")]:
    by_split[split].append(sample_id)
print(dict(by_split))                                # {'train': [1, 3], 'test': [2]}

try:
    {["a"]: 1}
except TypeError as err:
    print("lists can't be keys:", err)

How Hashing Works (Briefly)

  • hash(key) gives an integer; the dict uses it to pick a slot.

  • Equal keys must have equal hashes; collisions are resolved by probing other slots.

  • The table resizes as it fills, keeping lookups fast.

Common Pitfalls

  • Mutable keys (lists, dicts) raise TypeError.

  • d[key] on a missing key raises KeyError; use get, setdefault or defaultdict.

  • Modifying a dictionary’s size while iterating over it.

Interview Tips

  • Explain hash tables and average O(1) access.

  • Show get, comprehensions, Counter and defaultdict.

6

What is a list comprehension in Python? Give an example.

A list comprehension builds a new list from an iterable in a single expression, [expression for item in iterable if condition], which is shorter and usually faster than an equivalent for loop with append. The same syntax creates dictionaries ({k: v for ...}), sets ({x for ...}) and lazy generators ((x for ...)). In ML code they are handy for small transformations such as building file lists or label maps, but large numeric work should use vectorized NumPy or pandas operations instead.

Read full answer

Applies toPython 3.12

Forms

Form Example Result
List [x ** 2 for x in range(5)] [0, 1, 4, 9, 16]
With filter [x for x in range(10) if x % 3 == 0] [0, 3, 6, 9]
With if/else ["pos" if x > 0 else "neg" for x in [3, -1]] ['pos', 'neg']
Nested [(i, j) for i in range(2) for j in range(2)] [(0, 0), (0, 1), (1, 0), (1, 1)]
Dict {w: len(w) for w in ["ml", "data"]} {'ml': 2, 'data': 4}
Set {w.lower() for w in ["A", "a", "B"]} {'a', 'b'}
Generator sum(x * x for x in range(1_000_000)) Computed lazily

Code Example

import numpy as np

files = ["img_01.png", "notes.txt", "img_02.png", "img_03.jpg"]
images = [f for f in files if f.endswith((".png", ".jpg"))]
print(images)                                   # ['img_01.png', 'img_02.png', 'img_03.jpg']

sentences = ["Deep learning rocks", "Python is great"]
tokens = [word.lower() for s in sentences for word in s.split()]
vocab = {word: i for i, word in enumerate(sorted(set(tokens)))}
print(tokens[:3], vocab["python"])             # ['deep', 'learning', 'rocks'] 4

scores = [0.2, 0.8, 0.55, 0.4]
labels = [1 if s >= 0.5 else 0 for s in scores]
print(labels)                                   # [0, 1, 1, 0]

rng = np.random.default_rng(42)
big = rng.random(1_000_000)
loop_style = [1 if s >= 0.5 else 0 for s in big]   # works, but slow for large arrays
vectorized = (big >= 0.5).astype(int)              # much faster: runs in C
print(sum(loop_style) == vectorized.sum())          # True

Common Pitfalls

  • Deeply nested comprehensions that are hard to read; use a loop or helper function.

  • Using a list comprehension only for side effects ([print(x) for x in xs]).

  • Building huge lists when a generator expression would stream the values.

Interview Tips

  • Write the general form and a filtered example.

  • Mention dict/set comprehensions and when vectorization is better.

7

What are generators in Python and how do they differ from list comprehensions?

A generator produces values lazily, one at a time, only when asked (by next() or a for loop), instead of building the whole collection in memory; you create one with a function that uses yield or with a generator expression in parentheses, (x * x for x in data). A list comprehension builds the complete list immediately, so it uses memory proportional to the number of items and can be indexed and reused, while a generator uses constant memory, can represent infinite or streaming sequences, and can be iterated only once. In ML, generators stream batches from files too large for memory, read logs line by line, and power data pipelines.

Read full answer

Applies toPython 3.12

Comparison

Aspect List comprehension Generator
Syntax [f(x) for x in it] (f(x) for x in it) or def ...: yield
Evaluation Eager: all at once Lazy: on demand
Memory O(n) O(1) per item
Reusable Yes No: exhausted after one pass
Indexing / len Yes No
Infinite sequences No Yes

Code Example

import sys
from itertools import islice

import numpy as np

squares_list = [x * x for x in range(1_000_000)]
squares_gen = (x * x for x in range(1_000_000))
print(sys.getsizeof(squares_list) > 1_000_000, sys.getsizeof(squares_gen) < 1_000)   # True True

def batch_generator(X, y, batch_size, seed=42):
    """Yield shuffled mini-batches forever, like a training data stream."""
    rng = np.random.default_rng(seed)
    n = len(X)
    while True:
        order = rng.permutation(n)
        for start in range(0, n, batch_size):
            idx = order[start:start + batch_size]
            yield X[idx], y[idx]

rng = np.random.default_rng(42)
X, y = rng.normal(size=(10, 3)), np.arange(10)
batches = batch_generator(X, y, batch_size=4)
for xb, yb in islice(batches, 3):              # take only 3 batches from the infinite stream
    print(xb.shape, len(yb))                    # (4, 3) 4, then (4, 3) 4, then (2, 3) 2

def read_lines(path):
    with open(path, encoding="utf-8") as f:
        for line in f:                          # files are already lazy iterators
            yield line.rstrip("\n")

with open("log.txt", "w", encoding="utf-8") as f:
    f.write("epoch 1 loss 0.9\nepoch 2 loss 0.6\n")
print(max(float(line.split()[-1]) for line in read_lines("log.txt")))   # 0.9

gen = (x for x in range(3))
print(list(gen), list(gen))                     # [0, 1, 2] []: exhausted after one pass

Common Pitfalls

  • Iterating a generator twice and getting nothing the second time.

  • Calling len() on a generator.

  • Using a generator when you need random access; convert with list() if it fits in memory.

Interview Tips

  • Explain lazy evaluation and memory use.

  • Give a batch-streaming example for ML.

8

How do you use *args and **kwargs in Python function definitions?

In a function definition, *args collects any extra positional arguments into a tuple and **kwargs collects any extra keyword arguments into a dictionary, so a function can accept a variable number of inputs; in a call, * and ** do the reverse, unpacking a sequence into positional arguments and a dictionary into keyword arguments. In ML code they are used to forward options to another function (for example passing hyperparameters through a wrapper to a scikit-learn or PyTorch constructor), to write decorators that wrap any function, and to build models from configuration dictionaries with Model(**config).

Read full answer

Applies toPython 3.12

Parameter Order

def f(positional, /, normal, *args, keyword_only, **kwargs)

Part Meaning
Before / Positional-only
Normal Positional or keyword
*args Extra positional arguments → tuple
After *args (or a bare *) Keyword-only
**kwargs Extra keyword arguments → dict

Code Example

from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

def describe(*args, **kwargs):
    print("args:", args, "| kwargs:", kwargs)

describe(1, 2, lr=0.01, epochs=5)        # args: (1, 2) | kwargs: {'lr': 0.01, 'epochs': 5}

def evaluate(model_cls, X, y, *, cv=5, **model_params):
    """Forward any hyperparameters to the model constructor."""
    model = model_cls(**model_params)
    return cross_val_score(model, X, y, cv=cv).mean()

X, y = make_classification(n_samples=300, n_features=10, random_state=42)
config = {"n_estimators": 100, "max_depth": 5, "random_state": 42}
score = evaluate(RandomForestClassifier, X, y, cv=3, **config)    # unpack the dict
print(f"mean CV accuracy: {score:.3f}")

def weighted_mean(*values, weights=None):
    weights = weights or [1] * len(values)
    return sum(v * w for v, w in zip(values, weights)) / sum(weights)

scores = [0.8, 0.9, 0.7]
print(round(weighted_mean(*scores), 3))                    # 0.8
print(round(weighted_mean(*scores, weights=[1, 2, 1]), 3)) # (0.8 + 1.8 + 0.7) / 4 = 0.825

Common Pitfalls

  • Accepting **kwargs and silently ignoring misspelled options; validate or forward them to something that checks.

  • Confusing definition (*args collects) with call (*seq unpacks).

  • Overusing them, which hides a function’s real interface; type hints and explicit parameters are clearer.

Interview Tips

  • Explain packing in definitions and unpacking in calls.

  • Show forwarding hyperparameters with Model(**params).

9

How does Python's garbage collection work?

CPython frees most objects through reference counting: each object tracks how many references point to it and is deallocated the moment that count drops to zero; on top of that, a generational cyclic garbage collector (the gc module) finds and frees groups of container objects that reference each other but are no longer reachable from the program, which reference counting alone can’t free. The cyclic collector groups objects into generations (new objects start young and survivors move to older generations, which are scanned less often), runs automatically when allocation thresholds are crossed, and can be triggered with gc.collect() or tuned with gc.set_threshold.

Read full answer

Applies toCPython 3.12

Two Mechanisms

Mechanism Frees When
Reference counting Objects with no references Immediately
Cyclic GC Unreachable reference cycles (lists, dicts, class instances) Periodically, by generation thresholds

How the Cyclic Collector Finds Garbage

  1. Look only at container objects that can form cycles (lists, dicts, instances).

  2. For each object in the generation being scanned, subtract references that come from other objects in the same set.

  3. Objects still referenced from outside are reachable; everything reachable from them is kept.

  4. What remains is unreachable cyclic garbage and is freed (finalizers run first).

Code Example

import gc
import weakref

class Model:
    def __init__(self, name):
        self.name = name
        self.peer = None

    def __del__(self):
        print(f"freed {self.name}")

m = Model("solo")
del m                                   # prints "freed solo" right away: refcount hit 0

a, b = Model("a"), Model("b")
a.peer, b.peer = b, a                   # a cycle
watch = weakref.ref(a)                  # a weak reference doesn't keep a alive
del a, b
print("still alive before collect:", watch() is not None)   # True
gc.collect()                            # prints "freed" for a and b (order may vary)
print("alive after collect:", watch() is not None)          # False

print(gc.get_threshold())               # generation thresholds, e.g. (700, 10, 10) in 3.12

Practical Notes for ML Code

  • Most large ML objects (arrays, tensors, DataFrames) are freed by reference counting as soon as the last reference goes.

  • Lingering references, not the GC, are the usual cause of memory growth: lists of tensors, notebook Out[...] history, global caches, exception tracebacks.

  • gc.collect() plus torch.cuda.empty_cache() can help reclaim GPU memory in notebooks after deleting models.

  • Weak references (weakref) let caches refer to objects without keeping them alive.

Common Pitfalls

  • Calling gc.collect() in tight loops (slow, rarely needed).

  • Assuming del x frees memory when other references remain.

  • Disabling GC without understanding the cycles in your code.

Interview Tips

  • Explain reference counting first, then why cycles need a separate collector.

  • Mention generations and practical memory debugging.

10

What are decorators in Python and when would you use one?

A decorator is a function that takes another function (or class) and returns a modified version of it, applied with the @decorator syntax above a definition; it lets you add behaviour such as timing, logging, caching, retries, input validation or registration without changing the original function’s code. Python ships useful ones (@functools.lru_cache, @property, @staticmethod, @dataclass), and ML code commonly uses them for timing training steps, caching expensive feature computations, retrying flaky data downloads, registering models by name, and framework features such as @torch.no_grad() and @torch.compile.

Read full answer

Applies toPython 3.12

Code Example

import functools
import time

def timed(func):
    """Print how long each call takes."""
    @functools.wraps(func)                       # keep the original name and docstring
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        print(f"{func.__name__} took {time.perf_counter() - start:.4f}s")
        return result
    return wrapper

def retry(times=3, exceptions=(ConnectionError,)):
    """A decorator factory: takes arguments and returns a decorator."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, times + 1):
                try:
                    return func(*args, **kwargs)
                except exceptions:
                    if attempt == times:
                        raise
                    print(f"attempt {attempt} failed, retrying")
        return wrapper
    return decorator

MODELS = {}

def register(name):
    def decorator(cls):
        MODELS[name] = cls
        return cls
    return decorator

@register("baseline")
class MeanModel:
    def fit(self, y):
        self.mean_ = sum(y) / len(y)
        return self

@timed
@functools.lru_cache(maxsize=None)              # cache results of an expensive pure function
def expensive_feature(n):
    return sum(i * i for i in range(n))

calls = {"count": 0}

@retry(times=3)
def flaky_download():
    calls["count"] += 1
    if calls["count"] < 3:
        raise ConnectionError("network blip")
    return "data.csv"

print(expensive_feature(2_000_000))             # slow the first time
print(expensive_feature(2_000_000))             # near-instant: served from the cache
print(flaky_download())                         # two retries, then 'data.csv'
print(MODELS["baseline"]().fit([1, 2, 3]).mean_)   # 2.0

Common Uses

Use Example
Timing / logging @timed, logging inputs and outputs
Caching @functools.lru_cache, @functools.cache
Retries Network calls, cloud storage reads
Registration Model or dataset registries by name
Access control / validation Checking argument types or shapes
Framework hooks @torch.no_grad(), @torch.compile, @app.post (FastAPI)

Common Pitfalls

  • Forgetting functools.wraps, which hides the wrapped function’s name and docstring.

  • Caching functions that take unhashable arguments (lists, arrays) or have side effects.

  • Stacking decorators without realizing they apply bottom-up.

Interview Tips

  • Explain that @d above def f means f = d(f).

  • Write a timing decorator with wraps, and mention decorator factories.

11

Which Python libraries are most used in machine learning and what are they for?

The core Python ML stack is NumPy (fast n-dimensional arrays and linear algebra), pandas (tabular data loading, cleaning and transformation), Matplotlib and seaborn (plotting), SciPy (scientific routines: statistics, optimization, sparse matrices), scikit-learn (classical ML models, preprocessing, pipelines and evaluation), XGBoost / LightGBM / CatBoost (gradient-boosted trees for tabular data), statsmodels (statistical models and tests), and PyTorch or TensorFlow/Keras (deep learning), with Jupyter notebooks for interactive work. Around them sit Hugging Face Transformers for pre-trained NLP and vision models, Optuna for hyperparameter tuning, MLflow for experiment tracking, and Polars or Dask for larger data.

Read full answer

Applies toPython 3.12 ecosystem

Libraries by Purpose

Library Primary purpose Typical use
NumPy Arrays, vectorized math, linear algebra Feature matrices, custom algorithms
pandas DataFrames: load, clean, join, group CSV/SQL data preparation
Polars Fast DataFrames (multi-threaded, lazy) Large tables
Matplotlib / seaborn Static plots / statistical plots EDA, results
Plotly Interactive plots Dashboards
SciPy Stats, optimization, sparse, signal Hypothesis tests, sparse features
scikit-learn Classical ML, preprocessing, metrics Baselines, pipelines, CV
XGBoost / LightGBM / CatBoost Gradient boosting Tabular competitions and production
statsmodels Regression with inference, time series p-values, ARIMA
PyTorch Deep learning Research, custom models
TensorFlow / Keras Deep learning Production pipelines, mobile
Hugging Face Transformers Pre-trained models NLP, vision, LLMs
Optuna Hyperparameter search Tuning
MLflow / W&B Experiment tracking Reproducibility
Jupyter Notebooks Exploration, reports

Code Example: Several Libraries Together

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(42)                                  # NumPy: data
df = pd.DataFrame({"age": rng.integers(18, 70, 500), "income": rng.normal(50, 15, 500)})   # pandas
df["bought"] = ((df["income"] > 55) & (df["age"] < 50)).astype(int)

t_stat, p_value = stats.ttest_ind(df.loc[df.bought == 1, "income"], df.loc[df.bought == 0, "income"])
print(f"income differs by group: p = {p_value:.1e}")             # SciPy: statistics

X_train, X_test, y_train, y_test = train_test_split(df[["age", "income"]], df["bought"], random_state=42)
model = RandomForestClassifier(random_state=42).fit(X_train, y_train)   # scikit-learn
print(f"test accuracy: {model.score(X_test, y_test):.3f}")

df.plot.scatter(x="age", y="income", c="bought", colormap="coolwarm")   # pandas + Matplotlib
plt.savefig("customers.png", dpi=120)

Common Pitfalls

  • Reaching for deep learning on small tabular datasets, where boosted trees usually win.

  • Unpinned library versions, which break notebooks months later.

Interview Tips

  • Group libraries by stage: data, visualization, classical ML, boosting, deep learning, tooling.

  • Say which you’d pick for tabular vs image or text problems.

12

What is NumPy and how is it useful in machine learning?

NumPy is the foundational Python library for numerical computing: it provides the ndarray, a fixed-type, contiguous n-dimensional array, along with fast vectorized operations, broadcasting, linear algebra, random number generation and statistics, all running in compiled C instead of Python loops. In machine learning, datasets, feature matrices, images and model weights are arrays, and pandas, scikit-learn, SciPy and Matplotlib are built on NumPy; even PyTorch and TensorFlow tensors convert to and from NumPy arrays.

Read full answer

Applies toNumPy 2.x, Python 3.12

Why It Matters for ML

Feature Benefit
Vectorization 10–100× faster than Python loops
Broadcasting Operate on arrays of different shapes without copies (e.g. subtract feature means)
Linear algebra (@, np.linalg) Matrix products, solving, decompositions
Random generators Reproducible sampling (np.random.default_rng(42))
Fixed dtypes Compact memory (float32, int8)
Indexing / masking Fast filtering and selection

Code Example

import time

import numpy as np

rng = np.random.default_rng(42)
X = rng.normal(loc=10, scale=3, size=(100_000, 5))      # 100k samples, 5 features

X_std = (X - X.mean(axis=0)) / X.std(axis=0)           # broadcasting: standardize each column
print(X_std.mean(axis=0).round(6), X_std.std(axis=0).round(6))   # zeros and ones

start = time.perf_counter()
loop_sum = sum(x * x for x in X[:, 0])
loop_t = time.perf_counter() - start
start = time.perf_counter()
vec_sum = np.dot(X[:, 0], X[:, 0])
vec_t = time.perf_counter() - start
print(np.isclose(loop_sum, vec_sum), f"vectorized is ~{loop_t / vec_t:.0f}x faster")

# Linear regression with the normal equation: w = (X^T X)^-1 X^T y, solved stably with lstsq
true_w = np.array([2.0, -1.0, 0.5, 0.0, 3.0])
y = X_std @ true_w + 4 + rng.normal(scale=0.1, size=len(X_std))
X_b = np.column_stack([np.ones(len(X_std)), X_std])
w, *_ = np.linalg.lstsq(X_b, y, rcond=None)
print(w.round(2))                                       # close to [4, 2, -1, 0.5, 0, 3]

mask = y > 10
print(f"{mask.mean():.1%} of samples have y > 10")      # boolean masking

Common Pitfalls

  • Python loops over array elements instead of vectorized operations.

  • Shape mistakes: (n,) vs (n, 1) broadcasting into (n, n).

  • Views vs copies: slices share memory, so modifying them changes the original.

  • Using the legacy global np.random.seed; prefer a Generator from default_rng.

Interview Tips

  • Explain vectorization and broadcasting with an example.

  • Mention that most ML libraries build on NumPy arrays.

13

What is pandas and why is it important for data manipulation?

pandas is the standard Python library for working with tabular data: its DataFrame (a table of named, typed columns with an index) and Series (one column) let you load data from CSV, Excel, SQL, Parquet and JSON, then clean, filter, transform, join, reshape, group and aggregate it with concise, vectorized operations. In ML it is where most data preparation happens (handling missing values, encoding categories, creating features, splitting by time or group) before converting the result to NumPy arrays or passing DataFrames directly to scikit-learn.

Read full answer

Applies topandas 2.x, Python 3.12

Core Capabilities

Task pandas tools
Load / save read_csv, read_parquet, read_sql, to_parquet
Inspect head, info, describe, value_counts
Select / filter df["col"], df.loc[rows, cols], df.query(...), boolean masks
Clean isna, fillna, dropna, drop_duplicates, astype
Transform Vectorized arithmetic, .str and .dt accessors, assign
Aggregate groupby().agg(), pivot_table
Combine merge, concat, join
Reshape melt, pivot, stack
Time series DatetimeIndex, resample, rolling

Code Example

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
orders = pd.DataFrame({
    "order_id": range(1, 9),
    "customer": ["a", "b", "a", "c", "b", "a", "c", "d"],
    "amount": [120.0, 80.0, np.nan, 300.0, 45.0, 60.0, 210.0, 95.0],
    "date": pd.date_range("2026-01-01", periods=8, freq="W"),
})
customers = pd.DataFrame({"customer": ["a", "b", "c", "d"], "segment": ["gold", "silver", "gold", "new"]})

orders["amount"] = orders["amount"].fillna(orders["amount"].median())   # median of the 7 values = 95
orders["month"] = orders["date"].dt.month
df = orders.merge(customers, on="customer", how="left")

features = (df.groupby("customer")
              .agg(n_orders=("order_id", "count"), total=("amount", "sum"), avg=("amount", "mean"))
              .reset_index()
              .round(2))
print(features)
#   customer  n_orders  total    avg
# 0        a         3  275.0  91.67 (120 + 95 + 60)
# 1        b         2  125.0  62.50
# 2        c         2  510.0 255.00
# 3        d         1   95.0  95.00

print(df.pivot_table(index="segment", columns="month", values="amount", aggfunc="sum", fill_value=0))
X = features[["n_orders", "total", "avg"]].to_numpy()       # ready for a model
print(X.shape)                                              # (4, 3)

Common Pitfalls

  • Row-by-row loops with iterrows instead of vectorized operations.

  • Chained assignment (df[df.x > 0]["y"] = 1), which may not modify df; use df.loc[mask, "y"] = 1.

  • object dtype for strings and categories wasting memory; use category or the string dtype.

  • Silent row explosions from merges on non-unique keys (use validate="m:1").

Interview Tips

  • Name the main operations: load, clean, groupby, merge, reshape.

  • Show vectorized feature engineering with groupby().agg().

14

How does scikit-learn fit into the machine learning workflow?

scikit-learn is the standard Python library for classical machine learning, and it covers most of the workflow with one consistent API: preprocessing (scalers, encoders, imputers), feature selection and dimensionality reduction, models for classification, regression and clustering, Pipeline and ColumnTransformer to chain steps without leakage, model selection (train/test splits, cross-validation, grid and random search) and evaluation metrics. Every estimator follows fit / predict (or transform), so models are interchangeable, and fitted pipelines can be saved with joblib for deployment.

Read full answer

Applies toscikit-learn 1.5+, Python 3.12

Where It Fits

Workflow stage scikit-learn tools
Split data train_test_split, StratifiedKFold, TimeSeriesSplit, GroupKFold
Preprocess StandardScaler, OneHotEncoder, SimpleImputer, ColumnTransformer
Feature engineering PolynomialFeatures, SelectKBest, PCA
Model LogisticRegression, RandomForestClassifier, HistGradientBoostingClassifier, KMeans
Tune GridSearchCV, RandomizedSearchCV, HalvingGridSearchCV
Evaluate classification_report, roc_auc_score, cross_val_score, confusion_matrix
Persist joblib.dump / joblib.load

Code Example

import joblib
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.impute import SimpleImputer
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

rng = np.random.default_rng(42)
n = 800
df = pd.DataFrame({
    "age": rng.integers(18, 75, n).astype(float),
    "income": rng.normal(60, 20, n),
    "plan": rng.choice(["basic", "plus", "pro"], n),
})
df.loc[rng.choice(n, 40, replace=False), "income"] = np.nan
df["churn"] = ((df["plan"] == "basic") & (df["age"] < 35) | (df["income"] < 35)).astype(int)

X, y = df.drop(columns="churn"), df["churn"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y, random_state=42)

preprocess = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]),
     ["age", "income"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
])
pipe = Pipeline([("prep", preprocess), ("model", HistGradientBoostingClassifier(random_state=42))])

search = GridSearchCV(pipe, {"model__max_depth": [3, None], "model__learning_rate": [0.05, 0.1]},
                      cv=5, scoring="f1")
search.fit(X_train, y_train)                    # preprocessing is re-fit inside each CV fold
print(search.best_params_)
print(classification_report(y_test, search.predict(X_test), digits=3))

joblib.dump(search.best_estimator_, "churn_pipeline.joblib")
loaded = joblib.load("churn_pipeline.joblib")
print(loaded.predict(X_test.head(3)))

Limits

  • Not designed for deep learning or GPU training (use PyTorch/TensorFlow).

  • Datasets must usually fit in memory (use partial_fit models, Dask-ML or Spark for more).

Common Pitfalls

  • Fitting scalers or encoders on the full dataset before splitting (data leakage); use pipelines.

  • Ignoring random_state, so results aren’t reproducible.

  • Evaluating with accuracy on imbalanced data.

Interview Tips

  • Describe the fit/predict/transform API and pipelines.

  • Walk through split → pipeline → CV tuning → test evaluation → save.

15

What are Matplotlib and seaborn used for in data visualization?

Matplotlib is Python’s foundational plotting library: it gives full control over figures, axes, lines, labels and styles and can produce almost any static chart, at the cost of more code; seaborn is built on Matplotlib and adds a high-level, statistics-oriented API that works directly with pandas DataFrames to draw distributions, relationships, categorical comparisons and correlation heatmaps in one call, with good default styling. In ML work you use seaborn for fast exploratory analysis (feature distributions, pair plots, class balance) and Matplotlib for custom result plots (learning curves, ROC curves, publication figures) or fine-tuning seaborn output.

Read full answer

Applies toMatplotlib 3.9+, seaborn 0.13+

Comparison

Aspect Matplotlib seaborn
Level Low-level, flexible High-level, opinionated
Data input Arrays, lists pandas DataFrames with column names
Statistical plots Manual Built in (KDE, regression fits, confidence intervals)
Styling Manual or style sheets Attractive defaults and themes
Typical ML use Learning curves, custom figures EDA: histograms, box plots, pair plots, heatmaps

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

rng = np.random.default_rng(42)
df = pd.DataFrame({
    "feature_a": np.r_[rng.normal(0, 1, 200), rng.normal(2, 1, 200)],
    "feature_b": np.r_[rng.normal(0, 1, 200), rng.normal(1, 0.5, 200)],
    "label": ["neg"] * 200 + ["pos"] * 200,
})

fig, axes = plt.subplots(1, 3, figsize=(15, 4))
sns.histplot(data=df, x="feature_a", hue="label", kde=True, ax=axes[0])          # seaborn: distribution
sns.boxplot(data=df, x="label", y="feature_b", ax=axes[1])                       # seaborn: categorical
sns.heatmap(df[["feature_a", "feature_b"]].corr(), annot=True, vmin=-1, vmax=1, cmap="vlag", ax=axes[2])
axes[2].set_title("Correlation")
fig.tight_layout()
fig.savefig("eda.png", dpi=120)

epochs = np.arange(1, 21)
train_loss = 1 / epochs + 0.05
val_loss = 1 / epochs + 0.1 + 0.004 * epochs                  # starts to rise: overfitting
fig, ax = plt.subplots(figsize=(6, 4))                         # Matplotlib: custom result plot
ax.plot(epochs, train_loss, label="train")
ax.plot(epochs, val_loss, label="validation")
best = int(epochs[np.argmin(val_loss)])
ax.axvline(best, linestyle="--", color="grey", label=f"best epoch {best}")
ax.set(xlabel="epoch", ylabel="loss", title="Learning curve")
ax.legend()
fig.savefig("learning_curve.png", dpi=120)
print("best epoch:", best)

Common Pitfalls

  • Using the implicit plt.* state machine in complex scripts; the fig, ax = plt.subplots() style is clearer.

  • Unlabelled axes or missing units.

  • Overplotting large datasets (use transparency, hexbin or sampling).

  • Rainbow colormaps for ordered data (use perceptually uniform ones such as viridis).

Interview Tips

  • Say seaborn for quick statistical EDA, Matplotlib for control.

  • Name the plots you use for distributions, relationships and results.

16

What are the differences between SciPy and NumPy?

NumPy provides the core data structure, the n-dimensional array, together with basic operations on it: vectorized math, broadcasting, indexing, basic linear algebra, random numbers and simple statistics; SciPy is built on NumPy arrays and adds higher-level scientific algorithms: optimization, statistical distributions and tests, sparse matrices, integration, interpolation, signal and image processing, spatial algorithms and more complete linear algebra. In ML, NumPy is the data container and math layer, while SciPy supplies tools such as scipy.optimize.minimize for fitting custom models, scipy.stats for hypothesis tests, and scipy.sparse for large sparse feature matrices like bag-of-words, which scikit-learn uses directly.

Read full answer

Applies toNumPy 2.x, SciPy 1.13+

Comparison

Area NumPy SciPy
Role Array container + core numerics Scientific algorithms on arrays
Linear algebra np.linalg basics scipy.linalg: more decompositions, BLAS/LAPACK access
Statistics mean, std, percentile scipy.stats: distributions, tests (t-test, chi², KS)
Optimization None scipy.optimize: minimize, curve_fit, root finding
Sparse data None scipy.sparse matrices
Signal / image Basic FFT (np.fft) scipy.signal, scipy.ndimage
Spatial None KD-trees, distance matrices (scipy.spatial)
Integration / interpolation Trapezoid only scipy.integrate, scipy.interpolate

Code Example

import numpy as np
from scipy import optimize, sparse, stats
from scipy.spatial.distance import cdist

rng = np.random.default_rng(42)

a = rng.normal(0.0, 1.0, 200)                       # NumPy: arrays and basic stats
b = rng.normal(0.3, 1.0, 200)
print(round(float(a.mean()), 3), round(float(b.mean()), 3))

t, p = stats.ttest_ind(a, b)                         # SciPy: hypothesis test
print(f"t-test p-value: {p:.4f}")

x = np.linspace(0, 5, 50)                            # SciPy: fit a custom model
y = 3.0 * np.exp(-0.7 * x) + rng.normal(0, 0.05, 50)
params, _ = optimize.curve_fit(lambda x, a, k: a * np.exp(-k * x), x, y, p0=[1, 1])
print("fitted a, k:", params.round(2))               # close to [3.0, 0.7]

dense = np.zeros((1000, 5000))
dense[rng.integers(0, 1000, 3000), rng.integers(0, 5000, 3000)] = 1
sp = sparse.csr_matrix(dense)                        # SciPy: sparse storage
print(f"dense {dense.nbytes / 1e6:.0f} MB vs sparse {(sp.data.nbytes + sp.indices.nbytes + sp.indptr.nbytes) / 1e6:.2f} MB")

points = rng.random((4, 2))
print(cdist(points, points).round(2).shape)          # (4, 4) pairwise distance matrix

Common Pitfalls

  • Converting sparse matrices to dense (.toarray()) on large data and running out of memory.

  • Using NumPy for problems SciPy already solves robustly (optimization, distributions).

Interview Tips

  • Describe NumPy as the foundation and SciPy as the algorithm toolkit.

  • Give ML examples: sparse text features, statistical tests, custom optimization.

17

What are the benefits of Jupyter notebooks for machine learning projects?

Jupyter notebooks combine code, its output (tables, plots), and explanatory Markdown in one document that you run cell by cell, which makes them ideal for exploratory data analysis, quick experiments, visualizing results and sharing reproducible reports or tutorials. Their weaknesses are the other side of the same coin: cells run out of order create hidden state, notebooks are hard to diff and review in git, and they encourage untested copy-pasted code, so mature ML projects move reusable code into Python modules and keep notebooks for exploration and reporting.

Read full answer

Applies toJupyterLab 4.x, Jupyter Notebook 7.x, VS Code notebooks

Benefits

Benefit Why it helps in ML
Interactive execution Load data once, iterate on features and models quickly
Inline visualization See plots and tables next to the code that made them
Narrative Markdown explains decisions, useful for reviews and stakeholders
Rich outputs DataFrames render as tables; widgets for interactive exploration
Easy sharing GitHub renders notebooks; export to HTML/PDF
Kernel flexibility Python, R, Julia; remote GPUs (Colab, Kaggle, JupyterHub, SageMaker)

Drawbacks and Mitigations

Drawback Mitigation
Hidden state from out-of-order runs “Restart kernel and run all” before sharing
Hard to diff nbstripout, Jupytext (pair with .py files), nbdime
Poor code reuse Move functions to a src/ package and import them
No tests Test modules with pytest; keep notebooks thin
Long training jobs Run scripts or jobs; use notebooks to analyze results

Code Example: A Notebook-Friendly Pattern

# Cell 1: configuration in one place
from dataclasses import dataclass

import numpy as np
import pandas as pd

@dataclass
class Config:
    seed: int = 42
    n_samples: int = 1000

cfg = Config()
rng = np.random.default_rng(cfg.seed)

# Cell 2: logic in functions (later moved to src/features.py)
def make_features(df: pd.DataFrame) -> pd.DataFrame:
    return df.assign(ratio=df["a"] / (df["b"] + 1e-9), log_a=np.log1p(df["a"]))

# Cell 3: exploration
df = pd.DataFrame({"a": rng.exponential(2, cfg.n_samples), "b": rng.exponential(1, cfg.n_samples)})
features = make_features(df)
print(features.describe().round(2).loc[["mean", "std"]])

Useful notebook commands: %timeit for micro-benchmarks, %load_ext autoreload with %autoreload 2 to pick up edits in imported modules.

Common Pitfalls

  • Relying on variables defined in deleted or re-ordered cells.

  • Committing notebooks with large outputs or secrets in them.

  • Using a notebook as the production pipeline.

Interview Tips

  • Give the benefits (interactivity, visualization, narrative) and the risks (hidden state, versioning).

  • Explain how you move code from notebooks into tested modules.

19

What is data cleaning and why is it important in machine learning?

Data cleaning is the process of finding and fixing problems in raw data before modelling: missing values, duplicates, inconsistent formats and category spellings, wrong data types, impossible or outlier values, mislabelled targets, and leakage columns that wouldn’t be available at prediction time. It matters because models learn whatever is in the data: errors become learned patterns (“garbage in, garbage out”), many algorithms fail outright on missing or non-numeric values, and leakage produces models that look excellent in testing and fail in production. In practice it often takes more time than modelling and gives bigger accuracy gains.

Read full answer

Applies topandas 2.x, Python 3.12

Typical Cleaning Steps

Problem Detection Fix
Missing values df.isna().mean() Drop, impute (median/mode/model), add missing indicator
Duplicates df.duplicated() drop_duplicates()
Wrong types df.dtypes pd.to_numeric(errors="coerce"), pd.to_datetime
Inconsistent categories value_counts() Strip, lower-case, map synonyms
Impossible values Domain rules (age < 0) Set to NaN, then impute
Outliers IQR, z-scores, plots Cap, transform (log), or investigate
Label noise Disagreement checks, cleanlab Relabel or drop
Leakage Columns created after the outcome Remove

Code Example

import numpy as np
import pandas as pd

raw = pd.DataFrame({
    "id": [1, 2, 2, 3, 4, 5, 6],
    "age": ["34", "29", "29", "-5", "41", "n/a", "230"],
    "city": [" Pune", "delhi", "delhi", "Delhi ", "PUNE", "Chennai", "chennai"],
    "income": [52000, 61000, 61000, None, 75000, 58000, 64000],
    "signup": ["2026-01-03", "2026-02-11", "2026-02-11", "2026-03-15", "bad date", "2026-04-02", "2026-05-20"],
})

df = raw.drop_duplicates().copy()                                   # 1 duplicate row removed
df["age"] = pd.to_numeric(df["age"], errors="coerce")               # "n/a" -> NaN
df.loc[~df["age"].between(0, 110), "age"] = np.nan                  # -5 and 230 are impossible
df["city"] = df["city"].str.strip().str.title()                     # " Pune", "PUNE" -> "Pune"
df["signup"] = pd.to_datetime(df["signup"], errors="coerce")        # "bad date" -> NaT
df["income_missing"] = df["income"].isna().astype(int)              # keep the signal that it was missing
df["income"] = df["income"].fillna(df["income"].median())
df["age"] = df["age"].fillna(df["age"].median())

print(len(raw), "->", len(df), "rows")                              # 7 -> 6 rows
print(df["city"].value_counts().to_dict())                          # {'Pune': 2, 'Delhi': 2, 'Chennai': 2}
print(df.isna().sum().to_dict())                                    # only signup has 1 NaT left

In a real project, compute imputation values on the training split only (for example with scikit-learn’s SimpleImputer inside a pipeline) so that test data doesn’t leak into them.

Common Pitfalls

  • Dropping every row with any missing value, which can discard most of the data and bias it.

  • Cleaning with statistics from the whole dataset before splitting.

  • Removing “outliers” that are real and important (fraud, rare failures).

  • No record of the cleaning steps, so production data gets treated differently.

Interview Tips

  • List the problem types and a fix for each.

  • Stress leakage and fitting cleaning statistics on training data only.

20

What are the common data preprocessing steps for a machine learning model?

Common preprocessing steps are: split the data first (train/validation/test, by time or group when needed), clean it (duplicates, types, impossible values), impute missing values, encode categorical features (one-hot, ordinal or target encoding), scale or normalize numeric features for models that need it, transform skewed features (log, Box-Cox), engineer new features, handle class imbalance if necessary, and optionally select features or reduce dimensions. Every step that learns from data (imputation values, scaler means, encoders) must be fitted on the training set only and applied to the rest, which scikit-learn’s Pipeline and ColumnTransformer enforce.

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x

Steps and Tools

Step When needed scikit-learn tool
Train/test split Always, first train_test_split, TimeSeriesSplit, GroupShuffleSplit
Imputation Missing values SimpleImputer, KNNImputer, IterativeImputer
Categorical encoding Text categories OneHotEncoder, OrdinalEncoder, TargetEncoder
Scaling Distance- or gradient-based models (kNN, SVM, linear, neural nets) StandardScaler, MinMaxScaler, RobustScaler
Skew transforms Heavy-tailed features FunctionTransformer(np.log1p), PowerTransformer
Feature engineering Domain knowledge Custom transformers, PolynomialFeatures
Imbalance Rare classes class_weight, resampling (imbalanced-learn)
Selection / reduction Many features SelectKBest, PCA

Code Example

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer, OneHotEncoder, StandardScaler

rng = np.random.default_rng(42)
n = 1000
df = pd.DataFrame({
    "income": rng.lognormal(10, 0.8, n),                 # skewed
    "age": rng.integers(18, 80, n).astype(float),
    "region": rng.choice(["north", "south", "east", "west"], n),
})
df.loc[rng.choice(n, 50, replace=False), "age"] = np.nan
y = ((np.log(df["income"]) > 10.2) & (df["region"] != "south")).astype(int)

X_train, X_test, y_train, y_test = train_test_split(df, y, test_size=0.2, stratify=y, random_state=42)

preprocess = ColumnTransformer([
    ("income", Pipeline([("log", FunctionTransformer(np.log1p)), ("scale", StandardScaler())]), ["income"]),
    ("age", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), ["age"]),
    ("region", OneHotEncoder(handle_unknown="ignore", drop="if_binary"), ["region"]),
])
model = Pipeline([("prep", preprocess), ("clf", LogisticRegression(max_iter=1000))])
model.fit(X_train, y_train)                              # all statistics learned from training data only

print("test accuracy:", round(model.score(X_test, y_test), 3))
print(model.named_steps["prep"].get_feature_names_out())
# ['income__income' 'age__age' 'region__region_east' 'region__region_north' 'region__region_south' 'region__region_west']

Model-Dependent Choices

  • Tree-based models (random forests, gradient boosting) don’t need scaling and handle skew well.

  • Linear models, SVMs, kNN and neural networks need scaled features.

  • High-cardinality categories: target encoding or embeddings instead of one-hot.

Common Pitfalls

  • Preprocessing before the split (leakage).

  • Different preprocessing code paths in training and production; ship the fitted pipeline.

  • One-hot encoding high-cardinality IDs into thousands of sparse columns without need.

Interview Tips

  • List the steps in order, starting with the split.

  • Show a ColumnTransformer pipeline and explain why it prevents leakage.

21

How do you handle missing or corrupted data in a dataset using Python?

First detect and understand the problem: count missing values per column (df.isna().mean()), find corrupted entries by trying to parse them (pd.to_numeric(..., errors="coerce"), pd.to_datetime(..., errors="coerce")) and by checking domain rules (negative ages, future dates), and ask why values are missing, since data missing for a reason carries information. Then treat it: drop rows or columns only when the loss is small or a column is mostly empty, convert corrupted values to NaN, impute (median or mode, KNN or iterative model-based imputation, forward-fill for time series), add missing-indicator features, or use models that accept missing values natively (HistGradientBoosting, XGBoost, LightGBM), fitting every imputer on the training split only.

Read full answer

Applies topandas 2.x, scikit-learn 1.5+

Strategies

Situation Strategy
Few rows missing at random Drop rows (dropna)
Column mostly empty (> 60–80%) Drop the column or keep only an indicator
Numeric, skewed Median imputation + missing indicator
Categorical Most frequent value or a new “missing” category
Related features available KNNImputer, IterativeImputer
Time series Forward-fill, interpolation
Tree boosting models Leave NaN; models learn a default direction
Corrupted values Coerce to NaN, then treat as missing

Code Example

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.impute import KNNImputer, SimpleImputer
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(42)
n = 500
df = pd.DataFrame({
    "temp": rng.normal(25, 5, n).round(1).astype(object),
    "humidity": rng.uniform(30, 90, n).round(1),
    "sensor": rng.choice(["A", "B", "C"], n).astype(object),
})
df["load"] = 2 * df["temp"].astype(float) + 0.5 * df["humidity"] + rng.normal(0, 2, n)
df.loc[rng.choice(n, 20, replace=False), "temp"] = "ERR"          # corrupted readings
df.loc[rng.choice(n, 40, replace=False), "humidity"] = np.nan
df.loc[rng.choice(n, 15, replace=False), "sensor"] = None

df["temp"] = pd.to_numeric(df["temp"], errors="coerce")            # "ERR" -> NaN
df.loc[~df["humidity"].between(0, 100), "humidity"] = np.nan       # domain rule check
print(df.isna().sum().to_dict())                                   # {'temp': 20, 'humidity': 40, 'sensor': 15, 'load': 0}

X, y = df.drop(columns="load"), df["load"]
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
X_train, X_test = X_train.copy(), X_test.copy()

for frame in (X_train, X_test):
    frame["humidity_missing"] = frame["humidity"].isna().astype(int)   # keep the signal
cat_imp = SimpleImputer(strategy="most_frequent").fit(X_train[["sensor"]])
num_imp = KNNImputer(n_neighbors=5).fit(X_train[["temp", "humidity"]])   # fitted on train only
for frame in (X_train, X_test):
    frame[["sensor"]] = cat_imp.transform(frame[["sensor"]])
    frame[["temp", "humidity"]] = num_imp.transform(frame[["temp", "humidity"]])
print(X_train.isna().sum().sum(), X_test.isna().sum().sum())       # 0 0

# Alternative: a model that handles NaN natively (no imputation step)
native = HistGradientBoostingRegressor(random_state=42)
native.fit(X.loc[y_train.index, ["temp", "humidity"]], y_train)
print("R^2 with native NaN handling:", round(native.score(X.loc[y_test.index, ["temp", "humidity"]], y_test), 3))

Common Pitfalls

  • Imputing with statistics from the full dataset (leakage).

  • Filling with 0 when 0 is a meaningful value.

  • Dropping rows without checking whether missingness relates to the target.

  • Treating sentinel values (−999, “N/A”, “unknown”) as real data; convert them first.

Interview Tips

  • Separate detection, understanding why, and treatment.

  • Mention missing indicators and models with native NaN support.

22

What is feature scaling and why is it necessary?

Feature scaling transforms numeric features onto comparable ranges, most commonly by standardization (subtract the mean and divide by the standard deviation, giving mean 0 and standard deviation 1) or min-max normalization (map to [0, 1]); robust scaling with the median and interquartile range is used when outliers are present. It is necessary for algorithms that depend on distances or gradient magnitudes: without it, a feature measured in thousands (income) dominates one measured in units (number of children) in kNN, k-means, SVMs and PCA, and gradient descent for linear models and neural networks converges slowly; regularization penalties also assume comparable scales. Tree-based models don’t need it, and the scaler must be fitted on training data only.

Read full answer

Applies toscikit-learn 1.5+

Scalers

Scaler Formula Use when
StandardScaler (x − mean) / std Default for linear models, SVMs, neural nets, PCA
MinMaxScaler (x − min) / (max − min) Bounded inputs (images, some neural nets)
RobustScaler (x − median) / IQR Outliers present
MaxAbsScaler x / max(|x|) Sparse data (keeps zeros)
PowerTransformer / log Reduce skew, then standardize Heavily skewed features

Which Models Need It

Needs scaling Doesn’t
kNN, k-means, SVM, PCA Decision trees, random forests
Linear/logistic regression with regularization Gradient boosting (XGBoost, LightGBM)
Neural networks Naive Bayes (mostly)

Code Example

from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=600, n_features=5, n_informative=3, random_state=42)
X[:, 0] *= 1000                    # one informative-or-not feature on a huge scale
X[:, 3] *= 0.01                    # and one on a tiny scale

raw = cross_val_score(KNeighborsClassifier(), X, y, cv=5).mean()
scaled = cross_val_score(make_pipeline(StandardScaler(), KNeighborsClassifier()), X, y, cv=5).mean()
print(f"kNN accuracy without scaling: {raw:.3f}, with scaling: {scaled:.3f}")
# Scaling usually helps a lot here, because unscaled distances are dominated by feature 0

scaler = StandardScaler().fit(X[:480])          # fit on the training part only
X_test_scaled = scaler.transform(X[480:])
print(X_test_scaled.mean(axis=0).round(2))      # near 0, not exactly: test data uses training statistics

Common Pitfalls

  • Fitting the scaler on all data before splitting (leakage).

  • Scaling the target unintentionally, or forgetting to invert target scaling for predictions.

  • Min-max scaling with outliers, which squashes most values into a tiny range.

  • Scaling one-hot or binary columns without reason.

Interview Tips

  • Name standardization vs normalization and which models need them.

  • Mention fitting on training data inside a pipeline.

23

What is the difference between label encoding and one-hot encoding?

Label (ordinal) encoding replaces each category with an integer (red → 0, green → 1, blue → 2) in a single column, while one-hot encoding creates one binary column per category with a 1 marking the row’s category. Label encoding is compact but implies an order and spacing between categories, so it suits truly ordinal features (low < medium < high) and tree-based models, which can split on the integers; one-hot encoding makes no order assumption and is the safe choice for nominal categories in linear models, SVMs and neural networks, at the cost of many columns for high-cardinality features. In scikit-learn, use OrdinalEncoder for features and LabelEncoder only for target labels.

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x

Comparison

Aspect Label / ordinal encoding One-hot encoding
Output One integer column k binary columns (k categories)
Implies order Yes No
Dimensionality Low Grows with categories
Good for Ordinal features; tree models Nominal features; linear models, neural nets
scikit-learn OrdinalEncoder (features), LabelEncoder (target) OneHotEncoder
High cardinality Fine for trees Many sparse columns; consider target encoding or embeddings

Code Example

import pandas as pd
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder

df = pd.DataFrame({"color": ["red", "green", "blue", "green"], "size": ["S", "L", "M", "S"]})

size_enc = OrdinalEncoder(categories=[["S", "M", "L"]])      # explicit order for an ordinal feature
df["size_code"] = size_enc.fit_transform(df[["size"]]).ravel().astype(int)
print(df["size_code"].tolist())                             # [0, 2, 1, 0]

color_enc = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
onehot = color_enc.fit_transform(df[["color"]])
print(color_enc.get_feature_names_out())                    # ['color_blue' 'color_green' 'color_red']
print(onehot.astype(int).tolist())                          # [[0, 0, 1], [0, 1, 0], [1, 0, 0], [0, 1, 0]]

print(color_enc.transform(pd.DataFrame({"color": ["purple"]})).tolist())   # [[0.0, 0.0, 0.0]]: unseen category

naive = {"blue": 0, "green": 1, "red": 2}                   # label encoding a nominal feature
print(abs(naive["red"] - naive["blue"]) > abs(naive["green"] - naive["blue"]))   # True: a fake "distance"

Common Pitfalls

  • Label-encoding nominal features for linear models or kNN, which invents an order.

  • Using pd.get_dummies separately on train and test, giving mismatched columns; use a fitted OneHotEncoder.

  • The dummy-variable trap in unregularized linear regression (use drop="first" when needed).

  • Exploding dimensionality with IDs or zip codes.

Interview Tips

  • Explain the implied-order problem with an example.

  • Say which models tolerate integer codes and which need one-hot.

24

Why do we split data into training, validation and test sets?

Splitting data separates the jobs of learning, choosing and judging: the training set is used to fit model parameters, the validation set to compare models and tune hyperparameters, and the test set, used once at the end, to give an unbiased estimate of performance on new data. If you tuned on the test set, its score would be optimistically biased because you’d have chosen the model that happens to fit it best. Typical splits are 60/20/20 or 70/15/15 (or train/test plus k-fold cross-validation on the training part), and the split must match how the model will be used: stratified for imbalanced classes, chronological for time series, and grouped when several rows belong to the same user or patient.

Read full answer

Applies toscikit-learn 1.5+

Roles

Set Used for Touched
Training Fitting weights / trees Every training run
Validation Hyperparameters, early stopping, feature choices, model selection Many times
Test Final unbiased estimate Once

Choosing the Split Type

Data Split
i.i.d. rows, balanced Random
Imbalanced classes Stratified (stratify=y)
Time series Chronological: train on the past, test on the future
Repeated entities (users, patients) Group split (GroupShuffleSplit, GroupKFold)
Small datasets Train/test + k-fold CV on train

Code Example

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=1000, weights=[0.9, 0.1], random_state=42)

X_temp, X_test, y_temp, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.25, stratify=y_temp, random_state=42)
print(len(X_train), len(X_val), len(X_test))          # 600 200 200
print(round(y_train.mean(), 3), round(y_val.mean(), 3), round(y_test.mean(), 3))   # similar class ratios

best_C, best_score = None, -1.0
for C in [0.01, 0.1, 1, 10]:                           # tune on validation only
    score = LogisticRegression(C=C, max_iter=1000).fit(X_train, y_train).score(X_val, y_val)
    if score > best_score:
        best_C, best_score = C, score

final = LogisticRegression(C=best_C, max_iter=1000).fit(X_temp, y_temp)   # refit on train + validation
print(f"best C = {best_C}, test accuracy = {final.score(X_test, y_test):.3f}")   # reported once

Common Pitfalls

  • Preprocessing (scaling, imputation, feature selection) fitted before splitting.

  • Random splits for time series or grouped data, which leak information.

  • Repeatedly checking the test set and adjusting, which turns it into a validation set.

  • Duplicate or near-duplicate rows across splits.

Interview Tips

  • Explain each set’s role and why the test set must stay untouched.

  • Mention stratified, time-based and group splits.

25

How do scikit-learn pipelines streamline preprocessing?

A scikit-learn Pipeline chains preprocessing steps and a final model into one estimator, so a single fit learns every step in order on the training data and a single predict applies the same fitted transformations to new data; ColumnTransformer routes different columns to different preprocessing (scale numeric columns, one-hot encode categorical ones). Pipelines prevent data leakage in cross-validation (each fold re-fits the preprocessing on its own training part), make hyperparameter search over preprocessing and model settings possible with step__param names, and package everything into one object you can save and deploy.

Read full answer

Applies toscikit-learn 1.5+

Benefits

Benefit How
No leakage Transformers fitted only on training folds
Less code One fit / predict
Joint tuning GridSearchCV(pipe, {"prep__num__scale": [...], "model__C": [...]})
Deployable joblib.dump(pipe, "model.joblib")
Readable Steps named and ordered
Mixed column types ColumnTransformer + make_column_selector

Code Example

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder, StandardScaler

rng = np.random.default_rng(42)
n = 600
df = pd.DataFrame({
    "tenure": rng.integers(0, 72, n).astype(float),
    "monthly": rng.normal(70, 25, n),
    "contract": rng.choice(["monthly", "yearly", "two-year"], n),
    "payment": rng.choice(["card", "bank", "cash"], n),
})
df.loc[rng.choice(n, 30, replace=False), "monthly"] = np.nan
y = ((df["contract"] == "monthly") & (df["tenure"] < 24)).astype(int)

numeric = Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())])
categorical = Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                        ("onehot", OneHotEncoder(handle_unknown="ignore"))])
prep = ColumnTransformer([
    ("num", numeric, make_column_selector(dtype_include="number")),
    ("cat", categorical, make_column_selector(dtype_include=object)),
])
pipe = Pipeline([("prep", prep), ("model", LogisticRegression(max_iter=1000))])

X_train, X_test, y_train, y_test = train_test_split(df, y, stratify=y, random_state=42)
grid = GridSearchCV(pipe, {
    "prep__num__scale": [StandardScaler(), MinMaxScaler()],    # tune preprocessing too
    "model__C": [0.1, 1, 10],
}, cv=5)
grid.fit(X_train, y_train)
print(grid.best_params_)
print("test accuracy:", round(grid.score(X_test, y_test), 3))
print(grid.best_estimator_.predict(X_test.head(3)))          # raw DataFrame in, predictions out

Common Pitfalls

  • Fitting a scaler outside the pipeline and then cross-validating (leaky scores).

  • Wrong parameter names in grids; check pipe.get_params().keys().

  • Custom transformers that don’t implement fit/transform correctly (inherit from BaseEstimator, TransformerMixin).

Interview Tips

  • Lead with leakage prevention in cross-validation.

  • Show ColumnTransformer and tuning with step__param names.

26

How do you handle categorical data in machine learning models?

Most models need numbers, so categorical features are encoded according to their type, cardinality and the model: one-hot encoding for nominal features with few categories (especially for linear models and neural networks), ordinal encoding for ordered categories or for tree models, target (mean) encoding with cross-fitting or frequency encoding for high-cardinality features, and learned embeddings in neural networks. Some libraries handle categories natively: CatBoost, LightGBM, and scikit-learn’s HistGradientBoosting with categorical_features. Also plan for categories that are rare (group them as “other”) or unseen at prediction time (handle_unknown="ignore").

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x

Choosing an Encoding

Feature Encoding
Nominal, few levels (≤ ~15) One-hot
Ordinal (size, rating) Ordinal with explicit order
High cardinality (zip code, product ID) Target encoding (cross-fitted), frequency, hashing, embeddings
Tree boosting models Native categorical support or ordinal
Text-like categories Clean first (case, whitespace, typos)
Rare categories Group into “other” (min_frequency, max_categories)

Code Example

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, TargetEncoder

rng = np.random.default_rng(42)
n = 2000
cities = [f"city_{i}" for i in range(200)]
city_effect = dict(zip(cities, rng.normal(0, 1, len(cities))))
df = pd.DataFrame({
    "city": rng.choice(cities, n),                               # high cardinality
    "plan": rng.choice(["basic", "plus", "pro"], n),             # few nominal levels
    "rating": rng.choice(["low", "medium", "high"], n),          # ordinal
})
logit = df["city"].map(city_effect) + (df["plan"] == "pro") * 1.0 + df["rating"].map({"low": -1, "medium": 0, "high": 1})
y = (logit + rng.normal(0, 0.5, n) > 0).astype(int)

prep = ColumnTransformer([
    ("city", TargetEncoder(random_state=42), ["city"]),          # cross-fitted to avoid leakage
    ("plan", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
    ("rating", OrdinalEncoder(categories=[["low", "medium", "high"]]), ["rating"]),
])
linear = make_pipeline(prep, LogisticRegression(max_iter=1000))
print("logistic + mixed encodings:", round(cross_val_score(linear, df, y, cv=5).mean(), 3))

df_cat = df.astype("category")
native = HistGradientBoostingClassifier(categorical_features="from_dtype", random_state=42)
print("native categorical boosting:", round(cross_val_score(native, df_cat, y, cv=5).mean(), 3))

rare = OneHotEncoder(min_frequency=15, handle_unknown="infrequent_if_exist").fit(df[["city"]])
print("one-hot columns for city with rare grouping:", len(rare.get_feature_names_out()))

Common Pitfalls

  • Target encoding without cross-fitting, which leaks the target and overfits.

  • One-hot encoding thousands of IDs for a linear model without regularization.

  • Unseen categories in production crashing the encoder.

  • Inconsistent spellings creating fake categories.

Interview Tips

  • Choose encodings by cardinality, ordering and model type.

  • Mention target encoding leakage and native categorical support.

27

What is the process of building a machine learning model in Python?

Building a model in Python follows a repeatable workflow: define the problem and success metric; collect and explore the data (pandas, seaborn); split it into train/validation/test sets before anything else; preprocess and engineer features inside a scikit-learn pipeline; train a simple baseline, then stronger models; tune hyperparameters with cross-validation; evaluate once on the test set with the chosen metric and error analysis; then save the pipeline (joblib or the framework’s format) and deploy and monitor it. The code is easiest to maintain when each stage is a function or module with a fixed random seed and configuration.

Read full answer

Applies toPython 3.12, scikit-learn 1.5+

Workflow

Step Python tools
1. Define problem, metric Business goal → e.g. F1 or RMSE
2. Load and explore pandas, seaborn, df.describe()
3. Split train_test_split (stratified/time/group)
4. Preprocess + features Pipeline, ColumnTransformer
5. Baseline DummyClassifier, logistic regression
6. Better models Random forest, gradient boosting
7. Tune GridSearchCV, RandomizedSearchCV, Optuna
8. Evaluate Test metrics, confusion matrix, error analysis
9. Save + deploy joblib, FastAPI, batch jobs
10. Monitor Drift and performance tracking, retraining

Code Example

import joblib
from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, f1_score
from sklearn.model_selection import RandomizedSearchCV, cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

SEED = 42

def load_data():
    data = load_breast_cancer(as_frame=True)
    return data.frame.drop(columns="target"), data.frame["target"]

def compare_models(X_train, y_train):
    models = {
        "baseline": DummyClassifier(strategy="most_frequent"),
        "logistic": make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)),
        "boosting": HistGradientBoostingClassifier(random_state=SEED),
    }
    for name, model in models.items():
        score = cross_val_score(model, X_train, y_train, cv=5, scoring="f1").mean()
        print(f"{name:9s} CV F1 = {score:.3f}")

def tune(X_train, y_train):
    search = RandomizedSearchCV(
        HistGradientBoostingClassifier(random_state=SEED),
        {"learning_rate": [0.03, 0.1, 0.3], "max_depth": [3, 5, None], "l2_regularization": [0.0, 1.0]},
        n_iter=8, cv=5, scoring="f1", random_state=SEED,
    )
    return search.fit(X_train, y_train)

X, y = load_data()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=SEED)
compare_models(X_train, y_train)
search = tune(X_train, y_train)
best = search.best_estimator_
print("best params:", search.best_params_)
print(f"test F1 = {f1_score(y_test, best.predict(X_test)):.3f}")
print(classification_report(y_test, best.predict(X_test), digits=3))
joblib.dump(best, "model.joblib")

Common Pitfalls

  • Skipping the baseline, so you can’t tell whether the model adds value.

  • Leakage from preprocessing before the split.

  • Choosing a metric that doesn’t match the business goal.

  • Notebook-only code that can’t be re-run.

Interview Tips

  • Walk through the steps in order, naming a tool for each.

  • Emphasize baselines, leakage prevention and a single final test evaluation.

28

How do you make sure a model is not overfitting?

You detect overfitting by comparing training performance with performance on data the model hasn’t seen (a validation set, cross-validation folds or a final test set): a large gap, such as 99% training accuracy and 80% validation accuracy, or a validation loss that rises while training loss keeps falling, signals that the model has memorized noise. To prevent it, use more or augmented data, simpler models or fewer features, regularization (L1/L2, dropout, weight decay), tree limits (max depth, min samples per leaf), early stopping, ensembling, and proper validation that respects time order and groups so the estimate is honest.

Read full answer

Applies toscikit-learn 1.5+, Python 3.12

Detection

Check Overfitting sign
Train vs validation score Large gap
Learning curve (score vs training size) Validation score far below training, gap shrinks slowly with data
Validation curve (score vs complexity) Validation score falls as complexity increases
Cross-validation spread High variance across folds
Performance after deployment Worse than offline estimate

Prevention

Technique Example
More data / augmentation Collect, augment images or text
Simpler model Fewer features, smaller trees, lower polynomial degree
Regularization Ridge(alpha), LogisticRegression(C), dropout
Tree constraints max_depth, min_samples_leaf
Early stopping Boosting early_stopping, neural nets
Ensembles Bagging, random forests
Honest validation Group/time splits, no leakage

Code Example

import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_validate, validation_curve
from sklearn.tree import DecisionTreeClassifier

X, y = make_classification(n_samples=600, n_features=20, n_informative=5, flip_y=0.1, random_state=42)

for depth in [None, 4]:
    res = cross_validate(DecisionTreeClassifier(max_depth=depth, random_state=42), X, y,
                         cv=5, return_train_score=True)
    print(f"max_depth={depth}: train {res['train_score'].mean():.3f}, validation {res['test_score'].mean():.3f}")
# The unlimited tree scores 1.000 on training data with a big validation gap; depth 4 narrows the gap

depths = np.arange(1, 16)
train_scores, val_scores = validation_curve(
    DecisionTreeClassifier(random_state=42), X, y, param_name="max_depth", param_range=depths, cv=5)
best = int(depths[val_scores.mean(axis=1).argmax()])
print("depth with the best validation score:", best)

Common Pitfalls

  • Judging only on training accuracy.

  • Tuning against the test set until it overfits too.

  • Leakage that hides overfitting by making validation look good.

Interview Tips

  • Explain the train-validation gap and learning/validation curves.

  • List several remedies and match them to the model type.

29

What is cross-validation and where does it fit in model training?

Cross-validation estimates how well a model generalizes by repeatedly splitting the training data into training and validation parts: in k-fold cross-validation the data is divided into k folds, the model is trained on k − 1 of them and scored on the remaining one, and this repeats k times so every row is used for validation exactly once; the mean and spread of the k scores give a more reliable estimate than a single split. It sits between data splitting and final testing: you hold out a test set first, then use cross-validation on the training data to compare models, select features and tune hyperparameters (as GridSearchCV does), and finally evaluate the chosen model once on the test set.

Read full answer

Applies toscikit-learn 1.5+

Variants

Variant Use when
KFold (k = 5 or 10) General regression / balanced data
StratifiedKFold Classification, especially imbalanced
GroupKFold Several rows per user/patient; keep groups together
TimeSeriesSplit Ordered data: train on the past, validate on the future
RepeatedStratifiedKFold Small datasets; reduce variance of the estimate
Leave-one-out Very small datasets (expensive, high variance)
Nested CV Unbiased estimate when tuning hyperparameters

Code Example

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV, StratifiedKFold, cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000))   # scaler re-fit in every fold
scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc")
print("fold AUCs:", np.round(scores, 3), f"mean {scores.mean():.3f} ± {scores.std():.3f}")

search = GridSearchCV(pipe, {"logisticregression__C": [0.01, 0.1, 1, 10]}, cv=cv, scoring="roc_auc")
search.fit(X_train, y_train)                    # CV used for tuning
print("best C:", search.best_params_["logisticregression__C"])
print(f"test AUC (once): {search.score(X_test, y_test):.3f}")

outer = cross_val_score(search, X_train, y_train, cv=StratifiedKFold(5, shuffle=True, random_state=0),
                        scoring="roc_auc")          # nested CV: tuning inside each outer fold
print(f"nested CV AUC: {outer.mean():.3f}")

Common Pitfalls

  • Preprocessing the whole dataset before cross-validation (leakage); put it in a pipeline.

  • Shuffled k-fold on time series or grouped data.

  • Reporting the best CV score from a large search as the expected performance (optimistic); use nested CV or the test set.

Interview Tips

  • Explain k-fold with a diagram-like description and where it sits in the workflow.

  • Mention stratified, group and time-series variants.

30

What is the bias-variance tradeoff in machine learning?

The bias-variance tradeoff describes two sources of prediction error that move in opposite directions as model complexity changes: bias is error from overly simple assumptions (the model can’t capture the true pattern, so it underfits), and variance is error from sensitivity to the particular training sample (the model fits noise, so it overfits and changes a lot with different data). For squared error, expected error = bias² + variance + irreducible noise; increasing complexity lowers bias but raises variance, so the best model sits at the complexity that minimizes their sum, found in practice with validation data or cross-validation, and regularization, more data or ensembling shift the balance.

Read full answer

Applies togeneral concepts; example with scikit-learn 1.5+

Comparison

Aspect High bias High variance
Symptom Underfitting: poor train and validation scores Overfitting: great train, poor validation
Example Linear model on a curved pattern Deep unpruned tree, high-degree polynomial
Fixes More features, more complex model, less regularization More data, regularization, simpler model, bagging

Code Example: Measuring Both

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures

rng = np.random.default_rng(42)
true_f = np.sin
x_test = np.linspace(0, 3, 50)

for degree in [1, 4, 15]:
    preds = []
    for _ in range(200):                                   # many training sets from the same process
        x = rng.uniform(0, 3, 30)
        y = true_f(x) + rng.normal(0, 0.3, 30)
        model = make_pipeline(PolynomialFeatures(degree), LinearRegression()).fit(x[:, None], y)
        preds.append(model.predict(x_test[:, None]))
    preds = np.array(preds)
    bias_sq = float(((preds.mean(axis=0) - true_f(x_test)) ** 2).mean())
    variance = float(preds.var(axis=0).mean())
    print(f"degree {degree:2d}: bias^2 = {bias_sq:.4f}, variance = {variance:.4f}")
# Degree 1: high bias, low variance. Degree 15: low bias, much higher variance. Degree 4 balances them.

Ways to Manage It

  • Regularization (L1/L2, dropout) adds a little bias to cut variance.

  • Bagging and random forests average many high-variance models to reduce variance.

  • Boosting combines weak, high-bias learners to reduce bias.

  • More data reduces variance without adding bias.

Modern Note

Very large neural networks can show “double descent”: test error falls again past the point where they fit the training data perfectly, so the classic U-shaped curve isn’t the whole story for overparameterized models.

Common Pitfalls

  • Treating bias and variance as properties you can read from one training run (they’re defined over many possible training sets).

  • Assuming more complexity always means overfitting.

Interview Tips

  • Give the decomposition and the underfit/overfit symptoms.

  • Pair each side with remedies.

31

What steps do you take to improve a model's accuracy?

Improve a model systematically, starting from evidence rather than trying random changes: confirm the metric and validation setup are right (no leakage, a split that mirrors production), compare with a baseline, diagnose whether the model is underfitting or overfitting with learning curves, then act on the biggest weakness, usually data first (more or cleaner data, fixed labels, better features), then model choice and hyperparameter tuning, then ensembling and threshold tuning, and finally error analysis on the examples it gets wrong. Each change should be measured with the same cross-validation so improvements are real and not noise.

Read full answer

Applies toPython 3.12, scikit-learn 1.5+

A Step-by-Step Approach

Step Action
1. Check evaluation Right metric, leakage-free validation, enough validation data
2. Baseline Dummy model, simple linear model
3. Diagnose Train vs validation gap; learning curve
4. Data quality Clean errors, fix labels, handle missing values and outliers
5. Features Domain features, interactions, aggregates, encodings, text/date features
6. Model choice Try gradient boosting for tabular, pre-trained networks for images/text
7. Tuning Random/Bayesian search with CV
8. Imbalance / threshold Class weights, tuned decision threshold
9. Ensembles Averaging or stacking diverse models
10. Error analysis Inspect worst errors by segment; feed insights back to steps 4–5

Code Example: Measuring Each Change

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import Ridge
from sklearn.model_selection import KFold, cross_val_score

rng = np.random.default_rng(42)
n = 2000
df = pd.DataFrame({"length": rng.uniform(1, 10, n), "width": rng.uniform(1, 10, n),
                   "material": rng.integers(0, 3, n)})
price = 5 * df["length"] * df["width"] * (1 + 0.5 * df["material"]) + rng.normal(0, 10, n)
cv = KFold(5, shuffle=True, random_state=42)

def score(model, X):
    return -cross_val_score(model, X, price, cv=cv, scoring="neg_root_mean_squared_error").mean()

base = df[["length", "width", "material"]]
print(f"1) ridge, raw features:          RMSE {score(Ridge(), base):6.1f}")
featured = base.assign(area=df["length"] * df["width"])
featured = featured.assign(area_x_material=featured["area"] * featured["material"])
print(f"2) ridge, + domain features:     RMSE {score(Ridge(), featured):6.1f}")
print(f"3) boosting, raw features:       RMSE {score(HistGradientBoostingRegressor(random_state=42), base):6.1f}")
print(f"4) boosting, + domain features:  RMSE {score(HistGradientBoostingRegressor(random_state=42), featured):6.1f}")
# The domain features (area and its interaction with material) give the biggest single improvement

Common Pitfalls

  • Tuning hyperparameters before fixing data and features, where most gains usually are.

  • Accepting improvements smaller than the cross-validation noise.

  • Optimizing accuracy when the business cares about recall, cost or calibration.

  • Overfitting the validation set by trying hundreds of variants.

Interview Tips

  • Present a diagnosis-first process, not a list of tricks.

  • Stress data and feature work, then tuning and ensembles, measured consistently.

32

What are hyperparameters and how do you tune them?

Hyperparameters are settings chosen before training that control how a model learns or how complex it can be, such as the learning rate, the number and depth of trees, the regularization strength, or the number of neighbours in kNN; unlike parameters (weights, split thresholds), they are not learned from the data by the training algorithm itself. You tune them by searching over candidate values and scoring each with cross-validation on the training data: grid search for small spaces, random search for larger ones (it finds good regions with far fewer trials), and Bayesian optimization (Optuna, scikit-optimize) or successive halving to focus the budget on promising settings; the final model is then checked once on the test set.

Read full answer

Applies toscikit-learn 1.5+, Optuna 3.x/4.x

Parameters vs Hyperparameters

Parameters Hyperparameters
Set by Training algorithm You (or a search)
Examples Linear weights, tree splits, neural network weights learning_rate, max_depth, C, n_estimators, alpha
Selected with Training data Validation / cross-validation

Search Methods

Method How Best for
Grid search All combinations Few hyperparameters with few values
Random search Random samples from distributions Larger spaces, limited budget
Successive halving Many configs on little data, keep the best Expensive models
Bayesian (TPE, Gaussian processes) Model the score, sample promising regions Expensive evaluations
Early stopping Stop iterations when validation stops improving Boosting, neural networks

Code Example

import optuna
from scipy.stats import randint
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RandomizedSearchCV, cross_val_score, train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)

random_search = RandomizedSearchCV(
    RandomForestClassifier(random_state=42),
    {"n_estimators": randint(100, 500), "max_depth": randint(2, 20),
     "min_samples_leaf": randint(1, 10), "max_features": ["sqrt", "log2", None]},
    n_iter=20, cv=5, scoring="roc_auc", random_state=42, n_jobs=-1,
)
random_search.fit(X_train, y_train)
print("random search:", random_search.best_params_, round(random_search.best_score_, 4))

def objective(trial):
    model = RandomForestClassifier(
        n_estimators=trial.suggest_int("n_estimators", 100, 500),
        max_depth=trial.suggest_int("max_depth", 2, 20),
        min_samples_leaf=trial.suggest_int("min_samples_leaf", 1, 10),
        max_features=trial.suggest_categorical("max_features", ["sqrt", "log2"]),
        random_state=42, n_jobs=-1,
    )
    return cross_val_score(model, X_train, y_train, cv=5, scoring="roc_auc").mean()

optuna.logging.set_verbosity(optuna.logging.WARNING)
study = optuna.create_study(direction="maximize", sampler=optuna.samplers.TPESampler(seed=42))
study.optimize(objective, n_trials=20)
print("optuna:", study.best_params, round(study.best_value, 4))
print("test AUC of the random-search model:", round(random_search.score(X_test, y_test), 4))
# For learning rates or C, sample on a log scale: scipy.stats.loguniform(1e-4, 1e-1)

Common Pitfalls

  • Tuning on the test set.

  • Linear ranges for scale-like hyperparameters (use log scales for learning rate, C, alpha).

  • Searching too many hyperparameters at once with a tiny budget.

  • Ignoring the variance across CV folds when comparing close scores.

Interview Tips

  • Define the difference from parameters with examples.

  • Explain grid vs random vs Bayesian search and why random beats grid in big spaces.

33

How do ensemble methods work? Give an example where they are useful.

Ensemble methods combine the predictions of several models so their errors partly cancel out, producing a model that is more accurate and more stable than any single member; this works best when the members are individually decent and make different mistakes. The main families are bagging (train many high-variance models such as deep trees on bootstrap samples and average them; random forests add random feature selection), boosting (train models sequentially, each correcting the previous ones’ errors; gradient boosting, XGBoost, LightGBM), and stacking or voting (combine different model types through a vote, an average or a meta-model). A typical use is tabular credit-risk or churn prediction, where gradient-boosted trees, often blended with a linear model, are hard to beat.

Read full answer

Applies toscikit-learn 1.5+

Families

Method How it combines Main effect Examples
Bagging Average models trained on bootstrap samples Reduces variance BaggingClassifier, RandomForestClassifier
Boosting Sequential models fit to previous errors Reduces bias (and variance) HistGradientBoostingClassifier, XGBoost, LightGBM
Voting Majority vote or averaged probabilities Combines diverse strengths VotingClassifier
Stacking Meta-model learns how to weight base models Often best accuracy StackingClassifier

Code Example: Churn-Style Classification

from sklearn.datasets import make_classification
from sklearn.ensemble import (HistGradientBoostingClassifier, RandomForestClassifier, StackingClassifier,
                              VotingClassifier)
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.tree import DecisionTreeClassifier

X, y = make_classification(n_samples=2000, n_features=20, n_informative=8, weights=[0.8, 0.2],
                           flip_y=0.05, random_state=42)
logreg = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
models = {
    "single tree": DecisionTreeClassifier(random_state=42),
    "logistic": logreg,
    "random forest (bagging)": RandomForestClassifier(n_estimators=300, random_state=42, n_jobs=-1),
    "gradient boosting": HistGradientBoostingClassifier(random_state=42),
    "soft voting": VotingClassifier([("lr", logreg), ("rf", RandomForestClassifier(n_estimators=300, random_state=42)),
                                     ("gb", HistGradientBoostingClassifier(random_state=42))], voting="soft"),
    "stacking": StackingClassifier([("rf", RandomForestClassifier(n_estimators=300, random_state=42)),
                                    ("gb", HistGradientBoostingClassifier(random_state=42))],
                                   final_estimator=LogisticRegression(), cv=5),
}
for name, model in models.items():
    auc = cross_val_score(model, X, y, cv=5, scoring="roc_auc").mean()
    print(f"{name:24s} AUC {auc:.3f}")
# The ensembles typically beat the single tree by a wide margin and the logistic model by a smaller one

When Ensembles Are Useful

  • Tabular business problems (churn, fraud, credit risk, pricing).

  • Competitions, where small accuracy gains matter.

  • Noisy data, where averaging reduces variance.

  • Less useful when interpretability, latency or memory limits dominate.

Common Pitfalls

  • Combining near-identical models (no diversity, little gain).

  • Stacking without out-of-fold predictions, which leaks training labels into the meta-model.

  • Ignoring the added complexity and serving cost.

Interview Tips

  • Explain bagging (variance) vs boosting (bias) and stacking.

  • Give a concrete use case and mention diversity as the key ingredient.

34

How do you implement gradient descent in Python?

Gradient descent minimizes a loss by repeatedly moving the parameters a small step in the direction opposite to the loss’s gradient: θ ← θ − η·∇L(θ), where η is the learning rate. For linear regression with mean squared error, the gradient is ∇L = (2/n)·Xᵀ(Xθ − y), which NumPy computes in one vectorized line; the loop runs until the loss stops improving or a step limit is reached, and the result can be checked against the closed-form least-squares solution.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

import numpy as np

rng = np.random.default_rng(42)
n = 500
X = rng.normal(size=(n, 3))
true_theta = np.array([3.0, -2.0, 0.5])
y = X @ true_theta + 1.0 + rng.normal(0, 0.1, n)

def gradient_descent(X, y, lr=0.1, n_iters=1000, tol=1e-10):
    X_b = np.column_stack([np.ones(len(X)), X])        # add the intercept column
    theta = np.zeros(X_b.shape[1])
    history = []
    for i in range(n_iters):
        residual = X_b @ theta - y
        loss = float(np.mean(residual ** 2))
        history.append(loss)
        grad = (2 / len(y)) * X_b.T @ residual         # vectorized gradient
        theta -= lr * grad
        if i > 0 and abs(history[-2] - history[-1]) < tol:
            break
    return theta, history

theta, history = gradient_descent(X, y)
print("gradient descent:", theta.round(3))            # close to [1.0, 3.0, -2.0, 0.5]
print("iterations:", len(history), "final MSE:", round(history[-1], 4))   # MSE near the noise variance 0.01

X_b = np.column_stack([np.ones(n), X])
closed_form, *_ = np.linalg.lstsq(X_b, y, rcond=None)
print("closed form:     ", closed_form.round(3))
print("match:", np.allclose(theta, closed_form, atol=1e-4))              # True

for lr in [0.001, 0.1, 1.1]:                           # learning rate matters
    _, h = gradient_descent(X, y, lr=lr, n_iters=200)
    print(f"lr={lr}: loss after {len(h)} steps = {h[-1]:.3g}")
# 0.001 is still far from converged, 0.1 converges, 1.1 diverges (loss explodes)

Variants

Variant Gradient computed on Trade-off
Batch GD All samples Stable, slow per step on big data
Stochastic GD One sample Noisy, cheap steps
Mini-batch GD 32–512 samples Standard in deep learning
Momentum / Adam Mini-batches + running averages Faster, less tuning

Common Pitfalls

  • Unscaled features, which make the loss surface elongated and convergence slow.

  • A learning rate that is too large (divergence) or too small (no progress).

  • Forgetting the intercept column.

  • Python loops over samples instead of vectorized gradients.

Interview Tips

  • Write the update rule and the MSE gradient.

  • Verify against the closed-form solution and discuss the learning rate.

35

How do you assess a model's performance? Name at least three metrics.

Assess a model on data it hasn’t trained on (cross-validation and a held-out test set) with metrics that match the task and the business goal: for classification, accuracy, precision, recall, F1, ROC-AUC, PR-AUC and log loss; for regression, MAE, RMSE, R² and MAPE; and compare every number with a simple baseline. Beyond single numbers, check the confusion matrix, calibration of probabilities, performance per segment (fairness and robustness), and the cost of errors in the real application, for example missing a fraud case versus flagging a legitimate payment.

Read full answer

Applies toscikit-learn 1.5+

Classification Metrics

Metric Meaning Use when
Accuracy Share of correct predictions Balanced classes, equal error costs
Precision Of predicted positives, share truly positive False positives are costly
Recall Of actual positives, share found Missing positives is costly
F1 Harmonic mean of precision and recall Imbalanced data, one balanced score
ROC-AUC Ranking quality across thresholds Comparing models independent of threshold
PR-AUC Precision-recall trade-off Rare positive class
Log loss / Brier Quality of probabilities Calibrated probabilities needed

Regression Metrics

Metric Meaning
MAE Average absolute error, in target units; robust to outliers
RMSE Penalizes large errors more
R² Share of variance explained (1 = perfect, 0 = predicting the mean)
MAPE Percentage error; unstable near zero targets

Code Example

from sklearn.datasets import make_classification, make_regression
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.metrics import (accuracy_score, average_precision_score, f1_score, mean_absolute_error,
                             precision_score, r2_score, recall_score, roc_auc_score, root_mean_squared_error)
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=2000, weights=[0.9, 0.1], random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
for name, model in [("baseline", DummyClassifier(strategy="most_frequent")),
                    ("forest", RandomForestClassifier(random_state=42))]:
    model.fit(X_tr, y_tr)
    pred, proba = model.predict(X_te), model.predict_proba(X_te)[:, 1]
    print(f"{name:8s} acc {accuracy_score(y_te, pred):.3f}  precision {precision_score(y_te, pred, zero_division=0):.3f}  "
          f"recall {recall_score(y_te, pred):.3f}  F1 {f1_score(y_te, pred):.3f}  "
          f"ROC-AUC {roc_auc_score(y_te, proba):.3f}  PR-AUC {average_precision_score(y_te, proba):.3f}")
# The baseline gets about 0.90 accuracy with zero recall: accuracy alone is misleading here

Xr, yr = make_regression(n_samples=1000, n_features=5, noise=15, random_state=42)
Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, random_state=42)
pr = RandomForestRegressor(random_state=42).fit(Xr_tr, yr_tr).predict(Xr_te)
print(f"MAE {mean_absolute_error(yr_te, pr):.1f}  RMSE {root_mean_squared_error(yr_te, pr):.1f}  R2 {r2_score(yr_te, pr):.3f}")

Common Pitfalls

  • Reporting accuracy on imbalanced data.

  • Evaluating on training data.

  • Choosing the metric after seeing results.

  • Ignoring the decision threshold (precision and recall depend on it).

Interview Tips

  • Name metrics for both classification and regression and when each fits.

  • Always compare against a baseline and connect the metric to error costs.

36

What is a confusion matrix and how do you interpret it?

A confusion matrix is a table that counts a classifier’s predictions against the true labels: for binary classification it has four cells: true positives (TP, positives correctly flagged), false positives (FP, negatives wrongly flagged), false negatives (FN, positives missed) and true negatives (TN). Reading it shows which kinds of mistakes the model makes, not just how many, and every common metric comes from it: accuracy = (TP + TN) / total, precision = TP / (TP + FP), recall = TP / (TP + FN), specificity = TN / (TN + FP). For multi-class problems it is a k × k table whose off-diagonal cells show which classes get confused with each other.

Read full answer

Applies toscikit-learn 1.5+

Layout (scikit-learn convention: rows = actual, columns = predicted)

Predicted negative Predicted positive
Actual negative TN FP (Type I error)
Actual positive FN (Type II error) TP

Worked Example

A fraud model checks 1,000 transactions, 50 of them fraudulent. It flags 60, of which 40 are fraud.

  • TP = 40, FP = 20, FN = 10, TN = 930.

  • Accuracy = (40 + 930) / 1000 = 0.97.

  • Precision = 40 / 60 ≈ 0.667: one in three alerts is a false alarm.

  • Recall = 40 / 50 = 0.80: 20% of fraud is missed.

  • Specificity = 930 / 950 ≈ 0.979.

Code Example

import numpy as np
from sklearn.metrics import ConfusionMatrixDisplay, classification_report, confusion_matrix

y_true = np.array([0] * 950 + [1] * 50)
y_pred = np.array([0] * 930 + [1] * 20 + [0] * 10 + [1] * 40)   # matches the worked example

cm = confusion_matrix(y_true, y_pred)
tn, fp, fn, tp = cm.ravel()
print(cm)                          # [[930  20] [ 10  40]]
print(f"precision {tp / (tp + fp):.3f}, recall {tp / (tp + fn):.3f}, specificity {tn / (tn + fp):.3f}")
# precision 0.667, recall 0.800, specificity 0.979
print(classification_report(y_true, y_pred, digits=3))

print(confusion_matrix(y_true, y_pred, normalize="true").round(3))   # rows sum to 1: per-class recall

y3_true = ["cat", "dog", "bird", "cat", "dog", "bird", "cat"]
y3_pred = ["cat", "cat", "bird", "cat", "dog", "dog", "dog"]
print(confusion_matrix(y3_true, y3_pred, labels=["bird", "cat", "dog"]))
# [[1 0 1]
#  [0 2 1]
#  [0 1 1]]   e.g. one bird was predicted as dog
disp = ConfusionMatrixDisplay.from_predictions(y3_true, y3_pred)      # heatmap plot

Interpreting It

  • Large FN: the model misses positives; lower the threshold or improve recall.

  • Large FP: too many false alarms; raise the threshold or improve precision.

  • Off-diagonal clusters in multi-class: classes the model can’t separate; look for better features or merge labels.

Common Pitfalls

  • Mixing up the axis convention (some textbooks put predictions on rows).

  • Judging an imbalanced problem by accuracy instead of the FN and FP cells.

  • Forgetting that the matrix depends on the decision threshold.

Interview Tips

  • Draw the 2 × 2 table and derive precision and recall from it.

  • Tie FP and FN to real costs in the application.

37

What are precision and recall in classification?

Precision is the share of the model’s positive predictions that are actually positive, TP / (TP + FP), so it answers “when the model says yes, how often is it right?”; recall (also called sensitivity or true positive rate) is the share of actual positives the model finds, TP / (TP + FN), answering “of all the real positives, how many did it catch?” They trade off through the decision threshold: lowering it catches more positives (higher recall) but adds false alarms (lower precision). Which one matters more depends on error costs: spam filters favour precision, while cancer screening and fraud detection favour recall; F1, their harmonic mean, summarizes both.

Read full answer

Applies toscikit-learn 1.5+

Definitions

Metric Formula Question it answers Prioritize when
Precision TP / (TP + FP) How trustworthy are positive predictions? False positives are costly (spam, legal flags)
Recall TP / (TP + FN) How many positives are found? Missing positives is costly (disease, fraud)
F1 2·P·R / (P + R) Balance of both Imbalanced data, single summary
Fβ (1 + β²)·P·R / (β²·P + R) Weighted balance (β > 1 favours recall) Unequal costs

Hand Example

A screening model flags 80 patients; 60 truly have the condition, out of 100 who have it.

  • Precision = 60 / 80 = 0.75.

  • Recall = 60 / 100 = 0.60.

  • F1 = 2 · 0.75 · 0.60 / (0.75 + 0.60) = 0.9 / 1.35 ≈ 0.667.

Code Example: The Threshold Trade-Off

import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, precision_recall_curve, precision_score, recall_score
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=3000, weights=[0.9, 0.1], class_sep=0.8, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
proba = LogisticRegression(max_iter=1000).fit(X_tr, y_tr).predict_proba(X_te)[:, 1]

for threshold in [0.2, 0.5, 0.8]:
    pred = (proba >= threshold).astype(int)
    print(f"threshold {threshold}: precision {precision_score(y_te, pred, zero_division=0):.3f}, "
          f"recall {recall_score(y_te, pred):.3f}, F1 {f1_score(y_te, pred):.3f}")
# Raising the threshold increases precision and lowers recall

precision, recall, thresholds = precision_recall_curve(y_te, proba)
ok = recall[:-1] >= 0.8                                   # pick the most precise threshold with recall >= 0.8
best = int(np.argmax(np.where(ok, precision[:-1], -1)))
print(f"threshold for recall >= 0.8: {thresholds[best]:.3f} (precision {precision[best]:.3f})")

Common Pitfalls

  • Reporting precision without recall (or the reverse); either can be made perfect alone.

  • Using the default 0.5 threshold without checking the trade-off.

  • Averaging per-class scores without saying macro, micro or weighted.

Interview Tips

  • Give formulas, plain-language meanings and an application for each.

  • Explain the threshold trade-off and F1.

38

What is the ROC curve and what does AUC measure?

The ROC (receiver operating characteristic) curve plots a classifier’s true positive rate (recall) against its false positive rate (FP / (FP + TN)) as the decision threshold sweeps from high to low, showing every trade-off between catching positives and raising false alarms; the area under it (ROC-AUC) summarizes the curve in one number between 0 and 1. AUC equals the probability that a randomly chosen positive example receives a higher score than a randomly chosen negative one: 0.5 is random guessing, 1.0 is perfect ranking. It is threshold-independent and insensitive to class balance, which makes it good for comparing models, but on rare-positive problems the precision-recall curve and PR-AUC are often more informative.

Read full answer

Applies toscikit-learn 1.5+

Reading the Curve

Point / shape Meaning
(0, 0) Threshold so high nothing is flagged
(1, 1) Threshold so low everything is flagged
Diagonal Random classifier (AUC 0.5)
Curve hugging the top-left Strong separation
Below the diagonal Worse than random (labels or scores inverted)

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import RocCurveDisplay, roc_auc_score, roc_curve
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=2000, n_informative=5, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)

fig, ax = plt.subplots(figsize=(5, 5))
for model in [LogisticRegression(max_iter=1000), RandomForestClassifier(random_state=42)]:
    model.fit(X_tr, y_tr)
    RocCurveDisplay.from_estimator(model, X_te, y_te, ax=ax)
ax.plot([0, 1], [0, 1], "k--", label="random")
ax.legend()
fig.savefig("roc.png", dpi=120)

# AUC as a ranking probability, checked by hand on a tiny example
scores = np.array([0.9, 0.8, 0.6, 0.55, 0.4, 0.2])
labels = np.array([1, 1, 0, 1, 0, 0])
pairs = [(p, n) for p in scores[labels == 1] for n in scores[labels == 0]]
manual_auc = np.mean([p > n for p, n in pairs])
print(round(float(manual_auc), 4), round(roc_auc_score(labels, scores), 4))   # 0.8889 0.8889 (8 of 9 pairs)

fpr, tpr, thr = roc_curve(labels, scores)
print(np.c_[fpr, tpr].round(3).tolist())

The tiny example has 3 positives and 3 negatives, so 9 positive-negative pairs; only the pair (0.55 positive, 0.6 negative) is ranked wrongly, giving AUC = 8/9 ≈ 0.889.

ROC-AUC vs PR-AUC

ROC-AUC PR-AUC
Axes TPR vs FPR Precision vs recall
Random baseline 0.5 Positive class rate
Rare positives Can look optimistic More sensitive to false positives

Common Pitfalls

  • Computing AUC from hard 0/1 predictions instead of scores.

  • Treating a high AUC as proof of good calibration (it measures ranking only).

  • Relying on ROC-AUC alone for 1%-positive problems.

Interview Tips

  • Define the axes, the threshold sweep and the ranking interpretation of AUC.

  • Mention PR-AUC for imbalanced data.

39

How does evaluation differ between supervised and unsupervised learning?

In supervised learning, evaluation is direct: you compare predictions against known labels on held-out data with metrics such as accuracy, F1, ROC-AUC, RMSE or R². In unsupervised learning there are no true labels, so evaluation relies on internal measures of structure (silhouette score, Davies-Bouldin index, inertia for clustering; reconstruction error or explained variance for dimensionality reduction and autoencoders), external labels when some exist (adjusted Rand index, normalized mutual information), stability across samples or seeds, and above all whether the result is useful for the downstream task or makes sense to domain experts.

Read full answer

Applies toscikit-learn 1.5+

Comparison

Aspect Supervised Unsupervised
Ground truth Labels available Usually none
Typical metrics Accuracy, F1, AUC, RMSE, R² Silhouette, Davies-Bouldin, Calinski-Harabasz, inertia, reconstruction error
With partial labels Same ARI, NMI, homogeneity
Validation Train/validation/test, CV Stability (bootstrap, different seeds), downstream performance
Judged by Prediction error on new data Structure quality, usefulness, interpretability

Code Example

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, adjusted_rand_score, davies_bouldin_score, silhouette_score)
from sklearn.model_selection import train_test_split

X, y = make_blobs(n_samples=600, centers=4, n_features=6, cluster_std=1.5, random_state=42)

X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)            # supervised
clf = LogisticRegression(max_iter=1000).fit(X_tr, y_tr)
print("supervised accuracy:", round(accuracy_score(y_te, clf.predict(X_te)), 3))

for k in [2, 3, 4, 5, 6]:                                                    # unsupervised
    labels = KMeans(n_clusters=k, n_init=10, random_state=42).fit_predict(X)
    print(f"k={k}: silhouette {silhouette_score(X, labels):.3f}, Davies-Bouldin {davies_bouldin_score(X, labels):.3f}")
# The best silhouette (highest) and Davies-Bouldin (lowest) usually point to k = 4 here

labels4 = KMeans(n_clusters=4, n_init=10, random_state=42).fit_predict(X)
print("ARI against hidden labels:", round(adjusted_rand_score(y, labels4), 3))   # only possible when labels exist
pca = PCA().fit(X)
print("variance explained by 2 components:", round(float(pca.explained_variance_ratio_[:2].sum()), 3))

Common Pitfalls

  • Treating a good silhouette score as proof the clusters are meaningful for the business.

  • Comparing inertia across different numbers of clusters (it always decreases with k).

  • Evaluating clustering with accuracy (cluster IDs are arbitrary; use ARI or NMI).

Interview Tips

  • Contrast label-based metrics with internal and external clustering measures.

  • Stress downstream usefulness and stability for unsupervised results.

40

How do you use a learning curve to diagnose model performance?

A learning curve plots training and validation scores against the number of training examples; its shape shows whether more data would help and whether the model underfits or overfits: if both curves converge at a poor score, the model has high bias (underfitting) and needs more capacity or better features, not more data; if the training score is high and the validation score is much lower but still rising, the model has high variance (overfitting) and more data or regularization should help. In scikit-learn, learning_curve (or LearningCurveDisplay) computes these scores with cross-validation.

Read full answer

Applies toscikit-learn 1.5+

Reading the Curves

Pattern Diagnosis Action
Both low, close together, flat High bias (underfit) More complex model, better features, less regularization
Train high, validation much lower, gap shrinking High variance (overfit) More data, regularization, simpler model
Both high, close together Good fit Ship or fine-tune
Validation still rising at the largest size Data-limited Collecting more data is worthwhile
Validation flat for the last sizes Data-saturated More data won’t help much

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import learning_curve
from sklearn.tree import DecisionTreeClassifier

X, y = make_classification(n_samples=3000, n_features=20, n_informative=10, flip_y=0.05, random_state=42)
sizes = np.linspace(0.1, 1.0, 8)

fig, axes = plt.subplots(1, 2, figsize=(11, 4), sharey=True)
for ax, (name, model) in zip(axes, [("deep tree (high variance)", DecisionTreeClassifier(random_state=42)),
                                    ("logistic regression", LogisticRegression(max_iter=1000))]):
    n, train_scores, val_scores = learning_curve(model, X, y, train_sizes=sizes, cv=5, n_jobs=-1)
    ax.plot(n, train_scores.mean(axis=1), "o-", label="train")
    ax.plot(n, val_scores.mean(axis=1), "o-", label="validation")
    ax.fill_between(n, val_scores.mean(1) - val_scores.std(1), val_scores.mean(1) + val_scores.std(1), alpha=0.2)
    ax.set(title=name, xlabel="training examples", ylabel="accuracy")
    ax.legend()
    gap = train_scores.mean(axis=1)[-1] - val_scores.mean(axis=1)[-1]
    print(f"{name}: final train-validation gap {gap:.3f}")
fig.tight_layout()
fig.savefig("learning_curves.png", dpi=120)
# The deep tree shows a large gap (train accuracy 1.0); logistic regression shows two close curves

Learning Curve vs Validation Curve

  • Learning curve: score vs training set size; answers “will more data help?”

  • Validation curve: score vs a hyperparameter (e.g. tree depth); answers “what complexity is right?”

Common Pitfalls

  • Using a single split instead of cross-validation (noisy curves).

  • Reading curves on accuracy when the real metric is F1 or AUC.

  • Forgetting that a time-ordered dataset needs time-aware splits here too.

Interview Tips

  • Describe the two classic shapes and the action for each.

  • Mention that a still-rising validation curve justifies collecting more data.

41

What validation strategies exist, such as k-fold cross-validation?

Validation strategies differ in how they carve training data into fit and evaluation parts: a simple hold-out split (fast, but noisy on small data), k-fold cross-validation (k rotations so every row is validated once), stratified k-fold (keeps class proportions in each fold), group k-fold (keeps all rows of a user, patient or session in the same fold), time-series splits (always validate on data after the training window), repeated k-fold (averages several shuffles), leave-one-out (k = n, for tiny datasets), and nested cross-validation (an outer loop for evaluation around an inner loop for tuning). The right choice mirrors how the model will meet new data in production; a mismatch, such as random folds on time-ordered data, leaks information and inflates scores.

Read full answer

Applies toscikit-learn 1.5+

Strategies

Strategy scikit-learn Use when
Hold-out train_test_split Large datasets, quick checks
k-fold KFold(n_splits=5, shuffle=True) General regression
Stratified k-fold StratifiedKFold Classification, imbalanced classes
Group k-fold GroupKFold, StratifiedGroupKFold Repeated entities
Time series TimeSeriesSplit(gap=...) Forecasting, ordered events
Repeated RepeatedStratifiedKFold Small data, stable estimates
Leave-one-out LeaveOneOut Very small data
Nested CV cross_val_score(GridSearchCV(...), ...) Unbiased estimate with tuning

Code Example

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GroupKFold, KFold, StratifiedKFold, TimeSeriesSplit, cross_val_score

rng = np.random.default_rng(42)
n_users, rows_per_user = 60, 10
groups = np.repeat(np.arange(n_users), rows_per_user)
user_trait = rng.normal(size=n_users)[groups]                       # each user has a stable trait
X = np.column_stack([user_trait + rng.normal(0, 0.1, len(groups)), rng.normal(size=(len(groups), 3))])
y = (rng.random(n_users) > 0.5).astype(int)[groups]                 # label depends only on the user

model = RandomForestClassifier(n_estimators=200, random_state=42)
random_cv = cross_val_score(model, X, y, cv=KFold(5, shuffle=True, random_state=42)).mean()
group_cv = cross_val_score(model, X, y, cv=GroupKFold(5), groups=groups).mean()
print(f"random k-fold: {random_cv:.3f}  group k-fold: {group_cv:.3f}")
# Random folds put the same user in train and validation, so the model memorizes users and scores near 1.0;
# group folds show the honest result for new users, close to chance here

for fold, (tr, va) in enumerate(StratifiedKFold(3, shuffle=True, random_state=42).split(X, y)):
    print(f"stratified fold {fold}: positive rate train {y[tr].mean():.2f}, val {y[va].mean():.2f}")

for tr, va in TimeSeriesSplit(n_splits=3, gap=5).split(np.arange(40)):
    print(f"time split: train 0..{tr[-1]}, validate {va[0]}..{va[-1]}")   # validation always later

Common Pitfalls

  • Random folds on grouped or temporal data.

  • Preprocessing outside the cross-validation loop.

  • Too few folds on small data (noisy) or too many on large data (slow, little gain).

  • Reporting the tuned CV score as the final estimate without nested CV or a test set.

Interview Tips

  • Match the split to the deployment scenario.

  • Give the group-leakage example and mention nested CV.

42

How do you normalize an array to the range [0, 1] in Python?

Min-max normalization maps each value x to (x − min) / (max − min), so the smallest value becomes 0 and the largest 1; a robust function vectorizes this with NumPy, works per column for 2-D feature matrices, handles constant columns (where max − min = 0) without dividing by zero, ignores NaN values when finding the range, and can reuse the training minimum and maximum to transform new data. In real pipelines scikit-learn’s MinMaxScaler does the same, fitted on training data only.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

import numpy as np
from sklearn.preprocessing import MinMaxScaler

def min_max_normalize(x, axis=0, data_min=None, data_max=None):
    """Scale values to [0, 1] along `axis`.

    Pass data_min/data_max from training data to transform new data consistently.
    Constant columns become 0. NaN values are ignored when computing the range and stay NaN.
    """
    x = np.asarray(x, dtype=float)
    data_min = np.nanmin(x, axis=axis, keepdims=True) if data_min is None else data_min
    data_max = np.nanmax(x, axis=axis, keepdims=True) if data_max is None else data_max
    span = data_max - data_min
    safe_span = np.where(span == 0, 1.0, span)            # avoid division by zero
    return (x - data_min) / safe_span, data_min, data_max

print(min_max_normalize([2, 4, 6, 10])[0])             # [0.   0.25 0.5  1.  ]

X_train = np.array([[1.0, 200.0, 5.0],
                    [2.0, 400.0, 5.0],
                    [3.0, 300.0, 5.0]])
X_scaled, mins, maxs = min_max_normalize(X_train)
print(X_scaled)
# [[0.  0.  0. ]
#  [0.5 1.  0. ]
#  [1.  0.5 0. ]]   the constant third column becomes 0

X_new = np.array([[4.0, 250.0, 5.0]])
print(min_max_normalize(X_new, data_min=mins, data_max=maxs)[0])   # [[1.5  0.25 0.  ]]: outside [0, 1] is possible

print(min_max_normalize([1.0, np.nan, 3.0])[0])        # [0.  nan 1. ]

sk = MinMaxScaler().fit(X_train)
print(np.allclose(sk.transform(X_train), X_scaled))    # True

Design Notes

Choice Why
Vectorized NumPy Fast, no Python loops
keepdims=True Broadcasting works per column
Return min and max Reuse on validation/test data
Safe span Constant features don’t produce NaN or inf

Common Pitfalls

  • Computing min and max on the full dataset, including test data.

  • Expecting new data to stay in [0, 1]; clip if the model requires it.

  • Outliers squashing all other values toward 0; consider robust scaling.

Interview Tips

  • Write the formula, handle the zero-range case, and show reuse on new data.

  • Mention MinMaxScaler and fitting on training data only.

43

How do you build a Python class for a simple perceptron?

A perceptron is the simplest linear classifier: it computes a weighted sum of inputs plus a bias, predicts class 1 if the sum is at least 0 and class 0 otherwise, and learns by the perceptron rule, adjusting weights only when it misclassifies an example: w ← w + η·(y − ŷ)·x and b ← b + η·(y − ŷ). A clean class structure stores the hyperparameters in __init__, learns in fit, exposes net_input and predict, and records the number of mistakes per epoch; it converges when the classes are linearly separable and never fully converges (it keeps oscillating) otherwise.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

import numpy as np

class Perceptron:
    """Binary perceptron classifier with labels 0 and 1."""

    def __init__(self, learning_rate=0.1, n_epochs=50, random_state=42):
        self.learning_rate = learning_rate
        self.n_epochs = n_epochs
        self.random_state = random_state

    def fit(self, X, y):
        X, y = np.asarray(X, dtype=float), np.asarray(y)
        rng = np.random.default_rng(self.random_state)
        self.weights_ = rng.normal(0, 0.01, X.shape[1])
        self.bias_ = 0.0
        self.errors_ = []
        for _ in range(self.n_epochs):
            mistakes = 0
            for i in rng.permutation(len(X)):              # shuffle each epoch
                update = self.learning_rate * (y[i] - self.predict(X[i]))
                if update != 0:
                    self.weights_ += update * X[i]
                    self.bias_ += update
                    mistakes += 1
            self.errors_.append(mistakes)
            if mistakes == 0:                              # converged on separable data
                break
        return self

    def net_input(self, X):
        return np.asarray(X, dtype=float) @ self.weights_ + self.bias_

    def predict(self, X):
        return np.where(self.net_input(X) >= 0.0, 1, 0)

    def score(self, X, y):
        return float(np.mean(self.predict(X) == np.asarray(y)))

X_and = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
model = Perceptron().fit(X_and, [0, 0, 0, 1])               # AND is linearly separable
print(model.predict(X_and).tolist(), model.errors_)         # [0, 0, 0, 1], mistakes fall to 0

xor = Perceptron(n_epochs=20).fit(X_and, [0, 1, 1, 0])      # XOR is not
print("XOR accuracy:", xor.score(X_and, [0, 1, 1, 0]), "| mistakes in last epoch:", xor.errors_[-1])
# XOR never reaches 0 mistakes: a single perceptron can't separate it

rng = np.random.default_rng(42)
X = np.vstack([rng.normal(-2, 1, (50, 2)), rng.normal(2, 1, (50, 2))])
y = np.r_[np.zeros(50, int), np.ones(50, int)]
print("blobs accuracy:", Perceptron().fit(X, y).score(X, y))   # 1.0 or very close

Design Notes

Element Purpose
Trailing underscore attributes (weights_) scikit-learn convention for learned values
fit returns self Allows Perceptron().fit(X, y).predict(...)
errors_ Monitors convergence
Early stop when mistakes = 0 Saves time on separable data

Common Pitfalls

  • Unscaled features with very different ranges, which slow convergence.

  • Expecting probabilities (a perceptron gives hard labels; use logistic regression for probabilities).

  • Using it on non-separable data without a fixed epoch limit.

Interview Tips

  • Write the update rule and explain why updates happen only on mistakes.

  • Mention the XOR limitation that motivated multi-layer networks.

44

How do you implement k-means clustering using only the Python standard library?

K-means groups points into k clusters by alternating two steps until assignments stop changing: assign each point to its nearest centroid (by squared Euclidean distance), then move each centroid to the mean of the points assigned to it. With only the standard library you represent points as tuples, use random.Random(seed) for reproducible initialization (k-means++ seeding, which picks spread-out starting centroids, gives better results than purely random picks), compute distances with math.dist or a sum of squares, handle empty clusters by re-seeding them, and stop when centroids move less than a tolerance or a maximum number of iterations is reached.

Read full answer

Applies toPython 3.12 (standard library only)

Code Example

import math
import random

def squared_distance(a, b):
    return sum((x - y) ** 2 for x, y in zip(a, b))

def mean_point(points):
    n = len(points)
    return tuple(sum(coords) / n for coords in zip(*points))

def kmeans_plus_plus(points, k, rng):
    centroids = [rng.choice(points)]
    while len(centroids) < k:
        d2 = [min(squared_distance(p, c) for c in centroids) for p in points]
        centroids.append(rng.choices(points, weights=d2, k=1)[0])   # far points are more likely
    return centroids

def kmeans(points, k, max_iter=100, tol=1e-6, seed=42):
    rng = random.Random(seed)
    centroids = kmeans_plus_plus(points, k, rng)
    for iteration in range(1, max_iter + 1):
        clusters = [[] for _ in range(k)]
        for p in points:
            nearest = min(range(k), key=lambda j: squared_distance(p, centroids[j]))
            clusters[nearest].append(p)
        new_centroids = [mean_point(c) if c else rng.choice(points) for c in clusters]   # re-seed empty clusters
        shift = max(math.dist(a, b) for a, b in zip(centroids, new_centroids))
        centroids = new_centroids
        if shift < tol:
            break
    labels = [min(range(k), key=lambda j: squared_distance(p, centroids[j])) for p in points]
    inertia = sum(squared_distance(p, centroids[label]) for p, label in zip(points, labels))
    return centroids, labels, inertia, iteration

rng = random.Random(42)
true_centers = [(0, 0), (6, 6), (0, 8)]
data = [(rng.gauss(cx, 1), rng.gauss(cy, 1)) for cx, cy in true_centers for _ in range(100)]

centroids, labels, inertia, n_iter = kmeans(data, k=3)
print("centroids:", sorted((round(x, 1), round(y, 1)) for x, y in centroids))   # near the true centers
print("cluster sizes:", sorted(labels.count(j) for j in range(3)))              # about [100, 100, 100]
print("inertia:", round(inertia, 1), "| iterations:", n_iter)

for k in range(1, 6):                                   # elbow method: inertia drops sharply until k = 3
    print(k, round(kmeans(data, k)[2], 1))

Complexity and Notes

  • Each iteration costs O(n · k · d); NumPy or scikit-learn’s KMeans are far faster for real data.

  • The result depends on initialization; run several seeds and keep the lowest inertia.

  • K-means assumes roughly spherical, similar-size clusters; scale features first.

Common Pitfalls

  • Empty clusters crashing the mean computation.

  • Unscaled features dominating the distance.

  • Comparing inertia across k as if lower is always better (it always decreases with k).

Interview Tips

  • Describe the assign-update loop and the stopping rule.

  • Mention k-means++ initialization and the elbow or silhouette method for choosing k.

45

How do you perform linear regression on a dataset using NumPy?

With NumPy you fit linear regression by adding a column of ones for the intercept to the feature matrix X and solving the least-squares problem min ‖Xβ − y‖², either with np.linalg.lstsq (numerically stable, the recommended way) or with the normal equation β = (XᵀX)⁻¹Xᵀy (solve it with np.linalg.solve rather than computing an inverse). Then predict with Xβ and evaluate with R² and RMSE on held-out data; the script below also computes coefficient standard errors to show how uncertain each estimate is.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

import numpy as np

rng = np.random.default_rng(42)
n = 400
size = rng.uniform(50, 200, n)                  # m^2
rooms = rng.integers(1, 6, n)
age = rng.uniform(0, 40, n)
price = 20 + 1.5 * size + 8 * rooms - 0.6 * age + rng.normal(0, 10, n)   # in thousands

X = np.column_stack([size, rooms, age])
idx = rng.permutation(n)
train, test = idx[:320], idx[320:]

def add_intercept(X):
    return np.column_stack([np.ones(len(X)), X])

X_tr, X_te = add_intercept(X[train]), add_intercept(X[test])
y_tr, y_te = price[train], price[test]

beta, *_ = np.linalg.lstsq(X_tr, y_tr, rcond=None)              # least squares
beta_normal = np.linalg.solve(X_tr.T @ X_tr, X_tr.T @ y_tr)      # normal equation (same answer)
print("coefficients:", beta.round(2))                           # about [20, 1.5, 8, -0.6]
print("same as normal equation:", np.allclose(beta, beta_normal))

def r2(y, y_hat):
    return 1 - np.sum((y - y_hat) ** 2) / np.sum((y - y.mean()) ** 2)

pred = X_te @ beta
print(f"test R^2 = {r2(y_te, pred):.3f}, RMSE = {np.sqrt(np.mean((y_te - pred) ** 2)):.2f}")   # RMSE near 10

residuals = y_tr - X_tr @ beta
sigma2 = residuals @ residuals / (len(y_tr) - X_tr.shape[1])     # unbiased noise variance
std_err = np.sqrt(np.diag(sigma2 * np.linalg.inv(X_tr.T @ X_tr)))
for name, b, se in zip(["intercept", "size", "rooms", "age"], beta, std_err):
    print(f"{name:9s} {b:7.3f} ± {1.96 * se:.3f} (95% CI half-width)")

Why lstsq Over the Inverse

Method Notes
np.linalg.lstsq Uses SVD; handles nearly collinear features
np.linalg.solve(XᵀX, Xᵀy) Fine for well-conditioned problems
np.linalg.inv(XᵀX) @ Xᵀy Least stable; avoid for fitting

Common Pitfalls

  • Forgetting the intercept column.

  • Evaluating on training data only.

  • Collinear features making coefficients unstable (check the condition number, consider ridge).

  • Interpreting coefficients causally.

Interview Tips

  • Write the normal equation and say why lstsq is preferred.

  • Show prediction, R² and a held-out evaluation.

46

How do you write a function that minimizes a cost function with gradient descent?

A general gradient descent optimizer takes the cost function, its gradient (or a numerical approximation), a starting point and a learning rate, then repeats x ← x − η·∇f(x) until the gradient norm or the change in cost falls below a tolerance or an iteration limit is reached, recording the history so you can check convergence. Making it generic (functions passed in as arguments), adding optional momentum and a backtracking line search, and testing it on a function with a known minimum shows both the algorithm and good engineering.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

import numpy as np

def numerical_gradient(f, x, h=1e-6):
    """Central differences, useful when no analytic gradient is available."""
    grad = np.zeros_like(x)
    for i in range(len(x)):
        step = np.zeros_like(x)
        step[i] = h
        grad[i] = (f(x + step) - f(x - step)) / (2 * h)
    return grad

def gradient_descent(f, x0, grad=None, lr=0.1, momentum=0.0, tol=1e-8, max_iter=10_000):
    """Minimize f starting at x0. Returns (x, history of f values)."""
    x = np.asarray(x0, dtype=float).copy()
    velocity = np.zeros_like(x)
    grad = grad or (lambda z: numerical_gradient(f, z))
    history = [f(x)]
    for _ in range(max_iter):
        g = grad(x)
        if np.linalg.norm(g) < tol:
            break
        velocity = momentum * velocity - lr * g
        x = x + velocity
        history.append(f(x))
        if not np.isfinite(history[-1]):
            raise FloatingPointError("diverged: lower the learning rate")
    return x, history

# 1) A quadratic bowl with minimum at (3, -1)
def bowl(x):
    return (x[0] - 3) ** 2 + 10 * (x[1] + 1) ** 2

def bowl_grad(x):
    return np.array([2 * (x[0] - 3), 20 * (x[1] + 1)])

x_min, hist = gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.05)
print(x_min.round(4), len(hist))                # [ 3. -1.], converged

x_mom, hist_mom = gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.05, momentum=0.8)
print(x_mom.round(4), len(hist_mom))            # same minimum

# 2) Rosenbrock function (minimum at (1, 1)) with a numerical gradient
def rosenbrock(x):
    return (1 - x[0]) ** 2 + 100 * (x[1] - x[0] ** 2) ** 2

x_r, hist_r = gradient_descent(rosenbrock, [-1.0, 1.0], lr=1e-3, momentum=0.9, max_iter=50_000)
print(x_r.round(3), f"final cost {hist_r[-1]:.2e}")   # close to [1, 1]

# 3) Divergence is detected
try:
    gradient_descent(bowl, [0.0, 0.0], grad=bowl_grad, lr=0.2)   # 0.2 * 20 > 2: unstable along x[1]
except FloatingPointError as err:
    print(err)

Why the Last Call Diverges

Along x[1] the gradient is 20·(x[1] + 1), so each step multiplies the error by (1 − 20·lr). With lr = 0.2 that factor is −3, so the error triples in size every step; stability needs |1 − 20·lr| < 1, that is lr < 0.1.

Common Pitfalls

  • A single learning rate for badly scaled problems (use feature scaling, momentum or adaptive methods).

  • Stopping on a fixed number of steps without checking convergence.

  • Numerical gradients with too small or too large h (1e-6 to 1e-5 works for doubles).

Interview Tips

  • Write the generic loop with a stopping criterion.

  • Explain the learning-rate stability condition on a quadratic.

47

How do you read and clean a CSV file with pandas?

A clean pandas workflow reads the CSV with explicit options (dtype, parse_dates, na_values for codes like “N/A” or “-”), inspects it (info, describe, isna().sum(), value_counts), standardizes column names, removes duplicates, fixes types with pd.to_numeric(..., errors="coerce") and pd.to_datetime, cleans text categories, handles missing values and impossible values, derives useful columns, and finally saves the result in a typed format such as Parquet. Writing these steps as a function (or a chain of .pipe calls) makes the cleaning repeatable on new files.

Read full answer

Applies topandas 2.x, Python 3.12

Code Example

import io

import numpy as np
import pandas as pd

raw_csv = io.StringIO("""Order ID,Order Date,Customer Name,Region,Units,Unit Price,Status
1001,2026-01-05, alice ,North,3,19.99,shipped
1002,2026-01-06,BOB,south,N/A,5.50,Shipped
1002,2026-01-06,BOB,south,N/A,5.50,Shipped
1003,2026-01-07,Chen,East,2,-,pending
1004,not a date,dana,WEST,1,120.00,cancelled
1005,2026-01-09,Eve,North,-4,9.99,shipped
""")

def clean_orders(source):
    df = pd.read_csv(source, na_values=["N/A", "-", ""], skipinitialspace=True)
    df.columns = df.columns.str.strip().str.lower().str.replace(" ", "_")        # order_id, order_date, ...
    df = df.drop_duplicates()
    df["order_date"] = pd.to_datetime(df["order_date"], errors="coerce", format="%Y-%m-%d")
    df["units"] = pd.to_numeric(df["units"], errors="coerce")
    df["unit_price"] = pd.to_numeric(df["unit_price"], errors="coerce")
    df.loc[df["units"] < 0, "units"] = np.nan                                     # impossible value
    for col in ["customer_name", "region", "status"]:
        df[col] = df[col].str.strip().str.title()
    df["region"] = df["region"].astype("category")
    df["units"] = df["units"].fillna(df["units"].median())
    df["revenue"] = df["units"] * df["unit_price"]
    return df.dropna(subset=["order_date"])                                       # unusable without a date

orders = clean_orders(raw_csv)
print(orders[["order_id", "customer_name", "region", "units", "unit_price", "revenue"]])
print(orders.dtypes.to_dict())
print(orders.groupby("region", observed=True)["revenue"].sum().round(2).to_dict())
orders.to_parquet("orders_clean.parquet", index=False)     # keeps dtypes (requires pyarrow)

What happens to the sample rows: the duplicate 1002 row is removed, “N/A” and “-” become NaN, the −4 units become NaN and are filled with the median of the valid units (3, 2, 1 → 2), names and regions are title-cased, and order 1004 is dropped because its date can’t be parsed.

Checklist

Step pandas
Read with options read_csv(na_values=..., dtype=..., parse_dates=...)
Inspect info(), describe(), isna().sum(), value_counts()
Names df.columns.str.lower().str.replace(" ", "_")
Duplicates drop_duplicates()
Types to_numeric, to_datetime, astype("category")
Text .str.strip().str.title()
Missing / invalid fillna, masks with .loc
Features Vectorized arithmetic, .dt accessors
Save to_parquet

Common Pitfalls

  • Letting pandas guess types, so numbers with stray text stay as strings.

  • Chained assignment instead of .loc.

  • Filling missing values before splitting data for modelling (fit imputation on training data).

  • Saving cleaned data back to CSV and losing dtypes.

Interview Tips

  • Present cleaning as a repeatable function.

  • Mention na_values, errors="coerce" and saving to Parquet.

48

How do you implement a decision tree from scratch in Python?

A decision tree classifier from scratch recursively splits the data: at each node it tries every feature and candidate threshold, picks the split that most reduces impurity (Gini impurity 1 − Σp² or entropy), and recurses on the left and right subsets until a stopping rule applies (maximum depth, minimum samples, or a pure node), where it stores a leaf predicting the majority class. Prediction walks each sample from the root down to a leaf. A compact implementation uses a small Node dataclass, NumPy for the impurity calculations, and checks its accuracy against scikit-learn’s DecisionTreeClassifier.

Read full answer

Applies toPython 3.12, NumPy 2.x

Code Example

from dataclasses import dataclass

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

@dataclass
class Node:
    feature: int | None = None
    threshold: float | None = None
    left: "Node | None" = None
    right: "Node | None" = None
    value: int | None = None              # class label for leaves

def gini(y):
    _, counts = np.unique(y, return_counts=True)
    p = counts / counts.sum()
    return 1.0 - np.sum(p ** 2)

class DecisionTree:
    def __init__(self, max_depth=5, min_samples_split=2):
        self.max_depth, self.min_samples_split = max_depth, min_samples_split

    def fit(self, X, y):
        self.root_ = self._grow(np.asarray(X, float), np.asarray(y), depth=0)
        return self

    def _best_split(self, X, y):
        best_gain, best = 0.0, None
        parent = gini(y)
        for f in range(X.shape[1]):
            values = np.unique(X[:, f])
            for t in (values[:-1] + values[1:]) / 2:                  # midpoints between sorted values
                left = X[:, f] <= t
                n_left = left.sum()
                child = (n_left * gini(y[left]) + (len(y) - n_left) * gini(y[~left])) / len(y)
                if parent - child > best_gain:
                    best_gain, best = parent - child, (f, t)
        return best

    def _grow(self, X, y, depth):
        majority = int(np.bincount(y).argmax())
        if depth >= self.max_depth or len(y) < self.min_samples_split or len(np.unique(y)) == 1:
            return Node(value=majority)
        split = self._best_split(X, y)
        if split is None:
            return Node(value=majority)
        f, t = split
        mask = X[:, f] <= t
        return Node(feature=f, threshold=t,
                    left=self._grow(X[mask], y[mask], depth + 1),
                    right=self._grow(X[~mask], y[~mask], depth + 1))

    def _predict_one(self, x, node):
        while node.value is None:
            node = node.left if x[node.feature] <= node.threshold else node.right
        return node.value

    def predict(self, X):
        return np.array([self._predict_one(x, self.root_) for x in np.asarray(X, float)])

print(round(gini(np.array([0, 0, 1, 1])), 3), round(gini(np.array([0, 0, 0, 0])), 3))   # 0.5 0.0

X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=42)
ours = DecisionTree(max_depth=3).fit(X_tr, y_tr)
sk = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_tr, y_tr)
print("from scratch accuracy:", round(float(np.mean(ours.predict(X_te) == y_te)), 3))
print("scikit-learn accuracy:", round(sk.score(X_te, y_te), 3))   # very similar
print("root split: feature", ours.root_.feature, "<=", round(ours.root_.threshold, 2))

Complexity and Extensions

  • Naive split search is O(features · thresholds · n) per node; sorting once per feature makes it much faster.

  • Regression trees replace Gini with variance (MSE) and leaves predict the mean.

  • Pruning (cost-complexity) and min_samples_leaf reduce overfitting.

Common Pitfalls

  • No stopping rule, giving one leaf per sample (overfitting).

  • Splitting on every unique value instead of midpoints.

  • Forgetting the case where no split improves impurity.

Interview Tips

  • Explain Gini/entropy, the greedy split search and stopping rules.

  • Compare against scikit-learn to show correctness.

49

How do you write a train/test split function in Python?

A train/test split function shuffles row indices with a seeded random generator, takes the first fraction as the test set and the rest as training, and returns the matching feature and label subsets; useful options are stratification (preserving class proportions by splitting within each class), and a no-shuffle mode for time-ordered data. Using index arrays instead of shuffling the data itself works for both NumPy arrays and pandas DataFrames, and the result can be compared with scikit-learn’s train_test_split.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.x

Code Example

import numpy as np
import pandas as pd

def split_train_test(X, y, test_size=0.2, shuffle=True, stratify=None, seed=42):
    """Split X and y (arrays or DataFrames) into train and test parts.

    test_size: fraction in (0, 1) or an integer number of rows.
    stratify: labels to keep proportional across splits (usually y).
    shuffle=False keeps the original order (time series: the test set is the last rows).
    """
    n = len(X)
    if len(y) != n:
        raise ValueError("X and y must have the same length")
    n_test = int(round(test_size * n)) if isinstance(test_size, float) else int(test_size)
    if not 0 < n_test < n:
        raise ValueError("test_size must leave rows in both splits")
    rng = np.random.default_rng(seed)

    if not shuffle:
        idx = np.arange(n)
        train_idx, test_idx = idx[:n - n_test], idx[n - n_test:]
    elif stratify is None:
        idx = rng.permutation(n)
        test_idx, train_idx = idx[:n_test], idx[n_test:]
    else:
        labels = np.asarray(stratify)
        test_parts = []
        for cls in np.unique(labels):
            cls_idx = rng.permutation(np.flatnonzero(labels == cls))
            test_parts.append(cls_idx[:int(round(len(cls_idx) * n_test / n))])
        test_idx = np.concatenate(test_parts)
        train_idx = np.setdiff1d(np.arange(n), test_idx)
        rng.shuffle(train_idx)

    def take(data, rows):
        return data.iloc[rows] if isinstance(data, (pd.DataFrame, pd.Series)) else np.asarray(data)[rows]

    return take(X, train_idx), take(X, test_idx), take(y, train_idx), take(y, test_idx)

rng = np.random.default_rng(42)
X = rng.normal(size=(1000, 4))
y = (rng.random(1000) < 0.1).astype(int)                       # about 10% positives

X_tr, X_te, y_tr, y_te = split_train_test(X, y, test_size=0.2)
print(X_tr.shape, X_te.shape)                                  # (800, 4) (200, 4)

_, _, ys_tr, ys_te = split_train_test(X, y, test_size=0.2, stratify=y)
print(round(y.mean(), 3), round(ys_tr.mean(), 3), round(ys_te.mean(), 3))   # nearly identical rates

df = pd.DataFrame(X, columns=list("abcd")).assign(day=np.arange(1000))
d_tr, d_te, _, _ = split_train_test(df, pd.Series(y), test_size=0.2, shuffle=False)
print(d_tr["day"].max(), d_te["day"].min())                    # 799 800: test is strictly later

Common Pitfalls

  • Shuffling time-series data.

  • Splitting X and y with different random orders, which misaligns labels.

  • Rows from the same user or patient on both sides (use a group split).

  • Not seeding, so splits change every run.

Interview Tips

  • Shuffle indices with a seed, then index both X and y.

  • Mention stratification, time order and group splits.

50

How do you write a Python script that automates hyperparameter tuning with grid search?

An automated grid search script defines the model as a pipeline (so preprocessing is re-fitted inside every fold), a parameter grid using step__param names, a cross-validation strategy and a scoring metric that matches the goal, then runs GridSearchCV in parallel (n_jobs=-1), reports the top configurations with their mean and standard deviation, evaluates the refitted best model once on a held-out test set, and saves both the model and the results. Command-line arguments or a config file make it reusable, and a fixed random seed makes it reproducible.

Read full answer

Applies toscikit-learn 1.5+, Python 3.12

Code Example (tune.py)

import argparse
import json

import joblib
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV, StratifiedKFold, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

PARAM_GRID = [
    {"model__kernel": ["linear"], "model__C": [0.1, 1, 10]},
    {"model__kernel": ["rbf"], "model__C": [0.1, 1, 10, 100], "model__gamma": ["scale", 0.01, 0.001]},
]

def parse_args():
    parser = argparse.ArgumentParser(description="Grid search for an SVM classifier")
    parser.add_argument("--scoring", default="f1")
    parser.add_argument("--folds", type=int, default=5)
    parser.add_argument("--seed", type=int, default=42)
    parser.add_argument("--out", default="best_model.joblib")
    return parser.parse_args()

def main():
    args = parse_args()
    X, y = load_breast_cancer(return_X_y=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=args.seed)

    pipe = Pipeline([("scale", StandardScaler()), ("model", SVC())])
    search = GridSearchCV(
        pipe, PARAM_GRID, scoring=args.scoring, n_jobs=-1, refit=True,
        cv=StratifiedKFold(args.folds, shuffle=True, random_state=args.seed),
    )
    search.fit(X_train, y_train)

    results = pd.DataFrame(search.cv_results_)
    top = results.sort_values("rank_test_score")[["params", "mean_test_score", "std_test_score"]].head(5)
    print(top.to_string(index=False))
    print("best:", search.best_params_, f"CV {args.scoring} = {search.best_score_:.4f}")

    y_pred = search.predict(X_test)                    # test set used once, at the end
    print(classification_report(y_test, y_pred, digits=3))

    joblib.dump(search.best_estimator_, args.out)
    results.to_csv("grid_results.csv", index=False)
    with open("best_params.json", "w") as f:
        json.dump({k: v for k, v in search.best_params_.items()}, f, indent=2, default=str)

if __name__ == "__main__":
    main()

Run with python tune.py --scoring roc_auc --folds 5. The grid has 3 + 12 = 15 combinations, so with 5 folds it trains 75 models plus one final refit.

When Grid Search Isn’t Enough

Situation Alternative
Many hyperparameters RandomizedSearchCV
Expensive models HalvingGridSearchCV, Optuna with pruning
Continuous ranges Random or Bayesian search with log-uniform distributions

Common Pitfalls

  • Scaling outside the pipeline (leakage into CV scores).

  • Huge grids that take hours for tiny gains.

  • Picking the best mean score while ignoring a large standard deviation.

  • Evaluating on the test set during the search.

Interview Tips

  • Show a pipeline, step__param grid names and the final single test evaluation.

  • Mention random and Bayesian search for larger spaces.

51

What if a model does well in training but poorly on new data?

Good training performance with poor results on new data means the model is not generalizing, either because it overfits (memorizes noise) or because the new data differs from the training data; first confirm which: compare training, validation and production metrics, check for leakage that made validation look better than reality, and compare feature distributions between training and new data (data drift). For overfitting, simplify or regularize the model, get more or augmented data, reduce features, use early stopping and proper cross-validation; for drift or a mismatched split, retrain on recent, representative data, use validation splits that mirror deployment (time-based or group-based), and make sure preprocessing is identical in training and serving.

Read full answer

Applies toscikit-learn 1.5+, Python 3.12

Diagnosis

Check Points to
Train ≫ validation score Overfitting
Validation good, production bad Leakage, drift, or training-serving skew
Feature distributions differ (KS test, PSI) Data drift
Same user/time period in train and validation Leaky split
Different preprocessing in production Training-serving skew

Remedies

Cause Fix
Overfitting Regularization, simpler model, max_depth/min_samples_leaf, dropout, early stopping
Too little data More data, augmentation, transfer learning
Too many features Feature selection, dimensionality reduction
Leakage Remove leaky features; fit preprocessing on training folds only
Drift Retrain on recent data, monitor, time-based validation
Skew Ship the fitted pipeline; one feature code path

Code Example

import numpy as np
from scipy.stats import ks_2samp
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_validate

X, y = make_classification(n_samples=800, n_features=30, n_informative=5, flip_y=0.1, random_state=42)

for params in [{}, {"max_depth": 6, "min_samples_leaf": 5, "max_features": 0.3}]:
    res = cross_validate(RandomForestClassifier(random_state=42, **params), X, y, cv=5, return_train_score=True)
    print(params or "default", f"train {res['train_score'].mean():.3f}  validation {res['test_score'].mean():.3f}")
# The constrained forest gives up some training accuracy and narrows the gap

rng = np.random.default_rng(42)
train_feature = rng.normal(0, 1, 2000)
new_feature = rng.normal(0.4, 1.2, 2000)                 # production data has shifted
stat, p = ks_2samp(train_feature, new_feature)
print(f"KS statistic {stat:.3f}, p = {p:.1e} -> drift detected" if p < 0.01 else "no drift")

Common Pitfalls

  • Assuming it’s always overfitting and only adding regularization.

  • Re-tuning on the test set until it looks good.

  • Ignoring that the world changed (seasonality, new users, new products).

Interview Tips

  • Separate overfitting from distribution shift and leakage before fixing.

  • Mention monitoring and validation that mirrors production.

52

How would you approach feature selection in a large dataset?

Approach feature selection in stages, cheapest first: remove useless features (constant or near-constant, mostly missing, duplicates, ID-like columns and anything that leaks the target), drop one of each highly correlated pair, then rank the rest with fast filter methods (mutual information, ANOVA F-test, chi-squared), use embedded methods that select while training (L1/Lasso, tree-based or permutation importance), and finish with wrapper methods (recursive feature elimination or sequential selection) on the shortlist if compute allows. Do selection inside cross-validation so it doesn’t leak into the evaluation, and judge each step by the validation score and by domain sense, not by importance numbers alone.

Read full answer

Applies toscikit-learn 1.5+

Methods

Type Examples Cost Notes
Cleanup VarianceThreshold, missing-rate filter, correlation filter Very low Do first
Filter mutual_info_classif, f_classif, chi2 with SelectKBest Low Ignores feature interactions
Embedded Lasso/L1 logistic, tree importances, SelectFromModel Medium Model-specific
Permutation importance sklearn.inspection.permutation_importance Medium On validation data; less biased than impurity importance
Wrapper RFECV, SequentialFeatureSelector High Best on a shortlist
Dimensionality reduction PCA, embeddings Medium Creates new features, less interpretable

Code Example

import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.feature_selection import RFECV, SelectFromModel, VarianceThreshold
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=1500, n_features=200, n_informative=10, n_redundant=10, random_state=42)
X = pd.DataFrame(X, columns=[f"f{i}" for i in range(200)])
X["constant"] = 1.0
X["dup_f0"] = X["f0"]

X1 = X.loc[:, VarianceThreshold(1e-8).fit(X).get_support()]           # drop constant columns
corr = X1.corr().abs()
upper = corr.where(np.triu(np.ones(corr.shape, dtype=bool), k=1))
X2 = X1.drop(columns=[c for c in upper.columns if (upper[c] > 0.95).any()])   # drop near-duplicates
print(X.shape[1], "->", X1.shape[1], "->", X2.shape[1])                # 202 -> 201 -> about 200 (dup_f0 removed)

cv = StratifiedKFold(5, shuffle=True, random_state=42)
full = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
l1_select = make_pipeline(
    StandardScaler(),
    SelectFromModel(LogisticRegression(penalty="l1", C=0.05, solver="liblinear")),   # selection inside CV
    LogisticRegression(max_iter=2000),
)
print("all features AUC:", round(cross_val_score(full, X2, y, cv=cv, scoring="roc_auc").mean(), 3))
print("L1-selected AUC: ", round(cross_val_score(l1_select, X2, y, cv=cv, scoring="roc_auc").mean(), 3))

l1_select.fit(X2, y)
kept = X2.columns[l1_select.named_steps["selectfrommodel"].get_support()]
print("features kept by L1:", len(kept))

rfe = RFECV(LogisticRegression(max_iter=2000), step=5, cv=cv, scoring="roc_auc", min_features_to_select=5)
rfe.fit(StandardScaler().fit_transform(X2[kept]), y)                   # wrapper method on the shortlist
print("RFECV keeps:", rfe.n_features_)

Common Pitfalls

  • Selecting features on the full dataset and then cross-validating (optimistic scores).

  • Trusting impurity-based importances, which favour high-cardinality features.

  • Dropping a feature that looks weak alone but matters in interactions.

  • Removing features a regulator or the business needs to see.

Interview Tips

  • Describe a staged funnel from cheap filters to wrappers.

  • Stress doing selection inside cross-validation.

53

What strategies help with imbalanced datasets?

For imbalanced datasets (for example 1% fraud), start by choosing the right evaluation (precision, recall, F1, PR-AUC, cost-based metrics, never plain accuracy) and stratified splits; then rebalance the learning with class weights (class_weight="balanced"), resampling (random undersampling of the majority, oversampling or SMOTE for the minority, applied only to training folds), or algorithms that cope well, such as gradient boosting with scale_pos_weight; and finally tune the decision threshold to the business trade-off. Often the cheapest large gains come from threshold tuning and better features rather than resampling, and probability calibration may be needed after reweighting.

Read full answer

Applies toscikit-learn 1.5+, imbalanced-learn 0.12+

Strategies

Level Strategy Notes
Evaluation PR-AUC, recall at fixed precision, F1, cost matrix Accuracy misleads
Splitting Stratified splits and folds Keeps minority examples in every fold
Algorithm class_weight="balanced", scale_pos_weight No data duplication
Data Undersampling, oversampling, SMOTE Only on training data, inside CV
Decision Threshold tuning Matches the cost of FP vs FN
Problem framing Anomaly detection, more minority data For extreme imbalance

Code Example

import numpy as np
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import make_pipeline as make_imb_pipeline
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, f1_score, precision_recall_curve, recall_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=20_000, n_features=15, weights=[0.98, 0.02], class_sep=0.8, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, stratify=y, random_state=42)

models = {
    "plain": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
    "class weights": make_pipeline(StandardScaler(), LogisticRegression(class_weight="balanced", max_iter=1000)),
    "SMOTE": make_imb_pipeline(StandardScaler(), SMOTE(random_state=42), LogisticRegression(max_iter=1000)),
}
for name, model in models.items():
    model.fit(X_tr, y_tr)                                   # SMOTE resamples only the training data
    pred, proba = model.predict(X_te), model.predict_proba(X_te)[:, 1]
    print(f"{name:13s} recall {recall_score(y_te, pred):.3f}  F1 {f1_score(y_te, pred):.3f}  "
          f"PR-AUC {average_precision_score(y_te, proba):.3f}")

proba = models["plain"].predict_proba(X_te)[:, 1]           # threshold tuning on the plain model
precision, recall, thresholds = precision_recall_curve(y_te, proba)
f1 = 2 * precision[:-1] * recall[:-1] / np.clip(precision[:-1] + recall[:-1], 1e-12, None)
best = int(np.argmax(f1))
print(f"best F1 threshold {thresholds[best]:.3f} -> F1 {f1[best]:.3f}")
# In practice choose the threshold on a validation set, not the test set

Common Pitfalls

  • Oversampling before splitting, which puts copies of the same minority rows in train and test.

  • Reporting accuracy.

  • Using reweighted probabilities as if they were calibrated.

  • Choosing the threshold on the test set.

Interview Tips

  • Cover metrics, algorithm-level, data-level and threshold strategies.

  • Warn about resampling leakage.

54

How does regularization work in linear models? Give a Python example.

Regularization adds a penalty on the size of the coefficients to the linear model’s loss so the model can’t fit noise with extreme weights: ridge (L2) adds α·Σw², shrinking all coefficients smoothly toward zero and stabilizing correlated features; lasso (L1) adds α·Σ|w|, which drives some coefficients exactly to zero and so performs feature selection; elastic net mixes both. The strength α (or C = 1/α in scikit-learn’s logistic regression) trades a little bias for lower variance and is chosen by cross-validation, and features must be standardized first because the penalty treats all coefficients on the same scale.

Read full answer

Applies toscikit-learn 1.5+

Comparison

Model Penalty Effect Use when
Ridge α·‖w‖₂² Shrinks all weights; handles collinearity Many correlated useful features
Lasso α·‖w‖₁ Sets some weights to 0 Sparse signal, feature selection
Elastic net α·(ρ‖w‖₁ + (1 − ρ)/2·‖w‖₂²) Sparse and stable with correlated groups Many correlated features, some irrelevant
Logistic regression penalty="l2" / "l1", strength C (smaller = stronger) Same ideas for classification

Code Example

import numpy as np
from sklearn.linear_model import ElasticNetCV, LassoCV, LinearRegression, RidgeCV
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

rng = np.random.default_rng(42)
n, p = 120, 60                                  # few samples, many features: easy to overfit
X = rng.normal(size=(n, p))
X[:, 1] = X[:, 0] + rng.normal(0, 0.05, n)      # two highly correlated features
true_w = np.zeros(p)
true_w[[0, 2, 5]] = [3.0, -2.0, 1.5]            # only 3 features matter
y = X @ true_w + rng.normal(0, 1.0, n)

alphas = np.logspace(-3, 2, 30)
models = {
    "OLS": make_pipeline(StandardScaler(), LinearRegression()),
    "Ridge": make_pipeline(StandardScaler(), RidgeCV(alphas=alphas)),
    "Lasso": make_pipeline(StandardScaler(), LassoCV(alphas=alphas, cv=5, random_state=42, max_iter=50_000)),
    "ElasticNet": make_pipeline(StandardScaler(), ElasticNetCV(l1_ratio=0.5, alphas=alphas, cv=5,
                                                               random_state=42, max_iter=50_000)),
}
for name, model in models.items():
    r2 = cross_val_score(model, X, y, cv=5, scoring="r2").mean()
    model.fit(X, y)
    coef = model[-1].coef_
    print(f"{name:10s} CV R^2 {r2:6.3f}  non-zero coefs {int(np.sum(np.abs(coef) > 1e-6)):2d}  "
          f"largest |coef| {np.abs(coef).max():.2f}")
# OLS overfits with all 60 coefficients; Lasso keeps only a handful, close to the true 3

lasso = models["Lasso"][-1]
print("Lasso alpha:", round(float(lasso.alpha_), 4), "| top features:", np.argsort(-np.abs(lasso.coef_))[:4].tolist())

Choosing α

  • Use RidgeCV, LassoCV, ElasticNetCV or GridSearchCV with a log-spaced range.

  • Plot the regularization path (coefficients vs α) to see which features survive.

Common Pitfalls

  • Regularizing unscaled features, which penalizes features unevenly.

  • Reading lasso’s choice among correlated features as “the” important one (it picks somewhat arbitrarily).

  • Forgetting that in LogisticRegression a smaller C means stronger regularization.

Interview Tips

  • Write both penalties and explain sparsity vs shrinkage.

  • Mention scaling and choosing α by cross-validation.

55

What are the advantages of SGD over standard gradient descent?

Stochastic gradient descent (SGD) updates the parameters using the gradient from one example or a small mini-batch instead of the full dataset, so each step is far cheaper, the model starts improving after seeing only a fraction of the data, memory use doesn’t grow with dataset size, it supports online and streaming learning, and its gradient noise can help escape shallow local minima and saddle points. Full-batch gradient descent gives exact, smooth gradients and stable convergence, but each step requires a pass over all the data, which is impractical for large datasets and deep learning; the price of SGD is noisy convergence, which is handled with learning rate schedules, momentum and mini-batches.

Read full answer

Applies togeneral concepts; examples in Python 3.12 with NumPy 2.x and scikit-learn 1.5+

Comparison

Aspect Batch gradient descent SGD / mini-batch SGD
Gradient per step All n samples 1 sample or a batch (32–512)
Cost per step O(n) O(batch)
Progress per epoch 1 update n / batch updates
Memory Whole dataset One batch
Convergence path Smooth Noisy (needs decaying LR)
Online learning No Yes (partial_fit)
Escaping saddle points Harder Noise helps
Hardware use — Mini-batches use GPUs efficiently

Code Example

import time

import numpy as np
from sklearn.linear_model import SGDRegressor

rng = np.random.default_rng(42)
n, d = 200_000, 10
X = rng.normal(size=(n, d))
w_true = rng.normal(size=d)
y = X @ w_true + rng.normal(0, 0.1, n)

def batch_gd(epochs=20, lr=0.1):
    w = np.zeros(d)
    for _ in range(epochs):                        # one update per pass over all data
        w -= lr * (2 / n) * X.T @ (X @ w - y)
    return w

def minibatch_sgd(epochs=1, lr=0.05, batch=256):
    w = np.zeros(d)
    for _ in range(epochs):
        order = rng.permutation(n)
        for start in range(0, n, batch):           # ~780 updates in one pass
            idx = order[start:start + batch]
            w -= lr * (2 / len(idx)) * X[idx].T @ (X[idx] @ w - y[idx])
    return w

for name, fn in [("batch GD, 20 passes", batch_gd), ("mini-batch SGD, 1 pass", minibatch_sgd)]:
    start = time.perf_counter()
    w = fn()
    print(f"{name:22s} error {np.linalg.norm(w - w_true):.4f}  time {time.perf_counter() - start:.2f}s")
# One SGD pass usually gets closer to the true weights than 20 full-batch steps

online = SGDRegressor(learning_rate="invscaling", eta0=0.01, random_state=42)
for start in range(0, n, 10_000):                  # streaming chunks, like data arriving over time
    online.partial_fit(X[start:start + 10_000], y[start:start + 10_000])
print("online SGD error:", round(float(np.linalg.norm(online.coef_ - w_true)), 4))

Common Pitfalls

  • A constant learning rate that keeps the solution bouncing around the minimum; decay it.

  • Not shuffling data between epochs.

  • Unscaled features, which make SGD unstable.

Interview Tips

  • Contrast cost per step, updates per epoch and noise.

  • Mention mini-batches as the practical middle ground and partial_fit for streaming.

56

How can you parallelize computations in Python for machine learning?

Python ML code parallelizes at several levels: vectorized NumPy and pandas operations already run in optimized, often multi-threaded C code; libraries expose n_jobs=-1 (scikit-learn, XGBoost, LightGBM) to use all CPU cores for cross-validation, grid search and tree building; for your own CPU-bound functions you use processes (multiprocessing, concurrent.futures.ProcessPoolExecutor, joblib.Parallel) because the GIL stops threads from running Python bytecode in parallel; threads suit I/O-bound work such as downloads; Numba compiles numeric loops with prange; GPUs accelerate deep learning and RAPIDS; and Dask, Ray or Spark spread work across machines.

Read full answer

Applies toPython 3.12, joblib 1.4+, scikit-learn 1.5+

Options

Level Tool Best for
Vectorization NumPy, pandas Array and table math
Library built-in n_jobs=-1 CV, grid search, forests, boosting
Processes joblib.Parallel, ProcessPoolExecutor, multiprocessing.Pool CPU-bound Python functions
Threads ThreadPoolExecutor I/O-bound (APIs, files, downloads)
JIT compilation Numba (@njit(parallel=True)) Custom numeric loops
GPU PyTorch, CuPy, RAPIDS cuDF/cuML Deep learning, large arrays
Cluster Dask, Ray, Spark Data or tuning across machines

Code Example

import time
from concurrent.futures import ThreadPoolExecutor

import numpy as np
from joblib import Parallel, delayed
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

def slow_feature(seed):
    """A CPU-bound pure-Python task, e.g. a simulation per customer."""
    rng = np.random.default_rng(seed)
    total = 0.0
    for _ in range(200_000):
        total += rng.random() ** 2
    return total

def timed(label, fn):
    start = time.perf_counter()
    result = fn()
    print(f"{label:28s} {time.perf_counter() - start:.2f}s")
    return result

if __name__ == "__main__":                       # required for process pools on macOS/Windows
    seeds = range(8)
    serial = timed("serial", lambda: [slow_feature(s) for s in seeds])
    procs = timed("joblib processes (n_jobs=-1)", lambda: Parallel(n_jobs=-1)(delayed(slow_feature)(s) for s in seeds))
    with ThreadPoolExecutor(max_workers=8) as pool:
        threads = timed("threads (GIL-bound)", lambda: list(pool.map(slow_feature, seeds)))
    print("same results:", np.allclose(serial, procs) and np.allclose(serial, threads))

    X, y = make_classification(n_samples=5000, n_features=30, random_state=42)
    rf = RandomForestClassifier(n_estimators=200, random_state=42)
    timed("CV n_jobs=1", lambda: cross_val_score(rf, X, y, cv=5, n_jobs=1))
    timed("CV n_jobs=-1", lambda: cross_val_score(rf, X, y, cv=5, n_jobs=-1))
# Processes and n_jobs=-1 speed up CPU-bound work on multi-core machines; threads barely help here

Common Pitfalls

  • Nested parallelism (parallel CV around a model that also uses all cores) oversubscribing the CPU; parallelize one level.

  • Large data copied to every process; use shared memory, memory-mapped arrays (joblib does this for big NumPy arrays) or Dask.

  • Parallelizing tiny tasks, where process start-up costs more than the work.

  • Forgetting the if __name__ == "__main__": guard.

Interview Tips

  • Explain the GIL and why processes beat threads for CPU-bound Python.

  • Start with vectorization and n_jobs, then joblib, Numba, GPUs and Dask/Ray.

57

Why does model persistence matter and how do you save and load models?

Model persistence means saving a trained model to disk so it can be reused without retraining: it is what lets you deploy a model to a server, reproduce results, compare versions, roll back to an earlier model, and share models across teams. For scikit-learn, save the whole fitted pipeline (preprocessing plus model) with joblib.dump and load it with joblib.load in the same library versions; for PyTorch, save the state_dict with torch.save and load it into the same model class; for Keras, use model.save("model.keras"); and for cross-language or runtime-independent deployment, export to ONNX. Store metadata (library versions, training data version, metrics, feature names) next to the file, and never load pickle-based files from untrusted sources.

Read full answer

Applies toscikit-learn 1.5+, joblib 1.4+, Python 3.12

Formats

Library Save Load Notes
scikit-learn joblib.dump(pipe, "m.joblib") joblib.load(...) Pickle-based; same versions needed
scikit-learn (safer) skops.io.dump skops.io.load(trusted=...) Restricts what can be loaded
XGBoost / LightGBM model.save_model("m.json") load_model Version-stable native formats
PyTorch torch.save(model.state_dict(), "m.pt") load_state_dict(torch.load(..., weights_only=True))
Keras model.save("m.keras") keras.models.load_model
Any (inference) ONNX export ONNX Runtime Language-independent

Code Example

import json
import platform
from datetime import date

import joblib
import numpy as np
import sklearn
from sklearn.datasets import load_wine
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_wine(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)
pipe = make_pipeline(StandardScaler(), RandomForestClassifier(random_state=42)).fit(X_tr, y_tr)

joblib.dump(pipe, "wine_model.joblib", compress=3)        # preprocessing and model together
metadata = {
    "trained_on": str(date.today()),
    "sklearn_version": sklearn.__version__,
    "python_version": platform.python_version(),
    "test_accuracy": round(pipe.score(X_te, y_te), 4),
    "n_features": X.shape[1],
}
with open("wine_model.json", "w") as f:
    json.dump(metadata, f, indent=2)

loaded = joblib.load("wine_model.joblib")                  # only load files you trust
with open("wine_model.json") as f:
    meta = json.load(f)
if meta["sklearn_version"] != sklearn.__version__:
    print("warning: model was saved with a different scikit-learn version")
print("identical predictions:", np.array_equal(loaded.predict(X_te), pipe.predict(X_te)))   # True
print(meta["test_accuracy"])

Common Pitfalls

  • Saving only the model and not the fitted preprocessing, so production inputs are transformed differently.

  • Loading a joblib/pickle file with a different library version (warnings or wrong behaviour).

  • Loading pickles from untrusted sources (arbitrary code execution).

  • No versioning, so you can’t tell which model is in production.

Interview Tips

  • Explain why persistence matters (deployment, reproducibility, rollback).

  • Show saving the full pipeline plus metadata, and mention ONNX and security.

58

How do you interpret the coefficients of a logistic regression model?

A logistic regression coefficient is the change in the log-odds of the positive class for a one-unit increase in that feature, holding the other features constant; exponentiating it gives an odds ratio, so a coefficient of 0.7 means the odds are multiplied by e^0.7 ≈ 2.01 per unit, a negative coefficient lowers the odds, and zero means no effect. The intercept is the log-odds when all features are zero (or at their means if features are centred). To compare features, standardize them first so coefficients are per standard deviation, and remember that coefficients describe associations in the data, not causes, and become unreliable when features are strongly correlated or heavily regularized.

Read full answer

Applies toscikit-learn 1.5+, statsmodels 0.14+

Interpretation Guide

Quantity Meaning
β (coefficient) Change in log-odds per unit of x
e^β (odds ratio) Multiplicative change in odds per unit of x
e^β > 1 Feature raises the probability
e^β < 1 Feature lowers the probability
Standardized β Effect per standard deviation; comparable across features
Intercept Log-odds when all features are 0

Worked Example

With β_age = 0.05 per year: e^0.05 ≈ 1.051, so each extra year multiplies the odds by about 1.05 (+5%); ten years multiply them by e^0.5 ≈ 1.65. The effect on probability is not constant: it is largest near p = 0.5 and small near 0 or 1.

Code Example

import numpy as np
import pandas as pd
import statsmodels.api as sm
from sklearn.linear_model import LogisticRegression

rng = np.random.default_rng(42)
n = 5000
df = pd.DataFrame({
    "age": rng.uniform(20, 70, n),
    "smoker": rng.integers(0, 2, n),
    "exercise_hours": rng.uniform(0, 10, n),
})
log_odds = -6 + 0.08 * df["age"] + 1.2 * df["smoker"] - 0.15 * df["exercise_hours"]
df["disease"] = (rng.random(n) < 1 / (1 + np.exp(-log_odds))).astype(int)

X = df[["age", "smoker", "exercise_hours"]]
model = LogisticRegression(penalty=None, max_iter=1000).fit(X, df["disease"])   # no regularization
coefs = pd.Series(model.coef_[0], index=X.columns)
print(pd.DataFrame({"coef": coefs.round(3), "odds_ratio": np.exp(coefs).round(3)}))
# age close to 0.08 (odds x1.08 per year), smoker close to 1.2 (odds x3.3), exercise close to -0.15 (x0.86 per hour)

sm_model = sm.Logit(df["disease"], sm.add_constant(X)).fit(disp=False)        # p-values and intervals
ci = np.exp(sm_model.conf_int())
ci.columns = ["odds_ratio_2.5%", "odds_ratio_97.5%"]
print(ci.round(3))

X_std = (X - X.mean()) / X.std()
std_coefs = LogisticRegression(penalty=None, max_iter=1000).fit(X_std, df["disease"]).coef_[0]
print("per-SD effects:", dict(zip(X.columns, std_coefs.round(3))))          # comparable sizes

Common Pitfalls

  • Reading coefficients as changes in probability (they are log-odds).

  • Comparing raw coefficients across features with different units.

  • Interpreting regularized coefficients (shrunk toward zero) as unbiased effect sizes.

  • Multicollinearity making signs and sizes unstable.

  • Claiming causality from observational data.

Interview Tips

  • Convert a coefficient to an odds ratio out loud.

  • Mention standardization, regularization and correlation caveats.

59

What is dimensionality reduction and when would you use it?

Dimensionality reduction transforms data with many features into fewer features that keep most of the useful information, either by feature extraction (creating new combined features: PCA, truncated SVD, autoencoders, or t-SNE and UMAP for visualization) or by feature selection (keeping a subset of the original columns). You use it to visualize high-dimensional data in 2-D or 3-D, to speed up training and reduce memory, to remove noise and multicollinearity, to fight the curse of dimensionality in distance-based methods like kNN and clustering, and to compress inputs such as images or text embeddings, while accepting some loss of information and interpretability.

Read full answer

Applies toscikit-learn 1.5+; umap-learn optional

Methods

Method Type Preserves Use for
PCA Linear Global variance Compression, decorrelation, preprocessing
Truncated SVD Linear Variance (works on sparse) Text TF-IDF (LSA)
LDA (linear discriminant) Linear, supervised Class separation Classification preprocessing
t-SNE Non-linear Local neighbourhoods 2-D visualization only
UMAP Non-linear Local + some global structure Visualization, clustering preprocessing
Autoencoders Non-linear (neural) Learned structure Images, complex data
Feature selection Subset Original features Interpretability

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.manifold import TSNE
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_digits(return_X_y=True)                         # 1797 images, 64 pixel features
pca = PCA().fit(StandardScaler().fit_transform(X))
n95 = int(np.searchsorted(np.cumsum(pca.explained_variance_ratio_), 0.95) + 1)
print(f"components for 95% of variance: {n95} of 64")

for k in [64, n95, 10]:
    model = make_pipeline(StandardScaler(), PCA(n_components=k), LogisticRegression(max_iter=3000))
    print(f"{k:2d} components: CV accuracy {cross_val_score(model, X, y, cv=5).mean():.3f}")
# Accuracy stays close to the full model with far fewer features

emb = TSNE(n_components=2, random_state=42, init="pca").fit_transform(X)    # visualization only
plt.figure(figsize=(6, 5))
plt.scatter(emb[:, 0], emb[:, 1], c=y, cmap="tab10", s=5)
plt.colorbar(label="digit")
plt.title("t-SNE of handwritten digits")
plt.savefig("digits_tsne.png", dpi=120)

When Not to Use It

  • When interpretability of individual original features is required (prefer feature selection).

  • With tree-based models on moderate feature counts, which handle many features well.

  • Using t-SNE/UMAP coordinates as model features without care (distances between clusters aren’t meaningful).

Common Pitfalls

  • Running PCA on unscaled features (large-scale features dominate).

  • Fitting PCA on all data before the train/test split.

  • Reading t-SNE cluster sizes or distances literally.

Interview Tips

  • Separate extraction from selection and linear from non-linear.

  • Give use cases: visualization, speed, noise removal, curse of dimensionality.

60

What is a neural network and how would you implement one in Python?

A neural network is a model built from layers of simple units (“neurons”): each computes a weighted sum of its inputs plus a bias and passes it through a non-linear activation such as ReLU, so stacked layers can learn complex, non-linear functions; training adjusts the weights by backpropagation (computing gradients of a loss with the chain rule) and gradient descent. In Python you can write a small network from scratch with NumPy to understand the mechanics, use scikit-learn’s MLPClassifier for quick tabular baselines, and use PyTorch or Keras for real deep learning with GPUs, automatic differentiation and modern layers.

Read full answer

Applies toPython 3.12, NumPy 2.x, scikit-learn 1.5+

Building Blocks

Component Role
Layer (weights W, bias b) z = xW + b
Activation (ReLU, sigmoid, tanh, softmax) Adds non-linearity
Loss (cross-entropy, MSE) Measures error
Backpropagation Gradients of the loss for every weight
Optimizer (SGD, Adam) Updates weights
Epochs and mini-batches Repeated passes over data

Code Example: From Scratch with NumPy

import numpy as np
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier

rng = np.random.default_rng(42)
X, y = make_moons(n_samples=1000, noise=0.2, random_state=42)        # not linearly separable
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)

def init(n_in, n_hidden, n_out=1):
    return {"W1": rng.normal(0, np.sqrt(2 / n_in), (n_in, n_hidden)), "b1": np.zeros(n_hidden),
            "W2": rng.normal(0, np.sqrt(1 / n_hidden), (n_hidden, n_out)), "b2": np.zeros(n_out)}

def forward(p, X):
    z1 = X @ p["W1"] + p["b1"]
    a1 = np.maximum(0, z1)                                  # ReLU
    z2 = a1 @ p["W2"] + p["b2"]
    return z1, a1, 1 / (1 + np.exp(-z2))                    # sigmoid output probability

def train(X, y, n_hidden=16, lr=0.5, epochs=2000):
    p = init(X.shape[1], n_hidden)
    y = y.reshape(-1, 1)
    for epoch in range(epochs):
        z1, a1, out = forward(p, X)
        loss = -np.mean(y * np.log(out + 1e-12) + (1 - y) * np.log(1 - out + 1e-12))
        d_z2 = (out - y) / len(X)                           # gradient of BCE w.r.t. pre-sigmoid output
        grads = {"W2": a1.T @ d_z2, "b2": d_z2.sum(axis=0)}
        d_z1 = (d_z2 @ p["W2"].T) * (z1 > 0)                # backprop through ReLU
        grads.update({"W1": X.T @ d_z1, "b1": d_z1.sum(axis=0)})
        for k in p:
            p[k] -= lr * grads[k]
        if epoch % 500 == 0:
            print(f"epoch {epoch}: loss {loss:.4f}")
    return p

params = train(X_tr, y_tr)
acc = np.mean((forward(params, X_te)[2].ravel() > 0.5) == y_te)
print(f"from-scratch network test accuracy: {acc:.3f}")      # well above a linear model's accuracy

mlp = MLPClassifier(hidden_layer_sizes=(16,), max_iter=2000, random_state=42).fit(X_tr, y_tr)
print(f"scikit-learn MLP test accuracy: {mlp.score(X_te, y_te):.3f}")

For larger networks, the same model in PyTorch is nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 1)) trained with nn.BCEWithLogitsLoss and an optimizer, with autograd computing the gradients written by hand above.

Common Pitfalls

  • Unscaled inputs, which slow or break training.

  • Poor initialization (all zeros makes every neuron identical).

  • Learning rates that are too high (loss explodes) or too low (no progress).

  • Using neural networks on small tabular datasets where gradient boosting is usually better.

Interview Tips

  • Explain layers, activations, loss, backprop and gradient descent in order.

  • Show you can write the forward and backward pass, then say you’d use PyTorch or Keras in practice.

61

What is reinforcement learning and what are its implementation challenges?

Reinforcement learning (RL) trains an agent to make a sequence of decisions by interacting with an environment: at each step the agent observes a state, takes an action, and receives a reward and the next state, and it learns a policy that maximizes the expected cumulative (discounted) reward, using methods such as Q-learning and DQN (value-based), policy gradients and PPO (policy-based), or actor-critic algorithms like SAC. The main implementation challenges are designing a reward that produces the intended behaviour, sample inefficiency (millions of interactions), the exploration-exploitation trade-off, unstable training and high variance across random seeds, delayed credit assignment, and the gap between simulation and the real world; in Python, Gymnasium supplies environments and Stable-Baselines3 or TorchRL supply tested algorithms.

Read full answer

Applies toPython 3.12, Gymnasium 1.x, Stable-Baselines3 2.x

Core Concepts

Concept Meaning
State / observation What the agent sees
Action What it does (discrete or continuous)
Reward Immediate feedback signal
Policy π(a|s) The agent’s behaviour
Value V(s), Q(s, a) Expected future reward
Discount γ How much future rewards count

Implementation Challenges

Challenge Mitigation
Reward design / reward hacking Careful shaping, watching episodes, constraints
Sample inefficiency Off-policy methods with replay, model-based RL, simulators
Exploration ε-greedy, entropy bonuses, curiosity
Instability and seed variance Tested libraries, target networks, several seeds
Credit assignment Discounting, advantage estimation (GAE)
Sim-to-real gap Domain randomization, conservative policies
Safety Constrained RL, action shields, offline evaluation

Code Example

import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy

env = gym.make("CartPole-v1")
model = PPO("MlpPolicy", env, seed=42, verbose=0)

before, _ = evaluate_policy(model, env, n_eval_episodes=10)
model.learn(total_timesteps=50_000)
after, std = evaluate_policy(model, env, n_eval_episodes=20)
print(f"mean episode reward: before {before:.1f}, after {after:.1f} +/- {std:.1f}")   # 500 is the maximum

obs, info = env.reset(seed=42)                    # running the trained policy
total = 0.0
for _ in range(500):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    total += reward
    if terminated or truncated:
        break
print("one greedy episode:", total)

Common Pitfalls

  • Writing algorithms from scratch for production instead of using tested libraries.

  • Judging from a single seed or a single episode.

  • Environment bugs (wrong termination, leaking information) mistaken for algorithm problems.

Interview Tips

  • Define the agent-environment loop and the main algorithm families.

  • Name reward design, sample efficiency and exploration as the key challenges.

62

What are generative adversarial networks and what are they used for?

A generative adversarial network (GAN) consists of two neural networks trained against each other: a generator that turns random noise into synthetic samples and a discriminator that tries to tell real samples from generated ones; the generator improves by fooling the discriminator, and at equilibrium it produces samples that resemble the training data. GANs have been used for realistic image synthesis (StyleGAN faces), image-to-image translation (pix2pix, CycleGAN), super-resolution, data augmentation and synthetic tabular data (CTGAN), and anomaly detection. They are known to be hard to train (instability, mode collapse), and diffusion models have overtaken them for most image generation, though GANs remain popular where single-step, fast generation matters.

Read full answer

Applies togeneral concepts; example in PyTorch 2.x

How Training Works

  1. Sample real data and generate fake data from noise.

  2. Train the discriminator to output 1 for real and 0 for fake.

  3. Train the generator so the discriminator outputs 1 for its fakes.

  4. Repeat; neither network should overpower the other.

Use Cases

Use case Example
Image synthesis StyleGAN faces, artwork
Image-to-image Sketches to photos, day to night, map to satellite (pix2pix, CycleGAN)
Super-resolution SRGAN, ESRGAN
Data augmentation Extra training images for rare classes
Synthetic tabular data CTGAN for privacy-preserving data sharing
Anomaly detection Poor reconstruction marks unusual samples

Code Example (Compact)

import torch
import torch.nn as nn

torch.manual_seed(42)
real_data = lambda n: torch.randn(n, 1) * 0.5 + 3.0          # target distribution: N(3, 0.5)

G = nn.Sequential(nn.Linear(4, 32), nn.ReLU(), nn.Linear(32, 1))
D = nn.Sequential(nn.Linear(1, 32), nn.LeakyReLU(0.2), nn.Linear(32, 1))
opt_g = torch.optim.Adam(G.parameters(), lr=1e-3)
opt_d = torch.optim.Adam(D.parameters(), lr=1e-3)
bce = nn.BCEWithLogitsLoss()

for step in range(3000):
    real, fake = real_data(128), G(torch.randn(128, 4))
    d_loss = bce(D(real), torch.ones(128, 1)) + bce(D(fake.detach()), torch.zeros(128, 1))
    opt_d.zero_grad()
    d_loss.backward()
    opt_d.step()

    g_loss = bce(D(G(torch.randn(128, 4))), torch.ones(128, 1))
    opt_g.zero_grad()
    g_loss.backward()
    opt_g.step()

with torch.no_grad():
    samples = G(torch.randn(5000, 4))
print(f"generated mean {samples.mean():.2f}, std {samples.std():.2f}")   # should approach 3.0 and 0.5

Challenges

  • Mode collapse: the generator produces a few similar outputs.

  • Instability: oscillating losses; use spectral normalization, WGAN-GP, careful learning rates.

  • Evaluation: no single loss to trust; use FID, precision/recall and visual inspection.

  • Ethics: deepfakes and misuse; provenance and watermarking matter.

Interview Tips

  • Explain the generator-discriminator game and the alternating updates.

  • Name use cases and training challenges, and compare briefly with diffusion models.

63

What is transfer learning and how do you implement it with Python libraries?

Transfer learning reuses a model trained on a large source dataset as the starting point for a related target task, so you need much less labelled data and compute: the pre-trained network’s early layers already capture general features (edges and textures for images, grammar and meaning for text), and you adapt it by replacing the output layer and either training only the new head (feature extraction) or fine-tuning some or all layers with a low learning rate. In Python, torchvision and timm provide pre-trained vision models for PyTorch, Keras Applications provide them for TensorFlow, and Hugging Face Transformers provides pre-trained language and vision models through from_pretrained.

Read full answer

Applies toPyTorch 2.x / torchvision 0.17+, Keras 3, Hugging Face Transformers 4.40+

Approaches

Approach Trainable layers Data needed
Feature extraction New head only Small
Partial fine-tuning Top layers + head Medium
Full fine-tuning All layers, low LR Larger, or a different domain
Parameter-efficient (LoRA, adapters) Small added modules Large models, limited GPU

Code Example: Keras (Feature Extraction, Then Fine-Tuning)

import keras
import numpy as np

keras.utils.set_random_seed(42)
base = keras.applications.MobileNetV2(input_shape=(160, 160, 3), include_top=False, weights="imagenet")
base.trainable = False                                    # stage 1: freeze the pre-trained backbone

inputs = keras.Input(shape=(160, 160, 3))
x = keras.applications.mobilenet_v2.preprocess_input(inputs)
x = base(x, training=False)                               # keep BatchNorm in inference mode
x = keras.layers.GlobalAveragePooling2D()(x)
x = keras.layers.Dropout(0.2)(x)
outputs = keras.layers.Dense(3, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.compile(optimizer=keras.optimizers.Adam(1e-3), loss="sparse_categorical_crossentropy", metrics=["accuracy"])

rng = np.random.default_rng(42)
images = rng.uniform(0, 255, (32, 160, 160, 3)).astype("float32")   # stands in for a real image dataset
labels = rng.integers(0, 3, 32)
model.fit(images, labels, epochs=1, batch_size=8, verbose=0)

base.trainable = True                                     # stage 2: fine-tune the top layers
for layer in base.layers[:-20]:
    layer.trainable = False
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(images, labels, epochs=1, batch_size=8, verbose=0)
print("trainable weights after unfreezing:", len(model.trainable_weights))

Hugging Face Text Example

from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name, num_labels=2)   # new classification head
batch = tokenizer(["great product", "terrible support"], padding=True, return_tensors="pt")
print(model(**batch).logits.shape)                         # torch.Size([2, 2]); fine-tune with Trainer

Common Pitfalls

  • Wrong preprocessing for the pre-trained model (each family has its own).

  • A high learning rate during fine-tuning, which destroys learned features.

  • Forgetting to recompile (Keras) or rebuild the optimizer (PyTorch) after unfreezing.

Interview Tips

  • Explain feature extraction vs fine-tuning and when to use each.

  • Name the libraries for each framework.

64

What is the difference between batch learning and online learning?

Batch (offline) learning trains a model on the whole available dataset at once and then deploys it unchanged until it is retrained from scratch on a new snapshot; online (incremental) learning updates the model continuously as new examples or mini-batches arrive, without revisiting all past data. Batch learning is simpler, easier to validate and the default for most systems; online learning suits data streams too large to store, fast-changing environments (news recommendations, fraud, ad click prediction) and memory-limited devices, but it needs safeguards against noisy or malicious data and drift in quality, plus careful choice of learning rate (how fast it forgets).

Read full answer

Applies toscikit-learn 1.5+ (`partial_fit`), River 0.21+

Comparison

Aspect Batch learning Online learning
Data Full dataset in one go Streamed one example or chunk at a time
Update Periodic full retraining Continuous incremental updates
Memory Must handle the whole dataset Constant
Adapts to change At the next retrain Quickly
Validation Standard train/test, CV Prequential (test-then-train) evaluation
Risks Stale models Bad data immediately affects the model
Tools Any fit partial_fit (SGD, Naive Bayes, MiniBatchKMeans), River

Code Example

import numpy as np
from sklearn.linear_model import SGDClassifier
from sklearn.metrics import accuracy_score

rng = np.random.default_rng(42)

def make_chunk(n, shift):
    X = rng.normal(size=(n, 2))
    y = (X[:, 0] + shift * X[:, 1] > 0).astype(int)        # the decision rule drifts over time
    return X, y

chunks = [make_chunk(500, shift) for shift in np.linspace(-1, 1, 10)]

batch_model = SGDClassifier(loss="log_loss", random_state=42).fit(*chunks[0])   # trained once at the start
online_model = SGDClassifier(loss="log_loss", random_state=42)

for i, (X, y) in enumerate(chunks):
    if i > 0:                                               # test on new data before training on it
        batch_acc = accuracy_score(y, batch_model.predict(X))
        online_acc = accuracy_score(y, online_model.predict(X))
        print(f"chunk {i}: batch {batch_acc:.2f}  online {online_acc:.2f}")
    online_model.partial_fit(X, y, classes=[0, 1])          # incremental update
# As the rule drifts, the batch model's accuracy falls while the online model keeps up

Hybrid Practice

  • Many production systems retrain in batch on a schedule (daily or weekly) and add online updates or fast fine-tuning between retrains.

  • Keep a holdout stream and rollback ability for online models.

Common Pitfalls

  • Online learning without monitoring: one bad data source can quickly degrade the model.

  • Evaluating an online model with a static test set instead of prequential evaluation.

  • Forgetting to pass classes on the first partial_fit call.

Interview Tips

  • Contrast data handling, adaptation and risk.

  • Give an example of where each fits and mention partial_fit.

65

What role do attention mechanisms play in NLP models?

Attention lets a model decide, for each word (token) it processes, how much to draw on every other token in the sequence: each token’s query vector is compared with every token’s key vector to produce weights (scaled dot products passed through softmax), and the output is the weighted sum of the value vectors. This solved the bottleneck of RNN encoder-decoders, which squeezed a whole sentence into one fixed vector, and it captures long-range dependencies (linking a pronoun to its noun many words back) in a single step. Self-attention, stacked with multiple heads in the Transformer architecture, is the core of modern NLP models such as BERT and GPT, which are trained in parallel on large text corpora and power translation, summarization, question answering and chatbots.

Read full answer

Applies togeneral concepts; examples in Python 3.12 with NumPy 2.x and Hugging Face Transformers

Scaled Dot-Product Attention

Attention(Q, K, V) = softmax(QKᵀ / √d_k) · V

Part Role
Query (Q) What this token is looking for
Key (K) What each token offers
Value (V) The information passed along
√d_k scaling Keeps softmax from saturating
Masking Hides padding or future tokens (in GPT-style decoders)
Multi-head Several attention patterns in parallel (syntax, coreference, position)

Code Example

import numpy as np

def softmax(x, axis=-1):
    e = np.exp(x - x.max(axis=axis, keepdims=True))
    return e / e.sum(axis=axis, keepdims=True)

def attention(Q, K, V, causal=False):
    scores = Q @ K.T / np.sqrt(K.shape[-1])
    if causal:                                              # a token can't look at later tokens
        scores = np.where(np.triu(np.ones_like(scores), k=1) == 1, -1e9, scores)
    weights = softmax(scores)
    return weights @ V, weights

rng = np.random.default_rng(42)
tokens = ["the", "cat", "sat", "down"]
d = 8
X = rng.normal(size=(len(tokens), d))                       # token embeddings
Wq, Wk, Wv = (rng.normal(size=(d, d)) / np.sqrt(d) for _ in range(3))

out, w = attention(X @ Wq, X @ Wk, X @ Wv)
print(out.shape, w.sum(axis=1).round(6))                    # (4, 8) [1. 1. 1. 1.]: each row of weights sums to 1
_, w_causal = attention(X @ Wq, X @ Wk, X @ Wv, causal=True)
print(np.round(w_causal, 2))                                # zeros above the diagonal

With Hugging Face you can inspect real attention weights: AutoModel.from_pretrained("bert-base-uncased", output_attentions=True) returns one attention tensor per layer, shaped (batch, heads, tokens, tokens).

Impact on NLP

Before attention With attention / Transformers
RNNs process tokens sequentially All tokens processed in parallel (fast on GPUs)
Long-range dependencies fade Direct links between any two tokens
Fixed context vector Dynamic, token-specific context
Task-specific models Pre-trained models fine-tuned for many tasks

Common Pitfalls

  • Treating attention weights as a full explanation of model decisions (they are only one signal).

  • Ignoring the quadratic cost in sequence length (efficient attention variants address it).

Interview Tips

  • Write the attention formula and explain Q, K, V.

  • Connect attention to Transformers, BERT and GPT.

66

How do you implement a recommendation system in Python?

A recommendation system in Python usually starts with the interaction data (user, item, rating or implicit signal like clicks) and one of three approaches: content-based filtering (recommend items similar to what the user liked, using item features such as TF-IDF of descriptions), collaborative filtering (learn from patterns across users, via user/item similarity or matrix factorization such as SVD or ALS), or a hybrid of both. You evaluate with a time-based split and ranking metrics such as precision@k, recall@k and NDCG, compare against a popularity baseline, and handle cold-start users and items with content features or popular items. Libraries include scikit-learn and SciPy for building blocks, Surprise for explicit ratings, implicit for implicit-feedback ALS, and LightFM or TensorFlow Recommenders for hybrid and deep models.

Read full answer

Applies toPython 3.12, NumPy 2.x, SciPy 1.13+, scikit-learn 1.5+

Approaches

Approach Idea Strengths Weaknesses
Popularity Most popular items Simple baseline, cold start Not personal
Content-based Similar item features Works for new items Narrow recommendations
User/item kNN CF Similar users or items by interactions Intuitive Sparse data, scaling
Matrix factorization Latent user and item vectors Accurate, scalable Cold start
Hybrid / deep (two-tower) Combine signals Best quality Complexity

Code Example: Matrix Factorization with Truncated SVD

import numpy as np
from scipy.sparse import csr_matrix
from scipy.sparse.linalg import svds

rng = np.random.default_rng(42)
n_users, n_items, k_true = 300, 200, 5
U_true, V_true = rng.normal(size=(n_users, k_true)), rng.normal(size=(n_items, k_true))
scores = U_true @ V_true.T

rows, cols, vals = [], [], []
for u in range(n_users):                                  # each user interacts with their top items plus noise
    liked = np.argsort(-scores[u] + rng.normal(0, 2, n_items))[:15]
    rows += [u] * len(liked)
    cols += liked.tolist()
    vals += [1.0] * len(liked)
R = csr_matrix((vals, (rows, cols)), shape=(n_users, n_items))

test = {u: int(R[u].indices[rng.integers(R[u].nnz)]) for u in range(n_users)}   # hide one item per user
R_train = R.tolil()
for u, i in test.items():
    R_train[u, i] = 0
R_train = R_train.tocsr()
R_train.eliminate_zeros()                                # drop the hidden entries completely

U, s, Vt = svds(R_train, k=10)                            # low-rank factorization
pred = U @ np.diag(s) @ Vt

def hit_rate(pred, k=10):
    hits = 0
    for u, held_out in test.items():
        p = pred[u].copy()
        p[R_train[u].indices] = -np.inf                   # don't recommend already-seen items
        hits += held_out in np.argsort(-p)[:k]
    return hits / len(test)

popularity = np.tile(np.asarray(R_train.sum(axis=0)).ravel(), (n_users, 1))
print(f"hit rate@10  SVD: {hit_rate(pred):.3f}  popularity: {hit_rate(popularity):.3f}")
# The personalized factorization should beat the popularity baseline

Production Considerations

  • Candidate generation (fast retrieval of hundreds of items) followed by a ranking model.

  • Freshness: retrain or update embeddings regularly.

  • Diversity and business rules on top of scores.

  • Online A/B tests, since offline metrics don’t fully predict engagement.

Common Pitfalls

  • Random train/test splits that leak future interactions.

  • Recommending items the user has already bought.

  • No popularity baseline.

Interview Tips

  • Compare content-based, collaborative and hybrid methods.

  • Mention ranking metrics, cold start and time-based evaluation.

67

How would you build a spam detection system in Python?

A spam detector is a text classification pipeline: collect labelled messages (spam vs not spam), clean and tokenize the text, turn it into features (TF-IDF of words and character n-grams, plus metadata such as the number of links, capital-letter ratio or sender reputation), train a classifier (multinomial Naive Bayes or logistic regression as strong, fast baselines; fine-tuned transformers when accuracy matters most), and evaluate with precision and recall, because flagging a real email as spam is usually worse than letting one spam through. In production, the trained pipeline is saved, served behind an API or mail filter, monitored, and retrained regularly, since spammers change tactics.

Read full answer

Applies toscikit-learn 1.5+, Python 3.12

Design

Stage Choice
Data Labelled emails/SMS; time-based split
Text features TF-IDF word 1–2-grams, character 3–5-grams (catch obfuscations like “fr€e”)
Extra features URL count, attachments, sender domain, header checks
Models Multinomial NB, logistic regression, linear SVM; transformers for the best accuracy
Metric Precision on the spam class (few false positives), recall, PR-AUC
Threshold Tuned to a maximum false positive rate
Deployment Saved pipeline, API, feedback loop from “report spam” buttons

Code Example

import joblib
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.pipeline import FeatureUnion, Pipeline

rng = np.random.default_rng(42)
spam_templates = ["WIN a FREE {} now!!! click {}", "Cheap {} offer, limited time, visit {}",
                  "You have been selected for a {} prize, claim at {}", "URGENT: your {} account, verify at {}"]
ham_templates = ["Can we move the {} meeting to {}?", "Here are the notes from {} about {}",
                 "Lunch at {} tomorrow? Bring the {} report", "Thanks for the {} update, see you {}"]
words = ["iphone", "project", "loan", "monday", "budget", "team", "gift card", "friday", "vacation", "invoice"]
links = ["http://bit.ly/x9", "www.win-now.biz", "http://claim.example"]

def make(templates, n, is_spam):
    out = []
    for _ in range(n):
        t = templates[rng.integers(len(templates))]
        second = links[rng.integers(len(links))] if is_spam else words[rng.integers(len(words))]
        out.append(t.format(words[rng.integers(len(words))], second))
    return out

texts = make(spam_templates, 300, True) + make(ham_templates, 700, False)
labels = np.r_[np.ones(300, int), np.zeros(700, int)]
X_tr, X_te, y_tr, y_te = train_test_split(texts, labels, test_size=0.25, stratify=labels, random_state=42)

features = FeatureUnion([
    ("words", TfidfVectorizer(ngram_range=(1, 2), lowercase=True, min_df=2)),
    ("chars", TfidfVectorizer(analyzer="char_wb", ngram_range=(3, 5), min_df=2)),
])
spam_filter = Pipeline([("features", features), ("clf", LogisticRegression(max_iter=1000, C=5))])
spam_filter.fit(X_tr, y_tr)
print(classification_report(y_te, spam_filter.predict(X_te), target_names=["ham", "spam"], digits=3))

new = ["Claim your FREE gift card now at http://claim.example", "Can we review the budget on friday?"]
print(spam_filter.predict_proba(new)[:, 1].round(3))      # high for the first, low for the second
joblib.dump(spam_filter, "spam_filter.joblib")

Synthetic templates make the task easy here; real data is messier, so expect lower scores and more feature work.

Common Pitfalls

  • Random splits when messages are near-duplicates (template spam), which inflates scores; group or time-split.

  • Optimizing accuracy instead of the false-positive rate.

  • Never retraining as spam campaigns change.

Interview Tips

  • Walk through data, features, model, metric, threshold and deployment.

  • Emphasize precision and adversarial drift.

68

How would you design a Python system that predicts house prices?

Design it as a regression pipeline: gather data on past sales with features such as size, rooms, location, age, condition and sale date; clean it and engineer features (price per area by neighbourhood, age at sale, distance to the city centre, log of size); split by time so the model is tested on later sales; train a baseline (median price by area, then linear regression on log price) and stronger models such as gradient boosting; evaluate with MAE or RMSE in currency and MAPE for relative error; then package the fitted pipeline behind an API or batch job, monitor errors as the market changes, and retrain regularly. Predicting the log of price usually works better because prices are right-skewed and errors scale with price.

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x

System Components

Component Choice
Data Sales records, property attributes, location data
Target log(price), converted back for reporting
Split Time-based (train on earlier sales)
Features Area, rooms, age, location encodings, neighbourhood aggregates (from training data only)
Models Baselines, ridge, gradient boosting (HistGradientBoosting, LightGBM)
Metrics MAE, RMSE, MAPE; error by price band and region
Serving FastAPI or batch scoring; saved pipeline
Monitoring Error on new sales, feature drift, retraining schedule

Code Example

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer, TransformedTargetRegressor
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import Ridge
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_error
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

rng = np.random.default_rng(42)
n = 3000
df = pd.DataFrame({
    "area_m2": rng.uniform(40, 250, n),
    "rooms": rng.integers(1, 6, n),
    "age_years": rng.uniform(0, 60, n),
    "neighbourhood": rng.choice(["central", "north", "south", "suburb"], n, p=[0.2, 0.3, 0.3, 0.2]),
    "sale_month": rng.integers(0, 36, n),                 # three years of sales
})
base = {"central": 9000, "north": 6000, "south": 5000, "suburb": 3500}
df["price"] = (df["neighbourhood"].map(base) * df["area_m2"] * (1 - 0.004 * df["age_years"])
               * (1 + 0.005 * df["sale_month"]) * np.exp(rng.normal(0, 0.1, n)))

train, test = df[df["sale_month"] < 30], df[df["sale_month"] >= 30]   # time-based split
features = ["area_m2", "rooms", "age_years", "neighbourhood", "sale_month"]

prep = ColumnTransformer([
    ("num", StandardScaler(), ["area_m2", "rooms", "age_years", "sale_month"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["neighbourhood"]),
])
models = {
    "ridge on log price": TransformedTargetRegressor(make_pipeline(prep, Ridge()), func=np.log, inverse_func=np.exp),
    "boosting on log price": TransformedTargetRegressor(
        make_pipeline(prep, HistGradientBoostingRegressor(random_state=42)), func=np.log, inverse_func=np.exp),
}
median_by_area = train.groupby("neighbourhood")["price"].median()
baseline_pred = test["neighbourhood"].map(median_by_area)
print(f"baseline (median by neighbourhood): MAE {mean_absolute_error(test['price'], baseline_pred):,.0f}")
for name, model in models.items():
    model.fit(train[features], train["price"])
    pred = model.predict(test[features])
    print(f"{name:22s}: MAE {mean_absolute_error(test['price'], pred):,.0f}, "
          f"MAPE {mean_absolute_percentage_error(test['price'], pred):.1%}")

Common Pitfalls

  • Random splits that let the model see future market prices.

  • Neighbourhood averages computed with test sales (target leakage).

  • Reporting only RMSE, which is dominated by expensive houses.

  • Extrapolating to property types missing from training data.

Interview Tips

  • Cover data, target transform, time split, baselines, models, metrics and monitoring.

  • Mention log price and leakage in location features.

69

How would you build a sentiment analysis model in Python?

Build sentiment analysis as text classification: get labelled text (reviews or posts tagged positive, negative and possibly neutral), split it, and start with a fast baseline of TF-IDF features and logistic regression, which handles many cases well; then, for higher accuracy on nuanced language (negation, sarcasm, context), fine-tune a pre-trained transformer such as DistilBERT or RoBERTa with Hugging Face, or use an existing sentiment model if the domain matches. Evaluate with macro F1 and a confusion matrix, inspect errors (negations like “not bad”, domain-specific words), and deploy the chosen pipeline behind an API with monitoring.

Read full answer

Applies toscikit-learn 1.5+, Hugging Face Transformers 4.40+

Options

Approach Accuracy Speed Data needed
Lexicon (VADER) Moderate Very fast None
TF-IDF + logistic regression Good Fast Thousands of labels
Fine-tuned transformer Best Slower, GPU helpful Hundreds to thousands
Zero-shot / LLM prompting Good Slow per item None

Code Example: Baseline

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline

rng = np.random.default_rng(42)
pos = ["great", "excellent", "loved", "fantastic", "not bad", "works perfectly", "highly recommend"]
neg = ["terrible", "awful", "hated", "broke", "not good", "waste of money", "never again"]
items = ["phone", "laptop", "headphones", "service", "delivery"]

def review(words):
    return f"The {items[rng.integers(len(items))]} was {words[rng.integers(len(words))]}, " \
           f"{words[rng.integers(len(words))]} overall."

texts = [review(pos) for _ in range(500)] + [review(neg) for _ in range(500)]
labels = np.r_[np.ones(500, int), np.zeros(500, int)]
X_tr, X_te, y_tr, y_te = train_test_split(texts, labels, test_size=0.2, stratify=labels, random_state=42)

model = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)), LogisticRegression(max_iter=1000))  # bigrams catch "not bad"
model.fit(X_tr, y_tr)
print(classification_report(y_te, model.predict(X_te), target_names=["negative", "positive"], digits=3))
print(model.predict(["The delivery was not good", "The laptop was not bad at all"]))   # expected [0 1]

vocab = model[0].get_feature_names_out()
coefs = model[-1].coef_[0]
print("most positive terms:", vocab[np.argsort(-coefs)[:5]].tolist())

Transformer Option

from transformers import pipeline

classifier = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
print(classifier(["The battery life is fantastic", "Support never answered my emails"]))
# [{'label': 'POSITIVE', ...}, {'label': 'NEGATIVE', ...}]; fine-tune on your own labels for domain text

Common Pitfalls

  • Unigram-only features that miss negation.

  • A pre-trained model from another domain (movie reviews) applied to, say, finance text without checking.

  • Ignoring neutral or mixed sentiment.

Interview Tips

  • Present a baseline first, then a transformer for accuracy.

  • Mention error analysis for negation and domain shift.

70

How would you build and deploy a model that predicts customer churn?

Start by defining churn precisely (for example, no purchase or cancelled subscription within the next 30 days) and fixing the prediction point, then build features from data available before that point (tenure, usage trends, support tickets, payment failures, plan changes), split by time, train a baseline and a gradient-boosting model, and evaluate with PR-AUC, recall at the retention team’s capacity, and calibration, since the scores drive who gets contacted. Deploy the saved pipeline as a scheduled batch job (or an API) that scores customers and pushes the riskiest to a CRM, then monitor score drift and actual churn, and measure business value with a holdout or A/B test of the retention offers.

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x, FastAPI 0.110+

Plan

Step Detail
Define label Churn in the next N days after a snapshot date
Features Only data before the snapshot: usage trends, tenure, tickets, billing
Split Train on older snapshots, test on the latest
Models Logistic regression baseline, gradient boosting
Metrics PR-AUC, precision/recall at top-k, calibration
Explain SHAP or permutation importance for retention teams
Deploy Weekly batch scoring job or API; CRM integration
Monitor Drift, realized churn, campaign uplift

Code Example: Training and Scoring

import joblib
import numpy as np
import pandas as pd
from sklearn.calibration import CalibratedClassifierCV
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder

rng = np.random.default_rng(42)
n = 6000
df = pd.DataFrame({
    "snapshot": rng.integers(0, 12, n),                  # month of the snapshot
    "tenure_months": rng.integers(1, 72, n),
    "logins_last_30d": rng.poisson(12, n),
    "logins_trend": rng.normal(0, 1, n),                 # change vs previous month
    "support_tickets": rng.poisson(1, n),
    "plan": rng.choice(["basic", "plus", "pro"], n),
})
risk = (-2 - 0.03 * df["tenure_months"] - 0.08 * df["logins_last_30d"] - 0.8 * df["logins_trend"]
        + 0.5 * df["support_tickets"] + (df["plan"] == "basic") * 0.7)
df["churned_next_30d"] = (rng.random(n) < 1 / (1 + np.exp(-(risk + 2)))).astype(int)

features = ["tenure_months", "logins_last_30d", "logins_trend", "support_tickets", "plan"]
train, test = df[df["snapshot"] < 10], df[df["snapshot"] >= 10]

prep = ColumnTransformer([("plan", OneHotEncoder(handle_unknown="ignore"), ["plan"])], remainder="passthrough")
model = CalibratedClassifierCV(make_pipeline(prep, HistGradientBoostingClassifier(random_state=42)), cv=3)
model.fit(train[features], train["churned_next_30d"])

proba = model.predict_proba(test[features])[:, 1]
print(f"base churn rate {test['churned_next_30d'].mean():.3f}, PR-AUC {average_precision_score(test['churned_next_30d'], proba):.3f}")
top = test.assign(score=proba).nlargest(int(0.1 * len(test)), "score")      # retention team can call 10%
print(f"churn rate in the top 10% scored: {top['churned_next_30d'].mean():.3f}")   # well above the base rate

joblib.dump(model, "churn_model.joblib")

Serving Sketch

A weekly job loads churn_model.joblib, builds features for every active customer from the warehouse, writes customer_id, score, top_reasons to a table the CRM reads, and logs the score distribution for drift monitoring. For real-time use, the same model sits behind a FastAPI endpoint.

Common Pitfalls

  • Leakage: features computed after the snapshot (such as “cancellation reason”).

  • Random splits across snapshot dates.

  • Targeting customers who would have stayed anyway; measure uplift with a control group.

  • Uncalibrated scores used as probabilities in cost calculations.

Interview Tips

  • Start with the churn definition and prediction point.

  • Cover time splits, business-aligned metrics, deployment and impact measurement.

71

How would you develop an image classification system in Python?

An image classification system in Python goes from data to deployment in clear stages: collect and label images into class folders, split them into train/validation/test sets (by source or patient when images are related), build an input pipeline with resizing, normalization and augmentation, start from a pre-trained convolutional network or vision transformer (torchvision, timm or Keras Applications) and fine-tune it, evaluate with per-class precision, recall and a confusion matrix, and export the model (TorchScript, torch.export or ONNX) behind an API, then monitor its predictions. Transfer learning is the default because it reaches high accuracy with a few hundred to a few thousand images per class.

Read full answer

Applies toPyTorch 2.x, torchvision 0.17+

Pipeline

Stage Tools / choices
Data ImageFolder layout: data/train/<class>/*.jpg
Split Stratified; grouped if many photos per object
Preprocess Resize, normalize with pre-trained stats (weights.transforms())
Augment Random crop, flip, color jitter (training only)
Model Pre-trained ResNet / EfficientNet / ViT, new head
Train AdamW, cosine schedule, mixed precision, early stopping
Evaluate Accuracy, macro F1, confusion matrix, error gallery
Deploy ONNX / torch.export, FastAPI or TorchServe
Monitor Confidence distribution, drift, human review of low-confidence cases

Code Example

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset
from torchvision.models import ResNet18_Weights, resnet18
from torchvision.transforms import v2

torch.manual_seed(42)
device = "cuda" if torch.cuda.is_available() else "cpu"
num_classes = 4

weights = ResNet18_Weights.DEFAULT
model = resnet18(weights=weights)
model.fc = nn.Linear(model.fc.in_features, num_classes)
model = model.to(device)

train_tf = v2.Compose([v2.RandomResizedCrop(224, scale=(0.7, 1.0), antialias=True), v2.RandomHorizontalFlip(),
                       v2.ToDtype(torch.float32, scale=True),
                       v2.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

# Real data: datasets.ImageFolder("data/train", transform=train_tf). Here: random stand-in images.
images = torch.randint(0, 256, (64, 3, 256, 256), dtype=torch.uint8)
labels = torch.randint(0, num_classes, (64,))
loader = DataLoader(TensorDataset(images, labels), batch_size=16, shuffle=True)

optimizer = torch.optim.AdamW([
    {"params": [p for n, p in model.named_parameters() if not n.startswith("fc")], "lr": 1e-4},
    {"params": model.fc.parameters(), "lr": 1e-3},
], weight_decay=0.01)
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)

model.train()
for xb, yb in loader:
    xb, yb = train_tf(xb).to(device), yb.to(device)       # v2 transforms work on batched tensors
    optimizer.zero_grad()
    loss = loss_fn(model(xb), yb)
    loss.backward()
    optimizer.step()
print("last batch loss:", round(loss.item(), 3))

model.eval()
example = torch.randn(1, 3, 224, 224, device=device)
with torch.no_grad():
    probs = torch.softmax(model(example), dim=1)
print("predicted class:", int(probs.argmax()), "confidence:", round(float(probs.max()), 3))
torch.onnx.export(model.cpu(), (example.cpu(),), "classifier.onnx", dynamo=True)   # for serving

Common Pitfalls

  • Near-duplicate photos across train and test, inflating accuracy.

  • Mismatched preprocessing between training and serving.

  • Accuracy only, hiding poor performance on rare classes.

  • Training from scratch with little data.

Interview Tips

  • Walk through data, transfer learning, evaluation and deployment.

  • Mention leakage from duplicates and per-class metrics.

72

How would you detect fraudulent transactions with machine learning in Python?

Fraud detection is an extremely imbalanced classification problem (often well under 1% fraud) where data arrives as a stream and fraudsters adapt, so a solid approach combines feature engineering on transaction behaviour (amount relative to the customer’s usual spend, transaction velocity in the last hour, new device or merchant, distance from the last location, time of day), a time-based split, a gradient-boosting classifier with class weighting, and evaluation with PR-AUC and recall at a fixed alert budget or false-positive rate, plus an unsupervised anomaly detector such as Isolation Forest to catch new patterns. In production, scores feed rules and human review queues in real time, confirmed cases feed back as labels, and the model is retrained often.

Read full answer

Applies toscikit-learn 1.5+, pandas 2.x

Approach

Component Choice
Labels Confirmed fraud (chargebacks, investigations), which arrive with delay
Features Per-customer aggregates over time windows, velocity, novelty flags
Split By time; never random
Supervised model Gradient boosting with class weights / scale_pos_weight
Unsupervised Isolation Forest, autoencoders for unknown patterns
Metrics PR-AUC, recall at k alerts per day, cost of fraud missed vs review cost
Serving Low-latency API, rules + model, analyst queue
Feedback Analyst decisions become new labels

Code Example

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier, IsolationForest
from sklearn.metrics import average_precision_score

rng = np.random.default_rng(42)
n = 50_000
df = pd.DataFrame({
    "customer": rng.integers(0, 2000, n),
    "time": np.sort(rng.uniform(0, 90, n)),            # days
    "amount": rng.lognormal(3.5, 1.0, n),
    "new_device": rng.random(n) < 0.05,
    "night": rng.random(n) < 0.15,
})
fraud_logit = -7 + 1.5 * np.log(df["amount"] / 30).clip(0) + 2.0 * df["new_device"] + 1.0 * df["night"]
df["fraud"] = (rng.random(n) < 1 / (1 + np.exp(-fraud_logit))).astype(int)

df = df.sort_values(["customer", "time"])               # behavioural features from past transactions only
g = df.groupby("customer")
df["cust_mean_amount"] = g["amount"].transform(lambda s: s.shift().expanding().mean()).fillna(df["amount"].median())
df["amount_ratio"] = df["amount"] / df["cust_mean_amount"]
df["txn_count_before"] = g.cumcount()
df = df.sort_values("time")

features = ["amount", "amount_ratio", "txn_count_before", "new_device", "night"]
train, test = df[df["time"] < 70], df[df["time"] >= 70]   # train on the past, test on the future
print(f"fraud rate: {df['fraud'].mean():.3%}")

clf = HistGradientBoostingClassifier(class_weight="balanced", random_state=42).fit(train[features], train["fraud"])
proba = clf.predict_proba(test[features])[:, 1]
print(f"PR-AUC (supervised): {average_precision_score(test['fraud'], proba):.3f}")

alerts_per_day = 50
k = int(alerts_per_day * (test["time"].max() - 70))
top_k = np.argsort(-proba)[:k]
caught = test["fraud"].to_numpy()[top_k].sum()
print(f"recall with {alerts_per_day} alerts/day: {caught / test['fraud'].sum():.2%}")

iso = IsolationForest(contamination=0.01, random_state=42).fit(train[features])
anomaly = -iso.score_samples(test[features])            # higher = more unusual
print(f"PR-AUC (Isolation Forest, no labels): {average_precision_score(test['fraud'], anomaly):.3f}")

Common Pitfalls

  • Random splits, which leak future customer behaviour into training.

  • Aggregates computed with the current or future transactions.

  • Measuring accuracy (99% by predicting “not fraud”).

  • Ignoring label delay: recent transactions may not be labelled yet.

Interview Tips

  • Stress imbalance, time-aware validation and behavioural features.

  • Tie the metric to alert capacity and fraud cost.

73

How do local, nonlocal and global scopes affect variable access in Python?

Python looks names up with the LEGB rule: first the local scope of the current function, then enclosing functions’ scopes, then the module’s global scope, then built-ins; assigning to a name inside a function makes it local unless you declare it global (to rebind a module-level name) or nonlocal (to rebind a variable of an enclosing function). In ML code this matters for configuration and state: functions that silently read global variables (a global model, df or learning rate) are hard to test and break when notebook cells run in a different order, closures with nonlocal are handy for counters and callbacks such as tracking the best validation score, and passing values as arguments or keeping state in objects is the clearest option.

Read full answer

Applies toPython 3.12

Scopes

Scope Where Rebind with
Local Inside the current function Plain assignment
Enclosing Outer function of a nested function nonlocal name
Global Module level global name
Built-in len, print, sum… Don’t shadow them

Code Example

LEARNING_RATE = 0.1                     # module-level (global) constant
step_count = 0

def train_step(weight, grad):
    global step_count                   # rebinding a global needs a declaration
    step_count += 1
    return weight - LEARNING_RATE * grad    # reading a global works without one

def broken_counter():
    try:
        step_count += 1  # noqa: F823, F841  (assignment makes it local, so reading it first fails)
    except UnboundLocalError as err:
        print("UnboundLocalError:", err)

def make_early_stopper(patience=2):
    best = float("inf")
    bad_epochs = 0

    def should_stop(val_loss):
        nonlocal best, bad_epochs       # rebind variables of the enclosing function
        if val_loss < best:
            best, bad_epochs = val_loss, 0
        else:
            bad_epochs += 1
        return bad_epochs >= patience

    return should_stop

w = 1.0
for g in [0.5, 0.4, 0.3]:
    w = train_step(w, g)
print(round(w, 3), step_count)          # 0.88 3
broken_counter()

stop = make_early_stopper(patience=2)
for epoch, loss in enumerate([0.9, 0.7, 0.72, 0.71, 0.69], start=1):
    if stop(loss):
        print("stop at epoch", epoch)   # stop at epoch 4
        break

def evaluate(model_predict, X, y):      # better: pass dependencies explicitly
    return sum(model_predict(x) == t for x, t in zip(X, y)) / len(y)

print(evaluate(lambda x: x > 0, [-1, 2, 3], [False, True, False]))   # 0.666...

Common Pitfalls

  • UnboundLocalError from assigning to a name that also exists globally.

  • Functions that depend on notebook globals, which give different results after cells are re-run.

  • Mutable globals shared by threads or processes.

  • Shadowing built-ins (list = [...], sum = 0).

Interview Tips

  • Explain LEGB and when global and nonlocal are needed.

  • Recommend explicit arguments or classes for ML state.

74

How does the GIL affect Python concurrency in machine learning applications?

CPython’s Global Interpreter Lock (GIL) allows only one thread at a time to execute Python bytecode in a process, so threads cannot speed up CPU-bound pure-Python code such as feature loops or custom simulations; they still help for I/O-bound work (downloading data, calling APIs, reading files) because the GIL is released while waiting. The GIL matters less for most ML workloads than it seems: NumPy, pandas, scikit-learn, PyTorch and TensorFlow do their heavy computation in C, C++ or CUDA code that releases the GIL, and multi-core parallelism is available through processes (multiprocessing, joblib), library n_jobs settings, Numba’s nogil mode and GPUs. Python 3.13 added an experimental free-threaded build without the GIL, which libraries are gradually supporting.

Read full answer

Applies toCPython 3.12 (free-threaded builds from 3.13 are experimental)

Impact by Workload

Workload Threads help? Better option
Pure-Python CPU loops No Processes, vectorization, Numba
NumPy / BLAS math Yes (GIL released; BLAS is already multi-threaded) Usually nothing needed
PyTorch / TensorFlow training N/A (C++/CUDA) DataLoader(num_workers=...) uses processes
I/O: downloads, DB queries, APIs Yes Threads or asyncio
Model serving Partly Multiple worker processes (Gunicorn/Uvicorn workers)
scikit-learn CV / forests — n_jobs=-1 (processes or GIL-free threads internally)

Code Example

import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

import numpy as np

def python_cpu_task(n=3_000_000):
    total = 0
    for i in range(n):                  # pure Python: holds the GIL
        total += i * i
    return total

def numpy_task(seed):
    rng = np.random.default_rng(seed)
    a = rng.random((600, 600))
    return float(np.linalg.eigvalsh(a @ a.T).max())   # C/LAPACK code releases the GIL

def io_task(seconds=0.5):
    time.sleep(seconds)                 # stands in for network or disk waits; releases the GIL
    return seconds

def timed(label, fn):
    start = time.perf_counter()
    fn()
    print(f"{label:34s} {time.perf_counter() - start:5.2f}s")

if __name__ == "__main__":
    timed("CPU task x4, serial", lambda: [python_cpu_task() for _ in range(4)])
    with ThreadPoolExecutor(4) as pool:
        timed("CPU task x4, threads (GIL-bound)", lambda: list(pool.map(lambda _: python_cpu_task(), range(4))))
    with ProcessPoolExecutor(4) as pool:
        timed("CPU task x4, processes", lambda: list(pool.map(python_cpu_task, [3_000_000] * 4)))
    with ThreadPoolExecutor(8) as pool:
        timed("I/O task x8, threads", lambda: list(pool.map(io_task, [0.5] * 8)))   # about 0.5s, not 4s
    with ThreadPoolExecutor(4) as pool:
        timed("NumPy task x4, threads", lambda: list(pool.map(numpy_task, range(4))))

Practical Guidance

  • Vectorize first; it removes most Python-level loops.

  • Use processes for CPU-bound Python work; beware of copying large data between them.

  • Use threads or asyncio for I/O such as downloading datasets or calling model APIs.

  • In serving, scale with multiple worker processes rather than threads.

Common Pitfalls

  • Expecting threads to speed up pandas apply with Python functions.

  • Spawning many processes that each load a large model, exhausting memory.

  • Oversubscription: processes × BLAS threads exceeding the number of cores.

Interview Tips

  • Define the GIL and separate CPU-bound from I/O-bound work.

  • Explain why NumPy and deep learning frameworks are mostly unaffected.

75

How do you use context managers in Python? Give an ML example.

A context manager sets something up when a with block starts and reliably cleans it up when the block ends, even if an exception occurs; it is defined by __enter__ and __exit__ methods or, more simply, with the @contextlib.contextmanager decorator around a generator that yields once. Familiar examples are with open(...) for files, with torch.no_grad(): for inference, and with mlflow.start_run(): for experiment tracking; in your own ML code they are useful for timing training stages, temporarily changing settings (random seeds, NumPy error handling, model eval mode), managing database connections, and making sure resources such as GPU memory or temporary files are released.

Read full answer

Applies toPython 3.12

Code Example

import os
import tempfile
import time
from contextlib import contextmanager

import numpy as np

class Timer:
    """Class-based context manager that records elapsed time."""

    def __init__(self, label):
        self.label = label

    def __enter__(self):
        self.start = time.perf_counter()
        return self

    def __exit__(self, exc_type, exc, tb):
        self.elapsed = time.perf_counter() - self.start
        print(f"{self.label}: {self.elapsed:.3f}s")
        return False                    # don't swallow exceptions

@contextmanager
def temporary_seed(seed):
    """Run a block with a fixed NumPy global seed, then restore the previous state."""
    state = np.random.get_state()
    np.random.seed(seed)
    try:
        yield
    finally:
        np.random.set_state(state)

@contextmanager
def checkpoint_dir():
    """A scratch directory for checkpoints that is always removed afterwards."""
    with tempfile.TemporaryDirectory() as path:
        yield path

with Timer("matrix training stand-in"):
    X = np.random.default_rng(42).normal(size=(1500, 1500))
    _ = X @ X.T

with temporary_seed(42):
    a = np.random.rand(3)
with temporary_seed(42):
    b = np.random.rand(3)
print(np.array_equal(a, b))             # True: the same seed inside both blocks

with checkpoint_dir() as d:
    path = os.path.join(d, "model.npy")
    np.save(path, np.arange(5))
    print(os.path.exists(path))         # True
print(os.path.exists(path))             # False: cleaned up automatically

with np.errstate(divide="ignore", invalid="ignore"):   # built-in NumPy context manager
    ratio = np.array([1.0, 0.0]) / np.array([0.0, 0.0])
print(ratio)                            # [inf nan] without warnings

Common ML Context Managers

Context manager Purpose
open(...) Files closed automatically
torch.no_grad() / torch.inference_mode() No gradient tracking
torch.autocast(...) Mixed precision
mlflow.start_run() Experiment run lifecycle
np.errstate(...) Temporary floating-point error settings
tempfile.TemporaryDirectory() Scratch space
threadpoolctl.threadpool_limits(...) Limit BLAS threads temporarily

Common Pitfalls

  • Forgetting try/finally in a generator-based manager, so cleanup is skipped on errors.

  • Returning True from __exit__, which silently swallows exceptions.

  • Using a value outside the with block after its resource is closed.

Interview Tips

  • Explain __enter__/__exit__ and @contextmanager.

  • Give ML examples such as torch.no_grad() and a timing or seed manager.

76

What are __slots__ in Python classes and how can they help in ML code?

Defining __slots__ in a class tells Python to reserve fixed attribute slots instead of creating a per-instance __dict__, which cuts memory per object substantially and makes attribute access slightly faster, while preventing new attributes from being added by mistake (a typo like self.lable raises an error instead of silently creating a new attribute). In ML applications this helps when you create millions of small Python objects, such as records for samples, tokens, graph nodes, tree nodes in a from-scratch model, or events in a streaming pipeline; for large numeric data, NumPy arrays or DataFrames are still far more efficient than any per-object design. Dataclasses support it with @dataclass(slots=True).

Read full answer

Applies toPython 3.12

Code Example

import sys
import tracemalloc
from dataclasses import dataclass

class SampleDict:
    def __init__(self, idx, label, weight):
        self.idx, self.label, self.weight = idx, label, weight

class SampleSlots:
    __slots__ = ("idx", "label", "weight")

    def __init__(self, idx, label, weight):
        self.idx, self.label, self.weight = idx, label, weight

@dataclass(slots=True)
class Token:
    text: str
    position: int
    embedding_id: int

def memory_for(cls, n=200_000):
    tracemalloc.start()
    objs = [cls(i, i % 2, 1.0) for i in range(n)]
    current, _ = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return current / 1e6, objs

dict_mb, _ = memory_for(SampleDict)
slots_mb, _ = memory_for(SampleSlots)
print(f"200k objects: with __dict__ {dict_mb:.1f} MB, with __slots__ {slots_mb:.1f} MB")
# __slots__ typically uses noticeably less memory

s = SampleSlots(1, 0, 1.0)
try:
    s.lable = 1                         # typo
except AttributeError as err:
    print("AttributeError:", err)

t = Token("hello", 0, 42)
print(t, hasattr(t, "__dict__"))        # Token(text='hello', position=0, embedding_id=42) False
print(sys.getsizeof(SampleDict(1, 0, 1.0).__dict__) > 0)   # the per-instance dict that slots avoid

Trade-offs

Benefit Cost
Less memory per instance No dynamic attributes
Faster attribute access Inheritance needs care (each subclass defines its own __slots__)
Catches attribute typos Some libraries that rely on __dict__ (certain pickling or monkey-patching) need adjustments

When to Use

  • Millions of small, fixed-shape objects.

  • Long-running services where memory matters.

  • Not for a handful of objects, and not as a substitute for arrays of numbers.

Common Pitfalls

  • Expecting large savings for few objects.

  • Forgetting "__weakref__" in __slots__ if you need weak references.

  • Using lots of objects when a NumPy array or DataFrame would be better.

Interview Tips

  • Explain the missing __dict__ and the memory saving.

  • Mention @dataclass(slots=True) and when arrays are better.

77

How does the collections module help manage data structures for ML?

The collections module adds specialized containers that make common data-handling tasks shorter and faster: Counter counts items (class balance, word frequencies, vocabulary building), defaultdict groups values without key checks (samples by label, features by user), deque keeps a fixed-size sliding window with O(1) appends and pops at both ends (moving averages of loss, replay buffers in reinforcement learning, streaming features), namedtuple creates lightweight, readable records, OrderedDict offers order-aware operations such as move_to_end for LRU caches, and ChainMap layers configuration dictionaries (defaults, file, command-line overrides).

Read full answer

Applies toPython 3.12

Containers

Container Use in ML code
Counter Class counts, token frequencies, top-k
defaultdict(list) Group samples or predictions by key
deque(maxlen=n) Rolling windows, replay buffers
namedtuple Immutable records (experiment results)
OrderedDict LRU caches (move_to_end, popitem(last=False))
ChainMap Layered configs

Code Example

from collections import ChainMap, Counter, OrderedDict, defaultdict, deque, namedtuple

import numpy as np

labels = ["cat", "dog", "cat", "bird", "cat", "dog"]
counts = Counter(labels)
print(counts.most_common(2), round(counts["cat"] / len(labels), 3))   # [('cat', 3), ('dog', 2)] 0.5

text = "the model learns the data and the model improves"
vocab = {w: i for i, (w, _) in enumerate(Counter(text.split()).most_common())}
print(list(vocab)[:3])                                  # ['the', 'model', 'learns']

by_label = defaultdict(list)
for idx, label in enumerate(labels):
    by_label[label].append(idx)
print(dict(by_label))                                   # {'cat': [0, 2, 4], 'dog': [1, 5], 'bird': [3]}

window = deque(maxlen=3)                                # moving average of the last 3 losses
for loss in [1.0, 0.8, 0.6, 0.5, 0.45]:
    window.append(loss)
print(round(sum(window) / len(window), 3))              # (0.6 + 0.5 + 0.45) / 3 = 0.517

rng = np.random.default_rng(42)
replay = deque(maxlen=1000)                             # RL replay buffer: oldest transitions drop off
for t in range(1500):
    replay.append((t, int(rng.integers(2)), float(rng.random())))
print(len(replay), replay[0][0])                        # 1000 500

Result = namedtuple("Result", ["model", "accuracy", "seconds"])
results = [Result("logreg", 0.91, 1.2), Result("forest", 0.94, 8.5)]
best = max(results, key=lambda r: r.accuracy)
print(best.model, best.accuracy)                        # forest 0.94

defaults = {"lr": 1e-3, "epochs": 10, "batch_size": 32}
file_cfg = {"epochs": 20}
cli = {"lr": 3e-4}
config = ChainMap(cli, file_cfg, defaults)              # first mapping wins
print(config["lr"], config["epochs"], config["batch_size"])   # 0.0003 20 32

cache = OrderedDict()
for key in ["a", "b", "c", "a", "d"]:                   # tiny LRU cache with capacity 3
    cache[key] = True
    cache.move_to_end(key)
    if len(cache) > 3:
        cache.popitem(last=False)
print(list(cache))                                      # ['c', 'a', 'd']

Common Pitfalls

  • defaultdict silently creating keys when you only meant to read (d[k] on a missing key inserts it).

  • Using lists as queues (list.pop(0) is O(n)); use deque.

  • Counting with manual dictionaries instead of Counter.

Interview Tips

  • Name each container with an ML use.

  • Show deque(maxlen=...) for windows and replay buffers.

78

What is a microservices architecture for deploying ML models?

In a microservices architecture, an application is split into small, independently deployable services that talk over the network (HTTP/REST, gRPC or message queues); for machine learning, the model runs as its own inference service with a clear API, separate from services such as feature computation, user management or the web front end. This lets the model team deploy, scale and roll back models without redeploying everything, run the model on different hardware (GPUs), use different language stacks, and test new versions with canary releases or A/B tests; the costs are network latency, more operational complexity (monitoring, service discovery, versioned APIs), and the need for consistent features between training and serving, often solved with a feature store.

Read full answer

Applies togeneral architecture; examples with FastAPI 0.110+ and Docker

Typical Services

Service Responsibility
API gateway Authentication, routing, rate limiting
Feature service / feature store Same features for training and serving
Model inference service Loads model, validates input, predicts
Model registry Versions and metadata (MLflow)
Training pipeline Scheduled retraining jobs (Airflow, Kubeflow)
Monitoring Latency, errors, drift, prediction logs
Message queue Async or batch requests (Kafka, RabbitMQ)

Benefits and Costs

Benefit Cost
Independent deploys and rollbacks Network latency between services
Scale the model service separately (GPU nodes) More infrastructure to operate
Different tech stacks per service Distributed debugging and tracing
Clear ownership API versioning and backward compatibility
Safe experiments (canary, A/B) Data consistency across services

Code Example: A Minimal Inference Service

from fastapi import FastAPI
from pydantic import BaseModel
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression

X, y = load_iris(return_X_y=True)
MODEL = LogisticRegression(max_iter=1000).fit(X, y)     # in production: load from a model registry
MODEL_VERSION = "iris-logreg-1.0.0"

app = FastAPI(title="iris-inference")

class Features(BaseModel):
    sepal_length: float
    sepal_width: float
    petal_length: float
    petal_width: float

@app.get("/health")
def health():
    return {"status": "ok", "model_version": MODEL_VERSION}

@app.post("/v1/predict")
def predict(f: Features):
    row = [[f.sepal_length, f.sepal_width, f.petal_length, f.petal_width]]
    proba = MODEL.predict_proba(row)[0]
    return {"class_id": int(proba.argmax()), "probability": round(float(proba.max()), 4),
            "model_version": MODEL_VERSION}

# Run: uvicorn service:app --port 8080. Package with a Dockerfile; deploy behind a gateway,
# scale replicas on Kubernetes, and route a small share of traffic to a new version first.

When Not to Use Microservices

  • A small team with one model and low traffic: a single application or batch job is simpler.

  • Very low-latency paths where an extra network hop is too slow: embed the model in the calling service.

Common Pitfalls

  • Computing features differently in the training pipeline and the inference service.

  • No API versioning, so model changes break clients.

  • Too many tiny services for a small system.

Interview Tips

  • Describe the inference service and its neighbours (features, registry, monitoring).

  • Weigh the benefits against latency and operational overhead.

79

What should you consider when scaling a Python machine learning application?

Scaling an ML application means handling more data, more training compute and more prediction traffic while keeping latency, cost and reliability under control: for data, move from in-memory pandas to columnar formats (Parquet), chunked processing, Polars, Dask or Spark; for training, use vectorized code, all CPU cores (n_jobs), GPUs, distributed training and efficient hyperparameter search; for serving, run stateless model services with multiple worker processes behind a load balancer, autoscale replicas, batch requests, cache frequent predictions and optimize models (quantization, ONNX Runtime). Around all of this you need monitoring (latency, throughput, errors, drift), reproducible deployments (containers, pinned versions), and cost tracking.

Read full answer

Applies toPython 3.12 ecosystem

Considerations

Area Scaling techniques
Data volume Parquet, chunking, Polars, Dask, Spark, sampling for exploration
Feature computation Precompute in batch, feature store, incremental updates
Training Vectorization, n_jobs, GPUs, distributed (DDP, Ray Train), early stopping
Hyperparameter search Random/Bayesian search, successive halving, parallel trials (Ray Tune, Optuna)
Serving throughput Multiple worker processes, horizontal replicas, autoscaling, dynamic batching
Latency Model optimization, caching, keeping models in memory, async I/O
Python limits GIL (use processes), memory per worker, startup time
Reliability Health checks, timeouts, graceful degradation, rollbacks
Cost Right-size instances, spot instances for training, batch vs real-time choice
Observability Metrics, logs, tracing, data and prediction drift

Code Example: Scaling Data Processing with Chunks

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
pd.DataFrame({
    "user_id": rng.integers(0, 10_000, 1_000_000),
    "amount": rng.lognormal(3, 1, 1_000_000).round(2),
}).to_csv("transactions.csv", index=False)               # stands in for a file too big for memory

totals, counts = pd.Series(dtype=float), pd.Series(dtype=float)
for chunk in pd.read_csv("transactions.csv", chunksize=200_000, dtype={"user_id": "int32", "amount": "float32"}):
    grouped = chunk.groupby("user_id")["amount"]
    totals = totals.add(grouped.sum(), fill_value=0)       # combine partial aggregates
    counts = counts.add(grouped.count(), fill_value=0)
features = pd.DataFrame({"total": totals, "mean": totals / counts})
print(features.shape, features["total"].sum().round(0) > 0)   # (10000, 2) True

features.to_parquet("user_features.parquet")             # columnar and typed for the next stage

For larger data, the same aggregation in Polars (pl.scan_csv(...).group_by("user_id").agg(...)) or Dask runs lazily and in parallel.

Serving at Scale

  • Run several Uvicorn/Gunicorn worker processes per container; load the model once per worker.

  • Scale containers horizontally on Kubernetes or a managed platform with autoscaling on CPU or request rate.

  • Use batch scoring for predictions that don’t need to be real time.

Common Pitfalls

  • Scaling infrastructure before profiling the code.

  • Loading the full dataset into pandas when only a few columns are needed (usecols).

  • Each worker loading a huge model and exhausting memory.

  • No load testing before launch.

Interview Tips

  • Separate data, training and serving scaling.

  • Mention profiling first, then horizontal scaling and monitoring.

80

What are the options for deploying a Python machine learning model?

The main deployment options are batch scoring (a scheduled job that writes predictions to a database or file), a real-time REST or gRPC API (FastAPI, Flask, or model servers like TorchServe, NVIDIA Triton, BentoML, MLflow serving), streaming inference attached to a message queue (Kafka consumers), serverless functions (AWS Lambda, Google Cloud Functions) for spiky, light traffic, managed ML platforms (SageMaker, Vertex AI, Azure ML) that handle hosting and scaling, and edge or in-browser deployment (ONNX Runtime, TensorFlow Lite / LiteRT, Core ML, ONNX Runtime Web) for on-device predictions. The choice depends on latency needs, traffic patterns, model size, team skills and cost, and most options package the model and its dependencies in a Docker container.

Read full answer

Applies toPython 3.12 ecosystem

Options

Option Latency Best for Tools
Batch scoring Minutes to hours Daily churn scores, reports Airflow, cron, Spark
REST/gRPC API Milliseconds User-facing predictions FastAPI, Flask, BentoML
Model server Milliseconds, high throughput GPU models, many models TorchServe, Triton, TF Serving
Streaming Seconds Event-driven fraud, IoT Kafka, Flink, Faust
Serverless Cold starts possible Spiky, small models Lambda, Cloud Run, Cloud Functions
Managed platform Configurable Teams wanting less ops SageMaker, Vertex AI, Azure ML
Edge / mobile / browser Local, offline Privacy, no network ONNX Runtime, LiteRT, Core ML
Embedded in an app In-process Simple apps, notebooks to dashboards Streamlit, Gradio

Code Example: Same Model, Two Deployment Styles

import joblib
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier

X, y = make_classification(n_samples=1000, n_features=5, random_state=42)
joblib.dump(RandomForestClassifier(random_state=42).fit(X, y), "model.joblib")

# 1) Batch scoring job (run nightly by a scheduler)
def batch_score(input_path, output_path, model_path="model.joblib"):
    model = joblib.load(model_path)
    df = pd.read_parquet(input_path)
    df["score"] = model.predict_proba(df[[f"f{i}" for i in range(5)]])[:, 1]
    df[["customer_id", "score"]].to_parquet(output_path, index=False)
    return len(df)

rng = np.random.default_rng(42)
pd.DataFrame(rng.normal(size=(200, 5)), columns=[f"f{i}" for i in range(5)]).assign(
    customer_id=range(200)).to_parquet("customers.parquet")
print("scored rows:", batch_score("customers.parquet", "scores.parquet"))

# 2) Real-time API: the same model behind FastAPI (run with `uvicorn app:app`)
API_CODE = '''
from fastapi import FastAPI
import joblib
app = FastAPI()
model = joblib.load("model.joblib")

@app.post("/predict")
def predict(features: list[float]):
    return {"score": float(model.predict_proba([features])[0, 1])}
'''
print(API_CODE.strip().splitlines()[0])

Choosing

  • Predictions needed instantly by users → API or model server.

  • Predictions needed for many records at once → batch.

  • Data arrives as events → streaming.

  • No connectivity or strict privacy → edge.

Common Pitfalls

  • Building a real-time service when daily batch predictions would do.

  • Not containerizing, so dependencies differ between environments.

  • No monitoring or rollback plan.

Interview Tips

  • List batch, API, streaming, serverless, managed and edge with a use case for each.

  • Explain how latency and traffic drive the choice.

81

How does containerization with Docker benefit ML applications?

Docker packages an ML application with everything it needs (a specific Python version, pinned libraries such as NumPy, scikit-learn or PyTorch, system libraries like CUDA runtimes, the model file and the serving code) into an image that runs the same way on a laptop, a CI server, a cloud VM or a Kubernetes cluster. That removes “works on my machine” problems, makes training and inference environments reproducible, allows rollback by redeploying an older image tag, isolates dependencies between projects, and makes scaling straightforward because orchestrators start more identical containers; GPU workloads use NVIDIA’s container toolkit with CUDA base images.

Read full answer

Applies toDocker 24+, Python 3.12

Benefits

Benefit Why it matters for ML
Reproducibility Same library versions for training and serving
Portability Run anywhere Docker runs: local, cloud, on-prem
Isolation Conflicting dependencies (e.g. two CUDA versions) on one host
Versioning and rollback Image tags per model version
Scaling Kubernetes or ECS start more replicas
CI/CD Build, test and deploy the same artifact
GPU support CUDA base images with --gpus all

Example Dockerfile for a Model API

FROM python:3.12-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt   # pinned: fastapi==..., scikit-learn==..., joblib==...

COPY app/ ./app/
COPY models/churn_model.joblib ./models/

RUN useradd --create-home appuser
USER appuser
EXPOSE 8000
HEALTHCHECK CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]

Build and run: docker build -t churn-api:1.3.0 . then docker run -p 8000:8000 churn-api:1.3.0. For GPUs, start from an image such as pytorch/pytorch or nvidia/cuda and run with docker run --gpus all ....

Code Example: The App Inside the Container

from pathlib import Path

import joblib
from fastapi import FastAPI
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

MODEL_PATH = Path("models/churn_model.joblib")
if not MODEL_PATH.exists():                     # for local testing; the image ships a trained model
    MODEL_PATH.parent.mkdir(parents=True, exist_ok=True)
    X, y = make_classification(n_samples=500, n_features=4, random_state=42)
    joblib.dump(LogisticRegression().fit(X, y), MODEL_PATH)

model = joblib.load(MODEL_PATH)
app = FastAPI()

@app.get("/health")
def health():
    return {"status": "ok"}

@app.post("/predict")
def predict(features: list[float]):
    return {"churn_probability": round(float(model.predict_proba([features])[0, 1]), 4)}

Best Practices

  • Pin dependency versions and use small base images (-slim).

  • Copy requirements.txt before the code so the dependency layer is cached.

  • Don’t bake secrets into images; pass them as environment variables or secrets.

  • Run as a non-root user and add health checks.

  • Store large models in a registry or object storage if they change often.

Common Pitfalls

  • Huge images from including datasets, caches or full CUDA toolkits when not needed.

  • latest tags in production, which make rollbacks unclear.

  • Training and serving images with different library versions.

Interview Tips

  • Lead with reproducibility and portability.

  • Describe a Dockerfile and how images fit CI/CD and orchestration.

82

What is model versioning and how do you manage it in practice?

Model versioning means giving every trained model a unique, traceable identity linked to everything needed to reproduce and audit it: the code commit, training data version, hyperparameters, environment (library versions), evaluation metrics, and the model artifact itself, plus its lifecycle stage (staging, production, archived). It lets you compare versions, roll back quickly when a new model misbehaves, run A/B tests, and answer “which model made this prediction?” In practice teams use a model registry (MLflow Model Registry, Weights & Biases, SageMaker or Vertex AI registries), data versioning (DVC, lakeFS, or dated snapshots), git for code, semantic or incrementing version numbers, and they log the model version with every prediction.

Read full answer

Applies toMLflow 2.x, Python 3.12

What to Version

Item How
Code Git commit hash
Data DVC, lakeFS, snapshot paths with hashes
Parameters Logged config (YAML/JSON)
Environment requirements.txt / lock file, Docker image tag
Artifact Model file in a registry or object storage
Metrics Validation and test results
Stage / alias staging, production, champion / challenger
Lineage Which data and run produced it

Code Example: MLflow Tracking and Registry

import mlflow
import mlflow.sklearn
from mlflow.models import infer_signature
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split

mlflow.set_tracking_uri("sqlite:///mlflow.db")            # local registry for the example
mlflow.set_experiment("cancer-classifier")

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=42)

params = {"n_estimators": 300, "max_depth": 6, "random_state": 42}
with mlflow.start_run() as run:
    model = RandomForestClassifier(**params).fit(X_tr, y_tr)
    auc = roc_auc_score(y_te, model.predict_proba(X_te)[:, 1])
    mlflow.log_params(params)
    mlflow.log_metric("test_auc", auc)
    mlflow.set_tag("data_version", "breast_cancer_sklearn_v1")
    mlflow.sklearn.log_model(model, artifact_path="model", signature=infer_signature(X_tr, model.predict(X_tr)),
                             registered_model_name="cancer-classifier")

client = mlflow.MlflowClient()
latest = max(client.search_model_versions("name='cancer-classifier'"), key=lambda v: int(v.version))
client.set_registered_model_alias("cancer-classifier", "champion", latest.version)
print(f"registered version {latest.version} as champion (AUC {auc:.3f})")

loaded = mlflow.sklearn.load_model("models:/cancer-classifier@champion")   # serving code loads by alias
print(loaded.predict(X_te.head(3)))

Rolling back is then a matter of pointing the champion alias at the previous version.

Practices

  • Never overwrite a model file in place; every retrain creates a new version.

  • Log the model version in prediction logs and API responses.

  • Promote through stages with checks (tests, metric thresholds, approval).

  • Keep older versions available for rollback and audits.

Common Pitfalls

  • Versioning the model but not the data, so results can’t be reproduced.

  • Manual file names like model_final_v2_new.pkl.

  • No record of which version served which predictions.

Interview Tips

  • List what must be versioned and why (reproducibility, rollback, audit).

  • Name a registry and the promotion/rollback flow.

83

How would you investigate a machine learning model failure using Python?

Take a concrete scenario: a churn model that scored well offline suddenly flags far fewer customers, and actual churn rises. Investigate systematically: confirm the symptom with monitoring data, check the inputs first (schema changes, missing values, new categories, unit changes, upstream pipeline bugs), compare feature distributions between training data and recent production data for drift, compare the prediction distribution over time, evaluate the model on recently labelled data by segment, verify that the serving pipeline transforms features exactly as training did, and only then consider retraining. Python tools for this are pandas for slicing data, SciPy statistics or Evidently for drift, the model’s own predictions for slice analysis, and SHAP or permutation importance to see which features changed the model’s behaviour.

Read full answer

Applies topandas 2.x, scikit-learn 1.5+, SciPy 1.13+

Common Failure Causes

Cause Example How to detect
Data pipeline bug A column becomes all null after an upstream change Missing-value rates, schema checks
Unit or encoding change Amounts sent in cents instead of rupees Distribution shift in one feature
New categories A new plan type unseen in training Unknown-category counts
Concept drift Customer behaviour changes after a competitor launches Performance drops on fresh labels
Training-serving skew Different preprocessing code in production Compare features for the same records
Feedback loops Model decisions change future data Policy and outcome analysis

Code Example: Diagnosing Drift and a Broken Feature

import numpy as np
import pandas as pd
from scipy.stats import ks_2samp
from sklearn.ensemble import HistGradientBoostingClassifier

rng = np.random.default_rng(42)

def make(n, amount_scale=1.0, null_rate=0.0):
    df = pd.DataFrame({"tenure": rng.integers(1, 60, n), "monthly_amount": rng.normal(700, 150, n) * amount_scale,
                       "tickets": rng.poisson(1, n)})
    df.loc[rng.random(n) < null_rate, "tenure"] = np.nan
    logit = -1 - 0.05 * df["tenure"].fillna(30) + 0.004 * (df["monthly_amount"] / amount_scale - 700) + 0.6 * df["tickets"]
    df["churn"] = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    return df

train = make(5000)
prod = make(2000, amount_scale=100, null_rate=0.3)         # amounts now in cents; tenure often missing
features = ["tenure", "monthly_amount", "tickets"]
model = HistGradientBoostingClassifier(random_state=42).fit(train[features], train["churn"])

report = []
for col in features:
    stat, p = ks_2samp(train[col].dropna(), prod[col].dropna())
    report.append({"feature": col, "train_mean": train[col].mean(), "prod_mean": prod[col].mean(),
                   "prod_null_rate": prod[col].isna().mean(), "ks_stat": stat})
print(pd.DataFrame(report).round(3).to_string(index=False))
# monthly_amount: mean jumps about 100x (a unit change); tenure: 30% nulls (a pipeline problem)

print(f"mean predicted churn: train {model.predict_proba(train[features])[:, 1].mean():.3f}, "
      f"prod {model.predict_proba(prod[features])[:, 1].mean():.3f}")
print(f"actual churn: train {train['churn'].mean():.3f}, prod {prod['churn'].mean():.3f}")

fixed = prod.assign(monthly_amount=prod["monthly_amount"] / 100)            # after fixing the unit bug
print(f"mean predicted churn after the fix: {model.predict_proba(fixed[features])[:, 1].mean():.3f}")

Investigation Checklist

  1. Reproduce: same model version, same inputs, same outputs?

  2. Inputs: schema, nulls, ranges, categories, units.

  3. Drift: KS tests or PSI per feature; prediction distribution.

  4. Performance by segment on recent labels.

  5. Pipeline parity: recompute features offline for a sample of production requests.

  6. Fix the root cause, add a data validation check (Great Expectations, Pandera), then retrain if the world changed.

Common Pitfalls

  • Retraining immediately, which can bake a data bug into the next model.

  • Looking only at aggregate metrics and missing a broken segment.

  • No input validation in the serving path.

Interview Tips

  • Use a concrete scenario and a step-by-step investigation.

  • Check data and pipeline issues before blaming the model.

84

What are good strategies for logging and monitoring ML applications?

Effective ML monitoring covers four layers: system health (latency, throughput, error rates, CPU/GPU and memory), data quality (schema violations, missing values, out-of-range inputs), model behaviour (prediction distributions, confidence, feature and prediction drift against the training baseline), and business outcomes (accuracy on delayed labels, conversion or fraud loss). Logging should be structured (JSON with request ID, model version, latency, key inputs or their hashes and the prediction), sampled where volume is high, free of personal data or secrets, and shipped to a central store; metrics go to tools like Prometheus and Grafana, with alerts on thresholds and drift, and dedicated tools such as Evidently, WhyLabs or Arize help with model-specific monitoring.

Read full answer

Applies toPython 3.12 (`logging`), prometheus-client 0.20+

What to Monitor

Layer Metrics Alert example
System p50/p95/p99 latency, error rate, throughput, memory p99 > 300 ms for 5 min
Data Null rates, schema errors, unknown categories, ranges Null rate > 5% on a key feature
Model Prediction mean/distribution, confidence, drift (PSI, KS) PSI > 0.2 on top features
Outcomes Accuracy, precision/recall on labelled samples, business KPIs Recall drops 10% week over week

Code Example: Structured Logs and Metrics

import json
import logging
import time
import uuid

import numpy as np
from prometheus_client import Counter, Histogram, start_http_server
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

class JsonFormatter(logging.Formatter):
    def format(self, record):
        payload = {"level": record.levelname, "msg": record.getMessage(), "time": self.formatTime(record)}
        payload.update(getattr(record, "extra_fields", {}))
        return json.dumps(payload)

logger = logging.getLogger("inference")
handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)
logger.setLevel(logging.INFO)

PREDICTIONS = Counter("predictions_total", "Predictions served", ["model_version", "label"])
LATENCY = Histogram("prediction_latency_seconds", "Prediction latency")
INPUT_ERRORS = Counter("input_validation_errors_total", "Rejected requests")

X, y = make_classification(n_samples=1000, n_features=4, random_state=42)
model, MODEL_VERSION = LogisticRegression().fit(X, y), "v1.4.2"

def predict(features):
    request_id = str(uuid.uuid4())
    if len(features) != 4 or any(not np.isfinite(f) for f in features):
        INPUT_ERRORS.inc()
        logger.warning("invalid input", extra={"extra_fields": {"request_id": request_id}})
        raise ValueError("expected 4 finite features")
    start = time.perf_counter()
    proba = float(model.predict_proba([features])[0, 1])
    latency = time.perf_counter() - start
    label = int(proba >= 0.5)
    LATENCY.observe(latency)
    PREDICTIONS.labels(MODEL_VERSION, str(label)).inc()
    logger.info("prediction", extra={"extra_fields": {
        "request_id": request_id, "model_version": MODEL_VERSION,
        "score": round(proba, 4), "label": label, "latency_ms": round(latency * 1000, 2)}})
    return proba

start_http_server(9100)                     # Prometheus scrapes http://localhost:9100/metrics
for row in X[:3]:
    predict(row.tolist())
try:
    predict([1.0, float("nan"), 0.0, 2.0])
except ValueError:
    pass

Practices

  • Log model version and request ID with every prediction.

  • Store a baseline of training feature distributions to compare against.

  • Join predictions with delayed ground truth to track real performance.

  • Alert on trends, not single noisy points; route alerts to an owner.

  • Redact personal data; follow retention policies.

Common Pitfalls

  • Monitoring only uptime while the model silently degrades.

  • Unstructured text logs that can’t be queried.

  • Logging full raw inputs with personal data.

Interview Tips

  • Name the four layers (system, data, model, outcomes) with a metric each.

  • Mention structured logs, drift checks and delayed labels.

85

What are Python's profiling tools and how do they help optimize ML code?

Profilers measure where a program actually spends time or memory, so you optimize the real bottleneck instead of guessing: timeit times small snippets, cProfile (with pstats or the SnakeViz viewer) reports time per function, line_profiler shows time per line inside chosen functions, py-spy samples a running process with almost no overhead (useful in production), tracemalloc and memory_profiler track memory, Scalene reports CPU, memory and GPU together, and framework profilers such as torch.profiler break down operator and GPU time. In ML code they usually reveal Python loops that should be vectorized, slow data loading or preprocessing, repeated work that can be cached, and memory spikes from unnecessary copies.

Read full answer

Applies toPython 3.12

Tools

Tool Measures When
timeit / %timeit Snippet run time Comparing two implementations
cProfile + pstats Time per function First look at a whole script
line_profiler (@profile, kernprof) Time per line Inside a hot function
py-spy Sampling profiler, flame graphs Running or production processes
Scalene CPU, memory, GPU per line Mixed Python/native code
tracemalloc, memory_profiler Memory allocations Memory growth, OOM errors
torch.profiler Operator, CUDA and data-loading time Deep learning training loops

Code Example

import cProfile
import io
import pstats
import timeit
import tracemalloc

import numpy as np

rng = np.random.default_rng(42)
X = rng.normal(size=(20_000, 20))

def zscore_loop(X):
    out = np.empty_like(X)
    for j in range(X.shape[1]):
        col = X[:, j]
        mean, std = col.mean(), col.std()
        for i in range(X.shape[0]):           # slow Python loop
            out[i, j] = (X[i, j] - mean) / std
    return out

def zscore_vectorized(X):
    return (X - X.mean(axis=0)) / X.std(axis=0)

def pipeline():
    a = zscore_loop(X)
    b = zscore_vectorized(X)
    return np.allclose(a, b)

profiler = cProfile.Profile()
profiler.enable()
same = pipeline()
profiler.disable()
stream = io.StringIO()
pstats.Stats(profiler, stream=stream).sort_stats("cumulative").print_stats(5)
print(same)
print("\n".join(stream.getvalue().splitlines()[:15]))   # zscore_loop dominates the time

loop_t = timeit.timeit(lambda: zscore_loop(X[:2000]), number=3) / 3
vec_t = timeit.timeit(lambda: zscore_vectorized(X[:2000]), number=3) / 3
print(f"loop {loop_t * 1000:.1f} ms vs vectorized {vec_t * 1000:.2f} ms")

tracemalloc.start()
_ = [X.copy() for _ in range(5)]                     # an unnecessary 5 copies
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(f"peak memory during copies: {peak / 1e6:.1f} MB")   # about 5 x 3.2 MB

Run py-spy top --pid <PID> or py-spy record -o profile.svg -- python train.py to profile a script without changing it.

Common Pitfalls

  • Optimizing before profiling.

  • Profiling tiny inputs where the bottleneck differs from real data sizes.

  • Timing GPU code without synchronization (torch.cuda.synchronize()).

  • Profiler overhead distorting very fast functions (use sampling profilers).

Interview Tips

  • Name a time profiler, a line profiler, a sampling profiler and a memory profiler.

  • Give a typical finding, such as a Python loop that vectorization fixes.

86

How do unit and integration tests keep ML code correct?

Unit tests check small pieces of ML code in isolation with known inputs and outputs: a feature function returns the right values and shapes, a preprocessing step handles missing values and unseen categories, a custom loss matches a hand-computed result, and a model can overfit a tiny dataset; integration tests check that the pieces work together end to end: raw data goes through loading, feature engineering, training and prediction, the saved model reloads and gives the same predictions, and the serving API returns valid responses. Because ML outputs are numeric and partly random, tests use fixed seeds, tolerances (np.allclose, pytest.approx), and property checks such as probabilities summing to 1, alongside data validation tests and minimum-performance tests that guard against silent regressions.

Read full answer

Applies topytest 8.x, Python 3.12

What to Test

Test type Examples
Unit: features Known input → known output, edge cases (empty, NaN, zero division)
Unit: shapes and dtypes Model output shape, probability range
Unit: math Custom metric/loss vs hand calculation
Unit: learning sanity Overfits 10 samples; loss decreases
Data validation Schema, ranges, no duplicates, label balance
Integration: pipeline Train → save → load → predict on sample data
Integration: API HTTP request → valid JSON, error handling
Regression / performance Metric above a threshold on a fixed validation set

Code Example (test_pipeline.py, run with pytest -q)

import joblib
import numpy as np
import pandas as pd
import pytest
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder

def add_ratio_feature(df: pd.DataFrame) -> pd.DataFrame:
    """Feature under test: spend per visit, 0 when there are no visits."""
    visits = df["visits"].replace(0, np.nan)
    return df.assign(spend_per_visit=(df["spend"] / visits).fillna(0.0))

def build_pipeline() -> Pipeline:
    prep = ColumnTransformer([
        ("num", SimpleImputer(strategy="median"), ["spend", "visits", "spend_per_visit"]),
        ("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
    ])
    return Pipeline([("prep", prep), ("model", LogisticRegression(max_iter=1000))])

@pytest.fixture
def sample_data():
    rng = np.random.default_rng(42)
    df = pd.DataFrame({"spend": rng.uniform(0, 100, 200), "visits": rng.integers(0, 10, 200),
                       "plan": rng.choice(["a", "b"], 200)})
    y = (df["spend"] > 50).astype(int)
    return add_ratio_feature(df), y

def test_ratio_feature_values_and_zero_division():
    df = pd.DataFrame({"spend": [10.0, 5.0], "visits": [2, 0]})
    assert add_ratio_feature(df)["spend_per_visit"].tolist() == [5.0, 0.0]

def test_predict_proba_is_valid(sample_data):
    X, y = sample_data
    proba = build_pipeline().fit(X, y).predict_proba(X)
    assert proba.shape == (len(X), 2)
    assert np.allclose(proba.sum(axis=1), 1.0)

def test_handles_unseen_category_and_missing_values(sample_data):
    X, y = sample_data
    pipe = build_pipeline().fit(X, y)
    new = add_ratio_feature(pd.DataFrame({"spend": [np.nan], "visits": [3], "plan": ["zzz"]}))
    assert pipe.predict(new).shape == (1,)

def test_save_load_round_trip(sample_data, tmp_path):          # integration: train -> save -> load -> predict
    X, y = sample_data
    pipe = build_pipeline().fit(X, y)
    path = tmp_path / "model.joblib"
    joblib.dump(pipe, path)
    assert np.array_equal(joblib.load(path).predict(X), pipe.predict(X))

def test_minimum_quality(sample_data):
    X, y = sample_data
    assert build_pipeline().fit(X, y).score(X, y) >= 0.9         # guards against silent regressions

Common Pitfalls

  • Testing only that code runs, not that results are correct.

  • Exact float comparisons; use tolerances.

  • Tests that depend on random seeds not being fixed, causing flaky failures.

  • Slow tests on full datasets; use small fixtures and mark slow tests separately.

Interview Tips

  • Separate unit (functions, math, shapes) from integration (pipeline, persistence, API).

  • Mention data validation and performance thresholds in CI.

87

How do you handle exceptions and errors when deploying ML models in Python?

In a deployed model, error handling should validate inputs at the boundary (types, ranges, required fields, with Pydantic schemas) and reject bad requests with clear 4xx responses, catch only the specific exceptions you can handle (for example a missing feature store value or a timeout from a dependency), fall back safely when the model or a dependency fails (a cached prediction, a simple rule or a default decision), apply timeouts and retries with backoff to external calls, and log every failure with context (request ID, model version, stack trace) without leaking internals or personal data to the client. The model must also fail fast at start-up if it can’t load, and health checks should report readiness so the orchestrator doesn’t send traffic to a broken instance.

Read full answer

Applies toPython 3.12, FastAPI 0.110+, Pydantic 2.x

Error Categories

Category Example Handling
Invalid input Missing field, negative age, NaN Validate, return 422 with details
Model load failure Corrupt or missing file Fail at start-up; readiness check fails
Dependency failure Feature store timeout Timeout + retry, then fallback
Inference error Unexpected shape, numerical issue Catch, log with context, return 500 or fallback
Resource limits Out of memory, too-large batch Limit request size, return 413/503
Unknown Anything else Global handler logs and returns a generic 500

Code Example

import logging
import time
from contextlib import asynccontextmanager

import numpy as np
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

logger = logging.getLogger("model-api")
state = {}

class FeatureStoreTimeout(Exception):
    pass

@asynccontextmanager
async def lifespan(app: FastAPI):
    X, y = make_classification(n_samples=500, n_features=3, random_state=42)
    state["model"] = LogisticRegression().fit(X, y)       # a load failure here stops start-up (fail fast)
    yield

app = FastAPI(lifespan=lifespan)

class Request3(BaseModel):
    customer_id: int = Field(gt=0)
    amount: float = Field(ge=0, le=1_000_000)
    tenure_months: int = Field(ge=0, le=600)

def fetch_history_feature(customer_id, retries=2, timeout=0.2):
    for attempt in range(retries + 1):
        try:
            if customer_id % 13 == 0:                     # simulate a flaky dependency
                raise FeatureStoreTimeout(f"feature store timed out after {timeout}s")
            return float(customer_id % 7)
        except FeatureStoreTimeout:
            if attempt == retries:
                raise
            time.sleep(0.05 * 2 ** attempt)               # exponential backoff

@app.get("/ready")
def ready():
    return {"ready": "model" in state}

@app.post("/predict")
def predict(req: Request3):                               # Pydantic returns 422 for invalid input
    try:
        history = fetch_history_feature(req.customer_id)
        fallback = False
    except FeatureStoreTimeout:
        logger.warning("feature store unavailable, using fallback", extra={"customer_id": req.customer_id})
        history, fallback = 0.0, True                     # safe default
    x = np.array([[req.amount / 1000, req.tenure_months / 12, history]])
    if not np.isfinite(x).all():
        raise HTTPException(status_code=422, detail="features must be finite")
    score = float(state["model"].predict_proba(x)[0, 1])
    return {"score": round(score, 4), "fallback_used": fallback}

@app.exception_handler(Exception)
async def unhandled(request: Request, exc: Exception):
    logger.exception("unhandled error on %s", request.url.path)   # full trace goes to the logs only
    return JSONResponse(status_code=500, content={"error": "internal error"})

Common Pitfalls

  • Bare except: that hides bugs and swallows KeyboardInterrupt.

  • Returning stack traces or model internals to clients.

  • Silently returning a default prediction without logging or flagging it.

  • No timeouts on external calls, so one slow dependency blocks all workers.

Interview Tips

  • Separate validation errors, dependency failures and internal errors.

  • Mention fallbacks, retries with backoff, logging and health checks.

88

How do you write a Python generator that yields batches from a large dataset?

A batch generator reads the data lazily and yields one batch at a time, so memory stays constant no matter how large the dataset is: for a big CSV, read it in chunks with pd.read_csv(chunksize=...) and re-slice the chunks into fixed-size batches; for a large NumPy array on disk, use np.load(..., mmap_mode="r") and slice shuffled index ranges; and keep leftover rows so no data is lost between chunks. Options such as shuffling with a seed, a drop_last flag and an infinite mode for training loops make it a practical replacement for loading everything into memory.

Read full answer

Applies toPython 3.12, NumPy 2.x, pandas 2.x

Code Example

from collections.abc import Iterator

import numpy as np
import pandas as pd

def csv_batches(path: str, batch_size: int, chunk_size: int = 10_000, target: str = "label",
                drop_last: bool = False) -> Iterator[tuple[np.ndarray, np.ndarray]]:
    """Stream (X, y) batches from a CSV that may not fit in memory."""
    buffer = None
    for chunk in pd.read_csv(path, chunksize=chunk_size):
        buffer = chunk if buffer is None else pd.concat([buffer, chunk], ignore_index=True)
        while len(buffer) >= batch_size:
            batch, buffer = buffer.iloc[:batch_size], buffer.iloc[batch_size:]
            yield batch.drop(columns=target).to_numpy(np.float32), batch[target].to_numpy()
    if buffer is not None and len(buffer) and not drop_last:
        yield buffer.drop(columns=target).to_numpy(np.float32), buffer[target].to_numpy()   # final partial batch

def memmap_batches(X_path: str, y_path: str, batch_size: int, shuffle: bool = True, seed: int = 42,
                   epochs: int | None = 1) -> Iterator[tuple[np.ndarray, np.ndarray]]:
    """Batches from .npy files opened with memory mapping; epochs=None repeats forever."""
    X = np.load(X_path, mmap_mode="r")
    y = np.load(y_path, mmap_mode="r")
    rng = np.random.default_rng(seed)
    epoch = 0
    while epochs is None or epoch < epochs:
        order = rng.permutation(len(X)) if shuffle else np.arange(len(X))
        for start in range(0, len(X), batch_size):
            idx = np.sort(order[start:start + batch_size])      # sorted reads are faster on disk
            yield np.asarray(X[idx]), np.asarray(y[idx])
        epoch += 1

rng = np.random.default_rng(42)
data = pd.DataFrame(rng.normal(size=(25_000, 4)), columns=list("abcd"))
data["label"] = (data["a"] > 0).astype(int)
data.to_csv("big.csv", index=False)
np.save("X.npy", data[list("abcd")].to_numpy(np.float32))
np.save("y.npy", data["label"].to_numpy())

sizes = [len(yb) for _, yb in csv_batches("big.csv", batch_size=4096)]
print(len(sizes), sizes[-1], sum(sizes))         # 7 424 25000

gen = memmap_batches("X.npy", "y.npy", batch_size=1000, epochs=None)
for step, (xb, yb) in zip(range(3), gen):        # take 3 batches from the infinite stream
    print(step, xb.shape, xb.dtype, round(float(yb.mean()), 2))

The CSV has 25,000 rows: six full batches of 4,096 use 24,576 rows, leaving a final partial batch of 25,000 − 24,576 = 424 rows (dropped when drop_last=True).

Common Pitfalls

  • Losing the rows left over at the end of each chunk.

  • Converting the whole file to a list before batching, which defeats the purpose.

  • Shuffling only within chunks when data is sorted (for example by date or label); shuffle files beforehand or use a shuffle buffer.

  • Generators are single-use; create a new one per epoch or loop forever deliberately.

Interview Tips

  • Show yield, chunked reading and leftover handling.

  • Mention memory mapping and how frameworks (PyTorch IterableDataset, tf.data) build on the same idea.

89

How do you implement a convolutional neural network in PyTorch or TensorFlow?

A CNN stacks convolution layers, which slide small learnable filters over the image to detect local patterns, with non-linear activations and pooling layers that shrink the spatial size, and ends with a classifier head; in Keras you define it with Conv2D, MaxPooling2D, Flatten/GlobalAveragePooling2D and Dense layers and train with compile and fit, and in PyTorch you subclass nn.Module with nn.Conv2d layers and write the training loop. On MNIST, even a small CNN like the one below typically reaches about 99% test accuracy within a few epochs, because convolutions share weights across positions and exploit the spatial structure of images.

Read full answer

Applies toKeras 3 (TensorFlow backend), PyTorch 2.x

Code Example: Keras

import keras
import numpy as np

keras.utils.set_random_seed(42)
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = (x_train.astype("float32") / 255.0)[..., np.newaxis]     # (60000, 28, 28, 1)
x_test = (x_test.astype("float32") / 255.0)[..., np.newaxis]

model = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    keras.layers.Conv2D(32, 3, activation="relu", padding="same"),
    keras.layers.MaxPooling2D(),                                    # 28 -> 14
    keras.layers.Conv2D(64, 3, activation="relu", padding="same"),
    keras.layers.MaxPooling2D(),                                    # 14 -> 7
    keras.layers.Flatten(),
    keras.layers.Dropout(0.3),
    keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=3, batch_size=128, validation_split=0.1, verbose=2)
loss, acc = model.evaluate(x_test, y_test, verbose=0)
print(f"test accuracy: {acc:.4f}")                                  # about 0.99
model.summary()

Code Example: PyTorch (Same Architecture)

import torch
import torch.nn as nn

torch.manual_seed(42)

class SmallCNN(nn.Module):
    def __init__(self, n_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
        )
        self.head = nn.Sequential(nn.Flatten(), nn.Dropout(0.3), nn.Linear(64 * 7 * 7, n_classes))

    def forward(self, x):                 # x: (batch, 1, 28, 28), channels first
        return self.head(self.features(x))

model = SmallCNN()
x = torch.randn(8, 1, 28, 28)
print(model(x).shape)                     # torch.Size([8, 10])
print(sum(p.numel() for p in model.parameters()))   # 320 + 18,496 + 31,370 = 50,186

Layer-by-Layer Parameters (PyTorch version)

  • Conv 1: 32 filters × (3·3·1) + 32 biases = 320.

  • Conv 2: 64 × (3·3·32) + 64 = 18,496.

  • Dense: 64·7·7 = 3,136 inputs × 10 + 10 = 31,370.

Framework Differences

Aspect Keras PyTorch
Input layout Channels last (N, H, W, C) Channels first (N, C, H, W)
Training compile + fit Explicit loop
Output Softmax in the model Logits + CrossEntropyLoss

Common Pitfalls

  • Wrong channel order between frameworks.

  • Forgetting to scale pixel values to [0, 1] or normalize them.

  • Applying softmax before PyTorch’s CrossEntropyLoss.

Interview Tips

  • Explain convolution, pooling and weight sharing.

  • Compute output shapes and parameter counts for a layer.

90

How do you write a genetic algorithm in Python to optimize a simple problem?

A genetic algorithm searches for good solutions by evolving a population of candidates: it scores each candidate with a fitness function, selects parents that favour fitter candidates (for example by tournament selection), creates children by crossover (mixing two parents) and mutation (small random changes), keeps the best few unchanged (elitism), and repeats for many generations. It needs no gradients, so it suits discrete, noisy or non-differentiable problems such as feature selection, scheduling or hyperparameter search; the example below solves a 0/1 knapsack problem and shows the best fitness improving over generations.

Read full answer

Applies toPython 3.12, NumPy 2.x

Components

Component Knapsack choice
Representation Bit string: 1 = item packed
Fitness Total value, or 0 if over the weight limit
Selection Tournament of 3
Crossover Single-point, probability 0.9
Mutation Flip each bit with probability 1/n_items
Elitism Keep the best 2 each generation
Stopping Fixed number of generations

Code Example

import numpy as np

rng = np.random.default_rng(42)
n_items = 30
weights = rng.integers(5, 40, n_items)
values = rng.integers(10, 100, n_items)
capacity = int(weights.sum() * 0.4)

def fitness(pop):
    total_w = pop @ weights
    total_v = pop @ values
    return np.where(total_w <= capacity, total_v, 0)    # infeasible solutions score 0

def tournament(pop, fit, k=3):
    contestants = rng.integers(0, len(pop), k)
    return pop[contestants[np.argmax(fit[contestants])]]

def genetic_algorithm(pop_size=100, generations=150, crossover_rate=0.9, elite=2):
    pop = (rng.random((pop_size, n_items)) < 0.3).astype(int)
    history = []
    for _ in range(generations):
        fit = fitness(pop)
        history.append(int(fit.max()))
        children = [pop[i].copy() for i in np.argsort(-fit)[:elite]]   # elitism
        while len(children) < pop_size:
            p1, p2 = tournament(pop, fit), tournament(pop, fit)
            if rng.random() < crossover_rate:
                cut = rng.integers(1, n_items)
                child = np.concatenate([p1[:cut], p2[cut:]])
            else:
                child = p1.copy()
            flip = rng.random(n_items) < 1 / n_items           # mutation
            child[flip] = 1 - child[flip]
            children.append(child)
        pop = np.array(children)
    fit = fitness(pop)
    best = pop[np.argmax(fit)]
    return best, int(fit.max()), history

best, best_value, history = genetic_algorithm()
print("best value by generation:", history[::30])       # non-decreasing thanks to elitism
print("best value:", best_value, "| weight used:", int(best @ weights), "of", capacity)

def greedy():                                           # baseline: best value-per-weight first
    chosen, total = np.zeros(n_items, int), 0
    for i in np.argsort(-values / weights):
        if total + weights[i] <= capacity:
            chosen[i], total = 1, total + weights[i]
    return int(chosen @ values)
print("greedy baseline:", greedy())                     # the GA usually matches or beats it

Tuning Tips

  • Larger populations explore more but cost more per generation.

  • Too much mutation turns the search random; too little stalls in local optima.

  • Penalty or repair functions handle constraints better than discarding infeasible solutions in harder problems.

Common Pitfalls

  • No elitism, so the best solution can be lost.

  • A fitness function that gives every infeasible solution the same low score, leaving no gradient toward feasibility in hard problems.

  • Comparing against nothing; always include a simple baseline.

Interview Tips

  • Describe selection, crossover, mutation and elitism.

  • Say when GAs fit (no gradients, discrete search) and when gradient methods are better.

91

How do you simulate and compare optimization techniques on a fixed dataset?

To compare optimizers fairly, fix the dataset, the model, the loss, the starting weights and the random seed, then run each optimizer (for example batch gradient descent, mini-batch SGD, SGD with momentum, RMSprop and Adam) for the same budget of gradient evaluations or epochs, record the loss curve, and compare speed of convergence, final loss and stability, ideally over several seeds. Implementing the update rules yourself in NumPy for a logistic regression makes the differences visible: momentum smooths and accelerates steps, RMSprop scales each parameter’s step by its recent gradient size, and Adam combines both with bias correction.

Read full answer

Applies toPython 3.12, NumPy 2.x, Matplotlib 3.9+

Update Rules

Optimizer Update
GD / SGD w ← w − η·g
Momentum v ← βv + g; w ← w − η·v
RMSprop s ← ρs + (1 − ρ)g²; w ← w − η·g / (√s + ε)
Adam m ← β₁m + (1 − β₁)g; s ← β₂s + (1 − β₂)g²; bias-correct; w ← w − η·m̂ / (√ŝ + ε)

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_classification
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=2000, n_features=10, n_informative=6, random_state=42)
X = np.column_stack([np.ones(len(X)), StandardScaler().fit_transform(X)])

def loss_and_grad(w, Xb, yb):
    p = 1 / (1 + np.exp(-(Xb @ w)))
    loss = -np.mean(yb * np.log(p + 1e-12) + (1 - yb) * np.log(1 - p + 1e-12))
    return loss, Xb.T @ (p - yb) / len(yb)

def run(optimizer, lr, epochs=30, batch_size=64, seed=42):
    rng = np.random.default_rng(seed)
    w = np.zeros(X.shape[1])
    m, s, t = np.zeros_like(w), np.zeros_like(w), 0
    history = []
    for _ in range(epochs):
        order = rng.permutation(len(X)) if optimizer != "gd" else np.arange(len(X))
        size = len(X) if optimizer == "gd" else batch_size
        for start in range(0, len(X), size):
            idx = order[start:start + size]
            _, g = loss_and_grad(w, X[idx], y[idx])
            t += 1
            if optimizer in ("gd", "sgd"):
                w -= lr * g
            elif optimizer == "momentum":
                m = 0.9 * m + g
                w -= lr * m
            elif optimizer == "rmsprop":
                s = 0.9 * s + 0.1 * g ** 2
                w -= lr * g / (np.sqrt(s) + 1e-8)
            elif optimizer == "adam":
                m = 0.9 * m + 0.1 * g
                s = 0.999 * s + 0.001 * g ** 2
                m_hat, s_hat = m / (1 - 0.9 ** t), s / (1 - 0.999 ** t)
                w -= lr * m_hat / (np.sqrt(s_hat) + 1e-8)
        history.append(loss_and_grad(w, X, y)[0])          # full-data loss after each epoch
    return history

configs = {"gd": 0.5, "sgd": 0.1, "momentum": 0.01, "rmsprop": 0.01, "adam": 0.01}
results = {name: run(name, lr) for name, lr in configs.items()}
for name, hist in results.items():
    print(f"{name:8s} loss after 1 epoch {hist[0]:.4f}, after 30 epochs {hist[-1]:.4f}")
# Full-batch GD makes one step per epoch, so it lags far behind the mini-batch methods early on

plt.figure(figsize=(7, 4))
for name, hist in results.items():
    plt.plot(range(1, 31), hist, label=name)
plt.yscale("log")
plt.xlabel("epoch")
plt.ylabel("training log loss")
plt.legend()
plt.savefig("optimizer_comparison.png", dpi=120)

Fair Comparison Checklist

  • Same initialization, data order seeds and budget (epochs or gradient evaluations).

  • Tune each optimizer’s learning rate separately; a bad learning rate can make any optimizer look poor.

  • Repeat over several seeds and report mean and spread.

  • Compare validation loss too, not only training loss.

Common Pitfalls

  • Using one learning rate for all optimizers.

  • Comparing by iterations when per-iteration cost differs (full batch vs mini-batch).

  • Drawing conclusions from one run.

Interview Tips

  • Write the update rules and explain what momentum and adaptive scaling add.

  • Describe how you’d make the comparison fair.

92

How do you visualize a classifier's decision boundaries in Python?

To visualize decision boundaries for a two-feature classifier, create a fine grid covering the feature space with np.meshgrid, predict the class (or probability) for every grid point, reshape the predictions to the grid’s shape, draw them with plt.contourf, and overlay the training points with plt.scatter; scikit-learn’s DecisionBoundaryDisplay.from_estimator does the grid work for you. Comparing several models side by side shows how they generalize: linear models draw straight lines, k-nearest neighbours and deep trees draw jagged regions that can overfit, and RBF SVMs and ensembles draw smooth curves. For data with more than two features, plot boundaries on two selected features or on a 2-D projection (PCA), knowing the picture is only a slice.

Read full answer

Applies toscikit-learn 1.5+, Matplotlib 3.9+

Code Example

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_moons
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import DecisionBoundaryDisplay
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

X, y = make_moons(n_samples=400, noise=0.25, random_state=42)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=42)

models = {
    "Logistic regression": make_pipeline(StandardScaler(), LogisticRegression()),
    "k-NN (k=1)": KNeighborsClassifier(n_neighbors=1),
    "RBF SVM": make_pipeline(StandardScaler(), SVC(gamma=2, C=1, probability=True, random_state=42)),
    "Random forest": RandomForestClassifier(n_estimators=200, random_state=42),
}

fig, axes = plt.subplots(1, 4, figsize=(18, 4.2))
for ax, (name, model) in zip(axes, models.items()):
    model.fit(X_tr, y_tr)
    DecisionBoundaryDisplay.from_estimator(model, X, response_method="predict_proba", cmap="RdBu",
                                           alpha=0.6, ax=ax, grid_resolution=200)
    ax.scatter(X_tr[:, 0], X_tr[:, 1], c=y_tr, cmap="RdBu", edgecolors="k", s=18)
    ax.set_title(f"{name}\ntest acc {model.score(X_te, y_te):.2f}")
fig.tight_layout()
fig.savefig("decision_boundaries.png", dpi=120)

# The same idea by hand with meshgrid + contourf
model = models["RBF SVM"]
xx, yy = np.meshgrid(np.linspace(X[:, 0].min() - 0.5, X[:, 0].max() + 0.5, 300),
                     np.linspace(X[:, 1].min() - 0.5, X[:, 1].max() + 0.5, 300))
grid = np.c_[xx.ravel(), yy.ravel()]
zz = model.predict_proba(grid)[:, 1].reshape(xx.shape)
fig, ax = plt.subplots(figsize=(5, 4))
contour = ax.contourf(xx, yy, zz, levels=20, cmap="RdBu", alpha=0.7)
ax.contour(xx, yy, zz, levels=[0.5], colors="k")             # the decision boundary itself
ax.scatter(X[:, 0], X[:, 1], c=y, cmap="RdBu", edgecolors="k", s=15)
fig.colorbar(contour, label="P(class 1)")
fig.savefig("svm_boundary.png", dpi=120)
print("grid points predicted:", grid.shape[0])                # 90000

Reading the Plots

Pattern Meaning
Straight line Linear model; underfits curved data
Islands around single points Overfitting (k = 1, deep trees)
Smooth curve following the data Good fit
Wide uncertain band (probability ≈ 0.5) Region where the model is unsure

Common Pitfalls

  • Grids too coarse, making boundaries look blocky.

  • Forgetting to apply the same scaling to the grid as to the data (pipelines handle it).

  • Treating a 2-D slice of a high-dimensional model as the full picture.

Interview Tips

  • Explain meshgrid → predict → reshape → contourf.

  • Use the plots to discuss underfitting and overfitting.

93

How do you implement the A* search algorithm for grid pathfinding in Python?

A* finds a shortest path from a start cell to a goal by always expanding the cell with the lowest f(n) = g(n) + h(n), where g is the exact cost from the start and h is a heuristic estimate of the remaining cost; with an admissible heuristic (one that never overestimates, such as Manhattan distance on a 4-connected grid) the path it returns is optimal. A clean Python implementation uses heapq as the priority queue, a dictionary of best known g-costs, a came_from map to rebuild the path, and skips stale queue entries; with h = 0 it becomes Dijkstra’s algorithm, which explores many more cells.

Read full answer

Applies toPython 3.12 (standard library)

Code Example

import heapq

GRID = [
    "S...#......",
    ".##.#.####.",
    ".#..#....#.",
    ".#.###.#.#.",
    ".#.....#...",
    ".#####.###.",
    ".......#..G",
]

def parse(grid):
    walls, start, goal = set(), None, None
    for r, row in enumerate(grid):
        for c, ch in enumerate(row):
            if ch == "#":
                walls.add((r, c))
            elif ch == "S":
                start = (r, c)
            elif ch == "G":
                goal = (r, c)
    return walls, start, goal, len(grid), len(grid[0])

def manhattan(a, b):
    return abs(a[0] - b[0]) + abs(a[1] - b[1])

def a_star(grid, heuristic=manhattan):
    walls, start, goal, rows, cols = parse(grid)
    open_heap = [(heuristic(start, goal), 0, start)]      # (f, g, cell)
    g_cost = {start: 0}
    came_from = {}
    expanded = 0
    while open_heap:
        f, g, cell = heapq.heappop(open_heap)
        if g > g_cost.get(cell, float("inf")):
            continue                                       # stale entry
        expanded += 1
        if cell == goal:
            path = [cell]
            while cell in came_from:
                cell = came_from[cell]
                path.append(cell)
            return path[::-1], expanded
        r, c = cell
        for nr, nc in ((r - 1, c), (r + 1, c), (r, c - 1), (r, c + 1)):
            nxt = (nr, nc)
            if not (0 <= nr < rows and 0 <= nc < cols) or nxt in walls:
                continue
            new_g = g + 1
            if new_g < g_cost.get(nxt, float("inf")):
                g_cost[nxt] = new_g
                came_from[nxt] = cell
                heapq.heappush(open_heap, (new_g + heuristic(nxt, goal), new_g, nxt))
    return None, expanded                                  # no path

path, expanded = a_star(GRID)
_, expanded_dijkstra = a_star(GRID, heuristic=lambda a, b: 0)
print("path length (moves):", len(path) - 1)
print("cells expanded: A*", expanded, "| Dijkstra", expanded_dijkstra)   # A* expands fewer cells

canvas = [list(row) for row in GRID]
for r, c in path[1:-1]:
    canvas[r][c] = "*"
print("\n".join("".join(row) for row in canvas))

Both searches return a path of the same (optimal) length, because the Manhattan distance never overestimates the true remaining cost on a 4-connected grid; A* simply reaches it after expanding fewer cells.

Heuristics

Movement Admissible heuristic
4-connected, cost 1 Manhattan distance
8-connected, diagonal cost √2 Octile distance
Any direction Euclidean distance
Weighted terrain Heuristic × minimum step cost

Common Pitfalls

  • Inadmissible heuristics (e.g. Euclidean × 2), which can return non-optimal paths.

  • Not skipping stale heap entries, which wastes time.

  • Ties broken arbitrarily; adding g to the tuple (as above) or a counter keeps ordering stable.

Interview Tips

  • Explain f = g + h and admissibility.

  • Show the priority queue and path reconstruction, and compare with Dijkstra.

94

How do you implement a simple reinforcement learning agent for a basic game?

A simple RL agent for a small game uses tabular Q-learning: define the game’s states, actions and rewards, keep a Q-table of estimated returns for each state-action pair, choose actions ε-greedily (mostly the best known action, sometimes a random one), and after every step update Q(s, a) ← Q(s, a) + α·(r + γ·max Q(s′, ·) − Q(s, a)). The example below teaches an agent a “catch the falling fruit” game on a small grid and compares its catch rate with a random player, showing learning without any neural network.

Read full answer

Applies toPython 3.12, NumPy 2.x

Game Design

Element Definition
Board 5 columns, fruit falls from row 0 to row 4
State (fruit column, fruit row, basket column) = 5 × 5 × 5 = 125 states
Actions Move left, stay, move right
Reward +1 catch, −1 miss (at the bottom row), 0 otherwise
Episode One fruit drop (4 steps)

Code Example

import numpy as np

class CatchGame:
    def __init__(self, width=5, height=5, seed=42):
        self.width, self.height = width, height
        self.rng = np.random.default_rng(seed)

    def reset(self):
        self.fruit_col = int(self.rng.integers(self.width))
        self.fruit_row = 0
        self.basket = int(self.rng.integers(self.width))
        return self.state()

    def state(self):
        return (self.fruit_col, self.fruit_row, self.basket)

    def step(self, action):                                   # 0 = left, 1 = stay, 2 = right
        self.basket = int(np.clip(self.basket + action - 1, 0, self.width - 1))
        self.fruit_row += 1
        if self.fruit_row == self.height - 1:
            reward = 1.0 if self.basket == self.fruit_col else -1.0
            return self.state(), reward, True
        return self.state(), 0.0, False

def train(episodes=5000, alpha=0.2, gamma=0.95, seed=42):
    env, rng = CatchGame(seed=seed), np.random.default_rng(seed)
    Q = np.zeros((env.width, env.height, env.width, 3))
    for ep in range(episodes):
        s, done = env.reset(), False
        epsilon = max(0.05, 1 - ep / (0.6 * episodes))
        while not done:
            a = int(rng.integers(3)) if rng.random() < epsilon else int(np.argmax(Q[s]))
            s2, r, done = env.step(a)
            target = r if done else r + gamma * Q[s2].max()
            Q[s + (a,)] += alpha * (target - Q[s + (a,)])
            s = s2
    return Q

def catch_rate(policy, episodes=2000, seed=7):
    env, rng = CatchGame(seed=seed), np.random.default_rng(seed)
    caught = 0
    for _ in range(episodes):
        s, done = env.reset(), False
        while not done:
            s, r, done = env.step(policy(s, rng))
        caught += r > 0
    return caught / episodes

Q = train()
print(f"random player catch rate:  {catch_rate(lambda s, rng: int(rng.integers(3))):.2f}")
print(f"trained agent catch rate:  {catch_rate(lambda s, rng: int(np.argmax(Q[s]))):.2f}")
# The fruit takes 4 steps to fall and the basket can move up to 4 columns, so a perfect player
# always catches it; the trained agent gets close to 1.0

Why It Works

  • Rewards only arrive at the end, but the discounted update propagates value back to earlier states.

  • Exploration early on visits all states; decaying ε lets the agent exploit what it learned.

  • 125 states × 3 actions is small enough for a table; bigger games need function approximation (DQN).

Common Pitfalls

  • Bootstrapping from terminal states.

  • Evaluating with exploration on.

  • A state that leaves out needed information (e.g. omitting the basket position).

Interview Tips

  • Define state, actions and reward before coding.

  • Write the Q-update and compare with a random baseline.

95

How do you forecast stock market time series with a Python library?

For stock data, forecast with time-series tools while respecting what prices are: daily closing prices behave close to a random walk, so forecast returns (or log returns) rather than raw prices, use libraries such as statsmodels (ARIMA, exponential smoothing, GARCH via the arch package for volatility) or forecasting frameworks like Prophet, sktime, Darts or gradient boosting on lagged features, validate with walk-forward (rolling-origin) backtests, and always compare against the naive forecast (“tomorrow equals today”), which is very hard to beat for prices. Volatility is far more predictable than direction, and any model must be judged on out-of-sample errors after costs, never on in-sample fit.

Read full answer

Applies tostatsmodels 0.14+, pandas 2.x (yfinance for real data)

Approach

Step Detail
Data Daily OHLCV (for real data: yfinance.download("AAPL", start=...))
Transform Log returns: ln(Pₜ / Pₜ₋₁)
Baselines Naive (last price), zero return, historical mean
Models ARIMA on returns, ETS, GARCH for volatility, ML on lagged features
Validation Walk-forward: train on [0, t], predict t + 1, roll forward
Metrics MAE/RMSE of returns, directional accuracy, trading P&L after costs

Code Example

import warnings

import numpy as np
import pandas as pd
from statsmodels.tsa.arima.model import ARIMA

warnings.filterwarnings("ignore")                     # ARIMA convergence messages on noisy data
rng = np.random.default_rng(42)
dates = pd.bdate_range("2024-01-01", periods=600)
returns = 0.0003 + 0.1 * np.r_[0, rng.normal(0, 0.012, 599)]          # small AR structure + noise
for t in range(1, 600):
    returns[t] += 0.1 * returns[t - 1]
prices = pd.Series(100 * np.exp(np.cumsum(returns)), index=dates, name="close")
# Real data instead: prices = yfinance.download("AAPL", start="2024-01-01")["Close"].squeeze()

log_ret = np.log(prices).diff().dropna()
train_end = 500
naive_err, model_err, direction_hits = [], [], []
history = list(log_ret.iloc[:train_end])
for t in range(train_end, len(log_ret)):              # walk-forward evaluation
    fit = ARIMA(history, order=(1, 0, 0)).fit()
    forecast = float(fit.forecast(1)[0])
    actual = float(log_ret.iloc[t])
    model_err.append(abs(actual - forecast))
    naive_err.append(abs(actual - 0.0))               # "no change" forecast for returns
    direction_hits.append(np.sign(forecast) == np.sign(actual))
    history.append(actual)

print(f"MAE of returns: ARIMA {np.mean(model_err):.5f} vs naive {np.mean(naive_err):.5f}")
print(f"directional accuracy: {np.mean(direction_hits):.3f}")   # near 0.5 is typical for real prices
next_price = prices.iloc[-1] * np.exp(ARIMA(log_ret, order=(1, 0, 0)).fit().forecast(1).iloc[0])
print(f"next-day price forecast: {next_price:.2f} (last close {prices.iloc[-1]:.2f})")

The synthetic series has only weak structure, which is realistic: improvements over the naive forecast, if any, are small.

Common Pitfalls

  • Random train/test splits, which leak the future.

  • Forecasting prices and reporting a low error that just copies yesterday’s price.

  • Look-ahead bias in features (using today’s close to predict today).

  • Ignoring transaction costs and regime changes; past patterns often vanish.

Interview Tips

  • Mention returns vs prices, walk-forward validation and the naive baseline.

  • Be realistic about predictability; volatility models are more reliable than direction forecasts.

96

What does quantum computing mean for machine learning, from a Python view?

Quantum machine learning (QML) explores using quantum computers, which process information in qubits that can be in superpositions and become entangled, to speed up or improve learning tasks, for example through variational quantum circuits trained like neural networks, quantum kernels for classification, and quantum algorithms for sampling and optimization. Today it is a research field: available hardware is small and noisy (the “NISQ” era), proven speed-ups for practical ML problems are not established, and classical methods remain far better for real workloads; in Python you can experiment with simulators and cloud hardware through Qiskit (IBM), PennyLane (Xanadu), Cirq (Google) and Amazon Braket, which integrate with NumPy, PyTorch and JAX.

Read full answer

Applies toPennyLane 0.36+, Qiskit 1.x (concepts apply broadly)

Key Ideas

Concept Meaning for ML
Qubit State in superposition of 0 and 1
Entanglement Correlations between qubits with no classical equivalent
Variational circuit Parameterized quantum circuit trained with a classical optimizer
Quantum kernel Data mapped into a quantum state space; similarity used by an SVM
Barren plateaus Gradients vanish in large random circuits, making training hard
Noise / decoherence Limits circuit depth on current hardware

Implications (Realistic View)

Area Status
Speed-ups for standard ML (classification, regression) Not demonstrated on practical problems
Data loading Encoding large classical datasets into quantum states is a bottleneck
Optimization / sampling Active research (QAOA, quantum annealing)
Quantum data (chemistry, physics simulations) The most promising application area
Cryptography Future large quantum computers threaten some public-key schemes; post-quantum cryptography is being adopted

Code Example: A Variational Classifier with PennyLane (Simulator)

import pennylane as qml
from pennylane import numpy as pnp
from sklearn.datasets import make_moons
from sklearn.preprocessing import MinMaxScaler

X, y = make_moons(n_samples=60, noise=0.1, random_state=42)
X = MinMaxScaler((0, pnp.pi)).fit_transform(X)                 # angles for encoding
y_signed = 2 * y - 1                                           # labels -1 / +1

dev = qml.device("default.qubit", wires=2)                     # classical simulator

@qml.qnode(dev)
def circuit(weights, x):
    qml.AngleEmbedding(x, wires=[0, 1])                        # encode the 2 features
    qml.StronglyEntanglingLayers(weights, wires=[0, 1])        # trainable entangling layers
    return qml.expval(qml.PauliZ(0))                           # output in [-1, 1]

def cost(weights):
    preds = pnp.stack([circuit(weights, x) for x in X])
    return pnp.mean((preds - y_signed) ** 2)

pnp.random.seed(42)
weights = pnp.random.uniform(0, 2 * pnp.pi, size=qml.StronglyEntanglingLayers.shape(n_layers=3, n_wires=2),
                             requires_grad=True)
opt = qml.AdamOptimizer(0.1)
for step in range(40):
    weights, loss = opt.step_and_cost(cost, weights)
accuracy = pnp.mean(pnp.sign(pnp.stack([circuit(weights, x) for x in X])) == y_signed)
print(f"final loss {float(loss):.3f}, training accuracy {float(accuracy):.2f}")

This runs on a classical simulator; a logistic regression or small neural network solves the same task faster and better, which reflects the current state of the field.

Common Pitfalls in Answers

  • Claiming quantum computers already speed up mainstream ML.

  • Ignoring noise, qubit counts and data-loading costs.

  • Confusing quantum-inspired classical algorithms with quantum hardware results.

Interview Tips

  • Explain qubits and variational circuits briefly, then give a realistic assessment.

  • Name the Python libraries and say you’d experiment on simulators.

97

What is the role of Explainable AI and which Python libraries support it?

Explainable AI (XAI) provides methods that make a model’s predictions understandable to people: global explanations show which features drive the model overall, local explanations show why it made one specific prediction, and both help debug models, detect bias and leakage, build trust with users, and meet regulatory requirements (such as explanations for credit decisions). In Python, SHAP computes Shapley-value attributions for any model (fast exact versions for trees), LIME fits simple local surrogate models, scikit-learn provides permutation importance and partial dependence plots, InterpretML offers inherently interpretable boosting models (EBMs), Captum explains PyTorch networks (integrated gradients, saliency), and Fairlearn and AIF360 assess fairness across groups.

Read full answer

Applies toscikit-learn 1.5+, shap 0.45+

Methods

Method Scope Library
Interpretable models (linear, small trees, EBMs) Global + local scikit-learn, InterpretML
Permutation importance Global sklearn.inspection
Partial dependence / ICE plots Global / per sample sklearn.inspection
SHAP values Local, aggregated to global shap
LIME Local lime
Integrated gradients, saliency Local (deep learning) Captum, tf-explain
Counterfactuals Local (“what would change the outcome”) DiCE, Alibi
Fairness metrics Group level Fairlearn, AIF360

Code Example

import matplotlib

matplotlib.use("Agg")
import numpy as np
import pandas as pd
import shap
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.inspection import PartialDependenceDisplay, permutation_importance
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(42)
n = 3000
X = pd.DataFrame({
    "income": rng.normal(60, 20, n),
    "debt_ratio": rng.uniform(0, 1, n),
    "late_payments": rng.poisson(1, n),
    "noise": rng.normal(size=n),
})
logit = -0.04 * (X["income"] - 60) + 3 * (X["debt_ratio"] - 0.5) + 0.8 * X["late_payments"] - 1
y = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)              # 1 = default
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)
model = HistGradientBoostingClassifier(random_state=42).fit(X_tr, y_tr)

perm = permutation_importance(model, X_te, y_te, scoring="roc_auc", n_repeats=10, random_state=42)
print(pd.Series(perm.importances_mean, index=X.columns).sort_values(ascending=False).round(4))
# late_payments, debt_ratio and income matter; noise is close to 0

PartialDependenceDisplay.from_estimator(model, X_te, ["debt_ratio", "income"]).figure_.savefig("pdp.png")

explainer = shap.Explainer(model, X_tr)                                 # global and local attributions
shap_values = explainer(X_te.iloc[:200])
applicant = 0
contrib = pd.Series(shap_values.values[applicant], index=X.columns).sort_values(key=abs, ascending=False)
print("why applicant 0 got their score (log-odds contributions):")
print(contrib.round(3))

Practical Advice

  • Explain on held-out data, and check explanations against domain knowledge.

  • Correlated features split or swap attributions; group them or interpret together.

  • Explanations describe the model, not causal effects in the real world.

  • For high-stakes decisions, consider inherently interpretable models first.

Common Pitfalls

  • Trusting impurity-based tree importances, which are biased toward high-cardinality features.

  • Presenting SHAP values to non-experts without translation into plain language.

  • Using explanations as proof a model is fair.

Interview Tips

  • Separate global from local explanations and name a method for each.

  • Mention SHAP, LIME, permutation importance and fairness tools.

98

How have deep learning advances changed NLP work in Python?

Deep learning shifted NLP in Python from hand-built pipelines (tokenization rules, bag-of-words or TF-IDF features, and separate models per task) to pre-trained neural models that learn language representations from huge text corpora: word embeddings (word2vec, GloVe), then RNNs and LSTMs, then attention and Transformers, which led to BERT-style encoders for understanding tasks and GPT-style decoders, now large language models (LLMs), for generation. In practice this means most Python NLP work today starts from Hugging Face Transformers models and fine-tunes them, or uses an LLM through an API or an open-weights model with prompting, retrieval-augmented generation (RAG) and parameter-efficient fine-tuning (LoRA), while classic tools (spaCy, NLTK, scikit-learn) remain useful for preprocessing, fast baselines and resource-limited settings.

Read full answer

Applies toHugging Face Transformers 4.40+, PyTorch 2.x, Python 3.12

Timeline

Era Techniques Python tools
Classic (pre-2013) Rules, n-grams, TF-IDF, CRFs, SVMs NLTK, scikit-learn
Embeddings (2013–) word2vec, GloVe, fastText gensim
Recurrent (2014–) LSTMs, GRUs, seq2seq + attention Keras, PyTorch
Transformers (2017–) BERT, RoBERTa, T5, GPT Hugging Face Transformers
LLMs (2020–) Instruction-tuned and chat models, RAG, LoRA transformers, PEFT, vLLM, LangChain/LlamaIndex, provider SDKs

What Changed for Practitioners

Before Now
Feature engineering per task Fine-tune one pre-trained model
Thousands of labels needed Few-shot prompting or small labelled sets
One language at a time Multilingual models
Separate pipelines (NER, sentiment, QA) One model family or one LLM for many tasks
CPU-friendly models GPUs or hosted APIs; cost and latency matter

Code Example

from transformers import pipeline

ner = pipeline("ner", model="dslim/bert-base-NER", aggregation_strategy="simple")
print([(e["word"], e["entity_group"]) for e in ner("Ada Lovelace worked with Charles Babbage in London.")])
# e.g. [('Ada Lovelace', 'PER'), ('Charles Babbage', 'PER'), ('London', 'LOC')]

zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
result = zero_shot("The battery drains in two hours and support never replied.",
                   candidate_labels=["battery", "customer service", "price", "screen"], multi_label=True)
print({label: round(score, 2) for label, score in zip(result["labels"], result["scores"])})

summarizer = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6")
text = ("Transformers process all tokens in parallel using self-attention, which lets them capture "
        "long-range dependencies. Pre-training on large corpora followed by fine-tuning on small labelled "
        "datasets became the standard approach for NLP tasks such as classification and question answering.")
print(summarizer(text, max_length=40, min_length=10)[0]["summary_text"])

Remaining Challenges

  • Hallucinations and factuality in generative models.

  • Compute cost and latency; distillation and quantization help.

  • Bias, safety and evaluation of open-ended outputs.

  • Domain adaptation for specialized text (legal, medical).

Common Pitfalls

  • Using an LLM where a small fine-tuned classifier is cheaper and more reliable.

  • Skipping baselines; TF-IDF + logistic regression is still a strong check.

  • Evaluating generation only by eye.

Interview Tips

  • Trace the progression from TF-IDF to embeddings, RNNs, Transformers and LLMs.

  • Mention Hugging Face, fine-tuning vs prompting, and RAG.

99

How do big data technologies integrate with Python in ML projects?

When data outgrows a single machine’s memory, Python connects to big data systems that distribute storage and computation: PySpark runs DataFrame transformations and Spark MLlib models across a cluster, Dask scales pandas- and NumPy-style code with familiar APIs, Ray parallelizes Python functions, training and hyperparameter tuning, Polars and DuckDB process tens to hundreds of gigabytes quickly on one machine, and cloud warehouses (BigQuery, Snowflake, Redshift) run SQL at scale with Python clients. A common pattern is to do heavy filtering, joins and feature aggregation in Spark, Dask or the warehouse, store results in columnar formats (Parquet, Delta Lake, Iceberg), then train with scikit-learn, XGBoost or PyTorch on the reduced feature set, or with distributed training when the data is still too big.

Read full answer

Applies toPySpark 3.5+, Dask 2024+, Polars 1.x, DuckDB 1.x

Tools

Tool Strength Typical use
PySpark Cluster-scale ETL, SQL, MLlib Terabytes of logs, joins, feature pipelines
Dask pandas/NumPy-like API, scales out Larger-than-memory DataFrames, parallel scikit-learn
Ray General distributed Python, Ray Train/Tune/Serve Distributed training and tuning
Polars Fast multi-threaded single-machine DataFrames, lazy queries Up to hundreds of GB
DuckDB In-process SQL on Parquet/CSV Fast analytics without a cluster
Warehouses Managed SQL at scale Feature extraction close to the data
Kafka + Spark Streaming / Flink Streams Real-time features
Parquet / Delta Lake / Iceberg Columnar, versioned storage Shared data layer

Code Example: Same Aggregation, Three Engines

import duckdb
import numpy as np
import pandas as pd
import polars as pl

rng = np.random.default_rng(42)
n = 2_000_000
pd.DataFrame({
    "user_id": rng.integers(0, 50_000, n),
    "event": rng.choice(["view", "click", "buy"], n, p=[0.8, 0.17, 0.03]),
    "amount": rng.lognormal(3, 1, n).round(2),
}).to_parquet("events.parquet")                    # stands in for a partitioned data lake table

polars_features = (pl.scan_parquet("events.parquet")          # lazy: only needed columns are read
                   .group_by("user_id")
                   .agg(pl.len().alias("events"),
                        (pl.col("event") == "buy").sum().alias("purchases"),
                        pl.col("amount").filter(pl.col("event") == "buy").sum().alias("spend"))
                   .collect())

duck_features = duckdb.sql("""
    SELECT user_id, COUNT(*) AS events,
           SUM(CASE WHEN event = 'buy' THEN 1 ELSE 0 END) AS purchases,
           SUM(CASE WHEN event = 'buy' THEN amount ELSE 0 END) AS spend
    FROM 'events.parquet' GROUP BY user_id
""").df()

print(polars_features.shape, duck_features.shape)            # (about 50000, 4) for both
print(int(polars_features["purchases"].sum()) == int(duck_features["purchases"].sum()))   # True

# The same logic in PySpark on a cluster:
#   spark.read.parquet("s3://bucket/events/").groupBy("user_id").agg(
#       F.count("*").alias("events"),
#       F.sum(F.when(F.col("event") == "buy", 1).otherwise(0)).alias("purchases"))

Integration Patterns

  • Push computation to the data (SQL in the warehouse, Spark on the lake) and bring back features.

  • Sample for exploration in pandas, run full jobs at scale.

  • Convert between engines with Apache Arrow for zero-copy transfers.

  • Orchestrate pipelines with Airflow, Dagster or Prefect.

Common Pitfalls

  • Calling .toPandas() or .compute() on huge results and running out of memory.

  • Reaching for a cluster when Polars or DuckDB on one machine is enough.

  • Small-file problems in data lakes (millions of tiny Parquet files).

Interview Tips

  • Match tools to data size: pandas → Polars/DuckDB → Dask/Spark/Ray.

  • Describe the aggregate-at-scale, train-on-features pattern.

100

What is federated learning and how can you implement it in Python?

Federated learning trains a shared model across many devices or organizations (phones, hospitals, banks) without moving their raw data to a central server: each client trains the current global model on its local data for a few steps and sends only the model updates back, and the server aggregates them, typically by Federated Averaging (FedAvg, a weighted average of client weights by number of samples), then sends out the improved model for the next round. It protects privacy and helps with data residency rules, but brings challenges: non-identically distributed client data, unreliable or slow clients, communication cost, and updates that can still leak information, which is why it is combined with secure aggregation and differential privacy. In Python, Flower is the most widely used framework, alongside TensorFlow Federated, PySyft and NVIDIA FLARE.

Read full answer

Applies toPython 3.12, NumPy 2.x; Flower 1.x for production setups

How a Round Works

  1. Server sends global weights w to a sample of clients.

  2. Each client k trains locally on its n_k samples and returns w_k.

  3. Server computes w ← Σ (n_k / n) · w_k.

  4. Repeat for many rounds.

Challenges and Mitigations

Challenge Mitigation
Non-IID data across clients FedProx, personalization layers, more rounds
Stragglers / dropouts Client sampling, asynchronous aggregation
Communication cost Fewer, larger local steps; compression, quantized updates
Privacy leakage from updates Secure aggregation, differential privacy (clipping + noise)
Malicious clients Robust aggregation (median, trimmed mean)

Code Example: FedAvg Simulation with NumPy

import numpy as np
from sklearn.datasets import make_classification

rng = np.random.default_rng(42)
X, y = make_classification(n_samples=6000, n_features=10, n_informative=6, random_state=42)
X = np.column_stack([np.ones(len(X)), X])

order = np.argsort(X[:, 1])                       # non-IID split: clients see different feature ranges
client_sizes = [3000, 1500, 1000, 500]
splits = np.split(order, np.cumsum(client_sizes)[:-1])
clients = [(X[idx], y[idx]) for idx in splits]

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def local_train(w, X_k, y_k, epochs=5, lr=0.1, batch=64):
    w = w.copy()
    for _ in range(epochs):
        perm = rng.permutation(len(X_k))
        for s in range(0, len(X_k), batch):
            idx = perm[s:s + batch]
            w -= lr * X_k[idx].T @ (sigmoid(X_k[idx] @ w) - y_k[idx]) / len(idx)
    return w

def accuracy(w):
    return float(np.mean((sigmoid(X @ w) > 0.5) == y))

w_global = np.zeros(X.shape[1])
for round_ in range(1, 11):
    updates, sizes = [], []
    for X_k, y_k in clients:                      # in reality this runs on each device
        updates.append(local_train(w_global, X_k, y_k))
        sizes.append(len(y_k))
    w_global = np.average(updates, axis=0, weights=sizes)   # FedAvg: weighted by sample count
    if round_ in (1, 5, 10):
        print(f"round {round_}: global accuracy {accuracy(w_global):.3f}")

w_central = local_train(np.zeros(X.shape[1]), X, y, epochs=10)   # reference: data pooled centrally
print(f"centralized training accuracy: {accuracy(w_central):.3f}")
local_only = local_train(np.zeros(X.shape[1]), *clients[3], epochs=50)
print(f"smallest client alone: {accuracy(local_only):.3f}")      # usually worse on the full population

In a real deployment, Flower provides the client and server roles: a NumPyClient implements fit and evaluate on local data, and the server runs a FedAvg strategy over the network.

Common Pitfalls

  • Assuming federated learning is automatically private (updates can leak data without secure aggregation or differential privacy).

  • Testing only with IID splits, which hides the hard part.

  • Ignoring client availability and bandwidth.

Interview Tips

  • Explain the FedAvg round and why raw data never leaves clients.

  • Name non-IID data and privacy leakage as the key challenges, and Flower as the Python tool.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 100 PythonML answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All Machine Learning & Data Science subjects