TensorFlow is an open-source library for numerical computing and machine learning, developed by the Google Brain team and released in November 2015. It provides tensors, automatic differentiation, hardware acceleration on CPUs, GPUs and TPUs, and a large ecosystem for training and deploying models, with Keras as its high-level API. Its strength is production deployment across servers, mobile and browsers; the trade-off is a larger, more complex ecosystem than some research-focused alternatives.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Key Facts
Origin: successor to Google’s internal system DistBelief; now maintained by Google with a large open-source community under the Apache 2.0 license.
TensorFlow 2.0 (2019): made eager execution the default and Keras the main API.
TensorFlow 2.16 (2024): Keras 3 became the default Keras version.
What It Provides
Component
Purpose
Tensors and ops
N-dimensional arrays and math on CPU/GPU/TPU
tf.GradientTape
Automatic differentiation
tf.function
Compiles Python code into optimized graphs
Keras
High-level API to build and train models
tf.data
Fast, scalable input pipelines
tf.distribute
Multi-GPU, multi-machine and TPU training
TensorBoard
Visualization of training metrics, graphs and profiles
TensorFlow Serving
Serving models over REST/gRPC
TensorFlow Lite (LiteRT)
Mobile and edge inference
TensorFlow.js
Models in the browser and Node.js
TFX
End-to-end production ML pipelines
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
print(tf.__version__, "| GPUs:", tf.config.list_physical_devices("GPU"))
x = np.random.rand(1000, 4).astype("float32")
y = (x.sum(axis=1) > 2).astype("float32")
model = keras.Sequential([keras.Input(shape=(4,)), keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
history = model.fit(x, y, epochs=10, batch_size=32, validation_split=0.2, verbose=0)
print("validation accuracy:", round(history.history["val_accuracy"][-1], 3))
Common Uses
Image classification, detection and segmentation.
NLP and recommendation systems.
Time series forecasting.
On-device ML in mobile apps.
Interview Tips
Mention both Google Brain (origin) and the ecosystem (Keras, TFX, Lite, Serving, TensorBoard).
Be ready to compare with PyTorch: TensorFlow is strong in deployment tooling and TPUs; PyTorch is dominant in research.
2
What are the main features of TensorFlow?
KnownRevise
TensorFlow’s main features are tensor computation with hardware acceleration, automatic differentiation, a high-level Keras API, graph compilation for speed, scalable data pipelines, distributed training, and a complete deployment ecosystem covering servers, mobile, browsers and production pipelines. Together they let one framework cover research prototypes through to production. The trade-off is a broad, sometimes complex ecosystem with older (1.x) material still around that no longer applies.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Feature Overview
Feature
What it gives you
Eager execution (default)
Operations run immediately, like NumPy; easy debugging
tf.function
Turns Python functions into graphs for speed and export
Automatic differentiation
tf.GradientTape records operations and computes gradients
Keras API
Sequential, functional and subclassing models; compile and fit
import tensorflow as tf
w = tf.Variable(3.0) # trainable state
@tf.function # compiled into a graph on first call
def loss_fn(x):
return (w * x - 6.0) ** 2
with tf.GradientTape() as tape: # automatic differentiation
loss = loss_fn(tf.constant(1.0))
print("loss:", loss.numpy(), "gradient:", tape.gradient(loss, w).numpy()) # 9.0 and -6.0
ds = tf.data.Dataset.range(10).map(lambda i: i * 2).batch(4).prefetch(tf.data.AUTOTUNE)
print([b.numpy().tolist() for b in ds]) # [[0, 2, 4, 6], [8, 10, 12, 14], [16, 18]]
The gradient: loss = (3x - 6)^2 at x = 1 is 9, and d(loss)/dw = 2(3 - 6)(1) = -6.
Common Pitfalls
Following TensorFlow 1.x tutorials (tf.Session, tf.placeholder), which do not apply to TF 2.
Python side effects (like print or appending to lists) inside tf.function, which run only during tracing.
Assuming every feature works the same with Keras 3 multi-backend code; TensorFlow-specific APIs need the TensorFlow backend.
3
What is a computation graph in TensorFlow?
KnownRevise
A computation graph is a data structure in which nodes are operations (such as matrix multiplication or ReLU) and edges are the tensors flowing between them. TensorFlow uses graphs to optimize, parallelize and export computations independently of Python. In TensorFlow 2 you write normal eager Python, and tf.function traces it into a graph when you want graph benefits; the trade-off is that graph code is harder to debug and Python side effects behave differently.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Why Graphs Help
Optimization: Grappler rewrites the graph (constant folding, removing unused nodes, fusing operations); XLA can compile it further.
Speed: less Python overhead, especially for many small operations.
Portability: graphs are saved in SavedModel files and run in TensorFlow Serving, TensorFlow Lite or other languages without Python.
Parallelism and placement: independent operations can run concurrently on different devices.
Graphs in TensorFlow 1.x Versus 2.x
Aspect
TensorFlow 1.x
TensorFlow 2.x
How graphs are built
Explicitly, then run in a tf.Session
Automatically by tracing tf.function code
Default mode
Graph mode
Eager execution
Feeding data
tf.placeholder + feed_dict
Function arguments
Tracing
On the first call, tf.function runs the Python code once, records TensorFlow operations into a ConcreteFunction (a graph), and reuses it for later calls with compatible input shapes and types.
New input shapes or Python values cause retracing (building a new graph).
Code Example
import tensorflow as tf
@tf.function
def dense_relu(x, w, b):
return tf.nn.relu(tf.matmul(x, w) + b)
x = tf.random.normal([2, 3], seed=42)
w = tf.random.normal([3, 4], seed=42)
b = tf.zeros([4])
print(dense_relu(x, w, b).shape) # (2, 4), runs the traced graph
concrete = dense_relu.get_concrete_function(x, w, b)
ops = [op.type for op in concrete.graph.get_operations()]
print(ops) # includes 'MatMul', 'AddV2', 'Relu'
Common Pitfalls
Python print and list appends run only during tracing; use tf.print for values inside graphs.
Calling a tf.function with many different Python scalars or shapes causes retracing and slowdowns; pass tensors or set an input_signature.
Using Python if or for on tensor values: AutoGraph converts many cases to tf.cond and tf.while_loop, but not all Python constructs are supported.
4
What are tensors in TensorFlow?
KnownRevise
Tensors are TensorFlow’s core data structure: multi-dimensional arrays with a single data type (dtype) and a shape, similar to NumPy arrays but able to live on GPUs and TPUs and to take part in automatic differentiation. Scalars, vectors, matrices and higher-dimensional arrays (such as image batches) are all tensors. Regular tf.Tensor objects are immutable; trainable, changeable state is held in tf.Variable.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Key Properties
Property
Meaning
Example
Rank
Number of dimensions
0 scalar, 1 vector, 2 matrix, 4 image batch
Shape
Size of each dimension
(32, 224, 224, 3) for 32 RGB images
dtype
Element type
float32, int64, bool, string
Device
Where it is stored
/CPU:0, /GPU:0
Kinds of Tensors
tf.Tensor: immutable values from tf.constant or operations.
tf.Variable: mutable, trainable state (weights).
tf.RaggedTensor: rows of different lengths (e.g. sentences).
tf.SparseTensor: mostly-zero data stored efficiently.
String tensors: hold byte strings for text pipelines.
dtype mismatches: tf.constant(1) + tf.constant(1.0) fails; TensorFlow does not auto-promote types. Use tf.cast.
Trying to modify a tensor in place: tensors are immutable; use tf.Variable or tf.tensor_scatter_nd_update.
Shape errors: check .shape and use tf.debugging.assert_shapes when building models.
NumPy float64 defaults: NumPy arrays are often float64; cast to float32 for most models.
5
How does TensorFlow differ from other machine learning libraries?
KnownRevise
TensorFlow differs from other machine learning libraries mainly in scope: it is an end-to-end deep learning platform with strong production deployment tools (Serving, Lite, TensorFlow.js, TFX) and first-class TPU support, whereas scikit-learn focuses on classical machine learning and PyTorch is the leading choice for research with a very Python-native feel. In practice, TensorFlow and PyTorch overlap heavily for building neural networks; the choice depends on deployment targets, team experience and ecosystem.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, compared with PyTorch 2.x and scikit-learn 1.5+
Comparison
Aspect
TensorFlow (+ Keras)
PyTorch
scikit-learn
JAX
Main focus
Deep learning, production
Deep learning, research and production
Classical ML (trees, linear, clustering)
High-performance numerical computing, research
Execution
Eager by default, graphs via tf.function
Eager by default, torch.compile for speed
NumPy-based
Functional, compiled with jit (XLA)
High-level API
Keras (compile/fit)
Plain Python loops, Lightning
fit/predict estimators
Flax, Keras 3
Deployment
TF Serving, TF Lite (LiteRT), TF.js, TFX
TorchServe, ExecuTorch, ONNX export
Pickle/joblib, ONNX
Export via XLA, TF
Hardware
CPU, GPU, strong TPU support
CPU, GPU (TPU via XLA)
CPU
CPU, GPU, TPU
Research adoption
Smaller share than before
Dominant
n/a
Growing in research labs
Debugging
Easy in eager mode; harder inside graphs
Very Python-native
Simple
Requires functional style
Where TensorFlow Stands Out
Deployment breadth: the same model can serve in a datacenter, a phone app and a web page.
TPUs: mature support via TPUStrategy.
Production pipelines: TFX covers data validation, transformation, training, evaluation and pushing models.
Keras 3: the same Keras code can also run on JAX or PyTorch backends.
Where Others Stand Out
PyTorch: most new research code and pretrained models; intuitive debugging.
scikit-learn: best for tabular data with classical models and preprocessing pipelines.
XGBoost/LightGBM: usually better than neural networks on tabular data.
Interview Tips
Avoid claiming one framework is strictly better; explain trade-offs honestly.
Mention that ONNX allows moving models between frameworks for inference.
Point out that TensorFlow 1.x criticisms (sessions, static graphs) no longer apply to TensorFlow 2.
6
Which devices can TensorFlow run computations on?
KnownRevise
TensorFlow runs computations on CPUs, NVIDIA GPUs (through CUDA and cuDNN), Google TPUs, and, through TensorFlow Lite (LiteRT), on mobile CPUs, mobile GPUs, DSPs/NPUs and microcontrollers. TensorFlow.js adds browsers (WebGL, WebGPU, WebAssembly) and Node.js. Device placement is mostly automatic: operations run on a GPU when one is available, and you can control placement manually. The trade-off is that each hardware target has its own setup and supported operations.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Supported Hardware
Device
How
Notes
CPU
Default
x86 and ARM; oneDNN optimizations on Intel
NVIDIA GPU
pip install "tensorflow[and-cuda]" on Linux
Needs compatible CUDA/cuDNN; native Windows GPU support ended after TF 2.10 (use WSL2)
Apple silicon GPU
tensorflow-metal plugin
Community/Apple-maintained plugin
Google TPU
tf.distribute.TPUStrategy on Cloud TPU or Colab
Very fast for large matrix workloads
Mobile and edge
TensorFlow Lite (LiteRT) with delegates (GPU, NNAPI, Core ML, Hexagon)
Quantized models; Android, iOS, embedded Linux
Microcontrollers
TensorFlow Lite for Microcontrollers
Kilobytes of memory
Browser / Node.js
TensorFlow.js
WebGL, WebGPU, WASM backends
Device Placement
By default, TensorFlow places an operation on the first GPU if it has a GPU kernel.
tf.device("/CPU:0") or "/GPU:1" forces placement.
tf.debugging.set_log_device_placement(True) logs where each operation runs.
tf.config.experimental.set_memory_growth(gpu, True) stops TensorFlow from reserving all GPU memory at start-up.
Code Example
import tensorflow as tf
print("CPUs:", tf.config.list_physical_devices("CPU"))
gpus = tf.config.list_physical_devices("GPU")
print("GPUs:", gpus)
for gpu in gpus: # must run before any GPU is initialized
tf.config.experimental.set_memory_growth(gpu, True)
with tf.device("/CPU:0"):
a = tf.random.normal([1000, 1000], seed=42)
device = "/GPU:0" if gpus else "/CPU:0"
with tf.device(device):
b = tf.matmul(a, a)
print("result computed on:", b.device)
Common Pitfalls
CUDA/cuDNN version mismatches, which make TensorFlow silently fall back to the CPU; check list_physical_devices("GPU").
Setting memory growth after the GPU has been initialized (raises an error).
Operations without a GPU kernel, which run on the CPU and cause slow host-device copies.
Expecting every model to convert to TensorFlow Lite; unsupported ops need SELECT_TF_OPS or redesign.
7
What is a Session in TensorFlow 1.x and what replaced it?
KnownRevise
A Session in TensorFlow 1.x was the object that executed a computation graph: you first built a static graph of operations, then created a tf.Session and called session.run(...) to evaluate tensors, feeding inputs through placeholders. It managed resources such as variable values and device allocation. TensorFlow 2 removed sessions from the main API: eager execution runs operations immediately, and tf.function provides graph execution when needed.
Read full answerHide full answer
Applies toTensorFlow 1.x (historical) and TensorFlow 2.16+ (Python 3.12)
Role of a Session in TF 1.x
Execute graphs: sess.run(fetches, feed_dict=...) computed only the parts of the graph needed for the requested outputs.
Hold state: variable values lived inside the session; they had to be initialized with tf.global_variables_initializer().
Manage devices: placement on CPUs/GPUs and distributed execution.
Lifecycle: opened and closed explicitly, or used as a context manager.
import tensorflow as tf
# TensorFlow 2: eager execution, no session
a = tf.constant(2.0)
b = tf.constant(3.0)
print((a * b).numpy()) # 6.0, computed immediately
@tf.function # graph execution when you want it
def multiply(x, y):
return x * y
print(multiply(a, b).numpy()) # 6.0
# Legacy TF 1.x style, only through the compatibility module (for old code)
g = tf.Graph()
with g.as_default():
x = tf.compat.v1.placeholder(tf.float32, shape=())
y = x * 3.0
with tf.compat.v1.Session(graph=g) as sess:
print(sess.run(y, feed_dict={x: 2.0})) # 6.0
Migration Tips
Replace sess.run with direct calls or tf.function-decorated functions.
Replace placeholders with function parameters; use input_signature with tf.TensorSpec to fix shapes.
Use tf_upgrade_v2 to convert old scripts automatically as a first step.
Common Pitfalls
Copying TF 1.x tutorials into TF 2 code.
Mixing tf.compat.v1 graph code with Keras 3 models; keep legacy code isolated or migrate it fully.
8
What are the differences between TensorFlow 1.x and TensorFlow 2.x?
KnownRevise
TensorFlow 2.x redesigned the framework around ease of use: eager execution is the default, Keras is the standard high-level API, and graphs are created automatically with tf.function instead of being built and run through sessions. TensorFlow 1.x required building a static graph first, feeding data through placeholders and running it in a tf.Session. TF 2 code is shorter and easier to debug, while keeping graph performance and deployment through tf.function and SavedModel.
Python objects (tf.Variable), tracked by Keras layers
Initialization
global_variables_initializer()
Automatic
Gradients
tf.gradients, optimizers’ minimize on graphs
tf.GradientTape
Control flow
tf.cond, tf.while_loop by hand
Python if/for converted by AutoGraph
Contrib modules
tf.contrib
Removed; moved to separate projects or core
Saving
tf.train.Saver, frozen graphs
SavedModel, .keras files, tf.train.Checkpoint
Estimators
Recommended
Deprecated, removed in TF 2.16
Compatibility
—
tf.compat.v1 for legacy code; tf_upgrade_v2 script
Code Comparison
import tensorflow as tf
# TF 2.x: train a linear model y = 2x + 1 with eager execution and GradientTape
x = tf.constant([[1.0], [2.0], [3.0], [4.0]])
y = 2 * x + 1
w, b = tf.Variable(0.0), tf.Variable(0.0)
opt = tf.keras.optimizers.SGD(learning_rate=0.05)
for step in range(500):
with tf.GradientTape() as tape:
loss = tf.reduce_mean((w * x + b - y) ** 2)
opt.apply_gradients(zip(tape.gradient(loss, [w, b]), [w, b]))
print(round(float(w.numpy()), 2), round(float(b.numpy()), 2)) # close to 2.0 and 1.0
In TF 1.x, the same model needed placeholders for x and y, a minimize operation, a session, variable initialization and sess.run(train_op, feed_dict=...) inside the loop.
Migration Advice
Run tf_upgrade_v2 to rename APIs, then refactor sessions and placeholders into functions.
Replace tf.contrib usage with core APIs or add-ons.
Rewrite Estimator code with Keras models.
Common Pitfalls
Mixing tf.compat.v1 graph code with eager code in the same program.
Assuming TF 2 is slower: wrap hot code in tf.function to get graph performance.
9
How does TensorFlow handle automatic differentiation?
KnownRevise
TensorFlow handles automatic differentiation with tf.GradientTape: while the tape is active, it records the operations applied to watched tensors (trainable tf.Variables are watched automatically), and tape.gradient(target, sources) then replays that record backwards using the chain rule (reverse-mode differentiation) to compute exact gradients. This powers every optimizer step in Keras and in custom training loops. The trade-off is memory: the tape keeps intermediate values until the gradient is computed.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
How It Works
Operations inside with tf.GradientTape() as tape: are recorded.
Each TensorFlow operation has a registered gradient function.
tape.gradient(loss, variables) walks the recorded operations in reverse, multiplying local gradients (chain rule).
The tape is released after one gradient call unless persistent=True.
Reverse mode is efficient for machine learning because there is one scalar loss and many parameters: one backward pass gives all partial derivatives.
Key Features
Feature
Use
Automatic watching
Trainable tf.Variables are tracked; call tape.watch(t) for constant tensors
persistent=True
Compute several gradients from one tape
Nested tapes
Higher-order derivatives (e.g. second derivatives, gradient penalties)
tape.jacobian, tape.batch_jacobian
Full Jacobian matrices
tf.stop_gradient
Treat a value as constant
@tf.custom_gradient
Define your own gradient (e.g. for numerical stability)
Code Example
import tensorflow as tf
# 1. Gradient of a simple function: f(x) = x^3 + 2x, f'(x) = 3x^2 + 2
x = tf.constant(2.0)
with tf.GradientTape() as tape:
tape.watch(x) # constants are not watched automatically
y = x ** 3 + 2 * x
print("dy/dx:", tape.gradient(y, x).numpy()) # 14.0
# 2. Second derivative with nested tapes: f''(x) = 6x
with tf.GradientTape() as outer:
outer.watch(x)
with tf.GradientTape() as inner:
inner.watch(x)
y = x ** 3 + 2 * x
dy = inner.gradient(y, x)
print("d2y/dx2:", outer.gradient(dy, x).numpy()) # 12.0
# 3. Gradients for model weights
w = tf.Variable([[1.0], [2.0]])
b = tf.Variable(0.5)
inputs = tf.constant([[1.0, 3.0]])
with tf.GradientTape() as tape:
loss = tf.reduce_sum((tf.matmul(inputs, w) + b - 10.0) ** 2)
dw, db = tape.gradient(loss, [w, b])
print(dw.numpy().ravel(), db.numpy()) # [-5. -15.] -5.0
For the last case: prediction = 11 + 32 + 0.5 = 7.5, error = -2.5, so dL/db = 2 * (-2.5) = -5 and dL/dw = -5 times the inputs, giving [-5, -15].
Common Pitfalls
None gradients: the variable was not used in the computation, the path went through NumPy (which breaks the tape), or a non-differentiable op such as tf.argmax or an integer cast was used.
Computing outside the tape: the loss must be calculated inside the with block.
Integer tensors: gradients only flow through floating-point tensors.
Memory: persistent tapes keep resources until deleted (del tape).
10
What is a placeholder in TensorFlow and what replaced it?
KnownRevise
A placeholder (tf.placeholder) was a TensorFlow 1.x graph node that stood in for input data to be supplied later: you declared its type and shape when building the static graph and fed real values through feed_dict when running it in a tf.Session. Placeholders no longer exist in the TensorFlow 2 API: data is passed as ordinary function arguments, tf.function input signatures (tf.TensorSpec) fix shapes and types when needed, and keras.Input defines model inputs.
Read full answerHide full answer
Applies toTensorFlow 1.x (historical) and TensorFlow 2.16+ (Python 3.12)
How Placeholders Worked in TF 1.x
Declared with a dtype and a (possibly partially unknown) shape, e.g. shape=(None, 784) for any batch size.
Fed at run time: sess.run(output, feed_dict={x: batch}).
Separated graph definition from data, which made graphs reusable but code verbose.
import keras
import numpy as np
import tensorflow as tf
# TF 2: an input signature plays the role of a placeholder's shape and dtype
@tf.function(input_signature=[tf.TensorSpec(shape=[None, 3], dtype=tf.float32)])
def normalize(batch):
return (batch - tf.reduce_mean(batch, axis=0)) / (tf.math.reduce_std(batch, axis=0) + 1e-8)
print(normalize(tf.random.normal([4, 3], seed=42)).shape) # (4, 3); any batch size works
# Keras models declare inputs with keras.Input
inputs = keras.Input(shape=(3,), name="features") # batch dimension is implicit
model = keras.Model(inputs, keras.layers.Dense(1)(inputs))
print(model.predict(np.ones((2, 3), dtype="float32"), verbose=0).shape) # (2, 1)
# Legacy TF 1.x style, only through tf.compat.v1 for old code
g = tf.Graph()
with g.as_default():
x = tf.compat.v1.placeholder(tf.float32, shape=(None, 3))
total = tf.reduce_sum(x)
with tf.compat.v1.Session(graph=g) as sess:
print(sess.run(total, feed_dict={x: np.ones((2, 3))})) # 6.0
Common Pitfalls
Using tf.placeholder directly in TF 2 raises AttributeError; it only exists under tf.compat.v1.
Over-constraining input_signature shapes; use None for dimensions that vary (batch size, sequence length).
Converting old code mechanically while keeping session-based logic; refactor to functions and tf.data.
11
What is TensorFlow Lite and where is it used?
KnownRevise
TensorFlow Lite (now branded LiteRT) is TensorFlow’s runtime for running machine learning models on mobile phones, embedded devices and microcontrollers. A trained model is converted to a compact .tflite flatbuffer, often quantized to 8-bit integers, and executed by a small interpreter that can use hardware accelerators through delegates. It enables fast, private, offline inference on device; the trade-off is that not every TensorFlow operation is supported and quantization can reduce accuracy slightly.
Read full answerHide full answer
Applies toTensorFlow 2.16+ and TensorFlow Lite / LiteRT (Python 3.12)
Workflow
Train a model in TensorFlow/Keras.
Convert with tf.lite.TFLiteConverter.
Optimize: post-training quantization (dynamic range, float16, full integer) or quantization-aware training.
Deploy the .tflite file in an Android, iOS, embedded Linux or microcontroller app.
Run with the interpreter, optionally with delegates (GPU, NNAPI on Android, Core ML on iOS, Hexagon DSP, Edge TPU).
Real-time segmentation, pose estimation, face effects
IoT and edge devices
Raspberry Pi, Coral Edge TPU, industrial sensors
Microcontrollers
Keyword spotting (“wake words”), gesture detection (TensorFlow Lite for Microcontrollers)
Privacy-sensitive features
Health, keyboard prediction, where data stays on the device
Code Example
import keras
import numpy as np
import tensorflow as tf
model = keras.Sequential([keras.Input(shape=(28, 28)), keras.layers.Flatten(),
keras.layers.Dense(64, activation="relu"), keras.layers.Dense(10)])
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT] # dynamic-range quantization
tflite_bytes = converter.convert()
with open("model.tflite", "wb") as f:
f.write(tflite_bytes)
print(f"TFLite model size: {len(tflite_bytes) / 1024:.1f} KB")
interpreter = tf.lite.Interpreter(model_content=tflite_bytes)
interpreter.allocate_tensors()
inp, out = interpreter.get_input_details()[0], interpreter.get_output_details()[0]
interpreter.set_tensor(inp["index"], np.random.rand(1, 28, 28).astype(np.float32))
interpreter.invoke()
print("output shape:", interpreter.get_tensor(out["index"]).shape) # (1, 10)
Benefits and Limitations
Benefits: low latency, works offline, keeps data private, low power use, small binary.
Limitations: subset of operations (use SELECT_TF_OPS for others at a size cost), limited on-device training, accuracy loss from aggressive quantization.
Common Pitfalls
Not testing the converted model’s accuracy against the original.
Full-integer quantization without a representative dataset for calibration.
Assuming a delegate supports every layer; unsupported ops fall back to the CPU.
12
What is a tf.Variable in TensorFlow and why is it important?
KnownRevise
A tf.Variable is a tensor whose value can change during a program and persists across calls, which makes it the way TensorFlow stores model state such as weights, biases and optimizer statistics. Unlike an immutable tf.Tensor, a variable can be updated in place with assign, assign_add or assign_sub, is watched automatically by tf.GradientTape, and is saved in checkpoints. Without variables, models could not learn, because training is the repeated updating of these values.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Key Properties
Property
Description
Mutable
Updated in place with assign* methods
Persistent
Keeps its value between function calls and tf.function executions
trainable
If True (default), tapes watch it and Keras optimizers update it
Device placement
Stored on CPU or GPU like tensors
Checkpointing
Saved and restored with tf.train.Checkpoint and Keras saving
Shape and dtype
Fixed when created (unless shape=tf.TensorShape(None) is given)
Variable Versus Tensor
Aspect
tf.Tensor
tf.Variable
Mutability
Immutable
Mutable
Typical use
Inputs, intermediate results
Weights, counters, state
Gradient tracking
Only if watched
Automatic when trainable
Saved in checkpoints
No
Yes
Code Example
import tensorflow as tf
w = tf.Variable(tf.random.normal([3, 2], seed=42), name="weights")
b = tf.Variable(tf.zeros([2]), name="bias")
step = tf.Variable(0, trainable=False, dtype=tf.int64) # non-trainable state
x = tf.constant([[1.0, 2.0, 3.0]])
target = tf.constant([[1.0, 0.0]])
optimizer = tf.keras.optimizers.SGD(learning_rate=0.1)
for _ in range(3):
with tf.GradientTape() as tape:
loss = tf.reduce_mean((tf.matmul(x, w) + b - target) ** 2)
grads = tape.gradient(loss, [w, b]) # trainable variables are watched
optimizer.apply_gradients(zip(grads, [w, b])) # updates variables in place
step.assign_add(1)
print(f"step {step.numpy()}: loss={loss.numpy():.4f}")
ckpt = tf.train.Checkpoint(w=w, b=b, step=step)
path = ckpt.save("./ckpt/model") # variables persist to disk
print("saved to", path)
In Keras, layers create their variables automatically (layer.kernel, layer.bias), and model.trainable_variables lists them.
Common Pitfalls
Creating variables inside a tf.function on every call; create them once outside (or only on the first call).
Replacing a variable with a new tensor (w = w + 1) instead of w.assign_add(1); the first creates a new tensor and breaks tracking.
Forgetting trainable=False for counters or statistics that should not receive gradients.
13
Which data types does TensorFlow support?
KnownRevise
TensorFlow supports a range of data types (dtypes) for tensors: floating-point (float16, bfloat16, float32, float64), signed and unsigned integers (int8 to int64, uint8 to uint64), bool, string (byte strings), complex numbers (complex64, complex128) and quantized types used in model compression. Every tensor has exactly one dtype, and TensorFlow does not automatically mix types in operations, so you convert with tf.cast. Choosing the right dtype affects memory, speed and numerical stability.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Common Data Types
Category
dtypes
Typical use
Floating point
tf.float32 (default for floats)
Model weights and activations
tf.float16, tf.bfloat16
Mixed-precision training on GPUs/TPUs
tf.float64
High-precision scientific computing
Signed integers
tf.int8, tf.int16, tf.int32 (default for Python ints), tf.int64
Indices, counts, labels, quantized weights
Unsigned integers
tf.uint8, tf.uint16, tf.uint32, tf.uint64
Image pixels (uint8)
Boolean
tf.bool
Masks and conditions
String
tf.string
Raw text, file paths, serialized records (variable-length bytes)
Models: float32 by default; mixed precision (float16 or bfloat16 compute with float32 weights) for speed on modern accelerators.
Labels: int32 or int64 for sparse categorical losses.
Images: store as uint8, convert to float32 when training.
Text: tf.string in input pipelines, converted to integer token IDs.
Common Pitfalls
NumPy float64 arrays creating float64 tensors and mismatches with float32 weights.
Integer division surprises; cast to float first.
float16 overflow and underflow; use loss scaling in mixed precision (Keras does it automatically).
14
How do you build a neural network in TensorFlow?
KnownRevise
Building a neural network in TensorFlow is usually done with Keras, its high-level API: you define the layers (with the Sequential API, the functional API or by subclassing keras.Model), compile the model with an optimizer, a loss and metrics, train it with fit on your data, and evaluate it on held-out data. For full control you can also write a custom training loop with tf.GradientTape. Keras keeps simple models short while still allowing complex architectures.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Steps
Prepare data: split into train, validation and test sets; scale features; build a tf.data pipeline for large data.
Define the architecture: input shape, hidden layers with activations, output layer matching the task.
Compile: choose optimizer (e.g. Adam), loss (e.g. sparse categorical cross-entropy) and metrics.
Train: fit with validation data and callbacks such as EarlyStopping.
Evaluate and predict: evaluate on the test set; predict for new data.
Save: model.save("model.keras").
Choosing the Output Layer
Task
Output layer
Loss
Binary classification
Dense(1, activation="sigmoid")
binary_crossentropy
Multi-class (integer labels)
Dense(k) with softmax, or logits + from_logits=True
sparse_categorical_crossentropy
Regression
Dense(1) (linear)
mse or mae
Code Example
import keras
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
keras.utils.set_random_seed(42)
X, y = load_digits(return_X_y=True)
X = (X / 16.0).astype("float32") # scale pixel values to [0, 1]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
model = keras.Sequential([
keras.Input(shape=(64,)),
keras.layers.Dense(128, activation="relu"),
keras.layers.Dropout(0.2),
keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer=keras.optimizers.Adam(1e-3),
loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(X_train, y_train, validation_split=0.1, epochs=30, batch_size=32, verbose=0,
callbacks=[keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True)])
loss, acc = model.evaluate(X_test, y_test, verbose=0)
print(f"test accuracy: {acc:.3f}") # around 0.97
model.summary()
Common Pitfalls
Mismatched loss and labels (one-hot labels with a sparse loss, or the reverse).
Softmax output combined with from_logits=True, which applies softmax twice.
Unscaled inputs, which slow or prevent learning.
Evaluating on the training data, which hides overfitting.
15
What does compiling a model mean in TensorFlow Keras?
KnownRevise
Compiling a Keras model with model.compile(...) configures how it will be trained: which optimizer updates the weights, which loss function is minimized, which metrics are reported, and options such as JIT compilation or the number of steps per execution. It does not train the model or change its weights. Compilation must happen before fit or evaluate, and you must compile again after changing which layers are trainable.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
What compile Sets
Argument
Purpose
Examples
optimizer
Algorithm that updates the weights
"adam", keras.optimizers.SGD(0.01, momentum=0.9)
loss
Quantity to minimize
"binary_crossentropy", "sparse_categorical_crossentropy", "mse", custom function
fit builds a training step (forward pass, loss, gradients, optimizer update) and compiles it as a tf.function graph (or XLA program).
Optimizer state (e.g. Adam’s moment estimates) is created when training starts.
Code Example
import keras
import numpy as np
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(8,)), keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(1)]) # outputs logits (no sigmoid)
model.compile(
optimizer=keras.optimizers.Adam(learning_rate=1e-3),
loss=keras.losses.BinaryCrossentropy(from_logits=True), # matches the logit output
metrics=[keras.metrics.BinaryAccuracy(threshold=0.0), # threshold 0 on logits = 0.5 on probabilities
keras.metrics.AUC(from_logits=True)],
)
X = np.random.rand(500, 8).astype("float32")
y = (X[:, 0] + X[:, 1] > 1).astype("float32")
history = model.fit(X, y, epochs=5, verbose=0)
print({k: round(v[-1], 3) for k, v in history.history.items()})
model.compile(optimizer="adam", loss=keras.losses.BinaryCrossentropy(from_logits=True),
run_eagerly=True) # recompile to debug step by step
Common Pitfalls
Mismatched output activation and loss (from_logits set wrongly).
Forgetting to recompile after setting layer.trainable = False/True; the change is not applied to training.
Expecting metrics to influence training; only the loss does.
Using run_eagerly=True in production; it is much slower.
16
How do TensorFlow optimizers work, and which ones are common?
KnownRevise
Optimizers in TensorFlow decide how model weights are updated from their gradients to reduce the loss. Each step they take the gradients computed by tf.GradientTape (or by fit), apply an update rule that may use momentum or adaptive per-parameter learning rates, and write the new values into the model’s variables. Common choices are SGD with momentum, Adam, AdamW, RMSprop and Adagrad; the choice affects training speed, stability and how well the model generalizes.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Common Optimizers
Optimizer
Update idea
Typical use
SGD
w = w - lr * g
Simple; with momentum often generalizes well (vision)
SGD + momentum / Nesterov
Accumulates a velocity of past gradients
CNN training with learning rate schedules
RMSprop
Divides by a moving average of squared gradients
RNNs, noisy problems
Adagrad
Per-parameter rates shrink with accumulated squared gradients
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(1000, 10).astype("float32")
y = (X @ np.arange(10, dtype="float32") + np.random.normal(0, 0.1, 1000)).astype("float32")
schedule = keras.optimizers.schedules.CosineDecay(initial_learning_rate=1e-2, decay_steps=1000)
optimizers = {
"sgd_momentum": keras.optimizers.SGD(learning_rate=1e-2, momentum=0.9),
"adam": keras.optimizers.Adam(learning_rate=1e-2),
"adamw_cosine": keras.optimizers.AdamW(learning_rate=schedule, weight_decay=1e-4),
}
for name, opt in optimizers.items():
model = keras.Sequential([keras.Input(shape=(10,)), keras.layers.Dense(32, activation="relu"),
keras.layers.Dense(1)])
model.compile(optimizer=opt, loss="mse")
hist = model.fit(X, y, epochs=20, batch_size=32, verbose=0)
print(f"{name}: final training MSE = {hist.history['loss'][-1]:.4f}")
# Manual use in a custom loop
w = tf.Variable(5.0)
opt = keras.optimizers.Adam(learning_rate=0.1)
for _ in range(100):
with tf.GradientTape() as tape:
loss = (w - 2.0) ** 2
opt.apply_gradients([(tape.gradient(loss, w), w)])
print("w after Adam:", round(float(w.numpy()), 3)) # close to 2.0
Common Pitfalls
A learning rate that is too high (divergence, NaN loss) or too low (very slow training); the learning rate matters more than the optimizer choice.
Using Adam’s L2 regularization when you want decoupled weight decay (use AdamW).
Reusing one optimizer instance for two different models; each model needs its own optimizer state.
17
How do you do batch processing in TensorFlow?
KnownRevise
Batch processing in TensorFlow means grouping several examples into one tensor so the model processes them together, which uses the GPU efficiently and gives stable gradient estimates. You create batches with tf.data.Dataset.batch (or pass batch_size to model.fit), usually after shuffling and before prefetching. The batch size is a trade-off: larger batches are faster per example but use more memory and can generalize slightly worse without learning-rate adjustments.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Key Tools
Tool
Purpose
Dataset.batch(n, drop_remainder=False)
Stack n consecutive elements into one batch
Dataset.padded_batch(n)
Batch variable-length sequences by padding
Dataset.ragged_batch(n)
Batch into ragged tensors without padding
Dataset.shuffle(buffer)
Randomize order before batching
Dataset.prefetch(tf.data.AUTOTUNE)
Prepare the next batch while the current one trains
Dataset.unbatch() / rebatch()
Split or re-group existing batches
model.fit(x, y, batch_size=32)
Batching for in-memory NumPy data
model.predict(x, batch_size=256)
Batched inference
Code Example
import numpy as np
import tensorflow as tf
tf.random.set_seed(42)
features = np.random.rand(1000, 8).astype("float32")
labels = np.random.randint(0, 2, size=1000)
ds = (tf.data.Dataset.from_tensor_slices((features, labels))
.shuffle(buffer_size=1000, seed=42) # shuffle individual examples first
.batch(64) # then group into batches
.prefetch(tf.data.AUTOTUNE))
for x_batch, y_batch in ds.take(1):
print(x_batch.shape, y_batch.shape) # (64, 8) (64,)
print("batches per epoch:", ds.cardinality().numpy()) # 16 (the last batch has 40 examples)
# Variable-length sequences: pad within each batch
seqs = tf.data.Dataset.from_generator(lambda: ([1] * n for n in [3, 5, 2, 4]),
output_signature=tf.TensorSpec([None], tf.int32))
for padded in seqs.padded_batch(2):
print(padded.numpy()) # [[1 1 1 0 0] [1 1 1 1 1]], then [[1 1 0 0] [1 1 1 1]]
Choosing a Batch Size
Start with 32-256 for most models; increase until GPU memory is nearly full if speed matters.
With much larger batches, scale the learning rate up and use warm-up.
For multi-GPU training, the global batch is split across replicas.
Use drop_remainder=True when every batch must have the same shape (TPUs, some custom layers).
Common Pitfalls
Batching before shuffling, which shuffles only whole batches.
Batch normalization with very small batches, which gives noisy statistics.
Forgetting batch dimensions when predicting a single example (x[None, ...] or tf.expand_dims).
18
What role do loss functions play in TensorFlow, and which are common?
KnownRevise
A loss function measures how far a model’s predictions are from the true targets and turns that into a single number that training minimizes. Its gradients drive every weight update, so the loss defines what “better” means for the model. TensorFlow/Keras provides standard losses for regression, classification and other tasks, and you can write custom ones; choosing a loss that matches the output layer and the business goal is essential.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Common Losses
Task
Loss
Formula / idea
Regression
MeanSquaredError
mean((y - y_hat)^2); penalizes large errors
Regression (robust)
MeanAbsoluteError, Huber
Less sensitive to outliers
Binary classification
BinaryCrossentropy
-[y log p + (1 - y) log(1 - p)]
Multi-class, one-hot labels
CategoricalCrossentropy
-sum_k y_k log p_k
Multi-class, integer labels
SparseCategoricalCrossentropy
Same, without one-hot encoding
Imbalanced classification
BinaryFocalCrossentropy
Down-weights easy examples
Distribution matching
KLDivergence
Distillation, VAEs
Similarity learning
CosineSimilarity
Embedding alignment
Quantiles / intervals
Custom pinball loss
Probabilistic forecasting
Logits Versus Probabilities
If the last layer has no activation (it outputs logits), set from_logits=True; this is more numerically stable.
If the last layer uses sigmoid or softmax, keep from_logits=False (the default).
Code Example
import keras
import numpy as np
import tensorflow as tf
y_true = tf.constant([0, 2, 1])
logits = tf.constant([[2.0, 0.5, 0.1], [0.2, 0.3, 3.0], [0.1, 2.5, 0.3]])
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
print("cross-entropy:", round(float(loss_fn(y_true, logits)), 4))
print("MSE:", float(keras.losses.MeanSquaredError()([1.0, 2.0], [1.5, 2.5]))) # 0.25
print("Huber:", float(keras.losses.Huber(delta=1.0)([0.0], [3.0]))) # 2.5 (linear beyond delta)
def pinball_loss(q):
"""Quantile loss for predicting the q-th quantile."""
def loss(y, y_hat):
e = y - y_hat
return tf.reduce_mean(tf.maximum(q * e, (q - 1) * e))
return loss
model = keras.Sequential([keras.Input(shape=(4,)), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss=pinball_loss(0.9)) # custom loss
model.fit(np.random.rand(64, 4), np.random.rand(64, 1), epochs=1, verbose=0)
Using CategoricalCrossentropy with integer labels (or Sparse... with one-hot labels).
Double softmax: softmax output plus from_logits=True.
Optimizing MSE when the business cares about relative or asymmetric errors.
Custom losses that use NumPy or Python control flow on tensor values, which breaks gradients or graph tracing.
19
How do you implement a custom layer in TensorFlow Keras?
KnownRevise
A custom layer in Keras is a subclass of keras.layers.Layer that defines its own weights and computation. You create weights in build (called once the input shape is known) with self.add_weight, implement the forward pass in call, and add get_config so the layer can be saved and reloaded. Custom layers let you add operations Keras does not provide while keeping the benefits of Keras models: tracking of weights, saving, fit and serialization.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
The Structure
Method
Purpose
__init__
Store configuration (units, activation); create sublayers
build(input_shape)
Create weights with self.add_weight once the input size is known
call(inputs, training=None)
Forward computation; training switches behaviour such as dropout
get_config()
Return constructor arguments for saving
compute_output_shape (optional)
Needed only in some cases for shape inference
Code Example
import keras
import numpy as np
from keras import ops
@keras.saving.register_keras_serializable(package="custom")
class ScaledDense(keras.layers.Layer):
"""Dense layer with a learnable output scale per unit and optional activation."""
def __init__(self, units, activation=None, **kwargs):
super().__init__(**kwargs)
self.units = units
self.activation = keras.activations.get(activation)
def build(self, input_shape):
self.kernel = self.add_weight(shape=(input_shape[-1], self.units),
initializer="glorot_uniform", name="kernel")
self.bias = self.add_weight(shape=(self.units,), initializer="zeros", name="bias")
self.scale = self.add_weight(shape=(self.units,), initializer="ones", name="scale")
def call(self, inputs):
out = (ops.matmul(inputs, self.kernel) + self.bias) * self.scale
return self.activation(out)
def get_config(self):
config = super().get_config()
config.update({"units": self.units, "activation": keras.activations.serialize(self.activation)})
return config
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(4,)), ScaledDense(8, activation="relu"), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss="mse")
model.fit(np.random.rand(128, 4), np.random.rand(128, 1), epochs=2, verbose=0)
print([w.name for w in model.layers[0].weights]) # ['kernel', 'bias', 'scale']
model.save("custom_layer_model.keras")
reloaded = keras.models.load_model("custom_layer_model.keras") # works thanks to registration and get_config
print(np.allclose(model.predict(np.ones((1, 4)), verbose=0), reloaded.predict(np.ones((1, 4)), verbose=0)))
Using keras.ops keeps the layer backend-agnostic in Keras 3; with the TensorFlow backend you could also use tf operations.
Common Pitfalls
Creating weights in call, which creates new weights on every call.
Forgetting get_config or registration, which makes saving or loading fail.
Ignoring the training argument for layers that behave differently during training (dropout, noise).
Using NumPy inside call, which breaks gradients and graph compilation.
20
What is the difference between the Sequential and Functional APIs in Keras?
KnownRevise
The Sequential API builds a model as a simple linear stack of layers, each feeding the next, while the Functional API builds a model as a graph by calling layers on tensors, so it supports multiple inputs and outputs, shared layers, skip connections and branches. Sequential is the quickest way to define straightforward models; the Functional API is needed as soon as the architecture is not a single straight line. A third option, subclassing keras.Model, gives full Python flexibility.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Comparison
Aspect
Sequential
Functional
Subclassing
Structure
Linear stack
Directed acyclic graph of layers
Arbitrary Python code in call
Multiple inputs/outputs
No
Yes
Yes
Shared layers, skip connections
No
Yes
Yes
Model summary and plotting
Yes
Yes
Limited (graph not known in advance)
Shape checks when building
Yes
Yes
At call time
Serialization
Easy
Easy
Needs get_config
Best for
Simple feed-forward or CNN stacks
Most real architectures (ResNets, multi-task models)
Research models with dynamic behaviour
Code Example
import keras
# Sequential: a straight stack
seq = keras.Sequential([
keras.Input(shape=(32,)),
keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(1, activation="sigmoid"),
])
# Functional: two inputs, a skip connection and two outputs
numeric = keras.Input(shape=(32,), name="numeric")
text_ids = keras.Input(shape=(20,), dtype="int32", name="text_ids")
x = keras.layers.Dense(64, activation="relu")(numeric)
x = keras.layers.Add()([x, keras.layers.Dense(64)(numeric)]) # skip connection
t = keras.layers.Embedding(input_dim=5000, output_dim=16)(text_ids)
t = keras.layers.GlobalAveragePooling1D()(t)
merged = keras.layers.Concatenate()([x, t])
churn = keras.layers.Dense(1, activation="sigmoid", name="churn")(merged)
spend = keras.layers.Dense(1, name="spend")(merged)
func = keras.Model(inputs=[numeric, text_ids], outputs=[churn, spend])
func.compile(optimizer="adam",
loss={"churn": "binary_crossentropy", "spend": "mse"},
loss_weights={"churn": 1.0, "spend": 0.2})
print(len(seq.layers), "layers in Sequential |", len(func.inputs), "inputs,", len(func.outputs), "outputs in Functional")
When to Use Which
Sequential: prototypes, MLPs, simple CNNs.
Functional: anything with branches, multiple inputs/outputs, residual connections, or reused layers; also convenient for extracting intermediate outputs (keras.Model(model.input, model.get_layer("x").output)).
Subclassing: custom training logic or control flow that depends on inputs.
Common Pitfalls
Trying to add a second input to a Sequential model.
Reusing a layer instance by accident, which shares its weights.
Forgetting the Input layer, so the model is not built until it sees data.
21
How does TensorFlow support regularization to prevent overfitting?
KnownRevise
TensorFlow (through Keras) supports regularization with weight penalties (L1, L2) attached to layers, dropout layers, batch and layer normalization, data augmentation layers, early stopping, label smoothing and optimizers with decoupled weight decay. These techniques limit how closely the model can fit noise in the training data. Each adds a hyperparameter to tune, and too much regularization causes underfitting.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Techniques
Technique
Keras API
Effect
L2 weight penalty
kernel_regularizer=keras.regularizers.L2(1e-4)
Adds lambda * sum(w^2) to the loss; smaller weights
L1 weight penalty
keras.regularizers.L1(1e-5)
Adds ‘lambda * sum(
w
)‘; drives weights to zero
Decoupled weight decay
keras.optimizers.AdamW(weight_decay=1e-4)
Shrinks weights each step, independent of adaptive rates
Dropout
keras.layers.Dropout(0.3)
Randomly zeroes activations in training
Spatial dropout
SpatialDropout1D/2D
Drops whole feature maps in CNNs
Batch / layer normalization
BatchNormalization, LayerNormalization
Stabilizes training; mild regularizing effect
Data augmentation
RandomFlip, RandomRotation, RandomZoom
More varied training data
Early stopping
keras.callbacks.EarlyStopping
Stops when validation loss stops improving
Label smoothing
CategoricalCrossentropy(label_smoothing=0.1)
Prevents over-confident predictions
Gaussian noise
keras.layers.GaussianNoise(0.1)
Noise on inputs or activations
Code Example
import keras
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
keras.utils.set_random_seed(42)
X, y = make_classification(n_samples=600, n_features=50, n_informative=8, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X.astype("float32"), y, test_size=0.3, random_state=42)
def build(regularized):
reg = keras.regularizers.L2(1e-3) if regularized else None
layers = [keras.Input(shape=(50,)),
keras.layers.Dense(256, activation="relu", kernel_regularizer=reg)]
if regularized:
layers.append(keras.layers.Dropout(0.4))
layers += [keras.layers.Dense(256, activation="relu", kernel_regularizer=reg),
keras.layers.Dense(1, activation="sigmoid")]
model = keras.Sequential(layers)
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
return model
for regularized in (False, True):
callbacks = [keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True)] if regularized else []
hist = build(regularized).fit(X_train, y_train, validation_data=(X_val, y_val), epochs=100,
batch_size=32, verbose=0, callbacks=callbacks)
print(f"regularized={regularized}: train acc={hist.history['accuracy'][-1]:.3f}, "
f"best val acc={max(hist.history['val_accuracy']):.3f}")
Common Pitfalls
Applying dropout or augmentation at inference: Keras disables them automatically in predict, but custom code must respect the training flag.
Combining L2 regularization with Adam and expecting it to equal weight decay; use AdamW for decoupled decay.
Tuning regularization on the test set.
22
How do you save and restore models in TensorFlow?
KnownRevise
Saving and restoring models in TensorFlow can be done at three levels: the native Keras format (model.save("model.keras"), which stores architecture, weights and optimizer state in one file), weights only (model.save_weights and load_weights), and the SavedModel format (model.export(...) or tf.saved_model.save), which stores a language-neutral graph for TensorFlow Serving, TensorFlow Lite and other runtimes. Training checkpoints with tf.train.Checkpoint or the ModelCheckpoint callback let you resume interrupted training.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
In Keras 3, model.save requires a .keras (or legacy .h5) path; exporting a SavedModel directory uses model.export.
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X, y = np.random.rand(200, 5).astype("float32"), np.random.rand(200, 1).astype("float32")
model = keras.Sequential([keras.Input(shape=(5,)), keras.layers.Dense(16, activation="relu"), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss="mse")
model.fit(X, y, epochs=3, verbose=0,
callbacks=[keras.callbacks.ModelCheckpoint("best.keras", monitor="loss", save_best_only=True)])
# 1. Full Keras model
model.save("model.keras")
restored = keras.models.load_model("model.keras")
print("same predictions:", np.allclose(model.predict(X[:3], verbose=0), restored.predict(X[:3], verbose=0)))
# 2. Weights only
model.save_weights("model.weights.h5")
clone = keras.models.clone_model(model)
clone.load_weights("model.weights.h5")
# 3. SavedModel for serving
model.export("exported_model") # directory with saved_model.pb + variables/
served = tf.saved_model.load("exported_model")
print(served.serve(tf.constant(X[:2])).shape) # (2, 1)
Common Pitfalls
Custom layers or losses without get_config and registration, which fail to load.
Saving only weights and later changing the architecture code.
Using pickle for Keras models; use the built-in formats.
Loading a model saved with a much newer TensorFlow/Keras version in an older one.
23
How do you use callbacks in TensorFlow Keras?
KnownRevise
Callbacks are objects passed to fit, evaluate or predict that run code at specific points during training, such as the end of each epoch or batch. Keras includes callbacks for early stopping, saving checkpoints, adjusting the learning rate, logging to TensorBoard and stopping on NaN, and you can write your own by subclassing keras.callbacks.Callback. They automate training control without writing a custom loop.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
on_train_begin/end, on_epoch_begin/end, on_train_batch_begin/end, on_test_batch_*, on_predict_batch_*. Inside a callback, self.model gives access to the model and logs contains current metrics.
Code Example
import keras
import numpy as np
class GapMonitor(keras.callbacks.Callback):
"""Warn when validation loss is much higher than training loss (overfitting)."""
def __init__(self, ratio=1.5):
super().__init__()
self.ratio = ratio
def on_epoch_end(self, epoch, logs=None):
if logs["val_loss"] > self.ratio * logs["loss"]:
print(f"epoch {epoch + 1}: val_loss is {logs['val_loss'] / logs['loss']:.1f}x the training loss")
keras.utils.set_random_seed(42)
X = np.random.rand(500, 10).astype("float32")
y = (X.sum(axis=1) + np.random.normal(0, 0.5, 500)).astype("float32")
model = keras.Sequential([keras.Input(shape=(10,)), keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(1)])
model.compile(optimizer="adam", loss="mse")
callbacks = [
keras.callbacks.EarlyStopping(monitor="val_loss", patience=8, restore_best_weights=True),
keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.5, patience=3, min_lr=1e-5),
keras.callbacks.ModelCheckpoint("best_model.keras", monitor="val_loss", save_best_only=True),
keras.callbacks.CSVLogger("training_log.csv"),
GapMonitor(),
]
history = model.fit(X, y, validation_split=0.2, epochs=200, batch_size=32, verbose=0, callbacks=callbacks)
print("stopped after", len(history.history["loss"]), "epochs")
Common Pitfalls
Monitoring a metric that does not exist (e.g. val_accuracy without validation data or that metric); Keras warns and the callback does nothing.
EarlyStopping without restore_best_weights=True, which keeps the weights from the last (worse) epoch.
Heavy work in batch-level hooks, which slows training.
24
Which strategies does TensorFlow offer to handle overfitting during training?
KnownRevise
Overfitting happens when a model learns the training data, including its noise, so well that it performs worse on new data. TensorFlow and Keras offer strategies at every level: more and augmented data, simpler architectures, weight penalties and weight decay, dropout, normalization, early stopping, learning-rate schedules and careful validation. The key is to detect overfitting first, by watching the gap between training and validation metrics, and then apply the least intrusive fix.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Detecting Overfitting
Training loss keeps falling while validation loss rises.
Large gap between training and validation accuracy.
Plot history.history["loss"] and ["val_loss"], or watch them in TensorBoard.
Strategies
Category
Strategy
Keras tool
Data
More data, augmentation
keras.layers.RandomFlip/RandomRotation/RandomZoom, text augmentation
Model capacity
Fewer layers or units; transfer learning with a frozen backbone
Smaller Dense/Conv layers, base.trainable = False
Weight penalties
L1/L2, weight decay
kernel_regularizer, AdamW(weight_decay=...)
Noise
Dropout, Gaussian noise
Dropout, SpatialDropout2D, GaussianNoise
Normalization
Batch/layer normalization
BatchNormalization, LayerNormalization
Training control
Early stopping, learning-rate reduction
EarlyStopping, ReduceLROnPlateau
Targets
Label smoothing
label_smoothing=0.1 in cross-entropy losses
Evaluation
Proper validation and cross-validation
validation_data, k-fold splits outside Keras
Ensembles
Average several models
Multiple models or snapshot ensembles
Code Example: Augmentation + Dropout + Early Stopping for Images
import keras
import numpy as np
keras.utils.set_random_seed(42)
(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data() # downloads the data
x_train, y_train = x_train[:5000], y_train[:5000] # small subset: easy to overfit
augment = keras.Sequential([keras.layers.RandomFlip("horizontal"),
keras.layers.RandomTranslation(0.1, 0.1), keras.layers.RandomZoom(0.1)])
model = keras.Sequential([
keras.Input(shape=(32, 32, 3)),
augment, # active only during training
keras.layers.Rescaling(1.0 / 255),
keras.layers.Conv2D(32, 3, activation="relu", padding="same"),
keras.layers.MaxPooling2D(),
keras.layers.Conv2D(64, 3, activation="relu", padding="same"),
keras.layers.MaxPooling2D(),
keras.layers.GlobalAveragePooling2D(), # fewer parameters than Flatten + Dense
keras.layers.Dropout(0.3),
keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer=keras.optimizers.AdamW(1e-3, weight_decay=1e-4),
loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, validation_split=0.2, epochs=30, batch_size=64, verbose=0,
callbacks=[keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True)])
print("test accuracy:", round(model.evaluate(x_test, y_test, verbose=0)[1], 3))
Common Pitfalls
Adding every technique at once; change one thing at a time and measure validation performance.
Data leakage between training and validation sets, which hides overfitting.
Using the test set for early stopping or tuning.
25
How does TensorFlow integrate with Keras?
KnownRevise
Keras is TensorFlow’s high-level API for building, training and saving models, while TensorFlow supplies the tensors, automatic differentiation, compiled graphs, input pipelines and deployment tools underneath. Since TensorFlow 2.16, installing TensorFlow gives you Keras 3, a multi-backend library that can run on TensorFlow, JAX or PyTorch; with the TensorFlow backend, Keras models train with tf.function graphs, read tf.data datasets and export to SavedModel for TensorFlow Serving and LiteRT.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
How the Pieces Fit
Layer
Provided by
Examples
Model definition
Keras
keras.Sequential, Functional API, keras.layers.*
Training loop
Keras (fit, evaluate, predict)
Callbacks, metrics, losses, optimizers
Execution
TensorFlow backend
Ops, tf.function graphs, XLA via jit_compile
Gradients
TensorFlow
tf.GradientTape (used inside Keras’s training step)
Input pipeline
TensorFlow
tf.data.Dataset passed straight to fit
Scaling
TensorFlow
tf.distribute.MirroredStrategy and other strategies
import keras is the recommended import. In TF 2.16+, tf.keras also points to Keras 3 when it is installed.
Code written for Keras 2 that breaks under Keras 3 can use the tf_keras package with the environment variable TF_USE_LEGACY_KERAS=1 during migration.
The backend is chosen with KERAS_BACKEND (tensorflow by default when TensorFlow is installed). keras.ops functions work on every backend.
Code Example: Keras Model, TensorFlow Pipeline and Custom Training Step
import os
os.environ["KERAS_BACKEND"] = "tensorflow" # must be set before importing keras
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(1000, 20).astype("float32")
y = (X[:, 0] + X[:, 1] > 1).astype("float32")
ds = tf.data.Dataset.from_tensor_slices((X, y)).shuffle(1000, seed=42).batch(32).prefetch(tf.data.AUTOTUNE)
class GradClipModel(keras.Model):
"""Keras model whose training step uses TensorFlow's GradientTape directly."""
def train_step(self, data):
x, y = data
with tf.GradientTape() as tape:
y_pred = self(x, training=True)
loss = self.compute_loss(x=x, y=y, y_pred=y_pred)
grads = tape.gradient(loss, self.trainable_variables)
grads = [tf.clip_by_norm(g, 1.0) for g in grads]
self.optimizer.apply_gradients(zip(grads, self.trainable_variables))
for metric in self.metrics:
if metric.name == "loss":
metric.update_state(loss)
else:
metric.update_state(y, y_pred)
return {m.name: m.result() for m in self.metrics}
inputs = keras.Input(shape=(20,))
h = keras.layers.Dense(32, activation="relu")(inputs)
outputs = keras.layers.Dense(1, activation="sigmoid")(h)
model = GradClipModel(inputs, outputs)
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"], jit_compile=True)
model.fit(ds, epochs=3, verbose=0)
model.export("exported_model") # SavedModel for TF Serving / LiteRT conversion
print(tf.saved_model.load("exported_model").serve(tf.constant(X[:2])).shape) # (2, 1)
Interview Tips
Explain that Keras is the API and TensorFlow is the engine; mention the Keras 3 multi-backend change in TF 2.16.
Know the three ways to build models (Sequential, Functional, subclassing) and how to drop to tf.GradientTape when fit is not flexible enough.
Mention saving formats: .keras for Keras models, model.export for a SavedModel used in deployment.
26
What is eager execution in TensorFlow?
KnownRevise
Eager execution is TensorFlow’s default mode since 2.0: operations run immediately when Python calls them and return concrete values, just like NumPy. This makes code easy to write and debug with print statements, breakpoints and normal Python control flow. For speed and deployment you wrap functions in tf.function, which traces them once into a graph that TensorFlow can optimize, so modern TensorFlow combines eager development with graph execution.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Eager Versus Graph Execution
Aspect
Eager
Graph (tf.function)
When ops run
Immediately
After tracing, as one compiled graph
Results
Concrete EagerTensor values (.numpy() works)
Symbolic tensors during tracing
Debugging
Print, pdb, Python exceptions
Harder; use tf.print and run_functions_eagerly
Python control flow
Normal Python
Converted to tf.cond / tf.while_loop by AutoGraph
Performance
Per-op Python overhead
Op fusion, constant folding, optional XLA
Export
Not directly
Required for SavedModel signatures
Code Example
import tensorflow as tf
print(tf.executing_eagerly()) # True
a = tf.constant([[1.0, 2.0], [3.0, 4.0]])
b = tf.matmul(a, a) # computed right away
print(b.numpy()) # [[ 7. 10.] [15. 22.]]
# Gradients in eager mode
x = tf.Variable(3.0)
with tf.GradientTape() as tape:
y = x ** 2 + 2 * x
print(tape.gradient(y, x).numpy()) # dy/dx = 2x + 2 = 8.0
@tf.function
def scaled_sum(t, factor):
print("tracing") # Python side effect: runs only when tracing
tf.print("running") # graph op: runs on every call
if tf.reduce_sum(t) > 0: # AutoGraph converts this to tf.cond
return tf.reduce_sum(t) * factor
return tf.constant(0.0)
scaled_sum(a, 2.0) # prints "tracing" and "running"
scaled_sum(a, 2.0) # prints only "running": the graph is reused
scaled_sum(a, 3.0) # new Python float value -> retraces
# Debug a tf.function eagerly
tf.config.run_functions_eagerly(True)
scaled_sum(a, 2.0)
tf.config.run_functions_eagerly(False)
Why It Matters
Faster development: errors point at the line that failed, and intermediate values can be inspected.
Keras uses tf.function for fit by default, so you get eager convenience for model code and graph speed for training. Pass run_eagerly=True to compile when debugging.
Common Pitfalls
Passing Python numbers that change on each call to a tf.function, which causes retracing; pass tensors instead or fix an input_signature.
Relying on print or list appends inside a tf.function; they run only during tracing.
Calling .numpy() inside a tf.function, which fails because tensors are symbolic there.
27
What role does tf.data play in TensorFlow?
KnownRevise
tf.data is TensorFlow’s API for building input pipelines: it reads data from memory, files or generators, transforms it, and feeds batches to the model efficiently. Pipelines are chains of Dataset transformations (map, shuffle, batch, prefetch) that run in parallel C++ threads, so data loading overlaps with training and the accelerator does not wait. It handles datasets larger than memory and plugs directly into model.fit and custom training loops.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
List or read files (use interleave to read several at once).
map parsing and cheap deterministic preprocessing in parallel.
cache (in memory or to a file) if the result fits and is reused every epoch.
shuffle with a reasonably large buffer.
map random augmentation (after cache, so it changes each epoch).
batch, then prefetch(tf.data.AUTOTUNE).
Code Example: TFRecord Pipeline
import numpy as np
import tensorflow as tf
rng = np.random.default_rng(42)
# Write a small TFRecord file
with tf.io.TFRecordWriter("data.tfrecord") as writer:
for _ in range(500):
features = rng.random(4).astype("float32")
label = int(features.sum() > 2)
example = tf.train.Example(features=tf.train.Features(feature={
"x": tf.train.Feature(float_list=tf.train.FloatList(value=features)),
"y": tf.train.Feature(int64_list=tf.train.Int64List(value=[label])),
}))
writer.write(example.SerializeToString())
spec = {"x": tf.io.FixedLenFeature([4], tf.float32), "y": tf.io.FixedLenFeature([], tf.int64)}
def parse(record):
ex = tf.io.parse_single_example(record, spec)
return ex["x"], ex["y"]
def augment(x, y):
return x + tf.random.normal(tf.shape(x), stddev=0.01), y
ds = (tf.data.TFRecordDataset(["data.tfrecord"])
.map(parse, num_parallel_calls=tf.data.AUTOTUNE)
.cache()
.shuffle(500, seed=42)
.map(augment, num_parallel_calls=tf.data.AUTOTUNE)
.batch(32)
.prefetch(tf.data.AUTOTUNE))
for xb, yb in ds.take(1):
print(xb.shape, yb.shape) # (32, 4) (32,)
Pass ds directly to model.fit(ds, epochs=...). Do not also pass batch_size, because the dataset is already batched.
Common Pitfalls
Doing preprocessing in Python (from_generator or tf.py_function), which holds the GIL and slows the pipeline.
Putting random augmentation before cache, which freezes it.
A tiny shuffle buffer on data sorted by label.
Forgetting prefetch. Use the TensorBoard profiler’s input-pipeline analysis to find bottlenecks.
28
How do you use Transformers in TensorFlow for sequence modeling?
KnownRevise
Transformers model sequences with self-attention: each position computes weighted combinations of every other position, so the model captures long-range dependencies without recurrence and trains in parallel. In TensorFlow you can build them from Keras layers (MultiHeadAttention, LayerNormalization, Dense) or use KerasHub (formerly KerasNLP), which provides ready-made Transformer encoder and decoder blocks and pretrained models such as BERT and Gemma. Use an encoder for classification or tagging, and a decoder with a causal mask for generation or forecasting.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, KerasHub 0.18+ (Python 3.12)
Building Blocks
Component
Keras / KerasHub
Purpose
Token + position embedding
keras_hub.layers.TokenAndPositionEmbedding or two Embedding layers
Attention in words: for each token, compare its query with every key (dot product divided by the square root of the key size), turn the scores into weights with softmax, and take the weighted average of the values.
Code Example: Transformer Encoder for Sequence Classification
import keras
from keras import layers
keras.utils.set_random_seed(42)
VOCAB, SEQ_LEN, DIM, HEADS = 20000, 200, 64, 4
class PositionalEmbedding(layers.Layer):
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.tokens = layers.Embedding(VOCAB, DIM, mask_zero=True)
self.positions = layers.Embedding(SEQ_LEN, DIM)
def call(self, x):
positions = keras.ops.arange(keras.ops.shape(x)[-1])
return self.tokens(x) + self.positions(positions)
def compute_mask(self, x, mask=None):
return self.tokens.compute_mask(x)
def encoder_block(x):
attn = layers.MultiHeadAttention(num_heads=HEADS, key_dim=DIM // HEADS, dropout=0.1)(x, x)
x = layers.LayerNormalization()(layers.Add()([x, attn]))
ff = layers.Dense(DIM * 4, activation="gelu")(x)
ff = layers.Dropout(0.1)(layers.Dense(DIM)(ff))
return layers.LayerNormalization()(layers.Add()([x, ff]))
inputs = keras.Input(shape=(SEQ_LEN,), dtype="int32")
x = PositionalEmbedding()(inputs)
for _ in range(2):
x = encoder_block(x)
x = layers.GlobalAveragePooling1D()(x)
outputs = layers.Dense(1, activation="sigmoid")(layers.Dropout(0.1)(x))
model = keras.Model(inputs, outputs)
model.compile(optimizer=keras.optimizers.AdamW(3e-4), loss="binary_crossentropy", metrics=["accuracy"])
(x_train, y_train), _ = keras.datasets.imdb.load_data(num_words=VOCAB) # downloads the data
x_train = keras.utils.pad_sequences(x_train, maxlen=SEQ_LEN)
model.fit(x_train, y_train, validation_split=0.2, epochs=2, batch_size=64, verbose=0)
With KerasHub, fine-tuning a pretrained encoder takes a few lines: keras_hub.models.BertClassifier.from_preset("bert_base_en_uncased", num_classes=2), then fit on raw strings, because the preset includes its tokenizer and preprocessor.
Common Pitfalls
Forgetting positional information, which makes attention order-blind.
Missing padding masks, so the model attends to padding tokens.
Training large Transformers from scratch on small data; fine-tune a pretrained model instead.
Attention cost grows with the square of the sequence length, so long sequences need truncation, chunking or efficient attention.
29
What are TensorFlow distribution strategies, and when should you use them?
KnownRevise
tf.distribute.Strategy is TensorFlow’s API for spreading training across several GPUs, several machines or TPUs with few code changes. You create a strategy, build and compile the model inside strategy.scope(), and Keras’s fit handles splitting each batch across devices and combining gradients. Use it when training on one device is too slow or the data is too large, and pick the strategy that matches your hardware.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Strategies
Strategy
Hardware
How it works
MirroredStrategy
Several GPUs on one machine
Copies the model on each GPU; synchronous all-reduce of gradients
MultiWorkerMirroredStrategy
GPUs on several machines
Same idea across workers; configured with TF_CONFIG
TPUStrategy
TPU pods / Cloud TPU
Synchronous training on TPU cores
ParameterServerStrategy
Many CPU/GPU workers + parameter servers
Asynchronous updates; large embeddings, many workers
OneDeviceStrategy
A single chosen device
Testing code paths before scaling
Default strategy
No distribution
What runs when no strategy is set
Key Concepts
Data parallelism: every replica holds a full copy of the model and processes a slice of the global batch.
Global batch size = per-replica batch size × strategy.num_replicas_in_sync. Scale the learning rate roughly in proportion, often with warm-up.
Synchronous vs asynchronous: mirrored strategies keep replicas identical; parameter-server training lets workers progress independently.
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
strategy = tf.distribute.MirroredStrategy() # uses all visible GPUs (or the CPU if none)
print("replicas:", strategy.num_replicas_in_sync)
per_replica_batch = 64
global_batch = per_replica_batch * strategy.num_replicas_in_sync
X = np.random.rand(10000, 32).astype("float32")
y = (X.sum(axis=1) > 16).astype("float32")
ds = (tf.data.Dataset.from_tensor_slices((X, y))
.shuffle(10000, seed=42).batch(global_batch).prefetch(tf.data.AUTOTUNE))
with strategy.scope(): # variables are created on every replica
model = keras.Sequential([
keras.Input(shape=(32,)),
keras.layers.Dense(128, activation="relu"),
keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer=keras.optimizers.Adam(1e-3 * strategy.num_replicas_in_sync),
loss="binary_crossentropy", metrics=["accuracy"])
model.fit(ds, epochs=3, verbose=0) # fit distributes each batch automatically
For multi-worker training, each machine runs the same script with a TF_CONFIG environment variable that lists the cluster and its own task index; add keras.callbacks.BackupAndRestore so training can resume if a worker fails.
When to Use It
Training takes hours or days on one GPU and the model fits in a single GPU’s memory (data parallelism).
You have TPUs (TPUStrategy is required).
Do not use it when the input pipeline is the bottleneck; more GPUs will just wait for data.
Common Pitfalls
Creating the model or optimizer outside strategy.scope().
Keeping the per-replica batch size as the global batch size, which shrinks each GPU’s batch.
In custom loops, averaging the loss per replica instead of over the global batch (tf.nn.compute_average_loss).
30
What is TensorFlow Extended (TFX), and what are its main components?
KnownRevise
TensorFlow Extended (TFX) is an end-to-end platform for production ML pipelines. It chains standard components that ingest data, validate it, transform features, train and evaluate models, and push approved models to serving, while recording every artifact and run in ML Metadata (MLMD) for lineage and reproducibility. Pipelines run locally for development and on orchestrators such as Kubeflow Pipelines, Vertex AI Pipelines, Apache Airflow or Apache Beam in production.
Read full answerHide full answer
Applies toTFX 1.x with TensorFlow 2.x (Python 3.10-3.11; TFX releases pin specific TensorFlow versions, so check compatibility)
Standard Components
Component
Role
Built on
ExampleGen
Ingests data (CSV, TFRecord, BigQuery) and splits it into train/eval
Apache Beam
StatisticsGen
Computes dataset statistics
TensorFlow Data Validation (TFDV)
SchemaGen
Infers a schema (types, ranges, vocabularies)
TFDV
ExampleValidator
Detects anomalies, missing values and drift against the schema
TFDV
Transform
Feature engineering applied identically in training and serving
TensorFlow Transform (TFT)
Trainer
Trains the model from a user module file (run_fn)
TensorFlow / Keras
Tuner
Hyperparameter search
KerasTuner
Evaluator
Sliced metrics; “blesses” the model if it beats the baseline
TensorFlow Model Analysis (TFMA)
InfraValidator
Checks the model loads and serves in a sandbox
TF Serving
Pusher
Deploys blessed models to a serving location
Filesystem, TF Serving, Vertex AI
Why Transform Matters
TensorFlow Transform computes full-pass statistics (means, vocabularies) once and bakes them into the exported graph, so serving uses exactly the same preprocessing as training. This removes training/serving skew, a frequent source of silent production errors.
In production, add Transform and Evaluator, and give Pusher the Evaluator’s blessing so only models that pass the quality checks are deployed.
Interview Tips
Explain the flow: ingest → validate → transform → train → evaluate → push, with MLMD tracking lineage.
Mention training/serving skew and how TFT prevents it.
Know the trade-off: TFX is powerful but heavy and TensorFlow-centric; lighter teams often use MLflow, Kubeflow or Vertex AI Pipelines with custom steps.
31
How can TensorFlow be used for reinforcement learning?
KnownRevise
Reinforcement learning (RL) trains an agent to choose actions that maximize cumulative reward through trial and error in an environment. TensorFlow supports RL in two ways: you can write algorithms directly with Keras networks and tf.GradientTape (policy gradients, DQN, actor-critic), or use TF-Agents, a library with tested implementations of DQN, PPO, SAC, DDPG and others, plus replay buffers, drivers and environment wrappers. The neural network approximates the policy, the value function, or both.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, TF-Agents (Python 3.12)
Main Approaches
Family
Learns
Examples
Typical use
Value-based
Q-values Q(s, a)
DQN, Double DQN
Discrete actions (games)
Policy gradient
Policy ‘pi(a
s)‘ directly
REINFORCE, PPO
Discrete or continuous actions
Actor-critic
Policy + value baseline
A2C, PPO, SAC, DDPG
Robotics, control
Model-based
Environment dynamics
World models
Expensive environments
Code Example: REINFORCE on a Small Environment
The agent starts at position 0 on a line of 5 cells and must reach cell 4. Each step costs a small penalty, so shorter paths earn more reward.
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
rng = np.random.default_rng(42)
N_STATES, N_ACTIONS, GAMMA = 5, 2, 0.99 # actions: 0 = left, 1 = right
def step(state, action):
nxt = min(max(state + (1 if action == 1 else -1), 0), N_STATES - 1)
done = nxt == N_STATES - 1
return nxt, (1.0 if done else -0.05), done
policy = keras.Sequential([keras.Input(shape=(N_STATES,)),
keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(N_ACTIONS)]) # logits
optimizer = keras.optimizers.Adam(0.01)
for episode in range(300):
state, states, actions, rewards = 0, [], [], []
for _ in range(20):
logits = policy(tf.one_hot([state], N_STATES))
probs = tf.nn.softmax(logits)[0].numpy()
action = rng.choice(N_ACTIONS, p=probs)
nxt, reward, done = step(state, action)
states.append(state); actions.append(action); rewards.append(reward)
state = nxt
if done:
break
# Discounted return from each step: G_t = r_t + gamma * G_(t+1)
returns, g = [], 0.0
for r in reversed(rewards):
g = r + GAMMA * g
returns.insert(0, g)
returns = np.array(returns, dtype="float32")
returns = (returns - returns.mean()) / (returns.std() + 1e-8) # baseline-like normalization
with tf.GradientTape() as tape:
logits = policy(tf.one_hot(states, N_STATES))
log_probs = tf.nn.log_softmax(logits)
chosen = tf.reduce_sum(log_probs * tf.one_hot(actions, N_ACTIONS), axis=1)
loss = -tf.reduce_sum(chosen * returns) # raise probability of high-return actions
optimizer.apply_gradients(zip(tape.gradient(loss, policy.trainable_variables),
policy.trainable_variables))
print(tf.nn.softmax(policy(tf.one_hot(range(4), N_STATES))).numpy().round(2)) # "right" should dominate
The policy-gradient loss in words: increase the log-probability of each action in proportion to how good the return after it was.
Using TF-Agents
TF-Agents wraps environments (suite_gym, custom PyEnvironment), provides agents such as DqnAgent and PPOAgent, replay buffers and drivers that collect experience. It saves you from implementing target networks, epsilon schedules or clipping by hand. Check the release notes for supported TensorFlow versions before you start.
Common Pitfalls
High-variance gradients in REINFORCE; use baselines or actor-critic methods.
Forgetting target networks and replay buffers for DQN, which makes training unstable.
Judging progress from one seed; RL results vary a lot, so average over several seeds.
32
What is TensorFlow Serving, and how does it help deploy models?
KnownRevise
TensorFlow Serving is a high-performance server for deploying TensorFlow models in production. It loads models saved in the SavedModel format, exposes them through REST and gRPC APIs, and handles model versioning, hot-swapping new versions without downtime, and request batching for GPU efficiency. It separates the model from application code, so data scientists can publish a new version by copying it into a directory the server watches.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, TensorFlow Serving 2.x (Docker image `tensorflow/serving`)
Key Features
Feature
Benefit
SavedModel loading
Language-neutral model format with serving signatures
Versioned directories (/models/name/1, /2, …)
Automatic rollout of the newest version; rollback by removing a version
REST API (port 8501)
Easy JSON calls from any language
gRPC API (port 8500)
Lower latency, binary tensors
Server-side batching (--enable_batching)
Groups requests to use the GPU efficiently
Model config file
Serve several models or pin specific versions; labels such as stable and canary
Monitoring
Prometheus metrics endpoint
Deployment Workflow
Train and export: model.export("models/churn/1") creates a SavedModel with a serving_default signature.
import json
import keras
import numpy as np
import requests
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(4,)), keras.layers.Dense(8, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy")
model.fit(np.random.rand(200, 4), np.random.randint(0, 2, 200), epochs=2, verbose=0)
model.export("models/churn/1") # version 1 of the "churn" model
# After the Docker container is running:
payload = {"signature_name": "serving_default", "instances": np.random.rand(2, 4).round(3).tolist()}
resp = requests.post("http://localhost:8501/v1/models/churn:predict", data=json.dumps(payload), timeout=5)
print(resp.json()) # {"predictions": [[...], [...]]}
print(requests.get("http://localhost:8501/v1/models/churn", timeout=5).json()) # version status
Inspect a SavedModel’s inputs and outputs with saved_model_cli show --dir models/churn/1 --all.
Updating Models
Export the next version to models/churn/2. The server detects it, loads it, and switches traffic once it is ready; by default only the latest version is served. Use a model config file to keep several versions available for A/B tests or canary releases.
Common Pitfalls
Exporting without a version subdirectory, so the server finds no model.
Preprocessing done in client code that differs from training; put preprocessing layers inside the exported model.
Sending inputs whose names or shapes do not match the signature.
Using REST for very high throughput where gRPC would be faster.
33
How do you use TensorFlow's Estimator API, and what replaces it?
KnownRevise
The Estimator API (tf.estimator) was TensorFlow 1.x’s high-level training interface: you wrote input functions and a model_fn, and the Estimator handled training, evaluation, checkpointing and export. It was deprecated in TensorFlow 2 and removed in TensorFlow 2.16, so new code should use Keras (model.fit) with tf.data, tf.distribute and model.export. Interviewers still ask about it because older production systems use it, so you should know how it worked and how to migrate.
Read full answerHide full answer
Applies toTensorFlow 2.15 and earlier for `tf.estimator`; TensorFlow 2.16+ with Keras 3 for the replacement (Python 3.12)
Keras models; TF Decision Forests or XGBoost for trees
Input
input_fn returning a tf.data.Dataset
Pass the dataset to fit
Features
tf.feature_column
Keras preprocessing layers (Normalization, StringLookup, Discretization, FeatureSpace)
Train / evaluate
estimator.train, evaluate, train_and_evaluate
fit, evaluate, callbacks
Checkpoints
Automatic in model_dir
ModelCheckpoint, BackupAndRestore
Distribution
RunConfig(train_distribute=...)
strategy.scope()
Export
export_saved_model with a serving input receiver
model.export
Legacy Usage (for Recognition Only)
# Runs only on TensorFlow 2.15 or earlier
import tensorflow as tf
feature_cols = [tf.feature_column.numeric_column("x", shape=[4])]
estimator = tf.estimator.DNNClassifier(hidden_units=[16, 8], feature_columns=feature_cols,
n_classes=2, model_dir="est_model")
def input_fn(features, labels, training=True):
ds = tf.data.Dataset.from_tensor_slices(({"x": features}, labels))
return ds.shuffle(1000, seed=42).repeat().batch(32) if training else ds.batch(32)
# estimator.train(lambda: input_fn(X_train, y_train), steps=1000)
Modern Equivalent
import keras
import numpy as np
keras.utils.set_random_seed(42)
X = np.random.rand(1000, 4).astype("float32")
y = (X.sum(axis=1) > 2).astype("int32")
norm = keras.layers.Normalization()
norm.adapt(X) # replaces numeric feature-column normalization
model = keras.Sequential([keras.Input(shape=(4,)), norm,
keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(8, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(X, y, validation_split=0.2, epochs=5, batch_size=32, verbose=0,
callbacks=[keras.callbacks.ModelCheckpoint("best.keras", save_best_only=True)])
model.export("exported_model") # replaces export_saved_model
Migration Tips
Move feature columns to Keras preprocessing layers inside the model, so serving gets the same preprocessing.
Replace train_and_evaluate with fit(..., validation_data=...) plus callbacks.
Pin TensorFlow 2.15 only as a temporary measure while you migrate.
34
How do you approach optimizing TensorFlow model performance?
KnownRevise
Optimizing TensorFlow performance starts with measuring: profile to find whether time goes to the input pipeline, the device computation or host-device transfers, then fix the biggest bottleneck first. Common wins are a parallel, prefetched tf.data pipeline, compiling with tf.function and XLA, mixed precision on modern GPUs, larger batches, and multi-GPU strategies. For inference, add quantization, pruning or a smaller architecture. “Performance” can also mean accuracy, so be clear about which target you are optimizing.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Gradient checkpointing, smaller batches with accumulation, memory growth
Inference latency/size
Quantization and conversion with LiteRT, pruning, distillation, TF Serving batching
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
keras.mixed_precision.set_global_policy("mixed_float16") # faster on GPUs with Tensor Cores
X = np.random.rand(20000, 64).astype("float32")
y = np.random.randint(0, 10, size=20000)
ds = (tf.data.Dataset.from_tensor_slices((X, y))
.cache()
.shuffle(20000, seed=42)
.batch(256)
.prefetch(tf.data.AUTOTUNE))
model = keras.Sequential([
keras.Input(shape=(64,)),
keras.layers.Dense(512, activation="relu"),
keras.layers.Dense(512, activation="relu"),
keras.layers.Dense(10),
keras.layers.Activation("softmax", dtype="float32"), # keep outputs in float32 for stability
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"],
jit_compile=True, # XLA compilation
steps_per_execution=32) # fewer Python round trips per epoch
model.fit(ds, epochs=2, verbose=0,
callbacks=[keras.callbacks.TensorBoard(log_dir="logs", profile_batch=(5, 15))])
Measure before and after each change (time per epoch, examples per second) instead of assuming a gain; on CPUs, mixed float16 can be slower.
Common Pitfalls
Optimizing the model while the GPU sits idle waiting for data.
Retracing tf.function on every call because of changing Python arguments or shapes.
Turning on mixed precision without keeping the final softmax in float32.
Comparing speeds including the first epoch, which includes tracing and compilation.
35
What techniques does TensorFlow use for graph optimization?
KnownRevise
Graph optimization happens after tf.function traces Python code into a graph: TensorFlow’s Grappler optimizer rewrites the graph to remove waste, and the optional XLA compiler fuses operations into efficient device kernels. Because the whole computation is known in advance, TensorFlow can fold constants, drop unused nodes, fuse operations, reorder layouts and plan memory, which is not possible when running op by op in eager mode. Deployment tools add further optimizations such as quantization when converting models.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Grappler Optimizations
Optimizer
What it does
Constant folding
Computes parts of the graph that depend only on constants once
Arithmetic simplification
Removes redundant operations (e.g. x * 1, repeated transposes)
Common subexpression elimination
Reuses identical computations
Dependency / pruning
Removes nodes that do not affect outputs
Layout optimizer
Chooses NHWC or NCHW layouts for faster GPU convolutions
Remapper
Fuses patterns such as Conv2D + BiasAdd + ReLU into one kernel
Memory optimizer
Swaps or recomputes tensors to reduce peak memory
Loop optimizer
Hoists loop-invariant work out of loops
Auto mixed precision
Optional: rewrites eligible ops to float16
XLA (Accelerated Linear Algebra)
XLA compiles a whole function into fused kernels specific to the device, cutting memory traffic and launch overhead. Enable it with @tf.function(jit_compile=True) or model.compile(..., jit_compile=True). It works best with static shapes; changing shapes trigger recompilation.
Code Example
import time
import tensorflow as tf
tf.random.set_seed(42)
x = tf.random.normal([2048, 2048])
def block(t):
for _ in range(10):
t = tf.nn.relu(t @ tf.transpose(tf.transpose(t)) * 1.0 + 0.0) # redundant ops Grappler removes
t = t / (tf.norm(t) + 1e-6)
return t
eager_fn = block
graph_fn = tf.function(block)
xla_fn = tf.function(block, jit_compile=True)
for name, fn in [("eager", eager_fn), ("graph", graph_fn), ("xla", xla_fn)]:
fn(x) # warm-up: tracing and compilation
start = time.perf_counter()
for _ in range(5):
fn(x).numpy()
print(f"{name}: {(time.perf_counter() - start) / 5 * 1000:.1f} ms per call")
# Inspect Grappler options and turn individual passes on or off
print(tf.config.optimizer.get_experimental_options())
tf.config.optimizer.set_experimental_options({"constant_folding": True, "remapping": True})
Timings depend on hardware; on small CPU workloads the differences may be minor.
TensorRT integration (TF-TRT) builds optimized NVIDIA engines from SavedModels.
Freezing variables into constants enables more constant folding for inference graphs.
Common Pitfalls
Expecting speedups from eager code: optimizations apply only inside tf.function.
Dynamic shapes that force XLA to recompile repeatedly.
Python side effects in traced functions, which run once at trace time.
36
How do you optimize memory usage in TensorFlow?
KnownRevise
Memory optimization in TensorFlow keeps models and data within GPU (or host) memory so training does not crash with out-of-memory (OOM) errors and so you can use larger models or batches. Memory is used by weights, optimizer state, activations saved for backpropagation, and input batches; activations usually dominate during training. The main levers are batch size, numeric precision, recomputation of activations, efficient input pipelines and controlling how TensorFlow allocates GPU memory.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Techniques
Technique
How
Saves
Smaller batch + gradient accumulation
Accumulate gradients over k steps before applying (custom loop or gradient_accumulation_steps in Keras optimizers)
tf.data from files instead of loading everything with NumPy
Host memory
Efficient architecture
Global pooling instead of large Flatten + Dense; depthwise convolutions
Weights and activations
Free resources
keras.backend.clear_session() between experiments
Leaked graphs and models
Code Example
import keras
import tensorflow as tf
for gpu in tf.config.list_physical_devices("GPU"):
tf.config.experimental.set_memory_growth(gpu, True) # must run before GPUs are initialized
keras.utils.set_random_seed(42)
keras.mixed_precision.set_global_policy("mixed_float16")
class CheckpointedBlock(keras.layers.Layer):
"""Dense block whose activations are recomputed in the backward pass."""
def __init__(self, units, **kwargs):
super().__init__(**kwargs)
self.units = units
self.d1 = keras.layers.Dense(units, activation="relu")
self.d2 = keras.layers.Dense(units, activation="relu")
def build(self, input_shape):
# Create the weights here, not inside recompute_grad
self.d1.build(input_shape)
self.d2.build((*input_shape[:-1], self.units))
def call(self, x):
return tf.recompute_grad(lambda t: self.d2(self.d1(t)))(x)
model = keras.Sequential([keras.Input(shape=(1024,)), CheckpointedBlock(2048), CheckpointedBlock(2048),
keras.layers.Dense(10), keras.layers.Activation("softmax", dtype="float32")])
model.compile(optimizer=keras.optimizers.Adam(1e-3, gradient_accumulation_steps=4), # effective batch x4
loss="sparse_categorical_crossentropy")
if tf.config.list_physical_devices("GPU"):
print(tf.config.experimental.get_memory_info("GPU:0")) # current and peak bytes
Estimating Memory
Weights in float32 take 4 bytes per parameter; Adam adds two more copies, so roughly 16 bytes per parameter for weights + gradients + optimizer state, before activations. A 100M-parameter model therefore needs about 1.6 GB before counting activations.
Common Pitfalls
Calling set_memory_growth after the GPU has already been used, which raises an error.
Accumulating tensors in Python lists across steps (e.g. storing every loss tensor), which keeps memory alive.
Building new models in a loop without clear_session.
Loading a whole image dataset into a NumPy array instead of streaming it.
37
What is quantization in TensorFlow, and when should you use it?
KnownRevise
Quantization stores and computes a model’s weights (and often activations) with lower-precision numbers, such as 8-bit integers instead of 32-bit floats. This makes models about four times smaller and usually faster on CPUs, mobile devices and edge accelerators, at the cost of a small accuracy loss. In TensorFlow you apply it while converting to LiteRT (formerly TensorFlow Lite), either after training (post-training quantization) or by simulating it during training (quantization-aware training) when accuracy must be preserved.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, LiteRT converter `tf.lite.TFLiteConverter` (Python 3.12)
How It Works
A float value is mapped to an integer with a scale and zero point: q = round(x / scale) + zero_point, and recovered approximately as x ≈ scale * (q - zero_point). In words, the range of real values is divided into 256 steps, and each value is stored as the index of its step.
Types of Quantization
Method
Weights
Activations
Needs
Size
Typical accuracy cost
Dynamic range
int8
float (quantized on the fly)
Nothing
~4x smaller
Small
Full integer
int8
int8
Representative dataset (~100-500 samples)
~4x smaller
Small; required for some accelerators (Edge TPU, microcontrollers)
Float16
float16
float16
Nothing
~2x smaller
Negligible; good for GPU delegates
Quantization-aware training (QAT)
int8
int8
Retraining with fake-quant ops (TensorFlow Model Optimization Toolkit)
~4x smaller
Lowest
Code Example: Post-Training Full Integer Quantization
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(1000, 16).astype("float32")
y = (X.sum(axis=1) > 8).astype("float32")
model = keras.Sequential([keras.Input(shape=(16,)), keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy")
model.fit(X, y, epochs=5, verbose=0)
def representative_data():
for i in range(200):
yield [X[i:i + 1]] # real-looking inputs to calibrate activation ranges
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_data
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
with open("model_int8.tflite", "wb") as f:
f.write(tflite_model)
print("quantized size (KB):", round(len(tflite_model) / 1024, 1))
Always evaluate the quantized model on held-out data with the LiteRT interpreter and compare it with the float model before shipping.
When to Use It
Deploying to phones, browsers, microcontrollers or edge TPUs.
CPU inference where latency or cost matters.
Not usually needed for GPU training, where mixed precision is the right tool.
Common Pitfalls
A representative dataset that does not match real inputs, which gives poor calibration.
Forgetting to quantize inputs on the client when using int8 input types.
Assuming the accuracy is unchanged without measuring it, especially for small models and regression outputs.
38
How do you use mixed-precision training in TensorFlow?
KnownRevise
Mixed-precision training runs most computations in 16-bit floats (float16 or bfloat16) while keeping the weights and numerically sensitive operations in float32. On GPUs with Tensor Cores (NVIDIA Volta and newer) and on TPUs, this can speed training up significantly and roughly halve activation memory, usually with no loss in accuracy. In Keras you enable it with one global policy; float16 also needs loss scaling to stop small gradients from underflowing to zero, which fit handles automatically.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Policies
Policy
Compute
Weights
Use on
mixed_float16
float16
float32
NVIDIA GPUs (compute capability 7.0+)
mixed_bfloat16
bfloat16
float32
TPUs and NVIDIA Ampere+ GPUs; no loss scaling needed
float32
float32
float32
Default; CPUs
bfloat16 keeps float32’s exponent range with less precision, so gradients rarely underflow; float16 has a narrow range, which is why loss scaling multiplies the loss by a large factor before backpropagation and divides the gradients by it afterwards.
Code Example with fit
import keras
import numpy as np
keras.utils.set_random_seed(42)
keras.mixed_precision.set_global_policy("mixed_float16")
inputs = keras.Input(shape=(784,))
x = keras.layers.Dense(1024, activation="relu")(inputs)
x = keras.layers.Dense(1024, activation="relu")(x)
logits = keras.layers.Dense(10)(x)
outputs = keras.layers.Activation("softmax", dtype="float32")(logits) # final output in float32
model = keras.Model(inputs, outputs)
print(model.layers[1].compute_dtype, model.layers[1].variable_dtype) # float16 float32
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
# With mixed_float16, compile wraps the optimizer in a LossScaleOptimizer automatically.
model.fit(np.random.rand(2048, 784).astype("float32"), np.random.randint(0, 10, 2048),
batch_size=256, epochs=1, verbose=0)
Custom Training Loop
import keras
import tensorflow as tf
optimizer = keras.optimizers.LossScaleOptimizer(keras.optimizers.Adam())
loss_fn = keras.losses.SparseCategoricalCrossentropy()
@tf.function
def train_step(model, x, y):
with tf.GradientTape() as tape:
loss = loss_fn(y, model(x, training=True))
scaled_loss = optimizer.scale_loss(loss) # multiply by the current loss scale
grads = tape.gradient(scaled_loss, model.trainable_variables)
optimizer.apply(grads, model.trainable_variables) # unscales, skips steps with inf/NaN
return loss
Tips
Make layer sizes and batch sizes multiples of 8 so Tensor Cores are used.
Keep softmax, losses and normalization statistics in float32 (Keras does this for its built-in layers).
Measure: on CPUs or older GPUs, mixed precision can be slower.
Common Pitfalls
Leaving the output layer in float16, which can make the loss unstable or overflow.
Using float16 in a custom loop without loss scaling, so small gradients become zero.
Setting the policy after building the model; set it first.
39
How does TensorFlow support multi-GPU and distributed training?
KnownRevise
TensorFlow supports multi-GPU and distributed training through the tf.distribute API. A strategy object decides how variables and computation are placed across GPUs, machines or TPUs, and Keras’s fit uses it automatically when the model is built inside strategy.scope(). Most setups use synchronous data parallelism: each device holds a full copy of the model, processes part of every batch, and gradients are averaged with an all-reduce before every replica applies the same update.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Options
Setup
Strategy
Notes
One machine, several GPUs
MirroredStrategy
NCCL all-reduce; easiest to adopt
Several machines
MultiWorkerMirroredStrategy
Needs TF_CONFIG on each worker; collective ops over the network
TPUs
TPUStrategy
Connect with TPUClusterResolver
Many workers, async updates
ParameterServerStrategy
Variables live on parameter servers; tolerant of slow workers
Multi-Worker Setup
Each worker runs the same script with its own TF_CONFIG, for example:
import json
import os
os.environ["TF_CONFIG"] = json.dumps({
"cluster": {"worker": ["10.0.0.1:12345", "10.0.0.2:12345"]},
"task": {"type": "worker", "index": 0}, # index 1 on the second machine
})
Worker 0 is the chief and usually writes checkpoints and TensorBoard logs.
Code Example: Custom Loop on Multiple GPUs
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
strategy = tf.distribute.MirroredStrategy()
GLOBAL_BATCH = 64 * strategy.num_replicas_in_sync
X = np.random.rand(4096, 32).astype("float32")
y = np.random.randint(0, 2, size=4096).astype("float32")
dataset = tf.data.Dataset.from_tensor_slices((X, y)).shuffle(4096, seed=42).batch(GLOBAL_BATCH)
dist_dataset = strategy.experimental_distribute_dataset(dataset)
with strategy.scope():
model = keras.Sequential([keras.Input(shape=(32,)), keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(1)])
optimizer = keras.optimizers.Adam(1e-3)
loss_obj = keras.losses.BinaryCrossentropy(from_logits=True, reduction="none")
def train_step(inputs):
x, labels = inputs
with tf.GradientTape() as tape:
per_example = loss_obj(labels[:, None], model(x, training=True))
loss = tf.nn.compute_average_loss(per_example, global_batch_size=GLOBAL_BATCH)
grads = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
return loss
@tf.function
def distributed_step(inputs):
per_replica = strategy.run(train_step, args=(inputs,))
return strategy.reduce(tf.distribute.ReduceOp.SUM, per_replica, axis=None)
for epoch in range(2):
total = sum(float(distributed_step(batch)) for batch in dist_dataset)
print(f"epoch {epoch + 1}: summed loss {total:.3f}")
With fit, all of this is automatic: build and compile inside strategy.scope() and call model.fit(dataset).
Practical Tips
Scale the global batch size with the number of replicas and adjust the learning rate (often with warm-up).
Make sure the input pipeline can feed every GPU; use prefetch and sharded files.
Add BackupAndRestore for fault tolerance in multi-worker jobs.
For models too large for one GPU, data parallelism is not enough; consider model parallelism (for example the Keras 3 distribution API on the JAX backend) or smaller models.
Common Pitfalls
Averaging the loss over the per-replica batch instead of the global batch, which scales gradients wrongly.
Every worker writing checkpoints to the same path.
Expecting linear speed-ups when communication or input loading is the bottleneck.
40
What are best practices for writing efficient TensorFlow code?
KnownRevise
Efficient TensorFlow code keeps the accelerator busy and avoids Python overhead: express work as vectorized tensor operations, compile hot paths with tf.function, feed data with a parallel tf.data pipeline, and keep data on the device instead of converting to NumPy in loops. It also means writing code that is easy to profile and debug: develop eagerly, then compile, and measure every change with the profiler rather than guessing.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
tf.data with map(num_parallel_calls=AUTOTUNE), prefetch, cache
from_generator with heavy Python work
Host-device
Keep tensors on the GPU
.numpy() or float(tensor) inside training loops
Precision
Mixed precision on supported GPUs/TPUs
float64 unless needed
Training API
fit with callbacks; steps_per_execution
Hand-written loops without tf.function
Shapes
Static shapes, input_signature for serving
Unbounded variable shapes causing recompiles
Debugging
tf.debugging.assert_*, run_functions_eagerly(True) while developing
Leaving eager mode on in production
Reproducibility
keras.utils.set_random_seed(42)
Unseeded experiments
Code Example: Loop Versus Vectorized
import time
import tensorflow as tf
tf.random.set_seed(42)
points = tf.random.normal([2000, 3])
centers = tf.random.normal([50, 3])
def nearest_center_loop(p, c):
out = []
for i in range(p.shape[0]): # Python loop: slow
d = tf.reduce_sum((c - p[i]) ** 2, axis=1)
out.append(int(tf.argmin(d)))
return out
@tf.function(input_signature=[tf.TensorSpec([None, 3]), tf.TensorSpec([None, 3])])
def nearest_center_vectorized(p, c):
# Pairwise squared distances via broadcasting: (N, 1, 3) - (1, K, 3) -> (N, K)
d = tf.reduce_sum((p[:, None, :] - c[None, :, :]) ** 2, axis=-1)
return tf.argmin(d, axis=1)
start = time.perf_counter()
loop_result = nearest_center_loop(points, centers)
loop_time = time.perf_counter() - start
nearest_center_vectorized(points, centers) # trace once
start = time.perf_counter()
vec_result = nearest_center_vectorized(points, centers)
vec_time = time.perf_counter() - start
print("same result:", loop_result == vec_result.numpy().tolist())
print(f"loop {loop_time:.3f}s vs vectorized {vec_time:.4f}s")
The fixed input_signature lets the function accept any number of points without retracing.
More Habits
Use Keras preprocessing layers inside the model so the same logic runs in training and serving.
Profile with TensorBoard before optimizing, and compare examples per second rather than wall time of the first epoch.
Keep functions pure in tf.function: no hidden Python state, no printing for logic (use tf.print).
Common Pitfalls
Creating variables or models inside a tf.function that is called repeatedly.
Mixing NumPy and TensorFlow in hot loops, which copies data back and forth.
Building large Python lists of tensors and calling tf.stack at the end of each step; use tf.TensorArray inside graphs.
41
How do you handle image data in TensorFlow?
KnownRevise
Handling image data in TensorFlow means loading image files into tensors, resizing and scaling them consistently, augmenting them during training, and feeding them in efficient batches. Keras provides image_dataset_from_directory for folder-organized data and preprocessing layers (Resizing, Rescaling, RandomFlip) that can live inside the model, while tf.io and tf.image handle custom decoding and transformations. Images are represented as tensors of shape (batch, height, width, channels) by default.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
import keras
import numpy as np
import tensorflow as tf
from pathlib import Path
keras.utils.set_random_seed(42)
# Create a tiny folder dataset so the example is self-contained: data/cats, data/dogs
rng = np.random.default_rng(42)
for label in ["cats", "dogs"]:
folder = Path("data") / label
folder.mkdir(parents=True, exist_ok=True)
for i in range(20):
img = (rng.random((64, 64, 3)) * 255).astype("uint8")
keras.utils.save_img(folder / f"{i}.png", img)
train_ds, val_ds = keras.utils.image_dataset_from_directory(
"data", validation_split=0.2, subset="both", seed=42,
image_size=(128, 128), batch_size=8, label_mode="binary")
print(train_ds.class_names) # ['cats', 'dogs']
augment = keras.Sequential([keras.layers.RandomFlip("horizontal"), keras.layers.RandomRotation(0.1)])
train_ds = train_ds.prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.cache().prefetch(tf.data.AUTOTUNE)
model = keras.Sequential([
keras.Input(shape=(128, 128, 3)),
augment, # runs only in training
keras.layers.Rescaling(1.0 / 255), # same scaling at inference
keras.layers.Conv2D(16, 3, activation="relu"),
keras.layers.MaxPooling2D(),
keras.layers.GlobalAveragePooling2D(),
keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(train_ds, validation_data=val_ds, epochs=2, verbose=0)
# Manual decoding for a single file
raw = tf.io.read_file("data/cats/0.png")
img = tf.image.resize(tf.io.decode_png(raw, channels=3), (128, 128))
print(img.shape, img.dtype) # (128, 128, 3) float32
Common Pitfalls
Different preprocessing in training and serving; keep resizing and scaling inside the model.
Using the wrong preprocess_input for a pretrained backbone (some expect [-1, 1], others [0, 255]).
Applying augmentation to validation or test data.
Loading thousands of images into one NumPy array instead of streaming with tf.data.
42
What is the text preprocessing process in TensorFlow?
KnownRevise
Text preprocessing in TensorFlow turns raw strings into numeric tensors a model can use: standardize the text (lowercase, strip punctuation), split it into tokens, map tokens to integer IDs from a vocabulary, and pad or truncate sequences to a fixed length. The main tool is the Keras TextVectorization layer, which learns its vocabulary with adapt() on training data and can be placed inside the model so raw strings work at inference. For pretrained Transformers, use the matching subword tokenizer from KerasHub instead.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, KerasHub (Python 3.12)
Keep the most frequent tokens; reserve padding (0) and out-of-vocabulary (1)
adapt() on training text only
Encode
Tokens → integer IDs, multi-hot, counts or TF-IDF
output_mode="int", "multi_hot", "count", "tf_idf"
Pad/truncate
Fixed sequence length
output_sequence_length=...
Embed
IDs → dense vectors
Embedding(mask_zero=True)
Code Example
import re
import string
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
train_texts = np.array(["Great product, works well!", "Terrible. Broke after a day.",
"<br>Really great value", "Not good, would not buy again"])
train_labels = np.array([1, 0, 1, 0])
def custom_standardize(text):
text = tf.strings.lower(text)
text = tf.strings.regex_replace(text, "<br>", " ")
return tf.strings.regex_replace(text, f"[{re.escape(string.punctuation)}]", "")
vectorizer = keras.layers.TextVectorization(max_tokens=10000, output_mode="int",
output_sequence_length=8, standardize=custom_standardize)
vectorizer.adapt(train_texts) # vocabulary from training data only
print(vectorizer.get_vocabulary()[:6]) # ['', '[UNK]', ...]
print(vectorizer(["great value, unknownword"]).numpy())
model = keras.Sequential([
keras.Input(shape=(1,), dtype="string"),
vectorizer, # raw strings in, IDs out
keras.layers.Embedding(10000, 16, mask_zero=True),
keras.layers.GlobalAveragePooling1D(),
keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(train_texts[:, None], train_labels, epochs=5, verbose=0)
print(model.predict(np.array([["great product"]]), verbose=0))
For bag-of-words models, use output_mode="tf_idf" with ngrams=2 and a Dense classifier; it is a strong, fast baseline.
Pretrained Tokenizers
Transformer models need their own tokenizer, for example keras_hub.models.BertTokenizer.from_preset("bert_base_en_uncased"). Mixing a custom vocabulary with pretrained embeddings breaks the token-to-vector mapping.
Common Pitfalls
Calling adapt on the full dataset including test data, which leaks vocabulary information.
Removing words that carry meaning for the task (e.g. “not” in sentiment analysis).
Truncating long texts so much that the useful part is lost; check the length distribution first.
43
How would you approach time-series forecasting with TensorFlow?
KnownRevise
Time-series forecasting with TensorFlow frames the problem as supervised learning on sliding windows: the model sees the last N time steps and predicts the next value or the next H values. The workflow is a chronological train/validation/test split, scaling with statistics from the training period only, window creation with tf.data or keras.utils.timeseries_dataset_from_array, a simple baseline, then models of increasing complexity (dense, LSTM/GRU, 1D CNN, Transformer). Always compare against a naive forecast, because deep models often fail to beat it on small or noisy series.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Approach
Step
Details
Split
By time: e.g. 70% train, 15% validation, 15% test; never shuffle across the split
Scale
Fit mean/std on the training period only
Features
Lags, calendar features (day of week, holidays), known future inputs
Window
Input width (lookback), forecast horizon, stride
Baselines
Last value (naive), seasonal naive (value one season ago)
Models
Dense on flattened windows, LSTM/GRU, Conv1D, Transformer encoder
Multi-step output
Single-shot (Dense of size H) or autoregressive (feed predictions back)
Metrics
MAE, RMSE, MASE (error relative to naive)
Code Example
import keras
import numpy as np
keras.utils.set_random_seed(42)
rng = np.random.default_rng(42)
t = np.arange(2000)
series = (10 + 0.01 * t + 3 * np.sin(2 * np.pi * t / 50) + rng.normal(0, 0.5, t.size)).astype("float32")
LOOKBACK, HORIZON = 100, 10
n_train, n_val = 1400, 300
mean, std = series[:n_train].mean(), series[:n_train].std() # training statistics only
scaled = (series - mean) / std
def make_ds(data, start, end, shuffle):
# inputs: data[i : i + LOOKBACK]; targets: the next HORIZON values
segment = data[start:end]
targets = np.stack([segment[i + LOOKBACK:i + LOOKBACK + HORIZON]
for i in range(len(segment) - LOOKBACK - HORIZON + 1)])
return keras.utils.timeseries_dataset_from_array(
segment[:len(targets) + LOOKBACK - 1, None], targets, sequence_length=LOOKBACK,
batch_size=32, shuffle=shuffle, seed=42)
train_ds = make_ds(scaled, 0, n_train, shuffle=True) # shuffling windows within train is fine
val_ds = make_ds(scaled, n_train - LOOKBACK, n_train + n_val, shuffle=False)
test_ds = make_ds(scaled, n_train + n_val - LOOKBACK, len(scaled), shuffle=False)
model = keras.Sequential([
keras.Input(shape=(LOOKBACK, 1)),
keras.layers.Conv1D(32, 5, activation="relu"),
keras.layers.LSTM(32),
keras.layers.Dense(HORIZON), # single-shot multi-step forecast
])
model.compile(optimizer="adam", loss="mse", metrics=["mae"])
model.fit(train_ds, validation_data=val_ds, epochs=30, verbose=0,
callbacks=[keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True)])
# Compare with a naive forecast (repeat the last observed value) on the test set
model_mae, naive_mae = [], []
for x, y in test_ds:
pred = model.predict(x, verbose=0)
model_mae.append(np.abs(pred - y.numpy()).mean())
naive_mae.append(np.abs(x.numpy()[:, -1, :] - y.numpy()).mean())
print(f"model MAE: {np.mean(model_mae) * std:.3f} | naive MAE: {np.mean(naive_mae) * std:.3f}")
Validation and test windows start LOOKBACK steps early so their inputs use past values, but their targets never overlap the training targets.
Common Pitfalls
Shuffling before splitting or scaling with full-series statistics, which leaks the future.
Skipping the naive baseline, which hides models that learned nothing.
Using features at prediction time that would not be known then (e.g. future weather actuals).
Evaluating with a single split; use rolling-origin (walk-forward) validation when possible.
44
What support does TensorFlow offer for transfer learning?
KnownRevise
TensorFlow supports transfer learning through pretrained models you can load with one line and adapt to a new task: Keras Applications (ResNet, EfficientNet, MobileNet, ConvNeXt and more, with ImageNet weights), KerasHub (pretrained text and multimodal models such as BERT, Gemma and vision backbones via from_preset), and Kaggle Models / TensorFlow Hub. The usual workflow is feature extraction with a frozen backbone and a new head, followed by fine-tuning some top layers at a low learning rate.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, KerasHub (Python 3.12)
Load the SavedModel and wrap it, or use KerasHub presets
Two-Phase Workflow
Phase
Backbone
Learning rate
Purpose
1. Feature extraction
Frozen (trainable = False)
Normal (e.g. 1e-3)
Train the new head without destroying pretrained features
2. Fine-tuning
Top layers unfrozen
Low (e.g. 1e-5)
Adapt features to the new data
Code Example
import keras
keras.utils.set_random_seed(42)
NUM_CLASSES = 5
base = keras.applications.EfficientNetV2B0(include_top=False, weights="imagenet",
input_shape=(224, 224, 3)) # downloads weights
base.trainable = False
inputs = keras.Input(shape=(224, 224, 3))
x = keras.layers.RandomFlip("horizontal")(inputs)
x = base(x, training=False) # keep BatchNorm in inference mode
x = keras.layers.GlobalAveragePooling2D()(x)
x = keras.layers.Dropout(0.2)(x)
outputs = keras.layers.Dense(NUM_CLASSES, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.compile(optimizer=keras.optimizers.Adam(1e-3), loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
# model.fit(train_ds, validation_data=val_ds, epochs=10) # phase 1
base.trainable = True
for layer in base.layers[:-30]: # unfreeze only the last 30 layers
layer.trainable = False
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss="sparse_categorical_crossentropy",
metrics=["accuracy"]) # recompile after changing trainable flags
# model.fit(train_ds, validation_data=val_ds, epochs=5) # phase 2
print(len(model.trainable_weights), "trainable weight tensors")
EfficientNetV2 in Keras includes its own input rescaling, so it takes raw pixel values in [0, 255]; other backbones need their preprocess_input function.
Common Pitfalls
Not calling the frozen base with training=False, so BatchNormalization statistics change during fine-tuning.
Fine-tuning with a high learning rate, which destroys pretrained features.
Forgetting to recompile after changing trainable.
Using a backbone whose pretraining domain is very different (natural images for X-rays) without enough fine-tuning data.
45
How is TensorFlow deployed on mobile and edge devices?
KnownRevise
TensorFlow models are deployed to mobile and edge devices with LiteRT (formerly TensorFlow Lite): you convert a trained model to the compact .tflite flatbuffer format, optionally quantize it, and run it with the LiteRT interpreter on Android, iOS, embedded Linux or microcontrollers. Hardware delegates offload work to GPUs, DSPs and NPUs for speed. For browsers there is TensorFlow.js. On-device inference gives low latency, offline operation and better privacy, at the cost of model size and accuracy constraints.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, LiteRT (Python 3.12)
Deployment Options
Target
Runtime
Notes
Android
LiteRT (Java/Kotlin API, Google Play services runtime)
GPU delegate; vendor NPU delegates
iOS
LiteRT (Swift/Objective-C)
Metal GPU delegate, Core ML delegate
Embedded Linux (Raspberry Pi, Jetson, Coral)
LiteRT Python/C++; Edge TPU runtime
Edge TPU needs full int8 quantization
Microcontrollers
LiteRT for Microcontrollers (C++)
Kilobytes of memory; int8 models only
Browser / Node.js
TensorFlow.js
WebGL/WebGPU/WASM backends
Workflow
Train in Keras and keep preprocessing inside the model where possible.
Convert with tf.lite.TFLiteConverter.
Optimize: post-training quantization (dynamic, float16, full int8), pruning, or a smaller architecture such as MobileNet.
Check accuracy and latency of the converted model on real devices (use the LiteRT benchmark tool).
Bundle the .tflite file with the app or download it at runtime.
Code Example: Convert and Test
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(500, 10).astype("float32")
y = (X.mean(axis=1) > 0.5).astype("float32")
model = keras.Sequential([keras.Input(shape=(10,)), keras.layers.Dense(32, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy")
model.fit(X, y, epochs=5, verbose=0)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT] # dynamic-range quantization
tflite_bytes = converter.convert()
with open("model.tflite", "wb") as f:
f.write(tflite_bytes)
# Run it the way a device would (the standalone ai-edge-litert package offers the same Interpreter API)
interpreter = tf.lite.Interpreter(model_path="model.tflite")
interpreter.allocate_tensors()
inp, out = interpreter.get_input_details()[0], interpreter.get_output_details()[0]
interpreter.set_tensor(inp["index"], X[:1])
interpreter.invoke()
print("LiteRT:", interpreter.get_tensor(out["index"]), "| Keras:", model.predict(X[:1], verbose=0))
Design Considerations
Model size and memory: app-store limits, RAM on low-end phones.
Latency and battery: prefer smaller architectures and quantization.
Supported operations: some TensorFlow ops are not in LiteRT’s builtin set; SELECT_TF_OPS adds them but increases binary size.
Updates: download new models remotely; monitor on-device accuracy where possible.
Common Pitfalls
Preprocessing in the app that differs from training (resizing, normalization, color order).
Skipping on-device benchmarking; desktop timing is not representative.
Using unsupported custom ops that fail at conversion time.
46
How is TensorFlow used in healthcare? Give an example.
KnownRevise
TensorFlow is used in healthcare for medical image analysis (detecting diabetic retinopathy in retinal photos, finding lung nodules in CT scans, classifying skin lesions), clinical risk prediction from electronic health records, analysis of ECG and other signals, and genomics. A well-known example is Google’s research on deep learning for diabetic retinopathy screening from retinal fundus images, built with TensorFlow models. Healthcare ML differs from other domains because errors have clinical consequences, data is private and regulated, and models must be validated on data from the hospitals where they will be used.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, scikit-learn 1.5+ (Python 3.12)
Typical Applications
Area
Data
Model type
Radiology, ophthalmology, dermatology
X-rays, CT, MRI, retinal photos, skin images
CNNs / vision transformers with transfer learning
Pathology
Gigapixel slide images
Patch-based CNNs, multiple-instance learning
Clinical risk scores
EHR tabular and time-series data
Dense networks, RNNs, gradient-boosted trees
Cardiology
ECG signals
1D CNNs
Clinical text
Notes, reports
Transformers
Code Example: Malignant vs Benign Tumor Classifier
This uses the Wisconsin breast cancer dataset in scikit-learn (features computed from cell-nucleus images).
import keras
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import classification_report, roc_auc_score
from sklearn.model_selection import train_test_split
keras.utils.set_random_seed(42)
data = load_breast_cancer()
X, y = data.data.astype("float32"), (data.target == 0).astype("float32") # 1 = malignant
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
norm = keras.layers.Normalization()
norm.adapt(X_train[: int(len(X_train) * 0.8)]) # only the rows fit() trains on (validation_split takes the last 20%)
model = keras.Sequential([
keras.Input(shape=(X.shape[1],)), norm,
keras.layers.Dense(32, activation="relu"), keras.layers.Dropout(0.2),
keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=[keras.metrics.AUC(name="auc"),
keras.metrics.Recall(name="recall")])
model.fit(X_train, y_train, validation_split=0.2, epochs=100, batch_size=32, verbose=0,
callbacks=[keras.callbacks.EarlyStopping(monitor="val_auc", mode="max", patience=10,
restore_best_weights=True)])
proba = model.predict(X_test, verbose=0).ravel()
threshold = 0.3 # lower threshold: missing a malignancy costs more
print("ROC AUC:", round(roc_auc_score(y_test, proba), 3))
print(classification_report(y_test, (proba >= threshold).astype(int), target_names=["benign", "malignant"]))
The threshold should be chosen on validation data with clinicians, based on the acceptable trade-off between missed cancers (false negatives) and unnecessary follow-ups (false positives).
What Makes Healthcare Different
Metrics: sensitivity (recall), specificity, AUROC, calibration; accuracy is misleading with rare diseases.
Validation: external validation on other hospitals, devices and populations; prospective studies before clinical use.
Privacy and regulation: HIPAA/GDPR, de-identification, federated learning; medical-device regulation (e.g. FDA) for diagnostic software.
Fairness: check performance across age, sex and ethnicity groups.
Explainability: saliency maps or feature attributions help clinicians trust and audit predictions.
Common Pitfalls
Splitting by image instead of by patient, so the same patient is in train and test.
Shortcut learning, such as a model detecting the scanner type or hospital markers instead of disease.
Reporting only accuracy on imbalanced data.
47
Write TensorFlow code to multiply two tensors element-wise.
KnownRevise
Element-wise multiplication multiplies values at the same positions in two tensors, producing a tensor of the same shape. In TensorFlow you use tf.multiply(a, b) or the * operator, which also supports broadcasting: when shapes differ, dimensions of size 1 are stretched to match. It is different from matrix multiplication (tf.matmul or @), which combines rows and columns.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Code Example
import tensorflow as tf
a = tf.constant([[1, 2, 3],
[4, 5, 6]], dtype=tf.float32)
b = tf.constant([[10, 20, 30],
[40, 50, 60]], dtype=tf.float32)
product = tf.multiply(a, b) # same as a * b
print(product.numpy())
# [[ 10. 40. 90.]
# [160. 250. 360.]]
# Broadcasting: shape (2, 3) * shape (3,) multiplies each row by the vector
scale = tf.constant([1.0, 0.5, 2.0])
print((a * scale).numpy())
# [[ 1. 1. 6. ]
# [ 4. 2.5 12. ]]
# Broadcasting with a column: shape (2, 3) * shape (2, 1)
col = tf.constant([[2.0], [10.0]])
print((a * col).numpy())
# [[ 2. 4. 6.]
# [40. 50. 60.]]
# Contrast with matrix multiplication: (2, 3) @ (3, 2) -> (2, 2)
print(tf.matmul(a, tf.transpose(b)).numpy())
# [[ 140. 320.]
# [ 320. 770.]]
The matrix product entry [0, 0] is 1*10 + 2*20 + 3*30 = 140, the sum of the element-wise products of row 0 of a and row 0 of b.
Rules to Remember
Situation
Result
Same shapes
Element-wise product, same shape
Shapes compatible by broadcasting (align from the right; sizes equal or 1)
Expanded result shape
Incompatible shapes, e.g. (2, 3) and (2,)
InvalidArgumentError
Different dtypes, e.g. int32 and float32
Error: TensorFlow does not auto-cast; use tf.cast
Related Operations
tf.math.multiply_no_nan(a, b) returns 0 where b is 0 even if a is NaN or inf.
tf.einsum("ij,ij->ij", a, b) expresses the same product with index notation.
tf.reduce_sum(a * b) gives the sum of all element-wise products (a dot product when both are vectors).
Common Pitfalls
Mixing integer and float tensors without casting.
Expecting * to perform matrix multiplication.
Unintended broadcasting, for example shape (3,) with (3, 1) producing a (3, 3) result; check shapes with tf.debugging.assert_shapes.
48
Implement logistic regression using TensorFlow.
KnownRevise
Logistic regression predicts the probability of a binary outcome by passing a weighted sum of the features through the sigmoid function, p = 1 / (1 + exp(-(w·x + b))). It is trained by minimizing binary cross-entropy (log loss), which penalizes confident wrong predictions heavily. In TensorFlow you can implement it from scratch with variables and tf.GradientTape, or as a Keras model with a single Dense(1, activation="sigmoid") layer; both are shown below.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, scikit-learn 1.5+ (Python 3.12)
The Model in Words
Linear score: z = w·x + b, a weighted sum of the features plus a bias.
Probability: the sigmoid squashes z into (0, 1).
Loss: for each example, -(y log p + (1 - y) log(1 - p)), averaged over the batch.
Update: gradient descent moves w and b against the gradient of the loss; for logistic regression the gradient is (p - y) * x.
Code Example: From Scratch
import numpy as np
import tensorflow as tf
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
tf.random.set_seed(42)
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
scaler = StandardScaler().fit(X_train) # fit on training data only
X_train = scaler.transform(X_train).astype("float32")
X_test = scaler.transform(X_test).astype("float32")
y_train = y_train.astype("float32")
w = tf.Variable(tf.zeros([X_train.shape[1], 1]))
b = tf.Variable(0.0)
learning_rate, l2 = 0.1, 1e-3
@tf.function
def train_step(x, y):
with tf.GradientTape() as tape:
logits = tf.squeeze(x @ w + b, axis=1)
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=y, logits=logits))
loss += l2 * tf.reduce_sum(w ** 2) # L2 regularization
dw, db = tape.gradient(loss, [w, b])
w.assign_sub(learning_rate * dw) # plain gradient descent step
b.assign_sub(learning_rate * db)
return loss
ds = tf.data.Dataset.from_tensor_slices((X_train, y_train)).shuffle(1000, seed=42).batch(64)
for epoch in range(100):
for xb, yb in ds:
loss = train_step(xb, yb)
proba = tf.sigmoid(tf.squeeze(X_test @ w + b, axis=1)).numpy()
print("accuracy:", round(accuracy_score(y_test, proba >= 0.5), 3),
"| ROC AUC:", round(roc_auc_score(y_test, proba), 3))
sigmoid_cross_entropy_with_logits combines the sigmoid and the log loss in a numerically stable way, which avoids log(0) when probabilities saturate.
Code Example: Keras Version
import keras
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(X_train.shape[1],)),
keras.layers.Dense(1, activation="sigmoid",
kernel_regularizer=keras.regularizers.L2(1e-3))])
model.compile(optimizer=keras.optimizers.SGD(0.1), loss="binary_crossentropy", metrics=["accuracy"])
model.fit(X_train, y_train, epochs=100, batch_size=64, verbose=0)
print(model.evaluate(X_test, y_test.astype("float32"), verbose=0))
Interview Tips
Scale features; gradient descent converges much faster and regularization treats features fairly.
Explain why log loss is used rather than MSE: with a sigmoid, MSE is non-convex in the weights and gives weak gradients for confident mistakes.
For multi-class problems, replace the sigmoid with softmax and use categorical cross-entropy (multinomial logistic regression).
49
Build a simple CNN in TensorFlow for image classification.
KnownRevise
A convolutional neural network (CNN) classifies images by learning small filters that detect local patterns (edges, textures, shapes), stacking them so deeper layers see larger and more abstract features, and finishing with a classifier. A simple CNN for small images alternates Conv2D and pooling layers, then uses global average pooling and a dense softmax output. Below is a complete Keras example on CIFAR-10 (60,000 32×32 color images in 10 classes).
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Architecture
Layer
Output shape
Role
Input
32×32×3
RGB image
Rescaling + augmentation
32×32×3
Scale to [0, 1]; random flips/shifts in training
Conv block 1 (2 × Conv2D 32, BatchNorm, MaxPool)
16×16×32
Edges and colors
Conv block 2 (2 × Conv2D 64, MaxPool)
8×8×64
Textures and parts
Conv block 3 (2 × Conv2D 128, MaxPool)
4×4×128
Object parts
GlobalAveragePooling2D
128
One value per feature map
Dropout + Dense(10, softmax)
10
Class probabilities
Code Example
import keras
from keras import layers
keras.utils.set_random_seed(42)
(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data() # downloads the data
class_names = ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"]
def conv_block(x, filters):
for _ in range(2):
x = layers.Conv2D(filters, 3, padding="same", use_bias=False)(x)
x = layers.BatchNormalization()(x)
x = layers.Activation("relu")(x)
return layers.MaxPooling2D()(x)
inputs = keras.Input(shape=(32, 32, 3))
x = layers.Rescaling(1.0 / 255)(inputs)
x = layers.RandomFlip("horizontal")(x)
x = layers.RandomTranslation(0.1, 0.1)(x)
for filters in (32, 64, 128):
x = conv_block(x, filters)
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.3)(x)
outputs = layers.Dense(10, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.summary()
model.compile(optimizer=keras.optimizers.Adam(1e-3), loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
model.fit(x_train, y_train, validation_split=0.1, epochs=30, batch_size=128, verbose=2,
callbacks=[keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True),
keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=2)])
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
print(f"test accuracy: {test_acc:.3f}")
pred = model.predict(x_test[:1], verbose=0).argmax()
print("first test image predicted as:", class_names[pred], "| true:", class_names[y_test[0][0]])
Training on a GPU takes a few minutes; on a CPU, reduce the epochs.
Design Choices Explained
3×3 kernels with padding="same" keep spatial size inside a block; pooling halves it.
Doubling filters as resolution halves keeps computation balanced.
BatchNormalization stabilizes and speeds up training.
Global average pooling instead of Flatten + large Dense reduces parameters and overfitting.
Augmentation layers inside the model are active only during training.
Common Pitfalls
Forgetting to scale pixel values, which slows or breaks training.
Using categorical_crossentropy with integer labels (use the sparse version).
Evaluating on the validation split used for early stopping and reporting it as test accuracy.
50
Write a TensorFlow script to normalize a dataset's features.
KnownRevise
Feature normalization rescales input features so they have comparable ranges, which helps gradient-based training converge faster and stops large-valued features from dominating. The most common form is standardization: subtract each feature’s mean and divide by its standard deviation, giving mean 0 and standard deviation 1. In TensorFlow, the Keras Normalization layer learns these statistics with adapt() on the training data and applies them inside the model, so the same transformation is used at inference.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, scikit-learn 1.5+ (Python 3.12)
Normalization Methods
Method
Formula in words
Keras / TF tool
Use when
Standardization (z-score)
(value - mean) / standard deviation
keras.layers.Normalization
Default for neural networks
Min-max scaling
(value - min) / (max - min), into [0, 1]
Manual with tf.reduce_min/max or Rescaling for known ranges
import keras
import numpy as np
import tensorflow as tf
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
keras.utils.set_random_seed(42)
X, y = fetch_california_housing(return_X_y=True) # downloads the data
X = X.astype("float32")
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# 1. Keras Normalization layer (z-score), statistics from training data only
norm = keras.layers.Normalization(axis=-1)
norm.adapt(X_train)
print("means:", norm.mean.numpy().round(2))
scaled_train = norm(X_train)
print("after: mean", tf.reduce_mean(scaled_train, axis=0).numpy().round(2),
"std", tf.math.reduce_std(scaled_train, axis=0).numpy().round(2))
# 2. Manual min-max scaling with TensorFlow ops
x_min = tf.reduce_min(X_train, axis=0)
x_max = tf.reduce_max(X_train, axis=0)
minmax_test = (X_test - x_min) / (x_max - x_min + 1e-8) # test can fall slightly outside [0, 1]
# 3. Normalization inside the model: raw features in, predictions out
model = keras.Sequential([
keras.Input(shape=(X.shape[1],)),
norm,
keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(1),
])
model.compile(optimizer="adam", loss="mse", metrics=["mae"])
model.fit(X_train, y_train, validation_split=0.2, epochs=10, verbose=0)
print("test MAE:", round(model.evaluate(X_test, y_test, verbose=0)[1], 3))
Common Pitfalls
Computing statistics on the whole dataset (including test data), which leaks information.
Normalizing training data in a script but forgetting to apply the same statistics at serving time; putting the layer inside the model prevents this.
Dividing by zero for constant features; add a small epsilon or drop the feature.
Normalizing categorical IDs or one-hot columns as if they were continuous.
51
Create a recurrent neural network in TensorFlow for sequential data.
KnownRevise
A recurrent neural network (RNN) processes a sequence one step at a time while carrying a hidden state that summarizes what it has seen, so the output can depend on order and earlier context. In practice you use gated variants, LSTM or GRU, because plain RNNs struggle to learn long-range dependencies due to vanishing gradients. Keras provides SimpleRNN, LSTM, GRU and the Bidirectional wrapper; the example below classifies movie reviews as positive or negative.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Layer Choices
Layer
Idea
Notes
SimpleRNN
New state = tanh(weights × input + weights × previous state)
Fast, short memory
LSTM
Cell state with input, forget and output gates
Long dependencies; most common
GRU
Update and reset gates, no separate cell state
Fewer parameters, often similar accuracy
Bidirectional(LSTM(...))
Reads the sequence forwards and backwards
When the whole sequence is available (not streaming)
return_sequences=True
Output at every step
Needed to stack RNN layers or for per-step tagging
Code Example: Sentiment Classification
import keras
from keras import layers
keras.utils.set_random_seed(42)
VOCAB, MAXLEN = 20000, 200
(x_train, y_train), (x_test, y_test) = keras.datasets.imdb.load_data(num_words=VOCAB) # downloads data
x_train = keras.utils.pad_sequences(x_train, maxlen=MAXLEN) # pads/truncates at the start by default
x_test = keras.utils.pad_sequences(x_test, maxlen=MAXLEN)
model = keras.Sequential([
keras.Input(shape=(MAXLEN,), dtype="int32"),
layers.Embedding(VOCAB, 64, mask_zero=True), # mask tells the RNN to skip padding
layers.Bidirectional(layers.LSTM(64, return_sequences=True)),
layers.Bidirectional(layers.LSTM(32)), # last layer returns only the final state
layers.Dropout(0.3),
layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, validation_split=0.2, epochs=5, batch_size=64, verbose=2,
callbacks=[keras.callbacks.EarlyStopping(patience=2, restore_best_weights=True)])
print("test accuracy:", round(model.evaluate(x_test, y_test, verbose=0)[1], 3))
Sequence-to-Value on Numeric Data
For numeric sequences such as sensor readings, the input shape is (timesteps, features):
import keras
import numpy as np
keras.utils.set_random_seed(42)
X = np.random.rand(500, 30, 3).astype("float32") # 500 sequences, 30 steps, 3 features
y = X[:, -5:, 0].mean(axis=1) # target depends on recent steps
model = keras.Sequential([keras.Input(shape=(30, 3)), keras.layers.GRU(32), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss="mse")
model.fit(X, y, epochs=10, verbose=0)
Common Pitfalls
Forgetting return_sequences=True on every RNN layer except the last when stacking.
Not masking padding, so the RNN processes long runs of zeros.
Very long sequences: training is slow and memory-heavy; consider truncation, 1D CNNs or Transformers.
Keras uses the fast cuDNN kernel for LSTM/GRU only with default settings (tanh activation, no recurrent dropout); changing them can make training much slower.
52
Write a Python function using TensorFlow to compute a function's gradient.
KnownRevise
Computing a gradient in TensorFlow uses automatic differentiation with tf.GradientTape: operations run inside the tape are recorded, and tape.gradient(target, sources) applies the chain rule backwards to return the derivative of the target with respect to each source. Trainable tf.Variables are watched automatically; plain tensors need tape.watch. The same mechanism powers neural network training, but it works for any differentiable function you write with TensorFlow operations.
Read full answerHide full answer
Applies toTensorFlow 2.16+ (Python 3.12)
Code Example
import tensorflow as tf
def compute_gradient(fn, x):
"""Return fn(x) and the gradient of fn at x (x can be a scalar or a tensor)."""
x = tf.convert_to_tensor(x, dtype=tf.float32)
with tf.GradientTape() as tape:
tape.watch(x) # needed because x is a tensor, not a Variable
y = fn(x)
return y, tape.gradient(y, x)
# 1. Scalar function: f(x) = x^3 + 2x, so f'(x) = 3x^2 + 2
y, grad = compute_gradient(lambda x: x ** 3 + 2 * x, 2.0)
print(y.numpy(), grad.numpy()) # 12.0 14.0
# 2. Multivariate function: f(x, y) = x^2 * y + sin(y), gradient = [2xy, x^2 + cos(y)]
def f(v):
return v[0] ** 2 * v[1] + tf.sin(v[1])
y, grad = compute_gradient(f, [1.0, 0.0])
print(grad.numpy()) # [0. 2.] (2*1*0 = 0, 1 + cos(0) = 2)
# 3. Second derivative with nested tapes: f''(x) = 6x for f(x) = x^3 + 2x
x = tf.Variable(2.0)
with tf.GradientTape() as outer:
with tf.GradientTape() as inner:
y = x ** 3 + 2 * x
dy_dx = inner.gradient(y, x)
d2y_dx2 = outer.gradient(dy_dx, x)
print(dy_dx.numpy(), d2y_dx2.numpy()) # 14.0 12.0
# 4. Jacobian of a vector-valued function
x = tf.Variable([1.0, 2.0])
with tf.GradientTape() as tape:
out = tf.stack([x[0] * x[1], x[0] + x[1]])
print(tape.jacobian(out, x).numpy()) # [[2. 1.] [1. 1.]]
# 5. Check against a numerical (finite-difference) estimate
eps = 1e-3
numeric = ((2.0 + eps) ** 3 + 2 * (2.0 + eps) - ((2.0 - eps) ** 3 + 2 * (2.0 - eps))) / (2 * eps)
print(round(numeric, 3)) # close to 14.0
Useful Tape Options
Option
Purpose
tape.watch(t)
Track a constant tensor
GradientTape(persistent=True)
Call gradient more than once (delete the tape afterwards)
Getting None because the computation used NumPy or Python math instead of TensorFlow ops, or the source was not watched.
Integer tensors: gradients are only defined for floating-point types.
Calling gradient twice on a non-persistent tape.
Non-differentiable operations (tf.argmax, tf.round) give no useful gradient.
53
Write code to save and load a trained TensorFlow model.
KnownRevise
Saving and loading a trained model lets you reuse it without retraining, resume training later, or deploy it. In TensorFlow with Keras 3, use model.save("name.keras") and keras.models.load_model("name.keras") to store the architecture, weights, optimizer state and compile settings in one file; custom layers must be registered so they can be rebuilt. For serving, export a SavedModel with model.export(...), and for long training jobs use checkpoints to resume.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(500, 6).astype("float32")
y = (X @ np.arange(1, 7, dtype="float32") + 0.1).reshape(-1, 1)
@keras.saving.register_keras_serializable(package="demo")
class Scale(keras.layers.Layer):
"""Multiplies inputs by a constant factor (a custom layer to show serialization)."""
def __init__(self, factor=1.0, **kwargs):
super().__init__(**kwargs)
self.factor = factor
def call(self, x):
return x * self.factor
def get_config(self):
return {**super().get_config(), "factor": self.factor}
model = keras.Sequential([keras.Input(shape=(6,)), Scale(0.5),
keras.layers.Dense(32, activation="relu"), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss="mse")
model.fit(X, y, epochs=20, verbose=0)
# --- Save and load the whole model ---
model.save("regressor.keras")
loaded = keras.models.load_model("regressor.keras")
assert np.allclose(model.predict(X[:5], verbose=0), loaded.predict(X[:5], verbose=0))
loaded.fit(X, y, epochs=5, verbose=0) # optimizer state was restored, so training continues
# --- Weights only (architecture comes from code) ---
model.save_weights("regressor.weights.h5")
same_arch = keras.Sequential([keras.Input(shape=(6,)), Scale(0.5),
keras.layers.Dense(32, activation="relu"), keras.layers.Dense(1)])
same_arch.load_weights("regressor.weights.h5")
# --- SavedModel for TF Serving / LiteRT ---
model.export("regressor_savedmodel")
reloaded = tf.saved_model.load("regressor_savedmodel")
print(reloaded.serve(tf.constant(X[:2])).numpy()) # inference only; no Keras API
# --- Checkpoint to resume an interrupted training run ---
ckpt = tf.train.Checkpoint(model=model, optimizer=model.optimizer)
manager = tf.train.CheckpointManager(ckpt, "ckpts", max_to_keep=3)
manager.save()
ckpt.restore(manager.latest_checkpoint)
print("restored from", manager.latest_checkpoint)
Which Format to Use
Goal
Format
Share or reload a Keras model, continue training
.keras
Same code, just new weights
.weights.h5
Production serving, other languages, LiteRT conversion
SavedModel via model.export
Fault-tolerant training
ModelCheckpoint, BackupAndRestore, or tf.train.CheckpointManager
Common Pitfalls
Custom objects not registered (or not passed via custom_objects), which makes load_model fail.
Saving with model.save("dir") without an extension, which Keras 3 rejects; use .keras or model.export.
Loading weights into a model whose layers differ from the saved one.
Loading .keras files from untrusted sources: keep safe_mode=True (the default), which blocks arbitrary code in Lambda layers.
54
Implement a custom training loop in TensorFlow for a basic neural network.
KnownRevise
A custom training loop replaces model.fit with your own code: for each batch, run the forward pass under tf.GradientTape, compute the loss, get gradients, and apply them with an optimizer, while updating metrics and running validation yourself. It gives full control for things fit does not support easily, such as GANs, multiple optimizers, unusual losses or gradient manipulation. Wrapping the step in tf.function keeps it as fast as fit.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, scikit-learn 1.5+ (Python 3.12)
Code Example
import keras
import tensorflow as tf
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
keras.utils.set_random_seed(42)
X, y = make_classification(n_samples=3000, n_features=20, n_informative=10, n_classes=3, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
scaler = StandardScaler().fit(X_train)
X_train, X_val = scaler.transform(X_train).astype("float32"), scaler.transform(X_val).astype("float32")
train_ds = tf.data.Dataset.from_tensor_slices((X_train, y_train)).shuffle(len(X_train), seed=42).batch(64)
val_ds = tf.data.Dataset.from_tensor_slices((X_val, y_val)).batch(256)
model = keras.Sequential([
keras.Input(shape=(20,)),
keras.layers.Dense(64, activation="relu"),
keras.layers.Dropout(0.2),
keras.layers.Dense(3), # logits
])
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
optimizer = keras.optimizers.Adam(1e-3)
train_loss = keras.metrics.Mean()
train_acc = keras.metrics.SparseCategoricalAccuracy()
val_acc = keras.metrics.SparseCategoricalAccuracy()
@tf.function
def train_step(x, y):
with tf.GradientTape() as tape:
logits = model(x, training=True) # dropout active
loss = loss_fn(y, logits) + sum(model.losses) # include any regularization losses
grads = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
train_loss.update_state(loss)
train_acc.update_state(y, logits)
@tf.function
def val_step(x, y):
val_acc.update_state(y, model(x, training=False)) # dropout off
best_acc, patience, wait = 0.0, 5, 0
for epoch in range(50):
for metric in (train_loss, train_acc, val_acc):
metric.reset_state()
for x, y in train_ds:
train_step(x, y)
for x, y in val_ds:
val_step(x, y)
acc = float(val_acc.result())
print(f"epoch {epoch + 1}: loss {float(train_loss.result()):.4f} "
f"train acc {float(train_acc.result()):.3f} val acc {acc:.3f}")
if acc > best_acc: # simple early stopping
best_acc, wait = acc, 0
model.save_weights("best.weights.h5")
else:
wait += 1
if wait >= patience:
break
model.load_weights("best.weights.h5")
What fit Did for You
Responsibility
Where it is now
Forward pass and loss
train_step
Gradients and update
GradientTape + optimizer.apply_gradients
Training vs inference behaviour
training=True/False
Metrics
keras.metrics objects, reset each epoch
Validation, early stopping, checkpoints
Epoch loop
Speed
@tf.function
Common Pitfalls
Forgetting training=True, so dropout and batch normalization behave as at inference.
Not resetting metrics each epoch, which averages over all epochs.
Leaving out model.losses, which silently drops L1/L2 regularization.
Creating the optimizer or variables inside the tf.function.
55
Write a TensorFlow program using dataset shuffle, repeat and batch.
KnownRevise
shuffle, repeat and batch are the three core tf.data transformations for training input: shuffle(buffer_size) randomizes example order using a buffer, repeat(count) loops over the data for several epochs, and batch(size) groups consecutive examples into tensors. Their order matters: shuffling before batching shuffles individual examples, and putting repeat before or after shuffle changes whether epoch boundaries are blurred. The usual training order is shuffle → batch → prefetch, letting model.fit handle epochs.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Code Example
import keras
import numpy as np
import tensorflow as tf
ds = tf.data.Dataset.range(10) # 0, 1, ..., 9
# 1. shuffle -> batch: examples are mixed, then grouped
print("shuffle then batch:")
for batch in ds.shuffle(buffer_size=10, seed=42).batch(4):
print(batch.numpy()) # three batches: 4, 4, 2 elements
# 2. batch -> shuffle: only whole batches move; contents stay [0-3], [4-7], [8-9]
print("batch then shuffle:")
for batch in ds.batch(4).shuffle(buffer_size=3, seed=42):
print(batch.numpy())
# 3. shuffle -> repeat: each epoch is a full pass, reshuffled every epoch
epochs = ds.shuffle(10, seed=42, reshuffle_each_iteration=True).repeat(2).batch(5)
print("shuffle then repeat (2 epochs):", [b.numpy().tolist() for b in epochs])
# 4. repeat -> shuffle: epoch boundaries blur; an element may appear twice before another appears once
blurred = ds.repeat(2).shuffle(20, seed=42).batch(5)
print("repeat then shuffle:", [b.numpy().tolist() for b in blurred])
# 5. A training pipeline used with fit
keras.utils.set_random_seed(42)
X = np.random.rand(1000, 8).astype("float32")
y = (X.sum(axis=1) > 4).astype("float32")
train = (tf.data.Dataset.from_tensor_slices((X, y))
.shuffle(buffer_size=1000, seed=42) # buffer = dataset size: a full shuffle
.batch(32, drop_remainder=True)
.prefetch(tf.data.AUTOTUNE))
model = keras.Sequential([keras.Input(shape=(8,)), keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(train, epochs=5, verbose=0) # fit iterates the dataset once per epoch
# With an infinite dataset (repeat()), tell fit how many steps make an epoch
model.fit(train.repeat(), epochs=2, steps_per_epoch=1000 // 32, verbose=0)
The exact shuffled orders depend on the seed and TensorFlow version, so they are not listed here.
How Each Transformation Behaves
Transformation
Key arguments
Notes
shuffle
buffer_size, seed, reshuffle_each_iteration
Fills a buffer and samples from it; a buffer smaller than the dataset gives only a partial shuffle
repeat
count (None = forever)
Needed for custom loops counting steps, or with steps_per_epoch
batch
batch_size, drop_remainder
drop_remainder=True for fixed shapes (TPUs)
Common Pitfalls
A small shuffle buffer on data sorted by label, so batches contain one class.
repeat() without steps_per_epoch in fit, which never ends an epoch.
Shuffling validation or test data needlessly (harmless for metrics, but it makes outputs harder to match to rows).
56
Write a TensorFlow function for image dataset augmentation.
KnownRevise
Data augmentation creates modified versions of training images (flips, rotations, crops, brightness and contrast changes) so the model sees more variety and generalizes better, especially with small datasets. In TensorFlow you can augment with Keras preprocessing layers, which can run inside the model or in a tf.data pipeline, or with tf.image functions in a map step. Augmentation is applied only to training data, and each epoch sees different random variations.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Code Example
import keras
import tensorflow as tf
keras.utils.set_random_seed(42)
IMG_SIZE = 64
# Option 1: Keras preprocessing layers
keras_augment = keras.Sequential([
keras.layers.RandomFlip("horizontal"),
keras.layers.RandomRotation(0.1), # up to ±10% of a full turn (±36 degrees)
keras.layers.RandomZoom(0.1),
keras.layers.RandomContrast(0.2),
], name="augment")
# Option 2: a tf.image function for use in Dataset.map
def augment_image(image, label):
"""Random crop, flip, brightness and saturation on one image in [0, 1]."""
image = tf.image.resize_with_crop_or_pad(image, IMG_SIZE + 8, IMG_SIZE + 8) # pad by 4 px per side
image = tf.image.random_crop(image, size=[IMG_SIZE, IMG_SIZE, 3])
image = tf.image.random_flip_left_right(image)
image = tf.image.random_brightness(image, max_delta=0.1)
image = tf.image.random_saturation(image, lower=0.8, upper=1.2)
return tf.clip_by_value(image, 0.0, 1.0), label
# Example data: random "images" so the snippet runs anywhere
images = tf.random.uniform([100, IMG_SIZE, IMG_SIZE, 3], seed=42)
labels = tf.random.uniform([100], maxval=5, dtype=tf.int32, seed=42)
train_ds = (tf.data.Dataset.from_tensor_slices((images, labels))
.shuffle(100, seed=42)
.map(augment_image, num_parallel_calls=tf.data.AUTOTUNE) # per-image tf.image ops
.batch(32)
.map(lambda x, y: (keras_augment(x, training=True), y), # batched Keras layers
num_parallel_calls=tf.data.AUTOTUNE)
.prefetch(tf.data.AUTOTUNE))
for x, y in train_ds.take(1):
print(x.shape, float(tf.reduce_min(x)), float(tf.reduce_max(x)))
# Option 3: augmentation inside the model (active only during fit)
model = keras.Sequential([
keras.Input(shape=(IMG_SIZE, IMG_SIZE, 3)),
keras_augment,
keras.layers.Conv2D(16, 3, activation="relu"),
keras.layers.GlobalAveragePooling2D(),
keras.layers.Dense(5, activation="softmax"),
])
Pipeline Versus In-Model
Placement
Pros
Cons
In tf.data (CPU)
Runs in parallel with GPU training
Must remember to skip it for validation
Inside the model (GPU)
Automatically off at inference; travels with the model
Uses GPU time; exported model contains inactive layers
Choosing Augmentations
Use transformations that keep the label valid: horizontal flips suit cats and dogs, but not digits or text, and medical images may need domain-specific choices.
Start mild and increase strength if the model still overfits.
For detection or segmentation, transform boxes or masks together with the image.
Common Pitfalls
Augmenting validation or test data.
Augmenting before cache(), which freezes one random version.
Colour augmentations producing values outside the expected range; clip afterwards.
57
How would you debug a TensorFlow model that isn't learning?
KnownRevise
A model that isn’t learning shows a training loss that stays flat, decreases very slowly, oscillates or becomes NaN. Debug it systematically from the data to the model to the optimizer: confirm the data and labels are correct, check that the loss and output layer match the task, make the model overfit a tiny batch, then tune the learning rate and inspect gradients. Most failures come from data bugs, a wrong loss or activation, a bad learning rate, or preprocessing errors, not from the architecture.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Checklist
Area
What to check
Data
Inputs and labels aligned after shuffling; labels in the expected format; no NaNs; features scaled
Baseline
Loss at initialization matches theory (e.g. log(num_classes) for balanced softmax); compare with predicting the majority class
Output/loss pairing
Sigmoid + binary cross-entropy, softmax + categorical, from_logits set correctly; regression output with no activation
Overfit a tiny batch
A correct model drives the loss near zero on ~32 examples; if not, there is a bug
Learning rate
Too high: loss oscillates or explodes; too low: barely moves. Try 1e-4 to 1e-2 or a learning-rate range test
Gradients
Zero (dead ReLUs, frozen layers, trainable=False), vanishing or exploding norms
Architecture
Too small to fit, or too deep without normalization/residual connections
Training flags
training=True in custom loops; BatchNorm with tiny batches
Code Example: Overfit a Small Batch and Inspect Gradients
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
X = np.random.rand(32, 20).astype("float32")
y = np.random.randint(0, 5, size=32)
model = keras.Sequential([keras.Input(shape=(20,)), keras.layers.Dense(64, activation="relu"),
keras.layers.Dense(5)]) # logits
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
# 1. Initial loss should be close to log(5) = 1.609 for 5 balanced classes
print("initial loss:", float(loss_fn(y, model(X))), "expected about", round(np.log(5), 3))
# 2. Gradient norms per layer
with tf.GradientTape() as tape:
loss = loss_fn(y, model(X, training=True))
for var, grad in zip(model.trainable_variables, tape.gradient(loss, model.trainable_variables)):
print(f"{var.path:30s} grad norm {float(tf.norm(grad)):.4f}")
# 3. A correct model should memorize 32 random examples
model.compile(optimizer=keras.optimizers.Adam(1e-2), loss=loss_fn, metrics=["accuracy"])
hist = model.fit(X, y, epochs=200, verbose=0)
print("final loss on the tiny batch:", round(hist.history["loss"][-1], 4))
If the tiny-batch loss does not approach zero, look for a bug in labels, loss configuration or frozen layers before trying anything else.
Tools
TensorBoard: loss curves, weight and gradient histograms (histogram_freq=1).
tf.debugging.enable_check_numerics() for NaN losses.
keras.callbacks.TerminateOnNaN.
Common Pitfalls
Softmax in the last layer combined with from_logits=True.
Labels shuffled separately from features.
Unscaled inputs (e.g. raw pixel values or features in the thousands).
Changing many things at once, which makes it impossible to know what fixed the problem.
58
What are common TensorFlow errors, and how do you resolve them?
KnownRevise
Common TensorFlow errors fall into a few groups: shape and dtype mismatches, problems with tf.function tracing, GPU setup and memory errors, version and API changes (notably the move to Keras 3 in TensorFlow 2.16), and numerical issues such as NaN losses. Most error messages name the operation and the offending values, so reading the first and last lines of the traceback carefully usually points to the fix.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
import numpy as np
import tensorflow as tf
a = tf.constant([1, 2, 3]) # int32
b = tf.constant([0.5, 0.5, 0.5]) # float32
try:
a + b
except tf.errors.InvalidArgumentError as err:
print("dtype error:", type(err).__name__)
print((tf.cast(a, tf.float32) + b).numpy()) # [1.5 2.5 3.5]
x64 = np.random.default_rng(42).random((4, 3)) # NumPy defaults to float64
print(tf.constant(x64).dtype, "->", tf.constant(x64, dtype=tf.float32).dtype)
@tf.function
def relu_like(x):
if tf.reduce_sum(x) > 0: # fine: AutoGraph converts this to tf.cond
return x
return tf.zeros_like(x)
print(relu_like(tf.constant([1.0, -0.5])).numpy())
Debugging Habits
Pin package versions and check the TensorFlow, Keras, CUDA and Python compatibility table when upgrading.
Reproduce with a tiny batch in eager mode (run_eagerly=True), then switch back.
Search for the operation name in the error; it often identifies the exact layer.
59
How does TensorFlow's tf.debugging package help with debugging?
KnownRevise
tf.debugging is TensorFlow’s module of runtime checks and diagnostic tools. Its assertions (assert_shapes, assert_equal, assert_non_negative, check_numerics) stop execution with a clear message the moment a tensor has the wrong shape, type or value, instead of letting a bad value spread and fail far away. It also provides global switches: enable_check_numerics finds the first operation that produces NaN or infinity, and experimental.enable_dump_debug_info records tensors for inspection in TensorBoard’s Debugger V2.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Main Tools
Tool
What it checks
assert_shapes([(x, ("N", "D")), (w, ("D", "K"))])
Shapes and consistent symbolic dimensions across tensors
NaN/inf in every operation, with the op and Python stack
experimental.enable_dump_debug_info(logdir)
Dumps tensors and graphs for Debugger V2 in TensorBoard
set_log_device_placement(True)
Shows which device runs each op
Related helpers: tf.config.run_functions_eagerly(True) to step through tf.function code, and tf.print for printing inside graphs.
Code Example
import tensorflow as tf
tf.random.set_seed(42)
def dense_layer(x, w, b):
tf.debugging.assert_shapes([(x, ("N", "D")), (w, ("D", "K")), (b, ("K",))])
tf.debugging.assert_all_finite(x, "input contains NaN or inf")
return x @ w + b
x = tf.random.normal([8, 4])
w = tf.random.normal([4, 3])
b = tf.zeros([3])
print(dense_layer(x, w, b).shape) # (8, 3)
try:
dense_layer(x, tf.random.normal([5, 3]), b) # D mismatch: 4 vs 5
except (ValueError, tf.errors.InvalidArgumentError) as err:
print("shape check failed:", type(err).__name__)
def safe_log_loss(p):
tf.debugging.assert_non_negative(p, message="probabilities must be >= 0")
return tf.debugging.check_numerics(-tf.math.log(p), "log produced inf")
try:
safe_log_loss(tf.constant([0.5, 0.0])) # log(0) = -inf
except tf.errors.InvalidArgumentError as err:
print("numeric check failed:", err.message.splitlines()[0])
# Global NaN/inf detection: reports the first op that produces a bad value
tf.debugging.enable_check_numerics()
try:
tf.sqrt(tf.constant(-1.0)) # NaN
except tf.errors.InvalidArgumentError:
print("enable_check_numerics caught the NaN in Sqrt")
tf.debugging.disable_check_numerics()
Workflow for NaN Losses
Turn on tf.debugging.enable_check_numerics() and rerun a few steps.
Read the reported op and stack trace (often Log, Div, Exp or Sqrt).
Fix the cause: use from_logits=True, add epsilons, clip values, lower the learning rate, or check the input data.
For deeper inspection, record with enable_dump_debug_info("logs/debug", tensor_debug_mode="FULL_HEALTH") and open the Debugger V2 tab in TensorBoard.
Common Pitfalls
Leaving global debugging switches on in production; they slow execution considerably.
Using Python assert inside a tf.function: it runs only while tracing, whereas tf.debugging.assert_* ops run on every call.
Checking shapes with Python if on dynamic dimensions; use assert_shapes, which handles unknown sizes.
60
How would you investigate and fix a shape mismatch error in TensorFlow?
KnownRevise
A shape mismatch error means an operation received tensors whose dimensions do not fit its rules, for example a matrix multiply with incompatible inner sizes or a model fed inputs with the wrong number of features. The fix starts with reading the error message, which names the operation and both shapes, then tracing back to the layer or data step that produced the unexpected shape. Most cases come from a missing batch dimension, wrong label format, a flatten or reshape mistake, or data and model definitions that drifted apart.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Step-by-Step Investigation
Read the full error: it states the op (MatMul, Add, ConcatV2) or layer and the incompatible shapes, e.g. “expected axis -1 of input shape to have value 20, but received input with shape (None, 18)”.
Print shapes at each stage: model.summary() for layer outputs, and inspect a batch with for x, y in ds.take(1): print(x.shape, y.shape).
Check the data side: number of features, image size and channels, label shape and encoding.
Check the model side: Input shape, Flatten/Reshape, return_sequences, output units.
Add assertions with tf.debugging.assert_shapes in custom code, and run eagerly (run_eagerly=True in compile) for clearer tracebacks.
Fix at the source rather than reshaping blindly to make the error disappear.
Common Causes and Fixes
Symptom
Cause
Fix
Expected (None, 20), got (None, 18)
Feature columns changed after preprocessing
Rebuild features consistently; keep preprocessing inside the model
Expected 4 dims, got 3 (images)
Missing batch or channel dimension
x[None, ...] for one sample; x[..., None] for grayscale
Loss error with labels (32, 1) vs logits (32, 10)
One-hot vs integer label mismatch
sparse_categorical_crossentropy for integers, categorical_crossentropy for one-hot
Logits (32, 50, 10) vs labels (32,)
RNN with return_sequences=True before the classifier
Set return_sequences=False on the last RNN layer or add pooling
MatMul incompatible inner dims
Wrong transpose or weight shape
Check orders; use tf.einsum with named axes
Concatenation shape error
Branch outputs differ on non-concat axes
Align sizes with pooling or Resizing
Shape changes per batch in tf.function
Variable lengths
padded_batch, None in input_signature, or drop_remainder=True
Code Example
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(28, 28, 1)), keras.layers.Conv2D(8, 3, activation="relu"),
keras.layers.Flatten(), keras.layers.Dense(10, activation="softmax")])
image = np.random.rand(28, 28).astype("float32") # a single grayscale image without batch/channel
try:
model.predict(image, verbose=0)
except (ValueError, tf.errors.InvalidArgumentError) as err:
print("error:", str(err).splitlines()[0][:100])
fixed = image[None, ..., None] # add batch and channel dimensions
print("input shape:", fixed.shape) # (1, 28, 28, 1)
print("output shape:", model.predict(fixed, verbose=0).shape) # (1, 10)
# Inspect shapes layer by layer
for layer in model.layers:
print(f"{layer.name:12s} -> {layer.output.shape}")
# Catch problems early in custom code
labels = tf.constant([3, 1, 4])
tf.debugging.assert_shapes([(labels, ("N",))])
Interview Tips
Mention None as the batch dimension in Keras shapes and why it is flexible.
Distinguish static shapes (x.shape, known when tracing) from dynamic shapes (tf.shape(x), known at runtime).
Emphasize fixing the root cause and adding a shape assertion so the error cannot silently come back.
61
How can TensorBoard be used to debug TensorFlow programs?
KnownRevise
TensorBoard is TensorFlow’s visualization tool. For debugging it shows training and validation curves to spot overfitting, divergence or a stalled loss; histograms of weights and gradients to find dead or exploding layers; the model graph; images and text to check what the model actually receives; the Profiler to find performance bottlenecks; and Debugger V2 to trace where NaN or infinite values first appear. Logging is a one-line Keras callback, and custom values can be written with tf.summary.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, TensorBoard 2.16+ (Python 3.12)
Weights not changing, dead ReLUs, exploding activations or gradients
Graphs
Model structure, unexpected ops, tf.function graphs
Images
Verify inputs and augmentations look right; view predictions
Text
Check tokenization or generated outputs
HParams
Compare runs across hyperparameter settings
Profiler
Input-bound steps, slow ops, GPU idle time, memory use
Debugger V2
Find the first op producing NaN/inf, with its stack trace
Projector
Inspect embeddings
Code Example
import datetime
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
(x_train, y_train), _ = keras.datasets.mnist.load_data() # downloads the data
x_train = x_train[:5000, ..., None].astype("float32") / 255
y_train = y_train[:5000]
log_dir = "logs/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
model = keras.Sequential([keras.Input(shape=(28, 28, 1)), keras.layers.Conv2D(16, 3, activation="relu"),
keras.layers.Flatten(), keras.layers.Dense(10, activation="softmax")])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
# Log a few input images to confirm preprocessing
writer = tf.summary.create_file_writer(log_dir + "/inputs")
with writer.as_default():
tf.summary.image("training samples", x_train[:9], max_outputs=9, step=0)
tensorboard_cb = keras.callbacks.TensorBoard(
log_dir=log_dir,
histogram_freq=1, # weight histograms every epoch
profile_batch=(10, 20), # profile batches 10-20
)
model.fit(x_train, y_train, validation_split=0.2, epochs=3, verbose=0, callbacks=[tensorboard_cb])
# Custom scalar from your own code
with tf.summary.create_file_writer(log_dir + "/custom").as_default():
for step in range(3):
tf.summary.scalar("custom/learning_rate", 1e-3 * 0.5 ** step, step=step)
# Optional: record tensors for Debugger V2 before training
# tf.debugging.experimental.enable_dump_debug_info(log_dir + "/debug", tensor_debug_mode="FULL_HEALTH")
Start it with tensorboard --logdir logs and open http://localhost:6006, or use %load_ext tensorboard and %tensorboard --logdir logs in a notebook.
Reading the Signals
Validation loss rising while training loss falls: overfitting.
Loss flat from the start: data, loss or learning-rate bug.
Histograms frozen across epochs: layer not training (frozen or zero gradients).
Profiler shows a high “input” share: fix the tf.data pipeline.
Common Pitfalls
Writing all runs into the same folder, which mixes curves; use one subfolder per run.
Heavy histogram or image logging every batch, which slows training.
Leaving Debugger V2 dumping on for long runs; it uses a lot of disk.
62
How does GradientTape work in TensorFlow?
KnownRevise
tf.GradientTape records the operations executed inside its context so TensorFlow can compute gradients with reverse-mode automatic differentiation. During the forward pass, the tape stores each operation and the intermediate values it needs; when you call tape.gradient(target, sources), it walks the recorded operations backwards, applying the chain rule to get the derivative of the target with respect to each source. It is the engine behind Keras training and custom training loops.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
How It Works
Entering with tf.GradientTape() as tape: starts recording.
Trainable tf.Variables accessed inside are watched automatically; constant tensors need tape.watch(t).
Each op is recorded with references to its inputs and outputs.
tape.gradient(loss, vars) starts with d(loss)/d(loss) = 1 and propagates backwards: each op’s registered gradient function turns the gradient of its output into gradients of its inputs.
By default the tape releases its resources after one gradient call.
Reverse mode is efficient when there are many inputs (weights) and one scalar output (loss), which is exactly the neural network case.
Code Example
import tensorflow as tf
tf.random.set_seed(42)
# Basic: d/dw of (w * x - y)^2
w = tf.Variable(2.0)
x, y = tf.constant(3.0), tf.constant(5.0)
with tf.GradientTape() as tape:
loss = (w * x - y) ** 2 # (6 - 5)^2 = 1
print(tape.gradient(loss, w).numpy()) # 2 * (w*x - y) * x = 2 * 1 * 3 = 6.0
# Persistent tape: several gradients from one recording
a = tf.Variable(1.0)
b = tf.Variable(2.0)
with tf.GradientTape(persistent=True) as tape:
u = a * b
v = u + b ** 2
print(tape.gradient(u, a).numpy(), tape.gradient(v, b).numpy()) # 2.0 and a + 2b = 5.0
del tape # free resources
# Stopping gradients and unwatched values
c = tf.Variable(3.0)
with tf.GradientTape() as tape:
z = c * tf.stop_gradient(c) # second factor treated as a constant
print(tape.gradient(z, c).numpy()) # 3.0 instead of 6.0
# Gradient of a model's loss with respect to its weights
dense = tf.keras.layers.Dense(1)
inputs = tf.random.normal([4, 3])
with tf.GradientTape() as tape:
out = dense(inputs)
loss = tf.reduce_mean(out ** 2)
grads = tape.gradient(loss, dense.trainable_variables)
print([g.shape for g in grads]) # [TensorShape([3, 1]), TensorShape([1])]
# Gradient with respect to the input (saliency maps, adversarial examples)
with tf.GradientTape() as tape:
tape.watch(inputs)
score = tf.reduce_sum(dense(inputs))
print(tape.gradient(score, inputs).shape) # (4, 3)
Key Behaviours
Behaviour
Detail
Returns None
Source not used, not watched, or computation left TensorFlow (NumPy, .numpy(), Python math)
Integer tensors
Not differentiable; cast to float
Higher-order derivatives
Nest tapes
Control flow
Only the branch actually executed is recorded
Custom gradients
@tf.custom_gradient defines your own backward function
Common Pitfalls
Calling gradient twice on a non-persistent tape.
Computing the loss outside the with block, so nothing is recorded.
Converting to NumPy in the middle of the computation, which breaks the chain.
63
How would you implement attention mechanisms in TensorFlow?
KnownRevise
An attention mechanism lets a model decide, for each output position, how much to focus on each input position: it scores how relevant every input (key) is to the current query, turns the scores into weights with softmax, and returns the weighted average of the inputs’ values. In TensorFlow you can use the built-in Keras layers (MultiHeadAttention, Attention, AdditiveAttention) or write scaled dot-product attention yourself in a few lines, which is a common interview exercise.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Scaled Dot-Product Attention in Words
Compare each query with each key by a dot product.
Divide by the square root of the key size so large dimensions don’t push softmax into extreme values.
Optionally mask positions (padding, or future tokens in a decoder) by setting their scores to a large negative number.
Apply softmax over the keys to get weights that sum to 1.
Multiply the weights by the values to get the output.
Keras Layers
Layer
Type
Typical use
keras.layers.Attention
Dot-product (Luong-style)
Seq2seq decoder attending to encoder outputs
keras.layers.AdditiveAttention
Additive (Bahdanau-style)
Same, with a small learned scoring network
keras.layers.MultiHeadAttention
Several scaled dot-product heads in parallel
Transformers, self- and cross-attention
Code Example
import keras
import tensorflow as tf
tf.random.set_seed(42)
def scaled_dot_product_attention(q, k, v, mask=None):
"""q: (..., Tq, d), k: (..., Tk, d), v: (..., Tk, dv); mask: 1 = keep, 0 = block."""
scores = tf.matmul(q, k, transpose_b=True) # (..., Tq, Tk)
scores /= tf.math.sqrt(tf.cast(tf.shape(k)[-1], tf.float32))
if mask is not None:
scores += (1.0 - tf.cast(mask, tf.float32)) * -1e9
weights = tf.nn.softmax(scores, axis=-1)
return tf.matmul(weights, v), weights
batch, seq_len, dim = 2, 5, 8
x = tf.random.normal([batch, seq_len, dim])
# Causal mask: position i may only attend to positions <= i
causal = tf.linalg.band_part(tf.ones([seq_len, seq_len]), -1, 0)
out, w = scaled_dot_product_attention(x, x, x, mask=causal)
print(out.shape, w.shape) # (2, 5, 8) (2, 5, 5)
print(w[0, 0].numpy().round(3)) # first position attends only to itself: [1. 0. 0. 0. 0.]
# Built-in multi-head self-attention with a causal mask
mha = keras.layers.MultiHeadAttention(num_heads=2, key_dim=4)
y, scores = mha(x, x, use_causal_mask=True, return_attention_scores=True)
print(y.shape, scores.shape) # (2, 5, 8) (2, 2, 5, 5): one weight matrix per head
# Attention pooling for classification: learn which time steps matter
inputs = keras.Input(shape=(20, 16))
h = keras.layers.LSTM(32, return_sequences=True)(inputs)
query = keras.layers.GlobalAveragePooling1D(keepdims=True)(h) # (batch, 1, 32)
context = keras.layers.Attention()([query, h]) # (batch, 1, 32)
outputs = keras.layers.Dense(1, activation="sigmoid")(keras.layers.Flatten()(context))
model = keras.Model(inputs, outputs)
model.summary()
Self-Attention Versus Cross-Attention
Self-attention: queries, keys and values come from the same sequence (encoder layers).
Cross-attention: queries come from the decoder, keys and values from the encoder output (translation, captioning).
Multi-head: several attention heads with different learned projections capture different relationships, then their outputs are concatenated.
Common Pitfalls
Forgetting the scaling factor, which makes softmax too peaked and gradients small.
Missing padding masks, so the model attends to padding tokens.
Memory grows with the square of the sequence length; long inputs need truncation, chunking or efficient attention variants.
64
What is the difference between graph mode and eager mode in TensorFlow?
KnownRevise
Eager mode runs each TensorFlow operation immediately from Python and returns real values, while graph mode first builds a dataflow graph of the whole computation and then executes it as a unit. TensorFlow 2 is eager by default; you get graph mode by decorating a function with tf.function, which traces the Python code once per input signature into a graph that can be optimized, run without the Python interpreter, and exported. The usual pattern is to develop eagerly and run training and serving in graph mode.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12)
Comparison
Aspect
Eager mode
Graph mode (tf.function)
Execution
Op by op, from Python
Whole graph after tracing
Values
Concrete (.numpy() works)
Symbolic during tracing
Python code
Runs every call
Runs only while tracing
Control flow
Python if/for
AutoGraph converts tensor-dependent if/while to graph ops
Optimization
None across ops
Grappler rewrites, optional XLA fusion
Portability
Needs Python
Saved as SavedModel; runs in TF Serving, LiteRT, C++
Debugging
Easy
Harder; tf.print, run_functions_eagerly(True)
Speed
Python overhead per op
Faster for many small ops; similar for a few large ops
Tracing and Polymorphism
A tf.function creates one concrete function (graph) per distinct input signature: tensor shapes and dtypes, plus the values of Python arguments. Calling it with a new shape or a new Python value triggers a retrace. Fix the signature to control this.
Code Example
import tensorflow as tf
@tf.function
def square_sum(x):
print("Tracing with", x.shape) # Python: runs only while tracing
return tf.reduce_sum(x ** 2)
square_sum(tf.constant([1.0, 2.0])) # traces for shape (2,)
square_sum(tf.constant([3.0, 4.0])) # reuses the graph: no print
square_sum(tf.constant([1.0, 2.0, 3.0])) # new shape (3,) -> traces again
@tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.float32)])
def square_sum_any(x):
print("Tracing once")
return tf.reduce_sum(x ** 2)
for n in (2, 3, 10):
square_sum_any(tf.range(n, dtype=tf.float32)) # one trace for any length
# Inspect the graph behind a concrete function
concrete = square_sum_any.get_concrete_function()
print([op.type for op in concrete.graph.get_operations()][:6])
# Keras uses graph mode in fit by default; switch to eager for debugging
# model.compile(..., run_eagerly=True)
When to Use Which
Eager: exploring data, writing and debugging model code, small one-off computations.
Graph: training loops, inference servers, export to SavedModel, mobile conversion, anything performance-critical.
Common Pitfalls
Expecting Python side effects (prints, list appends, counters) to happen on every call of a tf.function.
Retracing on every step because a Python scalar such as the step number is passed instead of a tensor.
Creating tf.Variables inside a tf.function on each call.
65
How is TensorFlow used in natural language processing?
KnownRevise
TensorFlow is used in NLP to build and deploy models for text classification, sentiment analysis, named-entity recognition, translation, summarization, question answering and text generation. The toolkit covers the whole pipeline: the TextVectorization layer and TensorFlow Text for tokenization, Keras layers (Embedding, LSTM, MultiHeadAttention) for models, KerasHub for pretrained Transformers such as BERT and Gemma, and TF Serving or LiteRT for production. Today most NLP tasks start by fine-tuning a pretrained Transformer rather than training from scratch.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, KerasHub 0.18+ (Python 3.12)
import keras
import numpy as np
keras.utils.set_random_seed(42)
texts = np.array(["loved it, brilliant acting", "boring and far too long", "a great watch",
"terrible plot, awful ending", "wonderful soundtrack", "worst film this year"])
labels = np.array([1, 0, 1, 0, 1, 0])
vectorizer = keras.layers.TextVectorization(max_tokens=20000, output_mode="tf_idf", ngrams=2)
vectorizer.adapt(texts) # adapt on training texts only
model = keras.Sequential([keras.Input(shape=(1,), dtype="string"), vectorizer,
keras.layers.Dense(16, activation="relu"), keras.layers.Dropout(0.3),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(texts[:, None], labels, epochs=20, verbose=0)
print(model.predict(np.array([["a brilliant, great film"]]), verbose=0))
Code Example 2: Fine-Tuning a Pretrained Transformer with KerasHub
import keras
import keras_hub
keras.utils.set_random_seed(42)
texts = ["loved it, brilliant acting", "boring and far too long", "a great watch",
"terrible plot, awful ending"]
labels = [1, 0, 1, 0]
# The preset bundles the tokenizer and preprocessing, so raw strings go straight in
classifier = keras_hub.models.BertClassifier.from_preset("bert_tiny_en_uncased", num_classes=2)
classifier.compile(optimizer=keras.optimizers.AdamW(5e-5),
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"])
classifier.fit(x=texts, y=labels, batch_size=2, epochs=3, verbose=0)
print(classifier.predict(["what a brilliant film"], verbose=0)) # logits for the two classes
With real data, evaluate on a held-out test set using accuracy and F1, and compare against the TF-IDF baseline; a Transformer must beat it to justify its cost.
Common Pitfalls
Fitting the vocabulary or tokenizer on test data.
Training large models from scratch on small datasets.
Truncating long documents without checking where the useful text is.
Using a pretrained model with a different tokenizer from the one it was trained with.
66
Which recent TensorFlow features and changes are gaining traction?
KnownRevise
The biggest recent changes in the TensorFlow ecosystem are Keras 3 becoming the default high-level API (multi-backend, from TensorFlow 2.16), the renaming of TensorFlow Lite to LiteRT for on-device inference, and KerasHub as the home for pretrained models, including large language models. Alongside these, XLA compilation through OpenXLA, simpler GPU installation with tensorflow[and-cuda], and NumPy 2 support have matured. TensorFlow releases now focus on stability and deployment, while much new Google research uses JAX, which Keras 3 can also target.
Read full answerHide full answer
Applies toTensorFlow 2.16-2.18+ ecosystem (Python 3.12); check release notes for the latest versions
Notable Changes
Change
What it means
Keras 3 as default (TF 2.16)
One Keras API running on TensorFlow, JAX or PyTorch; .keras save format; model.export for SavedModel; legacy code via tf_keras
KerasHub (formerly KerasNLP and KerasCV)
Pretrained presets such as Gemma, Llama, Mistral, BERT and vision backbones; LoRA fine-tuning with enable_lora
LiteRT (renamed TF Lite)
On-device runtime for TensorFlow, JAX and PyTorch models; standalone ai-edge-litert package; GPU/NPU acceleration
AI Edge Torch / conversion tools
Convert PyTorch models to LiteRT for mobile
OpenXLA
Shared compiler stack; jit_compile=True speedups on GPUs and TPUs
tensorflow[and-cuda]
pip installs matching CUDA libraries on Linux; native Windows GPU support ended after TF 2.10 (use WSL2)
NumPy 2.x support (TF 2.18)
Compatibility with the modern scientific Python stack
Estimator API removed (TF 2.16)
Everything moves to Keras and tf.function
YDF (Yggdrasil Decision Forests)
Successor library to TensorFlow Decision Forests for tree models
Keras distribution API
Data and model parallelism (most complete on the JAX backend)
Code Example: Using Several of These Together
import os
os.environ["KERAS_BACKEND"] = "tensorflow" # or "jax" / "torch" with the same Keras code
import keras
import numpy as np
import tensorflow as tf
keras.utils.set_random_seed(42)
model = keras.Sequential([keras.Input(shape=(16,)), keras.layers.Dense(32, activation="relu"),
keras.layers.Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", jit_compile=True) # XLA
model.fit(np.random.rand(256, 16), np.random.randint(0, 2, 256), epochs=2, verbose=0)
model.save("model.keras") # Keras 3 native format
model.export("saved_model_dir") # SavedModel for serving
converter = tf.lite.TFLiteConverter.from_saved_model("saved_model_dir")
converter.optimizations = [tf.lite.Optimize.DEFAULT]
open("model.tflite", "wb").write(converter.convert()) # runs on the LiteRT runtime
# Pretrained LLM from KerasHub (needs keras-hub, model access on Kaggle, and a GPU)
# import keras_hub
# gemma = keras_hub.models.GemmaCausalLM.from_preset("gemma2_instruct_2b_en")
# gemma.backbone.enable_lora(rank=4) # parameter-efficient fine-tuning
Interview Tips
Show you know the Keras 3 migration issues: model.save needs .keras, tf.keras maps to Keras 3, and tf_keras keeps Keras 2 code running.
Mention the deployment story (SavedModel → TF Serving; LiteRT on device) as TensorFlow’s main strength today.
Be honest about trends: PyTorch dominates research, JAX is popular at Google, and Keras 3 lets teams move between them.
67
How would you use TensorFlow to predict stock market trends?
KnownRevise
Predicting stock market trends with TensorFlow is usually framed as forecasting future returns (or the direction of returns) from past prices, volumes and other features with sequence models such as LSTMs, 1D CNNs or Transformers. The honest answer in an interview is that markets are noisy and highly competitive, so most simple models do not beat naive baselines after costs. A credible approach therefore focuses on correct problem framing, strict time-based validation, leakage prevention and comparison with baselines rather than on model complexity.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, NumPy 2.x, pandas 2.2+ (Python 3.12)
Approach
Step
Good practice
Target
Predict returns or direction (up/down) over a horizon, not raw prices, which are non-stationary
Features
Lagged returns, volatility, volume changes, technical indicators, market index; all computed from past data only
Because these returns are random, a correct pipeline should find no real edge: the model should land near the baseline. That makes this a useful sanity check for leakage; a pipeline that “beats” random data has a bug.
Common Pitfalls
Look-ahead bias: features or scaling that use future data, or random shuffled splits.
Survivorship bias: training only on companies that still exist.
Ignoring transaction costs, slippage and market impact in backtests.
Overfitting by testing many models on the same test period.
68
What steps would you take to develop a chatbot using TensorFlow?
KnownRevise
Developing a chatbot with TensorFlow starts with defining what it must do, then choosing an architecture: a retrieval or intent-based bot that classifies the user’s message and returns or triggers a predefined response, or a generative bot built on a pretrained large language model, usually grounded in your documents with retrieval-augmented generation (RAG). TensorFlow and KerasHub cover intent classifiers, embedding models and LLM fine-tuning, and TF Serving or LiteRT handles deployment; the rest is data, evaluation and safety work.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, KerasHub (Python 3.12)
Steps
Step
Details
1. Scope
Users, tasks (FAQ, bookings, support), channels, languages, what must be escalated to a human
2. Data
Conversation logs, FAQs, documentation; label intents and entities; remove personal data
3. Architecture
Intent + entity models with dialogue rules; retrieval of answers; or LLM + RAG
4. Models
Intent classifier (TextVectorization or a fine-tuned encoder); embeddings for retrieval; LLM such as Gemma via KerasHub
5. Dialogue management
Track state (slots filled, previous turns), fallbacks for low-confidence predictions
Input/output filtering, refusal of out-of-scope requests, prompt-injection defences, rate limits
8. Deployment
TF Serving or a Python API; latency budget; logging and feedback loop for retraining
Code Example: Intent-Based Bot
import keras
import numpy as np
keras.utils.set_random_seed(42)
training = {
"greeting": ["hi", "hello there", "good morning", "hey"],
"hours": ["when are you open", "what are your opening hours", "are you open on sunday"],
"refund": ["i want my money back", "how do i get a refund", "return an item"],
}
responses = {
"greeting": "Hello! How can I help you today?",
"hours": "We are open 9am to 6pm, Monday to Saturday.",
"refund": "You can request a refund within 30 days from your Orders page.",
}
intents = list(training)
texts = np.array([t for intent in intents for t in training[intent]])
labels = np.array([intents.index(intent) for intent in intents for _ in training[intent]])
vectorizer = keras.layers.TextVectorization(output_mode="tf_idf", ngrams=2)
vectorizer.adapt(texts)
model = keras.Sequential([keras.Input(shape=(1,), dtype="string"), vectorizer,
keras.layers.Dense(16, activation="relu"),
keras.layers.Dense(len(intents), activation="softmax")])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")
model.fit(texts[:, None], labels, epochs=150, verbose=0)
def reply(message, threshold=0.6):
probs = model.predict(np.array([[message.lower()]]), verbose=0)[0]
if probs.max() < threshold: # low confidence: don't guess
return "Sorry, I didn't understand. Let me connect you to a person."
return responses[intents[int(probs.argmax())]]
print(reply("hello"))
print(reply("what time do you open"))
A real bot needs far more examples per intent, a held-out evaluation set and an “out of scope” class.
Generative Option
Load a pretrained instruction-tuned LLM, for example keras_hub.models.GemmaCausalLM.from_preset("gemma2_instruct_2b_en"), fine-tune it with LoRA on your support conversations if needed, and add RAG: embed your documents, retrieve the most relevant passages for each question, and put them in the prompt so answers are grounded and up to date.
Common Pitfalls
Letting a generative bot answer without grounding, which produces confident wrong answers.
No fallback or human hand-off.
Logging conversations without consent or redaction.
69
How would you use TensorFlow in an autonomous vehicle perception system?
KnownRevise
An autonomous vehicle perception system turns raw sensor data from cameras, LiDAR and radar into a structured view of the world: detected and tracked objects (cars, pedestrians, cyclists), drivable area and lanes, traffic signs and lights, and distances. TensorFlow would be used to train the deep learning models for these tasks, mainly detection, segmentation and depth estimation, and to export them as optimized graphs for in-vehicle hardware. The hard parts are real-time latency, robustness in rare conditions and safety validation, not just model accuracy.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3 (Python 3.12); in-vehicle inference typically via TensorRT or vendor runtimes
Early (raw data) or late (per-sensor detections) fusion
System Design
Data: large, diverse driving datasets (e.g. Waymo Open Dataset, nuScenes) plus your own fleet data; careful labelling; mining of rare cases (night, rain, unusual objects).
Training: multi-task models sharing a backbone to save compute; tf.data pipelines over TFRecords; multi-GPU training with tf.distribute.
Optimization: mixed precision, pruning, int8 quantization; export a SavedModel and convert with TF-TRT or TensorRT for the vehicle’s GPU.
Runtime: fixed latency budget per frame (tens of milliseconds), redundancy across sensors, timestamps synchronized.
Validation: metrics per class, distance and condition; simulation and closed-course tests; safety standards such as ISO 26262 and ISO 21448 (SOTIF).
Code Example: Lightweight Drivable-Area Segmentation Model
import time
import keras
import numpy as np
from keras import layers
keras.utils.set_random_seed(42)
H, W, NUM_CLASSES = 128, 256, 3 # classes: background, road, lane marking
def conv_bn(x, filters, stride=1):
x = layers.SeparableConv2D(filters, 3, strides=stride, padding="same", use_bias=False)(x)
x = layers.BatchNormalization()(x)
return layers.ReLU()(x)
inputs = keras.Input(shape=(H, W, 3))
x = layers.Rescaling(1.0 / 255)(inputs)
e1 = conv_bn(x, 16, stride=2) # 1/2 resolution
e2 = conv_bn(e1, 32, stride=2) # 1/4
e3 = conv_bn(e2, 64, stride=2) # 1/8
d2 = layers.Concatenate()([layers.UpSampling2D()(e3), e2])
d2 = conv_bn(d2, 32)
d1 = layers.Concatenate()([layers.UpSampling2D()(d2), e1])
d1 = conv_bn(d1, 16)
logits = layers.Conv2D(NUM_CLASSES, 1)(layers.UpSampling2D()(d1))
model = keras.Model(inputs, logits)
model.compile(optimizer="adam", loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[keras.metrics.MeanIoU(num_classes=NUM_CLASSES, sparse_y_pred=False)])
frame = np.random.randint(0, 255, (1, H, W, 3)).astype("float32")
model.predict(frame, verbose=0) # warm-up
start = time.perf_counter()
for _ in range(20):
model.predict_on_batch(frame)
print(f"{(time.perf_counter() - start) / 20 * 1000:.1f} ms per frame on this machine")
Measure latency on the target hardware after conversion; desktop numbers are only indicative.
Common Pitfalls
Optimizing average accuracy while missing rare but critical cases (children, unusual vehicles, glare).
Ignoring calibration of confidence scores, which downstream planning relies on.
Validating only on data from the same cities and weather as training.
70
What approach would you use for real-time object detection with TensorFlow?
KnownRevise
Real-time object detection means locating and classifying objects in each video frame fast enough to keep up with the stream, typically 15-30+ frames per second. The practical approach with TensorFlow is to use a single-stage detector designed for speed (SSD MobileNet, EfficientDet-Lite, YOLO-style models), run it with an optimized runtime (LiteRT on CPUs, phones and edge devices, or TensorRT on NVIDIA GPUs), and fine-tune it on your own classes. Speed comes from model choice, input resolution, quantization and an efficient capture-to-inference pipeline.
Read full answerHide full answer
Applies toTensorFlow 2.16+ with Keras 3, LiteRT, MediaPipe Tasks (Python 3.12)
Choosing a Detector
Model family
Speed
Accuracy
Where it runs well
SSD MobileNet (V2/V3)
Very fast
Moderate
Phones, Raspberry Pi, Edge TPU
EfficientDet-Lite0-4
Fast to moderate
Good, tunable by size
LiteRT on mobile and edge
YOLO-style single-stage models
Fast
Good
GPUs; also exportable to LiteRT
Two-stage (Faster R-CNN)
Slow
High
Offline analysis, not real time
Pipeline
Capture frames (OpenCV, camera API) in a separate thread so capture never waits for inference.
Resize to the model’s input size (e.g. 320×320); convert BGR to RGB.
Run inference with a quantized LiteRT model or a TensorRT engine.
Post-process: score threshold and non-maximum suppression (often built into the model).
Optionally track objects across frames to smooth boxes and skip detection on some frames.
Code Example: Webcam Detection with a LiteRT Model via MediaPipe
MediaPipe’s Object Detector runs LiteRT (.tflite) detection models such as EfficientDet-Lite0 and handles pre- and post-processing.
import time
import cv2
import mediapipe as mp
from mediapipe.tasks import python as mp_python
from mediapipe.tasks.python import vision
options = vision.ObjectDetectorOptions(
base_options=mp_python.BaseOptions(model_asset_path="efficientdet_lite0.tflite"), # downloaded model
running_mode=vision.RunningMode.VIDEO,
max_results=5,
score_threshold=0.5,
)
detector = vision.ObjectDetector.create_from_options(options)
cap = cv2.VideoCapture(0)
start = time.perf_counter()
frames = 0
while cap.isOpened():
ok, frame = cap.read()
if not ok:
break
rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
timestamp_ms = int((time.perf_counter() - start) * 1000)
result = detector.detect_for_video(mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb), timestamp_ms)
for det in result.detections:
box, cat = det.bounding_box, det.categories[0]
cv2.rectangle(frame, (box.origin_x, box.origin_y),
(box.origin_x + box.width, box.origin_y + box.height), (0, 255, 0), 2)
cv2.putText(frame, f"{cat.category_name} {cat.score:.2f}", (box.origin_x, box.origin_y - 5),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 1)
frames += 1
fps = frames / (time.perf_counter() - start)
cv2.putText(frame, f"FPS: {fps:.1f}", (10, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
cv2.imshow("detections", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
break
cap.release()
cv2.destroyAllWindows()
Custom Classes
Fine-tune a pretrained detector on your own labelled images (bounding boxes in COCO or Pascal VOC format) with MediaPipe Model Maker or the TensorFlow Model Garden, export to LiteRT with int8 quantization, and evaluate with mean average precision (mAP) on a held-out set before measuring FPS on the target device.
Common Pitfalls
Measuring speed on a desktop GPU and deploying to a much slower device.
Too high an input resolution for small gains in accuracy, or too low for small, distant objects.
Capture, drawing or display code being the real bottleneck rather than the model.
Choosing the score threshold without checking the precision/recall trade-off on validation data.