What is a convolutional neural network for image processing?
A convolutional neural network (CNN) is a neural network that learns small filters for detecting visual features such as edges, textures, and object parts. It solves image classification, object detection, segmentation, and related tasks by reusing the same filters across spatial locations. Its main trade-off is efficient, hierarchical feature learning versus higher training and inference costs, especially for large images or deep architectures.
Read full answerHide full answer
Applies toPyTorch 2.x (Python 3.12); CNN concepts are library-independent
How It Works
A CNN processes an image as a grid of values. During convolution, a learned filter slides across the input, performs element-wise multiplication and summation, and produces a feature map indicating where a particular feature appears.
For a one-dimensional input width W, the output width is:
output size = floor((W - K + 2P) / S) + 1
-
W: input width -
K: kernel or filter width -
P: padding added to each side -
S: stride between filter positions
The same formula applies to height. This calculation assumes no dilation.
For an input with C_in channels, each filter spans all input channels and produces one output channel. A layer with C_out filters therefore produces C_out feature maps. Most deep-learning libraries implement cross-correlation, rather than mathematically flipping the kernel before convolution, but “convolution layer” is the conventional term.
Convolutional layers use weight sharing: the same filter is applied at every spatial position. This greatly reduces the number of parameters compared with a fully connected image classifier.
Core Components
-
Convolution layer: Learns local filters and creates feature maps.
-
Activation function: Commonly
ReLU, which introduces nonlinearity. -
Batch normalization: Normalizes layer activations and usually makes optimization faster and more stable.
-
Pooling layer: Reduces spatial dimensions. Max pooling retains the strongest response in each window, providing some local translation invariance.
-
Linear head: Converts extracted features into predictions such as class logits.
-
Receptive field: The input region that can influence one activation. Deeper layers generally observe larger portions of the image.
-
Parameter count: For a convolution layer, it is
K × K × C_in × C_out + C_out. The first term counts filter weights, and the final term counts output biases.
Convolution provides translation equivariance: moving a feature in the image generally moves its activation in the feature map. Pooling and global aggregation contribute some translation invariance, but CNNs are not automatically invariant to every image transformation.
Major Architectures
| Architecture | Main idea | Typical advantage |
|---|---|---|
| LeNet | Early convolution and pooling design | Simple foundation for handwritten-digit recognition |
| AlexNet | Deeper CNN with ReLU and GPU training | Demonstrated large gains in image classification |
| VGG | Stacked small convolution blocks | Simple, uniform architecture |
| ResNet | Residual skip connections | Mitigates degradation in very deep networks |
| Inception | Parallel filters at multiple scales | Captures features at different receptive-field sizes |
| MobileNet | Depthwise and pointwise convolutions | Lower computation and memory use |
| EfficientNet | Balanced scaling of depth, width, and resolution | Strong efficiency for many model sizes |
| Vision Transformer | Self-attention over image patches | Common alternative when sufficient training data and compute are available |
Training and Evaluation
-
Split data into training, validation, and test sets before training. Do not use test data for model selection.
-
Apply augmentation only to training data. Random crops, flips, rotations, and color transformations reduce overfitting but must be realistic for the task.
-
Use
model.train()while training andmodel.eval()during validation or inference. This is essential becauseBatchNorm2dbehaves differently in the two modes. -
Use learning-rate schedules and, when appropriate, dropout or weight decay for regularization.
-
Transfer learning can improve results on small datasets. In current
torchvision, request weights withweights=..., not the obsoletepretrained=Trueargument. -
Accuracy can be misleading for imbalanced classification. Also consider precision, recall, F1, ROC-AUC, or PR-AUC as appropriate. Use weighted sampling or class-weighted loss when necessary.
-
CNN training usually benefits from GPUs, although small models and batch sizes can run on CPUs. Standard convolution work is roughly proportional to
H_out × W_out × C_out × K² × C_in. -
One-dimensional CNNs apply the same principles to sequences, time series, and audio using filters that slide along time.
Common Pitfalls
-
Incorrect tensor layout: PyTorch convolution expects
NCHW—batch, channels, height, width—whereas Keras commonly usesNHWC. -
Wrong normalization behavior: Forgetting
model.eval()produces incorrectBatchNorm2dbehavior. -
Data leakage: Applying preprocessing statistics or augmentation using validation or test data invalidates evaluation.
-
Incorrect flattening: The linear input size must match the final feature-map dimensions.
-
Confusing equivariance and invariance: Convolution shifts responses; it does not inherently make predictions invariant to every transformation.
-
Ignoring imbalance or the wrong metric: High accuracy can conceal poor performance on minority classes.
-
Overfitting the validation set: Repeatedly tuning against one validation set eventually turns it into a training signal; use a separate final test set or cross-validation.