Case study
LeNet-5 FashionMNIST Reproduction
A reproducible PyTorch implementation of the LeNet-5 architecture, adapted for FashionMNIST while preserving paper-specific features such as sparse convolutional connections, trainable subsampling, scaled tanh activations, Euclidean RBF outputs, and discriminative MAP loss.
- Project context
- Personal Project
- Year
- 2026
- Categories
- Machine Learning, Deep Learning, Computer Vision
- Source code
- View source code
- Technology stack
- Python, PyTorch, torchvision, NumPy, scikit-learn, Matplotlib, JupyterLab
System design
A public-safe view of the components, integrations, and data flow described in this case study.
Scroll horizontally to inspect the full diagram, or open it full size.
Problem
Reproducing LeNet-5 involves more than assembling a conventional convolutional network. Modern implementations often replace the architecture's sparse C3 connections, trainable average-subsampling layers, scaled hyperbolic tangent activations, and radial-basis-function classifier with simpler contemporary alternatives. This project implements those paper-specific characteristics in PyTorch and adapts the model from handwritten-digit recognition to the ten clothing categories in FashionMNIST.
Role
Abdul independently implemented and documented the machine-learning experiment. His responsibilities included translating the LeNet-5 architecture into PyTorch, designing the FashionMNIST data pipeline, implementing custom neural network layers and loss functions, configuring reproducible training, and evaluating the resulting classifier.
Architecture
The input pipeline pads each `28 × 28` FashionMNIST image to `32 × 32` and maps pixel intensities from `[0, 1]` to `[-0.1, 1.175]`. The network follows the LeNet-5 processing sequence: - C1: six `5 × 5` convolutional feature maps. - S2: trainable average subsampling with per-channel scale and bias. - C3: sixteen feature maps using the original sparse 60-connection pattern. - S4: a second trainable subsampling stage. - C5: 120 `5 × 5` convolutional units. - F6: an 84-dimensional fully connected representation. - Output: ten Euclidean RBF distances used for classification through `argmin`. Scaled tanh activations are applied throughout the network. The executed configuration uses trainable RBF prototypes and contains 60,840 trainable parameters.
Implementation
The project is delivered as a reproducible Jupyter notebook with pinned core PyTorch and torchvision versions. It includes deterministic random seeding, automatic CPU, CUDA, or Apple MPS device selection, shape and parameter-count assertions, gradient checks, and a structural smoke test. Because the original RBF prototypes represented handwritten digits, the FashionMNIST adaptation constructs ten deterministic, separated bipolar prototypes in the model's 84-dimensional feature space. Training uses the paper-inspired discriminative MAP/RBF objective, implemented with `torch.logsumexp` for numerical stability, rather than cross-entropy. The full experiment uses mini-batch SGD and a staged learning-rate schedule over 40 epochs. The notebook also provides optional affine data augmentation, a 600,000-view expanded training dataset, an experimental diagonal preconditioner, learning curves, and confusion-matrix visualization.
Results
On the saved reproducible CPU experiment, the model completed 40 epochs over 60,000 clean FashionMNIST training examples and achieved **88.96% test accuracy**, corresponding to an **11.04% test error**, on the 10,000-example test set. The notebook records the complete epoch-by-epoch training history and produces training curves, test curves, a class-level confusion matrix, and a final experiment report. These results apply to the documented configuration with sparse C3 connections, trainable RBF prototypes, no data augmentation, seed 42, and the staged SGD learning-rate schedule.