user@devops:~$ cat README.md
INT8 Quantization - PTQ and QAT to compress models 4x
# Description
INT8 quantization implemented with PyTorch FX quantization API on a custom CNN (421,834 params, 1.62 MB) trained on Fashion-MNIST (stratified split 4,000/1,000/5,000, seed 42, AdamW + cosine + early stopping, 53 s on CPU). Three methods compared: (1) dynamic PTQ (torch.quantization.quantize_dynamic) - weights only to int8, activations in fp32, no calibration: acc 0.8698 (-0.06 pp), 0.46 MB, no speedup; (2) static PTQ (FX API: fuse_fx + prepare_fx + calibration with 500 samples + convert_fx) - weights and activations to int8: acc 0.8694 (-0.10 pp), 0.41 MB, 2.6x faster (10.52 to 4.10 ms/batch) thanks to oneDNN INT8 kernels; (3) QAT (prepare_qat_fx + 8 fine-tune epochs + convert_fx) - FakeQuantize simulates quantization during training: acc 0.8712 (+0.08 pp vs baseline; quantization-noise regularization improves generalization), 0.41 MB. Per-layer error analysis: dense layers quantize much better than convolutional ones (MSE 10-8 vs 10-1); dense activation ranges are ~10x wider (FC1: [-32.8, 26.6] vs Conv1: [-3.2, 2.7]). INT8 quantization is nearly free for small networks: 4x compression with loss under 0.1 pp. 8 visualizations.
# Key features
$ Three INT8 quantization strategies with PyTorch: dynamic PTQ, static PTQ (FX API) and QAT - from 1.62 MB to 0.41 MB (-74%)
$ Custom CNN with fusable Conv-BN-ReLU pattern: 421,834 params trained on Fashion-MNIST (4,000/1,000/5,000, seed 42, AdamW + cosine + early stopping in 53 s CPU)
$ Static PTQ via FX API: fuse_fx + prepare_fx + calibration with 500 samples + convert_fx - int8 weights and activations with oneDNN kernels
$ QAT (prepare_qat_fx): FakeQuantize simulates quantization during training - the network learns to live with the noise and generalizes better (+0.08 pp)
$ Accuracy is preserved: loss under 0.1 pp with PTQ - INT8 quantization is nearly free for small networks
$ Static PTQ 2.6x faster on CPU (10.52 to 4.10 ms/batch) by quantizing activations too
$ Dense layers quantize much better than convolutional ones (MSE 10-8 vs 10-1) - per-layer error analysis
$ 8 visualizations: training curves, fp32 vs int8 histograms, per-layer error, activation ranges, accuracy, size, latency and confusion matrices
# Gallery
# Technologies used
