Lottery Ticket Hypothesis: magnitude pruning from scratch

# Description

Lottery Ticket Hypothesis implemented from scratch in PyTorch, no pruning libraries. MaskedLinear MLP (forward multiplies W ⊙ m) with 80,352 weights on Fashion-MNIST (4,000/1,000/2,000, seed 42, 10 epochs, CPU ~258 s). Part 1: train the dense net — test acc 82.35%. Part 2: global unstructured magnitude pruning at sparsities 50/75/90/95/98%. Part 3: winning ticket = apply the mask and rewind to the original initialization, then retrain. Part 4: random-mask control with the same per-layer density. Part 5: continued fine-tune (no rewind) and Kaiming reinit (Liu et al. 2019) on the same mask. Part 6: IMP, 4 rounds × 50% of surviving weights (remaining 50/25/12.5/6.25%). Results: ticket s=0.75 83.20% (beats dense); ticket s=0.90 81.30% (−1.05 pp vs dense, +9.45 pp vs random 71.85%); ticket s=0.95 78.55% vs random 56.05%; at s=0.98 both collapse (ticket 45.45%, random 10.25%). Rewind vs reinit at s=0.90: 81.30% vs 74.50% (−6.80 pp). IMP matches one-shot (round 2: 83.15%). Findings: tickets exist in the 50–90% regime; rewind to init is not replaceable by random reinit (topology is not enough); continued is not the LTH baseline (it starts from trained weights); at 98% the ticket also falls. 7 figures.

$ ./deploy.sh $ git clone

# Key features

$ MaskedLinear MLP 784→96→48→10: 80,352 weights, global unstructured magnitude pruning

$ Fashion-MNIST CPU subset: 4,000 train / 1,000 val / 2,000 test, seed 42, 10 epochs, ~258 s

$ One-shot LTH: ticket (rewind to init) vs random mask vs continued fine-tune at 6 sparsities

$ Ticket s=0.75 83.20% beats dense 82.35%; s=0.90 81.30% (−1.05 pp) vs random 71.85% (+9.45 pp)

$ Rewind ≠ reinit (Liu et al.): same mask s=0.90, original init 81.30% vs Kaiming 74.50% (−6.80 pp)

$ IMP 4 rounds × 50%: 82.50 / 83.15 / 81.75 / 77.85 — matches one-shot

$ Honest collapse at s=0.98: ticket 45.45%, random ≈ chance (10.25%)

$ 7 figures: sparsity-acc, |w| histograms, per-layer density, curves, IMP, rewind vs reinit, summary

# Gallery

Project terminal

Accuracy vs sparsity: ticket / random / continued

Magnitude histograms dense vs ticket

Remaining density per layer (s=0.90)

Val curves dense vs ticket vs random

Iterative IMP vs one-shot prune

Rewind to init vs Kaiming reinit

LTH summary

# Technologies used

Python PyTorch NumPy Pandas scikit-learn Matplotlib