user@devops:~$ cat README.md
RealNVP: Normalizing Flows from scratch
# Description
RealNVP (Dinh et al. 2016) implemented from scratch in PyTorch, no flow libraries: 6 affine-coupling layers with alternating masks (passthrough + y_c = x_c·exp(s(x_d)) + t(x_d)) and an exact triangular Jacobian (log|det J| = Σ s(x_d)), tanh·2-bounded scale and a 128-128 tanh MLP per layer (102,924 total params). Trained by direct maximum likelihood (exact NLL 0.5‖z‖² + log2π − logdet, AdamW lr 1e-3 + cosine, 150 epochs per density) on 4 synthetic 2D densities of 4,096 samples: two moons NLL 2.096 nats (+5.81 vs uniform), two gaussians 1.197 (+4.91), ring 3.035 (+6.75) and checkerboard 3.095 (+6.81, the hardest: 8 disjoint modes with zero-mass boundaries). Sampling is exact via the analytic inverse (x = f⁻¹(N(0,I))); the latent z = f(x) over train data is nearly N(0,I) (a diagnostic VAEs cannot give: here the normalization is exact by construction). The density evaluates in O(1) at any point (p(x) = p_z(f(x))·|det J|); the x→z→x roundtrip fails by 2.9e-06 (invertibility verified). Per-layer mean log|det J| shows how each coupling spreads the deformation (earlier layers flatten more). Seed 42, ~41 s total CPU. 7 visualizations.
# Key features
$ RealNVP (Dinh et al. 2016) from scratch: 6 affine-coupling layers with alternating masks — triangular Jacobian and exact NLL
$ Bijection f: z ~ N(0,I) → x with analytic inverse: exact sampling, no MCMC, no variational loss
$ 4 2D densities × 4,096 samples: moons 2.096 nats, gaussians 1.197, ring 3.035, checkerboard 3.095 (hardest: 8 disjoint modes)
$ vs uniform: +5.8 to +6.8 nats of mass assigned to data across all 4 densities
$ log p(x) = log p_z(f(x)) + log|det J| evaluable in O(1) anywhere; p(x=[0,0]) ≈ 0.064 on moons
$ Latent z = f(x) nearly N(0,I): normalization is exact by construction (not approximated as in a VAE)
$ Per-layer mean log|det J|: earlier couplings spread more deformation; tanh·2-bounded scale avoids blow-ups
$ Invertibility verified: max x→z→x error 2.9e-06 · 102,924 params · ~41 s CPU · 7 visualizations
# Gallery
# Technologies used
