Generative Models

Generative-model generalization depends on whether rule acquisition occurs before memorization dominates. Diffusion and autoregressive transformers both exhibit rule learning before later memorization. Model capacity accelerates both rule learning and memorization, so it does not necessarily expand innovation. Generative models trained on finite data face a tension between learning latent population rules and reproducing training examples. The study uses synthetic rule-governed distributions so that rule validity, novelty, and exact memorization can be measured.