Innovation Window

For G=6, rule learning sometimes succeeded only when larger datasets delayed memorization enough. The innovation window is the interval between rule learning and training-sample memorization. A model can innovate only if it learns the rule before the empirical training distribution becomes dominant. The innovation window narrows with higher rule complexity because rule learning occurs later. The innovation window widens with larger datasets because memorization occurs later.