Jonathan discusses the significance of learning rates in network pruning and emphasizes the need for empirical testing to understand why learning rate rewinding is effective. He shares his conservative approach in research design to ensure defensibility and avoid scrutiny from reviewers, highlighting the importance of anchoring choices in prior work.
