Pruning Neural Networks
The discussion revolves around the challenges of pruning neural networks effectively, particularly whether different methods should be applied at various training stages. Insights reveal that while empirical studies, like those from Hadi, suggest early masking of weights may be beneficial, the fundamental questions about efficient lottery ticket discovery remain unresolved. Both speakers express a sense of curiosity about the future of this research, acknowledging the limitations of current technology and the ongoing nature of scientific exploration.In this clip
From this podcast
The Gradient
Jonathan Frankle: From Lottery Tickets to LLMs
Related Questions