Neural Network Pruning
Tim and Robert discuss the surprising results of network pruning, highlighting how a subnetwork with fewer weights can outperform the original network. Keith introduces the concept of signal to noise ratio, raising questions about network efficiency and training distractions. Robert explains the limitations of pruning in larger tasks and introduces the concept of rewinding for improved performance.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Robert Lange on NN Pruning and Collective Intelligence
Related Questions
What is the main topic of the clip Neural Network Pruning from the episode The Lottery Ticket Hypothesis with Jonathan Frankle?
How does the size of a neural network affect its performance in deep learning, as discussed in the episode Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 and the clip Introduction to Deep Double Descent?