Tim and Robert discuss the surprising results of network pruning, highlighting how a subnetwork with fewer weights can outperform the original network. Keith introduces the concept of signal to noise ratio, raising questions about network efficiency and training distractions. Robert explains the limitations of pruning in larger tasks and introduces the concept of rewinding for improved performance.
