Keith and Robert discuss the challenges of identifying important subnetworks early in training. Robert highlights the use of aggressive learning schemes to find early bird tickets. The conversation delves into the practical implications of synaptic flow paper and the potential of iterative magnitude pruning in various contexts.
