Reusing Samples and Real World Performance
Preetum explains that reusing samples can be beneficial for training, but only up to a certain point. He claims that this holds true even for small sample sizes, as long as the optimizer is still improving. This insight sheds light on the relationship between overfitting and the effectiveness of training samples.In this clip
From this podcast
The Gradient
Preetum Nakkiran: An Empirical Theory of Deep Learning
Related Questions