Model Complexity and Training Procedures
Daniel and Preetum discuss the definition of model complexity and its relation to training procedures in deep learning. They explore various aspects of training procedures that can increase a model's complexity, such as regularization, time steps, and different optimization techniques. Preetum presents an ambitious conjecture that encapsulates these observations and highlights the importance of testing this conjecture in future research.In this clip
From this podcast
The Gradient
Preetum Nakkiran: An Empirical Theory of Deep Learning
Related Questions