Optimization Insights
Daniel and Jonathan delve into the concept that the outcome of optimization is confined to linearly connected regions. By training multiple copies of a network with varied data, they explore how to determine if these networks land in the same convex region. Jonathan highlights the significance of one-dimensional slices of the loss landscape, revealing their potential to distinguish networks with different properties and the implications for averaging networks effectively.In this clip
From this podcast
The Gradient
Jonathan Frankle: From Lottery Tickets to LLMs
Related Questions