Representation Learning Perspective
Hattie and Daniel discuss the relationship between transfer learning and representation learning, specifically focusing on the impact of reinitializing and retraining the last layer of a neural network. They explore whether the improvement in features comes from a better later layer or the amplification of features in earlier layers. Their experiments suggest that the features in earlier layers are continuously improved, leading to better generalization and stronger features.In this clip
From this podcast
The Gradient
Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs
Related Questions