Masking Pretrained Models
Hattie Zhou discusses the concept of using masks to intervene on pretrained models, allowing for specialization and steering in specific directions. She highlights the potential of this approach to tackle catastrophic forgetting and adapt models to different tasks by selecting subsets of connections within the model.In this clip
From this podcast
The Gradient
Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs
Related Questions