The Supermask
Hattie discusses the concept of the supermask, which suggests that randomly initialized networks contain subnetworks that can perform well on a task without any training. This opens up possibilities for alternative approaches to training neural networks and using masking as a way to probe pretrained models.In this clip
From this podcast
The Gradient
Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs
Related Questions