Understanding Model Representations
Elena discusses the distinction between models that encode information, like the sentiment neuron, and those that merely extract it. She emphasizes the importance of measuring both the accuracy and the effort required to extract information from representations, introducing the minimum description length framework as a practical evaluation tool. This approach sheds light on the complexity involved in understanding how different layers of a model contribute to its predictive capabilities.In this clip
From this podcast
Machine Learning Street Talk (MLST)
#039 - Lena Voita - NLP
Related Questions