Information Theoretic Probing
The discussion dives into the concept of minimum description length (MDL) and its significance in understanding model representations. By shifting from traditional accuracy metrics to information theoretic probing, insights are gained into how well representations capture linguistic properties and the complexity involved in achieving those representations. This approach offers a more stable measure of performance, highlighting the nuances often overlooked in standard probing methods.In this clip
From this podcast
Machine Learning Street Talk (MLST)
#039 - Lena Voita - NLP
Related Questions