Understanding Features
The discussion delves into the concept of features within language models, emphasizing how neurons can represent multiple topics through a process called polysemanticity. It highlights the manual exploration of these features and their activation in response to different texts, ultimately linking this understanding to the broader goal of model alignment. The conversation also touches on specific features like the Golden Gate Bridge and immunology, underscoring the importance of comprehending how models think.In this clip
From this podcast
ThursdAI
📅 ThursdAI - May 23 - OpenAI troubles, Microsoft Build, Phi-3 small/large, new Mistral & more AI news
Related Questions