Stacking State Space Models
Dan discusses the limitations of traditional state space models and how stacking two models can enhance their functionality. This approach allows for dynamic memory recall and comparison of tokens, enabling more complex associations within sequences. The conversation highlights the importance of multiplicative interactions in directing where to look within the stored memory, ultimately bridging the gap to real language applications.In this clip
From this podcast
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Language Modeling With State Space Models with Dan Fu - 630
Related Questions