State Space Models
Aidan discusses State Space Models (SSMs) as a middle ground between Transformers and RNNs, focusing on scalable internal memory usage. He highlights the need for mature software support for SSMs to potentially replace Transformers and enable models with larger context windows.In this clip
From this podcast
Gradient Dissent - A Machine Learning Podcast
Scaling LLMs and Accelerating Adoption: Interview with Aidan Gomez
Related Questions