Language Model Insights
Nathan and Michael discuss the surprising performance of language models like Mamba and striped hyena, highlighting the mathematical connections between linear attention and state space models. They delve into the breakdown of softmax and the exploration of different directions in model architecture, shedding light on the trade-offs involved in model design.In this clip
From this podcast
Interconnects Audio
Interviewing Tri Dao and Michael Poli of Together AI on the future of LLM architectures
Related Questions