Hierarchical Attention Models
David and Tim discuss the potential of applying attention to different hierarchies in machine learning models, envisioning future decoder models that can operate across multiple levels of hierarchy. They highlight the importance of dividing historical experiences into chunks for better embedding and the limitations of current recurrent approaches in retaining information over time.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Future of Generative AI [David Foster]
Related Questions