Language Model Complexity
Patrick discusses the challenges of processing long sequences in language models, emphasizing the need for active reading and working memory. He touches on the limitations of training models with long context windows due to signal drop-off, highlighting the importance of understanding complex relational structures in language processing.In this clip
From this podcast
Machine Learning Street Talk (MLST)
#100 Dr. PATRICK LEWIS (co:here) - Retrieval Augmented Generation
Related Questions