Dynamic Memory Insights
Jonas discusses a paradigm where memory is dynamic and tied to underlying computations, emphasizing the importance of model expectations in determining information gain. He highlights the limitations of current retrieval augmented generation systems, particularly their reliance on nearest neighbor searches, which can lead to redundant information. The debate on whether to use context or gradient steps for data ingestion remains an open question in the field.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Jonas Hübotter (ETH) - Test Time Inference
Related Questions
Do I get it right that a Retrieval Augmented Generation (RAG) system can retrieve data in addition to its training data as discussed in the episode Patrick Lewis on Retrieval-Augmented Generation - Weaviate Podcast #76! and the clip Unsupervised Retrieval Models?
Do I get it right that a Retrieval Augmented Generation (RAG) system can retrieve data in addition to its training data?
Do I get it right that a Retrieval Augmented Generation (RAG) system can retrieve data in addition to its training data, as discussed in the episode Vector Databases and the Power of RAG and the clip Evolution of AI, as well as in the episode with Cohere co-founder Nick Frosst on building LLM apps for business and the clip Model Evaluation Insights?