Evolution vs. Learning
Daniel questions the analogy between evolutionary processes and the pre-training of language models, expressing skepticism about its validity. Jacob acknowledges some behavioral parallels but emphasizes the differences in prediction mechanisms between human learning and current language model training. He also notes intriguing similarities in representations between visual models and the human brain, highlighting the complexity of understanding these correspondences in language.In this clip
From this podcast
The Gradient
Jacob Andreas: Language, Grounding, and World Models
Related Questions