Active Learning Models

John and Dwarkesh discuss the efficiency of learning models, exploring the balance between training and context learning. They ponder the idea of models with medium-term memory and specialization to tasks, aiming for a more deliberate and active learning approach.

In this clip
From this podcast
Dwarkesh Podcast
John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI
Related Questions
- How do deep learning models remember?
- Can you explain what in-context learning, chain-of-thought reasoning, and end-to-end agents are in the context of long context unlocking the potential of large language models?