Chelsea Finn on Meta Learning & Model Based Reinforcement Learning

Video Prediction

Video prediction models are pivotal in advancing reinforcement learning, yet they face significant challenges. Chelsea Finn highlights the potential of these models, drawing parallels to large language models that predict future outcomes using vast amounts of unlabeled data 1. However, she notes that video prediction models often struggle with underfitting, a rarity in deep learning, due to the diverse and extensive datasets required 2.

We were able to get the models to be used for planning for fairly simple tasks. But a big challenge that we ran into was that the models were underfitting a lot.

--- Chelsea Finn

This underfitting issue stems from the complexity of fitting training data from varied sources, such as the Robonet dataset, which includes data from multiple robot platforms and camera viewpoints.

   

Data Collection

Scaling data collection is crucial for enhancing robot learning, as Chelsea Finn explains. She describes the autonomous data collection process, where robots interact with their environment to gather diverse data, which can be easily scaled 3. This approach, however, limits the complexity of tasks that can be learned, prompting efforts to gather higher-quality data.

Once you set all that up, then it's mostly a matter of letting the robots run and collect the data, and they can do that fairly autonomously.

--- Chelsea Finn

Finn also emphasizes the importance of sharing data across institutions to improve generalization in robot learning. The Robonet dataset, inspired by ImageNet, exemplifies this by compiling data from multiple labs and robot platforms, enabling more effective pretraining and fine-tuning across different environments 4.