Future of Large Models

Aidan discusses the potential limitations of transformers due to sequence length constraints. He predicts consistent model performance scaling with increased compute and highlights data constraints as a significant challenge for building large language models.