Scaling Machine Learning

Andrew discusses the challenges and opportunities of conducting machine learning research in academia compared to industry giants like OpenAI. He emphasizes the importance of exploring fundamental questions at smaller scales, while also acknowledging the unique insights that large-scale experiments can provide. The conversation highlights the value of constraints in fostering innovative design and research methodologies.