Episode 13: Jonathan Frankle, MIT, on the lottery ticket hypothesis and the science of deep learning

Infrastructure

Jonathan Frankle, a PhD candidate at MIT, highlights the significant infrastructure challenges in conducting large-scale deep learning experiments. He emphasizes the importance of effectively utilizing resources like GPUs and TPUs, noting that many researchers struggle to keep these resources busy due to a lack of proper infrastructure and tools 1. Frankle shares his innovative solution, a Google spreadsheet that manages experiments by automatically deploying and monitoring TPUs, which has become an essential tool for his research group 1.

My wackiest tool is I have this Google spreadsheet, but it's not like any other Google spreadsheet. It's a little bit magical.

--- Jonathan Frankle

He also discusses the difficulties of accessing resources in academia, advocating for more accessible GPU pools to support diverse research ideas 2.

   

AI Balance

Frankle balances optimism and skepticism regarding AI advancements, acknowledging both the potential and limitations of current technologies. He argues that while neural networks have made significant strides, they are far from achieving true intelligence, urging the community to remain realistic about AI's capabilities 3. Frankle reflects on his early work with facial recognition and AI policy, noting how rapidly the field has evolved and the importance of staying informed to contribute meaningfully 4.

What I see in front of me is very promising, but what I see in front of me is not intelligent by any stretch right now.

--- Jonathan Frankle

He encourages a balanced approach, emphasizing the need for continued scientific progress without succumbing to unwarranted hype.