Will Falcon — Making Lightning the Apple of ML

Theory

Will Falcon shares his insights into the evolving landscape of self-supervised learning, emphasizing the role of data transformations over complex algorithms. He argues that many advancements in this field are driven by the effective use of transforms rather than intricate methods like negative sampling 1. Falcon reflects on his past research, noting how simple approaches can achieve results comparable to more complex techniques. He states, "I want a super simple pae loss or something super basic that I know why it works, and I can pinpoint exactly why it's doing what it's doing" 1.

   

Challenges

Falcon discusses the challenges faced in self-supervised learning, particularly the need to simplify algorithms while maintaining effectiveness. He highlights the importance of reducing algorithmic complexity, expressing frustration with the trend of minor tweaks being presented as major innovations 1. Falcon's journey through various frameworks, from Theano to PyTorch, underscores the evolution of tools in machine learning. He recalls, "My first version actually was built on top of Tensorflow, but the second that Pytorch came out, which was a few years later, I rewrote it all in Pytorch" 2.