Nicolas Koumchatzky — Machine Learning in Production for Self-Driving Cars

Deep Learning Shift

Nicolas Koumchatzky, Director of AI Infrastructure at NVIDIA, shares his journey from deploying deep learning models at Twitter to leading AI infrastructure at NVIDIA. He highlights the initial challenges of integrating deep learning into existing systems, particularly in areas like image filtering and ad placement, where traditional models fell short. Nicolas explains that deep learning enabled Twitter to achieve the high accuracy required by advertisers, which was previously unattainable with traditional methods.

Without deep learning, it was almost impossible to perform at the required accuracy.

--- Nicolas Koumchatzky

This transition marked a significant shift in how Twitter approached machine learning, moving from batch processing to real-time processing for certain applications 1 2.

   

Infrastructure Challenges

Deploying deep learning at scale presented numerous infrastructure challenges, especially in the early stages. Nicolas discusses how Twitter had to build new infrastructure to support deep learning, which included automating training processes to accommodate users unfamiliar with Lua Torch. This automation initially came at the cost of flexibility but was necessary for adoption.

We brought a lot of automation in the training phase at the cost of flexibility at the beginning.

--- Nicolas Koumchatzky

As the importance of deep learning became evident, Twitter invested in education and moved to TensorFlow for its comprehensive capabilities, marking a pivotal evolution in their machine learning strategy 3 1.

   

Framework Choices

The choice of machine learning frameworks played a crucial role in training and deployment. Nicolas explains that while Twitter initially used Lua Torch, they transitioned to TensorFlow due to its stability and inference capabilities. This shift was part of a broader strategy to streamline machine learning processes and improve performance.

We moved to TensorFlow pretty quickly after that for training and inference.

--- Nicolas Koumchatzky

For deployment, NVIDIA's TensorRT was utilized to optimize performance on NVIDIA hardware, showcasing the importance of selecting the right tools for specific tasks 4 3.