Lessons from Studying FAANG ML Systems // Ernest Chan // MLOps Coffee Sessions #84

Tech Choices

Choosing the right technology is crucial for building reliable ML systems. Ernest Chan emphasizes the value of selecting stable, well-established technologies over newer, less proven alternatives. He shares, "I choose MySQL because it hasn't broken. It hasn't lost any data for the last ten years." 1 This approach ensures that the technology stack remains dependable and minimizes risks associated with adopting cutting-edge tools. Demetrios Brinkmann adds that a simpler, opinionated platform often provides a better user experience, while flexibility allows engineers to tweak systems to meet specific needs 2.

   

Infrastructure

Adapting ML infrastructure involves balancing new capabilities with repeatability. Ernest Chan discusses the importance of distinguishing between infrastructure and platforms, noting that repeatability is key to transitioning from one to the other. He states, "We have to start with the infrastructure side and then from there think about how to turn into a platform." 3 At Duo, they improved data visibility by setting up a model registry, which enhanced access to model metrics and facilitated trend analysis 3. Vishnu Rachakonda highlights Chan's insights on ML platforms, praising his thoughtful content on MLOps and ML infrastructure 4.