SDS 619: Tools for Deploying Data Models into Production — with Erik Bernhardsson

Serverless Solutions

Erik Bernhardsson, the creator of Spotify's original recommendation system, shares insights into his company, Modal, which aims to revolutionize data operations with serverless infrastructure. Modal allows data teams to focus on coding and deploying solutions without worrying about traditional infrastructure, similar to what Vercel does for front-end teams 1. Erik's approach is driven by the desire to fill gaps in existing tools, making data science solutions more seamless and efficient 1.

I built modal to solve a bunch of stuff I always wanted working with data for I don't know how many years, 15 years maybe.

--- Erik Bernhardsson

Additionally, tools like Datalore are highlighted for enhancing team productivity through collaborative data science platforms, offering a comprehensive Jupyter notebook experience and facilitating business intelligence and team collaboration 2.

   

Productivity Dynamics

The discussion also explores how advancements in tools increase the demand for engineers by making them more productive. Erik notes that as engineers gain access to better tools, the demand for their skills rises, leading to more opportunities and applications 3. This paradox is evident in the rise of no-code tools, which were initially thought to reduce the need for engineers but have instead expanded their roles 3.

There's always this tension where there's a lot of demand for engineers, and then people try to invent tools to go around engineers.

--- Erik Bernhardsson

Furthermore, Erik emphasizes that tools like Modal enable data scientists to push projects to production more efficiently, reducing the need for extensive engineering teams and fostering innovation 4.