847: AI Engineering 101 — with Ed Donner

Serverless Deployment

Serverless deployment is revolutionizing how AI models are productionized, offering flexibility and cost-effectiveness. Ed Donner highlights the use of platforms like Modal, which allows AI engineers to deploy models in the cloud seamlessly. This serverless AI platform charges only for the clock cycles used, making it an efficient choice for startups and enterprises alike 1.

It stays warm for a couple of minutes in case other requests come in and then it calmly shuts down and then you stop paying.

--- Ed Donner

Jon Krohn adds that Modal's ease of use and free credits make it accessible for those experimenting with AI models 2.

   

Integrated Strategies

Integrated deployment strategies involve using a variety of platforms and technologies to optimize AI applications. Ed Donner explains that AI engineers often need to experiment with different models and techniques, such as fine-tuning and agentic AI, to find the best fit for their needs 3.

There are rules of thumb. If you're focused on trying to improve the accuracy and specialist skills, then it tends to lend itself towards rag.

--- Ed Donner

Jon Krohn emphasizes the importance of creating comprehensive test sets to ensure models meet user requirements, suggesting the use of real platform data to enhance simulations 1.