The new AI app stack

LLM Ecosystem

Large language models (LLMs) are often misunderstood as standalone applications, but they are part of a broader AI ecosystem. Daniel Whitenack emphasizes that while models like Llama 2 or Stable Diffusion provide functionality, they are not the complete application. Instead, a whole ecosystem of tools and frameworks supports these models, as illustrated by a diagram from Andreessen Horwitz 1. Chris Benson notes the importance of understanding this ecosystem to fully leverage LLMs' potential.

There's this perception that the model is the application, which is basically a falsehood.

--- Daniel Whitenack

This understanding is crucial for developers aiming to create value beyond the model itself.

   

App Integration

Integrating generative AI models into applications involves more than just deploying the model; it requires a comprehensive approach to app hosting and middleware. Daniel Whitenack discusses emerging trends in app hosting, highlighting platforms like Vercel that cater to new AI apps 2. Chris Benson adds that merging model hosting with app hosting simplifies management and enhances productivity 2.

Every app in the future is going to have models in it.

--- Chris Benson

Additionally, the concept of model middleware, which sits between the orchestration layer and model hosting, plays a critical role in this integration process 3.