This Week in ML & AI avatar

Dexa/This Week in ML & AI

Learn more
Published Jan 13, 2025

Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - 714

Abhijit Bose of Capital One delves into the evolution of their Generative AI platform, spotlighting a platform-centric approach that marries centralized governance with cutting-edge AI tools and Kubernetes integration, enhancing flexibility and resource optimization for enterprise operations and customer service.
Episode Highlights
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) logo

Topics covered

Popular Clips

Questions from this episode

Episode Highlights

  • Agentic Workflows

    Agentic workflows are poised to revolutionize enterprise operations by automating routine tasks and enhancing efficiency. highlights the potential of these workflows in transforming back-office functions and software engineering tasks at Capital One. He explains that the integration of open-source frameworks like LangChain and Langgraph into their platform allows for the orchestration of multiple agents, addressing scalability and regulatory challenges 1 2. Bose notes, "We are super excited about agentic and multi-agentic workflows. I truly believe it is probably the next frontier where you have one or more agents responsible for understanding intent or planning or executing a tool, taking some action."

    We are super excited about agentic and multi-agentic workflows. I truly believe it is probably the next frontier where you have one or more agents responsible for understanding intent or planning or executing a tool, taking some action.

    ---

    The development of these workflows also requires new roles and skillsets, blending business process knowledge with GenAI expertise 3.

       

    AI in Customer Service

    AI is significantly enhancing customer service efficiency at Capital One by streamlining information retrieval and improving response accuracy. describes how fine-tuning LLMs with domain-specific data has led to higher accuracy in summarization and search tasks, reducing the need for manual exploration by human agents 4 5. He explains, "With our new system, we took all of those knowledge bases that the agents look up, we index them and put them in a vector database. And we also fine-tuned our LLM with a lot of the knowledge and the type of question and answer tasks these agents perform."

    With our new system, we took all of those knowledge bases that the agents look up, we index them and put them in a vector database. And we also fine-tuned our LLM with a lot of the knowledge and the type of question and answer tasks these agents perform.

    ---

    This system is now widely deployed, supporting 20,000 agents and demonstrating the practical benefits of AI in customer service 4.

       

    Evolving AI Landscape

    The AI landscape is rapidly evolving, with significant reductions in inference costs and the emergence of new open-source models. notes that these developments are encouraging more creativity and innovation among companies and developers 6. He expresses enthusiasm for the current pace of AI advancements, stating, "It's a fun time. You know, like, I have been doing this for last, you know, 20 years probably, but never, ever in my career. You know, I have seen a phase where every week there is something new coming up."

    It's a fun time. You know, like, I have been doing this for last, you know, 20 years probably, but never, ever in my career. You know, I have seen a phase where every week there is something new coming up.

    ---

    These changes are paving the way for more complex problem-solving capabilities in AI, particularly in multimodal contexts 6.

Related Episodes