Published Jun 20, 2023

689: Observing LLMs in Production to Automatically Catch Issues — with Amber Roberts and Xander Song

Discover how Amber Roberts and Xander Song from Arize AI emphasize the critical role of ML observability in maintaining effective and ethical AI systems, discussing advanced strategies for managing data drift, biases, and model performance. Explore their insights on retraining and scaling large language models to ensure seamless AI operations.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

787: MLOps: The Job and The Key Tools — with Demetrios Brinkmann
Answers 383 questions
767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka
Answers 383 questions
SDS 599: MLOps: Machine Learning Operations — with @Miki_ML
Answers 383 questions
747: Technical Intro to Transformers and LLMs — with Kirill Eremenko
Answers 383 questions
661: Designing Machine Learning Systems — with Chip Huyen
Answers 383 questions
721: Quantum Machine Learning — with Dr. Amira Abbas
Answers 383 questions
710: LangChain: Create LLM Applications Easily in Python — with Kris Ograbek (@krisograbek)
Answers 383 questions
801: Merged LLMs Are Smaller And More Capable — with Arcee AI's Mark McQuade and Charles Goddard
Answers 383 questions
788: Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks — with @JonKrohnLearns
Answers 383 questions
713: Llama 2, Toolformer and BLOOM: Open-Source LLMs — with Meta's Dr. Thomas Scialom
Answers 383 questions
706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu
Answers 383 questions
SDS 541: Data Observability — with Dr. Kevin Hu
Answers 383 questions
797: Deep Learning Classics and Trends — with Dr. Rosanne Liu
Answers 383 questions
669: Streaming, reactive, real-time machine learning — with Adrian Kosowski
Answers 383 questions
707: Vicuña, Gorilla, Chatbot Arena and Socially Beneficial LLMs — with Prof. Joey Gonzalez
Answers 383 questions

Dexa/Super Data Science: ML & AI Podcast with Jon Krohn

689: Observing LLMs in Production to Automatically Catch Issues — with Amber Roberts and Xander Song

Topics covered

Popular Clips

ML Observability Insights

Model Maintenance Insights

Community Engagement Insights

Embedding Distribution Insights

ML Observability Essentials

Measuring Unstructured Drift

Monitoring Data Drift

AI Insights Unleashed

Scaling Data Insights

ML Observability vs. Monitoring

ML Observability Insights

Ad Hoc ML Monitoring

Understanding Drift Types

Model Retraining Strategies

Episode Highlights

Model Maintenance Strategies

Arize AI Features

Understanding ML Observability

Related Episodes

787: MLOps: The Job and The Key Tools — with Demetrios Brinkmann

767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka

SDS 599: MLOps: Machine Learning Operations — with @Miki_ML

747: Technical Intro to Transformers and LLMs — with Kirill Eremenko

661: Designing Machine Learning Systems — with Chip Huyen

721: Quantum Machine Learning — with Dr. Amira Abbas

710: LangChain: Create LLM Applications Easily in Python — with Kris Ograbek (@krisograbek)

801: Merged LLMs Are Smaller And More Capable — with Arcee AI's Mark McQuade and Charles Goddard

788: Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks — with @JonKrohnLearns

713: Llama 2, Toolformer and BLOOM: Open-Source LLMs — with Meta's Dr. Thomas Scialom

706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu

SDS 541: Data Observability — with Dr. Kevin Hu

797: Deep Learning Classics and Trends — with Dr. Rosanne Liu

669: Streaming, reactive, real-time machine learning — with Adrian Kosowski

707: Vicuña, Gorilla, Chatbot Arena and Socially Beneficial LLMs — with Prof. Joey Gonzalez

689: Observing LLMs in Production to Automatically Catch Issues — with Amber Roberts and Xander Song

Topics covered

Popular Clips

Episode Highlights

Model Maintenance StrategiesAmber Roberts and Xander Song discuss the importance of retraining strategies and model performance monitoring in machine learning. They highlight the need for observability to ensure models remain effective and efficient.

Model Maintenance Strategies

Arize AI FeaturesAmber Roberts and Xander Song from Arize discuss the intricacies of their platform, focusing on ML observability and data drift management. They highlight the platform's scalability and its ability to handle large datasets and LLMs efficiently.

Arize AI Features

Understanding ML ObservabilityAmber Roberts and Xander Song from Arize discuss the importance of ML observability in monitoring AI systems. They explore how it helps detect model drift, address biases, and ensure ethical AI development.

Understanding ML Observability

Related Episodes