Measuring Unstructured Drift
Xander explains how to measure unstructured drift in production by monitoring the distribution of embedding vectors. These vectors not only represent the content of images but also capture subtle nuances like image quality and semantic changes. By identifying new areas in the embedding space that weren't represented during training, one can effectively detect drift across various data types, including text and audio.In this clip
From this podcast
Super Data Science: ML & AI Podcast with Jon Krohn
689: Observing LLMs in Production to Automatically Catch Issues — with Amber Roberts and Xander Song
Related Questions