Cost Over Model Performance
Sayash emphasizes that the true measure of AI model success lies in the overall system performance rather than just model capabilities. He critiques the tendency to conflate model evaluation with downstream evaluation, highlighting that downstream users prioritize actual dollar costs over abstract metrics. The discussion reveals how overlooking these costs can mislead developers about the true efficiency and value of their models.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Sayash Kapoor - How seriously should we take AI X-risk? (ICML 1/13)
Related Questions