Benchmarking Efficiency

Sara and Daniel discuss the challenges of benchmarking efficiency in machine learning, highlighting the limitations of MLPeRF and hardware agnostic metrics. They explore the need for a more comprehensive approach to benchmarking that captures real-world efficiency gains and address the incentives of benchmarking for hardware vendors.