Phil discusses the potential efficiency gains of sparse training in machine learning models. Lukas questions the current hardware support for leveraging sparseness. Phil highlights the upcoming software release with static and dynamic sparse libraries for improved performance.