Warm Up Training
Introducing a novel bookkeeping scheme, Han discusses the impact of warm-up training on model accuracy, achieving a 0.4% improvement in both image classification and speech recognition. By gradually increasing sparsity during the initial training phases, the method not only recovers lost accuracy but also surpasses baseline performance. The conversation raises intriguing questions about the relationship between warm-up training and the total number of training iterations.In this clip
From this podcast
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Deep Gradient Compression for Distributed Training with Song Han - #146
Related Questions