Momentum Vector Masking
Han discusses the challenges of gradient clipping and compression in LSTMs versus CNNs, highlighting the importance of managing obsolete velocity terms. By implementing momentum vector masking, they significantly reduced accuracy loss in both region and speech recognition tasks. The insights reveal how tracking and managing older contributions can lead to more effective model training.In this clip
From this podcast
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Deep Gradient Compression for Distributed Training with Song Han - #146
Related Questions