Encouraging Optimization Stability
Kyunghyun and his postdoc CiFAr explore ways to ensure that optimization algorithms stay near the initial point provided by pretraining, while still being able to solve problems effectively. They discuss the insights gained from their analysis and the potential implications for pretraining, fine-tuning, and dropout techniques.In this clip
From this podcast
The Gradient
Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science
Related Questions