Unsupervised Objectives
Tim and Connor discuss the efficiency of using denoising objectives in self-supervised language models, highlighting the importance of short target sequences for computational efficiency. Yannic adds insights on the evolution of pre-training objectives in models like Bert, emphasizing the significance of varied tasks for model performance.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Related Questions