Pretraining in Natural Language Processing
Richard discusses the power of pretraining in natural language processing, starting with word vectors and expanding to include encoders and decoders. He highlights the influential Elmo paper and the vision of a single model for all of NLP.In this clip
From this podcast
The Gradient
Richard Socher: Re-Imagining Search
Related Questions
What's next in large language models (LLMs) as discussed in the episode Richard Socher: Re-Imagining Search and the clip Pretraining in Natural Language Processing?
What is the role of large models in predicting the next word in language tasks as discussed in the episode Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 and the clip History of Neural Networks in Language?