Published Feb 9, 2023

Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science

Kyunghyun Cho discusses the evolution of neural machine translation, emphasizing groundbreaking techniques like Adapter Fusion and the role of pretraining in model optimization, while offering a candid look at the nuanced balance between intellectual independence and serendipity in scientific discovery.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

Hugo Larochelle: Deep Learning as Science
Answers 383 questions
Been Kim: Interpretable Machine Learning
Answers 383 questions
Some Changes at The Gradient
Answers 383 questions
Eric Jang on Robots Learning at Google and Generalization via Language
Answers 383 questions
Zachary Lipton: Where Machine Learning Falls Short
Answers 383 questions
Chip Huyen: Machine Learning Tools and Systems
Answers 383 questions
Greg Yang on Communicating Research, Tensor Programs, and µTransfer
Answers 383 questions
François Chollet: Keras and Measures of Intelligence
Answers 383 questions
Soumith Chintala: PyTorch
Answers 383 questions
Linus Lee: At the Boundary of Machine and Mind
Answers 383 questions
Ed Grefenstette: Language, Semantics, Cohere
Answers 383 questions
Yann LeCun on his Start in Research and Self-Supervised Learning
Answers 383 questions
Ted Underwood: Machine Learning and the Literary Imagination
Answers 383 questions
Nicholas Thompson: AI and Journalism
Answers 383 questions
Sewon Min: The Science of Natural Language
Answers 383 questions

Dexa/Daniel Bashir

Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science

Topics covered

Popular Clips

Finding Balance

Detachment and Deep Learning

Exploiting Distributional Hypothesis

Unlocking Transformer's Success

Tackling the Vanishing Gradient

Intellectual Independence

Advances and Limitations

Discovering Machine Translation

Accidental Impact

Shifting Perspectives

Encouraging Optimization Stability

Generative vs Discriminative Policies

Evaluation Challenges

Adapter Fusion

Episode Highlights

Model Optimization Techniques

Scientific Approach

Neural Machine Translation

Related Episodes

Hugo Larochelle: Deep Learning as Science

Been Kim: Interpretable Machine Learning

Some Changes at The Gradient

Eric Jang on Robots Learning at Google and Generalization via Language

Zachary Lipton: Where Machine Learning Falls Short

Chip Huyen: Machine Learning Tools and Systems

Greg Yang on Communicating Research, Tensor Programs, and µTransfer

François Chollet: Keras and Measures of Intelligence

Soumith Chintala: PyTorch

Linus Lee: At the Boundary of Machine and Mind

Ed Grefenstette: Language, Semantics, Cohere

Yann LeCun on his Start in Research and Self-Supervised Learning

Ted Underwood: Machine Learning and the Literary Imagination

Nicholas Thompson: AI and Journalism

Sewon Min: The Science of Natural Language

Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science

Topics covered

Popular Clips

Episode Highlights

Model Optimization TechniquesKyunghyun Cho explores the intricacies of fine-tuning models and the innovative solutions like Adapter Fusion that address these challenges. He delves into the significance of pretraining and initialization in effective model training.

Model Optimization Techniques

Scientific Approach

Neural Machine TranslationKyunghyun Cho shares his journey into neural machine translation, highlighting the pivotal moments and technical innovations that shaped his career. He reflects on the impact of his work and the importance of scaling in advancing the field.

Neural Machine Translation

Related Episodes