Daniel Bashir avatar

Dexa/Daniel Bashir

Learn more
Published Feb 9, 2023

Kyunghyun Cho: Neural Machine Translation, Language, and Doing Good Science

Kyunghyun Cho discusses the evolution of neural machine translation, emphasizing groundbreaking techniques like Adapter Fusion and the role of pretraining in model optimization, while offering a candid look at the nuanced balance between intellectual independence and serendipity in scientific discovery.
Episode Highlights
The Gradient logo

Topics covered

Popular Clips

Episode Highlights

  • Timing & Serendipity

    Timing and serendipity have played pivotal roles in career. He reflects on how his journey into neural machine translation was largely accidental, emphasizing that his success was due to being in the right place at the right time 1. Cho's decision to study in Finland was influenced by a series of random events, including an automated assignment to a neural network lab, which set the course for his future research 2.

    Just a string of the random events, let's put it like that.

    ---

    These experiences underscore the unpredictable nature of scientific discovery and career development.

       

    Intellectual Independence

    journey highlights the importance of intellectual independence in scientific research. During his PhD, he worked in isolation, which helped him develop a unique perspective on his work and reduced his insecurity about not following mainstream trends 3. Cho emphasizes the necessity of detachment from popular methodologies to focus on niche areas that may not be immediately recognized but hold potential for future breakthroughs 4.

    We can't just dig one well too deep. But we got to always keep everything in the context, in the broader context.

    ---

    This approach has allowed him to contribute significantly to fields that others might overlook.

       

    Healthy Skepticism

    Healthy skepticism is crucial in the scientific process, as explains. He stresses the importance of detaching oneself emotionally from past work to maintain critical thinking and advance scientific progress 5. Cho admires pioneers like Yoshua Bengio for their ability to identify limitations in mainstream approaches and explore new research directions 6.

    It's not about what we like or what are the things that we have done so far, but it's more likely what we have to do in the future.

    ---

    This mindset fosters innovation and adaptability in an ever-evolving field.

Related Episodes