Published Jan 23, 2021

#039 - Lena Voita - NLP

Lena Voita delves into cutting-edge NLP visualization techniques and token representation dynamics, critiques traditional probing methods, and addresses challenges like hallucination in neural machine translation, offering innovative solutions to enhance model interpretability and translation accuracy.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

NLP is not NLU and GPT-3 - Walid Saba
Answers 383 questions
#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)
Answers 383 questions
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Answers 383 questions
Explainability, Reasoning, Priors and GPT-3
Answers 383 questions
OpenAI GPT-3: Language Models are Few-Shot Learners
Answers 383 questions
MLST #78 - Prof. NOAM CHOMSKY (Special Edition)
Answers 383 questions
#70 - LETITIA PARCALABESCU - Symbolics, Linguistics [UNPLUGGED]
Answers 383 questions
#029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence
Answers 383 questions
#92 - SARA HOOKER - Fairness, Interpretability, Language Models
Answers 383 questions
Jürgen Schmidhuber - Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs
Answers 383 questions
#038 - Professor Kenneth Stanley - Why Greatness Cannot Be Planned
Answers 383 questions
Capsule Networks and Education Targets
Answers 383 questions
#114 - Secrets of Deep Reinforcement Learning (Minqi Jiang)
Answers 383 questions
#035 Christmas Community Edition!
Answers 383 questions
Kernels!
Answers 383 questions

Dexa/Machine Learning Street Talk (MLST)

#039 - Lena Voita - NLP

Topics covered

Popular Clips

Transformer Relevance Patterns

Teacher Forcing Explained

Understanding Model Representations

Token Interaction Dynamics

Understanding AI Limitations

Interactive Learning Tools

Embracing Mathematical Beauty

NLP Course Overview

Engaging Learning Strategies

Training Objectives Unpacked

Visualizing Machine Learning

Evolving Token Representations

Meta Learning Insights

Information Theoretic Probing

Hallucination and Bias

Episode Highlights

NLP Visualization Techniques

Evolution of Token Representations

Probing and Interpretability

Machine Translation Challenges

Related Episodes

NLP is not NLU and GPT-3 - Walid Saba

#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Explainability, Reasoning, Priors and GPT-3

OpenAI GPT-3: Language Models are Few-Shot Learners

MLST #78 - Prof. NOAM CHOMSKY (Special Edition)

#70 - LETITIA PARCALABESCU - Symbolics, Linguistics [UNPLUGGED]

#029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence

#92 - SARA HOOKER - Fairness, Interpretability, Language Models

Jürgen Schmidhuber - Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs

#038 - Professor Kenneth Stanley - Why Greatness Cannot Be Planned

Capsule Networks and Education Targets

#114 - Secrets of Deep Reinforcement Learning (Minqi Jiang)

#035 Christmas Community Edition!

Kernels!

#039 - Lena Voita - NLP

Topics covered

Popular Clips

Episode Highlights

NLP Visualization TechniquesLena Voita's NLP course is a masterclass in using visual aids and interactive elements to enhance learning. Her innovative approach makes complex concepts accessible and engaging for students.

NLP Visualization Techniques

Evolution of Token RepresentationsElena Voita examines the intricate dynamics of token interactions and representation changes in transformer models. She highlights how training objectives and information bottleneck theory influence the flow of information through these models.

Evolution of Token Representations

Probing and InterpretabilityElena Voita critiques traditional NLP probing methods and introduces information-theoretic probing for better model evaluation. She emphasizes the importance of model interpretability and the influence of training objectives on information retention.

Probing and Interpretability

Machine Translation ChallengesElena Voita explores the challenges of neural machine translation, focusing on hallucination and exposure bias. She discusses methods to balance source and target contributions, enhancing translation accuracy and reducing errors.

Machine Translation Challenges

Related Episodes