Published Mar 21, 2018

53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle

Sergey Edunov and Myle Ott delve into classical structured prediction losses in NLP, innovatively employing pseudo references to enhance sequence generation accuracy, while sharing insights from their experiments and overcoming computational challenges.

Episode Highlights

Topics covered

Popular Clips

Episode Highlights

Related Episodes

52 - Sequence-to-Sequence Learning as Beam-Search Optimization, with Sam Wiseman
Answers 383 questions
31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
Answers 383 questions
12 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Answers 383 questions
117 - Interpreting NLP Model Predictions, with Sameer Singh
Answers 383 questions
51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae
Answers 383 questions
22 - Deep Multitask Learning for Semantic Dependency Parsing, with Noah Smith
Answers 383 questions
37 - On Statistical Significance, Training Variance, and Why Reporting Score Distributions Matters
Answers 383 questions
07 - Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks
Answers 383 questions
10 - A Syntactic Neural Model for General-Purpose Code Generation
Answers 383 questions
29 - Neural machine translation via binary code prediction, with Graham Neubig
Answers 383 questions
05 - Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Answers 383 questions
40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis
Answers 383 questions
81 - BlackboxNLP, with Afra Alishahi and Tal Linzen
Answers 383 questions
44 - Truly Low Resource NLP, with Anders Søgaard
Answers 383 questions
27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov
Answers 383 questions

Dexa/NLP Highlights

53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle

Topics covered

Popular Clips

Pseudo Reference in NLP

Model Optimization Insights

Sequence NLL Loss

Structured Prediction Losses

Loss Function Combinations

Classic Losses vs. Modern Techniques

Practical NLP Tips

Optimizing Loss Functions

Episode Highlights

Pseudo Reference Techniques

Structured Prediction Losses

Experimentation and Challenges

Related Episodes

52 - Sequence-to-Sequence Learning as Beam-Search Optimization, with Sam Wiseman

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

12 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

117 - Interpreting NLP Model Predictions, with Sameer Singh

51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae

22 - Deep Multitask Learning for Semantic Dependency Parsing, with Noah Smith

37 - On Statistical Significance, Training Variance, and Why Reporting Score Distributions Matters

07 - Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

10 - A Syntactic Neural Model for General-Purpose Code Generation

29 - Neural machine translation via binary code prediction, with Graham Neubig

05 - Transition-Based Dependency Parsing with Stack Long Short-Term Memory

40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis

81 - BlackboxNLP, with Afra Alishahi and Tal Linzen

44 - Truly Low Resource NLP, with Anders Søgaard

27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov

53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle

Topics covered

Popular Clips

Episode Highlights

Pseudo Reference TechniquesThe discussion shifts to the innovative use of pseudo references in NLP model training, highlighting their role in improving sequence generation accuracy. Sergey Edunov and Myle Ott explore the practical implications and performance benefits of this approach.

Pseudo Reference Techniques

Structured Prediction LossesSergey Edunov and Myle Ott discuss their research on classical structured prediction losses in NLP. They explore the effectiveness of these losses in modern neural seq2seq models and the benefits of combining different loss functions.

Structured Prediction Losses

Experimentation and Challenges

Related Episodes