Published Nov 7, 2017

40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis

Gábor Melis delves into the crucial role of hyperparameter tuning in optimizing neural language models, unveils surprising insights in comparing LSTMs and RHNs, and highlights the pressing challenges of evaluation, reproducibility, and dataset limitations in language model research.

Episode Highlights

Topics covered

Popular Clips

Episode Highlights

Related Episodes

63 - Neural Lattice Language Models, with Jacob Buckman
Answers 383 questions
27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov
Answers 383 questions
114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro
Answers 383 questions
81 - BlackboxNLP, with Afra Alishahi and Tal Linzen
Answers 383 questions
109 - What Does Your Model Know About Language, with Ellie Pavlick
Answers 383 questions
120 - Evaluation of Text Generation, with Asli Celikyilmaz
Answers 383 questions
64 - Neural Network Models for Sentence Pair Tasks, with Wuwei Lan and Wei Xu
Answers 383 questions
38 - A Corpus of Natural Language for Visual Reasoning, with Alane Suhr
Answers 383 questions
44 - Truly Low Resource NLP, with Anders Søgaard
Answers 383 questions
29 - Neural machine translation via binary code prediction, with Graham Neubig
Answers 383 questions
35 - Replicability Analysis for Natural Language Processing, with Roi Reichart
Answers 383 questions
37 - On Statistical Significance, Training Variance, and Why Reporting Score Distributions Matters
Answers 383 questions
25 - Neural Semantic Parsing over Multiple Knowledge-bases
Answers 383 questions
34 - Translating Neuralese, with Jacob Andreas
Answers 383 questions
130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn
Answers 383 questions

Dexa/NLP Highlights

40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis

Topics covered

Popular Clips

Evaluating Language Models

LSTM vs. RHN

Hyperparameter Tuning Insights

Efficient Hyperparameter Tuning

Model Evaluation Insights

Data Set Assumptions

Recurrent Networks Comparison

NLP Model Comparison

Episode Highlights

Hyperparameter Tuning

Model Comparisons

Language Model Challenges

Related Episodes

63 - Neural Lattice Language Models, with Jacob Buckman

27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov

114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro

81 - BlackboxNLP, with Afra Alishahi and Tal Linzen

109 - What Does Your Model Know About Language, with Ellie Pavlick

120 - Evaluation of Text Generation, with Asli Celikyilmaz

64 - Neural Network Models for Sentence Pair Tasks, with Wuwei Lan and Wei Xu

38 - A Corpus of Natural Language for Visual Reasoning, with Alane Suhr

44 - Truly Low Resource NLP, with Anders Søgaard

29 - Neural machine translation via binary code prediction, with Graham Neubig

35 - Replicability Analysis for Natural Language Processing, with Roi Reichart

37 - On Statistical Significance, Training Variance, and Why Reporting Score Distributions Matters

25 - Neural Semantic Parsing over Multiple Knowledge-bases

34 - Translating Neuralese, with Jacob Andreas

130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn

40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis

Topics covered

Popular Clips

Episode Highlights

Hyperparameter TuningGábor Melis discusses the intricacies of hyperparameter tuning in neural language models, emphasizing its crucial role in achieving optimal performance. He shares insights on efficient tuning strategies and the significant impact these have on model results.

Hyperparameter Tuning

Model ComparisonsGábor Melis examines the comparative performance of LSTMs and recurrent highway networks, revealing surprising insights into their effectiveness in language modeling. His research challenges previous assumptions, demonstrating that with proper tuning, LSTMs can outperform RHNs.

Model Comparisons

Language Model ChallengesGábor Melis explores the challenges in evaluating neural language models, focusing on reproducibility issues, dataset limitations, and evaluation criteria. He emphasizes the need for improved methodologies to enhance the reliability and applicability of research findings.

Language Model Challenges

Related Episodes