Published Apr 7, 2022

#73 - YASAMAN RAZEGHI & Prof. SAMEER SINGH - NLP benchmarks

Tim Scarfe hosts Yasaman Razeghi and Prof. Sameer Singh in a deep dive into NLP benchmarks, tackling challenges in reasoning, model explainability, and the impact of pre-training term frequencies while addressing the fascination with performance metrics and future directions in ML DevOps.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

Explainability, Reasoning, Priors and GPT-3
Answers 383 questions
#92 - SARA HOOKER - Fairness, Interpretability, Language Models
Answers 383 questions
Prof. Subbarao Kambhampati - LLMs don't reason, they memorize (ICML2024 2/13)
Answers 383 questions
OpenAI GPT-3: Language Models are Few-Shot Learners
Answers 383 questions
Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!
Answers 383 questions
Speechmatics CTO - Next-Generation Speech Recognition
Answers 383 questions
Jürgen Schmidhuber - Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs
Answers 383 questions
#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)
Answers 383 questions
#107 - Dr. RAPHAËL MILLIÈRE - Linguistics, Theory of Mind, Grounding
Answers 383 questions
#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]
Answers 383 questions
#032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!
Answers 383 questions
ICLR 2020: Yoshua Bengio and the Nature of Consciousness
Answers 383 questions
#91 - HATTIE ZHOU - Teaching Algorithmic Reasoning via In-context Learning #NeurIPS
Answers 383 questions
#94 - ALAN CHAN - AI Alignment and Governance #NEURIPS
Answers 383 questions
#039 - Lena Voita - NLP
Answers 383 questions

Dexa/Machine Learning Street Talk (MLST)

#73 - YASAMAN RAZEGHI & Prof. SAMEER SINGH - NLP benchmarks

Topics covered

Popular Clips

Reevaluating Metrics

Language Model Realities

Model Reasoning Analysis

Future of Reasoning

Model Performance Analysis

Language Model Performance

Pre-Training Corpus Analysis

Future of ML DevOps

Complex Model Explanations

Metric Theater Insights

Language Models and Intelligence

Defining Reasoning

Model Validation Innovations

Hybrid Neurosymbolic Models

Episode Highlights

Metric Obsession

ML DevOps and Future

Model Explainability

Pre-training Impact

Related Episodes

Explainability, Reasoning, Priors and GPT-3

#92 - SARA HOOKER - Fairness, Interpretability, Language Models

Prof. Subbarao Kambhampati - LLMs don't reason, they memorize (ICML2024 2/13)

OpenAI GPT-3: Language Models are Few-Shot Learners

Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!

Speechmatics CTO - Next-Generation Speech Recognition

Jürgen Schmidhuber - Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs

#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)

#107 - Dr. RAPHAËL MILLIÈRE - Linguistics, Theory of Mind, Grounding

#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]

#032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!

ICLR 2020: Yoshua Bengio and the Nature of Consciousness

#91 - HATTIE ZHOU - Teaching Algorithmic Reasoning via In-context Learning #NeurIPS

#94 - ALAN CHAN - AI Alignment and Governance #NEURIPS

#039 - Lena Voita - NLP

#73 - YASAMAN RAZEGHI & Prof. SAMEER SINGH - NLP benchmarks

Topics covered

Popular Clips

Episode Highlights

Metric Obsession

ML DevOps and Future

Model ExplainabilitySameer Singh and Yasaman Razeghi explore the intricacies of explainability in machine learning, focusing on techniques like LIME and SHAP. They discuss the challenges of interpreting complex models and the potential of hybrid neurosymbolic systems to enhance reasoning.

Model Explainability

Pre-training ImpactYasaman Razeghi and Sameer Singh explore the impact of pre-training term frequencies on language models, questioning their reasoning capabilities. They discuss the potential for improved model design and the need for transparency in pre-training data.

Pre-training Impact

Related Episodes