Published Sep 23, 2019

BERT is Shallow

Join Tim Niven as he critiques BERT's limitations in capturing world knowledge and managing linguistic artifacts, while proposing methodologies for more robust AI models and highlighting the importance of philosophical insights in enhancing AI's language reasoning capabilities.

Episode Highlights

Topics covered

Popular Clips

Episode Highlights

Related Episodes

BERT
Answers 383 questions
BERT is Magic
Answers 383 questions
What BERT is Not
Answers 383 questions
SpanBERT
Answers 383 questions
Easily Fooling Deep Neural Networks
Answers 383 questions
The Limits of NLP
Answers 383 questions
Serverless NLP Model Training
Answers 383 questions
[MINI] Natural Language Processing
Answers 383 questions
[MINI] Primer on Deep Learning
Answers 383 questions
Deep Fakes
Answers 383 questions
Machine Learning Done Wrong
Answers 383 questions
Annotator Bias
Answers 383 questions
Artificial Intelligence, a Podcast Approach
Answers 383 questions
Transfer Learning
Answers 383 questions
Understanding Neural Networks
Answers 383 questions

Dexa/Kyle Polich

BERT is Shallow

Topics covered

Popular Clips

Adversarial Data Insights

Learning with Brilliant

Research Directions Ahead

Adversarial Test Sets

Deep Learning Insights

Language Model Limitations

BERT Performance Insights

Episode Highlights

Understanding BERT

Research Methodologies

Future Research Directions

Argumentation in AI

Related Episodes

BERT

BERT is Magic

What BERT is Not

SpanBERT

Easily Fooling Deep Neural Networks

The Limits of NLP

Serverless NLP Model Training

[MINI] Natural Language Processing

[MINI] Primer on Deep Learning

Deep Fakes

Machine Learning Done Wrong

Annotator Bias

Artificial Intelligence, a Podcast Approach

Transfer Learning

Understanding Neural Networks

BERT is Shallow

Topics covered

Popular Clips

Episode Highlights

Understanding BERTTim Niven examines BERT's performance and the inherent challenges in deep learning models. He questions BERT's ability to capture world knowledge and suggests alternative approaches for deeper understanding.

Understanding BERT

Research MethodologiesTim Niven explores the limitations of BERT in handling adversarial attacks and systematic learning challenges. He highlights the need for more robust models that can effectively manage linguistic artifacts and achieve deeper understanding beyond statistical language modeling.

Research Methodologies

Future Research DirectionsTim Niven discusses the importance of debiasing datasets and developing innovative test sets to enhance model training. He shares insights on using out-of-distribution and adversarial test sets to better evaluate model performance.

Future Research Directions

Argumentation in AITim Niven, a PhD student with a background in philosophy, discusses the challenges AI faces in argument reasoning tasks. He explores how philosophical insights can enhance AI's ability to process complex language tasks.

Argumentation in AI

Related Episodes