Published Sep 11, 2023

The Defeat of the Winograd Schema Challenge

Vid Kocijan, a Machine Learning Engineer at Kumo AI, delves into the complexities of common sense reasoning, the impact of large language models on the Winograd Schema Challenge, and the future of AI benchmarking, questioning the true significance of NLP achievements and advocating for more meaningful intelligence assessments beyond the Turing Test.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

Winograd Schema Challenge
Answers 383 questions
Graph Databases and AI
Answers 383 questions
Machine Learning Done Wrong
Answers 383 questions
StrategyQA and Big Bench
Answers 383 questions
AI Fails on Theory of Mind Tasks
Answers 383 questions
Interpretable One Shot Learning
Answers 383 questions
Brain Inspired AI
Answers 383 questions
The Imitation Game
Answers 383 questions
Defending Against Adversarial Attacks
Answers 383 questions
Easily Fooling Deep Neural Networks
Answers 383 questions
The Master Algorithm
Answers 383 questions
AI Platforms
Answers 383 questions
Complexity and Cryptography
Answers 383 questions
Transfer Learning
Answers 383 questions
Deploying LLMs
Answers 383 questions

Dexa/Kyle Polich

The Defeat of the Winograd Schema Challenge

Topics covered

Popular Clips

Common Sense Reasoning

New Frontiers in AI

Evaluating Model Accuracy

Turing Test Challenges

Model Evaluation Challenges

Winograd Schema Challenge

Schema Challenge Insights

Ambiguity in Language

Language Model Advances

Schema Evolution

Episode Highlights

Common Sense Reasoning

Winograd Schema Milestone

Language Model Evolution

AI Benchmarking Future

Related Episodes

Winograd Schema Challenge

Graph Databases and AI

Machine Learning Done Wrong

StrategyQA and Big Bench

AI Fails on Theory of Mind Tasks

Interpretable One Shot Learning

Brain Inspired AI

The Imitation Game

Defending Against Adversarial Attacks

Easily Fooling Deep Neural Networks

The Master Algorithm

AI Platforms

Complexity and Cryptography

Transfer Learning

Deploying LLMs

The Defeat of the Winograd Schema Challenge

Topics covered

Popular Clips

Episode Highlights

Common Sense ReasoningVid Kocijan explores the intricacies of pretraining in large language models and the challenges of achieving reliable common sense reasoning. He provides insights into the evolution of the field and the dominance of LLMs in natural language processing.

Common Sense Reasoning

Winograd Schema Milestone

Language Model EvolutionThe discussion shifts to the historical development and progress in natural language processing, focusing on the impact of language models like BERT and GPT. Vid Kocijan shares insights into the evolution of schemas and their role in advancing NLP capabilities.

Language Model Evolution

AI Benchmarking FutureThe Turing Test and its alternatives, like the Winograd Schema Challenge, are scrutinized for their ability to measure AI's general intelligence. Vid Kocijan discusses the need for new benchmarks that truly assess AI capabilities beyond current limitations.

AI Benchmarking Future

Related Episodes