Published Sep 10, 2023

Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!

Prof. Melanie Mitchell explores the flaws in current AI benchmarks, advocating for a shift towards contextual understanding and cognitive psychology to redefine true machine intelligence. She challenges traditional perceptions of AI, proposing rigorous testing and dispelling existential threat myths.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

#57 - Prof. Melanie Mitchell - Why AI is harder than we think
Answers 383 questions
Francois Chollet - On the Measure of Intelligence
Answers 383 questions
Mahault Albarracin - Cognitive Science
Answers 383 questions
#65 Prof. PEDRO DOMINGOS [Unplugged]
Answers 383 questions
#111 - AI moratorium, Eliezer Yudkowsky, AGI risk etc
Answers 383 questions
#046 The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)
Answers 383 questions
#72 Prof. KEN STANLEY 2.0 - On Art and Subjectivity [UNPLUGGED]
Answers 383 questions
Neel Nanda - Mechanistic Interpretability
Answers 383 questions
#94 - ALAN CHAN - AI Alignment and Governance #NEURIPS
Answers 383 questions
Prof. Daniel Dennett - Could AI Counterfeit People Destroy Civilization? (SPECIAL EDITION)
Answers 383 questions
Pattern Recognition vs True Intelligence - Francois Chollet
Answers 383 questions
Taming Silicon Valley - Prof. Gary Marcus
Answers 383 questions
#58 Dr. Ben Goertzel - Artificial General Intelligence
Answers 383 questions
Can AI therapy be more effective than drugs?
Answers 383 questions
The Social Dilemma - Part 2
Answers 383 questions

Dexa/Machine Learning Street Talk (MLST)

Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!

Topics covered

Popular Clips

Brain Computations Debate

Evolving Benchmarks

Language Model Experimentation

Refining AI Understanding

Evolving Benchmarks

AI Intelligence Complexity

Testing Machine Intelligence

Scaling Social Systems

AI Solutions Discussion

AI Superintelligence Fallacy

Reporting Insights

Abstract Reasoning Challenges

Unraveling Goals

Understanding Information Leakage

Understanding AI Models

Episode Highlights

AI Benchmark Limitations

Machine Understanding

Redefining Intelligence

Related Episodes

#57 - Prof. Melanie Mitchell - Why AI is harder than we think

Francois Chollet - On the Measure of Intelligence

Mahault Albarracin - Cognitive Science

#65 Prof. PEDRO DOMINGOS [Unplugged]

#111 - AI moratorium, Eliezer Yudkowsky, AGI risk etc

#046 The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)

#72 Prof. KEN STANLEY 2.0 - On Art and Subjectivity [UNPLUGGED]

Neel Nanda - Mechanistic Interpretability

#94 - ALAN CHAN - AI Alignment and Governance #NEURIPS

Prof. Daniel Dennett - Could AI Counterfeit People Destroy Civilization? (SPECIAL EDITION)

Pattern Recognition vs True Intelligence - Francois Chollet

Taming Silicon Valley - Prof. Gary Marcus

#58 Dr. Ben Goertzel - Artificial General Intelligence

Can AI therapy be more effective than drugs?

The Social Dilemma - Part 2

Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!

Topics covered

Popular Clips

Episode Highlights

AI Benchmark LimitationsMelanie Mitchell critiques current AI benchmarks, arguing they often misrepresent machine intelligence by encouraging reverse engineering rather than genuine understanding. She advocates for more systematic testing and detailed reporting to truly assess AI capabilities.

AI Benchmark Limitations

Machine UnderstandingMelanie Mitchell critiques the limitations of AI models in achieving human-like understanding, emphasizing the need for rigorous experimental methods. She explores the philosophical implications of AI intelligence, arguing against the notion of AI as an existential threat.

Machine Understanding

Redefining Intelligence

Related Episodes