Robust Fit to Nature

Acting vs. Understanding

Neural networks are designed to act rather than understand, a concept that Uri Hasson explores by comparing them to the human brain. He explains that while neural networks perform specific functions like recognizing faces or predicting words, they lack the ability to truly understand the world, unlike the human brain, which aims to comprehend underlying structures 1. This distinction highlights the difference between acting and understanding, as Hasson notes, "Science tries to understand. Science doesn't try to feed the data."

Science tries to understand. Science doesn't try to feed the data.

--- Uri Hasson

The conversation also touches on the evolution of neural networks, emphasizing the need for biases and efficient designs to facilitate learning, much like the human brain's exposure to vast data structures 2.

   

Extrapolation Limits

The limitations of neural networks in extrapolating data reveal a significant gap compared to human cognitive abilities. Uri Hasson points out that while neural networks excel in interpolation, they struggle with extrapolation, failing to predict outcomes beyond their training data 3. This is contrasted with human intelligence, which can understand rules and apply them to new situations, as seen in the ability to grasp Newton's laws and apply them universally.

Humans have this capacity to extrapolate and to really understand the underlying structure.

--- Uri Hasson

Hasson suggests that the brain's higher cognitive functions, though slow and error-prone, can achieve rule-based thinking and extrapolation, a feat that neural networks have yet to master 4.