#56 - Dr. Walid Saba, Gadi Singer, Prof. J. Mark Bishop (Panel discussion)

Deep Learning Limits

The limitations of deep learning are evident in its struggle to achieve higher-level reasoning and understanding. Walid Saba points out that while deep learning can identify objects like horses and fences, it falls short in reasoning about these objects 1. Keith Duggar highlights the need for feedback between recognition and reasoning systems, suggesting that reasoning plays a crucial role in refining what we perceive 1. J. Mark Bishop adds that artificial neural networks, though inspired by the human brain, lack the dynamic detail necessary for true intelligence 2.

Learning is one part of what we do. Our cognitive capacities are much more complex. Until we understand the real problem, how big it is, like autonomous driving, that ignored the frame problem, it's all going to collapse.

--- Walid Saba

The panel agrees that without addressing these fundamental issues, AI progress will remain stagnant, akin to climbing trees in an attempt to reach the moon 3.

   

Image Interpretation

Image interpretation remains a significant challenge for deep learning systems, as they often fail to capture the context-dependent nature of images. J. Mark Bishop illustrates this with the example of a horse by a fence, where different observers might focus on different elements based on their background or interests 4. He argues that the meaning of an image is contingent on its use, and there is no single canonical interpretation 4. Walid Saba concurs, emphasizing that while the content of an image or text is not inherently ambiguous, our interpretations can vary widely 5.

The syntax of the words is not sufficient to pin down the meaning of the piece. And I know Walid sees things differently, but I don't. I think there is always a context and that helps us arrive at the deeper meaning of words.

--- J. Mark Bishop

This complexity highlights the gap between current deep learning capabilities and human-like understanding, underscoring the need for more sophisticated models that can account for context and interpretation.