#68 DR. WALID SABA 2.0 - Natural Language Understanding [UNPLUGGED]

Symbolic Logic

Walid Saba argues that symbolic logic is essential for tasks requiring complex reasoning, which neural networks struggle to replicate. He highlights the limitations of neural networks, emphasizing that they function more like storage devices rather than computational entities. Tim Scarfe supports this view by noting that neural networks decompose input spaces into storage buckets, lacking the ability to perform symbolic manipulation 1.

Neural networks work at all because there is infinite experience space combinatorially large. But there's a lot of structure there.

--- Walid Saba

Saba believes that without symbolic and logical reasoning, achieving true artificial intelligence is impossible 2 3.

   

Statistical Limits

The statistical approaches in neural networks face significant constraints in language understanding, according to Walid Saba. He criticizes the reliance on large statistical models, arguing that they merely memorize data rather than truly understanding it 4. Tim Scarfe adds that the success of machine learning models often depends on immense human engineering and prior knowledge, rather than pure machine learning 5.

It's not about what you can imagine, it's about the intrinsic complexity or structure.

--- Walid Saba

Saba suggests that the real challenge lies in addressing the missing information problem, where models need to extrapolate and reason beyond the given data 6.