How can AI learn common sense?

Sources:

AI can develop common sense primarily through techniques such as supervised learning and self-supervised learning.

  1. Supervised Learning: Traditional supervised learning requires human supervision where the AI is trained with labeled data. For example, in image recognition, the system is fed thousands of labeled images (e.g., images labeled as cats, dogs, etc.) and learns to identify these objects by recognizing patterns in the data. This method involves providing explicit truths to the AI about what each data point represents 1.

  2. Self-Supervised Learning: Unlike supervised learning, self-supervised learning minimizes human intervention. Here, the AI learns by observing and extracting patterns from large datasets without labeled inputs. This technique has been particularly successful in natural language processing and computer vision, allowing models to build a foundational understanding or "common sense" of the data they process. Essentially, it learns from the data structure itself rather than predefined labels, aiming to understand broader concepts and relationships inherent in the data 1.

  3. Lifelong Learning: Another aspect involves lifelong learning, where AI systems continuously learn and adapt over long periods, much like humans. Current machine learning systems struggle with maintaining and learning from long-term interactions and experiences. Innovations in this area aim to enable AI to remember and build on past interactions, crucial for developing a more nuanced common sense understanding 2.

These approaches help AI systems move beyond rigid, task-specific knowledge, allowing them to gain a more flexible and generalized understanding akin to human common sense.

Learning without Supervision

Lex discusses the differences between supervised and self-supervised learning in computer vision and language models. He explores the challenges of providing ground truth for supervised learning and the success of self-supervised learning in building common sense knowledge without human supervision.
Huberman Lab
Dr. Lex Fridman: Machines, Creativity & Love | Huberman Lab Podcast #29
1
2
RELATED QUESTIONS