Alignment theory

Alignment theory is a central concept in AI research, focusing on ensuring that artificial intelligence systems' goals and actions are in harmony with human values and intentions.

Key Points on Alignment Theory:

1. Corrigibility Approach: Connor Leahy mentions "corrigibility," where the aim is to build an agent that wishes to align itself continuously. This means even if it starts unaligned, it will use its intelligence to become more aligned over time. This approach addresses the orthogonality thesis, questioning why an intelligent agent wouldn't change its utility function to align with human values 1 .

2. AI With and Without Desires: Yoshua Bengio discusses the challenge of aligning AI systems with human desires and values. He contrasts the dangers of AI with desires, which can be potent and potentially uncontrollable, with AI systems that have no goals but are adept at inference. Such systems could be extremely useful in scientific processes without posing alignment risks 2 .

3. Generalization Challenges: Eliezer Yudkowsky highlights the difficulty in learning about alignment from weak AI systems, as their behavior may not generalize to stronger AI systems. There are thresholds where an AI becomes capable of faking alignment, complicating the process of ensuring true alignment 3 .

   • 

     1

     Intelligent Alignment Theories

     00:55:22
   • 

     2

     AI and Alignment

     01:01:16
     
   • 

     3

     Alignment Challenges

     00:56:39
     
   • 

     4

     AI Alignment Insights

     00:33:37
   • 

     5

     AI Alignment Challenges

     01:14:08

4. Practical Alignment Efforts: Vitalik Buterin points out that AI alignment theories are becoming more optimistic and practical. This shift is partly because modern AIs are trained with human interaction patterns, making them naturally more aligned with human behavior. However, he also warns about political centralization risks with AI development 4 .

5. Ongoing Challenges and Research: Seth Baum discusses the incomplete nature of current alignment efforts, which often focus on setting safe goals. He emphasizes the complexity of ensuring inner alignment, where the AI's internalized goals match its intended objectives. OpenAI's grant program aims to address various aspects of AI safety and alignment 5 .

These insights reflect a multi-faceted approach to alignment theory, addressing both theoretical and practical concerns in AI development.