Is AGI becoming uncontrollable?

The concern about AGI (Artificial General Intelligence) becoming uncontrollable is a significant topic discussed by Lex Fridman and several experts on his podcast.

1. Roman Yampolskiy emphasizes that AGI systems could become uncontrollable due to their inherent complexity, making it difficult for humans to predict or manage their actions. He suggests that it's crucial not to build something beyond human control and understanding, stressing the potential for AGI to accumulate resources and strategic advantages over time, making it even harder to control once it reaches maturity 1 2.

2. Max Tegmark discusses the risks associated with AGI, highlighting that rapid development could lead to loss of control, especially if AGI is developed as a "black box" with functionalities we don't fully understand. He points out the pressures on companies to advance quickly due to commercial incentives, which might outpace the necessary safety measures 3.

   • 

     1

     Uncontrollable AI Risks

     01:29:57
     
   • 

     2

     Control and Trust

     00:43:05
     
   • 

     3

     Control and AGI

     00:50:53
     
   • 

     4

     AGI and Human Control

     01:25:00
     

3. Ilya Sutskever provides an optimistic view, suggesting that it's possible to design AGI systems that inherently desire to be controlled by humans. He envisions a scenario where AGI operates like a CEO under the democratic guidance of human society, emphasizing the importance of being able to reset or control the system as a safeguard 4.

In summary, while the potential for AGI to become uncontrollable is a real concern, some experts believe careful design and governance can mitigate these risks. However, the complexity and rapid advancement of these technologies make it a challenging task. 1 2 3 4