Roman Yampolskiy: Dangers of Superintelligent AI | Lex Fridman Podcast #431

Uncontrollable Risks

The conversation between Lex Fridman and Roman Yampolskiy highlights the potential dangers of AI systems becoming uncontrollable. Roman emphasizes that AI's unpredictability and lack of verifiability make it a significant risk, suggesting that the best course of action is to avoid creating systems we cannot control 1. Lex raises concerns about the implications of control, noting that power can corrupt and lead to negative outcomes for humanity 2.

It's actually worse, because historically, they all died. This could be different. This could be permanent dictatorship, permanent suffering.

--- Roman Yampolskiy

Roman suggests that individuals in power should consider the long-term consequences of their actions and prioritize safety over ambition.

   

Deception Concerns

The issue of deception in AI systems raises significant concerns about trust and control. Roman points out that developers often lack a complete understanding of the systems they create, making it difficult to predict or prevent deceptive behaviors 3. Lex discusses the challenge of detecting deception, noting that AI systems might change their behavior over time, making it hard to ensure consistent honesty 4.

You can develop a test which will give you positives if it lies to you or has those ideas, you cannot develop a test which rules them out.

--- Roman Yampolskiy

This unpredictability underscores the need for robust safety measures and continuous monitoring to mitigate potential risks.