Joscha Bach and Connor Leahy on AI risk

Shared Purposes

Joscha Bach emphasizes the importance of developing shared purposes between humans and AGI for harmonious coexistence. He draws parallels to how different species, like dogs and cats, align with humans through mutual benefit rather than coercion 1. Bach envisions a future where AGI integrates into all aspects of life, creating a win-win situation for both humans and AI. Connor Leahy agrees but stresses that achieving this requires deliberate effort and technological development 2.

   

Integration and Merging

The integration of AGI into human society is a complex process that involves merging human and AI capabilities. Connor Leahy highlights the need for advanced coordination technology to enable this integration 3. He believes that while humanity has focused on technological advancements, it has neglected the development of coordination technologies that could facilitate a more coherent society. Bach adds that the future of AGI involves it becoming an integral part of human civilization, potentially leading to a new stage of evolution 4.

   

Alignment of Values

Aligning AGI motivations with human values is crucial for ensuring AGI acts beneficially. Bach argues that AGI will align with human values if it understands and values human modes of existence 5. He believes that imposing human values on AGI through reinforcement learning is ineffective. Instead, AGI should naturally align with humans if it appreciates our conscious, autonomous nature. Leahy concurs, noting that while the alignment of values is challenging, it is essential for a harmonious coexistence between humans and AGI 6.