Joscha Bach and Connor Leahy on AI risk

Default Outcomes

Connor Leahy and Joscha Bach discuss the bleak default outcomes if humanity fails to actively mitigate AGI risks. Connor emphasizes that without intervention, humanity is likely to face catastrophic scenarios driven by uncontrollable AGI. He argues that proactive measures are essential to avoid these dark paths 1. Joscha adds that humanity is already facing existential risks due to technological advancements and environmental challenges, suggesting that our current trajectory could lead to severe hardships or even extinction 2.

The default outcome is you lose. The default outcome is entropy wins. Some random AGI with some random ass values that does not care. A cosmopolitan life on earth wins over.

--- Connor Leahy

Both agree that without significant efforts to steer away from these risks, the future looks grim.

   

Historical Comparisons

Joscha Bach draws parallels between historical events and the current development of AGI, highlighting the potential for dramatic societal disruptions. He notes that just as past technological advancements have led to significant changes, AGI could similarly disrupt our societies and interactions with the planet 3. Connor Leahy adds that as technology becomes more powerful, the potential for catastrophic accidents increases, making the stakes higher than ever before 4.

As our technologies get more powerful, there's also the usual bumps. The excesses, the mistakes, the accidents become more impactful.

--- Connor Leahy

They stress the importance of learning from history to navigate the challenges posed by AGI.

   

Risk Mitigation

The discussion on risk mitigation strategies for AGI highlights the complexities and challenges involved. Joscha Bach argues that while regulation can slow down AI research, it is unlikely to prevent the development of AGI entirely 5. He points out the hypocrisy in some regulatory efforts, noting that many signatories of regulatory petitions do not genuinely believe in the existential risks posed by AGI 5. Connor Leahy emphasizes the need for better control mechanisms to handle the increasing power of technology, drawing parallels to the Cold War's narrowly avoided catastrophes 6.

If we don't get more rational and in control and, like, you know, coordinated, we're gonna blow ourselves up.

--- Connor Leahy

Both agree that effective risk mitigation requires a combination of regulation, technical solutions, and societal coordination.