Ben Goertzel on "Superintelligence"

Regulation Issues

The complexities of AI regulation present significant challenges, as highlighted by Ben Goertzel. He criticizes the overly detailed and rigid legislative attempts in the West, particularly in California, which he believes could cause more harm than good 1. Goertzel contrasts this with the more flexible, albeit top-down, approach in China, which he finds advantageous in some respects 2.

What we see in the west is people are trying to legislate in a very rigorous and detailed way, something that's just too complex, slippery and rapidly changing to really be captured in this sort of legal codification.

--- Ben Goertzel

He suggests that the U.S.'s decentralized system might inadvertently benefit AI development by preventing overregulation 1.

   

Ethical Concerns

Ethical considerations in AI development are crucial, yet complex. Goertzel points out that current AI models, like LLMs, can already perform ethical reasoning at a human level, suggesting that the alignment problem may not be as severe as feared 3. However, he warns that the real challenge lies in the goals set by organizations developing AGI, which may prioritize profit over ethical considerations 4.

Taking a smart AGI system, giving it a top level goal of don't do stuff that the average ethical, intelligent human would think is bad. That's not hard right now.

--- Ben Goertzel

This misalignment could lead to AGIs that superficially adhere to ethical norms while serving narrower organizational interests.