What is an intelligence explosion?

An intelligence explosion refers to the hypothetical, rapid increase in artificial intelligence (AI) capabilities where AI systems improve exponentially through positive feedback loops. This could occur once AI reaches human-level functionality and could potentially invent breakthroughs like new architectures or optimal training methods. The term suggests that such advances would allow AI to far surpass human intelligence in a relatively short time.

Carl Shulman discussed this idea, mentioning that an intelligence explosion might include AI improving itself or advancing in hardware and software technology. This exponential improvement is part of what fuels the race between shaping AI motivations and the autonomous increase of AI capabilities, which could lead to a scenario where AI could exceed human control and understanding. In essence, an intelligence explosion could occur as a result of a virtuous cycle of AI self-improvement, resulting in rapid and profound advancements in intelligence, potentially leading to superintelligent systems 1 2.