AI Risk Predictions

Sayash discusses how subjective probability estimates in the AI community may be influenced by cognitive biases, leading to inflated perceptions of extinction risk. He highlights a recent prediction tournament that revealed the justifications for these estimates are often no more reliable than casual opinions. Additionally, he points out a systematic tendency among forecasters to overestimate low-probability risks, raising questions about the accuracy of current risk assessments in AI.