Tag: Existential risk
-
Revisiting the shutdown problem (Part 1: Introduction)
The catastrophic shutdown problem is (roughly) the problem of designing AI systems that can be shut down when their acts would lead to existential catastrophe.
-
If anyone builds it, everyone dies (Part 4: We would lose)
Yudkowsky and Soares argue that we would lose a conflict with artificial superintelligence
-
If anyone builds it, everyone dies (Part 3: Remaining arguments for misalignment)
This post addresses the second of Yudkowsky and Soares’ two main arguments for misalignment in Chapter 4.
-
If anyone builds it, everyone dies (Part 1: Introduction and cruxes)
Yudkowsky and Soares lay out their case for AI risk. This post introduces the book and identifies key cruxes.
-
Exaggerating the risks (Part 20: AI 2027 timelines forecast, benchmarks and gaps)
The second part of the AI 2027 timelines model relies primarily on insufficiently evidenced forecasts.
-
Exaggerating the risks (Part 18: Introduction to AI 2027)
This post introduces the AI 2027 report.
-
Instrumental convergence and power-seeking (Part 2: Benson-Tilsen and Soares)
A leading power-seeking theorem due to Benson-Tilsen and Soares does not ground the needed form of instrumental convergence
-
Instrumental convergence and power-seeking (Part 1: Introduction)
Power-seeking theorems aim to formally demonstrate that artificial agents are likely to seek power in problematic ways. I argue that leading power-seeking theorems do not succeed.
-
The scope of longtermism (Part 5: A case study – Existential risk)
Many longtermists think that existential risk mitigation escapes the scope-limiting factors. To what extent is this true?