Eliezer Yudkowsky frames ASI alignment as an irretrievable one-shot problem
Yudkowsky argues that safe tests of weaker systems cannot validate alignment under the materially different conditions where an ASI could cause catastrophe, and a severe first failure would prevent a corrective retry.
0 comments · 0 votes
Sign in to join the discussion →
No comments yet. Start the discussion.
TOWARD DOOM22confidence 56/100
Why it moved the index
The post adds a distinct technical mechanism: distribution shift separates survivable development tests from the first capability regime in which alignment failure could remove the possibility of recovery. It is a detailed first-person argument, but not an empirical demonstration that the predicted critical transition will occur.
AUDIT TRAIL
Assessment history
- R1Toward 22 · confidence 56
Initial inclusion from a dated first-person source recovered during Eliezer Yudkowsky's historical backfill.
13 Aug 2026