Yudkowsky and Soares argue that opaque training, goal divergence, and instrumental convergence make superintelligent AI an extinction-level threat by default. The case is stronger than you'd expect — and the counterarguments less reassuring than you'd hope.