Logical induction: progress in AI alignment

The Machine Intelligence Research Institute (MIRI) is interested in reasoning about highly advanced AI systems before they exist, specifically for developing models of safety and control for such systems.

In this talk, MIRI's Andrew Critch examines some criteria for "ideal" logical induction, a new algorithm for logical induction that satisfies many desirable properties, and some implications for what a very powerful AI system is able to learn.