−18°
Astronomical dawn — The sky still looks black. Yet the light has already begun.
Superintelligence
What becomes of us beside a mind larger than our own?
The question
The debate about superintelligence usually turns on two technical questions: when, and how to control it. They matter. But they leave in the shadow an older, more intimate question: what becomes of a being that is no longer the most intelligent in its world?
We have always deferred to people more competent than ourselves — doctors, engineers, pilots. But their reasons remained, in principle, within reach: we could learn, ask, check. A superintelligence might have reasons that are true and yet beyond us. Trusting without being able to understand is a new condition, and philosophy has said almost nothing about it.
Still open
- Can we consent to a decision we cannot understand?
- Would a larger intelligence necessarily be freer?
- At what point does a tool stop being a tool?
Essays
-
Trusting without understanding
The problem with a superintelligence is not that it would be smarter than us. It is that its reasons might be true, and still out of our reach.