programs that are outside the boundaries of human comprehension. And because computer programs can test and process
information so much faster than humans can, those programs
will be able to analyze the results and refine their creations with
Imagine this scenario: Two people are asked to write a story.
One person is given pen and paper, while the other receives a
computer with a word processor. Assuming basic typing skills
and similar amounts of time needed to invent the story, the
person with the computer will finish writing the story sooner—
typing is generally faster than writing things out by hand. Then,
imagine each person is asked to revise the story, adding new
sections and resubmitting the entire piece. Now the comparison
is stark: The person with the computer can delete bad sections
and easily insert new ones where they need to be. The person
with pen and paper must write everything out again, inventing
new sections along the
way. Each new revision
widens the gap between
the writer with the com-
puter and the one with
implements. The person
with the word processor
might finish the sixth re-
vision while the other person is just beginning the third draft.
That is essentially the situation we face with artificial intel-
ligence—we are stuck with pen and paper, while machines can
create, analyze, and adapt programs at an alarmingly fast pace.
Which is precisely why we can’t wait to begin discussions of
policy: If we adopt a reactionary attitude, we’ll be so far behind
technology, we may never catch up.
Our human brains may be
unable to even imagine
the kind of intelligence
we may be creating,
and so we must be very
careful about the kinds of
programs we develop and
the kinds of moral, human
values we put into them .