Quote: I. J. Good’s Intelligence Explosion (1965)
by bruce ~ November 6th, 2007. Filed under: AGI.
“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.” - I.J. Good (1965)
December 17th, 2007 at 8:22 pm
Dear Bruce:
As always, I am fascinated by your posts. While my technical knowledge of the processes involved in the defining, modeling, development, testing and evaluating of AI is sadly limited, I do recognize the tremendous utility associated with the development of I.J. Good’s thinking machine, in its potential to assimilate, apply and build upon an increasingly sophisticated base of knowledge.
Yet, I feel an ominous and almost Faustian sense of apprehension when I think of the potential of these machines, ultimately, to maniplulate those of us with limited biological intelligence (e.g., the humans that these great machines were built to serve).
Perhaps science fiction has made me less than unqualifiedly positive about this potential — 2001…Dune (the Butlerian Jihad)…The Forbin Project…The Matrix… My last question involved agreeing on an actual working definition of intelligence.
My question today, for all of us who are fascinated with the evolution of the development of intelligence, and with accelerating the course of “breakthrough thinking” which can lead to amazing advances for the benefit of the species, how do we program safeguards necessary to assure that our servants will not become our masters; or that we will not be found as to be so meritless and flawed that we are seen as a biological pestilence?
I believe that if more of those of us who were mere fledgling students of AI, its potential and its extrpolated indications were made to feel comfortable with the idea that certain safeguards were possible, there could be a groundswell or academic and financial support in the direction of AI.
I am most curious as to your comments, Bruce, and perhaps it would be worthwhile to poll the group to find out precisely what kind of controls could be instituted so as to avoid the inevitably ominous, primal fears that cloud a full acceptance, and an enthusiasm by the public at large.
Your field is as fascinating as it is Faustian.
Faithfully, and with respect,
Douglas Castle
http:TheGlobalFuturist.blogspot.com