A Formal Measure of Machine Intelligence

Abstract A fundamental problem in artificial intelligence isthat nobody really knows what intelligence is. Theproblem is especially acute when we need to con-sider artificial systems which are significantly dif-ferent to humans. In this paper we approach thisproblem in the following way: We take a numberof well known informal definitions of human intelli-gence that have been given by experts, and extracttheir essential features. These are then mathemat-ically formalised to produce a general measure ofintelligence for arbitrary machines. We believe thatthis measure formally captures the concept of ma-chine intelligence in the broadest reasonable sense. 1 Introduction Most of us think that we recogniseintelligence whenwe see it, but we are not really sure how to pre-cisely define or measure it. We informally judgethe intelligence of others by relying on our past ex-periences in dealing with people. Naturally, thisnaive approach is highly subjective and imprecise.A more principled approach would be to use oneof the many standard intelligence tests that areavailable. Contrary to popular wisdom, these tests,when correctly applied by a professional, deliverstatisticallyconsistentresults andhaveconsiderablepower to predict the future performance of individ-uals in many mentally demanding tasks. However,while these tests work well for humans, if we wishto measure the intelligence of other things, perhapsof a monkey or a new machine learning algorithm,they are clearly inappropriate.One response to this problem might be to de-velop specific kinds of tests for specific kinds of en-tities; just as intelligence tests for children differto intelligence tests for adults. While this workswell when testing humans of different ages, it comesundone when we need to measure the intelligenceof entities which are profoundly different to eachother in terms of their cognitive capacities, speed,senses, environments in which they operate, and soon. To measure the intelligence of such diverse sys-tems in a meaningful way we must step back fromthe specifics of particular systems and establish theunderlying fundamentals of what it is that we arereally trying to measure. That is, we need to estab-lish a notion of intelligence that goes beyond thespecifics of particular kinds of systems.The difficulty of doing this is readily apparent.Consider, for example, the memory and numericalcomputation tasks that appear in some intelligencetests and which were once regarded as defining hall-marks of human intelligence. We now know thatthese tasks are absolutely trivial for a machine andthus do not test the machine’s intelligence. Indeedeven the mentally demanding task of playing chesshas been largely reduced to brute force search. Astechnology advances, our concept of what intelli-gence is continues to evolve with it.How then are we to develop a concept of intelli-gence that is applicable to all kinds ofsystems? Anyproposed definition must encompass the essence ofhuman intelligence, as well as other possibilities, ina consistent way. It should not be limited to anyparticular set of senses, environments or goals, norshould it be limited to any specific kind of hard-ware, such as silicon or biological neurons. It shouldbe based on principles which are sufficiently funda-mental so as to be unlikely to alter over time. Fur-thermore, the intelligence measure should ideally beformally expressed, objective, and practically real-isable.This paper approaches this problem in the fol-lowing way. In Section 2 we consider a range of def-initions of human intelligence that have been putforward by well known psychologists. From thesewe extract the most common and essential featuresand use them to create an informal definition ofintelligence. Section 3 then introduces the frame-1