The Turing Ratio : A Framework for Open-Ended Task Metrics

The Turing Test is of limited use for entities differing substantially from human performance levels. We suggest an extension of Turing’s idea to a more differentiated measure the "Turing Ratio" which provides a framework for comparing human and algorithmic task performance, up to and beyond human performance levels. Games and talent levels derived from pairwise comparisons provide examples of the concept. We also discuss the related notions of intelligence amplification and task breadth. Intelligence amplification measures total computational efficiency (the computational benefit gained relative to investment, including programmer time, hardware, and so on); we argue that evolutionary computation is a key amplifier of human intelligence. Task breadth is an attempt to weight Turing Ratios by the frequency and importance of the task they measure doing well at a broad range of tasks is an empirical definition of “intelligence”. Measuring Turing Ratios and considering task breadth, prior knowledge, and time series of the measures may yield long-term insight into both open-ended computational approaches and the underlying task domains being measured.

[1]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[2]  E. Berlekamp,et al.  Winning Ways for Your Mathematical Plays , 1983 .

[3]  Mark Glickman An Introductory Analysis of the United States Chess Federation Rating System , 1985 .

[4]  R. Lathe Phd by thesis , 1988, Nature.

[5]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[6]  David Scott Warren Rating Systems for Gameplayers, and Learning , 1994 .

[7]  J. Schaeffer One Jump Ahead , 1997, Springer New York.

[8]  M. Glickman Parameter Estimation in Large Dynamic Paired Comparison Experiments , 1999 .

[9]  David B. Fogel,et al.  Anaconda defeats Hoyle 6-0: a case study competing an evolved checkers program against commercially available software , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[10]  Nan Bahr IT and learning , 2000 .

[11]  R. Hanson Economic Growth Given Machine Intelligence , 2000 .

[12]  R. French,et al.  The Turing Test : The First Fifty Years , 2000 .

[13]  J. Pollack,et al.  Evolution of complexity in real-world domains , 2001 .

[14]  Bruno Bouzy,et al.  Computer Go: An AI oriented survey , 2001, Artif. Intell..

[15]  Peter J. Bentley,et al.  CREATIVE EVOLUTIONARY SYSTEMS , 2001 .

[16]  Lance Fortnow,et al.  Computational depth , 2001, Proceedings 16th Annual IEEE Conference on Computational Complexity.

[17]  Steffen Christensen,et al.  The Turing Ratio: Metrics For Open-ended Tasks , 2002, GECCO.

[18]  Varol Akman,et al.  Turing Test: 50 Years Later , 2000, Minds and Machines.