Experience Matters

Deciding which piece of information to acquire or attend to is fundamental to perception, categorization, medical diagnosis, and scientific inference. Four statistical theories of the value of information—information gain, Kullback-Liebler distance, probability gain (error minimization), and impact—are equally consistent with extant data on human information acquisition. Three experiments, designed via computer optimization to be maximally informative, tested which of these theories best describes human information search. Experiment 1, which used natural sampling and experience-based learning to convey environmental probabilities, found that probability gain explained subjects’ information search better than the other statistical theories or the probability-of-certainty heuristic. Experiments 1 and 2 found that subjects behaved differently when the standard method of verbally presented summary statistics (rather than experience-based learning) was used to convey environmental probabilities. Experiment 3 found that subjects’ preference for probability gain is robust, suggesting that the other models contribute little to subjects’ search behavior.

[1]  L. Stein,et al.  Probability and the Weighing of Evidence , 1950 .

[2]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[3]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[4]  P. Wason.  On the Failure to Eliminate Hypotheses in a Conceptual Task , 1960 .

[5]  L. Beach,et al.  Man as an Intuitive Statistician , 2022 .

[6]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[7]  P. Johnson-Laird,et al.  Psychology of Reasoning: Structure and Content , 1972 .

[8]  R. Lindsay,et al.  On Estimating the Diagnosticity of Eyewitness Nonidentifications , 1980 .

[9]  Y. Trope,et al.  Confirmatory and diagnosing strategies in social information gathering. , 1982 .

[10]  H. Heyer,et al.  Information and Sufficiency , 1982 .

[11]  R. Skov,et al.  Information-gathering processes: Diagnosticity, hypothesis-confirmatory strategies, and perceived hypothesis confirmation. , 1986 .

[12]  J. Klayman,et al.  Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing , 1987 .

[13]  J. Baron,et al.  Heuristics and biases in diagnostic reasoning: II. Congruence, information, and certainty☆ , 1988 .

[14]  J. Klayman,et al.  Information selection and use in hypothesis testing: What is a good question, and what is a good answer? , 1992, Memory & cognition.

[15]  M. Gluck,et al.  Probabilistic classification learning in amnesia. , 1994, Learning & memory.

[16]  Nick Chater,et al.  A rational analysis of the selection task as optimal data selection. , 1994 .

[17]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[18]  Stuart J. Russell.  Rationality and Intelligence , 1995, IJCAI.

[19]  J. Klayman.  Varieties of Confirmation Bias , 1995 .

[20]  N. Chater,et al.  RATIONAL EXPLANATION OF THE SELECTION TASK , 1996 .

[21]  J.,et al.  Dynamics of Rule Induction by Making Queries : Transition Between StrategiesIris , 1996 .

[22]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[23]  R. Nickerson.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[24]  J. Kruschke,et al.  A model of probabilistic category learning. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[25]  Lorenzo Magnani,et al.  Model-Based Reasoning in Scientific Discovery , 1999, Springer US.

[26]  Ulrich Hoffrage,et al.  Simplifying Bayesian Inference: The General Case , 1999 .

[27]  Javier R. Movellan,et al.  Active Inference in Concept Learning , 2000, NIPS.

[28]  G. Gigerenzer,et al.  Teaching Bayesian reasoning in less than two hours. , 2001, Journal of experimental psychology. General.

[29]  Joachim Denzler,et al.  Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  R. Hertwig,et al.  Decisions from Experience and the Effect of Rare Events in Risky Choice , 2004, Psychological science.

[31]  Nigel Harvey,et al.  Blackwell Handbook of Judgment and Decision Making , 2004 .

[32]  Aaron B. Hoffman,et al.  Eyetracking and selective attention in category learning , 2005, Cognitive Psychology.

[33]  Jonathan D. Nelson.  Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain. , 2005, Psychological review.

[34]  C. Mckenzie,et al.  Increased sensitivity to differentially diagnostic answers using familiar materials: Implications for confirmation bias , 2006, Memory & cognition.

[35]  Kiyohiko Nakamura.  Neural representation of information measure in the primate premotor cortex. , 2006, Journal of neurophysiology.

[36]  Garrison W. Cottrell,et al.  A probabilistic model of eye movements in concept formation , 2007, Neurocomputing.

[37]  Jonathan D. Nelson.  Towards a rational theory of human information acquisition , 2007 .

[38]  Craig R. M. McKenzie.  Hypothesis Testing and Evaluation , 2008 .

[39]  N.J. Butko,et al.  I-POMDP: An infomax model of eye movement , 2008, 2008 7th IEEE International Conference on Development and Learning.

[40]  James E. Corter,et al.  Observed attention allocation processes in category learning , 2008, Quarterly journal of experimental psychology.

[41]  Jonathan D. Nelson.  Naïve optimality: Subjects' heuristics can be better motivated than experimenters' optimal models , 2009, Behavioral and Brain Sciences.

[42]  Mark A. Pitt,et al.  Adaptive Design Optimization: A Mutual Information-Based Approach to Model Discrimination in Cognitive Science , 2010, Neural Computation.