Optimal indolence: a normative microscopic approach to work and leisure

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control-theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward. When the microscopic benefit-of-leisure increases nonlinearly with duration, the model generates behaviour that qualitatively matches various microfeatures of subjects’ choices, including the distribution of leisure bout durations as a function of the pay-off. We relate our model to traditional accounts by deriving macroscopic, molar, quantities from microscopic choices.

[1]  James L Olds,et al.  Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. , 1954, Journal of comparative and physiological psychology.

[2]  C. B. Ferster,et al.  Schedules of reinforcement , 1957 .

[3]  T. F. Gilbert Fundamental dimensional properties of the operant. , 1958, Psychological review.

[4]  K. Breland,et al.  The misbehavior of organisms. , 1961 .

[5]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[6]  W M Baum,et al.  Choice as time allocation. , 1969, Journal of the experimental analysis of behavior.

[7]  W M Baum,et al.  On two types of deviation from the matching law: bias and undermatching. , 1974, Journal of the experimental analysis of behavior.

[8]  R J Herrnstein,et al.  Formal properties of the matching law. , 1974, Journal of the experimental analysis of behavior.

[9]  W. Baum,et al.  Time-based and count-based measurement of preference. , 1976, Journal of the experimental analysis of behavior.

[10]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[11]  H. Rachlin A molar theory of reinforcement schedules. , 1978, Journal of the experimental analysis of behavior.

[12]  B. Skinner,et al.  Giving up the ghost , 1981, Behavioral and Brain Sciences.

[13]  W M Baum,et al.  Optimization and the matching law as accounts of instrumental behavior. , 1981, Journal of the experimental analysis of behavior.

[14]  J. Kagel,et al.  Income-Leisure Tradeoffs of Animal Workers , 1981 .

[15]  A. Hamilton,et al.  Reward, performance, and the response strength method in self-stimulating rats: Validation and neuroleptics , 1985, Physiology & Behavior.

[16]  J. J. McDowell On the falsifiability of matching theory. , 1986, Journal of the experimental analysis of behavior.

[17]  H. Minardi One day at a time. , 2006, Nursing times.

[18]  J. Kagel,et al.  Consumption-leisure tradeoffs in pigeons: Effects of changing marginal wage rates by varying amount of reinforcement. , 1987, Journal of the experimental analysis of behavior.

[19]  P. Soubrié,et al.  Effects of imipramine-like drugs and serotonin uptake blockers on delay of reward in rats. Possible implication in the behavioral mechanism of action of antidepressants. , 1988, The Journal of pharmacology and experimental therapeutics.

[20]  S. Wighton One day at a time. , 1990, Nursing.

[21]  C. Gallistel,et al.  Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. , 1991, Behavioral neuroscience.

[22]  R. Frank Microeconomics and behavior , 1991 .

[23]  L. Green,et al.  Economic substitutability of electrical brain stimulation, food, and water. , 1991, Journal of the experimental analysis of behavior.

[24]  C. Gallistel,et al.  The function relating the subjective magnitude of brain stimulation reward to stimulation strength varies with site of stimulation , 1992, Behavioural Brain Research.

[25]  Patsy Haccou,et al.  Statistical Analysis of Behavioural Data: An Approach Based on Time-structured Models , 1992 .

[26]  T. A. Mark,et al.  Subjective reward magnitude of medial forebrain stimulation as a function of train duration and pulse frequency. , 1993, Behavioral neuroscience.

[27]  Satinder Singh Soft Dynamic Programming Algorithms: Convergence Proofs Soft Dynamic Programming Algorithms: Convergence Proofs , 1993 .

[28]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[29]  C. Gallistel,et al.  Saturation of subjective reward magnitude as a function of current and pulse frequency. , 1994, Behavioral neuroscience.

[30]  J. Kagel,et al.  Economic Choice Theory: An Experimental Analysis of Animal Behavior , 1995 .

[31]  P. Fletcher Effects of combined or separate 5,7-dihydroxytryptamine lesions of the dorsal and median raphe nuclei on responding maintained by a DRL 20s schedule of food reinforcement , 1995, Brain Research.

[32]  R. Thaler,et al.  Labor Supply of New York City Cabdrivers: One Day at a Time , 1997 .

[33]  B. Richmond,et al.  Neuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials , 1998, The Journal of Neuroscience.

[34]  M. Ho,et al.  5-Hydroxytryptamine and impulse control: prospects for a behavioural analysis , 1998, Journal of psychopharmacology.

[35]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[36]  J. Bizot,et al.  Serotonin and tolerance to delay of reward in rats , 1999, Psychopharmacology.

[37]  Paul Weirich Economic Choice Theory: An Experimental Analysis of Animal Behavior, John H. Kagel, Raymond C. Battalio, and Leonard Green. Cambridge University Press, 1995, xii + 230 pages , 1999, Economics and Philosophy.

[38]  J. Richards,et al.  Serotonergic mediation of DRL 72s behavior: receptor subtype involvement in a behavioral screen for antidepressant drugs , 1999, Biological Psychiatry.

[39]  J. J. McDowell,et al.  Falsification of matching theory's account of single-alternative responding: Herrnstein's k varies with sucrose concentration. , 2000, Journal of the experimental analysis of behavior.

[40]  E. Diener,et al.  End Effects of Rated Life Quality: The James Dean Effect , 2001, Psychological science.

[41]  O. Hikosaka,et al.  Modulation of saccadic eye movements by predicted reward outcome , 2001, Experimental Brain Research.

[42]  P. Hineline Beyond the molar-molecular distinction: we need multiscaled analyses. , 2001, Journal of the experimental analysis of behavior.

[43]  R. Shull,et al.  Response rate viewed as engagement bouts: effects of relative reinforcement and schedule type. , 2001, Journal of the experimental analysis of behavior.

[44]  W. Baum From molecular to molar: a paradigm shift in behavior analysis. , 2002, Journal of the experimental analysis of behavior.

[45]  J. Salamone,et al.  Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine , 2002, Behavioural Brain Research.

[46]  David S. Touretzky,et al.  Long-Term Reward Prediction in TD Models of the Dopamine System , 2002, Neural Computation.

[47]  A. Kacelnik,et al.  Cost can increase preference in starlings , 2002, Animal Behaviour.

[48]  P. Shizgal,et al.  Growth of brain stimulation reward as a function of duration and stimulation strength. , 2003, Behavioral neuroscience.

[49]  W. Baum Molar and molecular views of choice , 2004, Behavioural Processes.

[50]  J J McDowell,et al.  On the classic and modern theories of matching. , 2005, Journal of the experimental analysis of behavior.

[51]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[52]  Peter Shizgal,et al.  Employing labor-supply theory to measure the reward value of electrical brain stimulation , 2005, Games Econ. Behav..

[53]  K. Berridge The debate over dopamine’s role in reward: the case for incentive salience , 2007, Psychopharmacology.

[54]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[55]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[56]  P. Shizgal,et al.  The reinforcement mountain: allocation of behavior as a function of the rate and intensity of rewarding brain stimulation. , 2008, Behavioral neuroscience.

[57]  P. Shizgal,et al.  Rattus Psychologicus: Construction of preferences by self-stimulating rats , 2009, Behavioural Brain Research.

[58]  Jonathan Williams,et al.  Dynamic behavioural changes in the Spontaneously Hyperactive Rat 2 Control by novelty , 2009, Behavioural Brain Research.

[59]  Jonathan Williams,et al.  Dynamic behavioural changes in the Spontaneously Hyperactive Rat 1. Control by place, timing, and reinforcement rate , 2009, Behavioural Brain Research.

[60]  D. Bavykin,et al.  List of Symbols , 2009 .

[61]  Jonathan Williams,et al.  Dynamic behavioural changes in the Spontaneously Hyperactive Rat: 3. Control by reinforcer rate changes and predictability , 2009, Behavioural Brain Research.

[62]  P. Shizgal,et al.  At What Stage of Neural Processing Does Cocaine Act to Boost Pursuit of Rewards? , 2010, PloS one.

[63]  N. Daw,et al.  Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions , 2011, Neuropsychopharmacology.

[64]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[65]  P. Shizgal,et al.  Cannabinoid Receptor Blockade Reduces the Opportunity Cost at Which Rats Maintain Operant Performance for Rewarding Brain Stimulation , 2011, The Journal of Neuroscience.

[66]  W. Baum Introduction to molar behavior analysis , 2011 .

[67]  K. Doya,et al.  Activation of Dorsal Raphe Serotonin Neurons Underlies Waiting for Delayed Rewards , 2011, The Journal of Neuroscience.

[68]  P. Dayan Instrumental vigour in punishment and reward , 2012, The European journal of neuroscience.

[69]  K. Doya,et al.  Activation of Dorsal Raphe Serotonin Neurons Is Necessary for Waiting for Delayed Rewards , 2012, The Journal of Neuroscience.

[70]  Jean-Paul Chilès,et al.  Wiley Series in Probability and Statistics , 2012 .

[71]  P. Shizgal,et al.  Role of Dopamine Tone in the Pursuit of Brain Stimulation Reward , 2012, The Journal of Neuroscience.

[72]  P. Dupas,et al.  Daily Needs, Income Targets and Labor Supply: Evidence from Kenya , 2013 .

[73]  Yannick-André Breton Molar and Molecular Models of Performance for Rewarding Brain Stimulation , 2013 .

[74]  B. Skinner,et al.  The Behavior of Organisms: An Experimental Analysis , 2016 .