A Batch, Off-Policy, Actor-Critic Algorithm for Optimizing the Average Reward

We develop an off-policy actor-critic algorithm for learning an optimal policy from a training set composed of data from multiple individuals. This algorithm is developed with a view towards its use in mobile health.

[1]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[2]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  K. Patrick,et al.  A Text Message–Based Intervention for Weight Loss: Randomized Controlled Trial , 2009, Journal of medical Internet research.

[6]  Dimitri P. Bertsekas,et al.  Convergence Results for Some Temporal Difference Methods Based on Least Squares , 2009, IEEE Transactions on Automatic Control.

[7]  L. Epstein,et al.  Variety influences habituation of motivated behavior for food and energy intake in children. , 2009, The American journal of clinical nutrition.

[8]  C. Depp,et al.  Mobile Interventions for Severe Mental Illness: Design and Preliminary Data From Three Approaches , 2010, The Journal of nervous and mental disease.

[9]  Martha White,et al.  Linear Off-Policy Actor-Critic , 2012, ICML.

[10]  C. Kennedy,et al.  Active Assistance Technology for Health-Related Behavior Change: An Interdisciplinary Review , 2012, Journal of medical Internet research.

[11]  Susan M. Kaiser,et al.  Mobile Technologies Among People with Serious Mental Illness: Opportunities for Future Services , 2013, Administration and Policy in Mental Health and Mental Health Services Research.

[12]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  E. Granholm,et al.  Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. , 2012, Schizophrenia bulletin.

[14]  S. Bauer,et al.  Technology-enhanced maintenance of treatment gains in eating disorders: efficacy of an intervention delivered via text messaging. , 2012, Journal of consulting and clinical psychology.

[15]  J. Ainsworth,et al.  Intelligent real-time therapy: Harnessing the power of machine learning to optimise the delivery of momentary cognitive–behavioural interventions , 2012, Journal of mental health.

[16]  Audie A Atienza,et al.  Mobile health technology evaluation: the mHealth evidence workshop. , 2013, American journal of preventive medicine.

[17]  Erika B. Litvin,et al.  Computer and mobile technology-based interventions for substance use disorders: an organizing framework. , 2013, Addictive behaviors.

[18]  Stavroula G. Mougiakakou,et al.  An Actor-Critic based controller for glucose regulation in type 1 diabetes , 2013, Comput. Methods Programs Biomed..

[19]  O. Kristjansdottir,et al.  A Smartphone-Based Intervention With Diaries and Therapist-Feedback to Reduce Catastrophizing and Increase Functioning in Women With Chronic Widespread Pain: Randomized Controlled Trial , 2013, Journal of medical Internet research.

[20]  David Silver,et al.  Concurrent Reinforcement Learning from Customer Interactions , 2013, ICML.

[21]  Jylana L. Sheats,et al.  Harnessing Different Motivational Frames via Mobile Phones to Promote Daily Physical Activity and Reduce Sedentary Behavior in Aging Adults , 2013, PloS one.

[22]  Wendy Nilsen,et al.  Dynamic Models of Behavior for Just-in-Time Adaptive Interventions , 2014, IEEE Pervasive Computing.

[23]  K. Witkiewitz,et al.  Development and evaluation of a mobile intervention for heavy drinking and smoking among college students. , 2014, Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors.

[24]  Cynthia Breazeal,et al.  Learning to Maintain Engagement: No One Leaves a Sad DragonBot , 2014, AAAI Fall Symposia.

[25]  Dhavan V. Shah,et al.  A smartphone application to support recovery from alcoholism: a randomized clinical trial. , 2014, JAMA psychiatry.