Understanding User-Bot Interactions for Small-Scale Automation in Open-Source Development

Small-scale automation tools, or "bots," have been widely deployed in open-source software development to support manual project maintenance tasks. Though interactions between these bots and human developers can have significant effects on user experience, previous research has instead mostly focused on project outcomes. We reviewed existing small-scale bots in wide use on GitHub. After an in-depth qualitative and quantitative evaluation, we compiled several important design principles for human-bot interaction in this context. Following the requirements, we further propose a workflow to support bot developers.

[1]  Marco Aurélio Gerosa,et al.  The Power of Bots , 2018, Proc. ACM Hum. Comput. Interact..

[2]  Meng Xia,et al.  Exploring how software developers work with mention bot in GitHub , 2018, CCF Transactions on Pervasive Computing and Interaction.

[3]  Christoph Matthies,et al.  An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives , 2019, 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE).

[4]  Margaret-Anne D. Storey,et al.  How Software Developers Mitigate Collaboration Friction with Chatbots , 2017, ArXiv.

[5]  Boyang Li,et al.  Reducing Interruptions at Work: A Large-Scale Field Study of FlowLight , 2017, CHI.

[6]  Amy J. Ko,et al.  How power users help and hinder open bug reporting , 2010, CHI.

[7]  Simon Urli,et al.  How to Design a Program Repair Bot? Insights from the Repairnator Project , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP).

[8]  Zhenchang Xing,et al.  AnswerBot: Automated generation of answer summary to developers' technical questions , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[9]  Riccardo Scandariato,et al.  Current and Future Bots in Software Development , 2019, 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE).

[10]  Marco Aurélio Gerosa,et al.  Should I Stale or Should I Close? An Analysis of a Bot That Closes Abandoned Issues and Pull Requests , 2019, 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE).

[11]  Margaret-Anne D. Storey,et al.  Defining and Classifying Software Bots: A Faceted Taxonomy , 2019, 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE).

[12]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[13]  Audris Mockus,et al.  What make long term contributors: Willingness and opportunity in OSS community , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[14]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[15]  Serge Demeyer,et al.  Among the Machines: Human-Bot Interaction on Social Q&A Websites , 2016, CHI Extended Abstracts.

[16]  Jinghui Cheng,et al.  How Do the Open Source Communities Address Usability and UX Issues?: An Exploratory Study , 2018, CHI Extended Abstracts.