Four presumed gaps in the software engineering research community's knowledge

Background: The state of the art in software engineering consists of a myriad of contributions and the gaps between them; it is difficult to characterize. Questions: In order to help understanding the state of the art, can we identify gaps in our knowledge that are at a very general, widely relevant level? Which research directions do these gaps suggest? Method: 54 expert interviews with senior members of the ICSE community, evaluated qualitatively using elements of Grounded Theory Methodology. Results: Our understanding of complexity, of good-enoughness, and of developers' strengths is underdeveloped. Some other relevant factors' relevance is apparently not clear. Software engineering is not yet an evidence-based discipline. Conclusion: More software engineering research should concern itself with emergence phenomena, with how engineering tradeoffs are made, with the assumptions underlying research works, and with creating certain taxonomies. Such work would also allow software engineering to become more evidence-based.

[1]  Premkumar T. Devanbu,et al.  An Investigation into Coupling Measures for C++ , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[2]  T.C. Lethbridge,et al.  Guide to the Software Engineering Body of Knowledge (SWEBOK) and the Software Engineering Education Knowledge (SEEK) - a preliminary mapping , 2001, 10th International Workshop on Software Technology and Engineering Practice.

[3]  Aurora Ramírez,et al.  A Systematic Review of Interaction in Search-Based Software Engineering , 2019, IEEE Transactions on Software Engineering.

[4]  Victor R. Basili,et al.  An Empirical Study of Communication in Code Inspections , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[5]  Yanzhao Wu,et al.  CCAligner: A Token Based Large-Gap Clone Detector , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[6]  Tracy Hall,et al.  Investigating the Role of Trust in Agile Methods Using a Light Weight Systematic Literature Review , 2008, XP.

[7]  Abdul Azim Abdul Ghani,et al.  Unit Testing Approaches for BPEL: A Systematic Review , 2009, 2009 16th Asia-Pacific Software Engineering Conference.

[8]  Hajimu Iida,et al.  "Was My Contribution Fairly Reviewed?" A Framework to Study the Perception of Fairness in Modern Code Reviews , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[9]  Pascale Le Gall,et al.  A Theory of Probabilistic Functional Testing , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[10]  André L. M. Santos,et al.  Six years of systematic literature reviews in software engineering: An updated tertiary study , 2011, Inf. Softw. Technol..

[11]  Tsun Chow,et al.  A survey study of critical success factors in agile software projects , 2008, J. Syst. Softw..

[12]  Mary Shaw Everyday Dependability for Everyday Needs , 2002 .

[13]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..

[14]  A. Strauss,et al.  CHAPTER 20 Constructionism and the Grounded Theory Method , 2007 .

[15]  Diomidis Spinellis,et al.  On the Dichotomy of Debugging Behavior Among Programmers , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[16]  Sylvain Frey,et al.  The Good, the Bad and the Ugly: A Study of Security Decisions in a Cyber-Physical Systems Game , 2018, IEEE Transactions on Software Engineering.

[17]  Jean-Pierre Jacquot,et al.  Early Specification of User-Interfaces: Toward a Formal Approach , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[18]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[19]  Alvin Cheung,et al.  How not to Structure Your Database-Backed Web Applications: A Study of Performance Bugs in the Wild , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[20]  Gregor Snelting,et al.  Assessing Modular Structure of Legacy Code Based on Mathematical Concept Analysis , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[21]  Mark Harman,et al.  Genetic Improvement of Software: A Comprehensive Survey , 2018, IEEE Transactions on Evolutionary Computation.

[22]  Sarfraz Khurshid,et al.  Towards Practical Program Repair with On-demand Candidate Generation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[23]  Tony Gorschek,et al.  Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review , 2012, IEEE Transactions on Software Engineering.

[24]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A tertiary study , 2010, Inf. Softw. Technol..

[25]  C. A. R. Hoare,et al.  How Did Software Get So Reliable Without Proof? , 1996, FME.

[26]  Gabriele Taentzer,et al.  Multi-granular Conflict and Dependency Analysis in Software Engineering Based on Graph Transformation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[27]  Shinji Kusumoto,et al.  A New Software Project Simulator Based on Generalized Stochastic Petri-net , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[28]  Diane L. Gibson,et al.  Demonstrating the Impact and Benefits of CMMI ® : An Update and Preliminary Results , 2003 .

[29]  Salma Imtiaz,et al.  A tertiary study: experiences of conducting systematic literature reviews in software engineering , 2013, EASE '13.

[30]  Marco Scavuzzo,et al.  Experiences and challenges in building a data intensive system for data migration , 2017, Empirical Software Engineering.

[31]  Jason Williams Integrating Technical Debt Management and Software Quality Management Processes: A Framework and Field Test , 2019 .

[32]  Tore Dybå,et al.  The effectiveness of pair programming: A meta-analysis , 2009, Inf. Softw. Technol..

[33]  Premkumar T. Devanbu,et al.  A Survey of Machine Learning for Big Code and Naturalness , 2017, ACM Comput. Surv..

[34]  June M. Verner,et al.  The Effect of Department Size on Developer Attitudes to Prototyping , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[35]  Yang Liu,et al.  From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[36]  Paul Ralph,et al.  Consensus in Software Engineering: A Cognitive Mapping Study , 2018, ArXiv.

[37]  Tim Menzies,et al.  The Strangest Thing About Software , 2007, Computer.

[38]  Nancy G. Leveson,et al.  Software Deviation Analysis , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[39]  Eran Toch,et al.  Privacy by designers: software developers’ privacy mindset , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[40]  Ming Wen,et al.  Context-Aware Patch Generation for Better Automated Program Repair , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[41]  Albert Endres,et al.  A handbook of software and systems engineering - empirical observations, laws and theories , 2003, The Fraunhofer IESE series on software engineering.

[42]  Terry Cooke-Davies,et al.  The “real” success factors on projects , 2002 .

[43]  Mehrdad Sabetzadeh,et al.  Software Engineering Research and Industry: A Symbiotic Relationship to Foster Impact , 2018, IEEE Software.

[44]  Sarah J. Tracy Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research , 2010 .

[45]  Claes Wohlin,et al.  Agile Practices in Global Software Engineering - A Systematic Map , 2010, 2010 5th IEEE International Conference on Global Software Engineering.

[46]  Thomas Leich,et al.  Do You Remember This Source Code? , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[47]  Paul G. Sorenson,et al.  Hooking into Object-Oriented Application Frameworks , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[48]  A Straw,et al.  Guide to the Software Engineering Body of Knowledge , 1998 .

[49]  Betty H. C. Cheng,et al.  Formalizing and Integrating the Dynamic Model within OMT , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[50]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..

[51]  Emerson R. Murphy-Hill,et al.  When Not to Comment: Questions and Tradeoffs with API Documentation for C++ Projects , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[52]  Michael R. Lyu,et al.  Online App Review Analysis for Identifying Emerging Issues , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[53]  Nenad Medvidovic,et al.  Leveraging Program Analysis to Reduce User-Perceived Latency in Mobile Applications , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[54]  Kweku Ewusi-Mensah,et al.  Software development failures : anatomy of abandoned projects , 2003 .

[55]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[56]  Helen Sharp,et al.  Motivation in Software Engineering: A systematic literature review , 2008, Inf. Softw. Technol..

[57]  K. Wiesner,et al.  What is a complex system? , 2012, European Journal for Philosophy of Science.

[58]  Meiyappan Nagappan,et al.  Roles and Impacts of Hands-on Software Architects in Five Industrial Case Studies , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[59]  Jun Sun,et al.  Towards Optimal Concolic Testing , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[60]  Eric W. T. Ngai,et al.  Examining the critical success factors in the adoption of enterprise resource planning , 2008, Comput. Ind..

[61]  Xin Huang,et al.  Synthesizing Qualitative Research in Software Engineering: A Critical Review , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[62]  E. Brink,et al.  Constructing grounded theory : A practical guide through qualitative analysis , 2006 .

[63]  B. Glaser Theoretical Sensitivity: Advances in the Methodology of Grounded Theory , 1978 .

[64]  Daniela Cruzes,et al.  Research synthesis in software engineering: A tertiary study , 2011, Inf. Softw. Technol..

[65]  Geir Kjetil Hanssen,et al.  Signs of Agile Trends in Global Software Engineering Research: A Tertiary Study , 2011, 2011 IEEE Sixth International Conference on Global Software Engineering Workshop.

[66]  Nancy G. Leveson,et al.  High-pressure steam engines and computer software , 1992, Computer.

[67]  Meir M. Lehman,et al.  On understanding laws, evolution, and conservation in the large-program life cycle , 1984, J. Syst. Softw..