Difficulty Factors of Obtaining Access for Empirical Studies in Industry

Context: The difficulty (not just effort) of obtaining access for software engineering empirical studies in industry varies greatly. Supposedly, some of this variance in difficulty is particular, stemming from properties of individual contexts (the industrial partners and their work), while the rest is repeatable, related to properties of the research question and research design. Question: What are these recurring difficulty factors that arise from research question and research design? What mechanisms produce their influence? Method: We use ideation and knowledge extraction from research experience to identify potential difficulty factors, use expert discussion to understand their mechanisms, and use concept analysis to arrange them into a taxonomy. We evaluate the result by comparatively applying it to two research efforts pursued by the same research group. Results: We find six scope factors, five problematic intervention effects factors, and seven helpful intervention (side-)effects factors. Conclusion: Considering these factors systematically during the formulation of a research question and the design of a research method can help with balancing data collection difficulty with results validity and relevance.

[1]  V. Zeithaml Consumer Perceptions of Price, Quality, and Value: A Means-End Model and Synthesis of Evidence: , 1988 .

[2]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[3]  Natalia Juristo Juzgado,et al.  Topic selection in industry experiments , 2014, CESI 2014.

[4]  Lutz Prechelt,et al.  What Happens During Pair Programming , 2008, PPIG.

[5]  G. Hofstede Culture′s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations , 2001 .

[6]  Timothy C. Lethbridge,et al.  The Management of University-Industry Collaborations Involving Empirical Studies of Software Enginee , 2008, Guide to Advanced Empirical Software Engineering.

[7]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[8]  William B. Gudykunst,et al.  Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations [2nd edition]: Geert Hofstede , 2005 .

[9]  Paul Grünbacher,et al.  Success factors for empirical studies in industry-academia collaboration: A reflection , 2013, 2013 1st International Workshop on Conducting Empirical Studies in Industry (CESI).

[10]  Daniela E. Damian,et al.  Selecting Empirical Methods for Software Engineering Research , 2008, Guide to Advanced Empirical Software Engineering.

[11]  Muhammad Ali Babar,et al.  Conducting empirical studies in industry: Balancing rigor and relevance , 2013, 2013 1st International Workshop on Conducting Empirical Studies in Industry (CESI).

[12]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[13]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1992 .

[14]  Xavier Franch,et al.  Proceedings of the 2nd International Workshop on Conducting Empirical Studies in Industry , 2014, ICSE 2014.

[15]  Lutz Prechelt Agile offsharing: Using pair work to overcome nearshoring difficulties , 2013, 2013 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE).

[16]  Pablo Romero,et al.  "Talking the talk": Is intermediate-level conversation the key to the pair programming success story? , 2007, Agile 2007 (AGILE 2007).

[17]  Pamela Jordan Basics of qualitative research: Grounded theory procedures and techniques , 1994 .

[18]  Lefteris Angelis,et al.  The Success Factors Powering Industry-Academia Collaboration , 2012, IEEE Software.

[19]  Robert M. Davison,et al.  Principles of canonical action research , 2004, Inf. Syst. J..

[20]  J. Calder Survey research methods , 1998, Medical education.

[21]  Christopher Oezbek,et al.  Saros: an eclipse plug-in for distributed party programming , 2010, CHASE '10.