A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking

Type checking is considered an important mechanism for detecting programming errors, especially interface errors. This report describes an experiment to assess the defect-detection capabilities of static, intermodule type checking. The experiment uses ANSI C and Kernighan & Ritchie (K&R) C. The relevant difference is that the ANSI C compiler checks module interfaces (i.e., the parameter lists calls to external functions), whereas K&R C does not. The experiment employs a counterbalanced design in which each of the 40 subjects, most of them CS PhD students, writes two nontrivial programs that interface with a complex library (Motif). Each subject writes one program in ANSI C and one in K&R C. The input to each compiler run is saved and manually analyzed for defects. Results indicate that delivered ANSI C programs contain significantly fewer interface defects than delivered K&R C programs. Furthermore, after subjects have gained some familiarity with the interface they are using, ANSI C programmers remove defects faster and are more productive (measured in both delivery time and functionality implemented).

[1]  Watts S. Humphrey,et al.  A discipline for software engineering , 2012, Series in software engineering.

[2]  Victor R. Basili,et al.  Software errors and complexity: an empirical investigation , 1993 .

[3]  B. A. Sheil,et al.  The Psychological Study of Programming , 1981, CSUR.

[4]  Webb Stacy,et al.  Cognitive bias in software engineering , 1995, CACM.

[5]  Walter F. Tichy,et al.  A Controlled Experiment Measuring the Effect of Procedure Argument Type Checking on Programmer Productivity , 1996 .

[6]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[7]  Curtis R. Cook,et al.  An analysis of the on-line debugging process , 1987 .

[8]  Elliot Soloway,et al.  Empirical Studies of Programmers: Second Workshop , 1991 .

[9]  James C. Spohrer,et al.  Empirical Studies of Programmers: Fifth Workshop , 1993 .

[10]  Elliot Soloway,et al.  Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers , 1986 .

[11]  Paul Hudak,et al.  An Experiment in Software Prototyping Productivity , 1994 .

[12]  M. Eisenstadt.  Tales of Debugging from The Front Lines , 1993 .

[13]  Alireza Ebrahimi,et al.  Novice programmer errors: language constructs and plan composition , 1994, Int. J. Hum. Comput. Stud..

[14]  John D. Gannon.  An experimental evaluation of data type conventions , 1977, CACM.

[15]  Claes Wohlin,et al.  Certification of Software Components , 1994, IEEE Trans. Software Eng..

[16]  Niklaus Wirth,et al.  Gedanken zur Software-Exlosion , 1994, Inform. Spektrum.

[17]  Eliot Soloway,et al.  Workshop on empirical studies of programmers , 1986, SOEN.

[18]  Robert B. Grady,et al.  Practical results from measuring software quality , 1993, CACM.

[19]  Elliot Soloway,et al.  Analyzing the high frequency bugs in novice programs , 1986 .

[20]  Victor R. Basili,et al.  Software errors and complexity: an empirical investigation0 , 1984, CACM.

[21]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[22]  Yi Pu-Yong,et al.  A Psychological Study , 1971 .