The Forum for Negative Results (FNR)Guest Editorial

In September 1997, J.UCS published an article titled "Why we Need an Explicit Forum for Negative Results" [Prechelt, 1997]. It argued that when a plausible approach for solving a computer science or software engineering problem had failed to work out, it was silly for the scientific system not to publish the attempt iff a useful insight had been gained along the way nevertheless. Due to the strong bias of essentially all Computer Science publication venues towards "successful" research results, it was thus required to call for such negative results explicitly in order to avoid that those results would either be misleadingly disguised as successes or disappear in some closet. The article declared that J.UCS had thus agreed to create the "Forum for Negative Results (FNR)" as a permanent special section of J.UCS. To be submitted to FNR, an article would explain an idea, argue why it was plausible to lead to success, describe its implementation and evaluation, describe how the result failed to meet the expectation, and then (and this would be the article's main research contribution) explain why this failure occurred and what had been wrong with the expectation. The submission would be subtitled "A contribution to the Forum for Negative Results" and would then be subjected to some additional review criteria besides J.UCS's usual ones; see the FNR homepage [FNRa]. The present J.UCS issue publishes an FNR paper: "Points-to Analysis: A Fine-Grained Evaluation. A contribution to the Forum for Negative Results" by Lundberg and Löwe. It pertains to static program analysis and investigates, for various styles of analysis of object references, the expectation that taking more than one level of call history into account will lead to improvements in analysis precision, at least when the precision metric distinguishes different instances of a source-level object. This would mean the quality of program analysis can be improved by doing such (expensive) k-level analyses. The article finds, however, that this is not the case. Its contribution is the explanation: An even much finer (and practically irrelevant) level of detail considered by the metric is needed before the differences become visible-so the improvement exists in principle, but is not relevant for practical purposes. This negative result holds a rather positive and useful message: There is no need to perform the expensive k-level-deep analysis. Overall, this is an exemplary FNR contribution. The article happens to be the very first contribution ever published by …