We propose an automated approach for design defect detection. It exploits an algorithm that automatically finds rules for the detection of possible design defects, thus relieving the designer from doing so manually. Our algorithm derives rules in the form of metric/threshold combinations, from known instances of design defects (defect examples). Due to the large number of possible combinations, we use a music-inspired heuristic that finds the best harmony when combining metrics. We evaluated our approach on finding potential defects in three open-source systems (Xerces-J, Quick UML and Gantt). For all of them, we found more than 80% of known defects, a better result when compared to a state-of-the-art approach, where the detection rules are manually specified.

A critical item of a bug report is the so-called "severity", i.e., the impact the bug has on the successful execution of the software system. Consequently, tool support for the person reporting the bug in the form of a recommender or verification system is desirable. In previous work we made a first step towards such a tool: we demonstrated that text mining can predict the severity of a given bug report with a reasonable accuracy given a training set of sufficient size. In this paper we report on a follow-up study where we compare four well-known text mining algorithms (namely, Naive Bayes, Naive Bayes Multinomial, K-Nearest Neighbor and Support Vector Machines) with respect to accuracy and training set size. We discovered that for the cases under investigation (two open source systems: Eclipse and GNOME) Naive Bayes Multinomial performs superior compared to the other proposed algorithms.

Testing is an important and cost-intensive part of the software development life-cycle. Defect prediction models try to identify error-prone components, so that these can be tested earlier or more in-depth, and thus improve the cost-effectiveness during testing. Such models have been researched extensively, but whether and when they are applicable in practice is still debated. The applicability depends on many factors, and we argue that it cannot be analyzed without a specific scenario in mind. In this paper, we therefore present an analysis of the cost-effectiveness for one case study based on detailed data from the testing phase of a software system from the avionic domain. An analysis of all defects found in the testing phase reveals that only less than half of them could actually be identified by a defect prediction model based on static code metrics. We investigate the predictive performance of different prediction models built using random forest. The small ratio of defective instances, however, results in relatively poor performance. Our analysis of the cost-effectiveness then shows that the prediction model is not able to outperform simple models, and hence, does not offer any advantage in practice.