This page is supplementary to the paper entitled "Evolving Human Competitive Spectra-Based Fault Localisation Techniques", which has been presented at the 4th International Symposium on Search-Based Software Engineering. This page contains additional results that could not be reported in the paper due to page limits. The data used in the paper is also made available here to enable replication and further research on the topic.

Background

The paper presents the first attempt to evolve human-competitive Spectra-Based Fault Localisation (SBFL). SBFL relies on risk evaluation formulas to predict the relative risk of each statement in System Under Test containing the fault. The widely studied form of risk evaluation formulas is a formula of four program spectra elements: \(e_p\) (number of passing tests that cover the statement), \(e_f\) (number of failing tests that cover the statement), \(n_p\) (number of failing tests that do not cover the statement), and \(n_f\) (number of failing tests that do not cover the statement). All existing formulas were designed by human tester/software engineer. Our Genetic Programming approach can evolve risk evaluation formulas that can significantly outperform many of the widely studied human-designed formulas. For details, please refer to the original paper.

Full Results

This section contains the full comparision of each individual GP-evolved formula against all human-designed metrics studied in the paper. Please click the ID of the GP experiment to see the full details.

Experimental Data

The program spectra data and the fault location data are available from here.