Our new paper "FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases" is accepted at ISSTA 2022
We’re delighted to announce that a COINSE paper titled “FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases” (Gabin An and Shin Yoo) has been accepted at ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022)! This work proposes a novel measure FDG to quantify the fault diagnosability gain of individual test cases. Using the test coverage and the ongoing Fault Localisation (FL) results, FDG can precisely measure how much diagnosability gain each test can bring to the current test suite. If you’re interested in our work, please checkout the latest preprint!
FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases, An, G. and Yoo, S., Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, 14–26.
Abstract
The performance of many Fault Localisation (FL) techniques directly depends on the quality of the used test suites. Consequently, it is extremely useful to be able to precisely measure how much diagnostic power each test case can introduce when added to a test suite used for FL. Such a measure can help us not only to prioritise and select test cases to be used for FL, but also to effectively augment test suites that are too weak to be used with FL techniques. We propose FDG, a new measure of Fault Diagnosability Gain for individual test cases. The design of FDG is based on our analysis of existing metrics that are designed to prioritise test cases for better FL. Unlike other metrics, FDG exploits the ongoing FL results to emphasise the parts of the program for which more information is needed. Our evaluation of FDG with Defects4J shows that it can successfully help the augmentation of test suites for better FL. When given only a few failing test cases (2.3 test cases on average), FDG can effectively augment the given test suite by prioritising the test cases generated automatically by EvoSuite: the augmentation can improve the acc@1 and acc@10 of the FL results by 11.6x and 2.2x on average, after requiring only ten human judgements on the correctness of the assertions EvoSuite generates.