Our new paper titled “Automatically Identifying Shared Root Causes of Test Breakages in SAP HANA” has been accepted to ICSE 2022 Software Engineering in Practice (SEIP) track. This work was done by Gabin An, Juyeon Yoon, Jeongju Sohn, and Shin Yoo from COINSE in collaboration with Jingun Hong and Dongwon Hwang from SAP Labs Korea.

In this paper, we target large-scale Continuous Integration (CI) systems such as SAP HANA with a non-trivial number of test breakages. Each breakage that newly occurs from daily runs needs to be manually inspected and assigned to developers for debugging. Automatically identifying breakages with shared root causes will be able to significantly reduce the duplicated effort in the post-breakage steps. We investigate the effectiveness of multiple similarity functions (of multiple information sources and similarity measures) for identifying whether the two breakages share the same root cause. In detail, we consider multiple information sources, such as static (i.e., the code itself), historical (i.e., whether the test results have changed in a similar way in the past), as well as dynamic (i.e., whether the coverage of test cases are similar to each other). We observe the complementary relationship between different information sources.

Using all individual similarity functions as input features, we construct a classification model that predicts whether two test breakages share the same root cause or not. Our model trained on the real-world test breakage history collected from the issue tracker of SAP HANA achieves an F1 score of 0.743 on unseen breakage pairs. This empirical result supports that a classification model based on multiple test similarity functions can successfully assist the bug triage stage of a CI pipeline.

The preprint is available in this link.