Background

This tool visualises the risk value surface of any given Spectrum-Based Fault Localisation technique. The spectrum data for any structural element $s$ is usually summarised as a tuple $(e_f, n_f, e_p, n_p)$, where $e_f$ and $n_f$ are the number of failing test cases that covered or not covered $s$, and $e_p$ and $n_p$ are the number of passing test cases that covered or not covered $s$.

The intuition behind this visualisation is that, given a set of finished test results, the number of passing and failing test cases (i.e. $P$ and $F$) are fixed. This allows us to substitute $n_f$ with $F - e_f$ and $e_p$ with $P - n_p$, reducing the spectrum data tuple to a spectral coordinate, $(e_f, n_p)$. Consequently, any SBFL formula $R$ can be visualised by mapping the spectral coordinates, $(e_f, n_p)$, to the risk evaluation score, $R(e_f, n_p)$.

The visualisation also provides an intuitive short-hand for the recent proof, which showed that there is no greatest SBFL formula, i.e. the one that always outperforms all other formulas in Expense metric, regardless of the fault, program, and test suite. A faulty statement that can be localised by SBFL can have any coordinate $(e_f, n_p)$ as long as its $e_f$ equals $F$. However, without knowing the fault a priori, it is not possible for any formula to always map the coordinate of the faulty statement to the highest risk evaluation value. Hence no such formula can exist. For details of the proof, please refer to the following techreport:

How to Use

Use the control pane to enter the formula you want to visualise. Spectrum elements are denoted by "ef", "nf", "ep", and "np" ("ep" and "nf" will be automatically replaced with "(P - np)" and "(F - ef)" when you click "Graph Function"). You can also use four parameters ("a", "b", "c", and "d"), whose values you can control through the pane to see the effect in real time. The risk evaluation score is normalised to $[0, 1]$ before visualisation. There are a set of pre-defined, existing formulas for your convenience.

The 3D visualisation is rendered using Three.js and WebGL. We had the best performance with Chrome web browser.

Credit

This tool is based on the Three.js example from http://stemkoski.github.io/Three.js/.