Should We Trust Automated Static Analysis Tools? A Comparison on the Agreement of Three Tools
Lujan, Savanna (2020)
2020
Degree Programme in Science and Engineering, BSc (Tech) - Degree Programme in Science and Engineering, BSc (Tech)
Tekniikan ja luonnontieteiden tiedekunta - Faculty of Engineering and Natural Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2020-05-14
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202004294600
https://urn.fi/URN:NBN:fi:tuni-202004294600
Tiivistelmä
Software developers use Automated Static Analysis Tools (ASATs) to detect issues in their code in the early code development stages. By investigating the code quality, these tools highlight the technical debt (TD) of the code. Currently, developers lack in-depth information regarding the current capabilities of ASATs which makes it harder to determine which tools they should use for their needs. In addition, there is a lack of information regarding what current ASATs identify and how the different ASATs agree among one another with respect to TD detection.
This study aims to bring in-depth knowledge concerning ASATs with respect to what TD items they identify as well compare and contrast how the tools differ in TD detection. This would bring more knowledge regarding the TD detection capabilities of current ASATs and in turn allow practitioners to be able to choose more precisely appropriate ASATs for their needs as well as encourage researchers to investigate other ASATs. A portion of this knowledge gap concerning current ASATs and TD detection will be filled by investigating three popular ASATs: SonarQube, Coverity Scan, and Better Code Hub.
A total of 53 Java projects from the open-source Java Qualitas Corpus set were analyzed and their results were processed to derive the scope of detectable TD items each of the tools identify as well as examine the agreement among the tools concerning common TD items. The results of the study show that ASATs vary with regards to the type of TD items they identify and hardly agree among one another with respect to TD identification. Further research is needed to examine the accuracy and precision of the tools as well as extending the study towards the TD identification capabilities of other ASATs.
This study aims to bring in-depth knowledge concerning ASATs with respect to what TD items they identify as well compare and contrast how the tools differ in TD detection. This would bring more knowledge regarding the TD detection capabilities of current ASATs and in turn allow practitioners to be able to choose more precisely appropriate ASATs for their needs as well as encourage researchers to investigate other ASATs. A portion of this knowledge gap concerning current ASATs and TD detection will be filled by investigating three popular ASATs: SonarQube, Coverity Scan, and Better Code Hub.
A total of 53 Java projects from the open-source Java Qualitas Corpus set were analyzed and their results were processed to derive the scope of detectable TD items each of the tools identify as well as examine the agreement among the tools concerning common TD items. The results of the study show that ASATs vary with regards to the type of TD items they identify and hardly agree among one another with respect to TD identification. Further research is needed to examine the accuracy and precision of the tools as well as extending the study towards the TD identification capabilities of other ASATs.
Kokoelmat
- Kandidaatintutkielmat [6531]