Evaluating security tools in the context of DevSecOps
Adhikari, Nikesh Bahadur (2024)
2024
Master's Programme in Computing Sciences
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2024-04-26
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202404153576
https://urn.fi/URN:NBN:fi:tuni-202404153576
Tiivistelmä
DevSecOps (Development, Security, and Operations) paradigm is simply an expansion of the DevOps (Development, and Operations), which intersects all the development, security, and operation activities under the same roof to develop a secure product continuously in an iterative approach. Recently, DevSecOps has started becoming a popular approach for most organizations. However, identifying the right security tools is always a concern in DevSecOps.
The primary objective behind this thesis is to determine the strengths and limitations based on general benchmarking and performance metrics of both Security Application Security Testing (SAST) and Infrastructure as Code (IaC) tools. We aspired to assist stakeholders interested in implementing the DevSecOps paradigm in Azure pipelines with a static analysis as a starting point. In addition, an attempt is made to map the OWASP Top 10 vulnerabilities with various security activities that detect them in DevSecOps.
We performed two case studies on two benchmarks in an Azure cloud environment, i.e., OWASP Juice Shop for SAST and Terragoat for IaC tools. Semgrep, Sonarcloud, and Snyk were selected as SAST tools, whereas Checkov, Snyk IaC, and Tfsec were chosen as IaC tools. Each tool has its Azure pipeline setup for scan which was executed independently to collect the data of the scan results. The collected data was analyzed for performance metrics, while general metrics were collected from their official documentation.
The outcomes of general and performance metrics reveal that each tool has its strengths and limitations, and the selection of security tools can be done according to one’s requirements. For instance, Semgrep stands out to be better in performance metrics in SAST analysis, while Snyk is the fastest in performing scans and SonarCloud has the largest programming language support for static analysis. Similarly, Tfsec is the easiest to get started for IaC scans. Snyk IaC has also a good number of updated Azure policies while Checkov is prominent in terms of performance due to its largest datasets of known Azure policies. On the other hand, OWASP's Top 10 vulnerabilities besides “A09:2021-Security Logging and Monitoring Failures” are detected by the SAST tools.
The primary objective behind this thesis is to determine the strengths and limitations based on general benchmarking and performance metrics of both Security Application Security Testing (SAST) and Infrastructure as Code (IaC) tools. We aspired to assist stakeholders interested in implementing the DevSecOps paradigm in Azure pipelines with a static analysis as a starting point. In addition, an attempt is made to map the OWASP Top 10 vulnerabilities with various security activities that detect them in DevSecOps.
We performed two case studies on two benchmarks in an Azure cloud environment, i.e., OWASP Juice Shop for SAST and Terragoat for IaC tools. Semgrep, Sonarcloud, and Snyk were selected as SAST tools, whereas Checkov, Snyk IaC, and Tfsec were chosen as IaC tools. Each tool has its Azure pipeline setup for scan which was executed independently to collect the data of the scan results. The collected data was analyzed for performance metrics, while general metrics were collected from their official documentation.
The outcomes of general and performance metrics reveal that each tool has its strengths and limitations, and the selection of security tools can be done according to one’s requirements. For instance, Semgrep stands out to be better in performance metrics in SAST analysis, while Snyk is the fastest in performing scans and SonarCloud has the largest programming language support for static analysis. Similarly, Tfsec is the easiest to get started for IaC scans. Snyk IaC has also a good number of updated Azure policies while Checkov is prominent in terms of performance due to its largest datasets of known Azure policies. On the other hand, OWASP's Top 10 vulnerabilities besides “A09:2021-Security Logging and Monitoring Failures” are detected by the SAST tools.