Testing Serverless Applications: A systematic Review of Frameworks, Tools & Best Practices

Abstract

Testing serverless applications presents unique challenges due to the distributed and event-driven nature of these architectures. This thesis aims to address these challenges by conducting a systematic literature review to explore testing techniques, patterns, anti-patterns, and tools specific to serverless applications. The research questions focus on identifying the available testing techniques, evaluating testing tools, and understanding the challenges encountered in testing serverless applications. The findings indicate that there are many different ways to test serverless applications. These include unit testing, integration testing, performance testing, and monitoring & debugging. The techniques provide comprehensive coverage, improve reliability and stability, enable faster feedback cycles, mitigate risks, and enhance system resilience. However, they come with certain restrictions, such as increased complexity, limited scope, test environment restrictions, and the need for specialized knowledge. The evaluation of testing tools reveals a diverse landscape of options, including performance testing, infrastructure and deployment, monitoring and observability, distributed tracing and debugging, language-specific testing, test case generation, cloud service-specific testing, and specialized graph modeling tools. Each tool has advantages and limitations, and developers must evaluate their specific demands and limits while selecting the proper tools. Based on the research, several best practices are synthesized for testing serverless applications. Emphasizing performance testing, automating testing processes, considering cross-platform compatibility, validating event flow and interactions, and adopting distributed tracing and debugging techniques are identified as crucial practices for ensuring the quality and reliability of serverless applications. The thesis acknowledges its limitations, such as the exclusion of gray literature, the lack of practical implementation, and the potential oversight of relevant studies. Future research is recommended to incorporate gray literature, conduct practical experiments, and investigate specific application scenarios to enhance the understanding and applicability of testing techniques for serverless applications. Overall, this thesis provides valuable insights into the landscape of testing techniques, tools, challenges, and best practices for serverless applications. The findings contribute to the existing body of knowledge and offer guidance to practitioners and researchers in improving testing practices and ensuring the quality and reliability of serverless applications.