Quality Assurance Process in AI Applications
Tanskanen, Atte (2025)
2025
Tietojenkäsittelyopin maisteriohjelma - Master's Programme in Computer Science
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2025-05-27
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202505266185
https://urn.fi/URN:NBN:fi:tuni-202505266185
Tiivistelmä
With the advancement of generative AI technology, new opportunities for increased efficiency across various business domains are emerging. As with traditional software, applications incorporating large language model (LLM) components must be developed and their quality assessed. However, the non-deterministic nature of AI models presents significant challenges for testing and quality evaluation. These challenges are further exacerbated by the relative novelty of the AI development field, where standardized best practices and frameworks are still under development and not yet widely adopted. Traditional software engineering and quality assurance methodologies may offer partial solutions - some existing metrics and quality criteria can be directly applied, while others require adaptation or the creation of entirely new approaches to address the unique characteristics of AI systems.
This study investigates the software quality assurance process in the context of applications that include AI components. A case study methodology is employed, comparing insights gained during the development of the APUOPE-RE teaching assistant application to recent academic research on AI quality.
The findings of this thesis highlight the specific challenges that AI components introduce to software development and propose strategies for addressing them. Key AI-related quality aspects are identified and discussed in terms of their application during the design and testing phases of development. For effective AI-focused quality assurance, it is recommended that software development teams place greater emphasis on the design phase and data management. The design phase should involve selecting appropriate learning models, defining clear requirements, applying prompt engineering techniques, and proactively assessing legal and ethical risks. Data management should ensure that training data is of high quality, balanced, and sufficiently comprehensive, as poor, biased, or limited datasets can result in suboptimal performance of LLM-based applications.
This study investigates the software quality assurance process in the context of applications that include AI components. A case study methodology is employed, comparing insights gained during the development of the APUOPE-RE teaching assistant application to recent academic research on AI quality.
The findings of this thesis highlight the specific challenges that AI components introduce to software development and propose strategies for addressing them. Key AI-related quality aspects are identified and discussed in terms of their application during the design and testing phases of development. For effective AI-focused quality assurance, it is recommended that software development teams place greater emphasis on the design phase and data management. The design phase should involve selecting appropriate learning models, defining clear requirements, applying prompt engineering techniques, and proactively assessing legal and ethical risks. Data management should ensure that training data is of high quality, balanced, and sufficiently comprehensive, as poor, biased, or limited datasets can result in suboptimal performance of LLM-based applications.