Performance of Model-Based Testing for an Android Application
Vaattovaara, Matti (2019)
2019
Tietotekniikan DI-ohjelma - Degree Programme in Information Technology
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2019-11-07
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-201910274110
https://urn.fi/URN:NBN:fi:tuni-201910274110
Tiivistelmä
The supply of Android applications is large, and the market is highly competitive. Bugs and performance issues in an application are reasons for a user to switch to a more stable competitor. Automated graphical user interface (GUI) testing of mobile applications is one of the ways to improve their quality. There are several techniques to implement GUI testing, each of which has its strengths and weaknesses.
In this study, a model-based GUI test automation was implemented for an Android application. The performance of model-based test automation was evaluated based on fault detection and coverage. This was done by using the test automation on an Android application under development over three months. As a point of reference, the same versions of the application were tested by a fairly sophisticated manually scripted UI test automation over the same period. To mitigate the effect of cost, the amount of effort put to implementing each of the solutions was evened out. In addition to fault detection and coverage, the characteristics of the implementations were compared in terms of applicability and cost-effectiveness over the development cycle.
The results of the study were in line with the general theory of model-based testing (MBT). It was concluded that model-based testing achieved better use case coverage and fault detection than the manually scripted test automation, provided that the effort was big enough for the advantages of model-based testing paradigm to manifest. Model-based testing provided the ability to run noticeably longer test cases with hardly any extra effort, thus allowing testing for memory leak issues. Another model-based advantage was the ability to focus or avoid testing on specified areas of the system under test (SUT) using strategies. On the other hand, the manually scripted test automation solution was easier to implement, faster to start yielding benefits and more flexible in terms of platform and device.
In this study, a model-based GUI test automation was implemented for an Android application. The performance of model-based test automation was evaluated based on fault detection and coverage. This was done by using the test automation on an Android application under development over three months. As a point of reference, the same versions of the application were tested by a fairly sophisticated manually scripted UI test automation over the same period. To mitigate the effect of cost, the amount of effort put to implementing each of the solutions was evened out. In addition to fault detection and coverage, the characteristics of the implementations were compared in terms of applicability and cost-effectiveness over the development cycle.
The results of the study were in line with the general theory of model-based testing (MBT). It was concluded that model-based testing achieved better use case coverage and fault detection than the manually scripted test automation, provided that the effort was big enough for the advantages of model-based testing paradigm to manifest. Model-based testing provided the ability to run noticeably longer test cases with hardly any extra effort, thus allowing testing for memory leak issues. Another model-based advantage was the ability to focus or avoid testing on specified areas of the system under test (SUT) using strategies. On the other hand, the manually scripted test automation solution was easier to implement, faster to start yielding benefits and more flexible in terms of platform and device.