Aspect-Oriented Approach to Testing - Experiences from a Smartphone Product Family

Abstract

The size of software running in smartphones has increased significantly during recent years. As a result the testing of such systems is inherently more difficult and expensive. Meanwhile, the quality expectations are more demanding and development life-cycles shorter. Since testing is commonly considered one of the most resource consuming activities of a modern software project, device manufacturers need to find solutions to more effectively manage the testing effort and related investments in order to succeed in a fiercely competitive environment. This thesis presents an approach to capturing system-wide, cross-cutting concerns to be modularized as manageable units for testing. Considering testing as a system-wide concern allows separating the testing concern from other concerns, thus modularizing testability issues into manageable units. In this thesis aspect-orientation is used in formulating such testability issues and in implementing testware. In the approach presented here the origin of test case definitions is revised from code and specifications to requirements and expectations. Expected behaviors and system characteristics are formulated as testing objectives and further developed as test aspects. These aspects provide representations of system-wide testing concerns, such as performance, and implement testing in a non-invasive manner, allowing system-wide testing issues to be addressed already at the unit testing level. Furthermore, visualizing the testing scenarios as sequence diagrams can be used to automatically generate the related testware in system testing, as well as in unit testing, without the need for understanding the original code. Based on the experiments the aspect-oriented approach proposes a technique to implement testware without actually touching the code. The system under test and the related testware are easier to separate, as the original implementation remains oblivious to testing in most cases. An aspect-oriented approach allows system-wide evaluation of testing concerns, but formulating the correct aspects can be dfficult. Furthermore, adopting the technique on an industrial scale requires changes to processes and additional tools.