A Conceptual Framework for the Comparison of Fully Automated GUI Testing Techniques

Abstract

Fully automated GUI testing techniques play an important role in the modern software development life cycles. These techniques are implemented by algorithms that automatically traverse the GUI by interacting with it, like robots discovering unexplored spaces. These algorithms are able to define and run test cases on the fly, while the application is in execution. Testing adequacy, performance or costs of such techniques may differ on the basis of different factors. In this paper we will propose an approach for comparing fully automated GUI testing techniques in a systematic manner. The approach is based on a generalized parametric algorithm that abstracts the key aspects of these techniques and provides a conceptual framework that can be used to define and compare different testing approaches. To validate the framework, we exploit it to compare the testing adequacy and the GUI models inferred by 9 fully automated testing techniques obtained by varying the configuration of the algorithm. The experiment is performed on a real Android application.