Automated model-based Android GUI testing using multi-level GUI comparison criteria

Automated Graphical User Interface (GUI) testing is one of the most widely used techniques to detect faults in mobile applications (apps) and to test functionality and usability. GUI testing exercises behaviors of an application under test (AUT) by executing events on GUIs and checking whether the app behaves correctly. In particular, because Android leads in market share of mobile OS platforms, a lot of research on automated Android GUI testing techniques has been performed. Among various techniques, we focus on model-based Android GUI testing that utilizes a GUI model for systematic test generation and effective debugging support. Since test inputs are generated based on the underlying model, accurate GUI modeling of an AUT is the most crucial factor in order to generate effective test inputs. However, most modern Android apps contain a number of dynamically constructed GUIs that make accurate behavior modeling more challenging. To address this problem, we propose a set of multi-level GUI Comparison Criteria (GUICC) that provides the selection of multiple abstraction levels for GUI model generation. By using multilevel GUICC, we conducted empirical experiments to identify the influence of GUICC on testing effectiveness. Results show that our approach, which performs model-based testing with multi-level GUICC, achieved higher effectiveness than activity-based GUI model generation. We also found that multi-level GUICC can alleviate the inherent state explosion problems of existing a single-level GUICC for behavior modeling of real-world Android apps by flexibly manipulating GUICC.