Automated GUI Testing in Industry

Every change to an application should be followed by a regression test if we want to assure that the application keeps functioning in conformance with its requirements. Executing such a test manually can be a time consuming and boring task. Time pressure often causes the regression test to be skipped, or only partially executed. Executing these regression tests automatically can be a solution to this problem. However, large costs, especially for maintenance, are involved in automating a big regression test. Also, many potential pitfalls can cause test automation projects to fail. A Dutch health insurance company called DSW has tried to incorporate automated GUI testing in their test process before. A testing tool called WinRunner was used to automate the regression test of a mainframe application called ISIS. We identified the causes for failure of this project to be high maintenance costs due to GUI object recognition problems, ignoring possibilities for reusing overlapping parts in test cases, and using hard-coded values in test scripts. These findings alongside with the pitfalls and guidelines identified by other researchers were used to avoid project failure and create a maintainable automated GUI testing solution. This automated GUI testing solution is called DARTH VADER. Key points in the design of DARTH VADER are separation of test case data and test case logic, an easily maintainable GUI mapping, the use of a tool to maintain the test cases, and restoring a fixed set of test data prior to each run. We concluded that the improved bug detection and work satisfaction, and the fact that no changes to the application under test are taken into production without being tested, outweighed the costs of developing and maintaining DARTH VADER.