A Statistical Analysis of Operational Profile Driven Testing

The amount of faults in software products is constantly increasing and it is not feasible to remove all faults. Within many organizations, operational profiles are used to focus the test activities. Prior art has claimed that when reliability growth tests are allocated using operational profiles, the reliability level will be the maximum practically achievable, but this claim has never been proven nor rejected. In this paper a statistical model is introduced that captures the relation between faults and failures. Using the model the effects of using operational profiles for reliability growth testing is analyzed. The analysis shows that the benefits of using operational profiles depend on a number of factors such as the number of faults and the failure probability. The results show that using operational profiles for reliability growth testing is beneficial, however not when a very high reliability is obtained, due to the effect of over testing.