On the estimation of reliability of a software system using reliabilities of its components

We report an experiment to evaluate a method, known as component based reliability estimation (CBRE), for the estimation of reliability of a software system using reliabilities of its components. CBRE involves computing path reliability estimates based on the sequence of components executed for each test input. Path reliability estimates are averaged over all test runs to obtain an estimate of the system reliability. In the experiment reported, three components of a Unix utility were seeded with errors and the reliability of each component was measured. The faulty components were then introduced systematically into the utility, in various combinations, to produce several faulty versions of the utility. For each faulty version, test cases were drawn from an operational profile to measure the component-based reliability. The true reliability of the faulty version was estimated using the frequency count approach. The goodness of CBRE was assessed in terms of the accuracy and efficiency of the estimates with respect to the true reliability. Results from this experiment suggest that CBRE yields reasonably accurate results at an efficient rate. However, the accuracy and efficiency of CBRE is sensitive to the dependency among successive calls to a component.