Replicate, Replicate, Replicate

Replication is a standard part of scientific experimentation. Unfortunately, in software engineering, replication of experiments is often considered an inferior type of research, or not even research at all. In this paper we describe four different types of replication that we have been performing as part of validating the effectiveness and applicability of our software fault prediction research. We discuss replication over time, replication by using different subject systems, replication by changing the variables in prediction models, and replication by varying the modeling algorithms.