Designing Experimental Research

Abstract This chapter is about experimental research design, or the way an experiment is organized. One must critically consider methods for reducing or eliminating threats to internal and external validity. “Internal validity” refers to a concern about whether an experimental treatment did, in fact, make a difference in the outcome of an experiment. “External validity” refers to generalizability, whether the results of an experiment can be applied broadly or apply only to a specific population and the situations circumscribed by the experimental procedures. Threats to external validity are more difficult to resolve with experimental design features. One approach is to conduct a delimited experiment with reasonable controls of the threats to internal validity that represents one population, one treatment approach, one compelling outcome, and one setting, and then—should that experiment yield promising results—conduct more research to systematically expand the generalizability of the findings by varying the population, treatment, outcome, or setting one step at a time. Pre-experimental designs lack controls over threats to internal validity and should rarely be used. True experimental designs are the strongest designs since they are the most resilient to threats of internal validity. Quasi-experimental designs are not quite as strong as the true experimental designs, but they do offer many useful possibilities.