Making Tradeoffs in Designing Scientific Experiments: A Case Study with Multi-Level Factors

Experimentation is an important way that scientists learn; i.e., the scientific method. Planning scientific experiments involves a variety of challenges, both statistical and logistical in nature. Interesting statistical questions arise in planning scientific experiments that involve assessing the tradeoffs between the number of runs performed, the selection of experiment factor levels, the ability to estimate effects of different experiment factors, and the degree to which statistical optimality criteria such as orthogonality can be achieved. The relative merits of different types of experiment designs such as fractional factorials, orthogonal arrays, and near-orthogonal arrays for achieving desirable statistical properties need to be considered while facing the realities of practical constraints. These issues are examined as they arise in the process of designing experiments for materials studies.