Analysis of Observational Data in Speech and Language Research Using Generalizability Theory

Most research in speech-language pathology relies on observational data collected by human observers or judges. The reliability and generalizability of such measurements are always important considerations. This article reviews classical methods of estimating reliability and proposes that a more powerful approach capable of estimating the dependability of behavioral measurements is available. This approach, based on generalizability theory, provides a practical framework for estimating multiple sources of measurement error in the collection of observational data. Concepts central to generalizability theory are discussed, and a hypothetical data set illustrates the usefulness of generalizability measurements in speech and language research.