Introduction into Item Response Theory and Computer Adaptive Testing

Item banks and Computerized Adaptive Testing (CAT) have the potential to greatly improve the assessment of health outcomes. CATs promise to provide more precise and less burdensome instruments, and IRT item banks will help comparing established tools to move away from an instrument defined measurement towards greater standardization of Patient- Reported Outcome (PRO) measures. This presentation describes the features of item banks and CAT and discusses how to develop item banks. In CAT, a computer selects the items from an item bank that are most relevant for and informative about the particular respondent; thus optimizing test relevance and precision. Item response theory (IRT) provides the foundation for selecting the items that are most informative for the particular respondent and for scoring responses on a common metric. The development of an item bank is a multi-stage process that requires a clear definition of the construct to be measured, good items, a careful psychometric analysis, and a clear specification of the final CAT. The psychometric analysis needs to evaluate the assumptions of the IRT model such as unidimensionality and local independence; that the items function the same way in different subgroups of the population; and that there is an adequate fit between the data and the chosen item response models. Although medical research can draw upon expertise for educational testing the development in the medical field encounters unique opportunities and challenges.