A Range-Null Hypothesis Approach for Testing DIF under the Rasch Model

A primary concern with testing differential item functioning (DIF) using a traditional point-null hypothesis is that a statistically significant result does not imply that the magnitude of DIF is of practical interest. Similarly, for a given sample size, a non-significant result does not allow the researcher to conclude the item is free of DIF. To address these weaknesses, two types of range-null hypotheses utilizing Lord's χ2 DIF statistic were presented. The first type tests a null hypothesis whose rejection implies the item exhibits a meaningful magnitude of DIF, while the second type tests a null hypothesis whose rejection implies the item is effectively free of DIF. A simulation study was performed to evaluate the empirical Type I error rate and power of both types of range-null hypothesis tests under two crossed factors: test length (20 and 60 items) and sample size per group (2500, 5000, 10,000, and 20,000 examinees). The proposed statistic controlled the Type I error rates over all conditions and demonstrated acceptable power for sample size conditions of 5000 and larger. The implications of using the range-null hypothesis approach in practice are discussed.