Classical and Bayesian Uncertainty Intervals for the Reliability of Multidimensional Scales

The reliability of a multidimensional test instrument is commonly estimated using coefficients 𝜔𝑡 (total) and 𝜔h (hierarchical). However, point estimates for the coefficients are rarely accompanied by uncertainty estimates. In this study, we compare bootstrap and normal-theory confidence intervals. In addition, we develop Bayesian versions of coefficients 𝜔𝑡 and 𝜔h by sampling from a second-order factor model. Results from a comprehensive simulation study show that the studied confidence intervals performed well when the sample size was sufficiently large (𝑁 ≥ 500). The Bayesian estimates performed well across most studied conditions. When the sample size was small and the reliability low, only the bias-corrected and accelerated bootstrap confidence interval approached a satisfactory coverage among all intervals. This study guides on 𝜔𝑡 and 𝜔h confidence intervals and introduces 𝜔𝑡 and 𝜔h credible intervals that are easy to use and come with the benefits of Bayesian parameter estimation.