A Q3-Based Permutation Test for Assessing Local Independence.

Item response theory (IRT) is a powerful statistical methodology used in the analysis of psychological and educational assessments. IRT rests on three fundamental assumptions about the data, including local independence, which means that after accounting for the latent trait(s) being measured, the item responses are independent of one another. Traditionally, this assumption is assessed using Yen's Q3 statistic. However, Q3 does not have a known sampling distribution, and thus, it is typically used in a descriptive fashion, such that values larger than an arbitrary cut-value (e.g., 0.2) indicate the presence of local dependence. The current study introduces a formal test of the null hypothesis that for a given item pair Q3 is 0, based on permutation test methodology. A small simulation study carried out to assess the Type I error and power rates of the Q3 permutation test found that this new statistic maintains good Type I error control, while also yielding power for detecting local dependence at a rate higher than that associated with the use of the 0.2 cut-value.