Item-Fit Statistic Based on Posterior Probabilities of Membership in Ability Groups.

A novel approach to item-fit analysis based on an asymptotic test is proposed. The new test statistic, χ w 2 , compares pseudo-observed and expected item mean scores over a set of ability bins. The item mean scores are computed as weighted means with weights based on test-takers' a posteriori density of ability within the bin. This article explores the properties of χ w 2 in case of dichotomously scored items for unidimensional IRT models. Monte Carlo experiments were conducted to analyze the performance of χ w 2 . Type I error of χ w 2   was acceptably close to the nominal level and it had greater power than Orlando and Thissen's S - x 2 . Under some conditions, power of χ w 2 also exceeded the one reported for the computationally more demanding Stone's χ 2 ∗ .