Identification of questionable data using validity indices and item response theory methods: Examinations with a teacher-rating scale.

To assess quality, psychological instruments often include additional scales or validity indices to alert users to the presence of questionable data. Although identified cases are often scrutinized further or removed, it is not fully understood how such cases may affect overall analyses if retained. Using a teacher-report instrument of child behavior, this study examined 5 different validity indices: F-Index, Consistency Index, Response Pattern Index, Rasch Outfit, and Rasch Infit. Results showed that methods largely did not identify similar cases. Removing flagged cases did not have a great effect on reliability estimates or validity coefficients. Hierarchical linear modeling illustrated that differences in validity indices were significantly related to years of teaching experience and level of education for the F-Index. In general, before using social-emotional screeners, teacher training may be useful to help minimize response effects. (PsycINFO Database Record (c) 2018 APA, all rights reserved).