OBJECTIVE: To assess the reliability of survey measures of organizational characteristics based on reports of single and multiple informants. DATA SOURCE: Survey of 330 informants in 91 medical clinics providing care to HIV-infected persons under Title III of the Ryan White CARE Act. STUDY DESIGN: Cross-sectional survey. DATA COLLECTION METHODS: Surveys of clinicians and medical directors measured the implementation of quality improvement initiatives, priorities assigned to aspects of HIV care, barriers to providing high-quality HIV care, and quality improvement activities. Reliability of measures was assessed using generalizability coefficients. Components of variance and clinician-director differences were estimated using hierarchical regression models with survey items and informants nested within organizations. PRINCIPAL FINDINGS: There is substantial item- and informant-related variability in clinic assessments that results in modest or low clinic-level reliability for many measures. Directors occasionally gave more optimistic assessments of clinics than did clinicians. CONCLUSIONS: For most measures studied, obtaining adequate reliability requires multiple informants. Using multiple-item scales or multiple informants can improve the psychometric performance of measures of organizational characteristics. Studies of such characteristics should report the organizational level reliability of the measures used.
OBJECTIVE: To assess the reliability of survey measures of organizational characteristics based on reports of single and multiple informants. DATA SOURCE: Survey of 330 informants in 91 medical clinics providing care to HIV-infectedpersons under Title III of the Ryan White CARE Act. STUDY DESIGN: Cross-sectional survey. DATA COLLECTION METHODS: Surveys of clinicians and medical directors measured the implementation of quality improvement initiatives, priorities assigned to aspects of HIV care, barriers to providing high-quality HIV care, and quality improvement activities. Reliability of measures was assessed using generalizability coefficients. Components of variance and clinician-director differences were estimated using hierarchical regression models with survey items and informants nested within organizations. PRINCIPAL FINDINGS: There is substantial item- and informant-related variability in clinic assessments that results in modest or low clinic-level reliability for many measures. Directors occasionally gave more optimistic assessments of clinics than did clinicians. CONCLUSIONS: For most measures studied, obtaining adequate reliability requires multiple informants. Using multiple-item scales or multiple informants can improve the psychometric performance of measures of organizational characteristics. Studies of such characteristics should report the organizational level reliability of the measures used.
Authors: Lawrence Casalino; Robin R Gillies; Stephen M Shortell; Julie A Schmittdiel; Thomas Bodenheimer; James C Robinson; Thomas Rundall; Nancy Oswald; Helen Schauffler; Margaret C Wang Journal: JAMA Date: 2003 Jan 22-29 Impact factor: 56.272
Authors: S M Shortell; J L O'Brien; J M Carman; R W Foster; E F Hughes; H Boerstler; E J O'Connor Journal: Health Serv Res Date: 1995-06 Impact factor: 3.402
Authors: Karen B Lasater; Olga F Jarrín; Linda H Aiken; Matthew D McHugh; Douglas M Sloane; Herbert L Smith Journal: Med Care Date: 2019-09 Impact factor: 2.983
Authors: Sara C Handley; Molly Passarella; Ashley E Martin; Scott A Lorch; Sindhu K Srinivas; Ingrid M Nembhard Journal: Health Serv Res Date: 2022-02-18 Impact factor: 3.734
Authors: Sarang Deo; Keith McInnes; Charles J Corbett; Bruce E Landon; Martin F Shapiro; Ira B Wilson; Paul D Cleary Journal: Med Care Date: 2009-09 Impact factor: 2.983
Authors: Edward P Post; Amy M Kilbourne; Robert W Bremer; Francis X Solano; Harold Alan Pincus; Charles F Reynolds Journal: Implement Sci Date: 2009-12-31 Impact factor: 7.327