Lynda S Robson1, Selahadin Ibrahim2, Sheilah Hogg-Johnson3, Ivan A Steenstra4, Dwayne Van Eerd5, Benjamin C Amick6. 1. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada. Electronic address: lrobson@iwh.on.ca. 2. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada. Electronic address: sibrahim@iwh.on.ca. 3. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada. Electronic address: shoggjohnson@iwh.on.ca. 4. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada. Electronic address: isteenstra@morneaushepell.com. 5. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada; School of Public Health and Health Systems, University of Waterloo, 200 University Ave West, Waterloo, Ontario, N2L 3G1, Canada. Electronic address: dvaneerd@iwh.on.ca. 6. Institute for Work & Health, 481 University Ave., Suite 800, Toronto, Ontario M5G 2E9, Canada. Electronic address: bamick@iwh.on.ca.
Abstract
INTRODUCTION: OHS management audits are one means of obtaining data that may serve as leading indicators. The measurement properties of such data are therefore important. This study used data from Workwell audit program in Ontario, a Canadian province. The audit instrument consisted of 122 items related to 17 OHS management elements. The study sought answers regarding (a) the ability of audit-based scores to predict workers' compensation claims outcomes, (b) structural characteristics of the data in relation to the organization of the audit instrument, and (c) internal consistency of items within audit elements. METHOD: The sample consisted of audit and claims data from 1240 unique firms that had completed one or two OHS management audits during 2007-2010. Predictors derived from the audit results were used in multivariable negative binomial regression modeling of workers' compensation claims outcomes. Confirmatory factor analyses were used to examine the instrument's structural characteristics. Kuder-Richardson coefficients of internal consistency were calculated for each audit element. RESULTS: The ability of audit scores to predict subsequent claims data could not be established. Factor analysis supported the audit instrument's element-based structure. KR-20 values were high (≥0.83). CONCLUSIONS: The Workwell audit data display structural validity and high internal consistency, but not, to date, construct validity, since the audit scores are generally not predictive of subsequent firm claim experience. Audit scores should not be treated as leading indicators of workplace OHS performance without supporting empirical data. PRACTICAL APPLICATIONS: Analyses of the measurement properties of audit data can inform decisionmakers about the operation of an audit program, possible future directions in audit instrument development, and the appropriate use of audit data. In particular, decision-makers should be cautious in their use of audit scores as leading indicators, in the absence of supporting empirical data.
INTRODUCTION:OHS management audits are one means of obtaining data that may serve as leading indicators. The measurement properties of such data are therefore important. This study used data from Workwell audit program in Ontario, a Canadian province. The audit instrument consisted of 122 items related to 17 OHS management elements. The study sought answers regarding (a) the ability of audit-based scores to predict workers' compensation claims outcomes, (b) structural characteristics of the data in relation to the organization of the audit instrument, and (c) internal consistency of items within audit elements. METHOD: The sample consisted of audit and claims data from 1240 unique firms that had completed one or two OHS management audits during 2007-2010. Predictors derived from the audit results were used in multivariable negative binomial regression modeling of workers' compensation claims outcomes. Confirmatory factor analyses were used to examine the instrument's structural characteristics. Kuder-Richardson coefficients of internal consistency were calculated for each audit element. RESULTS: The ability of audit scores to predict subsequent claims data could not be established. Factor analysis supported the audit instrument's element-based structure. KR-20 values were high (≥0.83). CONCLUSIONS: The Workwell audit data display structural validity and high internal consistency, but not, to date, construct validity, since the audit scores are generally not predictive of subsequent firm claim experience. Audit scores should not be treated as leading indicators of workplace OHS performance without supporting empirical data. PRACTICAL APPLICATIONS: Analyses of the measurement properties of audit data can inform decisionmakers about the operation of an audit program, possible future directions in audit instrument development, and the appropriate use of audit data. In particular, decision-makers should be cautious in their use of audit scores as leading indicators, in the absence of supporting empirical data.