A S Adams1, S B Soumerai, J Lomas, D Ross-Degnan. 1. Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, MA 02215, USA.
Abstract
OBJECTIVE: To assess trends in the use of self-report measures in research on adherence to practice guidelines since 1980, and to determine the impact of response bias on the validity of self-reports as measures of quality of care. METHODS: We conducted a MEDLINE search using defined search terms for the period 1980 to 1996. Included studies evaluated the adherence of clinicians to practice guidelines, official policies, or other evidence-based recommendations. Among studies containing both self-report (e.g. interviews) and objective measures of adherence (e.g. medical records), we compared self-reported and objective adherence rates (measured as per cent adherence). Evidence of response bias was defined as self-reported adherence significantly exceeding the objective measure at the 5% level. RESULTS: We identified 326 studies of guideline adherence. The use of self-report measures of adherence increased from 18% of studies in 1980 to 41% of studies in 1985. Of the 10 studies that used both self-report and objective measures, eight supported the existence of response bias in all self-reported measures. In 87% of 37 comparisons, self-reported adherence rates exceeded the objective rates, resulting in a median over-estimation of adherence of 27% (absolute difference). CONCLUSIONS: Although self-reports may provide information regarding clinicians' knowledge of guideline recommendations, they are subject to bias and should not be used as the sole measure of guideline adherence.
OBJECTIVE: To assess trends in the use of self-report measures in research on adherence to practice guidelines since 1980, and to determine the impact of response bias on the validity of self-reports as measures of quality of care. METHODS: We conducted a MEDLINE search using defined search terms for the period 1980 to 1996. Included studies evaluated the adherence of clinicians to practice guidelines, official policies, or other evidence-based recommendations. Among studies containing both self-report (e.g. interviews) and objective measures of adherence (e.g. medical records), we compared self-reported and objective adherence rates (measured as per cent adherence). Evidence of response bias was defined as self-reported adherence significantly exceeding the objective measure at the 5% level. RESULTS: We identified 326 studies of guideline adherence. The use of self-report measures of adherence increased from 18% of studies in 1980 to 41% of studies in 1985. Of the 10 studies that used both self-report and objective measures, eight supported the existence of response bias in all self-reported measures. In 87% of 37 comparisons, self-reported adherence rates exceeded the objective rates, resulting in a median over-estimation of adherence of 27% (absolute difference). CONCLUSIONS: Although self-reports may provide information regarding clinicians' knowledge of guideline recommendations, they are subject to bias and should not be used as the sole measure of guideline adherence.