George Davey Smith1, Thierry Mertens. 1. Department of Social Medicine, University of Bristol, Canynge Hall, Whiteladies Road, Bristol BS8 2PR, UK.
Abstract
OBJECTIVE: Indirect data collection methods, or approaches which disturb usual practice, are generally used in health care evaluation. We have compared what doctors report at interview what is observed by an identified researcher with an unobtrusive measure of their usual practice. DESIGN: Private practitioners who provide a service to sexually transmitted disease (STD) patients were interviewed regarding their usual case management. An identified researcher carried out structured observations of consultations between physicians and patients. Simulated clients then sought consultations, presenting a standardised history and symptom profile. Structured reporting of the history taking, examination, treatment and counselling aspects of these consultations was undertaken. SUBJECTS: Eighteen private practitioners in Madras (now Chennai), India. MAIN OUTCOME MEASURES: Comparisons between interviews, observations by identified researchers and the experiences of simulated clients were carried out. RESULTS: Interviews with physicians and observations by identified researchers indicated more favourable practice than was seen during simulated client visits. These differences were substantial and would lead to a severe misrepresentation of the actual situation-and thus of intervention needs, if data from interviews or observations were relied upon. CONCLUSIONS: The usual methods used in the evaluation of medical services and in carrying out medical audit may produce highly unreliable findings. STD services in the study area are failing to realise their potential of improving the sexual health of populations. The methodological and substantive findings of this study could be combined through the introduction of simulated client visits in the monitoring, improvement and licensing of STD (and perhaps other medical) services.
OBJECTIVE: Indirect data collection methods, or approaches which disturb usual practice, are generally used in health care evaluation. We have compared what doctors report at interview what is observed by an identified researcher with an unobtrusive measure of their usual practice. DESIGN: Private practitioners who provide a service to sexually transmitted disease (STD) patients were interviewed regarding their usual case management. An identified researcher carried out structured observations of consultations between physicians and patients. Simulated clients then sought consultations, presenting a standardised history and symptom profile. Structured reporting of the history taking, examination, treatment and counselling aspects of these consultations was undertaken. SUBJECTS: Eighteen private practitioners in Madras (now Chennai), India. MAIN OUTCOME MEASURES: Comparisons between interviews, observations by identified researchers and the experiences of simulated clients were carried out. RESULTS: Interviews with physicians and observations by identified researchers indicated more favourable practice than was seen during simulated client visits. These differences were substantial and would lead to a severe misrepresentation of the actual situation-and thus of intervention needs, if data from interviews or observations were relied upon. CONCLUSIONS: The usual methods used in the evaluation of medical services and in carrying out medical audit may produce highly unreliable findings. STD services in the study area are failing to realise their potential of improving the sexual health of populations. The methodological and substantive findings of this study could be combined through the introduction of simulated client visits in the monitoring, improvement and licensing of STD (and perhaps other medical) services.