Rosy Reynolds1, Paul C Lambert, Paul R Burton. 1. Department of Medical Microbiology, Southmead Hospital, Southmead Road, Bristol BS10 5NB, UK. rreynolds@bsac.org.uk
Abstract
OBJECTIVES: Logistic regression is commonly used to analyse resistance surveillance studies, but variation between collecting centres undermines its assumption that isolates are independent. We studied the impact of this problem and the ability of alternative methods to overcome it. We also investigated different study designs and estimated the statistical power of the BSAC Resistance Surveillance Programmes. METHODS: We simulated datasets with various combinations of study design, inter-centre variation, annual centre turnover, initial resistance level and odds ratio, and analysed 1000 repetitions of each for trends in resistance by five variants of logistic regression. RESULTS: Traditional analysis by unadjusted logistic regression was invalid because it gave very high type 1 (false-positive) error rates, up to 49%, in the presence of high levels of inter-centre variation and turnover. Of the other methods investigated, logistic regression with random effects for centre performed best: it had appropriate error rates for all study designs assessed and generally had higher power than fixed-effects or cluster-robust approaches. A 'Diffuse' study with more centres contributing fewer isolates was less susceptible to the ill-effects of inter-centre variation than a study of equal overall size with fewer centres contributing more, and had slightly higher power. CONCLUSIONS: Unadjusted logistic regression, ignoring inter-centre variation, is unsuitable for the analysis of trends in typical resistance surveillance studies, often leads to erroneous conclusions and should be avoided. Random effects logistic regression is an appropriate, widely applicable alternative, available in most standard statistical software. Collecting isolates from a larger number of centres has both statistical and scientific advantages.
OBJECTIVES: Logistic regression is commonly used to analyse resistance surveillance studies, but variation between collecting centres undermines its assumption that isolates are independent. We studied the impact of this problem and the ability of alternative methods to overcome it. We also investigated different study designs and estimated the statistical power of the BSAC Resistance Surveillance Programmes. METHODS: We simulated datasets with various combinations of study design, inter-centre variation, annual centre turnover, initial resistance level and odds ratio, and analysed 1000 repetitions of each for trends in resistance by five variants of logistic regression. RESULTS: Traditional analysis by unadjusted logistic regression was invalid because it gave very high type 1 (false-positive) error rates, up to 49%, in the presence of high levels of inter-centre variation and turnover. Of the other methods investigated, logistic regression with random effects for centre performed best: it had appropriate error rates for all study designs assessed and generally had higher power than fixed-effects or cluster-robust approaches. A 'Diffuse' study with more centres contributing fewer isolates was less susceptible to the ill-effects of inter-centre variation than a study of equal overall size with fewer centres contributing more, and had slightly higher power. CONCLUSIONS: Unadjusted logistic regression, ignoring inter-centre variation, is unsuitable for the analysis of trends in typical resistance surveillance studies, often leads to erroneous conclusions and should be avoided. Random effects logistic regression is an appropriate, widely applicable alternative, available in most standard statistical software. Collecting isolates from a larger number of centres has both statistical and scientific advantages.