OBJECTIVE: To examine interrater agreement for classifying emergency medical services transports as medically unnecessary using emergency department diagnosis as the sole determining factor. METHODS: Three emergency physicians and two family medicine physicians classified 913 International Classification of Diseases, Ninth Revision (ICD-9) codes as medically necessary, unnecessary, or uncertain. Overall agreement, interrater agreement, and agreement within 17 major disease categories were measured using kappa statistics in SAS. RESULTS: Physicians rated between 25% and 65% of diagnoses codes as medically unnecessary. Overall agreement was fair (kappa = 0.31). Agreement within specialties was higher among family medicine-trained physicians than among emergency physicians (kappa = 0.52 and kappa = 0.22, respectively). Agreement across all raters was highest for diseases classified as symptoms, signs, and ill-defined conditions (kappa = 0.40) and lowest for diseases of the blood and blood-forming organs (kappa = -0.17). Agreement was observably better between physicians with more experience. CONCLUSIONS: Considerable doubts about the utility of emergency department diagnosis as a criterion are raised from study findings. Further development of Neely Conference criteria is needed. Priority should be given to testing and validation of criteria as well as exploration of differences in judgment between specialists representative of the medical director profession.
OBJECTIVE: To examine interrater agreement for classifying emergency medical services transports as medically unnecessary using emergency department diagnosis as the sole determining factor. METHODS: Three emergency physicians and two family medicine physicians classified 913 International Classification of Diseases, Ninth Revision (ICD-9) codes as medically necessary, unnecessary, or uncertain. Overall agreement, interrater agreement, and agreement within 17 major disease categories were measured using kappa statistics in SAS. RESULTS: Physicians rated between 25% and 65% of diagnoses codes as medically unnecessary. Overall agreement was fair (kappa = 0.31). Agreement within specialties was higher among family medicine-trained physicians than among emergency physicians (kappa = 0.52 and kappa = 0.22, respectively). Agreement across all raters was highest for diseases classified as symptoms, signs, and ill-defined conditions (kappa = 0.40) and lowest for diseases of the blood and blood-forming organs (kappa = -0.17). Agreement was observably better between physicians with more experience. CONCLUSIONS: Considerable doubts about the utility of emergency department diagnosis as a criterion are raised from study findings. Further development of Neely Conference criteria is needed. Priority should be given to testing and validation of criteria as well as exploration of differences in judgment between specialists representative of the medical director profession.
Authors: Keith Flower; Anneke Post; Jeremy Sussman; Niels Tangherlini; John Mendelson; Mark J Pletcher Journal: Emerg Med J Date: 2010-07-08 Impact factor: 2.740
Authors: P Daniel Patterson; Matthew D Weaver; Rachel C Frank; Charles W Warner; Christian Martin-Gill; Francis X Guyette; Rollin J Fairbanks; Michael W Hubble; Thomas J Songer; Clifton W Callaway; Sheryl F Kelsey; David Hostler Journal: Prehosp Emerg Care Date: 2011-10-24 Impact factor: 3.077
Authors: P Daniel Patterson; Judith R Lave; Matthew D Weaver; Francis X Guyette; Robert M Arnold; Christian Martin-Gill; Jon C Rittenberger; David Krackhardt; Vincent N Mosesso; Ronald N Roth; Richard J Wadas; Donald M Yealy Journal: Prehosp Emerg Care Date: 2014-05-30 Impact factor: 3.077