BACKGROUND: Clinical practice guidelines quickly become outdated. One reason they might not be updated as often as needed is the expense of collecting expert judgment regarding the evidence. The RAND-UCLA Appropriateness Method is one commonly used method for collecting expert opinion. We tested whether a less expensive, mail-only process could substitute for the standard in-person process normally used. METHODS: We performed a 4-way replication of the appropriateness panel process for coronary revascularization and hysterectomy, conducting 3 panels using the conventional in-person method and 1 panel entirely by mail. All indications were classified as inappropriate or not (to evaluate overuse), and coronary revascularization indications were classified as necessary or not (to evaluate underuse). Kappa statistics were calculated for the comparison in ratings from the 2 methods. RESULTS: Agreement beyond chance between the 2 panel methods ranged from moderate to substantial. The kappa statistic to detect overuse was 0.57 for coronary revascularization and 0.70 for hysterectomy. The kappa statistic to detect coronary revascularization underuse was 0.76. There were no cases in which coronary revascularization was considered inappropriate by 1 method, but necessary or appropriate by the other. Three of 636 (0.5%) hysterectomy cases were categorized as inappropriate by 1 method but appropriate by the other. CONCLUSIONS: The reproducibility of the overuse and underuse assessments from the mail-only compared with the conventional in-person conduct of expert panels in this application was similar to the underlying reproducibility of the process. This suggests a potential role for updating guidelines using an expert judgment process conducted entirely through the mail.
BACKGROUND: Clinical practice guidelines quickly become outdated. One reason they might not be updated as often as needed is the expense of collecting expert judgment regarding the evidence. The RAND-UCLA Appropriateness Method is one commonly used method for collecting expert opinion. We tested whether a less expensive, mail-only process could substitute for the standard in-person process normally used. METHODS: We performed a 4-way replication of the appropriateness panel process for coronary revascularization and hysterectomy, conducting 3 panels using the conventional in-person method and 1 panel entirely by mail. All indications were classified as inappropriate or not (to evaluate overuse), and coronary revascularization indications were classified as necessary or not (to evaluate underuse). Kappa statistics were calculated for the comparison in ratings from the 2 methods. RESULTS: Agreement beyond chance between the 2 panel methods ranged from moderate to substantial. The kappa statistic to detect overuse was 0.57 for coronary revascularization and 0.70 for hysterectomy. The kappa statistic to detect coronary revascularization underuse was 0.76. There were no cases in which coronary revascularization was considered inappropriate by 1 method, but necessary or appropriate by the other. Three of 636 (0.5%) hysterectomy cases were categorized as inappropriate by 1 method but appropriate by the other. CONCLUSIONS: The reproducibility of the overuse and underuse assessments from the mail-only compared with the conventional in-person conduct of expert panels in this application was similar to the underlying reproducibility of the process. This suggests a potential role for updating guidelines using an expert judgment process conducted entirely through the mail.
Authors: Sheryl Davies; Patrick S Romano; Eric M Schmidt; Ellen Schultz; Jeffrey J Geppert; Kathryn M McDonald Journal: Health Serv Res Date: 2011-07-25 Impact factor: 3.402
Authors: Amy L Halverson; Morgan M Sellers; Karl Y Bilimoria; Mary T Hawn; Mark V Williams; Robin S McLeod; Clifford Y Ko Journal: J Gastrointest Surg Date: 2014-06-10 Impact factor: 3.452
Authors: Iain R Murray; Jorge Chahla; Marc R Safran; Aaron J Krych; Daniel B F Saris; Arnold I Caplan; Robert F LaPrade Journal: J Bone Joint Surg Am Date: 2019-05-15 Impact factor: 5.284
Authors: Karl Y Bilimoria; David J Bentrem; Keith D Lillemoe; Mark S Talamonti; Clifford Y Ko Journal: J Natl Cancer Inst Date: 2009-06-09 Impact factor: 13.506
Authors: Jorge Chahla; Betina B Hinckel; Adam B Yanke; Jack Farr; William D Bugbee; James L Carey; Brian J Cole; Dennis C Crawford; James E Fleischli; Alan Getgood; Andreas H Gomoll; Simon Gortz; Allan E Gross; Deryk G Jones; Aaron J Krych; Christian Lattermann; Bert R Mandelbaum; Peter R Mandt; Tom Minas; Raffy Mirzayan; Timothy S Mologne; John D Polousky; Matthew T Provencher; Scott A Rodeo; Oleg Safir; Seth Lawrence Sherman; Eric D Strauss; Sabrina M Strickland; Christopher J Wahl; Riley J Williams Journal: Orthop J Sports Med Date: 2020-03-26
Authors: Annemie Heselmans; Bert Aertgeerts; Peter Donceel; Stijn Van de Velde; Peter Vanbrabant; Dirk Ramaekers Journal: J Med Internet Res Date: 2013-01-17 Impact factor: 5.428