Tanya G K Bentley1, Joshua T Cohen2, Elena B Elkin3, Julie Huynh4, Arnab Mukherjea5, Thanh H Neville6, Matthew Mei7, Ronda Copher8, Russell Knoth8, Ioana Popescu6, Jackie Lee9, Jenelle M Zambrano9, Michael S Broder9. 1. Partnership for Health Analytic Research, LLC, Beverly Hills, CA, USA. Electronic address: tbentley@pharllc.com. 2. Tufts Medical Center, Boston, MA, USA. 3. Memorial Sloan Kettering Cancer Center, New York, NY, USA. 4. Hematology Oncology of San Fernando Valley, Encino, CA, USA. 5. Health Sciences Program, California State University, East Bay, Hayward, CA, USA. 6. Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA. 7. City of Hope National Medical Center, Duarte, CA, USA. 8. Eisai Inc., Woodcliff Lake, NJ, USA. 9. Partnership for Health Analytic Research, LLC, Beverly Hills, CA, USA.
Abstract
BACKGROUND: Several organizations have developed frameworks to systematically assess the value of new drugs. These organizations include the American Society of Clinical Oncology (ASCO), the European Society for Medical Oncology (ESMO), the Institute for Clinical and Economic Review (ICER), and the National Comprehensive Cancer Network (NCCN). OBJECTIVES: To understand the extent to which these four tools can facilitate value-based treatment decisions in oncology. METHODS: In this pilot study, eight panelists conducted value assessments of five advanced lung cancer drugs using the ASCO, ESMO, and ICER frameworks. The panelists received instructions and published clinical data required to complete the assessments. Published NCCN framework scores were abstracted. The Kendall's W coefficient was used to measure convergent validity among the four frameworks. Intraclass correlation coefficients were used to measure inter-rater reliability among the ASCO, ESMO, and ICER frameworks. Sensitivity analyses were conducted. RESULTS: Drugs were ranked similarly by the four frameworks, with Kendall's W of 0.703 (P = 0.006) across all the four frameworks. Pairwise, Kendall's W was the highest for ESMO-ICER (W = 0.974; P = 0.007) and ASCO-NCCN (W = 0.944; P = 0.022) and the lowest for ICER-NCCN (W = 0.647; P = 0.315) and ESMO-NCCN (W = 0.611; P = 0.360). Intraclass correlation coefficients (confidence interval [CI]) for the ASCO, ESMO, and ICER frameworks were 0.786 (95% CI 0.517-0.970), 0.804 (95% CI 0.545-0.973), and 0.281 (95% CI 0.055-0.799), respectively. When scores were rescaled to 0 to 100, the ICER framework provided the narrowest band of scores. CONCLUSIONS: The ASCO, ESMO, ICER, and NCCN frameworks demonstrated convergent validity, despite differences in conceptual approaches used. The ASCO inter-rater reliability was high, although potentially at the cost of user burden. The ICER inter-rater reliability was poor, possibly because of its failure to distinguish differential value among the sample of drugs tested. Refinements of all frameworks should continue on the basis of further testing and stakeholder feedback.
BACKGROUND: Several organizations have developed frameworks to systematically assess the value of new drugs. These organizations include the American Society of Clinical Oncology (ASCO), the European Society for Medical Oncology (ESMO), the Institute for Clinical and Economic Review (ICER), and the National Comprehensive Cancer Network (NCCN). OBJECTIVES: To understand the extent to which these four tools can facilitate value-based treatment decisions in oncology. METHODS: In this pilot study, eight panelists conducted value assessments of five advanced lung cancer drugs using the ASCO, ESMO, and ICER frameworks. The panelists received instructions and published clinical data required to complete the assessments. Published NCCN framework scores were abstracted. The Kendall's W coefficient was used to measure convergent validity among the four frameworks. Intraclass correlation coefficients were used to measure inter-rater reliability among the ASCO, ESMO, and ICER frameworks. Sensitivity analyses were conducted. RESULTS: Drugs were ranked similarly by the four frameworks, with Kendall's W of 0.703 (P = 0.006) across all the four frameworks. Pairwise, Kendall's W was the highest for ESMO-ICER (W = 0.974; P = 0.007) and ASCO-NCCN (W = 0.944; P = 0.022) and the lowest for ICER-NCCN (W = 0.647; P = 0.315) and ESMO-NCCN (W = 0.611; P = 0.360). Intraclass correlation coefficients (confidence interval [CI]) for the ASCO, ESMO, and ICER frameworks were 0.786 (95% CI 0.517-0.970), 0.804 (95% CI 0.545-0.973), and 0.281 (95% CI 0.055-0.799), respectively. When scores were rescaled to 0 to 100, the ICER framework provided the narrowest band of scores. CONCLUSIONS: The ASCO, ESMO, ICER, and NCCN frameworks demonstrated convergent validity, despite differences in conceptual approaches used. The ASCO inter-rater reliability was high, although potentially at the cost of user burden. The ICER inter-rater reliability was poor, possibly because of its failure to distinguish differential value among the sample of drugs tested. Refinements of all frameworks should continue on the basis of further testing and stakeholder feedback.
Authors: Elizabeth M Jaffee; Chi Van Dang; David B Agus; Brian M Alexander; Kenneth C Anderson; Alan Ashworth; Anna D Barker; Roshan Bastani; Sangeeta Bhatia; Jeffrey A Bluestone; Otis Brawley; Atul J Butte; Daniel G Coit; Nancy E Davidson; Mark Davis; Ronald A DePinho; Robert B Diasio; Giulio Draetta; A Lindsay Frazier; Andrew Futreal; Sam S Gambhir; Patricia A Ganz; Levi Garraway; Stanton Gerson; Sumit Gupta; James Heath; Ruth I Hoffman; Cliff Hudis; Chanita Hughes-Halbert; Ramy Ibrahim; Hossein Jadvar; Brian Kavanagh; Rick Kittles; Quynh-Thu Le; Scott M Lippman; David Mankoff; Elaine R Mardis; Deborah K Mayer; Kelly McMasters; Neal J Meropol; Beverly Mitchell; Peter Naredi; Dean Ornish; Timothy M Pawlik; Jeffrey Peppercorn; Martin G Pomper; Derek Raghavan; Christine Ritchie; Sally W Schwarz; Richard Sullivan; Richard Wahl; Jedd D Wolchok; Sandra L Wong; Alfred Yung Journal: Lancet Oncol Date: 2017-10-31 Impact factor: 41.316
Authors: Patricia Vella Bonanno; Michael Ermisch; Brian Godman; Antony P Martin; Jesper Van Den Bergh; Liudmila Bezmelnitsyna; Anna Bucsics; Francis Arickx; Alexander Bybau; Tomasz Bochenek; Marc van de Casteele; Eduardo Diogene; Irene Eriksson; Jurij Fürst; Mohamed Gad; Ieva Greičiūtė-Kuprijanov; Martin van der Graaff; Jolanta Gulbinovic; Jan Jones; Roberta Joppi; Marija Kalaba; Ott Laius; Irene Langner; Ileana Mardare; Vanda Markovic-Pekovic; Einar Magnusson; Oyvind Melien; Dmitry O Meshkov; Guenka I Petrova; Gisbert Selke; Catherine Sermet; Steven Simoens; Ad Schuurman; Ricardo Ramos; Jorge Rodrigues; Corinne Zara; Eva Zebedin-Brandl; Alan Haycox Journal: Front Pharmacol Date: 2017-08-23 Impact factor: 5.810
Authors: Howard L Kaufman; Michael B Atkins; Prasun Subedi; James Wu; James Chambers; T Joseph Mattingly; Jonathan D Campbell; Jeff Allen; Andrea E Ferris; Richard L Schilsky; Daniel Danielson; J Leonard Lichtenfeld; Linda House; Wendy K D Selig Journal: J Immunother Cancer Date: 2019-05-17 Impact factor: 13.751
Authors: Green Bae; SeungJin Bae; Donghwan Lee; Juhee Han; Dong-Hoe Koo; Do Yeun Kim; Hee-Jun Kim; Sung Young Oh; Hee Yeon Lee; Jong Hwan Lee; Hye Sook Han; Hyerim Ha; Jin Hyoung Kang Journal: Int J Environ Res Public Health Date: 2021-03-18 Impact factor: 3.390