Gabriela Martinez-Zayas1, Francisco A Almeida2, Lonny Yarmus3, Daniel Steinfort4, Donald R Lazarus5, Michael J Simoff6, Timothy Saettele7, Septimiu Murgu8, Tarek Dammad9, D Kevin Duong10, Lakshmi Mudambi11, Joshua J Filner12, Sofia Molina1, Carlos Aravena13, Jeffrey Thiboutot3, Asha Bonney14, Adriana M Rueda5, Labib G Debiane6, D Kyle Hogarth8, Harmeet Bedi10, Mark Deffebach11, Ala-Eddin S Sagar15, Joseph Cicenia2, Diana H Yu16, Avi Cohen6, Laura Frye17, Horiana B Grosu1, Thomas Gildea2, David Feller-Kopman3, Roberto F Casal1, Michael Machuzak2, Muhammad H Arain1, Sonali Sethi2, George A Eapen1, Louis Lam2, Carlos A Jimenez1, Manuel Ribeiro2, Laila Z Noor1, Atul Mehta2, Juhee Song18, Humberto Choi2, Junsheng Ma18, Liang Li18, David E Ost19. 1. Department of Pulmonary Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX. 2. Department of Pulmonary Medicine, Cleveland Clinic, Cleveland, OH. 3. Division of Pulmonary and Critical Care, Johns Hopkins University, Baltimore, MD. 4. Department of Respiratory Medicine, Royal Melbourne Hospital, Melbourne, Australia; Department of Medicine, University of Melbourne, Melbourne, Australia. 5. Department of Pulmonary, Critical Care, and Sleep Medicine, Baylor College of Medicine, Houston, TX. 6. Department of Pulmonary and Critical Care Medicine, Henry Ford Hospital, Detroit, MI. 7. Department of Pulmonary Disease and Critical Care Medicine, Saint Luke's Hospital of Kansas City, Kansas City, MO. 8. Division of Pulmonary and Critical Care Medicine, University of Chicago, Chicago, IL. 9. Department of Pulmonary Medicine, University of New Mexico, Albuquerque, NM; Department of Pulmonary and Critical Care Medicine, CHRISTUS St. Vincent Medical Center, Santa Fe, NM. 10. Department of Pulmonary, Allergy and Critical Care Medicine, Stanford University Medical Center and School of Medicine, Stanford, CA. 11. Division of Pulmonary and Critical Care, VA Portland Health Care System, Oregon Health and Science University, Portland, OR. 12. Department of Pulmonary Medicine, Northwest Permanente and The Center for Health Research, Kaiser Permanente Northwest, Portland, OR. 13. Department of Respiratory Diseases, Pontificia Universidad Catolica de Chile, Santiago, Chile. 14. Department of Respiratory Medicine, Royal Melbourne Hospital, Melbourne, Australia. 15. Department of Pulmonary Medicine, Banner MD Anderson Cancer Center, Gilbert, AZ. 16. Division of Pulmonary, Critical Care and Sleep Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA. 17. Division of Allergy, Pulmonary and Critical Care Medicine, University of Wisconsin, Madison, WI. 18. Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX. 19. Department of Pulmonary Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX. Electronic address: dost@mdanderson.org.
Abstract
BACKGROUND: Two models, the Help with the Assessment of Adenopathy in Lung cancer (HAL) and Help with Oncologic Mediastinal Evaluation for Radiation (HOMER), were recently developed to estimate the probability of nodal disease in patients with non-small cell lung cancer (NSCLC) as determined by endobronchial ultrasound-transbronchial needle aspiration (EBUS-TBNA). The objective of this study was to prospectively externally validate both models at multiple centers. RESEARCH QUESTION: Are the HAL and HOMER models valid across multiple centers? STUDY DESIGN AND METHODS: This multicenter prospective observational cohort study enrolled consecutive patients with PET-CT clinical-radiographic stages T1-3, N0-3, M0 NSCLC undergoing EBUS-TBNA staging. HOMER was used to predict the probability of N0 vs N1 vs N2 or N3 (N2|3) disease, and HAL was used to predict the probability of N2|3 (vs N0 or N1) disease. Model discrimination was assessed using the area under the receiver operating characteristics curve (ROC-AUC), and calibration was assessed using the Brier score, calibration plots, and the Hosmer-Lemeshow test. RESULTS: Thirteen centers enrolled 1,799 patients. HAL and HOMER demonstrated good discrimination: HAL ROC-AUC = 0.873 (95%CI, 0.856-0.891) and HOMER ROC-AUC = 0.837 (95%CI, 0.814-0.859) for predicting N1 disease or higher (N1|2|3) and 0.876 (95%CI, 0.855-0.897) for predicting N2|3 disease. Brier scores were 0.117 and 0.349, respectively. Calibration plots demonstrated good calibration for both models. For HAL, the difference between forecast and observed probability of N2|3 disease was +0.012; for HOMER, the difference for N1|2|3 was -0.018 and for N2|3 was +0.002. The Hosmer-Lemeshow test was significant for both models (P = .034 and .002), indicating a small but statistically significant calibration error. INTERPRETATION: HAL and HOMER demonstrated good discrimination and calibration in multiple centers. Although calibration error was present, the magnitude of the error is small, such that the models are informative.
BACKGROUND: Two models, the Help with the Assessment of Adenopathy in Lung cancer (HAL) and Help with Oncologic Mediastinal Evaluation for Radiation (HOMER), were recently developed to estimate the probability of nodal disease in patients with non-small cell lung cancer (NSCLC) as determined by endobronchial ultrasound-transbronchial needle aspiration (EBUS-TBNA). The objective of this study was to prospectively externally validate both models at multiple centers. RESEARCH QUESTION: Are the HAL and HOMER models valid across multiple centers? STUDY DESIGN AND METHODS: This multicenter prospective observational cohort study enrolled consecutive patients with PET-CT clinical-radiographic stages T1-3, N0-3, M0 NSCLC undergoing EBUS-TBNA staging. HOMER was used to predict the probability of N0 vs N1 vs N2 or N3 (N2|3) disease, and HAL was used to predict the probability of N2|3 (vs N0 or N1) disease. Model discrimination was assessed using the area under the receiver operating characteristics curve (ROC-AUC), and calibration was assessed using the Brier score, calibration plots, and the Hosmer-Lemeshow test. RESULTS: Thirteen centers enrolled 1,799 patients. HAL and HOMER demonstrated good discrimination: HAL ROC-AUC = 0.873 (95%CI, 0.856-0.891) and HOMER ROC-AUC = 0.837 (95%CI, 0.814-0.859) for predicting N1 disease or higher (N1|2|3) and 0.876 (95%CI, 0.855-0.897) for predicting N2|3 disease. Brier scores were 0.117 and 0.349, respectively. Calibration plots demonstrated good calibration for both models. For HAL, the difference between forecast and observed probability of N2|3 disease was +0.012; for HOMER, the difference for N1|2|3 was -0.018 and for N2|3 was +0.002. The Hosmer-Lemeshow test was significant for both models (P = .034 and .002), indicating a small but statistically significant calibration error. INTERPRETATION: HAL and HOMER demonstrated good discrimination and calibration in multiple centers. Although calibration error was present, the magnitude of the error is small, such that the models are informative.
Authors: Roberto F Casal; Boris Sepesi; Ala-Eddin S Sagar; Juerg Tschirren; Minxing Chen; Liang Li; Jennifer Sunny; Joyce Williams; Horiana B Grosu; George A Eapen; Carlos A Jimenez; David E Ost Journal: Eur Respir J Date: 2019-05-09 Impact factor: 16.671
Authors: David E Ost; Armin Ernst; Xiudong Lei; Kevin L Kovitz; Sadia Benzaquen; Javier Diaz-Mendoza; Sara Greenhill; Jennifer Toth; David Feller-Kopman; Jonathan Puchalski; Daniel Baram; Raj Karunakara; Carlos A Jimenez; Joshua J Filner; Rodolfo C Morice; George A Eapen; Gaetane C Michaud; Rosa M Estrada-Y-Martin; Samaan Rafeq; Horiana B Grosu; Cynthia Ray; Christopher R Gilbert; Lonny B Yarmus; Michael Simoff Journal: Am J Respir Crit Care Med Date: 2016-01-01 Impact factor: 21.405
Authors: Nithya Ramnath; Thomas J Dilling; Loren J Harris; Anthony W Kim; Gaetane C Michaud; Alex A Balekian; Rebecca Diekemper; Frank C Detterbeck; Douglas A Arenberg Journal: Chest Date: 2013-05 Impact factor: 9.410
Authors: Gerard A Silvestri; Anne V Gonzalez; Michael A Jantz; Mitchell L Margolis; Michael K Gould; Lynn T Tanoue; Loren J Harris; Frank C Detterbeck Journal: Chest Date: 2013-05 Impact factor: 9.410
Authors: Daniel P Steinfort; Shankar Siva; Kanishka Rangamuwa; Percy Lee; David Fielding; Phan Nguyen; Barton R Jennings; Shaun Yo; Nick Hardcastle; Gargi Kothari; Laurence Crombag; Jouke Annema; Kazuhiro Yasufuku; David E Ost; Louis B Irving Journal: BMC Pulm Med Date: 2022-09-24 Impact factor: 3.320