Simon R Turner1, Hollis Lai2, Basil S Nasir3, Kazuhiro Yasufuku4, Colin Schieman5, James Huang6, Eric L R Bédard7. 1. Division of Thoracic Surgery, University of Alberta, Edmonton, Canada. Electronic address: sturner@ualberta.ca. 2. Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada. 3. Division of Thoracic Surgery, Université de Montréal, Montréal, Canada. 4. Division of Thoracic Surgery, University of Toronto, Toronto, Canada. 5. Division of Thoracic Surgery, University of Calgary, Calgary, Canada. 6. Thoracic Surgery Service, Memorial Sloan Kettering Cancer Center, New York, New York. 7. Division of Thoracic Surgery, University of Alberta, Edmonton, Canada.
Abstract
BACKGROUND: To meet the need for competency assessment in thoracic surgery education, we developed and tested an instrument to assess trainees' ability to perform anatomic lung resection for cancer. METHODS: The Thoracic Competency Assessment Tool-Anatomic Resection for Lung Cancer (TCAT-ARC) was developed through a multistep process involving logical analysis, expert review, and simulation-based and clinical pilot testing. Validity evidence was gathered during a 6-month clinical study of trainees performing anatomic lung resections and assessments of practicing surgeons. Feedback was gathered via post-encounter questionnaires. RESULTS: A 35-item instrument was developed and was tested in the clinical validation study. Seven trainees in 4 North American institutions participated and completed 64 anatomic lung resections. Reliability was high (α = 0.93). Interobserver reliability (k = 0.73) and correlation with an existing global competency scale (k = 0.68) were moderately high. Item analysis revealed the most difficult and discriminatory items, which matched well with a conceptual understanding of lung resection. Both trainees and assessors viewed the instrument as highly educationally effective and user-friendly. Practicing surgeons outperformed trainees. CONCLUSIONS: The TCAT-ARC demonstrated early evidence of validity and reliability in assessing performance of anatomic lung resection. The instrument may be most useful early in training and as a means for providing fine-grained formative feedback about which steps have been mastered and which still require improvement. The TCAT-ARC may be used in training programs to aid in the development of trainees' competency and as a part of an aggregate assessment of trainees' overall mastery of the procedure and readiness for independent practice.
BACKGROUND: To meet the need for competency assessment in thoracic surgery education, we developed and tested an instrument to assess trainees' ability to perform anatomic lung resection for cancer. METHODS: The Thoracic Competency Assessment Tool-Anatomic Resection for Lung Cancer (TCAT-ARC) was developed through a multistep process involving logical analysis, expert review, and simulation-based and clinical pilot testing. Validity evidence was gathered during a 6-month clinical study of trainees performing anatomic lung resections and assessments of practicing surgeons. Feedback was gathered via post-encounter questionnaires. RESULTS: A 35-item instrument was developed and was tested in the clinical validation study. Seven trainees in 4 North American institutions participated and completed 64 anatomic lung resections. Reliability was high (α = 0.93). Interobserver reliability (k = 0.73) and correlation with an existing global competency scale (k = 0.68) were moderately high. Item analysis revealed the most difficult and discriminatory items, which matched well with a conceptual understanding of lung resection. Both trainees and assessors viewed the instrument as highly educationally effective and user-friendly. Practicing surgeons outperformed trainees. CONCLUSIONS: The TCAT-ARC demonstrated early evidence of validity and reliability in assessing performance of anatomic lung resection. The instrument may be most useful early in training and as a means for providing fine-grained formative feedback about which steps have been mastered and which still require improvement. The TCAT-ARC may be used in training programs to aid in the development of trainees' competency and as a part of an aggregate assessment of trainees' overall mastery of the procedure and readiness for independent practice.
Authors: Simon R Turner; Basil S Nasir; Hollis Lai; Kazuhiro Yasufuku; Colin Schieman; Brian E Louie; Eric L R Bédard Journal: Ann Thorac Surg Date: 2019-04-19 Impact factor: 4.330