BACKGROUND AND OBJECTIVES: Technical proficiency in regional anesthesia is often determined subjectively through in-training evaluations. Objective assessment tools improve these evaluations by providing criteria for measurement. However, any evaluation instrument needs to be valid and reliable before it is adopted into a curriculum. The purpose of this study is to determine the validity and reliability of a devised assessment of residents performing an interscalene brachial plexus block (ISB). METHODS: In this prospective study, 10 junior trainees and 10 senior trainees were videotaped performing an ISB. Junior trainees were defined as in their first year of anesthetic training and had performed less than 10 ISBs independently. Senior trainees had completed at least 1 year of anesthesia training and had performed greater than 10 ISBs independently. Two blinded expert raters independently evaluated the performance of the ISB using a checklist and global rating scale. Construct validity was established if the assessments were able to reliably discriminate between different levels of training. RESULTS: Senior trainees performed an ISB significantly better than junior trainees when assessed using the global rating scale (P < .05) and checklist (P < .001). The overall interrater reliability for the global rating scores was excellent (r = 0.85, P < .05) and was good for the checklist scores (r = 0.74, P < .05). CONCLUSIONS: Both assessment modalities were valid, in that they reliably discriminated between different levels of training. Objective measures of technical skills are feasible, timely, and improve the validity and reliability of competency assessments.
BACKGROUND AND OBJECTIVES: Technical proficiency in regional anesthesia is often determined subjectively through in-training evaluations. Objective assessment tools improve these evaluations by providing criteria for measurement. However, any evaluation instrument needs to be valid and reliable before it is adopted into a curriculum. The purpose of this study is to determine the validity and reliability of a devised assessment of residents performing an interscalene brachial plexus block (ISB). METHODS: In this prospective study, 10 junior trainees and 10 senior trainees were videotaped performing an ISB. Junior trainees were defined as in their first year of anesthetic training and had performed less than 10 ISBs independently. Senior trainees had completed at least 1 year of anesthesia training and had performed greater than 10 ISBs independently. Two blinded expert raters independently evaluated the performance of the ISB using a checklist and global rating scale. Construct validity was established if the assessments were able to reliably discriminate between different levels of training. RESULTS: Senior trainees performed an ISB significantly better than junior trainees when assessed using the global rating scale (P < .05) and checklist (P < .001). The overall interrater reliability for the global rating scores was excellent (r = 0.85, P < .05) and was good for the checklist scores (r = 0.74, P < .05). CONCLUSIONS: Both assessment modalities were valid, in that they reliably discriminated between different levels of training. Objective measures of technical skills are feasible, timely, and improve the validity and reliability of competency assessments.
Authors: Joseph M Neal; J C Gerancher; James R Hebl; Brian M Ilfeld; Colin J L McCartney; Carlo D Franco; Quinn H Hogan Journal: Reg Anesth Pain Med Date: 2009 Mar-Apr Impact factor: 6.288
Authors: David F Pepley; Hong-En Chen; Yichun Tang; Sanjib Das Adhikary; Scarlett R Miller; Jason Z Moore Journal: IEEE Trans Haptics Date: 2019-05-02 Impact factor: 2.487
Authors: Karthikeyan Kallidaikurichi Srinivasan; Anthony Gallagher; Niall O'Brien; Vinod Sudir; Nick Barrett; Raymund O'Connor; Francesca Holt; Peter Lee; Brian O'Donnell; George Shorten Journal: BMJ Open Date: 2018-10-15 Impact factor: 2.692