Scott C Watkins1, Paul J Nietert2, Elisabeth Hughes3, Eric T Stickles3, Tracy E Wester4, Matthew D McEvoy3. 1. Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN. Electronic address: Scott.watkins@vanderbilt.edu. 2. Department of Public Health Sciences, College of Medicine, Medical University of South Carolina, Charleston, SC. 3. Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN. 4. Department of Anesthesia and Perioperative Medicine, College of Medicine, Medical University of South Carolina, Charleston, SC.
Abstract
BACKGROUND: Pediatric perioperative cardiac arrests are rare events that require rapid, skilled and coordinated efforts to optimize outcomes. We developed an assessment tool for assessing clinician performance during perioperative critical events termed Anesthesia-centric Pediatric Advanced Life Support (A-PALS). Here, we describe the development and evaluation of the A-PALS scoring instrument. METHODS: A group of raters scored videos of a perioperative team managing simulated events representing a range of scenarios and competency. We assessed agreement with the reference standard grading, as well as interrater and intrarater reliability. RESULTS: Overall, raters agreed with the reference standard 86.2% of the time. Rater scores concerning scenarios that depicted highly competent performance correlated better with the reference standard than scores from scenarios that depicted low clinical competence (P < 0.0001). Agreement with the reference standard was significantly (P < 0.0001) associated with scenario type, item category, level of competency displayed in the scenario, correct versus incorrect actions and whether the action was performed versus not performed. Kappa values were significantly (P < 0.0001) higher for highly competent performances as compared to lesser competent performances (good: mean = 0.83 [standard deviation = 0.07] versus poor: mean = 0.61 [standard deviation = 0.14]). The intraclass correlation coefficient (interrater reliability) was 0.97 for the raters' composite scores on correct actions and 0.98 for their composite scores on incorrect actions. CONCLUSIONS: This study provides evidence for the validity of the A-PALS scoring instrument and demonstrates that the scoring instrument can provide reliable scores, although clinician performance affects reliability.
BACKGROUND: Pediatric perioperative cardiac arrests are rare events that require rapid, skilled and coordinated efforts to optimize outcomes. We developed an assessment tool for assessing clinician performance during perioperative critical events termed Anesthesia-centric Pediatric Advanced Life Support (A-PALS). Here, we describe the development and evaluation of the A-PALS scoring instrument. METHODS: A group of raters scored videos of a perioperative team managing simulated events representing a range of scenarios and competency. We assessed agreement with the reference standard grading, as well as interrater and intrarater reliability. RESULTS: Overall, raters agreed with the reference standard 86.2% of the time. Rater scores concerning scenarios that depicted highly competent performance correlated better with the reference standard than scores from scenarios that depicted low clinical competence (P < 0.0001). Agreement with the reference standard was significantly (P < 0.0001) associated with scenario type, item category, level of competency displayed in the scenario, correct versus incorrect actions and whether the action was performed versus not performed. Kappa values were significantly (P < 0.0001) higher for highly competent performances as compared to lesser competent performances (good: mean = 0.83 [standard deviation = 0.07] versus poor: mean = 0.61 [standard deviation = 0.14]). The intraclass correlation coefficient (interrater reliability) was 0.97 for the raters' composite scores on correct actions and 0.98 for their composite scores on incorrect actions. CONCLUSIONS: This study provides evidence for the validity of the A-PALS scoring instrument and demonstrates that the scoring instrument can provide reliable scores, although clinician performance affects reliability.
Authors: Estée C Grant; Vincent J Grant; Farhan Bhanji; Jonathan P Duff; Adam Cheng; Jocelyn M Lockyer Journal: Resuscitation Date: 2012-01-28 Impact factor: 5.262
Authors: J M Weller; B J Robinson; B Jolly; L M Watterson; M Joseph; S Bajenov; A J Haughton; P D Larsen Journal: Anaesthesia Date: 2005-03 Impact factor: 6.955
Authors: Diane B Wayne; Monica J Fudala; John Butter; Viva J Siddall; Joe Feinglass; Leonard D Wade; William C McGaghie Journal: Acad Med Date: 2005-10 Impact factor: 6.893
Authors: Joseph M Neal; Christopher M Bernards; John F Butterworth; Guido Di Gregorio; Kenneth Drasner; Michael R Hejtmanek; Michael F Mulroy; Richard W Rosenquist; Guy L Weinberg Journal: Reg Anesth Pain Med Date: 2010 Mar-Apr Impact factor: 6.288