BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion. OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion. METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings. RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99). CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.
BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion. OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion. METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings. RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99). CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.
Authors: Howard W Francis; Hamid Masood; Kashif N Chaudhry; Kulsoom Laeeq; John P Carey; Charles C Della Santina; Charles J Limb; John K Niparko; Nasir I Bhatti Journal: Otol Neurotol Date: 2010-07 Impact factor: 2.311
Authors: Stacey L Ishman; David J Brown; Emily F Boss; Margaret L Skinner; David E Tunkel; Rose Stavinoha; Sandra Y Lin Journal: Laryngoscope Date: 2010-11 Impact factor: 3.325
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Scott M Thomas; Wesley Burch; Sarah E Kuehnle; Robert G Flood; Anthony J Scalzo; James M Gerard Journal: Pediatr Crit Care Med Date: 2013-11 Impact factor: 3.624
Authors: Grace C Huang; Lori R Newman; Richard M Schwartzstein; Peter F Clardy; David Feller-Kopman; Julie T Irish; C Christopher Smith Journal: Acad Med Date: 2009-08 Impact factor: 6.893
Authors: Leigh V Evans; James L Morse; Cara J Hamann; Michael Osborne; Zhenqiu Lin; Gail D'Onofrio Journal: Acad Med Date: 2009-08 Impact factor: 6.893
Authors: Irene W Y Ma; Mary E Brindle; Paul E Ronksley; Diane L Lorenzetti; Reg S Sauve; William A Ghali Journal: Acad Med Date: 2011-09 Impact factor: 6.893
Authors: Ryan J Good; Danielle Mashburn; Erika Jekich; Kristen Miller; Matthew K Leroue; Jason Woods; Angela S Czaja Journal: MedEdPORTAL Date: 2022-09-27