Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: An assessor-blinded randomized controlled study.

BACKGROUND: We evaluated whether a simulation-based training with a vessel phantom improves the basic skills of a novice required for ultrasound-guided radial artery cannulation in real patients. In addition, we analysed whether repeated simulation training sets with an inter-training interval would accelerate the learning curve.
METHODS: From March 2019 to July 2019, twenty-one anesthesiology residents were randomized into either a simulation group (n = 11) or control group (n = 10). Residents performed a total of 84 ultrasound-guided radial artery cannulations in real patients. The simulation group participated in two sets of simulation training on a vessel phantom (10 sessions per set) with a one-month inter-training interval. Trainee's performance proficiency was scored using a developed checklist, and a learning curve for each training set was constructed. To evaluate the effectiveness of our training curriculum in skill transfer, each resident performed four ultrasound-guided radial artery cannulations in real patients. The primary outcome was first attempt success rate and the secondary outcome was dynamic needle-tip positioning ability in real patients.
RESULTS: The first attempt success rate and dynamic needle-tip positioning ability by ultrasound transducer were significantly higher in the simulation group than the control group (81.8% vs. 50%, P = 0.002 and 68.2% vs. 7.5%, P < 0.001, respectively). A reduced number of sessions was required to reach a plateau score on the learning curve in the repeated training set compared in the first-set (7 (5-8) vs. 3 (2-4), P = 0.003, respectively).
CONCLUSIONS: Simulation-based training using a vessel phantom effectively improved the first attempt success rate for ultrasound-guided radial artery cannulation in real patients and the dynamic needle-tip positioning ability by ultrasound transducer in novice anesthesiology residents. In addition, repeated training curriculum accelerated the learning curve for recall skill proficiency and reduced inter-individual variability for skill acquisition. CLINICAL TRIAL REGISTRATION: Clinical Research Information Service (KCT0003471, Principle investigator: Jeong Jin Min, Date of registration: 06/March/2019).

RCT Entities:   Population Interventions Outcomes

Introduction

In modern clinical practice, the use of ultrasound-guided technique is rapidly increasing for many procedures. Ultrasound-guided radial artery cannulation has grown in popularity and showed improved success rate at first attempt compared with the palpation method.[1-8] However, ultrasound-guided procedures require the ability to handle an ultrasound machine, proper identification of procedure-relevant anatomy on the ultrasound image, and a combination of visuospatial skills with hand-eye coordination.[9, 10] Therefore, an optimal training curriculum would be helpful to improve the success rate of ultrasound-guided radial artery cannulation.

In several previous studies, simulation-based ultrasound training has improved a trainee’s skill proficiency and the skill transferred well to the real-world clinical practice.[11-14] However, to our knowledge, no study has proven the effectiveness of phantom-based simulation training to improve ultrasound-guided radial artery cannulation performance with dynamic needle-tip positioning technique. In this randomized controlled trial, we evaluated whether simulation-based training with a vessel phantom model would improve a novice’s basic skills of ultrasound-guided radial artery cannulation in real patients. In addition, we analysed whether repeated simulation training sets with an inter-training interval would accelerate the learning curve.

Methods

This study was approved by the Samsung Medical Center's Institutional Review Board (SMC 2018-09-085-006, Chairperson Professor Lee Suk-Koo) and written informed consent was obtained from all subjects participating in the trial. The trial was registered prior to patient enrollment at Clinical Research Information Service (KCT0003471, Principal investigator: Jeong Jin Min, Date of registration: 06/March/2019).

Study population and randomization

From March 2019 to July 2019, anesthesiology residents (1–3 training years) with no experience in ultrasound-guided radial artery cannulation or simulation-based ultrasound-guided phantom training were enrolled in the study. Residents were randomly assigned to either a simulation group (n = 11) or a control group (n = 10) using sealed opaque envelopes after stratification based on training year as an anesthesiology resident (Table 1).

Table 1

Baseline characteristics of anesthesiology residents.

	Simulation group (N = 11)	Control group (N = 10)	P
Training year (year, 1 / 2 / 3)	4 / 4 / 3	4 / 3 / 3
Palpated artery cannulation (adult, times)
< 50 / 50–100 / ≥ 100	1 / 2 / 8	1 / 3 / 6	0.611
Palpated artery cannulation (pediatric, times)
None / < 5 / 5–10 / ≥ 10	4 / 3 / 1 / 3	4 / 2 / 2 / 2	0.912
US-guided central line cannulation (adult, times)
None / 1–10 / 10–50 / > 50	0 / 3 / 3 / 5	1 / 2 / 3 / 4	0.765

Values are presented as numbers.

To evaluate the effectiveness of simulation training in real clinical practice, adult patients who underwent elective surgeries requiring arterial blood pressure monitoring were enrolled. Patients with a wound near the insertion site, abnormal vascular circulation of the hand (a satisfactory modified Allen test result was ascertained), signs of skin infection, a history of radial artery cannulation within 1 month, and a history of peripheral artery disease were excluded. Independent investigator (E. J. Oh) enrolled the residents and patients for the study and assigned to intervention.

Equipment

A Blue phantom paediatric four-vessel ultrasound training block model (CAE healthcare®, Sarasota, FL, USA) was used for simulation training. This phantom model was developed for clinician training in psychomotor skills associated with ultrasound vessel cannulation. The model contains four branching blood vessels of various sizes ranging from two to six millimeter. Simulation training was performed on the two millimeter artificial vessel. During the study, all ultrasound procedures were performed with a 13- to 6-MHz linear transducer (Sonosite® M-turbo L25x transducer, Sonosite® Inc., Bothell, WA, USA). Sterile ultrasound transducer covers and sterile ultrasound gels were also used for ultrasound procedures in real patients.

Study protocol

The study flow in both groups is presented in Fig 1.

Fig 1

Flow diagram of study drawn in CONSORT.

Step 1. Pre-lecture for background knowledge in all residents

Background knowledge regarding ultrasound machine manipulation and ultrasound-guided vascular approach was standardised by a lecture given to residents. The session consisted of one video clip and an oral lecture. The video clip covered knowledge of the ultrasound machine and basic procedural skills of ultrasound-guided radial artery cannulation.[15] The oral lecture was about basic anatomy, preparation of the radial artery insertion site, procedural skills such as out-of-plane needling and transducer manipulation, and research regarding the clinical usefulness of ultrasound-guided radial artery cannulation. After the lecture, all participating residents took a mini quiz to verify that they understood the basic knowledge of the lecture (S1 Table). Residents who scored more than 9 of 10 in the mini quiz proceeded to the next step. All participants scored more than 9 of 10 and were enrolled to the study.

Step 2. Simulation-based training in simulation group vs. Standard curriculum in control group

The simulation group participated in two simulation training sets with a one-month interval between the sets. Each set consisted of ten simulation training sessions for ability to operate an ultrasound machine, manipulate an ultrasound transducer, and dynamic needle-tip positioning technique on a two millimeter vessel branch in the Blue phantom ultrasound training block (Fig 2). The dynamic needle-tip positioning technique is an ultrasound-guided cannulation technique with an out-of-plane approach. The angiocatheter needle is advanced while continuously tracing the needle-tip on the ultrasound monitor. Once the hyperechoic dot is visualized between the skin and the artery, the ultrasound transducer is slid slightly in the proximal direction until the hyperechoic dot disappears from the ultrasound monitor (tip position). Subsequently, the needle is advanced until the hyperechoic dot reappears on the ultrasound monitor. This process is repeated until the needle-tip is located on and punctures the anterior wall of the artery and a blood flashback is confirmed on the catheter hub. [16]

Fig 2

(A) Simulation training on Blue phantom paediatric 4 vessel ultrasound training block model. (B) Ultrasound image of the needle tip (hyperechoic dot, white arrow), located at midline of artificial vessel anterior wall. Out-of-plane method (short axis method).

Learning objectives were pre-defined as 1) Skill acquirement required in ultrasound machine manipulation for optimal cannulation condition and 2) Skill acquirement to trace the needle-tip in continuous motion with the ultrasound transducer during the procedure (dynamic needle-tip positioning technique) as a major performance milestone. Each simulation session was performed as individual training (1 tutor to 1 participant) for 5 days. The simulation tutor (E. J. Oh) is a skilled anesthesiologist with more than 200 experiences in ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning. In particular, more than half of the experiences of E. J. Oh in ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning were done in children under two years old, who are expected to have procedural difficulties. Also, each year, the tutor has been teaching dynamic needle tip-positioning skills with verbal assistance and skill demonstration as part of the standard curriculum in our tertiary medical center. The tutor gave procedural feedback at the end of each simulation and was not involved in the assessment. All participant simulation procedures were video recorded. After completing two simulation training sets, residents in the simulation group were surveyed on changes in self-confidence for performing the procedure according to a 5-point scale (S2 Table).

An independent investigator (J. J. Min) who was blinded to participant group allocation, reviewed the recorded video clips and assessed participant performance level scores using a pre-developed checklist (S3 Table). This blinded investigator has more than 500 experiences in ultrasound-guided radial arterial cannulation using the dynamic needle tip positioning technique and worked as a tutor in our medical center on practical skills and aspects of performing ultrasound-guided vascular cannulation. Our checklist was developed based on the American Society of Echocardiography and the Society of Cardiovascular Anaesthesiologists recommended training objectives for ultrasound-guided vascular cannulation.[10] The checklist included ultrasound skills, procedure proficiency, and whether a participant could dynamically position the needle-tip (hyperechoic dot) on the ultrasound image until puncturing the anterior wall of the vessel branch inside the block. Each question was scored as either 1 (performed correctly) or 0 (performed incorrectly) for 17 total questions.

The participants in the control group watched a video clip including operating an ultrasound machine, ultrasound transducer manipulation, and dynamic needle-tip positioning technique The video clip was provided to all participants in the control group, allowing participants to watch whenever they wanted over a month. In addition, during this period the participants in the control group also observed the actual ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning technique by a skilled researcher (E. J. Oh) in more than 10 real patients. This is a standard curriculum for training in ultrasound-guided radial artery cannulation in our medical center.

Step 3. Evaluation of effectiveness of simulation-based training for skill transfer into clinical practice

One month after study enrolment (when the second set was completed in the simulation group), all residents performed four ultrasound-guided radial artery cannulations in adult patients each. Before the procedure, the radial artery image was recorded to measure artery size, subcutaneous depth, and any anomalies. All radial artery cannulations were performed using an out-of-plane method, as residents were trained. The cannulation performance of each participant and the ultrasound monitor images were video recorded on one screen. After each performance, a blind assessor (J. J. Min) independently reviewed the video clip and scored the performance level score using the same checklist as during the simulation training. Procedure time measurement started when the ultrasound transducer contacted the skin and ended when the arterial waveform was confirmed on the monitor. Procedure time was limited to five minutes on the study protocol. If a procedure took more than five minutes, it was considered as a failure.

The primary outcome was to compare the first attempt success rate of ultrasound-guided radial artery cannulation in real patients between the two groups. It was considered as a cannulation attempt whenever the skin was newly punctured or when a blood flashback on the angiocatheter hub was confirmed. If blood flashback was not seen, the re-directioning process of the needle inside the subcutaneous space was defined as a single cannulation attempt. Secondary outcomes included performance level score defined by the checklist score, a participant’s dynamic needle-tip tracing ability, procedure time, and total number of attempts. The dynamic needle-tip tracing ability was evaluated based on whether the participant advanced the angiocatheter needle while continuously tracing the needle tip on the ultrasound monitor (successful acquisition of the dynamic needle-tip tracing ability) or identified the hyperechoic dot once during the whole performance (visualized at least once without continuous tracing).

Statistical analysis

The sample size of real patients was predetermined according to the difference in first attempt success rate between groups. Because there was no previous study on the effect of simulation training in ultrasound-guided radial artery cannulation, sample size was calculated based on the previously reported first attempt success rate in ultrasound-guided radial artery cannulation. The first attempt success rate was 65% in a previous study,[17] while a pilot study in our center involving only novices in ultrasound-guided radial artery cannulation had a first attempt success rate of 35%. Assuming a difference in means of 30% in first attempt success rate between the simulation group and the control group, the necessary minimum sample size to achieve the desired power of 0.8 and alpha error of 0.05 was 80 artery cannulations (40 per group). To account for a 5% drop out rate, four additional cannulations were recruited. Therefore, the first attempt success rate was measured in a total of eighty-four artery cannulations. Since this study consisted of 21 anesthesiology residents in our tertiary academic medical center, four real patients per resident were recruited. All ultrasound-guided radial artery cannulation on real patients were considered as separate cases.

Statistical analysis was executed using SAS version 9.4 (SAS Institute, Cary, NC) and R 3.4.4 (Vienna, Austria; http://www.R-project.org/). The chi-square test was used to compare the first attempt success rate and the dynamic needle-tip position ability between the two groups. Wilcoxon-signed rank sum test was used for continuous variables measured as secondary outcomes (procedure level score, procedure time, and total attempts). In addition, the learning curve of each simulation set was derived based on procedure level scores using a power model formula of Y = a·Xb, where Y represents procedure level score at number of attempt; X represents number of attempt; a represents the time required to produce the first unit of output; and b represents the slope of the learning curve (rate of improvement) when plotted on logarithmic scale.[18, 19] The slopes of the two learning curves were compared by testing the slope of the learning curve with respect to the differences between the two simulation data. P < 0.05 was considered significant.

Results

Twenty-one anesthesiology residents (11 residents in simulation group and 10 residents in control group) participated in and completed the study. Training year, baseline clinical experience of palpated radial artery cannulation, and ultrasound-guided central line cannulation were all comparable between the two groups (Table 1).

Simulation-based training in simulation group

Learning curves in the simulation group are presented in Fig 3. Compared to the learning curve of the first training set, the learning curve of the repeated training set started with higher performance level score and required fewer training sessions until reaching a plateau score. Therefore, the slopes of the two learning curves differed significantly (0.19 for first training set vs. 0.03 for repeated training set, P < 0.001), and the repeated training set showed a narrower confidence interval range reflecting reduced inter-individual variability (1.04 for first training set vs. 0.35 for repeated set) (Fig 3). However, all residents eventually reached full marks based on the performance level checklist within 10 simulation sessions in each set. The median number of sessions required to dynamically position the needle-tip was 4 (3–6) sessions in the first training set and significantly decreased to 1 (1–1) sessions in the second training set (P = 0.005). The average number of sessions required to reach the plateau in participant performance level score was 7 (5–8) in the first training set and was reduced to 3 (2–4) sessions in the second training set (P = 0.003).

Fig 3

Learning curves for first simulation training set (blue line) and second simulation training set (red line) after one-month inter-training interval.

CI, confidence interval.

After each simulation training set, all participants self-reported their changes in confidence for ultrasound-guided vascular cannulation performance. Ten of eleven participants reported that they sufficiently improved in self-confidence after the first simulation training set (S2 Table). After completing two sets of simulation training, all 11 participants in the simulation group responded that they were confident enough about the procedure.

Evaluation of simulation-based skill transfer into clinical practice

Each resident performed four ultrasound-guided radial artery cannulations on real patients. The age of allocated real patients in the control group were significantly younger than the simulation group (52.9 ± 14.4 vs. 61.0 ± 15.2, P = 0.015, respectively). However, the gender distribution and the radial artery characteristics in real patients were comparable between the groups (Table 2). In addition, radial artery diameters in patients were similar to those of vascular branches inside the Blue phantom model used during the simulation training.[20]

Table 2

Patients characteristics and their radial arterial characteristics.

	Simulation group (N = 44)	Control group (N = 40)	P
Age (years)	61.0 ± 15.2	52.9 ± 14.4	0.015
Male sex	22/44 (50.0)	19/40 (47.5)	0.819
Height (kg)	160.9 ± 8.8	162.1 ± 10.1	0.601
Weight (cm)	65.8 ± 12.5	62.9 ± 12.9	0.291
BMI (kg/m2)	25.4 ± 4.4	23.8 ± 3.6	0.074
Systolic blood pressure (mmHg)	125 (106, 142)	120 (107, 132)	0.516
Pulse pressure (mmHg)	39 (32, 57)	46 (32, 62)	0.446
Radial artery subcutaneous depth (cm)	0.34 ± 0.13	0.38 ± 0.16	0.168
Radial artery cross sectional area (cm2)	0.05 ± 0.02	0.05 ± 0.03	0.103

Values are presented as mean ± S.D. or median (IQR). Male sex is presented as numbers (proportions). BMI = Body mass index.

The detailed procedural data in real patients are presented in Table 3. The first attempt success rate in real patients was significantly higher in the simulation group compared to the control group (81.8% vs. 50.0%, P = 0.002). Among successful first attempt cases, the average procedure level score was also higher in the simulation group compared to the control group (16.1 ± 1.2 vs. 14.4 ± 1.4, P < 0.0001). Even for successful first attempt cases, participants of the control group failed to score points on questions such as whether the participant punctured the midline of the radial artery anterior wall or slid the ultrasound transducer in continuous motion while confirming the ultrasound image of the needle-tip (Dynamic needle-tip positioning ability). In six successful first attempt cases of the control group, the study participant was unable to visualise the ultrasound image of the needle-tip even once (S4 Table). The number of attempts performed until success was also higher in the control group than in the simulation group (Table 3, P = 0.017).

Table 3

Ultrasound-guided radial arterial cannulation performance data in real patients.

	Simulation group (N = 44)	Control group (N = 40)	P
First attempt success rate	36/44 (81.8)	20/40 (50.0)	0.002
Performance level score	15.8 ± 1.4	13.6 ± 1.7	< 0.0001
Successful first attempt cases	16.1 ± 1.2	14.4 ± 1.4	<0.0001
Number of attempts to success 1 / 2 / 3	36(81.8) / 7(15.9) / 1(2.3)	20(54.1) / 16(43.2) / 1(2.7)	0.017
Hyperechoic dot (Tip of the needle)
Visualize at least once	44/44 (100.0)	28/40 (70.0)	<0.0001
Dynamically positioning ability	30/44 (68.2)	3/40 (7.5)	< 0.0001
Procedure time, sec	65.5 (50.5–150.0)	134.5 (53.0–226.0)	0.082

Values are presented as mean ± S.D. or median (IQR). First attempt success rate, number of attempts to success, white dot (dynamically positioning ability) are presented as numbers (proportions). Three cases in control group, which failed to successfully place angiocatheter within five minutes, were excluded from the number of attempts to success analysis.

During the cannulation procedures, all residents in the simulation group visualised the ultrasound image of the needle-tip at least once, while only 70.0% of residents in the control group identified the ultrasound image of the needle-tip at least once (P < 0.0001). The proportion of residents who were able to dynamically position the needle-tip during cannulation was significantly higher in the simulation group compared to the control group (68.2% vs. 7.5%, P < 0.0001, respectively). However, there was no significant difference in procedure time between the two groups (P = 0.082) (Table 3). Also there were no artery cannulation related complications during the study.

Discussion

In this randomized controlled trial, we compared the simulation and the control group to demonstrate that training through practice in controlled situations, such as simulation, is more effective in transferring skills to actual performance than training through observation. Two sets of simulation-based training significantly improved the first attempt success rate and the dynamic needle-tip positioning ability on ultrasound-guided radial artery cannulation in real patients compared to the control group residents who were trained through watching a video clip on ultrasound-guided radial artery cannulation and observing actual performances of a skilled researcher. In addition, repeated training curriculum accelerated the learning curve for recall skill proficiency and reduced inter-individual variability for skill acquisition among residents in the simulation group.

The simulation-based training curriculum of the present study was constructed to include ten simulation sessions in each training set. All residents in the simulation group reached the plateau in performance level score before ten simulation sessions were completed. Ten simulation sessions were sufficient to demonstrate improvement in skill proficiency, which is in agreement with the findings of previous study suggesting range of six to ten simulation training sessions to demonstrate competence in ultrasound-guided radial artery cannulation.[10]

In addition, our curriculum was designed with repeated training sets and an one month inter-training interval based on previous results that skill acquisition is influenced by training distribution.[21] Residents retained the skill better when they were taught in a repeated training manner with a period of rest between sessions compared to a mass training including all training at a single session.[22, 23] This is thought to be a consequence of the motion neural process that continues during the rest period between training sessions.[24] In the present study, psychomotor skills were retained after the inter-training interval between simulation sets. Skill proficiency in ultrasound-guided cannulation of the vascular model improved as simulation sessions progressed in both training sets. However, a resident’s procedure level score reached the plateau after fewer training sessions and inter-individual variability decreased in the second simulation set. This result supports the importance of repeating simulation training with interval.

The effectiveness of simulation training curriculum to transfer skill into clinical practice was assessed through ultrasound-guided radial artery cannulation performance in real adult patients. Residents in the simulation group showed a similar first attempt success rate as the previously reported success rates of 71.4% in experienced cardiac anesthesiologists [25] and 83% in faculty anesthesiologists. On the other hand, the residents in the control group showed a similar first attempt success rates of 53% in anesthesia trainees with less than five experiences in ultrasound-guided radial artery cannulation [26] and 62% in anesthetists with ultrasound-guided central vein insertion experience but novice to ultrasound-guided radial artery cannulation.[27] Among successful first attempt cases, the control group showed less success in terms of dynamic needle-tip positioning ability during the procedure, and this group achieved lower overall procedure level scores than the simulation group. It can be deduced that some of the successful cases in the control group were carried out without following the key elements of ultrasound-guided procedures emphasized in this study. The key elements, such as dynamically positioning the needle-tip or midline puncture of the radial artery anterior wall are known to increase the success rate.[28] Thus, ultrasound-guided procedures which were performed with missing the key elements may be difficult to evaluate as proficient skill acquisition of ultrasound-guided cannulation.

Unexpectedly, procedure time was not significantly different between groups. Participants in the simulation group tended to follow knowledge acquired from the simulation training, such as moving the transducer and the angiocatheter in continuous motion guided by the hyperechoic image of the needle-tip on the ultrasound monitor. Consequently, the procedure was performed more accurately, but the procedure time was longer. Moreover, procedure time varied depending on each participant’s character. Some participants were more hesitant to advance the needle even while looking at the needle-tip in real time. These individual differences make procedure time an inaccurate tool to assess proficiency.[21]

Although several studies have investigated simulation-based training for ultrasound-guided procedures, there was no single optimised training curriculum for ultrasound-guided radial artery cannulation. To the best of our knowledge, the present study is the first randomized controlled trial to verify the Blue phantom vascular block model as a simulator for ultrasound-guided radial artery cannulation and to evaluate the different forms of training on ultrasound-guided radial artery cannulation with dynamic needle-tip positioning. The strength of this study is that performance assessments were conducted in an actual workplace among real patients by recorded video clips. Based on ‘Miller’s pyramid of competence’, the workplace-based assessment results show a significant correlation with skill proficiency in clinical practice. [29] Thus, this study may provide a basis for constructing a standard simulation training curriculum for ultrasound-guided radial artery cannulation which leads to enhanced clinical performance.

This study has several limitations. First, it is limited to a single center with a small number of participating residents. The number of residents in our department is an unchangeable condition that we tried to overcome by having each participant perform four ultrasound-guided radial artery cannulations in real patients resulting in eighty-four cannulations in the study. Second, a non-validated checklist was used to score participant performance level because there are no standardised assessment tools for ultrasound-guided radial artery cannulation performance. However, checklist questionnaires were objectively constructed based on the American Society of Echocardiography and the Society of Cardiovascular Anaesthesiologists recommended training objectives for ultrasound-guided vascular cannulation. In addition, the same score was given for all questions on the checklist; however, some questions regarding the main learning points may be weighted in future studies. Fourth, the age of real patients allocated in the simulation group were significantly older compared to the control group. This age difference between the two groups may be due to random chance in a finite sample. However, factors that can be associated with catheterization failures, such as artery tortuosity, or presence of atherosclerosis, usually increase with age. [30, 31] Thus, we suspect that the influence of age even supports the effectiveness of simulation training which is the conclusion of our study. Finally, we did not observe the long-term effect of simulation-based training. The participating residents continue clinical practice every day, which may improve their procedure skills and making it difficult to evaluate the sole effect of the simulation training curriculum.

Conclusion

Simulation-based training using a Blue phantom vascular block model effectively improved the first attempt success rate for ultrasound-guided radial artery cannulation as well as dynamic needle-tip positioning technique with an ultrasound transducer scan in novice anesthesiology residents. In addition, repeated training curriculum accelerated the learning curve for recall skill proficiency and reduced inter-individual variability for skill acquisition.

Quiz for background knowledge on ultrasound-guided radial artery cannulation.

(DOCX)

Click here for additional data file.

5-point scale for assessing the change in self confidence after the simulation training set.

(DOCX)

Click here for additional data file.

Checklist for procedures of ultrasound-guided radial artery cannulation.

(DOCX)

Click here for additional data file.

The questions that failed to score points on the checklist among successful first attempt cases.

(DOCX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

23 Mar 2020

PONE-D-19-35381

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

PLOS ONE

Dear Dr. Min,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript has been assessed by two reviewers; their comments are available below.

The reviewers have raised some major items that need attention in a revision, the reviewers feel that further information should be provided on the training that each group received, and they have also requested additional clarification on the secondary outcome measured and a number of aspects of the methodology.

Could you please carefully revise the manuscript to address the comments raised by the reviewers?

We would appreciate receiving your revised manuscript by May 05 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Iratxe Puebla

Deputy Editor-in-Chief, PLOS ONE

Journal Requirements:

1.  When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

* In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

3. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for allowing me to review " Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study" by Oh et al.

The authors compared first success rate of ultrasound guided artery cannulation by resident between simulation training versus control groups. The first attempt success rate and dynamic needle-tip positioning ability by ultrasound transducer were significantly higher in the simulation group than the control group.

This is very interesting study.

My suggestions are as follows.

The authors should describe the detail of dynamic needle tip positioning in the method section referring to previous reports.

Dynamic needle-tip tracing ability as the secondary outcome is unclear. Please clarlify.

There are terms of “procedure duration time”, “procedure duration”, and “procedure time”. Please unify the expression as it is confusing.

The question 1 and 2 of S1 table seem like the same question.

Table 3

Performance level→Performance lever score in successful first attempt cases?

Why two data are described? 16 (14.5-17) and 16.1 ± 1.2

Procedure time→please unify

Figure 2B

Please point the needle tip by arrow or something.

Reviewer #2: The paper does not make it very clear what types of training are being compared? The performance of the dynamic needle tip positioning technique was evaluated but more detailed information is needed on the actual training received by the residents.

Line 126: It is not clear what the experience level of the proctors was as far as the dynamic needle tip positioning technique.

Line 132: Did the video clips that were used for grading the participants show the resident technique or the ultrasound images or both. You would need a recording of the ultrasound images to properly grade the dynamic needle tip technique

Line 138: What is the experience level of the researcher and what ultrasound cannulation technique was he/she teaching the residents

Line 152: Was the timing of the procedure started upon contact of the ultrasound probe and then skin or upon puncturing of the skin by the needle?

Line 190: There was a statistically significant difference in the median ages of the patients in the simulation and the control group. This should be mentioned in the results section and in the discussion

Line 190: Gender has been shown to be an important factor affecting success of radial artery cannulation. This should have been looked at in the baseline characteristics table

Line 286: To draw a conclusion that the control group achieved radial arterial cannulation by chance is incorrect. They simply used a different ultrasound technique

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

20 Apr 2020

April 07, 2020

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

Dear Editor

We thank you and the reviewers of Plos One for taking the time to review our article. We made corrections and clarifications to the manuscript to address the reviewer comments. All authors have read and approved the revised manuscript. The changes are summarized below.

Point-to-Point Responses to The First Reviewer’s comments

Reviewer #1

Thank you for allowing me to review " Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study" by Oh et al.

The authors compared first success rate of ultrasound guided artery cannulation by resident between simulation training versus control groups. The first attempt success rate and dynamic needle-tip positioning ability by ultrasound transducer were significantly higher in the simulation group than the control group. This is very interesting study.

1. The authors should describe the detail of dynamic needle tip positioning in the method section referring to previous reports. Dynamic needle-tip tracing ability as the secondary outcome is unclear. Please clarlify.

Response: We have added the dynamic needle-tip positioning method and how we evaluated dynamic needle-tip tracing ability as a secondary outcome in the revised Methods section as follows.

“The dynamic needle-tip positioning technique is an ultrasound-guided cannulation technique with an out-of-plane approach. The angiocatheter needle is advanced while continuously tracing the needle-tip on the ultrasound monitor. Once the hyperechoic dot is visualized between the skin and the artery, the ultrasound transducer is slid slightly in the proximal direction until the hyperechoic dot disappears from the ultrasound monitor (tip position). Subsequently, the needle is advanced until the hyperechoic dot reappears on the ultrasound monitor. This process is repeated until the needle-tip is located on and punctures the anterior wall of the artery and a blood flashback is confirmed on the catheter hub. (1) ” In lines 122-128.

“The dynamic needle-tip tracing ability was evaluated based on whether the participant advanced the angiocatheter needle while continuously tracing the needle tip on the ultrasound monitor (successful acquisition of the dynamic needle-tip tracing ability) or identified the hyperechoic dot once during the whole performance (visualized at least once without continuous tracing).” In lines 176-180.

2. There are terms of “procedure duration time”, “procedure duration”, and “procedure time”. Please unify the expression as it is confusing.

Response: We appreciate your comment. We made the expression consistent as “procedure time” throughout the manuscript.

3. The question 1 and 2 of S1 table seems like the same question.

Response: Thank you for your comments. We have corrected the second question as follows.

“ 2) The clinician should place the ultrasound machine directly across from the operator to establish the best ergonomic position for the procedure.” (Supplementary Table 1).

4. In Table 3, Performance level→Performance level score in successful first attempt cases?

Why two data are described? 16 (14.5-17) and 16.1 ± 1.2

Response: After checking the data distribution normality, we have revised the performance level score presentation for the mean ± standard deviation (S.D.) in revised Table 3 as follows:

Simulation group (N=44) Control group (N=40) P-value

Performance level score

Successful first attempt cases 15.8 ± 1.4

16.1 ± 1.2 13.6 ± 1.7

14.4 ± 1.4 < 0.001

<0.001

5. In Figure 2B, Please point the needle tip by arrow or something.

Response: Thank you for your comments. We have added a new white arrow to Figure 2B to point to the needle tip.

Reviewer #2

1. The paper does not make it very clear what types of training are being compared? The performance of the dynamic needle tip positioning technique was evaluated but more detailed information is needed on the actual training received by the residents.

Response: We appreciate the reviewer for this critical comment. Currently, in the information era, there are numerous educational materials/videos available online that a trainee could view to obtain knowledge about an invasive procedure before performing it on real patients. In comparison to just knowing how to do it by learning through various educational materials or observing the expert’s practices (control group), we aimed to evaluate the effectiveness of simulation-based learning in which trainees actually practice the procedure several times before performing it on real patients (simulation group). Therefore, we compared the simulation group and the control group to demonstrate that training through practice in controlled situations, such as simulation training, is more effective in transferring skills to actual performance than training through observation. In our study, the simulation group underwent two sets of simulation training of ultrasound-guided radial artery cannulation on a phantom vascular model. On the other hand, the control group learned the technique through education materials/video clips or observing actual clinical performances by a skilled researcher.

Specifically, the contents of the ultrasound-guided radial artery training were ability to operate an ultrasound machine, ultrasound transducer manipulation without compressing the procedure relevant anatomy, hand-eye coordination, and dynamic needle-tip positioning technique.

After the two different forms of training in each group, we evaluated the effectiveness of education type by first attempt success rate in ultrasound-guided radial artery cannulation among real patients. Based on ‘Miller’s pyramid for assessing clinical competence’, the performance assessment was carried out at the actual workplace among real patients. We video recorded the performance and the video clip was assessed using a pre-developed checklist. The pre-developed checklist included ultrasound skills, procedure proficiency, and whether a participant could dynamically position the needle-tip (hyperechoic dot) on the ultrasound image until puncturing the anterior wall of the radial artery. We have clarified the training method in the revised Discussion section.

“In this randomized controlled trial, we compared the simulation and control groups to demonstrate that training through practice in controlled situations, such as simulation, is more effective in transferring skills to actual performance than training through observation. Two sets of simulation-based training significantly improved the first attempt success rate and the dynamic needle-tip positioning ability on ultrasound-guided radial artery cannulation in real patients compared to the control group residents who were trained through watching a video clip on ultrasound-guided radial artery cannulation and observing actual performances of a skilled researcher.” In lines 270-276.

“To the best of our knowledge, the present study is the first randomized controlled trial to verify the Blue phantom vascular block model as a simulator for ultrasound-guided radial artery cannulation and to evaluate the different forms of training on ultrasound-guided radial artery cannulation with dynamic needle-tip positioning. A strength of this study is that performance assessments were conducted in an actual workplace among real patients by recorded video clips. Based on ‘Miller’s pyramid of competence’, the workplace-based assessment results show a significant correlation with skill proficiency in clinical practice.(2) Thus, this study may provide a basis for constructing a standard simulation training curriculum for ultrasound-guided radial artery cannulation which leads to enhanced clinical performance.” In lines 319-327.

Also, we have added detailed training content to the revised Methods section.

“Each set consisted of ten simulation training sessions for ability to operate an ultrasound machine, manipulate an ultrasound transducer, and dynamic needle-tip positioning technique on a two millimeter vessel branch in the Blue phantom ultrasound training block” in lines 119-121 and “Participants in the control group watched a video clip including operating an ultrasound machine, ultrasound transducer manipulation, and dynamic needle-tip positioning technique, and they observed the actual performances of a skilled researcher (E. J. Oh) on real patients for a month.” in lines 151-153.

2. It is not clear what the experience level of the proctors was as far as the dynamic needle tip positioning technique.

Response: The blinded proctor (J. J. Min) in our study is an expert in cardiac anesthesiology with more than 500 experiences in ultrasound-guided vascular cannulation using the dynamic needle tip positioning technique. She has conducted a study comparing ultrasound-guided with palpation-guided techniques for radial arterial catheterization in infants (3) and studies related to ultrasound-guided vascular cannulation using the dynamic needle-tip positioning technique. We have added the information on proctor experience level to the revised Methods section.

“An independent investigator (J. J. Min) who was blinded to participant group allocation, reviewed the recorded video clips and assessed participant performance level scores using a pre-developed checklist (S3 Table). This blinded investigator has more than 500 experiences in ultrasound-guided radial arterial cannulation using the dynamic needle tip positioning technique and worked as a tutor in our medical center on practical skills and aspects of performing ultrasound-guided vascular cannulation.” In lines 139-144.

3. Did the video clips that were used for grading the participants show the resident technique or the ultrasound images or both. You would need a recording of the ultrasound images to properly grade the dynamic needle tip technique.

Response: The participating resident’s procedure technique and the ultrasound monitor image were simultaneously recorded on one screen with a video camera. Therefore, the blinded assessor (J. J. Min) was able to properly score if the residents performed dynamic needle-tip positioning accurately. We revised the indicated sentences in the Methods section.

“The cannulation performance of each participant was video recorded.” to “The cannulation performance of each participant and the ultrasound monitor images were video recorded on one screen.” In lines 163-165.

4. What is the experience level of the researcher and what ultrasound cannulation technique was he/she teaching the residents.

Response: The simulation tutor (E. J. Oh) in our study is a skilled anesthesiologist with more than 200 experiences in ultrasound-guided radial artery cannulation and more than 400 experiences in ultrasound-guided central venous catheter insertion. She has conducted studies related to ultrasound-guided vascular cannulation using the dynamic needle-tip positioning technique and also performed individual training (1 tutor to 1 participant) in a tertiary medical center. The individual training covered practical skills and aspects of performing ultrasound-guided vascular cannulation, i.e., combination of visuospatial skills with hand-eye coordination, dynamic needle-tip positioning technique, and two puncture techniques (anterior/posterior). We have added more details about this to the revised Methods section as follows:

“The dynamic needle-tip positioning technique is an ultrasound-guided cannulation technique with an out-of-plane approach. The angiocatheter needle is advanced while continuously tracing the needle-tip on the ultrasound monitor. Once the hyperechoic dot is visualized between the skin and the artery, the ultrasound transducer is slid slightly in the proximal direction until the hyperechoic dot disappears from the ultrasound monitor (tip position). Subsequently, the needle is advanced until the hyperechoic dot reappears on the ultrasound monitor. This process is repeated until the needle-tip is located on and punctures the anterior wall of the artery and a blood flashback is confirmed on the catheter hub.(1)” in lines 122-128 and “The simulation tutor (E. J. Oh) is a skilled anesthesiologist with more than 200 experiences in ultrasound-guided radial artery cannulation.” In lines 133-134.

5. Was the timing of the procedure started upon contact of the ultrasound probe and then skin or upon puncturing of the skin by the needle?

Response: The “procedure time” measurement started when the ultrasound transducer contacted the skin and ended when the arterial waveform was confirmed on the monitor. We have clarified this in the revised manuscript.

“Procedure time measurement started when the ultrasound transducer contacted the skin and ended when the arterial waveform was confirmed on the monitor.” In lines 167-168, and “It was considered as a cannulation attempt whenever the skin was newly punctured or when a blood flashback on the angiocatheter hub was confirmed. If blood flashback was not seen, the re-directioning process of the needle inside the subcutaneous space was defined as a single cannulation attempt.” In lines 171-174.

6. There was a statistically significant difference in the median ages of the patients in the simulation and the control group. This should be mentioned in the results section and in the discussion section.

Response: As the reviewer suggested we have added information on real patient age to the revised Results section and in the revised Discussion section.

“The age of allocated real patients in the control group were significantly younger than the simulation group (52.9 ± 14.4 vs. 61.0 ± 15.2, P = 0.015, respectively).” In lines 232-234.

“Fourth, the age of real patients allocated in the simulation group were significantly older compared to the control group. This age difference between the two groups may be due to random chance in a finite sample. However, factors that can be associated with catheterization failures, such as artery tortuosity, or presence of atherosclerosis, usually increase with age. (4, 5) Thus, we suspect that the influence of age even supports the effectiveness of simulation training which is the conclusion of our study.” In lines 337-342.

7. Gender has been shown to be an important factor affecting success of radial artery cannulation. This should have been looked at in the baseline characteristics table.

Response: Thank you for your comments. We are aware that gender affects the vascular access ability and failure rate for arterial catheter insertion. (6, 7) In our study, the distribution of gender did not differ between the simulation group and the control group (P = 0.819). We have added the data on gender distribution of the real patient participants to the revised Table 2 as follows and in the revised Results section.

Table 2. Patients characteristics and their radial arterial characteristics.

Simulation group (N=44) Control group (N=40) P

Male sex 22/44 (50.0) 19/40 (47.5) 0.819

“However, the gender distribution and the radial artery characteristics in real patients were comparable between the groups (Table 2).” In lines 234-235.

8. To draw a conclusion that the control group achieved radial arterial cannulation by chance is incorrect. They simply used a different ultrasound technique.

Response: We fully agree with your suggestion. We have changed the description in the revised Discussion section according to your comments.

“However, among successful first attempt cases, the control group showed less success in terms of dynamic needle-tip positioning ability during the procedure, and this group achieved lower overall procedure level scores than the simulation group. It can be deduced that some of the successful cases in the control group were carried out without following the key elements of ultrasound-guided procedures emphasized in this study. The key elements, such as dynamically positioning the needle-tip or midline puncture of the radial artery anterior wall are known to increase the success rate. (8) Thus, ultrasound-guided procedures which were performed with missing the key elements may be difficult to evaluate as proficient skill acquisition of ultrasound-guided cannulation.” In lines 301-308.

References

1. Clemmesen L, Knudsen L, Sloth E, Bendtsen T. Dynamic needle tip positioning - ultrasound guidance for peripheral vascular access. A randomized, controlled and blinded study in phantoms performed by ultrasound novices. Ultraschall in der Medizin (Stuttgart, Germany : 1980). 2012;33(7):E321-e5.

2. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. The Lancet. 2001;357(9260):945-9.

3. Min JJ, Tay CK, Gil NS, Lee JH, Kim S, Kim CS, et al. Ultrasound-guided vs. palpation-guided techniques for radial arterial catheterisation in infants: A randomised controlled trial. European journal of anaesthesiology. 2019;36(3):200-5.

4. Lee D, Kim JY, Kim HS, Lee KC, Lee SJ, Kwak HJ. Ultrasound evaluation of the radial artery for arterial catheterization in healthy anesthetized patients. Journal of clinical monitoring and computing. 2016;30(2):215-9.

5. Crow MT. Atherosclerosis and the vascular biology of aging. Aging Clinical and Experimental Research. 1996;8(4):221-34.

6. Santen SA, Yamazaki K, Holmboe ES, Yarris LM, Hamstra SJ. Comparison of Male and Female Resident Milestone Assessments During Emergency Medicine Residency Training: A National Study. Acad Med. 2020;95(2):263-8.

7. Eisen LA, Minami T, Berger JS, Sekiguchi H, Mayo PH, Narasimhan M. Gender disparity in failure rate for arterial catheter attempts. Journal of intensive care medicine. 2007;22(3):166-72.

8. Takeshita J, Yoshida T, Nakajima Y, Nakayama Y, Nishiyama K, Ito Y, et al. Dynamic Needle Tip Positioning for Ultrasound-Guided Arterial Catheterization in Infants and Small Children With Deep Arteries: A Randomized Controlled Trial. Journal of cardiothoracic and vascular anesthesia. 2019;33(7):1919-25.

We hope the revised manuscript will better meet your requirements for publication. We thank the editor and the reviewers of Plos One once again for their constructive review of our paper.

Warm regards,

Jeong Jin Min, M.D., Ph.D.

Samsung Medical Center,

Sungkyunkwan University School of Medicine

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

19 May 2020

PONE-D-19-35381R1

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

PLOS ONE

Dear Dr Jeong Jin Min,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR:

The authors adressed all of my concern (as reviewer 1). They still should adress the minor comment of reviewer 2.

==============================

We would appreciate receiving your revised manuscript by Jul 03 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Jun Takeshita, M.D., Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Dear Editor,

Thank you for the opportunity to review this manuscript. The authors have done a great job or responding to the past reviewer comments. My additional comments are:

1. Line 134-135 The authors responded by clarifying the experiences of E.J. Oh but because this is a study evaluating dynamic needle tip positioning, they should also add what the dynamic needle tip positioning experience of this provider was.

2. Line 153-154 Because this is an evaluation of simulation training vs other training methods, it should be clear what the control group experience was. How many procedures on average did they participants in the control group witness? Were these witnessed experiences all dynamic needle tip positioning or not?

3. Line 165-166 It seems like this line was edited but both the original and edited sentences remained in the manuscript. Please correct this

4. Line 212 "was all comparable" should read "were all comparable"

5. Supplement table 4. It is not clear how to read this table. Because these are all questions, maybe it would be better to create "yes" and "no" columns for the control and the simulation group and then fill in the number of responses

6. Line 284. The word "enhance" does not fit in the sentence well. Consider using enhancement, improvement, etc

7. Line 302. The 50% first pass success rate in novices is not higher than expected. Ueda et al in A randomized controlled trial of radial artery cannulation guided by doppler vs palpation vs ultrasound, achieved a first pass of 53% in anesthesia trainees with < 5 USG radial artery cannulations. levin et al in Use of ultrasound guidance in the insertion of radial artery catheters, Crit Care Med 2003, achieved a first pass success rate of 62% in operators with ultrasound guided central venous catheter experience but novices in radial artery cannulation with ultrasound guidance

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

26 May 2020

May, 26, 2020

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

Dear Editor

We thank you and the reviewers of Plos One for taking the time to review our article once again. We made corrections and clarifications to the manuscript to address the reviewer comments. All authors have read and approved the revised manuscript. The changes are summarized below.

Reviewer #2

1. Line 134-135. The authors responded by clarifying the experiences of E.J. Oh but because this is a study evaluating dynamic needle tip positioning, they should also add what the dynamic needle tip positioning experience of this provider was.

Response: As the reviewer suggested we have added information on dynamic needle tip positioning experience of the simulation tutor (E. J. Oh) in our study. The simulation tutor (E. J. Oh) has more than 200 experiences in ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning and have conducted a study on ultrasound-guided vascular cannulation using the dynamic needle-tip positioning technique in children under two year old. Also, each year, the tutor has been teaching dynamic needle tip-positioning skills with verbal assistance and skill demonstration as part of the standard curriculum in our tertiary medical center. We have added more details about this to the revised Methods section as follows:

“The simulation tutor (E. J. Oh) is a skilled anesthesiologist with more than 200 experiences in ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning. In particular, more than half of experiences of E. J. Oh in ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning were done in children under two years old, who are expected to have procedural difficulties. Also, each year, the tutor has been teaching dynamic needle tip-positioning skills with verbal assistance and skill demonstration as part of the standard curriculum in our tertiary medical center.” In lines 133-139.

2. Line 153-154. Because this is an evaluation of simulation training vs other training methods, it should be clear what the control group experience was. How many procedures on average did they participants in the control group witness? Were these witnessed experiences all dynamic needle tip positioning or not?

Response: We fully agree with your comments. The participants in the control group watched a video clip including operating an ultrasound machine, ultrasound transducer manipulation, and dynamic needle-tip positioning technique. The video clip was provided to all participants in the control group, allowing participants to watch whenever they wanted. In addition, the participants in the control group observed the actual performances of a skilled researcher (E. J. Oh) in more than 10 real patients over a month. We have added the information on proctor experience level to the revised Methods section.

“The participants in the control group watched a video clip including operating an ultrasound machine, ultrasound transducer manipulation, and dynamic needle-tip positioning technique. The video clip was provided to all participants in the control group, allowing participants to watch whenever they wanted over a month. In addition, during this period the participants in the control group also observed the actual ultrasound-guided radial artery cannulation using the dynamic needle tip-positioning by a skilled researcher (E. J. Oh) in more than 10 real patients.” In lines 156-161.

3. Line 165-166. It seems like this line was edited but both the original and edited sentences remained in the manuscript. Please correct this

Response: We appreciate the reviewer for the attention. As the reviewer pointed out we have erased the duplicated sentence below.

“The cannulation performance of each participant was video recorded.” to “ ” In line 171.

4. Line 212. "was all comparable" should read "were all comparable"

Response: We have revised the sentence as the reviewer pointed out in the revised Result section.

“Training year, baseline clinical experience of palpated radial artery cannulation, and ultrasound-guided central line cannulation were all comparable between the two groups (Table 1).” In lines 216-218.

5. Supplement table 4. It is not clear how to read this table. Because these are all questions, maybe it would be better to create "yes" and "no" columns for the control and the simulation group and then fill in the number of responses

Response: As the reviewer suggested we have revised the supplement table 4. Please refer to the attached file < Supplement table 4>.

6. Line 284. The word "enhance" does not fit in the sentence well. Consider using enhancement, improvement, etc.

Response: We appreciate the reviewer for the comment. We have changed the description in the revised Discussion section.

“Ten simulation sessions were sufficient to demonstrate improvement in skill proficiency, which is in agreement with the findings of previous study suggesting range of six to ten simulation training sessions to demonstrate competence in ultrasound-guided radial artery cannulation.” In lines 288-291.

7. Line 302. The 50% first pass success rate in novices is not higher than expected. Ueda et al in A randomized controlled trial of radial artery cannulation guided by doppler vs palpation vs ultrasound, achieved a first pass of 53% in anesthesia trainees with < 5 USG radial artery cannulations. levin et al in Use of ultrasound guidance in the insertion of radial artery catheters, Crit Care Med 2003, achieved a first pass success rate of 62% in operators with ultrasound guided central venous catheter experience but novices in radial artery cannulation with ultrasound guidance

Response: We appreciate the reviewer for the comment. We have changed the description in the revised Discussion section according to your comments.

“On the other hand, the residents in the control group showed a similar first attempt success rates of 53% in anesthesia trainees with less than five experiences in ultrasound-guided radial artery cannulation (1) and 62% in anesthetists with ultrasound-guided central vein insertion experience but novice to ultrasound-guided radial artery cannulation.(2) Among successful first attempt cases, the control group showed less success in terms of dynamic needle-tip positioning ability during the procedure, and this group achieved lower overall procedure level scores than the simulation group.” In lines 307-313.

Reference

1. Ueda K, Bayman E, Johnson C, Odum N, Lee JJ. A randomised controlled trial of radial artery cannulation guided by Doppler vs palpation vs ultrasound. Anaesthesia. 2015;70(9):1039-44.

2. Levin PD, Sheinin O, Gozal Y. Use of ultrasound guidance in the insertion of radial artery catheters. Critical care medicine. 2003;31(2):481-4.

We hope our additionally revised manuscript will better meet your requirements for publication. We thank the editor and the reviewer of Plos One once again for their constructive review of our paper.

Warm regards,

Jeong Jin Min, M.D., Ph.D.

Samsung Medical Center,

Sungkyunkwan University School of Medicine

Submitted filename: Response to Reviewers_2.docx

Click here for additional data file.

29 May 2020

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

PONE-D-19-35381R2

Dear Dr. Jeong Jin Min

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Jun Takeshita, M.D., Ph.D.

Guest Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

2 Jun 2020

PONE-D-19-35381R2

Simulation-based training using a vessel phantom effectively improved first attempt success and dynamic needle-tip positioning ability for ultrasound-guided radial artery cannulation in real patients: an assessor-blinded randomized controlled study

Dear Dr. Min:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jun Takeshita

Guest Editor

PLOS ONE