Evaluation of skills transfer in short-term phacoemulsification surgery training program by International Council of Ophthalmology -Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) and assessment of efficacy of ICO-OSCAR for objective evaluation of skills transfer.

Purpose: To evaluate skills transfer in short-term phacoemulsification surgery training program by International Council of Ophthalmology -Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) and assessment of efficacy of ICO-OSCAR for objective evaluation of skills transfer.
Methods: Prospective analysis of total 1300 phacoemulsification surgeries performed by 52 phacoemulsification trainees divided into three groups based on the previous cataract surgical experience. The pretraining and posttraining average mean scores, independent completion rates (ICR), and complication rates among the groups were calculated and statistically analyzed.
Results: Mean age of trainees (N = 52) was found to be 30.98 ± 2.44 years. Mean pretraining scores in the three groups were 42.8 ± 0.2, 46.6 ± 0.2, and 50.1 ± 0.1, respectively, and posttraining mean scores of the groups were 88.4 ± 0.2, 92.3 ± 0.2, and 94.3 ± 0.2, respectively, out of 100. Pretraining independent completion rates (%) were 22.1 ± 2.5, 52.1 ± 2.5, and 62 ± 3.7, respectively, and posttraining independent completion rates (%) were 74.8 ± 3, 79.7 ± 3, and 90.6 ± 3.5, respectively. Learning curves of all groups were noted to be steep. Rate of complications was within acceptable range, overall complication rate being 12.30% (160 out of 1300 cases). OSCAR scores correlate significantly (P = 0.0004) with ICR.
Conclusion: Steep learning curve for phacoemulsification can be made easy by the use of standardized rubrics, proper techniques, expert guidance, and adequate surgical volume. The previous SICS experience provides smoother learning curve. ICO-OSCAR is indicator of proficiency and provides constructive feedback and objective evaluation immediately after surgery as well as at the end of training. Short-term (1 Month) training program is an effective, efficient, and safe approach to progressively master phacoemulsification skills for trainees with SICS experience.

Phacoemulsification is the accepted standard surgery for cataract world over.[1] Surgical skills transfer is a core component of ophthalmic training. Proficiency in phacoemulsification is an essential requirement for a cataract surgeon. Learning curve for phacoemulsification is quite steep.[2] Lot of studies till now have concentrated on the visual outcome and complication rates as the measure of a successful surgery and training.[34] There are no fixed indicators to define surgical competency.[5] International Council of ophthalmology's Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) phacoemulsification is internationally valid, reliable, and objective surgical competency assessment tool.[4] This study was designed in order to objectively evaluate the skills acquisition by trainees undergoing phacoemulsification training with the help of ICO-OSCAR and study correlation between independent completion rate and ICO-OSCAR scores.

Methods

A prospective observational study evaluated groups of phacoemulsification trainees over a period of 10 months. The study was conducted at a tertiary eye care center in western India. The total number of trainees enrolled was 52. The trainees with the previous experience of at least 50 small incision cataract surgeries (SICS) performed prior to this training were admitted to this short-term phacoemulsification training program. Trainees unwilling for assessment of surgical skills transfer were planned to be eliminated from study. Permission to conduct this study was taken from Institutional ethics committee.

Written consent and biodata were collected from all the trainees. Trainees were divided in three groups A, B, and C based on the previous surgical experience.

Group A (19 Trainees): Trainees who had the experience of 50 to 500 SICS

Group B (20 Trainees): Trainees who had the experience of more than 500 SICS and less than 20 phacoemulsifications

Group C (13 Trainees): Trainees who had the experience of more than 500 SICS and more than 20 Phacoemulsifications.

A detailed preoperative evaluation was done for all patients. Wet lab training of trainees for hands-on was taken on first 3 days. Didactic lectures were given over phacodynamics, phacoemulsification surgery steps, and management of complications in first week of training. Patients having age-related cataract nuclear opacification grades 2–3 (LOCS III: Lens Opacity Classification System III) were considered for cases for trainees and complicated cases were avoided. All surgeries were performed under peribulbar anesthesia. A clear corneal incision was made with a 2.8-mm single bevel keratome. Two side port incisions were made with the help of 15-degree slit blade. After staining anterior capsule with Trypan Blue dye, a 5.5–6 mm continuous curvilinear capsulorrhexis was advised. Hydrodissection and hydrodelineation procedures were performed with balanced salt solution (BSS). Nucleotomy was performed with standard stop and chop technique. Initial trench was made and nucleus divided into two halves. Both halves were subsequently chopped and emulsified. All eyes underwent phacoemulsification using Zeiss Visalis 100 phaco machine system. The cortical matter was removed with the help of bimanual irrigation-aspiration (I-A) probe. A foldable hydrophilic acrylic single-piece intraocular lens was inserted and placed within the capsular bag. The IOL was injected with the help of manual injector. Viscoelastic substance was removed from the anterior chamber with I-A probe. Securing the incision with a 10-0 nylon suture was done, if necessary. Postoperative pinhole visual acuity was measured on day one and best-corrected visual acuity (BCVA) on day 30.

Trainees were asked to approach the trainer immediately after surgery and note down the scores for all 20 steps obtained by them as per ICO-OSCAR. If the surgery was independently performed by the trainee, it was separately noted in study proforma. Independent completion was defined as the phacoemulsification surgery performed by trainee without any intervention from the trainer. If trainee noted difficulty in performing any step like inability to complete continuous curvilinear capsulorrhexis (CCC), failure to separate the nucleus after three trials, difficulty keeping a stable anterior chamber depth during any step or any complication was noted during surgery, the case was taken over by the trainer and managed, keeping in mind safety of the patient. The details of the complications, if any, were noted. All surgeries were performed under continuous observation of trainer by sidescope and display monitor. Video recording of surgeries and assessment was done on the same day by trainer or reviewer and suggestions were given to improve surgical performance.

The data entered on ICO-OSCAR was evaluated for improvement in surgical skills by calculating average mean scores, improvement in mean scores in last five cases as compared to first five cases, rate of independent completion of case, and assessment of the rate of complications. Parameters like Mean ICO-OSCAR scores and independent completion rates in first five cases were noted as pretraining parameters and parameters noted in last five cases as posttraining parameters. Major complications assessed were anterior capsular runaway or tear, posterior capsular rupture (PCR), nucleus drop, iris tissue damage, and zonulodialysis.

The data was statistically analyzed using Statistical Package for the Social Sciences 20.0 for Windows software package (SPSS Inc., Chicago, IL). Prevalence of an outcome variable along with 95% confidence limits was considered significant. SEMI-LOG Transformation Regression Equation was used to correlate OSCAR scores with Independent Completion Rates (ICR), i.e., trainee skill levels and competency.

Results

A total of 1300 cases operated by 52 trainees, under supervision of trainer from November 2017 to August 2018, were included in the study. The mean age of trainees was found to be 30.98 ± 2.44 years. Each trainee performed an average of 25 cases, each case carrying maximum score of 100 (maximum score of five per step, each case was divided in 20 steps). Scores in the first and last five cases were considered as pre and posttraining scores, respectively.

Statistically significant improvement in mean scores was seen in all groups, as shown in Table 1. A difference of mean scores and percentage score improvements among groups suggests the impact of the previous surgical experience and inherent surgical skills of trainees.

Table 1

Pre and Posttraining Mean ICO-OSCAR Scores, and Percentage Improvement

Group	Pretraining Scores	Posttraining Scores	P	Percentage score improvement	Interpretation
A	42.8±0.2	88.4±0.2	0.001	110.1±9.3	Significant difference and improvement in pretraining posttraining mean scores, and percentage improvement of all groups (P<0.05).
B	46.6±0.2	92.3±0.2	0.005	100.4±7.8
C	50.1±0.1	94.3±0.2	0.015	90.8±4.6

Group A had pre and posttraining independent completion rates (%) of 22.1 ± 2.5 and 74.8 ± 3, respectively. Group B had pre and posttraining independent completion rates (%) of 52.1 ± 2.5 and 79.7 ± 3, respectively. Pre and posttraining independent completion rates (%) of 62 ± 3.7 and 90.6 ± 3.5, respectively, were noted in group C. On statistical analysis with paired t-test, significantly higher posttraining score was noted compared to pretraining score (P < 0.001) in all groups, as shown in Table 2.

Table 2

Pretraining, Posttraining, and Overall Mean Independent Completion Rates

Group	Pretraining ICR (%)	Posttraining ICR (%)	P	Overall ICR (%)	Interpretation
A	22.1±2.5	74.8±3	0.005	64.8±6.2	Significantly higher posttraining completion rates compared to pretraining completion rates (P<0.05).
B	52.1±2.5	79.7±3	0.018	75±5.6
C	62±3.7	90.6±3.5	0.035	85.9±3.5

Learning curve of trainees of three groups is shown in Fig. 1. Learning curve was charted according to their mean scores in groups of successive five cases. Chart shows three graphs. It was observed that Group C with more experienced trainees had more pre and posttraining scores than those of Groups A and B. The learning curves of all groups were noted to be steep. Learning curve of group A was steepest.

Figure 1

Learning curve of phacoemulsification trainees

Anterior capsular runaway/tear occurred in 18 (6.92%) and 6 (2.3%) cases in pre and posttraining cases, respectively. Posterior Capsular rupture (PCR) occurred in 32 (12.30%) cases in pretraining cases which reduced to 12 (4.61%) cases in posttraining cases. Total cases performed by trainees of groups A, B, and C were 475, 500, and 325, respectively, (25 cases each of 19, 20, and 13 trainees). Table 3 shows the rate of complications in all cases across three groups. LogMar pinhole visual acuity on postoperative day one and BCVA (Best Corrected Visual Acuity) of patients on one month visit were noted. Group A had pre and posttraining pinhole logMar visual acuity of (Mean ± SD) 1.2 ± 0.3 and 0.6 ± 0.3, respectively. Similarly, Group B had logMar visual acuity of 1 ± 0.2 and 0.4 ± 0.1 and Group C had logMar visual acuity of 0.6 ± 0.3 and 0.2 ± 0.2, respectively. Significantly lower posttraining score (improved visual acuity) was noted compared to pretraining score on statistical analysis with paired t-test (P < 0.001). At postoperative one month visit, Group A trainees' operated patients had mean logMar BCVA (Mean ± SD) of 0.12 ± 0.05 and 0.10 ± 0.07 in first 5 and last 5 cases. Similarly, Group B trainees' patients had mean logMar BCVA of 0.09 ± 0.07 and 0.07 ± 0.06 in first 5 and last 5 cases, while Group C trainees' had mean logMar BCVA of 0.04 ± 0.05 and 0.03 ± 0.05 in first 5 and last 5 cases, respectively. There was no statistically significant difference in BCVA of first 5 and last 5 patients of trainees of all groups, at 1-month postoperative visit.

Table 3

Pre, Posttraining, and Group Wise Complications

Complications	Pretraining-all groups (260 cases)	Posttraining all groups (260 cases)	P	Group A All cases (n=475)	Group B All cases (n=500)	Group C All cases (n=325)
Anterior capsular runaway/tear	18 (6.92%)	6 (2.3%)	0.049	28 (5.89%)	20 (4%)	8 (2.46%)
Posterior capsular rupture (PCR)	32 (12.30%)	12 (4.61%)	0.018	24 (5.05%)	16 (3.2%)	9 (2.76%)
Iris tissue damage	5 (1.92%)	3 (1.15%)	0.04	10 (2.1%)	5 (1%)	1 (0.31%)
Zonulodialysis	2 (0.76%)	1 (0.38%)	1	4 (0.84%)	3 (0.6%)	1 (0.31%)
Nucleus drop	2 (0.76%)	1 (0.38%)	1	2 (0.42%)	1 (0.2%)	0%
Overall	59 (22.69%)	23 (8.8%)	0.03	68 (14.31%)	45 (9%)	19 (5.84%)

We studied the relationship between ICR and OSCAR with Semi-log Transformation Regression Equation. OSCAR scores denoting skill levels of trainees correlate significantly (P = 0.0004) with their independent completion rates of surgery, as shown in Fig. 2.

Figure 2

ICR and OSCAR scores: SEMI-LOG Transformation Regression Equation

Discussion

Proficiency in phacoemulsification is an essential skill requirement for an ophthalmologist. Training in this surgery is usually acquired during post-graduation and during short courses or fellowships later. Assessment of uptake of surgical skills, during these exposures, is often subjective and often considered as superfluous. Evaluating skills transfer during these courses and the surgical proficiency achieved is no doubt difficult. There are no universally accepted parameters or a method to assess progress and proficiency during these training modules.[6] There are few formal standardized tools for evaluating the surgical competence of ophthalmology trainees like Objective Assessment of Skills in Intraocular Surgery (OASIS),[7] Global Rating Assessment of Skills in Intraocular Surgery (GRASIS),[8] Ophthalmic Clinical Evaluation Exercise (OCEX),[3] and International Council of ophthalmology's Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR)[4]. OASIS is one-page objective evaluation which takes into account multiple factors from wound placement and size, procedure done, total surgical time to postoperative vision, pin hole and spectacle corrected. It grades corneal edema, anterior chamber cells, and PCO-Posterior capsular opacification into 0/trace/1+/2+/3+/4+. GRASIS uses parameters like preoperative knowledge of patient and procedure, microscope centration, instrument handling, treatment of ocular structures and other tissues, flow of operation time and motion, use of nondominant hand, knowledge of phacoemulsification, vitrector equipment and instruments, interaction with assistants, handling of unexpected operative events, and overall performance, which are graded on rating scale of 1/2/3/4/5/NA-not applicable. OCEX is observed encounter between patient and resident doctor. Rating system used is 1-does not meet expectations, 2-meets some expectations, 3-meets all, and 4-exceeds all. In ICO-OSCAR, surgical procedures are divided into 20 individual steps including global indices. Each step is graded on a scale of novice, beginner, advanced beginner, competent, and done by trainer, having allotted scores of 2, 3, 4, 5, 0, respectively. Our study evaluated the surgical skills transfer during short-term phacoemulsification training using the original ICO-OSCAR and other traditionally used parameter such as independent completion rate (ICR), complication rate (CR), and visual acuity. The participants were further grouped according to their previous surgical experience. To our knowledge, this is first of such kind of studies which has evaluated learning curve of short-term phacoemulsification trainees by grading each step by ICO-OSCAR and calculating independent completion rate and complication rates in a single study. Comparison of ICO-OSCAR scores with more traditionally used surgical proficiency indicators like independent completion rate helps in validating the effectiveness of ICO-OSCAR as an indicator of proficiency and progress. Semi Log Mar Regression shows a direct relationship between Independent completion rate and Oscar Scores. ICO-OSCAR scores improved by almost 100% over the short-term (1 Month) training program. This could be objectively evaluated due to the use of ICO-OSCAR. As we had grouped the trainees as per experience, we could establish objectively the impact of the previous SICS surgical experience. We objectively found this short-term training to be an effective and safe approach to progressively master phacoemulsification skills. In the study by Farooqui et al., participating ophthalmologists were asked to complete an evaluation sheet (modified OSCAR) before starting the wet lab. The evaluation sheet had 22 questions to assess areas covered during training under various categories. Trainees completed the same evaluation sheet after the wet lab as well. A similar evaluation sheet was completed by the trainer for each trainee, and discussion was held at the end of the day between the trainer and the trainee evaluating the day's session. Their study found that the trainer's scores did not match that of the trainee's self-assessment scores for most of the task. They noted that modified OSCAR offers a reliable way to objectively assess the surgical skills acquired during wet lab training and observed that trainees tend to overestimate their skills.[9]

The overall complication rate in our study was 12.30%. Pretraining complication rate was 22.69% which reduced to 8.8% after training. Overall complication rate noted by Loncar et al. in their study was 15.12%.[2] In the study by Randleman et al., the PCR rate was 6.3% during the first 80 cases and reduced to 3.5% (P = 0.2) afterwards.[6] Dooley and O'Brien observed that PCR rate was 9% (4% with vitreous loss) in 100 consecutive cases.[10] PCR with or without vitreous loss is a common significant intraoperative complication during cataract surgery which can be associated with increased morbidity requiring further surgery or causing reduced final visual outcome. It is considered as the benchmark complication to judge surgical quality.[11]

This study also evaluated initial proficiency and progress of the trainees as per their previous surgical experience. Lesser experienced participants had lower pretraining and posttraining scores indicating the negative impact of lesser previous surgical exposure. They also had steeper learning curves as compared to the more experienced, as seen in Fig. 1.

The more traditional indicators of surgical skill proficiency used are ICR and CR. We compared ICO-OSCAR scores with these traditional indicators and found a very strong correlation between them, as shown in Fig. 2. However, we found ICO-OSCAR to be a more detailed, step-wise assessment of the entire surgery. It is a more valid and comprehensive assessment of the surgical prowess of the trainee. It helps to identify the weak areas for that particular participant so the trainer and trainee can put more emphasis on these. The study conducted by Ng et al. has also found ICO-OSCAR to be valuable learning and assessment tool.[12]

Conclusion

Learning curve for phacoemulsification is steep but it can be made smoother by use of standardized rubrics, proper techniques, and expert guidance. A short-term intensive phacoemulsification training program is of immense value for honing surgical skills for future practice. Wet-lab and didactic lectures provide a good take-off platform. Complication rates for novice surgeons can be high. Hence, it is important to have experienced trainers who can intervene at an appropriate time to ensure a good visual outcome for the patient. ICO-OSCAR for phacoemulsification is a very valuable asset for surgical training. It provides valuable inputs to the trainee and helps the trainer evaluate the trainee objectively. Comparison of ICO-OSCAR scores with more traditionally used surgical proficiency indicators like independent completion rate helps in validating the effectiveness of ICO-OSCAR as an indicator of proficiency and progress. Semi Log Mar Regression Analysis shows a direct relationship between Independent completion rate and Oscar Scores. Mean scores improved by almost 100% over the short-term (1 Month) training program. This could be objectively evaluated due to the use of ICO-OSCAR. We objectively found this short-term training to be an effective and safe approach to progressively master phacoemulsification skills.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.