Core competency in laparoendoscopic surgery.

BACKGROUND: Concern about patient safety and physician competence was highlighted by the Institute of Medicine report, revealing the prevalence of fatal medical errors. There is also awareness that technical difficulties specific to laparoendoscopic surgery can cause medical errors. Reported herein is a review of the evidence pertaining to objective assessment of core competency components in laparoendoscopic surgery: cognitive skills, technical skills, surgical performance, and judgment.
METHODS: PubMed and MedLine searches were performed to identify articles with combinations of the following key words: core competency, competency, laparoscopy, training, assessment, and curriculum. Further articles were obtained by searching reference lists of identified papers and through personal communication.
CONCLUSIONS: The available evidence suggests that it is currently possible to objectively assess core competency components in laparoendoscopic surgery: knowledge and clinical judgment with well-established tests and innate technical abilities with computer-based simulators with embedded metrics. Simulation training is conducted to a proficiency criterion regardless of the number of repetitions or practice hours. Reports indicate that skills learned on a simulator transfer to the operating room. However, to date, objective assessment of surgical performance can be obtained only through review of unedited video tapes of surgical procedures by disinterested experts as recently demonstrated by our Japanese colleagues in urology.

INTRODUCTION

Concern about patient safety and physician competence highlighted by the Institute of Medicine report[1] revealing the prevalence of fatal medical errors has prompted calls for closer scrutiny of surgical training and practice. In the case of laparoendoscopic surgery, there is awareness that technical difficulties associated with performing this new modality impose human-factor related problems that are prone to cause medical errors.[2]

Criteria Defining Competence in Medicine

The Accreditation Council of Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) have identified and endorsed 6 general core competencies defining competence:

Patient care: compassionate, appropriate, effective.

Medical knowledge about established and evolving sciences and its application to patient care.

Professionalism: commitment to professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.

System-based practice: ability to interact with and call upon resources of a health-care system to provide optimal care.

Practice-based learning and improvement; evaluation of one's own practice, appraisal and assimilation of scientific evidence for improvement of patient care.

Interpersonal/communication skills to team with and exchange information with patients, families and other health professionals.

Residents in training programs are expected to acquire competency in these areas to the level of a new practitioner. Practicing physicians are also expected to be evaluated along the same lines for certification and maintenance thereof. Evaluation tools are being developed to measure achievements of the competencies.[3-5]

Core Competency in Laparoendoscopic Surgery

The 6 core competencies are pertinent for endoscopists who are surgeons who are in turn physicians who provide competent diagnosis, effective treatment plans, and compassionate care of patients. Cognitive and technical skills modulated by judgment are the special components of competency in laparoendoscopic surgery within the patient care and medical knowledge categories. These are manifested in (1) preoperative care: diagnosis, preoperative preparation, and judgment; (2) operative performance: integrated cognitive and technical skills and judgment; (3) postoperative care: monitoring, treatment, and judgment.

Assessing Cognitive Skills

In laparoscopy, cognitive curriculum covers subjects that are of interest across specialties, such as entry, pneumoperitoneum, anesthesia, and complications, and those that are specialty specific, such as anatomy, pathology, and disease entities. Superior CD-ROM based, self-instructional, self-paced didactic programs are already in existence, such as one offered by SAGES. The programs contain didactic information as well as clinical scenarios. Well-recognized examination systems, such as the Multiple Choice Questions (MCQ) test, Patient Management Problems (PMP) instrument, and the Objective Structured Clinical Examination (OSCE) are in place to assess knowledge, judgment, and clinical performance.

Assessing Technical Skills

The skills required to perform laparoendoscopic surgery are very different from those of open surgery. They include the fundamental ability to operate on a 3D object from a 2D image projected on a remote video screen and adaptation to the restricted access, limitation of instrument manipulations and tactile feedback, and the fulcrum effect.[6] Acquiring these skills in the OR using the apprenticeship model is inadequate and potentially risky. Trainees need to develop their skills by practicing outside the OR rather than observing in the OR.[7] Animal and human cadaver training sessions offer realistic anatomy and tissue haptics; however, they are expensive, restricted, and lack objective assessment. Box trainers, although useful, also lack assessment metrics and objective feedback. The new paradigm in laparoscopic surgery training utilizes computer-based simulators with embedded assessment criteria for objective measurement of laparoscopic skills. Training and assessment are 2 sides of the same coin. Feedback helps the trainee to improve, and performance improvement reinforces the trainee.

Laparoscopic skill training using simulators is divided into 3 categories of ascending complexity[8]: a) basic coordination skills that assess the individuals’ inherent abilities comprising 3D to 2D vision spatial translation and psychomotor hand-eye coordination, demonstrated as translation, tracking, targeting, and pick and place; b) enabling skills and tasks that duplicate maneuvers of laparoscopic surgery and represent the building blocks for achieving technical proficiency. They represent combinations of inherent abilities and basic skills. Enabling skills include cannulation, clip application, and cutting. Enabling tasks include camera navigation, ligation, suturing, knot tying, and application of energy sources; c) simulating laparoscopic procedures by progressively building a series of primary tool/tissue interactions into a series of skills and tasks that combine into a series of manipulations resulting in a procedure.

Innate technical abilities represent the limiting factor that determines the ultimate level of operator skill in laparoendoscopic surgery. Some aspects of operative performance do not improve with practice. This was shown by McMillan and Cusheiri[9] using the Advanced Dundee Endoscopic Psychomotor Tester (ADEPT) with regard to error rates. Gettman et al[10] were able to predict operative performance of porcine laparoscopic nephrectomy, evaluated using an advanced modeling approach known as nonlinear causal resource analysis (NCRA). The prediction was based on measures of basic performance resources (BPRs) that assess innate abilities. Examples of BPRs include simple visual hand response speed, visual information processing speed, visual spatial short-term memory capacity as well as arm neuromotor channel capacity, which was the most common performance-limiting factor.

Pass/fail performance standards have been developed for the McGill Inanimate System for the Training and Evaluation of Laparoscopic Skills (MISTELS).[11,12] Participants were divided into competent and noncompetent groups, depending on level of clinical experience: clinically noncompetent surgeons (CNCS) comprising 83 medical students, PG1 and 2 residents, and clinically competent surgeons (CCS) comprising 82 chief residents, fellows, and practicing laparoscopic surgeons. A pass/fail criterion was established based on score distribution. Cutoff scores were evaluated through a range of scores within the overlapping score interval of 240 to 350. A pass/fail standard representing a score of 270, having a sensitivity of 82% and specificity of 82% was selected. The authors cautioned against using such a system for a high-stakes examination, as 18% of clinically competent surgeons would have failed and 18% of noncompetent surgeons would have passed. Lowering the cutoff score to 240 (lower limit) would have passed 28% of CNCS and failed 11% of CCS. Raising the cutoff score to 350 (upper limit) would have passed 1% of CNCS and failed 59% of CCS.[12]

These results may be an indication of simulation-based skill distribution among practicing laparoscopic surgeons. The pattern is reinforced by a study conducted at the American College of Surgeons meeting in 2001. Technical skills of 210 experienced laparoscopic surgeons were tested using MIST-VR and a Box-Trainer. Fifteen (7%) were unable to complete assessment and were excluded from the study. Of the 195, 12% performed >2 SDs from the mean.[13] These 2 studies used different protocols and simulator types. Both identified a segment of practicing laparoscopic surgeons who lacked technical skills when tested objectively with simulators. However, it should be noted that the learning curve required for adapting and adjusting to the peculiarities of the simulator interface plays a confounding role in how well one performs on it, especially if the exercise/task is not intuitive. In other words, we may be testing for abstract adaptive skills as well as for laparoscopic technical skills when we assess early simulation performances.

Proficiency defined as “error-free performance/time” can be established for any level of laparoendoscopic training and any simulator system[14] by having a group of recognized master surgeons practice on the simulator until the learning curve is essentially flat for 2 consecutive trials.[15]

The mean score is determined and outliers trimmed to establish values that define benchmark criteria. Variability among master surgeons is expected because an expert makes up for a relative deficiency in one skill set (and BPRs) by a relative surplus in another skill set (and BPRs). Wide variations in simulator-tested skills of experts have been reported.[13,16,17]

Once the proficiency level is established for a simulator, individuals can be trained to a criterion representing it regardless of number of trials or hours of practice.[15] Transfer of skills from virtual reality simulators or video trainers to the operating room has been confirmed in several studies.[18-20] In one study,[19] residents were trained to an established performance criterion regardless of number of trials. In the new paradigm for surgical training, residents would be trained to a skill proficiency criterion in the safe simulation environment before performing surgery on patients. Simulation practice would continue at regular intervals so as to consolidate and integrate gained skills within the learners established repertoire. Sustained deliberate practice helps to maintain proficiency over time.[21]

Errors

Identification of errors and creation of error metrics is a major contribution of virtual reality and hybrid computer-based simulators. However, at this time, most of the identified errors are fairly obvious. They include dropping objects, imprecise manipulation or cutting, incorrect application of clip, cautery, or ligature, as well as inaccurate suturing and slipped knots. Some simulators measure efficiency parameters globally as time to completion as well as individually as motion tracking, economy of motion, instrument path length, instrument collision, and other such things. Although efficiency measurements are important research tools, they may not relate to surgical outcome.

Identifying surgical errors is more complex. It is generally recognized that most surgical mishaps have 2 components: a) system component: an error or series of errors occurring within the health-care system culminating in or contributing to an adverse event (latent or chain of events); b) surgeon component: an error caused by a specific action of the surgeon (coface). This may be an error of commission, omission, or incorrect execution.[22] Surgical errors are further divided into those with consequences (mishaps) and those without [high-accident potential (HAPs) or near misses, latent]. Root cause analysis (RCA) is a concept used in aviation and manufacturing to investigate causal factors involved in actual or potential mishaps to make risk control decisions. It is being increasingly applied in medicine.

Assessing Surgical Performance

Surgical performance is a complex phenomenon representing constant interactions between the surgeon's cognitive and technical skills and the exercise of judgment. Traditional methods of credentialing physicians to perform new operations are based on the number of procedures performed and subjective evaluation of a proctor. In a recent review of the literature, Dagash et al[23] examined 3641 articles on credentialing and selected 37 on the basis of strict inclusion criteria. These included 25 777 patients. Initial experience was represented by a first group of patients and late experience by a later group of patients, after proficiency was deemed to have been achieved. The 2 groups were compared for operative time, conversion rates, complication rates, and length of hospital stay. In all articles, the definition of proficiency was highly subjective and the number of procedures required to reach it highly variable. For example, proficiency was claimed after a median of 30 procedures of laparoscopic cholecystectomy (range, 8 to 20), 28 procedures for fundoplication (range, 20 to 60), and 40 procedures for colectomy (range, 13 to 70). Wide variations in outcome parameters were noted at initial experience with convergence toward improved outcome at late experience. The magnitude of improvement varied widely between procedures.[23] Clearly, a shift toward more precise definitions based on objective assessments is needed.

Reznick and Regehr[24] modified the OSCE to assess technical skills and developed the Objective Structured Assessment of Technical Skill (OSATS) Exam. This is a multistation, task-specific, bell-ringer examination. The learner's performance is assessed by 2 examiners using 2 tools: a) task-specific checklist of relevant items (scored in 0/1) and b) global rating scale of 7 dimensions of operative performance: respect for tissue, time/motion, instrument handling, flow of operation, knowledge of instruments, knowledge of procedure, and use of assistants. These were scored from 0 poor to 5 excellent.[24]

To date, surgical proficiency can be assessed only by reviewing unedited videotapes of surgical procedures, a labor intensive and time-consuming process. The tapes are evaluated by 2 or more blinded experts for Errors and OSATS criteria. Tape reviewers must be in agreement at least 80% of the time (inter-rater reliability criterion P≥.80). Generic and procedure specific errors need to be identified and agreed upon by a consensus conference of recognized experts. Examples of generic errors include complete and incomplete transaction of tubular vital structure (ureter) by mechanical or other energy source, injury to hollow viscus (bowel) or solid organ (liver), inability to properly expose the operative field, identify anatomic structures within, obtain proper orientation thereof, enter proper cleavage dissection planes and secure hemostasis. Consideration should be given to distortion of anatomic features as a mitigating circumstance and to whether the error was recognized and corrected intraoperatively. The entire QA process must be shielded from legal discovery.

A system of reviewing unedited videotapes of laparoscopic nephrectomies or adrenalectomies on the basis of simplified criteria has been implemented in Japan, to assess laparoscopic surgical skills of urologists.[25] The score of a “perfect” procedure is 75 points, with 1 to 5 points deducted for each “dangerous maneuver” or error. More than 60 points are required to pass; 23 of 36 applicants passed the assessment test. A good correlation was found between the individual scores from the 2 referees evaluating each videotape with an average interexaminer difference of 4.4 points.[25]

CONCLUSION

It is now possible to objectively assess various components of core competency in laparoendoscopic surgery: cognitive skills, technical skills, surgical performance, and judgment. Knowledge and clinical judgment can be evaluated with well-established tests, such as MCQ, PMP, and OSCE. The available evidence suggests that innate technical abilities can be assessed/developed using computer-based simulators with embedded metrics. Novices can be simulator trained to a proficiency criterion regardless of number of repetitions or hours of practice. The characteristics of the learning curve will largely depend on their innate abilities and BPRs. Evidence also exists that skills learned on simulators transfer to the operating room in the form of better performance. However, at this time, objective assessment of surgical performance (including judgment) can only be obtained by reviewing unedited videotapes of surgical procedures for errors and quality of performance by at least 2 disinterested experts. Our Japanese colleagues in urology have already implemented the concept and started the trend.