European expert consensus on a structured approach to training robotic-assisted low anterior resection using performance metrics.

AIM: The aim was to develop and operationally define 'performance metrics' that characterize a reference approach to robotic-assisted low anterior resection (RA-LAR) and to obtain face and content validity through a consensus meeting.
METHOD: Three senior colorectal surgeons with robotic experience and a senior behavioural scientist formed the Metrics Group. We used published guidelines, training materials, manufacturers' instructions and unedited videos of RA-LAR to deconstruct the operation into defined, measurable components - performance metrics (i.e. procedure phases, steps, errors and critical errors). The performance metrics were then subjected to detailed critique by 18 expert colorectal surgeons in a modified Delphi process.
RESULTS: Performance metrics for RA-LAR had 15 procedure phases, 128 steps, 89 errors and 117 critical errors in women, 88 errors and 118 critical errors in men. After the modified Delphi process the final performance metrics consisted of 14 procedure phases, 129 steps, 88 errors and 115 critical errors in women, 87 errors and 116 critical errors in men. After discussion by the Delphi panel, all procedure phases received unanimous consensus apart from phase I (patient positioning and preparation, 83%) and phase IV (docking, 94%).
CONCLUSION: A robotic rectal operation can be broken down into procedure phases, steps, with errors and critical errors, known as performance metrics. The face and content of these metrics have been validated by a large group of expert robotic colorectal surgeons from Europe. We consider the metrics essential for the development of a structured training curriculum and standardized procedural assessment for RA-LAR.

The present study is the first to describe a structured approach to develop performance metrics for training in robotic‐assisted low anterior resection for rectal cancer.

Introduction

For the past decades, we have been in search of better outcomes for patients undergoing colorectal cancer surgery. From optimizing surgical techniques, the use of adjuvant and neoadjuvant therapy, the adoption of a minimally invasive approach to enhanced recovery after surgery [1], all these elements are incorporated into the principle of aggregation of marginal gains, which was popularized by Sir Dave Brailsford [2]. Furthermore, for more complex and lower volume procedures, there is evidence to encourage centralization to enhance surgeons’ and their team’s experience as it may help to improve surgical outcomes and survival [3].

Centralization of rectal cancer surgery improves long‐term survival [4]. However, this volume and outcome relationship may not always be directly correlated, and volume alone cannot be an indicator to assure good patient outcomes [5, 6]. Recent research into surgical skills has identified a strong correlation between surgeons’ technical performance, patients’ complication rates and clinical outcomes [7, 8]. Therefore, the focus has now turned to implementing optimal surgical training and improving surgeons’ skills.

The European School of Coloproctology (ESC) of the European Society of Coloproctology (ESCP) was set up to benchmark surgical training at a European level, and the robotic colorectal surgery curriculum is one of the training programmes being established [9]. In the search for a more objective and transparent approach to surgical training, we have identified a scientific approach, known as proficiency‐based progression (PBP) training [10, 11, 12]. The core of this approach is to deconstruct a surgical procedure to operational defined performance metrics [10, 11, 13, 14, 15, 16, 17]. These metrics can facilitate training and assessment and allow learners to progress in their training based on their proficiency, rather than the number of cases performed or duration of practice. This method has been applied to specialities including orthopaedics [15, 17], cardiology [14, 18], anaesthetics [16, 19] and interventional neuroradiology [13, 20]. PBP training has been shown to significantly reduce intra‐operative errors (> 40%) [18, 19, 21, 22, 23, 24, 25]. Therefore, the ESC aims to introduce this scientific training approach in the robotic colorectal surgery training programme.

Low anterior resection (LAR) for rectal cancer is an index operation in the field of coloproctology which is commonly performed with robotic‐assisted (RA) surgery. An optimally performed RA‐LAR potentially minimizes adverse clinical, oncological and functional outcomes for patients undergoing this procedure. Our study aimed to develop and objectively define performance ‘metrics’ that characterize a reference approach to RA‐LAR for rectal cancer, and to obtain face and content validity through a consensus meeting (i.e. with a Delphi panel) of very experienced and expert robotic colorectal surgeons.

Method

The principle of metric development and stress testing (face and content validation) for PBP training has been described in non‐colorectal specialities previously [10, 11, 13, 14, 15, 16, 17]. This principle was applied when developing the RA‐LAR metrics, and is described below.

Metrics team

Approval of the conduct of the study was granted by the Ethics Committee at the Instituto de Investigacion Marques de Valdecilla (IDIVAL), Santander, Cantabria, Spain (Código interno: 2020.026). The Metrics Group consists of three experienced colorectal surgeons (KM, MGR, ST) with special interest in robotic surgery, who are involved in setting up the ESCP Colorectal Robotic Surgery (ESCP CRS) Programme, a senior behavioural scientist and an education‐training expert (AGG).

RA‐LAR metrics development

A detailed task analysis and deconstruction process were used to break down a ‘reference’ approach to the performance of the complete RA‐LAR for rectal cancer procedure in small non‐overlapping parts [10, 11, 14, 17]. Published written guidelines, video teaching materials, manufacturers’ instructions and access of more than 10 anonymized unedited RA‐LAR videos performed by surgeons with different levels of experience supported the metrics development process. The goal was to characterize a reference approach to RA‐LAR procedures for rectal cancer in female and male patients. A reference procedure is assumed to be straightforward and uncomplicated to guide trainees to learn the ‘recipe’ to perform these procedures, before performing more complex operations such as patients with high body mass index or with locally advanced disease. The phases and steps are the same between women and men undergoing RA‐LAR apart from during the rectal dissection phase due to anatomical considerations.

A 1‐day preliminary face‐to‐face planning meeting, three 1½‐day face‐to‐face meetings for metrics identification and definition, and four 1½‐day face‐to‐face meetings for the metric stress test were conducted. Video‐conferencing using Zoom (San Jose, California, USA) and email exchange were used to complement face‐to‐face meetings for further clarification of the metrics development.

At the beginning of the metrics development, the Metrics Group agreed on some definitions as follows.

Performance metrics: Units of observable behaviour which together constitute a stepwise description of a reference approach to a procedure.

Procedure phase: A group or series of integrally related events or actions that, when combined with other phases, make up or constitute a complete operative procedure.

Step: A component task, the series aggregate of which forms the completion of a specific procedure.

Error: A deviation from optimal performance.

Critical error: Major deviation from optimal performance which has a likelihood of causing harm to the patient or compromising the safe completion of the procedure [13, 15, 17].

The metrics therefore consist of procedural phases involved in an RA‐LAR. Each phase comprises specific steps required to accomplish that phase. The importance of the metrics approach in defining these phases and steps is that they are clear and unambiguous. The step either occurred or did not occur, and can be scored as such by an external reviewer with a high degree of reliability [26, 27, 28]. Similarly, errors and critical errors were defined associated with particular steps within different phases of the procedure. For errors, behaviours exhibited by the operator may not necessarily in and of themselves lead to a bad outcome or an event with more serious consequences but their enactment sets the stage or increases the probability for a more serious event to occur or detracts from the efficient execution of the desired procedure. In contrast, a critical error is a more serious occurrence and represents operative performance that could either jeopardize the outcome of the procedure or lead to significant iatrogenic damage [13, 14, 17]. Figure 1 illustrates an example of a procedural phase characterized for the RA‐LAR. The sequence of the procedure assessed using metrics in the RA‐LAR is included in Appendix S1. In addition to the metrics, valuable knowledge and principles of the operation were compiled (such as applied anatomy) to facilitate the learning process; these formed the didactic component for the learner during the training process.

Figure 1

Example of a phase during robotic‐assisted low anterior resection that was characterized with steps, errors and critical errors. DNTT, damage to non‐target tissue.

Once the Metrics Group defined the metrics, they were then used to score four unedited anonymized RA‐LARs performed by four different surgeons. The scorings were performed by the members of the Metrics Group independently. Any difference in the scoring was discussed in order to identify discrepancies in interpretation or ambiguities in the metric definition. Based on this process and if agreed, changes were made in the metrics to facilitate the scoring agreement. This process was repeated for each video until the Metrics Group was satisfied with the metrics and they could be scored with a high degree of reliability (i.e. inter‐rater reliability > 0.8 – the internationally agreed gold standard) [29, 30]. In the end, there were 15 phases, 128 steps, 89 errors and 117 critical errors in women, 88 errors and 118 critical errors in men.

Metrics stress testing (face and content validation) with a modified Delphi approach

Once the metrics for the RA‐LAR were defined and characterized, face validity and content were verified by a group of experienced robotic colorectal surgeons. To provide a more objective and independent assessment of the metrics an international panel of expert minimally invasive colorectal surgeons with robotic experience were invited to join the Delphi panel [13, 14, 15, 16, 17].

Eighteen expert colorectal surgeons including the three Metrics Group members from 10 countries, a non‐voting behavioural scientist and a senior administrative member for the ESCP CRS Working Group attended a consensus meeting in Munich, 8 February 2020 (Table 1). The panel was chosen for their colorectal robotic surgical experience and their demonstrated educational interests and commitment. The age of the panel ranged from 40 to 62 years; there were three female surgeons. Ten panel members were head of their respective departments, and five were full professors affiliated with universities. The combined robotic colorectal procedures performed or supervised by the Delphi panel were more than 4000.

Table 1

Number of surgeons from each country represented in the Delphi panel.

Country	Number of surgeons
France	1
Spain	2
UK	5
Germany	3
Belgium	1
Italy	2
Poland	1
Ireland	1
Sweden	1
The Netherlands	1
Total	18

A brief overview of the project and meeting objectives was presented. Background information regarding proficiency‐based training, prior literature demonstrating the validity of this training approach for procedural specialties, and the specific objectives of the current Delphi panel were reviewed [31]. Each phase of the procedure, the procedural steps that were included in that phase, and the potential errors were presented. It was also explained that the associated metrics had been developed by the Metrics Group for a reference approach to RA‐LAR for rectal cancer. It was acknowledged that the designated reference procedure might not reflect the exact techniques employed by individual Delphi panellists, but that the operative steps presented accurately embodied the essential and key components of the procedure and ‘were not wrong’ [13, 14, 15, 17].

To assess the correlation of the procedure steps, errors and critical errors before and after the Delphi process, changes were analysed with the Pearson chi‐squared test (IPM SPSS Statistics for Windows, version 26, IBM Corp., Armonk, New York, USA). A P value of < 0.05 was considered statistically significant.

Results

The Metrics Group proposed 15 phases included in the RA‐LAR, and each phase had a defined beginning and end. During the Delphi process, two phases (stoma formation and wound closure) were combined, to allow accurate representation of the sequence of the steps during surgery. Therefore, there was a total of 14 phases in the final RA‐LAR procedure (Table 2).

Table 2

The beginning and end of the different phases of the reference approach to robotic‐assisted low anterior resection and the changes agreed and voted on by the Delphi panel.

Procedure phase	Title	Phase – begins	Phase – ends
I	Patient positioning and preparation	Completion of WHO checklist	Patient is on the table before prepping
II	Preparation of operative field	Creation of a sterile field	Patient is draped
III	Trocar position	Incision/insertion of trocars	Removal of laparoscopic instruments
IV	Docking	Advance the patient cart to the patient	Operating surgeon takes control at the console
V	IMA dissection/ligation	Visualize the working end of all three robotic instruments intraabdominally	Complete division of the IMA
VI	IMV exposure and ligation	Atraumatic instrument to retract the descending mesocolon	Complete division of the IMV
VII	Splenic flexure mobilization	Atraumatic instrument to retract the descending mesocolon	Left mesotransverse colon is completely mobilized
VIII	Complete mobilization of left colon	Atraumatic instrument to retract the descending mesocolon	Release the lateral attachment of the sigmoid colon
IX	Rectal dissection/rectal transection (TME/LAR) – separately for female/male patient	Visualize the working end of all three robotic instruments in the pelvis	The divided rectum is placed in the abdominal cavity and in view
X	Undocking the system	Robotic instruments removed	Patient cart removed
XI	Specimen extraction and re‐establishing the pneumoperitoneum (adjust the position of the table)	Make Pfannenstiel incision	Permanently or temporarily closing the transverse incision to re‐establish the pneumoperitoneum
XII	Anastomosis	Move the proximal bowel to pelvis	Check the anastomosis for leakage, e.g. air leak test, rigid or flexible sigmoidoscopy
XIII	Stoma formation and wound closure	Incision at the stoma site	Application of stoma bag/appliance
XIV	Transfer patient from operating table to bed	Transfer patient to bed	Patient out of the operating room
N = 14	Stoma formation and wound closure were combined to one phase during Delphi meeting

Changes are in italic.

IMA, inferior mesenteric artery; IMV, inferior mesenteric vein; LAR, low anterior resection; TME, total mesorectal excision; WHO, World Health Organization.

In the end, there are 14 phases, 129 steps performed during a RA‐LAR; one step was added (sterile draping of the cart in phase II), and 16 steps were modified during the Delphi meeting (Table 3).

Table 3

Steps before and after the Delphi meeting.

Procedure phase	Title	Steps before Delphi	Steps after Delphi	Added	Deleted	Modified
I	Patient positioning and preparation	11	11	0	0	1
II	Preparation of operative field	5	6	1	0	1
III	Trocar position	16	16	0	0	3
IV	Docking	7	7	0	0	2
V	IMA dissection/ligation	7	7	0	0	1
VI	IMV exposure and ligation	6	6	0	0	0
VII	Splenic flexure mobilization	10	10	0	0	0
VIII	Complete mobilization of left colon	4	4	0	0	0
IX	Rectal dissection/rectal transection (TME/LAR) – separately for female/male patient	19	19	0	0	0
X	Undocking the system	3	3	0	0	1
XI	Specimen extraction and re‐establishing the pneumoperitoneum (adjust the position of the table)	7	7	0	0	0
XII	Anastomosis	15	15	0	0	1
XIII	Stoma formation and wound closure	16	16	0	0	6
XIV	Transfer patient from operating table to bed	2	2	0	0	0
N = 14		128	129	1	0	16

IMA, inferior mesenteric artery; IMV, inferior mesenteric vein; LAR, low anterior resection; TME, total mesorectal excision.

There were 89 procedure errors identified for RA‐LAR in female patients and 88 in male patients. After the Delphi meeting, the final number of procedure errors in women undergoing RA‐LAR was 88, and 87 in men (Table 4).

Table 4

Errors before and after the Delphi meeting.

Procedure phase	Title	Errors before Delphi	Errors after Delphi	Added	Deleted	Modified
I	Patient positioning and preparation	3	3	0	0	0
II	Preparation of operative field	2	2	0	0	0
III	Trocar position	8	8	0	0	0
IV	Docking	6	5	0	1	0
V	IMA dissection/ligation	9	11	2	0	0
VI	IMV exposure and ligation	9	10	1	0	0
VII	Splenic flexure mobilization	5	5	0	0	0
VIII	Complete mobilization of left colon	5	5	0	0	0
IX	Rectal dissection/rectal transection (TME/LAR) – separately for female/male patient	Women – 13	Women – 13	Women – 1	Women – 1	Women – 0
		Men – 12	Men – 12	Men – 1	Men – 1	Men – 0
X	Undocking the system	3	3	0	0	0
XI	Specimen extraction and re‐establishing the pneumoperitoneum (adjust the position of the table)	8	7	0	1	0
XII	Anastomosis	6	5	1	2	0
XIII	Stoma formation and wound closure	12	11	0	1	0
XIV	Transfer patient from operating table to bed	0	0	0	0	0
N = 14		Women – 89	Women – 88	Women – 5	Women – 6	Women – 0
		Men – 88	Men – 87	Men – 5	Men – 6	Men – 0

IMA, inferior mesenteric artery; IMV, inferior mesenteric vein; LAR, low anterior resection; TME, total mesorectal excision.

Similarly, small changes were made in the Delphi meeting, and the numbers of procedure critical errors identified for RA‐LAR were 115 and 116, respectively, for women and men (Table 5). Furthermore, the number of procedure steps, errors and critical errors before and after the Delphi changes were highly correlated (Pearson correlation coefficient for procedure steps r = 0.999, P < 0.001; errors r = 0.98, P < 0.001; and critical errors r = 0.985, P < 0.001).

Table 5

Critical errors before and after the Delphi meeting.

Procedure phase	Title	Critical errors before Delphi	Critical errors after Delphi	Added	Deleted	Modified
I	Patient positioning and preparation	13	13	0	0	2
II	Preparation of operative field	5	5	0	0	1
III	Trocar position	7	7	0	0	0
IV	Docking	3	4	1	0	0
V	IMA dissection/ligation	14	14	0	0	0
VI	IMV exposure and ligation	16	15	0	1	1
VII	Splenic flexure mobilization	11	11	0	0	0
VIII	Complete mobilization of left colon	7	7	0	0	0
IX	Rectal dissection/rectal transection (TME/LAR) – separately for female/male patient	Women – 12	Women – 13	Women – 1	Women – 0	Women – 0
		Men – 13	Men – 14	Men – 1	Men – 0	Men – 0
X	Undocking the system	1	1	1	1	1
XI	Specimen extraction and re‐establishing the pneumoperitoneum (adjust the position of the table)	8	7	0	1	1
XII	Anastomosis	11	9	0	2	0
XIII	Stoma formation and wound closure	7	7	0	0	0
XIV	Transfer patient from operating table to bed	2	2	0	0	0
N = 14		Women – 117	Women – 115	Women – 3	Women – 5	Women – 6
		Men – 118	Men – 116	Men – 3	Men – 5	Men – 6

IMA, inferior mesenteric artery; IMV, inferior mesenteric vein; LAR, low anterior resection; TME, total mesorectal excision.

Table 6 shows the consensus from the Delphi panel in each procedure phase after incorporated changes during the meeting. Apart from the patient positioning and preparation, and docking of the robot, other phases received unanimous votes from the panel.

Table 6

Results from the Delphi meeting and consensus reached at the end of each phase.

Procedure phase	Title	Percentage consensus (%)
I	Patient positioning and preparation	83
II	Preparation of operative field	100
III	Trocar position	100
IV	Docking	94
V	IMA dissection/ligation	100
VI	IMV exposure and ligation	100
VII	Splenic flexure mobilization	100
VIII	Complete mobilization of left colon	100
IX	Rectal dissection/rectal transection (TME/LAR) – separately for female/male patient	100 (female TME), 100 (male TME) 100 (transection of rectum)
X	Undocking the system	100
XI	Specimen extraction and re‐establishing the pneumoperitoneum (adjust the position of the table)	100
XII	Anastomosis	100
XIII	Stoma formation and wound closure	100
XIV	Transfer patient from operating table to bed	100

IMA, inferior mesenteric artery; IMV, inferior mesenteric vein; LAR, low anterior resection; TME, total mesorectal excision.

Discussion

In this study, the performance metrics (procedure phases, steps, errors, critical errors) for RA‐LAR for rectal cancer could be characterized. This was achieved through a systematic and structured approach. A Metrics Group was formed with three expert colorectal surgeons with a particular interest in robotic surgery, involved in setting up a training curriculum at a European level (ESC), and a senior behavioural scientist who has more than two decades of experience in surgical training. The development process of the metrics was comprehensive and involved reviewing the published guidelines, training materials, manufacturers’ instructions and unedited anonymized videos of RA‐LAR performed by surgeons with different levels of experience. The metrics that have been developed are not confined to any specific robotic platform.

The metrics were also subjected to stress testing. First, the individual surgeons in the Metrics Group independently scored unedited videos of RA‐LAR performed by different surgeons. Differences in scoring resulted in further discussion and refinement of the metrics operational definitions to facilitate consensus and reliable scoring. Second, the metrics produced by the Metrics Group were then scrutinized by a panel of expert colorectal surgeons (18, from 10 different countries). The expert panel during the Delphi meeting provided an opportunity to optimize the metrics further. The Delphi panel understood that surgeons might work differently, but the goal was to achieve a standardized structure approach to a ‘reference’ RA‐LAR. The metrics presented in the Delphi meeting may not be in the same way or sequence that each surgeon is practising, but the metrics are not wrong and should be suitable for learners. The pre‐ and post‐Delphi metrics are highly correlated (Tables 2, 3, 4, 5). After incorporating the changes suggested by the Delphi panel voting was obtained at the end of the discussion of each phase. All the phases received unanimous agreement apart from phase I, Patient position and preparation, and phase IV, Docking. The main discussion in phase I was about the padding of the patient, their face, body and limb protection to ensure the safety of the patient. There was no suggestion to change the metrics, but the knowledge of compartment syndrome should be added to the didactic part of the training programme. There was no issue raised with regard to the docking phase of the RA‐LAR.

With the evolution of surgical training, we have witnessed a shifting paradigm of the training curriculum from the apprenticeship model to a more organized approach such as the UK LapCo programme [32]. However, it is not straightforward to determine when the learners can progress to the next phase of training and whether this is determined based on volume or duration. The volume–outcome relationship has attracted much attention in the procedural‐based intervention [5, 33]. While it is expedient to measure volume, the quality of the procedure and volume relationship may not always be straightforward [6]. One of the primary functions of the types of performance metrics reported on in this study is to accelerate the learning curve. They provide performance feedback to the trainee which affords deliberate practice rather than repeated practice [34]. This means that the trainee receives feedback which is proximate to their performance which informs them what they did wrong and how it should optimally be performed [35]. Furthermore, these types of metrics are the foundation of effective virtual reality simulation training [11, 13, 18, 25, 36]. Explicitly defined procedure performance metrics, scored in a binary fashion, are more difficult to develop than off‐the‐shelf Likert‐type scales (e.g. OSATS or GEARS) [37, 38]. They are more reliable, however (e.g. OSATS and GEARS) [26, 28] and provide the trainee with objective, transparent and explicit feedback which is derived from a consensus of expert senior surgeons [11, 12, 13, 15, 17]. In contrast to hand/tool ‘efficiency’ measures these types of performance metrics will also underpin and bolster the development of effective virtual reality simulation and machine learning [20, 39, 40, 41].

The approach used to characterize an RA‐LAR in this study provides structured and objective metrics that are explicit and transparent and allow learners to focus on achieving the steps required before progress to the next level – PBP training. This PBP training has been utilized in different specialities and has been shown to significantly reduce intra‐operative errors [18, 19, 21, 22, 23, 24, 25, 42]. Furthermore, the metrics will serve as a template for trainers to provide structured training for a surgical procedure. The approach also provides effective learning because the metrics underpin a ‘deliberate practice’ approach to training rather than the traditional repeated practice approach [34]. Equally important, performance assessment using these metrics will provide feedback to the trainee that is transparent, objective and unambiguous, disadvantages often associated with Likert‐type scale assessment [26, 27, 30]. A recently published consensus statement has highlighted the importance of standardization of robotic total mesorectal excision. This study provides a good starting point for training in robotic rectal surgery [43]. The metrics described by us have objectively measurable steps with defined errors and critical errors and will further enhance training through a ‘deliberate practice’ method.

The debate of the approach for cancer of the rectum is ongoing [44], but RA‐LAR has been increasingly performed [45]. Given the adverse outcomes associated with the new technique of rectal cancer surgery [46], although outcomes from large multicentre trials under way are awaited (COLOR III, RESET) [47, 48], the importance of a structured and quality assured approach to training is needed to safeguard patients’ outcomes. This is one of the fundamental remits of the ESCP CRS Working Group within the ESC.

The next phase, to continue this present work with the performance metrics for RA‐LAR, is to obtain construct validity, i.e. whether the metrics distinguish between the objectively assessed performances of novices and very experienced robotic colorectal surgeons. The results from the construct validity are vital as the validated metrics can be used to inform training and assessment, i.e. setting a benchmark in surgical training [9], using PBP approach.

There are several limitations to this study. The proposed metrics are for a standard straightforward (i.e. reference approach) RA‐LAR for female and male patients. The aim is to provide learners with a structured stepwise approach. The Delphi panel was aware occasionally that there are deviations from a ‘reference’ operation, and a slight variation of techniques may be needed. Equally, there is more than a single approach for a particular part of an operation. For example, the splenic flexure mobilization technique characterized in the metrics employs a supra‐pancreatic approach, commonly practised in Europe. The learners will probably be exposed to other approaches once they have mastered the recommended technique described in these metrics.

The number of errors and critical errors is not exhaustive, but these are considered by the Metrics Group and the Delphi panel to be important in RA‐LAR. There are no clinical outcomes to correlate with the scoring of these metrics. Therefore, it is not known at this stage which part of the metrics are more important than others, i.e. they are not weighted.

In conclusion, a commonly performed robotic rectal operation can be broken down to defined phases and steps and have measurable errors and critical errors, known as performance metrics. Evidence of the face and content of these metrics has been validated by a large group of expert robotic colorectal surgeons from Europe. Further development of these metrics is essential to guide the training curriculum and assessment for RA‐LAR.

Conflicts of interest

ST received education grants from Intuitive Surgical and Medtronic. MGR received education grants from Intuitive Surgical and Medtronic and currently is Medical Advisor to Intuitive Surgical, Medtronic and J&J. AGG holds education research grants from Medtronic (Dublin, Ireland), AO Education Institute (Davos, Switzerland) and the Arthroscopic Association of North America (Chicago, USA) for the investigation of metric‐based education and training. KEM is Medical Advisor to Medtronic and Boehringer Ingelheim.

Funding

The study is part of the ongoing project for the Colorectal Robotic Surgery Working Group, European School of Coloproctology, European Society of Coloproctology. Medtronic provided the educational grant for this study but did not influence in selecting the experts, design and conduct of the research, data collection, analysis and preparation of the manuscript.

Appendix S1. Sequence of the procedure assessed using the metrics in the robotic‐assisted low anterior resection.

Click here for additional data file.