Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men.

BACKGROUND: We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men.
METHODS: A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation.
RESULTS: The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains.
CONCLUSION: We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.