The Learning Curve of Computer-Assisted Free Flap Jaw Reconstruction Surgery Using 3D-Printed Patient-Specific Plates: A Cumulative Sum Analysis.

BACKGROUND: Computer-assisted jaw reconstruction (CAJR) has benefits in reducing operation time and improving reconstruction accuracy, compared to conventional freehand jaw reconstruction. However, no information is available regarding learning curves in CAJR with the use of 3D-printed patient-specific surgical plates (PSSP). The purpose of this study was to assess surgical outcomes and learning curve for the first 58 consecutive CAJR using 3D-printed PSSP performed by a single surgical team in a single institution.
METHODS: In a prospective study, consecutive patients who underwent free flap CAJR using 3D-printed PSSP were included. The determination of proficiency, based on the cumulative sum of surgical success (no major adjustment of 3D-printed PSSP, flap survival) passing the acceptable boundary line of cumulative sum analysis, was the primary outcome. To find out any potential factors influencing the learning curve, baseline characteristics of patients were compared before and after proficiency achievement. Secondary outcomes included inflexion points of the total operation time, blood loss, length of hospital stay, and bone graft deviation, measured by the cumulative sum analysis.
RESULTS: From December 2016 to November 2020, 58 consecutive cases underwent surgery performed by a single surgical team. The overall surgical success rate was 94.8% (55/58). A three-stage learning curve of primary outcome was observed. The proficiency was achieved after 23 cases. The proportions of advanced tumor staging and concomitant surgery after obtaining proficiency were significantly higher than those before achieving proficiency (p = 0.046 and p < 0.001, respectively). Mean values of operation time, intraoperative blood loss, length of hospital stay, and bone graft deviation were 532.5 ± 119.2 min, 1,006.8 ± 547.2 ml, 16.1 ± 6.3 days, and 0.9 ± 1.2 mm, respectively. Two trends of learning curve were observed in the CUSUM analyses of total operation time, length of hospital stay, and bone graft deviation, in which the first and second inflexion points occurred between 8 and 17 cases and between 43 and 46 cases, respectively.
CONCLUSION: Our results revealed a three-stage learning curve of CAJR with the use of PSSP, including initial learning, plateau, and overlearning. Based on CUSUM analysis, the surgical proficiency was achieved after 23 cases, and total operation time, length of hospital stay, and bone graft deviation stabilized after 8-17 cases.

Introduction

The emerging technique of computer-assisted jaw reconstruction (CAJR), which facilitates preoperative surgery simulation and transfers the virtual plan to a real operation, significantly impacted conventional surgical approaches (1, 2). Studies have reported the benefits of CAJR compared to conventional freehand jaw reconstruction, including reductions of ischemia time, operation time, and related costs, and improvement of reconstruction accuracy (3–10).

With the increasing popularization of CAJR surgery, the needs of standardizing surgical training and optimizing patient outcomes are urgent. The cumulative sum (CUSUM) control chart is a sequential analysis technique in statistical quality control, typically used for monitoring change detection (11). Now, the concept of CUSUM analysis has been used by surgeons to assess the learning curve in complex surgeries, such as robot-assisted surgery and endoscopic surgery (12–14). It allows surgeons to precisely detect potential imperfections and then improve surgical outcomes. However, very sparse data are available on learning curves of free flap jaw reconstruction (15–18). To our best knowledge, there is no study reporting the learning curve in the practice of CAJR.

Our team started performing CAJR using 3D-printed patient-specific surgical plates (PSSP) in December 2016. We reported our first experience in 2018 and indicated that CAJR with PSSP is feasible, safe, and precise (19). This study aimed to analyze the surgical outcomes and learning curve for the first 58 consecutive CAJR cases using 3D-printed PSSP performed by a single surgical team in a single institution.

Materials and Methods

This study was approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster (No. UW 16-315), registered in ClinicalTrials.gov with a No. of NCT03057223 and in The University of Hong Kong Clinical Trials Centre with a study identifier of HKUCTR-2113 (www.HKUCTR.com).

Surgical Interventions

From December 2016 to November 2020, all consecutive patients who underwent CAJR with 3D-printed PSSP performed by a single surgical team led by the same chief surgeon in the Queen Mary Hospital in Hong Kong were enrolled without dropout. Before December 2016, the surgical team had no previous experience in using PSSP.

The virtual planning, design, and fabrication of 3D-printed PSSP and surgical techniques have been described in our previous publications (19, 20). Patients indicated for jaw surgery were arranged to undertake contrast-enhanced CT scan of head and neck and the donor site. The virtual surgical planning was done using Proplan CMF 3.0 software (Materialise, Leuven, Belgium). To exactly transfer the surgical plan to the operation room, we designed PSSP in line with the planned surgery using the 3-Matic software (Materialise, Leuven, Belgium). The surgical templates were printed using the stereolithography technology from high-strength resin. The plates were printed using Grade 2 pure titanium by selective laser melting technology. All the surgical procedures and perioperative management were conducted in a routine manner, except that osteotomies, bone movements, and flap inset were guided by the prepared 3D-printed patient-specific surgical templates and fixed by the 3D-printed patient-specific pure titanium plate. Patients were followed up regularly. Plain x-ray image and CT/CBCT scanning were done approximately 1 month after surgery.

Outcomes

The primary outcome was the achievement of proficiency. All CAJR cases were recorded chronologically by operation date. Proficiency was based on the cumulative sum of surgical success reaching the acceptable boundary line of the CUSUM analysis. Surgical success was defined as no change of 3D-printed PSSP (major adjustment of plates, including the need for bending plate and conversion to conventional commercialized titanium plates) and flap survival. Cases were divided into two groups according to the achievement of proficiency, and the baseline characteristics were compared to analyze the potential factors influencing the learning curves.

The secondary outcomes were the stabilization of total operation time, intraoperative blood loss, length of hospital stay, and reconstruction accuracy to a steady state. The determination of stabilization was based on the inflection point of the cumulative graph. The accuracy of jaw reconstruction 1 month postoperatively was measured by calculating the deviation distance between preoperative planned and postoperative achieved bone graft positions using the same method described previously (4).

CUSUM Analysis

CUSUM analysis was performed to detect subtle deviations of the surgeon’s performance in primary and secondary outcomes (21).

The overall fibula flap failure rate was revealed as 7.0% (n = 161/2,305) by the latest systematic review and meta-analysis (22). Since fibula flaps were used in most of our cases, we utilized 7.0% as the current failure rate of free flap. We reviewed the available studies using PSSP in maxillofacial reconstruction () and pooled the current failure rate of PSSP as 4.4% (n = 6/136) (19, 23–27). Therefore, the overall success rate of implementing free flap and PSSP was 88.9%, and the acceptable level of surgical failure (p0) was set at 11.1% [100% − (100% − 7.0%) × (100% − 4.4%)] in the CUSUM analysis. A chosen level of surgical failure rate (p1) reflecting a change in surgical performance was set as two times the acceptable level of surgical failure (28). All calculation procedures and intermediate values are shown in . Briefly, with each surgical success obtained, the line would rise by 0.162; with each failure, the line would fall by 0.838. Type 1 and type 2 errors, the probabilities of wrongly accusing the surgeon of unacceptable performance and acceptable performance, were set as 0.10, which were considered rational (28). Boundary lines were calculated to determine whether the surgical performance was acceptable (H1) or unacceptable (H0). Once the line reached H1 or H0, proficiency was obtained or lost, respectively.

Table 1

Literature review on studies using PSSP in maxillofacial reconstruction.

Author	Total No. of Cases Using PSSP	Failed No. of Cases Using PSSP
Frank Wilde, 2015 (23)	32	6
Majeed Rana, 2017 (24)	22	0
Yang, 2018 (19)	10	0
David Öhman, 2019 (25)	5	0
Philipp Jehn, 2020 (26)	20	0
Zavattero, 2021 (27)	47	0

The sequential differences of total operation time, intraoperative blood loss, length of hospital stay, and bone graft deviation between each case and the mean value were also detected by CUSUM (29). The mean values of each variable were calculated and used as the reference. Briefly, with each variable more or less than the mean value, the line would rise or fall by the absolute difference, respectively. The best-fit curve and its corresponding equation were determined with the cftool command in Excel (version 2019; Microsoft Corporation, USA).

Statistical Analysis

All statistics were calculated using SPSS Statistics (version 26.0; IBM Corporation, Chicago, IL, USA). Data were presented as number (percentage) for categorical data and mean ± standard deviation for continuous data. Independent samples t test was used to detect differences in the means in the patients’ age and donor bone length. Chi-square test was used to detect differences in the proportions of gender, diagnosis, TNM classification, surgical site, bone segments, and concomitant surgery. p < 0.05 was considered statistically significant.

Results

A total of 58 consecutive patients who underwent CAJR using PSSP were included. Baseline characteristics of total cases are shown in .

Table 2

Baseline characteristics and surgical details.

Characteristics	Total cases (n = 58)
Gender (n, %)
Male	26 (44.8%)
Female	32 (55.2%)
Age (years)	59.3 ± 16.0
Diagnosis (n, %)
Malignant tumor	40 (69.0%)
Benign tumor	13 (22.4%)
Othersa	5 (8.6%)
TNM classificationb (40 cases) (n, %)
Stage I and II	15 (37.5%)
Stage III and IV	25 (62.5%)
Surgical site (n, %)
Mandible	44 (75.9%)
Maxilla	14 (24.1%)
Donor bone graft (n, %)
Fibula	54 (93.1%)
Iliac crest	4 (6.9%)
Donor bone length (mm)	89.7 ± 31.0
Bone segments (n, %)
One	12 (20.7%)
Two	31 (53.4%)
Three and more	15 (25.9%)
Concomitant surgery (n, %)
None	36 (62.1%)
Simultaneous dental implant	21 (36.2%)
Radial forearm flap	1 (1.7%)

Others: osteoradionecrosis of the jaw (n = 4) and mandibular defect secondary to malignancy resection (n = 1).

According to the AJCC (American Joint Committee on Cancer) Cancer Staging Manual (8th Edition).

Primary Outcome

Altogether, surgical failure occurred in 5.2% of patients (3/58). One was intraoperative flap failure due to sclerotic vessels and recurrent arterial thrombosis. The other two cases were postoperative flap failures caused by venous compromise of flap at postoperative day 4 and late-stage artery thrombosis at postoperative day 10. 3D-printed PSSP were successfully used in all the patients. shows the CUSUM analysis of the surgical success for the surgeon. After a surgical failure occurred in the 11th case, a steadily climbing line was seen and proficiency was first obtained after 23 cases. A new concomitant surgery of simultaneous dental implant placement with or without immediate loading was added to selective cases from the 23rd case. Two surgical failures occurred in the 26th and 33rd cases and the surgical performance line dropped below the acceptable boundary line. After that, proficiency was completely regained after 35 cases.

Figure 1

CUSUM analysis of surgical success for the CAJR with PSSP performed by the surgical team. The x-axis shows the chronological sequence of cases. The black line indicates the cumulative sum of surgical success. The horizontal red dotted lines represent the acceptable boundary line H1 and unacceptable boundary line H0. Proficiency was first achieved after 35 cases.

According to the achievement of proficiency, we divided the patients into two groups. From the baseline characteristics of the two groups in , there was no significant difference in the patient’s baseline features, except TNM stage (p = 0.046) and concomitant surgery (p < 0.001). From the 24th to 58th cases, the proportions of stage III & IV malignancy and concomitant surgery were significantly higher than the first 23 cases, indicating an increased proportion of cases with advanced stage malignancy and more complex surgery after first achieving the proficiency.

Table 3

Comparison of baseline characteristics among chronological 2 groups.

Characteristics	Group 1 (Case No.1-23)	Group 2 (Case No.24-58)	P value
Gender (n; %)
Male	8 (34.8%)	18 (51.4%)	0.212
Female	15 (65.2%)	17 (48.6%)
Age (years)	58.7±15.0	59.6±16.9	0.831
Diagnosis (n; %)
Malignant tumor	16 (69.6%)	24 (68.6%)	0.523
Benign tumor	4 (17.4%)	9 (25.7%)
Others	3 (13.0%)	2 (5.7%)
TNM classification (40 cases) (n; %)
Stage I & II	9 (56.3%)	6 (25.0%)	0.046
Stage III & IV	7 (43.8%)	18 (75.0%)
Surgical site (n; %)
Mandible	17 (73.9%)	27 (77.1%)	0.779
Maxilla	6 (26.1%)	8 (22.9%)
Donor bone graft (n; %)
Fibula	21 (91.3%)	33 (94.3%)	0.522
Iliac crest	2 (8.7%)	2 (5.7%)
Donor bone length (mm)	89.6±36.5	89.7±27.4	0.982
Bone segments (n; %)
One	4 (17.4%)	8 (22.9%)	0.771
Two	12 (52.2%)	19 (54.3%)
Three and more	7 (30.4%)	8 (22.9%)
Concomitant surgery (n; %)
No	22 (95.7%)	14 (40.0%)	<0.001
Yes	1 (4.3%)	21 (60.0%)

Secondary Outcomes

The mean operation time was 532.5 ± 119.2 min. The inflexion point is the 8th case, from which the operation time started to diminish, although there was a slight trend of increasing operation time from the 28th case to the 46th case. The linear and CUSUM analysis graphs of total operation time are shown in .

Figure 2

Linear graph and CUSUM analysis of total operation time in CAJR applying patient-specific surgical plates. The black line represents the curve of best fit in a general model with equation, y = 4E−06x 6 − 0.0006x 5 + 0.0299x 4 − 0.3358x 3 − 9.1732x 2 + 168.85x − 187.47, R² = 0.8506.

Mean intraoperative blood loss was 1,006.8 ± 547.2 ml. There was a trend of decreasing blood loss in the first 22 cases. After that, the intraoperative blood loss was increasing. The linear and CUSUM analysis graphs of intraoperative blood loss are shown in .

Figure 3

Linear graph and CUSUM analysis of intraoperative blood loss in CAJR applying patient-specific surgical plates. The black line represents the curve of best fit in a general model with equation, y = 4E−06x 6 − 0.0006x 5 + 0.0274x 4 − 0.2982x 3 − 1.1805x 2 − 94.942x + 133.67, R² = 0.7233.

Mean length of hospital stay was 16.1 ± 6.3 days. Four inflexion points were presented at the 5th, 17th, 28th, and 45th cases, respectively. The linear and CUSUM analysis graphs of hospital stay length are shown in .

Figure 4

Linear graph and CUSUM analysis of hospital stay length in CAJR applying patient-specific surgical plates. The black line represents the curve of best fit in a general model with equation, y = 6E−07x 6 − 0.0001x 5 + 0.0077x 4 − 0.2461x 3 + 3.7349x 2 − 23.636x + 31.376, R² = 0.7912.

Mean bone graft deviation was 0.9 ± 1.2 mm. After 15 cases, the bone graft deviation started to diminish as a general trend. However, from the 31st to 43rd cases, a slightly increasing trend of bone graft deviation was observed. The linear and CUSUM analysis graphs of bone grafts deviation are shown in .

Figure 5

Linear graph and CUSUM analysis of bone grafts deviation in CAJR applying patient-specific surgical plates. The black line represents the curve of best fit in a general model with equation, y = 5E−08x 6 − 8E−06x 5 + 0.0005x 4 − 0.0148x 3 + 0.1689x 2 − 0.2517x − 1.307, R² = 0.5936.

Discussion

The concept of learning curve in medicine was commonly defined as the time taken and/or the number of procedures an average surgeon needs to be able to perform a procedure independently with a reasonable outcome (30). As modern surgical training is always beset by problems like increased working hours, inadequate training facilities, lack of resources, and medicolegal issues (31), the understanding of learning curves on surgical procedures can make the training more efficient and standard. However, a study reviewed assessments of learning curve in health technologies and indicated that learning curves were rarely evaluated formally with a proper study design and statistical method (32). As a new tool, the CUSUM technique for learning curve has been introduced and proposed as a useful instrument in the field of surgical training, which allows quantitative monitoring of individual performance during the learning process (28, 33).

Nowadays, computer-assisted surgery is increasingly utilized in reconstructive surgery, while no literature has reported the learning curve for this surgical procedure. In the present study, we performed CUSUM analysis on the learning curve of CAJR using 3D-printed PSSP by a single surgical team. Although operation time is the most common determinant for the learning curve on surgery, the assessment on the basis of a single parameter might be simplistic (12, 34). We tried to analyze trends in multidimensional variables in the present study.

For the primary outcome, our study revealed a three-stage learning pattern of CAJR with the use of PSSP, including initial learning, plateau, and overlearning. The first stage is initial learning. The learning curve regarding surgical success stabilized after 11 cases, and first achieved proficiency after 23 cases. The second stage is plateau. When surgeons felt competent in this surgical procedure, the significantly increased proportions of tumor staging and concomitant surgery suggested that the complexity of surgery after 23 cases was higher than the earlier cases. The concomitant surgery, simultaneous dental implant placement with the aid of a “three-in-one” patient-specific surgical guide (35), was also a technically challenging procedure. Accordingly, a fluctuation of proficiency was observed at this stage. The final stage is overlearning. A steady proficiency was completely achieved after 35 cases. Overall, the surgical failures in our cohort were infrequent (5.2%), and no performance of the surgeon below the unacceptable boundary line was observed, which suggests that application of 3D-printed PSSP in CAJR with free flap is safe and feasible.

For the secondary outcomes, we found that although the four graphs of CUSUM analysis had different curve shapes, a similar ascending trend occurred from 23–31 cases to 43–46 cases. The increased complexity after 23 cases could also reasonably explain the increased total operation time and intraoperative blood loss. As a result, we actually observed two phenomena of “learning curve” in the CUSUM analysis graphs of the total operation time, length of hospital stay, and bone graft deviation. In the first learning curve (from 1st to 28th–31st cases), the inflexion points occurred between 8 and 17 cases, from which operation time, length of hospital stay, and bone graft deviation started to diminish. In the second learning curve (from 28th–31st to 58th cases), stabilizations of the operation time, length of hospital stay, and bone graft deviation occurred between 43 and 46 cases.

The main novelty of this study was using the CUSUM technique to analyze the learning curve of CAJR with PSSP. We utilized a chosen level of surgical failure rate as strict as two times the currently acceptable level (11.1%) to determine proficiency, which was one strength of our study (28). The cohort of patients from a prospective clinical trial without dropout was also the strength. Since no study described the learning curve of CAJR surgery, our work can provide the first quantitative assessment on this topic to the literature. It should be noted that our surgical team had previous experience in conventional free flap jaw reconstruction surgery and also CAJR without the use of PSSP (36), which could help shorten the learning process. Learning curve varies from different surgeons, but its stage pattern may be similar. Among three stages, the occurrence and persistence of a plateau depend on many reasons, such as the interference by previous experience, the nature of the task, and the motivation (37). Finding out the correct cause and getting over this period will be an important portion in surgical training.

There are certain limitations of the present study that need to be addressed. First, the CUSUM analysis included different diseases (malignant and benign tumors), surgeries (maxillary and mandibular reconstructions), and free flaps (fibular and iliac flaps), which influenced the homogeneity of enrolled cases. A multi-center clinical trial, with a big enough sample size of the same disease and surgery, might be needed to overcome this limitation. However, it will lead to other limitations of different expertise of multiple surgical teams and hospital setting. Second, post-operative oral function and quality of life are important outcomes for jaw reconstruction. However, there are a lot of confounding factors influencing these outcomes, which will need a well-designed prospective randomized control trial for further investigation. Thus, we did not include them in the present analysis. Last, the learning curve of time spent on preoperative preparation, such as virtual surgical planning and PSSP design, was not reported in this study. Previously, we reported that the time spent on virtual surgery and plate design was 18.8 ± 13.2 h, and the time taken for 3D printing, post-processing, and product delivery was 162.9 ± 55.2 h (38). The consecutive data of the time spent on preoperative preparation with a large sample size would provide a better understanding of the whole learning process of CAJR using PSSP.

Conclusion

A three-stage learning pattern of CAJR with the use of PSSP was revealed, including initial learning, plateau, and overlearning, which may guide the clinical teaching and training of CAJR using PSSP. Based on CUSUM analysis, surgical proficiency was obtained after 23 cases. Stabilization of total operation time, length of hospital stay, and bone graft deviation occurred after 8–17 cases. Our study provided evidence to guide the training of this new surgical procedure to ensure patient safety and clinical outcomes.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

W-yZ and Y-xS: study design. Y-xS: study supervision. W-yZ, WC, JP and W-fY: data collection. W-yZ and MW: manuscript preparation, statistical analysis, and data interpretation. All authors contributed to the article and approved the submitted version.

Funding

The study was funded by the Health and Medical Research Fund (No. 08192096), Hong Kong SAR, Guangdong Science and Technology Department (No. 2019A050516001), University Research Committee Platform Technology Funding 2021, and HKU Seed Fund for Translational and Applied Research (202010160032).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.