Evaluation of the Accuracy and Precision of a Next Generation Computer-Assisted Surgical System.

BACKGROUND: Computer-assisted orthopaedic surgery (CAOS) improves accuracy and reduces outliers in total knee arthroplasty (TKA). However, during the evaluation of CAOS systems, the error generated by the guidance system (hardware and software) has been generally overlooked. Limited information is available on the accuracy and precision of specific CAOS systems with regard to intraoperative final resection measurements. The purpose of this study was to assess the accuracy and precision of a next generation CAOS system and investigate the impact of extra-articular deformity on the system-level errors generated during intraoperative resection measurement.
METHODS: TKA surgeries were performed on twenty-eight artificial knee inserts with various types of extra-articular deformity (12 neutral, 12 varus, and 4 valgus). Surgical resection parameters (resection depths and alignment angles) were compared between postoperative three-dimensional (3D) scan-based measurements and intraoperative CAOS measurements. Using the 3D scan-based measurements as control, the accuracy (mean error) and precision (associated standard deviation) of the CAOS system were assessed. The impact of extra-articular deformity on the CAOS system measurement errors was also investigated.
RESULTS: The pooled mean unsigned errors generated by the CAOS system were equal or less than 0.61 mm and 0.64° for resection depths and alignment angles, respectively. No clinically meaningful biases were found in the measurements of resection depths (< 0.5 mm) and alignment angles (< 0.5°). Extra-articular deformity did not show significant effect on the measurement errors generated by the CAOS system investigated.
CONCLUSIONS: This study presented a set of methodology and workflow to assess the system-level accuracy and precision of CAOS systems. The data demonstrated that the CAOS system investigated can offer accurate and precise intraoperative measurements of TKA resection parameters, regardless of the presence of extra-articular deformity in the knee.

The long-term clinical success of the total knee arthroplasty (TKA) is critically dependent on the accurate positioning of the prosthesis.1,2,3) Studies have shown that malalignment can lead to various complications, such as component loosening and instability, polyethylene wear, and patellar dislocation.4,5,6) In the coronal plane, greater than 3° varus/valgus postoperative knee alignment has been found to increase the risk of negative outcomes,4,5) with mechanisms of failure generally being medial collapse for the varus malaligned knees, and ligament instability for the valgus malaligned knees.7,8) In the sagittal plane, malalignment of the components has been linked to an increased failure rate (3.3% and 4.5% for femur and tibia, respectively) compared to neutrally aligned components (0% and 0.2% for femur and tibia, respectively).9) Although numerous studies have stressed the importance of ensuring accurate component position and orientation, TKA performed using conventional instruments still largely relies on the surgeon's experience and skill level to achieve this goal. It has been reported that conventional implantation techniques involving the use of extramedullary or intramedullary mechanical instruments can only achieve satisfactory lower limb alignment (within ± 3° of varus/valgus relative to mechanical axis) in 60% to 80% of the cases.5,10,11) Another notable fact is that arthroplasty registry data indicates 20% to 25% of patients remain dissatisfied with the results of the surgery,12,13,14) which may partially be attributed to component malalignment.15)

Computer-assisted orthopaedic surgery (CAOS) has been shown to offer more accurate, reliable and reproducible component positioning compared to conventional techniques.16,17,18,19) In a 2008 cohort study, CAOS was found to provide both closer restoration to the neutral mechanical alignment and approximately double the cases of optimal alignment compared to the conventional instrumentation group.17) Sparmann et al.18) reported only 2% of outliers (> 3° in varus/valgus alignment) in the TKA cases using an imageless CAOS system (ExactechGPS, Blue-Ortho, Grenoble, France), compared to 22% of the conventional instrumentation cases. Also, the differences between the CAOS system and conventional alignment method were found to be significant in both coronal and sagittal planes. Similar conclusions were drawn in many other studies.16,19) However, despite CAOS have demonstrated its advantage over conventional instrumentation in term of implant positioning, wide application of this technology turned out to be challenging due to several limitations, including cost for the system, increased operating time, and bulky equipment. Furthermore, the intraoperative use of such systems can be cumbersome due to equipment is placed out of the surgical field, causing difficulties in user-system interaction. Furthermore, the optical markers in many tracker designs are prone to be easily blocked from the view of the camera by surgical staff and bloody fluid.

Recent advances in computer and optical technologies enabled the development of a next generation imageless CAOS system, which provides the solutions to multiple limitations of the traditional CAOS systems. Specifically, ExactechGPS allows the integrated camera and display unit to be located within the sterile field, providing maximum accessibility by the surgeon (Fig. 1A). The wireless active trackers are resistant to blood occlusion, ensuring optimum visualization by the optical camera. Also, the system enables the surgeon to easily customize individual preference in operative technique, instruments used, and surgical workflow, such that the surgery can be performed following the procedure he/she is trained and most comfortable with. Finally, patient specific TKA resections are enabled by the real-time guidance provided by the system in combination with the smart instrumentation, based on individual patient's anatomy.

Fig. 1

(A) A typical set up for the ExactechGPS system inside the sterile field. (B) A whole leg assembly used for this study. (C) A representative knee insert with anatomical landmarks identified using the metal probe. FDC: femur distal center, FDM: femur medial distal condyle, FDL: femur lateral distal condyle, FPM: femur medial posterior condyle, TPC: tibia proximal center, FPL: femur lateral posterior condyle, TPM: tibia lowest point on the medial peateau, TPL: tibia lowest point on the lateral plateau.

The introduction of such next generation CAOS system requires an understanding of its accuracy and precision, which can be dictated both in system level (hardware and software) and clinical alignment outcomes. Previous evaluations of CAOS systems for TKA have been mostly focused on the final implant position and alignment in the reconstructed knee joint.17,18,19) The intrinsic accuracy of the systems themselves has generally been overlooked. However, recent development in the CAOS related research pointed out that significant differences may exist across different CAOS systems in both coronal alignment and the number of radiographic outliers,20) and concluded that surgeons should not consider all the TKA CAOS systems to be equally accurate. This finding addressed the importance to evaluate the system accuracy for individual CAOS system to understand the errors generated by the hardware system and software algorithm on the surgical resection level. The purpose of this study was therefore to evaluate the accuracy and precision of the intraoperative surgical resection measurements performed by the ExactechGPS system under the lab setting, and assesses the impact of extra-articular knee deformity on measurement errors.

METHODS

Specimen Preparation

Twenty-eight synthetic knee inserts (MITA knee insert, Medical Models, Bristol, UK) were used in this study, including 12 neutral knees (Catalog no. M-00598), 12 varus knees (5° of deformity, Catalog no. M-00566), and 4 valgus knees (12° of deformity, Catalog no. M-00567). An artificial leg (MITA trainer leg, Catalog no. M-00058, Medial Models) was used to assemble with each insert to simulate the entire leg (Fig. 1B).

At the beginning of the study, a set of anatomical landmarks were annotated by firmly pressing a metal probe into the surface of the knee inserts to create small dimples with the same diameter as the tip of the ExactechGPS probe tracker (Fig. 1C). The landmarks acquired in this step and their abbreviations are listed in Table 1. Next, the knee inserts and the artificial leg were digitized individually using a three-dimensional (3D) scanner (Comet L3D, Steinbichler, Plymouth, MI, USA). The digitized surfaces were then created (Verify64 and DesignX 64 , Geomagic, Lakewood, CO, USA). Under the CAD environment (Unigraphics NX ver. 7.5, Siemens PLM Software, Plano, TX, USA), each insert was virtually assembled with the artificial leg to create a whole leg assembly. The preannotated surface landmarks were recreated on the knee inserts in Unigraphics by identifying the center of each surface dimple. Two additional landmarks were defined in Unigraphics (Table 1): (1) femoral head center (FHC): the center of the fitted sphere of the femoral head on the artificial leg; and (2) ankle joint center (TAC): the midpoint between medial and lateral malleoli, which were annotated on the surface of the ankle. Based on the landmarks, a set of anatomical axes and planes were established for each whole leg assembly in Unigraphics, serving as the reference for TKA resections. The detailed definition of the reference system is described in Table 2.

Table 1

Anatomical Landmarks Identified by Metal Probe and in Unigraphics

Landmark	Abbreviation
Femur (by metal probe)
Medial posterior condyle	FPM
Lateral posterior condyle	FPL
Medial distal condyle	FDM
Lateral distal condyle	FDL
Distal center	FDC
Femur (defined in Unigraphics)
Head center	FHC
Tibia (by metal probe)
Lowest point on the medial peateau	TPM
Lowest point on the lateral plateau	TPL
Proximal center	TPC
Tibia (defined in Unigraphics)
Ankle center	TAC

Table 2

Definition of the Reference Axes and Anatomical Planes for Total Knee Arthroplasty Resections

Variable	Abbreviation	Definition
Femur
Mechanical axis	FMA	Line connecting FHC and FDC
Posterior condylar line	FPCL	Line connecting FPM and FPL
Coronal plane	FCP	Plane parallel to both FMA and FPCL
Sagittal plane	FSP	Plane perpendicular to FCP and parallel to FMA
Tibia
Mechanical axis	TMA	Line connecting TPC and TAC
Coronal plane	TCP	Plane perpendicular to TSP and parallel to TMA
Sagittal plane	TSP	Plane passing through TMA and oriented to the second toe

FHC: femur head center, FDC: femur distal center, FPM: femur medial posterior condyle, FPL: femur lateral posterior condyle, TPC: tibia proximal center, TAC: tibia ankle center.

Computer-Assisted TKA Resection

Computer-assisted TKA resections were performed by a board-certified orthopaedic surgeon (RAL) using ExactechGPS guidance system on each physical whole leg assembly. The knees were prepared targeting following the Optetrak Logic PS knee implants operative technique (Exactech, Gainesville, FL, USA). First, a surgeon profile was set up in the CAOS guidance system according to the surgeon's preference on the philosophy, surgical flow, and instrumentation. Second, the probe tracker was used during the CAOS procedure to acquire the same set of anatomical landmarks by probing the precreated dimples on the surfaces of the whole leg assembly (Table 1), except for FHC, which was identified by the guidance system following the "rotational method." 21) The cutting blocks were then fixed onto the femur and tibia, adjusted individually to the desired resection parameters, and used to guide the saw blade for bone resections. The resections aimed for the restoration of the mechanical axis and accurate rotational alignment of the knee components. For this specific study, only the first resection on each bone type was made, namely, the distal resection of the femur and the proximal resection for the tibia.

The final surgical parameters were collected intraoperatively by the CAOS guidance system after the resections (intraoperatively measured surgical parameters) (Fig. 2A). The data recorded included the medial and lateral tibial resection depths, tibial varus/valgus alignment, tibial posterior slope, medial and lateral distal femoral resection depths, femoral varus/valgus alignment, and femoral flexion/extension angle. The measurement algorithms for the surgical parameters are summarized below: (1) Medial and lateral tibial resection depths: the perpendicular distances from tibia lowest point on the medial peateau (TPM) and tibia lowest point on the lateral plateau (TPL) to the tibial resection plane, respectively. (2) Tibial varus/valgus alignment: the angle between tibia mechanical axis (TMA) and the normal of the tibial resection plane, projected onto the tibia coronal plane (TCP). (3) Tibial posterior slope: the angle between TMA and the normal of the tibial resection plane, projected onto the tibia sagittal plane (TSP). (4) Medial and lateral distal femoral resection depths: the perpendicular distances from FPM and FPL to the distal femoral resection plane, respectively. (5) Femoral varus/valgus alignment: the angle between femur mechanical axis (FMA) and the normal of the distal femoral resection plane, projected onto the femur coronal plane. (6) Femoral flexion/extension angle: the angle between FMA and the normal of the distal femoral resection plane, projected onto the femur sagittal plane (FSP).

Fig. 2

Measurements of resection depths and alignment angles on the same representative tibia by ExactechGPS CAOS system (A) and using three-dimensional scan-based surface registration (B). CAOS: computer-assisted orthopaedic surgery, T: The CAOS system is referencing the Tibial Tracker in this step, G: The CAOS system is referencing the Guide Tracker in this step.

Postoperative Measurement

Following the TKA resections, 3D scans were repeated on each knee insert. The digitized postresection bone surfaces were registered with the corresponding whole bone surfaces. In Unigraphics, 3D model of the instrument used for intraoperative bone resection check was virtually placed on each resected tibia and femur. Surgical resection planes were recreated from the bone-contacting plane of the checker instrument. The same set of surgical resection parameters (actual surgical parameters) were measured in the predefined anatomical referencing system using Geomagic software platform (Fig. 2B). To assess the accuracy of the surface registration workflow, one tibia and one femur were selected from each deformity groups (neutral, varus, and valus). The surface distance error between each registered preoperative and postoperative bone surface pair was computed (3-matic 8.0, Materialise, Leuven, Belgium) and averaged across the 6 sampled bones. Both the mean surface distance (0.0007 mm) and its associated standard deviation (SD, 0.0037 mm) were found to be lower than the level of accuracy reported in this study (0.01 mm) (Fig. 3). The workflow was therefore confirmed to be sufficiently accurate.

Fig. 3

Representative tibia (A) and femur (B) demonstrating surface error (plotted on the resected bones) between registered preoperative and postoperative three-dimensional scanned surfaces. Distribution of the surface error is also shown for each bone.

Data Analysis

The unsigned and signed differences between the actual surgical parameters and intraoperatively measured surgical parameters were calculated. The unsigned differences represent the magnitude of error generated in the measurements performed by the CAOS guidance system. The signed differences however, identify any bias of the measurement error, such as a tendency of resecting towards varus (or valgus), flexion (or extension), more (or less) resection depth, or an increased (or reduced) posterior slope in the alignment. The accuracy (mean error) and precision (SD) of the CAOS guidance system on each surgical resection parameter were measured (for both the signed and unsigned difference). The 95% confidence interval (CI) was assessed for the signed differences for each resection parameter by a single sample Student t-test (Minitab, Minitab Inc., State College, PA, USA). A unitless error index was introduced as the overall indication of the error magnitude within a specific group of interest. It was calculated as the mean and SD of the unsigned errors combining all the dimensional and angular measurements within a specific bone type/deformity group. This definition deemed a difference of 1 mm and 1° in the surgical resections as of equivalent significance, as they are both at the same level of clinical detectability. The impact of preoperative knee condition on the accuracy and precision of the CAOS guidance system was investigated by comparing across the deformity groups. Statistical significance was defined as p < 0.05 (analysis of variance).

RESULTS

A summary of unsigned errors and error indexes can be found in Table 3. The data showed that minimum errors (≤ 0.68 mm for resection depths, ≤ 0.90° for angular measurements) were generated by the CAOS guidance system across all the deformity groups (neutral, varus, valgus, and deformed in general) and bone types (femur and tibia). The pooled mean errors were equal or less than 0.61 mm and 0.64° for resection depths and angular measurements, respectively. Regardless of bone type and deformity group, both the mean and SD of the error Indexes were small and clinically undetectable (means ≤ 0.57, SDs ≤ 0.39). No statistical difference was found in error index between tibia and femur, nor between the knee deformity groups.

Table 3

Unsigned Errors of the ExactechGPS System on Surgical Resection Parameter Measurements

Bone	Parameter	Varus	Neutral	Valgus	Deformed (varus + valgus)	Pooled
Tibia	Medial resection depth (mm)	0.35 ± 0.38	0.46 ± 0.29	0.40 ± 0.21	0.37 ± 0.34	0.41 ± 0.32
	Lateral resection depth (mm)	0.55 ± 0.63	0.38 ± 0.27	0.38 ± 0.38	0.51 ± 0.57	0.46 ± 0.46
	Varus/valgus alignment (°)	0.49 ± 0.35	0.54 ± 0.35	0.25 ± 0.14	0.43 ± 0.33	0.48 ± 0.33
	Posterior slope (°)	0.41 ± 0.29	0.90 ± 0.35	0.37 ± 0.21	0.40 ± 0.27	0.62 ± 0.39
	Error index (95% confidence interval)	0.45 ± 0.27 (0.28 to 0.62)	0.57 ± 0.11 (0.50 to 0.64)	0.35 ± 0.17 (0.08 to 0.63)	0.43 ± 0.39 (0.22 to 0.64)	0.49 ± 0.19 (0.42 to 0.56)
Femur	Medial resection depth (mm)	0.41 ± 0.21	0.34 ± 0.24	0.32 ± 0.38	0.38 ± 0.25	0.36 ± 0.24
	Lateral resection depth (mm)	0.53 ± 0.37	0.68 ± 0.46	0.63 ± 0.35	0.56 ± 0.36	0.61 ± 0.40
	Varus/valgus alignment (°)	0.25 ± 0.15	0.44 ± 0.32	0.22 ± 0.11	0.24 ± 0.14	0.33 ± 0.25
	Flexion/extension angle (°)	0.54 ± 0.41	0.68 ± 0.52	0.80 ± 0.55	0.60 ± 0.45	0.64 ± 0.47
	Error index (95% confidence interval)	0.43 ± 0.13 (0.35 to 0.52)	0.54 ± 0.23 (0.39 to 0.68)	0.49 ± 0.19 (0.21 to 0.78)	0.45 ± 0.34 (0.27 to 0.63)	0.49 ± 0.15 (0.43 to 0.55)

Values are presented as mean ± standard deviation.

Regardless of the nature of the knee deformity, the mean signed error of the CAOS guidance system was systematically less than 0.5 mm for bone resection depths, and equal or less than 0.9° for joint angle measurements (Table 4), with pooled means less than 0.5 mm and 0.5°, respectively. The guidance system was shown to have a slight tendency to measure more in distal femoral resection depth and increased femoral valgus compared to the actual resection (negative values in the mean errors). However, the biases were not clinically meaningful (< 0.50 mm in resection depths, < 0.50° in varus/valgus alignment measurements). No other biases were found in the rest of the surgical resection parameters. The 95% CIs were in the ranges of -1.44 to 0.67 mm for bone resection depths, and -0.64° to 1.67° for angular measurements. The differences across bone types and deformity groups were not found to be statistically significant.

Table 4

Signed Errors of the ExactechGPS System on Surgical Resection Parameter Measurements

Bone	Parameter*	Varus	Neutral	Valgus	Deformed (varus + valgus)	Pooled
Tibia	Medial resection depth (mm)	0.10 ± 0.52 (-0.23 to 0.43)	0.46 ± 0.29 (0.28 to 0.64)	-0.33 ± 0.34 (-0.87 to 0.21)	-0.01 ± 0.51 (-0.28 to 0.26)	0.19 ± 0.48 (0.00 to 0.38)
	Lateral resection depth (mm)	-0.35 ± 0.77 (-0.84 to 0.14)	0.23 ± 0.42 (-0.03 to 0.50)	-0.32 ± 0.45 (-1.04 to 0.40)	-0.34 ± 0.69 (-0.71 to 0.03)	-0.10 ± 0.65 (-0.35 to 0.15)
	Varus/valgus alignment (°)	0.05 ± 0.62 (-0.35 to 0.44)	-0.25 ± 0.61 (-0.64 to 0.14)	0.05± 0.32 (-0.46 to 0.56)	0.05 ± 0.55 (-0.24 to 0.34)	-0.08 ± 0.58 (-0.31 to 0.15)
	Posterior slope (°)	-0.07 ± 0.51 (-0.40 to 0.25)	0.90 ± 0.35 (0.68 to 1.12)	0.37 ± 0.21 (0.02 to 0.71)	0.04 ± 0.49 (-0.22 to 0.30)	0.41 ± 0.61 (0.17 to 0.65)
Femur	Medial resection depth (mm)	-0.13 ± 0.45 (-0.42 to 0.16)	-0.04 ± 0.42 (-0.31 to 0.23)	-0.13 ± 0.50 (-0.93 to 0.67)	-0.13 ± 0.50 (-0.40 to 0.14)	-0.09 ± 0.43 (-0.26 to 0.08)
	Lateral resection depth (mm)	-0.49 ± 0.43 (-0.77 to 0.22)	-0.49 ± 0.67 (-0.92 to 0.07)	-0.42 ± 0.64 (-1.44 to 0.59)	-0.42 ± 0.64 (-0.76 to -0.08)	-0.48 ± 0.55 (-0.69 to -0.27)
	Varus/valgus alignment (°)	-0.07 ± 0.29 (-0.26 to 0.11)	-0.41 ± 0.37 (-0.64 to 0.17)	-0.07 ± 0.27 (-0.50 to 0.35)	-0.07 ± 0.27 (-0.21 to 0.07)	-0.22 ± 0.36 (-0.36 to 0.08)
	Flexion/extension angle (°)	0.23 ± 0.66 (-0.19 to 0.65)	0.15 ± 0.87 (-0.40 to 0.70)	0.80 ± 0.55 (-0.08 to 1.67)	0.80 ± 0.55 (0.51 to 1.09)	0.28 ± 0.75 (-0.01 to 0.57)

Values are presented as mean ± standard deviation (95% confidence interval).

*Positive error indicates that compared to the actual surgical parameters, the intraoperative measured parameters had: (1) less bone resection depth, more varus alignment, and decreased posterior slople in the tibia and (2) less bone resection depth, more varus alignment, and less extension in the femur.

DISCUSSION

Accurate TKA bone resection is crucially important for the accurate placement of the component. In the TKA cases using conventional mechanical alignment guides, achieving proper bony resection depends on the design and accuracy of the instruments, surgical assumptions such as valgus alignment adjustment from the mechanical axis, as well as the experience and skill level of the surgeon. None of these factors can be free of error, nor is quantitative real-time feedback available during the procedure. Clinical studies have reported outliers in postoperative limb alignment to be ranged from 26% to 28% in the conventional group, compared to 0% to 3% in the navigation group.16,18,19) The results of this study demonstrated that ExactechGPS system can offer intraoperative imageless surgical assistance with both high accuracy and precision. The contribution of the system itself to the total surgical variability was shown to be clinically negligible (sub-millimeter for resection depth, and sub-degree for alignment). Also, the system does not significantly bias the measurements, providing highly reliable feedbacks during the surgical operation. Furthermore, the results showed that ExactechGPS consistently provides accurate and precise measurements regardless of the status of preoperative extra-articular knee deformity (neutral, varus, or valgus).

Intraoperative measurement of surgical resection parameters during imageless computer-assisted TKA surgery is a critical step, in which a surgeon directly relies on the real-time data obtained by the optical trackers to prepare the bony resections and check the final realized cuts. As pointed out by a previous study,22) computer-assisted surgical systems provides a "smart" user interface at the surgical application level, which tends to cause the overlooking of the underlying hardware setup and software algorithm during the assessment of CAOS systems. As the result, numerous studies have investigated the impact of landmarking and overall clinical accuracy of the computer-assisted surgical in knee arthroplasty,17,18,19,23,24) yet limited information is available on the error caused by the CAOS systems themselves during the intraoperative measurement of surgical resection parameters. In a 2004 study, Wiles et al.22) quantified the accuracy of an optical tracking system by assessing the distance error between position measurements performed by the system and the benchmark locations. Although the study provided great contribution to the methodology for assessing the accuracy of such systems, interpolation of the reported single-marker and rigid body based errors to clinical meaningful surgical resection parameters may be challenging. Another published investigation assessed the accuracy of an imageless navigation system by comparing measured alignment data between an imageless CAOS system and a digital caliper for various knee deformity types. However, the manual probing process may be subjected to human error, and only alignment angles were studied. This present study reported comparable or higher level of accuracy for the ExactechGPS system for the angular measurements with additional accuracy assessment on the surgical resection depths.

This study presented a set of methodology and workflow to assess the system-level accuracy of CAOS systems. The use of 3D scanned data provided a high resolution, non-contact method that eliminated errors associated with users or movement during data acquisition compared to using a digital caliper unit. Furthermore, anatomical landmarks were annotated and preserved on both pre- and postoperative knees, which ensures the anatomical based surgical referencing system to be consistent throughout the assessment for accurate registration and measurement of the bone resections. Both advantages offer improved accuracy of the measurement workflow. Especially given the small magnitude of the reported errors, errors from data acquisition and processing in this study were kept to a minimum. One limitation of the study was that it was performed in vitro. Since the error was calculated between the intraoperative measured and actual resection parameters on the finished bony cut, lack of soft tissue environment was not expected to affect the data. However, the impact of other factors in the operating room setting (e.g., blood occlusion, presence of surgical staff and other surgical equipment in the camera field) on the results may be further investigated.

In conclusion, this study demonstrated that the ExactechGPS system can offer both accurate and precise imageless intraoperative surgical resection measurements during computer-assisted TKA, regardless of the deformity status of the knees. The errors generated by this CAOS guidance system were clinically negligible.