Validating a Semi-Automated Technique for Segmenting Femoral Articular Cartilage on Ultrasound Images.

OBJECTIVE: To validate a semi-automated technique to segment ultrasound-assessed femoral cartilage without compromising segmentation accuracy to a traditional manual segmentation technique in participants with an anterior cruciate ligament injury (ACL).
DESIGN: We recruited 27 participants with a primary unilateral ACL injury at a pre-operative clinic visit. One investigator performed a transverse suprapatellar ultrasound scan with the participant's ACL injured knee in maximum flexion. Three femoral cartilage ultrasound images were recorded. A single expert reader manually segmented the femoral cartilage cross-sectional area in each image. In addition, we created a semi-automatic program to segment the cartilage using a random walker-based method. We quantified the average cartilage thickness and echo-intensity for the manual and semi-automated segmentations. Intraclass correlation coefficients (ICC2,k) and Bland-Altman plots were used to validate the semi-automated technique to the manual segmentation for assessing average cartilage thickness and echo-intensity. A dice correlation coefficient was used to quantify the overlap between the segmentations created with the semi-automated and manual techniques.
RESULTS: For average cartilage thickness, there was excellent reliability (ICC2,k = 0.99) and a small mean difference (+0.8%) between the manual and semi-automated segmentations. For average echo-intensity, there was excellent reliability (ICC2,k = 0.97) and a small mean difference (-2.5%) between the manual and semi-automated segmentations. The average dice correlation coefficient between the manual segmentation and semi-automated segmentation was 0.90, indicating high overlap between techniques.
CONCLUSIONS: Our novel semi-automated segmentation technique is a valid method that requires less technical expertise and time than manual segmentation in patients after ACL injury.

Introduction

After anterior cruciate ligament (ACL) injury and reconstruction (ACLR), patients are at an increased risk of knee osteoarthritis.[1,2] Approximately one-third of patients exhibit radiographic knee osteoarthritis within the first decade following ACLR. Alterations in femoral articular cartilage morphology (e.g., cartilage thinning or thickening) are a hallmark sign of knee osteoarthritis. Monitoring cartilage morphology alterations following ACL injury may be a way for identifying the patients at highest risk for early-onset knee osteoarthritis.[4,5] This earlier recognition of people at high risk for osteoarthritis is a needed next step for developing osteoarthritis prevention strategies that intervene early in the disease process and change the course of the disease.

Prior magnetic resonance imaging (MRI) studies indicate that the femoral trochlea is a common anatomical site for knee pathology (i.e., bone marrow lesions, cartilage lesions, osteophytes) in patients’ post-ACL reconstruction.[7-9] Diagnostic ultrasound is a clinically feasible imaging modality that is a valid tool for assessing femoral trochlear cartilage thickness. Our prior work has used diagnostic ultrasound to identify cartilage thickening following ACL reconstruction and has observed a relationship between pre-operative cartilage echo-intensity and osteoarthritis-related symptoms at 1-year post-ACL reconstruction.[11-13] However, these prior ultrasound studies relied on manual techniques to segment femoral cartilage images,[11-13] which are time-consuming and require a level of technical expertise that may limit translation of ultrasound cartilage imaging to a clinical setting.

To improve upon the prior manual segmentation techniques, we developed a new semi-automated technique for segmenting femoral cartilage on ultrasound images that simultaneously reduces the time needed to perform the segmentation and imaging expertise needed for the rater to successfully perform the segmentation ( ). The purpose of this study is to determine whether this semi-automated technique for segmenting femoral cartilage ultrasound images is a valid alternative when compared with a traditional manual segmentation technique. We hypothesized that our novel semi-automated segmentation technique for femoral cartilage on ultrasound images will produce similar average thickness and echo-intensity results compared with a traditional manual segmentation technique. Developing a time-efficient, validated semi-automated technique that requires less expertise to complete the segmentation is needed to translate ultrasound as a tool for monitoring femoral cartilage morphology in people at risk for osteoarthritis.

Description of Manual and Semi-Automated Femoral Cartilage Segmentation Techniques and the Associated Segmentation Masks. (A) Unenhanced B-mode ultrasound image used for the manual segmentation technique. (B) Enhanced ultrasound image used for the semi-automated segmentation technique using a frequency domain image filtering approach. (C) Manual segmentation technique using the freely available ImageJ software. Each asterisk (*) represents a single point used to outline the entire cartilage area. (D) Semi-automated segmentation technique using the custom Matlab program. The semi-automated technique requires identifying 10 seed points within the cartilage (green stars) and 10 seed points outside of the cartilage (red stars). (E) Segmentation mask associated with the manual segmentation technique. (F) Segmentation mask associated with the semi-automated segmentation technique. The semi-automated segmentation program uses the location and pixel intensity of the seed points in Figure D to segment the entire cartilage area.

Methods

Participants

We recruited participants with a primary unilateral ACL injury at a pre-operative visit with a single orthopedic surgeon. We included participants who were 18 to 35 years old and scheduled to undergo an ACL reconstruction. We excluded participants based on the following criteria: previous surgery of the lower extremity, knee injury within the prior 6 months (other than the ACL injury), a current knee injury that involved other knee ligaments, or previous diagnosis of any form of arthritis. This study was approved by our institution’s Institutional Review Board prior to the start of data collection. We obtained written informed consent from each participant prior to data collection.

Ultrasound Methodology Used to Image the Femoral Cartilage

A single investigator (MSH) performed a transverse suprapatellar ultrasound scan using a LOGIQe ultrasound machine (GE Healthcare, Chicago, IL) to acquire images of the femoral cartilage in the ACL injured knee. This investor has 7 years of experience using ultrasound to assess femoral cartilage and has demonstrated excellent intra-rater reliability (ICC2, k ≥ 0.93) using this technique.

We positioned participants on a hospital bed for 30 minutes prior to the ultrasound assessment. We instructed participants to position their ACL injured knee in maximal flexion (≥110°) to uncover the femoral cartilage surface from behind the patella and allow for visualization of the femoral cartilage. We recorded the maximum knee flexion angle for all participants. We placed a 12L-RS linear probe (GE Healthcare) in a transverse suprapatellar position in line with the apex of the femoral condyles and rotated the probe until it was perpendicular to the femoral cartilage surface ( ). We acquired three ultrasound images on their ACL injured knee with the probe being removed and repositioned between the acquisition of each image.

Manual Femoral Cartilage Segmentation

The investigator (MSH) who acquired the ultrasound images manually segmented the femoral cartilage cross-sectional area in all ultrasound images using the publicly available ImageJ software (https://imagej.nih.gov/). The same investigator (MSH) segmented the entire imaged cartilage cross-sectional area between the cartilage-bone (deep) and cartilage-soft tissue (superficial) interface ( ). The ImageJ segmentation regions of interest (ROIs) for all three images of the ACL injured knee were converted to binary segmentation masks that will be used in a later step to compare the manual and semi-automated segmentation techniques ().

Semi-Automated Cartilage Segmentation

We created a computational method, written in Matlab (MathWorks, Natick, MA), to semi-automatically segment the femoral cartilage based on a random walker image segmentation method. A novice reader (NM) used this semi-automated program to segment the same ultrasound images that were manually segmented by the expert reader. First, we roughly cropped the ultrasound image by dragging a rectangular box around the imaged cartilage to remove the rest of the ultrasound image prior to completed cartilage segmentation. The program then used a local phase-based image enhancement method to improve the contrast between the hypo-echoic femoral cartilage and the surrounding tissue ( ). Local phase image enhancement involved filtering the B-mode ultrasound images using a frequency domain image filtering approach. Specifically, we used a bandpass quadrature log-Gabor filter as the frequency domain filter. The enhanced images were used as an input to the random walker image segmentation method. Random-walker method is a graph-based interactive segmentation method where the user is asked to initialize random seed points belonging to foreground (object to be segmented) and background regions. The algorithm then computes the probability, for each seed point, that a random walker leaving that seed location will first arrive at a foreground seed before arriving at a background seed. In our study, the novice reader selected 10 seed points within the imaged cartilage (foreground) and 10 seed points outside the cartilage (background) ( ). The number of seed points were determined empirically and provided the best semi-automatic segmentation accuracy with optimum segmentation time. The number of seed points were kept constant during the segmentation of all the enhanced cartilage images. We set a threshold of 0.95 for the probability needed to be assigned as “cartilage” in the segmentation. The program used this information to create a binary segmentation mask using the same coordinate plane as the manual segmentation mask ().

Calculating Average Cartilage Thickness and Echo-Intensity

The program then imported the manual segmentation mask. To ensure the same region of the ultrasound image analyzed using the manual and semi-automated segmentations, the image crop applied by the novice reader during the semi-automated segmentation technique was also applied to the manual segmentation mask. The program then calculated the average cartilage thickness and echo-intensity throughout the segmented femoral cartilage for the manual and semi-automated segmentation masks. To calculate the average cartilage thickness, we divided the cross-sectional area of the segmentation by the cartilage length. To calculate the cartilage length, we first created a line that bisected the cartilage area throughout the entire length of the segmentation using morphological skeletonizing and then measured the distance of the extracted line using geodesic distance transform. The average echo-intensity was calculated as the average pixel intensity (0 [i.e., black] to 255 [i.e., white] arbitrary units [AU]) throughout the segmented cartilage. The average cartilage thickness and echo-intensity were averaged across each participant’s three images for the manual and semi-automated segmentation.

Statistical Analysis

Two-way random effect intraclass correlation coefficients based on absolute agreement (ICC2,k), standard error of the measurement (SEM), and Bland-Altman plots with 95% confidence limits were used to validate the average cartilage thickness and echo-intensity from the semi-automated segmentation to the values from the manual segmentation. ICC values less than 0.5 were considered poor reliability, values between 0.75 and 0.9 were considered good reliability, and values greater than 0.90 were considered excellent reliability. SEM was calculated between the semi-automated and manual segmentations to establish the measurements’ precision. Bland-Altman plots were used to provide an indication of the systematic error. The Bland-Altman plots graph the mean of the ultrasound measures between the manual and semi-automated techniques for each participant (x-axis) against the percent difference between the manual and semi-automated techniques (y-axis). The 95% upper and lower bound limits of agreement were determined for the mean percent difference, with excellent agreement defined as no more than 5% of all data points falling outside of the limits of agreement. A linear regression was used to assess the relationship between the percent difference and the mean of the measurements from the semi-automated and manual segmentations. The purpose of this analysis was to determine whether the amount of bias was dependent on the magnitude of the measurements (e.g., is there a greater difference between the manual and semi-automated techniques as the cartilage thickness increases). A dice correlation coefficient was used to quantify the overlap between the locations segmented with the semi-automated and manual techniques for each image. The mean and standard deviation dice correlation coefficient was calculated across all participants. We recorded the time it took to segment a separate set of 120 cartilage ultrasound images using the semi-automated and manual segmentation techniques to compare the average time it takes to complete a single image segmentation. The ICC and dice correlation coefficient analysis were performed with SAS Enterprise 9.4 (Cary, NC), while the Bland-Altman plot analyses were performed with MedCalc Statistical Software 19.2.6 (Ostend).

Results

We included 27 participants in this study. A majority of the participants were male (n = 16), with an average height of 173 ± 10 cm, mass of 74.5 ± 15.0 kg, age of 24.0 ± 4.7 years old, and 44.7 ± 49.3 days since ACL injury. For average cartilage thickness, there was excellent reliability (ICC2,k = 0.99) and precision (SEM = 0.03 mm), as well as a minimal mean difference (+0.8%, 95% confidence interval [CI]: −2.6% to 4.1%; 0.02 mm, 95% CI: −0.05 mm to 0.09 mm) between the manual and semi-automated segmentations ( ). For average cartilage thickness, the difference between the manual and semi-automated segmentation is not dependent on the magnitude of the mean of the two techniques (R2 = 0.01, P = 0.61, ). For average echo-intensity, there was excellent reliability (ICC2,k = 0.97) and precision (SEM = 1.18 AU), as well as a minimal mean difference (−2.5%, 95% CI: −5.5% to 0.60%; −1.89 AU, 95% CI: −4.05 AU to 0.46 AU) between the manual and semi-automated segmentations ( ). For average echo-intensity, the difference between the manual and semi-automated segmentation is not dependent on the magnitude of the mean of the two techniques (R2 = 0.14, P = 0.053). The mean ± standard deviation dice correlation coefficient between the manual segmentation and semi-automated segmentation was 0.90 ± 0.01 (range 0.87-0.92), which indicates high overlap between the spatial location of the two segmentations. The average time to complete a semi-automated segmentation of a single ultrasound image was 50 seconds compared with an average time of 400 seconds to complete a manual segmentation of a single ultrasound image. Therefore, to assess six images (i.e., the number of images needed for a bilateral assessment of three images per knee), it would take ~5 minutes using the semi-automated segmentation technique and ~40 minutes using manual segmentation technique.

Bland Altman plots comparing cartilage ultrasound characteristics between the manual and semi-automated segmentation techniques. The figures below plot the mean of the ultrasound measures between the manual and semi-automated techniques for each participant (x-axis) against the percent difference between the manual and semi-automated techniques (y-axis). The upper and lower bound limits of agreement were determined as 1.96 times the standard deviation of the mean differences, with excellent agreement determined as no more than 5% of all data points falling outside of the limits of agreement. A linear regression was used to assess the relationship between percent difference and the magnitude of the means. There were small mean differences between the manual and semi-automated techniques for both the average cartilage thickness (0.8% or 0.02 mm; Figure A) and average cartilage echo-intensity (–2.5% or −1.80 arbitrary units [AU]; Figure B). The difference between the manual and semi-automated segmentation is not dependent on the magnitude of the mean for average cartilage thickness (R2 = 0.01, P = 0.61) or echo-intensity (R2 = 0.14, P = 0.053).

Discussion

The results of this study highlight the agreement between our novel semi-automated technique and the traditional manual technique for segmenting femoral articular cartilage on transverse suprapatellar ultrasound images in patients after ACL injury. The mean dice correlation coefficient highlights the high spatial overlap between the manual and semi-automated segmentation techniques, which indicates that the location assessed by the novice reader using semi-automated segmentation is very similar to the location segmented manually by the expert reader. In addition, there was excellent reliability and minimal mean differences for both the average cartilage thickness and echo-intensity. Therefore, this semi-automated segmentation technique was validated to the manual segmentation technique for assessing femoral articular cartilage on ultrasound images in patients after ACL injury. This is important because the semi-automated segmentation technique requires less reader expertise and overall time to complete the segmentation, which will increase the translation of using ultrasound to quantitatively assess femoral cartilage morphology in future studies.

The results of this study highlight that our new semi-automated segmentation technique was validated to the manual technique for segmenting femoral cartilage on ultrasound images in patients after ACL injury. For average cartilage thickness, there was excellent reliability and a mean difference of 0.8% between the semi-automated and manual segmentation techniques, which equates to an average of only 0.02 mm difference between the two techniques ( ). Similarly, for average echo-intensity, there was also excellent reliability and a mean difference of 2.5% between the two techniques ( ). The dice correlation coefficient average of 0.90 indicates high overlap between the locations segmented by the semi-automated and manual segmentation techniques. To put this into perspective with other recent cartilage segmentation techniques, our dice correlation coefficient of 0.90 is similar to that observed in the 2019 International Workshop on Osteoarthritis Imaging Knee Segmentation Challenge. In this prior study, six teams of osteoarthritis imaging experts created automatic methods to create three-dimensional (3D) segmentations of knee cartilage using a standard set of magnetic resonance images from the Osteoarthritis Initiative that were previously segmented manually. Similar to our results, the average dice correlation between the automated and manual segmentation techniques for the femoral cartilage was between 0.87 and 0.90 for the different teams. In this prior study, the median percent difference between the automated and manual segmentation techniques in femoral cartilage thickness was approximately 5% for these 3D segmentations, while in our current study the median percent difference in femoral cartilage thickness was 0.8% for our two-dimensional (2D) segmentations. While one would expect more error in the more complicated 3D segmentations throughout the entire knee when compared with our 2D segmentations of a single ultrasound image, it is reassuring that our error is well below the error from previous cartilage segmentation studies. Therefore, the high dice correlation, excellent ICCs, and the minimal mean difference between the two techniques indicate that our semi-automated ultrasound segmentation technique was validated to the traditional manual ultrasound segmentation technique for assessing femoral cartilage in patients post ACL injury. These improvements will be extremely beneficial and cost-saving for future studies that will use ultrasound to monitor alterations in femoral cartilage.

This study builds upon the previous studies that have used manual segmentation techniques to quantify cartilage morphology on femoral ultrasound images.[10-14,27-36] The earliest quantitative methods required a reader to manually segment a straight line perpendicular to the cartilage surface at a subjective location in three regions of the imaged cartilage.[10,31,33] However, this approach only had modest intra- and inter-rater reliability likely due to small deviation in the subjective placement of the thickness lines resulting in large thickness differences that may not adequately represent thickness in the imaged cartilage. Therefore, we developed a cartilage cross-sectional area segmentation technique that required a reader to manually segment the entire cartilage cross-sectional area to calculate the average cartilage thickness. This segmentation technique removed some of the subjectivity of selecting the location to assess cartilage thickness, as well as quantifying average cartilage thickness with excellent intra- and inter-rater reliability. However, the translation of this manual technique is limited due to the amount of time needed to complete the segmentation of the entire imaged cartilage (~40 minutes for a bilateral assessment of three images per knee) and requires extensive training and technical expertise to adequately segment the imaged cartilage. As our results indicate that the semi-automated technique was validated to the manual technique in patients after ACL injury, future studies can deploy our semi-automated technique using novice readers to segment femoral cartilage rapidly (~5 minutes for a bilateral assessment of three images per knee) and accurately, which will reduce the overall costs of the study and improve the translation of this technique into clinical research environments.

The results of this study indicate that our semi-automated technique is a valid alternative to traditional manual segmentation of femoral cartilage ultrasound images in patients after ACL injury; however, there are some limitations that should be discussed. First, despite the high agreement and minimal mean difference in average echo-intensity between the semi-automated and manual segmentation techniques, there may be systematic error between the techniques as the semi-automated technique is almost always greater than the manual technique ( ). Future work may be needed to better understand the source of this systematic error and whether we can update the semi-automated technique to limit this bias. Further work is also needed to compare the sensitivity and responsiveness to change between the manual and semi-automated techniques. Demonstrating that the semi-automated technique is sensitive and responsive to change will be imperative in future studies that use this technique to monitor longitudinal alterations in cartilage thickness in pathological populations. In addition, our study focused on participants following an acute knee injury who are at an increased risk for cartilage alterations and future knee osteoarthritis. However, reliability and validity need to be tested in different pathological populations since other populations (e.g., diagnosed with osteoarthritis) may have worse cartilage health (e.g., more irregular nature of cartilage boundaries) and we will need to confirm that the semi-automated program can accurately segment less healthy cartilage. Currently, this study focused on a quantitative assessment of the average cartilage thickness throughout a localized portion of the femoral trochlea. However, this current technique does not allow for a localized analysis of cartilaginous lesions within the ultrasound image.

While this study is a necessary next step for decreasing the time and expertise needed to validly segment femoral articular cartilage on ultrasound images, further work is needed to increase the clinical translation of quantitative femoral cartilage morphologic assessment on ultrasound images. For example, even though we have removed most of the human interaction, there is still a minimal amount of human interaction needed to complete the segmentation. Further work and refinements of the program that integrate more complex machine learning techniques are needed to remove the human interaction and create a fully automated segmentation program. There has been significant work creating fully automated cartilage segmentation programs for magnetic resonance images,[26,37] but there has been minimal work attempting to apply these methods to cartilage ultrasound images.[38,39] Therefore, further work is needed to develop more advanced machine learning–based segmentations for femoral cartilage ultrasound images. For example, prior work indicates that combining local phase-based images with B-mode ultrasound data improves the segmentation accuracy of the state-of-the-art machine learning methods.[40-42] Our future work will involve the development of a fully automatic segmentation methods by incorporating the local phase cartilage images as an additional feature into the machine learning methods.

In conclusion, we demonstrated the agreement between our novel semi-automated technique and the traditional manual technique to segment femoral articular cartilage on ultrasound images. This highlights that our semi-automated technique was validated to the manual technique in patients after ACL injury. This is important because the semi-automated technique can be performed quickly by novice readers, which will help reduce the costs of image analysis needed for future studies that longitudinally monitor cartilage thickness in patients at risk for osteoarthritis and is an initial step to making quantitative cartilage thickness analysis more clinically feasible.