Artificial Intelligence-enabled, Real-time Intraoperative Ultrasound Imaging of Neural Structures Within the Psoas: Validation in a Porcine Spine Model.

STUDY
DESIGN: Experimental in-vivo animal study.
OBJECTIVE: The aim of this study was to evaluate an Artificial Intelligence (AI)-enabled ultrasound imaging system's ability to detect, segment, classify, and display neural and other structures during trans-psoas spine surgery. SUMMARY OF BACKGROUND DATA: Current methodologies for intraoperatively localizing and visualizing neural structures within the psoas are limited and can impact the safety of lateral lumbar interbody fusion (LLIF). Ultrasound technology, enhanced with AI-derived neural detection algorithms, could prove useful for this task.
METHODS: The study was conducted using an in vivo porcine model (50 subjects). Image processing and machine learning algorithms were developed to detect neural and other anatomic structures within and adjacent to the psoas muscle while using an ultrasound imaging system during lateral lumbar spine surgery (SonoVision,™ Tissue Differentiation Intelligence, USA). The imaging system's ability to detect and classify the anatomic structures was assessed with subsequent tissue dissection. Dice coefficients were calculated to quantify the performance of the image segmentation.
RESULTS: The AI-trained ultrasound system detected, segmented, classified, and displayed nerve, psoas muscle, and vertebral body surface with high sensitivity and specificity. The mean Dice coefficient score for each tissue type was >80%, indicating that the detected region and ground truth were >80% similar to each other. The mean specificity of nerve detection was 92%; for bone and muscle, it was >95%. The accuracy of nerve detection was >95%.
CONCLUSION: This study demonstrates that a combination of AI-derived image processing and machine learning algorithms can be developed to enable real-time ultrasonic detection, segmentation, classification, and display of critical anatomic structures, including neural tissue, during spine surgery. AI-enhanced ultrasound imaging can provide a visual map of important anatomy in and adjacent to the psoas, thereby providing the surgeon with critical information intended to increase the safety of LLIF surgery.Level of Evidence: N/A.

Lateral lumbar interbody fusion (LLIF) is an established and effective surgical technique.[1-5] However, the trans-psoas approach puts elements of the lumbar plexus at risk.[6-8] To mitigate this, electromyography (EMG) is used for detecting and avoiding motor nerves while traversing the psoas and accessing the disc space. Proximity to motor nerves is inferred using a stimulated probe within the psoas and measuring the EMG response.[6-8] Unfortunately, this process provides neither true spatial nerve location nor identification of sensory branches. Furthermore, patient comorbidities (e.g., diabetes), anesthetic paralytics, electrical interference, and technical issues can reduce the accuracy of EMG in this setting.[9,10,11] In addition, LLIF necessitates the use of C-Arm fluoroscopy or other ionizing radiation imaging modalities. There is increasing concern about the risk of radiation exposure to surgeons and patients in lateral spine surgery as well as in other spinal procedures.[12-17]

Ultrasound is an important diagnostic tool because it facilitates the visualization of patient anatomy without the risk of radiation exposure associated with other imaging modalities, such as fluoroscopy and computerized tomography (CT).[18-20] The use of ultrasound is ubiquitous in some disciplines of medicine.[18,21,22] Advances have occurred in many aspects of ultrasound technology including sensors, computer hardware, and software. It is used in spinal applications,[23,24] and has even been used in the cervical spine in outer space.[25,26]

A recent review summarized the use of ultrasound in spinal diagnostic and therapeutic applications[27] and another reviewed the use of ultrasound in spine surgery,[24] claiming the use of intraoperative ultrasound was mainly for tumors or calcified thoracic discs. A more recent study reported the clinical evaluation of ultrasound for lateral lumbar spine surgery.[28] The potential appeal of ultrasound for lateral spine surgery is its ability to visualize soft tissues and vasculature using Doppler imaging with color overlays indicating the direction and velocity of blood flow.[18,28,29] In addition, ultrasound has the potential to reduce radiation exposure to surgeons and patients.

Given the limitations of current methodologies for intraoperatively localizing and visualizing neural structures during trans-psoas surgery, a need exists for improving the safety of LLIF. Ultrasound technology, enhanced with artificial intelligence (AI)-derived neural detection algorithms, could prove useful for doing so. In the present study, we evaluated the use of an artificial intelligence (AI)-enabled, real-time intraoperative ultrasound system for localization of nerves within the psoas in an in vivo porcine model.

MATERIAL AND METHODS

Animal Model and Handling

In vivo tests were performed in porcine tissue[30,31] using an FDA-cleared ultrasound imaging system (SonoVision,™ Tissue Differentiation Intelligence, USA) to evaluate the performance of the system in imaging and identifying tissue structures. The experimental protocol was approved by the Institutional Animal Care and Use Committee at the Medical Education and Research Institute (MERI), Memphis, TN. The pigs were anesthetized and placed in a lateral decubitus position to simulate the LLIF approach. A retroperitoneal surgical approach exposed the relevant anatomy, including the psoas. The experiments were conducted in two phases with a total of 50 pigs. The first phase involved using ultrasound imaging and subsequent open dissection to train the AI software to distinguish neural anatomy in the psoas environment. In the second phase, five of the 50 pigs were used to test the algorithm performance.

Ultrasound Imaging

The AI-enhanced ultrasound imaging system shown in Figure 1 was used to image the target region to detect neurovascular structures during LLIF spine surgery. The ultrasound imaging was performed in conjunction with the porcine LLIF approach in the vicinity of the L4 to L6 vertebral bodies. The software was used to identify neurovascular features in the targeted region of interest. In this study, the imaging was performed using an ultrasound probe (Beluga1, Tissue Differentiation Intelligence, USA), with a 128-element linear array and operating at a central frequency of 10 MHz.

Figure 1

SonoVision™—an Artificial-Intelligence enhanced ultrasound imaging system.

After the lateral skin incision, the peritoneum was pushed anteriorly to access the psoas muscle. The probe was inserted through the incision to reach the surface of the psoas muscle. The surgeon used C-Arm fluoroscopy to locate the approximate region of interest for scanning. The psoas was then scanned utilizing the tissue identification features of the ultrasound system, helping the surgeon to visualize neurovascular structures in each scan location. C-arm fluoroscopy was used to verify the location of the probe above the target region of the spine (Figure 2) specifically with respect to the vertebral body.

Figure 2

C-arm fluoroscopic image of the ultrasound probe above the vertebral body of a pig spine.

Once the location was finalized, the surgeon used a surgical arm with attachment to hold the probe stationary at the target location. After the probe position was fixed, the surgeon utilized the ultrasound system to confirm the desired path and then inserted a metal pin (1.4 mm diameter, 10–15 cm long) into the vertebral body or intervertebral disc space with the intent of targeting either a nerve or a clear path. Here, “clear path” is defined as a region without the presence of neurovascular structure so that the metal pin could be inserted without damaging surrounding neurovascular structures. The metal pins were inserted such that, post-imaging and euthanasia, the tissue could be dissected to validate that the ultrasonically detected features matched the dissected tissue in proximity to the pins. All dissections were performed by a neurosurgeon and the dissected tissue served as the ground truth to the presence or absence of any anatomical features in the region of interest.

Image Processing and Classification

Segmentation

During the initial phase of the study, B-mode ultrasound images were acquired and compared to the ground truth information established through dissection. Specifically, the nerve regions in the acquired B-mode images were annotated based on the dissection notes. Through dissection, the nerve regions were verified and simultaneously segmentation algorithms were developed based on image processing techniques. The segmentation algorithm separated the B-mode image into one or multiple regions that corresponded with the presence or absence of neural tissue. These segmented regions were sent into a classification training process where each region was classified as either a “nerve” or “other” region.

Tapered windowing functions were used to suppress the edges of the given B-mode image followed by applying a Time Gain Compensation (TGC) to account for attenuation as a function of depth. As well, the brightness of the image was normalized. Dilation and erosion of the image were performed, which helped form different closed or open regions in the given B-mode image. Each of these segmented regions was selected and further analyzed with respect to shape, area, aspect ratio, solidity, and threshold values. The segments that satisfied these threshold criteria were used to define either a “nerve” or “not-nerve” region and passed into the machine learning model for classification. During the training phase each of the contours was refined based on ground truth data through dissection.

Real-time Image Annotation and Display

To increase computational efficiency and reduce computation time as well as to display detected regions in real-time, a GeForce RTX 2080 Graphics Card (NVIDIA, USA) was used in the ultrasound system to achieve a frame rate >15 frames per second (fps).

The U-Net Convolutional Neural Network (CNN) algorithm[32-34] was used to classify and detect bone and muscle regions in a given B-mode image. In the U-Net architecture the B-mode image was the input to the network and the output was a probability map of the image for both bone and muscle. This probability map was then color-coded into RGB and alpha mapped with appropriate transparency onto the ultrasound image and displayed on the graphical user interface (GUI) such that the background and the overlay were both visible.

The segmentation algorithm was used to partition the B-mode image into regions. Using the segmentation algorithm, the regional contours were collected. These collected contours were passed into the CNN model for classification of these segments as a nerve or other region (i.e., not a nerve). The CNN model was used TensorFlow (Google, USA)[35] to classify the segmented regions. The classified nerve regions were then overlaid on the B-mode image and indicated with a solid yellow color.

Training and Validation of SonoVision Algorithm

In the first phase, training of the classification algorithm was performed using 36,000 and 10,000 B-mode images for nerves and bone/muscle regions, respectively. To test various aspects of the system, both nerve regions and clear path regions were selected. The protocol was to target nerve for some of the metal pin insertions and, in other cases, to find a clear path to insert the metal pin without damaging any surrounding neurovascular tissue structures. The aim was to validate the software both when it indicated a nerve region and when it indicated a clear path. The second phase of the study was to test the trained detection algorithm using approximately 4800 B-mode images.

Figure 3A and B demonstrates the ultrasound system GUI identifying neural tissue within the psoas and the corresponding anatomical dissection. This validation process was followed throughout the experimental study for both training the CNN model in phase one and for validating and testing the trained CNN model in phase two.

Figure 3

(A) SonoVision screen identifying a nerve region (yellow) at approximately 20 mm depth in the muscle, (B) dissected validation of the nerve as identified in (A).

Figure 4A and B demonstrates an example of clear psoas anatomy with no neural tissue present along with the corresponding anatomical dissection. The Doppler color flow imaging mode was also enabled allowing the surgeon to check for any vessels along the insertion path. An example of Doppler mode indicating the blood flow during real-time imaging is shown in Figure 4c.

Figure 4

(A) SonoVision screen indicating clear path through the psoas without the presence of nerve regions and vertebral body cortical surface beneath the psoas, (B) dissected tissue indicating muscle region is clear of neural tissue, and (C) SonoVision Doppler mode showing blood flow in the vessel.

Quantitative Metrics

To determine the similarity of the anatomical region detected by the algorithm and the labeled annotated ground truth, the Dice coefficient was calculated as follows:[36,37]

where |X| and |Y| are the cardinalities of the two sets (i.e., number of elements in each set). Here the two sets refer to the labeled annotated ground truth region and the algorithm detected region on a given B-mode, respectively, for each of the tissue types. Based on the Dice score, the true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values were defined. The sensitivity was defined as a statistical measure of the proportion of the actual positives that were correctly identified and was calculated as follows:

Specificity was defined as a statistical measure of the proportion of the actual negatives that were correctly identified and was calculated as follows:

The accuracy was calculated as follows:

During the second phase of the experimental study (the validation testing phase of the system), metal pins were inserted in the pigs using a blinded study design. The imaging notes and the dissection notes for each of these pins were collected separately and subsequently analyzed by an independent observer. Since these pins were inserted through the psoas muscle and anchored into the bone or the disc, the detection of bone surface and muscle region was also validated using the same pin locations. If a nerve was seen within a radius of 1 to 2 mm from the pin then it was considered as nerve within the vicinity of the pin. Validation was performed via post-euthanasia dissection, visual inspection, and tactile assessment.

The AI-trained ultrasound system detected nerve, psoas muscle, vertebral body surface, and disc space with high sensitivity and specificity. For all the pins, the ultrasound imaging recorded during the surgical approach matched the open dissection and identification of anatomy. Thorough dissection and gross anatomy analyses were performed for each type of tissue: nerve, muscle, bone surface, and disc space.

The ultrasound system identification of anatomical features was deemed successful when the imaging notes matched the dissection notes. For example, if the imaging notes indicated that the user targeted a clear path and then, post-dissection, it was observed that a nerve was present in the vicinity of the pin, then the imaging notes and the dissection notes did not match. This was considered a FN indication from the system with respect to nerve identification. Similarly, if the imaging notes indicated the presence of nerve and the dissection notes also indicated the presence of nerve, then this would be considered a TP.

In the second phase of the study to test the trained detection algorithm, approximately 4800 B-mode images were used to estimate the various quantitative parameters to define the sensitivity, specificity and accuracy of the detection algorithm. These 4800 B-mode images were acquired using five pigs at different locations in the left and the right psoas environments from each pig. The Dice scores from all of the cases for nerve, bone, and muscle are shown in Table 1. The mean Dice score for each tissue type was >80%, indicating that the detected region and ground truth were >80% similar to each other. The sensitivity for each of the three tissue types was >95%. The mean specificity of nerve detection was 92%; for bone and muscle, it was >95%. The accuracy of nerve detection was >95%.

TABLE 1

Results

Tissue Type	Dice Score	Sensitivity	Specificity	Accuracy
Nerve	83.81 ± 0.20	100	93.13 ± 0.15	96.30 ± 0.10
Bone	90.60 ± 0.09	100	96.42 ± 0.38	97.12 ± 0.20
Muscle	88.60 ± 0.36	100	98.61 ± 0.21	98.29 ± 0.18

DISCUSSION

The results of this animal study suggest that a combination of image processing and machine learning algorithms can correctly detect different tissue types such as nerve, muscle, disc space, and vertebral bone surface from a given B-mode ultrasound image. The tissues can be detected, segmented, classified, and displayed in real-time during the ultrasound scanning process. It was observed that only 40% to 50% of the GPU card memory was used during the real-time detection of the anatomical features. In particular, a key feature of the algorithm is that segmentation through image processing reduces the amount of data feed into the CNN classifier and thus reduces computational burden. The high sensitivity and specificity indicate the stability of the algorithm.

Machine learning has been previously used for detecting various tissue architectures.[32,38,39] To our knowledge, it has never been used to enhance image guidance for spine procedures. In a previously conducted clinical study evaluating traditional ultrasound for intraoperative guidance in lateral lumbar spine surgery,[28] the investigators used a transvaginal probe to ensure the operative corridor was free of other soft tissues such as kidneys and bowel. These investigators reported that the major vessels anterior to the vertebral body were identified using Doppler-mode in all 100 patients in their cohort. They did not identify neural structures within the psoas muscle.

The present study focused on the lateral surgical approach to the lumbar spine. Ultrasound using Doppler mode can provide real-time information concerning the location of surgical instruments relative to critical vascular structures. This could be beneficial in cases where patients have aberrant anatomy or in more complex spine cases, such as procedures where there is greater concern for vascular structures (e.g., OLIF and more complicated deformity corrections,[3,40-47] especially since vascular complications have been reported in the 0.3%–8.6% range).[44,45,48]

The study focused on AI algorithms for neural anatomy detection. These algorithms are deep learning methods whose performance improves with training. In other words, the system is self-educating such that its output (in this case, anatomic feature detection within the psoas and elsewhere) improves with additional data input. One of the study's limitations was that the Doppler-mode was only used to detect the presence of flow and was not analyzed in a quantitative fashion. Also, this study did not focus on quantifying the potential of the AI-enhanced ultrasound system to reduce radiation exposure. This potential benefit for health care constituents (surgeons, patients, and OR personnel) will be evaluated in subsequent clinical trials.

The present study incorporated a lateral surgical approach that simulated LLIF procedures in a porcine model. AI-enhanced ultrasound imaging provided a real-time spatial map of the critical anatomy present in the surgical field. Most importantly, the real-time intraoperative imaging demonstrated the presence and location of neural structures within the psoas muscle. These nerves are not evident with other intraoperative imaging modalities and are only indirectly and incompletely localized with intraoperative electrical testing. Applied to humans, the real-time guidance provided by this technology should enable a surgeon to quickly identify neural structures and define a safe path to the disc space while minimizing radiation exposure.

The study evaluated the use of an AI-enabled, real-time intraoperative ultrasound imaging system for localization of nerves and other anatomic structures within and adjacent to the psoas muscle in an in vivo porcine model of LLIF.

AI-enhanced ultrasound imaging provided a real-time spatial map of the critical neural anatomy present in the surgical field during lateral spine surgery.

This technology is intended to enable a spine surgeon to choose a safe pathway to the lateral lumbar spine.