Artificial intelligence in functional imaging of the lung.

Artificial intelligence (AI) is transforming the way we perform advanced imaging. From high-resolution image reconstruction to predicting functional response from clinically acquired data, AI is promising to revolutionize clinical evaluation of lung performance, pushing the boundary in pulmonary functional imaging for patients suffering from respiratory conditions. In this review, we overview the current developments and expound on some of the encouraging new frontiers. We focus on the recent advances in machine learning and deep learning that enable reconstructing images, quantitating, and predicting functional responses of the lung. Finally, we shed light on the potential opportunities and challenges ahead in adopting AI for functional lung imaging in clinical settings.

Introduction

The advent of artificial intelligence (AI) heralds a new era in digital data analysis and empowers us to interpret complex systems through unprecedented modeling capabilities. This power of AI has led to an explosion of applications across multiple disciplines including computer vision, and more recently, health care. Clinical care stands to benefit tremendously from AI to expose meaningful relationships in complex data sets obtained from clinical imaging to molecular medicine. Although AI still is a nascent field in many health-care domains, initial applications and proof-of-concept studies have shown promising and impactful results in diagnosing different disease conditions using only raw data sources like diagnostic imaging.[1,2] Thus, the immense analytic capacity of AI technology based on machine learning and deep learning will power human decision-making and complement human cognitive capabilities. Beyond equipping physicians with new abilities, data-driven modeling, as opposed to just model-based methods, is serving as a robust paradigm that can further improve the current cutting-edge algorithmic approaches in image formation, reconstruction, and post-processing.

The functional lung imaging community is recognizing the transformative power of AI. The data-driven approaches are well-positioned to invigorate established techniques in this field, improving robustness and often surpassing existing capabilities. Current functional lung imaging modalities utilize the underlying physics of the image properties related to different disease conditions of the lung.[3] The amount of data elements generated in functional imaging acquisitions, such as multiple MRI snapshots during free-breathing acquisitions or different CT energies, is amenable to applying data-driven approaches to discover novel relationships across different imaging phases, which otherwise would be difficult to identify. Various functional imaging modalities rely on advanced acquisitions and post-processing approaches, and hence AI is attractive as a primary modeling strategy.

Although AI applications in diagnostic imaging have increased rapidly in the last few years,[4,5] its clinical application to functional lung imaging is currently more of an evolving opportunity than a tested reality. Farhat et al[6] recently reviewed the application of deep learning in pulmonary medicine imaging and noticed that the use of AI in lung imaging is mostly circumscribed to chest CT and X-rays (CXR). In this review, we take a comprehensive look at the growing interest in applying AI technology specifically to pulmonary functional imaging and assess the underlying concepts of the proposed methodologies that utilize machine- and deep learning for state-of-the-art image reconstructions, functional assessment, and functional imaging synthesis. We evaluate the opportunities AI presents and weigh in on the challenges ahead for successfully implementing AI in pulmonary functional imaging.

Deep learning in medical imaging

The emergence of AI as a key component in medical imaging techniques is largely propelled by vast improvements in machine learning, specifically, deep learning. Deep learning performs a wide variety of challenging tasks, including classification, regression, clustering, image reconstruction artifact reduction, lesion detection, segmentation, and registration.[7] Deep learning is an extension of artificial neural networks[8] as a core building block. Deep learning gained importance in computer vision when neural networks outperformed other methods on several visual recognition tasks. Deep learning in medical imaging is primarily based on the convolutional neural network (CNN) paradigm. LeCun[9] introduced the CNNs to extend the use of neural networks from 1D signals to multidimensional signals like 2D or 3D volumes and provide a powerful way to learn representations of images and solve recognition tasks. CNNs are constructed with units of a compact kernel of neurons that slides across an image to produce an output image map. Neurons act like logistic regressors that generate a response at each image location as a weighted sum of the image intensities. The kernels define the weight of each location, and these neural kernels are assembled in multiple channels to create a CNN convolutional layer. Several such layers that function differently but complementary make up the CNN (Figure 1A). Information flows in a forward fashion, and deeper and deeper layers aggregate it in a non-linear manner. The success of CNNs in medical imaging inspired the development of other deep learning paradigms to exploit the various aspects of the information flowing through the network. A few examples of such advanced network methods are recurrent neural networks (RNN), autoencoders (AE), and its variations like U-Nets, generative adversarial networks (GANs), and more recently, transformers, among others.[12-14] Figure 1B illustrates the architecture of a U-Net used in medical applications to generate an output image from an input image after aggregating information at different scales. For more information, we refer the readers to the recent reviews of deep learning in radiology.[5,10,12,15]

Figure 1.

Schematic of a CNN architecture. (A) Traditional CNN architecture is used for image classification or regression. An input image is decomposed into multiple globally aggregated features by a final-stage fully connected neural network. Convolutional layers are the main component in CNNs. Additional layers include data pooling to downsample the image domain, drop-out for model simplification, and batch normalization. (B) U-Net architecture is a type of fully convolutional network that is widely employed in medical imaging applications. U-Net contains two convolutional steps: an encoder and a decoder. The encoder reduces the input data to a latent space, and the decoder uses this information to recreate a new image. Adapted from Chartrand et al and Zha et al[10,11] with permission. CNN, convolutional neural network.

Machine learning approaches can be classified into four major categories depending on the nature of the problem being solved and the data elements used as part of the training, viz. supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. A model maps a set of inputs to given outputs in supervised learning and requires annotated data sets. Unsupervised learning aims at finding structure in a data set, as it is common in clustering problems. Semi-supervised learning has emerged as an exciting approach that combines supervised and unsupervised techniques to take advantage of non-annotated datasets that can improve supervised learning by matching specific characteristics of the non-annotated data set. An example of semi-supervised learning is image-to-image translation using GANs.[16] Finally, reinforcement learning is based on agents that learn from their environments through trial and error while optimizing some objective functions. An example of reinforming learning in medical imaging is landmark detection methods.[17]

Finally, machine learning approaches define the model parameters using training data to solve an optimization problem. The proper definition of the training data set in terms of characteristics, sample size, and image conditions are key to converge to a solution that can be generalized to other data sets beyond the training examples. This implies that machine learning needs a thorough validation and testing of the models using data points that have been employed in training. Different techniques known as cross-validation are used to check the stability of the model when the training data change. It is essential to understand the conditions under which the model was derived, and the modelers need to follow good practices and careful documentation of the training process.[18]

AI in functional image reconstruction

Magnetic resonance imaging (MRI)

MRI has been a primary modality in functional lung imaging because of its safety characteristics and the exceptional ability to discover functional properties.[19] The early challenges due to a lack of protons and signal inhomogeneities in the lungs have been overcome, and now MRI can be used for static and dynamic lung imaging.[20] The arrival of ultrashort TE (UTE) MRI with sophisticated clinical hardware has advanced lung imaging, both at the structural and functional levels.[21] From oxygen-enhanced and hyperpolarized gases MRI for ventilation imaging[21] to Fourier Decomposition proton MRI for ventilation/perfusion (V/Q) imaging and dynamic contrast enhancement (DCE) MRI for microvascular perfusion,[19] MRI has become the modality of choice to examine the complex ventilation and perfusion functions in different pathological conditions.[22] Essential to MRI pulse sequence design is the need for short echo times and the balance between acquisition time and signal-to-noise ratio (SNR) that can be achieved with parallel imaging.[23] Many of the computational approaches in MRI applications have been focused on improving optimal phase encoding from an under sampled version of the k-space that could reduce the acquisition time while keeping SNR levels compatible with image quality.[24] Compressed sensing techniques were developed two decades ago for fast MRI reconstruction, and using diffusion MRI with hyperpolarized 129Xe.[25,26] In the past few years, CNNs and Recurrent NNs have taken a prominent role in improving static and dynamic MRI reconstruction to learning the spatiotemporal dependencies in heavily undersampled k-space data.[27-31] Duan et al[32] showed improved ventilation imaging using a coarse-to-fine neural network from undersampled k-space.[32] Reconstruction can be achieved with higher SNR values than compressed sensing reconstruction, paving the way for real-time reconstruction of contrast-enhanced MRI of the lung. Unlike compressed sensing, CNN reconstruction models rely on incorporating prior information learned as part of the training process to solve the inverse reconstruction problem.[33] However, the reliance on data to define a model implies that rigorous validation is needed.[34]

Another area where deep learning can impact is the inherent need to perform motion correction in dynamic MRI acquisitions. For example, Fourier Decomposition MRI for V/Q Imaging relies on registration techniques as a critical step in their reconstruction paradigm. Likewise, different approaches have been proposed based on traditional functional optimization that shows stable quality results.[35] Deep learning registration offers an alternative with low computational cost during the inference stage once the registration model is trained.[36,37] Deep learning in MRI also has been attempted to estimate quantitative tissue parameters using quantitative susceptibility mapping (QMS) and MRI fingerprinting to achieve more standardized biomarkers.[38] Although these techniques are yet to be applied in both preclinical and clinical MRI lung imaging, deep learning could catalyze the translation of these advanced quantitative tools.

Computed tomography (CT)

Volumetric CT (VCT) has high-density contrast between air and tissue and is a mainstay of clinical chest radiology. The introduction of helical multislice CT scanning facilitated spatiotemporal 4DCT as a tool in radiation oncology for measuring and managing overall respiratory motion.[39] Patient safety is increased because only low dose radiation is required when combined with advanced iterative reconstruction techniques, and hence functional CT imaging (both 4D and dual-energy) is preferred for broader clinical use. Like MRI reconstruction, new AI methods are pushing ultra-low-dose CT image reconstruction to another level. Major manufacturers are introducing new deep learning schemes that show higher SNR and contrast and improved object detectability than standard statistical or model-based iterative techniques.[40-42] New techniques under development and current iterative reconstruction approaches capable of denoising CNNs promise to improve the image SNR further.[43] In addition to supporting low-dose image reconstruction, deep neural networks have also been employed to reduce breathing artifacts and enhance image quality.[44] All these advances will make temporal ultra-low CT a safer and more versatile functional modality in clinical applications of CT.

Cone-beam CT (CBCT) system is becoming a key device in the interventional suite due to portability and high reconstruction quality for volumetric images. In addition, deep learning is catalyzing dynamic applications with real-time reconstruction from sparse projection data permitting real-time ventilation imaging in image-guided radiotherapy.[45,46] The combination of these improvements can open the door for these preclinical CBCT applications to broader adoption as a lung functional imaging modality.[47]

Dual-energy CT scanning (DECT) with contrast agents (iodine or Xenon) has also enabled the assessment of regional ventilation and perfusion by taking advantage of the difference in linear attenuation coefficient at different X-ray energies.[48-52] CNNs are being applied to improve DECT imaging fundamentals related to material decomposition,[53,54] simplify dual-energy acquisitions based on single-energy material decomposition[55] and combine virtual single-energy structural imaging from dual-energy acquisitions. The translation of these techniques can expand the role of DECT in ventilation and perfusion imaging as dual-energy is more readily available.

Positron emission tomography (PET-CT) and single-photon emission computed tomography (SPECT) have also been employed to perform V/Q imaging to improve planar lung scintigraphy[56] and assess pulmonary inflammation.[57] Deep learning solutions are being developed to enhance PET reconstruction and attenuation correction[58,59] ; however, up to date, no validation studies have been performed to show the impact of AI-enhanced molecular imaging in the lung. Thus, this area remains an exciting opportunity for AI in the years to come.

AI in functional quantification

Automated lung segmentation in functional modalities

For a functional imaging modality, it is important to define the structural components of the lung, such as lung field, lobar compartments, fissures, and the bronchovascular tree, to locate and quantitate image-based data. Deep learning is significantly evolving and transforming the post-acquisition upstream operations necessary to resolve the lung’s structural components to interpret and quantify regional functional markers. Deep learning is indeed replacing the rule-based approaches to segment the lung[60] and the lobes with more precise and reliable mapping methods based on CNNs that have shown more consistent results across modalities.[6] In particular, the use of U-nets, a specialized neural network architecture for semantic segmentation, has provided compelling results in multiple medical and biomedical imaging segmentation tasks.[61,62] These new approaches to image segmentation are superior in part due to their enhanced ability to encode shape priors of the segmented organ based on the provided training data without explicitly modeling the shape. One example of the application of U-Nets to functional modalities is the use of a 2D U-Net to perform volumetric lung segmentation from UTE proton MRI in a multiplane fashion.[11] Despite reduced contrast around the lung boundaries, the lung volume estimates in a set of asthmatic and cystic fibrotic patients closely matched the reference values (Figure 2). One caveat for the application of deep learning is the limited availability of training data. Recently, Guo and colleagues showed increased robustness in UTE MRI lung segmentation by including an adaptive k-mean after the initial U-net segmentation.[63] Robust lung segmentation in MRI is essential for quantitative analysis of functional parameters and its use in clinical studies. Similarly, a multiresolution U-Net architecture has been proposed for robust lobar segmentation in CT images to enable regional quantification of dynamic CT series.[64,65]

Figure 2.

Segmentation of the lung field on oxygen-enhanced UTE MRI images using a multiplane (axial and coronal and final consensus) U-Net approach in a 37-year-old female with cystic fibrosis. The delineation of the lung boundaries can be achieved despite the reduced contrast. Adapted from Zha et al[11] with permission. UTE, ultrashort TE.

Deformable image registration (DIR)

Ventilation imaging. DIR is one of the most employed methods to assess ventilation defects from temporal imaging modalities like 4DCT, CBCT, and MRI. DIR-enabled CT-based ventilation assessment has been successfully used in radiation oncology to avoid damage from radiation therapy as well as performing dose–response assessment.[39] Recently, MRI-based mechanical assessment of the lung via elastic registration has also been used to assess SSc-related fibrosis.[66] Ventilation assessment using tissue expansion metrics based on the deformation fields generated by DIR or the differences in tissue density between the coregistered image pairs have shown reasonable correlation with the regional assessment of ventilation using Xenon CT[48,67] and Xenon-MRI.[68] However, variability between registration approaches has led to a poor correlation between DIR-based ventilation metrics and reference modalities at the voxel level.[69]

Traditional DIR approaches describe the mapping of two images via a deformable field by finding the elastic transformation parameters that minimize the difference between images acquired at different moments during the respiratory cycle. Deformable registration in the lung has been challenging by the complexities of describing the transformation in a parametric way when dealing with large displacements commonly found in registration between images acquired between TLC and FRC while preserving known invariants like lung mass.[70] Nevertheless, traditional methods have partially addressed lung registration with reasonable accuracy performance, albeit with complex methodologies that lack robustness and require long computation times due to the numerical minimization needed for each registration instance.[71]

Deep learning-based deformation image registration (DLDIR) has emerged in the last 5 years as a new paradigm for registration. One of the main advantages of DLDIR approaches is the explicit or implicit definition of the deformation field via a CNN that can better capture the complexities of the deformation in a particular problem with relatively low computational needs during the inference step. DLDIR can be classified into supervised and unsupervised registration methods. Supervised approaches that regress the displacement vector field between two images using a CNN model were initially employed in DLDIR[72,73]R These methods were trained with previously aligned images using either a reference method[72] or synthetic deformations.[73,74] Although these approaches improve the registration computing times from minutes to just a few seconds, their accuracy is defined by the characteristics of the reference approach used for learning. The reported registration errors on reference data sets are on par with their traditional techniques that have been extensively used in ventilation studies. Unsupervised registration approaches have been explored to overcome the limitation of using an explicit reference deformation. Among them, unsupervised DLDIR has captured the attention in the last few years because it needs only limited training data.[75] Unsupervised techniques use a mismatch metric between the moving image and the reference image within the training data, as occurs in a traditional registration framework. A CNN model encodes the deformation parameters, and the optimization is done over the parameters of the CNNs rather than the deformation parameters. Once the training is completed, the CNN is employed to generate new deformation parameters from unseen data sets. De Vos and colleagues[37] pioneered this framework in lung registration using the multiscale ConvNet architecture (Figure 3). A similar approach has been shown to be feasible to register CT to CBCT and CBCT to CBCT[76] and one-shot methods have been tailored to track periodic breathing motion patterns.[77] Finally, Fu et al[77] proposed a LungRegtNet for 4DCT registration that employs vascular landmarks to achieve superior performance compared to current methods based on unsupervised registration in the DIRLAb data set.[78]

Figure 3.

An example of Unsupervised Deep Learning Deformable Image Registration from an expiratory (moving) to an inspiratory (fixed) CT scan. The CNN models the deformation field depicted as a warped grid. The Jacobian map estimates the volume change and can be used to compute ventilation maps. Registration inference can be performed in a few seconds in comparison to classical techniques enabling real-time deployment. Adapted from Vos BD de et al[37] with permission. CNN, convolutional neural network.

The new breed of lung DLDIR approaches can lead to higher accuracy and more robust registration results that could improve the assessment of regional ventilation at the voxel level; however, extensive validation studies in larger prospective samples should be conducted to confirm this possibility.[69,79] Inaccurate registrations can result in lung tissue being mapped to blood vessel voxels which will cause artifacts in the CT-ventilation image in both the Jacobian and HU formulations. Without any doubt, the most exciting characteristic of DLDIR is the need for low computation to resolve a deformation field once the method has been trained. This opens the opportunity for bringing DIR-based ventilation metrics closer to the patient point-of-care when applied to lower-cost setups like 4D CBCT. These exciting techniques are potential modalities for ventilation assessment during treatment in the near future.[39]

Multiparametric assessment. Registration is also a fundamental processing component of multiparametric structural and functional imaging analyses to correlate structural changes with functional defects in lung pathophysiology.[80-82] MacNeil et al[83] used volume-matched CT and hyperpolarized helium-3 (3He) MRI using static and diffusion-weighted imaging to define a multiparametric response map (mPRM). Structural changes measured on CT were coupled with regional MRI-based ventilation and microstructure based on the apparent diffusion coefficient (ADC) as shown in Figure 4. mPRM metrics were able to reveal emphysema and small airways disease not otherwise identified with CT or MRI, reflecting the power of multimodal approaches in disease characterization. Registration approaches that include deep learning schemes will likely translate these upcoming multiparametric approaches to clinical applications beyond correlative studies.

Figure 4.

Multiparametric imaging mapping from 3He MRI and CT in COPD. Functional and structural images (top) are aligned to produce a multiparametric Response Map (bottom). DL Registration techniques can enable accurate and real-time response mapping assessment. Adapted from MacNeil et al[83] with permission. COPD, chronic obstructive pulmonary disease.

Functional prediction

Deep learning approaches have been also postulated to predict the functional parameters from structural modalities. Westcott and colleagues showed how textural-based features extracted from a volume of interest on CT scans can predict regional ventilatory effects in subjects with COPD.[84] The method was trained with ventilation defect labels obtained from 1H and 3He MRI using a k-mean approach. Different classifiers were compared, and the most relevant features were selected in a cross-validation experimental setup. The AUC of the best model was 82%, with high specificity (91%) and moderate sensitivity (49%). Ventilation-defect percentage (VPD) predicted by the model and the one computed using the reference MRI modality show a strong correlation (90%); an encouraging sign of the ability of these approaches to offer patient-specific information on functional impairment conditions. However, it is hard to ascertain how stable the features proposed by this study could be generalized to a larger COPD population with milder disease conditions because of the limited sample size used for training. Larger sample size and reproducibility studies are needed to define the generalization power of the proposed features.

CNNs have also been used to extract features from CT images that can define spirometric status in smokers with and without COPD. Gonzalez et al[85] used a three-layer feed-forward CNN to predict COPD functional status based on spirometry. The correlation between FEV1 measurements and deep learning CT-based measurements was 73%. Tang and colleagues used a more complex network—a residual Network with 152 layers—to diagnose COPD from CT volumetric imaging.[86] The AUC in the testing cohort for the best model was 86%. This result was consistent with the performance reported by Gonzalez and colleagues. These results suggest that different architectures can extract complementary feature information from CT imaging to predict an outcome.

In sum, the best network architecture design in terms of combinations of neural layers must strike a trade-off between model complexity and the ability to generalize to different populations and imaging characteristics. Meta-learning techniques are being actively researched and developed to improve upon the prediction of single learning techniques in multiple learning episodes that integrates different approaches.[87]

AI in functional assessment

Function assessment is one of the most exciting emerging applications of AI where a direct functional response is synthesized to mimic a target functional modality, e.g. dual energy CT pulmonary perfusion, from source modalities that require simpler or a more direct imaging reconstruction setup. These techniques aim to resolve intrinsic relations across functional modalities or even the resolution of functional information from structural modalities like CT. These approaches are referred to as image-to-image translation within the AI community. They are based on an array of supervised and semi-supervised techniques that range from fully CNNs like convolutional generators based on autoencoders and U-nets[61] to Generative Adversarial Networks (GAN)[13] that combine a generator and a discriminator network. Image-to-image translation techniques were borne off in the context of computer graphics applications[88] and one prominent application is artificial style representation from natural images using paired (conditional) or unpaired (cycle) GANs.[16,89] In paired approaches, the training is performed in a data set containing paired instances of the target and source modality, while unpaired approaches can use instances from the source and the target modalities that are not matched or even belong to the same population of subjects.

Supervised functional synthesis

One of the first demonstrations of image translation approaches in functional lung images has been synthesizing ventilation imaging from 4DCT without explicit use of DIR. Unfortunately, 4DCT-derived ventilation images are sensitive to the choice of DIR algorithm and its accuracy.[90] Direct approaches can overcome this limitation by directly learning tissue expansion characteristics from multiple snapshots across a breathing cycle. Zhong et al[91] proposed a fully convolutional model composed of seven layers without any downsampling step to preserve the image resolution. Despite reasonable results, fully convolutional networks are limited to local relations between the inspiratory and expiratory images around a voxel that could lead to inconsistent results if the mismatch between inspiratory and expiratory images is significant.

To overcome some of the limitations of fully convolutional approaches, encoder–decoder convolutional like the U-net architecture have been extensively applied in image-to-image reconstruction tasks. The U-Net architecture includes multiple convolutional steps followed by a data down-sampling step in the encoder phase and up-sampling layers in the decoder phase. Also, information from the encoding phase at a given level is transferred to the decoder phase, similar to the fully convolutional approach. These architectures have shown promising results in synthesizing different functional ventilation images.[92,93] Capaldi et al[93] demonstrated the use of U-nets to estimate hyperpolarized noble gas MRI ventilation maps from free-breathing proton (1H) MRI after breathing phase sorting and interpolation (Figure 5). Training, validation, and testing were done in a set of 114 subjects with different pulmonary conditions, i.e. asthma, COPD, bronchiectasis, and NSCLC, and healthy volunteers. The deep learning-based VDP estimation showed good agreement with reference values based on hyperpolarized 3He MRI. In a similar fashion to Zhong et al.,[89] Gerard et al[92] used a multiresolution U-net to provide a direct estimation of the ventilation response based on the deformation Jacobian without a DIR (Figure 6A). Unlike prior approaches, this network was trained in an extensive database of inspiratory and expiratory CT scans from the COPDGene cohort and showed high voxel-wide correlations with ventilation images based on a classical mass-preserving DIR approach.

Figure 5.

Deep learning ventilation MRI for the synthesis of 3He MRI ventilation imaging from free-breathing proton (1H) MRI. (A) Illustration of the MRI pipeline to register and sort out free-breathing MRI images before consumption by the image-to-image U-Net network. The training was performed to predict ventilation maps from 3He MRI. (B) Comparison between the reference ventilation maps and DL ventilation MRI synthetic imaging for subjects with different types of obstructive airway diseases. Agreement in ventilation defect percentage between modalities was high with good correspondence. Adapted from Capaldi et al[93] with permission.

Figure 6.

Illustration of image-to-image translation techniques for synthetic ventilation and perfusion assessment based on single energy CT. (A) Direct Jacobian ventilation map estimation using a multiresolution deep learning approach without deformation image registration from inspiratory and expiratory CT scans. (B) Estimation of dual-energy perfusion maps from single energy CT angiograms to assess perfusion defects using a functional consistency CycleGAN.

Like ventilation imaging, recent studies have also shown the use of CNN approaches to estimate lung perfusion from single energy CT scans. Ren et al[94] employed an attention U-net architecture to synthesize albumin SPECT/CT perfusion mapping from non-contrast CT scans to enable functional lung avoidance in radiotherapy planning.[94] Their proposed neural network is superior to the traditional U-Net architecture and is able to identify features from the CT domain that are compatible with perfusion defects with moderate correlation. Despite the limited size of the training (31 subjects) and testing data (11 subjects), these results illustrate the ability of deep learning approaches to estimate both ventilation and perfusion functional imaging from routine non-contrast CT scans under a common imaging platform.

Adversarial functional imaging

Semi-supervised approaches based on GANs are also under development as an improved alternative in image translation that aims at increasing the stability of the results of multi layered neural networks.[15] For example, Nardelli et al[95] illustrated the use of a modified conditional CycleGAN to synthesize dual-energy-derived iodine perfusion maps from single energy contrast CT scans (Figure 6B). The cycleGAN leverages both CT imaging and structural vascular information in a setup with three encoding CNNs and three discriminators to generate the functional output with moderate local correlations (0.52 and 0.66 in the core and peel lung regions, respectively). Although unpaired GAN approaches are more complex and more challenging to train due to the need to find an equilibrium point in a min-max optimization problem, they seem relevant to approximate the statistical characteristics of the image that is being estimated. Unpaired GAN approaches can be employed with larger databases of unpaired datasets to predict the target functional modality without the need for scanning the same subject with both modalities as required by plain convolutional approaches.[93,94] Thus, the application of GANs presents a greater opportunity in the context of functional imaging. GAN-based learning can also be applied in various domains related to image reconstruction and preprocessing stages like super-resolution and multimodal registration and modality synthesis for multiparametric analysis.

Opportunities and challenges

AI applications in medical imaging have exploded over the past 5 years, driven by multiple factors. First, the maturity of the deep learning approaches exploiting non-linear relations in the data has been instrumental. Second, advances in optimization and regularization techniques have made it tractable to fit models with a large number of parameters to a limited set of training data points. Third, the availability of methods in well-maintained open-source libraries has empowered a broad community with AI techniques, including non-experts in the field with limited skills. Finally, specialized computing architectures based on Graphics Processing Units (GPUs) have delivered the necessary computing power to train advanced models within a reasonable amount of time.

While AI is still an emerging discipline in functional lung imaging, there are clear and tangible opportunities worth mentioning:

Multifunctional assessment: AI has the potential to unleash the power of multiple functional assessments under a single imaging platform. Currently, ventilation and perfusion imaging require the use of different imaging contrast agents in CT. One potential integrated solution could be the emerging combination of 4D CBCT and simulated dual-energy imaging for functional imaging. The benefits of synthetic multifunctional assessment include reduced radiological tests that require hard-to-obtain radioactive contrast agents, reduced radiation exposure, and improved care delivery as imaging synthesis is performed without the patient as part of the radiological and clinical evaluation. However, realizing these opportunities will require an extensive validation process to define the interval confidence in which the synthetic images are consistent with the underlying functional ground truth. The outcome of the validation studies will also determine the potential of AI-enabled synthetic imaging for clinical adoption and whether it could eventually be circumscribed to narrower clinical scenarios where an initial triage based on a sub optimal approach might be useful.

Clinical translation to low footprint radiological setups: current functional imaging relies on advanced modalities that require specialized equipment like hyperpolarizers. The potential use of AI-driven image-to-image translation could bring the benefit of functional information to standard radiological imaging modalities that are available in primary and secondary care facilities.

Novel biomarkers: functional modalities provide voluminous multiparametric data that need to be laboriously synthesized into specific markers of disease. AI provides an alternative computational approach to define novel biomarkers of the disease. Supervised CNNs can be used to extract relevant image features that are associated with a specific outcome. Unsupervised autoencoder techniques can also be applied for dimensionality reduction to define novel biomarkers from multiparametric imaging sources.

Unraveling lung structure and function: the relationship between structure and function of the lung has been well-described, but we are still limited in linking the structural changes to the functional impairment and achieving a better characterization of the disease. Studies that combine structural and functional modalities[83,96] can take advantage of AI as an exploratory tool to gain further insight into the structure–function relationship.

Despite the exciting and compelling preliminary evidence promising a more significant and elaborate role for AI in pulmonary functional imaging, several challenges remain that need to be carefully evaluated and resolved before realizing AI as a reliable component of clinical functional lung imaging:

Validation: data-driven approaches require rigorous validation studies to gauge the generality and robustness of the methods. Until now, most of the studies that apply AI to functional lung imaging were performed with small datasets. Although they provide early evidence of what AI can do, they lack the rigor needed to qualify as bonafide approaches. Large databases on diverse populations will be required to train and validate the techniques before translating them into clinical use.

Model transparency: one of the major criticisms of deep learning is a lack of transparency and interpretability. In other words, users (clinicians and researchers) should be able to understand the “reasoning” of the AI model; why it renders one verdict and not the other. Model developers and data scientists must make didactic efforts to teach the users how the models operate and decide outcomes. Transparency is crucial to defining a modality’s operational realm and proactively restricting deviations from the model that can affect image quality and diagnostic interpretability.

Model robustness: one collateral effect of the lack of model transparency is model instability to adversarial attacks (negligible input variations resulting in significant changes of the model output) and intrinsic model biases. Adversarial attack prevention is an oft-discussed topic in AI and they pose a substantial barrier to the use of AI for image synthesis in critical applications like diagnostic imaging.[97] Careful model design and training considerations must be taken to avoid adversarial attacks overall if models are trained with off-the-shelve components.[98] In a similar fashion, biases and disparity in functional expression may be translated into AI systems trained with imaging data in which those underlying biases exist.[99] Understanding the specific performance characteristics of each model is crucial to move beyond the preclinical scenario and successfully introduce it into clinical practice.

Unlocking data silos: the unresolved complexities of functional imaging imply that the number of training cases is limited compared to training scenarios available for modalities like CT and CXR. Training sample size is a key factor in deep learning that depends on the specific characteristics of the problem begin addressed and the model that is used. Unlocking the available data silos is paramount for implementing new data-driven advances in functional lung imaging. Open data repositories and challenges like VAMPIRE[79] are crucial for developing machine learning-centric approaches that improve functional lung imaging quality and performance reasonably and reproducibly. Issues about data integrity and privacy could be overcome with federated solutions that enable de-centralized AI modeling to exploit pan-institutional datasets.[100,101]

Conclusion

AI continues to evolve rapidly and push the limits in many spheres, and its interest in medicine is growing exponentially in recent years, especially in the functional imaging domain. Public and private entities recognize this as a thrust area, and their initiatives have begun to catalyze this field.[100] The pulmonary functional imaging community can benefit from this frenetic activity in data science as novel approaches using rich data sets are proposed to redefine disease conditions. Machine learning models that link imaging, functional, biomarkers, and multiomics data can advance our understanding of the complex and intimate connection between structure and function.[102] AI can also play a transformative role in adopting functional imaging approaches to clinical settings that are now restricted to preclinical scenarios due to their complexity. The use of functional modalities in diseases like chronic obstructive pulmonary disease (COPD), asthma, Interstital Lung Disease (ILD), or Cystic Fibrosis (CF) can bring a new dimensionality to define relevant markers of disease heterogeneity and progression.[82,103,104] At the same time, the application of AI is not free of limitations and perils stemming from the experimental nature of current techniques. The reliance on vast amounts of data exemplars rather than well-understood “fixed” models could act as a double-edged sword if AI is applied without careful methodological consideration. This issue is even more relevant in functional imaging scenarios where functional metrics describe subtle pathophysiological processes that need to be well-understood by the AI developers. Therefore, a multidisciplinary approach is essential to introduce AI in functional pulmonary imaging. Successful incorporation of AI in functional imaging holds promise to transform the field, delivering significant benefits in the coming years.