Application of artificial intelligence in digital chest radiography reading for pulmonary tuberculosis screening.

Currently, the diagnosis of tuberculosis (TB) is mainly based on the comprehensive consideration of the patient's symptoms and signs, laboratory examinations and chest radiography (CXR). CXR plays a pivotal role to support the early diagnosis of TB, especially when used for TB screening and differential diagnosis. However, high cost of CXR hardware and shortage of certified radiologists poses a major challenge for CXR application in TB screening in resource limited settings. The latest development of artificial intelligence (AI) combined with the accumulation of a large number of medical images provides new opportunities for the establishment of computer-aided detection (CAD) systems in the medical applications, especially in the era of deep learning (DL) technology. Several CAD solutions are now commercially available and there is growing evidence demonstrate their value in imaging diagnosis. Recently, WHO published a rapid communication which stated that CAD may be used as an alternative to human reader interpretation of plain digital CXRs for screening and triage of TB.

Introduction

Tuberculosis (TB) remains one of the major infectious diseases that seriously endanger human health. Nearly 1.4 million people died from TB in 2019. Of the estimated 10 million people who developed TB that year, some 3 million were either not diagnosed, or were not officially reported to national authorities. The early diagnosis of pulmonary TB is of great significance for disease control, which is mainly based on comprehensive consideration of clinical suspected symptoms, laboratory examinations, and chest imaging.

In China, TB patients are mainly identified by the routine surveillance system through presenting themselves to health-care facilities with suspected TB symptoms. However, pulmonary TB patients without typical symptoms and those who have symptoms but do not visit health-care facilities might be missed. Symptom screening for TB was a key component for combating TB, and the health care provider-initiated active TB screening in high-risk populations had become an important strategy recommended by WHO. In China, the active case finding had been implemented in close contacts of sputum smear positive TB patients, mainly via suspected symptoms screening. Those who present suspected TB symptoms would accept the subsequent chest radiography (CXR) and laboratory examinations in routine practice. However, such active case finding based on suspected symptoms screening showed lower sensitivity and specificity (especially among people living with HIV), and even smear-positive TB cases might be missed.

WHO has recommended that CXR screening can be used for TB case finding. It not only can immediately check the presence of lesions suggestive of TB, but also can determine the location, nature and severity of the lesions. According to the previous studies, CXR examination based case finding showed higher accuracy than symptoms screening.4, 5, 6 However, despite the high sensitivity when interpreted by experienced radiologists, CXR has its challenges, mostly due to its modest specificity, high inter- and intra-reader variability.5, 6, 7, 8 Besides, it has been reported that there was a shortage of qualified certificated-radiologists in many TB prevalent settings, which may impair screening efficacy., High cost of CXR hardware also means that these tests are rarely available at the primary care level.

Along with the application of computer-aided detection (CAD) system in reading medical images, several CAD software products have been developed to interpret digital CXRs for abnormalities suggestive of TB or other diseases.

Application of CXR in TB detection

Once hardware has been acquired, digital CXR has the advantages of low running cost and easy operation. Especially in low-income areas, digital radiography (DR) may be the only available medical imaging modality. CXR is used for a variety of conditions as a good tool for strengthening health systems.

For pulmonary TB, the radiographic presentation is wide-ranging, making it a challenging diagnosis. Active pulmonary TB can be subdivided into different forms: primary TB, secondary TB, and miliary TB. Primary pulmonary TB always develops shortly after infection, and CXR images mainly manifest as lymphadenopathy, consolidation, pleural effusion, and miliary nodules. Secondary TB develops after a long period of latent infection, with CXR images that usually manifest as cavities, consolidations, and centrilobular nodules. Miliary TB is caused by hematogenous dissemination, which shows miliary lung nodules in a random distribution on CXR images. In addition, prior old TB lesions often manifests as fibronodular opacities in the apical and upper lung zones, and fibronodular change is associated with a considerably higher risk of TB reactivation.

As many diseases may present with similar radiologic patterns, differential diagnosis of diseases must be considered when reading a CXR, including different forms of pneumonia, occupational lung diseases, neoplastic disease, pleural effusion, and cardiac disease. Hence, experienced radiologists are needed to interpret the images. However, even among trained human readers, inter- and intra-reader variability is common because they are susceptible to fatigue, perceptual biases, and cognitive biases, which may lead to errors. Moreover, it is sometimes difficult to distinguish similar lesions or find very obscure nodules, particularly when the contrast between the lesion and the surrounding tissue is very low or the lesion overlaps the tissue structures or larger pulmonary blood vessels. Therefore, the examination of TB using CXR will cause a certain degree of over- and under-diagnosis of TB. This is why an abnormal CXR should be followed up with a microbiological test for TB (such as smear microscopy, molecular test or liquid cultures).

CAD with machine learning (ML) technology in TB detection

With the development of artificial intelligence (AI), CAD systems have become a hot research topic, which can analyze the data related to the patient under investigation and suggest a diagnosis based on the analysis.

ML and Deep Learning (DL) are two frequently used AI methods for creating CAD programs. ML is the scientific discipline, which relies less on human specification and instead allows algorithms to decide what variables are important. As the CAD system in CXR can help doctors detect suspicious lesions that are easy to miss, thereby improving the accuracy of detection, many CAD algorithms of CXRs have been developed for routine diagnostic procedures. Previous studies reviewed the development of CAD technology between 2001 and 2009, and found the CAD systems mainly focused on a single aspect, such as detecting lung cancer nodules.,

In terms of TB detection, many studies about ML-based CAD systems on CXRs have been published in the past two decades and two studies have reviewed them., A majority of the studies primarily focused on how to create a CAD program for TB, with the AUC ranged from 0.78 to 0.99. However, these studies usually used the same databases for training and testing as well as used human readers as the reference standard, which might overestimate the diagnostic accuracy of CAD. Thus, studies focusing on the assessment of the accuracy outside of the study setting are needed to evaluate the value of the systems in TB detection. Besides, CAD has also been used in conjunction with clinical information; achieving better performance than using CAD scores or clinical information alone for TB diagnosis. These previous works suggested the potential of using CAD systems in detecting TB.

CAD with DL technology in TB detection

DL is a subset of ML, which attempts to model brain architecture and has great application prospects in the field of AI in the past few years. It is expected to revolutionize CAD systems and to play an important role in precision medical imaging. Especially in the field of computer vision, deep convolutional neural networks (DCNNs) have been proven to be very promising algorithms for various visual tasks. Table 1 shows the DL CAD solutions that are commercially available.

Table 1

Characteristics of AI-based solutions for interpretation of digital CXRs.

AI Solution	Developer	Category	Certification	Commercially available
CAD4TB version 6	DIAG, Radbound University, The Netherlands	DL	CE	Yes
qXR	Qure.AI, India	DL	CE	Yes
Lunit INSIGHT CXR	Lunit, South Korea	DL	CE	Yes

CXR: chest radiography; CE: Conformite Europeenne; DL: deep learning.

Since DCNNs can perform end-to-end training from images to classification, there is no need for target-specific manual feature engineering. Several studies primarily focusing on reporting methods for creating CAD programs for TB have published since 2016.22, 23, 24, 25 In 2017, two DCNNs (AlexNet and GoogLeNet), including pretrained and untrained models were used to evaluate the efficacy for detecting TB on CXR, which found that DL with DCNNs could accurately classify TB from healthy controls with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.99. Later, a deep DCNN model was trained by a TB-specific CXR dataset of one population (National Library of Medicine Shenzhen No.3 Hospital, China) and was tested by non-TB-specific CXR dataset (ChestX-ray8) of another population (National Institute of Health Clinical Centers, USA). This DCNN model exhibited an AUC of 0.98 and 0.85 for detecting TB in the training and intramural test sets, respectively, but the AUC dropped dramatically to 0.70 on the ChestX-ray8 dataset. About one third (36.51%) of abnormal radiographs in the ChestX-ray8 dataset were estimated to be TB-related. This suggested weaknesses of DL, such as poor generalizability and overdiagnosis. Therefore, technical specification of CXR images, disease severity distribution, dataset distribution shift, and overdiagnosis should be evaluated before applying the model in practice; especially in settings that differ from those where training dataset were from.

CAD4TB (Delft Imaging Systems, Netherlands) is one of the earliest commercially available CAD for TB. However, DL was not employed until the development of version 6, which was released in 2018. More CAD software products with DL technology were released in the past three years in worldwide, including a few from China, thereby more clinical studies primarily focusing on the assessment of the accuracy of the already-developed CAD software were conducted. A prospective study was conducted in a tertiary hospital from Pakistan to evaluate the performance of qXR(v2) and CAD4TB(v6) with mycobacterial culture being a standard reference, which collected a dataset of 2187 CXRs and reported that both software achieved non-inferior accuracy (qXR: sensitivity = 0.93; specificity = 0.75; CAD4TB: sensitivity = 0.93; specificity = 0.69) to WHO-recommended minimum values, while sensitivity might be lower when smear-negative pulmonary TB was more prevalent. A retrospective case–control study was conducted in a tertiary hospital from India using microbiologically-confirmed TB as the reference standard, which reported that qXR (v2) could detect TB with an AUC of 0.81, a sensitivity of 71% and a specificity of 80%. As many patients who present at tertiary hospitals have symptoms suggestive of TB, accurate and rapid triage tests that can rule out the disease are needed. Thus, the CAD software as a triage test for TB might be more suitable for the primary care level in settings where access to radiologists is limited than the tertiary care level.

In 2019, the Stop TB Partnership published a retrospective evaluation of three DL systems (CAD4TB (v6), Lunit INSIGHT (v4.7.2), and qXR(v2)) for detecting TB-associated abnormalities in CXRs from outpatients in Nepal and Cameroon. All 1196 individuals received a Xpert MTB/RIF assay and a CXR read by two groups of radiologists and the DL systems, with all three systems performing high AUCs (Lunit: 0.94, qXR: 0.94 and CAD4TB: 0.92) and significantly better than human radiologists. A larger dataset of 23,566 CXRs from Bangladeshi was used to evaluate five AI software platforms specific to TB-associated abnormalities, including CAD4TB (v6), InferRead®DR (v2), Lunit INSIGHT CXR (v4.9.0), JF CXR-1 (v2) and qXR (v3) by the same study group one year later. All AI systems significantly outperformed the human readers, who were certified radiologists with different years of working experience, and the AUCs from high to low are qXR (0.91), Lunit INSIGHT CXR (0.89), InferReadDR (0.85), JF CXR-1 (0.85), and CAD4TB (0.82). Fig. 1 shows a screenshot of JF CXR-1, which automatically generates probability score for TB and a heat map localizing the most indicative region of the image, providing reference for radiologists to identify TB-related abnormalities on CXRs.

Fig. 1

A screeshot of JF CXR-1, a TB screening and triaging tool. This CAD system applies DL technology, which automatically generates a heat map and probability score for TB.

WHO endorsement of CAD for CXR

In December 2020, WHO published a rapid communication on systematic screening for TB. Evidence on the use of CXR as a screening tool for detection of TB disease in several populations were reviewed by WHO in 2020. Across all populations considered, “CXR was found to be a sensitive screening tool that, while lacking sufficient specificity to confirm a TB diagnosis, has an important role in the early detection of TB in children and adults at higher risk of TB, as well as to reduce the population burden of TB disease when combined with early treatment and other public health action.”

Three independent evaluations of CAD were reviewed by WHO for both the screening and triage use cases. Each evaluation considered all three CAD technologies that were CE marked at the start of 2020 and available on the market. Evaluations showed substantial variation in CAD performance (diagnostic accuracy) across different contexts, implying that the use of CAD may require adjustment for the purpose and setting in which CAD will be implemented. Nonetheless, the diagnostic accuracy and the overall performance of CAD software were similar to the interpretation of digital CXR by a human reader, in both the screening and triage contexts.

Based on the review, WHO now recommends that CAD may be used as an alternative to human reader interpretation of plain digital CXR for TB screening and triage among adults.

Conclusion and prospects

Accurate, accessible, and affordable tools are needed for precise detection and diagnosis of TB, the first and a very challenging step in ending TB. CAD system can automatically detect and localize abnormalities in CXRs at a level comparable to human radiologists, saving time for radiologists and more importantly addressing the challenge of human resource shortage. Therefore, CAD systems are more needed in low- and middle-income countries where TB burden are high and lack of trained radiologists.

As DL algorithm is more accurate and efficient than ML algorithm in the effective training of medical image big data, the number of commercially available CAD systems with DL technology have increased substantially in recent years. They have been proved with highly sensitive and could save large proportions of the subsequent confirmatory tests. However, DL system performance at similar thresholds might vary greatly with the prevalence of TB. Therefore, while the WHO guidance for CAD software for TB emphasizes the need of predefined threshold scores, the implementers need to conduct their own pilot study to tailor specific thresholds according to the characteristics of the screened populations.,

With the recent WHO endorsement, CAD solutions can now be implemented in TB programs, guided by implementation research and operational studies to optimize impact. As for the development of future CAD systems, the correlation of demographics, clinical data and AI abnormal scores should be explored to achieve individualized risk scores, thereby optimizing performance. Besides, future research should also pay attention to improve the sensitivity of detecting smear-negative TB and to improve the generalizability of such novel technologies by eliminating diagnostic heterogeneity. In addition, as computed tomography (CT) scans allow practitioners to examine both the pulmonary parenchyma and the lymph nodes in greater detail than can be done with plain CXR alone, the development and application of CAD for CT image reading should be attached importance as well along with the popularization of CT in TB diagnosis.

Funding

This study was supported by a grant from the National Science and Technology Major Project of China [2017ZX10201302-008].

Conflicts of interest

None.