Chest Radiographs Using a Context-Fusion Convolution Neural Network (CNN): Can It Distinguish the Etiology of Community-Acquired Pneumonia (CAP) in Children?

Clinical symptoms and inflammatory markers cannot reliably distinguish the etiology of CAP, and chest radiographs have abundant information related with CAP. Hence, we developed a context-fusion convolution neural network (CNN) to explore the application of chest radiographs to distinguish the etiology of CAP in children. This retrospective study included 1769 cases of pediatric pneumonia (viral pneumonia, n = 487; bacterial pneumonia, n = 496; and mycoplasma pneumonia, n = 786). The chest radiographs of the first examination, C-reactive protein (CRP), and white blood cell (WBC) were collected for analysis. All patients were stochastically divided into training, validation, and test cohorts in a 7:1:2 ratio. Automatic lung segmentation and hand-crafted pneumonia lesion segmentation were performed, from which three image-based models including a full-lung model, a local-lesion model, and a context-fusion model were built; two clinical characteristics were used to build a clinical model, while a logistic regression model combined the best CNN model and two clinical characteristics. Our experiments showed that the context-fusion model which integrated the features of the full-lung and local-lesion had better performance than the full-lung model and local-lesion model. The context-fusion model had area under curves of 0.86, 0.88, and 0.93 in identifying viral, bacterial, and mycoplasma pneumonia on the test cohort respectively. The addition of clinical characteristics to the context-fusion model obtained slight improvement. Mycoplasma pneumonia was more easily identified compared with the other two types. Using chest radiographs, we developed a context-fusion CNN model with good performance for noninvasively diagnosing the etiology of community-acquired pneumonia in children, which would help improve early diagnosis and treatment.

Background

Pneumonia is an acute infection of the lung parenchyma by one or more pathogens, such as viruses, bacteria, and mycoplasma [1]. According to the World Health Organization (WHO), the incidence in children under five is evaluated to be 0.05 episodes per child-year in developed countries and 0.29 episodes per child-year in developing countries. This means there are about 151 million new cases in children that occur annually in developing countries, including 21 million in China [2]. Community-acquired pneumonia (CAP) has high morbidity and mortality rates in both developed and developing countries [3]. It is reported that pneumonia and preterm birth complications are the principal causes of death in children under five [4].

Viruses, bacteria, and mycoplasma are the common etiologies of pneumonia, but they need different medications and treatments. Viral pneumonia is treated with supportive care, bacterial pneumonia requires immediate antibiotic therapy (Penicillin G and amoxicillin), and macrolides are often used in the treatment of mycoplasma pneumonia. The empirical use of antibiotics remains fundamental to the treatment of pneumonia [5]. Untimely diagnosis of pathogens can lead to overuse of antibiotics and the formation of drug resistance [6, 7]. Delayed diagnosis increases the risk of irreversible damage to the patient’s respiratory system. Therefore, accurate and timely diagnosis is the key to ensuring the most effective treatment.

In clinical practice, most diagnoses of CAP are based on radiology, clinical signs, and symptoms. In most patients with respiratory symptoms, a chest radiograph is the first choice [8]; it is also considered as the clinical reference standard for pneumonia [9]. C-reactive protein (CRP) and white blood cell (WBC) count are the most commonly used markers of inflammation in clinical practice, which are used to evaluate children suspected of pneumonia [10, 11]. However, the diagnostic challenge of childhood CAP is that although clinical symptoms and inflammatory markers and radiological signs are indicative of pathogens [12-14], pneumonia pathogens cannot be reliably distinguished [3, 5, 15–18]. Sputum culture, multiplex polymerase chain reaction, or specific mycoplasma antibody tests can diagnose pneumonia [19, 20] but have some limitations.

To assist pediatricians and improve their diagnostic performance, computer-aided diagnostic systems, e.g., deep learning or radiomics, have been increasingly utilized in radiology and medical image analysis [21-24]. Recently, it has been found that deep learning technology combined with big data have a better detection effect in chest image diagnosis [25-27], including detecting the presence and detection of pneumonia [28, 29], emphysema and quantification [30], and pneumothorax [31, 32].

In this study, we first investigated the clinical characteristics and imaging biomarkers that were often used in chest radiograph analysis. Further analysis showed that the characteristics and biomarkers with different etiologies of children with CAP were significantly different, and the intensity-related biomarkers from lung and lesion areas have a complementary presentation ability. Despite the statistical significance, the scatter plot of the biomarkers showed broad overlap between three classes (as shown in the Results). Hence, we improved the diagnosis ability of chest radiographs using deep learning methods, and we built a pediatric chest radiograph dataset and then developed the prediction models based on chest radiographs and clinical characteristics. The diagnostic ability of a full-lung image, local-lesion image, and clinical characteristics were evaluated. The complementarity ability between a full-lung image and local-lesion image and between image CNN signature and clinical characteristics were explored.

Materials and Methods

Patient Selection and Data Collection

The dataset was approved by our ethics committee of Beijing children’s hospital, where the requirement of informed consent was waived. The retrospective analysis of 1769 (viral pneumonia 487, bacterial pneumonia 496, and mycoplasma pneumonia 786) cases of childhood CAP was confirmed in our hospital between 2015 and 2018 including chest radiographs of the first examination, while CRP was measured and WBC count was determined, excluding patients with poor image quality and incomplete clinical data. The chest radiograph dataset only contained patients from newborns to 18-year-olds, and most of them were children under 10 years old. We included both posteroanterior and bedside anteroposterior chest radiographs, where all radiographs were obtained with a single dedicated radiography unit (DR7500; Kodak Healthineers). The patient recruitment procedure and workflow of this study are shown in Fig. 1. The patient demographics for the dataset and clinical information are shown in Table 1.

Fig. 1

The patient recruitment procedure and workflow of this study

Table 1

Characteristics of patients in the training, validation, and test cohorts

Characteristics	Training and validation cohort				Test cohort
	Virus	Bacteria	Mycoplasma	P value	Virus	Bacteria	Mycoplasma	P value
Sex, no. (%)				< 0.001				0.048
Male	271 (69.8)	259 (65.1)	359 (57.3)		70 (70.7)	60 (61.2)	88 (55.3)
Female	117 (30.2)	139 (34.9)	268 (42.7)		29 (29.3)	38 (38.8)	71 (44.7)
Age, mean ± SD, months	30.28 ± 36.89	28.84 ± 34.42	52.14 ± 36.29	< 0.001	27.51 ± 35.37	28.63 ± 39.44	51.53 ± 33.53	< 0.001
CRP, No. (%)				< 0.001				< 0.001
≤ 8	212 (54.6)	186 (46.7)	420 (67.0)		55 (55.6)	41 (41.8)	118 (74.2)
> 8	176 (45.4)	212 (53.3)	207 (33.0)		44 (44.4)	57 (58.2)	41 (25.8)
WBC NO. (%)				< 0.001				0.003
< 4	27 (7.0)	13 (3.3)	30 (4.8)		6 (6.1)	5 (5.1)	8 (5.0)
[4, 10]	213 (54.9)	178 (44.7)	386 (61.6)		57 (57.5)	43 (43.9)	103 (64.8)
> 10	148 (38.1)	207 (52.0)	211 (33.6)		36 (36.4)	50 (51.0)	48 (30.2)

P value was derived from univariable association analyses between each characteristic and pneumonia types. Sex was calculated with chi2 test. Age, CRP, and WBC were calculated with Kruskal–Wallis test

SD standard deviation

Statistical Analysis of Chest Radiograph Markers

We hypothesized that the chest radiograph appearance influenced by different etiologies may be reflected in the intensity of the radiograph. Thus, we investigated the intensity-related biomarkers from the chest radiograph image analysis, as shown in Fig. 2A. (1) Mean intensity of lung and lesion areas: The mean value of the pixel intensity was calculated from the segmented lung and lesion areas, respectively. (2) Standard deviation of the intensity of lung and lesion areas: The standard deviation (STD) was calculated from the intensity histogram of the lung and lesion area pixels, respectively.

Fig. 2

The process of study for distinguishing the etiology of children with CAP using chest radiographs: A segmentation network and intensity-related biomarker analysis and B classification models based on local-lesion image, full-lung image and clinical characters, and the prediction performances for the three etiologies

Research Design and Workflow

The patient cohort was stochastically divided into training, validation, and test cohorts in a 7:1:2 ratio. The training cohort was used to optimize the CNN. During training, the results of the validation cohort were used to select the optimal model. The test cohort was not disclosed until the model was finalized. In order to avoid overfitting, data augmentation techniques including random rotations, translations, and flipping were used and further increased the size of the training cohort. The prediction ability of the full-lung image, local-lesion image, and clinical characteristics were studied and the prediction performances for the three etiologies are shown (Fig. 2B).

Preprocessing and ROI Acquisition

Routinely-used chest radiographs include some non-lung areas (neck, abdomen, bone, etc.) and blank spaces outside the body. To ensure the consistency of the exposure field, we first segmented lungs from the chest radiographs using an automatic segmentor, XLSor [33], which is an end-to-end convolutional model that can perform robust and accurate lung segmentation. The chest radiographs were input into XLSor, and the lung mask annotations were output automatically according to the results of the segmentation. The lung region was cropped automatically, which was a lung region of interest (ROI). The pneumonia lesion region was drawn manually using ITK-SNAP software (version 3.2.0; www.itksnap.org) by two board-certified radiologists who were blinded to the histological diagnoses and patient clinical information. Their primary focus was hand-crafted segmentation of pneumonia lesions; in case of disagreement between the two radiologists, consensus was reached through discussion. The lesion region was cropped automatically according to the bounding box derived from manual segmentation, and we call this partial lesion region a lesion ROI. In order to match the neural network inputs, all lung ROIs were resized to a size of 224 × 224 and all lesion ROIs were resized to a size of 160 × 160. Image intensities were normalized using histogram equalization. Figure 3B shows the process of lung ROI and lesion ROI acquisition.

Fig. 3

Overall architecture of the proposed neural network approach: A the CNN structure of the context-fusion model and B the process of lung ROI and lesion ROI acquisition

Model Development and Training

Both the lung ROI and lesion ROI generated from the chest radiographs were analyzed, and two clinical characteristics including WBC count and CRP were used to build a clinical model. In clinical diagnosis, a radiologist usually makes a comprehensive analysis about the entire lungs and local lesions. To provide this contextual information, we developed an end-to-end CNN model (context-fusion CNN). Overall, based on the chest radiograph, we built three models including a full-lung model, a local-lesion model, and a context-fusion model; based on clinical characteristics, we built a clinical model. We then combined the signature of the context-fusion model with two clinical characteristics to study the complementarity between the radiograph and characteristics. Table 5

Table 5

Lesion areas intensity variance statistics

	Mean	STD	Statistical significance
			Virus	Bacteria	Mycoplasma
Virus	0.103	0.025
Bacteria	0.098	0.023	***
Mycoplasma	0.097	0.022	***	-

***p < 0.001

In our study, the CNN models were derived from DenseNet121 [34], which consisted of a densely connected CNN feature extractor and a classifier. The full-lung model and local-lesion model were the standard Densenet. The inputs of the full-lung model were the lung ROIs with a size of 224 × 224, and the outputs of the feature extractor were full-lung CNN features. Similar to the full-lung model, the inputs of the local-lesion model were the lesion ROIs with a size of 160 × 160 and the outputs of the feature extractor were local-lesion CNN features. A clinical model based on the clinical characteristics of the WBC count and CRP was built, which used the Catboost machine learning method [35].

The context-fusion CNN simultaneously extracted the lung features and lesion features with a two-branch CNN, and two branches did not share the weights for extracting different features from the lung and local lesion. Furthermore, the feature fusion module merged two features as context features with a concatenation operation and decreased the redundancy features between lung and local branch with a fully connected layer (the number of input nodes was much smaller than the number of output nodes). Finally, another fully connected layer was used as a classifier for predicting the etiology of CAP. Figure 3A shows the CNN structure and details of the context-fusion model.

The full-lung model, local-lesion model, and context-fusion model were trained based on the Pytorch platform and optimized via an Adam algorithm with a mini-batch size of 32. The learning rate was set to 0.001 with a momentum coefficient of 0.9. The weights of CNN were initialized stochastically. The clinical model was trained based on the Catboost platform, the learning rate was set to 0.25, and the depth was 2. The optimal model of every model was the one with the lowest validation loss during training.

Statistical Analysis

The performance of models was evaluated by assessing the accuracy (ACC) of training and test cohorts. In addition, softmax or logistic regression probabilities were used to calculate ACC, precision, recall, area under curve (AUC) of the ROC analysis, sensitivity, and specificity. Statistical analysis was conducted with a Python toolkit (scipy.stats). Chi2 test and Kruskal–Wallis test were used. A two-sided P value < 0.05 was used to indicate statistical significance. Wilcoxon rank sum test was used to compare chest radiograph markers among three classes with different data sizes.

Results

Clinical Characteristics

Clinical characteristics consist of sex, age, CRP, and WBC count are reported in Table 1. CRP and WBC count were significantly associated with the etiology of CAP after the univariate analysis (p < 0.05), and there was a significant difference in sex and age. Generally, a virus is the most common etiology of CAP in infants and young children, with mycoplasma occurring in children over the age of 5 [36]. Although mycoplasma pneumonia was included in this study according to the age of viral pneumonia and bacterial pneumonia, children with mycoplasma pneumonia were still older than those with viral and bacterial pneumonia. Stratified analyses for the subgroups were classified according to sex. Specifically, man and woman subgroups yielded AUC values of 0.844 and 0.816 for viruses, 0.850 and 0.874 for bacteria, and 0.918 and 0.908 for mycoplasma, respectively (Delong test p-values: 0.220, 0.231, and 0.260). The results of the above stratified analyses indicated that our model was not affected by sex.

Statistical Significance of Potential Etiologic Biomarkers

Four intensity-related biomarkers from the CXR image were extracted and analyzed. Mean pixel intensity of each lung and lesion area is shown in the scatter plot of Figs. 4A and 5A. In both areas, mycoplasma cases showed higher mean intensity compared to other cases with a statistical significance level (p < 0.001 for all). In the lesion area, the difference between viral and bacterial cases was statistically significant (p < 0.01). The STDs of pixel intensity of each lung and lesion areas are scattered in the plot in Figs. 4B and 5B. In the lesion area, the variance values of the viral cases were higher than other classes with a statistical significance (p < 0.001 for all). Tables 2, 3, 4, 5 describe the corresponding statistical results. Despite statistical significance, the scatter plots of the biomarkers showed broad overlap between three classes. Statistical significance levels are indicated as asterisks; * for p < 0.05, ** for p < 0.01, and *** for p < 0.001.

Fig. 4

Scatter and box plots for both the mean pixel intensity (left) and the STD of the pixel intensity from lung areas (right). Blue triangles in box plots show mean values, and statistical significance levels are indicated as asterisks; *p < 0.05, **p < 0.01, and ***p < 0.001

Fig. 5

Scatter and box plots for both the mean pixel intensity (left) and the STD of pixel intensity from lesion areas (right). Blue triangles in box plots show mean values and statistical significance levels are indicated as asterisks; *p < 0.05, **p < 0.01, and ***p < 0.001

Table 2

Lung area intensity statistics

	Mean	STD	Statistical significance
			Virus	Bacteria	Mycoplasma
Virus	0.405	0.082
Bacteria	0.405	0.072	-
Mycoplasma	0.439	0.074	***	***

***p < 0.001

Table 3

Lung areas intensity variance statistics

	Mean	STD	Statistical significance
			Virus	Bacteria	Mycoplasma
Virus	0.255	0.035
Bacteria	0.258	0.033	*
Mycoplasma	0.259	0.036	-	-

*p < 0.05

Table 4

Lesion areas intensity statistics

	Mean	STD	Statistical significance
			Virus	Bacteria	Mycoplasma
Virus	0.686	0.085
Bacteria	0.697	0.083	**
Mycoplasma	0.717	0.078	***	***

**p < 0.01 and ***p < 0.001

Overall Performance of the Models

The context-fusion model yielded the best performance with an ACC of 0.70 in the training cohort and 0.72 in the test cohort. Figure 6 shows the prediction distribution of patients in the three pneumonia types. It illustrated that the prognosis for the etiology of CAP needs a comprehensive analysis about the entire lungs and local lesions. The clinical model had poor diagnostic performance with an ACC of 0.60 in the training cohort and 0.51 in the test cohort. Compared with the local-lesion model, the full-lung model yielded better performance with an ACC of 0.65 in the training cohort and 0.62 in the test cohort. The results in the training and test cohorts are provided in Table 6.

Fig. 6

The prediction distribution of patients for the three pneumonia types

Table 6

Overall performance of the prediction models

Model	Train/test	Accuracy	Precision	Recall
Context-fusion	Train	0.70	0.71	0.70
	Test	0.72	0.73	0.72
Full-lung	Train	0.65	0.66	0.65
	Test	0.62	0.64	0.62
Local-lesion	Train	0.57	0.53	0.57
	Test	0.58	0.54	0.58
Clinical	Train	0.60	0.62	0.60
	Test	0.51	0.49	0.51

Prediction Performances

The prediction performances for every etiology of CAP were consistent with the overall performances, and the context-fusion model yielded the best performance and was significantly higher than the other models in the training cohort and test cohort. Mycoplasma is easily diagnosed with an AUC of 0.911 in the training cohort and 0.924 in the test cohort. The diagnostic ability for viruses and bacteria was equal. Table 7 shows the results of the three pneumonia types, and the ROCs are shown in Fig. 7.

Table 7

The prediction performance for the three etiologies of CAP

Model	Category	Training cohort			Test cohort
		AUC	SEN	SPE	AUC	SEN	SPE
Context-fusion	Virus	0.837	0.842	0.663	0.851	0.776	0.793
	Bacteria	0.858	0.733	0.804	0.876	0.702	0.902
	Mycoplasma	0.911	0.868	0.798	0.924	0.834	0.865
Full-lung	Virus	0.786	0.709	0.742	0.782	0.747	0.704
	Bacteria	0.785	0.732	0.735	0.781	0.784	0.669
	Mycoplasma	0.843	0.875	0.663	0.798	0.827	0.646
Local-lesion	Virus	0.687	0.764	0.530	0.717	0.608	0.735
	Bacteria	0.726	0.653	0.712	0.683	0.670	0.650
	Mycoplasma	0.812	0.795	0.686	0.819	0.885	0.624
Clinical	Virus	0.779	0.764	0.650	0.595	0.794	0.361
	Bacteria	0.797	0.793	0.648	0.584	0.309	0.894
	Mycoplasma	0.779	0.821	0.590	0.633	0.848	0.408
Context-fusion and clinical	Virus	0.838	0.744	0.766	0.843	0.753	0.802
	Bacteria	0.875	0.780	0.808	0.904	0.777	0.914
	Mycoplasma	0.920	0.887	0.792	0.928	0.873	0.822

AUC area under curve, SEN sensitivity, SPE specificity

Fig. 7

The receiver operating characteristic curves of A context-fusion model, B full-lung model, C local-lesion model, and D clinical model

Discussion

Early diagnosis of pneumonia is critical to prevent complications including death. In this retrospective research, we built deep learning prediction models and systematically analyzed the ability of chest radiographs and clinical characteristics for distinguishing the etiology of CAP in children. A context-fusion model combined full-lung features and local-lesion features showing good performance, and there was complementarity between the image CNN signature built from the context-fusion model and clinical characteristics. The adopted models can potentially improve the diagnostic speed and accuracy in a non-invasive way.

At first, we analyzed the clinical characteristics and intensity-related biomarkers from chest radiographs. A positive correlation was found between the etiology of childhood CAP and both the clinical characteristics and chest radiographs biomarkers. The results were consistent with the study of Oh et al. [37], and they described that the mean intensity and variance values of viral pneumonia were higher than bacterial pneumonia or tuberculosis cases with statistical significance (p < 0.001 for all). Despite statistical significance, the scatter plot of the biomarkers showed a broad overlap between several classes. In addition, the presentation ability of the biomarkers from lung and lesion areas was complementary. Hence, we built a diagnostic model using deep learning methods and explored the complementarity ability between the full-lung image and local-lesion image.

In our study, deep learning methods were used to mine valuable features from chest radiographs for differentiating the etiology of pediatric pneumonia. Some studies compared a variety of algorithms to obtain the most optimal model [8, 29]. Deep learning is a promising technique for analyzing medical imaging, and some research in chest radiograph analyses achieved excellent performance. For example, in some studies [38, 39], they built deep learning models to diagnose chest pathology in chest radiographs and the models were competitive with radiologists on some pathology. Some researchers proposed more effective models to localize diseases using limited location annotations [40, 41]. Our results are consistent with the above study. Our context-fusion model was derived from Densenet and adopted a dual-path construction to combine the full-lung and local-lesion features from the patients’ chest radiograph, which proved valuable.

The full-lung and local-lesion regions of the chest radiographs were analyzed separately, and the diagnostic ability of the local-lesion model was lower than that of the full-lung model. The reasons may be as follows: infection via different pathogens may lead to inflammation of the lobes, bronchi, alveoli, or interstitial lung areas, as well as bronchiolitis. The lesions vary in size and often permeate the whole lung, presenting on chest radiographs as whole-lung or partial pulmonary consolidation, bronchogenic inflation, pleural effusion, and interstitial infiltration [42], while the lesion that we chose to delineate on the chest radiograph was in the most severe areas, not the entire lung; secondly, limited by spatial resolution, a chest radiograph can directly show the trachea and large bronchus as well as related lesions, but cannot directly show the bronchioles and their lesions. When lesions occur in this part of the airway, the chest radiograph can show increased brightness and increased lung volume, which is not delineated; thirdly, we chose to depict the lesions by radiologists with diagnostic experience, which is subjective, and the full-lung model included the entire lung that was objectively unaffected by doctors. The context-fusion model had the highest diagnostic accuracy, indicating that not only the lesion delineation area, but also the entire chest radiograph had visual changes that were difficult to recognize visually due to pathological changes in pneumonia.

The pathological difference of different etiologies of CAP will affect the appearance of the chest radiograph. Viral pneumonia can be seen as a large lobe or multiple focal infiltrates, typical bacterial pneumonia is usually lobar pneumonia with pleural effusion [43], and both are indistinguishable on chest radiographs. The pathology of mycoplasma pneumonia is usually confined to the airway wall, and even small airways and respiratory bronchioles and chest radiographs showed peri-bronchial infiltration, reticular nodules, and patchy and focal consolidation [44, 45]. In our study, the diagnostic ability of the models based on chest radiographs was higher than the clinical model. In other published studies, WBC count and CRP could also be found to indicate the presence of pneumonia, but they did not play a significant role in the pathogen determination of pneumonia [46], which was also consistent with this study.

A conjoint analysis regarding the clinical characteristics and image signature built from the context-fusion model was conducted using logistic regression; the results showed that there was good complementarity between the image signature and clinical features. As can be seen from Table 3, the AUC of the context-fusion model for diagnosing viral pneumonia, bacterial pneumonia, and mycoplasma pneumonia was 0.851, 0.876, and 0.924, respectively; the sensitivity was 0.776, 0.702, and 0.834; and the specificity was 0.793, 0.902, and 0.865, respectively. Mycoplasma pneumonia is more easily identified from the three, followed by bacterial pneumonia and viral pneumonia. Bacterial pneumonia and viral pneumonia are more difficult to distinguish, but the clinical indicators of the differential efficacy are relatively low with AUC values being lower than 0.6.

Diagnosing the etiology of CAP in children via chest radiographs is practical in our study, since the local lesion may retain critical features of pneumonia and the lung region contains global information. The combination of global and local information contributes to diagnosing the etiology of CAP in children, which is significantly effective. It is promising for future research to extend its application to more tasks, such as the diagnosis between common viruses and coronavirus disease 2019.

There are also some limitations in this study. Our chest radiograph was the first chest radiograph of the patient admitted to the hospital. The patient had already had fever, cough, and other clinical manifestations before admission, so it is uncertain if the patient had pneumonia for several days when the chest radiograph was collected and the prognosis of the patient was not tracked further. Secondly, this study only limited pneumonia to three categories, which did not involve tuberculosis or fungal pneumonia, and did not make a more specific classification of bacterial or viral pneumonia. Finally, the clinical indicators we collected were relatively single, and it would be more meaningful if procalcitonin or percentage of neutrophils were added.

Conclusion

This study provides a deep learning-based etiological prediction technology for community-acquired pneumonia, which can realize the early diagnosis of CAP in children in a non-invasive and rapid manner, and is of great significance for guiding clinical medication and reducing the mortality rate of childhood pneumonia.