Diagnostic performance of congestion score index evaluated from chest radiography for acute heart failure in the emergency department: A retrospective analysis from the PARADISE cohort.

BACKGROUND: Congestion score index (CSI), a semiquantitative evaluation of congestion on chest radiography (CXR), is associated with outcome in patients with heart failure (HF). However, its diagnostic value in patients admitted for acute dyspnea has yet to be evaluated. METHODS AND
FINDINGS: The diagnostic value of CSI for acute HF (AHF; adjudicated from patients' discharge files) was studied in the Pathway of dyspneic patients in Emergency (PARADISE) cohort, including patients aged 18 years or older admitted for acute dyspnea in the emergency department (ED) of the Nancy University Hospital (France) between January 1, 2015 and December 31, 2015. CSI (ranging from 0 to 3) was evaluated using a semiquantitative method on CXR in consecutive patients admitted for acute dyspnea in the ED. Results were validated in independent cohorts (N = 224). Of 1,333 patients, mean (standard deviation [SD]) age was 72.0 (18.5) years, 686 (51.5%) were men, and mean (SD) CSI was 1.42 (0.79). Patients with higher CSI had more cardiovascular comorbidities, more severe congestion, higher b-type natriuretic peptide (BNP), poorer renal function, and more respiratory acidosis. AHF was diagnosed in 289 (21.7%) patients. CSI was significantly associated with AHF diagnosis (adjusted odds ratio [OR] for 0.1 unit CSI increase 1.19, 95% CI 1.16-1.22, p < 0.001) after adjustment for clinical-based diagnostic score including age, comorbidity burden, dyspnea, and clinical congestion. The diagnostic accuracy of CSI for AHF was >0.80, whether alone (area under the receiver operating characteristic curve [AUROC] 0.84, 95% CI 0.82-0.86) or in addition to the clinical model (AUROC 0.87, 95% CI 0.85-0.90). CSI improved diagnostic accuracy on top of clinical variables (net reclassification improvement [NRI] = 94.9%) and clinical variables plus BNP (NRI = 55.0%). Similar diagnostic accuracy was observed in the validation cohorts (AUROC 0.75, 95% CI 0.68-0.82). The key limitation of our derivation cohort was its single-center and retrospective nature, which was counterbalanced by the validation in the independent cohorts.
CONCLUSIONS: In this study, we observed that a systematic semiquantified assessment of radiographic pulmonary congestion showed high diagnostic value for AHF in dyspneic patients. Better use of CXR may provide an inexpensive, widely, and readily available method for AHF triage in the ED.

Introduction

Acute heart failure (AHF) is 1 of the leading causes of acute dyspnea in the emergency department (ED) [1] and is associated with a higher risk of morbidity and mortality [2,3]. In-hospital mortality is reported to be greater than 10% [4] and has remained stable in the last 30 years. As prognosis is associated with initiation time of specific therapies [5], current guidelines emphasize the importance of early diagnosis and treatment initiation to improve clinical outcomes [5,6]. However, a minority of patients with AHF receive treatment within 1 hour of admission [5], in contradiction with current recommendations [6]. In addition, a third of AHF diagnosis are missed in the ED [6], further delaying access to care. An increasing number of better diagnostic tools for AHF are available in the ED [7-9]. However, there is likely room for improving the diagnostic approach to AHF from widely available routine tools including chest radiography (CXR).

CXR is a fast and inexpensive method performed systematically in the ED in patients with acute dyspnea [10-12]. It is the first-line diagnostic imaging modality advocated in current guidelines [13]. However, its diagnostic accuracy for HF has been reported to be relatively low [14-16]. In particular, diagnosing HF in patients with concomitant lung diseases such as chronic obstructive pulmonary disease (COPD) and pneumonia still remains a challenge [17].

A new semiquantitative approach to pulmonary congestion has recently emerged in the field of HF. Congestion score index (CSI) is a semiquantitative approach to pulmonary congestion based on a 6-zone evaluation of CXR, scoring each zone from 0 (no congestion) to 3 (intense alveolar pulmonary edema). CSI is a strong risk stratifier in patients with stable or worsening HF [18-20]. However, there are little available data regarding its diagnostic value for AHF.

The aims of the present study are to investigate the diagnostic value of pulmonary congestion assessed with CSI for AHF in patients admitted for acute dyspnea in the ED and to assess its discriminative value comparatively to and on top of currently used clinical diagnostic models and natriuretic peptide measurements.

Methods

Study population

This study is reported as per the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline (S1 Checklist). The Pathway of dyspneic patients in Emergency (PARADISE) cohort is a retrospective cohort study including consecutive patients aged 18 years or older who were admitted for acute dyspnea in the ED of the Nancy University Hospital (France) between January 1, 2015 and December 31, 2015 as detailed previously [21,22]. The hospital’s electronic charts (resurgences) were systematically reviewed by investigators to search for the records of all patients admitted for acute dyspnea in the ED. All patients with signs or symptoms of dyspnea or requiring oxygen therapy for dyspnea during (or prior to) their ED stay were included. Patients with shock or cardiac arrest were excluded from this analysis as dyspnea was not the primary condition triggering ED admission. The PARADISE cohort is consequently a cohort of unselected consecutive patients with acute dyspnea in the ED. In the current study, a total of 1,333 dyspneic patients with available information on CXR at the ED were analyzed (S1 Fig). Demographic parameters, medical history, physical examination, laboratory findings, and treatment received in the ED were retrieved from the patients’ electronic records.

External validation was performed on the merged dataset from the HF disease management program entitled “Insuffisance CArdiaque en LORraine (ICALOR)” [23,24] (included at a different time period that the data from the Nancy University Hospital used in the derivation set) and data from the Epinal Hospital (a secondary care hospital located 70 km from the Nancy University Hospital) within the Pathway and Urgent caRe of Dyspneic Patient at the Emergency Department in LorrainE District (PURPLE) multicenter cohort (NCT03194243). Briefly, we used the data from the previously described [19] 117 patients included in the ICALOR disease management cohort during a hospitalization for acutely decompensated HF in the Cardiology Department of the Nancy University Hospital (France). We also used the data of 107 consecutive patients admitted for acute dyspnea with available CXR data and discharge diagnosis in the ED of the Epinal Hospital. This external validation set allowed us to test CSI in a significantly different setting (mixing cardiology department from a tertiary care hospital and an ED of a secondary care hospital) than our derivation set.

Under French law, no formal Institutional Review Board approval is required for data extraction from patient medical records in single-center cohorts (the PARADISE and ICALOR cohorts). For the PURPLE cohort, patients were informed through a notice at admission and could refuse their inclusion in the study, although no formal consent was required in keeping with the framework of the Commission Nationale de l’Informatique et des Libertés (CNIL). The PARADISE cohort was recorded by the local hospital corresponding agent of the CNIL (Number R2016-08) and was registered on clinicaltrials.gov (NCT02800122). The PURPLE cohort was approved by an ethical board (“Comité de Protection des personnes”—Number 2016–63, ID RCB 2016-A01877-44) and CNIL (Number DR-2017-098) and was registered on clinicaltrials.gov (NCT03194243).

Diagnosis of heart failure

HF was diagnosed according to the European Society of Cardiology (ESC) guidelines [25]. Diagnosis of AHF was coded independently by 2 medical physicians (GG and TH) according to the ESC guidelines [25]. Each physician had access to all ED medical charts as well as additional hospital admission test results and records (e.g., echocardiography, natriuretic peptide level, and patient response to diuretic/bronchodilator therapy) but were blinded to CSI quantification on CXR. Homogenous coding was ensured by a trained senior physician (TC). Importantly, to determine the discharge diagnosis, we focused on the main cause of acute dyspnea rather than background medical history or coexisting conditions.

Radiographic congestion score index

Radiographic CSI was used to quantify the severity of pulmonary congestion in CXR as previously published [18,19]. After dividing the lung field into 6 topographical zones, each area was assessed as follows: Score 0, no congestion sign; Score 1, cephalization (superior area), perihilar haze or perivascular/peribronchial cuffing, or Kerley A lines (middle area), Kerley B, or C lines (inferior area); Score 2, interstitial or localized/mild alveolar pulmonary edema; Score 3, intense alveolar pulmonary edema (Fig 1). To enhance the reproducibility of the severity of confluent edema, a portion of the divided lung fields which was visually similar to the cardiac silhouette was regarded as an intense zone, whereas the field with weaker density was regarded as a mildly intense zone. Lung areas were not scored when more than one-third of the divided lung fields were occupied by pleural effusion (including vanishing tumor), atelectasis, or cardiac silhouette. CSI was calculated as the sum of the scores in each zone divided by the number of available zones. An examiner also assessed the presence of pneumonia, pleural effusion, cardiomegaly by cardiothoracic ratio (>50%), and the difficulty in assessing CSI.

Fig 1

Radiographic CSI.

The scoring is performed on 6 lung fields. The absence of radiographic congestion signs in a lung filed is graded as a score of “0.” Panels A and B provide examples. (A) Example: CSI = (2+3+3+3+2+1)/6 = 2.33. There is diffuse alveolar edema, appearing as intense edema in the left superior field and middle fields. (B) Example: CSI = (1+1+1+1+1+0)/6 = 0.83. Cephalization in superior fields and peribronchial and perivascular cuffing are visible in middle fields, respectively. *Confluent edema was regarded as intense edema when the density in an area of the divided lung field was visually similar to that of cardiac silhouette. CSI, congestion score index.

CXR was analyzed by a single emergency physician (AD), blinded to clinical data and discharge diagnosis, with no previous training in congestion quantification on CXR prior to that in the present study. After a short training (approximately 3 hours) using a 20-patient sample with a radiographic CSI expert (MK), intraobserver and interobserver agreements (with MK) were tested on 30 randomly selected patients, while blinded to clinical status and diagnosis. Intraclass correlation coefficients showed good reproducibility (intraobserver agreement, 0.85, 95% CI 0.71 to 0.93 and interobserver agreement 0.81, 95% CI 0.64 to 0.90).

Brest score

The Brest score was calculated for every patient based on the patients’ medical charts. This diagnostic score for AHF in dyspneic patients is based on age, comorbidities (i.e., prior history of HF, myocardial infarction, and COPD), pattern of dyspnea, ST segment abnormalities, and signs/symptoms of congestion (i.e., rales and leg edema) [26,27].

Statistical analysis

Categorical variables are expressed as frequencies (percentages) and continuous variables as means ± standard deviation (SD) or median (25th and 75th percentiles) according to the distribution of the variables. Comparisons of demographic, clinical, and biological parameters among quartiles of radiographic CSI were analyzed using χ2 tests for categorical variables and ANOVA or Kruskal–Wallis test for continuous variables. Interobserver and intraobserver agreements of CSI were assessed with the intraclass correlation coefficient.

We analyzed 1,333 dyspneic patients with 289 AHF diagnosis—which provided a sizeable statistical power to assess diagnostic performance in a multivariable setting [28]. A logistic regression model was used to assess the association of CSI with diagnosis of AHF. Multivariable analyses included relevant confounders as previously shown [21]: model 1: age, sex, body mass index (BMI), presence of hypertension, diabetes mellitus, coronary artery disease, atrial fibrillation, prior HF admission, use of angiotensin converting enzyme inhibitor/angiotensin receptor blocker, beta-blocker, diuretics, leg edema, jugular venous distension, hemoglobin, white blood cell count, and estimated glomerular filtration rate (eGFR, as calculated by the Chronic Kidney Disease Epidemiology Collaboration formula [29]) at admission; model 2: Brest score. Receiver operating characteristics (ROC) curve was used to determine the diagnostic value of CSI in AHF. All correlation coefficients of variables included in the models were less than 0.50 with CSI, suggesting the absence of important in-model collinearity.

The increase in discriminative value of the addition of CSI for AHF diagnosis on top of the aforementioned potential covariates was assessed using continuous net reclassification improvement (NRI) [30]. In addition, in 498 (37.4%) patients who had available b-type natriuretic peptide (BNP) measurements, the added value of CSI on the top of the Brest score and BNP was assessed.

This analysis on the PARADISE cohort was planned in February 2019, although no formal analysis was written. The general analysis intention was to evaluate the diagnostic value of CSI for AHF. Of note, among all patients with available data, CXR was assessed, blinded for clinical data and diagnosis, and standard statistical approaches were used. Based on recommendations made during the peer-review process, we conducted restricted cubic spline regression analysis for the association between CSI and AHF diagnosis.

All analyses were performed using R version 3.4.0 (R Development Core Team, Vienna, Austria). A 2-sided p-value < 0.05 was considered statistically significant. No imputation was performed.

Results

Baseline characteristics

Less than 10% of the population had no available data on CXR (S1 Table). These patients were markedly younger and had less comorbidities than patients who underwent CXR. The characteristics of the PARADISE cohort population across different discharge diagnoses such as AHF, COPD, and pneumonia are depicted in S2 Table.

In a total of 1,333 patients included in this study, a half were men, mean age was 72.0 ± 18.5 years, mean BMI was 25.5 ± 5.5 kg/m2, and less than 10% had a prior admission for HF (7.1%) (Table 1). CXR was considered as difficult to interpret during CXR reviewing in 502 patients (37.7%). Mean CSI was 1.42 ± 0.79. Patients with a higher CSI had more cardiovascular risk factors, comorbidities, more frequent prior HF admission, more severe congestion, inflammation status, higher BNP, poorer renal function, and more respiratory acidosis at admission (Table 1).

Table 1

Baseline characteristics according to the radiographic CSI (quartiles).

		CSI quartiles
	Global(N = 1,333)	Quartile I,<0.84(N = 376)	Quartile II,0.84–1.40(N = 328)	Quartile III,1.40–2.00(N = 359)	Quartile IV,≥2.00(N = 270)	% missingvalue	p-value	Adjusted p-value*
Age, years	72.0 ± 18.5	57.4 ± 20.9	73.4 ± 15.4	79.3 ± 12.6	80.9 ± 11.8	0	<0.001	—
Men, N (%)	686 (51.5%)	212 (56.4%)	168 (51.2%)	166 (46.2%)	140 (51.9%)	0	0.06	0.26
BMI, kg/m2	25.5 ± 5.5	24.7 ± 4.8	25.3 ± 5.0	25.5 ± 5.8	26.8 ± 6.4	1.1	<0.001	—
Medical history, N (%)
Hypertension	729 (54.7%)	111 (29.5%)	194 (59.1%)	236 (65.7%)	188 (69.6%)	0	<0.001	0.03
Diabetes mellitus	302 (22.7%)	50 (13.3%)	62 (18.9%)	94 (26.2%)	96 (35.6%)	0	<0.001	0.03
Dyslipidemia	280 (21.0%)	51 (13.6%)	79 (24.1%)	82 (22.8%)	68 (25.2%)	0	<0.001	0.36
Coronary artery disease	162 (12.2%)	16 (4.3%)	36 (11.0%)	63 (17.5%)	47 (17.4%)	0	<0.001	0.001
Atrial fibrillation	313 (23.5%)	33 (8.8%)	70 (21.3%)	107 (29.8%)	103 (38.1%)	0	<0.001	0.003
HF	258 (19.4%)	21 (5.6%)	47 (14.3%)	93 (25.9%)	97 (35.9%)	0	<0.001	<0.001
Prior HF admission, N (%)	95 (7.1%)	6 (1.6%)	15 (4.6%)	28 (7.8%)	46 (17.0%)	0	<0.001	<0.001
Medication, N (%)
ACEi/ARB	444 (34.7%)	67 (18.3%)	112 (36.5%)	142 (40.8%)	123 (47.7%)	4.1	<0.001	0.02
Beta-blocker	306 (23.9%)	45 (12.3%)	70 (22.8%)	103 (29.6%)	88 (34.1%)	4.1	<0.001	0.047
Spironolactone	64 (5.0%)	16 (4.4%)	13 (4.2%)	19 (5.5%)	16 (6.2%)	4.1	0.65	0.88
Diuretics	368 (28.8%)	40 (10.9%)	81 (26.4%)	127 (36.5%)	120 (46.5%)	4.1	<0.001	<0.001
Calcium channel blocker	258 (20.2%)	24 (6.6%)	67 (21.8%)	82 (23.6%)	85 (32.9%)	4.1	<0.001	<0.001
Statin	300 (23.5%)	58 (15.8%)	81 (26.4%)	87 (25.0%)	74 (28.7%)	4.1	<0.001	0.58
O2 flow, L/min	4.0 (2.0–9.0)	3.0 (2.0–9.0)	3.0 (2.0–9.0)	3.0 (2.0–9.0)	5.0 (3.0–9.0)	53.0	0.005	0.001
Physical examination, N (%)
Leg edema	334 (25.1%)	28 (7.4%)	65 (19.8%)	117 (32.6%)	124 (45.9%)	0	<0.001	<0.001
Jugular venous distension	43 (3.3%)	4 (1.1%)	8 (2.4%)	12 (3.4%)	19 (7.3%)	1.4	<0.001	0.04
Rales	454 (35.2%)	63 (17.3%)	104 (32.3%)	148 (42.8%)	139 (54.5%)	3.4	<0.001	<0.001
Systolic BP, mmHg	132.1 ± 26.0	129.6 ± 22.7	132.3 ± 26.0	132.9 ± 26.8	134.5 ± 28.9	0.1	0.13	0.98
Diastolic BP, mmHg	73.5 ± 17.6	77.2 ± 16.3	72.4 ± 17.8	71.6 ± 18.1	72.4 ± 18.0	0.1	<0.001	0.40
Heart rate, bpm	95.7 ± 20.7	98.1 ± 19.0	94.7 ± 20.2	94.2 ± 20.1	95.5 ± 23.8	0.4	0.02	0.32
Respiratory rate, /min	26.3 ± 7.9	24.5 ± 7.6	26.5 ± 7.5	26.8 ± 8.0	27.5 ± 8.4	31.3	<0.001	0.19
CSI	1.4 ± 0.8	0.5 ± 0.3	1.2 ± 0.2	1.8 ± 0.2	2.5 ± 0.3	0	<0.001	<0.001
Laboratory findings
Hemoglobin, g/dl	12.8 ± 2.0	13.5 ± 1.9	12.8 ± 2.0	12.6 ± 2.0	12.2 ± 2.0	0.3	<0.001	<0.001
White blood cell count, μ/l	11,300(8,300–15,400)	10,765(8,420–14,100)	10,900(7,700–15,100)	11,575(7,800–15,600)	12,600(8,900–16,700)	2.0	0.006	0.007
C-reactive protein, mg/dl	6.6(1.8–14.0)	3.9(0.8–10.5)	7.7(2.3–14.8)	7.7(2.3–14.5)	7.1(2.1–15.4)	7.7	<0.001	<0.001
Sodium, mmol/l	136.9 ± 5.3	137.1 ± 4.9	136.3 ± 5.4	137.0 ± 5.7	137.2 ± 5.0	2.7	0.17	0.003
Potassium, mmol/l	4.1 ± 0.6	4.0 ± 0.5	4.1 ± 0.6	4.2 ± 0.6	4.3 ± 0.7	5.6	<0.001	<0.001
Blood glucose, mmol/l	7.7 ± 3.5	6.8 ± 2.6	7.4 ± 3.0	7.8 ± 3.4	9.3 ± 4.5	3.0	<0.001	<0.001
BUN, mg/dl	25.3 ± 18.0	19.3 ± 15.5	23.5 ± 15.9	27.7 ± 17.5	32.6 ± 20.8	2.9	<0.001	<0.001
eGFR, ml/min/1.73m2	82.6 ± 55.4	96.2 ± 38.4	85.0 ± 42.2	79.8 ± 83.1	65.4 ± 34.4	3.5	<0.001	0.03
BNP, pg/ml	274(133–590)	65(37–183)	155(83–262)	280(154–592)	480(286–837)	62.6	<0.001	<0.001
Blood gas
pH	7.41(7.34–7.45)	7.43(7.39–7.46)	7.42(7.37–7.46)	7.41(7.34–7.45)	7.36(7.28–7.42)	18.8	<0.001	<0.001
PaO2, mmHg	65.0(56.0–79.0)	64.0(58.0–78.0)	64.5(56.0–79.5)	64.0(53.0–79.0)	66.0(57.0–79.0)	19.0	0.68	0.57
PaCO2, mmHg	40.4(35.0–48.0)	39.0(34.0–44.0)	41.0(35.0–47.0)	41.0(35.2–51.0)	42.0(36.0–52.0)	18.8	<0.001	<0.001
Lactate, mmol/L	1.10(0.80–1.60)	1.00(0.70–1.50)	1.00(0.80–1.40)	1.10(0.80–1.70)	1.10(0.80–1.90)	19.5	0.008	<0.001

Values are mean ± SD, N (%), or median (25th and 75th percentiles).

*p-value adjusted for age and BMI at admission.

ACEi, angiotensin converting enzyme inhibitor; ARB, angiotensin receptor blocker; BMI, body mass index; BP, blood pressure; BPM, beats per minute; BUN, blood urea nitrogen; BNP, b-type natriuretic peptide; CSI, congestion score index; eGFR, estimated glomerular filtration rate; HF, heart failure; PaCO2, partial pressure of carbon dioxide; PaO2, partial pressure of oxygen; SD, standard deviation.

Association of congestion score index with adjudicated discharge diagnosis of acute heart failure

In this study, 289 (21.7%) patients were diagnosed with AHF at discharge. Higher CSI was significantly associated with AHF diagnosis (odds ratio [OR] [95% CI] for a 0.1 unit increase in CSI = 1.22 [1.19 to 1.25], p < 0.001) even after adjustment for potential clinical confounders (adjusted OR [95% CI] in CSI = 1.18 [1.15 to 1.22], p < 0.001) and the Brest score (adjusted OR [95% CI] in CSI = 1.19 [1.16 to 1.22], p < 0.001). The association of CSI with AHF diagnosis using restricted cubic spline regression analysis is shown in Fig 2. CSI had a linear association with AHF diagnosis (p > 0.05 for nonlinearity), and higher CSI showed an increased risk of AHF diagnosis (OR [95% CI] for CSI score 1.0 = 4.09 [2.50 to 6.71], p < 0.001; OR [95% CI] for CSI score 1.5 = 13.92 [6.32 to 30.65], p < 0.001; OR [95% CI] for CSI score 2.0 = 37.03 [16.09 to 85.26], p < 0.001—considering CSI score 0.5 as a reference). Similar results were observed after adjustment for the Brest score (adjusted OR [95% CI] for CSI score 1.0 = 3.26 [1.93 to 5.48], p < 0.001; adjusted OR [95% CI] for CSI score 1.5 = 9.07 [4.00 to 20.59], p < 0.001; adjusted OR [95% CI] for CSI score 2.0 = 20.88 [8.83 to 49.35], p < 0.001—considering CSI score 0.5 as a reference) (Fig 2).

Fig 2

Association between CSI and discharge diagnosis of AHF.

Multivariable model included the Brest score, which was calculated from age, comorbidity burden, dyspnea, ST segment abnormalities, and clinical congestion. Dotted lines/shaded regions represent 95% CI. AHF, acute heart failure; CSI, congestion score index.

Diagnostic value of the congestion score index

In the whole population, CSI exhibited high discrimination for AHF as reflected by an area under the curve (AUC) of 0.84 (0.82 to 0.86) (Figs 3 and 4). Similarly, high AUC was observed across subgroups of age, sex, and comorbidities (obesity and COPD) or associated diagnosis (pneumonia and pleural effusion) (Fig 3). In contrast, subgroups without cardiomegaly assessed by CXR had higher AUC compared to those with cardiomegaly (AUC [95% CI] = 0.85 [0.81 to 0.89] versus 0.75 [0.70 to 0.79], respectively). In addition, the diagnostic value of CSI was influenced by the patient’s position (AUC [95% CI] = 0.83 [0.80 to 0.89] in sitting position and 0.79 [0.73 to 0.84] in supine position) as well as the difficulty in assessing CSI (AUC [95% CI] = 0.92 [0.86 to 0.97] for easy, 0.84 [0.80 to 0.88] for moderate, and 0.80 [0.75 to 0.84] for difficult assessments).

Fig 3

Diagnostic value of radiographic CSI for AHF.

*Pneumonia was diagnosed at discharge. AHF, acute heart failure; AUC, area under the curve; COPD, chronic obstructive pulmonary disease; CSI, congestion score index.

Fig 4

Diagnostic performance of the radiographic CSI.

On the receiving operating characteristic curve for the association between CSI and AHF diagnosis, the red shaded region represents the 90% or greater specificity zone (CSI ≤ 1.3), whereas the green shaded region represents the 90% or greater sensitivity zone (CSI > 2.2). On the top portion of the figure, CXR panels illustrate typical examples of radiographies in the 3 zones, with CSI score of “1,” “2,” and “3,” respectively, from left to right. AHF, acute heart failure; AUC, area under the curve; BNP, b-type natriuretic peptide; CSI, congestion score index; CXR, chest radiography; NRI, net reclassification improvement.

AUC for the Brest score was 0.78 [0.75 to 0.81]. The combination of CSI and Brest score yielded a high AUC for AHF (AUC [95% CI] = 0.87 [0.85 to 0.90]).

Improvement in reclassification associated with acute heart failure diagnosis

The addition of CSI on top of the Brest score significantly improved reclassification of AHF diagnosis (NRI [95% CI] = 94.9 [83.5 to 106.2], p < 0.001) (Fig 4).

Furthermore, in patients with available BNP data (N = 496), CSI still significantly improved reclassification of AHF diagnosis on top of BNP and the Brest score (NRI [95% CI] = 55.0 [38.0 to 72.0], p < 0.001, delta AUC [95% CI] = 2.9 [0.6 to 5.2], p = 0.015) (Figs 4 and 5). The diagnostic value of the joint use of the Brest score and CSI was not significantly different than that of the joint use of the Brest score and BNP (NRI [95% CI] = 4.4 [−13.3 to 22.1], p = 0.63, delta AUC [95% CI] = −1.4 [−5.0 to 2.1], p = 0.42). In this subgroup, the Brest score had a moderate accuracy (AUC [95% CI] = 0.72 [0.75 to 0.81]), but the combination of CSI, BNP, and Brest score resulted in high diagnostic value for AHF (AUC [95% CI] = 0.85 [0.82 to 0.89]) (Fig 5).

Fig 5

Added values of CSI and BNP for the diagnosis of AHF on top of the Brest score in patients with available BNP (N = 496).

AHF, acute heart failure; AUC, area under the curve; BNP, b-type natriuretic peptide; CSI, congestion score index; NRI, net reclassification improvement.

Validation in external cohorts

In the validation cohorts (N = 224), more than a half of the patients (56.7%) were men, mean age was 75.8 ± 13.9 years, mean CSI was 1.85 ± 0.87, and 72.7% had a diagnosis of AHF at discharge. The diagnostic performance of CSI was externally validated (AUC [95% CI] = 0.75 [0.68 to 0.82]).

Discussion

Our results show that pulmonary congestion quantified by a simple standardized CXR scoring can efficiently identify patients with AHF in the ED, consistently across the various subgroups (e.g., elderly, overweight, COPD, and pneumonia). Furthermore, CSI significantly improved the reclassification of AHF diagnosis on top of the recognized clinical diagnostic markers of AHF and natriuretic peptides. Our findings suggest that a semiquantified assessment of congestion on CXR could represent a readily available and clinically useful diagnostic tool for AHF in acute dyspneic patients in the ED.

Radiographic congestion score index as a diagnostic tool for acute heart failure

CXR exhibited a higher diagnostic performance for AHF than usually reported in the literature [14-16]. Of note, previous reports assessed the diagnostic value of CXR using a single or combination of typical radiographic signs of congestion using a global evaluation of the lungs [15,16,31-33] rather than a systematic approach, with lung segmentation as used in the CSI method. Our group recently showed that more severe pulmonary congestion, quantified by either CSI or lung ultrasound at admission, was associated with higher pulmonary artery systolic pressure [19,34]. This association of CSI with hemodynamic data emphasizes the mechanistic plausibility of our results.

Recent registry data have shown that approximately 20% of patients hospitalized for AHF had concomitant lung diseases such as pneumonia and COPD [35-37], and these patients generally excluded in previous reports assessing the diagnostic value of CXR [16,17,38,39]. However, our subgroup analysis provided a remarkably homogenous diagnostic performance of CSI across various subgroups (i.e., elderly and BMI) and coexisting lung diseases (i.e., COPD and pneumonia). The only factor appearing to decrease the diagnostic value of CSI was cardiomegaly, possibly as a result of the overlapping of relevant information of the lung fields with cardiac silhouette in these patients.

Radiographic congestion score index and other diagnostic measurements: Clinical parameters and natriuretic peptide

In keeping with previous literature data [19,40], our results showed that patients with more severe pulmonary congestion were predominantly elderly, had higher BMI, more cardiovascular risk factors and comorbidities, severe congestion, poorer renal function, and higher inflammatory markers. In the latest guidelines, natriuretic peptide is recommended to rule out non-HF–related causes of acute dyspnea, although accumulated data suggested the overall diagnostic accuracy of natriuretic peptide [41-45]. However, multiple comorbidity burdens (i.e., older age and poor renal function) may lead to diagnostic uncertainty in a sizeable proportion of dyspneic patients [7]. BNP, in addition, requires time to measure and is not always available in routine clinical practice. In cases with unequivocal diagnosis of AHF based on clinical parameters, prompt treatment approach is recommended rather than wait for its result [6]. In this regard, the Brest score, based on clinical parameters, may be a pragmatic tool as well as good diagnostic accuracy for diagnosing HF as previously shown [27]. However, in the current study, more than half of dyspneic patients have intermediate scores, suggesting that this clinical score may need to be complemented by more refined strategies in this relatively frequent “grey zone.” The unmet need in clinical practice, even when using the Brest score and natriuretic peptides, thus remains high.

In the current study, CSI was found to improve reclassification of AHF diagnosis on top of the Brest score and BNP. In addition, the combination of CSI and the Brest score improved the diagnostic value (AUC 0.81) to a similar degree of the combination of BNP and the Brest score (AUC 0.82). CSI also significantly improved diagnostic accuracy on top of the Brest score and BNP, and the combined used of these 3 parameters resulted in an AUC of 0.85. Taken together, these findings further strengthen that CSI may play a complementary role to the clinical model and natriuretic peptides in diagnosing AHF.

Clinical implications

An early accurate diagnosis and consequently a prompt appropriate management improve outcomes in AHF patients admitted to the ED [5,46,47]. Our results show that a standardized evaluation of CXR, using CSI, improves diagnosis performance and potentially the ability to swiftly manage AHF in the ED. The assessment of radiographic pulmonary congestion requires training period. However, this approach may be easily scalable since training for this assessment is fairly simple; the operator (AD) who evaluated all CXRs of this cohort efficiently acquired the technique in the context of this study in a matter of a few hours.

Recent studies showed the clinical utility of lung ultrasound to diagnose AHF in dyspneic patients with high specificity and high sensitivity [9,26,48], and its diagnostic accuracy was better than that of CXR [8,50]. Of note, these previous studies did not include quantitative assessment of radiographic pulmonary congestion. In any event, based on the promising results herein, multicenter trials may be warranted to assess the impact of the implementation of CSI on AHF diagnosis in patients with acute dyspnea. Furthermore, further study to compare diagnostic value between CSI on CXR and lung ultrasound is a worthy undertaking.

Limitations and strengths

The main limitation of our derivation cohort was its single-center and retrospective nature, although the external validation of our results may lead to generalize our findings. The overall proportion of AHF diagnosis was relatively low, which may explain the fact that a low number of patients had congestion signs (i.e., leg edema, rales, and jugular venous distention) and cardiovascular diseases. In our center (as in most centers in France), some dyspneic patients known to have HF and all patients with obvious evidence of myocardial ischemia were admitted directly in the intensive cardiac care unit/cardiology ward, not through the ED [49]. The proportion of AHF diagnosis possibly due to the healthcare system may influence our results. However, it should be noted that hospitalization rates for worsening HF in the ED declined over the past decades [50], which may result from the development of a disease management program to prevent urgent HF hospitalization [51]. Indeed, the proportion of AHF diagnosis in the present study was similar to that of dyspneic patients in other contemporary cohorts [44,52]. Ejection fraction was not recorded; thus, we did not evaluate the diagnostic accuracy of CSI across levels of ejection fraction. This parameter, however, is usually not a major determinant of decision-making for acute dyspneic management in the ED [6,12].

In the derivation cohort, we had no data on CSI in 138 (9.4%) patients who did not undergo CXR or had no available lung field to assess CSI. These patients, however, had better clinical status (i.e., younger age and less severe congestion), and only 8% of these patients (N = 11) were diagnosed with AHF (S1 Table). In addition, all CSI readings were performed blinded for other parameters and diagnoses, suggesting that this limitation is unlikely to have major influence on our findings.

CSI is a semiquantitative tool with some subjectivity. Although accurate and reproducible scoring was achieved after about 3-hour training period in the current study and 1 of our previous study [19], more evidence may be needed to ascertain the appropriate learning period.

Lastly, the assessment of CSI was difficult in 502 (37.7%) patients, which may limit its applicability in routine clinical practice. However, the diagnostic value of CSI persisted in these patients (AUC = 0.80, 0.75 to 0.84) and the difficulty in assessing CSI did not influence its diagnostic accuracy (Pinteraction = 0.13).

Conclusions

Our study shows that a semiquantified assessment of radiographic pulmonary congestion provided diagnostic value for AHF in dyspneic patients of similar magnitude to that of BNP. These results suggest that implementing radiographic CSI in the diagnostic approach to AHF in addition to clinical parameters and BNP measurement could benefit the management of AHF patients. Better use of CXR may provide an inexpensive, widely, and readily available method for AHF triage in the ED. Multicenter prospective studies are nonetheless needed to confirm the diagnostic value of radiographic CSI.

TRIPOD checklist.

(DOCX)

Click here for additional data file.

Flowchart.

(DOCX)

Click here for additional data file.

Baseline characteristics of patients with available and unavailable chest radiograph.

(DOCX)

Click here for additional data file.

Patient characteristics across different discharge diagnoses.

(DOCX)

Click here for additional data file.

1 Jun 2020

Dear Dr Kobayashi,

Thank you for submitting your manuscript entitled "Diagnostic Performance of Congestion Score Index Evaluated from Chest Radiography for Acute Heart Failure in the Emergency Department" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff [as well as by an academic editor with relevant expertise] and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Caitlin Moyer, Ph.D.,

Associate Editor

PLOS Medicine

15 Jul 2020

Dear Dr. Kobayashi,

Thank you very much for submitting your manuscript "Diagnostic Performance of Congestion Score Index Evaluated from Chest Radiography for Acute Heart Failure in the Emergency Department" (PMEDICINE-D-20-02033R1) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Aug 05 2020 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Emma Veitch, PhD

PLOS Medicine

On behalf of Clare Stone, PhD, Acting Chief Editor,

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

*Please revise your title according to PLOS Medicine's style - ideally this should have the study design in a subtitle (after the main study question is summarised) after a colon - eg, "A randomized controlled trial," "A retrospective study," "A modelling study," etc.

*We'd ask the authors restructure the abstract using the PLOS Medicine headings (Background, Methods and Findings, Conclusions) - "methods and findings" should be a single subsection.

*In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

*At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

*Ideally, please reformat the in-text citations to use sequential numerals in square brackets (eg [1], [2] etc) rather than superscript numerals - if using referencing software this should be relatively straight forward.

*Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis-- including those made in response to peer review comments-- should be identified as such in the Methods section of the paper, with rationale.

*The authors could consider using a reporting guideline designed to help enhance reporting of diagnostic studies, eg either STARD (https://www.equator-network.org/reporting-guidelines/stard/) or TRIPOD (https://www.equator-network.org/reporting-guidelines/tripod-statement/) depending on which is more appropriate to this study. If using one of the guidelines please complete and upload the appropriate checklist as supporting information alongside the revised paper (and consider citing the paper for the guideline in the methods section, noting that the tool was used to guide reporting of your study).

-----------------------------------------------------------

Comments from the reviewers:

Reviewer #1: Authors reported and concluded that semi-quantified assessment of pulmonary congestion on chest radiography provided excellent diagnostic value for acute heart failure in dyspneic patients in the emergency settings. Furthermore, this assessment may be available method for acute heart failure triage in the emergency department. This finding is interesting, however, I have the following concerns.

1. Vascular congestion is the most obvious therapeutic target when approaching patients with acute decompensated heart failure. From the view point of evaluation methods to identify congestion, cardiac imaging including chest X-ray is highly needed. However, initial screening to diagnose congestion or acute heart failure is mainly performed by physical examinations in clinical practice; naturiuretic peptides is not routinely measured.

2. In this study, the signs and symptoms, including jugular venous distention, rales, and leg edema, are relatively low even in patients among Quartile IV. Although the authors indicated the accuracy of radiographic congestion index for acute heart failure, it may be unreasonable to conclude the utility of this index for acute heart failure triage in the emergency department.

3. Generally, the presence of pneumonia may obscure the appearance or degree of congestion in patients with decompensated heart failure. In this paper, the mean value of C-reactive protein was 6.6 mg/dL in global population, and no description of patients with clinically apparent pneumonia in the excluding criteria. And serum procalcitonin level could be useful to handle such patients, affecting key results.

4. They also concluded that the combination of congestion score index improved reclassification on top of the Brest score and BNP. However, the relationship between this index and prognosis should be presented.

5. In Methods, precise diagnosis of study population is unclear. Furthermore, selection bias may be present because the admission criteria in this study population is undecided.

6. In Methods, subjects receiving heart failure therapy and diagnosed "heart failure" at discharge during hospitalization due to other diseases were excluded?

7. Reference 1 (published in 2006) may not be reflected current practice.

8. The author selected various factors as model 1 in logistic regression analysis. These factors were previously shown as prognostic factors?

9. The authors described " an automated assessment of pulmonary congestion based on CSI could be developed・・・" in the discussion section. I may be a gap in their argument.

10. I wonder that a short time training of radiographic congestion score index is unconvincing.

11. The authors described "the large-sized population sample, the relatively homogenous management and adjudication of diagnosis" in the discussion section, there is no basis for these statements.

-----------------------------------------------------------

Reviewer #2: I confine my remarks to statistical aspects of this paper. The basic approach is fine but I do have a few issues to resolve before I can recommend publication.

Abstract - What numbers come after the +- sign? (it seems like SD). Also, specify that the numbers in the parameter estimate are 95% CI

p. 6 Don't use quartiles. Use the raw score and, if desired, evaluate nonlinarity with a spline. Categorizing a continuous variable is almost always a mistake. Frank Harrell in *Regression Modelling Stragies* says "nothing could be more disastrous".

Was collinearity evaluated?

p. 7 Last para - this doesn't look gradual! Also, see comment above.

Peter Flom

-----------------------------------------------------------

Reviewer #3: In this retrospective, single-center study, the authors evaluated the diagnostic value of Congestion Score Index (CSI) using a semi-quantitative method on chest X-rays. The authors conclude that the CSI is capable of improving heart failure prediction models in those patients who come to the emergency department with the diagnosis of acute dyspnea and that are based on the combination of clinical data (Brest scale) and natriuretic peptides (BNP).

The usefulness of chest radiography to evaluate patients with acute dyspnea in the emergency department is evident, however, the latest guidelines for heart failure recommend caution in their interpretation, since up to 20% of heart failure patients will not present radiographic alterations. (Eur J Heart Fail. 2016 Aug;18(8):891-975. doi: 10.1002/ejhf.592.). In addition, lung ultrasound has demonstrated its superiority over chest radiography for the evaluation of acute dyspnea, showing a better sensitivity and specificity values. (Crit Care Resusc . 2016 Jun;18(2):124., JAMA Netw Open. 2019 Mar; 2(3): e190703.).

Despite being a retrospective study, it has a good sample size and the statistical methodology is well developed. Furthermore, the authors validate their results in another cohort, which supposes an added value. However, despite being well executed methodologically, I think the following points should be clarified before being accepted:

Comment 1: It is striking that only 10% of patients had a history of admission for heart failure. Furthermore, the percentage of cardiovascular diseases was relatively low despite the fact that the patients had an average age of 72 years. Do the authors believe that this fact could positively influence the results? In fact, in the group of patients with older age and in those who were more symptomatic (higher Brest scale), the AUC decreased the most.

Comment 2: If it is possible, it would be interesting to perform a sub-analysis of the ejection fraction (HFrEF vs. HFpEF). Patients with HFrEF usually debut with increased congestion and it would be interesting to see how CSI behaves in these subgroups.

Comment 3: As the authors well emphasize in the limitations section, up to 38% of the X-rays were difficult to interpret, I think it is an important fact, what was the degree of variability in the interpretation of the CSI in this group? If it is significant it should be taken into account.

Comment 4: In the results and discussion section, it should be clear that the use of natriuretic peptides is useful because of their high negative predictive value. The degree of congestion (in this case measured by the CSI) should not be compared with the BNP concentration figures, BNP is not a good biomarker of congestion, this fact could explain the lower AUC.

Comment 5: Given that the authors acknowledge having experience in the use of lung ultrasound, it would be advisable to compare the results of this study, based on chest radiography, with other studies where lung ultrasound has been tested to identify patients with heart failure upon arrival at the emergency room. I think that such a comparison would make the discussion more attractive.

Comment 6: Figure 3 tries to include the AUCs of the different models, but in my point of view it is difficult to interpret. Probably a simpler table with the different variables included in each model and its AUC would be easier to interpret.

-----------------------------------------------------------

Any attachments provided with reviews can be seen via the following link:

[LINK]

3 Aug 2020

Submitted filename: CSI_PARADISE_responses_to_reviewers.docx

Click here for additional data file.

21 Sep 2020

Dear Dr. Kobayashi,

Thank you very much for re-submitting your manuscript "Diagnostic Performance of Congestion Score Index Evaluated from Chest Radiography for Acute Heart Failure in the Emergency Department: An analysis from the PARADISE retrospective cohort" (PMEDICINE-D-20-02033R2) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by two reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

Our publications team (plosmedicine@plos.org) will be in touch shortly about the production requirements for your paper, and the link and deadline for resubmission. DO NOT RESUBMIT BEFORE YOU'VE RECEIVED THE PRODUCTION REQUIREMENTS.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Sep 28 2020 11:59PM.

Sincerely,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

1. Response to reviewer 1: Point 3: Please do include the table “Patient Characteristics across Different Discharge Diagnoses” in the manuscript, if preferred this could be a supporting information file.

2.Data Availability Statement: “Under french law, deidentified data cannot be transferred to someone not authorized by the CNIL (comité national informatique et liberté) to perform the analysis. To comply with this national data reglementation, we will provide access to the data on a secured server held by our institution upon reasonable request to the primary investigator of the study. Importantly, Nancy CIC-P works with a number of international groups using this secured server.” At this time, please update your statement to include a web link or contact email address for data access. Please note that the contact cannot be one of the authors of the study. Please also capitalize the F in French.

3. Title: Thank you for revising your title. We suggest a minor change: “Diagnostic performance of congestion score index evaluated from chest radiography for acute heart failure in the emergency department: A retrospective analysis from the PARADISE cohort”

4.Abstract: Methods and Findings: Please provide the years during which the study took place and the setting/population of the individuals in the PARADISE cohort.

5. Abstract: Methods and Findings: Please include the p value for the following result: “CSI was significantly associated with acute HF diagnosis (adjusted-odds ratio for 0.1-unit CSI increase 1.19, 95%CI 1.16 to 1.22).”

6.Abstract: Methods and Findings: For the adjusted analyses presented, please include the important variables that are adjusted for in the analyses.

7.Abstract: Methods and Findings: For the following results, please remove subjective descriptions such as “excellent” and “good” and replace with quantitative terms. “The diagnostic accuracy of CSI for acute HF was excellent, whether alone [area under ROC curve (AUROC) 0.84, 95%CI 0.82 to 0.86] or in addition to clinical model (AUCROC 0.87, 95%CI 0.85 to 0.90). CSI improved diagnostic accuracy on top of clinical variables [Net reclassification improvement (NRI)=94.9%] and clinical variables plus BNP (NRI=55.0%). Good diagnostic accuracy was observed in the validation cohorts (AUROC 0.75, 95%CI 0.68 to 0.82).”

8.Abstract: Conclusions: Please replace the word “provides” with “provided” in the first sentence. Please remove the word “excellent” and replace with a more quantitative term. Please address the study implications without overreaching what can be concluded from the data; the phrase "In this study, we observed ..." may be useful.

9.Author Summary: The author summary should immediately follow the Abstract in your revised manuscript- although you have included it in the “response to reviewer/editor comments” section, it is missing from the manuscript.

10. Author Summary: What did the researchers do and find?: Please remove the word “strongly” as it is redundant: “This Congestion Score Index was significantly and strongly associated with acute heart failure diagnosis.”

11. Author Summary: What did the researchers do and find?: Please clarify to: “The Congestion Score Index also improved diagnostic accuracy over clinical parameters with or without inclusion of natriuretic peptide.” or similar.

12.Methods: Study Population: In the first sentence, please clarify what is meant by “consecutive patients”

13.Methods: Study Population: Please remove the trademark symbol from “Resurgences”

14.Methods: Study Population: Please specify the nature of participant consent, including whether informed consent was written or oral, or whether the requirement for participant consent was waived (and by whom).

15.Methods: Ethical approval: Please specify the nature of ethical approval obtained for your study (i.e. the outcomes reported here, including the validation cohort) rather than for the registered trial/ cohort.

16.Methods: Page 5: Thank you for your clarification that your study did not have a formal prospective analysis plan written. Please make sure that all pre-planned analyses are explicitly described as such, and note any changes in the analysis-- including those made in response to peer review comments-- in the Methods section of the paper, with rationale.

17.Methods: Page 6: Please note if the values in parentheses represent the confidence intervals, or other value: “Intra-class correlation coefficients showed good reproducibility [0.85 (0.71–0.93) and 0.81 (0.64–0.90) for intra and inter-observer reproducibility, respectively].”

18.Results: Page 8: For the following analysis, please present the p values to accompany the OR and 95% CIs: “...higher CSI showed an increased risk of AHF diagnosis [OR for CSI score 1.0=4.09 (2.50 to 6.71) , OR for CSI score 1.5=14.30 (6.50 to 31.82) , OR for CSI score 2.0 =37.03 (16.09 to 85.26) – considering CSI score 0.5 as reference]”

19.Results: Page 8: Please also present the ORs adjusted for Brest Score for the relationship between CSI and AHF diagnosis with CSI scores of 1.0, 1.5 and 2.0, as these aren’t easy to discern from the multivariable panel from Figure 2.

20.Results: Page 9: Please rename “Central illustration” as a numbered figure of the paper (for example, “Figure 5”)

21.Results: Page 10: Please revise this sentence to reflect the quantitative findings, rather than saying “the diagnostic performance of CSI was good” which is subjective.

22.Discussion: Page 13: Please change “provides” to “provided” and revise the Conclusion to avoid vague statements such as “excellent” diagnostic value- we suggest: “Our study suggests that a semi-quantified assessment of radiographic pulmonary congestion provided diagnostic value for AHF in dyspneic patients, similar magnitude to that of BNP. These results suggest that implementing radiographic CSI in the diagnostic approach to AHF in addition to clinical parameters and BNP measurement could benefit acute heart failure patients.” or similar. Please also add a sentence here that mentions that implementing a radiographic CSI in the diagnosis of AHF will require further evaluation, such as in a prospective study, before being applied in clinical practice.

23.Conflict of Interest: Please remove this section from the main text of the manuscript and instead provide this information in the relevant section of the manuscript submission form.

24.Acknowledgements: The information in this section is more appropriate for the “Funding” section of the manuscript submission form.

25.Figure 1: Please indicate in the legend what is indicated by a score of “0” and please augment the description in the legend to make it clear that these are examples illustrating the scoring system.

26.Figure 2: Please change “Odd ratio” to “Odds ratio” on the X axis (please indicate if there are missing decimal places). Please include in the legend all factors adjusted for in the mulitvariable model, and indicate (on the x axis) that these are the adjusted odds ratios. Please indicate that the dotted lines/shaded regions represent the 95% CIs.

27.Central Illustration: Please rename/number this figure. Please include a descriptive legend, describing what is shown in the radiograph panels, and defining the lines, shaded regions within the graph. Also, please indicate the nature of the values presented in parentheses alongside the AUC and NRI values in the figure.

28.Supplementary Table 1: Please include a legend and define abbreviations for AHF, ACEi/ARB, BP, BUN, eGFR, BNP, PH, PaCO2/PaO2

29.Checklist: Thank you for including the TRIPOD checklist in your response to reviewer/editor comments. However, the checklist does not seem to be included as a supporting information file.

Please revise and submit the checklist, using section and paragraph numbers, rather than page numbers, to refer to locations in the text.

Please add the following statement, or similar, to the Methods: "This study is reported as per the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline (S1 Checklist)."

Comments from Reviewers:

Reviewer #1: The manuscript has been improved by the revision.

I have no additional comments.

Thank you for giving a chance of re-reviewing the paper.

Reviewer #2: The authors have addressed my concerns and I now recommend publication

Peter Flom

Any attachments provided with reviews can be seen via the following link:

[LINK]

14 Oct 2020

Dear Dr. Kobayashi,

On behalf of my colleagues and the academic editor, Dr. Mitsutoshi Oguri, I am delighted to inform you that your manuscript entitled "Diagnostic performance of congestion score index evaluated from chest radiography for acute heart failure in the emergency department: A restrospective analysis from the PARADISE cohort" (PMEDICINE-D-20-02033R3) has been accepted for publication in PLOS Medicine.

PRODUCTION PROCESS

Before publication you will see the copyedited word document (within 5 busines days) and a PDF proof shortly after that. The copyeditor will be in touch shortly before sending you the copyedited Word document. We will make some revisions at copyediting stage to conform to our general style, and for clarification. When you receive this version you should check and revise it very carefully, including figures, tables, references, and supporting information, because corrections at the next stage (proofs) will be strictly limited to (1) errors in author names or affiliations, (2) errors of scientific fact that would cause misunderstandings to readers, and (3) printer's (introduced) errors. Please return the copyedited file within 2 business days in order to ensure timely delivery of the PDF proof.

If you are likely to be away when either this document or the proof is sent, please ensure we have contact information of a second person, as we will need you to respond quickly at each point. Given the disruptions resulting from the ongoing COVID-19 pandemic, there may be delays in the production process. We apologise in advance for any inconvenience caused and will do our best to minimize impact as far as possible.

PRESS

A selection of our articles each week are press released by the journal. You will be contacted nearer the time if we are press releasing your article in order to approve the content and check the contact information for journalists is correct. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact.

PROFILE INFORMATION

Now that your manuscript has been accepted, please log into EM and update your profile. Go to https://www.editorialmanager.com/pmedicine, log in, and click on the "Update My Information" link at the top of the page. Please update your user information to ensure an efficient production and billing process.

Thank you again for submitting the manuscript to PLOS Medicine. We look forward to publishing it.

Best wishes,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

plosmedicine.org