Diagnostic performance of chest computed tomography during the epidemic wave of COVID-19 varied as a function of time since the beginning of the confinement in France.

OBJECTIVE: To evaluate the diagnostic performance of the initial chest CT to diagnose COVID-19 related pneumonia in a French population of patients with respiratory symptoms according to the time from the onset of country-wide confinement to better understand what could be the role of the chest CT in the different phases of the epidemic. MATERIAL AND
METHOD: Initial chest CT of 1064 patients with respiratory symptoms suspect of COVID-19 referred between March 18th, and May 12th 2020, were read according to a standardized procedure. The results of chest CTs were compared to the results of the RT-PCR.
RESULTS: 546 (51%) patients were found to be positive for SARS-CoV2 at RT-PCR. The highest rate of positive RT-PCR was during the second week of confinement reaching 71.9%. After six weeks of confinement, the positive RT-PCR rate dropped significantly to 10.5% (p<0.001) and even 2.2% during the two last weeks. Overall, CT revealed patterns suggestive of COVID-19 in 603 patients (57%), whereas an alternative diagnosis was found in 246 patients (23%). CT was considered normal in 215 patients (20%) and inconclusive in 1 patient. The overall sensitivity of CT was 88%, specificity 76%, PPV 79%, and NPV 85%. At week-2, the same figures were 89%, 69%, 88% and 71% respectively and 60%, 84%, 30% and 95% respectively at week-6. At the end of confinement when the rate of positive PCR became extremely low the sensitivity, specificity, PPV and NPV of CT were 50%, 82%, 6% and 99% respectively.
CONCLUSION: At the peak of the epidemic, chest CT had sufficiently high sensitivity and PPV to serve as a first-line positive diagnostic tool but at the end of the epidemic wave CT is more useful to exclude COVID-19 pneumonia.

Introduction

On March 12th, 2020, the World Health Organization (WHO) declared COVID-19 to be a pandemic. After a rapid spread in Asian countries starting in December 2019, Europe was affected from the end of February 2020. The first French patient was officially diagnosed on February 19th, 2020 and very quickly the epidemic wave hit France with a peak during the months of March and April 2020. Strict nationwide confinement was ordered on March 17th, at this time the basic reproduction rate of the virus (R0) was 2.8 Simultaneously, dedicated COVID-19 patient pathways were organized throughout the healthcare system as well as specific imaging procedures and strategies in our institution, a prime referral center for COVID-19 patients in Paris, using chest computed tomography (CT) as a first line imaging modality for the management of symptomatic patients suspect of COVID-19. Ai et al. [1] reported early in February in a series of 1014 Chinese patients that chest CT had a 97% (CI95%: 95–98%) sensitivity and a 25% (CI95%: 22–30%) specificity for detecting COVID-19, whereas at that time real-time reverse transcriptase-polymerase chain reaction (RT-PCR), considered as the reference standard to diagnose SARS-CoV-2 infection had an imperfect sensitivity around 60–70% reported to be variable according to sample site [2]. Moreover, initial RT-PCR required long processing times and needed to be repeated over time. These figures were mostly confirmed in numerous other small Asian studies with sensitivities for CT ranging from 86 to 99% [3, 4] and specificities from 25 to 53% [5]. Chest CT was therefore recommended during the early phase of the epidemic to perform rapid triage for referral of patients to appropriate COVID-19 sectors to avoid emergency department overcrowding and initiate optimal therapy. On May 11th, the French government decided to begin a deconfinement process because of a dramatic decrease in viral spread with R0 estimation around 0.8. The role of chest CT in the deconfinement strategy remained in question. Indeed, a diagnostic test sensitivity and specificity are commonly believed not to vary with disease prevalence [6, 7] whereas positive predictive values are directly dependent on the prevalence of the disease and therefore cannot be directly transposed across different phases of the epidemic wave or across a country. Hence, the primary objective of our study was to evaluate the diagnostic performance of the initial chest CT to diagnose COVID-19 related pneumonia in a French population of consecutive patients with respiratory symptoms according to the time from the onset of confinement to take into account the variability of COVID-19 prevalence as a function of time and to better understand what could be the role of CT scan in different phases of the epidemic and potential future disease outbursts.

Material and method

This retrospective study was approved by our institutional review board, the “Comité d'éthique de la Recherche-Sorbonne Université” and recorded as NCT04320017. According to the French law concerning retrospective studies of medical records, the patient were informed by post-mail that anonymized data from their medical records would be reviewed within the framework of this retrospective study and that, in case of non-opposition their data would be included in the study. Fourteen patients expressed their opposition and were consequently not included, whereas the 1064 patients who did not express opposition were included. Data from their medical records was anonymized and then studied in this research.

Patients

The study population consisted of consecutive adult patients referred to a large tertiary hospital between March 18th and May 12th, 2020 for initial chest CT and nasopharyngeal RT-PCR test. Indications were: suspicion of COVID-19 with one or more of the following respiratory signs: dyspnea, polypnea, oxygen dependence. Patients suspected of having COVID-19 without respiratory signs were not admitted to the ER and did not have chest CT.

CT exams

Detailed acquisition parameters are available as S1 File. In summary, all scans were performed on a dedicated scanner (Siemens Somaton Edge) either with or without iodine contrast injection according to the pretest risk of pulmonary embolism.

Image analysis

A total of ten senior radiologists, aware of COVID-19 suspicion but unaware of RT-PCR results, read the CT exams in real time (prospectively). They used a standardized report which included typical, atypical and very atypical findings [8-10] listed in Table 1. Imaging findings used were very similar to the one described by Prokop et al. [11] as described in Covid-Rads which was published after the completion of this study. Pulmonary embolism was considered separately. Pleural effusion was considered neither in favor nor against the COVID-19 diagnosis. The conclusion of the report was therefore one of the following: 1) imaging patterns suggesting the presence of COVID-19; 2) imaging patterns suggesting an alternative diagnosis; 3) imaging patterns suggesting a combination of COVID-19 with underlying lung disease; 4) CT considered normal.

Table 1

Imaging findings used in the standardized reports.

Typical findings of Covid 19	Atypical findings	In favor of another diagnosis
Multifocal ground glass opacities (GGO)	Lymphadenopathies	Tree in bud micronodules, bronchiolitis
Peripheral and basal distribution of GGO	systematized condensation with aerial bronchogram	Micronodules with lymphatic or random distribution
Unsharp demarcation		Mass and nodules
Crazy paving		Mosaic perfusion
Association of ground glass and consolidations		Cavitations
Consolidations, rather linear and rather located at the periphery of the lung		Central and peribronchovascular distribution of GGO
		Calcifications

RT-PCR

Nasopharyngeal swab samples were also performed to allow detection of SARS-CoV-2 RNA (S2 File). In the case of a first negative RT-PCR test, repeat tests were performed. For patients with multiple RT-PCR tests, the diagnosis of COVID-19 was confirmed when at least one test was positive within 15 days of CT.

Epidemiological data

The exact prevalence of the disease in the Paris area was not known at the time of admission, thus we collected, on a weekly basis, the absolute number of COVID-19 patients admitted into hospitals in the Paris region (Ile de France) and in the city of Paris obtained from government statistics (https://www.gouvernement.fr/info-coronavirus/carte-et-donnees) between March 18th and May 12th. We also collected the estimation of R0 values every weeks for the same period of time (from https://www.qualitiso.com/coronavirus-analyse-des-risques/).

Statistical analysis

For more detail see S3 File. Using RT-PCR results as reference, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and overall accuracy of chest CT imaging were calculated. The performance measures of chest CT for identifying COVID-19 as a function of time elapsed from the onset of confinement was also calculated. The observation period of this study lasted 8 weeks and was split into 8 weekly periods from the first day to the last day of confinement. Percentage of positive PCR over the 8 weeks, as well as the accuracy of chest CT over the 8 weeks were compared using the Chi-square test.

Results

Between March 18, and May 12, 2020, 1787 CT scans were performed. As summarized in Fig 1, 1064 patients, 583 males (55%) and 481 females (45%), with a mean age of 64 years (range 18–100) and a mean weight (when known, n = 465) of 75kg (range 32–183) met the inclusion criteria (Fig 1). The time elapsed since the onset of respiratory symptoms averaged 48 hours (range 2–168 h).

Fig 1

Study flow chart.

The first RT-PCR was positive in 516 (48%) of cases. Considering subsequent RT-PCRs, 546 (51%) patients were ultimately found to be positive for SARS-CoV2. Fig 2 shows the absolute number of COVID-19 patients admitted in the Paris Metropolitan Area and Region hospitals and the absolute number of positive RT-PCR in our institution as a function of weeks elapsed since the beginning of the confinement Values of R0 estimated at the beginning of each week and at the end of W8 are also reported on Fig 2.

Fig 2

R0, number of COVID 19 patients admitted in the hospital and number of positive RT-PCR in our institution.

Number of COVID-19 patients admitted in the hospital in the whole area around Paris (Ile de France: IDF) and in the city of Paris (Paris) (left axis) and number of positive RT-PCR in our institution (PSL) (right axis) and value of R0 as a function of weeks elapsed since the beginning of the confinement W1 = 18–24 of March, W2 = 25–31 of March, W3 = 01–07 of April, W4 = 08–14 of April, W5 = 15–21 of April, W6 = 22–28 of April, W7 = 29 of April-5 of May, W8 = 06–12 of May.

Table 2 shows the concurrent number and percentage of positive CT and RT-PCR for COVID-19 as a function of weeks. The highest positive RT-PCR rate was recorded in our institution during the second week of confinement (25–31 of March) with 71.9%. After six weeks of confinement, the positive RT-PCR rate dropped significantly to 10.5% (p<0.001) and then 4.7% at week 7 and 0% at week 8. CT revealed imaging patterns suggesting the diagnosis of COVID-19, in 603 patients (57%), either alone (520/603, 86%) or in combination with another lung disease (83/603, 14%) (Fig 3). The most common CT patterns for COVID-19 pneumonia seen in this study were ground glass opacities, crazy paving and consolidation mostly distributed in subpleural regions and involving all lobes (Fig 3). Imaging patterns in favor of another diagnosis was found in 246 patients (23%) (Fig 3) while CT was considered as normal in 215 patients (20%). One CT was unconclusive.

Table 2

Contingency table of the results of CT and RT-PCR for COVID-19 as a function of weeks from country-wide confinement.

			RT-PCR
			Negative	Positive	Total
Chest CT	W1	Negative	35 (21)	10 (6)	45 (28)
		Positive	17 (10)	101 (62)	118 (72)
		Total	52 (32)	111 (68)	163 (100)
	W2	negative	49 (19)	20 (8)	69 (27)
		positive	22 (9)	162 (64)	184 (73)
		total	71 (28)	182 (72)	253 (100)
	W3	negative	60 (28)	17 (8)	77 (36)
		positive	28 (13)	109 (51)	137 (64)
		total	88 (41)	126 (59)	214 (100)
	W4	negative	55 (39)	8 (6)	63 (45)
		positive	22 (16)	55 (39)	77 (55)
		total	77 (55)	63 (45)	140 (100)
	W5	negative	84 (55)	9 (6)	93 (61)
		positive	12 (8)	48 (31)	60 (39)
		total	96 (63)	57 (37)	153 (100)
	W6	negative	36 (75)	2 (4)	38 (79)
		positive	7 (15)	3 (6)	10 (21)
		total	43 (90)	5 (10)	48 (100)
	W7	negative	34 (77)	1 (2)	35 (80)
		positive	8 (18)	1 (2)	9 (20)
		total	42 (95)	2 (5)	44 (100)
	W8	negative	40 (83)	0 (0)	40 (83)
		positive	8 (17)	0 (0)	8 (17)
		total	48 (100)	0 (0)	48 (100)
	total 8 weeks	negative	393 (37)	67 (6)	460 (43)
		positive	124 (12)	479 (45)	603 (57)
		total	517(49)	546 (51)	1063 (100)

Note: The table reports the number of patients n and percentage (%). W1 = 18–24 of March, W2 = 25–31 of March, W3 = 01–07 of April, W4 = 08–14 of April, W5 = 15–21 of April, W6 = 22–28 of April, W7 = 29 of April-5 of May, W8 = 06–12 of May

Fig 3

Lung CT patterns found in patients with and without COVID-19 pneumonia.

A) Unenhanced chest CT images of a 62-year-old man with fever and dyspnea revealing one of the typical CT patterns for COVID-19 pneumonia. Axial images show ground glass opacities, crazy paving pattern (arrows) and consolidation with air bronchogramm (arrowhead) mostly distributed in subpleural regions and involving all lobes. RT-PCR here was positive (suggesting true positive diagnosis of CT). B) Unenhanced chest CT images of a 56-year-old man with COVID-19 pneumonia and sarcoidosis. Axial CT images revealed bilateral ground glass opacities (arrows) and perilymphatic irregular nodular thickening in an upper/mid lung distribution (arrowheads). RT PCR was positive (suggesting true positive diagnosis of CT). C) Enhanced chest CT images of a 37-year-old woman with dyspnea and fever revealing Pneumocystis Jiroveci infection. Axial CT images shows central diffuse GGO bilateral falsely considered suggesting of COVID-19. RT-PCR was here negative (suggesting false positive diagnosis of CT). D) Unenhanced chest CT images of a 67-year-old woman with dyspnea for few days revealing a hypersensitivity pneumonitis. Axial CT images shows homogeneous GGO bilateral and symmetric with a bronchovascular distribution. CT was considered suggesting another diagnosis than COVID-19. RT-PCR was here negative (suggesting true negative diagnosis of CT).

Concerning the 30 cases with initially negative RT-PCR subsequently positive, 18 initial CT scans were positive for COVID-19 (60%). Using at least one positive RT-PCR as a reference standard, the overall sensitivity of CT was 88%, overall specificity 76%, PPV 79%, and NPV 85%. Table 3 shows the accuracies of CTs obtained during the 8 weeks of the study. During the second week of confinement, the sensitivity, specificity, PPV and NPV of CT were 89%, 69%, 88% and 71% respectively. After six weeks of confinement, the same figures were 60%, 84%, 30% and 95% respectively. At the end of confinement, when the rate of positive PCR became extremely low, the sensitivity, specificity, PPV and NPV of CT were 50%, 82%, 6% and 99% respectively. Overall, accuracy remained stable over the 8 weeks (p = 0.42).

Table 3

Diagnostic performance of chest CT as a function of weeks elapsed since the beginning of the country-wide confinement.

	Sensitivity	Specificity	PPV	NPV
W1	91	67	86	78
W2	89	69	88	71
W3	87	68	80	78
W4	87	71	71	87
W5	84	88	80	90
W6	60	84	30	95
W7 + W8	50	82	6	99
Overall	88	76	79	85

Note: PPV, NPV: positive and negative predictive values. Reported numbers are percentages, %. W1 = 18–24 of March, W2 = 25–31 of March, W3 = 01–07 of April, W4 = 08–14 of April, W5 = 15–21 of April, W6 = 22–28 of April, W7 = 29 of April-5 of May, W8 = 06–12 of May. W7 and W8 were merged because of the very low number of positive RT-PCR during these two weeks.

Discussion

Our study found that, during the national spring confinement, the PPV of CT to diagnose COVID-19 dramatically dropped simultaneously with the decrease of R0 and of the number of hospitalized patients in the area which are both indirect markers of the local disease prevalence during confinement.”Six weeks after the onset of confinement PPV was only 30%. This value dropped even lower at 6% during the two last weeks of confinement. This low value is related to the rise of false positive results from 9% during the epidemic peak (week 2), to 15% in week 6 and 17% in week 8 of confinement. The CT reader’s expectation, which is influenced by the supposed prevalence of the disease, plays a major role [12, 13]. During the epidemic, radiologists tend to conclude that any ground glass opacity or crazy paving or peripheral linear zone of condensation suggests the presence of COVID-19. This tendency persists even during the declining phase of the epidemic but these imaging features are not specific of COVID-19 pneumonia but shared with other viral etiologies as well as with hypersensitivity pneumonia [14]. This increases the false positive rate, especially as the prevalence decreases rapidly. Consequently, although the initial CT is a valuable tool for managing patients at the peak of the epidemic (in our study the two first weeks of confinement when R0 was greater than 2.0) without waiting for RT-PCR results, this is no longer the case once the epidemic wave has been contained by confinement. Extensive use of CT as an initial tool to manage new patients suspected of COVID-19 at the beginning of the deconfinement period could result in a significant number of false positive results leading to poor patient management including mistakenly placing the patient in a COVID-19 pathway.

The rise in false positive results should also induce a decrease of the specificity of CT and should have no consequence on sensitivity and negative predictive values. However, in this study we found the opposite, sensitivity decreased while specificity increased as COVID-19 prevalence decreased over time. Leeflang et al. [13] reported that sensitivity and specificity can be either higher or lower with lower prevalence. In our study, the increase in specificity with time elapsed since the beginning of confinement, despite a higher number of false positive results, is necessarily explained by an even greater increase in true negatives (19% at week 2, 75% at week 6 and 83% at week 8 of confinement), which can be mechanically explained by the decrease in RT-PCR positive cases. The decrease of positive cases was noteworthy particularly after W5 and was a result of the combination of two phenomena: 1) Although Fig 2 shows a similar evolution profile between the number of hospitalizations in the region around Paris and in the city of Paris and the number of patients referred to our CT for Covid19 suspicion and respiratory symptoms, it appeared that this number fell more rapidly between W5 and W6 (-67%) than the number of new patients hospitalized, which only decreased by 47% in Paris and 36% in the region over the same period of time. This rapid decrease had no explanation but does not seem to correspond to a statistical variation because at W7 and W8 the number of patients referred to our CT for respiratory symptoms remained similar between 44% and 48% regardless of the trend towards a pursuit of the decrease of R0 and in the number of patients hospitalized for Covid19 in the region and the city of Paris. 2) at the same time, the proportion of positive PCR among patients referred to the scanner with respiratory symptoms went from 37% at W5 to 10% at W6 (-73%) and even 4.7% at W7 (-90%) and 0% at W8. This can be explained by a strong efficacy of the nationwide confinement combined with a return to the emergency department of patients with other respiratory pathologies (COPD, asthma, bacterial pneumonia,…), a return that became more pronounced as the epidemic was losing ground. These patients with other respiratory diseases were considered to be clinically suspect for COVID19 but confirmed to be true negatives of CT and likely caused a significant drop in the percentage of positive PCRs.

This also explains the increase of the NPV as a function of time during the confinement. Finally, the decrease of sensitivity observed when prevalence decreased is explained by both drop down of FN and TP results. The TP rate was 64% at week 2,6% at week 6 and around 1% the two last weeks of confinement. These results show that in deconfinement period CT scan is more useful to exclude COVID-19 than to diagnose it in patients with respiratory symptoms. Our study has certain limitations: RT-PCR on nasophynrageal swab samples, considered as the reference, had a significant number of false negative results which could be explained by several factors such as an inappropriate sampling technique, an inadequate transportation procedure or by insufficient viral load according to the natural history of the COVID-19 with a shift of the preferential viral replication site from the upper respiratory tract to the lower respiratory tract during the course of the infection [15]. Repeated testing could be necessary to avoid misdiagnosis and improve RT-PCR sensitivity; therefore, false positive results of CT were overestimated, leading to an underestimation of specificity; however, since we took into account at least one repeat RT-PCR positive result as the final confirmatory diagnosis, we probably minimized this bias. We did not re-analyze the chest CT scans and only considered the result of the report done at the time of the CT. We therefore merged the individual performances of ten different senior radiologists with unequal experience of lung disease. However, the resulting overall accuracy of the chest CT scan was more reflective of the reality of a night shift and, in addition, two chest radiologists (SB and VD) were always available to analyze difficult imaging results before final report validation. In addition, the duty distribution between each radiologist was even, in other words there was no reading bias by radiologists with higher sensitivity at the beginning of the wave and by less sensitive radiologists at the end of the wave. We did not consider in the conclusion of the chest CT report an “indeterminate appearance” as recommended by the RSNA expert consensus statement [10]. Therefore we may have misclassified some “inderterminate” exams probably leading to an excess of false negatives at the peak of epidemic and to an excess of false positives at the end of the study. However, we had made the choice to conclude in a binary manner in favor or not in favor of COVID-19 in order to help clinicians for early triage avoiding the “third” choice “indeterminate” that is usually often use when made available. Finally, we did not use CT as a screening tool for asymptomatic patients; our results apply consequently only to the population of patients with respiratory symptoms.

Conclusion

In patients with respiratory symptoms, diagnostic performance by CT for COVID-19 varied during the epidemic wave. At the peak of the epidemic, Chest CT had sufficiently high sensitivity and PPV to serve as a first-line diagnostic and triage tool without waiting for RT-PCR results. Conversely, at the end of the epidemic wave and beginning of deconfinement period, when the prevalence of COVID-19 becomes low, CT is more useful for excluding COVID-19 and making alternate diagnoses.

CT scanning.

(DOCX)

Click here for additional data file.

RT PCR.

(DOCX)

Click here for additional data file.

Statistical analysis.

(DOCX)

Click here for additional data file.

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

17 Sep 2020

PONE-D-20-23438

Diagnostic Performance of Chest Computed Tomography during the Epidemic Wave of COVID-19 Varies as a Function of Time since the Beginning of the Confinement in France

PLOS ONE

Dear Dr. Lucidarme,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 01 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Ming-Ching Lee

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In ethics statement in the manuscript and in the online submission form, please provide additional information about the patient records used in your retrospective study. Specifically, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Thank you for stating the following financial disclosure:

"The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

At this time, please address the following queries:

Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.

State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

If any authors received a salary from any of your funders, please state which authors and which funders.

If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 3 in your text; if accepted, production will need this reference to link the reader to the Table.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In the study the authors described the impact of disease prevalence on the diagnostic performance of chest CT during the pandemic COVID-19 period. The data was analyzed reasonably. I have one major question and two minors for the authors.

1.The case numbers decreased substantially from week 1 to week 6. However, the confirmed COVID-19 cases decreased more than 90% within one week (from 57 cases in week 5 to 5 cases in week 6), which was relatively a sudden drop as compared with data in the previous weeks. This may raise the concern of sampling bias in week 6 and affect the final results of your study. Would you compare your data to the decline rate of prevalence in your region during the same period?

2.In the context of Results, the sensitivity, specificity, PPV and NPV in week 2 and week 6 were not compatible with those in table 3. Please clarify which is correct.

3.Please review your reference and make corrections as errors were noted (at least ref. 5: wrong authors and wrong DOI, and ref. 10: wrong authors and no DOI).

Reviewer #2: This manuscript concerns an important topic to validate the role of chest CT scan during the COVID-19 endemic. The study design is reasonable to practice and the research outcomes are clear. I think it can provide useful information to clinicians.

Some questions as following

Question 1: What is the most common presentation of CT findings regarding the COVID-19 pneumonia in the current study?

Question 2: As your findings in the current study, the interpretation of CT will be influenced by the supposed prevalence of the disease. What time is the optimal time to use CT scan as the initial screening tool for symptomatic patients who are suspected to have COVID-19 infection? For example, R0 value? in the period of local transmission or community spread?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Shaw-Woei Leu

Reviewer #2: Yes: Dr. Pin-Kuei Fu. M.D., Ph.D

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

20 Oct 2020

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

E1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://clicktime.symantec.com/3YVDg2FUmJPdJafS3bt6WuN6H2?u=https%3A%2F%2Fjournals.plos.org%2Fplosone%2Fs%2Ffile%3Fid%3DwjVg%2FPLOSOne_formatting_sample_main_body.pdf and

https://clicktime.symantec.com/3V18fZ6tFtcSdAr3Y2XCj8A6H2?u=https%3A%2F%2Fjournals.plos.org%2Fplosone%2Fs%2Ffile%3Fid%3Dba62%2FPLOSOne_formatting_sample_title_authors_affiliations.pdf

Done. The reviewed manuscript fulfills PLOS ONE style templates.

E2. In ethics statement in the manuscript and in the online submission form, please provide additional information about the patient records used in your retrospective study. Specifically, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

According to the French law concerning retrospective studies of medical records (Jardé law), the patient must be informed by post-mail that anonymized data from their medical records will be reviewed within the framework of a given retrospective study. Patients are included only if they do not oppose the use of their anonymized medical records for research purposes. For this study, all patients (or their families) received a letter of information. 14 expressed their opposition and were therefore not included. 1064 patients who did not express opposition were thus included and anonymized prior to study in this research.

In order to answer the first comment of R1 we analyzed also the medical records of patients referred to CT for suspicion of COVID19 with respiratory symptoms at week 7 and 8. Hence we included in this revised manuscript an additional set of 92 patients to whom we also sent a letter of information mid-September. None of them expressed an opposition (Cf. above).

The Ethics Committee approval was referenced CER-Sorbonne Université 2020-014. We know provide in the revised manuscript the NCT number: NCT04320017.

E3. Thank you for stating the following financial disclosure:

"The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

At this time, please address the following queries:

a. Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.

There was no specific source of funding for this study, the authors are all paid employees of Academic Institutions the AP-HP “Assistance-Publique-Hôpitaux de Paris” and/or “Sorbonne University” and the material support (computer, software, database) was also provided by these 2 institutions for the routine clinical practice and routine research work.

b. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

c. If any authors received a salary from any of your funders, please state which authors and which funders.

d. If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

The authors received no specific funding for this work.

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

E4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

Repository information for our data is : Lucidarme, Olivier (2020), “COVIDCT-PSL2”, Mendeley Data, V1, doi: 10.17632/t2s38pzn66.1

E5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 3 in your text; if accepted, production will need this reference to link the reader to the Table.

To answer R1.1 we changed tables and figures and we renumbered accordingly in the revised manuscript.

Responses to Reviewers

Reviewer #1: In the study the authors described the impact of disease prevalence on the diagnostic performance of chest CT during the pandemic COVID-19 period. The data was analyzed reasonably. I have one major question and two minors for the authors.

R1.1. The case numbers decreased substantially from week 1 to week 6. However, the confirmed COVID-19 cases decreased more than 90% within one week (from 57 cases in week 5 to 5 cases in week 6), which was relatively a sudden drop as compared with data in the previous weeks. This may raise the concern of sampling bias in week 6 and affect the final results of your study. Would you compare your data to the decline rate of prevalence in your region during the same period?

Thank you for this important question which allowed us to deepen our reflection by introducing a comparative analysis with the evolution of the epidemic in both the Paris area, the Ile de France (IDF) Region and the city of Paris. In the revised manuscript, we removed Figures 2 and 4, which described the number of positive PCR in our institution and the accuracy of CTs which were redundant with table 3. We added a new figure 2, with a double-y axis figure that simultaneously displays the number of COVID 19 patients admitted into hospitals in the Paris region (IDF) and the city of Paris obtained from government statistics (https://www.gouvernement.fr/info-coronavirus/carte-et-donnees) because the exact prevalence of the disease was not known. In addition we added data from two additional weeks (Week 7 & week 8) in order to show that results in W6 were not pitfalls but was part of a general trend and we have commented on these results in the new discussion section as follows.

. The decrease of positive cases was noteworthy particularly after W5 and was a result of the combination of two phenomena: 1) Although Figure 2 shows a similar evolution profile between the number of hospitalizations in the region around Paris and in the city of Paris and the number of patients referred to our CT for Covid19 suspicion and respiratory symptoms, it appeared that this number fell more rapidly between W5 and W6 (-67%) than the number of new patients hospitalized, which only decreased by 47% in Paris and 36% in the region over the same period of time. This rapid decrease had no explanation but does not seem to correspond to a statistical variation because at W7 and W8 the number of patients referred to our CT for respiratory symptoms remained similar between 44% and 48% regardless of the trend towards a pursuit of the decrease in the number of patients hospitalized for Covid19 in the region and the city of Paris.

2) at the same time, the proportion of positive PCR among patients referred to the scanner with respiratory symptoms went from 37% at W5 to 10% at W6 (-73%) and even 5% at W7 (-90%) and 0% at W8. This can be explained by a strong efficacy of the nationwide confinement combined with a return to the emergency department of patients with other respiratory pathologies (COPD, asthma, bacterial pneumonia,...), a return that became more pronounced as the epidemic was losing ground. These patients with other respiratory diseases were considered to be clinically suspect for COVID-19 but were eventually confirmed to be true negatives of CT and likely caused a significant drop in the percentage of positive PCRs.

We modified accordingly the flow chart and the results displayed in tables 2 &3 to take into account results obtained at W7 and W8 and we added the following paragraph into the material and method section

“Epidemiological additional data

The exact prevalence of the disease in the Paris area was not known at the time of admission, thus we collected, on a weekly basis, the absolute number of COVID-19 patients admitted into hospitals in the Paris region (Ile de France) and in the city of Paris obtained from government statistics (https://www.gouvernement.fr/info-coronavirus/carte-et-donnees) between March 18th and May 12th. We also collected the estimation of R0 values every weeks for the same period of time (from https://www.qualitiso.com/coronavirus-analyse-des-risques/).

R1.2.In the context of Results, the sensitivity, specificity, PPV and NPV in week 2 and week 6 were not compatible with those in table 3. Please clarify which is correct.

Thanks for identifying this mistake. All calculations were redone and modifications made accordingly to the revised manuscript.

R1.3. Please review your reference and make corrections as errors were noted (at least ref. 5: wrong authors and wrong DOI, and ref. 10: wrong authors and no DOI).

Our apologies for these mistakes. We double checked the references.

Reviewer #2: This manuscript concerns an important topic to validate the role of chest CT scan during the COVID-19 endemic. The study design is reasonable to practice and the research outcomes are clear. I think it can provide useful information to clinicians.

Some questions as following

R2.1: What is the most common presentation of CT findings regarding the COVID-19 pneumonia in the current study?

To answer this question we added the following sentence in the result section : “The most common CT patterns for COVID-19 pneumonia seen in this study were ground glass opacities, crazy paving and consolidation mostly distributed in subpleural regions and involving all lobes (figure 3a).”

These patterns are illustrated in Figure 3.

Question 2: As your findings in the current study, the interpretation of CT will be influenced by the supposed prevalence of the disease. What time is the optimal time to use CT scan as the initial screening tool for symptomatic patients who are suspected to have COVID-19 infection? For example, R0 value? in the period of local transmission or community spread?

According to Table 3, the optimal time to use CT scans as initial screening tools for symptomatic patient was at the beginning of the confinement period (W1 = 18-24 of March or W2 = 25-31 of March). This was also probably the time of the highest prevalence however its value was not known due to a lack of RT PCR tests in the population. The local prevalence is the key regardless of local transmission or a community spread once again unpredictable. To answer this comment, we now provide additional available information about the number of patients (precisely known) admitted in hospital in the Paris-region (see also R1.1) and in the city of Paris as well as the estimation of R0 as a function of time in the figure 2 and added the following sentences in the discussion section:

“Our study found that the PPV of CT to diagnose COVID-19 dramatically dropped simultaneously with decrease during confinement of R0 and of the number of hospitalized patients in the area which are both indirect markers of the local disease prevalence during confinement.”

“CT is a valuable tool for managing patients at the peak of the epidemic (in our study the two first weeks of confinement when R0 was greater than 2.0) without waiting for RT-PCR results, this is no longer the case once the epidemic wave has been contained by confinement.”

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

We do agree

Submitted filename: Reviewer answer PlosOne.docx

Click here for additional data file.

11 Nov 2020

Diagnostic Performance of Chest Computed Tomography during the Epidemic Wave of COVID-19 Varied as a Function of Time since the Beginning of the Confinement in France

PONE-D-20-23438R1

Dear Dr. Lucidarme,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ming-Ching Lee

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Shaw-Woei Leu

13 Nov 2020

PONE-D-20-23438R1

Diagnostic performance of chest computed tomography during the epidemic wave of covid-19 varied as a function of time since the beginning of the confinement in france

Dear Dr. Lucidarme:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Ming-Ching Lee

Academic Editor

PLOS ONE