Case-control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands.

BACKGROUND: Coronavirus disease 2019 is a serious respiratory virus pandemic. Patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state will differ between individuals. The primary aim of this study was to investigate these variables in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic and to compare the "COVID-19 suspected" (positive and negative tested group) with the "COVID-19 not suspected" (control group) and to compare in the "COVID-19 suspected" group, the positive and negative tested patients.
METHODS: Consecutive adult patients, visiting the emergency room at the Franciscus Gasthuis & Vlietland, Rotterdam, the Netherlands, were asked to fill out questionnaires on the abovementioned items on an iPad. The patients were either "COVID-19 suspected" (positive and negative tested group) or "COVID-19 not suspected" (control group).
RESULTS: This study included a total of 159 patients, 33 (21%) tested positive, 85 (53%) negative and 41 (26%) were COVID-19 not suspected (control group). All patients in this study were generally aware of transmission risks and virulence and adhered to the non-pharmaceutical interventions. Working as a health care professional was correlated to a higher risk of SARS-Cov-2 infection (p- value 0.04). COVID-19 suspected patients had a significantly higher level of anxiety compared to COVID-19 not suspected patients (p-value < 0.001). The higher the anxiety, the more seriously hygiene measures were followed. The anxiety scores of the patients with (pulmonary) comorbidities were significantly higher than without comorbidities.
CONCLUSION: This is one of the first (large) study that investigates and compares patient characteristics, knowledge, behaviour, illness perception, and mental state with respect to COVID-19 of patients visiting the emergency room, subdivided as being suspected of having COVID-19 (positive or negative tested) and a control group not suspected of having COVID-19. All patients in this study were generally aware of transmission risks and virulence and adhered to the non-pharmaceutical interventions. COVID-19 suspected patients and patients with (pulmonary) comorbidities were significantly more anxious. However, there is no mass hysteria regarding COVID-19. The higher the degree of fear, the more carefully hygiene measures were observed. Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19.

Introduction

Coronavirus disease 2019 (COVID-19) is caused by the SARS-Cov-2 virus and constitutes the most serious respiratory virus pandemic since the 1918 H1N1 influenza pandemic. COVID-19 is characterized by fever and respiratory symptoms like cough, sneeze, and shortness of breath [1]. SARS-Cov-2 virus spreads by transmission of respiratory (small) droplets containing the virus particles from person to person, mostly in close contact [2]. Symptomatic patients spread the virus very fast. The SARS-Cov-2 virus can also be transmitted by asymptomatic persons and by contact with contaminated surfaces. Occasionally, the virus can be transmitted from humans to animals and vice versa [3]. Prevention of transmission is very important in the absence of an effective COVID-19 vaccine or treatment.

Non-pharmaceutical interventions (NPIs) to avoid transmission of SARS-Cov-2 involve hygiene measures, limitation of human contact, and social distancing. The Dutch National Institute for Public Health and Environment (RIVM) recommends washing hands thoroughly and frequently, coughing and sneezing in the inside of the elbow, avoiding shaking hands, keeping a distance of 1.5 meters (2 arms’ lengths) from others, and staying at home as much as possible.

Knowledge of contamination risks, severity of SARS-Cov-2 infection, the chance of recovery and long-term consequences of the disease will differ between individuals, including those individuals with and without a SARS-Cov-2 infection, and might influence the (risk) behaviour of people. Also, the mental state and comorbidities of people may influence (risk) behaviour and consequently the chance of SARS-Cov-2 contamination. Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19.

The present report concerns an observational study on patient characteristics and knowledge of the disease, risk behaviour, illness perception, and mental state (at the time of assessment) of patients visiting an emergency room in the Netherlands, subdivided as “COVID-19 suspected” (positive and negative tested group) or “COVID-19 not suspected” (control group).

These variables were measured for three groups: those tested positive for SARS-Cov-2, those tested negative, and the control group. The degree of anxiety in the total group has also been correlated to comorbidities and the hygiene measures, human contact, and social distancing.

Material and method

Study design and patient selection

This study was designed as an observational questionnaire study and registered in the Dutch Trial Register as PAtient’s knowledge and behaviour oN the COVID-19 disease and as DeterMInants of Contamination (PANDEMIC) study (trial number NL8563). The study was submitted to the medical ethical review board and was considered not subject to the Medical Research Involving Human Subjects Act (WMO; W20.075). We adhered to the methods and procedures of the Strengthening the Reporting of Observational Studies in Epidemiology (STOBE) guidelines for reporting this study. It was not possible to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of our research, because of the very fast procedure in the COVID-19 crisis.

Consecutive adult patients (>18 years of age) visiting the emergency room at the Franciscus Gasthuis & Vlietland, Rotterdam, the Netherlands, were asked to participate in this study (n> 200). The patients were “COVID-19 suspected” or “COVID-19 not suspected” (control group). All patients with upper- or lower respiratory symptoms were considered as possible COVID-19 positive. Before completing the survey, participants gave their written informed consent. Reasons for non-participation were patients with a language barrier or patients who could not operate an iPad or who had a severe medical condition (direct transfer to intensive care) or lack of iPads due to the large number of patients arriving at the emergency room at the same time.

A total of 164 patients were included in the period between April 10th and June 20th, 2020. Patients were asked to fill out their general patient characteristics and answer questions on their knowledge of the disease, risk behaviour, illness perception, and mental status on an iPad with disposable cover to respect hygiene measures. Of these 164 patients, 5 patients did not complete the questionnaires because of logistic–or personal reasons (Fig 1). General patient characteristics and information consisted of year of birth, sex, the number of people in their household, and profession type (if applicable). Vital professions were defined as those in sectors necessary to keeping society running during the COVID-19 crisis, for example: health care, policing, food, waste, and transport. Comorbidities were derived from the electronic patient files.

Fig 1

Flow chart of patient inclusion.

Patients’ knowledge of SARS-Cov-2 was investigated with a self-developed questionnaire on the severity of the disease, route of contamination, and the importance of government measures. Illness perception was measured with questions on the patient’s perception of COVID-19 with respect to their specific situation. For example: ‘If you are infected with SARS-Cov-2, do you think that you could die from this infection?’ (Attachment 1).

A second questionnaire concerned risk behaviour in terms of hygiene, human contact, and social distancing. The questionnaire was designed to study whether patients adhered to the measures imposed by the Dutch National Institute for Public Health and Environment. For example: Do you wash your hands frequently; do you avoid group gatherings; do you maintain a 1.5-meter distance between yourself and others? (Attachment 2).

The last 6-item questionnaire was the validated State-Trait Anxiety Inventory–short (STAI-s) survey [4] to measure the mental state of the COVID-19 suspected patients from the beginning of the COVID-19 crisis (1st of March 2020) to the time of administering the questionnaires.

The questionnaires were filled out before test results (COVID-19 negative/positive) were known. Three patients did not complete the second questionnaire and 5 patients did not complete the third questionnaire. These data were not part of the analysis.

Patients with a positive nasal Polymerase chain reaction (PCR) swab to SARS-Cov-2 or having SARS-Cov-2 antibodies were considered to be COVID-19 positive.

A comparison is made between the “COVID-19 suspected” (positive and negative tested group) with the “COVID-19 not suspected” (control group) and in the “COVID-19 suspected” group, the positive and negative tested patients.

Statistical analysis

Patient characteristics were reported in terms of mean, ranges, and proportions. Knowledge, risk behaviour, and illness perception of COVID-19 were reported descriptively. The differences in terms of these factors were compared between 3 groups (those tested positive for SARS-Cov-2, those tested negative, and the control group) using the Kruskal-Wallis test, and were described with p-values. P-values <0.05 were considered as significant.

Mental state was measured by the STAI-s score questionnaire. The sum of the 6 questions was calculated. This score was multiplied by 20/6 to be compared with the long version of the STAI. The One-Way ANOVA test was used to compare the mean scores on anxiety of the 3 groups. A cut off point of 39–40 has been suggested to detect clinically significant anxiety symptoms.

The STAI-s scores were correlated with the risk behaviour score for the areas of hygiene, human contact, and social distancing using linear logistic regression analysis. The risk behaviour score was the sum of the questions divided by the number of questions. The score on a 5-point Likert scale was: 0 points = no risk to 4 points = most risk and on a 2-point Likert scale: 1 point = no risk and 3 points = most risk). The outcomes of the regression analysis were reported with the regression standardized coefficient (ß), and the CI and p-values. Also, the mean STAI-s score was related to comorbidities with linear logistic regression analysis. Statistical analyses were done using SPSS statistics 24.

Results

Study population

This study included a total of 159 patients including 118 COVID-19 suspected patients and 41 COVID-19 not suspected patients (control group). The mean age of all the patients (inclusive control group) was 50.42 years (range 19–86 years), with 70 males (44%) and 89 females (56%) (Table 1). The patients were referred to the Franciscus Gasthuis and Vlietland, Rotterdam, by their general practitioner or taken to hospital directly by ambulance. Of the COVID-19 suspected patients in this study, 33 (21%) patients tested positive and 85 (53%) tested negative for COVID-19.

Table 1

Characteristics of the SARS-Cov-2 positive-tested, negative-tested, and control groups.

	Positive-tested (N = 33)		Negative-tested (N = 85)		Control group (N = 41)
Age (Mean, range)	50.42	(25–86)	51.53	(21–85)	48.07	(19–77)
Males (N, %)	13	(39)	34	(40)	23	(56)
Household (N, %)
1–2	15	(46)	50	(59)	16	(39)
3–4	14	(42)	25	(29)	19	(46)
≥4	4	(12)	10	(12)	6	(15)
Vital profession (N, %)
Yes	16	(48)	21	(25)	11	(27)
Institution (N, %)	N = 16		N = 21		N = 11
General practice	1	(6)	3	(14)	0	(0)
Nursing home /care institution	9	(56)	5	(24)	0	(0)
Hospital	2	(13)	3	(14)	1	(9)
Not in the medical care sector	4	(25)	10	(48)	10	(91)

The average household was 2.76 people (range 1–7) in the total group and was similar for the positive and the negative group. Patient characteristics are summarized in Table 1. Of the SARS-Cov-2 positive cases, 16 patients (48%) had a vital profession (and this differed significantly (p-value 0.04) compared to 23 (26%) in the negative tested group (Fig 2A). More than 60% of these patients worked in the medical sector, mostly in nursing homes or care institutions (56%) (Fig 2B, Table 1). Some of these patients worked with COVID-19 patients and a substantial part worked with colleagues also infected with Sars-Cov2.

Fig 2

Patients working in a vital profession (A) and in the medical sector (B) in the SARS-Cov-2 positive-tested, negative-tested, and control groups.

Significantly more patients infected with SARS-Cov-2 work in a vital profession compared to uninfected patients and the control group (p-value 0.04), and most of them work in the medical sector.

Knowledge/Illness perception

COVID-19 was seen as a severe disease: almost 90% of the total study population believe the disease is worse than influenza. Ninety-eight percent of patients knew that contaminations most frequently occur through person to person contact. Approximately 18% of the patients considered wearing face masks useful in the prevention of SARS-Cov-2 infection. Ninety-four percent of patients thought that adhering to NPIs helps to prevent infection. There was no difference between the positive-tested, negative-tested, and control group with respect to knowledge about COVID-19 (Table 2).

Table 2

Group comparison for non-pharmaceutical interventions (NPIs) and SARS-Cov-2 infection in the positive-tested, negative-tested, and control groups.

Behaviour during COVID-19 crisis	N (%)		N (%)		N (%)		P-value
	Negative-tested		Positive-tested		Control
Hygiene	85		33		41
Washing hands (times a day)							0.11
0	0	(0)	1	(3)	1	(2)
1–3	6	(7)	2	(6)	3	(7)
4–6	24	(28)	4	(12)	16	(39)
7–9	20	(24)	7	(21)	6	(15)
>10	35	(41)	19	(58)	15	(37)
Missing
Washing hands (20 seconds)							0.59
Never—sometimes	15	(18)	5	(15)	10	(24)
Almost always- always	70	(82)	28	(85)	31	(76)
Stick to coughing rules							0.14
No-probably	6	(2)	5	(15)	4	(10)
Probably yes- yes	79	(98)	28	(85)	37	(90)
Wearing face mask							0.42
Never—sometimes	75	(88)	29	(88)	37	(90)
Almost always- always	10	(12)	4	(12)	4	(10)
Keeping 1.5-meter distance							0.03
Never—sometimes	2	(4)	2	(6)	4	(10)
Almost always- always	82	(96)	31	(94)	37	(90)
Missing	1
Contacts
Staying at home							< 0.001
Never—sometimes	16	(19)	14	(42)	13	(73)
Almost always- always	67	(81)	19	(58)	28	(27)
Missing	2
Shop visits							0.21
No	63	(72)	26	(79)	24	(60)
Yes	25	(28)	7	(21)	16	(40)
Missing					1
Park/market/ beach visits							0.07
No	67	(81)	32	(97)	35	(88)
Yes	16	(19)	1	(3)	5	(12)
Missing	2				1
Family visit
Receive visitors >60 year of ag							0.99
No	75	(89)	30	(91)	36	(88)
Yes	8	(9)	3	(9)	4	(10)
Missing	2				1
Visit people > 60 year of age							0.94
No	70	(84)	27	(82)	33	(83)
Yes	13	(16)	6	(18)	7	(17)
Missing	2				1
Group gathering							0.30
No	66	(79)	26	(79)	27	(67)
Yes, max 2	14	(17)	3	(9)	9	(23)
Yes 3–5	3	(4)	3	(9)	3	(7)
Yes 6–10	0	(0)	1	(3)	0	(0)
>10	0	(0)	0	(0)	1	(3)
Missing	2				1

Group comparison for non-pharmaceutical interventions (NPIs) and SARS-Cov-2 infection in the positive-tested, negative-tested (COVID-19 suspected), and control group (COVID-19 not suspected) analyses by the Kruskal-Wallis test.

In this study population (COVID-19 suspected and control group), 32% believed they would not be infected by SARS-Cov-2, 37% did not expect to become seriously ill and 42% did not expect to die from the infection. Of all the patients, 20% did not consider the virus to be contagious. The illness perception scores did not differ between the 3 tested groups.

Risk behaviour

For all patients (COVID-19 suspected- and control group), 43% washed their hands more than 10 times a day and approximately 81% reported washing their hands always/almost always for more than 20 seconds. More than 90% of patients reported using the inside of their elbow to cough and sneeze. Only 11% of patients reported wearing a face mask for protection. Almost all patients indicated always/almost always keeping a 1.5-meter distance from others (95%), however the control group reported adhering less to this 1.5- meter measure (p-value 0.02). There was no significant difference between the positive-tested, negative-tested, and control group in terms of other abovementioned risk behaviour (Table 2).

In this total study population, 72% reported always/almost always staying home during the COVID-19 pandemic; however, the control group reported staying home significantly less (p- value < 0.001). Thirty percent of all patients reported visiting shops (beyond the basic necessities of life) and 14% reported visiting markets, parks or beaches. The positive-tested group reported visiting markets, parks or beaches less often than the negative-tested group and control group (p-value 0.07). Approximately 16% and 9% of the study population reported visiting people older than 60 years and/or receiving visits from people aged 60 year or older, respectively. Attending group gatherings (> 6 people) was reported by 1% of the total study population.

Mental state

The mean anxiety score was 48.56 points (range 23–67) in the negative-tested group and 50.10 points (range 27–77) in the positive-tested group (not significantly different). These anxiety scores of these COVID-19 suspected patients (mean 49.01, range 23.33–76.67) were significantly higher (p-value < 0.001) than the anxiety score of the control group (mean 39.00, range 20–70).

The anxiety scores of all patients were significantly inversely correlated to risk behaviour in the hygiene domain (ß 0.20, CI 0.93–7.62, p-value 0.01). The domains human contact and distancing were not significantly related to the anxiety scores in study group overall, with p-values of 0.87 and 0.89, respectively. The anxiety scores of the patients without comorbidities (mean 41.25, range 20–70) were significantly lower (ß 0.28, CI 3.47–11.79, p-value < 0.001) than the patients with comorbidities (48.88, range 23–77). Patients with a history of respiratory disease had significantly higher (ß 0.23, CI 1.83–10.42, p-value 0.05) anxiety scores (corrected for other comorbidities) than those without comorbidities.

Discussion

The primary aim of this study was to investigate patient characteristics, knowledge of contamination risks, the severity of the disease, illness perception, and mental state in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic. The “COVID-19 suspected” (positive and negative tested group) was compared with the “COVID-19 not suspected” (control group) and in the “COVID-19 suspected” group, the positive and negative tested patients were compared.

The mean STAI-s score, which represents anxiety, was significantly higher in the COVID-19 suspected patients compared to the general population requiring emergency care. No significant difference in the anxiety score was observed in the positive-tested and negative-tested patients. Moreover, COVID-19 suspected patients and patients with (pulmonary) comorbidities were significantly more anxious, and the higher the degree of fear, the more seriously the hygiene measures were followed.

Remarkably, more than 50% of the positive-tested patients worked in health care and most of them in the medical sector. The percentage of the SARS-Cov-2 positive-tested group who reported having visited public places was lower than in the other two groups. The control group reported adhering less to the 1.5- meter measure and stayed at home less.

Furthermore, the severity of SARS-Cov-2 infection, the risk of contamination and the importance of NPIs were well understood by all patients, and the measures were therefore properly observed in the positive-tested, negative-tested, and control group. The age of the patients in this study was quite young (mean age of 50.42 years).

Significantly higher anxiety was observed in the COVID-19 suspected patients than reported in the control group and the average research population [5]. This is in line with the large degree of fear that was previously reported by Liu et al., stating that patients with COVID-19 experienced high levels of anxiety (mean STAI score of 58) and low sleep quality [6]. The current study adds to these findings that this anxiety is already present in suspected patients and not only in diagnosed patients or in the total population. There was thus no mass hysteria regarding COVID-19. In line with our study, Motta Zanin et al. investigated the public Italian perception of health risk through the administration of a questionnaire in more and less COVID-19 effected regions. They demonstrated that the participants mainly expressed uncertainty, fear, and sadness in the more effected regions [7]. Mainly, patients with (pulmonary) comorbidities were significantly more anxious in our study, and the higher the fear, the better hygiene measures were followed.

The percentage of SARS-Cov-2 infected health care professionals in our study is higher than that reported by Heinzerling et al. [8]. In that study involving 121 hospital health care workers who were exposed to COVID-19 patients in a Hospital in California, 36% developed symptoms during 14 days after exposure and only 3 people tested positive. Another study demonstrated that 92% of the health care personnel in the United States had at least one symptom (fever, cough, or shortness of breath) after exposure to SARS-Cov-2-infected patients, however this was not proven to be COVID-19-related [9]. Barrett et al. demonstrated that the prevalence of SARS-CoV-2 infection was higher among health care workers (7.3%) than in non-health care workers (0.4%) [10].

The severity of SARS-Cov-2 infection, the risk of contamination and the importance of NPIs were well understood by all patients and the measures to prevent infection were (therefore) reported as being adhered to in this study. In line with our study, a cross-sectional survey in Hong Kong with 765 participants demonstrated that the overall knowledge and understanding of COVID-19 was good and most respondents agreed that NPIs could reduce the transmission of COVID-19 [11].

There was clear communication from the Dutch government about the importance of NPIs to prevent transmission of SARS-Cov-2. The study of Gesser-Edelsburg et al. underscores this importance of the public trust in adhering to NPIs to prevent transmission of SARS-Cov-2. They showed in an online survey in the Israeli public with 1056 participants that the higher the public trust and evaluation of crisis management was, the greater the compliance of the public with the government guidelines [12].

The percentage who reported using a face mask in this study was much lower compared to a study in China, which found 97% wore face masks [13]. At the beginning of the COVID-19 period, face masks were not compulsory and often even not available in nursing homes and care institutions in the Netherlands.

The lower average age in this study compared to the SARS-Cov-2-infected populations reported in other studies can be explained by the selection criteria for referring patients to the hospital in the Netherlands. General practitioners refer patients with relatively severe symptoms and patients for whom health improvements could reasonably be expected, per the Dutch referral policy. This policy excludes patients with mild symptoms and vulnerable elderly patients not eligible for treatment in the hospital. General practitioners and nursing home doctors take into account whether medical treatment is desirable and useful in respect to the quality of life of each individual patient. This approach is in contrast to a retrospective cohort study included 124 patients who required 911 Emergency Medical Services care for COVID-19 in Washington. This study revealed that most patients with COVID-19 presenting to emergency medical services were older and had multiple chronic health conditions [14].

This is one of the first (large) study that investigates and compares patient characteristics, knowledge, behaviour, illness perception, and mental state with respect to COVID-19 of patients visiting the emergency room, subdivided as being suspected of having COVID-19 (who then tested either positive or negative) and a control group not suspected of having COVID-19. The limitations of this study include the state of COVID-19 testing and measures taken by the population in the Netherlands, which may not be generalizable for other countries. Also, seriously ill patients who were respiratory insufficient, patients with a language barrier, and patients who were not able to handle an iPad were excluded from this study. Finally, the data on risk behaviours were self-reported, which implies that social desirability could have played a role in the low risk behaviours described in this sample.

Regulatory agencies urge adherence to NPIs to reduce SARS-Cov-2 virus contamination [15, 16]. In the group suspected of infection, risk behaviour was no greater in the group that tested positive than in the group that tested negative. High exposure to SARS-Cov-2 in the health care sector was probably the most important factor in being infected with COVID-19. This specific patient group should try to prevent infection by using protective material, including face masks. However, at the beginning of the COVID-19 crisis most of the available protective equipment was destined for hospitals and there was a shortage in nursing homes and care institutions. This may explain the high number of infections in nursing homes and care institutions.

However, the disproportionately large number of infections in health care workers may also suggest a super spreading pattern [17] as observed in church choirs, gyms, and animal slaughterhouses. Health care professionals often work and stay in poorly ventilated rooms, as do workers in nursing homes and care institutions. Furthermore, aerosol generating procedures (tracheal intubation and non-invasive ventilation) increase the risk of transmission of SARS-Cov-2 to health care workers [18]. The health care workers in this study mainly worked in health care institutions not involved in these procedures.

Conclusion

This is the first (large) study that investigates and compares patient characteristics, knowledge, behaviour, illness perception, and mental state with respect to COVID-19 of patients visiting the emergency room, subdivided as being suspected of having COVID-19 and a control group not suspected of having COVID-19. All patients in this study were generally aware of transmission risks and virulence and adhered to the NPIs. Most patients suspected of having COVID-19 who visited our emergency department had a high degree of anxiety compared to the general population. There is thus no mass hysteria regarding COVID-19, but anxiety was higher among the patients suspected of having COVID-19. Patients with (pulmonary) comorbidities were significantly more anxious and experienced a higher degree of fear, which was correlated with better behaviour as regards hygiene measures. Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19.

Evidence before this study

We searched PubMed using the keywords “COVID-19”, “2019-nCoV”, “SARS-CoV-2” and “emergency”, “anxiety”, “mental state”, “fear”, “patient characteristics”, “knowledge”, “risk behaviour”. We screened 90 articles (updated 21th of August 2020) and considered 4 studies relevant to our research goal. We also checked references to relevant articles (‘snowballing’).

(COVID-19[Title] OR 2019-nCoV[Title] OR SARS-CoV-2[Title]) AND emergency[Title] AND (anxiety OR mental state OR fear) AND (patient characteristics OR knowledge OR risk behaviour).

Added value of this study

We investigate patient characteristics, knowledge of contamination risks, the severity of the disease, illness perception, and mental state in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic. We found that the severity of SARS-Cov-2 infection, the risk of contamination and the importance of NPIs were well understood. Most patients suspected of having COVID-19 had a high degree of anxiety compared to the general population. Patients with (pulmonary) comorbidities were significantly more anxious which was correlated with better behaviour as regards hygiene measures. There is thus no mass hysteria regarding COVID-19, but anxiety was higher among the patients suspected of having COVID-19.

Implications of all the available evidence

Clear communication from the government about the importance of NPIs to prevent transmission of SARS-Cov-2 ensures the adherence to NPIs. High degree of anxiety is present in COVID- 19 suspected patients and in the patients with (pulmonary) comorbidities. It is important to pay attention to this psychological aspect of COVID-19 in this particular part of the population. Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19.

Patient and public involvement

No patient involved.

Questionnaire on knowledge and illness perception.

(DOCX)

Click here for additional data file.

Questionnaire on risk behaviour.

(DOCX)

Click here for additional data file.

(PDF)

Click here for additional data file.

(PDF)

Click here for additional data file.

(DOCX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

(PDF)

Click here for additional data file.

(JPG)

Click here for additional data file.

(JPG)

Click here for additional data file.

18 Jan 2021

PONE-D-20-28005

Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands

PLOS ONE

Dear Dr. Johanna van der Valk,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Yuka Kotozaki

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.  If the original language is written in non-Latin characters, for example Amharic, Chinese, or Korean, please use a file format that ensures these characters are visible.

3. Please state whether you validated the questionnaire prior to testing on study participants. Please provide details regarding the validation group within the methods section.

4. For more information on PLOS ONE's expectations for statistical reporting, please see https://journals.plos.org/plosone/s/submission-guidelines.#loc-statistical-reporting. Please update your Methods and Results sections accordingly.

5. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

6. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

7. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Important note: This review pertains only to ‘statistical aspects’ of the study and so ‘clinical aspects’ [like medical importance, relevance of the study, ‘clinical significance and implication(s)’ of the whole study, etc.] are to be evaluated [should be assessed] separately/independently. Further please note that any ‘statistical review’ is generally done under the assumption that (such) study specific methodological [as well as execution] issues are perfectly taken care of by the investigator(s). This review is not an exception to that and so does not cover clinical aspects {however, seldom comments are made only if those issues are intimately / scientifically related & intermingle with ‘statistical aspects’ of the study}. Agreed that ‘statistical methods’ are used as just tools here, however, they are vital part of methodology [and so should be given due importance].

COMMENTS: It is (being KAP type cross-sectional survey only) a fairly simple [and so straight forward] study as ‘The primary aim of this study was to investigate “Patient characteristics, knowledge of the COVID-19 disease, risk behaviour and illness perception in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic”. However, as said in (lines 63-64) ‘Abstract-Conclusion’ that “This is the first (large) study that investigates these’ is not true. But in any case, I request the authors to consider/note following points:

It may please be noted {kindly confirm from field experts} that patients having SARS-Cov-2 antibodies are not capable to spread the disease [unless IgM result shows status of infection]. If that is so, then how correct is to combine them? with ‘Patients with a positive nasal Polymerase chain reaction (PCR) swab to SARS-Cov-2’ which are real ‘COVID-19 positive’.

Though measures/tools used as “Indicators/Measures of knowledge, risk behaviour, and illness perception of COVID-19” (lines 155-56), are appropriate, most of them yield data that are in [most likely] ‘ordinal’ level of measurement [and not in ratio level of measurement for sure {as the score two times higher does not indicate presence of that parameter/phenomenon as double}]. Then application of suitable non-parametric test(s) is/are indicated/advisable [even if distribution may be ‘Gaussian’ (i.e. normal) in these (such) cases. Therefore, as said in lines 157-8 that ‘differences in terms of these factors were compared between 3 groups (those tested positive for SARS-Cov-2, those tested negative, and the control group) using the One-Way ANOVA test’ is not correct and it is indicated/advisable to use non-parametric ‘One-Way ANOVA’ namely Kruskal-Wallis test.

Please read the following [from famous text book]:

“Inferential statistics (i.e. hypothesis testing + estimation of CI) is built on the population model (i.e. the underlying assumption is that there is a population and we are dealing with random sample(s) drawn from that population). Although in clinical trial (involving at least two groups) we do not really deal with random samples, ‘allocation’ to treatment groups is ‘randomly’ done which enable us to evoke the population model and we can use inferential statistics safely. But when there is only one group or in studies even with two/more groups ‘random allocation’ is out-of- question [like internal grouping as in this case] and with ‘non-random’ selection, it may be questionable to use inferential statistics [even if you have two measurement sets as ‘pre-post’].

By this I do not advice “not to use inferential statistics here”, but just to keep this limitation in mind while interpreting results as there is no guarantee of representation of population {example, line 336: The lower average age in this study compared to the SARS-Cov-2-infected populations reported in other studies}.

Test used to analyse data displayed in Table 2 [Group comparison for non-pharmaceutical interventions (NPIs) and SARS-Cov-2 infection in the positive-tested, negative-tested, and control groups] is not mentioned anywhere and so the question is ‘how ANOVA is applied as most data are categorical’ but if Chi-square is used then ‘how zero frequency (rather all low frequencies are/) is dealt with’ [remember that this a scientific/academic document and so all details should be clearly communicated].

Implications of this study [in backdrop of ‘Added value of this study’ described in lines 396-40] are questionable, in my opinion {though it is true (line 71-72) that ‘Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19’}. Few things/findings are ‘very obvious’ [ex. Line 186-7: ‘Significantly more patients infected with SARS-Cov-2 work in a vital profession compared to uninfected patients and the control group (p-value 0.04), and most of them work in the medical sector’].

Reviewer #2: This is an ambitious case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms and controls in the Netherlands. I admire the intent and work to perform this work, but unfortunately the study population, questionnaire with responses, and analyses require clarification. The study population as it is described decreases the study generalizability. Measuring behaviour by self report in patients with COVID-19 symptoms compared to those without COVID-19 symptoms might be too confounding to say anything about behaviour. Use of the anxiety measure without reference or definition of the meaning of the scoring system decreases its utility. Use of a questionnaire without validation or precision is worrisome. (For example when asked about staying home is that staying home aside from work or inclusive, is it in general (which is what I thought initially) or with symptoms or post exposure (which is what the questionnaire seems to imply ? Analysis of the questionnaire results with binary outcomes when they were measured on a 5 point likert scale is puzzling and very much decreases belief or precision in the results. For example do the essential worker patients, such as those who work in a nursing home really keep a 1.5m distance if they are working with patients? Is going to a park for excersie or to walk the dog an exposure? But similarly the only intervention reported on a 5 point scale was handwashing which I would imagine is very hard to recall correctly.

1) Study population: Consecutive patients presenting at one ED in the Netherlands between April 10 and June 20. Because this is one ED it is important to understand the ED and patient population in order to see the extent to which this study is generalizable. 159 patients were enrolled in approximately 70 days. This is 2.5 patients a day which seems like a very low volume ED or it is a convenience enrollment. In either case, it is important to understand if there was a difference between those that enrolled and those that did not enroll in the study. It would be good to understand the chief complaints/diagnostic categories of the cases and controls as well as their disposition and illness severity. I want to know what are the parameters for testing “covid suspected.” Would it be symptomatic patients or exposed? If so, there are some constellations of symptoms that are more worrisome than others (fever alone, vs. respiratory symptoms and fever.) What was the prevalence in the netherlads at that time?

2) Comparison is really between covid suspected and controls then a subanalysis could be done with positive and negatives.

3) Since this study involves patient recall regarding behavior, it is important to use validated questions /outcome measurements regarding behavior. It is important to understand the Netherlands rules/public health injunctions at this time (Were masks required?) The authors report using a 5 point likert scale but the report their results on a 2 point scale which seems a little blunt

4) Recall bias possible regarding covid suspected and covid positive (were they feeling sicker)

5) Since this is recall, how long of a time period are they asking the patient to recall?

6) Likewise, visiting beaches, markets and parks seems vague. I do not think of beaches and parks as places with increased transmission (in the US we are allowed to go to parks and beaches and out of doors.)

7) If the authors are going to use an anxiety score they should cite it and then describe its use. I had to look it up on the internet. While the authors state that the comparison of anxiety measurements are significantly different, I do not know what this means. My source says that the STAI-s is used to make clinical diagnoses but I don’t think that is the case here. The source I saw on the internet stated that the ranges found here are both “moderate”

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: renamed_dec77.docx

Click here for additional data file.

26 Jan 2021

Dear Editor

Thank you very much for giving us the opportunity to revise our article titled: Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands”

We are very thankful for the commentary of the referees. We would like to respond point by point to the comments. For your convenience, we have written the answers in blue.

COMMENTS – Manuscript PONE-D-20-28005

Title: “Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands”

Reviewer #1.COMMENTS: It is (being KAP type cross-sectional survey only) a fairly simple [and so straight forward] study as ‘The primary aim of this study was to investigate “Patient characteristics, knowledge of the COVID-19 disease, risk behaviour and illness perception in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic”. However, as said in (lines 63-64) ‘Abstract-Conclusion’ that “This is the first (large) study that investigates these’ is not true.

Thank you for this comment, we agree with you. We changed the sentence to:

‘this is one of the first (large) study that investigates …’

But in any case, I request the authors to consider/note following points:

It may please be noted {kindly confirm from field experts} that patients having SARS-Cov-2 antibodies are not capable to spread the disease [unless IgM result shows status of infection]. If that is so, then how correct is to combine them? with ‘Patients with a positive nasal Polymerase chain reaction (PCR) swab to SARS-Cov-2’ which are real ‘COVID-19 positive’.

We agree with you. However, only 3 patients were included only on the SARS-Cov-2 antibodies. All other patients were tested with COVID-19 PCR. Because, this study was performed in the beginning of the pandemic, we assume that these 3 patients had a very recent infection. We think that taking these 3 patients in to account will not influence our study results.

Though measures/tools used as “Indicators/Measures of knowledge, risk behaviour, and illness perception of COVID-19” (lines 155-56), are appropriate, most of them yield data that are in [most likely] ‘ordinal’ level of measurement [and not in ratio level of measurement for sure {as the score two times higher does not indicate presence of that parameter/phenomenon as double}]. Then application of suitable non-parametric test(s) is/are indicated/advisable [even if distribution may be ‘Gaussian’ (i.e. normal) in these (such) cases. Therefore, as said in lines 157-8 that ‘differences in terms of these factors were compared between 3 groups (those tested positive for SARS-Cov-2, those tested negative, and the control group) using the One-Way ANOVA test’ is not correct and it is indicated/advisable to use non-parametric ‘One-Way ANOVA’ namely Kruskal-Wallis test.

Thank you for this good comment. We agree with you and did the analysis again with the non-parametric One-Way ANOVA test, namely Kruskal-Wallis test. We have adjusted table 2. Fortunately, the new results of this statistical method did not change our conclusions.

Please read the following [from famous text book]:

“Inferential statistics (i.e. hypothesis testing + estimation of CI) is built on the population model (i.e. the underlying assumption is that there is a population and we are dealing with random sample(s) drawn from that population). Although in clinical trial (involving at least two groups) we do not really deal with random samples, ‘allocation’ to treatment groups is ‘randomly’ done which enable us to evoke the population model and we can use inferential statistics safely. But when there is only one group or in studies even with two/more groups ‘random allocation’ is out-of- question [like internal grouping as in this case] and with ‘non-random’ selection, it may be questionable to use inferential statistics [even if you have two measurement sets as ‘pre-post’].

By this I do not advice “not to use inferential statistics here”, but just to keep this limitation in mind while interpreting results as there is no guarantee of representation of population {example, line 336: The lower average age in this study compared to the SARS-Cov-2-infected populations reported in other studies}.

Test used to analyse data displayed in Table 2 [Group comparison for non-pharmaceutical interventions (NPIs) and SARS-Cov-2 infection in the positive-tested, negative-tested, and control groups] is not mentioned anywhere and so the question is ‘how ANOVA is applied as most data are categorical’ but if Chi-square is used then ‘how zero frequency (rather all low frequencies are/) is dealt with’ [remember that this a scientific/academic document and so all details should be clearly communicated].

Thank you very much for the book suggestion. We have read the book chapter with interest. We used indeed the ANOVA test and as you mentioned, it is better to use the Kruskal-Wallis test. We made this improvement and correct this in the statistical methods. We have also adapted table 2 and noted the statistical method below the table.

Implications of this study [in backdrop of ‘Added value of this study’ described in lines 396-40] are questionable, in my opinion {though it is true (line 71-72) that ‘Knowledge about the coping of the population during the COVID-19 pandemic is very important, certainly also in the perspective of a possible second outbreak of COVID-19’}. Few things/findings are ‘very obvious’ [ex. Line 186-7: ‘Significantly more patients infected with SARS-Cov-2 work in a vital profession compared to uninfected patients and the control group (p-value 0.04), and most of them work in the medical sector’].

Maybe, these conclusions of our study are obvious. However, we think it is important to pay attention to this important sector and therefore, we mentioned this point in our manuscript.

Reviewer #2: This is an ambitious case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms and controls in the Netherlands. I admire the intent and work to perform this work, but unfortunately the study population, questionnaire with responses, and analyses require clarification. The study population as it is described decreases the study generalizability. Measuring behaviour by self report in patients with COVID-19 symptoms compared to those without COVID-19 symptoms might be too confounding to say anything about behaviour.

Thank you very much for this extensive comment. The questionnaires were completed before the patients get there COVID-19 PCR result, therefore, there is no confounding in the questionnaire outcomes.

Use of the anxiety measure without reference or definition of the meaning of the scoring system decreases its utility.

We agree with you and added the reference to the manuscript: Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol. 1992;31(3):301-6.

A cut off point of 39–40 has been suggested to detect clinically significant anxiety symptoms and we clarified this in the manuscript.

Use of a questionnaire without validation or precision is worrisome. (For example when asked about staying home is that staying home aside from work or inclusive, is it in general (which is what I thought initially) or with symptoms or post exposure (which is what the questionnaire seems to imply ? Analysis of the questionnaire results with binary outcomes when they were measured on a 5 point likert scale is puzzling and very much decreases belief or precision in the results. For example do the essential worker patients, such as those who work in a nursing home really keep a 1.5m distance if they are working with patients? Is going to a park for excersie or to walk the dog an exposure? But similarly the only intervention reported on a 5 point scale was handwashing which I would imagine is very hard to recall correctly.

That is a very good comment. At the start of the study we were looking for validated questionnaires about behaviour during infection outbreaks. Unfortunately, validated questionaries don’t exist on this topic. And it is true that these questionnaires have a recall bias. However, the time between the start of the pandemic and completing the interview is only 3.5 month.

Study population: Consecutive patients presenting at one ED in the Netherlands between April 10 and June 20. Because this is one ED it is important to understand the ED and patient population in order to see the extent to which this study is generalizable. 159 patients were enrolled in approximately 70 days. This is 2.5 patients a day which seems like a very low volume ED or it is a convenience enrollment. In either case, it is important to understand if there was a difference between those that enrolled and those that did not enroll in the study. It would be good to understand the chief complaints/diagnostic categories of the cases and controls as well as their disposition and illness severity. I want to know what are the parameters for testing “covid suspected.” Would it be symptomatic patients or exposed?

All patients with upper- or lower respiratory symptoms were considered as possible COVID-positive and were tested with PCR. We added the following sentence to the manuscript:

All patients with upper- or lower respiratory symptoms were considered as possible COVID-19 positive.

If so, there are some constellations of symptoms that are more worrisome than others (fever alone, vs. respiratory symptoms and fever.) What was the prevalence in the netherlads at that time?

The prevalence of COVID-19 suspected patients in our emergency room was among 7 patients a day. There is indeed a selection bias, because the seriously ill patients who were respiratory insufficient, patients with a language barrier, and patients who were not able to handle an iPad were excluded from this study. Unfortunately, that was inevitable. We did mention this in the manuscript.

2) Comparison is really between covid suspected and controls then a subanalysis could be done with positive and negatives.

We have chosen to compare the positive, negative and not suspected COVID-19 positive patients to get a more complete vision of the metal state and behaviour of all patients visiting the emergency room. However, it is possible to get the comparison between the positive and negative patients from the tables.

3) Since this study involves patient recall regarding behavior, it is important to use validated questions /outcome measurements regarding behavior. It is important to understand the Netherlands rules/public health injunctions at this time (Were masks required?) The authors report using a 5 point likert scale but the report their results on a 2 point scale which seems a little blunt

We choose to report not all details to make it easy for the reader. We think is a more organized and less extensive table. However, it is indeed also possible to give the total overview of all numbers,

4) Recall bias possible regarding covid suspected and covid positive (were they feeling sicker)

That is a good point. That maybe the case. Hoverer, we have not scored the degree of illness in this study.

5) Since this is recall, how long of a time period are they asking the patient to recall?

The recall was maximal 3.5 month.

6) Likewise, visiting beaches, markets and parks seems vague. I do not think of beaches and parks as places with increased transmission (in the US we are allowed to go to parks and beaches and out of doors.)

In the Netherlands it is very busy on the beaches and parks. Therefore, these places are considered as places with a high risk of infection.

7) If the authors are going to use an anxiety score they should cite it and then describe its use. I had to look it up on the internet. While the authors state that the comparison of anxiety measurements are significantly different, I do not know what this means. My source says that the STAI-s is used to make clinical diagnoses but I don’t think that is the case here. The source I saw on the internet stated that the ranges found here are both “moderate”

We agree with you. We added a reference to the manuscript.

The mean anxiety score was 48.56 points (range 23-67) in the negative-tested group and 50.10 points (range 27-77) in the positive-tested group. The the anxiety score of the control group (mean 39.00, range 20-70). As we mentioned in the manuscript a cut off point of 39–40 has been suggested to detect clinically significant anxiety symptoms. We have clarified this in the text.

We hope we have answered all questions and really hope you will accept our improved manuscript in Plos One. I would like to ask you to take into consideration that it will be very difficult for us to get our article published in another journal after a 4.5 month waiting period for review at Plos one.

Looking forward to hear from you.

Kind regards,

Hanna van der Valk

Submitted filename: Letter_reviewer.docx

Click here for additional data file.

3 Mar 2021

PONE-D-20-28005R1

Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands

PLOS ONE

Dear Dr. Johanna van der Valk,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Apr 17 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Yuka Kotozaki

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: COMMENTS: Since most of the comments made on earlier draft by me (and hopefully by other respected reviewers also) are attended positively/adequately, I am fully satisfied and the manuscript is improved a lot.

While answering my comment, it is said that “Fortunately, the new results of this statistical method did not change our conclusions” is very good but remember that it does not indicate that any will do. Methodology used should be appropriate [always use which is indicated / most desired].

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Sanjeev Sarmukaddam

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

4 Mar 2021

Dear Editor

We would like to respond to the comment of the referee.

1. "While answering my comment, it is said that “Fortunately, the new results of this statistical method did not change our conclusions” is very good but remember that it does not indicate that any will do. Methodology used should be appropriate [always use which is indicated / most desired."

I fully agree with you. I am very pleased that you pointed this out to me, and indeed the correct methodology is very important. We want to apologize for the word choice we made.

Looking forward to hear from you.

Kind regards,

Hanna van der Valk

Submitted filename: Letter_reviewer2.docx

Click here for additional data file.

15 Mar 2021

PONE-D-20-28005R2

Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands

PLOS ONE

Dear Dr. Johanna van der Valk,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Apr 29 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Yuka Kotozaki

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: COMMENTS: As already said, it being a KAP type cross-sectional survey only [is fairly simple and so straight forward], there is hardly anything to comment, mention/point-out critical observation(s) or suggest few things for further improvement.

However, I just felt that the fact ‘this study also includes comparison between three groups [Negative-tested, Positive-tested, Control] is not adequately mentioned [except line 64 of ‘Abstract-Conclusion’ that “This is one of the first (large) study that investigates and compares patient characteristics”]

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Sanjeev Sarmukaddam

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

19 Mar 2021

Dear Editor

We would like to respond to the comment of the referee.

Reviewer #1: COMMENTS: As already said, it being a KAP type cross-sectional survey only [is fairly simple and so straight forward], there is hardly anything to comment, mention/point-out critical observation(s) or suggest few things for further improvement.

However, I just felt that the fact ‘this study also includes comparison between three groups [Negative-tested, Positive-tested, Control] is not adequately mentioned [except line 64 of ‘Abstract-Conclusion’ that “This is one of the first (large) study that investigates and compares patient characteristics”]

We agree with you. As you suggest, we mentioned now more adequately in our manuscript the fact that this study describes the variables in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic and that we also made a comparison between the “COVID-19 suspected” (positive and negative tested group) with the “COVID-19 not suspected” (control group) and in the “COVID-19 suspected” group, the positive and negative tested patients.

We added the following sentences to the manuscript:

Abstract (line 46-49)

The primary aim of this study was to investigate these variables in patients visiting an emergency department in the Netherlands during the COVID-19 pandemic and to compare the “COVID-19 suspected” (positive and negative tested group) with the “COVID-19 not suspected” (control group) and to compare in the “COVID-19 suspected” group, the positive and negative tested patients.

Material and methods (line 160-162)

A comparison is made between the “COVID-19 suspected” (positive and negative tested group) with the “COVID-19 not suspected” (control group) and in the “COVID-19 suspected” group, the positive and negative tested patients.

We clarified this also in the discussion (line 314-317)

The “COVID-19 suspected” (positive and negative tested group) was compared with the “COVID-19 not suspected” (control group) and in the “COVID-19 suspected” group, the positive and negative tested patients were compared.

We really hope -by addressing you comment- that we improved the message of the manuscript. We are looking forward to hear from you.

Kind regards,

Hanna van der Valk

Submitted filename: Letter_reviewer3.docx

Click here for additional data file.

26 Mar 2021

Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands

PONE-D-20-28005R3

Dear Dr. Johanna van der Valk,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yuka Kotozaki

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: COMMENTS: As already said, it being a KAP type cross-sectional survey is fairly simple and straight forward, therefore there is nothing much to comment as it describes factual information. I think if this info be useful [if at all] for clinicians involved in treating COVID patients, then we should not delay its publication.

Hope, you already have taken note of the fact {which was highlighted on very first occasion by including ‘important note’} that “This review pertains only to ‘statistical aspects’ of the study and so ‘clinical aspects’ [like medical importance, relevance of the study, ‘clinical significance and implication(s)’ of the whole study, etc.] are to be evaluated [should be assessed] separately/independently”.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Sanjeev Sarmukaddam

6 Apr 2021

PONE-D-20-28005R3

Case- control study of patient characteristics, knowledge of the COVID-19 disease, risk behaviour and mental state in patients visiting an emergency room with COVID-19 symptoms in the Netherlands

Dear Dr. van der Valk:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Yuka Kotozaki

Academic Editor

PLOS ONE