Circadian patterns of heart rate, respiratory rate and skin temperature in hospitalized COVID-19 patients.

RATIONALE: Vital signs follow circadian patterns in both healthy volunteers and critically ill patients, which seem to be influenced by disease severity in the latter. In this study we explored the existence of circadian patterns in heart rate, respiratory rate and skin temperature of hospitalized COVID-19 patients, and aimed to explore differences in circadian rhythm amplitude during patient deterioration.
METHODS: We performed a retrospective study of COVID-19 patients admitted to the general ward of a tertiary hospital between April 2020 and March 2021. Patients were continuously monitored using a wireless sensor and fingertip pulse oximeter. Data was divided into three cohorts: patients who recovered, patients who developed respiratory insufficiency and patients who died. For each cohort, a population mean cosinor model was fitted to detect rhythmicity. To assess changes in amplitude, a mixed-effect cosinor model was fitted.
RESULTS: A total of 429 patients were monitored. Rhythmicity was observed in heartrate for the recovery cohort (p<0.001), respiratory insufficiency cohort (p<0.001 and mortality cohort (p = 0.002). Respiratory rate showed rhythmicity in the recovery cohort (p<0.001), but not in the other cohorts (p = 0.18 and p = 0.51). Skin temperature also showed rhythmicity in the recovery cohort (p<0.001), but not in the other cohorts (p = 0.22 and p = 0.12). For respiratory insufficiency, only the amplitude of heart rate circadian pattern increased slightly the day before (1.2 (99%CI 0.16-2.2, p = 0.002)). In the mortality cohort, the amplitude of heart rate decreased (-1.5 (99%CI -2.6- -0.42, p<0.001)) and respiratory rate amplitude increased (0.72 (99%CI 0.27-1.3, p = 0.002) the days before death.
CONCLUSION: A circadian rhythm is present in heart rate of COVID-19 patients admitted to the general ward. For respiratory rate and skin temperature, rhythmicity was only found in patients who recover, but not in patients developing respiratory insufficiency or death. We found no consistent changes in circadian rhythm amplitude accompanying patient deterioration.

Introduction

Many elements of human physiology follow a circadian rhythm to anticipate and react to environmental changes throughout the day [1]. Acute disruption of this cycle is associated with immune dysregulation [2], delirium [3] and even mortality at the intensive care unit (ICU) [4, 5]. Hospitalization can contribute to disruption of circadian patterns due to artificial light, noise, (sedative) medication, and the fact that the individual sleep-wake cycle of a patient has to make way for the hospital routine [1]. In addition, the illness itself can cause circadian disruption, for example in the case of systemic inflammation [1, 6–9]. Neuroinflammation and neurodegeneration specifically might alter the regulation genes, or clock genes, responsible for a normal 24-hour cycle. Coronavirus disease 2019 (COVID-19) has several characteristics that may lead to disruption of circadian rhythms. COVID-19 is accompanied by sleep disturbance [10], neuroinflammation [11], and in severe cases systemic inflammation and encephalopathy [12-14]. Since July 2020, patients with COVID-19 are treated with dexamethasone [15], which can affect the circadian pattern of the human metabolism depending on time of administration [16]. Moreover, circadian patterns of heart rate and respiratory rate can be disturbed by acute hypoxia [17], a common symptom of severe COVID-19.

Several vital signs have shown to follow a circadian rhythm [18-20]. Even in critically ill patients admitted to the ICU, where vital signs are highly influenced by medication and ventilation, circadian patterns were found in respiratory rate, heart rate, blood pressure and temperature [21]. Previous research in ICU settings has shown that circadian rhythm becomes increasingly more pronounced in recovering patients (who will eventually be discharged home), as opposed to patients who will not survive or were discharged with palliative care [21]. However, circadian patterns in vital signs thus far have mainly been studied in either healthy volunteers, or in critically ill patients at the ICU (where continuously recorded data is readily available). Since the development of wireless sensors, continuous monitoring of vital signs at the general hospital ward has become more common [22]. Data can be used for visual monitoring by clinicians, and for the development of clinical decision support models, to detect deterioration of patients at an earlier stage. However, the alarm strategies of many systems are mainly based on single threshold breaches. Aspects of vital sign trends, like a circadian pattern, are not considered, even though incorporating vital signs trends has the potential to improve prediction models and alarm strategies considerably [23, 24]. Moreover, changes in circadian patterns themselves could be valuable predictors of deterioration. A recent study used changes in circadian rhythm characteristics to identify SARS-CoV-2 infection and predict COVID-19 diagnosis [25].

In this exploratory study, we aimed to answer three related research questions. First, we assessed whether circadian rhythms can be observed for heart rate, respiratory rate and skin temperature in COVID-19 patients admitted to a general hospital ward. Subsequently, we assessed to what extent these circadian rhythms exist in patients who develop respiratory insufficiency, patients who died, and patients who recovered without developing respiratory insufficiency. Lastly, we explored whether changes in the amplitude of circadian rhythms of vital signs can be observed in deteriorating patients, and could therefore be possible predictors of deterioration.

Methods

We performed a retrospective cohort study of patients who were diagnosed with COVID-19. Patients were offered the chance to opt-out of retrospective data analyses during hospital registration and again at hospital discharge, according to the institutional protocol. A waiver for ethical review was obtained from the medical ethical research committee Utrecht (MERC-20-365). The study was conducted according to the principles of the Declaration of Helsinki and the General Data Protection Regulation [26, 27].

Setting

During the pandemic, a continuous wireless monitoring system for vital signs was deployed at the COVID-19 cohort ward of a tertiary medical center in Utrecht, the Netherlands, starting April 1, 2020. This system recorded heart rate, respiratory rate and skin temperature twice per minute, using a wearable wireless patch sensor (Biosensor Voyage, Philips Electronics Netherlands BV) and peripheral oxygen saturation (SpO2) via a finger pulse-oximeter (EarlyVue VS30, Philips Electronics Netherlands BV). The patch sensor was attached on the left hemithorax, approximately 2 cm sub clavicular, and was replaced every three days following manufacturer instructions. Patients with a pacemaker did not receive a sensor since ECG-derived respiratory rate measurements are unreliable in paced rhythms. Heart rate, respiratory rate and oxygen saturation was real-time available for all caregivers to support care. The values for skin temperature were not directly available, since the clinical relevance of skin temperature is unsure and not yet integrated in general hospital care.

Data collection

Patients were included starting April 1, 2020 until March 1, 2021. Inclusion was stopped because the manufacturer stopped delivering these sensors to focus on the production of other sensors, but the replacement did not meet the accuracy requirements. All patients with confirmed COVID-19 and available continuous sensor data were included. To be able to describe the cohort, baseline characteristics were recorded from the electronic patient record, including the Charlson Comorbidity Index for predicting 1-year mortality [28].

Data selection

Patients were divided into three groups: patients who recovered without experiencing respiratory insufficiency, and patients with severe clinical deterioration, divided into patients who developed respiratory insufficiency and patients who died. We chose these three groups since respiratory insufficiency and mortality are both outcomes of severe patient deterioration, but follow a different course. Patients seldom died unexpectedly, and often received palliative care in the last days before death. Therefore we decided to analyze this group separately. If a patient developed respiratory insufficiency at any point during admission, he or she was included in the respiratory insufficiency cohort, and not in the recovered cohort. If a patient developed respiratory insufficiency and died while being monitored, he or she was included in the mortality cohort instead of the respiratory insufficiency. Respiratory insufficiency was defined as the need for 15 l/min oxygen therapy, high flow oxygen therapy or mechanical ventilation, whichever came first. We did not deem ICU admission a suitable endpoint since a substantial part of the population had treatment restrictions preventing them from ICU admission, and the hospital regularly struggled with capacity problems at the ICU. Instead, we chose the endpoint hypoxic respiratory insufficiency, which better reflects the starting point of severe illness in COVID-19. The time and date of onset of respiratory insufficiency was manually collected from the electronic patient record.

Since the length of stay and length of continuous monitoring varied among patients, we chose to only include 3 days (72 hours) of data for each patient. This way we aimed to avoid overrepresentation of patients with more data. For patients in the respiratory insufficiency cohort, we selected the 72 hours before onset of respiratory insufficiency. For patients who died, we selected the 72 hours of data preceding death. Since respiratory insufficiency usually occurred within the first 72 hours (median 33 hours) of admission, we selected the first 72 hours of data for patients in the recovery group as a comparable control. Since at least 4 hours of data was needed for statistical analysis, patients with less than 4 hours of continuous data in the selected 72-hour timeframe were excluded.

All continuous vital sign data was validated before use: physiologically improbable data was removed using a predefined computer algorithm. Since our cohort included dying patients, we used wide limits for improbable data (for respiratory rate <1/min & >80/min; for heart rate <30/min & >280/min; for skin temperature < 25°C). Artifacts in respiratory rate and heart rate were filtered by removing large abrupt changes that lasted for less than 2 minutes (for respiratory rate a change of >20/min, for heart rate a change of >25/min). To ensure we only used skin temperature data of periods that the wearable was attached to the patient, and not the data of the preparation period, we only used skin temperature data between the first and last valid heart rate measurements. The first 10 measurements (5 minutes) of skin temperature data of each patient were removed, since the sensor needed several minutes to warm up.

Statistical analysis

To limit the impact of short-lasting outliers and minutes with missing data further, the median of each vital sign per fifteen-minute segment was calculated for each patient. Subsequently we calculated the overall mean of these medians, including a 95% confidence interval (CI) and the 95% upper and lower limit of all measurements. Data was plotted for visual evaluation. For quantitative evaluation we made use of a cosinor model. A cosinor model is a type of non-linear model used to asses repetitive patterns, such as circadian rhythms [29]. A cosinor consists of several components. The MESOR (midline estimating statistic of rhythm) is the rhythm adjusted mean of the modelled variable, e.g. the rhythm adjusted mean heart rate. The amplitude is the measure of the extent of predictable change within the cycle, e.g. 2 heart beats/min. Two times the amplitude is the difference between the highest and lowest point of the cosinor regression line. The acrophase represents the timing of overall high values in a cycle, expressed in (negative) degrees, where the reference time is set to 0°, and a full period is 360°. The period is the (expected) duration of one cycle, which is 24 hours for circadian cycles. For this study, we fitted two separate cosinor models. First, we used a cosinor model of the population mean to estimate the mean coefficients of the three cohorts and to detect rhythmicity, using R package ‘cosinor2’ [29]. This model illustrates mean differences between the cohorts. Rhythmicity was determined by the fit of the cosinor model using the F-ratio. However, this model does not account for correlation within individual patients and cannot assess longitudinal changes in data. Therefore, we fitted a cosinor mixed effects model as second model, using R package ‘cosinoRmixedeffects’ [25, 30]. This allows for random MESOR, amplitude and acrophase per patient. We included an interaction term with the day on which measurements were taken, to see if coefficients changed over the three-day observation period. To estimate means and mean differences, we used a bootstrapping method with 500 simulations [30]. For more elaborate explanation of this method we refer to the article by Hirten et al. [25]. A p-value of 0.01 was deemed to be statistically significant for quantitative analysis. R software version 4.0.3 (R foundation for Statistical Computing, Vienna, Austria 2021) was used for all analyses.

Results

Between April 1st 2020 and March 1st 2021, a total of 429 COVID-19 patients were continuously monitored at the ward. Of these, 368 could be included for analysis: 296 patients who recovered without developing respiratory insufficiency, 27 patients who died, and 45 patients who developed respiratory insufficiency and either recovered, or died without being monitored (Fig 1). Table 1 shows a description of the cohort. Note that patients who died were older, had more comorbidities, received dexamethasone less often and had a higher rate of ‘Do not ventilate’ orders.

Fig 1

Flowchart of patient inclusion and data selection.

Table 1

Patient characteristics and median duration of recorded vital signs during three-day observation period.

		All	Recovery	Resp. insuf.	Mortality
Number of patients		368	296	45	27
Age (median, IQR)		65 (55–74)	63.5 (55–72)	64 (56–73)	76 (71–82)
Male sex (n, %)		221 (60.0%)	181 (61.1%)	25 (55.6%)	15 (55.6%)
CCI (median, IQR)		3 (1–4)	2 (1–4)	3 (2–4)	4 (4–6)
Dexamethasone administration (n, %)		279 (75.8%)	223 (75.3%)	38 (84.4%)	18 (66.7%)
‘Do not ventilate’ order (n, %)		91 (24.7%)	57 (19.3%)	10 (22.2%)	24 (88.9%)
Length of stay (median days, IQR)		7 (4–11)	6 (4–10)	15 (10–31)	8 (5–13)
Median (IQR) hours of data per patient during 72-hour timeframe	• Heart rate	72 (46.8–72)	72 (60–72)	34 (25.8–70.3)	68.8 (26.4–72)
	• Respiratory rate	62.1 (38.3–72)	63.5 (48.9–72)	31.5 (18.7–51.9)	60.5 (17.3–72)
	• Skin temperature	63.6 (37.8-63-6)	72 (52.5–72)	30.8 (12.4–51.9)	60 (15–72)

Resp. insuf.: hypoxic respiratory insufficiency, CCI: Charlson Comorbidity Index based on 1 year mortality, IQR: interquartile range

Assessment of rhythmicity

Fig 2 shows the raw overall mean of the vital signs in the three cohorts. Both the respiratory insufficiency and mortality cohort had a small sample size and wide confidence intervals. Rhythmicity in mean heart rate was found in all cohorts (recovery p<0.001, respiratory insufficiency p<0.001, mortality p0.002) (Table 2). Rhythmicity in mean respiratory rate and mean skin temperature was only found in the recovery cohort (resp. p<0.001 and p<0.001).

Fig 2

Mean of vital signs during three day observation period in each cohort.

Table 2

Coefficients of cosinor models.

Recovered patients are compared to patients with respiratory insufficiency and deceased patients.

	Recovered (95%CI)	Resp. insuf. (95%CI)	p-value of difference	Died (95%CI)	p-value of difference
Heart rate (/min)
• MESOR	74.7 (73.3–76.1)	78.9 (73.9–84.0)	0.04	95.3 (88.0–102.5)	<0.001
• Amplitude	6.9 (6.4–7.5)	5.1 (3.1–7.1)	0.76	4.0 (2.0–5.9)	0.58
➢ Rhythmicity	p<0.001	p<0.001		p = 0.002
Respiratory rate (/min)
• MESOR	20.7 (20.3–211)	22.7 (21.0–24.5)	0.001	26.0 (24.4–27.6)	<0.001
• Amplitude	1.0 (0.7–1.2)	1.4 (-0.24–2.9)	0.90	1.0 (-0.86–3.0)	<0.001
➢ Rhythmicity	p<0.001	p = 0.18		p = 0.51
Skin temperature (°C)
• MESOR	34.2 (34.1–34.3)	33.2 (31.4–34.8)	0.003	34.6 (34.1–35.1)	0.07
• Amplitude	0.39 (0.28–0.50)	1.5 (-0.2–3.2)	0.66	0.32 (0.02–0.62)	0.95
➢ Rhythmicity	p<0.001	p = 0.22		p = 0.12

Resp. insuf.: respiratory insufficiency, MESOR: midline estimation statistic of oscillation

Changes in circadian pattern amplitude

The cosinor characteristics for each cohort per day are presented in Fig 3 and S1 Fig. The MESOR values for heart rate and respiratory rate were lower in the recovery cohort than the respiratory insufficiency cohort, but higher in the mortality cohort. In the recovery cohort, an increase of amplitude was seen for all parameters over the course of the three days. The amplitude for heart rate significantly increased on day 2 (difference of 0.90 (99%CI 0.64–1.2, p<0.001)) and from day 2 to 3 (difference of 0.53 (99%CI 0.21–0.85, p<0.001)) (Table 3). Respiratory rate amplitude increased from day 2 to 3 (difference of 0.25 (99%CI 0.14–0.35, p<0.001)) and skin temperature amplitude increased from day 1 to 2 (difference of 0.10 (99%CI 0.06–0.13, p<0.001). For the respiratory insufficiency cohort, only heart rate showed a clear increase in amplitude (difference day 2 to day 3 of 1.2 (0.16–2.2, p = 0.002)). Skin temperature amplitude initially decreased (difference day 1 to 2 of -0.31 (99%CI -0.48- -0.14, p<0.001)) and later increased (difference day 2 to 3 of 0.16 (99%CI 0.00–0.23, p = 0.006). In the mortality cohort, heart rate amplitude decreased from day 1 to 2 (difference of -1.5 (99%CI -2.6- -0.42, <0.001), and respiratory rate amplitude increased from day 2 to 3 (difference of 0.72 (99%CI 0.27–1.3, p = 0.002).

Fig 3

Progression of cosinor characteristics over the course of three days for heart rate, respiratory rate and skin temperature, stratified by cohort.

Table 3

Differences in cosinor mixed effect model amplitudes (difference, 99%CI) between days for the A. recovery cohort, B. respiratory insufficiency cohort, and C. mortality cohort.

	day 1 vs day 2	p-value	day 2 vs day 3	p-value
A. Recovery
• Heart rate	0.90 (0.64–1.2)	<0.001	0.53 (0.21–0.85)	<0.001
• Respiratory rate	0.01 (-0.08–0.10)	0.82	0.25 (0.14–0.35)	<0.001
• Skin temperature	0.10 (0.06–0.13)	<0.001	0.00 (-0.04–0.03)	0.80
B. Respiratory insufficiency
• Heart rate	0.20 (-1.2–1.5)	0.71	1.2 (0.16–2.2)	0.002
• Respiratory rate	-0.15 (-0.60–0.26)	0.39	0.12 (-0.19–0.49)	0.36
• Skin temperature	-0.31 (-0.48- -0.14)	<0.001	0.16 (0.00–0.23)	0.006
C. Mortality
• Heart rate	-1.5 (-2.6- -0.42)	<0.001	0.40 (-0.75–1.6)	0.39
• Respiratory rate	0.01 (-0.34–0.35)	0.96	0.72 (0.27–1.3)	0.002
• Skin temperature	-0.04 (-0.19–0.09)	0.51	-0.02 (-0.16–0.11)	0.68

Discussion

In patients admitted with COVID-19, we could confirm the presence of a circadian rhythm of heart rate. For respiratory rate and skin temperature, a circadian pattern could only be observed in patients who ultimately recovered. The amplitude of heart rate circadian rhythm increased slightly the day before respiratory insufficiency. In dying patients, a slight decrease in heart rate amplitude and an increase in respiratory rate amplitude can be observed in the days before death. Although statistically significant, these differences were small.

The existence of a circadian rhythm in vital signs has been well established [18-20]. However, in daily clinical practice, this physiological rhythm is hardly considered. With the advent of wireless continuous vital signs monitoring, patterns in vital signs are gaining attention. A recent study on cardiovascular changes in COVID-19 found a repetitive pattern in cardiovascular parameters and hypothesized this to be part of a circadian rhythm [31]. Our study confirms the existence of a circadian pattern in vital signs of hospitalized COVID-19 patients. A study performed in multiple intensive care units demonstrated circadian patterns for blood pressure, heart rate, respiratory rate and temperature [21]. This study found that the difference between the peak and nadir of vital signs is reduced in patients who died compared to patients who recovered. This led to the hypothesis that a decrease in circadian rhythm amplitude might contain prognostic information. In our study, we could not confirm a consistent decrease of circadian rhythm amplitude in deteriorating COVID-19 patients. Some vital signs even showed a slight increase of circadian pattern amplitude during of the observation period. The method we used here, however, is different. In the study by Davidson et al., the peak-nadir excursion was used to quantify the circadian rhythm, which is somewhat different from the cosinor amplitude and might be more influenced by temporary peaks and troughs. These methodological differences might explain the observed differences in results.

Although we did not find a decrease in amplitude values for deteriorating patients, we did find a lack of rhythmicity in mean respiratory rate and mean skin temperature in the days leading up to respiratory insufficiency or death. This could be a sign of a generalized disturbed circadian rhythm in these patients. Changes in heart rate and respiratory rate during the day are mostly caused by changes in arousal and level of muscle activity, independent of the time of day [19, 32–34]. If patients are active during the night, e.g. due to severe illness and/or delirium, they could have similar vital signs during these periods as during the day, resulting in a lack of rhythmicity. Periods of fever and hypoxemia could also result in temporary deviations in heart rate and respiratory rate, disrupting the circadian pattern even further.

Patients who died at the hospital ward showed no rhythmicity of respiratory rate and skin temperature, and a decrease of heart rate amplitude two days before death. This is in accordance with the observations of Davidson et al. 2021 [21]. The decrease of circadian rhythm might be caused by several factors. Severe illness has shown to influence clock gene expression and melatonin excretion [35, 36]. Older age is also accompanied with lower levels of melatonin [37]. Comorbidities and medication suppressing the regulation of vital signs, such as metoprolol, could have influenced circadian patterns too. Furthermore, circadian rhythms are influenced by light input [37]. As part of palliative care, patients were often relocated to single rooms with closed blinds for comfort. These patients also often received sedative medication such as opioids and benzodiazepines, blurring the difference between wake and sleep. This might have played a role in the lack of rhythmicity in this cohort. Lastly, patients often died after more than 72 hours of admission. The selected data therefore represents a later part of the admission than the data of the other two cohorts. The longer hospitalization time might have added to the disruption of circadian rhythm. In dying patients, continuous monitoring was often discontinued as part of palliative care too, so unfortunately only few patients could be included for analysis.

Skin temperature showed a circadian pattern opposite from heart rate and respiratory rate, with its peak at night instead of during the day. Core temperature usually drops during the night due to an increase of skin temperature and the subsequent excess heat loss [38-40]. This is, however, only true for distal body parts. In our study, we used a sensor that was attached to the chest, two centimeter sub clavicular. In such a proximal location, the skin temperature is expected to follow the same pattern of the core temperature [38], instead of the inversed pattern that we observed. Why this phenomenon occurred is unknown.

Strengths and limitations

This study shows that a circadian rhythm of vital signs is present in hospitalized COVID-19 patients. All patients were admitted with the same disease, with a known pattern of deterioration, and for each patient a large set of data points was available for analysis. This made it possible to not only look at the differences between cohorts, but also to analyze more closely the changes of amplitude during deterioration. Our study also has multiple limitations. Even though the overall sample size was large, the respiratory insufficiency and mortality cohorts were relatively small, resulting in wide confidence intervals. The data selection of the mortality cohort was of a later stage of admission than the other two cohorts, introducing ‘hospitalization time’ as a possible confounder. In future research, this could be avoided by using a case-control design matched by length of hospital admission. Although previous studies have shown differences in vital signs patterns between men and women [21, 31, 41], we decided not to do a sub analysis based on sex due to the limited sample size of two of the cohorts. Secondly, all patients in our study were admitted with COVID-19, and therefore conclusions can only be drawn regarding this specific population. Lastly, skin temperature can be modified by many factors, including environmental temperature, clothing, showering, and exercise. The effect of miscellaneous factors, such as leakage of airflow from underneath an oxygen mask, are unknown. The clinical relevance and interpretation of skin temperature therefore is uncertain. Nonetheless, the observation that a circadian rhythm is present for skin temperature in COVID-19 patients who recover could be a valuable continuously measured vital parameter for the future.

Use in predictive modelling and clinical practice

Continuous monitoring is used increasingly outside high care units in an effort to detect deterioration timely [22]. In COVID-19 too, the trajectory of vital signs is hypothesized to aid in the detection of respiratory and cardiovascular decline [31]. Predictive models and alarm strategies could help clinicians to recognize deterioration, without producing too many false alarms [42]. The performance of these models might be influenced by the existence of a circadian rhythm. Previous research has already shown that accounting for differences in vital signs values between day and night may reduce alarm rate in various models at the general ward [24]. The next step in predictive modeling with continuous data is trend analysis, since changes of vital signs might be better predictors than single values [43, 44]. Both model builders and hospital professionals should be aware however that a rise in heart rate and respiratory rate in the morning, or a rise of skin temperature in the evening, might not be a deteriorating trend at all, but rather a part of a physiological rhythm. Even though this should be accounted for, changes in circadian rhythm themselves are unlikely to be useful as predictors of deterioration Lack of rhythmicity is not reflected in a decrease of amplitude, so a different metric should be used to express decrease of rhythmicity. Furthermore, one would need at least 24 hours’ worth of data before being able to assess a circadian pattern. Future research should focus on adequately predicting deterioration with vital sign trends despite the existence of circadian patterns. In clinical practice, several general wards have already implemented continuous monitoring for COVID-19 patients [31, 45, 46]. Alarm strategies and escalation protocols are often based on early warning scores, which could be influenced by physiological changes in vital signs over the day. Based on our clinical experience during the pandemic, the early warning scores of the majority of COVID-19 patients increase in the morning when patients become physically active. Awareness of the existence of a circadian rhythm in common vital signs might aid nurses and physicians in the interpretation of continuous data and continuous early warning scores.

In conclusion, a circadian rhythm is present in heart rate of COVID-19 patients admitted to the general ward. For respiratory rate and skin temperature, rhythmicity was only found in patients who recovered, but not in patients developing respiratory insufficiency or death. We found no consistent changes in circadian rhythm amplitude accompanying patient deterioration.

STROBE statement—checklist of items that should be included in reports of cohort studies.

(DOCX)

Click here for additional data file.

Progression of cosinor characteristics per vital sign per group, mean with 99%CI.

MESOR: midline estimation statistic of rhythm. MESOR and amplitude are in /min for heart rate and respiratory rate, and°C for skin temperature. Acrophase in (degree).

(TIF)

Click here for additional data file.

20 Dec 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Henrik Oster, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

Additional Editor Comments:

n/a

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: No

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Harriët MR van Goor and colleagues reported the existence of circadian patterns in heart rate, respiratory rate and skin temperature of hospitalized Covid-19 patients. In their study, they compared various stages of disease. Albeit the predictive power of circadian pattern amplitude for disease severity was low, the authors state that accounting for circadian patterns might improve general monitoring- and alarm strategies.

The overall writing style and the accuracy of language is sufficient. However, the novelty of the reported findings is questionable as most of the findings are not specific for Covid-19 and already reproduced several times in other cohorts. Moreover, additional improvements within methods and discussion addressing the recent literature are required.

1. Introduction: “Since September 2020, patients with covid-19 are treated with dexamethasone[15], which has an suppressive effect on the circadian pattern of the human metabolism[16].” The citation (16) is not applicable. In the cited study dexamethasone was administered in the afternoon. Within the clinical routine, dexamethasone is likely to be administered in the early morning which might rather result in a strengthening of circadian rhythms. How many patients received dexamethasone in this study? Why September 2020, not June 2020?

2. Methods: “Inclusion of patients stopped because the wearable sensor was no longer available.” Please specify: Was the availability of the sensor tied to funding for the study or did the company stop production due to unreliable measurement accuracy?

3. Methods: “Patients with a pacemaker did not receive a sensor since RR measurements might result unreliable in paced rhythms.” Please clarify: Were those patients completely excluded from the study or only RR measurements were excluded for those patients?

4. Methods: „Since our cohort included dying patients, we used wide limits for improbably 114 data (for RR <1/min & >80/min; for HR <30/min & >280/min; for sT < 25°C).” The lower limits for temperature and respiratory rate are extremely wide and should be critically revised.

5. Methods: “Data was divided in five cohorts based on different stages of disease…. Hypoxic respiratory insufficiency was defined as the need for 15 l/min oxygen therapy.” Why did the authors not stratify according to the WHO criteria in mild, intermediate and severe COVID19. Please discuss and reference, if this method has been used before.

6. Methods: “For quantitative assessment we divided the data in daytime (06:00-00:00) and nighttime (00:00-06:00).” The daytime period is proportionally much longer than the nighttime. Please discuss and reference, why this method was chosen and if this method has been published before.

7. Methods: Data collection included the Charlson Comorbidity Index, but the group differences were not further discussed within the manuscript. The use of further disease severity scores for ICU patients (SOFA, GCS) would complement the author’s analysis.

8. Statistics: Although the authors nicely removed several abrupt deviations before analysis, the use of PNex measurement might not be picking up circadian trends very well. It is correlated, however results are very noisy. For a better understanding the analysis needs to be discussed and compared with other methods applied in previous publications examining vital signs. Why did the authors not perform a regular rhythmicity analysis and sine curve fit to better estimate the amplitude?

9. Discussion: Studying a clinical cohort of Covid-19 patients exclusively, the authors conclude that a general knowledge of circadian patterns might improve general monitoring- and alarm strategies. This affirmation is not supported by data including other disease entities and therefore needs to be discussed including more recent literature and trials examining circadian patterns in clinical cohorts.

Daou M, Telias I, Younes M, Brochard L, Wilcox ME. Abnormal Sleep, Circadian Rhythm Disruption, and Delirium in the ICU: Are They Related? Front Neurol. 2020 Sep 18;11:549908. doi: 10.3389/fneur.2020.549908. PMID: 33071941

Lachmann G, Ananthasubramaniam B, Wünsch VA, Scherfig LM, von Haefen C, Knaak C, Edel A, Ehlen L, Koller B, Goldmann A, Herzel H, Kramer A, Spies C. Circadian rhythms in septic shock patients. Annals of Intensive Care. 2021 11: 64. PMID 33900485

Maas MB, Lizza BD, Abbott SM, Liotta EM, Gendy M, Eed J, Naidech AM, Reid KJ, Zee PC. Factors Disrupting Melatonin Secretion Rhythms During Critical Illness. Crit Care Med. 2020 Jun;48(6):854-861. PMID: 32317599

Maas MB, Iwanaszko M, Lizza BD, Reid KJ, Braun RI, Zee PC. Circadian Gene Expression Rhythms During Critical Illness. Crit Care Med. 2020 Dec;48(12):e1294-e1299. PMID: 33031153

Reviewer #2: PLOS ONE – D 21 34906

Circadian patterns of HR, RR, and skin temp in hospitalized COVID19 patients

This study evaluated circadian patterns in patients with COVID19 admitted to inpatient ward, using pulseox + “wireless sensor” measurements over 5 different time windows of their hospitalized illness. As a descriptive study this is interesting and provides rationale for continuing to investigate the utility of such measurements for risk prediction and patient monitoring.

The main issue is that the current study design does not support an analysis of risk for hypoxemic respiratory insufficiency as the authors propose. Evaluating different time periods a single cohort of patients, comparing a pre-hypoxemic “control” period and post-hypoxemic “outcome” period in those who developed hypoxemic respiratory insufficiency (defined in this study as >15L O2 therapy), we are able to establish only that these measurements differ in these two periods of time within these patients. To evaluate for risk of hypoxemic respiratory insufficiency, the authors must include a study population that did not develop hypoxemic respiratory insufficiency. This could be done in a variety of ways to create either a cohort or a case-control study design. The analysis should be revised to include such a cohort, or the manuscript needs to be rewritten with this significant change in mind.

It would also be helpful to include references using the methods employed (daytime peak vs. nighttime nadir of physiologic data) to evaluate circadian patterns in the data.

Abstract

- Please define “delta day” as a measurement in the abstract.

- Please revise to more accurately describe the logistic regression results (that VS changes are associated with worsened hypoxemia within in this select cohort of patients; they do not demonstrate risk of developing hypoxemia – as above, this cannot be evaluated unless the study also includes patients who did not develop hypoxemic respiratory insufficiency.)

Methods

- p5. Please clarify - if sT measurements were not directly available then how were they studied?

- Please include how many COVID19 patients were excluded for lack of sensor data.

- p.6. Please correct “improbably data.”

- The authors excluded patients with < 48 hours of continuous data during a three-day period, except for the post-hypoxemic respiratory failure period. This suggests that patients who developed hypoxemic respiratory failure very quickly (eg after 1-2 days) or who died shortly after admission without developing hypoxemic respiratory failure, etc. would have been excluded. This biases the study population to those patients who had a relatively slow progression of disease and may bias the study population away from those admitted to the ICU (it is unclear whether remote monitoring continued after ICU admission?). The authors should consider revising the analysis, using a shorter time window for each “period”.

- Consider including a flow diagram to describe the total number of potentially eligible patients, and those excluded at each stage (e.g. as suggested by the STROBE statement).

- Including patient data in more than one time-period “cohort” and then comparing differences between time periods is problematic from a study design perspective. This introduces bias towards the null – which makes it more difficult to identify true differences in vital signs between cohorts. The authors should consider revising their compared time periods so that no overlap occurs.

- How accurately was the onset of hypoxemic respiratory insufficiency defined? It would be helpful to describe how this information was obtained, e.g. if this was time stamped (hour and minute) to ensure that the vital sign data was assigned correctly as pre- vs. post-intervention.

- p.7. Were peaks and nadirs drawn from raw data? Typically, circadian analyses using peak/nadir are drawn from a cosinor or sinusoidal analysis that accounts for the entire dataset shifting + and – in a circadian fashion, which is less sensitive to random noise (e.g., Shoben Am J Epi 2011 for an excellent example.) If no cosinor analysis is applied, thenit seems like this peak-nadir measurement (PNex) will only provide the range of daytime vs. nighttime data. Please include references for using this measurement in circadian analyses.

Results:

- p. 7. Please rename the 5 time periods; they are not truly “cohorts” as they include the same patients, and the patient data is actually duplicated across some of these analytic groups. Suggest using “time periods” or something similar.

- How do more patients have respiratory insufficiency than deterioration? This might be clarified by using more distinct/separated definitions of time periods for comparison and/or a flow diagram as suggested above.

- Table 3 sugests that day 1 vs. 2 of respiratory insufficiency and day 1 vs 3 of respiratory insufficiency were compared in logisitc regression. This is different from the analyses described in the Methods. Please clarify.

-p.9; was mean PNex compared quantitatively between stages using a statistical test? If so, consider adding these comparisons to table 2.

-p.9, lines 189-191 and 191-192 should be edited for accuracy. Since patients without hypoxemic respiratory insufficiency were not included in this analysis, we cannot evaluate whether this development was associated with the development of hypoxemia. All we know is that the delta PNex was associated with skin temperature in this select cohort of patients who will go on to develop respiratory insufficiency.

-Table 3: Table 3 compares the difference in Pnex between days 1 and 2 and days 1 and 3. Please clarify – is this limited to stage III patients? If so, please add this detail and the number of patients included to the title.

Discussion:

- I agree that Figure 1 suggests qualitatively that there is circadian variability given the cyclical nature of overall VS pattern. However, it’s hard to claim definitively that there is no circadian variability based on a subjective assessment (eg for the patients with mortality.) From figure 1, stage I and stage III do not necessarily look all that different – it is not clear what the authors used as a cut off for circadian rhythm presence vs. absence. If a quantitative analysis is possible, one should be included. If not, the discussion should reflect this uncertainty throughout.

- The Discussion should be edited in several places to accurately reflect the results presented in Table 3. This analysis supports only an association between PNex and skin temperature, not an association with hypoxemic respiratory failure.

- If the PNex is not a validated method of measuring circadian amplitude, then the related portions of the discussion should be revised accordingly.

Figure 1

- Please provide a legend for the green, red, and black tracings. The caption says that the Y axis is hours; is this correct? (It seems like the X axis is hours?)

Figure 2

- Consider defining in the methods that you plan to compare day 1, 2 and 3 within the 5 time windows. Were any quantitative analyses applied to these 1/2/3 day comparisons? Clarifying why it was included and what these results mean would be helpful.

Reviewer #3: See attached file

**Review for PONE-D-21-34906**

In this retrospective study among 429 covid-19 patients admitted to the general ward, the authors aim to evaluate whether circadian rhythms can be observed in those patients with respect to heart rate, respiratory rate and skin temperature. Moreover, the authors explore a possible association between circadian pattern amplitude and hypoxic respiratory insufficiency. Data from five predefined time intervals were analyzed: (1) the first and (2) last three days of admission, the three days (3) preceding and (4) succeeding hypoxic respiratory insufficiency, and the (5) last three days before death.

Results revealed that heart rate and respiratory rate followed a circadian pattern in all stages of hospital admission, except for the days prior to death. Skin temperature only followed a circadian pattern on admission, discharge, and the days preceding respiratory insufficiency.

The authors conclude that these circadian patterns could improve monitoring- and alarm strategies. However, the predictive power of peak-nadir excursion however appears to be low.

The authors are to be complimented for their initiative to provide data in this important and new field of research. However, there are significant issues regarding explanation of methods, data analysis and the presentation of the results. Furthermore, part of the conclusion does not seem to be supported by the data provided.

*Major Comments*

1. We ask the authors to explain in more detail the rational for the definition of each predefined stages/cohorts? Why did the authors did not perform a longitudinal statistical analysis? It would be interesting to know if circadian patterns change over time depending on specific covariates (covariate analysis: e.g. respiratory insufficiency as a covariate). We suggest a separate statistical methods paragraph. Why are p-values not provided within the manuscript. A review by an independent statistician may be helpful.

2. The applied definition of _a circadian pattern_ should be explained in the manuscript. Sentences like _A circadian pattern was observed in HR and RR during stages I-IV._ are unclear. When should clinicians deem a pattern as a _normal circadian rhythm_? Did the authors look for indicators of circadian disturbances when analysing the data? A longitudinal analysis of patterns comparing patients with and without developing respiratory insufficiency would be very interesting for the readership.

3. The results section is very hard to "digest"; especially for readers who are not experts in the s very new field of research. We strongly suggest to amend the results section with figures, which illustrate the main results more clearly.

4. The authors conclude that circadian patterns of heart rate, respiratory rate and skin temperature could improve monitoring- and alarm strategies. At this stage of the manuscript, we do not think that this conclusion is supported by the data. We agree, that including informations of circadian profiles within in prediction models and monitoring algorithms may be an interesting and helpful approach in the future. We ask the authors to provide 1 or 2 concrete examples or clinical scenarios how this could be helpful in clinical decision making. However, the authors should be clear, that this is not supported by their data and should remove statements from the conclusion section.

5. Page 10, Line 204: „The mean circadian pattern amplitude showed differences between stages, but only an increasing PNex of skin temperature was associated with developing respiratory insufficiency and with only a small effect size." Please provide a P-value. Was the association statistical significant?

*Further Comments*

- Page 3, Line 42-44: „Since September 2020, patients with covid-19 are treated with dexamethasone, which has a suppressive effect on the circadian pattern of the human metabolism." What does _suppressive effect on circadian pattern_ mean? Please explain.

- Page, Line 226: „The decrease of circadian rhythm might be due to severe illness and extreme physical stress, but could also have been influenced by age, comorbidities and medication. Furthermore, circadian rhythms are highly influenced by light input. As part of palliative care, patients were often relocated to single rooms with closed blinds for comfort. This might have played a role in the decrease of circadian pattern seen in all vital parameters.“ Please explain in more detail why circadian rhythm disturbances can occur in the context of severe diseases, e. g. closed eyes, reduced physical activity, inflammation etc.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PLOS ONE D21-34906 notes.docx

Click here for additional data file.

Submitted filename: PONE-D-21-34906_R1.pdf

Click here for additional data file.

23 Feb 2022

Circadian patterns of heart rate, respiratory rate and skin temperature in hospitalized covid-19 patients - Response to reviewers’ and editors’ comments

Dear prof. dr. Oster, dear reviewers,

We would like to thank you for considering our manuscript for possible publication, and for the opportunity to improve our work using the detailed and specific reviewers’ comments. We noticed that all reviewers asked for substantial changes in the analytical approach and we have adjusted our analyses and the Methods section accordingly. Several reviewers suggested to use a cosinor model instead of peak-nadir excursion, and we believe the use of this type of modelling has improved our manuscript substantially. We have also redefined our cohorts, to avoid both bias and confusion. We decided to keep patients with respiratory insufficiency and patients who died separate, since we feel that the clinical circumstances of these patients were very different.

Because of the substantial revision of methodology and statistical analysis, some sections of the manuscript have been rewritten. We aimed to address as many concerns as possible, but you will find that in the new manuscript some comments are no longer applicable.

We believe that these changes have considerably improved the manuscript. If there are additional comments and required revisions, we will, of course, be happy to address each of these.

Looking forward to hearing from you,

Sincerely,

Harriët van Goor

h.m.r.vangoor-3@umcutrecht.nl

EDITOR

Fix issues with citations.

We have critically revised our references, and have adjusted citations in accordance with the suggestions made by the reviewers.

Clarify information on dexamethasone treatment and absent data.

We have added information on dexamethasone treatment, and have provided information on the amount of continuous data used in analysis.

Provide more details on methodology and statistics as specified by the reviewers.

We have considerably modified the analysis methodology as per the reviewers’ suggestions, and have added more elaborate explanations and references to substantiate our choices.

Improve discussion about alternative methodologies and include more recent references.

We have added references to the new methodological approach, which was recently been developed and has been used to assess circadian rhythm in HRV in covid-19 patients. We have added several more recent references in the discussion, especially regarding covid-19.

Consider including a study population that did not develop hypoxemic respiratory insufficiency – or at least discuss this omission and tone down conclusions.

We have redefined our cohorts to clarify the distinction between patients that did and did not experience respiratory insufficiency.

Consider including a flow diagram to describe the total number of potentially eligible patients, and those excluded at each stage (e.g. as suggested by the STROBE statement).

We have added a flow chart (figure 1) of patient inclusion.

Avoid including patient data in more than one time-period “cohort” and then comparing differences between time

We have redefined our cohorts to clarify the distinction between patients that did and did not experience respiratory insufficiency, and have aimed to model coefficient changes over time.

If no cosinor analysis is applied, thenit seems like this peak-nadir measurement (PNex) will only provide the range of daytime vs. nighttime data. Please include references for using this measurement in circadian analyses.

We have no longer used the method of PNex, but used cosinor modelling as suggested by several reviewers.

Include a quantitative analysis for rhythmicity.

We have added a rhythmicity test based on the goodness of fit of the cosinor model for the population mean cosinor models.

Discuss more specifically how findings could be translated into the clinics.

We have specifically addressed the implications for both predictive modelling and clinical practice.

Reviewer #1

1. Introduction: “Since September 2020, patients with covid-19 are treated with dexamethasone[15], which has an suppressive effect on the circadian pattern of the human metabolism[16].” The citation (16) is not applicable. In the cited study dexamethasone was administered in the afternoon. Within the clinical routine, dexamethasone is likely to be administered in the early morning which might rather result in a strengthening of circadian rhythms. How many patients received dexamethasone in this study? Why September 2020, not June 2020?

Thank you very much for this observation. We did not realize the difference in timing of administration of dexamethasone would have the opposite impact in our study population. We have adjusted the sentence to better reflect this. As for the amount of patients receiving dexamethasone, we have added this to our baseline table. We chose September 2020 since this was the date of the first scientific publication of the RECOVERY trial according to the website of the RECOVERY trial. We have adjusted this to July 2020 when the first preliminary report was published by the NEJM.

2. Methods: “Inclusion of patients stopped because the wearable sensor was no longer available.” Please specify: Was the availability of the sensor tied to funding for the study or did the company stop production due to unreliable measurement accuracy?

Unfortunately, the reason behind the availability of the sensor is too long a story to add to this paper. The wearable we used is reliable (Selvaraj et al., 2018). The distributor of the wearable (Philips) however had developed their own patch sensor and stopped the production of the old sensor (the one we used). Unfortunately, that new sensor turned out to be not reliable. The distributor then decided to stop selling these sensors all together and instead focused on a new type of sensor, leaving us without sensors. We have added a summary of the reason to the paper. Hopefully this provides enough clarification.

3. Methods: “Patients with a pacemaker did not receive a sensor since RR measurements might result unreliable in paced rhythms.” Please clarify: Were those patients completely excluded from the study or only RR measurements were excluded for those patients?

Since patients with a pacemaker did not receive a sensor, they had no available continuous data and therefore did not meet the inclusion criteria as described in the paragraph on ‘data collection’. We have added ‘sensor’ to the notion of ‘available continuous data’ to clarify that only patients who received a sensor could be included.

4. Methods: „Since our cohort included dying patients, we used wide limits for improbably 114 data (for RR <1/min & >80/min; for HR <30/min & >280/min; for sT < 25°C).” The lower limits for temperature and respiratory rate are extremely wide and should be critically revised.

Thank you for this feedback, we have discussed this extensively. The limits were chosen based on literature and clinical experience, but also on observations in the data. Especially in patients who were monitored while dying, it is hard to say which values are still valid and which are not. Therefore we have chosen to use wide limits, but also use additional methods to filter out possible artefacts (by deleting short lasting changes in HR and RR, and the ‘warming up’ period of skin temperature), and using a median filter of 15 minutes, before performing further analyses.

5. Methods: “Data was divided in five cohorts based on different stages of disease…. Hypoxic respiratory insufficiency was defined as the need for 15 l/min oxygen therapy.” Why did the authors not stratify according to the WHO criteria in mild, intermediate and severe COVID19. Please discuss and reference, if this method has been used before.

We did no use the WHO criteria for multiple reasons: first of all, patients with mild disease are not admitted to the hospital in the Netherlands, leaving us with only two categories. Secondly, the difference between moderate and severe disease according to the WHO guidelines is made based on respiratory rate (with a threshold of 30/min) and oxygen saturation (with a threshold of 90% on room air). Due to the nature of our data, which includes continuous respiratory rate values, we have observed that respiratory rate changes often and quickly. A patient could change between the moderate and severe group multiple times within one hour. Regarding the threshold for oxygen saturation, this threshold was made to use without room air. The vast majority of patients in our cohort however receive supplemental oxygen, so it would be almost impossible to decide in which category they belong. We could have used the situation at admission, however this does not reflect the aim of our study: to determine the change in circadian pattern during deterioration. The more conventional method to define deterioration would be intensive care admission. However, we do not believe that the time of admission to the intensive care adequately reflects the moment a patient becomes respiratory insufficient. A patient could have received 100% supplemental oxygen for hours without improvement of the situation, which unfortunately happened often during the covid-19 crisis. Furthermore, many patients had treatment restrictions, and 15L/min O2 was the highest level of care they were willing to receive or which was medically responsible.

Since we had such a specific aim and population, we decided to define a new endpoint. We hope this explanation clarifies our choice.

6. Methods: “For quantitative assessment we divided the data in daytime (06:00-00:00) and nighttime (00:00-06:00).” The daytime period is proportionally much longer than the nighttime. Please discuss and reference, why this method was chosen and if this method has been published before.

Thank you for this observation. Due to the new methodology, we have no longer defined daytime and nighttime.

7. Methods: Data collection included the Charlson Comorbidity Index, but the group differences were not further discussed within the manuscript. The use of further disease severity scores for ICU patients (SOFA, GCS) would complement the author’s analysis.

Thank you for this suggestion. Since our population is not an ICU population, but patients at the general ward, we have no routinely collected data on SOFA or GCS. Unfortunately, we also have not collected any other form of disease severity score. Instead we have tried to stratify our analysis based on disease severity: a cohort of patients who recovered without respiratory insufficiency, a cohort of patients who developed respiratory insufficiency, and a cohort of patients who died.

8. Statistics: Although the authors nicely removed several abrupt deviations before analysis, the use of PNex measurement might not be picking up circadian trends very well. It is correlated, however results are very noisy. For a better understanding the analysis needs to be discussed and compared with other methods applied in previous publications examining vital signs. Why did the authors not perform a regular rhythmicity analysis and sine curve fit to better estimate the amplitude?

Thank you very much for this helpful suggestion. Based on this comment and comments by the other reviewers, we have chosen to change our methodology altogether. We have fitted both a population mean cosinor model and a mixed-effect cosinor model to answer our different research questions. We hope our new methodology leads to better understanding.

9. Discussion: Studying a clinical cohort of Covid-19 patients exclusively, the authors conclude that a general knowledge of circadian patterns might improve general monitoring- and alarm strategies. This affirmation is not supported by data including other disease entities and therefore needs to be discussed including more recent literature and trials examining circadian patterns in clinical cohorts.

Daou M, Telias I, Younes M, Brochard L, Wilcox ME. Abnormal Sleep, Circadian Rhythm Disruption, and Delirium in the ICU: Are They Related? Front Neurol. 2020 Sep 18;11:549908. doi: 10.3389/fneur.2020.549908. PMID: 33071941

Lachmann G, Ananthasubramaniam B, Wünsch VA, Scherfig LM, von Haefen C, Knaak C, Edel A, Ehlen L, Koller B, Goldmann A, Herzel H, Kramer A, Spies C. Circadian rhythms in septic shock patients. Annals of Intensive Care. 2021 11: 64. PMID 33900485

Maas MB, Lizza BD, Abbott SM, Liotta EM, Gendy M, Eed J, Naidech AM, Reid KJ, Zee PC. Factors Disrupting Melatonin Secretion Rhythms During Critical Illness. Crit Care Med. 2020 Jun;48(6):854-861. PMID: 32317599

Maas MB, Iwanaszko M, Lizza BD, Reid KJ, Braun RI, Zee PC. Circadian Gene Expression Rhythms During Critical Illness. Crit Care Med. 2020 Dec;48(12):e1294-e1299. PMID: 33031153

Thank you very much for the suggested literature. We have used several to clarify and improve the discussion of our article. However, the suggested articles are all studies on critically ill patients in the intensive care unit. We feel strongly that the vital signs of critically ill patients at an intensive care unit cannot be compared to the vital signs of moderately ill patients at a low care ward, and that monitoring strategies for both wards are different, so we have avoided direct comparison. We did add several references of studies on general wards where wearables are used for continuous monitoring of covid-19 patients. Furthermore, we have highlighted why awareness of circadian patterns might aid interpretation of vital sign trends in clinical practice. The cited paper by van Rossum et. al. ((van Rossum et al., 2021) reference [24]), who found that the alarm strategy at a surgical ward could be improved by using different threshold for days and nights, underlines the potential of knowledge of circadian patterns to improve alarm strategies.

Reviewer #2

The main issue is that the current study design does not support an analysis of risk for hypoxemic respiratory insufficiency as the authors propose. Evaluating different time periods a single cohort of patients, comparing a pre-hypoxemic “control” period and post-hypoxemic “outcome” period in those who developed hypoxemic respiratory insufficiency (defined in this study as >15L O2 therapy), we are able to establish only that these measurements differ in these two periods of time within these patients. To evaluate for risk of hypoxemic respiratory insufficiency, the authors must include a study population that did not develop hypoxemic respiratory insufficiency. This could be done in a variety of ways to create either a cohort or a case-control study design. The analysis should be revised to include such a cohort, or the manuscript needs to be rewritten with this significant change in mind. It would also be helpful to include references using the methods employed (daytime peak vs. nighttime nadir of physiologic data) to evaluate circadian patterns in the data.

Thank you for pointing out this issue of the study design. Based on this comment, and comments made by the other reviewers, we have decided to apply a new analysis. We have included 3 cohorts: patients who recovered without experiencing respiratory insufficiency, patients who developed respiratory insufficiency and patients who died. We have not gone as far as to make a predictive model. We feel like this study is an explorative precursor study to see is a predictive model based on circadian pattern amplitude would be feasible. To better reflect this, we have adjusted the statement of objectives in the introduction.

Since we have abandoned the method of daytime peak and nighttime nadir, we have not included these references in the article.

Abstract

- Please define “delta day” as a measurement in the abstract.

- Please revise to more accurately describe the logistic regression results (that VS changes are associated with worsened hypoxemia within in this select cohort of patients; they do not demonstrate risk of developing hypoxemia – as above, this cannot be evaluated unless the study also includes patients who did not develop hypoxemic respiratory insufficiency.)

Since we have abandoned this method, the mentioned sentences are no longer part of the abstract.

Methods

- p5. Please clarify - if sT measurements were not directly available then how were they studied?

Skin temperature values were not available for clinical use. They were however stored for research purposes.

- Please include how many COVID19 patients were excluded for lack of sensor data.

We have added a flowchart of the patient inclusion (figure 1). We hope this will clarify the number of patients excluded during each stage.

- p.6. Please correct “improbably data.”

We have corrected improbably to improbable.

- The authors excluded patients with < 48 hours of continuous data during a three-day period, except for the post-hypoxemic respiratory failure period. This suggests that patients who developed hypoxemic respiratory failure very quickly (eg after 1-2 days) or who died shortly after admission without developing hypoxemic respiratory failure, etc. would have been excluded. This biases the study population to those patients who had a relatively slow progression of disease and may bias the study population away from those admitted to the ICU (it is unclear whether remote monitoring continued after ICU admission?). The authors should consider revising the analysis, using a shorter time window for each “period”.

Thank you for this helpful comment. We have redefined our cohorts, and have limited the amount of needed data to 4-hours. We believe this reduces the introduced bias. To be able to look at changes over time, we did include a three-day period of data.

- Consider including a flow diagram to describe the total number of potentially eligible patients, and those excluded at each stage (e.g. as suggested by the STROBE statement).

Thank you very much for this suggestion, we have included a flow chart (figure 1).

- Including patient data in more than one time-period “cohort” and then comparing differences between time periods is problematic from a study design perspective. This introduces bias towards the null – which makes it more difficult to identify true differences in vital signs between cohorts. The authors should consider revising their compared time periods so that no overlap occurs.

Thank you for this observation. As mentioned before, we have redefined the cohorts, limiting the introduced bias.

- How accurately was the onset of hypoxemic respiratory insufficiency defined? It would be helpful to describe how this information was obtained, e.g. if this was time stamped (hour and minute) to ensure that the vital sign data was assigned correctly as pre- vs. post-intervention.

The onset of respiratory insufficiency was defined as the first recording in the electronic patient record of either 15L/min oxygen therapy, high flow oxygen therapy, ventilation, or a cardiac arrest call. We have added this information to the data selection paragraph of the methods.

- p.7. Were peaks and nadirs drawn from raw data? Typically, circadian analyses using peak/nadir are drawn from a cosinor or sinusoidal analysis that accounts for the entire dataset shifting + and – in a circadian fashion, which is less sensitive to random noise (e.g., Shoben Am J Epi 2011 for an excellent example.) If no cosinor analysis is applied, thenit seems like this peak-nadir measurement (PNex) will only provide the range of daytime vs. nighttime data. Please include references for using this measurement in circadian analyses.

Thank you very much fort his comment. It has led us to revise our methodology, and we have now fitted a cosinor analysis (including references).

Results:

Thank you very much for the following comments regarding the results section. Since we have changed the methodology and therefore the results, these comments are no longer applicable. We have however tried to apply the feedback in general to the new result section.

- p. 7. Please rename the 5 time periods; they are not truly “cohorts” as they include the same patients, and the patient data is actually duplicated across some of these analytic groups. Suggest using “time periods” or something similar.

- How do more patients have respiratory insufficiency than deterioration? This might be clarified by using more distinct/separated definitions of time periods for comparison and/or a flow diagram as suggested above.

- Table 3 sugests that day 1 vs. 2 of respiratory insufficiency and day 1 vs 3 of respiratory insufficiency were compared in logisitc regression. This is different from the analyses described in the Methods. Please clarify.

-p.9; was mean PNex compared quantitatively between stages using a statistical test? If so, consider adding these comparisons to table 2.

-p.9, lines 189-191 and 191-192 should be edited for accuracy. Since patients without hypoxemic respiratory insufficiency were not included in this analysis, we cannot evaluate whether this development was associated with the development of hypoxemia. All we know is that the delta PNex was associated with skin temperature in this select cohort of patients who will go on to develop respiratory insufficiency.

-Table 3: Table 3 compares the difference in Pnex between days 1 and 2 and days 1 and 3. Please clarify – is this limited to stage III patients? If so, please add this detail and the number of patients included to the title.

Discussion:

- I agree that Figure 1 suggests qualitatively that there is circadian variability given the cyclical nature of overall VS pattern. However, it’s hard to claim definitively that there is no circadian variability based on a subjective assessment (eg for the patients with mortality.) From figure 1, stage I and stage III do not necessarily look all that different – it is not clear what the authors used as a cut off for circadian rhythm presence vs. absence. If a quantitative analysis is possible, one should be included. If not, the discussion should reflect this uncertainty throughout.

Thank you for this suggestion. We have added a quantitative analysis of rhythmicity to the methods of our study.

- The Discussion should be edited in several places to accurately reflect the results presented in Table 3. This analysis supports only an association between PNex and skin temperature, not an association with hypoxemic respiratory failure.

Since we have changed this analysis, these results are no longer part of the study.

- If the PNex is not a validated method of measuring circadian amplitude, then the related portions of the discussion should be revised accordingly.

Although other studies did use PNex to model circadian rhythms ((Davidson et al., 2021) reference [21]), we have decided not to use PNex as method but used a cosinor model instead. We have added to the discussion that the different methods might have different results.

Figure 1

- Please provide a legend for the green, red, and black tracings. The caption says that the Y axis is hours; is this correct? (It seems like the X axis is hours?)

Indeed, the X-axis was supposed to be hours, thank you for pointing this out. We have added the legend to the description of the figure.

Figure 2

- Consider defining in the methods that you plan to compare day 1, 2 and 3 within the 5 time windows. Were any quantitative analyses applied to these 1/2/3 day comparisons? Clarifying why it was included and what these results mean would be helpful.

Since we have changed this analysis, these comparisons are no longer part of the study.

Reviewer #3

*Major Comments*

1. We ask the authors to explain in more detail the rational for the definition of each predefined stages/cohorts? Why did the authors did not perform a longitudinal statistical analysis? It would be interesting to know if circadian patterns change over time depending on specific covariates (covariate analysis: e.g. respiratory insufficiency as a covariate). We suggest a separate statistical methods paragraph. Why are p-values not provided within the manuscript. A review by an independent statistician may be helpful.

Thank you for this valuable comment. We redefined the cohorts, and have paid attention to explaining why the new cohorts were chosen. Using the mixed-effect cosinor model, we have aimed to gain insight into changes of amplitude over time depending on the cohort (figure 3). Although this analysis does not provide prognostic information, we believe it does explore differences in circadian patterns over the days that might be used as input for prognostic studies.

2. The applied definition of _a circadian pattern_ should be explained in the manuscript. Sentences like _A circadian pattern was observed in HR and RR during stages I-IV._ are unclear. When should clinicians deem a pattern as a _normal circadian rhythm_? Did the authors look for indicators of circadian disturbances when analysing the data? A longitudinal analysis of patterns comparing patients with and without developing respiratory insufficiency would be very interesting for the readership.

Thank you for this feedback. We have chosen to change the method altogether, to avoid confusing and unclarity. We have added a quantitative measure of rhythmicity by doing a ‘goodness of fit’ test of the cosinor model. We have aimed to show longitudinal changes by using the mixed-effect cosinor model.

3. The results section is very hard to "digest"; especially for readers who are not experts in the s very new field of research. We strongly suggest to amend the results section with figures, which illustrate the main results more clearly.

Thank you very much for this suggestion. Since the methods have changed, the results section has changed too. We have added figure 3 and supplemental figure 1 to illustrate results.

4. The authors conclude that circadian patterns of heart rate, respiratory rate and skin temperature could improve monitoring- and alarm strategies. At this stage of the manuscript, we do not think that this conclusion is supported by the data. We agree, that including informations of circadian profiles within in prediction models and monitoring algorithms may be an interesting and helpful approach in the future. We ask the authors to provide 1 or 2 concrete examples or clinical scenarios how this could be helpful in clinical decision making. However, the authors should be clear, that this is not supported by their data and should remove statements from the conclusion section.

We agree that the improvement of monitoring- and alarm strategies was not the scope of this article and have removed this statement from the conclusion. In the paragraph ‘use in predictive modelling and clinical practice’ you will find a concrete example of how awareness of circadian patterns might influence clinical decision making.

5. Page 10, Line 204: „The mean circadian pattern amplitude showed differences between stages, but only an increasing PNex of skin temperature was associated with developing respiratory insufficiency and with only a small effect size." Please provide a P-value. Was the association statistical significant?

We have changed the analysis and provided p-values in the new result section.

*Further Comments*

- Page 3, Line 42-44: „Since September 2020, patients with covid-19 are treated with dexamethasone, which has a suppressive effect on the circadian pattern of the human metabolism." What does _suppressive effect on circadian pattern_ mean? Please explain.

We agree with you and reviewer 1 that this statement is unclear and incorrect. We have changed the sentence to ‘..dexamethasone, which can affect the circadian pattern of the human metabolism depending on time of administration’ to better reflect that dexamethasone can either suppress or amplify circadian patterns, depending on the parameter and the timing of administration.

- Page, Line 226: „The decrease of circadian rhythm might be due to severe illness and extreme physical stress, but could also have been influenced by age, comorbidities and medication. Furthermore, circadian rhythms are highly influenced by light input. As part of palliative care, patients were often relocated to single rooms with closed blinds for comfort. This might have played a role in the decrease of circadian pattern seen in all vital parameters.“ Please explain in more detail why circadian rhythm disturbances can occur in the context of severe diseases, e. g. closed eyes, reduced physical activity, inflammation etc.

Thank you for this suggestion. We have clarified these statements with details and added several references.

References

Davidson, S., Villarroel, M., Harford, M., Finnegan, E., Jorge, J., Young, D., Watkinson, P., & Tarassenko, L. (2021). Day-to-day progression of vital-sign circadian rhythms in the intensive care unit. Critical Care. https://doi.org/10.1186/s13054-021-03574-w

Selvaraj, N., Nallathambi, G., Moghadam, R., & Aga, A. (2018). Fully Disposable Wireless Patch Sensor for Continuous Remote Patient Monitoring. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. https://doi.org/10.1109/EMBC.2018.8512569

van Rossum, M. C., Vlaskamp, L. B., Posthuma, L. M., Visscher, M. J., Breteler, M. J. M., Hermens, H. J., Kalkman, C. J., & Preckel, B. (2021). Adaptive threshold-based alarm strategies for continuous vital signs monitoring. Journal of Clinical Monitoring and Computing. https://doi.org/10.1007/s10877-021-00666-4

Submitted filename: Response to comments - Circadian patterns in covid-19.docx

Click here for additional data file.

11 Apr 2022

19 Apr 2022

Response to reviewers

The authors have substantially revised their manuscript reflecting a complete revision of their analysis, now comparing circadian patterns in vital signs between three outcome-based groups: patients who recovered without respiratory insufficiency, those who developed respiratory insufficiency, and those who died. This analysis is much clearer and more readily interpreted. I have one remaining question regarding time frame definitions for analysis within each cohort, as well as some minor suggestions/clarifications.

Introduction

1. P.4 – This paragraph could likely be shortened without losing substantive information. For example, “Assuming a physiological difference in vital sign values between night and day might be closer to reality than assuming equal values throughout the entire 24-hour cycle” may no longer fit the analysis employed in this manuscript.

Thank you for the suggestion, we have critically reviewed the paragraph and have removed several sentences.

2. p.4 – minor detail, consider adding the 3rd cohort (patients who did not develop respiratory insufficiency) to the description of your 2nd research aim.

We have added the third cohort to the description of the research aim.

Methods

1. p.6 – Please clarify the sentence beginning “Patients were included if…”. The “respiratory insufficiency resp. death” is confusing.

We have clarified and elaborated on the choice of data for patients in each cohort.

2. Suggest moving the “4 hours of data” requirement to earlier in the paragraph – you restricted this analysis to patients who had at least 4 hours of (continuous?) data within the time frame of interest.

We decided to write down the methods in the same order as we analysed the data, to make reproduction of the study as easy as possible. Since we first selected the 72 hour timeframes, and later excluded those patients who appeared to have less than 4 hours of data within this timeframe, this is the chronological order of analysis.

3. Because you analyzed serial days within the 72h window per patient, did you require 4 hours of data per day?

No, we required 4 hours of data in total to include a patient in the population mean cosinor and rhythmicity test. For the stratification per day, a patient was only included in the day he/she had data during that day.

4. How were you able to do the cosinor modeling for patients with only 4h of data per day? It seems like this would require more data.

The cosinor model does not need a full cycle to estimate a cosinor curve. 4 hours of data, plus the indication where on the 24 hour cycle this data is located (the time) was enough to estimate the rest of the curve.

5. Choosing comparable time frames in these three outcome-defined cohorts is challenging. The authors have chosen three different time frames during the admission, in which the patient physiology could be expected to be very different – 72h after admission for patients who recovered, 72h before respiratory failure in those patients, and 72h before death in those patients. Because LOS in the different cohorts was quite different, to help understand the “comparability” of these patients, providing some information about where this window fell in the average length of stay for each cohort would be helpful. (For example – the 72h is the first 3 days out of an average 6 day LOS for the recovery cohort. But was it also on average the first 3 days for the respiratory failure cohort – e.g., when in the hospital stay did the respiratory failure occur?) This matters because patients who have been hospitalized longer have more opportunity for circadian disruption, and you may be finding changes in this cohort that are attributable solely to longer LOS/later time of evaluation and not attributable to clinical decline. If this is the case, this source of bias would need to be explicitly discussed in the discussion (and could be addressed in multiple locations in the discussion, as it may help explain the inconsistent findings between the respiratory failure and mortality cohorts, as well as the presence of respiratory and temperature variability only in the recovery cohort.) One could also consider trying to address this analytically, e.g. by matching recovery patients to patients in the other cohorts based on LOS, but this is probably not worth doing.

Thank you for your thorough consideration of the implications of timeframe selection. We share your opinion that the comparability of patients would be less if the timeframes were selected from different parts of the admission. Because respiratory insufficiency usually occurs within the first 72 hours of admission (median 33 hours), we chose the first 3 days of admission for the recovery (“control”) group too. We realise we failed to mention this in earlier versions of the manuscript, but we have now added it to the methods on page 6/7 of the manuscript.

Regarding the mortality cohort, mortality usually occurred later than 72 hours after admission. The comparability of this cohort with the other two cohorts could indeed be questioned. We have added this to the discussion on page 13 and to the limitations of the study on page 14, including the consideration to use a case-control design matched on length of hospitalization in future research.

Results

1. p.10 - Please clarify “stratified by day”. Do you mean stratified by cohort (as in the title of figure 3)?

Actually, figure 3 presents the cosinor characteristics stratified by cohort and by day. We have clarified this both in the manuscript and in the title of figure 3.

Minor comments

- Some editing for grammar would be helpful, e.g. p. 3 “circadian rhythm become increasingly more pronounced”, p.4 “three…research questions”, etc.

Thank you for your acuity. We have checked the entire manuscript and have corrected several grammatical mistakes.

Submitted filename: Rebuttal - Ciradian patterns in COVID-19.docx

Click here for additional data file.

22 Apr 2022

Circadian patterns of heart rate, respiratory rate and skin temperature in hospitalized COVID-19 patients

PONE-D-21-34906R2

Dear Dr. van Goor,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Henrik Oster, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

n/a

Reviewers' comments:

28 Jun 2022

PONE-D-21-34906R2

Circadian patterns of heart rate, respiratory rate and skin temperature in hospitalized COVID-19 patients

Dear Dr. van Goor:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Henrik Oster

Academic Editor

PLOS ONE