Curating evidence on mental health during COVID-19: A living systematic review.

The coronavirus disease (COVID-19) pandemic has disrupted the lives of people across the world by its rapid spread, high mortality, disruption of the social fabric, toll on health care systems, and devastating economic impact [1]. Fear of personal infection or infection of friends and family members is common among people exposed to any infectious disease outbreak [2]. For COVID-19, there are additional fears that health care systems may be overrun and that adequate medical care will not be available for all those affected; that isolation and movement restrictions will be long-lasting with a heavy toll on mental health and well-being, social functioning, and work; and that individual and societal economic resources will be insufficient or will not recover any time soon [3,4].

There will likely be serious mental health implications from the COVID-19 outbreak and that these could extend beyond the acute period of the outbreak for many people [2,5,6]. Adequately addressing mental health needs during and following COVID-19, as well as preparing for possible future outbreaks, requires an understanding of the nature and extent of mental health effects, factors associated with vulnerability to negative mental health outcomes, and evidence on the effectiveness of interventions that may be rapidly employed to prevent or address mental health concerns.

A February 2020 review [2] identified 24 studies from previous infectious disease outbreaks on psychological outcomes among people quarantined after being exposed to others who had been infected, including studies from severe acute respiratory syndrome in mainland China, Hong Kong, and Canada in 2003, equine influenza in Australia in 2007, H1N1 influenza in Australia in 2009, Ebola in West Africa in 2014, and Middle East Respiratory Syndrome (MERS) in Korea in 2015. There are important limitations, however, that reduce our ability to easily apply that evidence to decision-making in COVID-19; among them, (1) few studies used validated mental health outcome measures; (2) no studies compared outcomes during quarantine to pre-outbreak mental health data, which reduces the ability to draw conclusions about changes in mental health and associated factors; (3) and no trials tested interventions to improve mental health symptoms during or following infectious disease outbreaks. Furthermore, the scope of COVID-19 far exceeds that of other relatively recent outbreaks, and, subsequently, the overall threat it poses is much greater.

A major barrier to effectively using research on mental health during COVID-19, however, will likely be the large number and rapid publication of studies of variable quality, rather than a lack of evidence. As part of this, there will be challenges in rapidly separating informative evidence from evidence that may be less useful or misleading due to poor methodology, inadequate reporting, or both. A quick perusal of early journal and pre-print publications, trial registries, discussions with journal editors, and our own experience with peer review requests suggests that it will be crucial to be able to curate evidence rapidly, clearly delineate the kind of evidence that will answer pressing questions, and identify well-conducted and reported studies that can help us answer those questions.

To this end, in partnership with the Journal of Psychosomatic Research, we are launching a living systematic review [7] to evaluate (1) levels of mental health symptoms, prioritizing studies that assess changes in symptoms from pre-COVID-19 or compare concurrent samples between participants with different experiences with COVID-19 (e.g., those infected versus healthy comparison sample); (2) factors associated with levels or changes in symptoms during COVID-19, and (3) the effect of interventions on mental health symptoms during COVID-19.

Living systematic reviews [7] are systematic reviews that are continually updated and provide ongoing access to results via online publication. They are logistically challenging, but provide value beyond conventional systematic reviews in situations where (1) important decisions need to be made that merit the resources involved; (2) the certainty in existing evidence is low or very low, posing a barrier to decision-making; and (3) there is likely to be new research evidence emerging that would inform decisions [7]. This is precisely the scenario we face with COVID-19.

Our living systematic review has been registered in the PROSPERO prospective register of systematic reviews (CRD42020179703), and any changes to the study protocol will be registered as amendments with PROSPERO. A protocol has been uploaded to the Open Science Framework (https://osf.io/96csg/). Studies in any language that address review questions and meet inclusion criteria will be included. A comprehensive search strategy for the review was developed in English by a health sciences librarian and translated to Chinese by members of the research team. The search will be conducted in English-language databases (MEDLINE (Ovid), PsycINFO (Ovid), CINAHL, EMBASE (Ovid), Web of Science) and using both English and Chinese search terms in the China National Knowledge Infrastructure and Wanfang databases. Search alerts will be set in each database to send the research team daily updates when new results match search terms. Results will then be quickly uploaded onto a website dedicated to the project (https://www.depressd.ca/covid-19-mental-health). Our team will provide a narrative synthesis as evidence accumulates. As part of this, Dr. Sarah Markham, a member of the team and an experienced patient advisor [8] (diagnosed with anxiety and a depressive disorder) who survived a serious suicide attempt in 2007, will provide commentary from a patient perspective. In addition, experts in mental health research and care will contribute outside reviews and commentary, which will be made public, as the review progresses. Eventually, the review will undergo traditional academic peer review for publication in the Journal of Psychosomatic Research.

In evaluating evidence on the nature and level of mental health burden from COVID-19, we will prioritize studies that compare symptom levels or diagnoses among study participants during COVID-19 to pre-COVID-19 data. Evidence from cross-sectional studies that report percentages of participants with scores above cutoff thresholds on commonly used symptom questionnaires is sometimes considered. Conclusions that can be drawn from that type of data about mental health effects from COVID-19 and clinical implications, however, will be limited. This is because percentages of people who score above a threshold on standardized questionnaires vary, sometimes dramatically, between populations, even in normal times. For example, the percentage of participants with scores of at least 10 on the Patient-Health Questionnaire-9 [9], a commonly used measure of depressive symptoms, in large, randomly selected, regional or national general population samples, has been reported as 4% in Hong Kong (N = 6028) [10]; 6% in Germany (N = 5018) [11]; 7% in Shanghai, China (N = 1045) [12]; 8% in the United States (N = 10,257) [13]; 8% in the province of Alberta, Canada (N = 3304) [14]; 11% in Sweden (N = 3001) [15]; and 22% in Jiangsu Province, China (N = 8400) [16]. Even within populations from the same region, the percentage can vary dramatically depending on sample characteristics. In Jiangsu Province, for example, the percentage among rural residents (32%) is twice that of urban residents (16%); it is also several times higher for older adults (25% for 55–64 years; 87% for ≥65 years) than for young adults (8% for 18–34 years). Further complicating interpretation when there is not a time-based or other relevant comparator, percentages from symptom measures such as the PHQ-9 tend to dramatically overestimate prevalence that would be obtained from validated methods for ascertaining prevalence of mental health disorders, and there is too much heterogeneity between samples in the difference to correct for this statistically [17].

Ideally, investigators with pre-existing cohorts will be able to compare mental health symptoms prior to and during or after the COVID-19 outbreak. Other investigators who have recently completed relevant studies may be able to obtain permission to exceptionally contact study participants for follow-up assessments during or post-COVID-19.

For studies that assess factors associated with levels or changes in mental health symptoms, we will include only studies with multivariable analyses. Studies with cross-sectional and longitudinal designs will be eligible, although cross-sectional designs will only be included if factors assessed were present prior to the outcome (e.g., gender, pre-existing medical conditions). Studies with multivariable analyses that are predominated by concurrent mental health associations will be excluded, as these do not permit interpretation of directionality. In prioritizing evidence, studies with representative samples, sufficiently large sample sizes for precise estimates, an adequate number of participants per variable in models, and appropriate statistical methods will be emphasized.

We will track registrations and results from randomized trials and non-randomized controlled trials that evaluate the effects of any intervention designed to improve any aspect of mental health during the COVID-19 pandemic. As with any systematic review, evidence from trials that are registered and define outcomes prior to enrolling participants and that are well-conducted and reported will be emphasized.

For all of our review questions, evidence from studies that carefully characterize study populations and participants will be the most useful. In addition to characteristics that are expected to be reported in all studies, authors should provide details on the characteristics of the COVID-19 outbreak in the study locality during the time data were collected, considering issues such as the number of cases and deaths, the trajectory of both of these, and restrictions on social interaction and mobility. Characteristics of participants should, to the extent possible, describe the degree to which they have been affected by the outbreak in terms of their own health; the health of close relations; and their risk of infection due to reasons such as having a pre-existing medical condition or working as a health care provider for infected patients.

Our mental health research community has the important responsibility and the opportunity to dramatically expand our understanding of how large-scale health and other crises may influence mental health. The degree to which we are able to do this and our ability to effectively apply what we learn to preventive activities and to mental health intervention depends on the quality of the evidence we generate, our ability to identify the most informative studies, and how capably we use evidence from those studies to draw conclusions. We are hoping that our living systematic review will help towards that end.

Authors contributions

BDT, OB, DBR, JTB, MA, CH, SM, YS, YW, AK, AB, and IT-V contributed to the development of the living systematic review protocol. BDT drafted the editorial, and all other authors provided a critical review and approved the final version. BDT is the guarantor.

Funding statement

Ms. Rice was supported by a Canadian Institutes of Health Research Vanier Graduate Scholarship, and Dr. Wu was supported by Fonds de recherche du Québec – Santé Postdoctoral Training Fellowships. There was no funding for the project itself, and no sponsor or funder was involved in the study design; in the collection, analysis and interpretation of the data; in the writing of the report; or in the decision to submit the paper for publication.

Declaration of Competing Interest

All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare that they have no competing interests.