Probable depression and anxiety in seven European countries during the COVID-19: Probable overestimation of the problem due to sampling method.

Dear editor,

We read with interest the article by Hajek et al. (2022) which screened for mental health issues by self-rating instruments, showing the prevalence of probable depression and probable anxiety and determined their correlates during the Covid-19 pandemic in seven European countries (ECOS study). Probable depression and anxiety were quantified using the established and validated scales for the brief screening of negative affective states (PHQ-2 and GAD-2; Kroenke et al., 2009). The authors found that in three observations (4th, 5th, and 6th wave of the Covid-19 pandemic during November 2020, January 2021, and April 2021, respectively) a very high number of adult participants had probable depression (26.6%, 25.5%, and 23.8%, respectively) and probable anxiety (25.7%, 23.6%, and 22.1%, respectively). This screening survey clearly stated that studies based on clinical interviews for DSM-5 disorders would be desirable to validate given findings.

Diverse socioeconomic, demographic, and ethnic factors, mental health literacy, and the methodological issues (screening instruments vs. clinical diagnostic interviews, accuracy of the measurements, sampling procedures), could contribute to the differences in the prevalence of negative affective states across the world (Hajek et al., 2022; Wang et al., 2021) and we commend Hajek et al. (2022) for addressing this pressing topic. However, to adequately estimate prevalence rates in a population the sampling method is of crucial importance (Pierce et al., 2020). Thus, we would focus on it and call into question the generalizability of the results by Hajek et al. (2022) regarding probable depression and anxiety for the general populations of the aforementioned countries. We propose that probable depression and anxiety estimates in their study might be substantially inflated by the sampling method.

Namely, Hajek and colleagues conducted an online survey in which data were collected from about 1000 adult individuals in each of the participating countries via the market research company (7000 participants in total), and used several recruiting techniques (open recruitment, affiliate networks, mobile apps, or loyalty programs) to reach the general population. Representativeness of the samples was accomplished using quotas based on gender, region, education, and age in each country. However, quota sampling used in this study - violating the principle of random selection, i.e. equal chances of all those belonging to a particular quota to be selected - is associated with various threats to the representativeness, such as: a) its non-probabilistic nature precludes determination of sampling error leading to possible biased estimates of population parameters; b) a probability of selection bias precluding generalization to the population; and c) an unrealized bias introduced by a selection of traits based on which the quotas are formed (Sharma, 2017).

Researchers could weight their sample to match the target population but the adjustments could be missing elements of bias, particularly if the response rate is unknown. As shown, the use of weights can correct for proportionality but not the representativeness whenever demographic weighting variables are not associated with the main research variables (Haddad et al., 2022). Given the description of the sampling procedure in ECOS study, one is left wondering if, apart from quotas, the authors used any additional adjustments, which could have led to the imprecise estimates of probable depression and anxiety.

Recently, we collected a sample of individuals representative of the Serbian adult population (N = 1203) who were interviewed in person (CoV2Soul.rs study). We used multistage probabilistic household sampling and respondents were recruited in 135 randomly selected local communities in 60 out of the 180 municipalities in Serbia. The data collection took place between June and October 2021, the response rate was 67%, and we screened for probable depression and anxiety using the same instruments as in the abovementioned studies (for more details see Maric et al., 2021a, Maric et al., 2022). In our sample, the rate of probable depression (PHQ-2 ≥ 3) and probable anxiety (GAD-2 ≥ 3) was 6.3% and 5.5%, respectively (unpublished data, but data set is available at OSF: https://osf.io/f8sje/; PHQ-4 Cronbach alpha = 0.84).

When the rates of probable depression were explored using the longer version of the PHQ before the pandemic (2013–2015) in all countries involved in the ECOS study by the EHIS study (Hapke et al., 2019), a two-stage stratified cluster sampling approach was used. The data were collected in person. At that time, the average rates of probable depression (PHQ-8 ≥ 10) in the general population of Denmark, France, Germany, Italy, Portugal, and the UK were: 7.4; 7.2; 9.2; 4.6; 9.1 and 7.4%, respectively. Roughly speaking, ECOS study (Hajek et al., 2022) suggests 3–4 fold increase of probable depression during the pandemic. In 2013 EHIS study was conducted in Serbia, too (Boričić et al., 2013) and 3% of the adult general population had probable depression (PHQ-8 ≥ 10). Relying on the same instrument and on the same sampling method of the adult general population in mid-2021 we found 5.7% cases with PHQ-8 ≥ 10. This suggests that the rate of the negative affective states in the population of Serbia almost doubled in the second year of Covid-19 pandemic (Marić et al., 2021).

Even a two-fold increase becomes very challenging for mental health services when transformed into absolute numbers. A three or four-fold increase, indirectly suggested by ECOS study, could be a mental health service disaster. Is it happening? We believe that the negative affective states prevalence rate changes in reality are not that dramatic and that the ECOS findings could be, at least partially, explained by elements of the sampling bias.

Hajek and colleagues tackled an important topic which warrants investigation. However, many authors have been emphasizing that really good mental health epidemiology is hard to do. As Pierce et al. (2020) suggested at the beginning of the pandemic, transparency in methodology is fundamental, and all surveys (including screening surveys) would describe in detail their sampling strategy and show comparative statistics with the population they are sampled from. This is how informed conclusions are made about representativeness. Collecting data from representative samples is always challenging, especially during pandemics. Regardless, transparency in methodology and careful consideration of the results is fundamental to minimize potentially misleading results.

Role of funding source

This work was supported by the Science Fund of the Republic of Serbia, grant number #7528289. The special research program on Covid-19 is financed by a World Bank loan through Serbia Accelerating Innovation and Entrepreneurship Project – SAIGE.

CRediT authorship contribution statement

All authors - conceptualization

NM - writing the first draft

LJM, LL, GK directly accessed and verified the underlying data reported in the manuscript

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.