What happened to the predicted COVID-19-induced suicide epidemic, and why?

Two years ago, in the early stages of the COVID-19 pandemic, there were widespread and grim predictions of an ensuing suicide epidemic. Not only has this not happened but also by the end of 2021 in the majority of countries and regions with available data, the suicide rates had, if anything, declined. We discuss four reasons why the predictions of suicide models were exaggerated: (1) government intervention reduced the economic and mental costs of lockdowns, (2) the pandemic itself and lockdowns had less of an effect on mental health than assumed, (3) the evidence for a link between economic downturns, distress and suicide is weaker and less consistent than the models assumed and (4) predicting suicide is generally hard. Predictive models have an important place, but their strong modelling assumptions need to acknowledge the inherent high degree of uncertainty which has been further augmented by behavioural responses of pandemic management.

In the early 2020, as the COVID-19 pandemic raged across the Northern Hemisphere and threatened the entire world, researchers, advocates and modellers predicted an ensuing suicide epidemic (Bartone et al., 2020; McIntyre and Lee, 2020; Moutier, 2021; Reger et al., 2020), widely reported in the media (Benson, 2020). In Australia’s case, the predictions were terrifyingly grim: c. 1000 extra suicide deaths per year for 5 years in a population of 25.7 million (Road to Recovery BMC). To put this number into context, in Europe, with a population of 735 million in 2009, there were 4884 excess suicide deaths in the year following the Global Financial Crisis (Chang et al., 2013).

Despite these high-end predictions, by the end of 2020, there was no such epidemic (Pirkis et al., 2021). After 15 months, the suicide rates in most countries that produced data had remained stubbornly static and in some cases declined (Pirkis et al., 2022). In Victoria (which by the end of 2020 had endured the world’s second longest lockdown), New South Wales and Tasmania, the rates were either flat or fell. One exception is Queensland, where a small increase in suicides was observed (13%), primarily among young men (Pirkis et al., 2022). Presentations of deliberate self-harm (DSH) to health services have also been inconsistent with these predictions. The most recent numbers reported in the International Association for the Study of Pain’s (IASP)s living review showed sustained reductions in service utilisation for DSH up until the first half of 2021 (Steeg et al., 2022).

This good news seems to have gone under the media and advocacy radar. How might we account for this? To stimulate some scientific discourse about why this predicted epidemic has not manifested 2 years later, we present some reasons, with a particular focus on the pandemic’s health effects in Australia, and the knowns and unknowns that could have guided suicide prediction models.

The predicted increase in suicide assumed that the pandemic and its sequelae, in particular lockdowns, would increase a range of stressors associated with distress and/or suicidal behaviour. These were not unreasonable assumptions. By the end of April 2020, half of humanity was in some form of lockdown; almost 4 billion people across 90 countries were asked by their governments to stay at home (Sandford, 2020), a policy commonly predicted to have a major impact on mental health. This concern was supported by a systematic review of the effects of quarantine (from historic outbreaks) published in the early 2020, which demonstrated, albeit small, negative impacts of quarantine on mental health (Brooks et al., 2020). Subsequently, rises in population mental ill-health were, indeed, reported for countries following the start of the COVID-19 pandemic (Robinson et al., 2022).

The impact of lockdowns on mental ill-health and suicide was assumed to come through not only social disconnection and loneliness but also economic downturns. In some countries, suicide rates are observed to increase during economic downturns (Miller et al., 2009; Ruhm, 2000; Stevens et al., 2015). The Great Lockdown was predicted to be as destructive as the Great Depression (Gopinath, 2020). By April 2020, the International Monetary Fund (IMF, 2020) forecast a contraction of global output in 2020 by 3% and a spike in unemployment rates for the Organisation for Economic Co-operation and Development (OECD) countries most affected by the Great Lockdown. Among advanced economies, Australia, France, Germany, Italy and the United States were all flagged to be heavily affected by economic downturn. By April 2020, the United States recorded its highest spike in unemployment since the Great Depression (14.7%), and the IMF (2020) predicted a doubling of unemployment in Australia to 10%.

It was therefore reasonable to assume that lockdowns would cause unemployment, and that such stressors would increase mental distress, and therefore the risk of suicide. For instance, one of the models developed in Australia that predicted an increase in suicides by 13.7% between 2020 and 2025 included multiple variables associated with suicidal behaviour, but focussed predominately on unemployment, which the authors assumed to spike between 11.7% (best-case) and 17% (worst-case) (Atkinson et al., 2020). The influence of unemployment and other factors (domestic violence, substance abuse, homelessness) on suicide behaviour was then assumed to be mediated through a balance of psychological distress and mental health service use, meaning that these two factors, particularly a rise in distress, became paramount in explaining the predicted high suicide rates.

So, were these assumptions wrong? We suggest four explanations for why the suicide models may have exaggerated their predictions of suicide. First, the economic downturn expected from pandemic lockdown did not happen due to government intervention. Second, the mental ill-health crisis of the pandemic and lockdowns did not happen. Third, economic downturns – even if they had happened – may not cause suicide, or at least as much as predicted, after all. Fourth, suicide is an inherently difficult (stochastic) event to predict (Kessler et al., 2020).

Governments buffered the economic downturn and provided more health services

One crucial reason for why the suicide epidemic may not have happened is that governments counterbalanced the economic and mental costs of lockdowns by increasing public expenditure (Witteveen and Velthorst, 2020). In some countries, these resulted in sustained incomes and limited job dislocation, as seen in Australia in 2020, where the unemployment rate actually fell in 2020 as foreign workers left the country. However, in other countries, the unemployment rate rocketed, e.g. the United States, with claims going from a few hundred thousand to over 6 million per week (Tooze, 2021: 104–106) and widespread poverty, only partially ameliorated by a one-time direct cash payment of US$1200 per person (+US$500 per child). Yet the United States, like Australia, showed no increase in its suicide rates, although an increase in the ‘deaths of despair’ not seen elsewhere (Sterling and Platt, 2022), suggesting little consistency that could help predictions.

Furthermore, there was some limited expansion of mental health service provision and use in many countries, albeit again with high levels of variation across countries. In Australia, AU$76 million was provided in 2020 in the National Mental Health and Wellbeing Pandemic Response Plan (a 0.72% increase on a total 2018–2019 mental health expenditure of AU$10.6 billion). The Medicare telephone consultation policy response did enable psychological service provision to stabilise. Data from the Australian Institute of Health and Welfare (AIHW) show a continuation of the long-term trend of year-on-year increases in PBS mental health–related prescriptions. However, demand and waiting lists soared. Conversely in the United Kingdom, also with a decline in suicide rates, access to Improved Access to Psychological Therapies (IAPT; their version of Better Access) (Bauer-Staeb et al., 2021) fell by over 50% in the first few months of the 2020 lockdown, as did both referrals and acute adult admissions to secondary mental health services (Bakolis et al., 2021).

Determining the causal role of the various government economic and health service interventions on the observed suicide rate is fraught with problems, as the timing and severity of sequelae varied enormously from country to country and state to state, with the only constant being the threat of COVID-19. The health service and economic responses differed dramatically between countries where the suicide rates remained constant, suggesting these factors provide little or no consistent rationale. Confirming this lack of explanatory power, the most recent meta-analysis from the International COVID-19 Suicide Prevention Research Collaboration (ICSPRC) showed that there was no association of the COVID-19 mortality rate, stringency of public health response, level of economic support or presence of a national suicide prevention strategy with any changes in suicide after the onset of the pandemic (Pirkis et al., 2022).

People were not as distressed by the pandemic and lockdown as predicted

Another reason for lower suicide rates than predicted could have been that people were more resilient in their mental health responses to the lockdowns than predicted by the model. Indeed, in some countries, positive changes in mental health were observed during the earlier lockdowns as was found for the United Kingdom (Fancourt et al., 2021). Mental health in the United States recovered within 4 months (i.e. by June 2020) (Daly and Robinson, 2021). A subsequent meta-analysis of all of the global studies of the mental health pandemic lockdown demonstrated only a very small and transient (effect size of 0.17) increase in the rates of mental ill-health (Prati and Mancini, 2021), some of which could also be attributed to other confounders, e.g. fear of the virus, and no reduction in positive psychological functioning.

Perhaps the most convincing and rigorous evidence for this hypothesis comes from Australia, where a natural experiment occurred because of the way one state, Victoria, was locked down for up to 111 days, while other states were kept open. Exploiting this natural experiment, Butterworth et al. (2022) showed that lockdown led only to a small increase in mental ill-health (Mental Health Index-5 [MHI-5]) on average, but that mental ill-health effects were observed for mothers of young dependent children. This may have been because of the additional pressure imposed on mothers due to home-schooling for prolonged periods of time. We have confirmed that the decline in mental health and loneliness of this group was notable, despite an increase in feelings of safety and doing more exercise (Schurer et al., 2022). Thus, the mental health effects were concentrated among women with child-caring duties, among whom suicide rates are very low, limiting any absolute population level signal in suicide rates even if there may have been a relative risk increase (AIHW, 2022).

So, is the best explanation for why the suicide models got it wrong simply that people were not as distressed by lockdown as imagined? This post hoc speculation is supported by the observation that suicide and self-harm did not increase even among minorities who felt the burden of lockdown more than anyone else (mothers, young adults, etc.). This confirms a systematic review of the effect of previous epidemics by Rogers et al. (2021), showing a quite limited effect on suicidal thoughts or behaviours. Other factors are likely at play here. For instance, as any clinician on call will know, suicidal behaviour is commonly associated with acute intoxication (Chong et al., 2020) and the lockdown’s prevention of socialising in pubs and clubs may have had a protective effect, even if male alcohol use increased slightly overall during lockdown (Schurer et al., 2022).

Economic downturn may not cause suicide after all

Another potential cause of the overprediction of suicide may stem from the fact that model assumptions exaggerated not only the spikes in unemployment but also the relationship between economic downturns and suicide. The two major assumptions of the suicide predictions were that (1) there is a strong temporal association between unemployment and suicide (Ando and Furuichi, 2022; McIntyre and Lee, 2020) and (2) that this is mediated by an increase in psychological distress.

Evidence from the late 1990s South East Asia financial crisis showed variable impacts of recessions on the suicide rate, with increases in Japan and Korea but no change in Taiwan (Gunnell and Chang, 2016). A recent analysis of Australia’s unemployment–suicide link utilising administrative time series data for 40 years from 1979 to 2017 showed no relationship between unemployment and mortality on average and, if anything, a slightly lower level of suicide deaths during periods of higher unemployment (Atalay et al., 2021). Although evidence on the unemployment–suicide nexus exists for the United States (Miller et al., 2009; Ruhm, 2000; Stevens et al., 2015), no such link is found in Asia-Pacific countries (Lin, 2009), France (Brüning and Thuilliez 2019) and studies of the OECD countries overall have resulted in conflicting results (Gerdtham and Ruhm, 2006), with the best fit models suggesting suicide rises precede increases in unemployment (Nordt et al., 2015). In reviews and meta-analyses (e.g. Huikari & Korhonen, 2021; Mathieu et al., 2022), there are studies showing results inconsistent with the direction of the point estimate. In Germany (Neumayer, 2004), as in Australia (Atalay et al., 2021), suicides decline during spells of higher unemployment. As such, this fundamental assumption of the models has inconsistent evidential support, with recent contradictory data from several countries with very diverse socioeconomic contexts including South Africa (Phiri and Mukuku, 2020), and India (Arya et al., 2018), as well as Australia.

The second major assumption of the models is that population levels of mental ill-health are causally and temporally associated with suicide rates, i.e. suicide is viewed as the end of the distribution of distress with an inevitable rise as the bell curve of distress shifts to the right. It then follows that trends in one will be mirrored in trends in the others. Over the past 5 or 10 years, we have seen a decoupling of these time trends in Australia, with some increases in mental ill-health (Burns et al., 2020; Butterworth et al., 2020) while the suicide rate has fluctuated little and usually within the margin of error (AIHW, 2022). As such, any model’s assumption that changes in the population levels of distress would be associated with suicide rates is circumspect.

Suicide modelling is inherently very inaccurate

All predictive models in behavioural sciences are wrong to some extent, due to the impossibility of accurately predicting the future of complex human behaviour, but some are useful. A recent systematic review of 64 unique mathematical prediction models found a predictive validity below 0.01 for suicide mortality in most models (Belsher et al., 2019). A meta-review of over 25,000 social psychology studies with over 8 million subjects found correlation coefficients between risks and outcomes of 0.21, insufficient for any predictive accuracies with model predictive abilities not much better than chance (Richard et al., 2003). Furthermore, what is commonly reported is the point estimate without presenting the degree of confidence that we have of this point estimate. In the above review, the standard deviations of the correlations were almost as great as the estimate at 0.15.

Prediction models have different ways of dealing with uncertainty. Dynamic systems models, for instance, are subject to structural error propagation, whereby each of the individual domains that feeds into a further domain produces a point estimate and error, but these errors become compounded as the model becomes more complex, potentially leading to wide confidence intervals by the time the final point estimate of a prediction is derived (Engelhardt et al., 2016). Presenting the credible intervals around the forecast estimate would enable us to evaluate how confident we can be of the inferences. Other predictive models (e.g. machine learning, Bayesian approaches) have different problems.

This is not unique to mental health by any stretch of the imagination. As John Ioannidis and University of Sydney colleagues have shown, much of the COVID-19 forecasting modelling failed due to similar issues with poor data inputs, incorrect modelling assumptions, high sensitivity of estimates, lack of incorporation of appropriate epidemiological features, poor past evidence of the effects of available interventions and a lack of transparency, as well as consideration of only one or a few dimensions of the problem, compounded by group think and bandwagon effects (Ioannidis et al., 2020). Their team further showed that estimating the impact of behavioural interventions for limiting COVID-19 was more dependent upon the type of model used than the data that were used in the model (Chin et al., 2021). And this is an area where the data are far more robust and available in real-time than those struggling to provide mental health predictions.

Where to from here?

These concerns, by no means, are meant to undermine the use of statistical modelling to better understand mental health and suicide or that we should revert to expert ‘opinion’. Recchia et al. (2021) showed quite comprehensively how statistically informed predictions substantially out-perform expert opinion, which, in turn, out-perform lay opinion, a point made repeatedly by the Nobel prize winning economist Daniel Kahneman in his recent book, Noise (Kahneman et al., 2021). One of their most important points is that prediction models should not be static but continually informed by the changing context, re-evaluating assumptions and utilising new data. Although the prediction of greatly increased suicide rates proved incorrect, the models might have actually been completely accurate; it was just that the assumptions and parameters put into the model need revising with our subsequent knowledge about, e.g. what really happened to our unemployment rate, income and the impact of lockdowns. All too often, like fortune tellers, many of us who work in prediction modelling make predictions and then do not evaluate whether these predictions were actually accurate or if they were inaccurate to update them. This may be because of interventions, but the effect of these can be incorporated into the model validations. One argument from mental health advocates may be that it does not actually matter if the predictions were inaccurate as they spurred government intervention and spending in our Cinderella area. We would argue that as scientists this is not good enough, particularly when such prediction models and computer simulations are being promoted as one of the top 10 major advances in mental health science (Occhipinti and Skinner, 2021). Some estimate of the model dependence on the various assumptions should be very explicit in order to qualify as a ‘scientific’ prediction, e.g. what would have been the predicted suicide rate if unemployment rose by only 2% or, as in the case of Australia, fallen. We need an honest evaluation of the assumptions and performance of the models currently used to inform policy, and also to consider how the findings from other models utilising different methods, e.g. machine learning or Bayesian approaches can either be triangulated or incorporated. For instance, we should be estimating the uncertainty of our posterior evidence and finding ways of communicating this to policymakers and the public. In other disciplines (e.g. economics) or other areas of medicine (e.g. infectious disease), extraordinary statements require extraordinary evidence derived from alternative statistical models and assumptions, data sets and research teams. We too should not be reliant on just one modelling approach from one team if interest lies in producing policy-relevant mental health and suicide predictions.

Summary

Suicide is a highly complex behaviour, and even sophisticated models appear very limited in predicting it. In the case of COVID-19 and its sequelae for each potential explanation, we have outlined how the predicted effects of the pandemic varied by context and setting yet the suicide rates declined or remained static almost everywhere, Japan being a notable exception, suggesting none of these sequelae are either sufficient, or even necessary, to understanding what, if any, impact COVID-19 has had on suicide rates. More likely, this area is so complex with so many interactions that making confident and accurate predictions is possibly futile. Daniel Kahneman puts it best (Noise, pp. 372): People cannot be faulted for failing to predict the unpredictable but they can be blamed for a lack of predictive humility. The limitations and uncertainty of our assumptions and predictions need to be acknowledged.