Effect of lockdown on mental health in Australia: evidence from a natural experiment analysing a longitudinal probability sample survey.

BACKGROUND: Many studies have examined population mental health during the COVID-19 pandemic but have been unable to isolate the direct effect of lockdowns. The aim of this study was to examine changes in the mental health of Australians aged 15 years and older during the COVID-19 pandemic using a quasi-experimental design to disentangle the lockdown effect.
METHODS: We analysed data from ten annual waves (2011-20) of the longitudinal Household, Income and Labour Dynamics in Australia (HILDA) Survey to identify changes in the mental health of respondents from the pre-COVID-19 period (2011-19) to the COVID-19 period (2020). Difference-in-differences models were used to compare these changes between respondents in the state of Victoria who were exposed to lockdown at the time of the 2020 interviews (treatment group) and respondents living elsewhere in Australia (who were living relatively free of restrictions; control group). The models included state, year (survey wave), and person-specific fixed effects. Mental health was assessed using the five-item Mental Health Inventory (MHI-5), which was included in the self-complete questionnaire administered during the survey.
FINDINGS: The analysis sample comprised 151 583 observations obtained from 20 839 individuals from 2011 to 2020. The treatment group included 3568 individuals with a total of 37 578 observations (34 010 in the pre-COVID-19 and 3568 in the COVID-19 period), and the control group included 17 271 individuals with 114 005 observations (102 867 in the pre-COVID-19 and 11 138 in the COVID-19 period). Mean MHI-5 scores did not differ between the treatment group (72·9 points [95% CI 72·8-73·2]) and control group (73·2 points [73·1-73·3]) in the pre-COVID-19 period. In the COVID-19 period, decreased mean scores were seen in both the treatment group (69·6 points [69·0-70·2]) and control group (70·8 points [70·5-71·2]). Difference-in-differences estimation showed a small but statistically significant effect of lockdown on MHI-5 scores, with greater decline for residents of Victoria in 2020 than for those in the rest of Australia (difference -1·4 points [95% CI -1·7 to -1·2]). Stratified analyses showed that this lockdown effect was larger for females (-2·2 points [-2·6 to -1·7]) than for males (-0·6 [-0·8 to -0·5]), and even larger for women in couples with children younger than 15 years (-4·4 points [-5·0 to -3·8]), and for females who lived in flats or apartments (-4·1 points [-5·4 to -2·8]) or semi-detached houses, terraced houses, or townhouses (-4·8 points [-6·4 to -3·2]).
INTERPRETATION: The imposition of lockdowns was associated with a modest negative change in overall population mental health. The results suggest that the mental health effects of lockdowns differ by population subgroups and for some might have exaggerated existing inequalities in mental health. Although lockdowns have been an important public health tool in suppressing community transmission of COVID-19, more research is needed into the potential psychosocial impacts of such interventions to inform their future use. FUNDING: US National Institutes of Health.

Introduction

The adverse mental health consequences of COVID-19 were recognised early in the pandemic,1, 2, 3 with research focused on both the direct effects of COVID-19 (eg, fear of catching the virus1, 4 and the psychiatric sequelae of infection) and the indirect effects of the policies used to contain the spread of the virus. Although there are limitations to much of the mental health research conducted during the pandemic (eg, use of non-probability samples and absence of comparable pre-pandemic measures of mental health),6, 7 robust evidence documenting negative effects has been derived from representative longitudinal studies involving fieldwork before and then again during the COVID-19 pandemic. These studies, however, are unable to disentangle the indirect effects of the policy response (lockdowns) from the direct effects of the pandemic.

A few studies have examined the effect of lockdowns using quasi-experimental methods.9, 10, 11 This approach not only considers mental health at different points in time, but contrasts the effect of time between different groups exposed to different policy settings. Federal systems of government, in which states or provinces have policy autonomy, can generate the type of exogenous variation in policy needed for such natural experiments. However, attempts to evaluate the mental health effects of lockdown using these methods have been problematic, either not directly testing the interaction term that evaluates the difference (between states) in the difference (over time), or examining a proxy measure of mental health (eg, calls to a telephone crisis line). Not subject to these problems is a study of the lifting, rather than introduction, of lockdown restrictions in the UK, which capitalised on the fact that England lifted lockdown restrictions earlier than Scotland. Nevertheless, this experiment was relatively weak, with just a 2-week difference in the lifting of restrictions, and was probably affected by anticipation effects.

Evidence before this study

Although lockdowns have proven an important public health tool for reducing the transmission of COVID-19 during the pandemic, concerns about their potential impact on population mental health have also been raised. However, to date no study has been able to robustly quantify the association of lockdown measures with public mental health. We did a literature search using both Google Scholar and PubMed for all research output (including preprints) available between Jan 1, 2020, and Oct 15, 2021, with no language restrictions, using the following terms: “COVID-19” or “coronavirus” with “mental health”; and “lockdowns and mental health”. Many studies of changes in mental outcome measures following the COVID-19 outbreak were found, as well as one meta-analysis; however, with only one exception, none of these studies disentangle the effect of lockdowns. Furthermore, the one exception described an “experiment” that involved a very small difference between treated and control groups. Anticipation effects also probably meant that observed differences understated true differences.

Added value of this study

This quasi-experimental study is the first to provide robust evidence for the independent association of lockdown with mental health outcomes. Furthermore, the study identified how the magnitudes of these associations varied across different demographic and socioeconomic groups. The results show that, although the overall population effects were small, there were much larger effects for certain subgroups. However, the groups most affected by lockdown were not those often thought to be at greatest risk of psychological distress (eg, not adolescents, single mothers, or low-income households).

Implications of all the available evidence

Although COVID-19 has been shown to increase psychological distress, who bears the burden of that increased distress might differ depending on whether the source of the distress is exposure to and fear of the disease or a by-product of policies designed to contain the spread of the virus. If lockdowns and other policies intended to restrict population movement are to remain part of the policy toolkit for responding to pandemics, more attention needs to be given to providing support to alleviate the potential negative side-effects of control measures; for example, by focusing on equitably delivering childcare services and schooling. Research is also needed into the potential modifying role that environmental factors, such as housing and access to outdoor space, might have on the psychological effects of lockdowns.

Our use of data from Australia provides a more compelling context for a quasi-experimental approach estimating the mental health effects of lockdowns. First, Australia's federal system means lockdown decisions were made independently by the government of each state or territory. Second, Australia's national strategy of so-called aggressive suppression involved the early application of strict lockdown restrictions when community transmission was observed.13, 14 As a consequence, the nexus between the introduction of lockdowns and very high community rates of COVID-19 cases was weaker in Australia than in most other countries. The third reason is the geographical specificity of the lockdown. After a nationwide lockdown during the initial wave of COVID-19 (in March and April, 2020), restrictions were eased and COVID-19 was largely controlled through international border closures and quarantine. However, community transmission of COVID-19 was detected in the state of Victoria in July, 2020, following quarantine breaches. At this time, a series of lockdown measures, including business closures, stay-at-home orders, remote schooling, and evening curfews, were introduced exclusively in Victoria. The lockdown successfully suppressed community transmission and, after peaking with 725 cases on Aug 5, 2020, new daily case numbers in Victoria gradually declined and lockdown restrictions were lifted in late October, 2020. Fourth, the opportunity to apply quasi-experimental methods is often dependent on coincidence: the timing of a routine data collection aligning with an exogenous event. Our study reflects this coincidence of timing, with the planned annual fieldwork for the 20th wave of Australia's major household panel survey commencing just as the state of Victoria entered this second lockdown.

To date, two other Australian studies have taken advantage of the second wave in Victoria to investigate the mental health consequences of lockdown.15, 16 However, these studies each had significant methodological limitations, including the absence of any pre-COVID data, not following the same individuals over time, using convenience rather than probabilistic sampling methods, or employing samples restricted to Victoria and thus without an untreated control group for comparison.

To advance our understanding of the mental health consequences of lockdown, we analysed data from the 20th annual release of Australia's household panel study. We tested the hypothesis that, relative to pre-pandemic levels, the mental health of individuals in the state of Victoria during lockdown (the treatment group) worsened more than those in the remainder of Australia (the control group). We also assessed whether the mental health effects of lockdowns differed by respondent characteristics. On the basis of previous COVID-19 research, we considered age and gender, income,17, 18 pre-COVID-19 health, family and living arrangements,18, 20 and urban versus rural location.19, 21

Methods

Study design, participants, and collection procedures

The data came from the Household, Income and Labour Dynamics in Australia (HILDA) Survey, a household panel survey that commenced in 2001. After weighting for initial non-response, this survey provides a nationally representative sample of Australian households, albeit excluding very remote Australia and those in non-private dwellings. The first wave comprised 13 969 participants from 7682 households. Interviews were then sought, on an annual basis, with all members of participating households who were aged 15 years or older, along with any individuals who subsequently joined a household in which an original household member resided. A top-up sample, providing another 2153 households, was added in 2011. Rates of sample loss and attrition were low, with the re-interview rate rising from 87% in wave 2 to over 95% by wave 8 and remaining above that level in subsequent waves.

Data collection occurred between August and February, with over 90% of interviews in wave 20 (n=13 758) completed before the end of October. Before wave 20, more than 90% of interviews were done face-to-face. In wave 20, because of the pandemic, the fieldwork shifted to telephone. Respondents also completed a separate self-completion questionnaire (SCQ). Over the period covered by this analysis, the return rate for the SCQ averaged 90%.

The analysis used data from the ten waves spanning the period from 2011 to 2020 (appendix p 3). Observations with missing information on the outcome variable (mainly due to non-return of the SCQ) and from those no longer part of the in-scope population (moved to a very remote region or non-private dwelling) were excluded. The sample was also limited to respondents who provided data during 2020 and on at least one previous occasion. An analysis of selected respondent characteristics revealed only minor differences between this analytical sample and the larger sample (appendix p 6). Furthermore, these differences were largely eliminated after the application of the appropriate population weight.

Individuals gave oral informed consent for participation in the study. Additionally, consent was sought from parents or guardians before seeking the involvement of household members younger than 18 years. Ethics approval was granted by the Human Research Ethics Committee of the University of Melbourne in 2001 and has been updated or renewed on annual basis since (ID number 1955879).

Measures

The outcome measure was the five-item Mental Health Inventory (MHI-5), a subscale of the 36-item Short Form Survey, which has been administered in every wave of the HILDA Survey as part of the SCQ. The five items assess the frequency of symptoms of anxiety and mood over a 4-week period. Scores on each item were summed and scale values transformed to range from 0 to 100, with low values indicative of poor mental health. The MHI-5 is an effective screening instrument for people with mental health problems25, 26, 27 and is a much validated and widely used measure of population mental health.

The treatment variable identified respondents in Victoria who completed the SCQ in the second half of 2020 during the lockdown or within 2 weeks of the lockdown being lifted. This group covered all people living in metropolitan Melbourne who completed the SCQ on or before Nov 11, 2020, and all people living in regional Victoria who completed the SCQ on or before Oct 1, 2020.

Statistical analysis

Sample statistics are presented with weighted estimates of mean MHI-5 scores, overall and by the covariates considered in the analysis, and over time for both the treatment and control groups. To estimate the effects of lockdown on mental health, we used difference-in-differences (DiD) estimation to compare the MHI-5 scores of people in the treatment group (most Victorians) with those in the control group (people living in the rest of Australia) before and during the lockdown. As is standard in the literature, DiD estimation included state and year (survey wave) fixed effects. Capitalising on the longitudinal nature of the data, the analysis also incorporated person-specific fixed effects, which, although not necessary for isolating the causal impact of the lockdown, help to absorb residual variation in the MHI-5 score. Estimation was done using reghdfe, an add-on to the Stata econometric package (version 15) that runs regressions with many levels of fixed effects. SEs were adjusted for heteroskedasticity and clustered at the state level. Data were weighted using SCQ respondent weights, which adjust estimates in a way that accounts for the initial complex survey design as well as non-random response and attrition (including non-return of the SCQs). Key analyses were repeated using a binary measure of probable mental disorder (MHI-5 score <52). For ease of interpretation, these models were estimated using a linear probability approach.

Reliable DiD estimation was possible because decisions about lockdown differed between states. The crucial modelling assumption was that individuals' mental health developed in the same way in treated and control states before lockdown and would have continued on similar trajectories in the absence of the lockdown. The attribution of the treatment effect to lockdown, rather than broader COVID-19 health effects, relied on the assumption that Victoria's lockdown came into effect before community transmission exposed Victorians to SARS-CoV-2 and related health problems. These assumptions were tested first by examining state differences in mental health trends before 2020, and second by examining differences in participants' reports of having received a COVID-19 diagnosis or reports of concern about their likelihood of serious COVID-19 illness if infected.

The analysis also examined how the size of the estimated treatment effect varied with selected personal and household characteristics. DiD models were stratified by the following characteristics: gender, age (15–19, 20–29, 30–54, 55–69, and ≥70 years), family structure (couple aged <65 years without children aged <15 years, couple aged 65 years with children aged <15 years, single parent with children aged <15 years, single person aged <65 years, couple aged ≥65 years [including at least one partner aged ≥65 years], single person aged ≥65 years, and older dependent child [aged 15–24 years]), dwelling type (separate [detached] house; semi-detached house, terraced house, or townhouse; flat or apartment; and other housing types), country of birth (Australia, foreign mainly English-speaking country, and other foreign country), region of residence (major city, inner regional, and outer regional or remote), presence of a long-term health condition, poor mental health (MHI-5 score <52) in 2019, and equivalised household disposable income quintile (measured over the financial year preceding the lockdown). An analysis by race or ethnicity was not possible because data on these characteristics were not collected in the survey. Statistical tests of the equality of coefficients for the levels of each characteristic were also done.

We conducted sensitivity analyses on the DiD model using alternative treatment periods and sample restrictions and allowing for state-specific year fixed effects. Results from these analyses (along with other supporting material, including a description of the statistical model) are reported in the appendix.

Role of the funding source

The funder of this study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Results

174 932 observations from 25 520 individuals were obtained from the HILDA Survey from 2011 to 2020 (appendix p 3). After exclusion of observations with missing outcome data (18 480 observations), observations from people no longer part of the in-scope population (1643 observations), and observations from respondents without data from 2020 and one or more previous occasions (3226 observations), the final analysis sample comprised 151 583 observations from 20 839 individuals. The treatment group, comprising Victoria residents with available SCQ responses from around the time of the lockdown in 2020, represented 3568 (93·8%) of the 3804 Victorians interviewed in wave 20, and provided 37 578 observations (34 010 in the pre-COVID-19 period and 3568 in the COVID-19 period). The remaining 114 005 observations (from 17 271 individuals) formed the control group.

The mean MHI-5 score across the pre-COVID-19 period (2011–19) did not differ between the treatment group (72·9 points [95% CI 72·8–73·2]) and control group (73·2 points [73·1–73·3]; table 1 ). An analysis of pre-COVID-19 trends also revealed that MHI-5 scores did not differ significantly between treatment and control groups at any point during this period relative to their 2019 values (appendix p 4). Furthermore, respondents in the treatment group were less likely than those in the control group to report that they had been diagnosed with COVID-19 (0·22% vs 0·54%; p=0·033) and reported a lower percentage chance of having a serious COVID-19 illness (36·5% vs 40·5%; p<0·0001).

Table 1

Sample characteristics and mean MHI-5 scores of treatment and control groups

		Pre-COVID-19 period, 2011–19					Treatment period, 2020
		Overall sample size	Treatment group	Control group	Mean MHI-5 score (95% CI)		Overall sample size	Treatment group	Control group	Mean MHI-5 score (95% CI)
					Treatment group	Control group				Treatment group	Control group
Total		136 877	34 010	102 867	72·9 (72·8–73·2)	73·2 (73·1–73·3)	14 706	3568	11 138	69·6 (69·0–70·2)	70·8 (70·5–71·2)
Gender
	Male	64 095	15 768 (46·4%)	48 327 (47·0%)	74·3 (74·1–74·6)	74·3 (74·1–74·4)	6785	1650 (46·2%)	5135 (46·1%)	71·6 (70·7–72·5)	72·3 (71·8–72·8)
	Female	72 782	18 242 (53·6%)	54 540 (53·0%)	71·7 (71·4–71·9)	72·2 (72·0–72·3)	7921	1918 (53·8%)	6003 (53·9%)	67·8 (66·9–68·6)	69·4 (68·9–69·9)
Age, years
	15–19	10 586	2663 (7·8%)	7923 (7·7%)	70·9 (70·3–71·6)	71·0 (70·6–71·4)	743	198 (5·5%)	545 (4·9%)	63·2 (60·7–65·8)	65·4 (63·7–67·0)
	20–29	24 162	6113 (18·0%)	18 049 (17·5%)	70·6 (70·2–71·1)	71·0 (70·7–71·3)	2455	611 (17·1%)	1844 (16·6%)	65·0 (63·6–66·4)	65·8 (65·0–66·7)
	30–54	55 778	14 251 (41·9%)	41 527 (40·4%)	72·4 (72·1–72·7)	72·8 (72·6–72·9)	5992	1500 (42·0%)	4492 (40·3%)	68·8 (67·9–69·7)	70·5 (69·9–70·9)
	55–69	29 530	6966 (20·5%)	22 564 (21·9%)	74·7 (74·3–75·2)	75·4 (75·2–75·7)	3357	776 (21·7%)	2581 (23·2%)	73·4 (72·1–74·7)	73·6 (73·0–74·3)
	≥70	16 821	4017 (11·8%)	12 804 (12·4%)	76·9 (76·4–77·5)	75·9 (75·6–76·2)	2159	483 (13·5%)	1676 (15·0%)	75·8 (74·2–77·3)	75·9 (75·1–76·7)
Family structure or relationship
	Couple aged <65 years without young children*	35 561	8365 (24·6%)	27 196 (26·4%)	74·6 (74·2–74·9)	75·0 (74·8–75·2)	3665	878 (24·6%)	2787 (25·0%)	71·6 (70·3–72·8)	72·6 (71·9–73·3)
	Couple aged <65 years with young children*	37 923	9660 (28·4%)	28 263 (27·5%)	74·1 (73·7–74·4)	74·7 (74·5–74·8)	4144	1021 (28·6%)	3123 (28·0%)	70·3 (69·1–71·3)	73·1 (72·5–73·7)
	Single parent with young children*	4965	1146 (3·4%)	3819 (3·7%)	65·5 (64·3–66·7)	66·3 (65·7–66·9)	558	122 (3·4%)	436 (3·9%)	64·5 (61·3–67·8)	63·8 (62·1–65·6)
	Single person aged <65 years without young children*	26 321	6782 (19·9%)	19 539 (19·0%)	69·2 (68·8–69·7)	69·5 (69·2–69·8)	2784	700 (19·6%)	2084 (18·7%)	65·4 (64·0–66·9)	65·1 (64·2–65·9)
	Couple aged ≥65 years†	13 615	2179 (6·4%)	7044 (6·8%)	77·9 (77·4–78·5)	76·8 (76·5–77·1)	1736	234 (6·6%)	877 (7·9%)	77·8 (76·2–79·5)	77·0 (76·1–77·9)
	Single person aged ≥65 years	9223	3287 (9·7%)	10 328 (10·0%)	74·6 (73·8–75·4)	75·0 (74·6–75·4)	1111	397 (11·1%)	1339 (12·0%)	70·5 (68·1–72·8)	74·1 (72·9–75·3)
	Older dependent child‡	9269	2591 (7·6%)	6678 (6·5%)	71·5 (70·9–72·2)	70·9 (70·5–71·3)	708	216 (6·1%)	492 (4·4%)	63·7 (61·2–66·1)	63·7 (61·9–65·3)
Dwelling type
	Separate house	111 221	27 870 (81·9%)	83 351 (81·0%)	73·2 (73·0–73·4)	73·5 (73·4–73·6)	11 879	2884 (80·8%)	8995 (80·8%)	70·3 (69·6–71·0)	71·0 (70·6–71·3)
	Semi-detached house, terraced house, or townhouse	9582	1989 (5·8%)	7593 (7·4%)	72·5 (71·7–73·3)	71·4 (71·0–71·8)	1051	218 (6·1%)	833 (7·5%)	67·0 (64·5–69·6)	70·7 (69·5–71·8)
	Flat or apartment	15 039	3968 (11·7%)	11 071 (10·8%)	71·5 (70·9–72·0)	72·5 (72·2–72·9)	1667	446 (12·5%)	1221 (11·0%)	66·4 (64·6–68·2)	70·4 (69·3–71·4)
	Other housing types	1014	180 (0·5%)	834 (0·8%)	72·1 (69·4–74·7)	72·2 (70·8–73·5)	108	20 (0·6%)	88 (0·8%)	66·5 (55·3–77·7)	63·3 (58·8–67·7)
	Missing	21	3 (<0·1%)	18 (<0·1%)	..	..	1	0	1 (<0·1%)	..	..
Country of birth
	Australia	108 599	27 368 (80·5%)	81 231 (79·0%)	72·9 (72·7–73·1)	73·2 (73·0–73·3)	11 822	2916 (81·7%)	8906 (80·0%)	69·1 (68·4–69·8)	70·6 (70·1–70·9)
	Foreign country, mainly English-speaking	12 998	2457 (7·2%)	10 541 (10·2%)	74·4 (73·7–75·1)	75·2 (74·9–75·5)	1298	236 (6·6%)	1062 (9·5%)	70·8 (68·4–73·2)	71·4 (70·3–72·5)
	Other foreign country	15 239	4185 (12·3%)	11 054 (10·7%)	72·7 (72·2–73·3)	72·3 (71·9–72·6)	1582	416 (11·7%)	1166 (10·5%)	71·5 (69·7–73·2)	71·5 (70·5–72·6)
	Missing	41	0	41 (<0·1%)	..	..	4	0	4 (<0·1%)	..	..
Region of residence
	Major city	85 604	23 502 (69·1%)	62 102 (60·4%)	72·8 (72·5–73·0)	73·1 (73·0–73·3)	9023	2432 (68·2%)	6591 (59·2%)	69·1 (68·4–69·8)	70·8 (70.3–71·2)
	Inner regional	34 765	8919 (26·2%)	25 846 (25·1%)	73·4 (72·9–73·8)	73·2 (73·0–73·5)	3984	975 (27·3%)	3009 (27·0%)	71·0 (69·8–72·1)	70·6 (69·9–71·3)
	Outer regional or remote	16 508	1589 (4·7%)	14 919 (14·5%)	74·8 (74·0–75·6)	73·8 (73·5–74·1)	1699	161 (4·5%)	1538 (13·8%)	72·9 (70·3–75·7)	71·4 (70·5–72·4)
Long-term health condition
	Yes	39 410	8724 (25·7%)	30 686 (29·8%)	66·0 (65·6–66·4)	67·2 (67·0–67·4)	4439	950 (26·6%)	3489 (31·3%)	63·4 (62·2–64·8)	65·1 (64·4–65·8)
	No	97 441	25 278 (74·3%)	72 163 (70·2%)	75·5 (75·3–75·7)	75·6 (75·5–75·7)	10 263	2618 (73·4%)	7645 (68·6%)	71·9 (71·2–72·5)	73·3 (72·9–73·7)
	Missing	26	8 (<0·1%)	18 (<0·1%)	..	..	4	0	4 (<0·1%)	..	..
Poor mental health in 2019§
	Yes	19 811	4881 (14·4%)	14 930 (14·5%)	40·9 (40·6–41·2)	41·1 (40·9–41·3)	2718	721 (20·2%)	1997 (17·9%)	41·5 (40·7–42·3)	41·3 (40·9–41·8)
	No	117 066	29 129 (85·6%)	87 937 (85·5%)	78·8 (78·6–78·9)	78·8 (78·8–78·9)	11 988	2847 (79·8%)	9141 (82·1%)	77·1 (76·6–77·5)	77·7 (77·5–77·9)
Equivalised household disposable income in year ending June 30, 2019
	Quintile 1 (poorest)	29 325	6395 (18·8%)	22 930 (22·3%)	69·5 (69·0–69·9)	69·6 (69·3–69·8)	3170	671 (18·8%)	2499 (22·4%)	68·7 (67·2–70·2)	67·8 (67·1–68·6)
	Quintile 2	26 977	6458 (19·0%)	20 519 (19·9%)	71·8 (71·3–72·2)	72·1 (71·9–72·4)	3023	636 (17·8%)	2387 (21·4%)	67·5 (66·0–68·9)	69·8 (69·1–70·6)
	Quintile 3	26 728	6818 (20·0%)	19 910 (19·4%)	72·5 (72·1–72·9)	73·3 (73·1–73·6)	2771	736 (20·6%)	2035 (18·3%)	71·1 (69·7–72·4)	70·6 (69·9–71·4)
	Quintile 4	26 462	7113 (20·9%)	19 349 (18·8%)	74·6 (74·2–75·0)	74·9 (74·7–75·2)	2842	758 (21·2%)	2084 (18·7%)	69·8 (68·5–71·0)	72·1 (71·4–72·9)
	Quintile 5 (richest)	27 385	7226 (21·2%)	20 159 (19·6%)	75·9 (75·7–76·4)	76·3 (76·1–76·6)	2900	767 (21·5%)	2133 (19·2%)	70·6 (69·3–71·8)	73·9 (73·2–74·6)

Data are n, n (%), or mean (95% CI), with n representing the number of observations. With exception of sample sizes, all figures are weighted, adjusting for both complex survey design and non-response (including non-return of self-completion questionnaires). MHI-5=five-item Mental Health Inventory.

Young children is defined as at least one child younger than 15 years.

Any couple in which at least one person is aged 65 years or older.

Any dependent full-time student aged 15–24 years.

Poor mental health was defined as an MHI-5 score <52.

Both Victoria and other (control) states had a downward trend in MHI-5 scores in the years leading up to the pandemic (2011–19) and saw a disproportionately large and statistically significant drop in 2020 (figure 1 ). In Victoria, from 2019 to 2020, the mean MHI-5 score declined from 71·6 (95% CI 70·8–72·3) to 69·4 (68·6–70·1), a decrease of 2·2 points (p<0·0001), whereas the corresponding change in the other Australian states was from 72·0 (71·5–72·5) to 70·9 (70·5–71·4), a decrease of 1·1 points (p=0·0025).

Figure 1

Mean MHI-5 scores by wave of data collection in Victoria and other Australian states and territories

MHI-5 scores range between 0 and 100. Error bars represent 95% CIs. MHI-5=five-item Mental Health Inventory. The horizontal dashed line is the MHI-5 sample mean over the period 2011 to 2020.

The DiD estimation showed a significant overall treatment effect of the lockdown on MHI-5 scores of –1·4 points (95% CI –1·7 to –1·2; table 2 ). The DiD model explained two-thirds of the variation in MHI-5 scores (R2=0·67). The treatment effect was significantly larger for females (−2·2 [–2·6 to –1·7]) than for males (−0·6 [–0·8 to –0·5]; difference 1·5 points, p<0·0001).

Table 2

Effect of Victoria's lockdown on mental health by sex (difference-in-differences estimation results)

	Observations, n	MHI-5 score			Prevalence of poor mental health*
		Lockdown effect, points difference (95% CI)	R2	Pre-COVID-19 period mean (95% CI)	Lockdown effect, percentage points difference (95% CI)	R2	Pre-COVID-19 period mean (95% CI)
Overall	151 583	−1·4 (−1·7 to −1·2)	0·67	73·2 (73·1 to 73·2)	2·6 (1·3 to 3·9)	0·50	15·0 (14·8 to 15·2)
Females	80 703	−2·2 (−2·6 to −1·7)	0·66	72·1 (71·9 to 72·2)	4·3 (3·0 to 5·5)	0·49	16·4 (16·1 to 16·7)
Males	70 880	−0·6 (−0·8 to −0·5)	0·67	74·3 (74·2 to 74·4)	0·7 (−0·8 to 2·3)	0·50	13·5 (13·3 to 13·8)

All analyses are weighted, adjusting for both complex survey design and non-response including non-return of self-completion questionnaires), and include state, year, and person-specific fixed effects. 95% CIs for lockdown effect are based on robust SEs clustered at the state level. MHI-5=five-item Mental Health Inventory.

Poor mental health was defined as an MHI-5 score <52.

A treatment effect was also evident in models examining the binary measure of probable mental disorder. During the lockdown, the prevalence of poor mental health increased by 2·6 percentage points (table 2). This effect, however, was driven by females, with the percentage in poor mental health increasing by 4·3 percentage points (95% CI 3·0 to 5·5). Among males, the effect was small and non-significant (0·7 percentage points [–0·8 to 2·3]).

The estimates of mean MHI-5 scores were found to be insensitive to considering wider treatment windows allowing for a more persistent lockdown effect (appendix p 7), to excluding the 253 individuals from Victoria who were interviewed in wave 20 outside the lockdown period (appendix p 8), or to omitting the 81 persons reporting having tested positive for COVID-19 (appendix p 8). Allowing for state-specific year fixed effects (appendix p 8) resulted in a similar, but slightly larger, negative treatment effect (−2·2 points [–2·3 to –2·1]).

Analysis by age revealed that the effects of lockdown on MHI-5 scores were largest in women aged 20–29 years (−3·7 points [–4·5 to –2·9]) and 30–54 years (−3·0 points [–4·0 to –2·0]; figure 2A ; appendix p 9). No significant lockdown effects were observed for adolescents (aged 15–19 years) of either gender, or for young men (aged 20–29 years). Older males (ie, those aged ≥70 years) were the only group that showed evidence of significantly better mental health during lockdown (1·5 points [0·9 to 2·2]).

Figure 2

Effect of Victoria's lockdown on mental health by gender and other selected characteristics

Graphs show estimated lockdown effects (changes in MHI-5 scores in locked down areas relative to change in MHI-5 scores in other areas) and 95% CIs (error bars) by gender and by age (A), family structure or relationships (B), type of dwelling (C), and region of residence (D). An estimate for single fathers is not reported owing to the small sample size. Each estimate stems from a separate difference-in-differences regression of MHI-5 score on a treatment group indicator and individual, state, and wave fixed effects. All analyses are weighted, adjusting for both complex survey design and non-response (including non-return of self-completion questionnaires). Robust SEs clustered at the state level were used for CI calculation. See appendix (pp 9–12) for estimation results and sample sizes. MHI-5=five-item Mental Health Inventory. *Young children is defined as at least one child younger than 15 years. †Any couple in which at least one person is aged 65 years or older. ‡Any dependent full-time student aged 15–24 years.

Stratifying estimates by family structure showed that the negative lockdown effects were generally greatest for women in coupled households with children (−4·4 points [–5·0 to –3·8]; figure 2B; appendix p 10). More modest negative treatment effects were also observed for men in coupled households, independently of whether they had dependent children (−1·2 points [–1·8 to –0·7]) or not (−1·2 points [–1·9 to –0·5]). The positive treatment effect observed for men aged 65 years or older was restricted to those living in coupled households (1·4 points [0·7 to 2·2]). No treatment effect was found for single mothers.

Analysis stratified by household dwelling type showed pronounced negative effects on MHI-5 scores among females who lived in flats or apartments (−4·1 points [–5·4 to –2·8]) or in semi-detached houses, terraced houses, or townhouses (−4·8 points [–6·4 to –3·2]) at the time of the lockdown whereas a smaller negative effect was seen in females living in separate houses (−1·7 points [–1·9 to –1·4]; figure 2C; appendix p 11). In males, a negative effect of lockdown was also observed for those living in flats or apartments (−2·1 points [–3·2 to –1·0]) but not for those living in semi-detached houses, terraced houses, or townhouses (0·1 points [–1·6 to 1·8]). A negative effect was also seen for males living in separate houses (−0·6 points [–1·0 to –0·3]). However, effects were not significantly different between dwelling types among males.

Lockdown effects were more pronounced in those who lived in major cities (−1·7 points [–2·2 to –1·2]), with the effects for those in inner (−0·8 points [–1·4 to –0·2]) and outer (0·8 points [0·0 to 1·6]) regional Victoria being around half the magnitude (figure 2D; appendix p 12).

Additional analyses showed no difference in treatment effects based on an individual's pre-pandemic health: effects were similar for participants with and without a long-term health condition (appendix p 13) or with and without poor mental health (appendix p 14). There were also no marked differences between Australian-born and foreign-born respondents (appendix p 15). The effects were largest for people in the highest income quintile among both males (−2·3 points [–3·5 to –1·1]) and females (−3·7 [–4·6 to –2·7]). By contrast, no treatment effect was found in the lowest income quintile in males or females (appendix p 16).

Discussion

This study investigated the effect on mental health of lockdowns during the COVID-19 pandemic. It capitalised on the Australian policy approach at the time of study that responded early and vigorously to any evidence of community transmission of COVID-19. The focus of the analysis was on an outbreak of community transmission that resulted in a localised lockdown in the state of Victoria and coincided with the annual data collection for the 20th wave of the HILDA survey. This confluence of events and happenchance enabled a quasi-experimental approach, which revealed an overall decrement in mental health in the Victorian lockdown treatment group relative to the rest of Australia that was statistically significant but small in size: a decline of 1·4 points on the MHI-5 scale, which represents a standardised mean difference of less than 0·1. Applying an established cut-point on the MHI-5 scale to generate an estimate of probable mental disorder, we found that lockdown led to an additional 2·6% of the treatment group being identified as likely to have mental disorder.

The mental health consequences of lockdown were not consistent across all groups in society. Most notably, lockdown was associated with a greater impact on females than males, and this gendered effect was particularly pronounced for those in early to mid-adulthood (aged 20–54 years), in couples with dependent children younger than 15 years, those living in metropolitan Melbourne, and in those living in flats or apartments, or in semi-detached houses, terraced houses, or townhouses. The average treatment effect for these females was of a magnitude that would be considered clinically relevant and equivalent to that of major life events such as being laid off from work (ie, >4 scale points). By contrast, and despite the absence of a partner who can both contribute to household income and share the burden of caring for children, no negative effect of lockdown was found for single mothers. We speculate that this might reflect a greater reliance on income support among this group compared with other groups, the amount of which was greatly boosted in 2020 as part of the federal government's economic response to the pandemic.

Although lockdown had statistically significant effects on MHI-5 scores in men, these effects mostly did not reach this clinically relevant threshold. In contrast to females, there was also no evidence that the lockdown effect was different for males in couples with or without dependent children.

The existing research on the mental health effects of COVID-19 has studied the change in mental health at the onset of the pandemic, examined heterogeneity in these effects, and examined how mental health changed over time during the pandemic.6, 8 The current study had a different focus, using quasi-experimental methods to disentangle the effect of lockdown from other COVID-19-related exposures, such as living in an area with very high rates of community transmission, fear of catching the virus, or direct health effects of COVID-19.1, 5 This ability to separate out the effect of lockdown from other pathways sets this study apart from almost all previous studies of the impact of the COVID-19 pandemic on mental health.

We found that state differences in mental health were limited to the pandemic period (when effects were predicted), and that time trends before 2020 were similar between treatment and control states. This finding supports the common trend assumption, which is the main requirement to interpret DiD as a causal estimate. By comparing the lockdown effect in Victoria to the rest of Australia, the analysis accounts for underlying secular trends in mental health (eg, worsening mental health among youth and young adults) and general effects of the pandemic common to both treatment and control groups.

The observed effects of lockdown on mental health were not driven by the most vulnerable populations; no evidence was found that lockdown was disproportionately associated with worse mental health among older adults living on their own, those with chronic physical or mental health conditions, or those living in the most disadvantaged socioeconomic circumstances. Similar to previous COVID-19 studies,17, 18 our analysis found that the mental health effects of lockdown were greater among those with higher income, which might reflect lockdown restrictions and the associated economic shock having a greater impact on the lifestyle of those in better socioeconomic circumstances. Alternatively, it might be further evidence of the protective effect of the increase in income support payments for those with the least resources. We also found that location and housing circumstances were associated with the magnitude of the mental health effects, with the decline in mental health greatest for those living in urban areas and those living in apartments. The difference in effect by housing type might be related to access to green and other outdoor spaces.

There was also no evidence of a lockdown effect on the mental health of adolescents aged 15–19 years, despite this concern being widespread in the public health debate.2, 17 It must be emphasised that our focus is on the specific effect of lockdown. Although not a focus of the current analysis, our data indicate, in line with conclusions from other research,6, 32 a substantial decline in the mental health of youth and young adults in 2020 compared with 2011–19 (see table 1); however, this decline was evident for both treatment and control groups and, therefore, not a change attributable to lockdowns.

Others have documented the gendered impact of COVID-19 on mental health. The current study confirms the gendered nature of the lockdown experience. Research from the UK found that with the closure of childcare and schools, parents working from home or on furlough were undertaking more than 40 additional hours per week of childcare (including teaching) activities. The majority of this additional work was done by females, reinforcing pre-pandemic gender norms and inequalities. Our results show a clinically relevant decline in mental health associated with the lockdown effect for females with dependent children, suggesting the mental health consequences of lockdown are tied to role overload.

Despite the strengths of our analytic approach, there are limitations. First, the study includes only a single COVID-19 measurement occasion for each individual and, therefore, no conclusion about the duration or persistence of the mental health lockdown effect can be drawn. It can only be interpreted as a short-run effect.

Second, despite the strengths of our natural experiment, the counterfactual is uncertain. The estimated treatment effects were produced in the context of an aggressive COVID-19 suppression policy. It is possible that, in the absence of lockdowns and other associated restrictions to supress virus transmission, the rate of COVID-19 infection would have been higher, which could have led to great COVID-19 morbidity and mortality and resulted in a much greater impact on population mental health.

Third, although we used methods to correct for both non-random sample attrition and non-random response to the SCQ, this reweighting approach will not address the under-representation of immigrants arriving in Australia after 2011 (the time of the last refreshment sample). Further, the restriction of analysis to those living in private dwellings means older adults who had moved into supported accommodation during the previous 20 years were not in scope.

Finally, the strict suppression approach adopted by Australia in implementing lockdowns, and the country-specific nature of the economic, employment, and welfare policy responses to COVID-19, means the current findings are specific to the Australian context in 2020 and therefore might not be generalisable to other settings. Nevertheless, there is a commonality in the types of lockdown policy responses implemented around the world, involving stay-at-home orders, school and business closures, cancellation of public events, and restrictions on movement. Therefore, although the novel Australian context provided the opportunity to isolate the mental health consequences of lockdowns, these policy responses and their mental health effects might be widespread.

In conclusion, although the effects of lockdowns on overall population mental health were small, there were substantial and clinically relevant effects for some groups. The adverse mental health effects were largely seen in women with dependent children, who are likely to have borne the burden of the additional workload associated with working from home, as well as caring for and educating children. As such, the lockdown exaggerated existing inequalities in the responsibility for household and caring duties. These mental health effects should be accounted for when evaluating the merits and costs of the Australian COVID-19 policy approach.

Data sharing

The data used are available free of charge to researchers through the National Centre for Longitudinal Data Dataverse at the Australian Data Archive (https://dataverse.ada.edu.au/dataverse/ncld). Access is subject to approval by the Australian Government Department of Social Services and is conditional on signing a license specifying terms of use.

Declaration of interests

We declare no competing interests.