Public health guideline compliance and perceived government effectiveness during the COVID-19 pandemic in Canada: Findings from a longitudinal cohort study.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic has imposed enormous adversity worldwide. Public health guidelines have been a first line of defense but rely on compliance with evolving recommendations and restrictions. This study sought to characterize adherence to and perceptions of public health guidelines over a one-year timeframe during the pandemic.
METHODS: Participants were 1435 community adults in Ontario who completed assessments at five time points (April 2020, July 2020, October 2020, January 2021, and April 2021; 92% retention). Participants were assessed for self-reported adherence to government protocols and perceptions of government response (importance, compliance, and effectiveness). Analyses used general linear mixed-effects modelling of overall changes by time and examined differences based on age and sex.
FINDINGS: Over time, participants reported high or increasing behavioural engagement in public health guidelines, including physical distancing, restricting activity, and masking. In contrast, participants exhibited significant reductions in perceived importance and compliance, with evidence of more negative changes in younger participants. The largest changes were a substantial reduction in perceived government effectiveness, from predominantly positive perceptions to predominantly negative perceptions.
INTERPRETATION: These results illuminate evolving trends in public health compliance and perceptions over the course of the pandemic in Canada, revealing the malleability of public perceptions of public health recommendations and government effectiveness. FUNDING: This research was funded by a grant from the Canadian Institutes of Health Research (CIHR). CIHR had no role in study design, data collection, data analysis, interpretation, or writing of the report.

Evidence before this study

Public health guidelines have been a first line of defense in addressing the coronavirus disease 2019 (COVID-19) pandemic, but largely rely on population compliance with evolving recommendations and restrictions in liberal democratic societies. Few studies have longitudinally examined the uptake of responses and perceptions of the government response. This study sought to characterize adherence to and perceptions of public health guidelines over a one-year timeframe during the COVID-19 pandemic in Canada.

Added value of this study

In a Canadian sample of community adults, this study adds value by empirically revealing high behavioural responsiveness to public health guidance, but also significant decreases inperceived importance of and compliance with public health recommendations over time. Moreover, the study also reveals a large magnitude decrease in perceived government effectiveness over time.

Implications of all the available evidence

High adoption of public health measures in this sample contributes to understanding the comparatively moderate disease burden in Canada, but the collective decreases in importance, compliance, and government effectiveness also provide an empirical signature of ‘pandemic fatigue.’

Alt-text: Unlabelled box

Introduction

Since late 2019, coronavirus disease 2019 (COVID-19) has proliferated globally, causing major adversity in terms of health and economic distress. , As of Februrary 2022, there have been more than 380 million cases and five million deaths worldwide, with numbers continuing to rise. In response to the rapid increase in cases, governments have implemented a variety of public health measures to slow transmission of the virus, decrease the burden on the healthcare system, and reduce rates of mortality. , Such policies have included physical distancing, restriction of activities, mask-wearing, and stay-at-home orders.

While the implementation of protective measures is essential to controlling the spread of COVID-19, governments are confronted with the delicate task of balancing both public health needs and individual rights. This balance is essential to enhancing a population's perceptions of the importance of following public health guidelines, willingness to comply with those guidelines, and ultimately controlling the spread of COVID-19. Studies conducted following past pandemics have consistently indicated a link between government trust and adherence to health guidelines and a willingness to adopt protective measures.7, 8, 9 During the COVID-19 pandemic, Wright et al.,(2021) observed an association between increased trust and confidence in the government, and an increase in self-reported compliance with government recommendations, albeit using a global indicator that did not distinguish among different public health policies. In addition, Bargain & Aminjonov (2020) found enhanced political trust in Europe was associated with a significant reduction in engagement in non-essential mobility during lockdown. Similarly, increased government trust has been associated with increased adherence to rigorous public policies compared to those reporting low trust in the government. The contrapositive also appears to be the case, with evidence that low trust in government is associated with lower compliance with public health guidelines.

Over the course of the pandemic, public health guidance has been dynamic, necessarily evolving in response to new knowledge about the virus, and public responsiveness to this guidance has appeared to change over time also. Media reports highlight increasing ‘pandemic fatigue’ and public conflict over measures like masking. For these reasons, longitudinal studies are needed to inform and capture continuously changing trends in public health behaviour and perceptions. In addition to exploring general trends over time, subpopulation differences in adherence to COVID-19 recommendations also warrants further attention. Many studies conducted within the past year have found younger adults are generally less compliant with public health measures, such as social distancing.14, 15, 16 In addition, Khubchandani et al., (2020) found women and older adults were more likely to follow restrictions including wearing a mask and gloves compared to men and younger adults. Nivette et al., (2021) observed decreased compliance in younger adults who reported less trust in the government. These findings highlight the potential for substantive differences in following government recommendations based on age and sex.

In this context of evolving public health guidance and population responsiveness, there have been numerous cross-sectional studies examining public health compliance and attitudes (e.g.,), but virtually no longitudinal cohort studies examining these outcomes over time in the same individuals. Rather than just snapshots, longitudinal cohort studies have within-person continuity and can reveal how groups of individuals change over time and can therefore more clearly reveal changes in the continuously evolving landscape of the COVID-19 pandemic.

In Canada, there have been five waves of COVID-19: March 2020 (first wave), September 2020 (second wave), March 2021 (third wave), September 2021 (fourth wave), and January 2022 (fifth wave). Over time, there have been numerous substantive changes in public health guidelines, including varying policies on physical distancing, mask wearing, in-person shopping/school, gathering sizes, and vaccination status, among others. In particular, across Canada, physical distancing was recommended at the start of the pandemic and continues to be a key recommendation. Restrictions on non-essential travel began in March 2020 and persisted until September 2021. Masks became mandated for indoor spaces in July 2020, and outdoor masking recommendations have varied over time. There have been approximately 1.74 million cases of COVID-19 and approximately 29,203 deaths in Canada to date (Public Health Canada, 2021). Canada's mortality rate is 77.80/100,000 individuals, making it, in the context of G7 and other high-income nations, lower than the United States (230.21) and United Kingdom (212.90), similar to Israel (89.74) and Ireland (111.14), and higher than Norway (17.22), Australia (7.26) and New Zealand (0.65). As such, Canada is generally representative of the impact of the pandemic in a first-world economy. Given the paucity of longitudinal cohort studies of public health compliance and perceptions of government response globally, the current study sought to characterize patterns over time, both to understand the unfolding COVID-19 pandemic in Canada and the implications of these changes more broadly. In addition, to illuminate potential subgroup differences in these domains, the study examined systematic differences in public health compliance and attitudes based on sex and age group.

Methods

Participants & study design

The sample comprised participants in an ongoing longitudinal observational cohort study of health behaviour in general community adults. The initial longitudinal cohort of 1502 participants were recruited from an existing research registry at St. Joseph's Healthcare Hamilton (ascertained in 2015-2018), comprising adults in the Hamilton Ontario region. Participants in the registry were recruited through advertisements including social media, print, and other online advertising platforms. Core eligibility criteria were: 1) between the ages of 18 to 65; 2) greater than or equal to 9th-grade education (i.e., adequate literacy to complete written assessments); 3) willingness to receive invitations for future studies; and 4) no current terminal illness (i.e., ability to volunteer to participate in future studies). No specific clinical diagnoses were required. Participants enrolled in the ongoing study were provided with supplemental assessments regarding the COVID-19 pandemic. These assessments were administered at specific time points that fell within the pandemic. Specifically, assessments were administered during five time points: 1) April 15th – May 15th, 2020 (wave 1 and lockdown in Ontario); 2) July 15th – August 15th, 2020 (relaxation of public health restrictions), 3) October 15th – November 15th, 2020 (beginning of wave 2 and lockdown in Ontario) 4) January 15th, 2021 – February 15th, 2021 (beginning of Wave 3), and 5) April 15th – May 15th, 2021 (Wave 3 lockdown in Ontario).

The current sample included 1435 individuals who participated in at least two time points throughout the pandemic. For testing age group as a moderator, subgroups were divided as follows: <25, 25-34, 35-49, 50 and above. All data were collected using the web-based Research Electronic Data Capture (REDCap) software. Participants received online gift cards ($40 CAD) upon completion of the survey. All participants underwent informed consent. First, they received an information email describing the key elements of the research study and what their participation would involve. If participants agreed, they were asked to provide explicit consent. Only participants who provided informed consent were enrolled in the study. The study's procedures were reviewed and approved by the Hamilton Research Ethics Board (Protocol #4699) and complied with the Helsinki Declaration.

Measures

All COVID-19-specific questions were purpose-built and added to the survey after the pandemic commenced (see supplemental Table 2 for a copy of the survey). This survey was piloted, but did not undergo formal psychometric validation prior to use because of the pandemic. Questions on public health adherence and perceptions were embedded in each of the five time points.

For COVID-19 public health guidance adherence, participants were asked “What has your experience been during the COVID-19 pandemic?” and were then instructed to respond either yes or no to whether they enacted certain guidelines including whether they practiced physical distancing, restricted movements to reduce exposure, or wore a mask in public indoor or outdoor spaces.

For perceptions of government response with regard to effectiveness, importance, and rates of compliance, participants were asked three questions. The first was “How important do you think it is to follow government recommendations to address COVID-19” (response options: 1 = very unimportant, 2 = somewhat unimportant, 3 = neither important, nor unimportant, 4 = somewhat important, and 5 = very important). The second was “How much have you followed the government recommendations to address COVID-19” (response options: 1 = did the opposite of government recommendations, 2 = did not follow, 3 = somewhat closely, and 4 = very closely). The third was, “What is your opinion on the government's response to the COVID-19 outbreak” (response option: 1 = very ineffective, 2 = somewhat ineffective, 3 = no opinion, 4 = somewhat effective, and 5 = very effective).

Other measures included participant demographics and five quality control items with unambiguously correct answers to flag for low attention and effort. For example, one question asks: “In response to this question, please choose option 'nearly every day'", and participants are provided with four possible response options “Not at all”, “Several days”, “More than half of days”, and “Nearly every day”. If participants answered incorrectly to 2+ questions, they were excluded from that time point.

Data analysis

No data were missing on any of the variables. Age and education were analysed as subgroups in accordance with commonly used representations as well as balanced for sample size. Generalized linear mixed-effects modelling (GLM) was used with a random intercept to evaluate the moderating role of age, sex, and time on each of the outcomes. This approach accounts for within-individual correlations among multiple assessments (2 to 5 assessments) of outcomes by incorporating a random effect variable into the model. We used generalized mixed modelling with a binary link for public health practices (0/1). Perceptions of government responses to the pandemic was considered as an underlying score which was measured as a 5-option response. Of note, the data was not normally distributed, however, linear mixed modelling provides relatively robust deviations from a normal distribution. As such, the skewness was not extreme and did not warrant zero-inflated Poisson models or dichotomous outcomes. To assess for goodness-of-fit, area under the curve (AUC) was implemented for binary variables (i.e., adherence to government recommendations), and residual error histograms and Q-Q plots were implemented for continuous variables (i.e., government effectiveness, importance, and compliance). The analysis was conducted using GLIMMIX procedure of the SAS® software Version 9.4. To account for multiple testing and to reduce the Type 1 error rate, a conservative significance threshold of p < 0.01 was implemented.

Results

Participants

Participant demographics are in Table 1 and indicate the sample comprises of, on average, adults in their late-30s of moderate education and income of whom approximately 80% self-identified as being racially white. Table 1 also presents demographics for the original recruitment catchment area (i.e., Hamilton, Ontario), as well as provincial and national demographics. The cohort demographics are similar to municipal, provincial, and national demographics, albeit with education that is somewhat higher than the national and provincial average. The demographics suggest modest overrepresentation of younger individuals and females, and underrepresentation of racial minorities. Individual wave demographics are in supplemental materials and suggest similarity over time. In addition, Supplemental Table 3 presents comparisons of case rates and fatalities in Canada, Ontario, and Hamilton, respectively.

Table 1

Comparison of total sample demographics (N = 1435) with the recruitment catchment area (Hamilton, ON), provincial, and national demographics based on data from the 2016 Canada Population Census.

	Total Sample	Municipal	Provincial	National
Age (mean)	36.7	41.6	41.0	41.0
Sex (%female)	60.3%	51.3%	51.2%	50.9%
Race (% white)	79.2%	82%	71%	77.7%
Education⁎
< Bachelor's Degree	36.2%	75.0%	68.2%	71.5%
Bachelor's Degree	49.3%	16.5%	21.0%	19.0%
> Bachelor's Degree	14.6%	8.5%	10.9%	9.5%
Household income (median)	$75,000-90,000	$70,000- 79,999	$70,000- 79,999	$70,000- 79,999

Education for municipal, provincial, and national data was obtained for individuals age 25-64.

Retention was high across time points (Time 1 = 92.7%; Time 2 = 92.1%; Time 3 = 92.5%; Time 4 = 91.8%; and Time 5 = 90.4%). Of the 1435 participants, most participants completed all assessments (95.54%). Validity rates based on the quality control items were high (Time 1 = 99.3%; Time 2 =98.3%; Time 3 = 99.1%; Time 4 = 99.2%; and Time 5 = 99.4%).

Public adherence to government recommendations

Results from the general linear mixed-effects model for engagement in public health guidance and the interactions with sex and age group are presented in Table 2 . AUCs indicate good model fit for each of the government recommendations (see Supplemental Figure 4). In the analysis exploring whether participants wore a mask in outdoor spaces, because of very few (nearly zero) individuals endorsing this behaviour, the model did not converge. For this variable only three time points of data from July and October 2020 and January 2021 were used. The results suggest significant time effects for restricting movement to reduce exposure and mask wearing in both indoor and outdoor spaces, reflecting increases in adherence over the five time points. Figure 1 presents the temporal trends for each of the recommendations. There were no significant interactions with sex or age, reflecting a global increase in adherence without differences in these subgroups.

Table 2

Main effects of time and interactions with sex and age group for engagement in public health guidance to reduce the spread of COVID-19. Both F statistics and p values are presented for each category.

	Time		Time x Sex		Time x Age
	F	p	F	p	F	p
I practiced physical distancing	1.46	0.212	0.46	0.603	0.51	0.910
I reduced my movement to reduce exposure	14.61	<0.001	0.73	0.571	1.26	0.238
I wore a mask in public indoor spaces	151.08	<0.001	2.93	0.020	1.27	0.232
I wore a mask in public outdoor spaces	138.59	<0.001	1.17	0.321	0.81	0.608

Figure 1

Engagement in public health COVID-19 transmission prevention strategies including physical distancing, reduced movement, mask wearing in indoor public spaces, and mask wearing in outdoor public spaces from April 2020 to April 2021

Public health perceptions

The results for the general linear mixed-effects models for perceptions of public guidance in terms of importance, compliance, and government effectiveness in managing the pandemic from April, 2020 to April, 2021 are in Table 3 . Residual error histograms and Q-Q plots indicate good model fit for each response indicator (Supplemental Figure 5). Figures. 2 , 3 , and 4 present the temporal trends for importance, compliance, and effectiveness, respectively. There were significant effects of time for all three variables, with perceptions of importance, compliance, and effectiveness decreasing over the five time points.

Table 3

Perceptions of public guidance: importance, compliance, and government effectiveness in managing the pandemic from April, 2020 to April, 2021 over time and by sex and age group.

	Time		Time x Sex		Time x Age
	F	p	F	p	F	p
Importance	188.23	<0.001	1.04	0.383	4.55	<0.001
Compliance	53.78	<0.001	1.31	0.263	2.78	<0.001
Effectiveness	559.68	<0.001	1.10	0.355	1.79	0.045

Figure 2

Mean perceptions of the importance of public health guidance from April 2020 to April 2021.

Figure 3

Mean endorsement of compliance with public health strategies from April 2020 to April 2021.

Figure 4

Mean perceptions of government effectiveness from April 2020 to April 2021.

Comparing to the first assessment, effect sizes varied widely. Effect sizes for decreases in compliance were generally small (T1:T2 d = 0.204; T1:T3 d = 0.302; T1:T4 d = 0.259; T1:T5 d = 0.444). Importance effect sizes ranged from small to large (T1:T2, d = 0.202; T1:T3 d = 0.313; T1:T4 d = 0.437; T1:T5, d = 0.716). Government effectiveness effect sizes for significant changes ranged from medium to very large (T1:T2, d = 0.032 [nonsignificant]; T1:T3, d = 0.458; T1:T4 d = 1.177; T1:T5 d = 1.812). That is, an almost two standard deviation decrease in perceived government effectiveness was observed over the course of the year. In the units of the item itself, 87.7% (n = 1211) rated the government as somewhat or very effective in April 2020, but this dropped to 22.8% (n = 307) one year later. Changes in compliance, importance, and effectiveness are further illustrated in alluvial plots in supplemental materials.

In terms of subpopulation differences, small magnitude or nonsignificant changes were present. There was a time-by-age interaction for importance and compliance. The most notable change occurred in younger adults (i.e., below 25 years of age) who demonstrated a decrease over time in perceiving the importance of following government recommendations. In general, over 80% of people endorsed “very important” with regard to following government recommendations across all ages, with the most substantial reduction observed in individuals below 25. In this age group, importance of following government recommendations dropped to 37% compared to 64% in adults 50 and above. For compliance with government recommendations, there was a significant time effect suggesting a decrease in compliance over the five time points. There was also a significant time by age interaction, and the most substantial decrease in compliance was again observed in younger adults. The time by sex interactions were not significant for importance, compliance, or effectiveness. There was no significant time by sex or by age interactions for perceptions of government effectiveness.

Discussion

Using a longitudinal observational cohort design, the current study examined the temporal periodicity of compliance with and perceptions of public health COVID-19 guidelines and government response. Overall, the results revealed diverging trends. On one hand, participants reported either initially high or dramatically increasing self-reported behavioural compliance with government recommendations. More specifically, in terms of adherence, wearing a mask in public indoor spaces demonstrated the highest increase over time, followed by wearing a mask in public outdoor spaces. This finding is in line with the shift in public policy regarding wearing a mask from acceptable, to recommended, to required. Restricting movement also demonstrated a significant increase over time, with physical distancing remaining consistently high.

On the other hand, perception of importance and compliance with public health guidelines and perceptions of government response all significantly decreased over the five time points. These results can be thought of as an empirical signature of so-called pandemic fatigue in response to lifestyle changes in response to the pandemic. These changes were observed across the yearlong period, with the most substantial decline observed in government effectiveness over time. There are many factors that may influence people's perceptions of government effectiveness, including public health communication and disparities in opinions in ways of handling the crisis. Mizrahi et al., (2021) found that people's satisfaction with the government and perceived participation in decision making were directly related to perceptions of government effectiveness.

In this case, although necessarily conjecture, effectiveness of government response may be related to short-term changes that enhance people's sense of satisfaction. For example, implementation of the Canadian Emergency Response Benefit (CERB; a financial support program for employed Canadians who were directly affected by COVID-19) may have initially enhanced people's perception of effectiveness insofar as it reflected the government taking rapid tangible steps to offset the adverse effects of the pandemic; however, stringent subsequent measures, such as mandatory hotel quarantine for travellers, perceptions of ineffective timing of communications, and reduced financial support for individuals who were unable to work may have negatively affected perceptions of response at later time points. In other words, while programs like CERB may have initially generated high collective trust in the government's response to the pandemic, decreased financial support over time in conjunction with waxing and waning of public health restrictions may have eroded perceptions of government effectiveness. A slower than anticipated vaccine roll-out is a further factor that may have led to the large effect size decrease in perceived government effectiveness. Fortunately, the changes in perception do not appear to have substantially attenuated behavioural adherence during the current time points, but future impacts cannot be known. Furthermore, beyond the COVID-19 pandemic, the diminution of perceived importance of public health guidance and government effectiveness may have adverse consequences in the context of future pandemics.

While sex was not significantly related to adherence or response, age significantly interacted with time for multiple indices. In particular, government importance and compliance had a significant age by time interaction. In line with previous studies, younger people consistently reported a decrease in following recommendations and not believing the recommendations are very important more than middle and older aged adults. However, in terms of overall trends, the main effects of time were substantially larger than the subgroup differences observed.

These findings should be considered in light of strengths and limitations. A strength of the study is its longitudinal design, revealing changes over a one-year period at five separate time points. As the COVID-19 pandemic has changed substantially in the past year, these findings can provide novel insight into relevant patterns of response and adherence. In addition, the relatively large sample size and demographic characteristics suggest generalizability to large segments of the Canadian population and beyond. However, this sample comprised non-clinical adults, reflecting a general community sample, not a high-risk group. The sample was also not highly racially diverse, with more limited generalizability to minority populations. In addition, the study asked participants about government response in general, which does not permit unpacking potential differences between the local, provincial, and federal government response. However, with regard to provincial comparisons, the findings from this study are in line with individuals’ perceptions of government response to COVID-19 in Ontario, which dramatically declined at the beginning of the second wave of the pandemic according to a provincial- wide survey in Ontario, and these results are also in line with national data that suggest relatively high rates of adherence to changing government recommendations. Of note, a recent study by McCoy et al., (2020) suggested substantive heterogeneity in government response subnationally, which emphasizes the need for more exploration of response at both the provincial and municipal levels. It is also important to consider the potential bias inherent in self-report measures, such as social desirability, possibly overestimating participants adherence to government recommendations. While this is an important consideration, other studies conducted in Canada have suggested similarly high rates of compliance with policies such as wearing a mask, albeit over a narrower time window. Other studies on government compliance including social distancing have also suggested cohesion between self-reports and actual observed behaviour, which suggests that self-reports, when no contingencies for reporting certain outcomes are present, are acceptably valid.

The current study provides a unique perspective on the temporal patterns of compliance with and perceptions of public health COVID-19 guidelines in Canada. Substantive increases in behaviours to reduce disease transmission were observed but decreases in perceptions of public health guidelines and government response were also present. This was most notable in perception of government effectiveness which effectively collapsed over the course of the year. In general, the findings from this study illuminate the dynamic intersection of public health guidelines and the population's response as the COVID-19 pandemic unfolds and evolves in Canada and globally.

Contributors

Emily E. Levitt: conceptualisation, methodology, investigation, writing – original draft, visualization and validation.

Mahmood Gohari: conceptualisation, investigation, data curation, formal analysis, writing – review & editing, visualization.

Sabrina K. Syan: conceptualisation, investigation, methodology, project administration, writing – review & editing.

Kyla Belisario: conceptualisation, investigation, data curation, formal analysis, writing – review & editing.

Jessica Gillard: investigation, methodology, project administration, writing – review & editing.

Jane DeJesus: investigation, methodology, project administration, writing – review & editing.

Anthony Levitt: investigation, writing – review & editing.

James MacKillop: conceptualisation, funding acquisition, investigation, methodology, project administration, resources, supervision, writing – review & editing, visualization and validation.

Data sharing statement

Data from this study will be made available upon request to qualified researchers with scientifically appropriate goals for further analysis.

Declaration of interests

No authors declare any competing interests associated with this study. JM is a principal and senior scientist in BEAM Diagnostics, Inc. and a consultant to ClairvoyantRx, Inc. but no associated products or services were used in this study.