Governing evidence use in the nutrition policy process: evidence and lessons from the 2020 Canada food guide.

Nutrition guideline development is traditionally seen as a mechanism by which evidence is used to inform policy decisions. However, applying evidence in policy is a decidedly complex and politically embedded process, with no single universally agreed-upon body of evidence on which to base decisions, and multiple social concerns to address. Rather than simply calling for "evidence-based policy," an alternative is to look at the governing features of the evidence use system and reflect on what constitutes improved evidence use from a range of explicitly identified normative concerns. This study evaluated the use of evidence within the Canada Food Guide policy process by applying concepts of the "good governance of evidence" - an approach that incorporates multiple normative principles of scientific and democratic best practice to consider the structure and functioning of evidence advisory systems. The findings indicated that institutionalizing a process for evidence use grounded in democratic and scientific principles can improve evidence use in nutrition policy making.

INTRODUCTION

Integrating scientific knowledge into nutrition policy is a longstanding challenge. It raises questions about bridging the gap between science and politics in a way that enables crafting nutrition policy that is informed by systematically gathered, rigorous, and high-quality evidence – while doing so within democratic processes that uphold public values, preferences, and interests. As noted by Austin and Overholt in 1988, these sets of concerns can sometimes be seen to be at odds in nutrition policy making, usually to the dismay and frustration of those who condemn politics as a barrier to scientifically driven policy development (eg, see Cullerton et al 2016). Yet, as policy scholars of evidence use have increasingly argued, it is important to not let the utilization of evidence obscure the political nature of policy decisions and the competing social values and concerns that they address. For example, scientific evidence alone cannot say what the specific goals of nutrition policy should be in the first place, including the relative importance of promoting health, economic, and/or cultural objectives. Nor can it alone dictate how to balance the competing preferences and interests of the various stakeholders, such as consumers, industries, and nutrition professionals, since these are decidedly social choices that policy makers must make. The relative importance of other key factors alongside scientific validity in the decision-making process, such as speed, transparency, and representation, is also an important social consideration.

Additionally, as noted by several nutrition scholars, significant technical challenges also surround the generation and use of evidence in nutrition policy making. For example, authors have noted the challenge of establishing definitive causal claims due to the presence of confounding and contextual variables when conducting nutrition studies, as well as the challenge of addressing industry-funded research that often produces results that favor industry interests. Thus, while there is general agreement that nutrition policy should be based on scientific evidence and made in line with public values and preferences, the need to improve evidence use in nutrition policy considering these political and technical challenges represents a paramount concern for ensuring effective and legitimate nutrition policy going forward., This is of particular concern in an era in which both scientific knowledge and democratic practices are increasingly questioned, challenged, and undermined. The question of how to overcome these challenges and strike a balance among competing priorities in a way that forefronts the potentially synergistic relationship between science and democracy is increasingly important as governments continue to look for strategies to improve how evidence informs nutrition policy.

Indeed, it has been argued that many domains of public health policy, not just nutrition, stand to benefit from improving the way that they systematically incorporate evidence in their decision-making processes. One challenge, though, is that it is not clear what constitutes an “improvement” in evidence use, and, thus, how that might be indicated. And despite growing calls for improved evidence use in health and nutrition policy, there exists little practical guidance for policy makers to inform them of what such an improvement would entail. There is also little evidence on the practical experience of those who have tried to improve their evidence use in their policy processes.

This paper aims to contribute to this area by focusing on a recent policy experience to improve evidence-informed guideline development for nutrition policy. In 2012, Health Canada (the department of the Canadian federal government responsible for national health policy) overhauled the policy process that creates the Canada Food Guide – Canada’s national nutrition guide that aims to provide citizens with basic guidance for eating a healthy diet. The revised policy process included new rules for advisory committee membership, new and regular evidence review cycles, as well as new stakeholder consultation mechanisms. After publishing findings from the first evidence review conducted in 2015, a set of guiding principles in 2017, and a second evidence review in 2018, the policy process produced the 2020 Canada Food Guide, the first-ever guide created by the newly revised process. But while the overhaul changed several aspects of the policy process by which Health Canada creates the Canada Food Guide, the question of whether these changes served to improve evidence utilization has yet to be considered.

This study applies Parkhurst’s concept of “the good governance of evidence” to evaluate improvements in evidence-use within the Canada Food Guide revision process. The good governance of evidence was specifically developed to incorporate multiple normative principles of scientific and democratic best practices to consider the structure and functioning of evidence advisory systems. Evaluating whether and how procedural changes for a technical guide of this nature constitute an improvement in evidence use is a neglected area of policy inquiry, yet it can be important to inform thinking about where future adjustments to the process are needed. This analysis of changes in the Canada Food Guide process can serve as an example for other domains that wish to explicitly consider how evidence is used within guideline development processes.

The paper consists of 2 key parts. The first introduces the concept of the good governance of evidence and its 8 principles of quality, appropriateness, rigor, representation, stewardship, deliberation, contestability, and transparency. It then establishes a set of indicators to evaluate these principles in relation to nutrition policy. It begins by considering the challenges that surround the question of improving evidence use in nutrition policy and health policy more generally. Taking the perspective that using evidence in policy making is a multifaceted challenge that must balance both technical and political concerns, the first part argues that any attempt to measure improvement in evidence use must do so using standards that integrate both scientific and democratic principles. It then proceeds to adapt the 8 principles of the good governance of evidence framework to the specific context of nutrition guideline development – resulting in a framework consisting of a set of 28 indicators.

The second part employs this newly developed set of indicators to undertake an empirical comparative analysis of the 2007 and 2020 Canada Food Guide creation processes in relation to the governance of evidence. It uses the newly created tool to examine whether or how elements of the 8 principles were met, and to compare the presence (or absence) of these principles during the development of the 2007 and 2020 food-based dietary guidelines in Canada.

THE GOOD GOVERNANCE OF EVIDENCE IN NUTRITION POLICY

From evidence-use to the governance of evidence

There have been a number of authors advocating for evidence utilization in nutrition policy over the past decades. However, while many of them call for increased or more evidence use, they typically fail to explicitly clarify what better evidence utilization would look like for nutrition policy, or which principles can be elaborated to make such judgments. Rather, many proponents of evidence use in the nutrition sector call for uptake of evidence, which appears to simply equate more science with better policy. This perspective has been criticized in the health policy arena more broadly, however, as representing an oversimplified idea of the role of evidence in policy change, by failing to account for the multiple social concerns faced by decision makers (each with its pieces of relevant evidence), and the limits on perfect information, which means that there can be multiple ways that evidence may be utilized to achieve policy goals.,

Other scholars have worked to identify and study the political forces that shape evidence use and policy outcomes in nutrition policy in an attempt to identify strategies to train and inform evidence advocates.,,, These studies, however, often adopt the perspective that policy will be evidence-based if entrepreneurial evidence advocates are equipped with the skills to harness and/or tame politics when constrained by time-sensitive policy windows. Among scholars who study the political nature of evidence-use in health policy more broadly, there has been a move away from the idea that policy can simply be judged as evidence-based or not, with shifts to try and understand the features of political contexts that might shape the forms of evidence used and the ways that evidence feeds into policy. Yet research in neither of these streams address questions of how to pass judgment on different forms or processes of evidence utilization. Even if they explore the nature of the political process shaping when or how evidence might be used, there is much less work explicitly considering whether evidence use of one form or another can be judged good or bad, or more or less appropriate for political needs. So, for example, a recent paper by Tudisca and colleagues develops a set of indicators said to enhance evidence-informed decision-making in health policy, but which do not address the political and scientific principles against which evidence utilization can be judged. Similarly, Blake and colleagues and Cullerton and colleagues advance evidence-based frameworks that do not explicitly incorporate political considerations such as for what/whose purposes evidence is marshalled, under what conditions, and to whose benefit and loss. While noting that policy making is political, these frameworks and indicators again fall back on the idea that what matters is simply evidence uptake, implying that enhancement simply means more and faster evidence use.,,

In contrast to these approaches, this paper argues that assessments for improvement require some explicit normative principles against which to judge multiple elements of evidence use within political environments characterized by multiple competing interests and political needs. This changes the focus on improving evidence use away from just considering whether a preidentified piece or body of research evidence was used or not in a binary way, and instead requires explicit consideration of principles by which appropriate or better evidence utilization can be judged. One recent attempt to turn attention to the normative principles by which evidence use can be judged has been the development of the “Good Governance of Evidence” framework of Parkhurst (2017) – an approach that argues for explicit consideration of normative principles of both scientific and democratic good practice to judge evidence advisory processes and systems that govern the use of evidence in policy-making spaces. The approach takes the perspective that good evidence use depends on the use of “rigorous, systematic and technically valid pieces of evidence within decision-making processes that are representative of, and accountable to, populations served.”

This framework provides a key starting point for evaluating changes to guideline development processes, such as those seen in the Canadian Food Guide. While there are tools aimed at evaluating or informing healthcare guidelines and nutrition guidelines, these tools focus on a combination of process and output characteristics and reflect mostly on technical aspects of guideline development. They typically do not consider the inherently political context in which guidelines are created, nor do they ground their indicators on explicitly normative principles that enable judging whether one use is better than another. Rather, this study follows an approach similar to that of Shaxson (2019), who applied these governance of evidence principles to undertake a comparative analysis and render judgements on evidence-related practices in UK and US government agencies. In this study, however, 1 process of guideline development followed in a single country is compared across 2 instances in time.

Eight principles of good governance of evidence in nutrition policy

Nutrition policy and guideline formation rely on robust scientific evidence free from technical bias. However, nutrition policy and guideline formation are also political insofar as they embody and project an authoritative conception of the common good, a conception that inevitably favors some values and preferences at the expense of others. On a more theoretical level, the very act of publishing nutritional guidelines presupposes a political position about the role of government in relation to society. More pragmatically and depending on the goals of the specific policy process, different arrangements of priorities could see health promoted at the expense of particular social or cultural practices or economic activities. While improving democratic participation through increased deliberation could potentially help to clarify priorities or incorporate a range of knowledge and experiences, policy decisions will inevitably result in outcomes that favor some groups and interests at the expense of others. This inherently political nature of policy making underscores the need to ensure democratic principles are maintained in any process by which scientific evidence is brought to bear on social policy concerns – in addition to established scientific principles to ensure rigorous and valid uses of relevant evidence.

The Good Governance of Evidence framework of Parkhurst identifies 8 normative principles deriving from concepts of scientific best practice, as well as concerns over democratic legitimacy. These 8 principles are quality, appropriateness, rigor, representation, stewardship, deliberation, contestability, and transparency. Each of them will be discussed in turn here in relation to nutrition policy and planning to inform a set of indicators that can be applied to the Canada Food Guide case.

Quality.

Many note the importance of ensuring that nutrition policy is based on the highest quality evidence available, as well as the negative consequences when it is not. Quality criteria typically reflect the methodological principles pertaining to the form of research utilized. One challenge in nutrition science, however, is that randomized control trials, the gold standard for inferring causality, suffer from many challenges in the context of nutrition research, most notably around controlling bias in relation to blinding, accounting for random error, and achieving participant adherence. Alternative designs such as observational studies are available, but they offer limited confidence of causal inference, and most dietary measurements in observational studies rely on participant memory and do not measure the health impact of dietary changes over time. The different kinds of evidence and their respective limitations underscore the importance of assessing quality by criteria appropriate to the method and in relation to the potential for informing policy.,, Ordering evidence according to evidential hierarchies is standard practice for studies of intervention effect, yet nutritional evidence reviews may need to include types of evidence outside meta-analysis and randomized trials, given the nature of the evidence base. Although multiple quality evaluation criteria exist, the GRADE criteria are widely used in nutrition research and can provide an example of how formal quality rankings can be applied when appropriate to the tasks at hand (whether it is the exact GRADE criteria or a version adapted for specific needs).

Another important consideration in guideline development can be the role that industry-funded evidence plays in supporting recommendations. This is particularly relevant in nutrition policy formation in light of a growing body of evidence that industry-funded research tends to produce findings that favor industry products, which may result from deliberate design bias. Although it may be nearly impossible to completely exclude industry evidence, there should be processes that are explicitly focused on how to handle conflicts of interest that can generate technical bias.

Appropriateness.

While the importance of using high-quality evidence is widely accepted, policy makers must also consider the appropriateness of the evidence to their policy needs., Appropriateness can only arise inasmuch as the choice of evidence follows an initial assessment of the needs of the policy decision at hand. Given the importance of explicitly considering multiple values and social interests, calls to simply “follow the evidence” risk depoliticizing the policy process by projecting an objective understanding of the issue and a universally desired outcome. In reality, evidence can present a particular understanding of the issue and the resulting solution, which may or may not reflect how stakeholders conceptualize the policy problem or how best to address it. Appropriateness is thus key in nutrition policy since people live and eat in contexts highly influenced by political, social, and economic forces that must be accounted for in policies. Guidelines should attempt to include evidence that is appropriate by incorporating evidence that accounts for these diverse determinants of health within food systems.,

Rigor.

Appropriate and high-quality evidence for nutrition policy must still be gathered and applied rigorously according to high scientific standards. As a tool in the political process, evidence can be weaponized through selective cherry-picking when actors look for findings that support predetermined policy outcomes. Often, these instances occur when a particular actor or group has a strong interest in a particular outcome. Having a rigorous process that is shielded from these potential influences is a key concern. Rigor in the guideline development process would thus involve formalizing a mechanism to ensure comprehensive, systematic, and thorough gathering and synthesizing of evidence in a way that avoids selective cherry-picking or exclusion of relevant information.

Representation (of the people).

While often framed as a technocratic process, recognizing the political nature of guideline development and implementation means that there is a need to ensure that the needs and values of the population served are fundamentally reflected in the resultant policy. The concept of representation fundamentally recognizes that in democratic systems, this is best ensured when final decision authority for policies informed by evidence lies with representative and publicly accountable officials. Thus, while there may be a range of technical components to nutrition guideline development, the legitimacy of the guidelines comes from the government who, by virtue of their office, exercise power by making claims to represent the public good.

Stewardship (for the people).

A related issue to representation is the concept of stewardship. While democratic principles hold that legitimacy in policy arenas comes about when ultimate decision-making authority lies in representative officials, scientific advice and evidence synthesis can rarely be undertaken by elected officials. Considering the political issues surrounding nutrition policy raises further considerations about who uses evidence, in what ways, and under whose directive. Nutrition is a policy domain that encompasses multiple stakeholders with competing political and financial stakes, and many governments must manage strong, often unbalanced lobbies representing consumer and producer interests. Stewardship reflects the principle that the advisory bodies that curate evidence should be guided by a formal and public-serving mandate; as such, the design and composition of the advisory body is also an important consideration for how evidence is used in the policy process, particularly to preserve against industry influence – a key concern for ensuring the trustworthiness of nutrition science and policy. At the very least, conflicts of interest should be disclosed, yet it may also be important to limit the number of individuals on guideline advisory committees who have financial and nonfinancial conflicts of interest within the food sector.

Deliberation (by the people).

Evidence can take multiple forms, with varying connections to consumer needs and lived experiences. While ensuring decision-making representation can help to ensure the legitimacy of final decisions, democratic theory has further highlighted the importance of deliberative processes to also ensure the legitimacy of the ongoing operation of decision-making systems as well. Indeed, involving the public is already recognized as important in developing nutrition guidelines. Emphasizing the importance of deliberation starts from the assumptions that the public has the right to participate in the determination of policy and that individuals have valuable input for policies that affect their health. In addition to being a fundamental component that legitimizes the process, involving consumers through deliberation can also improve the implementation and uptake of guidelines. Though there is no standard approach to engaging the public through deliberation, some key beliefs have gained wide acceptance in research and policy circles, as noted by Health Canada’s own internal resources. Specifically, the public should be actively engaged throughout the process, including through involvement in meetings and ongoing dialogues with guideline makers, not just passively like through surveys and participation in feedback forums.

Contestability.

Contestation is inherent to both nutrition science and democratic politics alike. Nutrition is among the most contentious fields of science, and making nutrition policy requires navigating large, often conflicting bodies of evidence sometimes associated with controversy. But contestation is a driving motivator for scientific discovery and provides important quality assurance in the scientific endeavor more broadly. Nutrition policy processes that use evidence, therefore, cannot shy away from contestation, but rather there needs to be room for scientific disagreement when bringing knowledge to bear on policy. This principle underscores the need to subject evidence and its review processes to scrutiny through the process of peer review. Contestation is also a fundamental feature of democracy. It arises from the belief that citizens should have the opportunity to formulate and signify their views and preferences to other citizens through individual or collective action, and to have “their preferences weighed equally in the conduct of the government.” To ensure legitimacy through contestability, therefore, nutrition policy processes should aim to include mechanisms for considering dissent, as well as appealing policy decisions.

Transparency.

Finally, transparency is often considered important to ensure that the policy process operates fairly, and that evidence is used appropriately by, among other things, laying bare how and under what conditions decisions are arrived at, as well as whose interests are being served by those decisions – with nutrition policy being no exception. Transparency is typically achieved when there are clear and open ways for the public to see how the evidence bases informing a decision are identified and utilized. Indeed, there is general agreement that transparency is a fundamental component of legitimate democratic governance, and thus it is important to consider how guideline development processes are rendered transparent.,

Developing indicators

To apply these principles and evaluate the use of evidence for the 2007 and 2020 Canada Food Guide revision processes, the 8 principles described above are transformed into the set of 28 measurable indicators shown in Table 1. The attributes and components of attributes are organized logically based on their level of abstractness in the table. The good governance of evidence serves as the overarching concept (see Table 1, column 1) and is the most abstract; the next level down (column 2) consists of the 8 principles previously elaborated. These 8 principles, however, require further specification to enable the comparison of empirical cases. For example, appropriateness, as described, is a principle capturing how well evidence meets the needs of the policy decision at hand. Conceptually it can be seen to have 3 components: the significance of the policy problem to the population; the significance of the evidence to the policy problem; and the applicability of the evidence to the local context (column 3). From these components, specific indicators can be identified for evaluation: whether an evidence utilization process explicitly clarifies its goals, whether it tries to prioritize between multiple competing concerns, and whether it considers local applicability of a body of evidence, and the like (column 4). This process of specifying each principle into components and then identifying indicators to capture them is done for each of the 8 principles in the Table (Table 1).

Table 1

A framework to evaluate the good governance of evidence in nutrition policy

Level of abstraction
More abstract (less concrete)			Less abstract (more concrete)
Concept	Attributes	Components of attributes	• Indicators
Good governance of evidence in nutrition policy	Appropriateness	• The significance of policy problem to population	• The clarification of goals
			• The initial statement of the relevant decision criteria
			• The attempt to prioritize a number of considerations
		• The significance of the evidence to policy problem	• The critical questioning of evidence sources in terms of their relevance and use
			• The application of methods akin to those of multicriteria decision analysis
		• The applicability of the evidence to the local context	• The differentiation between internal and external validity of evidence
			• The requirement of assessments of local applicability prior to utilization
	Quality	• The use of appropriate methods to generate evidence in relation to the research question asked and data generated	• The application of GRADE or similar quality criteria appropriate to the research question
		• The use of different types of evidence and a recognition of their limitations	• If evidence includes an assessment of an intervention’s impact, then the ordering of evidence and use of evidential hierarchies.
			• A clearly described statement or strategy for dealing with and assessing the quality of industry-funded research
	Rigor	• The comprehensive gathering and synthesizing of evidence through a systematic process	• The use of practices with strict adherence to the scientific method and mechanisms to ensure comprehensive, systematic, and thorough gathering and synthesizing of evidence for systematic review, rapid review, realist review, and other synthesis methods where appropriate
			• Clearly described selection and inclusion criteria
			• Clearly described methods for formulating recommendations
	Stewardship	• The establishment and/or formalization of a public mandate for the body and rules that shape the advisory system	• Statement about the role of democratically elected agents or their representatives in the design or alteration of government evidence advisory bodies
			• The accountability of agents to the public
		• The independence of review and advisory bodies	• Statement about how the advisory body handles conflicts of interest
			• Robust defenses against imposition of institutional structures by nonmandated or unaccountable agents
	Representation	• The decision makers are representativeness of the public, or determined democratically	• The maintenance of decision authority in public representatives
			• The ability of legislatures and representatives to veto or override technical agencies when necessary
	Transparency	• Information used and generated is open and accessible	• The accessibility of information
			• The freedom of information
			• The publication of transcripts or minutes of expert body deliberations
	Deliberation	• Public engagement and participation during the policy process	• Formalized mechanism for active public participation
			• Formalized mechanisms for passive public participation
	Contestability	• The openness of the evidence or evidence use process to critical questioning	• The subjection of the evidence process to peer review or public scrutiny
			• The subjection of expert conclusions to peer review or public scrutiny
		• The openness of the policy decision to appeal	• The establishment of formal appeals procedures and rules for decisions of evidence-synthesizing bodies
			• Inclusion or publication of dissenting opinions or alternative viewpoints

METHODS

Data

Table 2 summarizes the data used for the analysis.,,,, For the 2007 Canada Food Guide creation process, the analysis relied on 4 sources for information: first, the 2007 Canada Food Guide itself; second, a document published by Health Canada that details the history of the Canada Food Guides and their creation since 1947; third, a peer-reviewed journal article commissioned by Health Canada and published in Canadian Journal of Dietetic Practice and Research, which outlines the 2007 policy process; and fourth, a peer-reviewed journal article commissioned by Health Canada and published in Nutrition Reviews, which outlines the 2007 evidence intake pattern.

Table 2

Summary of data sources

	Reference title	Date	Author	Description
2007 Food Guide	History of Canada’s Food Guides: From 1942–200715	2007	Health Canada	A policy brief that summarizes the history of Canada Food Guides
	Eating Well with Canada’s Food Guide53	2007	Health Canada	The 2007 Food Guide Policy Document
	Eating Well with Canada’s Food Guide: “a tool for the times”54	2007	Bush et al	Journal article published on behalf of the Office of Nutrition Policy and Promotion
	Eating Well with Canada’s Food Guide (2007): development of the food intake pattern55	2007	Katamay et al	Journal article published on behalf of the Office of Nutrition Policy and Promotion
2020 Food Guide	2020 Canada Food Guide19	2020	Health Canada	The 2020 Food Guide Policy Document
	Evidence Review for Dietary Guidance: Summary of Results and Implications for Canada’s Food Guide16	2016	Health Canada	Summary of results from evidence review cycle
	Food, Nutrients and Health: Interim Evidence Update 2018 : For Health Professionals and Policy Makers18	2019	Health Canada	Updated summary of results from evidence review cycle
	Developing an evidence review cycle model for Canadian Dietary Guidance56	2016	Colapinto et al	Journal article published on behalf of the Office of Nutrition Policy and Promotion

Since it decided to update the Guide’s revision process in 2012, Health Canada has published several documents outlining the new process, including reports from stakeholder consultations, evidence review reports, and commissioned consultation reports, all of which are not available for previous Canada Food Guides. To ensure comparison across similar sources of data, the analysis relied on 4 sources of information for the 2020 food guide process that were similar to those available for 2007. They were the 2020 Guide, Health Canada’s review process document, an interim update on the evidence base, and a peer-reviewed journal article published in Journal of Nutrition Education and Behaviour that outlined the 2020 Guide’s creation process.

Analysis

The processes that created the 2007 and 2020 Guides were evaluated using document analysis to uncover the presence of the indicators, which were assumed to provide information on whether or not a principle had been met. Two readings of all the documents were conducted to carry out the analysis. First, an initial scan was used to gain a general sense of the framework and the 2 processes. In the second reading, documents were reviewed in depth to identify the presence of indicators and practices. In-text searches were conducted for each of the 8 principles within each document.

RESULTS

Prior to the adoption of a formal process for evidence review in 2012, revisions to the Canada Food Guide and its evidence base occurred periodically on an ad hoc and incremental basis. Every revision before 2012 required the establishment of new revision criteria, which were not held to any overarching standards. The new standards adopted in 2012 not only institutionalized a clear logic and process for reviewing evidence, but also included the need for periodic reviews of the review process itself. To varying extents, 7 of the 8 principles of the framework (all except for representation) were addressed in the new institutionalized process (Table 3). In total, the 2007 process appeared to meet 6 of the 28 indicators, while the 2020 process met 21 out of 28.

Table 3

Summary of indicators present in the 2007 and 2020 Canada Food Guide Process

Principle	2007 process	2020 process
Appropriateness	Goals sufficiently clarified	Goals sufficiently clarified
	Relevant decision criteria not stated at outseta	Relevant decision criteria stated at outset
	A variety of considerations prioritized	A variety of considerations prioritized
	Little indication of critical questioning of evidence sources in terms of their relevance and usea	Critical questioning of evidence sources in terms of their relevance and use
	Application of methods akin to those of multicriteria decision analysis	Application of methods akin to those of multicriteria decision analysis
	Little indication of differentiation between internal and external validity of evidencea	Differentiation between internal and external validity of evidence
	No required assessment of local applicability prior to utilizationa	Local applicability assessed prior to utilization
Quality	Unclear how quality criteria were applieda	Application of strict quality criteria that were appropriate to the research question and in accordance with methods akin to GRADE
	Food intake pattern relied on data modeling	Evidence of impacts ordered according to evidential hierarchies
	No statement and strategy for dealing with and assessing the quality of industry-funded researcha	A clear statement indicating that industry-funded research was explicitly excluded from the evidence base
Rigor	Unclear whether or what practices were used for systematic review, rapid review, realist review, or other synthesis methodsa	Process followed good practices for systematic review and other synthesis methods
	Criteria for evidence selection and inclusion uncleara	Selection and inclusion criteria clearly described
	Methods for formulating recommendations uncleara	Methods for formulating recommendations clearly described
Stewardship	The role of democratically elected agents or their representatives in the design or alteration of government evidence advisory bodies uncleara	The role of democratically elected agents or their representatives in the design or alteration of government evidence advisory bodies uncleara
	Unclear whether and how agents were accountable to the publica	Unclear whether and how agents were accountable to the publica
	Unclear how conflicts of interest were handled or mitigateda	Process explicitly banned conflicts of interest
	Lack of robust defenses against imposition of institutional structures by nonmandated or unaccountable agentsa	Process included robust defenses against imposition of institutional structures by nonmandated or unaccountable agents
Representation	Unclear if decision authority was with public representativesa	Unclear if decision authority was with public representativesa
	Unclear whether legislatures and representatives had ability to veto or override technical agencies when necessarya	Unclear whether legislatures and representatives had ability to veto or override technical agencies when necessarya
Transparency	No links to the journal articles that overview the review process and intake patterna	Information easily accessible
	One journal article was behind a paywalla	Information free to access
	Unable to locate publication of transcripts or minutes of expert body deliberationsa	Unable to locate publication of transcripts or minutes of expert body deliberationsa
Deliberation	Process sought passive public input on an ad hoc basis	Process sought passive public input through formalized deliberative mechanisms
	No active public engagementa	No active public engagementa
Contestability	Review process not subject to peer reviewa	Review process subject to peer review
	Expert conclusions subject to peer review	Expert conclusions subject to peer review
	No established formal appeals procedures or rules for decisions of evidence-synthesizing bodiesa	No established formal appeals procedures or rules for decisions of evidence-synthesizing bodiesa
	No inclusion or publication of dissenting opinions or alternative viewpointsa	Dissenting opinions and alternative viewpoints gathered during process available

Indicators that were absent in the documents analyzed are noted with an ‘a’.

The total number of indicators and the number of indicators met per principle in the 2020 process were higher than those of the 2007 process. At times, these increases were achieved by making the rules and criteria of the process more explicit. For example, only the 2020 process questioned the relevance of the evidence to the Canadian context, including Canada-specific socioeconomic, cultural, and lifestyle factors, and made clear how a framework similar to GRADE was applied to judge evidence quality. Rigor could be seen to be improved through an explicit application of systematic review methods, including 2 rounds of searching for relevant and valid information. New rules also changed the processes of stakeholder deliberation by limiting the ways that industry actors could influence the process. Industry influence was also mitigated in the new process by rules that explicitly banned conflicts of interest on the food guide advisory committee.

In some areas, there were fewer changes in the number of indicators present. For example, the relationship between the advisory group and public authority was not made clear in either the 2007 or 2020 processes. Nor was the relationship between the advisory committee and elected and government officials discussed, meaning both the 2007 and 2020 processes lacked these indicators. Finally, changes in relation to transparency permitted greater access and insight into the 2020 process than the 2007 process. For the 2020 process, all documents were published open access and centrally available through a Health Canada webpage. These documents included not only the 2020 Food Guide’s evidence base, but also a meticulous referencing of how evidence was assessed for quality and appropriateness, and how each piece of evidence was used to support specific recommendations.

DISCUSSION

Principal findings

The 2020 Guide creation process represents several improvements in multiple areas of evidence governance when compared with the 2007 Guide creation process, as assessed through our framework. The most significant aspect of this transformation was the institutionalization of an evidence review cycle, which mandates regular and periodic evidence reviews according to an overarching guiding framework, and that marginalizes the influence of industry by banning industry-commissioned reports in the evidence base (to ensure quality and reduce potential bias), restricting their ability to lobby the advisory committee, and prohibiting industry interests in committee memberships through financial conflicts of interests (ensuring stewardship). The framework proved instrumental for analyzing differences in evidence use between the 2 processes.

Some of the governance of evidence principles saw clearly greater change than others. In policy making, legitimacy is crucial for maintaining public trust, which can have significant ramifications for implementing and achieving policy outcomes. Indeed, the proximity of industry representatives to prior food guide creation processes, as well the inclusion of industry-commissioned reports in the food guide evidence base, were identified as potentially contributing to public distrust in previous food guides. Curtailing the influence of industry, while also incorporating more evidence and periodic updates on social challenges in Canada, could be seen to help achieve a range of governance principles in relation to an appropriate, rigorous, high-quality, and well-stewarded process. Many references were made to pursuing the highest quality of evidence available, and review reports meticulously detailed the evidence underlying the final policy output. These trends could be a result of improvement in research, better knowledge translation techniques, and/or a response to the removal of the influence of industry.

The Canada Food Guide’s previously exclusive focus on evidence that linked health to food consumption, while not considering social, economic, and environmental conditions, framed the policy problem primarily as one of uninformed individual behavior, solvable by more and better knowledge for consumers. But – as the new policy review process acknowledges – this focus occludes the various social realities in which individual consumption decisions are made. Indeed, the previous focus on individual consumption did not capture all the challenges that Canadian consumers faced in achieving healthy diets. By seeking evidence that also accounts for socioeconomic and cultural factors that affect food choices, Health Canada expanded its conception of the relationship between individual choices, food, and health within complex social settings., In doing so, it arguably achieved an improvement in the appropriateness of evidence utilized for the policy needs by broadening the type of evidence and sources of evidence included in the review in direct relation to social needs judged relevant. Furthermore, by including the use of evidence that relates to food choices and the environment, Health Canada expanded its conceptualization of the relationship between population health, consumption patterns, and the environment.

Policy implications

The findings suggest lessons for policy specific to the Canada Food Guide and to health and nutrition policy making globally. In relation to the Canada Food Guide, these findings suggest that the next review process could be further improved by targeting the areas of representation and deliberation. The shortcomings in deliberation could be improved by including more active participation mechanisms that establish ongoing dialogues between consumers and guideline developers. Shortcomings in representation could be improved if the role of publicly elected decision makers on or in relation to the advisory committee was more clearly established. Also, knowing more details on the composition of the advisory committee would enable assessing it against accepted design principles for scientific advisory committees.

Beyond these specific changes that can improve the Canada Food Guide, the analysis revealed 3 implications for improving evidence use in health policy more broadly. First, institutionalizing a process for evidence use that is grounded in democratic and scientific principles can provide opportunities to improve the use of evidence in the policy-making process in several key dimensions. Second, the Good Governance of Evidence framework can be useful for guiding and evaluating evidence use in nutrition policy processes. Finally, the analysis illustrates how improving evidence use in accordance with the good governance of evidence framework can improve the legitimacy of the policy process.

CONCLUSION

The recent revisions to the process that created the Canada Food Guide implemented in 2012 mark an improvement in evidence utilization when assessed through our framework. The framework synthesizes scientific and democratic principles and can act as both an evaluative and guiding tool for improving evidence use in the policy process. One limitation is that the principles used here are not universally agreed upon, but rather derive from a recent attempt to consider how to judge evidence use from multiple normative perspectives. Thus, the framework is only as convincing as far as one agrees with the principles that were selected as the most important for good governance. However, since this study provides a general method that can be translated across principles, a similar method could be followed for evaluation based on other important principles relevant to the subject or context at hand.

Ultimately, improvements to the Canada Food Guide revision process, gained from its newly institutionalized evidence-use mechanisms and processes, suggest that the institutionalization of evidentiary processes that are based on normative principles of democratic and scientific best practice can improve evidence use in policy-making processes. With the limitations noted above, the transformation of the Canada Food Guide can serve as a model for other areas of policy that seek to improve their evidence use, while the framework developed here can potentially be adapted or applied to guide evidence use in other policy processes.