Supporting the diffusion of healthy public policy in Canada: the Prevention Policies Directory.

Healthy public policy plays an essential role in a comprehensive public health approach to preventing cancer and chronic disease. Public policies spread through the 'policy diffusion' process, enabling governments to learn from another's enacted policy solutions. The Prevention Policies Directory (the Directory), an online database of municipal, provincial/territorial, and federal cancer and chronic disease prevention policies from across Canada, was developed to facilitate the diffusion of healthy public policies and support the work of prevention researchers, practitioners, and policy specialists. This information technology solution was implemented, through a participatory engagement approach, as a communication channel or policy knowledge transfer tool. It also addressed the intrinsic shortcomings of environmental scanning for policy surveillance and monitoring. A combination of quantitative web metrics and qualitative anecdotal evidence have illustrated that the Directory is becoming an important tool for healthy public policy surveillance and policy diffusion in Canada.

Introduction

Regulation through healthy public policy is a fundamental component of a comprehensive public health approach to addressing the mounting global burden of chronic diseases. The role of healthy public policies in health promotion gained momentum in 1986 through the Ottawa Charter where incorporating health into the mandate of policymakers across all sectors and at all jurisdictional levels was recognized as paramount to the prevention of chronic diseases [1]. Healthy public policies play a role in targeting and addressing the modifiable risk factors of chronic diseases, such as cancer, including alcohol consumption, exposure to occupational and environmental carcinogens, infectious agents, unhealthy eating, physical inactivity, tobacco use, and ultraviolet/ionizing radiation [2]. While the importance of policy action for health promotion and cancer and chronic disease prevention is clearly demonstrated through successes seen in areas such as tobacco control [3], policy adoption and non-adoption is characterized by a complex interplay of factors dependent on the unique characteristics in each different jurisdiction.

Diffusion of Health Policies

Policies are rarely adopted by governments based solely on internal considerations; instead policy decisions are often influenced by the choices made by other jurisdictions – a process referred to as policy diffusion [4]. The occurrence of policy diffusion has been demonstrated in a multitude of healthy public policy cases, such as wellness policies moving between school district and state levels [5] and through the spread of smoke-free spaces bylaws amongst municipalities in Alberta and Ontario [6]. Beyond an elementary understanding that policy diffusion exists, a growing collection of literature on the impetus behind the movement of policies has shed light on why policies diffuse.

Initially, the study of policy diffusion identified and focused on geographic proximity, with policies moving and being adopted amongst regional players in clusters or webs, as the primary causative agent behind the process [4,7]. While sometimes applicable – for instance jurisdictions in close proximity can share common population and environmental characteristics thus making policies appealing to neighbouring governments – there are numerous examples (described below) where policies have disseminated beyond neighbouring governments [4,7]. More recent theories have put forward policy diffusion drivers arguably more powerful than locality to explain the variability in diffusion patterns.

A modest number of studies have focused on the policies themselves and how characteristics such as the: relative advantage over existing policies; compatibility with current values, experiences and needs; complexity; ability to observe results; possibility of implementation on a trial basis; and perceived success in other jurisdictions affect the movement of policies between jurisdictions, [8,9]. Another school of policy diffusion thought examines the role political actors, individuals and associations, play in advocating for policy change and shifting the agendas of government as ‘policy entrepreneurs’ [10,11]. These theories that narrowly focus in on components of policy diffusion (e.g., the policy, stakeholders) as driving forces have largely been overshadowed by emerging work on the interrelationships between jurisdictions and how those relationships act as mechanisms for policy diffusion.

Four relationships (learning, imitation, coercion and competition) have been identified as motivators for governments to base their policy decisions on the choices made by other jurisdictions. The relational mechanism of learning is grounded in the concept that the successes and failures of adopted policies in one jurisdiction provide additional information about the consequences of similar policy action or inaction in other jurisdictions [12]. This has been demonstrated across diverse policy arenas, from criminal justice to the liberalization of emerging market economies [12,13]. A second mechanism based on the interplay between governments is imitation. Imitation occurs when a jurisdiction ‘copies’ the policy choice of an external body regardless of any observed or learned outcomes often due to the similarities among the political and ideological environments of the jurisdictions in question or as a result of an overwhelming perception that the policy is a ‘social norm’ or the ‘right thing to do’ [7,14]. The third policy diffusion motivator based on governmental relationships, coercion, involves a concerted effort by one agent (e.g., government, financial institution, international organizations, etc.) to directly influence the policy decisions of another through the employment of threats or incentives [4,15]. Competition, the final relational driver, captures the struggle between governments in contending for economic advantages through taxes and attracting business [16-18]. Policies that bestow a benefit on one jurisdiction, for example through establishing a lottery system and thus a new revenue stream that has the capacity to poach money from neighbours without a lottery, are thought to quickly diffuse amongst competitors in a race between governments for larger tax bases and financial gains [16-18]. These four relational mechanisms, in conjunction with theories describing the impact of policy entrepreneurs and the attributes of a policy, all have the capacity to affect the process of policy diffusion. But what does the policy diffusion process itself look like?

Many scholars subscribe to variations of the Diffusion of Innovations (DI) theory to model how the policy diffusion mechanisms outlined above alter the relative effectiveness and payoffs of a policy choice in the minds of decision-makers [19-21]. The result is key actors in the policy process are influenced to adopt or reject a policy alternative and ultimately decide whether a policy successfully diffuses. Beyond describing the interaction between policy diffusion mechanisms and decision-makers, the DI theory also explains the diffusion of an innovation process through the combination of four key components: (1) the innovation, (2) communicated via an assortment of channels, (3) over time, (4) amongst a social system [22]. Thus, in the context of policy diffusion the DI theory would see policy decision-makers (the social system) learn about an innovation (the policy) from another jurisdiction. This learning would occur through one or more communication channels (e.g., newspapers, colleagues). The decision-makers weigh the relative effectiveness and payoffs of the policy, determined by mechanisms such as policy attributes, policy entrepreneurs and relational factors between jurisdictions (i.e., competition, learning, coercion, or imitation), and after a period of time decide whether to adopt the policy. While all four components are required for policy diffusion, the communication channel component is particularly important because it makes the previously unaware social system of policy decision-makers within a jurisdiction conscious of possible external policy options and therefore sets policy diffusion in motion [22]. Furthermore, the Internet has revolutionized communication channels with the enablement of quick access to vast amounts of specialized information, resulting in reduced decision times around policy adoption [22].

The concept of a communication channel can be explained through the lens of knowledge management as well. Knowledge transfer (KT) is described as the movement of knowledge between a source and a receiver, with the receiver accumulating new knowledge [23]. Explained this way, a policy option is a unit of knowledge that can be transferred from a source – a colleague or newspaper for instance – to the receiving decision-makers in other jurisdictions. In this example, the source plays the role of the communication channel outlined in the DI theory. Considering that communication channels, in a way, play a gateway role in the policy diffusion process, a communication channel centered on healthy public policy would support the diffusion of such policies. The Prevention Policies Directory (the Directory), an online KT tool, was constructed to fulfill this role as a specialized communication channel facilitating the diffusion of healthy public policies amongst Canadian jurisdictions.

The Prevention Policies Directory

Recognizing the importance of policy action on cancer prevention, in 2009 the World Cancer Research Fund (WCRF) and the American Institute for Cancer Research (AICR) published an extensive document entitled Policy and Action for Cancer Prevention: Food, Nutrition, and Physical Activity, containing 48 cancer prevention policy recommendations across nine different sectors [24]. The Canadian Partnership Against Cancer (the Partnership), funded by the Canadian federal government to accelerate action on cancer control and to implement Canada’s national cancer control strategy, commissioned a series of environmental scans to understand the Canadian cancer and chronic disease prevention policy landscape in an effort to translate the WCRF/AICR recommendations into a Canadian context. The environmental scans, focusing on the modifiable risk factors (e.g., nutrition, physical activity, environmental and occupational exposures, etc.) for cancer and chronic disease, yielded 771 policy initiatives from federal, provincial/territorial, municipal, and school governing bodies from across Canada between 1997 and 2007 (and as far back as the 1980s for toxic use reduction policies) [25-27]. Results of the environmental scans highlighted areas where Canadian policy makers had a burgeoning appetite to act; however, a key finding was the need for expanded monitoring and surveillance of Canadian cancer and chronic disease prevention policies to build capacity for tracking progress as well as identifying new policy approaches [25,26]. Essentially, this recommendation called for a mechanism, or communication channel, for Canadian research, practice, and policy specialists to learn of up-to-date policy options outside their jurisdiction.

It was apparent that further environmental scans were not the solution to address this required need around policy surveillance. Despite the valuable snapshot of the Canadian cancer and chronic disease prevention policy landscape provided by the environmental scans, these scans were time-consuming, resource intensive and ultimately possessed a very limited shelf life. Due to the ever-changing nature of policy development, environmental scans are often out-of-date by the time they are published. In addition, the scans did not provide a communication channel among research, practice, and policy specialists in different jurisdictions. Environmental scan reports are usually disseminated through traditional knowledge dissemination means such as professional presentations and interactions, and therefore would reasonably be expected to have a more limited effect on the diffusion of the collected policy information to new jurisdictions.

With these challenges of collecting and communicating policy information in mind, a new Canadian conduit for the transfer and exchange of cancer and chronic disease prevention policy information, with capacity for regular updating, was envisioned. The National Cancer Institute’s State Cancer Legislative Database (SCLD), a repository of state-level cancer-related health policies established in 1989 in the United States [28], provided a model for the creation of a similar Canadian resource: the Prevention Policies Directory (the Directory). Although the scope of the SCLD, which encompassed state policy information addressing topics such as access to cancer treatment, health disparities, genetics, cancer registries, tobacco use and a range of specific cancer disease sites, differed from the prevention of cancer and chronic disease via modifiable risk factor approach conceived for the Directory, the SCLD was an appropriate model for the development of an online database of Canadian healthy public policy alternatives.

In order to further ensure the utility of the Directory, research, practice, and policy specialists were involved throughout the development in a participatory engagement approach. Numerous methods, from across the social sciences, have been identified in which participants are purposefully engaged throughout a research, planning, or evaluation process and all share the objective of a final product with the greatest value to participants and creators [29]. This is accomplished via participant engagement methodologies by including the perspectives and experiences of future participants to ensure the needs of the target audience are met, in turn resulting in greater uptake, impact, and success [30,31]. Research, practice, and policy specialists helped shape the Directory, through early and frequent engagement, explained in greater detail later in this paper, into a useful healthy public policy tool.

The Directory was conceived and created through the combination of four key elements that included a foundation in policy diffusion theory, findings from several environmental scans calling for increased policy surveillance and tracking, the precedent set by the SCLD in the United States, and the participatory engagement of research, practice and policy specialists to meet the needs of the intended target audiences. The result is a policy diffusion communication channel, or KT tool, that regularly collects and updates prevention policies from jurisdictions across Canada related to the modifiable risk factors associated with cancer and chronic diseases and indexes them within an online, bilingual, and searchable database.

Methods

Directory Design

In designing the Directory, the web-based SCLD and the shortcomings of the policy environmental scans were instrumental. Internal IT specialists were responsible for developing the online user interface for the Directory (Figure 1) that would enable research, practice, and policy specialists to explore the database by any combination of user-defined keywords and four search fields: risk factor, policy type, jurisdiction, and geographic location. With the Directory positioned as an online resource, following the SCLD model, there was an opportunity to address limitations encountered through the environmental scanning approach associated with policy monitoring, primarily the large investment of time required and the quick obsolescence, through a custom web crawling IT solution. Built by an external IT firm with unique legal and policy expertise, the custom web crawler automated the policy scanning operation by applying a set of search criteria to find and capture relevant documents on, currently fifty, prescribed websites. This allowed policy scanning to occur twenty-four hours a day and seven days a week with no human resources required, alleviating some of the resource-intensive burden associated with environmental scanning. Moreover, the continuous nature of the web crawler allowed for regular updating of the Directory to prevent content from becoming outdated.

Figure 1

The Prevention Policies Directory website landing page, which includes the search fields to access over 1,700 Canadian cancer and chronic disease prevention policies.

Part and parcel of having a web crawler conduct automated policy scanning was the requirement of a robust quality assurance procedure to ensure the Directory was populated with credible and relevant policy information for the Directory. A set of inclusion criteria (Figure 2) defined the scope of the Directory and provided a checklist for a quality assurance analyst to cross-reference against policies captured by the web crawler to determine relevancy. The inclusion criteria ensured that Directory content was solely composed of seven policy document types and were adopted within Canadian jurisdictions. It also required that content was related to prevention, and specifically the prevention of cancer and/or chronic diseases. Finally, content had to be related to one of eight modifiable risk factors associated with cancer and chronic disease, or be aimed at more broadly impacting public health and health promotion, such as Public Health Acts. When captured policies met the inclusion criteria they were considered relevant and published to the online Directory.

Figure 2

Prevention Policies Directory Inclusion Criteria

An important challenge in constructing a resource aimed at knowledge transfer and exchange between jurisdictions across Canada is being inclusive of both official languages: English and French. This was addressed through the Directory design in a two-step fashion. First, the web crawler was able to employ both English and French search terms and therefore was able to capture documents in both languages. Secondly, a bilingual analyst was responsible for implementing the quality assurance criteria against captured French content. While bilingual human resources and skills were still necessary for the quality assurance procedure, the bilingual nature of the web crawler provided an innovative way to readily scan for policies in more than one language.

Target Audiences

The Directory is positioned as a tool for research, practice, and policy specialists working in the area of chronic disease prevention and the related modifiable risk factors. These target audiences were engaged continuously to inform the scope and direction of the Directory, primarily via two mechanisms: formal external advisory structures and ad hoc expert consultations. A steering committee was formed at the outset of the development work with a mandate to provide advice regarding the implementation of the web crawler as an automated monitoring mechanism for chronic disease and cancer policies and relevant knowledge transfer and exchange strategies. Membership on the pan-Canadian steering committee consisted of twelve individual experts in cancer and chronic disease prevention and the modifiable risk factors. The members also represented the research, practice, and policy communities and came from a variety of academic, non-governmental organizations, and government organizations. These experts played a significant role in brainstorming search terms sufficient for finding cancer and chronic disease prevention policies, as well as a list of relevant Canadian web sites where those policy documents were hosted. Furthermore, the group of advisors was consulted on issues of scope, for instance the decision to include municipal content (e.g., bylaws and policies) beginning with a selection of Canadian municipalities and the resolution to exclude school board level policies due in large part to web accessibility difficulties. The steering committee was also provided an opportunity to have regular updates with members of the target audience to ensure the Directory’s development remained aligned with their needs.

Apart from the formal steering committee, several decision points arose where additional advice was solicited from the target audiences on an as-needed basis. In the case of usability testing, a mix of research, practice, and policy specialists, ranging in familiarity with the Directory, were utilized to evaluate the user experience in working with the tool. When building or changing the search criteria, in addition to guidance from the steering committee, experts from the cancer and chronic disease modifiable risk factor areas were enlisted to help with the process. By engaging cancer and chronic disease prevention experts from the research, practice, and policy communities other than those present on the steering committee, fresh and different perspectives were incorporated into Directory-development decision-making. This participatory engagement approach attuned the development of the Directory to the intended target audiences and at the same time built a network of champions comprised of invested research, practice, and policy specialists knowledgeable with the resource.

Phase 1: Initial Launch

The development of the Prevention Policies Directory can be separated into three phases: the Initial Launch Phase, the Search Methodology Refresh Phase, and the Expansion to Municipalities Phase. The Setup Phase consisted of building the required information technology foundation for the new tool – previously described in the Directory Design section – as well as determining the Directory’s scope and search methodology to be employed by the web crawler. The involvement of expert advisors in the development of the search terms and relevant web site list through the participatory engagement approach, as previously described produced a search methodology consisting of 138 keywords to be applied to 280 web sites, representing federal, provincial and territorial policy sources. The final step in preparing the search methodology was the French cultural translation of the keywords. With these two critical pieces in place, the Directory was launched in the Spring of 2010.

Phase 2: Search Methodology Refresh

Approximately two years into the Directory’s operation, and based on continuous evaluation of the web crawler monitoring platform, the search methodology was refreshed. The change in search methodology applied by the web crawler consisted of a shift from simple keyword searches to search strings incorporating Boolean operators. Boolean operators function by relating individual search terms to each other to influence the retrieved results, either by reducing, expanding or improving the precision of those results [32]. For example, Figure 3 shows how a search string for active transportation (under the built environment risk factor) was created by combining a risk factor search term prefix with a policy search term suffix. The “OR” Boolean operator linked affiliated risk factor and policy terms to form a prefix and suffix respectively, while the “AND” Boolean operator combined the two into a search string – requiring that at least one term from the prefix and one from the suffix would need to be identified in a document for it to be deemed relevant. The combination of risk factor terms with policy terms in Boolean search strings equipped the web crawler to not only locate documents pertaining to the modifiable risk factors of cancer and chronic disease, which the previous keyword methodology accomplished, but to specifically recognize policy documents dealing with the cancer and chronic disease risk factors. Also of note in the construction of the search strings, the Boolean “AND NOT” operator enabled the exclusion of common terms found on the internet, such as “privacy policy”. Expert advice was leveraged to create 101 search strings which in turn underwent the French cultural translation process applied to the original keywords, and in 2012 the Directory was re-launched with a new search methodology.

Figure 3

Exemplar Boolean Search String for Active Transportation

Phase 3: Expansion to Municipalities

The latest evolution in the Directory’s development was an expansion of the scope of policy content. After the launch, through ongoing discussions with steering committee experts and feedback from the Directory’s target audience, the need for access to municipal policy content by research, practice, and policy specialists was identified. The importance of municipal content to research, practice and policy specialists was succinctly put by U.S. Supreme Court Justice Louis Brandeis when he described how a local government could act as a, “laboratory; and try novel social and economic experiments” [33]. Variability in the policies adopted at the local level provide provincial, territorial and federal governments with more sources of evidence to inform their own decisions [34]. In fact, state and provincial governments have routinely looked to local level policies as trial runs and used their success or failure to inform their own political decision in a vertical policy diffusion process [35]. This upward diffusion has been demonstrated in smoke-free spaces policies in the United States and in the restriction of cosmetic pesticide use in Canada [34,36]. By expanding the Directory’s scope to include municipal content, emerging policy directions not yet on the provincial, territorial or federal radar would be captured and available to research, practice, and policy specialists.

As of 2011, there were 5,253 municipalities in Canada [37]. Scanning all of these jurisdictions for municipal bylaws and policies was not feasible for a variety of reasons. For instance, the time required by the web crawler to search all of the relevant web sites and web pages, the time required by the analyst who would review captured policies, and the simple fact that many Canadian municipalities lack up-to-date, online archives of their bylaws were barriers to collecting municipal content for the Directory. In consultation with the Urban Public Health Network (UPHN), a group of Chief Medical Officers of Health from the eighteen largest Canadian cities, a collection of thirty-one municipalities were initially selected for inclusion (Table). The thirty-one municipalities constituted the eighteen UPHN member municipalities plus thirteen additional municipalities to ensure each province and territory was represented by at least two municipalities. The municipalities selected for inclusion in the Directory’s scope showcase a wide range of municipal environments present in Canada, from large urban centres with populations in the millions (e.g., Calgary, Montreal, and Toronto) to small provincial cities/towns with populations under thirty thousand (e.g., Conception Bay South and Summerside) to remote northern communities with less than five thousand inhabitants (e.g., Arviat and Hay River). With a pan-Canadian range, the selected municipalities also provide a diverse representation of demographics, socioeconomic status, industries, natural environments, political considerations, and more. The expansion in scope to include municipal content, following the shift of the search methodology to Boolean search strings, signaled the final chapter in the Directory’s development.

Results

Content

With development complete, a significant amount of policy content has already been reviewed and published to the Directory. By December 2013, 1,742 policies had been published to the Directory, 50 of which were unilingual French policies and the other 1,692 a mix of English and bilingual policy documents. The Directory’s policies are categorized according to four dimensions: risk factor, policy type, geographic location, and jurisdiction (previously discussed as search options for Directory users).

Figure 4 shows the breakdown of policies in the Directory by the risk factors they act upon to prevent cancer and chronic disease. Policies can address multiple risk factors, therefore when examining the number of policies in each risk factor the total will sum to more than the number of policies indexed in the Directory – this is the only dimension where this is the case. Also of note is the ‘General’ risk factor, which is not a modifiable risk factor of cancer or chronic disease, but a category where broad policies affecting health promotion and cancer and chronic disease prevention capacity and authority can be found, such as provincial and territorial Public Health Acts. Occupational and environmental exposures was the risk factor with the largest number of policies (n=623, 29%) at the time of publication, with built environment policies the second largest category (n=355, 16%). Cancer-causing infectious agents (e.g., Hepatitis C, Human Papillomavirus, etc.) prevention policies number the fewest in the Directory with only forty-seven (2%).

Figure 4

Prevention Policies Directory Content - Policies by risk factor

However, the prevalence of policies by risk factor, seen in Figure 4, is not consistent across each level of government in Canada. Figure 5 shows the 146 federal policies in the Directory according to the risk factor they address. Figure 5 illustrates that at the Federal level, there are no built environment policies, while nearly half (n=70, 48%) of the federal policies deal with occupational and environmental exposures. While the occupational and environmental exposures category remains the largest, as in the cross-section including all jurisdictions combined (Figure 4), the federal level is more skewed with 80% of the policies addressing occupational and environmental exposures, nutrition (n=20, 14%), and tobacco use (n=26, 18%) categories.

Figure 5

Prevention Policies Directory Content - Policies by risk factor (federal) Note: There are no built environment policies at the Federal jurisdiction level.

The spread of the 1,376 policies by risk factor at the provincial/territorial level (Figure 6) closely resembles the cross-section of all jurisdictions (Figure 4). Again, the occupational and environmental exposure group is the largest (n=395, 29%) and cancer-causing infectious agents prevention policies make-up the smallest group (n=39, 3%). However, the remaining seven risk factors are relatively uniform in that they each account for between 9-12%.

Figure 6

Prevention Policies Directory Content - Policies by risk factor (provincial/territorial)

The final jurisdictional slice of 664 municipal policies by risk factor is shown in Figure 7. The variation of policies by risk factor differs from the federal level and at this level the built environment risk factor is the largest group of policies (n=233, 35%). Similar to the federal level, three risk factors, built environment (n= 233, 3%), occupational and environmental exposures (n=158, 24%), and physical activity (n=122, 18%) account for the majority of municipal policies within the Directory. The infectious agents and general categories are the smallest at the municipal level each representing 1% of the policies.

Figure 7

Prevention Policies Directory Content - Policies by risk factor (municipal)

Seven different types of policy documents are within the scope of the Directory; the definitions of each can be found in Table 2. The types of included policy documents run the gamut of legal instruments (e.g., statutes, regulations, and bylaws), to so-called ‘soft’ policies that act to guide and standardize decision-making, to evaluations of implemented policies. Figure 8 shows that the majority of the Directory is composed of the three legal instruments (79%) with federal, provincial and territorial statutes and regulations responsible for 58% of the policy content. Codes account for only 1% of the Directory’s make-up.

Table 2

Types of Policy Documents Indexed by the Prevention Policies Directory

Legal Instruments

Bylaw

A subordinate legislation made by any authority subordinated to a legislature. The most frequently referenced bylaws are those made by municipalities.

Regulation

A subordinate legislation adopted by a government according to lawmaking powers conferred by a statute. Regulations clarify information that is found in the statute by providing more details or definitions.

Statute

An act of a legislature adopted pursuant to constitutional authority. These are written laws that are also referred to as Acts or legislation. Statutes usually permit the enactment of regulations. In terms of hierarchy, these have more legal authority than regulations, bylaws, or bills (in that order).

Other Policies

Bill

A document submitted to a legislature for its consideration and/or enactment. It is simply a proposed piece of legislation and is subject to change before it is enacted as a statute. Unlike statutes, regulations, or bylaws, they are not binding legal instruments.

Code

Any set of standards put forth and enforced by government for the protection of public safety and health (i.e., building codes for ventilation or sanitary requirements).

Policy

A set of organizational rules intended to promote health. These can be in the form of a plan or a course of action.

Other

Evaluation

Any document that provides a progress or final report on an implemented policy with the aim of describing its efficacy and achievement in reaching its prescribed goals and objectives, excluding academic publications.

Figure 8

Prevention Policies Directory Content - Policies by policy type

Evaluation Findings

The Directory is evaluated both as an IT tool and as a KT vehicle. Tracking the usage of the Directory with web metrics gives a sense of whether the tool is being used and the size of the audience. Since the search refresh and municipal policy addition phases, from April 2012 to March 2013, there have been 3,815 unique visits to the Directory (Google Analytics - an average of 318 unique visits per month) as illustrated in Table 3. When compared to the web traffic experienced by the SCLD (2,200 unique visits per month),the Directory’s web traffic is in line with the level of usage in the United States as Canada has a population that is approximately 10% the size of the United States population. The total number of visitors to the website (n = 5,374) was also split into first-time visitors to the Directory (n = 1,266) and repeat visitors (n = 4,108) who access the website at least one more time after their initial visit. Over the same time period, a total of 2,945 searches of the Directory were completed (Table 3). Among all visitors to the Directory, 54.8% conducted a search.

Table 3

Prevention Policies Directory Web Metrics

Web Metrics (2012-2013)
Unique Visitors	3,815
Unique Visitors per Month	318
New Visitors	1266 (23.56%)
Repeat Visitors	4108 (76.44%)
Total Visitors	5374
Total Searches	2945

Web metric data provides insight into the usage of the Prevention Policies Directory, but of equal, if not greater importance, is the impact the Directory has on policy decision-making. One useful indicator for measuring the impact of the Directory is tracking when it is cited in a report or document from another organization. For example, the Directory has been cited on numerous occasions by several publications, including the journal Healthcare Quarterly, as a useful tool and credible source of policy information [38-40]. In fact, the Directory has also been cited as a resource used in healthy public policy analyses [41,42]. Further to documented citations, anecdotal evidence that the Directory has played a role in policy diffusion and political decision-making is also valuable. One example from the eastern coast of Canada involved a municipal jurisdiction using the Directory to find a policy restricting alcohol advertising on public transit property from a central Canadian municipality and used this policy to inform their own alcohol advertising policy development and council debates. Through a variety of citations and qualitative evidence, the Directory is increasingly being recognized as a valuable resource in policy-making and is demonstrating its influence on Canadian healthy public policy diffusion.

Limitations

While the utility of the Directory lies in its ability to describe the Canadian policy landscape compiled through examples of cancer prevention policies, the absolute counts of policies should not be interpreted as reflecting the real prevalence of policies in a given jurisdiction or for a given risk factor area. The Directory’s content is populated with a convenience sample of policies, in part determined by the search methodology that includes prescribed websites for the web crawler to search. The selected websites are pre-vetted to ensure that the website contains policy content and that the website architecture is accessible by the web crawler. There remains the likelihood that additional relevant policies in a given jurisdiction or risk factor area have not been captured due to this web-crawling method of monitoring and surveillance. This limitation will continue to be mitigated in the future through the addition of new websites to search as the Directory continues to mature. Nonetheless, the relative proportions and examples of policies in a given jurisdiction or risk factor area provide a basis for informing comparative policy analysis and policy decision-making.

There also remains the obstacle, as mentioned earlier in the description of the municipalities expansion phase of the Directory’s development, that it is not feasible to systematically collect policies from the websites of 5,243 Canadian municipalities (pre-supposing that they all have websites with up-to-date policies). With municipalities acting as important sources for emerging public health policy solutions, this is an important gap in the positioning of the Directory as a credible and valuable resource. To this end, a mechanism allowing user contributed submissions from any municipality in Canada to a complementary database and plotted on a map was developed. This approach is described in the ‘Further Developments’ section below.

Another limitation, with respect to the evaluation of the Directory, is understanding the role it plays in the policy diffusion and policy development process. Using web metric data in conjunction with tracking citations and qualitative anecdotes provides clear evidence of the tool’s reach and impact; however, the value the Directory adds to specific policy decision-making processes is currently undocumented. Conducting key informant interviews with members of the Directory’s target audience who have indicated they have used the resource may provide a way to collect qualitative case studies on the role this resource plays in the policy decision-making process.

Discussion and Conclusion

Challenges and Opportunities

In creating a tool to facilitate the diffusion of healthy public policies in Canada and act as a policy surveillance platform, challenges included: the resource burden of scanning for policy documents, accommodating a bilingual and national audience, and ensuring that the Directory is valuable to research, practice, and policy specialists whose work impacts chronic disease prevention (e.g., urban planners, transportation engineers, environmentalists, etc.). The custom web crawler was a unique IT solution aimed at reducing the required time and human resources associated with environmental scanning. The web crawler also addressed finding French policies in addition to English, through the easy application of a bilingual search methodology, enabling the Directory to meet the needs of a national Canadian audience.

However, an IT approach was not without its own set of obstacles, chief among them is the differences between public health and IT vocabulary. To mitigate the effects of a language barrier between the external IT firm and internal public health specialists, in-house expertise was mobilized to ensure effective communication. It was crucial to have expertise in public health, but it was equally as important to have expertise in IT to comprehend technical terminology and translate expectations to the vendor. In addition, internal IT specialists were heavily relied upon for their expertise in dealing with the external IT firm, through reviewing technical documents and consulting prior to and after updates to the Directory’s infrastructure. Recognizing the differences in the languages of IT and public health was a key step that allowed internal resources to be leveraged to minimize this barrier.

There were additional human resources and financial considerations associated with the web crawler and database IT solutions that may impact the feasibility of utilizing this approach in different contexts. For instance, while the web crawler automated the search methodology and the database facilitates the organization of a large volume of documents, human resources were still necessary for reviewing content. A full-time analyst was required to implement the quality assurance process and manage the database; French documents were handled by a part-time bilingual analyst. In the same vein, unless an organization has existing internal IT capacity for web crawling, financial resources could be a barrier to replicating the implementation of this custom IT solution. Costs are not only connected with the development of a web crawler, but also the ongoing hosting, maintenance and support agreements required to continue its operation. These human and financial resource considerations must be made when determining whether an IT option is the correct solution.

Opportunities stem from the value of the Directory which lies in part in the credibility and comprehensiveness of the information it contains. With over 1,700 policies from three levels of government in jurisdictions across Canada, the Directory provides a large cross-section of the healthy public policy landscape in the country. Already it contains nearly 1,000 more cancer and chronic disease prevention policies than were captured in the three environmental scans used to inform the Directory’s development. The breakdown of the policies within the Directory by risk factor showcases the ability of the resource to provide a summary of the Canadian policy landscape. From the policy cross-sections provided in Figures 5, 6, and 7, it can be surmised which areas of cancer prevention each jurisdictional level is, or has been, active within. For instance, nearly half of the federal policies in the Directory are related to occupational and environmental exposures which reflects the role this level of government plays in regulating chemicals, products, and environmental pollutants. At the same time, the provincial/territorial level is characterized by greater uniformity across the risk factors and suggests this jurisdiction acts in each, but does not necessarily focus specifically on any particular cancer prevention domain. Likewise, the municipal level, which is the level of government closest to local infrastructure planning and development, is the only jurisdiction to have a majority of built environment policies. On the other hand, risk factor areas with fewer policies indicate untapped policy realms, potential areas for action. In the case of alcohol policy at the municipal level, a relatively small number of policies (n = 15, 2%) were adopted in the 31 municipalities scanned by the Directory at the time of this analysis, which suggests that alcohol control is a policy arena that municipalities have entered, but are not necessarily fully engaged in as of yet [43].

However, one caveat with this analysis is that hundreds more policies have been captured by the Directory’s web crawler and await rigorous quality assurance review. For instance, at the time this article was written, 875 federal government documents have been flagged for assessment. The fact that 7% of the total number of policies in the Directory are from the federal government, at the time of this article’s publication, affirms that the backlog of policy content is the leading opportunity to improve the resource’s integrity. It is also indicative of the leading challenge – efficient human resources able to keep up with the policies being identified by the regular web crawls. With the development of the Directory completed, the focus has shifted to aggressively adding and updating content. At the time of writing, all relevant policies for the 31 included municipalities and eight provinces and territories have been added to the Directory. It is anticipated that the quality assurance process for the four remaining provinces and territories and federal jurisdictions will be complete by March 2014.

A participatory engagement approach to the development of the Directory proved particularly fruitful in the uptake of the resource by researchers, practitioners, and policy specialists. By engaging members of the target audience at all stages of the construction of the tool, the Directory has been shaped by its eventual users and made as utilitarian for that group as possible. This is evidenced, in part, by the large number of repeat visitors. The “unique visitors” indicator provides a measure of the number of individuals aware of the Directory and the size of the audience it has reached, while the “new visits” indicator demonstrates how the audience is growing. Perhaps the most important indicator is “repeat visitors”, which represents the core users who value, and return to use, the tool. Over 75% of the 5,374 user visits to the Directory were repeat visitors suggesting it resonates with the target audience and addresses a need amongst the user base. Furthermore, citations and mixed methods qualitative feedback marking the Directory as a credible and reliable source of prevention policy information lends credence to the notion that participatory engagement in the design and implementation of the Directory has helped create a resource of value to its target audience.

Further Developments

With development complete and the Directory positioned as an important resource for Canadian researchers, practitioners and policy specialists, the focus has shifted to knowledge transfer and exchange (KTE) efforts. KTE activities will play a significant role in adding value to, and repackaging, the wealth of existing policy information in the Directory to increase its utility amongst the target audience. From consultations with, and feedback from, the research, practice, and policy communities, evidence related to healthy public policy was identified as a clear priority. To address this need, two pathways forward were devised; one would see policies in the Directory linked directly to sources of evaluation evidence of the need for the policy or its impact, and the other was a mechanism to collect contextual policy process evidence to shed light on policy development challenges and successes.

A partnership was formed with CAREX Canada, a national surveillance project funded by CPAC’s Prevention Program that estimates the exposure of Canadians to carcinogenic substances in the workplace and community environments, to explore the feasibility of linking Directory policies to a source of exposure evidence. CAREX Canada provides profiles and estimates for 85 carcinogens that detail carcinogenic evidence, uses of the substances, and potential for exposure to Canadians. Many of the 85 carcinogens CAREX Canada monitors are regulated to some degree by the occupational and environmental exposure policies in the Directory. As a first step in linking Directory policies to sources of evidence, the occupational and environmental exposure policies have been cross-linked with the CAREX Canada profiles and estimates. In practice this entailed adding hyper linkages between occupational and environmental exposure policies that address one or more of the 85 carcinogens and the corresponding CAREX Canada carcinogen profiles and vice-versa. Linkages between the Directory and CAREX Canada will provide users of the CAREX Canada carcinogen profiles with examples of regulatory action taken across Canada, while Directory users will have access to carcinogen exposure evidence providing some context for why regulation is, or is not, occurring.

The second KTE activity will provide a mechanism to capture qualitative evidence around the policy development and implementation processes involved in enacting healthy public policies as well as provide a space for collaboration amongst researchers, practitioners, and policy specialists. Using Google Maps, Directory policies are being plotted across the country to create policy maps on specific risk factor subjects. In addition to providing a visually enhanced interface to view policies in the Directory, functionality for users to submit their own policies will address scoping limitations of the Directory (e.g., 31 municipalities out of over 5,000 in Canada). User contributions will also enable context-specific, policy development and implementation process evidence related to healthy public policies to be captured. While policy-related evidence was a priority identified through exchanges with the research, practice, and policy communities, the experiences of colleagues with the policy process were viewed as particularly valuable. By enabling users to contribute their own policy development and implementation information to a map, they can include, context-specific information around the challenges faced and the solutions employed through the policy process. The first policy map was recently launched and focused on municipal-level active transportation policies. Future maps may target other aspects of the built environment, specific carcinogens addressed by environmental and occupational exposure policies, or other policy areas related to the cancer and chronic disease modifiable risk factors where the demand from users for this type of information is high.

Conclusion

The Directory, as a centralized source of existing Canadian healthy public policies, has proven an important tool for cancer and chronic disease prevention researchers, practitioners, and policy specialists. A higher proportion of Directory searches amongst return users coupled with numerous citations and user-reported anecdotal evidence supports this conclusion. Additional developments have focused on expanding the credible content in the Directory, as well as leveraging that information through the production of knowledge products, such as cross-linkages with sources of evidence and policy maps. The multifaceted approach that informed the development of the Directory, consisting of combining policy diffusion theory, addressing gaps in conventional environmental scanning and policy surveillance, and utilizing participatory engagement to position the tool to best serve the needs of its target audience, played a key role in this robust outlook for the Directory moving forward.

Availability

The Prevention Policies Directory can be found at: http://www.cancerview.ca/preventionpolicies.

Table 1

Prevention Policies Directory’s 31 Municipalities

Note: Italicized municipalities are UPHN members. * Technical difficulties with website architecture on the websites of the largest municipalities in Yukon have prevented the inclusion of more than one municipality for that territory.

Province/Territory	Municipality
Alberta	• Calgary• Edmonton
British Columbia	• Surrey• Vancouver• Victoria
Manitoba	• Brandon• Winnipeg
New Brunswick	• Fredericton• Moncton• Saint John
Newfoundland and Labrador	• Conception Bay South• St. John’s
Northwest Territories	• Hay River• Yellowknife
Nova Scotia	• Halifax• Region of Queens Municipality
Nunavut	• Arviat• Iqaluit
Ontario	• Hamilton• London• Ottawa• Regional Municipality of Peel• Toronto
Prince Edward Island	• Charlottetown• Summerside
Quebec	• Longueuil• Montreal• Quebec City
Saskatchewan	• Regina• Saskatoon
Yukon*	• Whitehorse

Box 1: Key Requirements for a Web-based Policy Scanning and Document Management Tool

Key Requirements for a Web-based Policy Scanning and Document Management Tool

Identify the extent to which relevant policy documents are available online

Develop custom IT solution to web crawl for policy documents

Develop effective search methodology for web crawler to find relevant policy documents

Secure human resources for monitoring and quality assurance, and financial resources for ongoing hosting, maintenance, and support of custom IT solution

Engage target audience/knowledge users early and regularly to align tool with audience needs

Evaluate impact of tool through combination of quantitative web metrics and qualitative indicators (e.g., citations, anecdotal user statements, etc.)