Improving public health policy through infection transmission modelling: Guidelines for creating a Community of Practice.

BACKGROUND: Despite significant research efforts in Canada, real application of modelling in public health decision making and practice has not yet met its full potential. There is still room to better address the diversity of the Canadian population and ensure that research outcomes are translated for use within their relevant contexts.
OBJECTIVES: To strengthen connections to public health practice and to broaden its scope, the Pandemic Influenza Outbreak Research Modelling team partnered with the National Collaborating Centre for Infectious Diseases to hold a national workshop. Its objectives were to: understand areas where modelling terms, methods and results are unclear; share information on how modelling can best be used in informing policy and improving practice, particularly regarding the ways to integrate a focus on health equity considerations; and sustain and advance collaborative work in the development and application of modelling in public health.
METHOD: The Use of Mathematical Modelling in Public Health Decision Making for Infectious Diseases workshop brought together research modellers, public health professionals, policymakers and other experts from across the country. Invited presentations set the context for topical discussions in three sessions. A final session generated reflections and recommendations for new opportunities and tasks.
CONCLUSIONS: Gaps in content and research include the lack of standard frameworks and a glossary for infectious disease modelling. Consistency in terminology, clear articulation of model parameters and assumptions, and sustained collaboration will help to bridge the divide between research and practice.

The International Society for Pharmacoeconomics and Outcomes Research defines a mathematical model as a framework “representing some aspects of reality at a sufficient level of detail to inform a clinical or policy question” (1). Mathematical, computational and statistical models and techniques have been applied in the Canadian public health system, especially after the 2003 severe acute respiratory syndrome (SARS) epidemic, but it is unclear to what degree their outcomes have been used to shape policy and improve practice. Furthermore, the diversity of the Canadian population has not been adequately addressed in public health models, and research outcomes are often not translated for use within their relevant contexts. To improve the applicability and impact of models in public health, it is necessary to understand areas where modelling results are unclear, the value of a common language between modelling and public health, and how to sustain and enhance the application of modelling in public health.

Established during the early stages of the 2009 H1N1 pandemic in Canada, the Pandemic Influenza Outbreak Research Modelling (Pan-InfORM) team has a mandate to develop innovative modelling frameworks and knowledge translation methods that inform public health by linking theory, policy and practice. Aligned with its mandate, on October 6 and 7, 2014, Pan-InfORM held its fourth biannual workshop (2–4) cohosted by the National Collaborating Centre for Infectious Diseases. This workshop brought together public health practitioners and leading research modellers (a list of attendees is available at participants.html>) to enhance cross-discipline communications by providing a forum for knowledge to flow freely in a ‘jargon-free’ setting. The expected outcome was to identify the infrastructure, expertise and resources necessary to establish a ‘Communities of Practice’ (CoP) network. The CoP concept, initially developed by Jean Lave and Etienne Wenger, refers to groups of people who share a concern, a set of problems or a passion about something they do, and learn how to do it better by interacting regularly (5–8). The proposed CoP, as a new initiative to be catalyzed from this workshop, would offer new approaches to addressing problems at different levels of health care and population health and enable the development of strategic plans to move evidence to action. An important role for this CoP is to develop a common language that can be used in understanding the outcomes of health research and disease modelling.

The workshop objectives were to: understand areas where modelling terms, methods and results are unclear; share information on how modelling can best be used in informing policy and improving practice, particularly regarding the ways to integrate a focus on health equity considerations; and sustain and advance collaborative work in the development and application of modelling in public health.

The two-day event unfolded in four sessions. The first two, “Modelling in public health: Opportunities and challenges” and “Mathematical modelling in public health practice”, helped to set the context regarding scientific methods and research applications, particularly for the evaluation of research uptake and provided public health perspectives on the utility of modelling in decision making. The third session, “Muddling through modelling: Communication, common language and health equity”, spurred discussion regarding the need for common language between researchers and knowledge users, to improve the use of modelling study results. This included foundational issues regarding access to data and the involvement of indigenous and other community representation. In the final session, “Developing our network and communities of practice”, participants reflected on earlier presentations and discussions to clarify what is needed to continue collaboration and knowledge exchange that can increase the value of research modelling in public health.

The presentations and discussions that ensued created compelling arguments that the most prominent and observable outcomes can be achieved when communication barriers between disciplines are eliminated. The present report discusses key presentations and discussions that took place, and summarizes the outcomes and action plans that emerged from the workshop.

Setting the context: opportunities and challenges for modelling in public health

Sessions 1 and 2 of the workshop began with a presentation regarding the development of infectious diseases modelling in Canada. Before the 2003 SARS epidemic, modelling activities were largely driven by research interests of individuals or small groups, with a significant emphasis on the theoretical aspects of exploring complex mathematical phenomena. For the most part, these activities were carried out in isolation, with minimal communication and engagement with public health professionals and policymakers (2). During and following the SARS epidemic, various groups of disease modellers were formed to engage with, and develop models for application to public and population health in more specific contexts. Despite the importance and relevance of these initiatives, knowledge translation remained a challenge that the Pan-InfORM was established in part to address (9).

The Canadian Institutes of Health Research supported the establishment of Pan-InfORM to address the limited knowledge exchange between modelling researchers and those who could potentially make use of models to inform health policy and improve practice. Since its inception, Pan-InfORM has undertaken several national initiatives for knowledge brokering, including the evaluation of Canada’s response to the spring and winter waves of the novel H1N1 pandemic, identification of strategies for protecting vulnerable populations from emerging infectious diseases, and development of approaches that can enrich existing links with Aboriginal health organizations and foster multijurisdictional collaborative efforts in Canada (2–4).

In Canada, public health decision making occurs within orders of government at all levels. Situations are often very complex for a number of reasons including the availability and adequacy of health resources; inconsistent or absent evidence regarding the effectiveness and cost effectiveness of intervention strategies; pressure from the public, media and government under which public health must operate; other competing public health services; and ethical considerations to balance the protection of community health against individuals’ rights and freedoms. Other pertinent challenges include the lack of data to estimate potential outcomes of a public health program, paralysis resulting from having too much information on occasion, differing opinions and short timelines. In this context, as one presenter put it, there are three questions decision makers face: what is the benefit of the public health program or intervention; who will benefit from the program and; is the program cost-effective? Often, the evidence to answer these questions is not available in a timely manner.

Ideally, one would address these questions by investigating the effects experimentally. However, controlled trials may not be feasible or ethical, and can also be time consuming, laborious, expensive or inconclusive. Models provide a useful tool to overcome these challenges and systematically evaluate possible effects by using existing data and knowledge, generating quantitative outcomes and mapping out interdependencies that may be key factors for determining policy needs. Given these capabilities, models can be used to identify key uncertainties in the parameters and generate qualitative predictions, such as the effect of behavioural changes on the trends and distribution of an infection in the population. Indeed, the overarching goal of modelling is to support evidence-based public health policy.

To enhance the utility of models, communication and collaboration between modellers and public health leaders must take place early in a decision making process. Models are more valuable when end users are engaged in formulating the questions because models are built so that they truly reflect a public health question. End users who understand a model are likely to be better able to assess the results. During the construction and validation of a model, the relevance and importance of input parameters must be understood, and the sources for their values and ranges, uncertainty about the parameters, and sensitivity of the model outcomes with respect to parameter variation and original model assumptions, must be determined. New knowledge generated by a model should address the target question and be translated and disseminated for uptake and action appropriate to the context. Furthermore, when data are limited, it is essential to quantify any uncertainty in parameterization, because different sets of parameters may fit equally well. Ideally, the process to improve the model structure and its outcomes is iterative.

The value of direct conversations between modellers and public health leaders, in particular with regard to the availability and access to data and other critical information that are essential for model inputs of real-time scenarios (10), was exemplified in the use of modelling and the implementation of model recommendations for antiviral use and vaccination in Canada’s response to the 2009 H1N1 pandemic (11,12). Table 1 summarizes key issues presented and discussed for modelling in public health during the workshop.

TABLE 1

Summary of issues arising from presentations and discussions on the use of modelling in public health

Areas of concern or interest	Consensus discussion	Recommendations
Applying models in public health	Modelling has an important place in public health policy and practice. The utilization of modelling has been far less that its potential in the Canadian context	Create a national infrastructure or network in Canada to develop useful and applicable models based on realistic assumptions and quality data
Closer working relationships	Collaboration, engagement and exchange between modellers, and policymakers are needed to facilitate iterative processes that optimize the value and understanding of models and their results	Identify partners at the provincial level within acute care, emergency services and public health divisions. Formalize exchange processes for regular communication and education
Applying health equity and other lenses	Limited attention has been paid to using health equity or sex and gender analyses. The availability of Aboriginal-specific information has been inconsistent at best	Modellers and users can be called on to create model frameworks and ask questions that will provide better information about where there are inequities and inequalities. Involve the people who understand equity issues
Data quality and access	Access to good-quality, population level data is essential to validate a model and its outcomes. Such data may not necessarily be available or accessible in a timely fashion during an emerging infectious disease	Evaluate data quality and the type of information provided by surveillance for its potential to be used for research modelling. Engage with provinces to determine the nature and availability of data required for modelling
Coherent and consistent descriptions of diseases as well as modelling terminology	Across 13 jurisdictions, public health in Canada does not have universal definitions or natural histories of diseases, such as tuberculosis, to input in models	Undertake the work needed to standardize descriptions, which can inform a standard lexicon of terminology and protocols for infectious disease modelling
Cost effectiveness	Public health personnel and governments do not have enough information about the economics of interventions for comprehensive decision making	Strengthen the existing infrastructure (eg, Canadian Immunization Registry Network, National Advisory Committee on Immunization) to include economic modelling that will inform policy decisions
Standardization of approaches	To develop useful models, three aspects of the modelling will need to be standardized: what (ie, frameworks that are context specific and take into account the population demographic and geographic characteristics); who (ie, involvement of policymakers, knowledge users and modellers with relevant expertise), and how (ie, develop an iterative process from the formulation of health policy questions to the dissemination of model outcomes)	A Communities of Practice network can be tasked with the standardization of this process to ensure that synergies exist when models are formulated to inform clinical or health policy decisions
Roles and responsibilities	Clarification on the roles of health agencies and jurisdictions are needed to engage partners from academic institutes, government health organizations and health industries	National Collaborating Centre for Infectious Diseases will lead the initiative to forge the linkages and develop appropriate channels and effective methods of communication between the involved partners
Capacity	Some jurisdictions lack modelling capacity. There is also a lack of information about which modellers are available to work with public health and their expertise	A centralized list or network could contribute to greater capacity for public health jurisdictions. Develop opportunities for public health personnel to learn more about models and their value

Muddling through modelling: finding a common language

The International Society for Pharmacoeconomics and Outcome Research guidelines highlight the importance of a common language for drafting a health decision question and addressing it through a modelling framework. The guidelines for transparency and validation state:

Every model should have non-technical documentation that is freely accessible to any interested reader. At a minimum, it should describe in nontechnical terms, the type of model and intended applications; funding sources; structure of the model; inputs, outputs, other components that determine the model’s function and their relationships; data sources, validation methods and results; and limitations (1).

Good communication flows to and from knowledge producers and users, and requires a common language to build effective partnerships and understanding of the groups’ respective concerns. There are a number of challenges to developing a common language: determining a common lexicon; understanding priorities and contributions, which may shift depending on the political climate or population health status; asking the right questions that are appropriate to the given context; knowing the right audience; and being able to communicate findings to others outside the research community.

The lack of such common language may have been an impediment to addressing key parameters in ‘determinants of health’ and ‘health equity’. In the Canadian context, one needs to take into account differential health status and population structure of First Nations, Inuit and Métis people, population-level patterns of abuse, poverty and historical trauma, challenges regarding access to health services in rural and remote areas, and limits in identifying “vulnerable” populations in available datasets with no real markers. Building partnerships and an iterative exchange allows for goals and facts to be clearly identified, and outcomes to be assessed for their value to inform decisions about the potential benefits and risks of policy development and program delivery. Effective partnerships require willingness and commitment, alignment of values, mechanisms to engage early and continuously, and plans to regularly review goals, objectives, roles, and responsibilities and outcomes.

Developing commonalities in modelling research and application

A recent review of literature highlights the inconsistency in definitions and interpretations of epidemiological terms in several modelling studies and the need for common language to sustain and enhance the application of models in public health (13). The review found that disparate outcomes and interpretations for policy decisions may arise from inconsistent use of terms in model structures, even when the assumptions and input parameters are identical. Discrepancies in how terms are used for modelling are generally associated with two main reasons. First, it is often assumed that the particular terms are well defined or well understood. For example, ‘infectiousness’ and ‘infectious’ were found to be used interchangeably; the former describes a characteristic of the disease and/or how readily the disease is transmitted, while the latter describes a patient state (13). Second, definitions of some terms have drifted over time as understanding of the mechanisms of disease processes and control has evolved. For example, the way terms such as ‘prevention’, ‘protection’ and ‘reduced susceptibility’ are used related to communicable disease may lead to different results depending how they are used in modelling. Developing a common language will help to reduce possible variation in study results produced by different research communities. This will in turn decrease misinterpretation of the outcomes by allowing for comparisons of scientific evidence from multiple disciplines involving health research, and helping knowledge users and policymakers to better understand research outcomes and their applicability to policy and practice.

There are other factors responsible for variation in model findings, including different strategies or approaches and assumptions, different population demographic variables, and the objectives for evaluating policy effectiveness that can vary from one situation to another. The latter can be exemplified in two recent studies on the effectiveness of school closure during pandemic influenza outbreaks. When assessing the effect of school closure strategies in reducing community attack rates, Halder et al (14) found that due to the difficulty in determining the true degree of epidemic spread and its severity in the early stages of an outbreak, a strategy of individual school closures would be more effective than simultaneous closures across a region. The outcomes are drawn from an agent-based simulation model of Albany, a small community in Western Australia with a population of approximately 30,000 individuals. In contrast, to evaluate the impact of local reactive school closures on critical care provision in the United Kingdom population setting, House et al (15) concluded that school closures should be coordinated in time (simultaneous) and location (all schools within a school district) to become an effective strategy to reduce infection transmission and, consequently, relieve capacity pressures of hospital intensive care unit admissions. The population demographics and the objectives for closing schools are distinctly different between the two studies, suggesting that different modelling approaches are required for measuring the effectiveness of school closures.

Understanding scenario-specific outcomes and their applications requires a critical evaluation to address the following questions:

Is the methodology appropriate for the specific population setting?

Do the assumptions and parameters address the reality of demographic and geographic characteristics?

Can the outcomes be compared with other studies and validated with observed data?

How generalizable are the outcomes to address different scenarios or population settings?

A consensus emerged during the workshop regarding the need to develop a common language for modelling to enhance its application in a public health context and promote bidirectional communication (Table 2). To address this need, the fourth session of the workshop provided an opportunity for participants to discuss the establishment and potential impact of a Community of Practice.

TABLE 2

Summary of challenges and actions identified to improve consistency in terminology in public health modelling

Opportunities and strengths a Community of Practice can provide

Develop a unified infrastructure to inform complex decision making and improve health practice based on quality data, evidence and scientific knowledge

Avoid duplication, use resources wisely and harness power of complementary disciplines to work collaboratively to common goals

Develop and foster the use of a common language for modelling in public health, and help identify similarities and differences in modelling approaches and their outcomes

Identify research priorities and data gaps (it is important to note that the lack of evidence and/or data may provide opportunities to exploit factors that affect model outcomes and their policy consequences. These opportunities can enhance our learning and, more importantly, may suggest novel data collection, better models and improved collaborations through a Community of Practice)

Facilitate data sharing and critical information in a timely fashion

Help address diversity of Canadian contexts

Identify relevant partners and stakeholders and engage them in formulating research questions

Integrate resources for knowledge translation and bidirectional communication

Increase uptake of knowledge and promote best practices for modelling collaborations

Community of Practice members

Exchange key documents, resources, and expertise

Evaluate research outcomes, and synthesize theoretical and practical knowledge gained from national and international collaboration

Organize targeted efforts for integrating modelling, surveillance, planning and decision making

Partnerships

Community of Practice will support the development of partnerships with federal/provincial/territorial health agencies and departments in Canada and other relevant organizations by enabling exchanges of information on the type of evidence used in decision making; assess the policy relevance of research outcomes; and support dissemination and implementation of the health policy recommendations

Research and knowledge translation priorities for modelling

Intervention strategies for outbreaks of influenza and other respiratory pathogens (eg, tuberculosis, pertussis)

Potential benefits of immunization of target groups (eg, school-aged children, or health care workers)

School closures strategies during emerging diseases with consideration of population characteristics

Cost-effectiveness of immunization programs

Strategies for early identification and treatment of active tuberculosis

Priorities for modelling consistency

Develop a glossary of definitions of terms used in modelling consistent with dictionary of terms for infectious disease epidemiology

Develop conceptual frameworks for natural history model of important diseases using standard terminologies

Provide guidelines and develop criteria for assessing the quality and relevance of databases for modelling

Develop guidelines for review and assessment of modelling research quality and the relevance to public health policy and practice in Canadian contexts

Host organization

National Collaborating Centre for Infectious Diseases

FINAL DISCUSSION

During the final discussion session, a number of important issues related to the development of a CoP network were discussed, including its structure and governance, leadership and research capacity, memberships and partnerships, strategic plans for sustainability and resources, and the impact and uptake of outcomes (Table 3).

TABLE 3

Summary of final discussion session with action plans for further interaction and integration through a Community of Practice

Challenges

Inadequate methods and/or analysis for applying results to public health policy

Lack of evidence and/or data

Variability and uncertainty of model outcomes (results)

Validation and applicability of the results

Inadequate collaborative research or expertise

Insufficient knowledge translation and communication

Consensus

Need for consistent use of infectious disease terms in modelling studies

Need for greater use of modelling approaches and results beyond scientific discoveries

Action plans

Develop a process to standardize terms to be used in modelling of infectious diseases

Develop plans to generate and translate new knowledge with engagement of relevant stakeholders

Formalize the structures and linkages initiated by the Pandemic Influenza Outbreak Research Modelling team to tackle a wider range of public health issues facing Canadians

CONCLUDING REMARKS

The October 2014 national workshop propelled new discussion on the value of mathematical models in public health planning and the need for greater cohesion and collaboration among stakeholders. The workshop concluded with a consensus among participants that there is work to be done and a willingness to continue to work together. The creation of a common lexicon is a tangible, initial task that should be undertaken as an immediate response to the workshop discussions.

We expect that through sustained cross-disciplinary dialogues, a CoP will initially produce a ‘book of terminology’ that describes current usage and proposes common terminology (community standards) in different areas, including medical and infectious diseases epidemiology, public health and disease modelling. This reference book can then be updated regularly when new terms need clarification for shared understanding and agreement in use. Furthermore, in times of uncertainty, the virtual CoP network will provide opportunities to access, analyze, synthesize and utilize reliable information and databases in a timely fashion, and drive a broad consensus around plausible alternatives and integrated courses of action. It is also true, however, that ongoing discussions between modellers and public health personnel will help to clarify language use and break down perceived barriers.

Postscript

Within a number of weeks after the workshop, a new virtual Community of Practice, created by National Collaborating Centre for Infectious Diseases through LinkedIn, called mod4PH (modelling for public health) (16) was launched generating questions and comments to further the workshop discussions. The online network is open to new public health and research members.