Building the field of population health intervention research: The development and use of an initial set of competencies.

Population health intervention research (PHIR) is a relatively new research field that studies interventions that can improve health and health equity at a population level. Competencies are one way to give legitimacy and definition to a field. An initial set of PHIR competencies was developed with leadership from a multi-sector group in Canada. This paper describes the development process for these competencies and their possible uses. Methods to develop the competencies included key informant interviews; a targeted review of scientific and gray literature; a 2-round, online adapted Delphi study with a 24-member panel; and a focus group with 9 international PHIR experts. The resulting competencies consist of 25 items grouped into 6 categories. They include principles of good science applicable though not exclusive to PHIR, and more suitable for PHIR teams rather than individuals. This initial set of competencies, released in 2013, may be used to develop graduate student curriculum, recruit trainees and faculty to academic institutions, plan non-degree professional development, and develop job descriptions for PHIR-related research and professional positions. The competencies provide some initial guideposts for the field and will need to be adapted as the PHIR field matures and to meet unique needs of different jurisdictions.

Introduction

Competencies help give shape, legitimacy, and definition to a field, in combination with many other elements, such as research and ethical frameworks, publishing venues, funding mechanisms, and standards (Isaacs and Knickman, 2005, Ottoson et al., 2009, Carnevale et al., 2013 ). Competencies typically consist of knowledge, skills, and attitudes that affect a major part of one’s job (a role or responsibility), correlate with performance on the job, can be measured against some accepted standard, and can be improved through training and development (Gebbie et al., 2008).

Population health intervention research (PHIR) is a relatively new research field with a focus on studying interventions, including policies, programs, and events that have the potential to improve health and health equity at the population level (Hawe et al., 2012). PHIR shares characteristics with other fields, including the well-established field of evaluation (Hawe and Potvin, 2009), and more emerging fields such as implementation research and delivery science (Statement on Advancing Implementation Research and Delivery Science, 2014). It is also sufficiently distinct as a field, as affirmed through several international forums (Canadian Institutes of Health Research (CIHR), 2010a; Canadian Institutes of Health Research, 2010b, Canadian Institutes of Health Research, 2012a, Canadian Institutes of Health Research, 2012b). For example, in contrast to clinical epidemiology, PHIR focuses on the study of interventions that attempt to modify the distribution of risk in a population instead of individual outcomes. In contrast to evaluation that tends to study interventions one at a time, PHIR is concerned with developing a cumulative body of knowledge about classes of interventions and their generative mechanisms. In contrast to implementation science, PHIR concentrates exclusively on population health interventions, recognizing that a set of unique features of these interventions, such as the involvement of actors from diverse sectors, the multiplicity of interacting components, the unique characteristics of public health as a key delivery system, the need to take into account the influence of context on both intervention implementation and its effective mechanisms, and the specific ethical issues raised with population health interventions pose specific problems for research. Also, questions of implementation are a subset of PHIR.

Within Canada, leadership for advancing PHIR and its use is provided by the Population Health Intervention Research Initiative for Canada (PHIRIC), a strategic alliance established in 2006 (CIHR, 2012a) whose strategic objectives included building capacity for and championing PHIR and its use. One activity to achieve these objectives, and the focus of this article, was the development of an initial set of core competencies for PHIR. This paper describes the development process for the competencies and their possible uses. Its aim is to increase dialog within and beyond Canada on the definition of PHIR as a field in general, and the content and use of the PHIR competencies in particular.

Methods

The competency development process was guided by a working group, consisting of researchers leading CIHR-funded strategic training programs in health research (STIHRs) with a focus on PHIR and with linkages to schools of public health, national organizations with mandates to support the generation, and use of PHIR (i.e. CIHR, Canadian Population Health Initiative of the Canadian Institute for Health Information, and the Public Health Agency of Canada [PHAC]), and evaluators. The competencies were developed using an iterative multi-stage process.

Stage 1: key informant interviews with principal investigators and program coordinators of STIHRs with a PHIR focus. This group of informants was purposefully selected since STIHRs were designed to provide learning opportunities that complement existing graduate curriculum; to pioneer areas of health research needing special attention and development. PHIR was the focus of seven funded programs, whose leads participated in an interview or shared relevant written materials as part of stage 1. Interviews and documents provided information on program objectives, research competencies that trainees participating in the programs were expected to gain, and other documents that could usefully inform the development of PHIR competencies.

Stage 2: review of competencies developed by professional and research organizations in PHIR-related fields. Particular attention was given to competencies for program evaluation (Competencies for Canadian Evaluation Practice, Canadian Evaluation Society) and public health (Core Competencies for Public Health in Canada, PHAC) as two fields most pertinent to PHIR.

Six primary documents were selected for identifying competency-related items (Australian Health Promotion Association, 2009, Canadian Evaluation Society, 2010, Gebbie et al., 2008, Israel et al., 1998, Public Health Agency of Canada, 2007, Social Research Association, 2009), from which a preliminary set of 30 competency items were extracted. By addressing overlap and duplication, this list was reduced to 28 competencies that related to the management and conduct of PHIR, the integration of knowledge translation and exchange within the population health intervention research process, and on being a reflective practitioner.

Stage 3: an adapted Delphi study designed to seek feedback and develop consensus on the competencies. A 24-member panel was recruited through a snowball technique, starting with nominations from the PHIR competencies working group. Panel members included research funder, researcher, and research user perspectives, primarily from Canada. A majority were from university settings (almost 60%), with others from public health and government agencies. All members were involved in PHIR for more than 3 years, and half had more than 10 years of experience with PHIR. Women and men were evenly represented.

Two rounds of online adapted Delphi surveys were conducted. The surveys asked panel members to rate the usefulness of the set of competencies as a whole, as well as rate each competency individually on dimensions of clarity, applicability and importance. Open-ended survey questions allowed panel members to justify their responses and provide suggestions to improve the competencies. After Round 1, a summary of responses was sent to the panel. For Round 2, minor revisions to the wording of some competencies were made to improve clarity. The meaning of the competencies remained the same. In Round 2, panel members were encouraged to reconsider and revise their answers in light of Round 1 feedback.

Stage 4: a focus group with a convenience sample of 9 international experts who attended the 2012 International Conference to Advance Population Health Intervention Research (Cathexis Consulting Inc, 2012). Participants were from the United Kingdom (n = 4), United States (n = 3), France (n = 1), and Australia (n = 1). The focus group sought feedback on the results of the second round of the Delphi, and possible uses of the competencies. Results were consistent with feedback from the Delphi and were used to refine the competencies, especially a preamble regarding intended uses and principles to accompany the competencies. Focus group participants also acknowledged that different countries may need to tailor use of the competencies to fit graduate student training and other professional development, including on-the-job training.

The initial set of PHIR competencies, therefore, is based on a systematic process that involved multiple perspectives. While there is no general agreement on the optimal size of a Delphi panel, 24 participants is at the high end of the range of 15–30 participants reported in a majority of Delphi studies (Hsu and Sandford, 2007). Also, retention was high (75%) from Round 1 to Round 2, and attrition did not compromise the representativeness of the panel. No known biases were present, although the perspectives most well-represented were researchers from central Canada. As a consensus-building instrument, the Delphi process provided anonymity for respondents and no explicit pressure to conform to group ratings. Even so, subtle pressure to conform and the potential of molding opinions are known limitations to any Delphi process and may have influenced results (Hsu and Sandford, 2007).

Results and discussion

The competencies resulting from the 4-stage process (see Table 1) consist of 25 items grouped into six categories. They include principles of good science applicable to PHIR, though not exclusive to PHIR. The uniqueness of the competencies and their specificity to PHIR is mostly based on areas of emphasis and their interactions. For example, the PHIR competencies emphasize collaboration and a highly context-sensitive science. They also emphasize interactions among interdisciplinary science, participatory approaches, a focus on health and health equity at a population level, integrated knowledge translation, and the central role of contextual influences on interventions. The competencies are not mutually exclusive; one competency may build on or overlap with another. Any given intervention study may require only a subset of the competencies. Also, the competencies are for the field of PHIR, rather than for individuals, since PHIR typically includes a team of researchers and knowledge users (e.g. public health practitioners). This orientation for the competencies is consistent with field building for applied research. For example, the competencies are intended to strengthen both academic and professional disciplines (Isaacs and Knickman, 2005), and they aim to link research and action, which is essential in applied fields with a focus on policy and practice interventions (Gutman et al., 2009).

Table 1

Population Health Intervention Research Competencies (released 2013 http://www.cihr-irsc.gc.ca/e/47223.html, and also available in French http://www.cihr-irsc.gc.ca/f/47223.html).

1 Demonstrate foundational knowledge relevant to PHIR:

1.1 Demonstrate relevant public and population health knowledge to inform PHIR (including knowledge of health equity, social determinants of health, and planning and/or implementation of population health interventions).

1.2 Demonstrate and apply knowledge of research and evaluation theories, methods, and approaches.

1.3 Demonstrate understanding of effective partnership models.

1.4 Demonstrate understanding of the unique characteristics of PHIR.

2 Manage PHIR projects:

2.1 Prepare PHIR funding proposals in partnership with community partners, policy leaders, and interdisciplinary team members.

2.2 Manage and coordinate project teams, tasks, budgets, and timelines.

2.3 Identify and mitigate potential risks to the research.

2.4 Develop and maintain partnerships.

3 Plan PHIR:

3.1 Describe the particular intervention or focus of the research.

3.2 Identify relevant research questions in partnership with community partners, policy leaders, and interdisciplinary team members.

3.3 Assess how situational and contextual factors and previously conducted research may influence the focus of the research, as well as the research itself.

3.4 Assess anticipated and unanticipated outcomes of the intervention.

3.5 Identify and select appropriate sources of data.

3.6 Select research methods that are

a. Appropriate to the research questions;

b. Appropriate to the population and/or focus of the research; and

c. The most rigorous possible in light of feasibility.

4 Integrate knowledge translation and exchange within the PHIR process:

4.1 Facilitate knowledge exchange throughout the research process by collaborating with policy leaders, community partners, decision-makers, and other researchers.

4.2 Synthesize individual research study findings and relate them back to the larger body of knowledge on the topic.

4.3 Tailor communication to the audience and local context.

4.4 Disseminate PHIR results to the relevant community, policy and academic audiences.

5 Carry out PHIR:

5.1 Collect data that is relevant to the question being asked and takes the context into account.

5.2 Analyze and interpret data.

5.3 Formulate and seek feedback on appropriate conclusions based on questions, methods, and data.

5.4 Justify decisions made in the course of designing and carrying out the research.

6 Be a reflective researcher:

6.1 Adhere to a relevant set of professional standards of practice.

6.2 Apply a relevant set of ethical principles to the research process.

6.3 Develop professional networks within and outside one’s field.

6.4 Seek out learning opportunities within and outside one’s field.

6.5 Reflect on one’s own strengths and limitations as a researcher.

6.6 Reflect on one’s role in improving population health and health equity.

Timely use of the PHIR competencies will help to capitalize on early successes in defining and building PHIR as a field. The competencies may be used in several ways, tailored to academic and professional training programs and practices or needs of different jurisdictions. Academic institutions may use the competencies to develop curriculum for graduate student training in PHIR and to recruit trainees, post-doctoral fellows, and faculty. The PHIR competencies may be especially relevant for schools of public health and other applied health science faculties and departments. The competencies may also be used to plan non-degree professional development or continuing education opportunities, which could be targeted and tailored to those working in research, policy, and practice sectors. Another use of the competencies may be developing job descriptions for PHIR-related staff positions in university, research institute, policy, and practice settings. These are initial ideas only and the authors encourage the growing community of those who fund, conduct, and use PHIR to consider how the competencies may inform their work.

The competencies described in this brief research article provide some initial guideposts for the field. As PHIR matures as a field within a dynamic environment for population health research, policy, and practice, the competencies will need to be questioned, regularly reviewed, and adapted, as appropriate. An early example is a constructive critique published on the competencies for a reflective researcher: Tremblay and Parent (2014) argue that a more nuanced definition of what constitutes a reflexive researcher is needed to continually advance the field of PHIR (e.g. someone who periodically questions their role in the research process). This and other feedback on the content and application of the PHIR competencies, shared through informal and formal venues, will inform a next iteration of PHIR competencies and help build a robust field of research that has a meaningful and enduring impact on population health and health equity.

Inquiry into the development, use, and usefulness of competencies more generally is also suggested. While competencies are generally accepted as giving shape, definition, and legitimacy to fields, their acceptance, adoption, adaptation, and usefulness are not well understood. For example, how does their use and usefulness vary by policy or practice setting, stakeholder groups, and/or by their development process? Exploring these questions over time will help to adapt current sets of competencies and provide insights for the role of competencies in field building more generally.

Conflict of interest

The authors declare that there are no conflicts of interest.