Developing a framework of core competencies in implementation research for low/middle-income countries.

The field of implementation research (IR) is growing. However, there are no recognised IR core competencies in low/middle-income countries (LMICs), nor consistent curriculum across IR training programs globally. The goal of this effort is to develop a framework of IR core competencies for training programs in LMICs. The framework was developed using a mixed-methods approach consisting of two online surveys with IR training coordinators (n = 16) and academics (n = 89) affiliated with seven LMIC institutions, and a modified-Delphi process to evaluate the domains, competencies and proficiency levels included in the framework. The final framework comprised of 11 domains, 59 competencies and 52 sub-competencies, and emphasised competencies for modifying contexts, strengthening health systems, addressing ethical concerns, engaging stakeholders and communication especially for LMIC settings, in addition to competencies on IR theories, methods and designs. The framework highlights the interconnectedness of domains and competencies for IR and practice, and training in IR following the outlined competencies is not a linear process but circular and iterative, and starting points for training may vary widely by the project, institution and challenge being addressed. The framework established the need for a theory-based approach to identifying proficiency levels for IR competencies (ie, to determine proficiency levels for IR based on generalisable educational theories for competency-based education), and the relevance of various IR competencies for LMICs compared with high-income settings. This framework is useful for identifying and evaluating competencies and trainings, and providing direction and support for professional development in IR.

The field of implementation research (IR) holds great promise for facilitating effective delivery of evidence-informed interventions and maximising the benefits of those interventions for improving population health. Several IR courses and trainings are available. However, there is no recognised set of core competencies for training in IR in low/middle-income countries (LMICs), nor is there a consistent curriculum across different IR training programme globally.

A framework of core competencies in IR for LMIC contexts differs from earlier frameworks developed for high-income countries in its emphasis on contexts, health systems, ethics, communication, advocacy skills and equity issues.

There is a statistically significant association between respondent’s self-reported proficiency level in IR and their perception of the proficiency level needed to achieve a competency in an LMIC context. Respondents tend to frequently assign proficiency needed to achieve a competency to the same level as their own self-reported proficiency.

This framework will help in identifying competencies needed to successfully respond to implementation challenges surrounding effective delivery of life-saving interventions and health services in LMICs. The framework can guide the objective assessment of training needs and the effectiveness of IR training programme, as well as guiding the development of future programme in IR in LMICs.

Introduction

The field of implementation research (IR) holds a great promise for facilitating effective delivery of evidence-informed interventions and enabling their scale up. IR also maximises the benefits of these interventions for improving population health, especially health of vulnerable populations. The field is growing globally with an array of courses and training programme that cater to researchers and practitioners seeking to learn more about IR methodologies and approaches.1 2 However, there is no recognised set of core competencies in IR in low/middle-income countries (LMICs), nor is there a consistent curriculum across different IR training programme globally. While there is a consensus on the key principles and characteristics of IR as applied to LMIC contexts,3 without a recognised set of core competencies, it is difficult to identify indicators and milestones for measuring the effectiveness of IR training programme, evaluate competency gaps, customise trainings to fit the specific needs of individuals and institutions in LMICs and provide direction and support for professional development in IR. It is also difficult to systematically develop future training programme in IR for LMICs and grow the field efficiently. The lack of a consistent approach to training in IR in LMICs limits the impact of researchers and practitioners in facilitating the effective delivery of life-saving interventions and strengthening of health systems in LMICs.

A review of the literature yielded six relevant articles describing competencies in IR4–9; all of these focused on IR in high-income countries (HICs) except one, the UNICEF, United Nations Development Programme (UNDP), World Bank and WHO Special Programme for Research and Training in Tropical Diseases (TDR) IR toolkit,9 which identified broad learning domains for IR researchers in LMICs, but did not explicitly define competencies under these domains. Four articles identified competencies for IR education and training programme in HICs,4–7 one of which defined four competencies that were focused exclusively on the conduct of IR,4 while the other three included competencies covering skills required in all phases of IR, including the pre-IR phase (eg, planning of IR) and post-IR phase (eg, disseminating IR results).5–7 Padek et al 6 described a comprehensive list of 43 competencies with inputs from over 100 IR experts from HICs; these competencies overlapped the seven competencies identified by Gonzales et al,5 and the nine competency clusters identified by Tabak et al 7 with inputs from 120 IR researchers and practitioners. However, due to the unique contexts in LMICs, for example, weak and unstable health systems, distinctive institutional arrangements and sociopolitical frameworks, these 43 relevant competencies cannot be readily employed for advancing IR globally without some adaptation to IR as applied in LMICs.

IR is typically embedded within real-life programmatic operations and project cycles,3 10 11 and addressing health systems and contextual issues are major considerations in IR in LMICs.3 11 12 The need to understand health system bottlenecks and contextual barriers in IR further makes it necessary to adapt IR competencies developed in HIC settings to LMICs. For example, only one of the 43 competencies developed by IR researchers in HICs addressed competencies related to understanding health systems in conceptualising IR.6

There have been successful attempts to develop core competencies for other forms of research in LMICs, including clinical research.12 However, such competencies are not readily transportable for training programme in IR. Unlike clinical research, which may emerge outside of practice, most IR arises from practice in attempts to implement or maximise the impact of evidence-informed interventions.10 13 Leadership in clinical research also tends to be investigator-led while IR emphasises team leadership during every phase of the research and includes individuals from policy and practice.3 14–16 Stakeholders represented in IR teams in LMICs are also uniquely derived based on the context including community gatekeepers, policymakers, service providers and other real end-users. IR teams may require equitable collaboration among these stakeholders in decision-making and shared resources to maximise impact at scale.

The goal of this study is to develop a framework of core competencies in IR relevant for education and training programme in LMICs. This framework will help teams identify the competencies needed to successfully respond to implementation challenges surrounding the effective delivery of life-saving interventions and health services, in real time and under real-world conditions in LMICs. The framework will also provide a guide for measuring the effectiveness of training programme in IR, providing direction and support for professional development and guiding the development of future training programme in IR in LMIC contexts.

Developing a framework of core competencies in IR for LMICs

The framework was developed over three phases using a mixed-methods approach consisting of a survey of IR researchers, trainers and trainees in selected academic institutions from mainly LMICs, and a consultative Delphi process with selected IR experts from both LMICs and HICs.

Phase 1: identifying learning domains and competencies for IR in LMICs

An initial list of broad learning domains for IR in LMICs was identified and defined (box 1), drawing from domains outlined in the TDR IR Toolkit9 and other reviewed literature.3 4

Apply a structured process to identify known bottlenecks and barriers in the health system.

Identify emerging challenges related to implementation of specific evidence-informed interventions.

Contextualise health systems bottlenecks and barriers relevant for addressing implementation of evidence-informed interventions.

Identify and engage relevant stakeholders.

Constitute an implementation research (IR) team to lead the research enterprise.

Formulate appropriate IR questions.

Determine the applicable study design.

Articulate a proposal to help leverage required resources.

Conduct and monitor IR in a robust, rigorous and ethical manner.

Feed the solutions/adaptations back into the health system.

Communicate and advocate effectively throughout the IR process.

The learning domains for evidence-informed interventions in box 1 are exemplified by interventions for addressing the burden of infectious diseases of poverty (IDoP) in LMICs such as onchocerciasis, lymphatic filariasis, leprosy and soil-transmitted helminthiasis. However, these interventions could be substituted with interventions addressing any other diseases or health challenges to adapt the learning domains for IR around other health problems.

Seven LMIC institutions that have been involved with the UNICEF, UNDP, World Bank and WHO TDR were selected to identify IR researchers and trainers based in LMICs. These institutions were selected because they are involved in IR training progams, conducting IR grant calls, implementing IR grants and offering graduate programme in IR in LMICs. These institutions also represent different geographical regions and sociopolitical contexts in LMICs (box 2).

BRAC James P Grant School of Public Health, BRAC University, Bangladesh.

National School of Public Health, Universidad de Antioquia, Colombia.

School of Public Health, University of Ghana, Ghana.

Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia.

Faculty of Health Sciences, American University of Beirut, Lebanon.

School of Public Health, University of the Witwatersrand, South Africa.

Department of Public Health, University of Zambia, Zambia.

The 43 IR competencies described for HIC settings by Padek et al 20146 were drafted in Qualtrics, an online survey platform,17 and shared with 16 IR training programme coordinators based at the seven training institutions in LMICs to assess the relevance of these competencies for IR programme offered at their institutions and addressing IR in LMICs contexts more broadly. During this first round of survey, respondents were further asked to review the list of competencies in Qualtrics between 3 and 15 August 2018, focusing on adding, deleting or clarifying concepts for LMIC contexts. The core study team grouped the 43 IR competencies under the 11 broad domains (box 1) as an initial version of the framework (V.1.0 in online supplementary table 1). In total, 12 (75%) of the 16 IR training programme coordinators assessed the relevance of the domains and competencies outlined in this version of the framework for LMIC contexts. The core study team collected and synthesised all responses gathered from the IR training programme coordinators. The feedback resulted in the deletion of 3 competencies, addition of 10 new competencies and the merging of 2 domains: ‘Apply a structured process to identify known bottlenecks and barriers in the health system’ was merged with ‘Contextualise health systems bottlenecks and barriers relevant for addressing implementation of evidence-supported IDoPs’. In addition, sub-competencies were added to provide more actionable steps for competencies, where applicable. The suggested changes made during this phase are summarised (online supplementary table 2), and were implemented to produce the V.2.0 of the framework (online supplementary table 3). The V.2.0 of the framework comprised of 10 domains and 62 actionable competencies/sub-competencies.

The IR training programme coordinators were requested to nominate 4–5 randomly selected IR professionals, including grantees, trainees and experts within and outside of their institutions to be included in another round of survey in phase 2 of the framework development. In all, 89 individuals were identified for the second round of survey, including the 16 IR training coordinators from the 7 LMIC institutions.

Phase 2: assessing proficiency levels for IR competencies and subcompetencies

Five proficiency levels were identified using the National Institutes of Health competencies proficiency scale18: basic awareness (1), beginner (2), intermediate (3), advanced (4) and expert (5). A second round of survey was conducted in Qualtrics with the 89 individuals identified in phase 1, between 25 October 2018 and 22 February 2019, to further revise the V.2.0 of the framework. Respondents were asked to assign a proficiency level to each competency/sub-competency that best expressed the level needed to address that particular competency/sub-competency. The proficiency scale was applied as an ordinal scale, with ‘Basic awareness’ equalling 1, ‘Beginner’ equalling two, and so on. The mean value of assigned proficiency levels was estimated for each competency/sub-competency statement, and then standardised to a mean of 0 and SD of 1. Competencies/sub-competencies were sorted by their standardised mean score in ascending order and then divided into quintiles based on the distribution of scores. Quintiles 1–5 corresponded to an ordinal proficiency scale of 1–5 (ie, basic awareness, beginner, intermediate, advanced and expert), respectively. In addition, respondents were also asked to self-identify their overall proficiency level in IR, and provide any additional suggestions on the framework.

A χ2 test was used to test the differences in the frequency of assignment of all competencies and sub-competencies to different proficiency levels (basic awareness, beginner, intermediate, advanced and expert) within sub-groups of respondents defined by their self-reported overall proficiency level and geographical regions. For each competency/sub-competency statement, an one-way Analysis of Variance (ANOVA) test was conducted to determine if there were statistically significant differences in the mean proficiency scores for sub-groups of respondents based on their self-reported overall proficiency level (ie, basic awareness, beginner, intermediate, advanced and expert), and pairwise comparison of mean proficiency scores between sub-groups was done with Bonferroni correction for competency statement showing significant differences. Similarly, the differences in the mean proficiency scores for each competency statement were compared for respondents grouped by geographical regions (Asia-Pacific, sub-Saharan Africa and Latin America). All quantitative analyses were conducted in STATA V.14 I/C statistical software.

In total, 50 (56%) out of the 89 individuals identified for the second round of survey responded, including individuals from 14 countries and 18 institutions (table 1). Of the 50 individuals that responded, 3 (6.7%), 7 (15.6%), 16 (35.6%), 17 (37.8%) and 2 (4.4%) self-identified their IR proficiency as basic awareness, beginner, intermediate, advanced and expert, respectively.

Table 1

Characteristics of survey respondents

Characteristic	n=50 (%)
Countries
Bangladesh	2 (4.0)
Colombia	9 (18.0)
Costa Rica	1 (2.0)
Ghana	4 (8.0)
Honduras	7 (14.0)
Indonesia	9 (18.0)
Lebanon	4 (6.0)
Philippines	1 (2.0)
South Africa	6 (12.0)
Sweden	1 (2.0)
Tanzania	1 (2.0)
Timor Leste	1 (2.0)
USA	2 (6.0)
Zambia	2 (4.0)
LMIC geographic region*
Asia-Pacific	17 (34.0)
Sub-Saharan Africa	13 (26.0)
Latin America	17 (34.0)
Institution
United Nations Population Fund (Bangladesh)	1 (2.0)
BRAC University	1 (2.0)
National Institute of Health (Colombia)	1 (2.0)
University of Antioquia (Colombia)	6 (12.0)
CIDEIM (Colombia)	2 (4.0)
Hospital San Juan de Dios (Costa Rica)	1 (2.0)
University of Ghana (Ghana)	4 (8.0)
The National Autonomous University (Honduras)	6 (12.0)
Hospital San Felipe (Honduras)	1 (1.0)
Universitas Gadjah Mada (Indonesia)	9 (18.0)
American University of Beirut (Lebanon)	4 (8.0)
Research Institute for Tropic Medicine (Philippines)	1 (2.0)
University of Witswatersrand (South Africa)	6 (12.0)
Uppsala University (Sweden)	1 (2.0)
Sokoine University of Agriculture (Tanzania)	1 (2.0)
Ministry of Health (Timor Leste)	1 (2.0)
Johns Hopkins University (USA)	2 (2.0)
University of Zambia (Zambia)	2 (2.0)
Self-reported proficiency in IR†
Basic awareness	3 (6.0)
Beginner	7 (14.0)
Intermediate	16 (32.0)
Advanced	17 (34.0)
Expert	2 (4.0)

*This excludes three individuals that were based in high-income countries, including individuals based in Sweden and USA.

†Five respondents (6%) did not self-identify their proficiency level.

CIDEIM, Centro Internacional de Entrenamiento e Investigaciones Medicas; IR, implementation research; LMIC, Low/middle-income country.

Of the 62 competencies and sub-competencies statements assessed during the second round of survey, 13 (21.0%) were assigned to the ‘basic awareness’ level, 15 (24.2%) were assigned to the ‘beginner’ level, 11 (17.7%) were assigned to the ‘intermediate’ level, 11 (17.7%) were assigned to the ‘advanced’ level and 12 (19.4%) were assigned to the ‘expert’ level based on the quintiles of standardised mean score (online supplementary table 3). There were statistically significant associations observed between respondents’ self-reported proficiency level and their perception of the proficiency level needed to achieve a competency as described by the frequency of assignment of competencies to different proficiency levels (table 2). Respondents tend to frequently assign proficiency needed to achieve a competency to the same level as their own self-reported proficiency. However, no such associations were observed between respondents’ geographic region and their perception of the proficiency level needed to achieve a competency.

Table 2

Association between self-reported proficiency level and the level assigned to competencies

Assigned proficiency level to competencies*
Self-reported proficiency level†	Basic awareness	Beginner	Intermediate	Advanced	Expert	χ2 (p value)
Basic awareness(n=186) (%)	8 (4.3)	39 (21.0)	56 (30.1)	38 (20.4)	45 (24.2)	34.16 (p<0.001)
Beginner(n=434) (%)	45 (10.4)	158 (36.4)	108 (24.9)	64 (14.7)	59 (13.6)	98.60 (p<0.001)
Intermediate‡(n=992) (%)	6 (0.6)	80 (8.1)	398 (40.1)	259 (26.1)	243 (24.5)	489.51 (p<0.001)
Advanced(n=1054) (%)	39 (3.7)	63 (6.0)	273 (25.9)	480 (45.5)	197 (18.7)	607.83 (p<0.001)
Expert(n=124) (%)	0 (0.0)	1 (0.8)	21 (16.9)	13 (10.5)	89 (71.8)	220.03 (p<0.001)

*Assigned proficiency level was determined by identifying the frequency of assignment of all competencies and sub-comptencies to different levels (ie, basic awareness, beginner, intermediate, advanced, expert) by survey respondents who self-reported their proficiency to a specific level. For example, the frequency total for all competencies for those who identified as basic awareness is 186, and of this total, 8 competencies were assigned to the basic awareness level, 39 competencies were assigned to the beginner level, and so on.

†Self-reported proficiency level was determined by identifying the frequency of survey respondents that self-identified their overall proficiency in IR to different levels (ie, basic awareness, beginner, intermediate, advanced, expert) for all competencies and sub-competencies. For example, there were three respondents who reported their proficiency level as basic awareness, and they each assigned proficiency levels to 62 statements for a frequency total of 186.

‡Assignment data were missing (<0.1%) for some respondents under these categories.

IR, implementation research.

For 8 out of the 62 competencies and sub-competencies statements, the mean proficiency scores were significantly different comparing sub-groups of respondents based on their self-reported proficiency level at p value<0.05 (table 3). The pairwise comparisons showed that these significant differences where mostly due to differences between respondents who self-reported their proficiency level as beginners (and who assigned lower proficiency to these competencies) compared with those who self-reported as experts (and who assigned higher proficiency to these competencies). There was no statistically significant difference for any of the 62 competency/sub-competency statements comparing mean proficiency scores for sub-groups of respondents based on their geographical regions.

Table 3

Competencies with differences in mean proficiency scores comparing sub-groups of respondents based on their self-reported proficiency level

Domain and competencies (based on V.2.0 of framework)	Global F, P value
Domain: Identify emerging challenges related to implementation of evidence-supported IDoP interventions
2.3. Identify the potential impact of scaling down (aka de-implementing) an ineffective but often used intervention for addressing that outcome.	0.04
2.4.2. Be able to strategise to address inequities specific to the implementation of a given set of efficacious interventions, and thereby achieving the desired health outcome.	0.02
Domain: Identify and engage relevant stakeholders
3.1.1. Understand key constructs of participatory research, that is, collaborative, equitable, community-based, co-learning, capacity building and so on.	0.04
3.3.1. Be aware of models and methods for facilitating stakeholders’ engagement and participation in IR process.	0.03
3.3.2. Be able to engage stakeholder groups appropriately to gather perspectives and opinions.	0.02
3.3.3. Be able to incorporate stakeholder input into IR practice.	0.03
Domain: Conduct and evaluate IR in a robust, rigorous and ethical manner
8.5.1. Identify potential ethical issues in IR such as safety of participants, power relationships, literacy, disruption of services.	0.03
Domain: Communicate and advocate effectively throughout the IR process
10.3.2. Understand methods of scaling up and what is required for each such as technical assistance, interactive systems, novel incentives and ‘pull’ strategies.	0.03

IDoP, infectious diseases of poverty; IR, implementation research.

Phase 3: developing a conceptual model for IR competencies in LMICs

Using a modified Delphi method (multiple rounds of review and aggregated anonymous responses to reach group consensus), a 2-day review meeting was conducted with 23 IR professionals from LMICs and HICs on 6–7 February 2019 to evaluate the draft V.2.0 of the framework. The review meeting focused on the description of the competencies, sequencing of domains and competencies, and assigned proficiency levels. The 23 IR experts included 11 of the 16 IR programme coordinators from the 7 LMIC institutions, and other IR experts from institutions in HICs and from TDR, Alliance for Health Policy and Systems Research and the WHO Department of Reproductive Health and Research.

Participants at the meeting were introduced to the meeting’s purpose, preceding efforts in developing competencies and methodology used for developing the draft framework. They were then asked to silently reflect on the first two domains and write specific notes regarding the domain names, domain definitions and competencies under that domain. Notes were collected and a facilitated discussion followed, moderated by the principal investigator for the study team until consensus was reached. This process was repeated until all domains were reviewed in the same fashion. Suggested changes to the draft framework of core competencies were noted and organised under five categories: (1) addition/deletion of domains and/or competencies; (2) merging/combining domains and/or competencies; (3) re-wording/re-describing domains/or competencies; (4) re-assigning competencies to a different domain; and (5) changes to sequencing of competencies and domains for training and other recommendations. The final draft of the framework (V.3.0) was then prepared to reflect these changes.

Based on the consensus and recommendations from the 2-day review meeting, the competency on ‘Applying ethical principles in conducting IR’ was elevated to a domain, 7 competencies were deleted and 25 competencies/sub-competencies were added, and nine competencies were re-described. The suggested changes made during this phase are summarised (online supplementary table 4), and were implemented to produce the final version (V.3.0) of the framework (online supplementary table 5). The final version of the framework comprised of 11 domains, 59 competencies and 52 sub-competencies, and these were organised in a conceptual model based on the recommendations from the review meeting (figure 1).

Figure 1

Circles of knowledge and skills relevant for implementation research education and training programme.

The conceptual model depicts the IR core competencies for LMICs as concentric circles of knowledge and skills (figure 1), that is, training in IR following the outlined competencies is not a linear process but circular and iterative, and starting points in IR training may vary widely by the project, institution and challenge being addressed. Stakeholders (innermost circle) are a core component to all aspects of IR. Trainees in IR will need to know how to identify appropriate stakeholders, engage with them meaningfully, form robust collaborations and implement change via these collaborations throughout the IR process. The IR process (middle circle) can be conceptualised as inclusive of four interconnected thematic areas: scientific inquiry, systems, resources, and communication and advocacy. For example, scientific inquiry, where many IR enterprises may start, includes domains focused on development of research questions, understanding and applying appropriate theories, frameworks and study designs to address those questions, possessing a comprehensive knowledge of methodology and the capacity to conduct IR in a rigorous manner (outer circle). Similarly, researchers must possess an understanding of the resources available and required to conduct IR in their settings, and be able to leverage those resources for the other thematic areas included in the IR process, just as skills to conduct scientific inquiry may be relevant for leveraging resources and facilitating the other thematic areas. This interconnectedness among thematic areas (represented by four rectangular blocks in figure 1) highlights the lack of a clear dichotomy between practice and research in IR,13 and the need for trainees to be proficient in knowledge and skills to do both. The systems thematic area encompasses the health systems, implementing organisations, and communities affected by implementation, and an understanding of their associated bottlenecks and barriers and how to identify facilitators for ameliorating these barriers and integrate solutions back into the systems where neccessary. Other results of the IR process that are developed in collaboration with stakeholders through scientific inquiry should be integrated into systems and this further requires skills in communication and advocacy.

Ethical engagement and context are two of the cross-cutting concepts that trainees must understand and apply throughout their work in IR. Implementation researchers need to know how to assess context, including stakeholders (innermost circle), the IR process (middle circle) and the domains under each thematic areas representing the IR process (outer circle), and determine aspects that may be modifiable, implementing change to context as needed and as appropriate. The application of ethical principles for conducting biomedical and clinical research may be different compared with IR.19 It is important for IR trainees to recognise the ethical considerations that are peculiar to IR, and work with key stakeholders to address these considerations throughout the IR process. These include assessing the risks and benefits to the study population, from whom and how best to obtain informed consent, provision of ancillary care during IR, upfront assessment of sustainability and scalability of the intervention if proven successful in a given context.19

How does this framework of core competencies in IR for LMICs compare with others developed for HIC settings?

The core competencies framework presented in this study covers most of the competencies that have been identified from HIC settings,4–8 but it differs in its emphasis on contexts, health systems, ethics, equity issues, communication and advocacy skills (while the other frameworks emphasise IR theories, methods and designs). This difference may reflect how IR is viewed differently in LMIC settings compared with HICs. IR is defined mostly by broad principles that encompass intervention-specific and broader health system issues in LMICs,3 11 and by set of theories, models and approaches, applied to intervention-specific issues in HICs.6 This distinction in how IR is applied further highlights the weaknesses of health systems in LMICs, and the greater need to address health systems and contextual issues in IR in LMICs. There may be opportunity for future studies in IR to learn how theories, models and approaches developed for HIC settings are adapted and applied in LMICs, and how lessons from these adaptations are applied for addressing implementation questions in HICs.

Unlike one of the HIC studies,6 this study showed significant differences in the proficiency mean score for some competencies comparing respondents’ self-reported proficiency level. Respondents with higher self-reported proficiency assigned higher proficiency to competencies around strategies and methods for scaling up or down, addressing inequities, facilitating stakeholders engagement and addressing ethical issues in the conduct of IR. There were also associations between respondents’ self-reported proficiency level and their perception of the proficiency level needed to achieve a competency. Respondents tended to assign most competencies to the same proficiency level they self-reported as having achieved in IR. These findings suggest that perception about proficiency needed to achieve a specific competency may vary by individual expertise level for stakeholders in LMIC settings. ‘Proficiency’ may have been interpreted differently by different respondents, and a majority appear to have interpreted proficiency as ‘difficulty level’ in demonstrating a certain competency and not necessarily the ‘ability’ to perform the competency.7 Hence, it may not be adequate to define proficiency levels for IR competencies based on perception of survey respondents in an LMIC setting as was done in a previous HIC study,6 and an alternative approach should be explored. For example, proficiency levels may be defined based on educational theories or models, for example, the Osler’s model for assessing clinical skills and competencies in medical education could be adapted to IR competencies.20 Such adaptation would assume that there are different levels of proficiency (ie, ability) within each competency and sub-competency, and that these levels can be independently assessed, for example, through knowledge-testing, demonstrated ability to successfully perform the competency under a testing situation or supervised condition, and/or a demonstrated ability to successfully perform the competency without supervision.

Study limitation and next steps

The overall response rates of 75% (n=16) and 56% (n=89) in the first and second rounds of survey may appear low; however, these are well above the typical range for both online and paper-based surveys (about 32%–33%).21 Moreover, the study population had representation from all of the institutions that were previously identified, and prior studies have suggested that a response rate of 40% may be adequate for sorting competencies by proficiency levels.6 There was an over-representation of respondents from institutions in Latin America (34%) because of the existence of a network of IR institutions in Latin America that included the Universidad de Antioquia Colombia, which was one of the seven LMICs institutions that was purposively selected for this project.

A major limitation of this study is the lack of involvement of practitioners from LMICs in the development of the framework. To make the framework to be more relevant for practitioners and policymakers in LMICs, an immediate next step would be to review the list of competencies with networks of practice and key practitioners in LMICs, with the view of understanding how the framework can be tailored for training and supporting practitioners. Other important next steps may involve the development of a conceptual model showing the pathways through which the existing IR training programme in LMICs may produce impact at the population level, and clarifying the roles that the competencies play along these pathways. Such a conceptual model will be useful for two purposes: for strengthening the evaluation of existing IR training programme, and to serve as the basis of future validation studies for the outlined core competencies.

The sequence of domains and competencies presented in the framework suggests a linear sequence for teaching and training researchers with little or no prior exposure to IR. However, this sequence can be customised to the experience of a trainee or group of trainees based on the implementation objective or phases of implementation. For example, the first four domains of the framework may be targeted to trainees involved in planning an IR project, while the next five domains could be targeted to trainees involved in the implementation of IR, and the last two domains could be targeted to those involved in the post-IR phase. On the whole, it is unlikely that all the competencies would be embodied in a single implementation researcher, and these competencies have been laid out such that different members within an IR team can combine expertise to demonstrate these competencies.

This framework of IR competencies extend the work that has been done previously by others,4–6 and will be useful for objectively assessing training needs and the effectiveness of IR training programme in LMICs, and tailoring training programme to address specific needs among implementation researchers in LMICs.

Conclusion

This study is the first attempt to develop core competencies for the rapidly growing field of IR in LMICs. The framework developed in this study will be useful for identifying competencies needed by teams based in LMICs to respond successfully to implementation challenges surrounding effective delivery of life-saving programme and health services, in real-time and under real-life conditions, through research embedded in their local contexts. This framework will also be useful for measuring effectiveness of training programme in IR, providing direction and support for professional development and guiding the development of future training programme in IR for LMIC contexts. Unlike other studies, our study findings highlight the need for a theory-based approach to identifying proficiency levels for IR competencies (ie, to determine proficiency levels for IR based on generalisable educational theories for competency-based education), and the relevance of various IR competencies for LMIC settings compared with HIC settings.