Developing a tool to measure the reciprocal benefits that accrue to health professionals involved in global health.

Research to date on global health collaborations has typically focused on documenting improvements in the health outcomes of low/middle-income countries. Recent discourse has characterised these collaborations with the notion of 'reciprocal value', namely, that the benefits go beyond strengthening local health systems and that both partners have something to learn and gain from the relationship. We explored a method for assessing this reciprocal value by developing a robust framework for measuring changes in individual competencies resulting from participation in global health work. The validated survey and evidence-based framework were developed from a comprehensive review of the literature on global health competencies and reciprocal value. Statistical analysis including factor analysis, evaluation of internal consistency of domains and measurement of floor and ceiling effects were conducted to explore global health competencies among diverse health professionals at a tertiary paediatric health facility in Toronto, Canada. Factor analysis identified eight unique domains of competencies for health professionals and their institutions resulting from participation in global health work. Seven domains related to individual-level competencies and one emphasised institutional capacity strengthening. The resulting Global Health Competency Model and validated survey represent useful approaches to measuring the reciprocal value of global health work among diverse health professionals and settings. Insights gained through application of the model and survey may challenge the dominant belief that capacity strengthening for this work primarily benefits the recipient individuals and institutions in low/middle-income settings.

Participation in global health research and service delivery efforts has increased and involves diverse stakeholders and institutions, including universities and hospitals.

The reciprocal sharing of knowledge and innovations among partners in low/middle-income and high-income settings has been increasingly recognised.

Few frameworks or tools are available to measure reciprocal value and global health competencies among diverse health professional and institutions resulting from this work.

A novel and validated tool was developed to measure the reciprocal benefits of international health work among diverse healthcare professionals.

Understanding that global health work benefits both providers and recipients challenges the dominant rhetoric and belief that only recipients benefit, and this could transform future approaches to global health.

Introduction

Global health represents an assimilation of educational, humanitarian and research interests, with participation of diverse stakeholders, including academic institutions, hospitals, and private and non-governmental organisations.1 Global priorities over the last 15 years, including the Millennium Development Goals and the more recent Sustainable Development Goals,2 have catalysed an increase in global financial resources and political will to support such endeavours. As a result, a strengthened focus on partnership and collaboration to address global health challenges has been established.3 4 Reflecting this renewed focus, several authors have documented an increased expansion and prioritisation of global health academic degrees, programmes and research efforts led by universities and academic institutions,5 as well as substantial variability in the expertise and focus of health professionals involved in global health work.6

Research on the impact of global health partnerships and capacity development initiatives frequently emphasises that knowledge and competencies are translated from individuals and institutions in high-income settings to those in low- or middle-income countries (LMICs).7 Recently, the perception that capacity and resources are transferred in a unidirectional process has been challenged, as the reciprocal value and benefits for both partners are increasingly recognised.7–11 Research on ‘reverse or frugal innovation’10 12–15 and institutional ‘twinning’, collaborative learning, and partnerships between stakeholders in LMICs and high-income countries10 16 17 underscores the importance of shared learning and highlights the mutual benefits for all partners involved in global health work.

Despite this emphasis on partnerships in global health, little is known about approaches to evaluating or measuring the ‘reciprocal value’ for institutions or diverse health professionals working in global health. Research with nurses and physicians who have returned from field work in LMICs reports changes in ‘soft’ skills relating to personality or interpersonal skills, and a positive impact on the individuals’ clinical practice. However, neither the impacts on allied professionals or institutional competencies are well documented.8 In addition, few existing frameworks have conceptualised global health competencies at the individual level,18 institutional level19 or across levels and professions.6 8 Further limiting this area of research is that the global health research community has yet to agree on and adhere to a set of standardised core competencies, nor have they developed or endorsed validated tools for measuring changes.6 The paucity of evidence on the skills and competencies gained by individuals who participate in global health efforts on both sides of the partnership poses an important challenge and opportunity for further research in this area.

To this end, we reviewed the literature, developed a validated survey and proposed a conceptual model of reciprocal value. The Global Health Competency Survey was developed and validated to measure the reciprocal value of global health work among diverse healthcare professionals within a paediatric tertiary health facility. These findings subsequently informed an evidence-based conceptual framework, the Global Health Competency Model. The study by Carbone et al 20 used this model and survey to measure the experiences of health professionals participating in global health work, and found gains in personal, professional and organisational competencies. The survey was revised, condensed and validated and may be appropriate for other research settings and stakeholders, including academic and healthcare institutions pursuing global health research, capacity building and global partnerships.

Development and validation of model and survey

Step 1: Comprehensive literature review

We searched peer-reviewed publications to determine commonly reported benefits and value of international partnerships. We used the Queen’s University Library Catalogue to search key databases including PubMed, PubMed Health, Embase, Proquest, Access Medicine, Cochrane Library, Medline, ScienceDirect and GoogleScholar (online supplementary appendix A). To inform the initial survey development, we reviewed 78 articles written in English and published between 1990 and 2015. Eleven publications included discussion of the core competencies required by health professionals, with examples of general or global competency development frameworks.18 19 Studies between 2015 and May 2018 were also reviewed and 11 additional articles included, using the same search strategy to ensure coverage of recent literature to inform the finalisation of the conceptual model, background and discussion. Most studies were published in Canada, the USA and the UK, and referenced partnerships with organisations in LMICs. This review informed the initial development of the Global Health Competency Survey and Model.

Step 2: Development and validation of global health competency survey

The survey was initially composed of 22 questions and 106 items measuring self-reported competencies and values gained through participation in global health work (online supplementary appendix B). Survey questions spanned personal and professional impact and were measured on a 5-point Likert scale, ranging from ‘strongly disagree’ to ‘strongly agree’ or ‘yes’ or ‘no’. Five additional questions were included to assess the value attributed to individuals’ global health work by colleagues and hospital management, to understand the nature of internal institutional support for global health work and to inform future strategic direction. Five participants were purposively selected to pilot test the questionnaire and provide feedback on survey clarity, content and length. Pilot responses were excluded from the survey analysis and results. Participant eligibility criteria included nursing professionals (clinical, educator, nurse practitioner, etc), physicians, research staff or allied health professionals who were currently employed at the Hospital for Sick Children, a paediatric tertiary healthcare facility located in Toronto, Canada. Participants were required to have undertaken fieldwork internationally in an LMIC or low-resource setting. The web-based survey was sent to 478 health professionals by email and administered through SurveyMonkey in July 2015. Respondents were given 3 weeks to complete the survey. The response rate was 34%, with 161 surveys completed; however, only 156 individuals (33%) met the eligibility criteria for inclusion in the analysis. An assessment of sampling bias by gender and occupation was conducted by comparing the demographic characteristics (online supplementary appendix C) of the survey sample and the target population and showed no statistically significant differences between respondents and non-respondents. Research ethics approval for this study was obtained from the Hospital for Sick Children Ethics Review Board in July 2015. All statistical analyses were conducted using SPSS version 23.0, and type 1 error rates were controlled at 0.05.

Survey validation

A factor analysis was conducted to reduce and categorise questionnaire items into descriptive, meaningful categories, based on underlying constructs.21 A maximum likelihood method of analysis was performed, based on the data from 156 health professionals.21 22 The Kaiser-Meyer-Olkin (KMO) measure, Bartlett’s test of sphericity, cumulative per cent of variance explained, Cronbach’s alpha and the component correlation matrix were used to assess the suitability of exploratory factor analysis for the Global Health Competency Survey.

Exploratory factor analysis is a variable reduction technique used to identify underlying constructs or factors within the Global Health Competency Survey.23 Factors were composed of at least three items,21 and missing data were replaced with the variable mean. Sensitivity analyses of approaches for handling missing data were conducted, with no meaningful differences identified between replacement with mean, pairwise and listwise deletion of missing data. Eigenvalues of ≥1 and scree plots were used to determine the number of underlying constructs.23 An orthogonal varimax rotation was used to improve interpretation of factors, as the analysis involved uncorrelated factors.21 24

Factor loadings represent a measure of the contribution of specific items to the factor, and values greater or equal to approximately 0.4 were used to identify contributing items. Items that experienced split-loading, or those that loaded onto more than one factor, were assessed using a cut-off of factor loading of 0.4. Questions with small sample sizes (n<40) were removed, as were items that addressed overlapping factors. Questions that demonstrated floor or ceiling effects, where at least 33% of the responses fell within the most extreme categories were also removed25–27 (online supplementary appendix D). Multicollinearity of variables was assessed through analysis of coefficients in the correlation matrix, and items having a correlation greater than 0.70 were removed to ensure items did not overlap. These strategies were employed to condense the survey and mitigate potentially skewed results. In total, 17 questions were removed. The final survey is composed of 80 items, including demographic information (7 items), institutional value (5 items) and 68 items distributed across personal, professional and institutional capacity development domains (online supplementary appendix E).

Overall factor analysis

A complete factor analysis conducted with all questionnaire items using eigenvalues of ≥1 and the scree plot (online supplementary appendix F1) produced a seven-factor solution (online supplementary appendix F2). However, the KMO test failed to verify the adequacy of the sampling for factor analysis. As a result, a three-phased factor analysis was conducted to overcome the limitations of a small sample size.

Factor analysis phase 1

Phase 1 of the factor analysis found a statistically significant KMO (0.874) and Bartlett’s test (p<0.000) and confirmed an adequate sample size for analysis (table 1). Four factors were identified to explain 53.8% of the total variance by analysing the scree plot (online supplementary appendix G) and based on eigenvalues of at least 1. These factors were defined (table 4) and include: (1) knowledge of global health (29.6% of variance explained); (2) cultural awareness and promotion (4.5%); (3) teamwork and motivation (8.7%) and (4) personal capacity development (11.0%).

Table 1

Factor analysis phase 1

Phase 1—factor analysis
KMO measure of sampling adequacy	0.874
Bartlett’s test of sphericity	P<0.000
Cumulative % of variance explained	53.789

*Indicates where an item is found to split-loading, that the selected item will be where the factor item will be loaded onto (based on theory and literature).

KMO, Kaiser-Meyer-Olkin.

Table 4

Factor descriptions, items and measure of internal consistency

Factor	Definition	Items loaded	Cronbach’s alpha (95% CI)
Factor 1: Knowledge of global health, n=142	Knowledge of global health is a broad understanding of the needs, resources and experiences of diverse societies to address health challenges worldwide.	Understanding of global health issues Health challenges Needs Resources Complexity of research	0.869 (0.831 to 0.900)
Factor 2: Cultural awareness and promotion, n=120	Cultural competence represents the ability to demonstrate cultural awareness and knowledge of diverse settings, and respect and work constructively with people from all backgrounds and orientations.	Attitudes and behaviours Share with colleagues Multicultural work	0.794 (0.720 to 0.850)
Factor 3: Teamwork and collaboration, n=68	Teamwork and collaboration is the ability to develop effective relationships with colleagues and team members and to lead and motivate others to achieve objectives and overcome challenges.	Help colleagues Manage team Motivate team Inform team Encourage innovation within team Working on a team	0.863 (0.806 to 0.908)
Factor 4: Personal capacity development, n=150	Personal capacity development is a process of enhancing self-awareness, self-reflection and confidence to empower individuals and enable personal growth. This domain includes items on changes in personal growth, relationships, desires and attitudes.	Desire for education Changes in personal life New friendships Personal growth Desire to travel Attitudes Cultural competence Cultural sensitivity	0.859 (0.823 to 0.891)
Factor 5a: Soft skills, n=90	Soft skills are interpersonal qualities, personal attributes, attitudes and behaviours.	New challenges Receptive to criticism Controlling emotions Interpersonal skills Share opinions Organisation Problem-solving Communication	0.894 (0.857 to 0.924)
Factor 5b: Job skills, n=46	Hard skills represent technical capacities, or skills, within a specific vocation or field of expertise.	Basic skills Assessing patients Patient care Clinical skills Curriculum development Research proposal writing Research skills Teaching skills	0.858 (0.786 to 0.912)
Factor 5c: Self-management and awareness, n=92	Self-awareness and management is the ability to adapt to new and diverse pressures, use challenges and criticism as a catalyst for improvement and take responsibility for tasks.	Exposure to ideas Apply new methods to improve work processes Take responsibility for actions Work under pressure Adapt to changing priorities Receptive to new ideas Adjust to new situations Complete tasks	0.866 (0.820 to 0.904)
Factor 5d: Ethical reasoning, n=63	Ethical reasoning is the capacity to identify and respond with integrity to ethical issues in diverse economic, political and cultural contexts.	Ethical complications Navigating ethical implications Transparency	0.819 (0.724 to 0.884)
Factor 6: Patient-centred care, n=98	The provision of health services that places patients at the centre of care through prioritising their needs, expectations and preferences.	Social determinants and patients Time for patients and families Patient relationship Caring attitude	0.870 (0.823 to 0.908)
Factor 7: Strategic analysis and planning, n=89	Strategic analysis and decision-making is the ability to analyse complex and inter-related factors that shape health to develop and manage goals and priorities.	Stand-by decisions Priorities and objectives Direct team Critical thinking Anticipate trends and identify goals	0.889 (0.848 to 0.922)
Factor 8: Opportunities for improvement, n=112	Opportunities for organisational-level improvement involves the broad sharing of knowledge, skills and resources to enhance global health programmes to address current and global health needs. This domain includes opportunities for preparation for global health fieldwork, partnership, support and resources to maximise global health programmes.	Partner collaboration Patient contact Student contact Skills Programme feedback Individual feedback Teaching opportunities Training Length of travel Support	0.864 (0.823 to 0.899)

Internal consistency of phase 2

Based on eigenvalues (≤1), phase 2 of the factor analysis loaded onto six factors. However, considering the scree plot (online supplementary appendix H1 and H2) and that fewer than three items loaded onto factors 5 and 6, a four-factor solution was proposed. Items analysed in this phase included wide-ranging capacities relating to soft, job-related, self-management, awareness and ethical reasoning skills, and together represent a larger ‘professional capacity development’ domain. This domain evaluated individuals’ responses to job-related skills, which due to the variety of occupations and roles held by respondents, led to extremely diverse responses. In addition, certain skills were not applicable for all respondents. As a result, findings from the factor analysis did not elicit a meaningful solution, and four subdomains were proposed based on the literature review and conceptual framework. Analysis of the internal consistency of the four subdomains reinforce the scale’s reliability to group competencies (table 2). All subdomains reported high levels of internal consistency (0.819 to 0.894).

Table 2

Internal consistency of proposed professional capacity development factor

Professional capacity development
Domain	Cronbach’s alpha (95% CI)
Domain 5a—Soft skills (n=90, 8 items)	0.894 (0.857 to 0.924)
Domain 5b—Job-related skills (n=46; 8 items)	0.858 (0.786 to 0.912)
Domain 5 c —Self-management and awareness (n=92; 8 items)	0.866 (0.820 to 0.904)
Domain 5d—Ethical reasoning (n=63; 3 items)	0.819 (0.724 to 0.884)

Factor analysis phase 3

Phase 3 of the factor analysis identified three factors, which together explained 48.6% of the cumulative variance (table 3; online supplementary appendix 1). The KMO measure of sampling adequacy (0.830) and Bartlett’s test of sphericity (p<0.000) confirmed the sample size as suitable. Factors were named and defined to include: (1) strategic analysis and decision-making (14.8% of variance explained); (2) patient-centred care (6.5%) and (3) institutional capacity development (27.3%).

Table 3

Factor analysis phase 3

Phase 3—factor analysis
KMO measure of sampling adequacy	0.830
Bartlett’s test of sphericity	P<0.00.000
Cumulative % of variance explained	48.570

*Indicates where an item is found to split-loading, that the selected item will be where the factor item will be loaded onto (based on theory and literature).

Description of factors and internal consistency of survey

Cronbach’s alpha coefficient was used to determine the internal consistency of the entire reciprocal value survey, and the 95% CI is provided by the intraclass correlation coefficient average measure. Overall, good internal consistency was demonstrated with Cronbach’s alpha coefficients ranging from 0.794 to 0.894, with values of 0.7–0.8 generally considered to be acceptable values.28 This indicates that the survey scale is suitable and that the items represent good domain measures. Rotated factor loadings and eigenvalues, scree tests and our key theoretical frameworks18 19 informed the interpretation of the factor analysis results. Factors were named and defined based on the existing models of global health competencies8 18 19 and were reviewed and validated by a multidisciplinary research team composed of global health experts and health professionals (table 4).

Step 3: Development of Global Health Competency Model

The proposed Global Health Competency Model is based on a review of global health competencies’ literature, models and frameworks,18 19 and our survey results. The model consists of eight overall domains across personal, professional and institutional competencies and skills (figure 1).

Figure 1

The Global Health Competency Model: an evidence-based framework to measure reciprocal value of global health work.

Application of model and survey

The Global Health Competency Model and Survey were used to assess the perceived reciprocal value of global health work among diverse health professionals working in various professions.20 Engagement in global health work was found to improve personal, professional and institutional competencies and skills, across different health professions, length of engagement in global health work and location of work in both low- and high-resource settings.

Both the model and the survey represent a novel approach to measuring the impact of global health experiences among health professionals and may be used for monitoring, evaluation and transformations in policy, programmes and training. This approach could be used to identify gaps and gains in knowledge and competencies by academic institutions, professional associations, researchers and policy-makers to ensure that health professionals are adequately prepared to engage in global health work. Results demonstrate that the Global Health Competency Survey is a reliable, suitable tool for measuring the impact of global health experiences among doctors, nurses and allied healthcare professionals. Survey validation involved a multipronged approach assessing face validity, pilot testing, implementation and factor analysis to inform the revision, resulting in a final survey tool.

Past research in this area has primarily defined or evaluated competency in global health knowledge and skills among student health practitioners (family physicians, nurses, physiotherapists and occupational therapist),25 29–38 volunteers8 or nurses.38 39 Focused on identifying core competencies, no attempts have been made to develop tools for monitoring and evaluation of reciprocal benefits, to inform education and training curricula,33 40 or to address potential career benefits for health professionals. The Global Health Competency Model and Survey address this gap, identifying a range of benefits experienced by healthcare professionals and their institutions that may prove useful for further investment and support for global health efforts.

Recent research efforts by a subcommittee of the Consortium of Universities for Global Health identified and defined global health competencies for students.6 The competencies identified in our study are consistent with this research, but our work extends to the development and validation of a survey method for assessing knowledge and skill benefits. Our research also identifies unique secondary benefits for organisations that facilitate and engage in global health work. The Global Health Competency Survey and Model may be applied in diverse institutional and country contexts to tailor training and capacity building efforts for health professionals, as well as to support future global health partnerships and initiatives.

Limitations and future research

Potential limitations of the survey tool and factor analysis include the original format of the survey, phased analytical approach and challenges in naming and defining constructs. The original survey used questions with ‘yes’ or ‘no’ as filters. Changes to the survey design eliminated filter questions in order to increase responses to all questions. A smaller sample size for management and administrative professions and technological support limited our analyses of other professionals working in the health sector. Despite high internal consistency, potential shortcomings of the domain relating to professional skills (Factor 5) and its poor performance in the factor analysis could be addressed through replication with a larger sample size. The use of a three-phased factor analysis approach may have limited the discovery of other underlying constructs across domains, yet the differences to the underlying constructs identified were minimal. Cross-cutting factors both within and across the overall domains of personal, professional and institutional competencies may also occur; however, given the strong conceptual basis used to inform the design of the conceptual framework and the statistical fit of overall items to domains, we feel that cross-cutting factors are few. Further, in the case of variables that split-loaded onto two domains, there was sound theoretical justification for the selected domain. Therefore, the analyses conducted should mitigate cross-cutting factors, as they were theoretically and statistically rationalised into the appropriate domain.

Previous research overwhelmingly report positive perceptions emerging from international fieldwork experiences.41–43 To address the potential for this bias, our survey respondents included individuals who volunteered for international work in addition to those whose job required and/or expected to travel. Many survey respondents reported experience in global health work earlier in their careers, and thus, attribution and evaluation of competencies gained while employed at The Hospital for Sick Children, instead of other international work, may be limited. Consideration of competencies gained while employed at The Hospital for Sick Children may be limited.

This study challenged the assumption that global health efforts benefit only the target or ‘recipient’ countries, institutions and individuals. Future applications of the Global Health Competency Model and Survey should consider the feasibility of its use within LMICs. Although the primary aim in global health is to address inequity in health of populations in many LMICs, the concept of mutuality and the maximisation of benefits by all partners involved in international development activities arguably influences partnership sustainability, improved legitimacy and sense of ownership by all partners involved.10

Conclusion

The development of a novel model and related tool to explore and measure the reciprocal value and emerging competencies among diverse health professionals engaged in global health work represents a significant contribution to global health efforts. Increased prioritisation and funding of global health initiatives5 6 underscore the importance of gaining insight on the complexity of international health partnerships and shared benefits. Recent outbreaks and epidemics (ie, Ebola and Zika virus) highlight the importance of ensuring the sustainability of development efforts, effectiveness of international collaborations and partnerships and improved opportunities for the development of global health competencies among diverse health professionals. The reciprocal benefits of global health work challenge the dominant rhetoric regarding the impacts of this work for institutions and individuals in the Global North and South and could lead to transformations in the approach to global health in the future.