Improving the performance of community health workers in humanitarian emergencies: a realist evaluation protocol for the PIECES programme.

INTRODUCTION: Understanding what enhances the motivation and performance of community health workers (CHWs) in humanitarian emergencies represents a key research gap within the field of human resources for health. This paper presents the research protocol for the Performance ImprovEment of CHWs in Emergency Settings (PIECES) research programme. Enhancing Learning and Research in Humanitarian Action (ELRHA) funded the development of this protocol as part of their Health in Humanitarian Crises (R2HC) call (No.19839). PIECES aims to understand what factors improve the performance of CHWs in level III humanitarian emergencies. METHODS AND ANALYSIS: The suggested protocol uses a realist evaluation with multiple cases across the 3 country sites: Turkey, Iraq and Lebanon. Working with International Medical Corps (IMC), an initial programme theory was elicited through literature and document reviews, semistructured interviews and focus groups with IMC programme managers and CHWs. Based on this initial theory, this protocol proposes a combination of semistructured interviews, life histories and critical incident narratives, surveys and latent variable modelling of key constructs to explain how contextual factors work to trigger mechanisms for specific outcomes relating to IMC's 300+ CHWs' performance. Participants will also include programme staff, CHWs and programme beneficiaries. Realist approaches will be used to better understand 'what works, for whom and under what conditions' for improving CHW performance within humanitarian contexts. ETHICS AND DISSEMINATION: Trinity College Dublin's Health Policy and Management/Centre for Global Health Research Ethics Committee gave ethical approval for the protocol development phase. For the full research project, additional ethical approval will be sought from: Université St. Joseph (Lebanon), the Ethics Committee of the Ministry of Health in Baghdad (Iraq) and the Middle East Technical University (Turkey). Dissemination activities will involve a mixture of research feedback, policy briefs, guidelines and recommendations, as well as open source academic articles. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Introduction

Community health workers

Community health workers (CHWs) are unpaid or paid lay health workers, with a varied range of training, experience and scope of practice.1 Often employed to mitigate against the ongoing human resource for health (HRH) crisis,2–4 CHWs provide essential primary care at the household and community level. While the training received and roles performed by CHWs differ across contexts, their purpose within local healthcare systems is universal:5 to improve the delivery and extend the reach of primary healthcare services in a cost-effective and equitable manner. More often used in low-income and middle-income countries (LMICs), governments and humanitarian organisations alike implement CHW programmes to increase access to care for marginalised populations and to bridge communities with facility-based services.3

It is well established that CHWs can make a positive impact on the health and well-being of the communities they serve6 and an extensive body of literature supports their effectiveness in the delivery of primary healthcare programmes.7–9 Specifically, there is a large body of work on CHWs for maternal and child health10 11 and HIV and AIDS programming.12 13 Recent studies have also drawn attention to the challenges in implementing CHW programmes including ensuring regular and supportive supervision (F Vallières, E McAuliffe, P Hyland, et al. Measuring the supervision of community health workers: developing and validating the perceived supervision scale. PLoS ONE in review),14 sustaining CHW motivation,15 high attrition rates7 16 17 and optimising CHW performance,18 19 all of which are necessary to ensure successful CHW programmes.20

The performance of CHWs and how it relates to motivation and programme implementation is not well understood. For this study, performance is considered in terms of the WHO's dimensions of a well-performing workforce whereby staff are available (they are retained and are regularly present) and competent (they are productive and responsive).21 22 Kok et al19 23 24 provide insight into the performance of CHWs in LMICs, and highlight that contextual factors can influence CHW performance. Aligned to this, it is also recommended that a minimum set of standardised skills, which are context-specific and respond to community needs, are central to the performance management of CHWs.25 In contrast, ineffective performance is characterised by variable quality in delivery of services, which is thought to have substantial effects on health.22

CHW motivation and performance are linked and appear to be determined by a number of inter-related factors23 including access to resources, community embeddedness, ongoing training and manageable workloads.26 Motivation and interventions that improve motivation and job satisfaction are considered likely determinants of CHW performance.17 27 Similarly, ineffective performance has been attributed to a lack of incentives, poor supervision, demotivation and the absence of ongoing training.5 28 29 Despite these considerations, human resource management for improving CHW performance in health interventions and programmes remains inadequately understood.30 31 While the current literature offers some guidance on what factors are involved in determining the performance of CHWs, little is known about how these factors interact to influence CHW performance. This is partially due to the methodological challenges of measuring motivation and performance and due to a preference for assessing the effects of an intervention solely on health outcomes.

Currently, there is a paucity of studies rigorously examining the determinants of CHW performance in humanitarian emergencies, where the need for such evidence is pressing. Health services in humanitarian emergencies are frequently non-existent or under pressure because of the ongoing violence and conflict,32 yet the needs for healthcare are increased. The impact of humanitarian emergencies on a population's health is severe and exacerbated by increases in food insecurity, population displacement, crowding and poor access to water and sanitation, lack of resistance to infection, the physical and psychological effects of weapons and exposure to violence, and the collapse of basic healthcare services.33 The impact of humanitarian emergencies on health workers and service provision is also extensive and includes the destruction of health facilities, infrastructure, frequent and prolonged shortages in drugs and equipment, loss of qualified health staff, and restricted access to healthcare.34 Numerous humanitarian organisations have established community health programmes as a means to increase access to health services during and after humanitarian emergencies in a bid to overcome infrastructural weakness, promote healthy behaviours and task-shift primary care to available cadres.10 35 Specifically, CHWs in emergency settings are often used to provide essential services under restrictive and sometimes dangerous situations, and have the potential to contribute to the sustainability of health programmes in the postconflict and recovery stages.36 Optimising the performance of CHWs in humanitarian emergencies is likely to be critical to achieving good health outcomes across health conditions, age groups and contexts.37 38

Challenges in CHW programming have been documented in Afghanistan, whereby CHWs reported difficulties with resource supplies, community recognition and health systems functioning.39 They also reported that the social, gender and cultural norms of CHWs can impact on their responsibilities and duties. For example, the authors noted that some CHWs were reluctant to engage in mental health activities, given its stigmatising nature in most contexts. Similarly, health workers operating in Northern Uganda during the conflict faced physical and emotional trauma and other demotivating factors such as insecurity, a disconnect from social systems, and unstable and under-resourced working conditions. Despite these challenges, CHWs continued to demonstrate innovative coping strategies and strong resilience, such as de-identifying themselves as health workers by sleeping in patients’ quarters and not wearing uniforms, finding strength in their faith or turning simple items such as plastic bags into medical supplies.40 A better understanding of how to support and motivate CHWs in humanitarian contexts, how to ensure their motivation is sustained and how motivation impacts performance requires methodologies that are (1) reflective of the complexity and variability of CHW programmes and that (2) can respond to the contextual conditions of the environment. Since contextual factors have been found to influence the performance of CHWs in development settings,41 understanding what enhances the motivation and performance of CHWs working in humanitarian emergencies represents a key research gap within the field of HRH.

In this paper, we present the protocol of a realist evaluation and describe an initial programme theory (IPT) that aims to explain CHW performance. The research background is presented first, followed by the methodology, which describes how we derived our IPT, followed by an explanation of the planned approach and research design. The protocol ends with a discussion of the methodological issues of the study. Taken together, this protocol aims to describe a realist evaluation that answers the question: What improves performance of CHWs in humanitarian contexts?

Background

Intervention and study setting

Starting in September 2016, the proposed research will be carried out over 2 years across three countries (Iraq, Lebanon and Turkey). Since the beginning of the crises in Syria and Iraq, International Medical Corps (IMC) has used CHW interventions to address a shortage in the health workforce, provide access to healthcare for the most hard-to-reach populations and ensure that services are aligned to beneficiary needs. Operating out of field hospitals, primary healthcare clinics and mobile medical units, CHWs are locally recruited from refugee populations to help deliver health education and medical outreach to conflict-affected beneficiaries. In addition to providing CHWs with a stipend and non-financial incentives, IMC also trains CHWs on maternal and child health, chronic non-communicable diseases, child protection and psychosocial support, recognition of diseases prone to outbreak (ie, cholera, measles) and behaviour change communication. Each CHW then serves a population of ∼1000 displaced and conflict-affected persons, providing (1) referrals to IMC-supported services for treatment, (2) delivery of timely and effective health messaging and (3) public health surveillance.

As part of IMC's CHW programmes, over 90 CHWs have been selected from the Syrian refugee population in Southern Turkey to carry out household visits among Syrian refugees in urban areas in the Syrian border cities of Mersin, Reyhanli, Kilis, Nizip and Sanliurfa.

In Iraq, the study will take place in Erbil, Duhok and Ninewa Governorates, in camps and communities among refugees and internally displaced persons (IDPs) displaced from Syria and areas occupied by the Islamic State of Iraq and the Levant (ISIL) as well as in the Baghdad region. In Northern Iraq, a further 90 CHWs work in three formats: from Mobile Medical Units in collective centres in towns and cities, in urban settings serving IDP populations that have fled to the Kurdish Region of Iraq, and in those in formal refugee camps.

In Lebanon, the study will take place in Tripoli, Akkar, Bekaa, Beirut and Mount Lebanon, and the South of Lebanon among the Syrian refugee population. Here, IMC has enlisted more than 100 CHWs to carry out health education across the country, mostly in informal tent settlements. In total, ∼300 CHWs are working across these three countries, directly serving over 300 000 community members across camp and non-camp settings, with refugees and IDPs, and with low-income and middle-income community members.

Aims and objectives

The aim of this study is to provide evidence that will inform the development of interventions to support and improve the performance of CHWs and improve CHW programmes in humanitarian crises settings. The objectives of this study are (1) to address current knowledge gaps in terms of what enhances the performance of CHWs in humanitarian emergencies, and (2) to contribute to the evidence base for the better design of CHW programmes within humanitarian contexts. This evaluation, while expanding our knowledge of what works to improve CHW performance, will further elucidate the specific needs of CHWs in specific contexts within humanitarian emergencies (refugee vs IDP camps, urban vs rural non-camp settings, etc) and inform the design of strategies that will improve performance, with a view to improving healthcare outcomes for the populations which CHWs serve. The study is timely when a number of scholars are calling for a shift from more traditional empirical studies to ones that consider the complex nature of such interventions and the importance of whole systems thinking.42–46

Methods

This study employs a realist evaluation using multiple cases across purposively selected humanitarian emergency contexts where IMC is currently implementing CHW programmes. The complexity and variability of CHW programmes across settings lends itself particularly well to realist studies, and realist methods have been recently recommended for the study and understanding of CHW motivation and performance.19 Realist methods are particularly applicable to humanitarian emergencies, where CHW motivation and performance are likely to show a different pattern.

The cycle of a realist evaluation, adapted from Van Belle et al,47 is outlined in figure 1. Within realist evaluations, initial theories around programmes, or IPTs, are first developed. As realist evaluation sees programmes as theories incarnate, the IPT describes how the programme is expected to work. The IPT is subsequently refined through research conducted across various sites, with the results of this process being more contextually relevant and evidence-based programme theories.48 These refined programme theories may in turn be compared to developing a middle-range theory (MRT) on CHW performance in humanitarian emergencies. The MRT produced through this process of programme specification is defined as the “theory that lies between the minor but necessary working hypotheses…and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change” (ref. 49, p. 39). The MRT therefore acts as an explanatory framework describing which inputs (ie, components of the IMC intervention) and contextual conditions produce the subsequent mechanisms required to generate change (ie, CHW performance). As stated by Pawson and Tilley,50 realist evaluation research follows the ‘traditional research cycle’ of hypothesis (theory) generating and testing. Similar to more traditional types of research, realist evaluations encourage multiple rounds, or iteration in data collection, with each round using previous findings to provide more programme specification.

Figure 1

Realist evaluation research cycle (adapted from Van Belle et al47). CMO, context-mechanism-outcome.

The methods used in realist studies are informed by the IPT, with realist evaluations themselves being methods neutral.47 In the case of this particular study, a combination of quantitative and qualitative methods was selected. Specifically, quantitative methods will be used to (1) measure the various latent constructs described in the IPT using confirmatory factor analysis, (2) ascertain the association between these latent constructs using multiple mediation analysis. Qualitatively, life histories and critical incident reviews with CHWs will be used to understand how these variables influence one another to impact on CHW performance. Research sites were purposefully selected to ensure that they provide sufficient opportunities to test parts of the IPT. Intraprogramme studies (eg, the same programme implemented across different groups) or the same programme being run in different contexts is useful to refine and develop further programme theories and increase the transferability of the findings.51

Formation of IPT

In line with suggestions from Pawson and Sridharan,52 the extraction of the IPT for PIECES' protocol was conducted through an in-depth analysis of literature and programme documents, and through interviews with programme developers and managers held between October and December 2015. Additionally, several focus group discussions (FGDs) and semistructured interviews (SSIs) were conducted with CHWs working within IMC's CHW programmes. Table 1 highlights the documentation reviews and methods used to assist in the formation of the IPT. Important to note is that document collection and analysis was done prior to stakeholder input, with context-mechanism-outcome configurations (CMOCs) being developed and further refined through the FGDs and SSIs. The interview guides were developed based on literature/document findings. This process was done to provide contextual information for further refinement and elicitation of the IPT.

Table 1

Literature and stakeholder input consulted as part of the IPT development process

Source	Emerging theories and performance factors	Notes
Literature and documentation
CHW motivation and performance literature	Self determination theory53Measures of burnout54Individual factorsIncentives (financial and non-financial)Recognition/respectWork factors (training, support and supervision, recruitment processes)Performance literature and performance outcome monitoring55Trust/relationships (community and health service)	Little information from emergency contexts, majority from development contexts
IMC CHW reports	Outcome (performance) indicators	Outcomes
IMC CHW programme design	Programme design and intervention inputsOutcome (performance) indicators	Context and outcomes
Stakeholder input
IMC CHW programme architect for Middle East	Work factors (specifically recruitment)Organisational commitment56	Continual feedback into IPT
IMC CHW programme managers SSIs (n=5)	Organisational justice57Community commitmentOutcome feedback (need to see change)Communication skills	(2) With managers from Turkey(1) With manager from Iraq(2) With managers from Lebanon
CHW focus group discussions (FGD) (n=3)	Organisational justicePsychosocial support/trauma burnoutIncentivesRecognition/respectOutcome feedback/acknowledgement (need to see change)	(2) With CHWs in Turkey, working in 2 programme sites (Rayhanli and Kilis)(1) With CHWs in Lebanon (from Bekaa and Beirut/Mt. Lebanon)
CHW SSI (n=1)	IncentivesOrganisational justiceCommunity commitmentPsychosocial support/trauma burnout	(1) via Skype with CHW in Erbil, Iraq

CHW, community health worker; IMC, International Medical Corps; IPT, initial programme theory; SSI, semistructured interview.

After the completion of IPT extraction, analysis using the CMOC framework as an analytical tool, and the articulation of the associated programme theories was used to create a visual representation.48 PIECES' IPT postulates that a CHW programme's inputs should promote organisational commitment, need satisfaction, psychosocial support and a sense of organisational justice, while also mitigating against burnout among CHWs. These factors combine to influence motivation, which acts as a key determinant of CHW performance. The IPT further suggests that improved CHW performance in the short term will lead to improved health outcomes in the long term. Conditions for a good CHW programme include appropriate incentives, strong work factors (ie, CHW training, supervision, recruitment, security) and consideration for the individual and community contexts. Figure 2 visually presents the above description of PIECES' initial theory that was elicited through the process described in table 1.

Figure 2

PIECES' initial programme theory. The bidirectional errors depict that change (or producing an outcome) does not necessarily happen in a unilateral direction. The context, mechanisms and outcome all influence one another. CHW, community health worker.

Field study design

In line with realist evaluation methodology, our field study design is based on the developed IPT. For this particular study, we will adopt a multiple case study design,58 whereby the IPT will be further refined within each case study (ie, Iraq, Lebanon and Turkey) to develop more contextually relevant programme theories of how to enhance CHW performance in humanitarian settings. Case study comparisons can then compare similar CMOCs and produce transferable recommendations across contexts.

Methods and tools

Table 2 describes the methods and tools that will be used to explore and refine the concepts that emerged from the IPT. Each process will be completed within all three case studies. First, and aligned to the International Test Commission principles,59 existing and, where possible, validated scales measuring the indicators for the mechanisms outlined in the IPT will be translated and back-translated. These indicators for the mechanisms will then be measured at three different time points throughout the proposed 24-month study: baseline (month 3), midterm (month 12) and endline (month 21). Validity will be assessed at baseline using measurement modelling procedures and internal reliability assessed using composite reliability measures. Important to note, however, is that changes to the methods may occur to best refine the programme theories based on findings from the baseline data collection.

Table 2

Methods for theory refinement

Concept/theory to be explored	Proposed methods and tools
Intervention inputs	Document reviewsKIIs with managers, supervisors and CHWs
CHW performance and outcome	KIIs with managers
CHW performance factors	Life histories and critical incident narratives with CHWsKIIs with managers and supervisorsLatent variable modelling
Outcome: quality	Household surveysSemistructured interviews with programme beneficiaries
Outcome: competency	CHW knowledge attitude and practice survey
Outcome: availability, productivity	CHW weekly reporting check cards

CHW, community health worker; KII, key informant interview.

Life histories and critical incident reviews with CHWs will be used to understand their experiences and perceptions relating to performance and their work. These have been used previously for health worker research during conflict and in postconflict settings.40 SSIs with programme beneficiaries and key informant interviews (KIIs) with programme staff will be used to explore particular issues of performance that relate to beneficiaries, and provide specialised knowledge on the programme functioning, respectively.

Sampling

Sampling will be done at the level of the IMC CHW intervention; as such, all participants will be either working with the programme (managers and CHWs) or programme beneficiaries. For all quantitative CHW surveys, the sample of CHWs included will be an estimated 300+ across the three settings, as there are ∼90 CHWs in Turkey, 90 in Iraq and 100 in Lebanon. CHWs will be contacted from current IMC records and all CHWs working for IMC will be invited to partake in the survey. Programme beneficiaries will be selected using convenience sampling methods to participate in the household survey measuring for quality of care. For all qualitative interviews, participants will be selected using purposive sampling. Here, respondents will be deliberately selected on the basis of features or characteristics that will represent a range of stakeholders and enable a detailed understanding of the topic.60 CHWs and managers/supervisors will be purposively sampled based on age, gender, place of work and length of time as a CHW in order to obtain the maximum variation in types of experiences. Programme beneficiaries will be selected based on whether they are considered IDPs or refugees, whether they are in a camp or non-camp setting, age, gender and type of services received.

Data collection

Realist techniques

When possible and in line with the methods, data collection will adhere to realist evaluation techniques. Qualitative information from life histories, critical incident narratives, SSIs and (KIIs) will incorporate the realist interview technique,48 similar to the ‘teacher–learner’ method. This includes teaching the participant the study team's programme theories, having the participant learn and subsequently teach their own theories regarding the question, accompanied by their own theory refinement. Similarly, all quantitative results will be fed back to CHWs and beneficiaries to stimulate discussion based on the findings. Data in the form of records and documents will also be collected to provide additional contextual information including programme inputs and outputs.

Analysis

Analysis will first be undertaken at the case study level. Measurement modelling will be used to model the link between the observed measures (items) and their hypothesised underlying factors as identified from the literature (ie, latent constructs). Multiple mediation conditional process analysis will be used to estimate both the total and specific indirect effects, and to contrast different indirect effects.61 The total effect of the independent variables on the dependent variable (CHW performance) will therefore be apportioned to (1) the direct effect of the independent variable on the dependent variable and to (2) the indirect effect of the independent variable on multiple mediating variable(s), followed by the effect of the mediating variables on the dependent variable. Multivariate regression will then be applied to explore how changes in independent and mediating variables at midterm predict changes in dependent variables at midterm and endline while controlling for country, type of CHW programme (rural, urban, camp, settings), gender, age, income and education. Findings from the quantitative results will then be further explored through further discussions with CHWs, CHW managers and beneficiaries to complement the development of the CMOCs.

Qualitative data analysis (including document reviews) will use approaches to investigate CMOCs and refine the theory. This will be done by thematically analysing the data, with initial codes being informed deductively by the IPT (codes, eg, ‘burnout’ and ‘organisational commitment’). Using the CMOCs concurrently as an analytical framework will help to identify patterns in the themes, and work inductively to refine the initial theory. Stakeholders (actors) inherently have mechanisms and these, combined with the right contextual factors, produce the generative mechanisms related to outcomes.62 It is understanding and explaining this interaction (of mechanisms from individuals and society, combined with contextual factors), which is an outcome of the analysis.

Once the initial analysis of each case is complete, an intraprogramme case study comparison will occur. The IPT will subsequently be refined through each case study, and the comparison of the refined programme theory may help formulate a programme theory that is of middle range. Our refined programme theories will highlight how to enhance CHW performance, by describing the CMOCs, which detail what works (outcome), as well as how (mechanisms) and under what conditions (context). By undergoing this process across three case studies, our programme theories will have a stronger evidence base and therefore be transferable to other CHW programmes within IMC and in other similar contexts.

Ethics and dissemination

Trinity College Dublin's Health Policy and Management/Centre for Global Health Research Ethics Committee (HPM/CGH REC) gave ethical approval for the protocol development phase of this research. For the full research project, ethical approval will be sought within each research country, in addition to Trinity College Dublin: Université St. Joseph in Lebanon, the Ethics Committee of the Ministry of Health in Baghdad, Iraq and the Middle East Technical University in Turkey. In addition, an early context analysis will inform if other permissions, for example, at camp or community level, are required. Informed consent will be required from all participants prior to data collection.

Three main audience groups have been targeted for dissemination activities: research participants, practitioners (non-governmental organisations) and academics who work in the field of health in humanitarian contexts. Dissemination activities will therefore involve a mixture of research feedback, policy briefs, guidelines and recommendations, as well as open source academic articles. Additionally, built into the research strategy is a social media platform, including a website, formation of a working group and workshop/meeting presentations at relevant international conferences.

Discussion

Community health researchers are emphasising the need for contextually relevant and explanatory methodologies to provide insight into community health programmes to strengthen programme design.19 63 64 Realist evaluations are well placed to address this demand due to its epistemological approach and assumptions of health programmes, with community health research standing to benefit from adapting such methodologies.

The lack of previous studies on CHWs within emergency contexts implies that most of what is presented in the IPT has been largely conjectured from studies of CHWs in LMICs, most of which is less specific to the emergency context literature. This has led to the development of a more ‘generic’ IPT, which may be relevant to the emergency and development contexts. It was, however, also refined though consultation with stakeholders within the programme, with findings having been incorporated into its current form. From these early findings, it is possible that there is some overlap of factors affecting performance of CHWs. However, there were other factors such as CHW safety, camp versus non-camp environments and the need for regular security updates that may be unique to humanitarian emergency contexts. The detailed contextual exploration in the full study phase will work to understand how these factors influence performance, with an emphasis on factors specific to emergency contexts.

This study will provide important information relating to the study question, as well as contribute to the methodological advancement of realist evaluations specifically within humanitarian emergencies. Since there is little precedent to follow in terms of conducting Realist Evaluations (REs) within such environments, the authors are cautious of unforeseen methodological issues that may arise. To this end, important ‘methodology checks’ such as periodical reviews from an outside realist expert, presentations to realist working groups for additional advice and frequent consortium check-ins, have been designed into the protocol, as discussed above. Accordingly, the research team will also be reporting on the use of this methodology in addition to the findings from the study itself. Additionally, working across three complex humanitarian settings may present unforeseen challenges and requires a level of flexibility and/or adaptability of the protocol during the research process. Any alterations to the schedule will be discussed with the research consortium and efforts will be made to maintain the integrity of the overall protocol, with rigour and realist evaluation standards kept as a priority. If the level of flexibility required to continue the research deviates too far from the protocol, the consortium will evaluate the ethical and methodological consequences before agreeing on the best way forward.

Conclusions

Factors that contribute to health worker performance, including the motivational and contextual factors that create an enabling environment for CHWs to perform effectively, are poorly understood in humanitarian emergencies.65 The development of robust, context-informed, evidence-based guidelines for CHW programmes in humanitarian emergencies will therefore help ensure the delivery of high-quality services, while also being reflective of CHW needs. Realist evaluations offer a useful way of doing this due to their flexibility and usefulness within complex interventions and can be adapted flexibly to humanitarian emergencies. To the best of the authors' knowledge, this is the first attempted realist evaluation conducted in an emergency context.