eLearning for health system leadership and management capacity building: a protocol for a systematic review.

INTRODUCTION: Health leadership and management capacity are essential for health system strengthening and for attaining universal health coverage by optimising the existing human, technological and financial resources. However, in health systems, health leadership and management training is not widely available. The use of information technology for education (ie, eLearning) could help address this training gap by enabling flexible, efficient and scalable health leadership and management training. We present a protocol for a systematic review on the effectiveness of eLearning for health leadership and management capacity building in improving health system outcomes. METHODOLOGY AND ANALYSIS: We will follow the Cochrane Collaboration methodology. We will search for experimental studies focused on the use of any type of eLearning modality for health management and leadership capacity building in all types of health workforce cadres. The primary outcomes of interest will be health outcomes, financial risk protection and user satisfaction. In addition, secondary outcomes of interest include the attainment of health system objectives of improved equity, efficiency, effectiveness and responsiveness. We will search relevant databases of published and grey literature as well as clinical trials registries from 1990 onwards without language restrictions. Two review authors will screen references, extract data and perform risk of bias assessment independently. Contingent on the heterogeneity of the collated literature, we will perform either a meta-analysis or a narrative synthesis of the collated data. ETHICS AND DISSEMINATION: The systematic review will aim to inform policy makers, investors, health professionals, technologists and educators about the existing evidence, potential gaps in literature and the impact of eLearning for health leadership and management capacity building on health system outcomes. We will disseminate the review findings by publishing it as a peer-reviewed journal manuscript and conference abstracts. TRIAL REGISTRATION NUMBER: PROSPERO CRD42017056998. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

We will perform a systematic review of the literature on the use of eLearning (ie, use of digital technology in education) for health leadership and management in healthcare.

We will follow the ‘gold standard’ Cochrane systematic review methodology and perform a comprehensive search of a range of relevant databases, robust data extraction, risk of bias and quality assessment and a meta-analysis of the outcome data.

In case of a limited number of eligible studies, our review will serve to highlight the evidence gaps and provide detailed recommendations for future research studies.

Introduction

Description of the condition

Health leadership and management capacity is a key component of health systems strengthening. It is essential for provision of cost-effective and equitable healthcare as well as translation of research and innovation into successful public health action.1 2 Effective health leadership and management hold promise for achieving health system goals of improved health outcomes and sustainable development goal of universal health coverage by optimising the use of existing human, technological and financial resources. Healthcare leaders and managers play an important role in enabling effectiveness of care and patient safety, and in ensuring health worker motivation and retention.3 4 Despite an obvious importance in delivery and development of health services, health leadership and management learning, particularly in developing countries, is often self-directed and on-the-job.1 This inattention towards formal health leadership and management training is coupled with a widespread shortage of health management cadre in terms of the amount, diversity and skills.1 In its new strategy on human resources for universal health coverage, the WHO highlights building capacity for effective public policy stewardship, leadership and governance as one of its four key objectives.5 The WHO also underscores the role of public health managers as critical for provision of evidence and technical advice, as well as successful administration and oversight of healthcare policies and guidelines.5

Description of the intervention

The WHO distinguishes two overarching healthcare workforce groups: health service providers and health management and support workers.6 Health service providers are largely concerned with provision of services on an individual level and patient outcomes, while health managers deal with efficiency at the population level, incorporating also patient outcomes as part of a mix to be met with finite resources. The global worldwide shortage of both cadres is expected to rise steeply in the coming years, accompanied by the exponential growth in healthcare spending due to an ageing society and growing burden of non-communicable diseases.7 Effective and efficient management of health systems is critical worldwide to ensure sustainability.

While conceptually distinct, in practice, management and leadership often overlap in many job positions.2 8 In theory, leadership is aimed at inspiring, motivating and bringing together various stakeholders and the organisation with the aim of achieving a shared vision. Management, on the other hand, typically focuses on administrative processes such as planning, budgeting, organising, staffing, controlling and problem solving. It deals with health services, resources and stakeholders and can be further conceptualised into three complementary levels: top management in charge of policy, middle management with supervisory role and operations management responsible for health service delivery.3 For the purpose of this systematic review, we consider health leadership to be closely interlinked to management.

Health leadership and management roles are commonly assumed by clinicians.1 2 And yet, these competences are largely absent in clinical training, prompting some to plea for their inclusion in formal training at both preservice and inservice level.8–11 Some even argue that clinicians have a professional obligation to leadership and management of systems of healthcare, whether on team, departmental, organisational or health authority level.12 Recommendations on core competences for health management training include: strategic thinking, problem solving, governance, leadership, political analysis and dialogue, community and customer assessment and engagement as well as management of human resource, finances, operations and performance.1 Others propose active learning, to reflect on and embrace the complexity of health systems and to allow for innovation.13 Inclusion of self-management skills, such as prioritisation and time management, personal workload planning, delegation and use of new technology as well as ethics training, is also seen as pertinent to health management training.8 14

How the intervention might work

Health leadership and management capacity building does not need to be limited to formal degree programs. The application of information technology in the educational context, referred to as eLearning, could enable a flexible, efficient and scalable training as well as delivery of latest evidence, innovation as part of both preservice and continuous, inservice education. The potential transformative impact of eLearning on health workforce education has been widely acknowledged and endorsed.15

The use of eLearning for health professionals’ education is becoming increasingly important with the advances in information technology and an expotential growth in human knowledge and information. eLearning combined with the traditional education approach (eg, didactic, face-to-face learning) is often referred to as blended learning. eLearning encompasses a number of modalities that differ in terms of their delivery, content, learning objectives, pedagogy and setting. A common eLearning classification differentiates offline and online computer-based eLearning, digital game-based eLearning, massive open online courses, virtual reality environments, virtual patient simulations, psychomotor skills trainers and mLearning among others.16 17 These various eLearning modalities have their own specificities, advantages, limitations and challenges. While eLearning is the medium by which learning is delivered, it is also a complex blend of technology, pedagogy and content knowledge.

Below we present a framework on how capacity building (in form of both traditional and eLearning) for health management leadership and management can affect health system outcomes (figure 1). We build on the Human Resources for Health Action framework, the UNAIDS Conceptual Model: Leading, Managing and Governing for Results and the President’s Emergency Plan for AIDS Relief Capacity Building Framework.18–21 The listed components in the framework are not meant to be exhaustive but to provide an example of the relevant activities. The framework differentiates three concentric levels at which capacity building can take place, system, organisational and individual level, and acknowledges the impact of various external factors. Healthcare workforce is divided into leadership and management capacity building and technical capacity building reflecting the WHO classification of the workforce into health management workers and health service providers.6 The immediate impact of the capacity building activities at various levels addresses competences, structure and quality of the healthcare workforce. The long-term impact of the capacity building activities includes changes in health system outcomes, for example, health outcomes, financial protection, user satisfaction, equity, efficiency, effectiveness and responsiveness. The impact of the various components of the framework is not unidirectional (eg, from the environment to health systems or from the organisational level to the individual level capacity building). Instead, the framework’s facets should be seen as interdependent. For example, skilful and strong health workforce influences activities undertaken at the system and organisational level and by improving health outcomes also contributes to economic development.

Figure 1

Conceptual framework for the use of eLearning for health workforce (HW) capacity building on health system outcomes.

Why it is important to do this review

The increasing interest in and recurring calls for health leadership and management capacity building are accompanied by a shortage of evidence to inform best practices. While health management and leadership clearly have an important role in quality and performance improvement, there is a dearth of methodologically rigorous research evaluating their long-term impact on healthcare.1 Health leadership and management have, to date, mostly been unheeded in research on health systems strengthening as well as health professionals’ education.21 Research on the use of eLearning for health workforce in general targets clinicians and adopts the Miller’s pyramid for evaluation focusing on changes in knowledge, skills, attitudes and satisfaction.22 The impact of capacity building on health system outcomes in general is often omitted from both the primary and secondary research literature, however. Despite significant ongoing investments into health sector capacity development worldwide, there are only a few attempts at synthesis of the relevant evidence, with evidence to date mostly focused on district-level management, in-person training and changes in competences rather than health system outcomes.23–26 In this protocol, we outline a framework on the impact of health management capacity building on health system outcomes and a protocol for a systematic review.

Objective

To assess effectiveness of eLearning for health leadership and management capacity building on health system outcomes including patient and population outcomes, care practice and behaviour, user satisfaction and experience, equity and economic outcomes.

Methods

We will follow the Cochrane Collaboration methodology in this systematic review.27 eLearning is defined as the use of any form of information technology for education. We will use the following definition for health management: ‘Healthcare management is the profession that provides leadership and direction to organisations that deliver personal health services and to divisions, departments, units, or services within those organisations’.28 We will follow John Kotter’s definition of leadership as ‘a set of processes that creates organisations in the first place or adapts them to significantly changing circumstances. Leadership defines what the future should look like, aligns people with that vision, and inspires them to make it happen despite the obstacles’.29 Capacity building is defined as an evidence-driven process of developing and strengthening the abilities of new as well as current workforce, organisations and systems.30

Type of studies

We will therefore include experimental study designs such as randomised controlled trials (RCTs), cluster RCTs and quasi RCT. Studies describing development of an eLearning programme without an assessment will be excluded. The eligible studies will primarily include programme evaluations and studies at organisational, regional or national scale assessing the relative performance of different digital technology-delivered interventions. We will not exclude studies on the basis of high risk of bias.

Type of participants

Most health leaders and managers in developing countries are trained health professionals (eg, doctors, nurses, medical officers and pharmacists) who often lack prior training or experience in health leadership or a management.2 We will therefore include studies focusing on the preservice and inservice education of all carders with health management and leadership responsibility including staff formally trained in health management as well as clinicians working as managers.

Type of intervention

As management and leadership are often combined in managerial positions in health systems, we will consider eligible intervention focusing on both. The eligible health management interventions will include the use of eLearning for facility management, regional management, partnership management, quality management, general management, resource management, financing, leadership, governance, health service delivery and organisation, human resources for health, pharmaceutical management and health information. We will also include studies assessing the use of eLearning for training on self-management, peer-to-peer communication, supervision, problem solving, team building, change management, decision-making and risk management as part of the leadership capacity building. We will consider ineligible studies assessing the effectiveness of eLearning for education on prevention, diagnosis and treatment of disorders and conditions as well as the use of eLearning for delivery, conduct and methodology of scientific research as they are the focus of other systematic reviews.

For comparison, we will consider eligible any intervention including no training, traditional training (ie, didactic, face-to-face learning such as lectures or seminars as well as textbook-based learning) as well as other forms of eLearning. We will consider eligible studies from both developed and developing countries but will present and analyse their findings separately taking into consideration the diverse context, resources and infrastructure of these settings.

Type of the outcomes

We will include studies reporting on a range of primary and secondary outcomes in line with the pertinent literature.31 Primary outcomes of interest will be:

Patient and population health outcomes (eg, mortality, morbidity, disease prevalence, quality of life, etc)

Financial protection (eg, out-of-pocket healthcare payments, catastrophic health expenditure)

User satisfaction (eg, service users’ and patients’ reports and ratings)

Secondary outcomes of interest are:

Effectiveness (eg, technical quality of care)

Economic outcomes (eg, efficiency, costs, cost-effectiveness, cost-utility, cost-benefit, etc)

Equity, that is, the fair distribution of healthcare among individuals or groups (eg, utilisation measures, rates of access, etc)

Responsiveness, that is, non-clinical and non-financial quality of care facets that convey interpersonal aspects of the provided healthcare and respect for human dignity(eg, patient experience reports and ratings)

Capacity-building outcomes (eg, participants’ knowledge, skills, attitudes and behaviours)

We will not exclude studies based on outcomes but will also present their findings in a separate section of the review.

Search strategy

We will liaise with the Harvard University librarians to develop a robust search strategy. The search strategy will include the following databases:

MEDLINE (Ovid)

Embase (Elsevier)

The Cochrane Central Register of Controlled Trials (CENTRAL) (Wiley)

PsychINFO (Ovid)

Educational Resource Information Centre (Ovid)

Cumulative Index to Nursing and Allied Health Literature (Ebsco)

Web of Science Core Collection (Thomson Reuters) Google Scholar (first 500 references)

Global Health (Ovid)

Health Systems Evidence

Joint Bank-Fund Library (between the International Monetary Fund and World Bank)

WHO

USAID

Health Systems 20/20

Management Sciences for Health

As healthcare management and leadership embrace a range of different competences described using a diverse terminology, we have decided to focus our search strategy on terms relating to eLearning and health professionals in order to increase its sensitivity. The search strategy for this review will partially overlap with a highly sensitive search strategy we developed for several other systematic reviews focusing on eLearning for health professionals’ education.32 33 The MEDLINE search strategy will be adapted to other electronic databases (see online supplementary material 1). All steps of screening the references will be conducted by two authors independently. For all references with unclear eligibility criteria, we will retrieve a full text of the study and invite a third author to act as an arbiter where uncertainties remain. We also intend to search reference lists of the included studies and topic-related systematic reviews. We also plan to search ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform to identify any potentially relevant ongoing trials. We will search the aforementioned databases from and including the year 1990 to present as the functionality and use of the information technology prior to this year were limited. We will consider eligible studies published in any language. We will include both published as well as grey literature.

Study selection

On running the searches, we will import all identified references from different electronic databases into a single to EndNote library. The duplicate records will be removed. We will calibrate the screening process between the reviewers on the first 500 citations. Two people will independently screen titles and abstracts to identify studies that potentially eligible studies. Next, we will retrieve the full texts of those studies and assess their eligibility in parallel and independently. Any disagreements will be discussed and resolved between the two authors, and if they persist, a third author will act as arbiter. We will contact study authors for clarification of data or missing information. Studies excluded at the full-text stage will be presented in the ‘Characteristics of excluded studies’ table with the exclusion reason. We will present the screening process by using an adapted Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.34

Data extraction

Data will be extracted using a date extraction form. Initially, two researchers will independently extract data and compare the results to assure a common data extraction strategy and for quality control. The data extraction form will be piloted and amended in line with the feedback. We will extract data on the study design, methodology, intervention, control and participants as well as eLearning-related information including the type of device, delivery method, content (text, multimedia, images etc) and eLearning mode. We will contact the study authors for missing or unclear information. Disagreements between review authors will be resolved by discussion and a third review author will act as an arbiter if the disagreements persist.

Risk of bias

The risk of bias will be performed independently by two review authors. The risk of bias assessment will be piloted on 10 studies to ensure consistency and clarity of judgements. Risk of bias in RCTs will be assessed in relation to the following domains: random sequence generation, allocation sequence concealment, blinding (participants, personnel), blinding (outcome assessment), completeness of outcome data, selective outcome reporting and other sources of bias. For cluster RCTs, we will also assess the risk of these additional biases: recruitment bias, baseline imbalance, loss of clusters, incorrect analysis and comparability with individually randomised trials. We will judge each risk of bias for each study using ‘yes’, ‘no’ or ‘unclear’ to indicate high, low or unclear risk of bias, respectively. We will present the risk of bias assessment data in the review using risk of bias tables, graph and summary as well as describe them in the text. The risk of bias assessment will also inform quality of evidence evaluation as part of the Grading of Recommendations,Assessment, Development and Evaluations (GRADE) assessment. Any potential disagreements will be resolved by discussion to reach consensus. We will request clarification or more data from the study authors as required.

Assessment of publication biases

We will aim to reduce the possibility of publication bias with the use of a comprehensive search strategy. If we include more than 10 eligible studies reporting the same outcome, we will create a funnel plot to assess reporting bias.

Data synthesis and analysis

We organised the review findings around the proposed framework. We will aim to pool the findings of studies together in form of a meta-analysis. We will assess appropriateness of pooling of the studies statistically in a meta-analysis by determining the heterogeneity in the contents (eg, type of eLearning interventions, controls used, study design setting and outcomes) and results of the included studies. In case meta-analysis is unsuitable, we will narratively summarise and analyse the findings of included studies using a forest plot with suppressed pooled estimate to facilitate visualisation of the results.

We will use OR and 95% CIs as measures of effect for dichotomous outcomes. As for continuous outcomes, we will use standardised mean differences and 95% CIs. The data reported in medians will be converted to means if possible.35 We will contact the study authors for missing information. We present separately studies with missing data. For studies reporting more than one outcome measure, we will report the measure defined as primary but the study authors. In case of a meta-analysis, we intend to use a random effects model as it provides a more conservative estimate of effect and can be employed in case of moderate heterogeneity.

We will adopt intention-to-treat analytic approach. Two authors will independently rank the quality of evidence using the GRADE criteria.

We will also attempt to conduct subgroup analyses, stratified by:

countries’ income (low/middle-income countries vs high-income countries)

professional groups (type of health professionals, ie, doctors, nurses, pharmacists, etc)

number of repeated interventions (one-off vs repeated interventions)

eLearning modality used (online, offline, mLearning, blended, Massive Open Online Course (MOOC)).

In addition, sensitivity analyses will be employed, if possible, to explore the impact of the study quality on the review findings.27 We will remove studies at high risk of bias as well as small sample studies to determine the potential effect on the pooled effects of the intervention.

Discussion

There is a widespread and growing recognition of the importance of health leadership and management in health system function and strengthening coupled with a lack of evidence on effective capacity building interventions that impact on health system outcomes. eLearning holds promise of an efficient, adaptable and accessible training approach that could be leveraged for expanding training in health leadership and management capacity building. This systematic review aims to synthesise the evidence on the impact of eLearning for health leadership and management capacity building on health system outcomes.

We will follow the Cochrane Handbook, that is, the gold standard for systematic review methodology and use a transparent and reproducible methodological approach. The challenge that we may encounter relate to identifying eligible studies on health management education and health system strengthening due to the interdisciplinary nature of this research area, encompassing economics, public health and education. To address this concern, we will employ a highly sensitive search strategy of a variety of sources. We intend to disseminate the findings of this systematic review by peer-reviewed journal publication, conferences and formal presentations.