Investigating the cost-effectiveness of health information technologies: a systematic review protocol.

INTRODUCTION: There is a need to develop new, more cost-effective models of healthcare and in this vein there is a considerable international interest in exploiting the potential offered by major developments in health information technologies (HITs). Very substantial investments are, as a result, now being made globally, but these still probably only represent a fraction of the investments needed if healthcare is to make the transition from the paper to the digital era. Investing greater resources is, however, inherently challenging and unpopular at a time of financial austerity and this is furthermore complicated by the thus far variable evidence of health benefits and demonstrable short-term to medium-term returns associated with investments in HITs.
OBJECTIVES: Building on our related systematic overviews investigating the impact of HITs, we now seek to estimate the cost-effectiveness of HITs and as a secondary aim to identify potentially transferable lessons in relation to how to realise returns on investments in these technologies.
METHODS: We will conduct a systematic review to identify the empirical evidence base surrounding the return on investments from implementing HITs. Two reviewers will independently search major international databases for published, unpublished and on-going experimental and quasi-experimental studies of interest published during the period 1990-2013. These searches of bibliographic databases will be supplemented by contacting an international panel of experts. There will be no restriction on the language of publication of studies. Studies will be critically appraised using the Critical Appraisal Skills Programme (CASP) Economic Evaluations checklist. In view of the anticipated heterogeneity in intervention investigated, study design and health system contexts, we will undertake a descriptive, narrative and interpretative synthesis of data. ETHICS AND DISSEMINATION: Ethical approval is not required.
RESULTS: These will be presented in one manuscript. The protocol is registered with the International Prospective Register for Systematic Reviews (PROSPERO) CRD42013005294.

We will systematically identify and critically appraise the available evidence and grey literature on this important policy question and identify research in progress.

The main limitations are those that are common to all systematic reviews, namely the possibility of failing to identify all relevant evidence and the need to ensure that the synthesis of evidence is appropriate and meaningful.

Introduction

Health systems globally face substantial challenges associated with, among other issues, rapidly changing demographic profiles (in particular, ageing populations and the reduced proportion of the population, ie, economically active), increasing numbers of people living with long-term conditions and associated spiralling healthcare costs, and the substantial, persistent variation in the quality and safety of care provision.1 2

There is, in response to these challenges, considerable policy interest in the potential of health information technologies (HITs) to support fundamental and far-reaching transformations of health systems, but achieving change on the scale required is, as our recent work and that of others has clearly demonstrated, challenging, time-consuming and expensive.3–6 Policy makers are therefore increasingly facing major challenges. These include questions surrounding the justification of substantial investments in HIT, particularly in times of austerity and unprecedented cuts in public investment, and associated pressures to achieve returns on investment in the short to medium terms.7 8

For example, as in many other parts of the world, the USA and the UK governments have already committed to making the transition from paper-based health systems to digital health economies.9 10 Substantial investments have thus been made through the Health Information Technology for Economic and Clinical Health (HITECH) Act in the USA and the National Programme for Information Technology (NPfIT) in the UK.11 These however represent first-wave investments in basic underpinning technologies and infrastructures associated with electronic health record (EHR) functionality. Very substantial subsequent investments will be needed in order to build on these infrastructures and utilise data held within them to redesign health systems.4 12

Given these challenges, there is a pressing need to better understand issues relating to the cost-effectiveness of HITs and also to identify strategies of proven value in enhancing economic returns on investments in HITs. This systematic review builds on our recent systematic overviews of the evidence investigating the impact of HITs,4 13 and is the first step in a related phased programme of work investigating approaches to enhancing returns on investments in HITs.

Methods and analysis

Design

We will undertake a systematic review of the published, unpublished and in-progress empirical research literature.

Eligibility criteria

Interventions of interest

We are interested in a broad range of HIT, which include tools to support the management of data (eg, EHRs), support and enhance clinical decision- making (eg, computerised physician order entry and computerised decision support systems) and provide care at a distance (eg, telehealthcare).4 13 14

Study design

The following study designs will be potentially eligible for inclusion:

Systematic reviews, randomised controlled trials (RCTs), quasi-RCTs, controlled clinical trials, controlled before-after-studies, interrupted time series and before-after or cohort type evaluations, undertaken with formal health economic evaluations from the perspective of healthcare providers.

Economic modelling studies.

Primary outcome

Estimates of the range of cost-effectiveness of HITs.

Secondary outcomes

Description of HITs that have been subjected to formal economic evaluation, the geographical, policy and clinical contexts in which these deployments have been made, the time frames over which the economic evaluations have been undertaken and the lessons learned.

Insights into interventions that are potentially cost-effective based on economic modelling studies.

Search methods

We will search the published empirical and grey literature from 1990 until 2013 for work investigating the returns on investment in HITs. This start date has been chosen to ensure that we capture the period when policy makers particularly became interested in the potential of HIT and also because the technologies that existed before this time period are likely to have only limited relevance to contemporary health systems.4 13 We will search the following major biomedical databases: The Cochrane Library (and its associated databases, namely: Cochrane Database of Systematic Reviews; Cochrane Central Register of Controlled Trials (CENTRAL); Cochrane Methodology Register; Database of Abstracts of Reviews of Effects (DARE); Health Technology Assessment Database; and the NHS Economic Evaluation Database), MEDLINE, EMBASE, Google Scholar, ClinicalTrials.gov and Current Controlled Trials. Specific search strategies will be employed for each database; an example of a search strategy developed for MEDLINE is shown in online supplementary appendix 1.

Study selection

Titles and abstracts of studies identified from the searches will be checked by two investigators. The full text of all retrieved potentially eligible studies will be independently assessed against the above criteria by these two investigators. The investigators will decide which of the studies satisfy the inclusion criteria and the methodological quality of eligible studies will be recorded (discussed below). Any disagreements will be resolved by discussion between the investigators, with referral to a third member of the research team, if necessary.

Quality assessment and analysis

Formal quality assessment of eligible studies will be independently undertaken by two reviewers using the Critical Appraisal Skills Programme Economic Evaluations checklist.15 Disagreements will be resolved through discussion, with arbitration by a third reviewer if agreement cannot be reached through discussion.

Data extraction

Data will be abstracted onto a customised data extraction sheet by two independent reviewers; any disagreements will be resolved through discussion, with arbitration by a third reviewer if necessary. Variables to be extracted will include: author and year; title of the study; country of origin; healthcare setting; HIT application(s); estimates of costs and cost-effectiveness; time frame over which evaluations are undertaken; lessons learned. Key findings from each study will be summarised and presented in summary tables.

Data analysis

A quantitative synthesis is likely to prove inappropriate due to the heterogeneity of technologies, study designs and care contexts. Data will therefore be descriptively summarised and narratively synthesised. We will follow four main steps in conducting an interpretive synthesis of our findings: (1) noting the range of functions and uses of existing systems; (2) developing a preliminary synthesis of the findings of included studies; (3) exploring relationships in the findings and (4) exploring the potential transferability of findings to other contexts and settings.16 There are likely to be challenges associated with integrating heterogeneous data sources, due to for example, differences in interventions studied, methods, outcomes, study population and context.17 We will attempt to address these by judging the relevance of heterogeneity, and the quality of evidence and consistency of findings of included studies.17