Economic evaluations on centralisation of specialised healthcare services: a systematic review of methods.

OBJECTIVE: To systematically review and appraise the quality of economic evaluations assessing centralisation of specialised healthcare services.
METHODS: A systematic review to identify economic evaluations on centralisation of any specialised healthcare service. Full economic evaluations comparing costs and consequences of centralisation of any specialised healthcare service were eligible for inclusion. Methodological characteristics of included studies were appraised using checklists adapted from recommended guidelines.
RESULTS: A total of 64 full-text articles met the inclusion criteria. Two studies were conducted in the UK. Most of the studies used volume of activity as a proxy measure of centralisation. The methods used to assess centralisation were heterogeneous. Studies differed in terms of study design used and aspect of centralisation they considered. There were major limitations in studies. Only 12 studies reported the study perspective. Charges which are not true representation of costs were used by 17 studies to assess cost outcomes. Only 10 reported the detailed breakdown of the cost components used in their analysis. Discounting was necessary in 14 studies but was reported only in 7 studies. Sensitivity analyses were included by less than one-third of the studies. The applicability of the identified studies to a setting other than the one they were conducted in is questionable, given variations in the organisation of services and healthcare costs. Centralisation as a concept has also been variably and narrowly defined as activity of specific services which may not reflect the wider aspects of centralisation.
CONCLUSIONS: Confounded and biased information coming from studies without standardised methods may mislead decision-makers towards making wrong decisions on centralisation. It is important to improve the methodology and reporting of economic evaluations in order to provide more robust and transferable evidence. Wider aspects of healthcare centralisation should be considered in the estimates of costs and health outcomes. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This is the first systematic review to comprehensively attempt to appraise the methodological quality of economic evaluations assessing centralisation of specialised healthcare services irrespective of their speciality.

Checklists adapted from recommended standard guidelines were used to assess included studies.

While rigorous searches were adopted, it is possible that some relevant publications may have been missed.

This review only examines the quality based on the information reported, and judgement could not be made on the quality of conduct from what was reported by studies.

Background

Centralisation of specialised healthcare services is typically characterised by reorganisation of healthcare services into fewer specialised units serving a higher volume of patients and aims to improve patient outcomes and efficiency. Lord Darzi's 2007 report1 recommended the reorganisation of London stroke and major trauma services into specialised centralised higher volume units catering for large populations with high-tech facilities and senior medical continuity. A number of studies2–9 suggest that reconfiguring healthcare services into fewer consolidated units will lead to increase in high-quality care and better patient outcomes. Cost savings arguably resulting from the economies of scale is also one of the driving forces behind centralisation of healthcare services.10 11 However, redistribution of limited resources could have secondary or unpredictable effects such as increased costs of access for patient and their carer. Increased journey distance to hospital may reduce healthcare utilisation particularly in some groups of population such as the elderly, those with poor socioeconomic status and those with poor access to transport.10 12–14 Increased journey distance may also lead to increased risk of mortality in patients with life-threatening medical emergencies, whereas in some cases, the benefits of specialist care in centralised healthcare services may outweigh the detriments of increased travel times.15 Thus, it is important to understand the trade-off between the quality of care and cost of centralisation. Economic evaluation methods can be used to explore this trade-off and inform decisions as to whether the resources required or redistributed to centralise services are ‘worth’ the health outcomes achieved.

Economic evaluations compare the costs and consequences of two or more competing healthcare interventions to identify which makes best use of limited resources. On the basis of the consequences measured and valued, the full economic evaluation techniques are classified as cost-effectiveness analysis (CEA), cost-consequence analysis (CCA), cost-benefit analysis (CBA) and cost-utility analysis (CUA) and can be based on primary empirical studies such as trials, decision analytic modelling or a combination of the two approaches.16

Economic evaluations are increasingly used in decision-making,17–19 but the methodological rigour varies and this can lead to erroneous conclusions being drawn. Therefore, when considering the economic evidence on centralisation of healthcare services, it is important to systematically identify relevant economic evidence and appraise the methodological quality. Such a review has not hitherto been published. The aim of this study was to systematically review and critically appraise the methodological quality of economic evaluations considering centralisation of specialised healthcare services.

Methods

Search strategy

A comprehensive and systematic literature search in the database of PubMed, National Health Service Economic Evaluation Database (NHS EED), Health Economic Evaluations Database (HEED) and Excerpta Medica Database (EMBASE) was first undertaken in January 2015 to identify studies on economic evaluation of centralisation of any healthcare services. The search was updated on 10 March 2016 to check for any new publications. However, HEED and NHS EED ceased their searches in December 2014, so 31 December 2014 is the date of last search of HEED and NHS EED. In addition, the reference lists of the retrieved articles from the search were also manually searched for relevant publications. Google was also used to check for relevant articles. The search was performed using an extensive search strategy using keywords and free text for each of the databases, with no restrictions on date and year of publication. A detailed search strategy is shown in box 1. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)20 guidelines were closely followed during this review.

EMBASE

#1 ((Emergency adj (medici* OR service* OR department)) OR Trauma OR (Speciali* adj (health OR center OR care OR unit OR medical OR service)) OR care OR health OR clinical OR Hospital OR Acute)

#2 (Centrali* OR Regionali* OR Reconfigurat* OR Reorgani* OR Redesign* OR Consolidat* OR Merg*)

#3 1 AND 2

#4 (cost OR cost adj (effective* OR utility* OR saving* OR minimi* OR analysis* OR benefi* OR illness*))

#5 3 AND 4

PubMed

#1 ((Emergency adj (medici* OR service* OR department) OR “ED” OR “A&E” OR Trauma OR (Speciali* adj (health OR center OR care OR unit OR medical OR service)) OR care OR health OR “clinical service*” OR Hospital OR Acute))

#2 (Centrali* OR Regionali* OR Reconfigurat* OR Reorgani* OR Redesign* OR Consolidat* OR Merg* OR Volume)

#3 1 AND 2

#4 (cost OR cost adj (effective* OR utility* OR saving* OR minimi* OR analysis* OR benefi* OR illness*)

#5 3 AND 4

NHS EED

#1 (centralisation OR concentration OR volume OR specialization OR regionalization OR multidisciplinary OR reconfiguration OR Reorganization)

#2 (“Emergency Medical Services” OR “Emergency Department” OR A&E OR Specialized OR Trauma OR “Acute Care” OR Hospital OR “Healthcare” OR “Health Care Service” OR “Clinical Service” OR “Vascular Service” OR Cancer OR Stroke)

#3 Cost OR Economic OR Saving OR Outcome OR Impact OR Implication OR “health economic”

HEED

#1 Hospital OR “health care” OR “health care service” OR Emergency OR Trauma OR “Acute Care” OR “Vascular Service” OR Centre OR Stroke OR Cancer

#2 Centrali* OR Region* OR Reconfigur* OR Consolidat* OR Merg* OR Concentrat* OR Reorgani* OR Redesign* OR “high volume”

#3 1 AND 2

#4 Cost OR Economic OR “health economic” OR Impact OR Implication OR Evaluation OR Outcome OR Effect OR Efficiency OR Sustain

#5 3 AND 4

Selection criteria

Inclusion/exclusion criteria were used to select studies. Studies were included in the review according to the following criteria:

Full economic evaluations comparing costs and consequences of at least two alternatives were included, that is, those using the following methodologies: CCA, CEA, CUA or CBA. Partial economic evaluations, that is, those not involving comparison between alternatives or not relating costs to benefits or those studies which only considered costs analysis/comparison/description were excluded.

All relevant health economic evaluations were considered, including those alongside high-quality randomised trials, modelling studies based on a meta-analysis of data from high-quality randomised trials or using secondary data from literature and those based on observational studies or analysis of large administrative databases.

Literature reviews and studies focusing only on methodology were excluded. Reviews, letters, comments, conference abstracts and other general articles which limited the assessment of their methodological quality were also excluded.

Only publications in the English language were included.

An assessment of centralisation of any specialised healthcare service was included; assessment of institutional/hospital volume of activity as a proxy for centralisation was included but excluded those assessing centralisation on the basis of surgeon/physician volume which measure individual clinician impact rather than the service. It was assumed that the outcomes achieved by a surgeon/physician are mainly driven by the resources provided at the institutional level.

Only publications from peer-reviewed journals were included; it is expected that studies published in peer-reviewed journals have already undergone some basic checks on their quality.

Data extraction

Data extraction from the full texts included in the review was focused on key methodological features such as study objective, population, type of economic evaluation, overall design, economic perspective, time horizon, comparator and intervention, incremental costs and outcomes and handling of uncertainties. One researcher (NB) independently carried out the extensive search and extracted the information from the selected studies, reviewed the evidence and wrote the manuscript with all other authors (PM, CP and LV) who contributed to the critical revision of the manuscript and provided critical comments.

Quality assessment

Studies were assessed for their reporting quality using a template (see online supplementary table S1) based on the Consolidated Health Economic Evaluation Reporting Standards (CHEERS)21 and Drummond's22 checklists. Scoring systems for quality ratings of included studies were not used, but important aspects of economic evaluations were summarised.

Results

Literature search

The literature search of the databases initially generated 11 544 hits of potentially relevant articles. Screening of titles and abstracts resulted in 86 articles which appeared to meet the eligibility criteria. After a detailed review of full text of these articles, 27 articles were excluded. The reasons for exclusion of these full-text articles were: not a full economic evaluation (12), only analysing surgeon volume (8), not analysing centralisation aspects (5), not an economic evaluation (1) and not a peer-reviewed journal publication (1). References from identified papers were also crosschecked and a further five full-text articles were identified and included. A total of 64 full-text articles met the inclusion criteria and were finally considered in this review (figure 1). A list of studies included in the review is available as online supplementary table S2.

Figure 1

PRISMA flow diagram of the study selection. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Key characteristics of economic evaluations

An overview of the key characteristics of economic evaluations included in the review is presented in table 1.

Table 1

Summary of key characteristics of studies included in the review

Characteristics	Studies (n)	Study reference (see online supplementary table S2)
Study country
UK	2	10, 31
Germany	1	6
Netherlands	3	7, 11, 55,
Canada	1	13
USA	47	1, 2, 3, 4, 5, 8, 9, 12, 14, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 33, 34, 35, 36, 37, 38, 39, 40, 44, 45, 46, 48, 49, 50, 52, 53, 54, 56, 57, 58, 59, 60, 62, 63, 64
Taiwan	4	15, 32, 43, 51
Japan	4	16, 21, 42, 47
South Korea	2	41, 61
Aspect of centralisation
Hospital volume	51	1, 2, 3, 4, 6, 8, 9, 12, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26 ,27, 28, 29, 30, 32, 33, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 57, 58, 59, 60, 62, 63, 64
Hospital location	2	11, 31
Healthcare specialisation	10	5, 7, 10, 13, 14, 23, 34, 54, 55, 61
Multidisciplinary care	1	56
Study design
Decision analytic modelling	11	4, 5, 6, 8, 10, 11, 12, 27, 54, 55, 60
Cohort	34	1, 2, 3, 7, 8, 13, 14, 15, 17, 18, 19, 23, 24, 25, 26, 29, 32, 35, 36, 39, 42, 43, 44, 47, 51, 52, 53, 56, 57, 58, 59, 62, 63, 64,
Case control	2	33, 45
Cross-sectional	15	9, 16, 20, 21, 22, 28, 30, 37, 38, 40, 41, 46, 48, 49, 50,
Before and after	3	10, 31, 61
Case series	1	34
Economic evaluation type
Cost utility analysis (CUA)	10	4, 5, 6, 10, 11, 13, 14, 27, 54, 55
Cost-effectiveness analysis (CEA)	4	7, 8, 12, 56
Cost-consequence analysis (CCA)	50	1, 2, 3, 9, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 57, 58, 59, 60, 61, 62, 63, 64
Economic perspective
Societal	4	7, 8, 12, 54
Societal and health insurance	1	6
Healthcare	4	10, 13, 58, 63
Third-party payer	3	27, 56, 60
Not stated	52	1, 2, 3, 4, 5, 9, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 59, 61, 62, 64
Targeted health services
Emergency	14	4, 5, 6, 10, 13, 14, 17, 30, 31, 32, 33, 34, 46, 61
Non-emergency	50	1, 2, 3, 7, 8, 9, 11, 12, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 62, 63, 64

It is recommended that studies provide a clear description of the location, setting and other relevant aspects of the healthcare system so that external validity, generalisability and transferability of study results in a different setting can be assessed.21 All included studies have clearly reported their location and setting. While the majority of studies were undertaken in the USA (n=47), two studies were carried out in the UK, one in Canada, four in European countries and the rest (n=10) in Asian countries. Most (n=51) used hospital volume (defined in terms of activity level) as a proxy for centralisation of healthcare services; however, there was a wide variation in the definition of high-volume and low-volume hospitals across studies. The variation in definition of volume may further complicate the impact of the findings generated from these studies. Volume alone cannot be a proper measure of quality,23 and the improvement in quality is rather explained by underlying mechanisms of care such as staff expertise, resource availability and specific processes of care correlated with volume.24 25 The theme of centralisation assessed was location of health service in two studies, specialisation of healthcare services in nine studies and multidisciplinary care in one study.

It was observed that the majority of the studies followed non-randomised designs, including retrospective cohort (n=34) with one (13) also using cross-sectional effectiveness data; cross-sectional (n=15) of which one (9) was repeated cross-sectional; comparative case series (n=1); case control (n=2) of which one (33) was matched case control; and before-and-after studies (n=3). Cross-sectional designs are particularly prone to selection and measurement bias. Uncontrolled before-and-after study designs are generally considered to have poor internal validity because they may fail to account for any prevailing temporal trend which might confound and bias the outcomes of the intervention.26 27 Only one study (10) which used a before-and-after study design attempted to account for potential confounder and bias by modelling intervention effects before and after centralisation using several independent, population-based datasets and by conducting sensitivity analysis. However, this may be insufficient because it has been argued that controlled before-and-after studies should have at least two intervention sites and two control sites to control for site-specific confounding effects on the observed differences between intervention and control groups.28 Altogether, 11 studies used some form of modelling approach as their analysis method.

Looking at the economic evaluation methods used in the studies, most of the studies used CCA (n=50), where costs and outcomes such as mortality, length of stay, complications and readmissions were assessed in a disaggregated form. CCA, a variant of CEA, presents a range of outcomes of an intervention in a disaggregated form that decision-makers are likely to use in a trade-off of their importance; however, it can be challenging for decision-makers to weigh up different outcomes against each other and compare alternatives.16 CEA is an appropriate economic evaluation method when the outcomes of alternatives can be expressed in common measures such as cost per life years saved, costs per cases averted or costs per readmissions avoided.16 CEA was used in only in four studies. Seven studies (1, 2, 3, 26, 28, 45, 60) which were reported as CEA were in fact CCA. CUA is considered the best approach for decision-making in healthcare because it allows broader comparisons to be made across widely differing alternatives;16 29 nevertheless, it is limited to measuring only health benefits. The results of this review showed that CUA method was used only in 10 studies. CBA was not applied by any of the studies.

Studies assessed a wide range of healthcare services or procedures such as stroke, cancers, trauma, accident and emergency services, congestive heart failure, coronary artery bypass grafting, pancreatitis, organ transplantations and other condition-specific surgical procedures. A majority of studies (n=50) assessed elective care centralisation issues and the remaining (n=14) focused around those appearing to be emergency healthcare need.

Methodological quality of economic evaluations

The results of the methodological quality assessment are presented in table 2 and reflect inconsistency across the studies assessing centralisation of healthcare services.

Table 2

Quality assessment criteria for economic evaluations included in the review

Dimension of quality	Reported (study reference in online supplementary table S2)	Not reported (study reference in online supplementary table S2)	Note
A clear description of the study objective and comparators is provided	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64	52	52: Not clearly stated
Characteristics of target population and subgroups are described and analysed	1, 2, 3, 4, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64	5, 7, 14, 22, 31	55: Reference for target population given; 22: only stated no difference between groups
Setting and location of the study is stated	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64
Study perspective is clearly stated	6, 7, 8, 10, 12, 13, 27, 54, 56, 58, 60, 63	1, 2, 3, 4, 5, 9, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 59, 61, 62, 64	7: Has only healthcare payer costs; 12: not what is stated; 60: not what is stated
Time horizon of costs and benefits is clear	1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 26, 27, 28, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 49, 50, 51, 53, 54, 55, 56, 57, 58, 60, 61, 62, 63, 64	8, 18, 24, 25, 29, 32, 43, 48, 52, 59
Discount rate is stated or an explanation is given if costs or benefits are not discounted (where applicable)	4, 5, 6, 10, 12, 13, 14, 27, 54, 55	3, 7, 15, 51	5, 54, 13: Benefits only
Source and methods used to collect effectiveness data described	1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64	8	8: Not clear
The primary outcome measure(s) for economic evaluation are clearly stated	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64
Quantities of resources are reported separately from their unit costs	6, 7, 10, 11, 12, 54, 55, 54, 56,	1, 2, 3, 4, 5, 8, 9, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 57, 58, 59, 60, 61, 62, 63, 64	7: Only unit costs reported separately, 55 ref to web table given
Approaches used to estimate resource use and cost are clear	1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64	9, 18, 20, 21
Cost components clear	6, 7, 10, 11, 12, 14, 42, 54, 55, 56	1, 2, 3, 4, 5, 8, 9, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 57, 58, 59, 60, 61, 62, 63, 64
Currency/price year are explicitly stated	2, 4, 5, 6, 9, 10, 12, 14, 19, 20, 26, 27, 29, 32, 35, 37, 40, 41, 43, 45, 47, 50, 51, 53, 54, 55, 58, 61, 62, 63	1, 3, 7, 8, 11, 13, 15, 16, 17, 18, 21, 22, 23, 24, 25, 28, 30, 31, 34, 33, 36, 38, 39, 42, 44, 46, 48, 49, 52, 56, 57, 59, 60, 64
Analytic methods supporting the evaluation including methods for dealing with skewed, missing or censored data, extrapolation methods, methods for pooling data, approaches to validate or make adjustments to a model, methods for handling population heterogeneity and uncertainty described in details	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64	7, 13, 14
Incremental costs and outcomes reported (if applicable, incremental cost-effectiveness ratios reported)	2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 26, 41, 48, 54, 55, 56	1, 3, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 52, 53, 57, 58, 59, 60, 61, 62, 63, 64	11: Only in graphs
Sensitivity analysis carried out	1, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 27, 33, 42, 45, 54, 55, 57, 60	2, 3, 9, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 56, 58, 59, 61, 62, 63, 64
Limitations clearly discussed	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63	22, 37	Not clearly reported in 22, 37
Generalisability of findings discussed	1, 2, 6, 10, 15, 16, 17, 19, 20, 21, 28, 30, 31, 32, 33, 34, 35, 42, 53, 54, 60, 61	3, 4, 5, 7, 8, 9, 11 ,12, 13, 14, 18, 22, 23, 24, 25, 26, 27, 29, 36, 37, 38, 39, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 55, 56, 57, 58, 59, 62, 63

Study objectives and comparators

Cost-effectiveness may vary by population characteristics, and it is recommended that these studied should be clearly reported.21 It was observed that most of the studies clearly reported their objectives, comparators and description of the target population characteristics, but statement of objective in one study (52) and population characteristics reported by two studies (55, 22) were incomplete; one (55) referred to another study for the characteristics and the other (22) only stated that there was no difference between the two population groups analysed. Five studies (5, 7, 14, 22, 28) did not provide any description and analysis of the target population characteristics.

Perspective of the study

Perspective is the viewpoint from which the costs and outcomes are evaluated and should be clearly stated in an economic evaluation.21 Only 12 studies clearly reported the perspective of the analysis. Among these, five (7, 54, 9, 12, 60) reported the use of societal perspective, one (6) reported societal perspective along with statutory health insurance perspective, four (11, 13, 58, 63) reported using healthcare perspective and two (27, 56) reported third-party payers’ perspective. However, on a closer examination, it was clear that three (7, 9, 12) and one (60) of those studies which reported using a societal perspective actually followed a healthcare and payers' perspective, respectively.

Effectiveness data

It was observed that the source and methods of effectiveness data were clearly reported in all studies but one (9). The majority (n=52) of studies analysed the data derived from a single-source database. One modelling study (10) used information from three different databases. Eight (4, 5, 6, 7, 11, 12, 54, 55) modelling and two (13, 56) observational studies used information from multiple sources including literature review and databases, but one (13) also carried out a cross-sectional survey to derive effectiveness data. A major limitation of eight (4, 5, 6, 7, 12, 54, 55, 13) of these studies which reported using effectiveness estimates from literature review was the failure to report whether a systematic review was conducted, making it unclear whether the best available evidence was used.

Outcomes assessed

The commonly used outcome measures in studies were in-hospital mortality, length of stay in the hospital, readmissions, life years gained or deaths averted, complications, length of stay in the intensive care unit (ICU), long-term survival and quality-adjusted life years (QALYs). Only 11 studies assessed QALYs as one of their outcome measures. Three studies (21, 43, 61) assessed length of stay in the hospital as the only outcome measure (see online supplementary table S2).

Costing

The costs resulting from healthcare centralisation can be broadly divided into direct and indirect costs. Direct costs are healthcare-related costs that result from inpatient or outpatient healthcare services used to address the health problem—for example, costs of surgery, drug treatment, laboratory tests, staff and equipment. Indirect costs are other types of costs that occur outside the healthcare sector—for example, value of time loss such as productivity/wages lost or leisure time lost, travel costs and costs associated with other aspects such as childcare. Direct costs were included in all studies included in this review (see online supplementary table S2). These were the hospital costs such as costs of diagnosis, treatment and surgery associated with the illness; however, 18 of the studies used hospital charges and did not report any conversions to actual incurred costs. Eight (1, 6, 10, 11, 12, 13, 14, 54) studies appeared to have considered indirect costs in their analysis (see online supplementary table S1); however, in one (11) of these, economic evaluation was restricted to travel costs to hospitals borne by patients, two (1,13) were not clear in what indirect costs were included, one (54) included costs of lost productivity and one (6) included costs of hospital infrastructure and informal carer travel, but explicitly excluded the productivity costs citing an average patient age of 74 years. All studies mentioned how the resource use and costs were estimated and all seem to have used activity-based costing method, but three (18, 20, 21) were not clear enough. Only 10 (6, 7, 10, 11, 12, 14, 42, 54, 55, 56) of the studies reported the detailed breakdown of components of the total costs assessed. Quantities of resource use and unit costs were reported in eight studies (6, 7, 10, 11, 12, 54, 55, 56); however, in two (7, 56) of these studies, only unit costs were presented and resource use was not presented separately with one (56) only presenting fee codes.

Adjustments for timing of costs and benefits

It is recommended that the time horizon over which the costs and consequences of an intervention occur and any discount rate used should be stated.21 In this review, it appeared that 40 studies had a time horizon of less than a year. Discounting was not reported, nor is it necessary in these studies with a time horizon shorter than 1 year. However, it is recommended that analysts should report this as 0% discount rates for clarity.21 Fourteen studies (3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 27, 51, 54, 55) had a time horizon longer than 1 year, but only seven (4, 6, 11, 12, 14, 27, 55) reported discounting costs and effects. In three studies (5, 13, 54), the time horizon for costs and effects appeared to be different and effectiveness data only was discounted. Costs in two (13, 54) appeared to have incurred within a year and discounting was not necessary. In the other study (5), the costs were expected for 10 years, but discounting was still not reported. Justification on why costs were not discounted was not provided. It was difficult to ascertain the time horizon in 10 studies (9, 18, 24, 25, 29, 32, 43, 44, 52, 59).

Price changes over time and research settings. Reporting dates, currency and any conversions facilitate comparisons of results from studies carried out at different times and jurisdictions. It is recommended that price year, the currency used and any currency conversions are reported.21 Currency or price dates and conversions were explicitly stated by only 30 studies.

Statistical analysis

All the analytic methods used in the studies must be reported to allow judgement of the appropriateness of the methods and the corresponding results in economic evaluations.21 Single study-based economic evaluations should report regression methods that are used in their analysis.21 The majority of studies (n=53) in this review were primarily single study-based economic evaluations and among these only 45 studies explicitly reported the regression methods and other statistical tests used in their analysis. Reporting of statistical analysis in five studies (31, 39, 46, 52, 56) was very limited. Among these, one (46) only stated using logistic regression analyses for demographic and clinical variables, but nothing was reported on costs. Three studies (31, 52, 56) reported the test of statistical significance in the difference. One (39) of these studies reported the tests of significance but also indicated accounting for clustering effects in groups, one (52) only stated using non-parametric analyses in testing continuous variables but failed to explain further, while another one (56) only stated that comparisons between groups were made by analysis of variance (ANOVA) and did not provide further details. Three studies (7, 13, 14) did not report any statistical methods in analyses of data. Altogether, 24 single empirical studies reported using statistical methods to account for skewed distribution of costs and resource utilisation data, and 14 of them also indicated accounting for potential clustering effects in the groups studied.

Model-based economic evaluations should describe and report methods used in estimating parameters used in the model.21 Among the 11 model-based economic evaluations in this review, seven studies (4, 6, 10, 11, 27, 54, 55) clearly reported the model parameter estimation methods; however, in four (27, 54, 55, 60) of these studies, there was no indication that any possible heterogeneity effects in parameters were addressed. Three studies (5, 8, 12) reported parameter estimation methods poorly, and reported limited explanation on estimation of transition probabilities; however, one (12) study indicated that heterogeneity was addressed. Ignoring heterogeneity effects in parameters may influence the results of an economic evaluation.30 Only two (6, 12) reported half-cycle corrections to account for transition events occurring halfway through the cycle. Not incorporating half-cycle corrections in states may lead to overestimation or underestimation of economic evaluation outcomes.31 Several methods for analysing healthcare costs and handling patient heterogeneity are currently in practice.30 32 33 Appropriateness of these methods is dependent on the data used by each study, but it is beyond the scope of this review.

Consideration of uncertainty

Effects of uncertainties in parameters arising out of methodological assumptions, sampling variation or structure of analyses should be described in economic evaluations.21 It was observed that analyses in 54 studies were mainly based on patient-level data; however, only 15 studies presented CIs for cost and effect results and one presented cost-effectiveness acceptability curves. Some form of sensitivity analysis was included by less than one-third of the studies (n=19). Among the eight studies (1, 7, 13, 14, 33, 42, 45, 57) which were mainly analysis of patient-level data, only seven (1, 7, 13, 14, 33, 42, 57) performed sensitivity analysis: six used one-way sensitivity analysis and one (13) used two-way sensitivity analysis.

Eleven studies (4, 5, 6, 8, 10, 11, 12, 27, 54, 55, 60) were model-based analysis. Six (4, 5, 8, 11, 27, 60) performed one-way sensitivity analysis, although reporting was limited to threshold graphs in one (27). In another five (6, 10, 12, 54, 55), one-way or two-way sensitivity analysis was required along with probabilistic sensitivity analysis.

Presentation of results

The recommended practice in economic evaluation studies is to report mean values for the main categories of estimated costs and outcomes of interest as well as the mean differences between the comparator groups and—if applicable—incremental cost-effectiveness ratios (ICERs).21 Although all of the studies in this review presented mean values for the estimated costs and major outcomes, it was observed that only 18 studies reported results in terms of incremental costs and outcomes or ICERs. One (11) of these only presented the results graphically, and the economic evaluation results were not reported clearly in numerical terms. Study limitations were clearly discussed by all studies but two (32, 37). Generalisability of study findings was discussed only by 22 studies.

Discussion

The main purpose of this review was to critically appraise the methodological quality of published economic evaluation considering centralisation of specialised healthcare services. The review identified 64 studies undertaking some form of economic evaluations of the centralisation of healthcare services. Considerable variation was observed in the methodological approaches used. The studies reviewed generally did not adhere to current standards for reporting economic evaluations, and while they might have been sufficient for the purpose they were originally conducted, they provide a very limited evidence base to guide decision-makers in other settings.

Most of the studies in this review used non-experimental designs. It may be impossible or impractical to undertake randomised trials to assess centralisation; therefore, quasi-experimental designs such as controlled before-and-after studies, interrupted time series and repeated-measures studies may generate more robust and convincing evidence compared with non-experimental designs.26 Modelling is also useful when there is a need to understand the long-term costs and effects of an intervention; however, validity of the modelling results depends on the evidence and assumptions on which they are based.16

The health outcomes assessed in studies were generally appropriate to their objectives. However, QALY, which is widely preferred as a summary measure of quantity and quality of life outcomes in health economics,34 was assessed in only 11 studies. Length of stay is widely used as quality and efficiency measures of healthcare services. It is significantly influenced by several other factors, and it has been argued that length of stay alone should not be used as a quality measure to compare healthcare services unless appropriately adjusted.35 Nevertheless, length of stay was the only health outcome measure assessed in three studies (21, 43, 61) in this review. Most of the studies had short-term follow-up, and it is possible that their outcomes would have been different if they had considered longer time frame.

It is important to clearly state the perspective of analysis in studies because cost-effectiveness may vary with the perspective and an intervention which is cost-effective from one viewpoint and may not be from another. For example, from the National Health Service (healthcare) perspective which typically includes direct medical costs, centralisation of hospitals may appear cost-effective, while from a societal perspective which includes broader costs to the society, such as lost productivity and leisure time costs due to additional travel time, costs to family members or caregivers, centralisation may not appear cost-effective. Nevertheless, a large majority (n=52) of studies fail to state their analysis perspective. Studies always did not consider all costs relevant to a particular perspective. One study(6) in this review followed a societal perspective but did not consider the lost productivity costs in the elderly citing their age; however, leisure time forgone may not be valueless to the elderly and not considering it may discriminate against healthcare interventions aimed towards the elderly population. Furthermore, it is also important that the source and methodology used in generating effectiveness are well reported because validity of the estimate of the measure of effectiveness used in the studies depends on the methodology used to generate those estimates.

A number of studies in this review have used hospital charges as costs. However, hospital charges are not considered a good approximation of costs because charges are essentially the list prices which are set to compensate costs such as free or discounted care to the poor, facility costs and other community service costs incurred by any healthcare organisation.36 In this context, it is possible that the charges set by a centralised or a high-volume hospital for a particular treatment are higher than that of a low-volume local hospital to compensate for the better facilities and services it provides to its patients. Alternatively, high-volume central hospitals may charge less than small-volume local hospitals by spreading out their costs of facilities and equipment over a very large volume of patients they cater to. Therefore, using charges in place of costs to determine if centralisation reduces healthcare costs could lead to unsubstantiated conclusions on healthcare centralisation.

Transparency in the costing methodology and cost components is another important aspect in economic evaluations. Depending on the purpose and the context, there exists variation in estimates of commonly used costing approaches;37 therefore, unclear reporting of estimation of resource use and costs may make it hard to judge applicability of estimates in a different healthcare setting. It is recommended that costing approaches and data sources are clearly described in economic evaluations.21 Limited data may lead to biased conclusions, but incomplete reporting may make it hard to judge applicability of study results for a different setting. Nevertheless, very few studies (n=10) in this review have presented the cost components clearly.

Centralisation would also be expected to impact on the travel distance to hospitals, resulting in changes in cost to health services and patients as well as potentially health outcomes. The impact on financial and other costs of accessing healthcare caused by centralisation may have equity implications as increases in cost may be disproportionately felt by those less well-off who coincidently may also be in the greatest need.38 Furthermore, costs generated by the increased distance to hospitals would become a significant component of the total costs of centralisation. Thus, it is important that wider aspects of healthcare centralisation are considered in the estimates of costs and health outcomes.

Notwithstanding the limitations as presented earlier, economic evaluations assessing QALYs as well as other relevant health outcomes and direct and indirect costs would give a better picture of costs and benefits of healthcare centralisation (see online supplementary table S1). Sensitivity analysis helps to understand the robustness of their findings by varying the assumptions in the values of major variables.

Many health services are facing an increasing pressure to centralise healthcare services into fewer but more specialised units. The case for greater concentration of some of the specialised health services reflects a relationship between quality of care and patient volume, derived from the greater clinical expertise, more specialised facilities and greater standards of care in the concentrated services.39 While it may be attractive and fairly acceptable to transfer findings from one setting to another, healthcare costs and outcomes often have limited transferability across settings because of differences in health and economic systems.40 In the light of the methodological limitations and differences, decision-makers should take caution while making decisions on the basis of existing economic evaluation studies on centralisation of healthcare services. Decisions on centralising any healthcare speciality should be informed by findings from methodologically strong economic evaluation studies considering multifaceted aspects of centralisation. Studies adhering to standard guidelines on economic evaluations and with a clearly reported methodology in terms of cost components, outcomes, analysis perspective, time horizon, robustness of the findings, limitations and generalisability aspects of the findings would assist decision-making.

Strengths and limitations

A number of economic evaluations on centralisation of specialised healthcare services have been conducted; however, studies assessing the methodological quality of these economic evaluations are lacking. An earlier review41 of economic evaluations considering centralisation was narrower and focused only on specific health issues such as cancer. To the best of our knowledge, this is the first systematic review to comprehensively attempt to assess the methodological quality of economic evaluations assessing centralisation of specialised healthcare services irrespective of their speciality. One of the strengths of this review is the use of a checklist adapted from recommended standard guidelines. While we have adopted rigorous searches, we caution that we may have left out some key search terms, and it is possible that we may have missed some relevant English language publications. Databases may be sensitive to different search terms, and use of generally similar search terms across databases could have missed key papers. A more substantive limitation is the exclusion of unpublished evaluation, grey literature and non-English language studies. However, it is also likely that unpublished and grey literature may either be of lower methodological quality or be no different compared with the published studies.42 Since quality assessment was primarily conducted by only one researcher, the likelihood of bias in quality assessment cannot be ignored. Length of stay, also a surrogate for cost, was assessed as a quality outcome measure in this review, and because of this, some of the studies were classified as CCA. Finally, though the checklist used to assess the methodological quality was adapted from standard guidelines, it only examines the quality reported by studies. It was not possible to judge the quality of conduct from what was reported by studies and was beyond the scope of this study. Although arguably, complete reporting is a part of good conduct and the safe inference from incomplete reporting is to assume the quality of conduct was poor.

Conclusion

This paper demonstrates economic evaluations on centralisation of specialised healthcare services have limited methodological quality and their results should be interpreted with caution in other settings. The rationale behind centralisation of specialised healthcare services is the improvement in healthcare quality and its efficiency. However, evidence coming from methodologically poor studies may force decision-makers to make uninformed decisions on centralisation. It is important to improve the methodology and reporting of economic evaluations so that decisions to centralise specialised healthcare services are informed by robust evidence of improvements in healthcare quality and efficiency arising out of centralisation. Future economic evaluations of specialised healthcare centralisation should adhere to standard guidelines on economic evaluations. Estimates of costs and health outcomes of specialised healthcare centralisation should consider wider aspects of centralisation.