Economic evaluation of enhanced asthma management: a systematic review.

OBJECTIVES: To evaluate and compare full economic evaluation studies on the cost-effectiveness of enhanced asthma management (either as an adjunct to usual care or alone) vs. usual care alone.
METHODS: Online databases were searched for published journal articles in English language from year 1990 to 2012, using the search terms '"asthma" AND ("intervene" OR "manage") AND ("pharmacoeconomics" OR "economic evaluation" OR "cost effectiveness" OR "cost benefit" OR "cost utility")'. Hand search was done for local publishing. Only studies with full economic evaluation on enhanced management were included (cost consequences (CC), cost effectiveness (CE), cost benefit (CB), or cost utility (CU) analysis). Data were extracted and assessed for the quality of its economic evaluation design and evidence sources.
RESULTS: A total of 49 studies were included. There were 3 types of intervention for enhanced asthma management: education, environmental control, and self-management. The most cost-effective enhanced management was a mixture of education and self-management by an integrated team of healthcare and allied healthcare professionals. In general, the studies had a fair quality of economic evaluation with a mean QHES score of 73.7 (SD=9.7), and had good quality of evidence sources.
CONCLUSION: Despite the overall fair quality of economic evaluations but good quality of evidence sources for all data components, this review showed that the delivered enhanced asthma managements, whether as single or mixed modes, were overall effective and cost-reducing. Whilst the availability and accessibility are an equally important factor to consider, the sustainability of the cost-effective management has to be further investigated using a longer time horizon especially for chronic diseases such as asthma.

INTRODUCTION

Asthma is a heterogeneous disease, usually characterized by chronic inflammation that is associated with a history of respiratory symptoms such as shortness of breath, chest tightness, wheezing, and coughing, which vary over time and in intensity, together with variable expiratory airflow limitation.1 It is estimated that around 235 million people in the world currently suffer from asthma.2 In 2011, Barnett and Nurmagambetov stated that the direct cost of asthma was USD3259 (2009 dollars) per person per year.3 As this number is expected to increase with urbanization4, so too will its economic burden on the population and country.

In the past three decades, many guidelines have been published and updated periodically to improve the care of this disease.5 There are two main dimensions in the management of asthma: pharmacological and non-pharmacological. Traditionally, asthma management focuses on pharmacological strategies (hereafter called ‘usual care’). Although there is no doubt that more recent and advanced pharmacological treatment is available, it is the non-pharmacological management of asthma that is currently gaining attention because of its potential to enhance a patient’s level of asthma control (hence the name ‘enhanced management’) as adjuncts to usual care, by preventing future asthma symptoms and acute attacks.1

However, wide varieties of non-pharmacological modalities for asthma have been developed since the 1980s, when the first non-pharmacological modalities were published. Some of the general types of modalities include education, environmental control, and self-management. These variations can be specifically distinguished by delivery and content. The contents vary in term of their topic and comprehensiveness, which may cover the disease and its management, medications, how to use inhaler devices, environmental control (where avoidable risk factors or asthma triggers are dealt with, such as cockroach extermination), inhaler labelling technique6, engagement with other personnel such as asthma educators or counsellors7, peak-flow self-management8, dietary advice and ‘complementary and alternative medicines’.9 These contents can be delivered through many ways e.g. by specific health care professionals, online media, and workshops. Enhanced management of asthma can thus involve many forms of non-pharmacological treatments, which are formally incorporated into the overall care to provide a better management of asthma. The range of varieties, however would, mean that there is a corresponding wide range of cost and effectiveness implications in their implementation. This would complicate the uptake of non-pharmacological treatments in other settings. In addition, there are not as many economic studies as there are effectiveness studies comparing an enhanced management/usual care combination to usual care alone’.6,10,11,12,13

This current review does not intend to replicate the works of the most recent review on asthma-related economic studies14, which focused on both dimensions of asthma management, although it is similar in certain of its methodological ways. Rather, this review is hoped to add or update current knowledge in non-pharmacological areas. In addition to focusing solely on this area of care, this review also included a greater number of studies ranging within the past three decades, hence allowing determining the best value-for-money enhanced management available. As such, this review aims to evaluate and compare full economic evaluation studies on the cost-effectiveness of enhanced management (either as an adjunct to usual care or alone) vs. usual care alone.

METHODS

The whole process of article search, selection, extraction, and assessment in this systematic review was done by two independent reviewers to avoid inconsistencies and potential biases that might arise. Discussions among the two reviewers were held when there were inconsistencies. If they failed to reach a consensus, a third independent reviewer was engaged into the discussion before changes to coding instructions were deemed necessary.

Search strategy

Online databases (ScienceDirect, Wiley Online Library, EbscoHost, OvidSP, and Scopus) were searched for published journal articles in the English language from year 1990 to 2012, using the search terms ‘“asthma” AND (“intervene” OR “manage”) AND (“pharmacoeconomics” OR “economic evaluation” OR “cost effectiveness” OR “cost benefit” OR “cost utility”)’. Booleans, wildcards, proximities, quotation marks, and field search were used to increase the specificity and sensitivity of the search terms. The deduplication function was selected in the OvidSP database to remove duplicates in the retrieved searches. It was believed that the year 1990 is a sensible starting period for this review, given that the first asthma guideline was published in the mid-1980s.15 This would allow a substantial amount of time for the spreading of the guideline to the public, and for the implementation of changes in practices according to the guideline. For these reasons, it is possible that the interventions in the studies published from 1990 onward are based on the earlier published guideline.

A hand search from reference lists of articles was not carried out because it is believed that a saturation point had been reached when the duplicates of the same studies gradually became more frequent from one online database to the next.

In addition, a hand search was done for local published/unpublished articles, abstracts and proceedings in the Malaysia Clinical Research Centre databases, Malaysian Journal of Pharmaceutical Sciences, Malaysian Journal of Pharmacy, and National Research and Development Conference. A hand search from a list of studies conducted by pharmacists in public health facilities for all 14 states in Malaysia was also carried out.

Attempts were made to search for the articles’ full text by all means possible, including contacting the author and the publisher.

Selection criteria

First stage screening was done at title-keyword-abstract level. Studies on both the cost and the outcome of enhanced management(s) (either as an adjunct to usual care or alone) were included. Duplication was screened for all the included studies. In the event of ambiguities in the abstract, the full-text article was referenced for better clarity.

Studies were excluded if:

they involve only cost

they involve only outcome

the author or publisher is not contactable

they do not contain original research, including newsletters and magazines

they are review articles

they involve pharmacological asthma intervention or management only

they involve other diseases in addition to asthma

they are irrelevant to the study question e.g. development of guidelines, qualitative studies

Subsequently, the full texts of the included studies were screened for study selection by reviewing the methods and results section of the studies for true full economic evaluation:

Must involve two or more alternatives16

Both cost and outcomes are examined16

The four types of economic evaluations that were considered in this review are as follows:

Cost consequences analysis (CCA), involves a way of reporting cost and an array of outcomes in a separate and disaggregated way, so that no incremental ratios are involved.17

Cost effectiveness analysis (CEA), involves incremental analysis between the calculated differences in costs and outcomes.17

Cost benefit analysis (CBA), values both measured health and non-health outcomes in monetary units.16,17

Cost utility analysis (CUA), involves utilities, quality-adjusted life years (QALY) or its variants as the measured outcomes. 16,17

Studies that applied more than one type of economic evaluation analysis were also included even if only one of the types meets the definition criteria. The final selection of studies then proceeded to the data extraction step.

Data extraction

Data were extracted and categorized according to country origin, economic evaluation analysis design (trial- or modelling-based), type of economic evaluation (CCA, CEA, CBA, or CUA), perspective, time horizon, intervention and follow-up period, study population, alternatives compared, costs, and outcomes. The type of economic evaluation was assigned according to the definition given above. All costs were converted to international dollars by dividing the local currency unit with the purchasing power parity rates for the mentioned price year, and subsequently inflated to 2012 year dollars, as defined by the World Bank Group.18,19 Publication year was used instead if the price year was not stated in the study. Final costs displayed were rounded up to the nearest whole number.

Quality assessment of economic evaluation and evidence

Each study was assessed for its economic evaluation quality using the Quality of Health Economic Studies (QHES) instrument. 20 QHES is a tool that assesses an economic evaluation quality quantitatively.21 In its original form, it has 16 weighted criteria scored on a scale of 1 to 100, where full weight is awarded for a ‘yes’, and no weight for a ‘no’ response to a particular criterion. One special characteristic of this tool is that the weightings are according to the relative importance of each criterion. This leads to a better discrimination between poor and good quality of economic evaluation, and it is suitable for both trial- and modelling-based evaluation.22 It is this implication which drives the choice to use QHES over all other quality assessment tools. In addition to having good reliability23 and construct validity21 evidences, QHES is also a commonly used tool in the literature.22,24,25,26 However, perhaps its major drawback is its double-barrelled items in a single criterion: multiple items related to the same criterion under one single weight. For the purpose of this review, the QHES scoring system was modified without changing the original weights to overcome this drawback. Criteria with double-barrelled questions were allocated sub-weights for each question, but still added up to the original weight (Table 1). Criteria 12 and 13 were made suitable for both trial-based and modelling-based economic evaluation. Criterion 6 was modified to also answer a CBA type of evaluation; this was not applicable to CCA, hence the total base score was 94 for this type of economic evaluation. The score of Criterion 4 did not count whenever it was not applicable, but the total base score remained 100 since its weight was small and did not affect the overall score much. After converting to percentage, a total QHES score of 75 to 100 indicated high quality, 50 to 74 indicated fair quality, 25 to 49 indicated poor quality, and 0 to 24 indicated an extremely poor quality of economic evaluation. A second QHES assessment was made for the same study if it had more than one type of economic evaluation analysis that met the definition criteria above. The average of the scores from both reviewers was taken.

Table 1

Modified Quality of Health Economic Studies (QHES) checklist. In this version, the ‘scoring system’ was modified to cater for the double barrelled items (questions/criteria and its respective scores in bold). A similar question/criteria and scores was added (in italics) for different types of analysis design or economic evaluation. The total points remained as in original version.

	Questions/ Criteria	Scoring system	score
1	Was the study objective presented in a clear, specific, and measurable manner?	Clear, specific, measurable = 7Any two = 5Any one = 2None = 0	7
2	Were the perspective of the analysis (societal, third-party payer, etc.) and reasons for its selection stated?	(1) Perspective = 2 (2) Reasons = 2	4
3	Were variable estimates used in the analysis from the best available source (i.e., randomized control trial - best, expert opinion - worst)?	Randomized control trial = 8Non-Randomized control trial = 7Cohort studies = 6Case-control/case report /case series = 4Expert opinion = 2	8
4	If estimates came from a subgroup analysis, were the groups prespecified at the beginning of the study?	Yes = 1No = 0	1
5	Was uncertainty handled by (1) statistical analysis to address random events, (2) sensitivity analysis to cover a range of assumptions?	(1) statistical analysis = 4.5 (2) sensitivity analysis = 4.5	9
6	Was incremental analysis performed between alternatives for resources and costs?	Yes = 6No = 0	6
	If the case is CBA, then the question shall ask “Was net monetary benefit / cost benefit ratio performed between alternatives for resources and costs?”	CCA type of economic evaluation = NA
7	Was the methodology for data extraction (including the value of health states and other benefits) stated?	Yes = 5No = 0	5
8	Did the analytic horizon allow time for all relevant and important outcomes? Were benefits and costs that went beyond 1 year discounted (3% to 5%) and justification given for the discount rate?	If less than 1 year, only answer for the time horizon. Yes=7, No=0; If more than 1 year, done for (1) Time horizon = 3 (2) Cost discounting = 1 (3) Benefit discounting = 1 (4) Justification = 2	7
9	Was the measurement of costs appropriate and the methodology for the estimation of quantities and unit costs clearly described?	Done for (1) appropriateness of cost measurement = 4 (2) clear description of methodology for the estimation of quantities = 2 (3) clear description of methodology for the estimation of unit costs = 2	8
10	Were the primary outcome measure(s) for the economic evaluation clearly stated and did they include the major short-term? Was justification given for the measures/scales used?	Done for (1) primary outcome clearly stated = 2 (2) include major short-term outcome = 2 (3) justification = 2	6
11	Were the health outcomes measures/scales valid and reliable? If previously tested valid and reliable measures were not available, was justification given for the measures/scales used?	Yes = 7No = 0	7
12	Were the economic model (including structure), study methods and analysis, and the components of the numerator and denominator displayed in a clear, transparent manner?	If modelling study, done for (1) economic model = 2 (2) study methods = 1.5 (3)analysis = 1.5 (4) components of numerator = 1.5 (5) components of denominator = 1.5If not a modelling study, done for (1) study methods = 2 (2) analysis = 2 (3) components of numerator = 2 (4) components of denominator = 2	8
13	Were the choice of economic model, main assumptions, and limitations of the study stated and justified?	If modelling study, done (stated and justified) for (1)economic model = 2 (2)assumptions = 2.5 (3)limitations = 2.5If not a modelling study, done (stated and justified) for (1)assumptions = 3.5 (2)limitations = 3.5	7
14	Did the author(s) explicitly discuss direction and magnitude of potential biases?	(1) direction = 3 (2) magnitude = 3	6
15	Were the conclusions/recommendations of the study justified and based on the study results?	Yes = 8No = 0	8
16	Was there a statement disclosing the source of funding for the study?	Yes = 3No = 0	3
	Total		100

The quality of evidence source was assessed based on Cooper et al’s study27, because it has hierarchies of appropriateness which are suitable for this review. The ranks are informative as to how different evidence sources are appropriate for each data component. There are 6 data components involved: clinical effect size, baseline clinical data, adverse events and complications, resource use, costs, and utilities. High-ranked evidence is ranked 1 or 2, medium is ranked 3 or 4, and low is ranked 5 or 6. If it is unclear to the reviewer, then that source is ranked 9. There is one limitation of using this tool; Cooper et al. did not advise what happens when there is more than one evidence source for a particular data component. As such, in this review, should there be more than one choice for any one component, then the higher-ranked one would be chosen. Similarly, the highest rank would be chosen should there be any discrepancies between the reviewers. Although this assessment is intended for assessing key parameters in economic models, it is still deemed appropriate to be used against non-modelling-based economic evaluation that uses multiple evidence sources in their analysis. All other studies were labelled as Not Applicable (NA) for this quality component.

Background of reviewed studies

The process of article search and selection is detailed in Figure 1. A total of 1169 studies from the databases were retrieved. After 648 (55%) of these were excluded according to type of articles, language, year of publication, and duplicates, 522 studies from online proceeded to the first stage screening. An additional 28 studies from manual searches were included, bringing the grand total to 550 studies. Around 426 (77%) of the remaining studies were excluded due to types of articles, assessment of pharmacological management only, involvement of other diseases in addition to asthma, and irrelevance to the study question. There were 65 excluded studies that investigated either cost or outcome alone. Lastly, full texts for 8 studies were irretrievable as their authors and publishers were not contactable. Subsequently, 51 studies were included in the second stage screening. One of those studies28 was excluded because it only investigated one alternative instead of two or more, as defined by Drummond et al.16 Another study was not considered a full/true CBA (as it claimed to be) because it did not value the measured health outcomes in monetary units.29 Therefore, a total of 49 studies were included in the data extraction process.

Figure 1

Flow diagram of retrieval of searches

The majority of the studies originated from the United States of America7,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54, followed by Europe8,55,56,57,58,59,60,61,62,63,64,65,66,67, Asia68-72, and Australia.73,7475,76 There was one economic evaluation concerning enhanced asthma management30 done in 1991, and at least one between 1994 and 2012. All of the studies were trial-based, except 3 (6%) modelling-based studies.40,65,75 Only 6 studies conducted a mixed type of analysis40,58,59,60,62,64, 5 were CEA studies7,43,51,55,56, 3 CUA studies8,65,75, and 3 were CBA studies62,63,66, whilst all others were CCA studies.

Among the CEA, CUA, and CBA studies, a mixture of education and self-management implemented by an integrated team of healthcare and allied healthcare professionals is deemed to be the most cost-effective (reported to be dominant). In CCA, costs and outcomes are presented separately without involving incremental analysis. Hence, it will not be possible to deduce whether the intervention is cost-effective or not. It all depends on how the decision-maker prefers to value the desired outcomes from their perspective, on the basis of the reported costs and outcomes. Therefore, although some CCA studies reported a reduction in costs and an improvement in outcome measures, it did not mean that that particular intervention was cost-effective.

Among the 12 studies that conducted CEA and/or CUA, there were 4 studies that reported dominance55,56,64,65, which means that the intervention is more effective than the comparator but at a lower cost. The interventions involved were education and self-management. For the remaining 8 studies, both of the outcomes and costs were better and higher than their comparators7,8,40,43,51,58,62,75; the lowest incremental cost-effectiveness ratio (ICER) was Int$14 per symptom-free day (SFD) gained for environmental control intervention7, while the highest was Int$29600 per Quality-Adjusted Live Years (QALY) from a societal perspective for internet-based self-management intervention.8 Although incremental analysis was done and cost-effectiveness was concluded in these 8 studies, they could not be deemed as cost-effective because none reported the willingness-to-pay (WTP) threshold. Without the threshold as a benchmark, it was impossible to tell if the ICER was enough to be cost-effective. At the least, 5 of these studies7,8,43,51,64 presented a cost-effectiveness acceptability curve to determine whether the probability of the intervention was cost-effective at a particular WTP threshold. This, together with the ICER, is of important aid to the decision-makers regarding whether to adopt the particular intervention.

There were 5 CBA studies that reported a positive net benefit for every dollar spent.35,60,62,63,66 In addition to the excluded non-full CBA study29, there was another study that also did not value the health outcomes in monetary units.40 However, this particular study was not excluded from the review, because CEA was part of the study too. The study by Kauppinen et al.58 was the only one that reported a negative net benefit, but this was most probably due to its high implementation cost during the first year, which was taken into account because eduction intervention only took place during the first year, and the study reported a positive net benefit five years later.60

There were generally 3 types of intervention reported: education, environmental control, and self-management. Asthma education was the most common and the earliest reported, followed by self-management. Self-management can be delivered either by written plans or via the internet. Peak-flow and symptom-based are the two most common types of self-management. Economic evaluation of environmental control intervention was not reported until the early 21st century, and it is comparatively rare in that only 3 studies were available prior to 2012.7,43,54 It is difficult to deduce which type of intervention is the most effective, because none yielded consistent results. Except for 3 studies that found no significant differences between their comparators8,74,76, findings from other studies showed that these interventions benefited all severity levels of asthma (from mild to potentially fatal asthma).

While most of these reviewed studies had reported their interventions as cost-reducing in relation to their comparators, there were 2 studies that reported no significant differences in the total costs between their alternatives.41,57 It is regrettable that a number of studies did not report the total cost of intervention or implementation per patient.31,33,34,36-39,41,45,48-50,53,57,61,66,67,68,69,70,71,72,74 Based on what is available, the cheapest intervention among these 48 studies was the enhanced services Nurse Support group with only Int$16 per patient76, followed by a symptom-based self-management plan with Int$43 per patient.40 The most expensive was an Int$3140 self-management program per patient.58

An provides more specific details on the data extraction of the 49 studies, all arranged in ascending manner according to publication year. All costs were adjusted to the 2012 price year. Around 19 (39%) studies did not state the price30,32-36,40,41,44,48,49,52,57,61,66,67,69,70,72,73, year, hence their costs were adjusted according to publication year.

Click here for additional data file.

A total of 53 economic evaluation analyses were undertaken. The mean QHES score was 73.7 (SD=9.7). The maximum and minimum scores were 94.746 and 59.068 respectively. There were 32 (60%) economic evaluations that scored within 50-74 (fair quality)31-39,42,44,48-50,52-54,57-61,66-68,70,717273,74 and the remaining 21 (40%) evaluations scored within 75-100 (high quality).7,8,30,40,41,43,45-47,51,55,56,62,63,64,65,69,75,76 There was no obvious pattern of the QHES scores across the years 1991 to 2012 (Online Appendix). The bar chart in Figure 2 shows the proportion of evaluations that obtained mode and full scores for each criterion. Equal mode and full scores were achieved in Criteria 1, 3, 7, 9, 11, 12, 15 and 16, in which Criteria 11 and 15 had full scores for 100% of economic evaluations. Other criteria had a higher proportion of mode scores than full scores. Studies that did not obtain full score for a specific criterion are listed in Table 2.

Figure 2

Proportion of economic evaluations (n=53) that obtained QHES mode and full scores for each criterion. The black- and white-coloured numbers in the bar chart represent the mode and full score respectively, for each criterion. These criteria are listed and named according to the QHES instrument; ‘objectives’ represents Criterion 1, ‘perspectives’represents Criterion 2, and so on.

Table 2

Proportion of economic evaluations (n=53) that did not obtain full score, for each criterion. These criteria are listed and named according to the QHES instrument; ‘objectives’ represents Criterion 1, ‘perspectives’ represents Criterion 2, and so on.

Question/Criterion	N (%) did not obtain full score	Reference(s)
Objectives	14 (26)	(8, 34, 35, 43, 44, 47, 52, 54, 57, 64, 66, 73)
Perspectives	34 (64)	(30-33, 35,36,37,38,39, 42, 44, 45, 47,48,49, 52, 53, 57-61, 63, 67,68,69,70,71,72, 74)
Variable estimate	25 (47)	(31, 33-39, 41, 42, 44, 46, 49, 50, 52,53,54,55, 61, 63, 65, 67, 68, 72)
Subgroup analysis	4 (67) out of 6 studies did not pre-specify subgroups.	(43, 46, 63, 73)
Uncertainty	36 (69)	(30,31,32,33,34,35,36,37,38,39,40,41,42, 44, 45, 47,4849,50, 52-54, 58, 61,62,63,64,65,66,67, 70,717273,74) For ref. #51, only applies to CEA.
Incremental	18 (35) did incremental analysis. Others NA due to CCA studies.	(7, 8, 35, 40, 43, 51, 55, 56, 58, 60, 62,63,64,65,66, 75)
Methodology	1 (2)	(66)
Time horizon	29 (56) did not do discounting.	(31, 32, 34-39, 41, 44, 49-51, 54, 59,60,61,62, 67, 68, 70, 71)
Costs	22 (42)	(34, 35, 37, 39, 41, 42, 44, 47,4849,50, 52,53,54, 57, 61, 67, 68, 70-72, 74)
Primary outcome	36 (68)	(8, 32, 33, 35, 37,38,39, 41, 42, 44, 45, 47,4849,50, 52, 54, 57, 59,60,61,62,63, 65,66,67,68, 71, 73, 74)
Valid reliable outcome	0	-
Model, methods, analysis	16 (30) did not clearly display numerator and/or denominator components.	(37, 39, 40, 42,44, 47,50,52,57, 61,67, 68,70, 72, 74)
Assumptions and limitations	20 (38) did not state and justify both main assumptions and limitations.	(33,34,35,38,43,57,58,59,60,61,67-69,71,7273,74)
Bias	39 (74)	(8, 31-38, 40, 43, 44, 48, 49, 51, 53, 55,56,57,58,59,60,61, 63, 64, 66,67,68,69,70,71,72, 74, 76)
Conclusion	0	-
Source of funding	15 (29)	(30, 33, 34, 36, 37, 40, 42, 49, 53, 54, 67, 70, 72, 73)

Only 26 (53%) studies were assessed for their quality of evidence sources using Cooper et al. These included 3 modelling-based studies40,65,75 and 23 non modelling-based studies7,8,31,35,37,38,43,44,45,46,47,51,55,56,57,61,63,66,67,70,73,74,76 that involved evidence sources other than their study population in any data component. The stacked bar chart in Figure 3 shows that NA aside, all data components had high quality evidence sources labelled as Rank 1 and Rank 2. The data components ‘Baseline clinical data’ and ‘Resource use’ for the study by De Asis et al. also had a medium quality evidence source because different jurisdiction was involved.

Figure 3

Proportion of ranked evidence sources for each data component in the assessed studies (n=26). Not all components are applicable to the evidences reviewed in the studies.

DISCUSSION

This review informs various modes of enhanced management with a varying degree of cost-effectiveness. Overall, this management is effective, cost-reducing, and may be applicable to different care settings worldwide and to different levels of asthma severity.

Limitations and bias

This review had two limitations. Firstly, the 57 non-English language studies of possible relevance were excluded. The true number of included studies could have been underestimated due to this exclusion. Secondly, about 40% of the studies did not state their price year. Hence, the associated costs in the Online Appendix were underestimated when adjusted according to the publication year. Selection and analytical biases were minimized as much as possible because two reviewers were involved.

Data extraction on reviewed studies

It is not unexpected that education is the most common type of management, as it has been emphasized since the early development of asthma guidelines such as the British Thoracic Society Guidelines 1990 and the Guidelines for the Diagnosis and Management of Asthma 1991 by the National Heart, Lung, and Blood Institute of the United States of America.5 Despite that, this review shows inconsistent results of education intervention alone, although they have similar interventions period. A review by Welsh et al showed similar inconsistent evidence of education intervention delivered at home, compared with standard care or outside the home.77 For the studies in this review that have significant positive outcomes, the sustainability issue is equally important as the time horizon for these studies are mostly one year. The five-year follow-up study by Kauppinen et al. showed that even when controlling for the peak-flow self-management effects of both intervention and control groups, the impact of education on the intervention group in the first year could not be sustained. Having said that, this result may be confounded by the fact that a peak-flow meter was expected to be self-purchased after the first year; hence the possibility of ‘lack’ of self-management itself could likely affect the overall final outcomes. By contrast, a study that was modelled for 5 years by Gordois et al.75 showed that the impact of education can indeed be sustained despite no annual review of the patients by the pharmacists. Another educational approach that may be considered to have an advantage over the sustainability issue is the handbook by Tschopp et al.66,67 In a way, the informative handbook that included a personalized action plan allowed an all-time reference in hand. Provided that the handbook is easy to comprehend and is made suitable for all ages and groups of patients, this approach may be worth adapting. Although education delivered by telephone calls did not demonstrate significant outcomes76, this was probably due to the small sample size (though the power of the study was not specified). It is worth having larger studies to investigate this delivery mode, because there was a significant cost reduction in the female subgroup within the intervention group. After childhood, asthma severity tends to increase more in females than in males.78

More often than not, education and self-management were not mutually exclusive. This relationship has long demonstrated to improve lung function and healthcare utilization79, as shown in this review. In fact, the outcomes and costs reported may have been underestimated for those which had a year or less of follow-up period. On the contrary, the dominance of a peak-flow self-management over two years as reported by Schermer et al.64 could have been overestimated, as neither the costs nor the outcomes were discounted. The direct cost of a similar management that was reported by Ghosh et al.69 was also inaccurate because the costs of physician visits were not taken into account. Although the costs and outcomes were presented clearly for both intervention and control groups, this is a disadvantage to the healthcare providers who wish to evaluate the outcomes from their perspective. The presence of bias or ‘contamination’ could have been the main cause of insignificant differences in outcomes between the three groups in Drummond et al’s study.74 This is justified by the fact that each physician could have several patients randomized into one of the three groups. This performance bias could have been better managed or avoided if the physicians were randomly allocated instead, as was the case in Schermer et al. Overall, the self-management in this review which involved peak-flow and/or symptoms monitoring was promising and also mostly cost-reducing. However, Willems et al found the cost-effectiveness of this type of management inconclusive.80 Nevertheless, the positive outcomes from a trial-based peak-flow self-management may not be the case in real-life situations if owning a peak-flow meter is a burden to the patients. Bearing in mind that peak-flow meters were provided to the study patients by the researchers, the availability of these really depend on the country’s healthcare system. For example, in Malaysia, a peak-flow meter is not subsidized by the government and needs to be self-purchased by patients.81 On the other hand, peakflow meters are freely available in New Zealand82 and covered by insurance such as Medicare in the United States.83

One of the delivery modes of self-management reviewed here is the internet-based mode. Two studies, Runge et al. and Van der Meer et al. have shown that this type of management via internet is suitable for a range of ages, from paediatrics as young as 8 years old to 50-year-old adults. The results of the latter study could have been more attractive if there were more outcome measures than just QALY, such as emergency department (ED) visits or workdays missed due to asthma. Furthermore, the QALYs measured using EuroQol (EQ-5D) may not be the most suitable tool to measure the asthma-specific QoL. This is because EQ-5D is insensitive towards changes in quality of life (QoL) in mild asthma84,85; it is assumed that all stages of asthma were included in this study because this factor was neither specified nor reported in the baseline characteristics. The likelihood of recruiting mostly mild asthmatics could have explained the non-statistically significant differences in QALYs measured in that study. About 35% of the patients did not complete the trial in Runge et al. As there are not many studies examining this type of management, it may be useful to gain a deeper insight on why the patients chose to withdraw / dropout from the study, in order that a better design or implementation strategy for this type of management could be planned and tested. Nevertheless, the positive outcomes in the real world from this type of management rely on constant internet accessibility. Although the number of worldwide internet users is increasing86, it must be ensured that asthma patients residing in rural areas have equal accessibility to participate in this management. Meanwhile, multilingual countries such as Malaysia may need to incur higher costs for the development of multilingual software support for this type of management. Also, the cost of time spent on utilizing the software for training patients that have low literacy levels should not be forgotten.

The low ICERs for both studies on environmental control management are appealing. Although the costs and outcomes in SFD for both studies are comparable, it should be noted that Sullivan et al. did not include medication costs incurred, unlike in Kattan et al. It is difficult to estimate whether the ICERs would still be comparable when the medication costs were taken into account in Sullivan et al. This is because of the possibility that clinical practices and medication costs changed over time; what was practiced during Sullivan’s study period might no longer be the same as during Kattan’s. Interestingly for both of the studies, there were no significant differences in other outcome measures such as ED visits between the intervention and control groups. Although these findings did not seem to greatly support the GINA fact that the risk of asthma exacerbations is lower when exposure to trigger factors is reduced or avoided1, this merely indicates a lack of reported economic evaluation on this type of management. This is because there are a number of studies that reported successful interventions but are also non-economic evaluations.87 Although Woods et al.54 demonstrated a positive return of investment (ROI) of a similar intervention over a year, it might be appreciated more if the outcomes had been valued using incremental analysis. Perhaps a more relevant issue here is to what extent the healthcare payer is willing to pay for the maintenance of the equipment supplied during the intervention period.

Quality assessment

There may be arguments regarding the choice to use QHES to assess the quality of economic evaluation over many other existing checklists or guidelines. Undoubtedly, this instrument has been criticized for its difficulty to score between a ‘Yes’ and ‘No’ response and for its double-barrelled items issue.21 Though these issues have been tackled in this review by modifying the scoring system, the QHES scores from the quality assessment should be interpreted with caution based on two main reasons. Firstly, the modified scoring system is not validated. Having said that, the modifications made are still compliant with the criterion of each item and relative weights assigned by the original developers, thus it is believed that the validity of the original QHES is still preserved. This belief may be explored through future usage of this modified QHES against other similar assessment tools. Secondly, many of its criteria are closely related to reporting quality in general rather than the study quality in particular.21 Hence, a low QHES score does not necessarily mean that the study has a low quality in its design or methodology.

The use of this modified QHES tool highlights the importance of having an explicit scoring system made accountable for each sub-item in a criterion, in order to have an easier and better comparison of the scores between similar systematic reviews. As an example, a similar review by Campbell et al.14 also used the QHES tool but in its unmodified form. The QHES scores for those studies that were also included here were mostly lower than the scores in this review. The differences in scores are difficult to explain, because Campbell et al.14 did not describe how the double-barrelled questions were dealt with, and hence the scoring system used by them was unknown.

Some of the criteria that more than half of these reviewed studies did not comply with include perspectives, sensitivity analysis, discounting, and incremental analysis. Without a stated perspective, the reasons behind an absence of certain measure of cost or benefit in the analysis could be ambiguous. After all, what constitutes the cost and benefit outcomes measures depends on the analyzed perspective.88 This review also showed that the researchers did not address uncertainties that are inevitably raised in all studies, and thus they did not test the stability of their economic data. The results of a study will not adequately inform a decision if sensitivity analysis is not performed. Around half of the reviewed studies that spanned more than a year did not implement discounting by comparing their cost and benefit outcomes for the same period of time; this omission could have resulted in overestimation of the cost savings and benefits measured.

For a cost-effectiveness or cost-utility study, it is the incremental analysis, and not the individual difference of costs and benefits or the net difference of the cost and benefit ratio, that matters to inform health-related decision making. ICER is meant to determine how much cost is needed in order to gain one extra unit of outcome.16 It is worth noting that the ‘ICER’ (as claimed) calculated by Chan et al (68), based on the difference among average cost-effectiveness ratios could lead to misinterpretation of the outcome. It is also disagreed that Kauppinen et al. did not conduct ICER merely due to statistically insignificant differences of costs and outcomes; they should have considered the uncertainties that present around their data.

Given these flaws in the reviewed studies, it is not surprising that the overall quality as assessed by QHES was of a fair level. However, there is no apparent pattern of the QHES scores despite the fact that criteria such as perspective, incremental analysis, sensitivity analysis, and objectives are common in at least 8 of the 10 reviewed assessment tools from 1992 to 2011.21 This suggests a lack of awareness of a common standard for economic evaluation among researchers during the period.

A quick search on related articles that cited Cooper et al. showed that all of them were modelling-based studies. This review is perhaps the first to adopt this assessment tool for assessing evidence sources in non-modelling based studies. The results from De Asis et al.40 may not be fully applicable to their target population because the evidence taken was from another country where different settings apply.

CONCLUSIONS

The overall quality of economic evaluation studies in literature are fair but used good quality evidence sources for all data components. Despite their limitations, this review showed that the delivered enhanced asthma managements, whether as single or mixed modes, were effective and cost-reducing overall. Among education, self-management, and environmental control, the most cost-effective enhanced management was a mixture of education and self-management by an integrated team of healthcare and allied healthcare professionals. Whilst the availability and accessibility are equally important factors to consider, the sustainability of the cost-effective management must be further investigated using a longer time horizon, especially for chronic diseases such as asthma. In addition, future research on the economic evaluation of asthma management should adhere to both methodological and reporting guidelines of the methods, in order to improve their validity and generalizability to researchers and policymakers alike.