Are complementary therapies and integrative care cost-effective? A systematic review of economic evaluations.

OBJECTIVE: A comprehensive systematic review of economic evaluations of complementary and integrative medicine (CIM) to establish the value of these therapies to health reform efforts. DATA SOURCES: PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE were searched from inception through 2010. In addition, bibliographies of found articles and reviews were searched, and key researchers were contacted. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Studies of CIM were identified using criteria based on those of the Cochrane complementary and alternative medicine group. All studies of CIM reporting economic outcomes were included. STUDY APPRAISAL
METHODS: All recent (and likely most cost-relevant) full economic evaluations published 2001-2010 were subjected to several measures of quality. Detailed results of higher-quality studies are reported.
RESULTS: A total of 338 economic evaluations of CIM were identified, of which 204, covering a wide variety of CIM for different populations, were published 2001-2010. A total of 114 of these were full economic evaluations. And 90% of these articles covered studies of single CIM therapies and only one compared usual care to usual care plus access to multiple licensed CIM practitioners. Of the recent full evaluations, 31 (27%) met five study-quality criteria, and 22 of these also met the minimum criterion for study transferability ('generalisability'). Of the 56 comparisons made in the higher-quality studies, 16 (29%) show a health improvement with cost savings for the CIM therapy versus usual care. Study quality of the cost-utility analyses (CUAs) of CIM was generally comparable to that seen in CUAs across all medicine according to several measures, and the quality of the cost-saving studies was slightly, but not significantly, lower than those showing cost increases (85% vs 88%, p=0.460).
CONCLUSIONS: This comprehensive review identified many CIM economic evaluations missed by previous reviews and emerging evidence of cost-effectiveness and possible cost savings in at least a few clinical populations. Recommendations are made for future studies.

Given the limited nature of previous systematic reviews, what is the extent of evidence on the economic impacts of complementary and integrative medicine (CIM)?

What are the range of therapies and populations studied, and the quality of published economic evaluations of CIM?

What are the results of the higher-quality, more recent (and likely most cost-relevant) economic evaluations of CIM?

This study's comprehensive search strategy identified 338 economic evaluations of CIM, including 114 full evaluations published 2001–2010.

The cost-utility analyses found were of similar or better quality to those published across all medicine.

The higher-quality studies indicate potential cost-effectiveness, and even cost savings across a number of CIM therapies and populations.

The strengths of this study are the comprehensive search strategy, the use of two reviewers, the use of multiple measures of study quality and the identification of higher-quality studies, for which results are reported in detail, via an objective short-list of quality criteria, which reduced the potential for bias.

The weaknesses of this study are similar to those of the other systematic reviews: reviewers were not blinded to journals and article authors, and some aspects of what makes a quality economic evaluation could not be judged from what was reported.

Publication bias was not assessed. However, it is not clear as to whether publication bias is relevant, given the purposes of this review.

Introduction

Between 1990 and 2007, four nationally representative surveys demonstrated that a third or more of US adults routinely used complementary and alternative medicine (CAM) therapies to treat their principal medical conditions.1–4 Total expenditures for CAM therapies were estimated at US$14 billion in 1990,1 US$27 billion in 19972 and US$34 billion in 2007.4 The 2007 US National Health Inventory Survey found that out-of-pocket expenditures for CAM therapies accounted for 11% of all out-of-pocket healthcare expenditures by Americans.4 Similar use numbers are seen in other countries.5–8 However, despite the popularity of and substantial expenditures on CAM therapies, their cost-effectiveness remains ill-defined and controversial.

Economic evaluations allow costs to be included, alongside data on safety and effectiveness, in healthcare policy decisions. As healthcare costs rise, the availability of these economic evaluations becomes increasingly important to the formulation of disease management strategies which are both clinically effective and financially responsible. According to the National Center for Complementary and Alternative Medicine (NCCAM), CAM is ‘a group of diverse medical and healthcare systems, practices and products that are not generally considered part of conventional medicine’.9 In not being part of conventional medicine, individual complementary therapies and emerging models of integrative medicine (ie, coordinated access to both conventional and complementary care)—collectively termed as complementary and integrative medicine (CIM)—are often excluded in financial mechanisms commonly available for conventional medicine,2 and are rarely included in the range of options considered in the formation of healthcare policy. The availability of economic data could improve the consideration and appropriate inclusion of CIM in strategies to lower overall healthcare costs. In addition, economic outcomes are relevant to the licensure and scope of practice of practitioners, industry investment decisions (eg, the business case for integrative medicine), consumers and future research efforts (ie, through identifying decision-critical parameters for additional research10).

A number of systematic reviews of economic evaluations of CIM have been published.11–23 Five of these prior reviews attempted to capture all economic evaluations of CIM therapies across all conditions.11 19–21 23 However, it is unclear as to whether all or even the majority of economic evaluations of CIM have been identified by these reviews. The searches are dated; the search strategy in the most recent review only captured articles published through 2007.23 The databases searched were limited—for example, only one used CINAHL,21 and only two others used EMBASE,19 23 in addition to Medline and AMED. Finally, these reviews generally used limited search terms to identify CIM studies. All but one only used variations on the terms ‘complementary’ or ‘alternative’ ‘medicine’ or ‘therapy’. Unfortunately, other reviewers have found that these search terms do not capture all CIM studies,24 25 which may be a reflection of the difficulty in defining what is and is not CIM.26 The search by Maxion-Bergemann et al11 also added individual therapies as search terms, but only included homeopathy, phytotherapy, traditional Chinese medicine, anthroposophic medicine and neural therapy. No search included ‘integrative medicine’.

The goal of this paper is to identify, to the extent possible, all published economic evaluations of CIM, describe the types of CIM evaluated and the clinical conditions for which they have been evaluated, and identify the recent (and therefore, most cost-relevant) higher-quality studies and highlight their results for policy makers. We also make recommendations for future economic evaluations of CIM.

Methods

Six electronic databases were searched from their inception through December 2010: PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE. To be as comprehensive as possible, a combination of 11 medical subject headings (MeSH) and 39 other search terms were used (box 1). In addition, bibliographies of found articles and reviews were searched, and key researchers in various areas of CIM were contacted for their lists of known studies. Although non-English language articles were collected, they are not analysed in this review.

Search terms used for the PubMed search: (Complementary Therapies (medical subject headings (MeSH)), Dietary Supplements (MeSH), Micronutrients (MeSH), Trace Elements (MeSH), Vitamins (MeSH), acupuncture, alternative medicine, ayurvedic medicine, chiropractic, biofeedback, collaborative medicine, complementary and alternative medicine, botanical medicine, complementary medicine, diet, energy medicine, herbal medicine, herbs, homeopathy, hypnosis, integrated medicine, integrative medicine, massage, meditation, mind-body medicine, minerals, naturopathic medicine, naturopathy, nutrients, nutritional supplements, relaxation, spa therapy, traditional Chinese medicine OR vitamins) AND (Cost-Benefit Analysis (MeSH), Cost Control (MeSH), Cost Savings (MeSH), Costs and Cost Analysis (MeSH), Economics (MeSH), economics (Subheading), Insurance (MeSH), cost benefit, cost effectiveness, cost identification, cost minimisation, cost utility, economic evaluation, insurance claims, managed care OR technology assessment). Searches in the other five databases used the same text words and (where available) analogous controlled vocabulary terms. All searches were restricted to human studies.

Defining a comprehensive search strategy for CIM is challenging.24 27–29 There have been a number of efforts to develop a concise definition of CAM.26 30 This review used the one developed by the members of the Cochrane CAM Field31 and then added the terms ‘integrative’, ‘integrated’ and ‘collaborative’ medicine. The Cochrane CAM definition starts with the NCCAM definition9 and then refines it by specifically including all therapies ‘based upon the theories of a medical system outside the Western allopathic medical model’ (eg, traditional Chinese medicine and Reiki), and including others depending on the context and setting of their use. The context of use considers treatment/condition combinations and excludes those ‘currently considered to be standard treatment’, and the setting of use generally includes self-care and therapies delivered by CIM providers, but excludes therapies ‘delivered exclusively by conventionally credentialed medical personnel or exclusively within hospital settings’. Therefore, therapies such as chemotherapy regimens (eg, chronotherapy32), and therapies requiring surgical implantation (eg, neuroreflexotherapy33) or the placement of a feeding tube34 were not included even though these therapies appeared in our search. In cases where CIM therapies (eg, biofeedback or hypnosis) were included as part of a package of care (eg, with cognitive behavioural therapy), a judgement was made as to whether the CIM portion of the treatment made up half or more of the overall package of care under study. If so, the package of care was included as CIM. Because more than half of CIM users use multiple CIM therapies,35 studies of packages of therapies and coordinated care were identified as such.

Articles were categorised as full economic evaluations if they compared the costs (inputs) and consequences (economic, clinical and/or humanistic outcomes36) of two or more therapeutic alternatives applied to the same patient population (ref.37, p. 11). Otherwise, they were considered partial evaluations, for example, cost-identification or cost-comparison studies.38 Studies that estimated resource utilisation were included as economic evaluations if the utilisation data were detailed enough to allow monetary valuation.

Two reviewers (PMH and BLP) evaluated all articles for inclusion and extracted all data. Disagreements were resolved by discussion between the two review authors, or, if needed, by the other coauthors. Because the results of economic evaluations can rapidly lose relevance with time, mainly due to changes in practice patterns and cost structures, data were extracted only from the economic evaluations published 2001–2010. Extracted data were entered into an Excel template developed for a previous review.20 The type(s) of CIM evaluated and the target population were captured for all economic evaluations. Various indicators of study quality were captured for all full economic evaluations, and more detailed data and results were captured only for those full economic evaluations that met five quality criteria.

The quality of an economic evaluation can be judged along two general dimensions: (1) whether the study was a quality measure of outcomes for its target population and location—that is, whether it was internally valid; and (2) whether enough information is provided for the study's results to be transferable (‘generalisable’).39 Health outcomes are to some extent considered generalisable across settings; however, because resource availability, practice patterns and relative prices can vary greatly, economic outcomes usually are not.40 Therefore, one goal in economic evaluation is to ensure the transferability of study results—that is, to provide enough study detail so that results can be adapted (usually via modelling) to apply to other settings.39 The 35-item BMJ checklist captures components of both dimensions of quality and was applied to all full economic evaluations.41 We also chose five quality criteria by which to identify a subset of full economic evaluations to highlight as being of most interest to policy makers. These quality criteria are based on recommendations made by the US Panel on Cost Effectiveness in Health and Medicine 42 and by well-known experts in the field,37 and focus on the quality of the underlying study (the first type of quality):

Because cost-effectiveness analysis (CEA) is comparative, to ensure that results are useful to decision makers, one of the alternatives to which the CIM intervention was compared must be some version of commonly available (routine, standard or usual) care.

The analysis must explicitly or implicitly use (and include all relevant costs from) at least one recognised perspective—for example, society, third-party payer, hospital or employer.

Since ‘an economic evaluation of a healthcare programme is only as good as the effectiveness data it is built upon’ (ref.43, p. 232), health outcomes must be from randomised controlled trials or non-randomised controlled trials using either statistical adjustment or matching to address baseline differences.

Since having patient-specific data on both costs and outcomes is an advantage for internal validity,44 resource use must be a measured outcome of the study. Modelling studies utilise the results of other published studies, therefore, are exempt from this criterion.

Because uncertainty in an economic evaluation comes not just from sample variation, but also from assumptions made,45 a sensitivity analysis is required.

Because the prices used to value resources are highly location-specific and setting-specific,39 46 we also note, for the articles meeting the above criteria, the presence of a study reporting criterion essential for the transferability of study results (usually via modelling):39 40 separate reporting of unit costs from resource use for economic evaluations alongside trials, or from model parameters (eg, transition probabilities) for economic evaluations using models.

Other data extracted for the economic evaluations which meet the five study-quality criteria are: treatment and study duration, primary clinical and economic outcome measures, the setting in which treatment took place, study design and sample size, the type (table 1) and perspective (ie, the point of view used to define costs) of the economic analysis, and incremental cost-effectiveness of the CIM alternative compared to usual care. Incremental cost-effectiveness is reported in 2011 US$ and is calculated from reported results by first converting the study currency to US$ using the Federal Reserve annual exchange rate47 for the study's currency year and then inflated to 2011 values using the medical care component of the Consumer Price Index.48

Table 1

Types of full economic evaluations

	Cost-benefit analysis	Cost-effectiveness analysis (CEA)	Cost-utility analysis (a special case of CEA)
Unit of health outcome	Monetary units (eg, US$)	Natural units (eg, life-years gained)	Units of overall impact on length and quality of life (eg, QALY)
Results	Net benefits	Incremental cost-effectiveness ratio*	Incremental cost-utility ratio*
	(B1−B2)−(C1−C2−S1+S2)	(C1−C2−S1+S2)/(E1−E2)	(C1−C2−S1+S2)/(QALY1−QALY2)

*Ratios are calculated when both the costs and the effects (health improvements) of one therapy alternative are higher than those of another. When the costs are lower and the effects are better for one therapy, it is said to dominate the alternative (and the alternative is said to be dominated) and no ratio is presented. B1, monetary value of health outcomes of alternative 1; B2, monetary value of health outcomes of alternative 2; C1, total input costs of alternative 1; C2, total input costs of alternative 2; S1, total cost savings (economic outcomes) for alternative 1; S2, total cost savings (economic outcomes) for alternative 2; E1, health effects of alternative 1; E2, health effects of alternative 2; QALY1, quality-adjusted life-years of alternative 1; QALY2, quality-adjusted life-years of alternative 2.

Finally, up to three additional quality measures are included for each of these studies. The Tufts CEA Registry49 quality score is recorded when it was available (note it is only available for cost-utility analyses, CUAs). A Jadad score50 with minor modifications (the two possible points for blinding were replaced with one point for the use of a blinded assessor)51 was calculated for the economic evaluations that included a randomised trial. The percentage of the applicable items from the 35-item BMJ checklist that were met by each article is also reported.41

Results

As shown in figure 1, the database search identified 270 published economic evaluations. An additional 68 articles were added through the bibliography and expert-supplied list search for a total of 338 economic evaluations of CIM. Of these, 204 (60%) were published from 2001 through 2010 (114 full and 90 partial economic evaluations). Of the recent full economic evaluations almost all (103, 90%) examined the effect of one CIM therapy and most of the balance (10, 9%) examined the effect of two or more CIM therapies provided by the same practitioner. Only one looked at the effect of multiple CIM therapies provided by different CIM providers.52 CIM was generally evaluated as an adjunct to usual care.

Figure 1

The flow of records and articles through the systematic review.

As shown in table 2, the 204 economic evaluations published in the past 10 years are spread across a wide range of CIM therapies applied to a number of different study populations. The biggest concentration of full economic evaluations (19 in number) pertained to the use of NCCAM's definition of manipulative and body-based practices (eg, chiropractic, osteopathic manipulation, massage, etc) for the treatment of back pain.53–72 However, even this subgroup is fairly heterogeneous in terms of the therapy (or therapies) tested and/or the type of back pain treated. Eight of these comparisons involved chiropractic care for back pain; one for chronic,53 one for acute57and six for either type.59 60 63 64 67 68 Five evaluated spinal manipulation and manual therapy provided by physiotherapists for chronic back pain (one),65 acute back pain (two)58 69 or either (two).56 68 Four involved osteopathic manipulation; one for chronic71 and one for subacute back pain72 and two for musculoskeletal conditions including back pain.66 68 Three evaluated massage; two for chronic55 62 and one for acute back pain.57 The last two studies evaluated a musculoskeletal physician (treatment ‘with a combination of manual therapy, injections, acupuncture and other pain management techniques’) for orthopaedic referrals;54 and a Finnish folk medicine practice called ‘bone setting’ for the treatment of patients with chronic back pain.61

Table 2

Types of individual complementary and integrative medicine (CIM) therapies studied for various conditions and in various populations: 2001–2010 (reported as the ratio of the total number of economic evaluations to the number of full economic evaluations)

	Manipulative and body-based practices	Acupuncture	Natural products	Other mind-body medicine	Homeopathy	CIM in general	Other CIM therapies*	Totals†
Back pain	28 : 19	11 : 10	2 : 2	–	1 : 1	3 : 0	2 : 2	42 : 29
Rheumatic disorders	9 : 5	6 : 4	6 : 6	2 : 2	–	1 : 0	4 : 3	27 : 19
Mixed populations	4 : 1	6 : 1	2 : 1	3 : 1	9 : 5	2 : 1	3 : 2	24 : 9
Cardiovascular disease and diabetes	–	1 : 0	8 : 6	6 : 4	1 : 1	–	3 : 1	18 : 12
Infection (various)	–	–	6 : 4	–	7 : 4	–	–	13 : 8
Surgery	1 : 1	2 : 2	4 : 3	5 : 4	–	–	–	12 : 10
Members of insurance plans	3 : 0	2 : 0	–	–	1 : 0	7 : 0	–	12 : 0
Mental disorders (various)	–	2 : 2	–	5 : 3	1 : 1	1 : 0	2 : 0	11 : 6
Older populations	–	–	6 : 3	2 : 0	–	–	3 : 1	11 : 4
Headaches	1 : 0	3 : 3	–	4 : 3	1 : 1	–	–	9 : 7
Children (various conditions)	1 : 0	–	–	–	6 : 4	1 : 0	1 : 0	9 : 4
Cancer	2 : 1	2 : 1	1 : 1	2 : 2	–	2 : 0	–	8 : 4
Pregnancy and women's health	–	5 : 5	1 : 0	1 : 0	–	–	–	7 : 5
Allergies	–	1 : 1	–	–	3 : 1	–	1 : 1	5 : 3
Other conditions‡	1 : 1	1 : 1	3 : 3	5 : 4	2 : 1	2 : 0	6 : 2	19 : 11
Totals†	45 : 25	41 : 29	38 : 28	27 : 16	24 : 13	18 : 1	25 : 12	204 : 114

*Other CIM therapies included aromatherapy, healing touch, Tai Chi, Alexander technique, spa therapy, music therapy, electrodermal screening, clinical holistic medicine, naturopathic medicine, anthroposophic medicine, water-only fasting, Ornish Program for Reversing Heart Disease, use of a corset and use of a traditional mental health practitioner.

†Totals across (down) columns will not add to numbers in the totals column (row) due to individual studies addressing more than one CIM therapy (patients in more than one group).

‡Other conditions studied included patients with multiple chemical sensitivities, respiratory disease, pharyngeal dysphagia, dyspepsia, functional bowel disorders, other functional disorders, venous leg ulcers, major burns and constipation; patients who rated themselves as physically ill or having low quality of life; patients in home hospice or with home nursing; long-term care workers and prisoners.

Table 3 shows the results of the application of the 35-item BMJ checklist to the full economic evaluations published 2001–2010.41 On average, the number of applicable items met by each article stayed fairly constant during this period. However, the application of two key items (ie, the proper use of discounting and the inclusion of sensitivity analysis) and the disclosure of funding sources improved significantly, and reporting of the study time horizon worsened significantly. As expected, the average overall and individual-item percentages were higher for the higher-quality articles (those meeting the five study-quality criteria) and for CUAs of CIM. It is not surprising that CUA's quality is higher. They generally involve more effort than other CEAs and are required or recommended by various national guidelines.42 73–75 Nevertheless, it seems as though the quality of CUAs of CIM is generally comparable to, or slightly better than, that seen in CUAs across all medicine, at least in terms of the Tufts quality score, disclosure of funding sources and the five items where comparable data are available.76 77

Table 3

Comparison of various quality measures between economic evaluations of complementary and integrative medicine (CIM) and conventional medicine

	Economic evaluations of CIM					Cost-utility analyses (CUAs) across all medicine†
	All full	2001–2005	2006–2010	Higher quality	CUAs	1998–2001	2002–2005
	n=114	n=59	n=55	n=31	n=27	n=300	n=637
Average percentage met of applicable items on BMJ checklist	72	71	73	87	89
Presented the study perspective clearly (%)	61	58	64	87	93**	74	83**
Presented the study time horizon (%)	96	98*	93*	100	100*	75	87*
Conducted and reported sensitivity analysis (%)	32	22**	44**	100	93**	93	84**
Discounted costs and health effects, where appropriate (%)§	60	25*	76*	94	100*	85	84*
Stated year of currency for resource costs (%)	59	54	60	77	78**	83	85**
Separate reporting of resource use (trials), parameters (models) and unit costs (for transferability)	52	51	53	71	70
Disclosed funding sources (%)	72	58*	76*	84	93*	65‡*
Industry sponsored (%)	10	12	11	10	7	18‡
Average Tufts quality score (CUAs only)					4.75***	4.25‡***

*χ2 Test p value<0.001.

**χ2 Test p value<0.01.

***t Test p value=0.002; comparisons were made between CIM economic evaluations published 2001–2005 and those published 2006–2010, and between CUAs of CIM 2001–2010 and CUAs of all medicine 2002–2005.

†Data from table 3 in Neumann.77

‡Data from table 3 in Neumann et al.76 Industry sponsored was calculated as a percent all studies 1976–2001.

§Denominators for the percentages reported in this row are the number of studies which evaluated impacts past 1 year in either the base case or in sensitivity analyses. For the first five columns the denominators are 25, 8, 17, 16 and 11, respectively. This information was not available for the last two columns.

The number of full evaluations meeting each of the five study-quality criteria are: comparison to usual care 97 (85%), all costs from a recognised perspective 96 (84%), health outcomes from a randomised or matched-control trial 86 (75%), patient-specific data on costs and outcomes 105 (92%) and sensitivity analyses 37 (32%). Sixty-two (54%) of full evaluations met the first four of these and 31 (27%) met all five. A summary of the results of these 31 higher-quality articles (covering 28 different studies) is shown in table 4.54 60 62 68 71 78–103 Twenty-two of these articles (19 of the studies) reported resource use (trials) or model parameters (models) separate from unit prices—a minimum measure of study transferability.54 62 68 71 78 80–85 87–93 95 100 101 103 For those studies which included a randomised trial, the modified Jadad scores ranged from 2 to 4 on a scale from 0 to 4. The Tufts CEA Registry quality scores for the studies containing a CUA ranged from 4 to 6.5 on a scale from 1 to 7. The percentage of the applicable items on the BMJ checklist met by these studies ranged from 66% to 97%.

Table 4

Summary of results of complementary and integrative medicine (CIM) economic evaluations that met five study-quality criteria (31 articles representing 28 studies)

	CIM therapy compared to usual care alone*	Treatment duration/study duration	Patient population	Primary outcome(s)	Setting (information often limited by what was reported)	Sample size	Study design and quality scores†	Resource use (trials), parameters (models), and unit costs (both) reported separately?	Form and perspective of economic evaluation	Incremental cost-effectiveness ratio (2011 US$)‡
Acupuncture studies
Brown et al54	Adjunctive acupuncture, manual therapy, injections and other pain management	Up to 1 year/1 year	Patients referred for an orthopaedic outpatient consultation who were classified as unlikely to require surgery	Clinical: SF-36 and, if appropriate, Aberdeen Low Back Pain Scale or Edinburgh Knee Function Scale; economic: EQ5D	Individualised care from one 'physical medicine’ physician in a hospital outpatient clinic in Scotland	829	R (2) 81%	Yes	CEA-H	Cost saving
							BMJ		CUA-H	Cost saving
van den Berg et al91	Adjunctive breech version acumoxa	2 visits/from 33 weeks to delivery	Pregnant women with breech presentation at 33 weeks	Economic: percentage of breech presentations at delivery—two ‘main analyses’—with and without the option of external cephalic versions	2 instructional visits to an acupuncturist followed by daily home self-care, the Netherlands	NA	Decision tree model	Yes	CEA-P	Cost savings
							81% BMJ		CEA-P	Cost savings
Ratcliffe et al85 and Thomas et al89	Adjunctive acupuncture	3 months/2 years	Patients with low-back pain	Clinical: bodily pain fm SF-36; economic: QALYs fm SF-6D	Up to 10 treatments from a TCM-trained acupuncturist in acupuncture clinic in the UK	239	R (3)	Yes	CUA-S	Cost saving
							Tufts 5		CUA-P	US$8755/QALY
							94%/94% BMJ
Kim et al81	Adjunctive acupuncture	10 treatments in 3-month cycles/5 years	60-year-old women with first time acute low-back pain	Clinical: Roland-Morris Disability, symptom bothersomeness; economic: QALYs fm literature	Hospital-based licensed oriental medical doctors in South Korea	NA	Markov model	Yes	CUA-S	US$3086/QALY
							Tufts 4.5
							94% BMJ
Witt et al97	Adjunctive acupuncture	3 months/6 months	Patients with dysmenorrhoea	Clinical: pain intensity VAS; economic: QALYs fm SF-6D	Up to 15 sessions with a physician trained in acupuncture (A-diploma) in Germany	201	R (3)	No	CUA-S	US$4708/QALY§
							Tufts 5.5
							77% BMJ
Witt et al96	Adjunctive acupuncture	3 months/6 months	Patients with chronic low-back pain	Clinical: Hannover Functional Ability Questionnaire; economic: QALYs fm SF-6D	Up to 15 sessions with a physician trained in acupuncture (A-diploma) in Germany	2518	R (3)	No	CUA-S	US$16230/QALY§
							Tufts 4.5
							73% BMJ
Witt et al99	Adjunctive acupuncture	Up to 15 treatments/3 months	Patients with headache	Economic: QALYs fm SF-6D	10–15 sessions with physician trained in acupuncture (A-diploma) in Germany	3182	R (2)	No	CUA-S	US$18225/QALY§
							Tufts 5.5
							88% BMJ
Willich et al94	Adjunctive acupuncture	Up to 15 treatments/3 months	Patients with chronic neck pain	Clinical: Neck Pain and Disability Scale; economic: QALYs fm SF-6D	10–15 sessions with physician trained in acupuncture (A-diploma) in Germany	3451	R (2)	No	CUA-S	US$19226/QALY§
							Tufts 5
							88% BMJ
Wonderling et al100 and Vickers et al93	Adjunctive acupuncture	3 months/1 year	Patients with chronic headache	Clinical: headache severity score; economic: QALYs fm SF-6D	Acupuncture-trained physiotherapists in own clinics in the UK	401	R (3)	Yes	CUA-S	US$19785/QALY
							Tufts 5		CUA-P	US$21074/QALY
							97%/93% BMJ
Reinhold et al86	Adjunctive acupuncture	3 months/3 months	Patients with chronic hip or knee osteoarthritis	Economic: QALYs fm SF-6D	10–15 sessions with physician trained in acupuncture (A-diploma), Germany	489	R (3)	No	CUA-S	US$27900/QALY§
							Tufts 4
							87% BMJ
Witt et al98	Adjunctive acupuncture	Up to 15 treatments/3 months	Patients with allergic rhinitis	Economic: QALYs fm SF-6D	10–15 sessions with physician trained in acupuncture (A-diploma) in Germany	981	R (3)	No	CUA-S	US$28137/QALY§
							Tufts 4
							94% BMJ
Manipulative and body-based practices—see also Brown et al
Korthals-de Bos et al82	Manual therapy	6 weeks/1 year	Patients with neck pain	Clinical: perceived recovery, pain VAS, and Neck Disability Index; economic: All clinical plus QALYs fm EQ-5D	Up to 6 weekly 45 min sessions with a physiotherapist who is also a registered manual therapist in the Netherlands	183	R (3)	Yes	CEA-S	Cost saving
							Tufts 6.5		CEA-S	Cost saving
							83% BMJ		CEA-S	Cost saving
									CUA-S	Cost saving
Williams et al71	Adjunctive osteopathic spinal manipulation	2 months/6 months	Patients with subacute (2–12 week) back pain	Clinical: Extended Aberdeen Spine Pain Scale; economic: QALYs fm EQ-5D	3 or 4 sessions with a general practitioner who is a registered osteopath at own clinic in UK	187	R (3)	Yes	CUA-P	US$8730/QALY
							Tufts 5
							89% BMJ
UK BEAM Trial Team68	Adjunctive spinal manipulation and exercise	3 months/1 year	Patients with low-back pain	Economic: QALYs fm EQ-5D	8 sessions with a chiropractor, osteopath, or physiotherapist at a private or NHS site in the UK	1287	R (3)	Yes	CUA-P	US$8425/QALY
	Adjunctive spinal manipulation						Tufts 6		CUA-P	US$10642/QALY
							93% BMJ
Hollinghurst et al62	Alexander technique	6 lessons/1 year	Patients with chronic or recurrent non-specific back pain	Clinical: Roland-Morris Disability Questionnaire (RMDQ); economic: above plus QALYs fm EQ-5D	Alexander technique teachers and massage therapists at own locations in the UK	579	R (3)	Yes	CUA-P	US$13300/QALY
									CEA-P	US$255/RMDQ pt
	Alexander technique plus exercise¶	6 lessons/1 year					Tufts 5.5		CUA-P	US$12022/QALY
									CEA-P	US$144/RMDQ pt
	Massage	6 sessions/1 year					97% BMJ		CUA-P	Dominated
									CEA-P	US$1010/RMDQ pt
	Massage plus exercise¶	6 sessions/1 year							CUA-P	US$11959/QALY
									CEA-P	US$354/RMDQ pt
Haas et al60	Treatment in a chiropractic clinic	Unspecified/1 year	Patients with acute low-back pain	Clinical and economic: pain severity 100 mm VAS and revised Oswestry Disability Questionnaire	Doctors of Chiropractic in own clinics in Oregon, the USA	1943	MC	No	CEA-P	US$21/pain mm
			Patients with chronic low-back pain			837	66% BMJ		CEA-P	US$0.73/pain mm
Natural products
Braga et al102	Adjunctive preoperative arginine and ω-3 fatty acid supplementation	5 days/5 days plus hospital stay	Patients with gastrointestinal cancer undergoing surgery	Economic: percentage of patients without complications	12.5 g arginine, 3.3 g ω-3 fatty acids and 1.2 g RNA in liquid daily taken orally for 5 days before surgery, Italy	204	R (3)	No	CEA-H	Cost saving
							88% BMJ
Stevenson et al103 and Stevenson et al88	Vitamin K1	10 years/10 years	Postmenopausal women with osteoporosis/osteopenia	Clinical: osteoporotic fracture; economic: QALYs fm the literature	10 mg/day of vitamin K1 daily, the UK	NA	Patient-level simulation model	Yes	CUA-P	Cost saving
							Tufts 4.5
							81%/84% BMJ
Trevithick et al90	Adjunctive antioxidants (vitamins C and E and β-carotene)	25 years/25 years	Cohort of Ontario residents aged 50–54 (prevention of cataracts)	Clinical: cataract formation	750 mg/day vitamin C, 600 mg/day vitamin E and 18 mg/day β-carotene daily, Canada	NA	Markov-type cohort model	Yes	CEA-P	Cost saving
							79% BMJ
Schmier et al87	Adjunctive ω-3 fatty acid supplementation	42 months/42 months	Males with a history of a heart attack	Economic: fatal MIs and cardiovascular deaths	‘Fish oil pills', the USA	NA	Decision analytic model	Yes	CEA-S	Cost saving
							77% BMJ		CEA-P	US$11903/fatal MI avoided
Lamotte et al83	Adjunctive ω-3 polyunsaturated fatty acids	3.5 years/lifetime	Patients after an acute myocardial infarction	Economic: life-years saved	∼465 mg EPA and ∼385 mg DHA ethyl esters in a daily gelcap, Australia, Belgium, Canada, Germany and Poland	NA	Decision tree model	Yes	CEA –P	US$5413/LYG Australia
							89% BMJ		CEA –P	US$8184/LYG Belgium
									CEA –P	US$4476/LYG Canada
									CEA –P	US$6750/LYG Germany
									CEA –P	US$7747/LYG Poland
Quilici et al84	Adjunctive ω-3 polyunsaturated fatty acids	4 years/lifetime	Patients after an acute myocardial infarction	Economic: life-years gained (LYG), QALYs fm the literature, deaths avoided	∼465 mg EPA and ∼385 mg DHA ethyl esters in a daily gelcap, the UK	NA	Markov model	Yes	CEA –P	US$28420/LYG
							Tufts 5		CUA-P	US$35940/QALY
							93% BMJ
Franzosi et al79	Adjunctive ω-3 polyunsaturated fatty acids	3.5 years/3.5 years	Patients with recent myocardial infarction	Clinical: death and non-fatal MI or stroke; economic: LYG	∼465 mg EPA and ∼385 mg DHA ethyl esters in a daily gelcap, Italy	5664	R (4)	No	CEA-P	US$41867/LYG
							85% BMJ
Black et al78	Adjunctive glucosamine sulphate	22.6 years/22.6 years	Patients with osteoarthritis of the knee	Clinical: pain, function, joint space loss; economic: QALYs fm the literature	Glucosamine sulphate powder 1500 mg daily in oral solution, the UK	NA	Cohort simulation model	Yes	CUA-P	US$59053/QALY
							84% BMJ
Other complementary and integrative medicine therapies
Wilson and Datta95	Adjunctive yang-style tai chi	1 year/1 year	Nursing home residents at average risk for a fall	Economic: hip fractures avoided	2 classes/week monitored by a certified tai chi instructor and an assistant, the USA	NA	Decision tree model	Yes	CEA-P	Cost saving
							96% BMJ
Herman et al80	Adjunctive naturopathic care including acupuncture, relaxation exercises, dietary and exercise advice	3 months/6 months	Patients with chronic low-back pain	Clinical: Oswestry Disability Questionnaire; economic: QALYs fm SF-6D	Twice weekly visits to licensed naturopathic doctors also trained in acupuncture in a worksite clinic in Canada	70	R (3)	Yes	CUA-S	Cost saving
							Tufts 5		CEA-E	US$191/absentee day avoided
							96% BMJ		CBA-E	Cost saving
Van Tubergen et al92	Combined spa-exercise therapy	3 weeks/40 weeks	Patients with ankylosing spondylitis	Clinical: Bath Ankylosing Spondylitis Functional Index (BASFI 10pts), pain VAS, well-being VAS and morning stiffness in minutes; economic: above plus QALYs fm EQ-5D	3-week stay at one of two spa-resorts with therapy provided by trained physiotherapists, the Netherlands	120	R (3)	Yes	CEA-S	US$2159/BASFI pt (spa in Austria)
							Tufts 4.5		CEA-S	US$4215/BASFI pt (spa in the Netherlands)
							90% BMJ		CUA-S	US$12703/QALY (spa in Austria)
									CUA-S	US$31609/QALY (spa in the Netherlands)
Zijlstra et al101	Adjunctive spa therapy	2.5 weeks/1 year	Patients with fibromyalgia	Economic: QALYs fm VAS and SF-6D	18-day stay at a spa in Tunisia with a variety of treatments, the Netherlands	128	R (3)	Yes	CUA-S	US$46443/QALY (VAS)
							Tufts 4		CUA-S	US$92886/QALY (SF-6D)
							97% BMJ

*The use of the term ‘adjunctive’ in this column indicates complementary and alternative medicine (CAM) therapies used in addition to usual care for that condition unless otherwise indicated.

†Study design: R, randomised; MC, matched controls and/or results statistically adjusted for baseline differences. A modified Jadad score (maximum score = 4) is provided if the study was randomised. If the study was a CUA and a quality score was available from the Tufts Medical Center Institute for Clinical Research and Health Policy Studies CEA Registry (https://research.tufts-nemc.org/cear/Default.aspx), it is reported. Quality scores range from 1 to 7 with 7 representing the highest quality. The last number is the percent of the applicable items on the BMJ 35-item quality checklist that this study met. If a study had more than one publication, both percentages were reported. The BMJ checklist is found in Drummond et al.41

‡The costs reported in each study were first converted to US$ using the Federal Reserve annual exchange rate (http://www.federalreserve.gov/releases/g5a/20090102/, accessed 30 Jan 2012) for the study's currency year and then inflated to 2011 values using the medical care component of the Consumer Price Index (http://www.bls.gov/cpi/cpi_dr.htm#2007, accessed 30 Jan 2012). In comparisons labelled as cost saving the CIM therapy both improved health and lowered costs compared to usual care. In the comparison labelled dominated the CIM therapy had worse health outcomes and higher costs than usual care.

§These studies did not report a currency year so it was estimated as being 1 year prior to publication.

¶Compared to usual care plus exercise.

CBA, cost-benefit analysis; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; DHA, Docosahexaenoic acid; E, employer perspective; EPA, Eicosapentaenoic acid; H, hospital perspective; MI, myocardial infarction; P, payer perspective; QALY, quality-adjusted life-year; S, societal perspective; VAS, visual analogue scale.

Of the 56 comparisons made in these studies, 16 (29%) are cost saving—that is, the added CIM therapy had better health outcomes and lower costs than usual care alone. Cost savings were seen for acupuncture alone (instructional visits with an acupuncturist followed by home self-care by the partner for pregnant women with breech presentations at 33 weeks in terms of reductions in both breech presentation at birth and ceasareans in the Netherlands,91 and treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain in terms of quality-adjusted life-years or QALYs from the societal perspective85) and in combination with other therapies (along with manual therapy, injections and other pain management for patients referred to an orthopaedic surgeon's office in Scotland who were unlikely to need surgery in terms of both improvements in health-related quality of life and QALYs54). Cost savings were also seen for manual therapy delivered by a physiotherapist, who is also a registered manual therapist, for neck pain in terms of perceived recovery, pain, neck disability and QALYs82; for preoperative oral supplementation with arginine and ω-3 fatty acids for patients with gastrointestinal cancer undergoing surgery102; for vitamin K1 supplementation for postmenopausal women with osteopenia and osteoporosis in terms of QALYs103; for supplementation with vitamins C and E and β-carotene for cataract prevention90; for fish oil supplementation in men with a history of heart attack87; for tai chi to prevent hip fractures in nursing home residents95 and for naturopathic care offered through a worksite clinic for chronic low-back pain in terms of both reductions in absenteeism and gains in QALYs.80

Of the 28 cost-utility comparisons, one (massage for low-back pain62) was dominated— that is, had worse health outcomes and higher costs than usual care. Five (18%) are cost saving,54 80 82 85 103 5 (18%) have incremental cost-effectiveness ratios (ICERs) between US$0 and US$10 000 per quality-adjusted life-year (QALY),68 71 81 85 97 and 89% had ICERs less than US$50 000/QALY, a threshold often considered to represent the upper limit of society's value for a QALY.104 The cost-saving cost-utility studies were included in the paragraph above (ie, those that mention QALYs). The studies with cost-utility ICERs between US$0 and US$10 000 per QALY were: treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain.85 hospital-based acupuncture by licensed oriental medical doctors in South Korea for 60-year-old women with first-time acute low-back pain,81 acupuncture from physicians with at least 140 h of training (A-diploma) in Germany for patients with dysmenorrhoea,97 osteopathic spinal manipulation by a general practitioner who is a registered osteopath in the UK for patients with subacute back pain,71 and an exercise programme plus spinal manipulation from a chiropractor, osteopath or physiotherapist at a private or National Health Service (NHS) site in the UK for low-back pain.68 The average percentage of applicable BMJ checklist items met by each study was slightly lower for those studies with at least one cost-saving comparison (85% vs 88%), but the difference was not statistically significant (t test=0.75, p value=0.460).

Discussion

This comprehensive systematic review identified 338 economic evaluations of CIM; 204 of which were published recently (2001–2010) covering a wide range of CIM therapies for a variety of populations. Although most patients who use CIM use more than one modality35 and despite the attention given to integrative medicine (coordinated access to conventional medicine and CIM),105 this systematic review found only one study that examined the effects of use of multiple CIM practitioners.52 In general, the quality of the recent full economic evaluations has held constant and is in line with what is seen in economic evaluations in conventional medicine. Details of the 31 recent higher-quality full economic evaluations indicate potential cost-effectiveness and cost savings across a variety of CIM therapies applied to different conditions. Owing to the non-generalisable nature of economic evaluations, the cost estimates shown are specific to their study settings.40 However, 22 articles provided at least the minimum information for study transferability. Therefore, their results could be adapted via modelling to determine the economic impact of these interventions in other settings.

The strengths of this study are the comprehensive search strategy, which revealed a substantial number of published economic evaluations of CIM, the use of two reviewers and the use of multiple measures of study quality. Higher-quality studies were identified and highlighted for policy makers using a simple objective list of quality criteria, which reduced the potential for bias. The weaknesses of this study are similar to those of the other systematic reviews. The reviewers were not blinded to journals and article authors, which may have influenced results. Also, some aspects of what makes a quality economic evaluation could not be judged from what was reported. For example, ideally, pragmatic trials enrol patients typical of normal caseload in typical settings with typically trained and experienced practitioners following them under routine conditions (ref.37, p. 251). Judgements as to whether these criteria were met were not always possible from the reports, and were beyond the scope of this review. Finally, publication bias was not assessed. However, since the major goal of this study was to establish the extent of the published literature on this topic and to highlight the results of the higher-quality studies, it is not clear that publication bias is relevant here.

The number of economic evaluations of CIM found and reviewed by this study far exceeds the numbers found in previous studies.11 19–21 23 This study found a total of 338 economic evaluations of CIM published between and including 1979 and 2010; 211 of these were full economic evaluations. White and Ernst19 identified 34 economic evaluations of CAM published 1987–1999; 11 of which were full economic evaluations. Between 1999 and October 2004, Herman et al20 identified 56 economic evaluations of CAM (39 full evaluations). Between 1994 and May 2004 Hulme and Long21 identified 19 full economic evaluations of CAM, and over a similar period (1995–2007) Doran et al23 found 43 economic evaluations (15 full evaluations). Maxion-Bergemann et al11 identified 5 (1 full) economic evaluations over an unspecified search period. The large number of economic evaluations found in this study reflects the facts that: (1) all evaluations from previous reviews were included; (2) a number of studies have been published since the last search dates of prior reviews and (3) a more extensive search strategy was used. It should be noted that 20% of the articles (68 of 338) in this review were identified through bibliography searches and from expert lists. Therefore, even the application of a long list of search terms to multiple databases does not guarantee that all CIM studies will be identified. However, there is some evidence that the indexing of these articles in medical databases is improving; studies from bibliographies and expert lists made up 32% of found articles published 2000 and before, but only 12% recent articles.

There are several implications of this study for policy makers, clinicians and future researchers. First, there is a large and growing literature of quality economic evaluations in CIM. However, although indexing is improving in databases, finding these studies can require going beyond simple CIM-related search terms. Second, the results of the higher-quality studies indicate a number of highly cost-effective, and even cost saving, CIM therapies. Almost 30% of the 56 cost-effectiveness, cost-utility and cost-benefit comparisons shown in table 4 (18% of the CUA comparisons) were cost saving. Compare this to 9% of 1433 CUA comparisons found to be cost saving in a large review of economic evaluations across all medicine.106 Third, by meeting the five study-quality criteria, the studies shown in table 4 can each be considered a reasonable indicator of the health and economic impacts of the CIM therapy studied, at least in that population and setting. These studies, especially those showing cost savings, should be considered further for applicability in other settings. This requires the study to be transferable.39 Fortunately, the majority of the higher-quality studies met our measure of study transferability—resource use or model parameters, and unit costs were reported separately.

Given the substantial number of economic evaluations of CIM found in this comprehensive review, even though it can always be said that more studies are needed, what is actually needed are better-quality studies—both in terms of better study quality (to increase the validity of the results for its targeted population and setting) and better transferability (to increase the usefulness of these results to other decision makers in other settings). Therefore, the following recommendations are made.

That all studies measuring the effectiveness of CIM at least consider also measuring input costs and economic outcomes.

That at least one arm of the study be some version of commonly available (usual) care, and that usual care and all interventions studied be described in sufficient detail that decision makers in other settings can determine what was done and whether the study's usual care comparator is applicable in their setting.

That consideration be given to how CIM is typically used (eg, multiple CIM therapies) or can be used (eg, coordinated integrative care models) when designing studies.

That changes in resource use be reported separately from unit costs in economic evaluations alongside clinical trials and that model parameters and unit costs be clearly reported in decision-analytic modelling studies.

That all economic evaluations contain sensitivity analyses to increase the reliability of results.

That more consideration be given to modelling as a method to estimate economic outcomes for existing effectiveness trial results, and to generalise existing quality economic evaluation results to other jurisdictions.