Value within otolaryngology: Assessment of the cost-utility analysis literature.

OBJECTIVE: To assess the characteristics and quality of cost utility analyses (CUA) related to otolaryngology within the CEA registry and to summarize their collective results.
METHODS: All cost-utility analyses published between 1976 and 2011 contained in the Cost-Effectiveness Analysis Registry (CEA Registry) were evaluated. Topics that fall within the care of an otolaryngologist were included in the review regardless of the presence of an otolaryngologist author. Potential associations between various study characteristics and CEA registry quality scores were evaluated using the Pearson product moment correlation coefficient.
RESULTS: Sixty-one of 2913 (2.1%) total CUA publications screened were related to otolaryngology. Eighteen of 61 (29.5%) publications included an otolaryngologist as an author. Fourteen studies agreed on the cost effectiveness of at least unilateral cochlear implantation and six of seven (85.7%) studies demonstrated the cost effectiveness of continuous positive airway pressure (CPAP) for obstructive sleep apnea (OSA). Forty-six percent (28 of 61) of all manuscripts were published between 2008 and 2011. A more recent publication year was associated with a higher CEA registry quality score while the presence of an otolaryngologist author and journal impact factor had no significant correlation with the quality of the CUA.
CONCLUSION: Based on current evidence in the CEA registry, unilateral cochlear implantation for hearing loss and CPAP for OSA are both cost-effective therapeutic interventions. Although CUAs in otolaryngology have increased in quantity and improved in quality in more recent years, there is a relative lack of CUAs in otolaryngology in comparison to other subspecialties.

Introduction

The cost of medicine in the United States and abroad is increasing at an exponential and economically unsustainable rate. Technological advances leading to more expensive diagnostic and therapeutic tools have contributed to this increase, which in turn has led to rising pressure to demonstrate the value of such interventions. This has ultimately led to growing governmental, professional, organizational, and academic interests in the value propositions in the healthcare system. Cost-effectiveness analysis (CEA) is the primary modality by which investigators assess the value of an intervention. CEA evaluates the price of an intervention, either to the payer or to society, for an individual measure of effectiveness of that intervention. This can include the years of life added and quality of life added, among others.

A subset of cost-effectiveness analysis is cost-utility analysis (CUAs), which expresses the effectiveness of an intervention using a uniform unit of cost per quality adjusted life year (QALY). The QALY describes the time spent in a certain health state, multiplied by the quality of each state (with 1 QALY being perfectly healthy for one year). In this way, both a treatment that improves health related quality of life from 0.5 to 1.0 for 5 years and a treatment that leads to 5 additional years of life with a health condition of 0.5 both yield 2.5 QALYs. One general way to look at value is to assess the cost of an intervention and to correlate this with the benefits rendered, either in life gained or in quality of life improved. In cost-utility analysis, interventions are considered of favorable value if their cost is less than $50,000 (USD) per QALY gained. As the cost per QALY decreases, the intervention becomes more cost effective. When comparing two interventions with the same intended goal, the intervention with the lower cost per QALY is the more economic choice.

Within all aspects of medical literature, there are an increasing number of studies evaluating cost utility. However, this is challenging within subspecialties such as otolaryngology due to a limited number of investigators and conditions compared to other specialties. Nevertheless, because otolaryngology utilizes costly diagnostic and therapeutic strategies for managing conditions such as head and neck cancer, hearing loss, and chronic sinusitis, it provides a fertile landscape for the assessment of cost effectiveness. The objectives of this study are to detail specific characteristics of CUAs within otolaryngology, to evaluate the quality of these studies and to summarize the collective results of the most common topics of economic evaluations in otolaryngology.

Methods

We performed a quantitative and qualitative assessment of studies within the spectrum of otolaryngology between 1976 and 2011 using the CEA registry. The CEA registry is a database updated three times per year with publically available data on all publications that are published in English, are original cost-effectiveness analyses, and measure health benefits of QALYs. The CEA registry is supported by the Center for the Evaluation of Value and Risk in Health (CEVR) and is part of the Institute for Clinical Research and Health Policy Studies at Tufts Medical Center. The total number of studies available in the registry at the time of analysis was 2913. The registry's rigorous methodology for screening cost utility analysis manuscripts is described on the website. In short, a MEDLINE search is performed with the keywords, “QALYs” “quality,” and “cost-utility analysis.” The CEA registry team screens abstracts to assess if there is an original cost-utility estimate. Each article is then abstracted for methodology, cost-effectiveness ratios, and utility weights. Two trained readers audit each article independently and a consensus audit resolves discrepancies.

Author MAC systematically reviewed all of the articles within the CEA registry and screened for publications that fall within the field of otolaryngology, which includes head and neck surgery, endocrine surgery, otology, pediatric otolaryngology, rhinology, allergy and sleep medicine. The presence of an otolaryngologist author, as determined per affiliations listed on the manuscript, was not a factor in inclusion. In the unique case that affiliations were not named, an Internet search was conducted. Study characteristics, including year of publication, journal of publication, author affiliation, country of research, type of funding, analysis perspective, intervention type, and CEA registry quality score (numbered from 1 (low) to 7 (high) by expert readers). The criteria used to determine the CEA registry quality score of each study includes: 1. accurate computation of incremental cost-effectiveness ratios, 2. comprehensive characterization of the uncertainty of results, 3. explicit specification of health economic assumptions used in the study, and 4. appropriate and explicit estimation of utility weights (Table 1). The strength and direction of association between characteristics of each study and CEA registry quality scores were measured using the Pearson product moment correlation coefficient using the Statistical Package for Social Sciences (SPSS Version 22.0, Chicago, IL). The collective results of CUAs of the most commonly evaluated interventions were also assessed in order to identify economically attractive management options within otolaryngology.

Table 1

CEA registry quality score criteria, adapted from the Tufts CEA Registry.

CEA registry quality score criteria (in order of importance)
1 Did the study authors correctly compute the incremental cost-effectiveness ratios? 2 Did the authors comprehensively characterize the uncertainty of the results? 3 Were the health economic assumptions used in the study (discount rate, currency, time horizon) explicitly specified? 4 Was there an appropriate and explicit estimation of utility weights?

Results

Assessment of 2913 studies revealed 61CUAs that evaluated interventions related to otolaryngology. The earliest study was published in 1991 and assessed the cost-effectiveness of tympanostomy tubes versus antibiotic prophylaxis for acute otitis media (AOM). Eighty-five percent (52 of 61) of studies were published later than 2000, with 28 (45.9%) published between 2008 and 2011 (Fig. 1). The 61 publications addressed topics within the subspecialties of otology (31.1%), endocrine surgery (19.6%), sleep medicine/surgery (18.0%), head and neck surgery (13.0%), pediatric otolaryngology (8.2%), allergy (6.6%), and rhinology (3.3%) (Fig. 2). Of the 61 manuscripts related to otolaryngology, 18 (29.5%) studies had at least one author who was an otolaryngologist. Eight (13.0%) manuscripts had a first author and seven (11.4%) had a final author primarily affiliated with a department of otolaryngology. Seventy-two percent of publications with an otolaryngologist as a first author were related to otology.

Fig. 1

CUA publications per time period.

Fig. 2

Publication subspecialties.

The otolaryngology CUAs were published in 41 journals, with only five journals having three or more manuscripts (Table 2). Thirty-two (52.4%) economic analyses had the United States as the country of interest. Eight studies focused on the United Kingdom and five on Canada. Ninety-eight percent of manuscripts had at least one author with an academic affiliation. Seventy-one percent of analyses had the perspective of healthcare payer. The funding sources of the manuscripts were stated as none or could not be determined in 29 (47.5%), government funding in 17 (27.9%), pharmaceutical or device in 14 (22.9%), foundation in 6 (9.8%) and healthcare organization in 3 (4.9%) (Table 2). Two studies (3.3%) evaluated primary prevention strategies, which are defined as efforts to prevent disease prior to its occurrence. One of these primary prevention strategies analyzed oral cancer screening programs for high-risk males and the second evaluated candidate vaccines for prevention of pediatric acute otitis media. Thirteen studies assessed secondary prevention interventions (methods that identify and treat asymptomatic individuals with risk factors or preclinical disease) and 46 (75.4%) evaluated tertiary prevention interventions (methods that limit disability after harm has occurred). A majority of the studies evaluated the cost utility of devices (47.5%) or pharmaceuticals (29.5%) (Table 2).

Table 2

Demographic characteristics of CUAs.

Study characteristic	No. Studies (%)	Mean CEA registry quality score (range)
Subspecialty
Otology	19 (31.1)	3.8 (1.5–6.0)
Endocrine surgery	12 (19.7)	4.2 (3.5–5.5)
Sleep medicine/surgery	11 (18.0)	4.2 (2.5–6.0)
Head and neck surgery	8 (13.0)	3.4 (2.5–5.0)
Pediatric otolaryngology	5 (8.2)	4.3 (3.0–5.5)
Allergy	4 (6.6)	4.5 (3.5–6.0)
Rhinology	2 (3.3)	4.5 (4.0–5.0)
Journal (2011 impact factor)
Laryngoscope (1.752)	6 (9.8)	3.3 (1.5–4.5)
Arch of Otolaryngology Head Neck Surg (1.63)	4 (6.6)	4.1 (1.5–5.0)
Ear Hearing (2.578)	3 (4.9)	4.5 (3.0–6.0)
Sleep (5.051)	3 (4.9)	4.2 (2.5–5.0)
Value Health (2.191)	3 (4.9)	5.0 (4.0–6.0)
Others	36 (59)	4.1 (1.5–6.0)
Total No. of Journals	41	4.0 (1.5–6.0)
Year of publication
1976–1991	1 (1.6)	3.0
1992–1995	4 (6.6)	2.1 (1.5–3.0)
1996–1999	4 (6.6)	3.9 (2–5.5)
2000–2003	10 (16.4)	3.9 (1.5–5.5)
2004–2007	14 (22.9)	4.0 (3.0–6.0)
2008–2011	28 (45.9)	4.4 (2.5–6.0)
Country of analysis
United States	32 (52.4)	3.9 (1.5–6.0)
United Kingdom	8 (13.1)	4.7 (1.5–6.0)
Canada	5 (8.2)	4.1 (2.5–5.5)
Australia	3 (4.9)	3.3 (3.0–4.0)
Germany	3 (4.9)	4.2 (4.0–4.5)
France	2 (3.3)	4.5 (3.5–5.5)
Netherlands	2 (3.3)	3.5 (2.0–5.0)
New Zealand	1 (1.6)	4.0
Austria	1 (1.6)	3.5
China	1 (1.6)	3.0
Finland	1 (1.6)	3.0
Belgium	1 (1.6)	5.0
Taiwan	1 (1.6)	5.0
Funding source
Government	17 (27.9)	4.1 (1.5–6.0)
Pharmaceutical or device	14 (22.9)	3.8 (1.5–5.5)
Could not be determined	29 (47.5)	3.9 (1.5–6.0)
Foundation	6 (9.8)	4.3 (3.0–5.5)
Healthcare organization	3 (4.9)	3.5 (1.5–4.5)
Perspective of study
Healthcare payer	43 (70.5)	3.9 (1.5–6.0)
Societal	17 (27.9)	4.4 (3.0–5.5)
Could not be determined	1 (1.6)	1.5
Intervention type
Primary	2 (3.3)	4.8 (4.0–5.5)
Secondary	13 (21.3)	3.8 (2.0–6.0)
Tertiary	46 (75.4)	4.0 (1.5–6.0)
Authors affiliation
Academic	60 (98.4)	4.0 (1.5–6.0)
Consultant	7 (11.5)	4.4 (3.5–5.0)
Government	1 (1.6)	3.0
Intervention assessed
Device	29 (47.5)	3.8 (1.5–6.0)
Diagnostic	6 (9.8)	4.0 (3.0–5.0)
Screening	6 (9.8)	3.4 (2.0–5.0)
Health education	1 (1.6)	4.0
Medical procedure	9 (14.7)	4.1 (3.0–5.0)
Pharmaceutical	18 (29.5)	4.6 (3.5–6.0)
Surgical	12 (19.7)	3.9 (2.5–5.5)
Immunization	2 (3.3)	5.0 (4.5–5.5)
Care delivery	2 (3.3)	3.3 (3.0–3.5)

The mean CEA registry quality score (on a scale of 1–7) for all 61 studies was 4.00. A more recent publication year was associated with a higher CEA registry quality score (r = 0.412, P < 0.01) (Table 3). The mean quality score for studies with at least one otolaryngology author was 3.77 and 4.09 for those without an otolaryngology author. There was no significant correlation between the number of otolaryngologist authors and the CEA quality score. The impact factor of the journal in which each study was published also had no significant association with the quality of the CUA. The references and topics for all 61 studies that relate to otolaryngology in the CEA database are listed in Table 4. Fourteen studies agreed on the cost-effectiveness of at least unilateral cochlear implantation and 6 of 7 studies demonstrated CPAP to be a cost-effective strategy for treating patients with OSA.

Table 3

Correlations between CUA characteristics and CEA registry quality score.

Variable	Correlation coefficient, r	P value
# of Otolaryngology authors	−0.043	0.749
Publication year	0.412	0.001a
Journal impact factor	0.184	0.160

Statistically significant.

Table 4

Summary of evaluated CUAs.

Year	Specialty	Journal	Reference	Topic of CUA publication
1991	General	Fam Pract Res J	Bisonni et al9	Tympanostomy tubes vs antibiotic prophylaxis for AOM
1994	Sleep	Sleep	Tousignant et al10	Impact of nasal CPAP on quality of life for OSA
1995	Otology	Ann Otol Rhinol Laryngol Suppl	Evans et al11	Adult unilateral cochlear implant
1995	Otology	Med Prog Technol	Lea et al12	Cochlear implantation vs vibrotactile devices
1995	Otology	Arch Otolaryngol Head Neck Surg	Harris et al13	Cochlear implantation for profound deafness
1996	Otology	Laryngoscope	Wyatt et al14	Multichannel cochlear implants
1996	Pediatrics	Clin Ther	Oh et al15	Second-line antibiotics for pediatric AOM
1999	Otology	Int J Technol Assess Health Care	Carter et al16	Pediatric and adult cochlear implantation
1999	Otology	Arch Otolaryngol Head Neck Surg	Palmer et al17	Adult cochlear implantation
2000	Otology	Laryngoscope	O'Neill et al18	Pediatric cochlear implantation
2000	Otology	JAMA	Cheng et al19	Pediatric cochlear implantation
2001	Head and neck	Cancer	Hollenbeak et al20	FDG-PET for N0 HNSCC
2002	Endocrine	Eur J Endocrinol	Vidal-Trecan et al21	Management of toxic thyroid adenomas
2002	Otology	Otol Neurotol	Bichey et al22	Cochlear implantation for large vestibular aqueduct syndrome
2002	Otology	Laryngoscope	Francis et al23	Cochlear implantation in older adults
2002	Head and neck	Community Dent Oral Epidemiol	Van der Meij et al24	Cancer screening of patients with oral lichen planus
2002	Otology	Arch Otolaryngol Head Neck Surg	Summerfield et al25	Unilateral vs bilateral cochlear implantation
2003	Otology	Arch Otolaryngol Head Neck Surg	Joore et al26	Fitting of hearing aids
2003	Otology	Laryngoscope	Wilson et al27	Intraoperative facial nerve monitoring for otologic surgery
2004	Otology	Ear Hear	Group, UKCIS et al28	Unilateral cochlear implantation in postlingually deafened adults
2004	Endocrine	Thyroid	Vidal-Trecan et al29	Radioiodine vs surgery for toxic thyroid adenoma
2005	Endocrine	Eur J Endocrinol	Sejean et al30	Surgery vs medical follow-up for primary hyperparathyroidism
2005	Endocrine	ANZ J Surg	Blamey et al31	Recombinant human TSH for diagnosis of recurrent thyroid cancer
2005	Sleep	Stroke	Brown et al32	Sleep study screening of stroke victims for OSA
2006	Pediatrics	Pediatrics	Van Howe et al33	Observation without testing for pediatric pharyngitis
2006	Sleep	Arch Int Med	Ayas et al34	CPAP for moderate to severe OSA
2006	Otology	Ear Hear	Barton et al35	Pediatric cochlear implantation
2006	Endocrine	Surgery	Zanocco et al36	Management of asymptomatic primary hyperparathyroidism
2007	Pediatrics	Ann Fam Med	Coco et al37	Management of pediatric acute otitis media
2007	Endocrine	Am J Kidney Dis	Narayan et al38	Parathyroidectomy vs cinacalcet for hyperparathyroidism in ESRD
2007	Allergy	Curr Med Res Opin	Keiding et al39	Immunotherapy for seasonal allergic rhinoconjunctivitis
2007	Rhinology	Am J Rhinol	Anzai et al40	Management of acute sinusitis
2007	Otology	Genet Med	Veenstra et al41	Testing for mitochondrial mutation (A155G) in cystic fibrosis
2008	Sleep	J Int Med Res	Lojander et al42	Nasal CPAP for OSA
2008	Otology	Otol Neurotol	Chang et al43	Hearing aid outcome in the elderly
2008	Allergy	Ann Allergy Asthma Immunol	Bruggenjurgen et al44	Subcutaneous immunotherapy for allergic rhinitis and allergic asthma
2008	Head and neck	Value Health	Brown et al45	Cetuximab plus radiotherapy for head and neck cancer
2008	Sleep	Thorax	Guest et al46	CPAP for OSA
2008	Sleep	Can Respir J	Tan et al47	CPAP for OSA
2008	Endocrine	Surgery	Zanocco et al48	Parathyroidectomy vs observation for primary hyperparathyroidism
2009	Endocrine	Value Health	Mernagh et al49	Recombinant human TSH before RAI ablation for thyroid cancer
2009	Head and neck	Ann Oncol	Sher et al50	CT and PET-CT for determining need for neck dissection in HNSCC
2009	Head and neck	Dermatol Surg	Seidler et al51	Mohs vs traditional surgery for nonmelanoma skin cancer
2009	Rhinology	Appl Health Encon Health Policy	Kneis et al52	Sinfrontal, homeopathic medication, for acute maxillary sinusitis
2009	Head and neck	Acad Radiol	Yen et al53	MRI vs PET vs MRI-PET for diagnosis of recurrent NPC
2009	Sleep	Int J Technol Assess Health Care	Weatherly et al54	CPAP vs dental devices and lifestyle advice for OSA
2009	Allergy	Am J Epidemiol	Witt et al55	Acupuncture for allergic rhinitis
2009	Sleep	Sleep Breath	Sadatsafavi et al56	CPAP vs oral appliances for OSAH
2009	Otology	Fam Pract	Hernandez et al57	Management of Bell's palsy
2009	Pediatrics	Pediatrics	O'Brien et al5	Candidate vaccines for prevention of pediatric AOM
2009	Sleep	Sleep	Snedecor et al58	Eszopiclone for primary chronic insomnia
2009	Endocrine	J Am Coll Surg	In et al59	Treatment options for Graves disease
2010	Endocrine	Ann Surg Oncol	Wang et al60	Oral calcium and calcitriol following total thyroidectomy
2010	Otology	Ear Hear	Summerfield et al61	Bilateral pediatric cochlear implantation
2010	Endocrine	JCEM	Wang et al60	Recombinant TSH prior to RAI for thyroid cancer
2010	Allergy	Value Health	Petrou et al62	Topical intranasal steroids for pediatric OME
2011	Sleep	Sleep	Pietzsch et al63	Diagnostic and therapeutic strategies for OSA
2011	Sleep	Cost Eff Resour Alloc	Scott et al64	Treatment of insomnia
2011	Endocrine	JCEM	Li et al65	Novel molecular test for indeterminate thyroid nodules
2011	Head and neck	Laryngoscope	Higgins et al66	Radiation vs transoral laser surgery for early-stage glottic carcinoma
2011	Head and neck	Laryngoscope	Dedhia et al4	Oral cancer screening programs for high-risk males

Abbreviations: AOM, acute otitis media; PAP, continuous positive airway pressure; T, computed tomography; CUA, cost-utility analysis; ESRD, end stage renal disease; FDG-PET, 18-F fluoro-2-deoxyglucose positron emission tomography; HNSCC, head and neck squamous cell carcinoma; MRI, magnetic resonance imaging; NPC, nasopharyngeal carcinoma; OME, otitis media with effusion; OSA, obstructive sleep apnea; OSAH, obstructive sleep apnea-hypopnea; PET-CT, positron emission tomography-computed tomography; TSH, thyroid stimulating hormone; RAI, radioiodine.

Discussion

In an effort to limit healthcare expenditures and to allocate resources efficiently, many groups have focused their work on the economic appraisal of clinical interventions. As healthcare costs rise, it remains unclear how the increasing economic burden will be handled. There is an increasing need for policy makers, administrators, and physicians alike to evaluate the cost-effectiveness of the services provided. Assessment of value has been investigated for decades, with an exponential increase in cost effectiveness analyses published in the past 10 years. Cost effectiveness research in otolaryngology may have an especially profound impact on limiting healthcare expenditures, as otolaryngologists manage many conditions associated with high costs. The cost of allergic rhinitis alone has been estimated to be close to 5.3 billions dollars per year in the United States. Interestingly, these costs are far surpassed by that of managing sinusitis, which impacts one in seven adults and has direct costs alone estimated at 5.8 billion dollars per year. Within otolaryngology, the management of head and neck cancer and thyroid disease is also especially costly, with only a portion of the costs reflected in direct expenditures of imaging, surgery, and radiation therapy. As healthcare costs continue to rise, it is vital that otolaryngologists take on a more active role in assessing the cost effectiveness of various management options as more expensive innovative technologies continue to be developed.

We have evaluated the CEA registry to assess the characteristics, results and quality of CUAs, which include only those studies measuring health benefits in QALYs, in otolaryngology. Despite the high cost of managing conditions in the practice of otolaryngology, only 2% of the total CUA literature in the CEA registry evaluates intervention sutilized by otolaryngologists and only 0.61% of the cost utility literature in the CEA registry included an otolaryngologist as an author. The relative paucity of otolaryngologists with published studies in the CEA registry may be related to a relative infrequency of conditions with well-established QALYs, fewer clinicians in otolaryngology with training to perform these investigations, or perhaps decreased awareness of these issues among otolaryngologists. Regardless of the reason, it is important that otolaryngologists become more involved in conducting these studies in order to actively participate in discussions regarding the allocation of heath care resources.

In a recent review assessing the quality of 50 economic evaluations published in otolaryngology, Liu and colleagues found that study characteristics such as journal impact factor and presence of an author with a PhD in health economics were associated with higher quality studies. Interestingly, in our study, the subjective quality score bestowed by the CEA registry revealed no statistically significant correlation between the quality of the manuscripts and the number of otolaryngologist authors or the impact factor of the journal in which the study was published. Our review did reveal that studies with a more recent publication year are associated with a higher quality score, indicating that despite the relative lack of CUAs related to otolaryngology, the studies have been improving in both quantity and quality in recent years. Assessment of the topics addressed by all of the studies and their collective results revealed that unilateral cochlear implantation is cost effective in all settings evaluated. Furthermore, 86% of studies identified CPAP to be a cost effective strategy for the management of OSA (Table 3).

Although the relatively large number of studies evaluated and the use of the CEA registry make this review unique, there are several limitations. The CEA registry is a limited database with regards to cost effectiveness literature as a whole. It is possible that many other cost-effectiveness analyses related to otolaryngology that do not adhere to the stringent CUA criteria, yet have made important contributions to understanding the cost of interventions in otolaryngology, have not been evaluated in this particular study. We did not perform our own manual search of all English literature to ensure that no publications were missing from the CEA registry, nor did we perform our own assessment of the individual studies included in this study. Despite this, we are the first group to perform a review of the CUA literature in otolaryngology that has fit the inclusion criteria of the CEA registry. Future reviews of CUAs in otolaryngology may wish to combine search results from multiple databases in order to more comprehensively review the literature in this field.

Conclusion

Based on current evidence in the CEA registry, there is consensus that unilateral cochlear implantation for hearing loss and near consensus that CPAP for OSA are both cost effective interventions. Although CUAs in otolaryngology have increased in quantity and quality in more recent years, there is a lack of CUAs evaluating interventions in otolaryngology. A significant need exists for more otolaryngologists to become involved in evaluating the cost effectiveness of the therapeutic interventions they utilize.

Conflicts of interest/disclosures

None to report.

Funding information

No grant support was received in the preparation of this study.