Toward a Global Consensus on Outcome Measures for Clinical Trials in Tinnitus: Report From the First International Meeting of the COMiT Initiative, November 14, 2014, Amsterdam, The Netherlands.

In Europe alone, over 70 million people experience tinnitus; for seven million people, it creates a debilitating condition. Despite its enormous socioeconomic relevance, progress in successfully treating the condition is somewhat limited. The European Union has approved funding to create a pan-European tinnitus research collaboration network (2014-2018). The goal of one working group is to establish an international standard for outcome measurements in clinical trials of tinnitus. Importantly, this would enhance tinnitus research by informing sample-size calculations, enabling meta-analyses, and facilitating the identification of tinnitus subtypes, ultimately leading to improved treatments. The first meeting followed a workshop on "Agreed Standards for Measurement: An International Perspective" with invited talks on clinimetrics and existing international initiatives to define core sets for outcome measurements in hearing loss (International classification of functioning, disability, and health core sets for hearing loss) and eczema (Harmonizing outcome measures for eczema). Both initiatives have taken an approach that clearly distinguishes the specification of what to measure from that of how to measure it. Meeting delegates agreed on taking a step-wise roadmap for which the first output would be a consensus on what outcome domains are essential for all trials. The working group seeks to embrace inclusivity and brings together clinicians, tinnitus researchers, experts on clinical research methodology, statisticians, and representatives of the health industry. People who experience tinnitus are another important participant group. This meeting report is a call to those stakeholders across the globe to actively participate in the initiative.

Background

Tinnitus—a perception of phantom sound—is a widespread auditory condition (Martinez, Wallenhorst, McFerran, & Hall, 2014). A variety of risk factors (hearing loss, diabetes, cardiovascular conditions, etc.) as well as potential mental comorbidities, such as depression or anxiety (Zirke et al., 2013), contribute to the complexity of tinnitus. This complexity results in diagnostic and therapeutic challenges and also causes problems with the assessment of treatment outcome (Baguley, McFerran, & Hall, 2013). Epidemiological studies performed in Western industrialized countries have found the prevalence rates of tinnitus to be roughly 10% of the adult population. Of all individuals affected by tinnitus, one third reported tinnitus as disturbing or very disturbing (Baguley et al., 2013). The incidence of tinnitus increases with age, as tinnitus closely accompanies any form of hearing loss: in this case, age-related hearing loss. For the industrialized countries, this translates into an ever-growing number of tinnitus sufferers (Martinez et al., 2014). Because of the high incidence rate, the economic impact of tinnitus on the health-care system is relatively large (Vio & Holme, 2005).

Two major factors contribute to the negative impact of tinnitus on economy, health care, and the quality of life of affected individuals: The first is a lack of a singularly effective treatment, and the second is the variation in routine daily practices applied by various clinical units (see Baguley et al., 2013 for a review). Although pharmacotherapy for tinnitus is the subject of intense efforts (Salvi, Lobarinas, & Sun, 2009), the development of an international standard treatment protocol for chronic tinnitus is still in its early stages. Several protocols have been developed for tinnitus treatment such as audiological management, tinnitus retraining therapy (TRT), or cognitive behavioral therapy, but the protocol application varies between practitioners, reflecting country-specific differences in health care or personal preferences. Of 335 studies that reported outcomes of TRT, only one could be assessed in a systematic review, as it was the only one that strictly followed the original protocol. As a result, a comparison between studies was impossible (Phillips & McFerran, 2010). The situation is slightly better for cognitive behavioral therapy, where eight published trials have been included in a Cochrane review and could be compared (Martinez-Devesa, Waddell, Perera, & Theodoulou, 2007; Martinez-Devesa, Perera, Theodoulou, & Waddell, 2010). Unfortunately, several discrepancies remained in the outcome measurements, increasing the difficulties in comparative analysis.

Why Are Outcome Measures for Clinical Trials Important?

The fact that tinnitus is a complex symptom associated with a variety of risk factors and affects not only auditory but also emotional systems makes the choice of outcomes challenging. Some studies or practitioners chose to assess only audiological aspects of tinnitus, others concentrated on psychological and mental health aspects of tinnitus, and some used both types of domains. The use of psychometric instruments widely differs among published studies. The Tinnitus Handicap Inventory, Tinnitus Handicap Questionnaire, Tinnitus Severity Index, verbally administered TRT interview forms, Iowa Tinnitus Handicap Questionnaire, Tinnitus Severity Scale, Tinnitus Reaction Questionnaire, and the Subjective Tinnitus Severity Scale are but a few questionnaires used to measure various parameters of therapy outcomes (Phillips & McFerran, 2010).

Landgrebe et al. (2012; pp.116) described an outcome measurement as “the single most important factor in conducting a clinical trial in tinnitus.” Clinicians use knowledge about how well treatment works when deciding how best to treat their patients. Researchers create this knowledge when they develop and test treatments. Researchers discover the effect of treatments on patients by measuring an outcome or a number of outcomes. Outcomes have two facets. The first is the domain defined as a distinct element (or topic) of tinnitus that is deemed sufficiently important to be evaluated in all clinical trials. Core outcome domains must be of equal relevance to health professionals and to patients who have first-hand experience of living with the condition. Examples include how loud or how emotionally distressing a patient may find his or her tinnitus. The second facet is the outcome instrument defined as the tool used to assess the domain. Outcome instruments should be reliable, validated, and responsive to treatment-related change. Making a quality-based evaluation about existing outcome instruments engages with a different branch of science known as clinimetrics. This is a methodological discipline with a focus on the quality of measurements in medical research and health-care practice (de Vet, Terwee, Mokkink, & Knol, 2011).

Despite the high public health relevance of tinnitus, it is still unclear which aspects of tinnitus are most relevant for assessment in clinical trials and in clinical record keeping. Different clinical trials that test treatments for tinnitus often choose to measure different outcome domains or select different outcome instruments purported to assess the same domain (Dobie, 1999; Hoare & Hall, 2011; Hoare, Kowalkowski, Kang, & Hall, 2011; Williamson, Altman, Blazeby, Clarke, & Gargon, 2012). For example, if one tinnitus study measured loudness and another measured emotional distress, then we could not compare or combine results. Difficulties also arise when researchers measuring several outcomes in a study decide to publish their results for only some of the outcomes (Clarke, 2007). This may happen when the results for some of the outcomes are not what the research team had hoped for. These actions present problems for other researchers who would struggle to work out exactly how effective a treatment is because they do not have full results from the study, and the information they do have might be biased.

If all tinnitus studies used and reported results for the same outcomes, they could be compared and combined. This would make it much easier to make sense of all the knowledge produced. A set of outcome domains and instruments that has been agreed upon for a health condition is called a Core Outcome Set (COS). A COS does not necessitate that outcomes in a particular trial should be restricted to only those in the COS. Rather, there is an expectation that the core outcomes will be collected and reported, but additional outcomes could be included as well. Deciding which outcomes should be core outcomes requires a great deal of discussion and so lends itself well to an international effort.

TINNET—COST Action BM1306: Better Understanding the Heterogeneity of Tinnitus to Improve and Develop New Treatments

European Cooperation in Science and Technology (COST, www.cost.eu) is an intergovernmental framework aimed at facilitating the collaboration and networking of scientists and researchers at European level. It was established in 1971 by 19 member countries and currently includes 35 member countries across Europe, and Israel as a cooperating state. COST funds pan-European, bottom-up networks of scientists and researchers across all science and technology fields. These networks, called COST Actions, promote international coordination of nationally funded research. By fostering the networking of researchers at an international level, COST enables breakthrough scientific developments leading to new concepts and products, thereby contributing to strengthening Europe’s research and innovation capacities. COST’s mission focuses, in particular, on the following:

Building capacity by connecting high-quality scientific communities throughout Europe and worldwide;

Providing networking opportunities for early career investigators;

Increasing the impact of research on policy makers, regulatory bodies, and national decision makers as well as the private sector.

Through its inclusiveness, COST supports the integration of research communities, leverages national research investments, and addresses issues of global relevance. Every year, thousands of European scientists benefit from being involved in COST Actions, allowing the pooling of national research funding to achieve common goals. As a precursor of advanced multidisciplinary research, COST anticipates and complements the activities of EU Framework Programmes, constituting a bridge toward the scientific communities of emerging countries. In particular, COST Actions are also open to participation by non-European scientists coming from neighboring countries (e.g., Albania, Algeria, Armenia, Azerbaijan, Belarus, Egypt, Georgia, Jordan, Lebanon, Libya, Moldova, Montenegro, Morocco, the Palestinian Authority, Russia, Syria, Tunisia, and Ukraine) and from a number of international partner countries. COST’s budget for networking activities has traditionally been provided by successive EU RTD Framework Programmes. COST is currently executed by the European Science Foundation through the COST Office on a mandate by the European Commission, and the framework is governed by a Committee of Senior Officials representing all its 35 member countries.

COST is currently funding a pan-European tinnitus research collaboration network through COST Action BM1306. The project is called TINNET, http://tinnet.tinnitusresearch.net/, which stands for TINnitus research NETwork. TINNET’s main objective is to create a research network for identifying pathophysiologically and clinically meaningful subtypes of tinnitus and their neurobiological underpinnings in order to develop and improve new effective treatments for tinnitus. Within the COST Action, Short-Term Scientific Missions are organized to allow participants to develop expertise in skills required for the project and to strengthen European collaboration.

Members also seek to learn from one another and to disseminate their work through the organization of workshops and conferences allowing tinnitus professionals to benefit from clinical and research training. To provide longer term sustainability to the research, there is an expectation that members will codevelop grant proposals to national, European, and international funding agencies. At present, the participants in the Cost Action BM1306 come from 27 European countries (see Figure 1).

Figure 1.

Twenty-seven participating countries across Europe: Austria, Belgium, Cyprus, Czech Republic, Denmark, Finland, France, fYR Macedonia, Germany, Greece, Ireland, Israel, Italy, Lithuania, Malta, Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovenia, Spain, Sweden, Switzerland, Turkey, and the United Kingdom.

TINNET comprises five working groups (WG): WG1 Clinical, WG2 Database, WG3 Neuroimaging, WG4 Genetics, and WG5 Outcomes. WG1 brings together experts from different disciplines to develop easy and meaningful guidelines for detailed clinical and phenotypic characterization of tinnitus cases and controls (symptom scoring instruments and specific validated self-report questionnaires). WG2 brings together clinicians, clinical trialists, data documentation specialists, biostatisticians, and mathematicians to standardize data management (statistical analysis) and quality control processes and to develop strategies for hypothesis-driven and data-driven meta-analyses. WG3 concerns neuroimaging. This group establishes Standard Operating Procedures for data acquisition and analysis of brain imaging data in order to identify the neurobiological mechanisms of the different forms of tinnitus. WG4 concerns genetics. This group joins clinicians and experts from molecular genetics, statistics, and bioinformatics with the goal of creating a pan-European biobank to study the underlying genetic basis of tinnitus.

Directly relevant to the purpose of this report, the main goal of WG5 is to develop standards for tinnitus clinical trials and outcome measurements in clinical trials and everyday clinical practice. Adoption of the WG5 actions across different tinnitus groups and countries would support data collection in a central database in order to enable further meta-analysis of patient responses to specific treatment options. The WG5 participants are responsible for expanding and extending the common central TINNET database to longitudinal data. WG5 actions will result in implementation and dissemination of guidelines and recommendations for assessing tinnitus management.

At the time of writing, WG5 is an international group of 27 clinicians (mainly specialists in otorhinolaryngology and audiology), psychologists, clinical researchers, and statisticians from 14 COST TINNET member countries. The participants have a wide range of expertise in tinnitus assessment and management, including randomized clinical trials. The group also has some experience in systematic review, Delphi surveys, clinimetrics, guideline development, instrument development and evaluation, and preclinical research.

Main Aims of the First Meeting: Workshop and Meeting

Members of WG5 were invited to participate in a workshop and meeting that were held in Amsterdam on November 13 and 14, 2014, respectively. The aim of the workshop was to provide an overview of (a) existing questionnaire tools for measuring tinnitus outcomes, (b) the methodological approaches available to evaluate the psychometric properties of those tools, and (c) the experience of similar groups that had been through the procedure of core set of outcome measures determination for other clinical entities (e.g., hearing loss, eczema). Representatives presented talks on the methodologies promoted by a number of different initiatives: the COMET (Core Outcome Measures in Effectiveness Trials, http://www.comet-initiative.org/about/overview), initiative, the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments, http://www.cosmin.nl/cosmin-taxonomy.html) initiative, and the ICF (International Classification of Functioning, Disability and Health, http://www.icf-research-branch.org/about-us/our-mission) Research Branch, a World Health Organization collaborator. The purpose of these presentations was to inform and familiarize WG5 participants about the current state-of-the-art regarding tinnitus outcome measures, as well as to give an in-depth overview of tools and methodologies that other groups have used to determine core outcome measure sets. The discussion was fruitful, and the participation was broad and active.

One of the speakers at the workshop used the analogy of a flower to describe the two-step process of defining a COS (Figure 2). Step 1. Domain: This defines the process of HOW to draw the flower by identifying how many petals, and what domain each petal represents. Each petal should relate to a topic or domain in which tinnitus could have an effect (e.g., on quality of life, emotional distress, or loudness). Step 1 seeks to achieve agreement on what is important and essential. The process for Step 1 can be supported by the frameworks outlined by the COMET initiative or by the WHO ICF branch. Step 2. Measurement Tool: This defines the process of WHAT will be the detail in the flower. Each petal needs to be matched to some sort of instrument or measurement tool (e.g., a questionnaire or a behavioral test). Step 2 seeks to achieve a quality-based evaluation on what instrument could be recommended and can be supported by the checklist outlined by the COSMIN initiative. For many TINNET members, this was quite a new way of thinking about approaching the problem of measuring tinnitus. The separation of how (domain) from what (measurement tool) raised discussion across the group, but overall, the group was in favor of this two-step approach. One advantage is that the first step avoids any direct comparison between outcome measures and potential disagreements over the psychometric validity of the different questionnaire tools.

Figure 2.

A good analogy for the research process is that of creating a flower. Each petal of the flower denotes a core domain. Each of those petals is associated with a particular measurement tool, denoted by the arrows. The identification of the domain set and outcome measure set reflect two distinct and separate processes (how and what). The example here denotes five domains and associated tools. There may be more or fewer depending on the outputs at each stage of the project, but the aim is to always nominate just one tool per domain.

Nineteen experts from 15 different countries participated in the subsequent meeting, including audiologists, otolaryngologists, psychologists, and researchers. The aims of the WG5 meeting were to decide how the group would work toward a COS determination, the role of participants, and a set of milestones. Methodologies presented the previous day were discussed and evaluated, decision-making procedures were defined, stakeholders were identified, and ways to approach them were proposed. Furthermore, participants discussed ways to engage clinical and patient stakeholders across countries on an international level.

Summary of Key Discussion Points From the Meeting

Members of the group agreed to be respectful of other people’s knowledge, skills, and expertise, and to approach the collaboration with an open, nonjudgmental, and prejudice-free mind. WG5 made the following key decisions during the meeting:

Methodological approach: It was agreed that the process of establishing standard outcome measurements would use already existing methodology informed by those working on other health conditions (Gargon, Williamson, Altman, Blazeby, & Clarke, 2014). That methodology would include a variant of the Delphi survey starting off with a set of predefined domains informed by previous work (e.g. Tyler & Baker, 1983).

Scope: The group agreed on establishing a COS for all types of tinnitus relevant for an adult (but not pediatric) population.

Group management: WG5 agreed unanimously on the creation of a Core Project Team to advise and manage the project. This group will be methodology-oriented and include four to six individuals, some of them being members of TINNET and others being experts in the field of methodology (where the expertise does not lie within TINNET). In addition, WG5 agreed on the importance of a patient representative(s) on the Core Project Team.

International reach: Although based in Europe and managed by the European authorities, WG5 agreed to extend the invitation to engage in WG5 activities to all tinnitus researchers from all over the world and to all relevant international and national organizations that are involved in tinnitus research, diagnosis, or treatment.

Stakeholder groups: WG5 agreed on identifying and actively engaging the following stakeholder groups: academics who research tinnitus, clinicians dedicated to tinnitus, founder and commissionaires of health services, industry representatives, health insurance companies, journal editors, tinnitus patients, and regulatory body representatives.

Roadmap of the Overall Project

In summary, the overall ambition of the WG is to establish an international standard for outcome measurements in clinical trials of tinnitus. The standard will be achieved by a two-step effort to produce a COS of domains and instruments that harmonize viewpoints across both professional and patient stakeholder groups. A roadmap has been proposed, which sets out a provisional plan for delivery (Figure 3). This roadmap reflects the two-step process with Stage 1 identifying and agreeing on outcome domains, and Stage 2 identifying and agreeing on outcome instruments. Stages 3 and 4 are somewhat more speculative. They would not be required if sufficient numbers of instruments passed the Stage 2 evaluation criteria set and if there was agreement to adopt them into the COS. The short-term goal of the WG is to achieve at least the first stage of the roadmap supported by the present European Union funding.

Figure 3.

Anticipated roadmap for the project. COS = Core Outcome Set.

To achieve a credible international standard requires widespread engagement from stakeholders across the globe, as well as transparent and rigorous methodology. Our aspiration is that the standard will be adopted by all workers in this field; thus, we acknowledge the importance of enabling everyone to contribute their viewpoint and for a consensus to be reached from those multiple perspectives.

Plans for delivery of Stage 1 are as follows. A Core Project Team will manage the project forward, meeting regularly by teleconference. Members have not yet been nominated, but the group will comprise the following expertise from within the WG members: a clinician with expertise in tinnitus, a researcher with expertise in tinnitus, a researcher with experience in consensus methodology, and a clinimetrician. In addition, the Core Project Team will receive external advice from a member of the Harmonizing Outcome Measures for Eczema (HOME, www.homeforeczema.org/) initiative.

The next tier of project management will include national advocates acting on behalf of individual countries across the globe to ensure that as many cultural perspectives as possible are represented. The role for advocate participants in the project would be to translate the Delphi survey from English into the native language, engage with relevant national stakeholder groups (including patient groups) to participate in the Delphi survey, and contribute to dissemination.

Without external funding or a dedicated research staff, the project currently relies on the enthusiasm and motivation of its participants. Many of us see this as an exciting new venture, which has the possibility to make a ground-breaking contribution to the field.

Next Steps

The first meeting focused on defining the question, on delineating the scope of the project and on agreeing on some of the fundamental methodological principles to be adopted. Prior to the second meeting, a small team of members will conduct a systematic review to identify, without bias, what outcome measurement tools have been so far used in randomized controlled trials of therapeutic interventions for tinnitus (http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015017525). The European Union grant will fund a small number of short-term working visits to facilitate this work over the next year.

The project is open to views from all stakeholders interested in the development of COS for tinnitus. There is a strong passion and shared optimism for working together and engaging with tinnitus experts outside the European Union in order to achieve a true international consensus. Therefore, the WG particularly encourages stakeholders from nations currently not represented to join the group, attend meetings, and participate in the Stage 1 Delphi survey. Participants from countries outside the European Union are particularly welcome to act as a national advocate for the project. For more details, see http://tinnet.tinnitusresearch.net/