Uniform Registration Agreements on Cholesteatoma Care: A Nationwide Consensus Procedure.

BACKGROUND: To coordinate and align the content for registration of cholesteatoma care.
METHODS: Systematic Delphi consensus procedure, consisting three rounds: two written sessions followed by a face-to-face meeting. Before this procedure, input on important patient outcomes was obtained. Consensus was defined as at least 80% agreement by participants. Hundred-thirty-six adult patients who had undergone cholesteatoma surgery and all ENT surgeons of the Dutch ENT Society were invited. The consensus rounds were attended by ENT surgeons with cholesteatoma surgery experience. Feasibility and acceptability of outcome measures and reporting agreements were assessed in round 1 by 150 ENT surgeons. In round 2 definitions were narrowed and context information to interpret outcome measure were questioned. In round 3, the results, amendments, and the open-ended points were discussed to reach agreement.
RESULTS: Most important outcome measures are: 1) the presence or absence of a cholesteatoma in the first 5 years after surgical removal of cholesteatoma, 2) hearing level after surgical removal of cholesteatoma, and 3) the documented assessment of patient's complaints with a validated patient reported outcome measures questionnaire (PROM). Furthermore, consensus was reached on the registration of cholesteatoma type (residual/recurrent), localization of cholesteatoma, and reporting of the presence of cholesteatoma in the follow-up.
CONCLUSION: Consensus was reached on the content and method of registration of cholesteatoma care based on patient's and ENT surgeons input. Three outcome measures were defined. National agreements on the method and content of registration will facilitate monitoring and feedback to the ENT surgeon about the cholesteatoma care.

Despite the low incidence of cholesteatoma (1), every ENT practice will see patients with this condition because of its recurrent nature. In the Netherlands, 1,000 to 3,000 cholesteatoma-related operations are performed every year. Although cholesteatoma occurs at all ages and all over the world (2), there are no national or international guidelines for the registration, treatment, or follow-up of cholesteatoma. However, the Dutch ENT Society has now taken the initiative to develop a cholesteatoma guideline for registration and follow-up of cholesteatoma patients. Part of such guideline is a plan to monitor the actual performance. That is why simultaneously steps have been taken in setting up a national otology quality registry.

A quality registry generates information on care (structure, process, and outcomes) based on the perspectives of both health providers and their patients on quality of care (3). This information, often expressed as quality indicators, can be used to obtain feedback on the results of the care provided by the individual participating ENT surgeons in relation to a benchmark. This comparison stimulates improvement as goals can be set based on best practices (4).

The definition, classifications, pathology, and treatment are an ongoing discussion in international scientific literature but also to improve uniform registration (5–8). Recent published international classifications are STAMCO, ChOLE, and SAMEO-ATO (7–9).

This project aims to identify suitable outcome measures for cholesteatoma care from both the patient's and the ENT surgeon's perspective. Describing outcome measures requires uniform definitions and terminology as well as registration agreements, which is why the study will also sharpen definitions. Moreover, it is important that data can be interpreted correctly, so that outcome differences may be related to differences in pathology, surgery, or other relevant (context) parameters.

To be able to register cholesteatoma care in the Netherlands in a better and uniform manner, an attempt has been made to find answers to the following research questions:

What are suitable outcome measures for cholesteatoma care?

What context information is required for the selected outcome measures?

Which terminology used requires further standardization of definitions?

MATERIALS AND METHODS

A Delphi consensus procedure was performed to decide on the data for monitoring cholesteatoma care (10–12). This systematic method consisted of two written sessions (rounds 1 and 2) followed by a face-to-face meeting (round 3). The preliminary set of information was based on input from a professional expert team (authors) as well as a patient survey on important outcomes on cholesteatoma care. The potential participants approached for the Delphi procedure were ENT surgeons who, at the time, were members of the Dutch ENT Society with experience in cholesteatoma surgery.

For all the consensus rounds in the Delphi study, consensus was defined as at least 80% of the ENT surgeons reaching agreement, which is a relatively strict cut-off point (13,14). All digital surveys were sent out using Castor-EDC (Amsterdam, the Netherlands). ENT surgeons had 2 weeks to complete the survey with a reminder after 1 week. After each round, ENT surgeons were asked to provide additional relevant information, and at the end of each round the results were shared online. The entire Delphi process (shown in Fig. 1) was completed within 6 months (November 2018–May 2019) (15).

FIG. 1

Contents of Delphi procedure for cholesteatoma care.

Preparations

First of all, an expert team was appointed: three ENT surgeons from university hospitals, three ENT surgeons from non-university hospitals, a Ph.D. researcher, a Dutch ENT society board member, a scientific researcher specialized in developing and selecting quality indicators, and a Professor of Auditory Functioning and Participation (authors, n = 9).

To obtain a better understanding of outcome measures that are important for patients, a list of the most frequently reported symptoms (both pre- and postoperative) was drawn up from the literature (16–19). This information was combined with the information on overall patients’ symptoms taken from the national Dutch Cholesteatoma Data study (DCD) (trial 80-83700-98-16504). The input of literature and this cohort was used to develop a patient survey on relevant outcome and process measures in cholesteatoma care from the patient's perspective (see table, Supplemental Digital Content 1, which demonstrates patient survey).

Which symptoms should be included in the patient record (both before and after surgery);

Should these symptoms be discussed by the ENT surgeon in a particular order;

What determines the success of cholesteatoma treatment for the patient;

What other factors, according to the patient, contribute to the quality of care?

Questions Asked Were

For each question, the patients were able to fill in several answers and, if necessary, to give additional information. A total of 136 patients from DCD were asked to answer these questions (METc approval VUmc, no. 2016.523). The participants were adult patients who had undergone surgery for cholesteatoma removal. These patients either had primary, recurrent, or residual cholesteatoma and the survey was sent out during their first year of follow-up after their (last) cholesteatoma surgery. Some of the patients had already several surgeries and multiple Magnetic resonance imaging or computed tomography (CT) scans in the past 4 years. Others just had the first surgery and their first MRI. This group of 136 patients had different complaints, hearing levels, and impact. Within this cohort of adult patients, age, social status, profession, and sex were well distributed.

After anonymization, the survey was analyzed using descriptive statistics. The percentages for each question were calculated for each answer category. The expert team was then given the top answers per question to enable them to draw up the list of possible outcome measures in round 1.

Round 1

All 556 ENT surgeons of the Dutch ENT society were invited by email to participate in the survey of round 1. The aim of this round was to define a limited set of outcome measures and to determine the required reporting on pathology, surgery, and aftercare. The participants were asked whether the availability of this information contributes to the quality of care. And asked whether the requested information is already in the electronic medical record (EMR), or whether the ENT surgeon is willing to register this information in the EMR. In addition, two questions were asked about current follow-up (no follow-up performed or the use of CT scan). These questions were asked in preparation for round 2.

Round 2

For the second round, ENT surgeons who had participated in the first round and also had experience with cholesteatoma surgery (150) were invited. Based on the results of the first round, consensus was sought on the contextual information required to interpret the proposed outcome and process measures properly.

Round 3

All participants who responded in rounds 1 or 2 received an invitation to attend the final joint meeting. The purpose of the joint meeting was to present the results of the previous rounds and to discuss them jointly. Issues from round 2 with a consensus percentage between 55 and 80% were presented to the participants again to clarify any ambiguities and were reassessed with the help of a digital voting system. In addition, five groups were formed to discuss and select: the use of a certain classification (ChOLE or STAMCO) to register the cholesteatoma (7,8), the classification for surgery type (SAMEO-ATO) (9) and about validated patient reported outcome questionnaires (PROM) to uniformly register the patients’ symptoms. These were the COMQ-12 and OQUA (20,21). This third round was led by an independent process consultant with experience in consensus discussions and setting up care registries.

RESULTS

Patient Survey

Ninety-six out of 136 surveys were completed (70.5%); no incomplete surveys were returned. Both before and after the cholesteatoma operation, the most frequently reported symptom is “hearing loss.” In addition, the two other most frequently reported symptoms are preoperative “otorrhea (ear discharge)” and “feeling of pressure in the ear.” Postoperative symptoms are “tinnitus” and “feeling of pressure in the ear.” The survey also showed that the factors determining whether surgery is perceived as being successful by patients are “no recurrence of the cholesteatoma,” “improved hearing,” and “no complications.” The factors that patients considered crucial in determining the quality of care are “communication with the physician,” “being able to discuss the fear of recurrence of the cholesteatoma,” and “number of visits to outpatient clinic” (see Table 1). From the patient's perspective, it is important to include this data for the uniform registration of cholesteatoma care.

TABLE 1

Patient survey: ear problems before/after surgery, success factors, and cholesteatoma care quality, (n = 96)

Round 1: ENT Surgeon Survey

Three outcome measures were drawn up based on the patient survey results, the literature, and the discussion in the expert team (Fig. 2). These outcome measures were presented in round 1 next to the reporting on pathology, diagnostics, surgery, and aftercare statements (see Table 2).

FIG. 2

Defined outcome measures.

TABLE 2

ENT surgeons survey 1: degree of consensus (%) on the proposed statements (n = 150)

This was first expressed in general terms, so that in the next round, the statements for which agreement had been reached, were specified further.

Of the 556 ENT surgeons, 192 completed the first survey and 150 ENT surgeons indicated that they also performed cholesteatoma surgery.

Table 2 shows that the proposed statements met the consensus norm of 80%. The PROM raised questions, as many surgeons do, or do not, use a PROM in cholesteatoma care. Furthermore, 43% of the ENT surgeons indicated that there may be situations in which no follow-up is performed (i.e., no MRI or second look surgery) after cholesteatoma surgery. 61% of the ENT surgeons occasionally use a CT scan in the follow-up. Numerous comments were made about the definitions and terminology. The expert team took these comments into account in the second round survey.

Round 2: ENT Surgeon Survey

In this round, consensus was sought on the context information for the three selected outcome measures (see Table 3).

TABLE 3

ENT surgeons survey 2: degree of consensus (in %) on the context information (n = 131)

A 70% consensus was reached on the definitions of primary acquired, recurrent, and residual cholesteatoma. In the definition the word “visible” was mistakenly used instead of “poorly visible,” which was often commented. The definition was modified and approved in round 3. 87% of the respondents agreed with the proposed way of reporting the presence of cholesteatoma during the follow-up.

No consensus was reached on the use of the STAMCO or ChOLE classification for the pathology and the SAMEO-ATO classification for the surgical procedures. However, the ENT surgeons did agree on the appropriate intervals for MRI in the follow-up. The answers indicate that 82% of ENT surgeons consider it desirable to perform at least two MRIs in the first 5 years and 73% propose to perform an additional MRI in the second or third year after surgical removal. There was no consensus on MRI monitoring after these first 5 postoperative years.

The comments made in survey 2 showed that some questions were not entirely clear, which meant that not all respondents were able to answer these questions in the same way. Many comments pertained to the third outcome measure, the PROM. This was discussed during the joint meeting.

Round 3: Joint Meeting

A total of 36 ENT surgeons from 25 different hospitals were present at the joint meeting, including five members of the expert team. The expert team members refrained from voting, because of involvement in the development of certain classification that were voted on during the joint meeting and prevent bias due to preferences. The results of rounds 1 and 2 were presented. Statements with a score between 55 and 80% were further explained, discussed, and reassessed. The definitions of the types of acquired cholesteatoma were jointly corrected and approved. Figure 3 shows the agreed definitions.

FIG. 3

Definition acquired cholesteatoma and categories agreed upon for presence of cholesteatoma in follow-up.

Next to this, it was decided in the meeting to add the type of cholesteatoma to the context information for the 1st outcome measure (cholesteatoma presence) (96%). And it was decided that a preoperative audiogram should be performed no more than 6 months before the surgery (83%).

The classifications and PROMs were discussed intensively in the meeting.

Classifications

The discussion showed that selecting a classification is not most important, but rather the information provided in the operative report. If the operative report contains all matters from the existing classifications, then this should be enough information for registration.

PROMs

The ENT surgeons were unable to decide between the two validated questionnaires identifying the patients’ problems (OQUA or COMQ-12). Various issues emerged from the discussion of the questionnaires. According to those present, a shorter questionnaire that is presented more often is more likely to be completed than a longer one. The OQUA is suitable for all types of ear procedures, which could make things easier for a practice than having a different questionnaire for each type of procedure/pathology. It was decided to first obtain more clarity on the choice of the PROM, before identifying the context information needed.

DISCUSSION

The study was carried out to reach a consensus on uniform registration of cholesteatoma care and is, as such, the first documented consensus in this field (both nationally and internationally). Three rounds of the Delphi method were used to systematically involve ENT surgeons to achieve registration agreements. The study provides a clear picture of the outcome measures in the treatment of primary acquired and other cholesteatomas that can possibly be used as quality indicators. These are: 1) the presence/absence of a cholesteatoma in the first 5 years after surgical removal of cholesteatoma, 2) hearing level after surgical removal of cholesteatoma, and 3) the documented assessment of patient's complaints with a validated patient reported outcome measures questionnaire (PROM).

During the whole study, two of the three outcome measures were adjusted. The first outcome measure did initially not include a time frame. Because this outcome measure can only be evaluated after a longer follow-up period, it was decided by the expert team to add a fixed time period. Various follow-up time frames were proposed and a follow-up of 5 years was eventually agreed upon. There is sufficient evidence that a long follow-up of at least 5 years is useful after cholesteatoma surgery (22) and “this five-year period may prevent the early discharge of follow-up after cholesteatoma surgery.” Furthermore it is in line with the current majority for MRI follow-up period and it is also comparable to cancer survival rates (23). The second outcome measure represents the most important symptom for patients, namely, the degree of hearing loss. For the last outcome measure, it was decided to monitor the patients’ problems using a validated questionnaire that was completed by the patients (PROM). A PROM is the basis for uniform registration of the subjective measure (24). In addition, the definitions of cholesteatoma types and the method of reporting the presence of cholesteatoma were established which prevent “contamination” of the registration. The definitions distilled from this study for the types of cholesteatoma are more specific than those mentioned in the consensus paper by Yung et al. (25).

To be able to judge the outcome measures properly, it had to be decided which context information is important. ENT surgeons need to relate their results to the case-mix of patients, so that the outcome measures can be used in a nuanced way. A total of three context items were defined for the first outcome measure and four for the second outcome measure. No further context items were agreed upon for the third outcome. Ideally, a validated questionnaire should include questions about the hearing, which would then give a complete picture. However, the use of a questionnaire or the systematic questioning of patients regarding their symptoms does not yet appear to be general practice. The national cholesteatoma study and the literature (16,17) show that hearing loss with or without otorrhea is the most important health concern. In addition, to improve hearing, the cessation of otorrhea is a determining factor for the success of surgery according to patients. These success factors are also mentioned in the studies of Lailach et al. (18) and Dornhoffer et al. (19), and are questioned items in both proposed PROMs. In view of these three acknowledgments, it seems that the structural request for information on patients’ problems using a PROM, before and after surgery, is an essential part of the quality monitoring.

The method used has the potential of systematically achieving a nationwide consensus between ENT surgeons (after input by patients and preparations by an expert team). The use of two consecutive written rounds in the Delphi method was highly conducive to the in-depth study. Another advantage of this “bottom-up” method is that input from the ENT surgeons was completely anonymous and not hindered by the opinions of (inter)national experts. Which meant that “having to follow the norm” and the conviction that strong opinions are decisive could be avoided (26,27). The expert team noticed, in accordance with the literature (28), that both feedback and reminders after each round increased participation. The joint meeting in the last round reinforced the results for several reasons. The meeting encouraged discussions, which helped clarify the argumentation, and in turn led to clear agreements on reporting, necessary for the implementation of the monitoring program aimed to improve the cholesteatoma care. In the literature, it is suggested that consensus can be reached in a joint meeting of at least seven persons (11). This requirement was amply met with a large representative group of ENT surgeons for the joint meeting. The high cut-off point for consensus (80%), increases the chance of reproducibility of the research (29). Furthermore, we think that this group of patients can be representative for the adult cholesteatoma patient in a Western country, because patients in different stages of the disease (primary, recurrent, or residual disease), with different complaints and impact, different surgical approaches, and different social and economic status are included. These patients were included from a multicenter study of 16 participating centers spread across the Netherlands and the distribution of included patients between peripheral and university medical centers was normal. Next to this, the DCD had ethical permission to send out questionnaires to these patients, whereas patient participation according to a newly set-up international consortium for health outcomes measurement framework would not have been feasible for this study. Furthermore, all patients were questioned at the same moment in time, but during different stages of individual follow-up.

The study also had a number of limitations. The second survey included additional information on both classifications and the validated questionnaires. However, the information had to be read in a too short period (2 weeks) and some statements were interpreted differently than intended. Clearer formulation and longer time could possibly have prevented these misinterpretations. The expert group was looking for the best classification for Dutch ENT practice in line with international classifications. However, consensus could not be reached. Combining separate elements of these classifications seemed a viable compromise but needs additional debate. Furthermore, it was decided to include the symptoms and only the complications caused by the cholesteatoma itself and not complications caused by the surgeon. This was a conscious decision by the expert team, because there are only a few national registries within the ENT field and the response rate is often related to how “sensitive” the information is. A next step would be to include the surgical complications.

This report will be presented to the Dutch ENT Society to support the development of a Otologic Quality registry.

CONCLUSION

National consensus has been achieved on outcome measures, definitions, and a uniform way of registering cholesteatoma care by using a Delphi consensus method with input from both patients and ENT surgeons. This consensus likely contains many valuable elements for the uniform registration of cholesteatoma care worldwide as well as for the establishment of (inter)national otologic quality registry to ensure and improve cholesteatoma patient care.