Comparability of European League Against Rheumatology-Recommended Pharmacological Treatments of Oral Ulcers Associated with Behçet's Disease: A Systematic Literature Review of Randomized Controlled Trials.

OBJECTIVE: Oral ulcers are the cardinal manifestation in Behçet's disease (BD). The 2018 European League Against Rheumatism (EULAR) recommendations describe treatments for BD-associated oral ulcers with mucocutaneous involvement; however, little comparative effectiveness information for these agents is available. In the absence of head-to-head trials, an indirect treatment comparison (ITC) could provide useful evidence regarding comparative effectiveness of BD treatments. The purpose of this study was to conduct a comparative systematic literature review (SLR) and similarity assessment of randomized controlled trials (RCTs) investigating the oral ulcer-related efficacy outcomes of EULAR-recommended treatments for BD-associated oral ulcers to determine the feasibility of an ITC.
METHODS: An SLR was performed to identify relevant RCTs indexed in MEDLINE/Embase before May 29, 2019. RCT similarities for the ITC were assessed based on a step-wise process recommended by the International Society for Pharmacoeconomics and Outcomes Research.
RESULTS: In total, 317 articles were identified, of which 14 RCTs, reflecting 11 EULAR-recommended treatments, were evaluated in a similarity assessment. Number of oral ulcers, resolution of oral ulcers, and healing time for oral ulcers were identified as the possible oral ulcer-related outcomes. After completing the similarity assessment of these outcomes, it was determined that a robust ITC was infeasible for the three oral ulcer-related outcomes due to heterogeneity in outcomes reporting, study design, and/or patient characteristics. More broadly, the results underscore the need for and consistent use of standardized measures for oral ulcer outcomes to facilitate comparative research.
CONCLUSION: In the absence of head-to-head RCTs and infeasibility of quantitative ITC, comparative assessments for BD-associated oral ulcers are limited, including comparative effectiveness and cost-effectiveness evaluations. Healthcare decision-makers must continue to base treatment decisions on the extent and strength of available evidence (eg, robust RCTs), clinical guidelines, real-world experience, and patient considerations.

Plain Language Summary

Little information is available on the comparative effectiveness of treatments recommended by the 2018 European League Against Rheumatism (EULAR) for oral ulcers in patients with Behçet’s disease (BD), given the lack of comparative randomized controlled trials (RCTs).

An indirect treatment comparison (ITC) may be evaluated when RCTs directly comparing treatments are unavailable.

A comprehensive review of the scientific literature and assessment of the similarity of RCTs investigating the oral ulcer-related efficacy outcomes of EULAR-recommended treatments in patients with BD was first conducted. Three oral ulcer-related outcomes were relevant and appropriate for assessing the feasibility of an ITC: number of oral ulcers, resolution of oral ulcers, and healing time of oral ulcers.

Results of our similarity assessment of RCTs identified differences in (a) the ways outcomes are reported, (b) the ways studies are designed, and/or (c) the characteristics of the patients included in the studies (eg, average age, severity of disease).

A robust ITC was determined to be infeasible for the oral ulcer-related outcomes identified in BD RCTs.

Results of the similarity assessment also highlight the need for standardized measures for oral ulcer outcomes and comparative research for oral ulcer treatments in BD.

In the absence of an ITC, the results of this study suggest that treatment decisions for patients with BD and oral ulcers must continue to be based on the extent and strength of available scientific evidence (eg, high-quality RCTs), clinical guidelines, real-world experience, and patient considerations.

Introduction

Behçet’s disease (BD) is a rare, systemic variable vessel vasculitis that involves multiple organ systems.1 Manifestations of BD may involve the skin and mucous membranes (ie, mucocutaneous involvement), eyes, vascular system (both veins and arteries), gastrointestinal tract, nervous system, and joints.1 Oral ulcers are the first manifestation experienced by a large majority of patients, appear in the oral cavity,2 and can be experienced throughout a patient’s lifetime.3–7 Oral ulcers are also the most common manifestation of BD.8 Results from a systematic literature review (SLR) showed that the median prevalence of oral ulcers among 27 observational studies was 100% (interquartile range: 84–100%), suggesting that oral ulcers are experienced by essentially all patients with BD.8 Furthermore, recurrent oral ulcers are required for diagnosis under the most commonly used set of criteria (the International Study Group criteria).9,10 Evidence suggests patients with BD may experience on average approximately 13 oral ulcers per year,11 and oral ulcers can also occur concurrently with other symptoms, such as fatigue- and sleep-related issues, pain, headache, and other skin lesions (eg, pseudofolliculitis, papulopustular lesions).8 As BD progresses, patients may experience additional manifestations, including ocular, vascular, and neurological involvement.12,13

The high prevalence and recurrence of oral ulcers and their negative impact on the clinical and humanistic burden of disease represent an important opportunity for treatment. Oral ulcers can lead to poor dental hygiene,14–16 limit essential daily activities like eating and talking,16 and impair quality of life (QoL) overall and across several domains, such as physical and social functioning, sleep and fatigue, and mental health.8,17–19 Furthermore, patients with BD who have oral ulcers have reduced QoL compared with healthy controls and patients with other chronic conditions.17,20

The European League Against Rheumatism (EULAR) is the primary source of treatment recommendations for BD-associated oral ulcers.1 In 2018, the EULAR recommendations suggested that topical treatments (eg, corticosteroids) should be considered first for the treatment of oral ulcers.1 Colchicine was recommended to be tried first for the prevention of recurrent mucocutaneous lesions, including oral ulcers, due to its good safety and tolerability profile; however, the EULAR recommendations also note conflicting efficacy data for colchicine in oral ulcers.1,21 Immunomodulatory and immunosuppressive drugs, such as azathioprine, thalidomide, interferon-α, tumor necrosis factor (TNF)-α inhibitors, and apremilast, are recommended by EULAR based on evidence from at least one randomized controlled trial (RCT) for select cases, such as patients whose oral ulcers continue to recur despite the use of colchicine. In 2019, Phase 3 RCT (RELIEF) data became available for apremilast, leading to a formal indication for BD-associated oral ulcers and subsequent approvals in the United States and Japan (2019) and in the European Union (2020).22–24 Although real-world evidence indicates oral corticosteroids may be used to treat BD, they are not recommended by EULAR to treat BD-associated oral ulcers; furthermore, a variety of published research challenges their use as chronic therapy due to substantial associated morbidity.1,25,26

Evidence-based healthcare decision-making requires a comparison of relevant competing interventions, and systematic reviews of RCTs are a standard method of analyzing information in the healthcare setting.27 Comparative evidence is also typically included in health technology assessments (HTAs), which is a multidisciplinary research process designed to inform policy decisions and to provide decision-makers with a summary of information related to the use of a health technology.28–30 The SLR underlying the EULAR recommendations for mucocutaneous manifestations found heterogeneously reported outcomes for BD-associated oral ulcers (eg, number, duration, frequency, severity), discrepancies in statistical significance of efficacy outcomes, a dearth of adverse events reporting, and lack of head-to-head comparisons.21 The authors noted that the limitations observed in the literature hindered a comparison of the efficacy of treatments and that a formal comparative effectiveness analysis would be valuable.21

Although the SLR underlying the EULAR recommendations provides a valuable assessment of efficacy and safety outcomes for individual treatments for mucocutaneous manifestations (including oral ulcers), the objective of the SLR was not to perform an in-depth comparison of trial similarity (eg, trial design, patient and treatment characteristics, outcome reporting methods) for oral ulcer treatments.21 Thus, we conducted a comparative SLR and similarity assessment of RCTs investigating the oral ulcer-related efficacy outcomes of EULAR-recommended treatments for oral ulcers associated with BD to determine the feasibility of a comparative effectiveness study via an indirect treatment comparison (ITC). Given that no direct comparative effectiveness evidence is available, the comparative SLR and RCT similarity assessment would inform the potential to proceed with an ITC and would provide useful evidence regarding comparative effectiveness of treatments for consideration by healthcare decision-makers, including HTA bodies.27

Materials and Methods

Systematic Literature Review: Search Strategy and Article Selection

An SLR was performed in accordance with standard practice31 using a predefined search strategy () to identify RCTs (articles and their supplements or letters) indexed in MEDLINE/Embase, written in English, and published any time up to May 29, 2019. The references identified in the literature search were reviewed for potential inclusion.

Articles were included if they i) were RCTs, ii) enrolled adult (≥18 years of age) patients with BD and oral ulcers, iii) assessed EULAR-recommended1 systemic or topical pharmacologic treatment for mucocutaneous manifestations (ie, topical steroids, colchicine, azathioprine, thalidomide, interferon-α, TNF-α inhibitors, and apremilast) or placebo, and iv) described efficacy or patient-reported outcomes related to improvement in oral ulcer-related outcomes. Excluded were non-randomized trials, single-arm or dose-escalation trials lacking placebo, observational studies, or results published only as conference abstracts. Full inclusion and exclusion criteria can be found in Table 1.

Table 1

Eligibility Criteria for Randomized Controlled Trials (PICOS Model)

	Inclusion	Exclusion
Population	Patients with Behçet’s disease aged ≥18 years
Intervention	EULAR-recommended therapiesa	Lifestyle/behavioral interventions
Comparator	Other EULAR-recommended therapiesa or placebo	Non-EULAR-recommended treatments or none
Outcome	Any relating to oral ulcers, including but not limited to: Number Frequency Healing time/duration Pain Patient-reported outcomes	Outcomes other than those relating to BD-associated oral ulcers
Other	Randomized, parallel-arm, placebo-controlled, withdrawal trials; randomized, crossover trials reporting results at first crossbEnglish languageArticles or letters indexed in MEDLINE/EmbaseAny publication date	Non-randomized trialsSingle-arm trialsDose-escalation trials lacking placeboObservational studiesConference abstracts

Notes: aIncludes topical and systemic measures such as steroids, colchicine, azathioprine, thalidomide, interferon-α, tumor necrosis factor-α inhibitors, and apremilast, per the 2018 EULAR recommendations for the management of mucocutaneous involvement (including oral ulcers) in Behçet’s disease. bOnly parallel-arm trials or crossover trials reporting results at first cross can be considered for the indirect treatment comparison.

Abbreviations: BD, Behçet’s disease; EULAR, European League Against Rheumatism; PICOS, participant, intervention, control, outcome, study design.

The literature search results were downloaded into a single Excel file, and one reviewer (EH) independently screened all titles and abstracts resulting from the search methodology to identify articles for full-text review. Citations selected for full-text evaluation were reviewed by two independent reviewers for potential inclusion in the feasibility assessment (EH and SR).

Data Extraction

One reviewer extracted all relevant data into an extraction sheet developed in Excel (EH), and a second reviewer performed quality assurance (SR). Extracted data included key trial characteristics, oral ulcer-related outcomes (eg, frequency, number, size, duration, pain, response, severity), and key baseline patient demographics and clinical characteristics. Such key baseline clinical characteristics included oral ulcer-related outcomes (eg, duration, number, healing time, pain), prior treatments, concomitant treatments, disease duration, and presence of other BD manifestations.

Similarity Assessment for an ITC

Evaluation of RCTs in the similarity assessment for an ITC was conducted using the step-wise process (Figure 1) recommended by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).32 First, RCTs reporting oral ulcer-related outcomes that were unique to one treatment were excluded from further review, as these would not be comparable against other treatments. Second, oral ulcer-related outcomes commonly reported for at least two treatments were identified. Third, for each commonly reported outcome, the following steps were followed to determine whether RCTs reporting that outcome should be excluded in an ITC: i) exclude RCTs if the outcome reporting method was different from the other RCTs, such as different outcome definitions (eg, new oral ulcers vs any oral ulcers) or different units of measure (eg, mean number vs probability of an event, where the endpoint could not be easily transformed into another commonly reported outcome); ii) exclude RCTs if the outcome was reported at different time points (ie, by at least 2 weeks; the 2-week time point was chosen because individual oral ulcers heal in approximately 2 weeks) compared with other RCTs; and iii) exclude RCTs if the patient demographics and clinical characteristics were not similar to other RCTs. Expert opinion was applied to qualitatively determine whether differences were clinically meaningful. After completion of the step-wise process described above, an ITC would be feasible for an oral ulcer-related outcome if at least two treatments were included.

Figure 1

Step-wise process for determining feasibility of an ITC for oral ulcer-related outcomes in BD.

Results

Study Selection

Overall, 317 articles were identified, including 310 identified through a pre-specified search of MEDLINE/Embase and seven articles identified in published systematic reviews (Figure 2). Of these 317 articles, 302 were excluded during the title/abstract screening. Thus, 15 articles were screened with full text, and one article assessing topical cyclosporine-A was excluded because of its publication type (ie, conference abstract).33 The remaining 14 articles (representing 14 RCTs) were evaluated in the similarity assessment for an ITC and included the following treatments: apremilast,34,35 colchicine,36–40 colchicine + benzathine penicillin,39 interferon-α-2a,41 triamcinolone acetonide ointment,42 interferon-α lozenges,43 cyclosporine,38 interferon-α-2c hydrogel,44 thalidomide,45 etanercept,46 and azathioprine.47

Figure 2

PRISMA diagram.

Efficacy Outcomes

In the first step of the similarity assessment for an ITC (Figure 3), six RCTs only reporting oral ulcer-related outcomes unique to one treatment were excluded. The excluded RCTs were i) an RCT evaluating colchicine, which rated aphthous stomatitis on a scale ranging from 0 to 3+ (numerical values used in this scoring system were not clearly defined)36; ii) an RCT evaluating colchicine, which reported Iranian BD dynamic activity measure scores for oral ulcers37; iii) an RCT evaluating colchicine and cyclosporine, which reported the percentage of patients who experienced “alleviated oral ulcers” (“alleviated” was not defined)38; iv) an RCT evaluating interferon-α lozenges, which reported the mean change in the total surface area of all oral ulcers present43; v) an RCT evaluating triamcinolone acetonide ointment, which reported the percentage of patients experiencing a “positive response” (“positive” was not defined)42; and vi) an RCT evaluating interferon-α-2a, which reported the number of patients who responded to treatment (authors considered all signs and symptoms of BD when determining response and results were not specific to oral ulcers).41 In the last RCT listed here, additional oral ulcer-specific endpoints were assessed; however, these outcomes were not further assessed for comparability because the authors only reported P values and no estimates were provided.41

Figure 3

Overviewof the results of the similarity assessment to determine the feasibility of performing an ITC for oral ulcer-related outcomes in RCTs.

In the second step of the similarity assessment (Figure 3), eight RCTs reporting on seven active treatments were evaluated.34,35,39,40,44–47 The following BD-associated oral ulcer-related outcomes were reported for multiple treatments: i) number of oral ulcers; ii) resolution of oral ulcers (ie, patients who were oral ulcer-free); and iii) healing time of oral ulcers. Not all eight RCTs reported on each of the three outcomes.

In the third step of the similarity assessment, we separately assessed whether an ITC would be feasible for each of the commonly reported oral ulcer-related efficacy outcomes.

Number of Oral Ulcers

Seven RCTs evaluating six active treatments reported the number of oral ulcers (Table 2).34,35,39,40,44–46 Four RCTs were not comparable to the other three RCTs due to differences in outcome reporting methods. Differences in outcome reporting methods included reporting any vs only new oral ulcers and reporting the sum of oral ulcers over the entire study period vs at specified time points.

Table 2

Comparison of Outcome Reporting Methods for Oral Ulcers: Number, Resolution, and Healing Time

Treatment Assessed (vs Placebo)	Citation	Number of Oral Ulcers	Comparability
		Outcome Reporting Method
Colchicinea Colchicine + benzathine penicillina	Calguneri 199639	Number and frequency of oral ulcers before/after treatment; definitions of number, frequency, and before/after were not defined	X
Colchicine	Yurdakul 200140	Mean number of oral ulcers during the 24-month study period	X
Thalidomide 100 mg/day Thalidomide 300 mg/day	Hamuryudan 199845	Number of oral ulcers reported as combined data obtained at study visits and from reports by patients on occurrence of lesions between study visits; only new oral ulcers were reported	X
Interferon-α-2c hydrogel	Hamuryudan 199144	Mean sum of oral ulcers observed over all visits during the 24-week study period	X
Apremilast	Hatemi 201534	Mean number of oral ulcers at visit	✓
Apremilast	Hatemi 201935	Least-squares mean number of oral ulcers at visit	✓
Etanercept	Melikoğlu 200546	Mean number of oral ulcers at visit	✓
Treatment Assessed(vs Placebo)	Citation	Resolution of Oral Ulcers	Comparability
		Outcome Reporting Method
Thalidomide 100 mg/d Thalidomide 300 mg/d	Hamuryudan 199845	Complete response defined as absence over the 24-week study period of oral or genital ulcers of any size at and between visits	X
Apremilast	Hatemi 201534	Complete response defined as the percentage of patients with no oral ulcers	✓
Apremilast	Hatemi 201935	Complete response defined as the percentage of patients with no oral ulcers	✓
Etanercept	Melikoğlu 200546	Percentage of patients free of oral ulcers	✓
Azathioprine	Yazici 199047	Percentage of patients with oral ulcers	✓
Treatment Assessed(vs Placebo)	Citation	Healing Time for Oral Ulcers	Comparability
		Outcome Reporting Method
Apremilast	Hatemi 201935	Time to resolution of oral ulcers (ie, no oral ulcers) in weeks	X
Colchicinea Colchicine + benzathine penicillina	Calguneri 199639	Healing time in days of oral ulcers determined for each episode	X
Colchicine	Yurdakul 200140	Distribution of time to first occurrence of sustained absence of oral ulceration	X

Note: aNot a placebo-controlled trial.

Of the four RCTs, one trial evaluating colchicine and colchicine + benzathine penicillin reported the number of oral ulcers and the frequency of oral ulcers before and after treatment.39 The definitions of the number and frequency of oral ulcers, the definitions of before and after treatment, and the measure (eg, mean vs median) for the number of oral ulcers were unclear. Another RCT evaluating colchicine reported the mean number of oral ulcers during the entire 24-month study period; however, results at specific interim time points were not reported.40 A third RCT evaluating thalidomide reported the number of new oral ulcers as combined data obtained at study visits and from reports by patients on the occurrence of lesions between study visits.45 The fourth RCT evaluating interferon-α-2c hydrogel was excluded because it reported the mean sum of oral ulcers observed over all visits during the 24-week study period.44 Results at interim time points were not reported.

Three of the seven RCTs (evaluating two active treatments) reporting the number of oral ulcers had comparable outcome reporting methods with regard to the outcome metric and reporting interval (Table 2). Two multi-country RCTs evaluating apremilast34,35 and one RCT in Turkey evaluating etanercept46 reported the mean number of oral ulcers at specified time points, and all three RCTs reported the mean number at Week 4. However, the two apremilast RCTs34,35 had dissimilar baseline patient demographics and disease characteristics compared with the etanercept RCT46 (Table 3). First, both of the multi-country apremilast RCTs required the presence of at least two oral ulcers as an inclusion criterion, whereas the Turkish etanercept RCT required mucocutaneous disease and/or arthritis and a positive pathergy test.34,35,46 As such, it is unclear how many patients in the etanercept RCT had oral ulcers, while all patients in the apremilast RCTs had oral ulcers; the mean number of oral ulcers per patient at baseline was lower in the etanercept RCT (placebo group=1.8 and etanercept group=2.0)46 than in the apremilast RCTs (Phase 2 RCT: placebo group=3.1 and apremilast group=3.2; Phase 3 RCT “RELIEF”: placebo group=3.9 and apremilast group=4.2).34,35 Second, patients in the etanercept RCT had shorter disease duration compared with the apremilast RCTs (~3 years vs ~6 years). Third, the etanercept RCT allowed patients to use topical/oral corticosteroids and azathioprine for severe disease (eg, presence of deep vein thrombosis or painful arthritis), whereas the apremilast RCTs did not permit the use corticosteroids during the double-blind phase of the RCTs. Lastly, the etanercept RCT46 included only male patients (N=40; n=20 in the active arm), whereas the apremilast RCTs34,35 included male and female patients and did not report data by sex (ie, apremilast Phase 2 RCT comprised 31% male [34/111] and 69% female [77/111], and the apremilast Phase 3 RCT comprised 39% male [80/207] and 61% female [127/207]).34,35 After following the step-wise approach for the similarity assessment, based on these differences, it was determined that an ITC would not be feasible for the number of oral ulcers.

Table 3

Comparison of Trial Characteristics, Baseline Patient Demographics, and Baseline Patient Clinical Characteristics Among Trials Assessing the Mean Number of Oral Ulcers

Trial Characteristics	Melikoğlu 200546		Hatemi 201534		Hatemi 201935
Country	Turkey		Turkey, United States		Multi-Country
Manifestations required for inclusion	Mucocutaneous disease and/or arthritis + positive pathergy test		≥2 oral ulcers		≥2 oral ulcers
Major organ involvement	None permitted		None permitted		None permitted
Have joint involvement/arthritis	NR but allowed		NR but allowed		NR but allowed
Concomitant treatment use during trial	Corticosteroids and azathioprine were allowed for patients with relatively severe diseasea during the entire study period		None related to BD were allowed in placebo-controlled period		Colchicine, glucocorticoids, immunosuppressants (including azathioprine), andbiologics were not permitted in placebo-controlled period
Baseline Patient Characteristics	Melikoğlu 200546		Hatemi 201534		Hatemi 201935
Treatment	Etanercept	Placebo	Apremilast	Placebo	Apremilast	Placebo
N	20	20	55	56	104	103
Oral ulcers at baseline, %	NR	NR	100	100	100	100
Age, mean (SD), years	28.5 (5.3)	30.8 (6.2)	34.3 (NR)	34.7 (NR)	39.4 (NR)	40.6 (NR)
Male, %	100	100	29	32	38.5	38.8
Disease duration, mean (SD), years	2.8 (2.6)	4.3 (3.2)	4.92 (NR)	5.72 (NR)	6.7 (NR)	6.9 (NR)
Baseline number of oral ulcers, mean (SD)	2.0 (1.3)	1.8 (1.6)	3.2 (2.0)	3.1 (1.3)	4.2 (NR)	3.9 (NR)
Colchicine use prior to trial, %	55	35	NR		50	54
Concomitant treatment use during trial, %
Colchicine	0	0	NR		0	0
Azathioprine	5	5			7.7	5.8
Topical corticosteroids	0	20			0	0
Oral corticosteroids (prednisolone)	10	10			0	0
Indomethacin	5	0			NR	NR
Paracetamol	5	0			NR	NR
Naproxen	0	5			20.5	16.5
Ibuprofen	NR	NR			8.7	8.7

Notes: aPatients who had relatively severe disease include two patients with recent exacerbation of deep vein thrombosis (one patient each in etanercept and placebo arm) and one patient in the placebo arm with painful arthritis.

Abbreviations: BD, Behçet’s disease; N, number; NR, not reported; SD, standard deviation.

Oral Ulcer Resolution

Five RCTs evaluating four active treatments reported the resolution of oral ulcers (Table 2).34,35,45–47 One RCT evaluating thalidomide was not comparable to the other four RCTs due to differences in outcome reporting method; the complete response was defined as the absence of oral or genital ulcers of any size at and between visits over a 24-week period, and results were not reported by ulcer type.45 The remaining four RCTs assessed the absence of oral ulcers only and had comparable outcome reporting methods to discern the percentage of patients who were oral ulcer-free. Two RCTs evaluating apremilast34,35 defined complete response as the percentage of patients who were oral ulcer-free; one RCT evaluating etanercept46 reported the percentage of patients who were oral ulcer-free, and one RCT evaluating azathioprine47 reported the percentage of patients who had oral ulcers. Therefore, the percentage of patients who were oral ulcer-free could be calculated. Although these four RCTs had similar outcome reporting methods to discern the percentage of patients who were oral ulcer-free, they were not comparable due to differences in time reporting. The two RCTs evaluating apremilast34,35 reported results at week 12, the RCT evaluating etanercept46 reported results at week 4, and the RCT evaluating azathioprine47 reported results at month 24. After following the step-wise approach for the similarity assessment, it was determined that an ITC is not feasible for the resolution of oral ulcers.

Healing Time of Oral Ulcers

Three RCTs evaluating three active treatments reported the healing time for oral ulcers (Table 2).35,39,40 However, these RCTs were not comparable due to differences and lack of clarity in outcome reporting methods, specifically whether healing time referred to the time needed to be oral ulcer-free or the duration of an oral ulcer. An RCT evaluating apremilast reported the time to resolution of oral ulcers (ie, oral ulcer-free) in weeks.35 An RCT evaluating colchicine and colchicine + benzathine penicillin reported the healing time in days for oral ulcers for each attack, but did not clarify whether the healing time was the duration of individual oral ulcers or all oral ulcers; thus, this RCT could not be compared with the apremilast RCT.39 Lastly, an RCT evaluating colchicine reported the distribution of time to the first occurrence of the sustained absence of oral ulceration.40 The probability of complete resolution of oral ulcers was reported, but not the time it took for patients to become oral ulcer-free; thus, this RCT could not be compared with the apremilast RCT. After following the step-wise approach for the similarity assessment, it was determined that an ITC is not feasible for the healing time of oral ulcers.

Summary of Results

Among the 14 RCTs identified in the SLR, three oral ulcer-related efficacy outcomes (ie, number of oral ulcers, resolution of oral ulcers, and healing time for oral ulcers) were identified as possible oral ulcer-related outcomes to assess in an ITC. After completion of the similarity assessment, it was determined that a robust ITC was infeasible for the three oral ulcer-related outcomes due to heterogeneity in outcomes reporting, study design, and/or patient characteristics. More broadly, the results demonstrate that standardized measures for oral ulcer outcomes and comparative research for oral ulcer treatments are lacking.

Discussion

Comparative effectiveness is an important component of regulatory and reimbursement decision-making. ITCs in particular are commonly used in HTAs to provide comparative evidence, especially in the absence of direct evidence from head-to-head trials; ITCs are also important in providing robust estimates that combine direct and indirect evidence when head-to-head data are available.29,30 However, comparative effectiveness research is often challenging to conduct and therefore unavailable, particularly in rare diseases such as BD, and in the clinical trial setting.48 We performed a comparative SLR and similarity assessment to determine the feasibility of an ITC of the EULAR-recommended treatments for BD-associated oral ulcers. Results demonstrated that, among the 14 RCTs identified in our similarity assessment, a robust ITC was not feasible based on heterogeneous outcomes reporting, study design, and baseline patient demographics and clinical characteristics. Therefore, our results provide an additional comparative analysis to consider along with findings from the SLR informing the 2018 EULAR recommendations for mucocutaneous involvement in BD.21

Although a robust indirect comparison with apremilast and treatments used off-label for BD-associated oral ulcers would provide additional insight for healthcare decision-makers, our study results demonstrate that this is not possible without additional data from RCTs specifically designed to support such comparisons (ie, data which are currently unavailable). The inability to perform such an assessment dramatically limits the comparative-effectiveness and cost-effectiveness evaluations of various treatments for BD-associated oral ulcers. Furthermore, the dissimilarities found in this study highlight the lack of standardized measures in BD overall and for BD-associated oral ulcers specifically (eg, disease activity, severity). The Outcome Measures in Rheumatology (OMERACT) Behçet’s Syndrome Working Group also identified the variability and lack of standardized definitions of outcomes across trials evaluating treatments in BD as an area of unmet need.49,50 The OMERACT Behçet’s Syndrome Working Group proposed a main core set of outcomes that evaluate all organ systems and should be used in all trials of BD, as well as organ-specific subdomains that can be used in trials focusing on a particular organ system (eg, for assessing oral ulcer-related efficacy).49,50 In 2016, the OMERACT Behçet’s Syndrome Working Group suggested that the most important measures related to oral ulcers included oral ulcer number, duration, and associated pain.49 Standardized definitions would ideally be adopted for frequently used outcomes to evaluate oral ulcer-related efficacy, such as response, relapse, and remission.49 Furthermore, patients expressed a need to assess the effect of oral ulcers on difficulty with household tasks, stress, and issues in relationships with partners.49 A reliable and validated patient-reported outcome measure that incorporates domains endorsed by patients and physicians would also be valuable.49 Most of the EULAR-recommended therapies for oral ulcers were not robustly nor recently studied among patients with BD-associated oral ulcers. Similarly, no validated patient-reported outcome instrument as suggested by OMERACT currently exists.

The RCTs identified in our literature search had several limitations that ultimately contributed to the infeasibility of an ITC, including lack of clarity in study designs, particularly in ambiguous inclusion criteria; baseline characteristics that were not fully reported; and outcomes and time points that were often not clearly defined. In addition, only the apremilast RCTs included patients from more than one country,34,35 and several RCTs had relatively short study durations (eg, 4 weeks) that may not sufficiently predict patient experience on treatment intended to be used as chronic therapy (). Finally, some studies had very small sample sizes (eg, the etanercept RCT from Turkey with 20 patients in the active, non-placebo arm); if undertaken, an ITC performed on small sample sizes would lead to imprecise effect estimates with wide confidence intervals and inconclusive results.51

There are significant unmet needs in patients with BD. Real-world evidence further suggests that approximately 10% of patients with BD do not receive treatment,8 possibly due to concerns about treatment (ie, actual or potential ineffectiveness of therapy, cost of treatment, and/or side effects).52 Oral ulcers are the most common manifestation among all patients with BD,8 yet evidence suggests the treatments commonly prescribed for patients with BD to address oral ulcers (eg, colchicine, corticosteroids) are used off-label, indicated for short-term use, have non-specific actions, and/or temporarily relieve symptoms.1,8,25 Many of the treatments for BD are also associated with serious safety concerns, for example, long-term use of topical or oral corticosteroids with several serious side effects,25 short-term use of oral corticosteroids with sepsis, venous thromboembolism, and fracture,26 and immunosuppressants with serious adverse events.25

Since the publication of the EULAR guidance in 2018,1 the only new RCT evaluating BD-associated oral ulcers is the multi-country Phase 3 RCT RELIEF study assessing apremilast for the treatment of active oral ulcers in patients with BD.35 Results of the Phase 3 study are consistent with the Phase 2 RCT in which patients receiving apremilast experienced a significant and sustained reduction in the number of oral ulcers, pain from oral ulcers, and disease activity, as well as a significant and sustained improvement in QoL.34,35 The safety profile of apremilast in RELIEF was consistent with other currently approved indications (ie, chronic plaque psoriasis and psoriatic arthritis), with approximately 509,000 patients treated as of March 31, 2020, since its first approval in 2014.23,24 Apremilast has subsequently gained regulatory approval for the treatment of oral ulcers in BD in several countries, including the United States, Japan, Switzerland, and the European Union.22–24

Acknowledging the practical challenges with conducting assessments focused on this rare disease, future comparative studies and real-world studies assessing the efficacy, safety, and humanistic value of treatments used for BD-associated oral ulcers across longer time frames would be valuable in informing the comparative effectiveness of available therapies. The consistent use of a recommended set of relevant endpoints and reporting intervals to consider for implementation for each manifestation of BD, including oral ulcers, given the near-universal prevalence8 (such as the Core Domain Set proposed by the OMERACT Behçet’s Syndrome Working Group50), would help to facilitate consistency in research, foster comparability of treatments, and ultimately help to guide healthcare decision-making.

This study has some limitations. First, the research question is primarily focused on EULAR-recommended treatments as opposed to any treatment for BD that might be used off label for BD-associated oral ulcers; however, we believe EULAR to be an important foundation for the treatment set evaluated, and no evidence from the SLR performed for this study suggested that non-recommended agents would be sufficiently similar and appropriate for an ITC. Second, the similarity assessment relies on qualitative comparison using expert feedback, given that there is no recommended quantitative method for determining similarity; however, our approach utilized a rigorous step-wise process recommended by ISPOR, and criteria for RCT exclusion were objective and defined a priori. Third, the evidence was limited to RCTs and no observational data were included; however, ITC methodology typically relies on RCT evidence to ensure a robust analysis.

Conclusions

Results of this evaluation show the clinical evidence for EULAR-recommended treatments for oral ulcers associated with BD cannot be robustly (ie, quantitatively) compared; therefore, comparative effectiveness insight via an ITC is currently not possible, thereby also limiting cost-effectiveness insights as a result. Where formal assessments are not possible, healthcare decision-makers, including HTA bodies, payers, and clinicians, must continue to base treatment decisions on the extent and strength of available evidence (eg, robust RCTs), clinical guidelines, real-world experience, and patient considerations. Consistent use of standardized measures for oral ulcer outcomes (such as those proposed by the OMERACT Behçet’s Syndrome Working Group50) across clinical trials is needed to facilitate comparative research of BD-associated oral ulcer treatments.