Impact of Changes of the 2020 Consensus Definitions of Invasive Aspergillosis on Clinical Trial Design: Unintended Consequences for Prevention Trials?

BACKGROUND: Consensus definitions for the diagnosis of invasive fungal diseases (IFDs) were updated in 2020 to increase the certainty of IFD for inclusion in clinical trials, for instance by increasing biomarker cutoff limits to define positivity. To date, there is a paucity of data as to the impact of the revised definitions on clinical trials.
METHODS: In this study, we sought to determine the impact of the new definitions on classifying invasive aspergillosis (IA), the most common invasive mold disease in immunocompromised patients. We reclassified 226 proven and probable IA cases plus 139 possible IFD cases in the Aspergillus Technology Consortium (AsTeC) and in an antifungal prophylaxis trial (BMT CTN 0101) using the new criteria.
RESULTS: Fewer cases met the more stringent diagnostic 2020 criteria after applying the reclassification criteria to define probable IA. Of 188 evaluable probable cases, 41 (22%) were reclassified to 40 possible IA and 1 probable IFD. Reclassification to possible IFD occurred in 22% of hematologic malignancy (HM) patients, 29% of hematopoietic cell transplant (HCT) patients, and in no lung transplant (LT) patients. Date of diagnosis was established a median (range) of 3 (1-105) days later in 15% of probable IA cases using the new criteria. Applying the new definitions to the BMT CTN 0101 trial, the power to detect the same odds ratio decreased substantially.
CONCLUSIONS: The updated IA consensus definitions may impact future trial designs, especially for antifungal prophylaxis studies.

Invasive fungal diseases (IFDs), especially invasive aspergillosis (IA), are among the most serious infections confronting immunocompromised patients, with high rates of morbidity and mortality. The diagnosis of IA remains challenging. Historically, invasive procedures to establish tissue diagnosis have been necessary, but biopsies of infected sites are fraught with the risk of serious complications and are insensitive [1]. Today, fungal biomarker blood assays are commonly used to establish the diagnosis [2].

Clinical trials to evaluate antifungal therapy have been plagued by uncertainties as to whether participants truly have the infection being treated. Such uncertainty has hampered the ability to establish guidelines for the best therapy to optimize treatment outcomes. For more than 2 decades, the European Organization for Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) has established standardized criteria that can guide clinical research trials [3-5]. With scientific advances, diagnostic criteria have evolved over time. The 2002 EORTC/MSG diagnostic criteria were revised in 2008 and more recently in 2020 [4, 5].

There are 2 substantive changes in the mycologic criteria for IA in the 2020 EORTC/MSG update: Higher thresholds of galactomannan (GM) positivity are required to establish probable IA, and, for the first time, Aspergillus polymerase chain reaction (PCR) assays are accepted as a mycologic criterion. The GM index threshold increased from the regulatory cleared positivity of 0.5–1.0 for a single serum or BAL specimen or a serum index of 0.7 plus a BAL sample index of ≥0.8. The GM assay is widely used worldwide, while some centers, particularly in Europe, rely on PCR assays. PCR assays are not widely used in the United States and other countries. This study was undertaken to determine the impact of the change in GM positivity of the new definitions on classifying IA.

METHODS

Objectives

We set out to determine the impact of the revised definitions on (1) the reclassification of cases, (2) effect on different key patient groups, (3) the date of diagnosis and its effect on survival rates, and (4) the magnitude of the effect on IA rates in antifungal prophylaxis trials.

Ethical Approvals and Patient Consent

This study describes secondary analyses of earlier studies that were conducted under the auspices of the University of Florida Institutional Review Board (IRB) and in accordance with the ethical standards of the Helsinki Declaration. The study was approved by local ethical committees at participating sites. All patients (or their designee) signed written informed consents, and patient information has been anonymized.

IA Cases

Cases were drawn from 2 data sets of patients treated in the United States: the National Institute of Allergy and Infectious Disease (NIAID)–funded Aspergillus Technology Consortium (AsTeC) and the National Heart, Lung, and Blood Institute and National Cancer Institute (NHLBI/NCI)–funded Blood and Marrow Transplant Clinical Trials Network (BMT CTN) 0101 clinical trial.

AsTeC Biorepository

Patients were scored as having no, possible, probable, or proven IFD using the EORTC/MSG 2008 diagnostic criteria [4]. A total of 872 patients were enrolled (282 HM, 432 HCT, 132 LT, and 26 other diseases) between 2008 and 2013. A total of 188 patients developed IA (27 proven and 161 probable) (Supplementary Table 1), 89 patients developed other types of IFD (65 proven and 24 probable), and 89 patients developed possible IFD. Illustrative publications [6-13] describe the study strategy, patient selection, data collection, sample procedures, and several uses of these specimens.

BMT CTN 0101Trial

This was a randomized, double-blind, multicenter study comparing fluconazole with voriconazole for the prevention of IFD in allogeneic HCT recipients enrolled at 35 US centers participating in the BMT CTN 0101 trial (NCT00075803) [14]. A total of 600 patients were randomized (voriconazole, n = 305; fluconazole, n = 295) from 2003 to 2006. Infections were scored using the 2002 MSG/EORTC criteria, with an additional criterion of “presumptive” IFD, which was defined as meeting the host and clinical criteria (including radiographic abnormalities) but lacking mycologic confirmation and with no alternative etiologies identified by extensive evaluation including bronchoscopy [3]. At 1 year post-HCT, 79 patients had developed IFDs (28 proven, 33 probable, and 18 presumptive IFDs). Aspergillus spp. accounted for 38 (48%) IFDs at 365 days. In addition, there were 75 patients with possible IFDs (Supplementary Table 1). For this study, the 18 presumptive IFD cases were placed in the possible IFD category as per EORTC/MSG criteria.

Definitions

The day of diagnosis was defined as the first day that all components of the diagnostic criteria were met. Survival from IFD was calculated at 6 and 12 weeks after diagnosis. In the BMT CTN 0101 trial, a GM optical density index value ≥0.5 in serum or BAL was considered positive for the GM index. Although the 2002 criteria stipulated the need for GM to be positive in ≥2 blood samples to eliminate false-positive results, subsequent studies have shown that false positivity could be eliminated by repeating the assay on the same blood sample (Wheat, personal communication); accordingly, samples that were repeat positive were used to satisfy this GM criterion for all AsTeC and BMT CTN 0101 samples rather than requiring collection of a second sample: 15 of 51 cases in AsTeC and 8 of 19 cases in BMT CTN 0101 met the criterion on the basis of repeat positive testing of the same sample rather than 2 separate samples. The 2002 definitions [3] allowed host factors plus either mycologic or clinical factors to satisfy possible IFD, but in the 2008 definitions [4], possible IFD required both host and clinical factors to be present in the absence of mycologic criteria. Thus, the number of possible IFD cases in the BMT CTN 0101 was reduced from 75 to 50 using the 2008 definitions for this study. Supplementary Table 3 summarizes the substantive changes in host, clinical, and mycologic criteria in IA definitions between 2002, 2008, and 2020.

Study Procedures

The cases were scored for IFDs using the 2008 and 2020 sets of MSG/EORTC definitions [4, 5]. In the BMT CTN 0101 trial, 8 cases were reported as having a positive GM (≥0.5) without quantification (7 serum and 1 BAL). We retested 4 available samples drawn within 3 days of the original sample for reclassification, but 4 cases were not able to be reclassified and were excluded from the analysis.

BMT CTN 0101 Analysis

The original 0101 trial targeted at least 80% power to detect an increase in fungal-free survival (FFS) at 6 months from 50% to 62% (a 12% absolute increase or delta, or equivalently an odds ratio [OR] of 1.632). The trial results (2002 definition) indicated FFS at 6 months, which was much higher (77% overall). The revised 2020 definitions indicated an FFS rate at 6 months of 79% overall. We recomputed power to detect a range of delta (absolute differences in 6-month FFS) for 3 scenarios of different baseline 6-month FFS rates: original trial assumption (50%), 2002 definition (77%), and 2020 definition (79%).

RESULTS

Our study population consisted of 226 proven and probable IA cases in the AsTeC and BMT CTN 0101 trial (Supplementary Table 1). Most of the cases were in the probable IA category. Supplementary Table 2 provides the characteristics of the patients with these infections.

Tables 1 and 2 demonstrate that fewer cases meeting the EORTC/MSG 2008 criteria met the more stringent 2020 diagnostic criteria. As shown in Figure 1, no cases with proven IA changed. However, 41 of the 188 (22%) evaluable probable cases were reclassified (40 possible IA, 1 probable IFD). Using 2008 criteria, 108 of 161 (67%) cases in AsTeC and 22 of 31 (71%) cases in the BMT CTN 0101 trial met probable IA classification based on serum and/or BAL GM EIA tests that were positive at the ≥0.5 index. However, under the revised higher limits in the new 2020 definitions, 32 of the 108 (30%) AsTeC cases and 9 of the 31 (29%) BMT CTN 0101 cases no longer met probable IA criteria. Of the 32 reclassified AsTeC cases, 3 possible IFD cases also had concomitant proven IFDs (2 candidiasis and 1 mucormycosis). Those 3 cases were included within the “reclassified to possible IFD” group for this analysis due to the focus on IA. Among BMT CTN 0101 cases, 9 of 27 (33%) evaluable probable IA cases were reclassified; 8 cases were reclassified to possible IFD, and 1 case was reclassified to probable IFD. As previously noted, 4 cases were not able to be evaluated for reclassification.

Table 1.

Impact of Changes in EORTC/MSG on IA Definitions in AsTeC Cases and BMT CTN 0101 Cases Between the 2008 and 2020 Definitions; Reclassification of Cases Between 2008 and 2020

Type of IA	AsTeC	No. (%) Reclassified	BMT CTN 0101	No. (%) Reclassified	Totals	No. (%) Reclassified
Proven IA	27		7		34
Probable IA	161	32 (20)	31	9a (29)	192	41a (21)
Possible IFD	89		50		139
Totals	277		88		365

Abbreviations: AsTeC, Aspergillus Technology Consortium; BMT CTN, Blood and Marrow Transplant Clinical Trials Network; EORTC/MSG, European Organization for Research and Treatment of Cancer/Mycoses Study Group; IA, invasive aspergillosis; IFD, invasive fungal disease.

aIncluding 1 case reclassified from probable IA to probable IFD.

Table 2.

Impact of Changes in EORTC/MSG on IA Definitions in Breakdown Between Different Patient Groups; Distribution of Cases Between Different Patient Groups With Numbers Reclassified

Patient Group	IA Category	2008	2020	No. (%) Reclassified
HM	Proven	9	9
	Probable	72	56	16 (22)
	Possible	49	65
HCT	Proven	13	13
	Probable	97	69	28 (29)
	Possible	84	107
LT	Proven	6	6
	Probable	15	15	0 (0)
	Possible	4	4
Other	Proven	6	6
	Probable	8	7	1 (12)
	Possible	2	3
All	Proven	34	34
	Probable	192	148a	41a,b (21)
	Possible	139	179b

Abbreviations: EORTC/MSG, European Organization for Research and Treatment of Cancer/Mycoses Study Group; HCT, hematopoietic cell transplant; HM, hematologic malignancy; IA, invasive aspergillosis; LT, lung transplant.

aIncluding 1 case reclassified from probable IA to probable IFD.

bExcluding 4 cases that were unable to be reclassified.

Figure 1.

Recategorization of the 365 cases. Left bar: classification of cases by 2008 definitions. Right bar: reclassification by 2020 definitions. aIncludes 1 case reclassified from probable IA to probable IFD. Abbreviations: IA, invasive aspergillosis; IFD, invasive fungal disease.

The effect of reclassification was different in various patient groups: the percentage of probable IA cases reclassified to possible IFD was greatest in HM and HCT patients (22 and 29%, respectively), and there was no effect in LT patients (Tables 1 and 2). The type of antifungal prophylaxis was examined to determine if it influenced the likelihood of reclassification of cases. In the BMT CTN 0101 trial, at 6 months, 4 of 14 (28%) probable IA cases in the fluconazole arm compared with 3 of 5 (60%) cases in the voriconazole arm (P = NS) required reclassification (Tables 3 and 4). For the entire 1-year follow-up (with most patients having completed antifungal prophylaxis at day 100), the difference was comparable with 5 of 16 (31%) in the fluconazole arm vs 4 of 11 (36%) in the voriconazole arm (P = NS) being reclassified.

Table 3.

Proven/Probable/Presumptive/Possible IFD Cases in BMT CTN 0101 Trial at 180 Days Used in Recalculating 6-Month FFS (Note: Both IA and Non-IA IFD Cases Were Used for This Analysis)

Types of IFD	IFD Cases Scored by Modified 2002 Definitions	IFD Cases Scored by 2008 Definitions	IFD Cases Scored by 2020 Definitions
Proven	14	14	14
Probable	24	24	14a
Presumptive	17b	NA	NA
Possible	70	43	49

Abbreviations: BMT CTN, Blood and Marrow Transplant Clinical Trials Network; EORTC/MSG, European Organization for Research and Treatment of Cancer/Mycoses Study Group; IA, invasive aspergillosis; IFD, invasive fungal disease.

aAmong the 10 fewer probable IFD cases (24 vs 14), 6 were reclassified to possible IFD and had infection date within 180 days of transplant; 4 cases were never reclassified because data were insufficient.

bThe presumptive IFD cases in the modified 2002 definitions were moved to possible IFD for reclassification using the 2008 and 2020 definitions in this analysis.

Table 4.

Numbers of Probable IA Cases According to Antifungal Prophylaxis Cohort With Percent Reclassified by the 2020 Definitions

Numbers of Patients With Invasive Fungal Infection Through Day 180 and Day 365
	0101 Study Definition				2020 Classification (% Probable IA Reclassified)
	(Modified 2002 Classification)
IFI Category	Days 0–180		Days 0–365		Days 0–180				Days 0–365
	FLU	VORI	FLU	VORI	FLU		VORI		FLU		VORI
					Unchanged	Reclassifieda	Unchanged	Reclassifieda	Unchanged	Reclassifieda	Unchanged	Reclassifieda
Proven
Aspergillus	3	0	5	2	3		0		5		2
Non-Aspergillus IFDb,c	6	5	8	13	6		5		8		13
Subtotal	9	5	13	15	9		5		13		15
Probable
Aspergillus	14	9	16	15d	10	4e (28)	2f	3f (60)	11	5e (31)	7f	4f (36)
Non-Aspergillus IFDg	1	0	2	0	1	0	0	0	2	0	0	0
Presumptive	9	8	10	8	h	h	h	h	h	h	h	h
Subtotal (probable/presumptive)	24	17	28	23	11	4	2f	3f	13	5	7f	4f
Total IFIs
Proven/probable/presumptive	33	22	41	38	20	4	7f	3f	26	5	22f	4f

Abbreviations: BMT CTN, Blood and Marrow Transplant Clinical Trials Network; FLU, fluconazole; GM, galactomannan; IA, invasive aspergillosis; IFD, invasive fungal disease; IFI, invasive fungal infection; VORI, voriconazole.

aMost of the reclassified cases were due to serum samples; there were only 4 BAL samples. For those on antecedent antifungal drugs for at least 72 hours, fluconazole was the most common; few were on voriconazole, other antimould antifungals, or echinocandins.

b Chaetomium, Pseudallescheria boydii, Alternaria, and hyphae invading tissue with negative culture.

cZygomycetes followed by Candida krusei, Zygomycetes followed by Candida glabrata, and Candida albicans followed by Zygomycetes.

dIncludes 1 mixed infection due to Aspergillus and Zygomycetes.

eIncludes 3 cases reclassified as possible IFD and 1 case as probable IFD.

fExcludes 4 cases that did not have quantitative GM values to determine if they met new criteria.

g Paecilomyces/Nocardia and Paecilomyces/Nocardia and Pneumocystis jiroveci.

hPresumptive cases in the BMT CTN 0101 modified definition were reclassified to possible IFD in the 2008 and 2020 classification.

The date the EORTC/MSG diagnostic criteria were satisfied for probable IA using the 2020 definitions was delayed compared with using 2008 definitions (median delay [range], 3 [1-105] days) in 22 of 147 (15%) evaluable patients (Supplementary Figure 1). Combining these 22 cases in which the diagnosis was delayed with the 40 cases (Tables 1 and 2) that would have been treated for IFD using regulatory cleared biomarker positivity but which never met the research end point, 62 of 188 (33%) probable IA patients based on 2008 criteria would have had antifungal treatment delayed or not started if the 2020 criteria had been used to initiate antifungal treatment.

The survival rates of probable IA were 66% and 65% at 6 weeks and 42% and 43% at 12 weeks, respectively, for cases scored using the 2008 criteria and retained as probable IA using the 2020 criteria (Supplementary Table 4). Survival rates of probable IA nonreclassified cases did not significantly differ from those reclassified to possible IFD at 6 weeks (65% vs 75%; P = .82) or 12 weeks (43% vs 46%; P = NS).

The original 0101 trial targeted at least 80% power to detect an increase in FFS at 6 months from 50% to 62% (a 12% absolute increase or delta, or equivalently an OR of 1.632). The trial results (2002 definition) indicated FFS rates at 6 months that were much higher (77% overall). The revised 2020 definitions indicated FFS at 6 months of 79% overall. We recomputed power to detect a range of delta (absolute differences in 6-month FFS) for 3 scenarios of different baseline 6-month FFS: original trial assumption (50%), 2002 definition (77%), and 2020 definition (79%) (Table 5). Note that power increases with baseline FFS further away from 50%. However, this is power to detect the same absolute difference in FFS. This comes with a caveat that targeting a particular absolute difference in FFS when FFS is 50% is much different than targeting the same absolute difference in FFS when FFS is 77%. The OR for 6-month FFS was 1.632 for the original 0101 trial design (corresponding to an FFS of 62% vs 50%). Using the same OR, the power to detect the same OR when the baseline FFS shifts to 77% or 79% decreases to 65% and 62%, respectively.

Table 5.

BMT CTN 0101 Power Calculations Using Various Estimates of Fungal-Free Survival

Delta (Absolute Differences in 6-Month FFS)	Original Trial (50% Baseline FFS)	2002 Definition (77% Baseline FFS)	2020 Definition (79% Baseline FFS)
9%	60%	81%	85%
10%	69%	89%	92%
11%	77%	95%	96%
12%	84%	98%	99%
OR, 1.632 (absolute difference dependent on baseline FFS)	84%	65%	62%

Abbreviations: BMT CTN, Blood and Marrow Transplant Clinical Trials Network; FFS, fungal-free survival; OR, odds ratio.

DISCUSSION

Defining IFD is difficult and continually evolving, prompting definitions to be established and updated by consensus groups, such as the EORTC/MSG. These definitions are intended principally for antifungal treatment trials to ensure that individuals enrolled in clinical trials truly have IFD and to standardize reporting and enable cross-trial comparisons of outcomes. The new definitions will have important implications for antifungal trials, as setting goalposts in mycologic criteria more conservatively leads to people with higher fungal burden. Also, prophylaxis trials stress sensitivity/specificity characteristics of testing differently than testing to confirm a diagnosis.

One implication is that, as expected, fewer patients with probable IA meet the more stringent EORTC/MSG 2020 criteria. The 22% and 29% reduction in probable IA cases in HM and HCT patients, respectively, who meet the new definitions will necessitate screening more candidates for antifungal treatment trials in HM and HCT patients to achieve comparable absolute risk reductions of IFDs. In contrast, there was no change in the LT cohort as all subjects (n = 15) recovered Aspergillus spp. from culture. The reasons for this are likely related to the lack of routine use of the GM assay in solid organ transplant patients owing to its reduced sensitivity in serum and specificity in BAL compared with its performance in HM populations [15].

A second implication is the divergence between the GM assay’s regulatory cleared threshold for GM positivity and the new research criteria, which is especially problematic for antifungal prophylaxis trials. Most clinicians will not delay start of antifungal therapy for suspected IFD in order to wait for the more stringent 2020 GM (index = 1.0) diagnostic criteria to be met because accepted practice uses the GM assay’s licensed threshold (index = 0.5). The frequent use of “empiric” antifungal therapy for “possible” IA will also need to be taken into consideration in the trial design. As noted, 62 of 188 (33%) patients with probable IFD based on the regulatory approved GM index criterion would have had treatment delayed or not given if the research criteria had been used to initiate antifungal treatment.

A third implication is how to evaluate possible IFD cases. Possible IA cases are typically not included in antifungal treatment trials, due to diagnostic uncertainty. In the past, most antifungal prophylaxis trials excluded possible IFD cases in the primary end point. This will need to be rethought as many possible IA cases that meet the regulatory GM criteria but not the 2020 criteria will be treated with empiric antifungal therapy, and such cases treated empirically will confound the study results.

There are several ways to address this quandary. One would be to embrace inclusion of possible IFD cases into the study primary end point. This, of course, has the downside of increasing the uncertainty of how many true IFD cases one is actually preventing. A second would be to adopt the more stringent criteria but to increase sample size to accommodate the lower event rate for probable or proven IFDs. This would still not address the confounding effect of all the empiric antifungal therapy courses given by clinicians for positive GM tests using the regulatory criteria of positivity in accordance with accepted clinical practice and necessitate a substantially higher number of patient enrollments (and increase the cost of the trial). A third would be to retain the 2008 criteria (which align with the regularly cleared threshold of the GM assay) for prophylaxis trials. A fourth solution is to incorporate the use of empiric antifungal therapy into the end point. This has been done in the past, although it was also criticized due to the variability of clinician decision-making.

Aspergillus PCR assays were not used in the diagnostic assessment of these cases; it is possible that some of the cases reclassified from the 2008 definitions might have continued to meet the mycologic criteria for probable IA if PCR assays had been used. Enormous strides in identifying components of the assay that affect the performance and standardization of these components have improved reliability [16-19]; still, numerous centers continue to use in-house assays. Unfortunately, at present no commercial Aspergillus PCR has been approved in the United States, and usage is patchy worldwide. If the PCR assay is used in a clinical trial, consistency of use across all participating centers would be highly desirable.

A fourth implication is that it is possible, perhaps likely, that cases meeting the more stringent criteria are either more aggressive infections or more extensive infections that might be less responsive to antifungal therapy. Survival of patients with IA is generally assessed at 6 or 12 weeks in antifungal trials [20]. Although we did not find a survival difference between patients with probable IA scored by the 2008 criteria and meeting the more stringent 2020 criteria in survival at 6 or 12 weeks in this analysis, a difference could have been obscured as all patients were treated using the 2008 criteria. A difference, if there is one, could have implications for response estimates in trials that include possible IFD with probable and proven IFD.

A limitation of this analysis is that these cases were reclassified >10 years ago. It is possible that changes in transplant practice or other factors may have had impacts on fungal-free survival in substantive ways other than the reclassification system, which could also affect sample size collection.

In conclusion, the more stringent 2020 EORTC/MSG criteria provide greater certainty of diagnosis, an important consideration for treatment trials; however, these new criteria also pose new challenges for clinical trial design for IA, particularly for prevention studies, where end points should reasonably approximate clinical practice. The findings in this study suggest that several important considerations are affected by the new definitions: the intent of the trial (treatment vs prevention), the type of patient group being studied, the type of mycologic biomarker used in diagnostic assessment (GM vs PCR), divergences between licensed and research criteria for biomarker assays, and the estimates of the expected rates of infection and response and survival rates for power sample size calculations. Until the impact of the new criteria on IFD rates in antifungal prophylactic trials has been determined, we suggest classifying IFD by both the older and the new consensus criteria.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.

Click here for additional data file.