Identifying the optimal cutoff point for MGMT promoter methylation status in glioblastoma.

Aim: To define the optimal cutoff point for determining methylation status of O6-methylguanine-DNA methyltransferase (MGMT) by pyrosequencing in glioblastoma. Patients & methods: A retrospective study of 109 glioblastoma patients was performed to determine the optimal cutoff point for MGMT methylation status.
Results: Receiver operating characteristic (ROC) analysis revealed 21% as the optimal cutoff (sensitivity: 68%; specificity: 59%) for MGMT methylation corresponding with the highest likelihood ratio of 1.66 and accuracy of 0.65. Methylation status (hazard ratio: 0.453; 95% CI: 0.279-0.735; p = 0.001) was associated with better overall survival. The crude model indicated linearity between methylation percent and survival rate; an increase of 10% of methylation resulted in a reduction of risk of death by 20% (p = 0.004).
Conclusion: ROC analysis determined 21% as the optimal cutoff point for MGMT methylation status by pyrosequencing.

Glioblastoma (GB) is the most common malignant primary brain tumor in adults and accounts for the majority of gliomas [1]. The tumor is highly invasive and is associated with relatively poor prognosis and overall survival (OS), commonly less than two years with a 5.6% 5-year survival rate [2]. The standard treatment for GB patients includes maximal safe tumor resection, radiation with concurrent temozolomide (TMZ) followed by adjuvant TMZ therapy with or without Optune tumor treating field (TTField) therapy. The therapeutic efficacy of TMZ depends on its ability to methylate DNA at different sites, including N7-methylguanine, N3-methyladenine3 and O6-methylguanine (most cytotoxic lesion) which consequently leads to DNA damage by triggering the DNA mismatch repair (MMR) pathway and tumor cell death [3-6]. However, some tumor cells can reverse the effect of DNA damage by expressing and overexpressing the DNA-repair protein O6-methylguanine-DNA methyltransferase (MGMT). This protein removes the alkylation of the O6 position of guanine, thus reversing and neutralizing the cytotoxic effect of TMZ [5,6]. Epigenetic silencing by cytidine phosphate guanosine dinucleotides (CpG) island promoter methylation prevents the synthesis of MGMT enzyme and consequently improves the cytotoxic efficacy of TMZ. As a result, MGMT promoter methylation status is a valuable biomarker predicting the effect of TMZ in the treatment of GB, and is also an important prognostic factor [7].

Patients with methylated MGMT have improved OS compared with unmethylated MGMT patients [8]. MGMT methylation status serves as an important decision point for therapeutic choices, especially in the elderly or patients with poor performance status. The NOA-08 trial of the Neuro-oncology Working Group (NOA) of the German Cancer Society compared standard postsurgical involved-field radiation to TMZ 100 mg/m2 in a one-week-on/one-week-off schedule in patients >65 years old with anaplastic astrocytoma or glioblastoma. The NOA-08 study showed a longer event-free survival in patients with methylated MGMT who received TMZ versus standard radiation therapy and concluded that MGMT promoter methylation status is a useful biomarker for outcomes by treatment [9]. More recently, a randomized phase 3 trial conducted in Germany (the NOA-09 study) used MGMT methylation status to choose treatment and suggested using TMZ with lomustine/CCNU(1-(2-chloroethyl)-3-cyclohexyl-1-nitrosourea) for hypermethylated patients [10]. MGMT status, therefore, serves as an important tool for decision-making and is an accurate, standardized and widely available method for defining methylated versus unmethylated patients.

MGMT status can be determined at the DNA level with promoter methylation status, RNA level by reverse transcriptase PCR (RT-PCR) or protein level by immunohistochemistry (IHC). Many clinical trials have used methylation-specific polymerase chain reaction (MSP) and pyrosequencing (PSQ). The latter has been shown to have good prediction of survival, high sensitivity and reproducibility [11,12]. PSQ is the method of choice for MGMT promoter methylation analysis in routine clinical practice [13]. This sequencing method detects methylation levels of individual CpG sites and provides accurate quantitative information on the percentage of CpGs methylation [14]. However, the main unresolved issue is the lack of a clear optimal cutoff value for the percentage of methylation required to classify patients into MGMT methylated and MGMT unmethylated subgroups. Due to the lack of optimal cutoff point, clinicians have difficulty in deciding whether a patient would benefit from TMZ or not. Defining the optimal cutoff value helps clinicians to predict the survival benefit and response when TMZ is given as adjuvant therapy. The aim of this study was to determine the optimal cutoff point for MGMT promoter methylation status. The authors also examined whether there was linearity between methylation percentage and survival.

Methods

The authors performed a retrospective study of 109 patients diagnosed with GB from 2000 to 2018 at the West Cancer Center and Research Institute who had MGMT testing at the time of diagnosis. All patients received standard treatment for GB, including surgical resection (total resection, partial resection or biopsy alone) and radiation with concurrent TMZ, followed by adjuvant TMZ therapy with or without Optune® tumor-treating field therapy. CARIS® Life Sciences (Irving, TX) was used for MGMT methylation testing by pyrosequencing at the time of diagnosis. Pyrosequencing is a sequence-based detection technology that quantifies in real-time each added nucleotide to give the percent methylation at each CpG. The pyrosequencing is done on DNA extracted from paraffin-embedded tumor samples after bisulfite treatment and PCR amplification with primers specific for exon 1 of MGMT (GRCh37/hgI9-chr10:131,265,448–131,265,560). Methylation status of PCR amplified products was determined using the PyroMark system (Qiagen, Hilden, Germany) for analysis of 5 CpG sites (CpG 74–78) which are part of the differentially methylated region, and of DMR2 sites which are also one of the most influential sites for transcriptional silencing [15]. The demographic profile of the patient population is shown in Table 1. The study was approved by the Institutional Review Board. Patient consent was not required, as this was a retrospective study in which patient information was deidentified. Statistical consultation was obtained from the Department of Preventive Medicine at the institution.

Table 1.

Patient characteristics.

Variable	Patients (n)	Percentage (%)
Age range (years)	27–83 (mean age 62)	NA
Age ≥60	70	64.2
Age <60	39	35.8
Sex
Female	33	30.3
Male	76	69.7
Race
White	80	73.4
Black	29	26.6
ECOG score
≤2	77	70.6
≥3	32	29.4
Extent of surgical resection
Total resection	66	60.6
Partial resection	28	25.6
Biopsy only	15	13.8

ECOG: Eastern Cooperative Oncology Group; NA: Not available.

Statistical analysis

Receiver operating characteristic (ROC) curve analysis was used to identify an optimal cutoff point for MGMT promoter methylation. Mean percentage methylation of each patient was plotted on a ROC curve. The area under the curve (AUC), as well as the test accuracy, were calculated. The likelihood ratio (LR+) and hazard ratios (HR), sensitivity, specificity and accuracy for several cutoff points ranging from 9 to 25% were calculated. LR+ expresses the probability of a person who has the disease testing positive divided by the probability of a person who does not have the disease testing positive. Likelihood ratios are one of the best measures of diagnostic accuracy [16]. The higher the LR+ value is, the greater the information value for a diagnostic test. OS was defined from the date of diagnosis to the date of death or censored at the date of last follow-up. Progression-free survival (PFS) was defined from the start of the first treatment to the time of radiographic progression based on response assessment in neuro-oncology (RANO) criteria. Based on the determined optimal cutoff point, the authors compared OS for the methylated and unmethylated subgroups using the Kaplan–Meier method with log rank test. Multivariate analysis using the Cox regression model was used to identify factors that had a significant impact on OS. Those factors include MGMT methylation status, gender, age (≤60 vs >60 years), surgical resection status (total gross resection vs partial resection vs biopsy alone) and Eastern Cooperative Oncology Group scores (ECOG ≤2 or >2). The crude model was used to determine if there wa linearity between methylation percentage and survival rates. The R Statistical Computing and SPSS Statistics 26 software were used to conduct statistical analysis.

Results

A total of 109 patients with GB were included in the study. The mean age was 62 years (range: 27–83 years; Table 1). Seventy-six patients were male and 33 were female. Of 109 patients, 66 received gross total surgical resection (60.6%), 28 had partial surgical resection (25.6%) and 15 had biopsy only (13.8%). A total of 77 patients had an ECOG score equal to or less than 2 at the time of diagnosis (70.6%).

The ROC curve for representative cutoff values of MGMT methylation percentage was plotted (Figure 1). The optimal cutoff point was selected based on the highest LR+ and accuracy. In this dataset, a cutoff point of 21% was the optimal value with the highest LR+ (1.66) and accuracy (0.65) and a Youden index of 0.27 (Table 2). Table 2 shows the other cutoff points with high LR+ were at methylation percent of 11 and 15 (LR+: 1.66, and 1.63, respectively), but both had relatively lower accuracy, sensitivity and specificity. The most commonly used cutoff point of 9% gave a lower LR+ (1.62) and accuracy (0.59) compared with a cutoff point of 21%. According to this cutoff point, 45 patients were methylated and 64 were unmethylated. Kaplan–Meier plots for OS and PFS using MGMT promoter methylation status as threshold are shown in Figures 2 & 3. MGMT methylation status was significantly associated with better OS (HR: 0.453, 95% CI: 0.279–0.735, p = 0.001). Compared with unmethylated patients, methylated patients had a significantly better OS (20.4 months vs 13.3 months, p = 0.001) but not significantly improved PFS (7.4 months vs 6.8 months, p = 0.3; Figures 2 & 3).

Figure 1.

Receiver operating characteristic curve for representative cutoff levels of MGMT promoter methylation status.

MGMT promoter methylation levels were plotted with true positives on the vertical axis (sensitivity) and false positives (1-specificity) on the horizontal axis. Compared with other cutoff points between 9 and 25%, a cutoff point of 21% correlated with the highest likelihood ratio and accuracy.

Table 2.

Positive likelihood ratio, sensitivity, specificity and accuracy for different cutoff points.

MGMT methylation (%)	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy	LR+
9	0.56 (0.441–0.67)	0.66 (0.468–0.81)	0.59	1.62
11	0.57 (0.454–0.68)	0.66 (0.468–0.81)	0.60	1.66
13	0.58 (0.4665–0.69)	0.62 (0.4375–0.78)	0.60	1.56
15	0.61 (0.492–0.72)	0.62 (0.437–0.78)	0.61	1.63
17	0.65 (0.531–0.75)	0.59 (0.408–0.76)	0.63	1.60
19	0.65 (0.531–0.75)	0.59 (0.408–0.76)	0.65	1.60
21	0.68 (0.558–0.78)	0.59 (0.408–0.76)	0.65	1.66
23	0.68 (0.558–0.78)	0.56 (0.359–0.73)	0.64	1.54
25	0.68 (0.558–0.78)	0.56 (0.359–0.73)	0.64	1.54

LR+: Likelihood ratio.

Figure 2.

Overall survival with a methylation cutoff point of 21%.

Figure 3.

Progression-free survival with a methylation cutoff point of 21%.

Multivariate Cox regression was used to determine factors with significant survival impact. Methylation status (HR: 0.453, 95% CI: 0.279–0.735, p = 0.001) and total surgical resection (HR: 0.083, 95% CI: 0.01–0.665, p = 0.019) had significant impact on OS (Table 3). Sex, age, race and performance status (ECOG score ≤2 vs >2), did not significantly affect OS.

Table 3.

Multivariate analyses of factors affecting survival.

Variable	HR	95% CI	p-value
Age ≥60 years	0.931	0.581–1.491	0.765
Age <60 years	1.075	0.671–1.722	0.766
Methylation status	0.453	0.279–0.735	0.001
ECOG ≤2	0.649	0.403–1.046	0.083
ECOG >2	1.540	0.956–2.482	0.076
Race (white)	1.56	0.88–2.74	0.126
Race (black)	0.571	0.273–1.198	0.138
Extent of surgical resection
Total	0.083	0.01–0.665	0.019
Partial	0.142	0.017–1.167	0.069
Biopsy	0.159	0.018–1.443	0.102

p-values in bold are significant.

ECOG: Eastern Cooperative Oncology Group; HR: Hazard ratio.

Last, the crude model indicated that linearity exists between the percentage of methylation and survival rate. Crude model analysis showed that the hazard ratio was 0.98, signifying that an increase in methylation percent by one unit results in a reduction of risk of MGMT death by 2%. The association is statistically significant (p-value = 0.004). In other words, higher methylation percentage was associated with better survival outcome. Figure 4 illustrates the correlation between methylation percent and survival rate. The correlation coefficient (R) of 0.23 indicates a positive linear association between methylation percent and survival rates with a statistically significant p-value of 0.017.

Figure 4.

Linear regression illustrating the correlation between methylation percent and survival time.

Discussion

MGMT promoter methylation, regarded as the most valuable molecular prognostic factor for GB, has been shown to improve the efficacy of TMZ in the treatment of GB. That is, MGMT promoter methylated patients have improved OS compared with MGMT promoter unmethylated patients [8]. In a recent survey of methods used to determine MGMT methylation status, the most used methods were MSP (37%) and PSQ (34%). Selection of the cutoff points has been either the cutoff points defined in the literature or decided by the laboratory, and varies from 1 to 30%. Only 4% use the cutoff value suggested by the company supplying the kit [17].

Of the different modalities used to detect MGMT promoter methylation status, PSQ has been suggested as the method of choice [18]. PSQ detects methylation levels of individual CpG sites and is preferred to MSP, IHC and quantitative real-time MSP (qMSP) due to its superior reproducibility, sensitivity and prediction of survival [9,10]. However, the optimal methylation level cutoff value to differentiate MGMT promoter methylated and MGMT promoter unmethylated subgroups has yet to be clearly established. Defining this cutoff value in PSQ is important for guiding treatment decisions. The cutoff values used in previous studies have ranged from 4% to 35% [19].

In an effort to further explore the prognostic impact of MGMT promoter methylation status and to define the optimal cutoff value of MGMT promotor methylation, the authors performed PSQ to determine methylation status for all patients in the study. Mean percentage methylation of each patient was plotted on a ROC curve, after which the AUC and test accuracy were calculated. The LR+, which accounts for both sensitivity and specificity, is one of the best measures of diagnostic accuracy [12]. The optimal cutoff point is associated with the highest LR+ and accuracy. In this dataset, a cutoff point of 21% was found to be the optimal value for determining MGMT promoter methylation status and corresponds with LR+ (1.66) and accuracy (0.65). According to this cutoff point, 45 patients were methylated and 64 were unmethylated. OS was compared for the methylated and unmethylated subgroups using the Kaplan–Meier method with log rank test; methylated patients had a statistically significant improvment in OS compared with unmethylated patients (20.4 vs 13.3 months, p = 0.001). There was a significant difference in OS but not PFS between patients who were methylated and unmethylated. The PFS data can be skewed by pseudo-progression, which would have led to a lack of significant differences between these arms, however, the response to TMZ is evident by the difference in OS. Through Cox multivariate survival analysis, methylation status (HR: 0.453, 95% CI: 0.279–0.735, p = 0.001) and total surgical resection (HR: 0.083, 95% CI: 0.01–0.665, p = 0.019) were confirmed to be independent prognostic factors for OS (Table 3).

Previous studies have suggested variable cutoffs. In their series of 100 patients, Quillien et al. found that the best prognostic effect for OS was observed at 9% [20]. That is, 9% was the best cutoff value for stratifying patients into MGMT promoter methylated and unmethylated subgroups determined by an outcome-based method using ROC curve analysis. Kim et al. used ROC curve analysis to determine the cutoff value for the mean percentage of methylation and showed that the optimal threshold for distinguishing long survival versus short survival was 9% [20]. However, they used sensitivity and specificity separately, rather than LR+ [21,22]. In a study by Brigliadori et al., this approach was shown to be associated with an overestimation of responders. When teased out by the extent of resection, the 9% cutoff value lost significance when patients had gross total resection, and outcomes seemed to be affected by surgery alone [23]. They did not use ROC analysis, but divided MGMT methylation status into 3 groups; 0–9%; 10–29% and 30–100%, and used OS outcomes to conclude that the best methylation cutoff was ≥30%. Another study using ROC analysis identified a cutoff point of 12.5%. While the cutoff point of 9.5% in this study had the highest LR, it was rejected for low sensitivity [24]. The present study showed that a cutoff of 21% is associated with the highest LR+ and accuracy as compared with 9% (LR+ 1.66 and accuracy of 0.65 vs LR+ 1.62, accuracy of 0.59, respectively). When we used a cutoff point of 21% versus 9%, as expected, more patients were classified as unmethylated (64 vs 53 patients). However, the difference in their OS was minimal (13.3 months vs 13.1 months; Table 4). This further underscores the need for identification of an ideal cutoff, as overestimation using a cutoff of 9% will lead to the risk of treating patients with ineffective chemotherapy at the cost of increased toxicity and financial burden. Additionally, in elderly patients with unmethylated MGMT, exposure to TMZ can cause rapid progression of disease. In the NOA-08 study, PFS was shorter in patients with unmethylated MGMT promoter who received TMZ than in those who underwent radiation therapy (3.3 months [3.0–3.5] vs 4.6 months [3.7–6.3]). A recent trial by Perry et al. looked at the addition of TMZ to radiation therapy in treating GB in the elderly. They showed that the addition of TMZ had the greatest benefit in MGMT methylated patients (median OS 13.5 vs 7.7 months, HR: 0.53, CI: 0.38–0.73, p < 0.001), but the OS did not reach statistical significance in the MGMT unmethylated group (median OS 10 vs 7.9 months; HR: 0.75, CI: 0.56 to 1.01, p = 0.055, p = 0.08 for interaction) [25].

Table 4.

Survival data with methylation cutoff points of 21 and 9%.

	Patients (n)	OS (months)	PFS (months)
Cutoff point of 21%
Methylated	45	20.4	7.4
Unmethylated	64	13.3	6.8
Cutoff point of 9%
Methylated	56	18.5	7.1
Unmethylated	53	13.1	6.4

OS: Overall survival; PFS: Progression-free survival.

As an exploratory analysis, we used a crude model to determine whether there was a linear association between the degree of methylation and OS. We found that for every percent increase in methylation, there is a 2% risk of reduction of death (p = 0.004). In other words, a higher methylation percentage correlates with better OS. Based on the results of this study, by using ROC curve analysis, the best cutoff value for defining MGMT promoter methylation status in this dataset was 21%. The promoter methylation level of MGMT by PSQ and surgical resection in GB patients had prognostic value. The determination of a well-established and widely accepted cutoff value for determining MGMT promoter methylation status by PSQ testing will allow for more concrete bases for decisions made in the clinical setting regarding the utility of therapeutic options available to GB patients.

Conclusion

Determining MGMT methylation status is of vital importance in the management of patients with glioblastoma, however, there is an unmet need to define a standard method and cutoff. Pyrosequencing is a commonly used method. The current dataset showed that by using ROC analysis, the optimal cutoff value to define methylated versus unmethylated status was 21%. Further, the crude model reveals that methylation percentage is linearly proportional to the reduction of risk of death. Additionally, methylation status and total surgical resection are proven to have significant positive impacts on survival.

Future perspective

The difficulties with MGMT testing in the clinical setting are increasingly being recognized. While current guidelines often include recommendations based on MGMT methylation results, in the future, it is expected that expert consensus and guidelines would recommend the ideal method for testing and clinically relevant cutoff points. Standardization of testing methods and interlaboratory control programs are needed. Using different methods, such as pyrosequencing and MSP to reanalyze and compare in large, randomized trials would provide high-level evidence supporting the use of these techniques.

MGMT promoter methylation status in glioblastoma is a key prognostic factor and predicts the effect of temozolomide.

MGMT methylation status serves as an important decision point for therapeutic choices, especially in the elderly or patients with poor performance status.

Pyrosequencing and methylation-specific polymerase chain reaction are the most commonly used methods for determining MGMT promoter methylation.

There is an unmet need to identify an optimal cutoff point for MGMT methylation status as determined by pyrosequencing.

Receiver operating characteristic analysis showed that the optimal cutoff for MGMT methylation was 21%, which corresponded to the highest likelihood ratio.

Methylated patients have better overall survival (OS) compared with unmethylated patients (20.4 months vs 13.3 months, p = 0.001).

Methylation status (HR: 0.453, 95% CI: 0.279–0.735, p = 0.001) and total surgical resection (HR: 0.083, 95% CI: 0.01–0.665, p = 0.019) had significant impacts on OS.

Multivariate analysis showed that sex, age, race and performance status do not significantly affect OS.

Crude model analysis demonstrated that the degree of methylation percentage is linearly proportional to the reduction of risk of death; an increase of 10% methylation resulted in a reduction of risk of death by 20% (p = 0.004).