Sensitivity and specificity of the macimorelin test for diagnosis of AGHD.

OBJECTIVE: The macimorelin test is approved for the diagnosis of adult growth hormone deficiency (AGHD) based on its efficacy vs the insulin tolerance test (ITT). Macimorelin has a significant advantage over ITT in avoiding hypoglycemia. Analyses were conducted to determine whether macimorelin performance is affected by age, BMI, or sex, and evaluate its performance vs ITT over a range of GH cutpoints.
DESIGN: Post hoc analyses of data from a previous randomized phase 3 study included participants aged 18-66 years with BMI <37 kg/m2 and high (Group A), intermediate (Group B), or low (Group C) likelihood for AGHD based on pituitary history, and matched controls (Group D).
METHODS: Probability of AGHD was estimated using unadjusted, age-adjusted, BMI-adjusted, and sex-adjusted logistic models. Area under the curve (AUC) of the estimated receiver operating characteristic (ROC) curve (range, 0-1; 1 = perfect) was compared for adjusted vs unadjusted models. Separate analyses evaluated agreement, sensitivity, and specificity for macimorelin and ITT using cutpoints of 2.8, 4.0, 5.1, and 6.5 ng/mL.
RESULTS: For participants in Group A (n = 41) and Group D (n = 29), unadjusted, age-adjusted, BMI-adjusted, and sex-adjusted models had ROC AUCs (95% CIs) of 0.9924 (0.9807-1), 0.9924 (0.9807-1), 0.9916 (0.9786-1), and 0.9950 (0.9861-1), respectively.
CONCLUSIONS: Macimorelin performance was not meaningfully affected by age, BMI, or sex, indicating robustness for AGHD diagnosis. Of the 4 GH cutpoints evaluated, the cutpoint of 5.1 ng/mL provided maximal specificity (96%) and high sensitivity (92%) and was in good overall agreement with the ITT at the same cutpoint (87%).

Introduction

Adult growth hormone deficiency (AGHD) is a clinical syndrome characterized by abnormal body composition, unfavorable cardiovascular risk, cardiac dysfunction, decreased bone mineral density, and glucose intolerance (1, 2). Findings from recent studies suggest that hypopituitarism is associated with excess mortality (3, 4, 5, 6, 7), in part due to AGHD (8, 9). In addition, if untreated, AGHD can increase the risk of future bone fractures, obesity, diabetes, dyslipidemia, and cardiovascular and cerebrovascular disease (10, 11, 12). Quality of life can also be negatively affected in patients with untreated AGHD as a result of fatigue, depression, anxiety, sleep impairment, sexual dysfunction, weight gain, reduced cognition, lack of energy, and/or social isolation (12, 13).

Treatment of AGHD with growth hormone (GH) replacement therapy has been shown to improve many, but not all, of the clinical features of AGHD (14). However, the diagnosis of AGHD is challenging because the clinical presentation of this syndrome is often nonspecific, with many features resembling those of metabolic syndrome (15, 16). Furthermore, when AGHD is suspected, there are no reliable biomarkers to guide clinicians on diagnosis and disease progression (17). Thus, except in patients with panhypopituitarism and low insulin-like growth factor I (IGF-I) levels, the diagnosis of AGHD often requires confirmation using a provocative GH stimulation test (GHST) (18, 19, 20). The current gold standard GHST advocated by several consensus guidelines is the insulin tolerance test (ITT) (18, 19, 20, 21). However, the widespread use of the ITT is limited by several factors (22), including the requirement for close medical supervision by a physician throughout the test. The ITT may also be unpleasant for some patients and is associated with increased risks for severe hypoglycemia and hypoglycemia-related seizures, as well as exacerbation of the cardiovascular and cerebrovascular disease. Notably, patients with high BMI often require higher doses of insulin (≥2 IU/kg) to achieve an adequate hypoglycemic response, which in turn predisposes these patients to delayed hypoglycemia after test completion. As a result of these safety concerns, the ITT is contraindicated in elderly patients and in those with underlying seizure disorders or cardiovascular disease (11, 23, 24, 25). If AGHD is suspected and the ITT is contraindicated or not feasible, the glucagon stimulation test can be used as an alternative GHST. However, the diagnostic accuracy of the glucagon stimulation test is unclear, and the test is time-consuming (3–4 h), requires intramuscular injection, and is associated with bothersome side effects, including nausea, vomiting, and headache (19).

Macimorelin is an orally active ghrelin receptor agonist that is indicated for the diagnosis of AGHD in the United States (26) and Europe (27). Macimorelin is well-tolerated; the most frequent side effect is transient and mild dysgeusia (28). The macimorelin GHST has a more favorable safety profile, is more convenient, only requires four blood draws for serum GH measurements, and takes less time to conduct than the ITT and the glucagon stimulation test (28, 29, 30, 31). In a recent phase 3 study, the diagnostic performance of macimorelin was comparable to that of the ITT when using an a priori cutpoint value of 2.8 ng/mL for macimorelin and 5.1 ng/mL for the ITT (28).

Previous studies have reported that in order to improve the reliability of the GH-releasing hormone plus arginine (GHRH-arginine) (32, 33, 34) and glucagon stimulation tests (30, 35, 36, 37), BMI-dependent GH cutpoints should be used. Herein we report results of post hoc analyses of data from the phase 3 study (28) of macimorelin, evaluating its performance by age, BMI, sex, and different GH cutpoint values.

Materials and methods

These post hoc analyses were performed using data from the previously published phase 3, open-label, randomized, multicenter, 2-way crossover study that validated the efficacy and safety of macimorelin by comparing this test with the ITT for the diagnosis of AGHD (28). The study protocol was approved by institutional review boards at each of the included study sites in the United States and Europe. The study was conducted in compliance with the Declaration of Helsinki and its amendments and the International Conference on Harmonisation Guideline for Good Clinical Practice (28).

Study participants and procedures

Detailed methods of the study have been reported previously (28). Briefly, analyses were performed using data from 140 participants aged 18–66 years with BMI <37 kg/m2 and varying degrees of likelihood for AGHD. Participants were classified as having high (Group A), intermediate (Group B), or low (Group C) likelihood for AGHD and were compared with matched healthy controls (Group D). Participants were considered to have a high likelihood of AGHD (Group A) if they had a structural hypothalamic or pituitary lesion and low serum IGF-I levels plus ≥3 other pituitary hormone deficiencies, or childhood-onset GHD with structural lesions and low serum IGF-I levels. Participants were considered to have a low likelihood of AGHD (Group C) if they had 1 risk factor for AGHD (e.g. history of distant traumatic brain injury, only 1 other pituitary hormone deficiency, or childhood-onset isolated GHD). Participants were considered to have an intermediate likelihood of AGHD (Group B) if they did not meet the criteria for Groups A or C. Matching participants in Group A with participants in Group D was based on age, sex, BMI, and estrogen status (28).

Participants were randomized to undergo either the macimorelin GHST (Aeterna Zentaris, Frankfurt, Germany) followed by the ITT or the ITT followed by the macimorelin GHST. The ITT was performed with regular human insulin obtained from pharmacy stock. Serum GH concentrations were measured using an ultrasensitive validated immunochemiluminescence assay (IDS-iSYS human GH) (38) that is standardized to the World Health Organization recombinant GH calibration standard (98/574) and complies with recommendations for assay standardization (39). GHSTs were performed 7–28 days apart under fasted conditions. Test results were considered ‘positive’ for GHD if the peak GH value was less than a cutpoint established a priori; tests were considered ‘negative’ for GHD if the peak GH value was greater than or equal to this cutpoint (28). In the primary analysis (28), the GH cutpoints determined a priori were 2.8 ng/mL for the macimorelin test and 5.1 ng/mL for the ITT.

Analysis of the effects of age, BMI, and sex on macimorelin performance

This analysis included participants in the high likelihood of AGHD group (Group A) and the matched controls (Group D). Study participants in Group A were assumed to be ‘true AGHD-positive’ subjects, and those in Group D were assumed to be ‘true AGHD-negative’ subjects (28).

The probability of AGHD was estimated using four logistic models, with peak GH level as the explanatory variable, fitted to the data: unadjusted, age-adjusted, BMI-adjusted, and sex-adjusted. Each model considered all subjects as independent observations, not accounting for matching.

The area under the curve (AUC) of the estimated receiver operating characteristic (ROC) curve (range of 0–1, where 1 is perfect) was measured after administration of the macimorelin GHST. ROC AUC results from each adjusted model were compared with results from the unadjusted model. Estimated sensitivity and specificity were calculated for each model using macimorelin cutpoint values of 2.8 and 5.1 ng/mL. Estimated sensitivity and specificity from the model adjusted for BMI were calculated at the minimum, mean, median, and maximum BMI values of study participants.

Analysis of macimorelin vs ITT performance over a range of GH cutpoints

The percentage agreement (negative, positive, and overall) between tests was determined using GH cutpoint values of 2.8, 4.0, 5.1, and 6.5 ng/mL for both the macimorelin GHST and the ITT in participants from all study groups (A, B, C, and D). The cutpoint of 2.8 ng/mL was selected based on a previous post hoc analysis (40). The cutpoint of 5.1 ng/mL was evaluated because it is a validated and widely referenced cutpoint for the ITT (23, 41). The cutpoint of 4.0 ng/mL was selected because it is the approximate midpoint between 2.8 and 5.1 ng/mL. The cutpoint of 6.5 ng/mL was evaluated because it is the lowest cutpoint that corresponded to an estimated sensitivity of 100% for the ITT. The percentage agreement between tests was calculated as the percentage of participants with the same finding (positive, negative, or overall (i.e. both positive and negative)) using a specified macimorelin cutpoint and a specified ITT cutpoint. Two-sided 95% CIs of the percentage agreement between tests were calculated based on the Clopper–Pearson method (42).

The estimated specificity and sensitivity of both tests were determined at all cutpoints. Specificity was calculated as the percentage of participants in Group D (healthy controls) with a negative finding using the specified GH cutpoint. Sensitivity was calculated as the percentage of participants in Group A (high likelihood of AGHD) with a positive finding using the specified GH cutpoint.

Results

Effects of age, BMI, and sex on macimorelin performance

Overall, the analysis included 41 participants with a high likelihood of AGHD (Group A) and 29 healthy controls (Group D). Demographic characteristics were comparable between participants in Group A and Group D. Overall, the mean (SD) age was 41.7 (13.9) years, and the ages of all participants ranged from 18 to 66 years. The mean (SD) BMI was 27.1 (4.0) kg/m2 (range, 20.4–36.6 kg/m2). Thirty-nine of 70 (55.7%) participants were male. As expected, mean (SD); range peak GH concentration was substantially lower in Group A (0.91 (1.9); 0.1–8.6 ng/mL) than in Group D (16.2 (7.4); 2.2–34.6 ng/mL).

Performance of the macimorelin test was not meaningfully affected by age, BMI, or sex (Fig. 1). The ROC AUC (95% CI) for the unadjusted model was 0.9924 (0.9807–1) compared with 0.9924 (0.9807–1) for the age-adjusted model (P = 1), 0.9916 (0.9786–1) for the BMI-adjusted model (P = 0.6861), and 0.9950 (0.9861–1) for the sex-adjusted model (P = 0.4207).

Figure 1

Macimorelin test ROC curves for the unadjusted model and models adjusted for age, BMI, and sex. ROC, receiver operating characteristic.

Using the macimorelin cutpoint of 2.8 ng/mL, estimated sensitivity was 88% and specificity was 97% for the unadjusted model (Table 1). These values remained the same when adjusting for age and for mean or median BMI. Adjusting for maximum BMI (36.6 kg/m2) resulted in sensitivity of 76% and specificity of 100%. When adjusting for sex, sensitivity was 88% for both males and females; specificity was 100% for males and 93% for females (Table 1).

Table 1

Estimated sensitivity and specificity at the prespecified macimorelin cutpoint of 2.8 ng/mL in unadjusted and adjusted models.

	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Covariate value
Unadjusted	88 (74–96)	97 (82–100)	–
Adjusted for age	88 (74–96)	97 (82–100)	–
Adjusted for BMI	90 (77–97)	97 (82–100)	Minimum BMI = 20.4 kg/m2
Adjusted for BMI	88 (74–96)	97 (82–100)	Mean BMI = 27.1 kg/m2
Adjusted for BMI	88 (74–96)	97 (82–100)	Median BMI = 26.7 kg/m2
Adjusted for BMI	76 (60–88)	100 (88–100)	Maximum BMI = 36.6 kg/m2
Adjusted for sex	88 (64–99)	93 (66–100)	Female sex
Adjusted for sex	88 (68–97)	100 (78–100)	Male sex

Using the macimorelin cutpoint of 5.1 ng/mL, sensitivity was 93% and specificity was 97% for the unadjusted model. These values remained the same when adjusting for age and mean BMI. When adjusting for sex, sensitivity was 92% for males and 94% for females, and specificity was 100% for males and 93% for females (Table 2).

Table 2

Estimated sensitivity and specificity at the macimorelin cutpoint of 5.1 ng/mL in unadjusted and adjusted models.

	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Covariate value
Unadjusted	93 (80–99)	97 (82–100)	–
Adjusted for age	93 (80–99)	97 (82–100)	–
Adjusted for BMI	95 (84–99)	93 (77–99)	Minimum BMI = 20.4 kg/m2
Adjusted for BMI	93 (80–99)	97 (82–100)	Mean BMI = 27.1 kg/m2
Adjusted for BMI	93 (80–99)	93 (77–99)	Median BMI = 26.7 kg/m2
Adjusted for BMI	90 (77–97)	97 (82–100)	Maximum BMI = 36.6 kg/m2
Adjusted for sex	94 (71–100)	93 (66–100)	Female sex
Adjusted for sex	92 (73–99)	100 (78–100)	Male sex

Macimorelin vs ITT performance over a range of GH cutpoints

This analysis included participants in the modified intent-to-treat population (n = 140; all randomized participants with evaluable data from both the macimorelin GHST and the ITT). Participants included those with a high (Group A; n = 38), intermediate (Group B; n = 37), or low (Group C; n = 40) likelihood for AGHD and healthy matched controls (Group D; n = 25). Using an ITT cutpoint of 5.1 ng/mL, 74 participants were classified as GH-deficient and 66 were classified as GH-sufficient.

Selecting the same cutpoint values for macimorelin and the ITT yielded high positive (Table 3), negative (Table 4), and overall (Table 5) agreement rates. At a GH cutpoint value of 2.8 ng/mL for both tests, positive agreement was 87.1% (95% CI, 76.2–94.3%), negative agreement was 93.6% (95% CI, 85.7–97.9%), and overall agreement was 90.7% (95% CI, 84.6–95.0%). At a GH cutpoint value of 5.1 ng/mL for both tests, positive agreement was 82.4% (95% CI, 71.8–90.3%), negative agreement was 92.4% (95% CI, 83.2–97.5%), and overall agreement was 87.1% (95% CI, 80.4–92.2%).

Table 3

Percentage (95% CI) positive agreement between the macimorelin GHST and the ITT at different cutpoints among participants with a positive finding for each ITT cutpoint.

Macimorelin cutpoint, ng/mL	ITT cutpoint, ng/mL
	2.8 (n = 62/140)	4.0 (n = 67/140)	5.1 (n = 74/140)	6.5 (n = 82/140)
2.8	87.1% (76.2–94.3%)	80.6% (69.1–89.2%)	74.3% (62.8–83.8%)	68.3% (57.1–78.1%)
4.0	88.7% (78.1–95.3%)	82.1% (70.8–90.4%)	78.4% (67.3–87.1%)	72.0% (60.9–81.3%)
5.1	90.3% (80.1–96.4%)	86.6% (76.0–93.7%)	82.4% (71.8–90.3%)	75.6% (64.9–84.4%)
6.5	93.6% (84.3–98.2%)	89.6% (79.7–95.7%)	87.8% (78.2–94.3%)	81.7% (71.6–89.4%)

Results in italics are based on using the same cutpoints for macimorelin and the ITT. n = number of participants with a positive finding using the specified ITT cutpoint.

GHST, growth hormone stimulation test; ITT, insulin tolerance test.

Table 4

Percentage (95% CI) negative agreement between the macimorelin GHST and the ITT at different cutpoints among participants with a negative finding for each ITT cutpoint.

Macimorelin cutpoint, ng/mL	ITT cutpoint, ng/mL
	2.8 (n = 78/140)	4.0 (n = 73/140)	5.1 (n = 66/140)	6.5 (n = 58/140)
2.8	93.6% (85.7–97.9%)	93.2% (84.7–97.7%)	93.9% (85.2–98.3%)	94.8% (85.6–98.9%)
4.0	91.0% (82.4–96.3%)	90.4% (81.2–96.1%)	93.9% (85.2–98.3%)	94.8% (85.6–98.9%)
5.1	87.2% (77.7–93.7%)	89.0% (79.5–95.2%)	92.4% (83.2–97.5%)	93.1% (83.3–98.1%)
6.5	80.8% (70.3–88.8%)	82.2% (71.5–90.2%)	87.9% (77.5–94.6%)	89.7% (78.8–96.1%)

Results in italics are based on using the same cutpoints for macimorelin and the ITT. n = number of participants with a negative finding using the specified ITT cutpoint.

GHST, growth hormone stimulation test; ITT, insulin tolerance test.

Table 5

Percentage (95% CI) overall agreement between the macimorelin GHST and the ITT at different cutpoints.

Macimorelin cutpoint, ng/mL	ITT cutpoint, ng/mL
	2.8 (n = 140)	4.0 (n = 140)	5.1 (n = 140)	6.5 (n = 140)
2.8	90.7% (84.6–95.0%)	87.1% (80.4–92.2%)	83.6% (76.4–89.3%)	79.3% (71.6–85.7%)
4.0	90.0% (83.8–94.4%)	86.4% (79.6–91.6%)	85.7% (78.8–91.1%)	81.4% (74.0–87.5%)
5.1	88.6% (82.1–93.3%)	87.9% (81.3–92.8%)	87.1% (80.4–92.2%)	82.9% (75.6–88.7%)
6.5	86.4% (79.6–91.6%)	85.7% (78.8–91.1%)	87.9% (81.3–92.8%)	85.0% (78.0–90.5%)

Results in italics are based on using the same cutpoints for macimorelin and the ITT. n = number of participants with findings for both the macimorelin GHST and the ITT.

GHST, growth hormone stimulation test; ITT, insulin tolerance test.

Assuming that all participants in Group A (n = 38) were AGHD cases and that all participants in Group D (n = 25) were healthy controls, the macimorelin GHST and the ITT had identical estimated specificities of 96% at GH cutpoint values of 2.8, 4.0, or 5.1 ng/mL. At the GH cutpoint of 5.1 ng/mL, the estimated sensitivity was 92% with the macimorelin GHST and 97% with the ITT. Increasing the GH cutpoint to 6.5 ng/mL increased the sensitivity of the macimorelin test to 97% and the ITT to 100%. However, these increases in sensitivity were at the expense of decreased specificity (92 and 88%, respectively, for the macimorelin test and the ITT at 6.5 ng/mL). Sensitivity and specificity values varied slightly when calculated using the regression models (data not shown).

Discussion

The results of these post hoc analyses confirm the robust performance of the macimorelin GHST for the diagnosis of AGHD by sex, within the ranges of age (18–66 years) and BMI (20.4–36.6 kg/m2) evaluated, and across a range of GH cutpoints. At the 4 GH cutpoints evaluated, the macimorelin test maintained its sensitivity and specificity, with high levels of agreement with the ITT.

In contrast to the GHRH-arginine test (32, 33, 34) and glucagon stimulation test (30, 35, 36, 37, 43), for which BMI-dependent GH cutpoints have been recommended to improve their reliability (19), the findings from this study indicate that the performance of the macimorelin test is neither affected by BMI (≤36.6 kg/m2) nor sex using the cutpoints of 2.8 and 5.1 ng/mL. This is particularly noteworthy given that endogenous GH secretion is known to be affected by body weight and estrogen levels (25, 44, 45). For example, recent data indicate that for the oral glucose tolerance test, BMI, sex, and use of oral contraceptives containing estradiol can significantly affect GH nadir levels, suggesting that the GH cutpoints used to diagnose acromegaly with this test may need to be adjusted based on patient characteristics (46, 47). Importantly, there is also a strong negative correlation between BMI and GH response when using the ITT, and some researchers have proposed the need to use substantially higher GH cutpoints for individuals without vs individuals with obesity (25, 44).

Peak GH levels after administration of ghrelin are significantly reduced in older men and women compared with younger individuals (48). Nevertheless, our results suggest that, in the age range studied (18–66 years) and with the GH cutpoints selected in our study population, age does not significantly affect the performance of the test.

However, several limitations of these analyses must be acknowledged. These results may not be generalizable to elderly or pediatric patients or to individuals with severe obesity. The study inclusion criteria restricted participants to the age range of 18–66 years (28). The highest recorded baseline BMI was 36.6 kg/m2, and most participants had a BMI of <30 kg/m2. Furthermore, analysis by sex did not include consideration for menopausal status among female participants. Additionally, this study involved a relatively small sample size and excluded patients with poorly controlled diabetes.

In conclusion, these post hoc analyses of macimorelin for the diagnosis of AGHD in patients with a high likelihood of AGHD vs matched controls indicate that the test performance is robust and is not meaningfully affected by age, baseline BMI (≤36.6 kg/m2), or sex over a range of GH cutpoints. In addition, when evaluating the entire study population, using the same cutpoint of 2.8 ng/mL for both the macimorelin test and the ITT resulted in high levels of positive (87.1%), negative (93.6%), and overall (90.7%) agreement between tests, which were higher than agreement levels using a cutpoint of 5.1 ng/mL for both tests. Finally, sensitivity of the macimorelin test was greatest at a cutpoint of 6.5 ng/mL but was at the expense of a decline in specificity from 96 to 92%, which may be undesirable if the primary consideration is the minimization of false-positive diagnosis of AGHD. Of the 4 GH cutpoints evaluated, the cutpoint of 5.1 ng/mL provided maximal specificity (96%) and high sensitivity (92%) and was in good overall agreement with the ITT at the same cutpoint (87%).

Declaration of interest

J M G has served as an advisor and/or investigator for Aeterna Zentaris, Pfizer, and Novo Nordisk. B M K B has served as a consultant and/or an investigator for OPKO Biologics, Novo Nordisk, Strongbridge Biopharma, Aeterna Zentaris, Ascendis, Merck Serono, and Pfizer. M K has served as an advisor, an investigator, and/or a speaker for Pfizer, Ono, Ipsen, Novo Nordisk and Novartis. V P has served as an advisor, an investigator, and/or a speaker for Novo Nordisk, Pfizer, and Novartis. A L has served as an advisor, an investigator, and/or a speaker for Aeterna Zentaris, Ipsen, Merck Serono, Novo Nordisk, Pfizer, and Sandoz. C J S has served as a consultant for Aeterna Zentaris, Ascendis, Chiasma, Ipsen, Merck Serono, Novartis, Novo Nordisk, Pfizer, Sandoz, and Strongbridge Biopharma. P C has served as an advisor, an investigator, and/or a speaker for Aeterna Zentaris, Ipsen, Novartis, Novo Nordisk, Pfizer, Italpharmaco, Antisense, Prolor Biotech, Sandoz, and Eli Lilly. R S has been an investigator for Clarus, Antares, and Novartis. C W has received research support from Clarus Therapeutics, Antares, and TesoRx. R R F and F C are employees of Strongbridge Biopharma. N A is an employee of Aeterna Zentaris GmbH. G M was an employee of Aeterna Zentaris GmbH at the time of study conduct and study data analysis. N K, F S, and V O are employees of Novo Nordisk Inc. K C J Y has served as an advisor, an investigator, and/or a speaker for Pfizer, Novo Nordisk, Strongbridge, Ascendis, Ipsen, and Novartis.

Funding

This work was sponsored by Aeterna Zentaris. The commercial rights to macimorelin in the United States and Canada have been licensed from Aeterna Zentaris to .