Diagnostic performance of basal cortisol level at 0900-1300h in adrenal insufficiency.

OBJECTIVE: An ACTH stimulation test is the standard diagnostic test for adrenal insufficiency (AI). We aimed to investigate the diagnostic performance between serum morning (0800 h) cortisol and serum basal (0900-1300 h) cortisol levels and determine the proper cut-off point to facilitate AI diagnosis to reduce the number of tests.
METHODS: A six-year retrospective study was performed in a tertiary care medical center. We identified 416 patients who had undergone either low (LDT) or high dose (HDT) ACTH stimulation outpatient tests. AI was defined as a peak serum cortisol level of <500 nmol/L at 30 or 60 minutes after LDT or HDT. The associations between AI and serum basal and morning cortisol levels were demonstrated by logistic regression model. Diagnostic performance was evaluated by ROC analysis.
RESULTS: Of the 416 patients, 93 (22.4%) were categorized as having AI. The adjusted area under the curve (AUC) for the basal cortisol level for the diagnosis of AI was significantly higher than that for the morning cortisol (0.82 vs 0.69, p <0.001) level. The proposed cut-off values for the basal cortisol were <85 nmol/L (specificity 99.7%) and >350 nmol/L(sensitivity 98.9%). By using these proposed cut-off points, approximately 30% of the ACTH stimulation tests could be eliminated.
CONCLUSION: The serum basal cortisol level with the proposed cut-off points were considered as an alternative option for diagnosis of AI. Utilizing the serum basal cortisol level can facilitate AI diagnosis as it is convenient, is not a time-specific test and has a high diagnostic performance.

Introduction

Adrenal insufficiency (AI) is a lethal disease if left undiagnosed. A variety of tests have been proposed to identify inadequate levels of glucocorticoid production [1]. One of the variables most commonly used to screen for AI is the serum morning cortisol level (0800 h cortisol). The dynamic tests that have been reported to properly assess AI were high-dose (HDT) and low-dose ACTH stimulation tests (LDT) [2-5]. To perform these tests, serum cortisol is drawn at 0 minutes (basal cortisol), and then either the standard 250 μg [3] or 1–5 μg [6, 7] of ACTH is administered before 30- and 60-minute serum cortisol levels are drawn.

Serum morning cortisol tests with a proper cut-off level can be used to screen for AI. If a serum morning cortisol level falls into the intermediate category, additional dynamic testing is required. Using ACTH stimulation as a reference standard for AI, some studies have proposed lower and upper cut-off values for serum morning cortisol levels of <100–145 nmol/L (3.5–5.2 μg/dL) and >375–500 nmol/L (13.6–18.0 μg/dL), respectively [8, 9]. The wide variance in the serum morning cortisol cut-off levels is caused by different cortisol assays used in multiple groups of studies, which makes it difficult to set a definite morning cortisol cut-off level. Currently, clinical practice guideline state that there is no evidence supporting the use of basal cortisol levels to diagnose AI [3]. However, using basal levels for screening has many advantages because the timing of the test is less important. Thus, it is more convenient for patients and health care providers. In fact, one study showed evidence supporting the use of basal cortisol levels to predict AI. They proposed that if the basal cortisol was >450 nmol/L (16.3 μg/dL), AI could be ruled out, with a negative predictive value of 98.7%. If the basal cortisol level was <100 nmol/L (3.6 μg/dL), AI can be diagnosed with a positive predictive value of 93.2% [10]. Still, there is no report comparing the diagnostic performance between serum morning cortisol and basal cortisol levels for the diagnosis of AI.

The objectives of our study were twofold: a) to compare the diagnostic accuracy for AI between the serum morning cortisol level and the basal cortisol level in an outpatient setting and b) to determine the appropriate cut-off values for the serum morning cortisol and basal cortisol levels that would preclude the need for dynamic testing in the outpatients settings.

Methods

A six-year retrospective observational cohort study was performed. Data were collected from electronic medical records for all patients referred to the Adult Endocrinology Outpatient Department Unit in a tertiary care referral hospital in the northern part of Thailand between January 2010 and December 2015. The study protocol was approved by the Faculty of Medicine, Chiang Mai University, Ethics Committee. The need for consent was waived by the Ethics Committee. The patients who had undergone either LDT or HDT ACTH stimulation tests were included. Only the first tests during this period were used. The exclusion criteria were patients with incomplete data for 0800 h serum morning cortisol and ACTH stimulation testing. Women who took oral contraceptives containing estrogen, patients with suspected congenital adrenal hyperplasia and patients who had undergone pituitary surgery within the past two months were excluded. Patients receiving glucocorticoids or other traditional medicines suspected of containing glucocorticoids were told to discontinue those substances at least 24 hours before testing. Serum cortisol levels were measured by an electrochemiluminescence immunoassay (ECLIA) using an Elecsys model 1010 (Roche Diagnostic, Laval, Quebec). The intra- and inter-assay coefficients of variation for serum cortisol were <10%. In addition, the serum albumin, total cholesterol and serum creatinine data were also collected within three months before or after ACTH stimulation testing.

ACTH stimulation testing protocol

Those who had serum morning (0800 h) cortisol levels that fell into intermediate levels of 83–499 nmol/L (3–17.9 μg/dL) were classified as either AI and normal adrenal response (non-AI) based on either LDT or HDT. All tests were performed between 0900 h-1300 h by well-trained medical nurses. The serum cortisol level was determined at 0 (basal), 30 and 60 minutes after intravenous administration of either 1 μg or 250 μg ACTH (Synacthen®, Tetracosin®). Because of the ACTH shortage in Thailand during the period from May 2010-March 2014, only LDT (1 μg) was used during that period and HDT was employed from April 2014-December 2015. ACTH 1 μg was prepared under sterile conditions by the hospital pharmacy. Briefly, a 250 μg ampule of ACTH was diluted with normal saline and transferred to a 1 ml syringe and stored at 2–8 °C.

Definitions

A serum morning cortisol level sample was defined as a serum cortisol sample drawn at 0800 h while serum basal cortisol was a sample drawn between 0900 h-1300 h and before ACTH administration (0-minute cortisol). AI was defined as a peak serum cortisol level <500 nmol/L (18 μg/dL) at 30 or 60 minutes after LDT or HDT [3]. A history of glucocorticoid use was defined as the use of any type and form of glucocorticoids for at least three weeks before the tests. A history of traditional medicine use was defined as a documented history of personal use of any herbal or traditional medicine suspected of being adulterated with glucocorticoids. Symptoms of AI were defined as any symptom of fatigue, weight loss, syncope, intractable nausea and vomiting, or orthostatic hypotension documented in the medical record.

Statistical analysis

The data were analyzed by STATA (Stata Corp., College Station, TX, USA). The statistical significance level was set as P-value < 0.05 for two-tailed tests. Categorical variables are presented as counts or percentages, and numerical variables are presented as means and ranges. Categorical variables were analyzed by the Fisher exact test and numerical variables by the t-test or Mann-Whitney U test, as appropriate. The 95% confidence interval (95% CI) was provided. The associations between AI and the serum morning cortisol level, serum basal cortisol level were analyzed using a logistic regression model. The areas under the ROC curves (AUC) of the model were plotted to determine the diagnostic performance of each value. Each cut-off value was calculated as the lowest (lower cut-off) or highest (upper cut-off) values, based on the highest sensitivity or specificity. The interval for each cut-off level was more than 10% of the prior cut-off levels based on the coefficients of variation of the serum cortisol test or 25 nmol/L, as appropriate. The sensitivity, specificity, PPV, NPV, likelihood ratio of positive (LHR+), likelihood ratio of negative (LHR-) and AUC were reported for each proposed cut-off point. Missing values >5% were inferred with multiple imputation analysis.

Results

Baseline characteristics

This study included 416 suspected AI patients (191 males and 225 females). The baseline characteristics are depicted in Table 1. Among all patients, 22.4% (n = 93/416) had a definite diagnosis of AI. The mean age was 50.0 (16.0–94.0) years. The mean age in the AI group was significantly higher than that in the no AI group. Most of the patients had the presence of AI symptoms as the indication for testing (35.3%). Those with a history of exogenous glucocorticoids use, traditional medicine use and pituitary tumor as the indication for testing had a higher probability of AI than no AI. Approximately 55% of the patients had undergone HDT. There was no significant difference in the AI results between LDT and HDT.

Table 1

Baseline characteristics.

Characteristics	Adrenal Insufficiency(n = 93)	Normal Response(n = 323)	P-Value
Age, Mean (range) (years)	55.6 (16.0–94.0)	48.4 (16.0–88.0)	< 0.001
Gender, N (%)
○ Male	40 (43.0)	151 (46.8)	0.524
○ Female	53 (57.0)	172 (53.2)
Body Weight, Mean (range) (kg)	58.4 (28.0–95.0)	61.5 (29.6–150.3)	0.111
Underlying Diseases, N (%)
○ Autoimmune Diseases	14 (15.1)	34 (10.5)	0.229
○ Diabetes Mellitus	12 (12.9)	48 (14.9)	0.628
○ Hypertension	25 (26.9)	70 (21.7)	0.298
○ Coronary Artery Disease	11 (11.8)	11 (3.4)	0.001
○ Malignancy	2 (2.2)	6 (1.9)	0.856
○ Others	69 (74.2)	205 (63.7)	0.059
○ No known underlying disease	1 (1.8)	5 (1.5)	0.823
Indication for Testing, N (%)
○ Exogenous Steroid Use	44 (47.3)	62 (19.2)	<0.001
- Prednisolone	14 (27.6)	24 (16.6)	0.090
- Dexamethasone	1 (2.6)	2 (1.6)	0.685
- Topical	0 (0.0)	3 (0.9)	0.351
- Herb or Traditional Medicine Use	27 (45.8)	32 (21.5)	<0.001
○ Post-Pituitary Surgery	11 (11.8)	69 (21.4)	0.040
○ Pituitary Tumor	8 (8.6)	6 (1.9)	0.001
○ Other Pituitary Hormonal Deficiencies	25 (26.9)	103 (31.9)	0.357
○ Symptoms of Adrenal Insufficiency	33 (35.5)	114 (35.3)	0.973
Baseline SBP, Mean (range) (mmHg)	119.9 (72.0–202.0)	121.1 (78.0–199.0)	0.646
Baseline DBP, Mean (range) (mmHg)	70.9 (83.0–113.0)	73.4 (50.0–115.0)	0.125
ACTH Stimulation Dose, N (%)
○ 1μg	37 (39.8)	148 (45.8)
○ 250μg	56 (60.2)	175 (54.2)	0.302
Serum Morning Cortisol, Mean (range) (nmol/L)	217.6 (83.8–470.6)	266.2 (88.0–492.9)	< 0.001
Serum Basal Cortisol, Mean (range) (nmol/L)	172.7 (1.4–406.5)	304.2 (25.7–891.8)	< 0.001
Cortisol at 30 Min, Mean (range) (nmol/L)	326.7 (44.1–491.2)	671.6 (280.1–1749.8)	< 0.001
Cortisol at 60 Min, Mean (range) (nmol/L)	351.4 (72.6–497.6)	743.6 (370.9–1749.8)	< 0.001
Serum Albumin, Mean (range) (g/L)	36.2 (12.0–53.0)	39.6 (10.0–65.2)	0.001
Serum Cholesterol, Mean (range)(mmol/L)	4.8 (1.3–8.6)	4.8 (1.5–14.4)	0.883
Serum Creatinine, Mean (range)(μmol/L)	94.4 (35.4–583.4)	80.1 (26.5–822.1)	0.098

SBP: systolic blood pressure

DBP: diastolic blood pressure

Range = Minimum—Maximum

Data for the 0800 h serum morning cortisol level, basal cortisol level, 30-minute cortisol level and 60-minute cortisol level after ACTH stimulation tests categorized by the presence of absence of AI are depicted in Fig 1. In those with a normal adrenal response higher levels of cortisol were observed for all values compared to those with AI (all p-values were <0.001).

Fig 1

Box plot graph of serum basal cortisol, serum morning cortisol, 30 and 60-minutes cortisol after ACTH stimulation test categorized by adrenal insufficiency status.

Diagnostic performance and cut-off values for serum morning and basal cortisol levels

The diagnostic performance of both the serum morning and basal cortisol levels after adjustment for age, sex, serum albumin level, cholesterol level, serum creatinine level and ACTH dose yielded significantly different AUCs (p <0.001). The covariate-adjusted AUC for the serum basal cortisol level was 0.82 (95% CI: 0.75–0.88), while that for the serum morning cortisol was 0.69 (95% CI: 0.62–0.78) (Fig 2). The data for univariate and multivariate analyses of the diagnostic performances of the morning and basal cortisol levels are shown in Table 2.

Fig 2

Covariate-adjusted AUCs of 0800 h serum morning cortisol and basal cortisol levels for the diagnosis of AI (adjusted for age, sex, serum albumin level, cholesterol level, creatinine level and ACTH dose).

Table 2

Univariate and multivariate analyses for 0800h serum morning and basal cortisol levels.

Cortisol	Univariate analysis model			Multivariate analysis model*
	AUC	95%CI	p-value	AUC	95%CI	p-value
Serum morning cortisol	0.65	0.58–0.71	<0.001	0.69	0.62–0.78	<0.001
Serum basal cortisol	0.79	0.74–0.85		0.82	0.75–0.88

*Multivariate analysis was adjusted for age, gender, serum albumin, cholesterol, creatinine and ACTH stimulation test doses

The proper lower cut-off value for the use of the 0800 h serum morning cortisol level to rule in AI which gave the highest specificity of 99.7% was ≤90 nmol/L (3.3 μg/dL) with a sensitivity of 4.3%. The optimal cut-off value for the serum morning cortisol level which had the highest sensitivity to rule out AI, was ≥380 nmol/L (13.8 μg/dL), with a sensitivity of 98.9% and specificity of 12.4%. The serum morning cortisol level was below the proposed lower cut-off in 1.2% (n = 5/416) of the subjects, whereas it was above the proposed upper cut-off value in 9.9% (n = 41/416). If the 0800 h serum morning cortisol cut-off value of <90 nmol/L was applied, only one patient would have received a false positive diagnosis of AI. Likewise, only one patient would have received a false negative diagnosis of AI if the 0800 h serum morning cortisol cut-off value of >380 nmol/L was applied.

For the serum basal cortisol level, the cut-off to diagnose AI with the highest specificity was ≤85 nmol/L (3.1 μg/dL), with a specificity of 99.7% and a sensitivity of 24.7% while the cut-off to rule out AI with the highest sensitivity was ≥350 nmol/L (12.7 μg/dL), with a sensitivity of 98.9% and specificity of 32.2%. Using these proposed cut-off levels, the serum basal cortisol level was below the lower cut-off point in 5.8% of patients (n = 24/416), while it was above the upper cut-off level in 25.4% (n = 106/416). If the serum basal cortisol cut-off level of <85 nmol/L was used, only one patient would have been falsely diagnosed as not having AI, while if the serum basal cortisol cut-off level of >350 nmol/L was applied, one patient would have been falsely diagnosed as not having AI.

Based on the AUCs, at each cut-off level, the serum basal cortisol had higher diagnostic accuracy than the 0800 h serum morning cortisol level. The data for other cut-off levels are presented in Table 3.

Table 3

Accuracy of the cut-off level for 0800 h serum morning and basal cortisol levels.

	Level (nmol/L)	Sensitivity (95%CI)	Specificity (95%CI)	PPV (%)	NPV (%)	LHR+	LHR-	TP (n)	FN (n)	FP (n)	TN (n)	AUC (95% CI)	Adjusted AUC (95%CI)
Serum Morning Cortisol
- Lower	<90	4.3 (1.2–10.6)	99.7 (98.3–100)	80.0	78.3	13.89	0.96	4	89	1	322	0.52 (0.50–0.54)	0.66 (0.56–0.74)
	<100	7.5 (3.1–14.9)	99.1 (97.3–99.8)	70.0	78.8	8.10	0.93	7	86	3	320	0.53 (0.51–0.56)	0.66 (0.57–0.75)
	<125	15.1 (8.5–24.0)	96.0 (92.6–97.3)	51.9	79.7	3.74	0.89	14	79	13	310	0.56 (0.52–0.59)	0.67 (0.58–0.76)
	<150	20.4 (12.8–30.1)	88.5 (84.6–91.8)	33.9	79.4	1.78	0.90	19	74	37	286	0.54 (0.50–0.59)	0.66 (0.56–0.75)
	<175	33.3 (23.9–43.9)	83.3 (78.8–87.2)	36.5	81.3	1.99	0.88	31	62	54	269	0.58 (0.53–0.64)	0.67 (0.58–0.76)
- Upper	>380	98.9 (94.2–100)	12.4 (9.0–16.5)	24.5	97.6	1.13	0.09	92	1	283	40	0.56 (0.54–0.58)	0.68 (0.60–0.76)
	>340	93.5 (86.5–97.6)	22.3 (17.9–27.2)	25.7	92.3	1.20	0.29	87	6	251	72	0.58 (0.54–0.61)	0.69 (0.60–0.77)
	>300	84.9 (76.0–97.5)	34.4 (29.2–39.8)	27.1	88.8	1.29	0.44	75	14	212	111	0.60 (0.55–0.64)	0.71 (0.63–0.78)
	>270	68.8 (58.4–78.0)	44.3 (38.8–49.9)	26.2	83.1	1.23	0.70	64	29	180	143	0.57 (0.51–0.62)	0.66 (0.57–0.75)
Serum Basal Cortisol
- Lower	<85	24.7 (16.4–34.8)	99.7 (98.3–100)	95.8	82.1	79.88	0.76	23	70	1	322	0.62 (0.58–0.67)	0.72 (0.63–0.81)
	<100	24.7 (16.4–34.8)	99.4 (97.8–99.9)	92.0	72.1	39.94	0.76	23	70	2	321	0.62 (0.58–0.66)	0.72 (0.63–0.81)
	<125	31.2 (22.0–41.6)	96.3 (93.6–98.1)	70.7	82.9	8.39	0.71	29	64	12	311	0.64 (0.59–0.69)	0.73 (0.64–0.82)
	<150	36.6 (26.8–41.2)	91.6 (88.1–94.4)	55.7	83.4	4.37	0.69	34	59	27	296	0.64 (0.59–0.69)	0.73 (0.65–0.82)
	<175	50.5 (40.0–61.1)	83.9 (79.4–87.7)	47.5	85.5	3.14	0.59	47	46	52	271	0.67 (0.62–0.73)	0.76 (0.68–0.84)
- Upper	>350	98.9 (94.2–100)	32.2 (27.1–37.6)	29.6	99.0	1.46	0.03	92	1	219	104	0.66 (0.63–0.68)	0.76 (0.70–0.83)
	>310	93.5 (86.5–97.6)	41.8 (36.4–47.4)	31.6	95.7	1.61	0.15	87	6	188	135	0.68 (0.64–0.71)	0.78 (0.71–0.84)
	>275	86.0 (77.3–92.3)	50.8 (45.2–56.4)	33.5	92.7	1.75	0.28	80	13	159	164	0.68 (0.64–0.73)	0.76 (0.68–0.83)
	>250	80.6 (71.1–88.1)	57.3 (51.7–62.7)	35.2	91.1	1.89	0.34	75	18	138	185	0.69 (0.64–0.74)	0.75 (0.68–0.83)

PPV: Positive predictive value

NPV: Negative predictive value

LHR+: Likelihood ratio positive

LHR-: Likelihood ratio negative

TP: True positive

FN: False negative

FP: False positive

TN: True negative

Discussion

This study highlighted a major finding, which is that the basal cortisol level between 0900 h and 1300 h has a statistically higher diagnostic performance than the serum morning cortisol level at 0800 h. Moreover, the proper cut-off values for both basal and morning cortisol levels to establish a diagnosis of AI with appropriate sensitivity and specificity were proposed.

We demonstrated that there is a diagnostic benefit of using the basal cortisol level to establish a diagnosis of AI. Additionally, the use of basal cortisol to diagnose AI has been discussed in multiple studies [3, 10]. Intriguingly, our study found novel data that the basal cortisol level has a statistically higher diagnostic accuracy for AI than the morning cortisol level. In our opinion, the basal cortisol level determined from samples taken during the period 0900 h-1300 h can be used as an alternative method for screening patients for AI, obviating the need to perform serum morning cortisol testing at a specific timepoint (0800 h), resulting in more flexibility and applicability in clinical practice. However, using the AUC alone to compare the diagnostic accuracy may mislead clinicians. The results provided were only the diagnostic accuracy and the comparison of that accuracy. However, in real life, clinicians may be interested in using the lower and upper cut-off levels to rule in and rule out AI and should know the error that may occur when using these cut-off levels.

We have proposed the appropriate diagnostic cut-off values for both the morning and basal cortisol levels. The populations with suspected AI who had a serum morning cortisol levels between 83–499 nmol/L (3–17.9 μg/dL) were included. It was demonstrated that 22.4% of those who were tested with ACTH stimulation failed the test. Therefore, more than two-thirds of unnecessary procedures could be eliminated by defining the appropriate cut-off points for unstimulated serum cortisol levels. Our study established the optimal cut-off for the 0800 h morning cortisol levels as <90 nmol/L (3.3 μg/dL) and >380 nmol/L (13.8 μg/dL) for the lower and upper values, respectively. Moreover, the proper cut-off levels for basal cortisol were <85 nmol/L (3.1 μg/dL) and >350 nmol/L (12.7 μg/dL), respectively. The lower cut-off value with the highest specificity was selected to reduce the number of patients falsely diagnosed with AI because treating AI with physiologic dose glucocorticoids may harm the patients who do not need the treatment. Likewise, as AI is a lethal and life-threatening disease if left undiagnosed, the upper cut-off level to rule out AI that had the highest sensitivity was chosen to reduce the number of false negative diagnoses. Only one patient fell into the false negative groups, and one was categorized in the false positive group when the proposed upper and lower cut-off levels were employed. This demonstrated the very low risk of error. However, as having a higher sensitivity may lead to a greater number of false positive patients who may need further ACTH stimulation tests, we suggest that these proposed values may need to be applied together with clinical symptoms to form the basis for clinical decisions regarding further investigations. Our proposed upper cut-off value was different from the conventional cut-off level and from the values proposed in other studies [8-10]. Of note, the difference could be, in part, the results of the diverse population, variability in the serum cortisol assays performed and differences in the criteria for the diagnosis of AI applied in each study. If our proposed cut-off levels for serum morning and basal cortisol levels were employed instead of the conventional cut-off levels, nearly 10% and 30% (true positive and true negative rate) of the number of ACTH stimulation tests could be reduced, respectively, while maintaining high levels of sensitivity and specificity. The reduction in the number of dynamic tests needed was in concordance with the findings of other studies [9, 10]. According to our data, basal cortisol levels had a statistically higher diagnostic performance than morning cortisol levels. Hence, in terms of using the proposed cut-off levels, the basal cortisol tests appear to be an option for use in diagnosis of AI.

The present study diagnosed AI based on a peak cortisol <500 nmol/L (18 μg/dL) which was based on the first-generation cortisol assay [3]. The newer second generation assay (Cortisol II) used in other studies revealed an approximately 30% lower serum cortisol level which may lead to the overdiagnosis of AI if the conventional cut-off level was employed [11, 12]. Therefore, our proposed lower and upper cut-off levels may not be applied to the newer assays or the first-generation cortisol assays other than the Roche Diagnostic assay. Thus, we recommend that the basal cortisol levels with the proposed lower and upper cut-off levels be employed as the screening test option and that AI symptoms should be taken into consideration before proceeding to further dynamic tests.

Our study has multiple strengths. We have demonstrated novel finding that the application of basal cortisol levels results in a high diagnostic value. Moreover, the optimal diagnostic cut-off for the basal cortisol level has been presented if this value was applied in clinical practice, it would be possible to avoid many unnecessary dynamic tests. This study also had a large number of patients, which makes the results broadly applicable to real-world practice. The confounding variables have been appropriately adjusted for in multivariable analysis, including serum creatinine, cholesterol and albumin levels, which multiple studies have demonstrated their associations with the changes in serum cortisol levels [13-15].

We acknowledge some limitations in our study. Both LDT and HDT were included in our study; thus, there was a lack of uniformity in the test procedures. In our institution, we have no specific criteria to choose whether LDT or HDT is appropriate for a specific group of patients. Although both LDT and HDT are still the standard dynamic tests that have been endorsed in many institutions [10, 16–19], whether the difference in the dosage of ACTH can influence the outcomes especially when using the same cut-off levels for both LDT and HDT to diagnose AI has not been definitively demonstrated. This might lead to the inaccuracy of the outcomes However, the ACTH dosage was used as one of the co-variables in our final model, and the potential confounding from dosage variability was thereby reduced. The gold standard test (insulin tolerance test) was not performed in our cohort. Therefore, patients with partial or recent secondary AI may have been misdiagnosed. Additionally, the population in this study was composed of patients with indeterminate results of the 0800 h serum cortisol level (83–499 nmol/L). Thus, these results may be applied only to this subgroup of the population with equivocal serum cortisol levels. Another limitation is that a diverse group of patients with both primary and secondary AI were included. Thus, the results may not be accurate for some subgroups of populations. On the other hand, this limitation makes the results more generalizable to the population usually encountered in clinical practice. Another limitation was that the exact times at which the basal cortisol level and ACTH stimulation test results were obtained were not provided, although most of the patients were tested before noon. Because the serum cortisol level follows a diurnal pattern and the timing of the test could affect the results [11], this limitation may alter the outcomes. However, a prior study stated that a single measurement of serum cortisol at 0800–1200 h can demonstrate a high sensitivity of >95%. Therefore, testing for the serum cortisol level in this proposed time range may still be beneficial in terms of convenience and flexibility [20]. The retrospective nature of our study was another limitation. Accordingly, the findings of this study should be validated in future prospective studies.

In conclusion, the basal cortisol level is considered a good alternative screening test for diagnosis of AI, with a diagnostic value which is not inferior to 0800 h serum morning cortisol level. Although, a number of ACTH stimulation tests are still required, utilizing this value makes the diagnostic procedure even easier. The proper cut-off values for the basal cortisol level were suggested, and these values have very high sensitivity and specificity. Apart from the number of dynamic tests that can be omitted, performing basal cortisol level tests is convenient, not time-dependent and can reduce expenses. Larger prospective studies in the future may be needed to confirm and validate our results.

16 Aug 2019

PONE-D-19-19551

Basal cortisol level at 0900h-1300h has higher diagnostic performance than 0800h cortisol level to diagnose adrenal insufficiency

PLOS ONE

Dear Dr. manosroi,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The authors address a potential important practical issue regarding the timing of cortisol measurement.

The reviewers have been rather divided on this paper, but I feel it may be suitable for publication if thoroughly revised.

In addition to the points raised by the reviewers, please focus on the difference in measurement of cortisol depending on assay methods. I presume that the assay used was Elecsys® Cortisol II assay (Roche Diagnostics GmbH, Mannheim, Germany), is the suggested cut-off values for this assay correct? You may consult Vogeser, M., Kratzsch, J., Bae, Y. J., Bruegel, M., Ceglarek, U., Fiers, T., ... & Suhr, A. C. (2017). Multicenter performance evaluation of a second generation cortisol assay. Clinical Chemistry and Laboratory Medicine (CCLM), 55(6), 826-835.

Furthermore, please use SI units (with conventional units in parenthesis) and carefully proof read the manuscript before re-submission. The paper could benefit of professional English proof reading. I can recommend AJE https://www.aje.com/.

We would appreciate receiving your revised manuscript by Sep 30 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Pal Bela Szecsi, M.D. D.M.Sci.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. Thank you for including your ethics statement:

"The study protocol was approved by the local Research Ethics Committee".

i) Please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

ii) Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript:

This study was supported by the Faculty of Medicine, Chiang Mai University.

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

The author(s) received no specific funding for this work.

5. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Manosroi et al. have examined the diagnostic performance of the time 0 cortisol drawn during a standard or low dose ACTH stimulation test undertaken between 0900h-1300h compared to a 0800h cortisol, to diagnose adrenal insufficiency in a patient cohort from Thailand. They conclude that a cortisol drawn between 0900h-1300h has a higher diagnostic performance than an 0800h cortisol in diagnosing adrenal insufficiency. Specific comments:

1. The main limitation of the study is the inappropriate use of a cortisol cut-off of 500 nmol/L in the ACTH stimulation test with the Roche Electrosys Cortisol II assay. Two separate publications have suggested a much lower cut off of 350 nmol/L (12.7 mcg/dL, Kline et al. Clin Biochem 2017) and 374 nmol/L (13.6 mcg/dL, Raverot et al. Ann Endocrinol 2016). Therefore using the 500 nmol/L or 18 mcg/dL as generically recommended in Ref 3 will markedly over-estimate the true prevalence of adrenal insufficiency in this cohort. It is now well established that assay-specific reference ranges are required to be used (El-Farhan et al. Clin Endocrinol 2013). This then makes the interpretation of any relationship between diagnosed adrenal insufficiency and basal cortisol (be it at 0800h or 0900-1300h) almost meaningless.

2. None of the cortisol data are depicted, the only figure is the ROC curve. Could the authors clarify if the criteria for a passed test was a cortisol of >18 mcg/dL at either 30 or 60 minutes, with the same criteria for both low dose and high dose tests?

3. Generally when establishing reference intervals, the 95% confidence interval in a normal population is used, so the upper cut point would be a 97.5% CI. Why did the authors use a >99% sensitivity/specificity when proposing their upper and lower cut points?

4. The findings are of interest, and the potential flexibility of being able to usefully interpret a cortisol drawn any time before 1300h would be beneficial for patients and clinicians alike. The main difference in the study between the "morning" cortisol and "basal" cortisol was the upper cut off of 16 vs 12.5 mcg/dL. My suspicion is that many of the patients who failed the ACTH stimulation test whose "morning" cortisol was between 12.5 mcg/L and 16 mcg/L probably in fact had normal function if an appropriate stimulated cortisol cut-off was applied.

Reviewer #2: 1) See attached file -> It is not clear why analysis was adjusted for cholesterol and creatinine.

2) Please see attached file.

3) If I am correct, there was no reason stated why the data should be available only upon request.

4) Please see attached file.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Tristan Struja

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

27 Aug 2019

Responses to reviewers

We thank the reviewers for their careful reading of the manuscript and their constructive comments. We have taken all the comments to improve and clarify the manuscript. Please find below a detailed point-by-point response to all comments (reviewers’ comments in bold black, our replies in non-bold black).

Since the reordering and restructuring of the manuscript was substantial, we have written bullet points of our major changes to the manuscript. Also, line numbering was referred to the revised manuscript without track changes.

• The manuscript has been edited by English language editing service (please see the attached certificate from AJE)

• In order to reduce the over estimation of AI prevalence, the cut-off levels to diagnose AI for serum cortisol post-ACTH stimulation test was modified to 420 nmol/L (instead of 500 nmol/L) according to proposed cut-point for second generation Cortisol II assay

• All the units have been changed to SI with conventional units in parentheses

• Accordingly, 29 patients were excluded from the cohort, as the inclusion criteria was changed to patients with morning cortisol between 86-420 nmol/L

• Statistical power for our final sample size was still more than 80%

• The main concept and results are still the same

• New Table 2 was created to demonstrated uni- and multivariate analyses of basal and morning cortisol

• New Table 3 for each cut-off levels was created, and multiple cut-off levels were added with the interval of 25 nmol/L or lower

• New Figure 1 was created to demonstrate and compare multiple serum cortisol values between patients with and without AI

Reviewer#1

Reviewer #1: Manosroi et al. have examined the diagnostic performance of the time 0 cortisol drawn during a standard or low dose ACTH stimulation test undertaken between 0900h-1300h compared to a 0800h cortisol, to diagnose adrenal insufficiency in a patient cohort from Thailand. They conclude that a cortisol drawn between 0900h-1300h has a higher diagnostic performance than an 0800h cortisol in diagnosing adrenal insufficiency. Specific comments:

1. The main limitation of the study is the inappropriate use of a cortisol cut-off of 500 nmol/L in the ACTH stimulation test with the Roche Electrosys Cortisol II assay. Two separate publications have suggested a much lower cut off of 350 nmol/L (12.7 mcg/dL, Kline et al. Clin Biochem 2017) and 374 nmol/L (13.6 mcg/dL, Raverot et al. Ann Endocrinol 2016). Therefore using the 500 nmol/L or 18 mcg/dL as generically recommended in Ref 3 will markedly over-estimate the true prevalence of adrenal insufficiency in this cohort. It is now well established that assay-specific reference ranges are required to be used (El-Farhan et al. Clin Endocrinol 2013). This then makes the interpretation of any relationship between diagnosed adrenal insufficiency and basal cortisol (be it at 0800h or 0900-1300h) almost meaningless.

Thank you for this very helpful comment.

After thoroughly reviewed the literatures regarding the assay-specific cut-off levels for cortisol II assay, we totally agree with your comment (Javorsky et al. J Endocr Soc. 2019 Apr 15; Klein et al. Clin Biochem 2017; El-Farhan et al. Clin Endocrinol (Oxf).2013 May; Vogeser et al. Clin Chem Lab Med. 2017). Endocrine Society Guideline for adrenal insufficiency also stated that the appropriated cut-off for ACTH stimulation tests should be assay-specific. After discussing with all the authors, we agreed to adjust the diagnostic criteria of adrenal insufficiency by using a lower peak serum cortisol of 420 nmol/L (15.1 µg/dL) instead of 500 nmol/L which based on the data from El-Farhan et al. Clin Endocrinol (Oxf).2013. This cut-off was derived from the lower reference limit at 30 min after ACTH stimulation standardized by GC-MS method. Therefore, this could avoid the overdiagnosis prevalence of adrenal insufficiency in our population and our main results could be applied in real practice

Therefore, the analyzed data have been changed accordingly including the number of total populations. We have changed our inclusion criteria to those who had 0800hr serum cortisol between 83-420 nmol/L (3-15.1 µg/dL) instead of 83-500 nmol/L (3-18 µg/dL). Therefore, only 29 patients were excluded from the cohort. The prevalence of adrenal insufficiency was decreased from 23.8% to 21.5% and there were some changes in the proposed cut-off levels as well. However, the main idea and results are still the same. Also, the new acquired co-variate adjusted ROC for serum basal cortisol was higher than the previous result. Also, the calculated power for the new sample size was still adequate (>80%) (Bujang et al. J Clin Diagn Res. 2016 Oct). We hope that you will agree with our new peak cortisol cut-off levels for ACTH stimulation tests.

Using 500 nmol/L for peak cortisol level is the number we normally use in our institution according to the Endocrine Society Guideline. We hope that we will change our practice of using the new cut-off levels for peak cortisol after ACTH stimulation soon. Thank you.

2. None of the cortisol data are depicted, the only figure is the ROC curve. Could the authors clarify if the criteria for a passed test was a cortisol of >18 mcg/dL at either 30 or 60 minutes, with the same criteria for both low dose and high dose tests?

Thank you for raising this point. We have provided the data of all cortisol levels categorized by adrenal insufficiency or no adrenal insufficiency status in box-plot diagram in Fig 1(new).

The Endocrine Society Guideline of adrenal insufficiency recommended the same cut-off for peak cortisol after ACTH stimulation tests of both low and high dose tests. Therefore, the authors came to the conclusion that we will use the same criteria for adrenal insufficiency for both low and high dose ACTH test are the same which is 420 nmol/L. We have added that in the manuscript in L99-100. Also, we have used ACTH dose (low or high dose) as one of the covariates and were adjusted in multivariate model in order to reduce the variability of the dose ACTH used.

3. Generally when establishing reference intervals, the 95% confidence interval in a normal population is used, so the upper cut point would be a 97.5% CI. Why did the authors use a >99% sensitivity/specificity when proposing their upper and lower cut points?

Normally, the 95%CI was used to estimate the range of the point estimate or to document the percentile. The way we chose each cut-off level is not depend on the 95%CI of overall patients in the cohort. We manually chose a very high sensitivity (>99%) for upper cut-off level to rule-out adrenal insufficiency as this is a lethal condition if we misclassify them. Likewise, the lower cut-off to rule in adrenal insufficiency should have a very high specificity, because to prescribe physiologic dose of corticosteroid may do harm to the patients if we over-diagnose them. Therefore, we finely adjusted each cut-off value level by level with the interval of 25 nmol/L or lower, as appropriate. Therefore, each cut-off level has its own sensitivity, specificity and 95%CI.

We have revised Table3 (new) and provided the data of 95%CI for each proposed cut-off sensitivity and specificity. We also have discussed the reason why we chose a very high sensitivity and specificity for each cut-off levels in L220-230 (discussion section).

4. The findings are of interest, and the potential flexibility of being able to usefully interpret a cortisol drawn any time before 1300h would be beneficial for patients and clinicians alike. The main difference in the study between the "morning" cortisol and "basal" cortisol was the upper cut off of 16 vs 12.5 mcg/dL. My suspicion is that many of the patients who failed the ACTH stimulation test whose "morning" cortisol was between 12.5 mcg/L and 16 mcg/L probably in fact had normal function if an appropriate stimulated cortisol cut-off was applied.

Thank you for raising this issue.

We have applied the new cut-off level for stimulated cortisol and the new proposed upper cut-off to rule-out AI in our study were >350 nmol/L (12.6 µg/dL) for 0800h morning cortisol and >337 nmol/L (12.2 µg/dL) for serum basal cortisol. Comparing to our previous results, the upper cut-off for 0800h morning cortisol has decreased from 440 to 350 nmol/L.

Further analysis

Serum morning cortisol

Basal cortisol >350 nmol/L <350 nmol/L

>337 nmol/L 14 91

<337 nmol/L 31 306

The new upper cut-off for both morning and basal cortisol (350 and 337 nmol/L) showed smaller interval compared to our previous one (440 and 347 nmol/L)

There were 91 patients who had levels between 337-350 nmol/L. Based on our analysis, only one (1/91) patient in this group was truly diagnosed with adrenal insufficiency and the other 90 patients had normal adrenal function. This could presume that our proposed cut-off level to rule out AI particularly from basal cortisol had fairly high accuracy.

Reviewer#2

Manosroi et al. present an interesting piece of research. In my opinion, it can add substantial guidance to the community, but it needs thorough revision first.

Thank you for the positive and encouraging comments.

Minor:

1) L 50: Please provide cortisol reference ranges in international units at least the first time they appear in the text.

Thank you. We have changed all units in the text to SI with conventional units in the parentheses.

2) L56: remove “there is” as it is unnecessary

It was removed as suggested.

3) L73 and 74: Rewrite to ….estrogen, patients with suspected congenital adrenal hyperplasia…

Thank you. The sentence has been rewritten as suggested.

4) L99: Rewrite “…history of personal using of…” to “…history of personal use of…”

The sentence has been rewritten as suggested.

L105 & 118 : Make sure to use the correct temporal form, multiple uses of past tense although present would be appropriate, for instance “..characteristics were depicted in Table 1.” Instead of “characteristics are depicted”.

We have corrected the sentence as suggested. Thank you.

5) Discussion: Although not a native speaker myself, I detected various semantic, and grammar mistakes in this section. They fall in the same category as those above. Please, revise your manuscript carefully.

Thank you. According to this issue, we apologize for all the grammatical mistakes and the revised manuscript has been edited by the English language editing service (please see the attached certificate from AJE).

Major:

Table 1: Numbers of categorical variables do not add up to 471. For instance, underlying diseases adds up to 268 only. Please provide on missing data. It is ok if this piece of information is retrospectively not retrievable anymore, as long it is clearly stated.

Thank you for noticing this.

Reviewer#1 has raised issue regarding the diagnostic cut-off point for Cortisol II electrosys assay We have thoroughly reviewed multiple literatures about this second generation assay and the specific cut-off levels for ACTH stimulation test which are lower than the usual cut-off points (Javorsky et al. J Endocr Soc. 2019 Apr 15; Klein et al. Clin Biochem 2017; El-Farhan et al. Clin Endocrinol (Oxf).2013 May). If the previous cut-off for cortisol (500 nmol/L) was used, this will markedly overestimate the real prevalence of adrenal insufficiency. Therefore, we agreed with Reviewer#1 and we have modified the diagnostic criteria for adrenal insufficiency by using the peak cortisol cut-off at 30 or 60 min at 420 nmol/L (15.2 µg/dL) instead of 500 nmol/L (El-Farhan et al. Clin Endocrinol (Oxf).2013 May).

Therefore, the data have been modified accordingly and the new number of all population were 442 patients as only 29 patients were excluded. Some numbers have been changed but the main idea and results are still the same. Also, the power of our new sample size is still >80% (Bujang et al. J Clin Diagn Res. 2016 Oct).

We also added the data in Table 1 for patients with no known underlying disease and other diseases. Please note that the add up numbers are higher than number of total populations as some patients may have more than one underlying disease. Also, other demographic data have been revised. There was no missing value for demographic variables. For biochemical variables apart from serum cortisol (e.g. albumin, cholesterol, creatinine), the missing variables of more than 5% were dealt with multiple imputation analysis as stated in L120-121.

L193 Rewrite “Of note, the different could be” to “Of note, the difference could be”

Thank you. The sentence has been rewritten.

Table 2: Authors should provide cortisol in international units in the legend to facilitate understanding for readers from Europe. Also, it would be prudent extend the table to include various ranges of cut-offs (maybe in steps of 2ug/dl?). This way readers can judge for themselves which cut-off would be optimal for their need.

As per Plos One manuscript guideline, we have changed all units to SI units with conventional units in parentheses.

We have modified an old Table 2 to the (new) Table 3 which included multiple cut-off values with the interval of 25 nmol/L (~1 µg/dL) or lower. 95%CI of sensitivity, specificity, adjusted AUC and non-adjusted AUC for each cut-off level were provided.

What I liked is that numbers of true negatives, true positives etc. were presented. This really helps to find out how many patients could potentially be misclassified, if the proposed cut-off were applied in practice.

Your encouraging comment is greatly appreciated. Thank you.

Authors should include a discussion how they arrived at the proposed cut-offs. Were they stated a priori or a posteriori?

We have added the statement in the statistical analysis section L116-118. Also, in the discussion section, we have stated the reason why we have to choose the upper cut-off with the highest sensitivity and the lower cut-off with the highest specificity in L220-230.

Furthermore, it is misleading to state an AUC of 0.82 in the abstract that is derived from figure 1, but nowhere presented (such as in table 2). Additionally, discussion should include a statement on the predictive abilities of ROC curves themselves. In case of rare events (what an AI in clinical practice gladly is), it tends to give overoptimistic results. Also, the overall AUC might not be what a reader is interested in, as it can be deceptive. Although AI is rare but potentially deadly, a clinician wants to minimize the numbers of false negatives. Therefore, it is important that these figures are presented in table 2.

We have created (new) Table 2 with data of both univariate and multivariate analyses of diagnostic accuracy for serum morning and basal cortisol. Also, both covariate-adjusted and non-adjusted AUCs were demonstrated in Table 2.

The discussion regarding the diagnostic accuracy based on ROC alone was added on L207-211 and L223-227.

I am honestly unsure how creatinine and serum cholesterol could affect the level of cortisol. Authors should either remove these variables form analysis or clearly state why they included them. I might rather be sensible to include the dose of ACTH (LDT vs. HDT) used as a covariate.

Thank you for raising this issue.

We agree with you on your helpful comment on the ACTH dosage as a covariate. As the ACTH dosage itself could affect serum cortisol level, therefore, we have re-analyzed the data and adjusted for ACTH dosage in multivariate analysis as suggested.

The associations between serum cholesterol and adrenal insufficiency in cirrhotic patients have been reported in multiple literatures including human and animal studies with varying results. However, there was no report of this issue in non-cirrhotic patients (Park et al. Medicine. 2018;97(26); Spadaro et al. Scandinavian journal of gastroenterology. 2015;50(3); Ouweeneel et al. Atherosclerosis. 2017;261). As cholesterol is the precursor for glucocorticoids synthesis (Bochemet al. J Lipid Res. 2013 Jun), we assume that cholesterol levels may cause an interference with cortisol levels. We have stated this issue in the discussion part (L256-258).

However, multiple studies had reported normal findings of adrenal function in chronic renal failure patients, physiologic changes of serum cortisol have been reported in multiple studies (Clodi et al. Am J Kidney Dis. 1998;32(1); Raff et al. Endocr Connect. 2013;2(1)). Serum half-life of cortisol is prolonged in chronic renal failure and both elevated and normal cortisol levels had been reported (Bacon et al. The Johns Hopkins medical journal. 1973; Nolan et al. JCEM. 1981;52(6); Sianmopoulous et al. Peritoneal dialysis international. 1990;10(2)). Therefore, in those with poor eGFR (high serum creatinine), serum cortisol may be altered. So, we have used serum creatinine as one of the covariates. We also have stated this issue in the discussion part (L256-258).

Furthermore, results from unadjusted analysis are not presented. Please include them in a separate column in table 2.

The new Table 3 for the unadjusted and adjusted data (AUCs) was created as suggested. Thank you.

L202 I assume that Cortisol II stands for second generation cortisol immunoassays. Please elaborate more on the differences between assays.

Thank you. We also discussed more about the differences between assays and the application of the proposed cut-off levels in other assays in the discussion section. (L242-250)

L 227 and 228 “Another limitation was the exact time at which the basal cortisol and ACTH stimulation had been tested were not provided but most of the patients were tested before noon.” This sentence questions the whole validity of the study. Please present the testing times in table 1. Table 1 should also include two additional columns splitting the population into 2 groups according to a morning cortisol group and a basal cortisol group.

Thank you for raising this issue.

We apologized that the data of the specific time point at which basal cortisol was tested were not available in our cohort. The time was recorded in the range between 0900h and 1300h. Therefore, we put this issue as one of our limitation.

However, one study in year 2019, also demonstrated a high sensitivity (>95%) of serum cortisol testing in the time range between 0800h-1200h (Mackenzie et al. Clin Endocrinol. 2019). The discussion regarding this limitation was added in L277-281. Therefore, we presumed that to test cortisol with our proposed time range still has validity. We plan to perform a prospective study regarding multiple time testing if serum cortisol in the future.

Each patient in our cohort had been performed both basal and morning cortisol levels as stated in the ACTH stimulation testing protocol section. Therefore, we cannot categorize the patients into 2 groups as you suggested. To performed both tests in the same patients support our results validity as this could reduce the confounders which occur from different patients.

If I am correct, there was no reason stated why the data should be available only upon request.

After discussing with other authors, the was no restriction to our data access. We have edited that in the submission form. The URL for the data access is https://drive.google.com/open?id=1XutNq1CD57xqjOInx_VM46LLXCCkEaDa

Thank you.

Submitted filename: responses to reviewers.docx

Click here for additional data file.

19 Sep 2019

PONE-D-19-19551R1

Basal cortisol level at 0900 h-1300 h has higher diagnostic performance than the 0800 h morning cortisol level for adrenal insufficiency

PLOS ONE

Dear Dr. manosroi,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript has improved, but some issues remain. Please focus on the cut-off problem.

We would appreciate receiving your revised manuscript by Nov 03 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Pal Bela Szecsi, M.D. D.M.Sci.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Manosroi et al. have undertaken a substantial revision of the manuscript and it is significantly improved. Some issues remain to be addressed:

1. Ref 11 (El-Farhan et al.) did not use the Roche cortisol II assay, they used the older Roche (cortisol I) E170 assay. Their cut-off of 420 nmol/L is based on GCMS. The papers cited by the authors which did use cortisol II, namely Javorsky et al. and Kline et al. reported cortisol cut offs of 400 nmol/L and 350 nmol/L respectively (and a third study previously mentioned by this reviewer, Raverot et al. was 374 nmol/L). Therefore it is puzzling why the authors chose 420 nmol/L as their cortisol cut-off. Furthermore, since the data are from 2010-2017, it seems unlikely that the authors' laboratory used a cortisol II assay for this entire period.

2. I think it is misleading if not frankly incorrect to state that their proposed cortisol cut-offs demonstrated that the basal cortisol drawn later in the day had a higher diagnostic performance than a 0800h cortisol. Firstly neither the lower nor upper proposed cut-offs are significantly different from each other, when taking into account the cortisol assay CVs. The lower cut off of 89 vs 86 nmol/L and the upper cut-off of 350 vs 337 nmol/L are well within even a 5% CV boundary.

Secondly, even if one takes these as different numbers, on the analysis performed there was 1 false positive and 1 false negative in each group, with identical sensitivity at the upper cut-off and specificity at the lower cut-off.

Point 1 is essential to resolve in my opinion - the cut-off of 420 nmol/L is still at least 20 nmol/L higher than any published data on the cortisol II assay in the ACTH stimulation test, if this was in fact the assay used throughout the study period.

Taken on face value with the current data set, the conclusion of the study should be that a basal cortisol taken any time during the morning up to 1200h (it is unclear how many were done between 1200h and 1300h) provides useful information in assessing the HPA axis integrity, though the cut-offs are not significantly different from the 0800h cortisol.

Reviewer #2: My concerns have been addressed. I appreciate the extensive work of the authors in regard to language, statistics and readability.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Tristan Struja

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

21 Sep 2019

Response to reviewers

Reviewer #1: Manosroi et al. have undertaken a substantial revision of the manuscript and it is significantly improved. Some issues remain to be addressed:

1. Ref 11 (El-Farhan et al.) did not use the Roche cortisol II assay, they used the older Roche (cortisol I) E170 assay. Their cut-off of 420 nmol/L is based on GCMS. The papers cited by the authors which did use cortisol II, namely Javorsky et al. and Kline et al. reported cortisol cut offs of 400 nmol/L and 350 nmol/L respectively (and a third study previously mentioned by this reviewer, Raverot et al. was 374 nmol/L). Therefore it is puzzling why the authors chose 420 nmol/L as their cortisol cut-off. Furthermore, since the data are from 2010-2017, it seems unlikely that the authors' laboratory used a cortisol II assay for this entire period.

Response:

We would like to apologize for our mistake in stating the wrong generation of assay used for cortisol testings in the manuscript. After we have double checked with the former head of the Endocrinology Laboratory at our institution, the Elecsys Cortisol II assay has been used since January 2016 until present time. Therefore, the period between January 2010-December 2015, the cortisol tests were performed by electrochemiluminescence immunoassay using an Elecsys model 1010 (Roche Diagnostic, Laval, Quebec) which is an older assay. Please find the attached certify letter from the former head of Endocrinology laboratory from our institution.

As the serum cortisol in the main population was performed by first-generation assay, we decided to exclude those with second-generation assay. The overall population were 471 patients. There were 55 patients excluded and 416 patients included. Also, as the assay used was the first generation, the conventional cut-off (peak cortisol <500 nmol/L) for peak cortisol after ACTH stimulation test was used to diagnose AI and we have included those with indeterminate serum cortisol between 83-499 nmol/L (Bornstein. J Clin Endocrinol Metab. 2016 Feb). Although, the total sample size has reduced, the statistical power is still more than 80% (Bujang. J Clin Diagn Res. 2016 Oct)

We have re-analyzed all the data according to the new cut-off level to diagnose AI (peak cortisol <500 nmol/L).

- All the Tables (Table 1-3) and Figures (Figure1 and 2) have been modified accordingly.

- In the method section, the period of data collection L72 and 75 has been changed. The method of cortisol assay used in the study has been modified (L85-86)

- In the ACTH stimulation testing protocol section, the indeterminate cortisol levels have been adjusted to level 83-499 nmol/L (L91)

- In the definitions section, the peak serum cortisol for AI diagnosis has been changed to >500 nmol/L (L103-105)

- In the discussion section, L248-256 have been modified.

- Minimal changes of some values in baseline characterisitics, results and discussion section.

- Only minimal changes in upper and lower cut-off levels were observed with the same sensitivity and specificity.

- The main concept of the manuscript is still the same.

We again apologized for our mistake and misunderstanding regarding the assay used in the manuscript which bring to some confusions. We hope that to include only those with older assay would benefit in terms of data homogeneity. Thank you.

2. I think it is misleading if not frankly incorrect to state that their proposed cortisol cut-offs demonstrated that the basal cortisol drawn later in the day had a higher diagnostic performance than a 0800h cortisol. Firstly neither the lower nor upper proposed cut-offs are significantly different from each other, when taking into account the cortisol assay CVs. The lower cut off of 89 vs 86 nmol/L and the upper cut-off of 350 vs 337 nmol/L are well within even a 5% CV boundary.

Secondly, even if one takes these as different numbers, on the analysis performed there was 1 false positive and 1 false negative in each group, with identical sensitivity at the upper cut-off and specificity at the lower cut-off.

Response:

The reason why we stated that basal cortisol had higher diagnostic performance than morning cortisol was based on the adjusted area under ROC (Fig 2). Area under ROC is the degree of agreement between the index test (basal or morning cortisol) and the reference standard (ACTH stimulation test) in which basal cortisol level showed higher rate of agreement to ACTH stimulation tests than morning cortisol (Florkowski. Clin Biochem Rev. 2008 Aug). This AuROC in Fig2 was calculated by the overall values for basal and morning cortisol, not for just the specific cut-off level.

The upper and lower cut-off levels we have proposed for basal and morning cortisol per se did not provide data on diagnostic accuracy for the whole values. The proposed cut-off level that we have chosen to report is just the spot on the ROC curve which gave the highest sensitivity or specificity for basal or morning cortisol. Therefore, these spots cannot demonstrate the whole picture of diagnostic accuracy unlike AuROC in Fig2. (Unal. Comput Math Methods Med. 2017)

We agree with you that our proposed cut-off levels between basal and morning cortisol were almost the same number which gave the same sensitivity and specificity. However, after statistical analysis by comparing an adjusted AuROC between the lower cut-off for morning (<90 nmol/L) and basal cortisol (<85 nmol/L) as shown in the table and figure below, basal cortsol showed statistically higher AuROC (diagnostic accuracy) than morning cortisol (p<0.01). These data were not shown in the manuscript.

AuROC p-value

Morning cortisol 0.65 0.01

Basal cortisol 0.71

Figure demonstrated AuROC between proposed lower cut-off level for basal and morning cortisol

Likewise, the upper cut-off levels for basal (>350 nmol/L) and morning cortisol (>380 nmol/L) demonstrated significantly different in diagnostic accuracy as shown in the table below.

AuROC p-value

Morning cortisol 0.68 0.003

Basal cortisol 0.76

Figure demonstrated AuROC between proposed upper cut-off level for basal and morning cortisol

Also, when looking at the absolute number of patients using these cut-off levels for morning and basal cortisol, nearly 10% and 30% of the patients (true positive and true negative numbers) could bypass the ACTH stimulation tests (as mentioned in the discussion section L241).

Based on the abovementioned data, these may be inferred that serum basal cortisol and the proposed cut-off levels have higher diagnostic performance than serum morning cortisol.

Point 1 is essential to resolve in my opinion - the cut-off of 420 nmol/L is still at least 20 nmol/L higher than any published data on the cortisol II assay in the ACTH stimulation test, if this was in fact the assay used throughout the study period.

We have addressed and clarified this issue on question#1.

Taken on face value with the current data set, the conclusion of the study should be that a basal cortisol taken any time during the morning up to 1200h (it is unclear how many were done between 1200h and 1300h) provides useful information in assessing the HPA axis integrity, though the cut-offs are not significantly different from the 0800h cortisol.

Thank you for raising this issue, we have discussed this issue in the first paragraph of question#2.

Reviewer #2: My concerns have been addressed. I appreciate the extensive work of the authors in regard to language, statistics and readability.

Thank you. We are really appreciate all your helpful comments which help improved our manuscript.

Submitted filename: responses to reviewer.docx

Click here for additional data file.

18 Oct 2019

PONE-D-19-19551R2

Basal cortisol level at 0900 h-1300 h has higher diagnostic performance than the 0800 h morning cortisol level for adrenal insufficiency

PLOS ONE

Dear Dr. manosroi,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The reviewers are still not satisfied, especially with your conclusions.

Usually we only allow two revisions, but I find that the findings might have some clinical impact on the more practically level. Accordingly; allow one final revision, but please take the issues raised by the first reviewer into account.

We would appreciate receiving your revised manuscript by Dec 02 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Pal Bela Szecsi, M.D. D.M.Sci.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper has gone though a number of revisions as the authors sort out exactly how cortisol was measured at their institution. Now they have confined their analysis to the old cortisol I assay and present an entirely new set of results.

1. While I accept the authors' statistical analysis, I do not accept the conclusion that a cortisol done at any time of the morning is inherently superior and a more accurate predictor of adrenal insufficiency than a 0800h cortisol. It makes no sense to me how in the "normal" group, the 0800h cortisol was numerically lower than the basal cortisol (Table 1). I do agree that it is feasible to establish cortisol levels at different times of the day above and below which an ACTH stimulation test is not required to rule in or rule out adrenal insufficiency.

2. As outlined in my previous comments, the proposed cut-offs for "morning" and "basal" cortisol lie well within the overlapping coefficients of variation of the assay. The analysis is based on an assay no longer in use and cannot necessarily be extrapolated to the new cortisol II assay.

3. While acknowledged in the Discussion, mixing the low and standard dose ACTH test and using the same cortisol cut-off for each is a weakness and source of inaccuracy.

4. In the Abstract (Results), please change the cortisol units from mcg/dL to nmol/L - should read 350 nmol/L

5. In Table 1, please remove the decimal points when examining cortisol and other parameters (inappropriate to measure cortisol to 2 decimal points). Furthermore, the way the data are now presented, mean and range would be more illustrative than mean and SD.

Reviewer #2: L.90-91

Those who had serum morning (0800 h) cortisol levels that fell into intermediate levels of

83-499 nmol/L (3-17.915.1 μg/dL) proceeded to either LDT or HDT.

As a retrospective study, and in light that the cut-offs have changed during the review process, it is misleading to state that patients were not subjected to a LDT/HDT. Rather write that these cases were classified as AI/non-AI.

L 246-247 Hence, in terms of using the proposed cut-off levels, the basal cortisol tests appear to be preferable over the 0800 h morning cortisol tests to facilitate AI diagnosis.

I understand my co-reviewers concerns on the question of morning cortisol vs. basal cortisol. There is no considerable difference between these two measurements. Also, clinical application is questionable, as slot for blood draw need to be filled anyway. Therefore, I suggest rephrasing above lines (and similar ones) into a more conciliatory tone. For instance, one could stipulate that the basal cortisol is as good as the morning cortisol, and that there is no specific need for drawing distinct morning cortisol values.

Furthermore, this issue could be alleviated by updating figure 2 with 95% confidence bands around the ROC curves.

Table 1

As your paper is based on the reliability question of basal cortisol, it might be prudent to introduce a row for the time of blood draw in table 1.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Tristan Struja

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

30 Oct 2019

Response to reviewers

Reviewer #1: This paper has gone though a number of revisions as the authors sort out exactly how cortisol was measured at their institution. Now they have confined their analysis to the old cortisol I assay and present an entirely new set of results.

1. While I accept the authors' statistical analysis, I do not accept the conclusion that a cortisol done at any time of the morning is inherently superior and a more accurate predictor of adrenal insufficiency than a 0800h cortisol. It makes no sense to me how in the "normal" group, the 0800h cortisol was numerically lower than the basal cortisol (Table 1). I do agree that it is feasible to establish cortisol levels at different times of the day above and below which an ACTH stimulation test is not required to rule in or rule out adrenal insufficiency.

We agree with the reviewer that it looks strange that the data in the normal adrenal response group showed that 0800h cortisol was lower than the basal cortisol. We re-checked the raw data and the entire data set again and found that the data shown in Table 1, although perhaps unexpected, is correct.

Regarding the conclusion, we have modified the sentence in the abstract and have removed comparative words like “superior” and “higher”. We have further noted that basal cortisol can be used as a screening option to diagnose AI as you suggested. Other sentences related to this issue have been similarly modified (L203, 209, 211, 245, 256, 247-248, 259).

The title has been changed to “Diagnostic Performance of Basal cortisol Level at 0900-1300h in Adrenal Insufficiency” and the word “higher diagnostic performance” has been removed.

2. As outlined in my previous comments, the proposed cut-offs for "morning" and "basal" cortisol lie well within the overlapping coefficients of variation of the assay.

Thank you for mentioning this. We have modified Table 3 and the interval of the cut-off levels have been changed to more than 10% of the prior cut-off level based on the coefficient of variation of the assay or 25 nmol/L, as appropriate. The statement in the statistical anlysis section has been modified accordingly (L118-L119).

The analysis is based on an assay no longer in use and cannot necessarily be extrapolated to the new cortisol II assay.

Thank you for your comment. We have included this issue in the Discussion section (limitations). At least, our results can be used as a guide for the future studies using the second generation cortisol assay (L249-L257).

3. While acknowledged in the Discussion, mixing the low and standard dose ACTH test and using the same cortisol cut-off for each is a weakness and source of inaccuracy.

According to the meta-analysis from Ospina NS, JCEM 2016 (ref. #19), no significant difference in diagnostic accuracy between LDT and HDT has been demonstrated when the same cut-off point is used. However, there are still some debate about this issue.

In our study, we used ACTH dosage as one of the potentially counfounding factors and adujsted for it using multivariable regression analysis. That adjustment somewhat reduces the confounding potential related to the different ACTH dosages. We have included this potential source of diagnostic inaccuracy as a limitation in the Discussion section (L271-272).

4. In the Abstract (Results), please change the cortisol units from mcg/dL to nmol/L - should read 350 nmol/L

Thank you. The units have been changed on L32.

5. In Table 1, please remove the decimal points when examining cortisol and other parameters (inappropriate to measure cortisol to 2 decimal points). Furthermore, the way the data are now presented, mean and range would be more illustrative than mean and SD.

Thank you. The data are now shown with one decimal point and are now presented with mean and range as suggested (Table 1). The Statistical Analysis and Results sections have been modified accordingly (L111 and L128)

Reviewer #2:

L.90-91

Those who had serum morning (0800 h) cortisol levels that fell into intermediate levels of 83-499 nmol/L (3-17.9 μg/dL) proceeded to either LDT or HDT.

As a retrospective study, and in light that the cut-offs have changed during the review process, it is misleading to state that patients were not subjected to a LDT/HDT. Rather write that these cases were classified as AI/non-AI.

Thank you. The sentence has been modified as suggested (L89-L90).

L 246-247 Hence, in terms of using the proposed cut-off levels, the basal cortisol tests appear to be preferable over the 0800 h morning cortisol tests to facilitate AI diagnosis.

I understand my co-reviewers concerns on the question of morning cortisol vs. basal cortisol. There is no considerable difference between these two measurements. Also, clinical application is questionable, as slot for blood draw need to be filled anyway. Therefore, I suggest rephrasing above lines (and similar ones) into a more conciliatory tone. For instance, one could stipulate that the basal cortisol is as good as the morning cortisol, and that there is no specific need for drawing distinct morning cortisol values.

Thank you for the suggestion. We have rephrased the conclusion in the abstract as you suggested and the title of the manuscript has been changed to “Diagnostic Performance of Basal cortisol Level at 0900-1300h in Adrenal Insufficiency”. Other sentences related to this issue have been modified (L203, 209, 211, 245, 255, 247-248, 259). We have used the phrase “statistically higher diagnostic performance,” but in describing the clinical implications we have adopted a more concilialtory tone, stating that basal cortisol level can be employed as a screening option to diagnose AI.

Furthermore, this issue could be alleviated by updating figure 2 with 95% confidence bands around the ROC curves.

Thank you for the suggestion. The 95%CI bands for ROC curves have been added in Figure 2.

Table 1

As your paper is based on the reliability question of basal cortisol, it might be prudent to introduce a row for the time of blood draw in table 1

Thank you for the suggestion. Unfortunately, as we mentioned in our response to comments on the first version of the manuscript, data regarding the specific time of the test is not available. We have included this issue in the Limitations section (L281-L283).

Submitted filename: responses to reviewer.docx

Click here for additional data file.

1 Nov 2019

Diagnostic Performance of Basal Cortisol Level at 0900-1300h in Adrenal Insufficiency

PONE-D-19-19551R3

Dear Dr. manosroi,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Pal Bela Szecsi, M.D. D.M.Sci.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

8 Nov 2019

PONE-D-19-19551R3

Diagnostic Performance of Basal Cortisol Level at 0900-1300h in Adrenal Insufficiency

Dear Dr. manosroi:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Pal Bela Szecsi

Academic Editor

PLOS ONE