Investigation of single and paired measurements of adrenocorticotropic hormone for the diagnosis of pituitary pars intermedia dysfunction in horses.

BACKGROUND: Paired measurement of ACTH concentration may be more reliable than a single measurement. HYPOTHESIS/
OBJECTIVES: To determine whether the mean of 2 measurements of ACTH concentration is more reliable in assessing pituitary pars intermedia dysfunction (PPID) than a single measurement. ANIMALS: Paired ACTH measurements were performed on (1) 148 occasions from 124 horses being investigated for PPID, (2) 90 occasions from 76 horses with PPID that were receiving treatment with pergolide, and (3) 63 occasions from 50 horses in which there was no clinical suspicion of PPID. Histologic examination of the pars intermedia was performed in 67 of the untreated horses.
METHODS: Outcome of testing using single and the mean of paired samples was compared directly and both methods were compared against histology, which was considered the gold standard.
RESULTS: Paired ACTH measurement altered binary classification as healthy or diseased in 6 of 211 cases, all off which had equivocal initial ACTH concentrations between 20 and 39 pg/mL. Using histology as the gold standard, optimal sensitivity and specificity for diagnosing PPID were 69.4 and 80.9%, respectively, for a single measurement and 72.2 and 76.2%, respectively, for paired measurements. The area under the receiver operating characteristic curve was 0.72 and 0.73 for single and paired measurements compared with histopathologic diagnosis, respectively. CONCLUSIONS AND CLINICAL IMPORTANCE: Paired measurement of ACTH concentration offers no advantage over a single measurement.

confidence interval

coefficient of variation

pituitary pars intermedia dysfunction

receiver operating characteristic

standard deviation

thyrotropin‐releasing hormone

Pituitary pars intermedia dysfunction (PPID) is a neurodegenerative condition of horses that is associated with aging.1 Measurement of plasma ACTH concentration is a common means of diagnosing the condition and offers practical and financial advantages over the use of dynamic tests such as the dexamethasone suppression test,2 thyrotropin releasing hormone (TRH) stimulation test,3, 4 or domperidone stimulation test.1, 5

The reliability of a single measurement of ACTH concentration has been questioned,2, 6 and variation in resting ACTH concentration has been reported to occur in horses when paired samples have been collected minutes apart.4, 7 Pulsatile release of ACTH into pituitary venous blood may result in variation in ACTH concentration,8, 9, 10 and fluctuations of over 50% of the mean ACTH concentration were identified in jugular venous blood in 1 study.11 Calculation of the mean ACTH concentration from 2 plasma samples collected 5 minutes apart has been proposed as a method of decreasing the influence of endogenous fluctuations in ACTH concentration and increasing the accuracy of PPID diagnosis.4 The objective of the following study was to compare the diagnostic utility of the mean of 2 ACTH measurements, obtained 5–15 minutes apart, with that of a single measurement of ACTH concentration.

Methods

Data were obtained from a retrospective review of clinical case records or prospectively from horses belonging to the institutions' research herds, with ethical approval from the relevant ethics and welfare committees and informed client consent. Records from the 3 participating institutions in Europe, North America, and Australia were reviewed to identify horses in which paired measurements of ACTH had been performed to diagnose PPID or assess the response to treatment with pergolide. Horses and ponies of all ages, breeds, and sex were included with Thoroughbreds, Standardbreds, Quarterhorses, Shetland ponies, other mixed native pony breeds, and other mixed horse breeds being represented in similar proportions. Ages ranged from 3 to 33 years with a median of 16 years.

Results were identified for 148 paired samples from 124 horses being investigated for PPID (Group 1), 90 paired samples from 76 horses with PPID that were receiving treatment with pergolide (Group 2), and 63 paired samples from 50 horses in which there was no clinical suspicion of PPID (Group 3). Samples were collected at all times of the year including the fall. Histologic examination of the pars intermedia was performed postmortem in 67 of the untreated horses, of which 44 had histologic changes and 23 were considered normal. Examinations were performed by board‐certified pathologists at a single institution, and a diagnosis of PPID was made if there was evidence of adenomatous hyperplasia, microadenoma, or macroadenoma.

After collection 5–15 minutes apart, plasma samples were chilled and centrifuged. Plasma samples were then refrigerated and analyzed within 24 hours or frozen at −20°C or −80°C for analysis within 4 weeks of collection. Although some variation in timings and equipment occurred among institutions, all protocols were consistent with currently accepted standards12 and both samples from each pair were handled in an identical manner such that any in vitro effect on ACTH concentration would have been equivalent for both samples. Reference ranges were corrected for season (29 pg/mL November to July inclusive and 47 pg/mL for August, September, and October).13 The same make model of analyzer (Immulite; Siemens)1 was used by all participating institutions. The coefficient of variation for the assay was calculated from the ratio of the standard deviation to the mean using repeated samples from 23 horses that were not included in the current investigation.

Statistical analysis was performed using Excel (Microsoft Corporation)2 and Rv2.15 software (R Development Core Team).3

For untreated horses (Groups 1 and 3):

The results of analysis of the first sample were used as the single sample result and compared to the mean of the 2 paired results to simulate clinical practice in which clinicians could consider waiting and collecting a second sample. A seasonally adjusted reference range13 was used to classify horses as diseased or nondiseased. If the classification was the same using both test methods, the tests were considered to be in agreement.

Receiver operating characteristic (ROC) analysis was performed and sensitivity and specificity were calculated for both single and paired measurements using histology as the gold standard.

For all groups, Bland‐Altman analysis was performed to compare single versus the mean of 2 ACTH measurements in untreated (Groups 1 and 3) and treated (Group 2) horses.

Results

Coefficient of Variation for the Assay

The coefficient of variation for the assay over a range of concentrations was 6.0%, consistent with results obtained when the analyzer was validated for use in horses.14 The assay had a mean bias of 2.9 pg/mL (95% CI: 31.4–37.2). In the region between 20 and 39 pg/mL the median coefficient of variation was 9.0% (mean, 9.5%; range, 0–26.0%).

Direct Comparison of Single and Paired Samples

The differences among the results of paired samples are presented in Table 1 and Fig 1. Of the 211 samples from untreated horses, single and paired tests were in agreement in 205 cases. All of the 6 cases in which results were not in agreement were tested in the nonautumn period. Five results changed from positive to negative and 1 result changed from negative to positive after inclusion of the second result. All of these results were in close proximity to the “nonautumn” cut‐off of 29 pg/mL13 (median, 29.2 pg/mL; range, 20.2–38.7 pg/mL). An additional 61 horses had ACTH results between 20 and 39 pg/mL on the first sample. Therefore, of all horses that had single ACTH results between these values, clinical interpretation could have been different in 9.8% (6/61) if second samples had been analyzed.

Table 1

Differences in absolute and percentage ACTH concentrations in paired plasma samples collected 5–15 minutes apart

	Number of Tests	Differences between ACTH Concentrations			Difference between ACTH Concentrations (% of Mean)
		Median Difference	Minimum Difference	Maximum Difference	Median Difference	Minimum Difference	Maximum Difference
Untreated	211	3.1	0	411	10.7	0	118.4
Histologic evidence of PPID	44	3.1	0	66.5	9.8	0	59
No histologic evidence of PPID	23	1.4	0.2	6.5	8.8	0.95	25.3
Pergolide‐treated	90	4.8	0.1	67.5	10.9	0.3	68

PPID, pituitary pars intermedia dysfunction.

Figure 1

Histogram of the distribution of ACTH measurements from untreated horses. First ACTH concentrations are shown on the x axis with the number of horses with any given ACTH concentration is indicated on the y axis. Unshaded boxes represent results that would indicate the absence of pituitary pars intermedia dysfunction. Gray boxes represent positive results. Black boxes indicate either negative or positive first results for which there was discordance between the results of single and paired sampling. The figure demonstrates the clustering of discordant results around the cut‐off value of 29 pg/mL and within a “grey zone” reported previously. Horses with an initial ACTH concentration of >100 pg/mL have been omitted improve visualization of the data around the cut‐off value.

Agreement with Histopathology

The result of a single ACTH measurement did not agree with the results of histopathology in 21/67 cases (31.3%; 19 false negatives and 2 false positives). Nine of these cases had borderline ACTH concentrations within a “gray zone” of 20 and 39 pg/mL reported previously.15 Of the remaining 12 cases, 11 were <20 pg/mL and one was >39 pg/mL. The mean result of paired measurements did not agree with the results of histopathology in 22 cases (32.8%; 20 false negatives and 2 false positives) and only 10 of these had concentrations within a 20–39 pg/mL “grey zone.” Of the remaining 12 cases, 11 had ACTH concentrations of <20 pg/mL and 1 >39 pg/mL.

Area under the ROC curve was 0.72 (95% CI: 0.59–0.86%) for single measurement of ACTH concentration (Fig 2). The single ACTH measurement cut‐off that correctly classified the highest number of horses was 21.3 pg/mL (giving a sensitivity of 69.4% and specificity of 80.9%). A small change in the cut off to 23.6 pg/mL increased specificity considerably to 95.2% with a decrease in sensitivity to 61.1%. Area under the ROC curve for paired measurement was 0.73 (95% CI: 0.60–0.87%). The cut‐off for mean of paired ACTH measurements that correctly classified the highest number of horses was 21.9 pg/mL, which provided a sensitivity of 72.2% and specificity of 76.2%. Specificity was increased by changing the cut‐off to 23 pg/mL, which resulted in sensitivity and specificity of 66.7 and 85.7%, respectively.

Figure 2

Receiver operating characteristic analysis for a single (solid line) and paired (dashed line) measurement of ACTH concentration using histology as the gold standard.

Bland‐Altman Analysis

Bland‐Altman analysis identified a mean bias of 1.5 pg/mL (95% CI: 40.3–43.3) in untreated horses (Fig 3), whereas for treated horses the mean bias was 1.3 pg/mL (95% CI: 19.1–16.4) (Fig 4). These results suggest minimal difference among methods.

Figure 3

Bland‐Altman plot of a single measurement of ACTH concentration against the mean of the measurement of 2 paired samples obtained 5–15 minutes apart in horses not receiving pergolide treatment. Results of 211 tests from 174 horses.

Figure 4

Bland‐Altman plot of a single measurement of ACTH concentration against the mean of the measurement of 2 paired samples obtained 5–15 minutes apart in horses with pituitary pars intermedia dysfunction that were receiving treatment with pergolide. Results of 63 tests from 50 horses.

Discussion

Using the mean value of paired measurements of ACTH concentration did not offer a substantial benefit over a single measurement of ACTH concentration. This was demonstrated by very similar areas under the curve in the 2 ROC curves and the results of Bland‐Altman analysis. In addition to there being no statistical difference, the use of paired measurements would not have influenced clinical decision making in the population of horses investigated. The increased costs associated with paired sampling appear difficult to justify given these results.

Using the mean of 2 ACTH tests compared with a single ACTH test could have changed the clinical interpretation in 6 (2.8%) of 211 samples from untreated horses. However, these 6 samples were all close to the cut‐off value of 29 pg/mL and other studies have already suggested that uncertainty exists for ACTH concentrations between 19 and 40 pg/mL.15 Results close to existing cut‐offs should be interpreted in light of signalment, history, and other clinical findings, and when unexpected or borderline results occur, most clinicians will consider further confirmatory testing. In the diagnosis of PPID, measurement of ACTH concentration after injection of TRH has been demonstrated to have greater diagnostic accuracy than measurement of resting ACTH concentration4, 16 and therefore is a useful confirmatory test. A TRH stimulation test need take no longer than performing paired ACTH measurements. Therefore, when a single measurement of ACTH is not considered sufficient in the diagnosis of PPID, the TRH stimulation test would provide a more logical alternative to the mean of 2 measurements of ACTH concentration. However, the use of TRH in horses is not licensed and if it is not available in small quantities it may be expensive to purchase.

Beech et al4 reported variation in baseline ACTH concentrations in some horses with PPID and little variation in clinically normal horses. A similar trend was observed in the current investigation that included the cases previously reported by Beech et al.4, 16, 17 The Bland‐Altman plots indicated a pattern of increased absolute variation as ACTH concentration increased for higher values of the mean of the 2 determinations. There did not appear to be an increase in percentage variation with increased baseline ACTH concentration, and the increased variation in PPID cases appears therefore to be a result of the increased production of ACTH from the pars intermedia rather than being because of large fluctuations in pituitary gland activity in horses with PPID. In response to treatment with pergolide, the degree of absolute variation in paired ACTH concentrations decreased as the mean concentration decreased.

Comparison of ACTH concentrations, both single and mean of 2 measurements, using ROC analysis with histology as the gold standard indicated that an ACTH concentration above a cut‐off of 29 pg/mL has a low sensitivity for the diagnosis of PPID. By contrast, specificity was moderate to high using this cut‐off value. This decreases concerns that single samples may result in more false‐positive diagnoses (using typical upper reference limits of approximately 29 pg/mL) than the mean of 2 measurements. However, the predictive value of the test will be influenced by the prevalence of the disease in the population, which in the general equine population will be much lower than in the study population. Factors such as age and clinical signs must be considered with diagnostic testing results. Only 2 of the 6 cases in the current investigation that would have been misdiagnosed were available for further testing and both had a positive response to TRH stimulation confirming the presence of PPID. The low sensitivity of basal ACTH for detecting changes in the pars intermedia is in accordance with the results of previous investigations in which ACTH and other endocrine tests were poor indicators of low‐grade pituitary histopathology.5, 18

In the current investigation, optimal cut‐offs for the diagnosis of pituitary histopathology derived from ROC analysis were much lower than those calculated from populations of clinically normal horses.13, 19 This observation may indicate that some of the clinically normal horses included in these previous investigations13, 19 had early pituitary abnormalities, but no clinical signs. However, because PPID is a change that is insidious in onset, the ability to identify the presence of pituitary pathology per se may not be that helpful clinically. Additional studies are required to identify markers, or panels of markers, that give an indication of early pituitary disease, and more importantly, indicate when pituitary dysfunction is of sufficient magnitude to result in clinical disease. Using ROC analysis of the data in the current investigation, when the cut‐off for the diagnosis of PPID was increased in accordance with recently published results,13, 19 sensitivity decreased to 53%, whereas specificity remained high at 95%.

In the healthy state, the pars distalis is the primary source of ACTH and, whereas changes in the pars distalis giving rise to increased ACTH concentrations have not been reported, it is possible that PPID may not be the only process that can cause increased ACTH concentrations. Ectopic ACTH production has not been reported in the horse; hence, based on current knowledge, increases in ACTH that are not attributable to PPID are likely to be physiologic rather than pathologic or because of sample deterioration in vitro. Increases in ACTH in venous blood leaving the pituitary gland have been identified in response to experimentally induced hypoglycemia and isolation stress.10, 20 Transport, exercise, and anesthesia may all increase plasma ACTH concentrations,21, 22, 23, 24, 25 but these influences are unlikely to be relevant in the context of testing for PPID. The results of investigations into the effects of pain and disease on ACTH concentration are conflicting,26, 27, 28, 29, 30 but are worthy of consideration in a clinical setting even if they were not relevant to the horses in the current investigation. The effects of stress may have been relevant in the 6 cases in the current study that may have been diagnosed differently if paired rather than single samples had been used. In 5 of the 6, the first ACTH concentration was higher than the second, which might be explained by a stress effect at the initial sampling. There was no general trend for the second results to be lower; median results for the group under investigation for PPID were 22.7 and 24.2 pg/mL at first and second sampling, respectively.

A limitation of the study is the population that was examined. Disease prevalence within a population will influence estimates of predictive values, sensitivity, and specificity. The purpose of the study was to compare diagnostic accuracy between single and paired samples rather than determine estimates that are applicable to the wider equine population. Ethical and financial constraints precluded random prospective sampling. The study population did, however, contain horses from different locations with a range of ages and breeds, and both horses with clinical evidence of PPID and those without were included. Neither age31 nor breed or body type29, 30, 32, 33 is thought to affect ACTH concentration in healthy horses.

In conclusion, this study has provided additional evidence that a single measurement of ACTH concentration is a valuable screening test in the diagnosis of PPID and there is no value in performing paired measurements of ACTH concentration. However, when unexpected or borderline results are obtained, clinicians should consider performing additional diagnostic evaluation such as a TRH stimulation test.