Influence of anticardiolipin and anti-β2 glycoprotein I antibody cutoff values on antiphospholipid syndrome classification.

BACKGROUND: Anticardiolipin (aCL) and anti-β2 glycoprotein I (aβ2GPI) immunoglobulin (Ig) G/IgM antibodies are 2 of the 3 laboratory criteria for classification of antiphospholipid syndrome (APS). The threshold for clinically relevant levels of antiphospholipid antibodies (aPL) for the diagnosis of APS remains a matter of debate. The aim of this study was to evaluate the variation in cutoffs as determined in different clinical laboratories based on the results of a questionnaire as well as to determine the optimal method for cutoff establishment based on a clinical approach.
METHODS: The study included samples from 114 patients with thrombotic APS, 138 patients with non-APS thrombosis, 138 patients with autoimmune disease, and 183 healthy controls. aCL and aβ2GPI IgG/IgM antibodies were measured at 1 laboratory using 4 commercial assays. Assay-specific cutoff values for aPL were obtained by determining 95th and 99th percentiles of 120 compared to 200 normal controls by different statistical methods.
RESULTS: Normal reference value data showed a nonparametric distribution. Higher cutoff values were found when calculated as 99th rather than 95th percentiles. These values also showed a stronger association with thrombosis. The use of 99th percentile cutoffs reduced the chance of false positivity but at the same time reduced sensitivity. The decrease in sensitivity was higher than the gain in specificity when 99th percentiles were calculated by methods wherein no outliers were eliminated.
CONCLUSIONS: We present cutoff values for aPL determined by different statistical methods. The 99th percentile cutoff value seemed more specific. However, our findings indicate the need for standardized statistical criteria to calculate 99th percentile cutoff reference values.

Cutoff values for antiphospholipid antibodies remain a matter of debate.

Cutoff values were derived from testing 200 normal controls with 4 commercial assays.

Large cutoff variations were observed by different methods of calculation.

Standardized statistical criteria to calculate 99th percentile cutoff reference values are needed.

INTRODUCTION

According to the updated Sapporo criteria, the diagnosis of antiphospholipid syndrome (APS) implies vascular thrombosis and/or pregnancy morbidity and at least 1 of the 3 antiphospholipid antibodies (aPLs) (ie, lupus anticoagulant [LAC], anticardiolipin [aCL]immunoglobulin (Ig) G/IgM antibodies and anti‐β2 glycoprotein I [aβ2GPI] IgG/IgM antibodies).1 The “classical” clinical characteristics of APS include vascular thrombosis and pregnancy morbidity. However, APS may be associated with a wide variety of other clinical symptoms, including thrombocytopenia, heart valve disease, livedo reticularis, nephropathy, and neurological manifestations, which are considered noncriteria manifestations of APS.1, 2 In view of the lack of specificity of the clinical manifestations (in particular when the classical clinical characteristics are absent), laboratory tests for aPL are crucial to diagnose APS. The persistent positivity of laboratory tests is important because the transient presence of epiphenomenal aPL may give rise to misclassification. Thereby, classification criteria emphasize the importance of repeating positive tests at an interval of >12 weeks.1

Traditionally, aCL and aβ2GPI antibodies are detected by enzyme‐linked immunosorbent assay (ELISA), although new, fully automated technologies such as chemiluminescent and fluorescence enzyme immunoassays have been introduced. The new assays show good analytical performance.3, 4 For all assays, the cutoff value should be carefully chosen because the sensitivity and specificity of diagnostic laboratory testing for aPL strongly depend on the cutoff value.5 In analogy with LAC, the Scientific and Standardization Committee on Lupus Anticoagulant/Antiphospholipid Antibodies (SSC‐aPL) of the International Society of Thrombosis and Haemostasis (ISTH) recommends the use of the 99th percentile of a reference population as the cutoff to maximize specificity.6, 7 Therefore, a minimum of 120 reference subjects should be used, taking into consideration the age and type of population most representative for each laboratory.6 Unfortunately, these in‐house calculated cutoff values may be significantly different from those recommended by the manufacturers.8, 9, 10 In fact, the value may depend on the performance characteristics, the statistical method and the reference population used to establish cutoff values. Reaching a consensus on the method for cutoff establishment is important from the viewpoint of harmonization of aPL measurement.

To study cutoff establishment in clinical laboratories, we prepared a questionnaire sent to SSC‐aPL members and participants of the Lupus Anticoagulant/Antiphospholipid Antibodies Program of the ECAT (External Quality Control for Assays and Tests) (ECAT Foundation, Voorschoten, The Netherlands). Subsequently, we set up a study to further investigate the extent of differences in cutoff values determined by different methods of calculation. A patient population was used to assess the impact of different cutoff values on the analytical and clinical performance of 4 commercial assays detecting aPL.

MATERIALS AND METHODS

Questionnaire on cutoff values for aPL with solid‐phase assays

A questionnaire (see the Supplementary Data) was sent by email to 318 SSC‐aPL members and 575 participants of the “Lupus Program” external quality exercises of the ECAT Foundation. SSC‐aPL members are workers in the field of aPL who expressed their interest in the SSC‐aPL by signing up on the ISTH website.

Selection of normal controls

A total of 120 normal controls (71 females and 49 males; mean age, 56 years; range, 44‐65 years) were recruited from healthy local volunteers at Ghent University Hospital (Ghent, Belgium). In collaboration with Jagiellonian University Medical College (Krakow, Poland), the normal population was expanded with another 80 healthy volunteers, giving rise to a sample size of 200 (129 females and 71 males; mean age, 53 years; range, 20‐71 years). All healthy volunteers were selected for blood sampling after declaring good health by a questionnaire. They were free from overt cardiovascular disease, conventional risk factors, and medication.

Selection of patient groups

Patients with APS, patients with non‐APS disease, and healthy controls were enrolled at Ghent University Hospital and Jagiellonian University Medical College. The patients were selected by the enrollment center out of a population referred for autoimmune disease, hypercoagulability, or prolonged clotting time with a request for aPL measurement. They were assigned to the following categories based on both clinical data and results of local laboratory investigations:

Group A consisted of patients with thrombotic APS according to the Sydney revised Sapporo guidelines.1 Of these patients, 24% had arterial thrombosis, 67% had venous thrombosis, and 10% had thrombosis in both vascular beds. Numbers in each center were as follows: Ghent, n = 64; Krakow, n = 50.

Group B consisted of patients who had a history of thrombosis and were negative for laboratory criteria of APS (“diseased controls”). Numbers in each center were as follows: Ghent, n = 138; Krakow, n = 0.

Group C consisted of patients with an autoimmune disease without thromboembolic or pregnancy complications. Numbers in each center were as follows: Ghent, n = 97; Krakow, n = 41.

Group D included patients fulfilling neither the clinical nor the laboratory criteria for APS and tested for aPL due to an accidentally found prolonged activated partial thromboplastin time or the appearance of clinical symptoms not included in the APS classification criteria, such as chorea, migraine, infertility, etc. (“healthy controls”). Numbers in each center were as follows: Ghent, n = 183; Krakow, n = 0.

The study was approved by the ethics committees of both centers included in the study.

Assays

Citrated plasmas from all patients and controls were retested for aCL and aβ2GPI IgG/IgM antibodies at 1 site (Ghent University Hospital) using 4 commercial assays:

The HemosIL AcuStar antiphospholipid assay (Werfen/Instrumentation Laboratory, Bedford, MA) is a fully automated 2‐step immunoassay using chemiluminescent technology for detecting aCL and aβ2GPI IgG/IgM antibodies on the ACL AcuStar.

The BioPlex 2200 APLS IgG and IgM kits (Bio‐Rad Laboratories, Hercules, CA) use multiplex bead technology for the detection of IgG/IgM antibodies to cardiolipin and β2GPI.

The EliA Cardiolipin IgG, IgM and EliA β2‐Glycoprotein I IgG, IgM (Thermo Fisher Scientific/Phadia, Uppsala, Sweden) use fluorescence enzyme immunoassay technology to measure aCL and aβ2GPI IgG/IgM antibodies on the Phadia 250 instrument.

The QUANTA Lite ACA IgG, IgM III and QUANTA Lite β2GPI IgG, IgM (Werfen/Inova Diagnostics, San Diego, CA) ELISAs were performed manually according to the manufacturer's instructions.

Table 1 gives information on the assays, including the test principle, mode of detection, and manufacturer's cutoff.

Table 1

Assay characteristics

	HemosIL AcuStar		BioPlex		EliA		QUANTA Lite
Assay	HemosIL AcuStar Anti‐β2 glycoprotein I IgM, IgG	HemosIL AcuStar Anti‐Cardiolipin IgM, IgG	APLS IgM kit	APLS IgG kit	EliA β2‐Glycoprotein I IgM, IgG	EliA Cardiolipin IgM, IgG	QUANTA Lite β2GPI IgM, IgG	QUANTA Lite ACA IgM, IgG III
Technology	Automated 2‐step chemiluminescent immunoassay	Automated 2‐step chemiluminescent immunoassay	Automated multiplex flow immunoassay	Automated multiplex flow immunoassay	Fluorescence enzyme immunoassay	Fluorescence enzyme immunoassay	Enzyme‐linked immunosorbent assay	Enzyme‐linked immunosorbent assay
Coating well/particle	Human β2GPI	Bovine cardiolipin with human β2GPI	1: Human β2GPI 2: Synthetic cardiolipin with human β2GPI	1: Human β2GPI 2: Synthetic cardiolipin with human β2GPI	Human β2GPI	Bovine cardiolipin and bovine β2GPI	Purified β2GPI	Purified cardiolipin and bovine β2GPI
Conjugate	Isoluminol‐labeled anti‐human IgM/IgG antibody	Isoluminol‐labeled anti‐human IgM/IgG antibody	Phycoerythrin conjugated murine monoclonal anti‐human IgM	Phycoerythrin conjugated anti‐human IgG	β‐galactosidase mouse monoclonal anti‐ IgM, IgG	β‐galactosidase mouse monoclonal anti‐IgG	Peroxidase‐labeled anti‐human IgM, IgG	Peroxidase‐labeled anti‐human IgM, IgG
Signal detection	Chemiluminescence	Chemiluminescence	Fluorescence	Fluorescence	Fluorescence	Fluorescence	Chromogenic	Chromogenic
Calibration	Internal standard traceable towards Koike's monoclonal antibodies (HCAL for IgG and EY2C9 for IgM)	Internal standard traceable towards Koike's monoclonal antibodies (HCAL for IgG and EY2C9 for IgM)	Internal standard containing human antibodies in serum matrix	Internal standard containing human antibodies in serum matrix	Internal standard containing human IgG or IgM in PBS, traceable to the WHO International Reference Preparation 67/86	Internal standard containing human IgG or IgM in PBS, traceable to the WHO International Reference Preparation 67/86	Internal standard containing human serum antibodies to β2GPI referenced to the reference calibrators for IgG/IgM β2GPI available from the Rheumatology Laboratory, Seton Hall University, St. Joseph's Hospital and Medical Center	Internal standard containing human serum antibodies to cardiolipin
Manufacturer's cutoff	20 U/mL	20 U/mL	20 MPL‐U/mL (aCL) 20 U/mL (aβ2GPI)	20 GPL‐U/mL (aCL) 20  U/mL (aβ2GPI)	10 U/mL	10 MPL‐U/mL, GPL‐U/mL	20 SMU, SGU	20 MPL, GPL
Calculation	250‐262 blood bank donors, 99th percentile	250‐252 blood bank donors, 99th percentile	300 blood bank donors, a recommended value	300 blood bank donors, a recommended value	400 healthy subjects, a recommended value	400 healthy subjects, a recommended value	11‐313 normal donors, a recommended value	488‐489 normal donors, a recommended value

aβ2GPI, anti‐β2 glycoprotein I antibodies; aCL, anticardiolipin antibodies; β2GPI, β2 glycoprotein I; GPL, IgG phospholipid units; MPL, IgM phospholipid units; PBS, phosphate‐buffered saline; SGU, standard IgG units; SMU, standard IgM units.

Statistical analysis

All results from normal controls were tested to assess deviation from the normal distribution by means of the Kolmogorov‐Smirnov test. Cutoff values for aPL were obtained by determining 95th percentiles (with 90% confidence intervals [CIs]) and 99th percentiles (with 90% CIs) by using 2 methods. One method (method A), cited by the Clinical Laboratory Standards Institute (CLSI),11 calculates an index p (n + 1) with p representing the percentile and n the sample size. Method B calculates an index pn + 0.5. Secondary analyses were performed when outliers were first eliminated. To identify outliers Reed's modification of the Dixon test12 and Tietjen‐Moore (TM) test13 were used. To determine the number of outliers for the TM test, we chose the position of the largest gap. Tukey's box method was not applied because it requires Box‐Cox transformation of the data to obtain a Gaussian distribution that was not possible for our data sets.

Cohen's kappa agreement test was carried out to assess analytical agreement among aPL assays. To evaluate the ability of the assays to predict thrombotic complications, odds ratios (with 95% CIs), sensitivities (with 95% CIs), specificities (with 95% CIs), and Youden indexes (sensitivity+specificity−100%) were calculated, considering groups A and B as clinically positive and all other groups as clinically negative. Clinically affected (groups A and B) and non–clinically affected patients (groups C and D) were set as outcome variable rather than APS/non‐APS in order to be independent of aPL presence previously detected to minimize selection bias. P values associated with odds ratios were calculated by Fisher's exact test. A P < 0.05 was considered to be statistically significant.

All statistical analyses were performed using Analyse‐it 4.81.1 for Microsoft Excel (Analyse‐it Software, Leeds, UK), MedCalc 17.5.5 (MedCalc Software, Ostend, Belgium), and DATAPLOT software package 6/2013 (National Institute of Standards and Technology, Gaithersburg, MD).

RESULTS

We received 139 answers from all over the world, yielding a response rate of 15.5%, including 72.7% hospital laboratories. A total of 61.4% of the responses originated from Europe, 17.0% from the United States, 11.4% from Asia, 7.9% from South America, and 2.3% from Australia. Over 85% of the participating laboratories performed all 4 parameters (aCL IgG/IgM, aβ2GPI IgG/IgM) with various methods. Tests were mainly performed in coagulation departments (40.9%) or clinical chemistry/immunology departments (51.5%).

Furthermore, 41.1% of the laboratories calculated in‐house cutoff values. Most of the laboratories that did not calculate in‐house cutoff values (58.9%) used the manufacturer's cutoff (75.7%). The cost and availability of normal donors were mentioned as the main drawbacks hampering the in‐house calculation. Only 38.2% of the laboratories checked the manufacturer's cutoff according to the CLSI guideline before transference6, 11; 44.1%, 30.3%, and 44.1% verified the number of donors, the demographic specifications, and the statistical method used, respectively. The minority (25% and 38.7%) of the laboratories rejected the manufacturer's cutoff if fewer than 120 donors were used or the statistical method did not conform to the recommendations.6 Furthermore, 53.7% of the laboratories that calculated in‐house cutoff values used 120 or more normal donors; in 81% of these laboratories, these normal donors originated from a local population (laboratory personnel) or blood bank donors.

The question “Which statistical method do you use?” revealed a parametric method in 41.6% of the laboratories: without data transformation (mean + 2SD) in 19.4%, without data transformation (mean + 3SD) in 13.9%, and after data transformation to achieve normality (mean + 2SD) in 8.3%. In contrast, 58.4% of the laboratories used a nonparametric method: right‐sided percentile estimation without data transformation in 41.7% and right‐sided percentile estimation after data transformation to achieve normality in 16.7% (Figure 1A). Of those laboratories applying a nonparametric method, 82.4% used the 99th percentile (p [n + 1] [47.2%] or pn + 0.5 [35.2%]), and 17.6% used the 95th percentile.

Figure 1

Results of the questionnaire. (A) Which statistical method do you use? (B) Which method do you use to identify outliers? IQR, interquartile range; SD, standard deviation

The question “Which method do you use to identify outliers?” showed that 61.5% of the laboratories checked for outliers by different methods as illustrated in Figure 1B, of which 68.4% effectually excluded outliers; 31.6% followed the recommendations to check the calculated cutoff value by a clinical approach in the local patient population by calculating sensitivity and specificity regarding the association with thrombotic/pregnancy complications,6 and 72.7% adapted their cutoff values accordingly, based on the criterion of sensitivity >95% (37.5%), specificity >95% (37.5%) or choosing the highest odds ratio (25%).

Calculation of cutoff values on a normal population

The results for aCL and aβ2GPI IgG/IgM antibodies from normal controls showed a nonparametric (positively right skewed) distribution. Not all data sets could be normalized due to the clustering of identical low values at the bottom of the distribution. Hence, it was decided to calculate cutoff values by means of nonparametric procedures.

The derived cutoff values for aPL are presented in Tables S1 and S2. As expected, higher cutoff values were found when calculated as 99th rather than 95th percentiles. Notably, the statistical method had considerable impact on the estimated cutoffs. When all different methods for percentile calculation were combined with all different outlier elimination methods and the 99th (95th) percentiles obtained were compared, >11‐ (>2‐) and >8‐ (>1‐) fold difference was obtained using data from 120 and 200 normal controls, respectively.

The 99th percentiles of 120 normal controls by method A were 31.1 U/mL for aβ2GPI IgM, 209.2 U/mL for aCL IgM, 136.2 U/mL for aβ2GPI IgG, and 42.5 U/mL for aCL IgG using HemosIL AcuStar. For BioPlex, these values were found to be 65.4 U/mL, 59.8 MPL‐U/mL, 123.6 U/mL and 132 GPL‐U/mL, respectively; for EliA, 21.3 U/mL, 22.0 MPL‐U/mL, 36.1 U/mL, and 48.1 GPL‐U/mL, respectively; and for QUANTA Lite, 24.6 SMU, 25.9 MPL, 16.4 SGU, and 60.0 GPL, respectively. The 99th percentiles with the use of method A were up to 2‐fold higher than those of method B. Moreover, method A produced the most heavily biased estimates compared to the manufacturers’ cutoffs, especially for the lower sample size of 120.

Of even more importance was the effect of outlier exclusion on the calculated 99th percentiles. The Reed method classified only 1 subject as a possible outlier, lowering the 99th percentile by 1‐ to 5.9‐fold (method A) and by 1‐ to 6.1‐fold (method B). However, using the TM method, 1 to 3 subjects were classified as possible outliers. Excluding these subjects from the calculations lowered the 99th percentile by 1.3‐ to 5.9‐fold (method A) and by 1.2‐ to 7.6‐fold (method B). The number of outliers is presented in Table 2.

Table 2

Excluded outliers

HemosIL AcuStar				BioPlex				EliA				QUANTA Lite
aβ2GPI IgM	aCL IgM	aβ2GPI IgG	aCL IgG	aβ2GPI IgM	aCL IgM	aβ2GPI IgG	aCL IgG	aβ2GPI IgM	aCL IgM	aβ2GPI IgG	aCL IgG	aβ2GPI IgM	aCL IgM	aβ2GPI IgG	aCL IgG
U/mL	U/mL	U/mL	U/mL	U/mL	MPL‐U/mL	U/mL	GPL‐U/mL	U/mL	MPL‐U/mL	U/mL	GPL‐U/mL	SMU	MPL	SGU	GPL
7.6	26.0	8.2	4.8	10.0	12.3	<1.4	1.9	2.5	6.3	1.4	2.6	7.0	8.1	3.9	6.8
8.4	26.1	8.2	5.4	10.6	12.5	1.7	2.3	2.5	8.1	1.4	2.7	7.5	8.1	3.9	6.9
11.8	26.1	8.3	5.9	12.0	12.5	1.9	2.4	2.6	8.3	1.4	3.2	7.6	8.7	3.9	6.9
12.2	32.5	10.3	7.4	12.5	12.8	2.2	2.6	3.3	8.8	1.5	3.9	7.9	9.4	4.0	7.0
14.1	33.5	11.0	7.9	13.6	19.9	2.3	2.7	3.5	9.6	1.8	6.5	8.0	11.7	4.1	7.1
15.0	45.1	11.5	7.9	18.0	21.2	2.4	4.3	3.8	10.0	1.9	10.0	10.1	12.1	4.2	7.2
16.6	53.5	27.0	8.4	20.7	33.7	2.9	5.3	4.6	12.0	2.1	11.0	10.7	12.8	4.3	8.5
16.9	63.5	31.3	21.1	27.8	43.9	3.2	6.4	4.9	17.0	7.5	16.0	20.8	15.3	5.8	10.2
20.0	64.2	49.0	24.5	29.1	46.8	26.3	26.6	11.0	18.0	25.0	37.0	24.0	21.9	6.6	11.7
34.0	247.8	159.4	47.3	75.0	63.2	149.5	≥ 160.0	24.0	23.0	39.0	51.0	24.8	27.0	19.0	72.8

The 10 highest concentrations are shown for each antibody. Outliers as identified by the Reed outlier test are underlined. Outliers as identified by the Tietjen‐Moore outlier test are indicated in bold.

aβ2GPI, anti‐β2 glycoprotein I antibodies; aCL, anticardiolipin antibodies; GPL, IgG phospholipid units; MPL, IgM phospholipid units; SGU, standard IgG units; SMU, standard IgM units.

A graphical presentation of the local derived cutoff values based on 120 normal controls is given in Figure 2. The value recommended by the manufacturer is indicated for comparison. In general, the 95th percentile cutoffs were lower than the manufacturers’ cutoffs. In contrast, the 99th percentile cutoffs were equal to or higher than the manufacturers’ cutoffs.

Figure 2

Dot plot of aPL results from normal controls. (A) AcuStar aβ2GPI IgM (n = 120), (B) AcuStar aCL IgM (n = 120), (C) AcuStar aβ2GPI IgG (n = 120), (D) AcuStar aCL IgG (n = 120), (E) BioPlex aβ2GPI IgM (n = 120), (F) BioPlex aCL IgM (n = 120), (G) BioPlex aβ2GPI IgG (n = 120), (H) BioPlex aCL IgG (n = 120), (I) EliA aβ2GPI IgM (n = 120), (J) EliA aCL IgM (n = 120), (K) EliA aβ2GPI IgG (n = 120), (L) EliA aCL IgG (n = 120), (M) QUANTA Lite aβ2GPI IgM (n = 120), (N) QUANTA Lite aCL IgM (n = 120), (O) QUANTA Lite aβ2GPI IgG (n = 120), (P) QUANTA Lite aCL IgG (n = 120). aβ2GPI, anti‐β2 glycoprotein I antibodies; aCL, anticardiolipin antibodies; GPL, IgG phospholipid units; method A, p (n + 1), where p indicates the percentile and n indicates the sample size; method B, pn + 0.5, where p indicates the percentile and n indicates the sample size; MPL, IgM phospholipid units; P, percentile; SGU, standard IgG units; SMU, standard IgM units; TM, Tietjen‐Moore

Agreement of the 4 assays for aPL testing

HemosIL AcuStar, BioPlex, EliA, and QUANTA Lite were compared using 573 samples. Figure 3 shows the kappa statistics among the HemosIL AcuStar, BioPlex, EliA, and QUANTA Lite aPL panels at different cutoffs defining positivity.

Figure 3

Kappa agreement among aPL assays at manufacturer and in‐house cutoff values. aβ2GPI, anti‐β2 glycoprotein I antibodies; aCL, anticardiolipin antibodies; method A, p (n + 1), where p indicates the percentile and n indicates the sample size; method B, pn + 0.5, where p indicates the percentile and n indicates the sample size; P, percentile; TM, Tietjen‐Moore. Dotted lines represent kappa agreement at manufacturers’ cutoff values

With the manufacturers’ cutoff (dotted lines in Figure 3), Cohen's kappa indicated good agreement for aβ2GPI IgM (kappa coefficient, 0.69‐0.79), aβ2GPI IgG (kappa coefficient, 0.64‐0.87) and aCL IgG (kappa coefficient, 0.71‐0.86) with all assays. The lowest agreement (kappa coefficient, 0.44‐0.65) was obtained for aCL IgM, except for the comparison between HemosIL AcuStar and BioPlex.

With all 99th percentile cutoffs investigated (based on 120 normal controls), kappa ranged from 0.52 to 0.90 for aβ2GPI IgM; from 0.22 to 0.73 for aCL IgM; from 0.71 to 0.97 for aβ2GPI IgG; and from 0.63 to 0.95 for aCL IgG. With all 95th percentile cutoffs investigated (based on 120 normal controls), kappa ranged from 0.55 to 0.82 for aβ2GPI IgM, from 0.29 to 0.74 for aCL IgM, from 0.29 to 0.72 for aβ2GPI IgG, and from 0.37 to 0.61 for aCL IgG.

Diagnostic performances

We compared the results for aCL and aβ2GPI antibodies obtained with HemosIL AcuStar, BioPlex, EliA, and QUANTA Lite with the presence of clinical features of the patients. Odds ratios, sensitivities, specificities, and Youden indexes are shown in Tables S3 through S6. A summary of these data is shown in Figure 4, showing the test statistics when at least 1 of the aPL panel tests was positive.

Figure 4

Global odds ratio, sensitivity, specificity, and Youden index for prediction of thrombotic events using manufacturer and in‐house cutoff values (A) calculated on 120 normal controls; (B) calculated on 200 normal controls. Method A, p (n + 1), where p indicates the percentile and n indicates the sample size; method B, pn + 0.5, where p indicates the percentile and n indicates the sample size; OR, odds ratio; P, percentile; sens, sensitivity; spec, specificity; TM, Tietjen‐Moore; YI, Youden index

All aCL and aβ2GPI IgM assays showed a poor relationship with thrombotic events. In contrast, IgG positivity was found to be strongly associated with thrombosis. With increasing aPL titers (eg, from the 95th to the 99th percentile), a trend of an increased risk for thrombosis was obtained.

With the manufacturers’ cutoff, the sensitivity and specificity varied, respectively, from 7.1% to 9.9% and 94.7% to 96.3% for aβ2GPI IgM, from 8.7% to 16.3% and 88.5% to 96.6% for aCL IgM, from 16.3% to 26.2% and 96.0% to 99.4% for aβ2GPI IgG, and from 16.7% to 22.2% and 96.9% to 99.1% for aCL IgG. This corresponds to Youden indexes varying from 3.1% to 5.8% for aβ2GPI IgM, from 4.4% to 5.3% for aCL IgM, from 15.6% to 22.1% for aβ2GPI IgG, and from 15.7% to 19.5% for aCL IgG.

With all 95th percentile cutoffs investigated (based on 120 normal controls), the sensitivity and specificity varied from 8.7% to 19.4% and 85.7% to 95.0% for aβ2GPI IgM, from 8.3% to 27.8% and 76.9% to 96.3% for aCL IgM, from 25.8% to 41.3% and 76.9% to 91.0% for aβ2GPI IgG, and from 25.0% to 36.1% and 82.2% to 95.6% for aCL IgG. This corresponds to Youden indexes varying from 3.7% to 10.7% for aβ2GPI IgM, from 3.3% to 8.9% for aCL IgM, from 12.8% to 29.2% for aβ2GPI IgG, and from 15.9% to 23.8% for aCL IgG.

With all 99th percentile cutoffs investigated (based on 120 normal controls), the sensitivity and specificity varied from 4.0% to 13.9% and 91.3% to 97.8% for aβ2GPI IgM, from 0.8% to 15.9% and 88.8% to 99.1% for aCL IgM, from 14.3% to 23.0% and 95.3% to 99.7% for aβ2GPI IgG, and from 10.7% to 23.0% and 95.0% to 100.0% for aCL IgG. This corresponds to Youden indexes varying from 1.8% to 5.8% for aβ2GPI IgM, from −0.1% to 6.8% for aCL IgM, from 14.0% to 19.2% for aβ2GPI IgG and from 10.7% to 18.7% for aCL IgG. The highest Youden index was obtained when cutoffs were calculated on outlier deleted data.

DISCUSSION

Defining cutoff reference values for aPL solid‐phase assays is one of the factors that determines the classification of a patient as an APS patient or not. The interpretation of results is determined by the cutoff values and plays a major role in classifying a sample as aPL positive or negative. aCL and aβ2GPI results are not expressed in International Units because of the lack of an international reference standard; rather, they are expressed in arbitrary units according to the calibration curve used in the method. The test signal is converted into antibody units derived from the calibration curve. Usually, these assay results are called semiquantitative despite the use of calibration curves, and results of aCL and aβ2GPI tests are expressed in units on a continuous scale. Each test result above the cutoff value calculated as higher than the 99th percentile should be regarded as positive, according to the SSC recommendations for solid‐phase assays.6

SSC‐aPL recommendations were published in 2014, providing detailed information on the execution of solid‐phase assays for aPL, including recommendations on the calculation of cutoff values.6 The questionnaire revealed that in daily practice, 84.5% (109/129) of the laboratories apply cutoff values according to the recommendations, either by calculating in‐house cutoffs (48.6%; 53/109) or by transference of the manufacturer's cutoff (51.4%; 56/109). Moreover, the in‐house cutoff is calculated by the 99th percentile by 82.4% of the laboratories, as advised, although not always using at least 120 normal controls (53.7%); 61.5% of the laboratories check for outliers by various methods, and 38.2% of those using the manufacturer's cutoff check the cutoffs before transferring them.

The goal of this study was to calculate and compare cutoff values for aCL and aβ2GPI antibodies analyzed on 4 different platforms and to define the optimal method of calculation by analyzing the diagnostic performance in a case‐control design. Previous studies defining optimal aPL cutoffs focused on the optimal separation of cases and controls.9, 14 However, this approach has inherent weaknesses.15, 16 Instead, we propose a different approach that takes advantage of the Youden index to select the optimal cutoff value.

The aPL results from the normal controls were not normally distributed. Therefore, cutoff values were determined by nonparametric evaluation based on centiles. The cutoffs obtained differed between assays and, in most of the cases, from the values given by the manufacturer. Moreover, we found an up to 11‐fold difference in 99th percentile cutoff values depending on which statistical method was used. Because the main aim of our analysis was to compare different statistical methods for cutoff establishment, we based our calculations on 120 normal controls. However, different values were obtained when using data from 200 normal controls, pointing to an inherent degree of uncertainty. The recommended number of 1206 actually comes from the number needed to determine the 90% CIs of the 2.5th and 97.5th percentiles of a population using nonparametric statistics.11 In fact, the minimum sample size for a reliable estimation of the 99th percentile is at least 300.17

Currently, only medium‐ or high‐titer antibodies are considered clinically relevant.18, 19 This study confirms an increased risk for thrombosis with increasing aPL titers (eg, from the 95th to the 99th percentile), although the application of the 99th percentile may be inappropriate for clinical use due to low sensitivity. For example, for HemosIL AcuStar aβ2GPI IgG, the titer of 136.2 U/mL calculated as the 99th percentile of 120 normal controls is associated with a disappointingly low sensitivity, dropping to <20%.

When selecting subjects for a reference range study, we assume that the reference values represent a “homogeneous” collection of observations. The question remains how to treat those subjects that are apparently aberrant. Many papers evaluating cutoff values for aPL simply assume the absence of outliers.20, 21 This is based on the assumption that samples from normal donors are negative for all aPL. However, given the high titer of aPL, the subjects classified as outliers in this study were regarded as true biological outliers. Of note, the presence of aPL in a patient can precede the occurrence of the typical clinical manifestations.

Our findings suggest the need for outlier removal as a necessary step before cutoff calculation, as the reference study is prone to biological outliers on top of analytical outliers. The CLSI guideline11 supports the use of the Reed method, in which the suspected outlier is rejected if the distance between the value and its closest neighbor is more than one third of the range of all values. However, this test can be used only when 1 outlier is suspected.22 Tukey's box method involves the labeling of extreme values by using only the middle 50% of the sample, thus reducing the possible masking effect of multiple outliers at 1 site of the distribution. However, Tukey's box method requires transformation to yield a parametric distribution. Alternatively, a block procedure, such as the TM test,13, 23 is suggested. No further details are given in the CLSI document about the use of this method that is also not available in conventional statistical software programs. The existence and support of multiple outlier detection methods complicates which method should be used, given the impact on the resulting cutoff value and hence the classification of patients.

Some limitations of our study should be recognized. First, the optimal minimum number of 300 normal donors was not reached. This may explain the considerable differences in cutoff estimations depending on the method used. Second, we did not include obstetric APS patients in this study. Some of those patients may have lower, yet persistent, aPL levels and could be more adversely affected by cutoff calculations. A small number of reports24, 25, 26 associate low titers of aPL with an increased risk of obstetric complications, though further confirmation in additional studies is necessary.

Considering the diagnostic importance of the 99th percentile in the prediction of APS‐related thrombosis, we emphasize the need for standardized criteria concerning the statistical analysis to define the 99th percentile. Based on our results, we recommend the use of a nonparametric procedure based on at least 300 samples from normal donors. To identify outlier data, we recommend the use of the Reed method given its simplicity. Moreover, applying this outlier exclusion method results in a high Youden index.

The sample size requirement causes a problem concerning the feasibility of local laboratories to establish cutoff values. Verification of the manufacturer's cutoff using a small number may be an alternative when local cutoff calculation is not feasible.6 However, this would assume that manufacturers’ cutoffs are established by appropriate statistical models using a sufficiently large donor population. In our experience, this is not always the case. A better alternative to establish cutoff values is a multicenter approach. Previously, it was demonstrated that a multicenter approach can determine the cutoff values with a higher accuracy by increasing the number of healthy blood bank donors.21 An interlaboratory cutoff value established by users of an identical automated system, applying the same sample type with comparable demographic characteristics of patients, was evaluated as a valuable alternative by 90% of the participants of the questionnaire. Joint efforts of independent organizations, such as national or international external quality control organizations or standardization committees such as the SSC‐aPL, can render this approach more cost‐effective and achievable, having the advantage of pooling results of higher numbers of normal donors, and applying a good design.

Uniformity in the calculation of cutoff values will lead to an improved and more standardized interpretation of assays and benefit the complex diagnosis of APS.

RELATIONSHIP DISCLOSURES

The authors have nothing to disclose.

AUTHOR CONTRIBUTIONS

KD and LV designed the study. KD organized the sample collection of the different centers. KD, LV, and JM collected samples and identified sample characteristics. Samples were analyzed under the supervision of KD. LV and KD interpreted the data, performed the statistical analyses, and wrote the manuscript. KD, HK, and BdL organized the SSC questionnaire. HK, BdL, and JM critically reviewed the manuscript.

Click here for additional data file.