Comparison of multiplex cytokine assays in a pediatric cohort with epilepsy.

BACKGROUND: Multiplex analyses allow for detection of dozens of cytokines/chemokines in small sample volumes. Although several commercially available assay kits are available, there are no comparative data in plasma measurements among pediatric or epilepsy cohorts. NEW
METHOD: Cohort study of 38 children with epilepsy. We evaluated plasma levels of cytokines/chemokines using three different assays: Luminex® xMAP high-sensitivity (HS) and standard-sensitivity (SS) assays, and Meso-Scale Discovery (MSD). We calculated recovery rates of each analyte, correlation coefficients between assays, and level of agreement between measurements. We repeated analyses in a subset of samples after a single freeze-thaw cycle.
RESULTS: Among ten analytes common to all assays, HS had high recovery (<15% of values extrapolated or out-of- range [OOR]) for all analytes, SS for 50%, and MSD for 40%. While several analytes had a high correlation between assays, Bland-Altman plots demonstrated assays were not interchangeable. For most analytes, a single freeze-thaw cycle decreased cytokines/chemokine measurements. There was good correlation of measurements after a freeze-thaw cycle with acceptable agreement between measurements for six of 13 (46%) analytes using HS, one of 9 (11%) for SS, and none for MSD. COMPARISON WITH EXISTING
METHODS: HS assays may optimize yield in plasma for proteins of particular interest in epilepsy research, limit values extrapolated beyond the standard curve, and improve precision compared to other SS and MSD assays.
CONCLUSION: Our results demonstrate assay choice may be critical to study results and support the need for a standardized approach to biomarker assessment across epilepsy research and other domains.

Introduction

Fluid biomarkers are increasingly investigated in neurologic disease to assist in diagnosis, monitor disease progression, and predict outcome (2016; Esenwa and Elkind, 2016; Malekzadeh et al., 2017; Numis et al., 2019; Rodney et al., 2018). Enzyme-linked immunosorbent assays (ELISA) are the gold standard for evaluating protein concentration, but require relatively large volumes of fluid and are limited to evaluation of one analyte at a time. Multiplex analyses are increasingly used for detection of dozens of analytes in small sample volumes, allowing a more complete understanding of complex interactions at lower cost with lower biofluid volumes (Ray et al., 2005). In pediatric cohorts, where only small volumes of biofluid are available due to safety concerns with larger blood draws, multiplexing is particularly useful in research investigations. Multiplexing includes both liquid and solid phase assays. Bead based assays (i.e. Luminex® xMAP) are pre-coated to capture specific antibodies using spectrally distinct beads (Ellington et al., 2010). After incubation, lasers identify the bead and detection agent in order to quantify each analyte. Another class of assays include meso-scale discovery (MSD) arrays where primary antibodies are bound to specified carbon spots on plates followed by electrochemical stimulation of bound ruthenium-conjugated secondary antibodies. Light is then emitted, amplified and measured (Burguillos, 2013). Multiplex platforms offer high-sensitivity assays for select analytes, with improved dynamic range and better recovery of physiologic levels of cytokine and chemokine values.

There are several commercially available products for multiplexing. Correlations of analytes between kits vary with measurements dependent on analyte, biofluid, and cohort (Belzeaux et al., 2017; Malekzadeh et al., 2012). Age- and disease-specific considerations must also be taken into account in study design to optimize protein recovery in key cytokines and chemokines of interest (Kothur et al., 2016; Larsson et al., 2015; Lee et al., 2017; Mizutani et al., 2017), but there are limited data in evaluation of plasma in neurologic cohorts and none in pediatric and epilepsy populations (Malekzadeh et al., 2012). Plasma concentrations of cytokines of particular interest in neurology are frequently out of range using standard sensitivity assays (Belzeaux et al., 2017; Chowdhury et al., 2009; Parkitny et al., 2013). Freeze-thaw cycles can further decrease yields, but these changes also widely depend on platform, manufacturer, and analytes of interest, with limited investigations of high-sensitivity assays (de Jager et al., 2009; Henno et al., 2017; Huang et al., 2017; Parkitny et al., 2013; Ray et al., 2005; Shen et al., 2018; Simpson et al., 2020).

Reproducibility is key to biomarker development and identification of best practice parameters is necessary to replicate results in multiple cohorts. In this study, we compared three assays to measure plasma cytokines and chemokines in a pediatric cohort with epilepsy. We evaluated the assays' ability to detect circulating analytes as well as the role of freeze-thaw cycles on protein recovery in each platform.

Materials and methods

Study design and population

We performed an ancillary investigation in a prospective longitudinal cohort study of children diagnosed with new-onset and intractable infantile spasms (IS). All participants were recruited from a clinic-based convenience sample of patients diagnosed with IS at the University of California, San Francisco Benioff (UCSF) Children's Hospital, Pediatric Epilepsy Center of Excellence between January 2017 and December 2018. IS was defined according to the International League Against Epilepsy (ILAE) criteria as seizures characterized by “epileptic spasms … a sudden flexion, extension, or mixed extension–flexion of predominantly proximal and truncal muscles” occurring in clusters and confirmed with electroencephalography (EEG) (Fisher et al., 2017). The study protocol was approved by the UCSF Institutional Review Board and a parent of each child provided written informed consent. This work has been carried out in accordance with The Code of Ethics of the World Medical Association.

Sample collection and storage

One to three milliliters of whole blood were drawn by peripheral venipuncture into a K2-EDTA vacutainer tube. Plasma collection was selected to optimize yield of cytokines of interest in epilepsy, in particular those within the IL-1 family and NLRP3 inflammasome signaling pathways including IL-1β and IL-6 (de Jager et al., 2009; Ulusoy et al., 2020; Wu et al., 2019). Within one hour of collection (Henno et al., 2017), blood was centrifuged for 10 minutes at 1,300 g at 4°C, the supernatant transferred to a separate tube, and then centrifuged for 10 minutes at 16,000 g at 4°C. The resulting platelet-poor plasma was stored in 0.5 mL aliquots at -80 °C until analysis.

Multiplex assays

Aliquots of plasma were thawed, and analytes evaluated using three different assays. For Luminex® xMAP assays we used MilliporeSigma (MilliporeSigma, Burlington, MA, USA) Human High Sensitivity T-cell Panel (HSTCMAG-28SK; herein called HS) and Human Cytokine/chemokine Magnetic Bead Panel (HCYTMAG-60K-PX41; herein called SS) based on our groups' experience, ability to detect low levels of cytokines and chemokines of interest in epilepsy, and correlation with ELISA (Breen et al., 2011; dupont et al., 2005). For MSD® (Meso Scale Discovery, MD, USA) analyses we used the Proinflammatory Panel 1 Kit (K15049D). Details of assay sensitivity for common analytes provided by the manufacturer are summarized in Table 1. For each assay, all reagents were prepared and used according to the manufacturer's guidelines. For Luminex® assays, the plate was incubated with wash buffer for 10 min at room temperature. After removal of washing buffer, sample wells were filled with assay buffer, diluted standard or 25 μL sample (in duplicate), then premixed beads. Subsequently the plate was sealed and incubated at 4 °C (shaking) overnight. The next day, the unbound well content was removed, and beads washed with buffer. Next, the beads were incubated with detection antibodies followed by Streptavidin-Phycoerythin solution at room temperature. After washing and addition of sheath fluid, plates were run on the Luminex® 200™. For MSD assays, prepared samples (in duplicate), calibrators or controls were added to plate wells. The plate was sealed and incubated for two hours at room temperature. Then, after washing, detection antibody added to each well, and plates were sealed and incubated at room temperature for two hours. Plates were again washed, read buffer added to each well, and the plate analyzed on the MSD instrument.

Table 1

Manufacturer provided minimum detectable concentrations (pg/mL) for selected analytes across three cytokine/chemokine assays.

	IL-1β	IL-2	IL-4	IL-6	IL-8	IL-10	IL-12p70	IL-13	IFN-γ	TNF-α
HS	0.14	0.19	1.12	0.11	0.13	0.56	0.15	0.23	0.48	0.16
SS	0.8	1.0	4.5	0.9	0.4	1.1	0.6	1.3	0.8	0.7
MSD	0.05	0.09	0.02	0.06	0.07	0.04	0.11	0.24	0.37	0.04

Abbreviations: HS = MilliporeSigma Human High Sensitivity T-cell Panel; MSD = MesoScale Discovery Proinflammatory Panel 1 Kit.SS: SS = MilliporeSigma Human Cytokine/chemokine Magnetic Bead Panel.

Freeze-thaw cycle

In subjects with an adequate volume of plasma remaining after initial recovery experiments, we evaluated the effects of a freeze-thaw cycle on analytes. After the initial sample thaw and analysis, remaining plasma was refrozen and stored at −80 °C until later assay. We evaluated proteins using identical methods described above using the MSD assay and custom versions of MilliporeSigma panels derived from the previously used kits and limited to cytokines and chemokines most associated with epilepsy and with adequate recovery in our initial experiments. The custom MilliporeSigma Human High Sensitivity T-cell Panel included fractalkine, GM-CSF, IL-1β, IL-4, IL-6, IL-8, IL-10, IL-12p70, IL-13, IL-17A, MIP1-α, MIP1-β, and TNF-α; the custom MilliporeSigma Human Cytokine/chemokine Magnetic Bead Panel included sCD40L, eotaxin/CCL11, GRO, IL-1RA, IL-9, IP-10, MCP-1, MCP-3, and RANTES.

Data analysis and statistics

Cytokine concentrations were determined using platform specific software, Bioplex Manager and Bioplex DataPro (Bio-Rad, Hercules, CA, USA), for Luminex and Discovery Workbench (Rockville, MD, USA) for MSD. Intra-assay coefficient of variances (CV) were calculated for each protein between duplicate measurements. An average CV<20% for each analyte was considered acceptable as previously described (Chowdhury et al., 2009). Samples with analyte levels with CV>20% were averaged and included in all analyses. Analyte concentrations were considered 'out of range' (OOR) if they fell in regions of the curve where standards fell outside the recovery percentage range of 70–130%. "OOR<" values were replaced with values that were half of the minimum detectable level in an assay and "OOR>" values were replaced with values that were double the highest standard. Analyte concentrations were considered to be "extrapolated" if they were below the lowest or highest standard for the assay but not OOR.

All analyses were performed using Graphpad Prism (v8.4.2) and Stata (v15.1). Cytokine concentrations were not expected to be normally distributed and non-parametric tests were used. The sensitivity of each assay in analyte recovery was assessed by calculating the frequency of values OOR and/or extrapolated. A high recovery of an analyte was defined as <15% of values that were extrapolated or OOR. For inter- and intra-assay analyses, Spearman's Rho was calculated with exclusion of analyte levels OOR. P-values were corrected using the false discovery rate with the Benjamini–Hochberg procedure, with significance defined with an alpha <0.05. For analytes with statistically significant correlations between measurements across platforms or after a freeze-thaw cycle, Bland-Altman plots were created to quantify agreement between measurements across platforms or after a freeze-thaw cycle, with the difference of repeated measurements of cytokine levels plotted against the average value for each pair of measurements. Limits of agreement (LOA) were defined as the 95% confidence interval for the difference between measurements, representing the maximum increase or minimum decrease in value on repeat measurement (Bland and Altman, 1986). In this analysis, we classify agreement as “moderately good” when the LOA are less than the mean concentration (100%) of the tested cytokine; and, "poor agreement" when the LOA is greater than a two-fold difference in mean concentrations (Massaro et al., 2019; Parkitny et al., 2013).

Any data not published within this article will be shared by reasonable request from any qualified investigator.

Results

Thirty-eight children with epileptic spasms were enrolled in the parent study. The median age in the cohort was 1.5 years (Interquartile Range [IQR] 0.6–2.1 years). Twenty-one participants had plasma analyzed for ten common analytes (IL-1β, IL-2, IL-4, IL-6, IL-8, IL10, IL-12p70, IL-13, IFN-γ, and TNF-α) using the HS, SS and MSD assays. An additional 17 participants had plasma analyzed for six common analytes (fractalkine (CX3CL1), GM-CSF, IL-5, IL-7, MIP-1α, and MIP-1β) using the HS and SS assays.

Recovery of circulating cytokines and chemokines in plasma of pediatric patients

Recovery was high on all assays for IL-8, IL-10, IFN-γ, and TNF-α (Figure 1). Recovery for the remaining analytes varied by platform. The HS assay had high recovery for the six remaining common analytes (IL-1β, IL-2, IL-4, IL-6, IL-12p70, IL-13, IFN-γ, and TNF-α), SS had high recovery for IL-4 only, and MSD had no other common analytes with high recovery. Average CV was less than 20% for all cytokines and chemokines evaluated except IL-12p70 (29%) and IL-13 (34%) using the MSD assay.

Figure 1

Among ten cytokines and chemokines evaluated using the Luminex® xMAP high-sensitivity assay (HS), Luminex® xMAP standard sensitivity assay (SS), and Meso-Scale Discovery assay (MSD): A) scatterplots of analyte levels plotted on a logarithmic scale, B) percentage of values extrapolated beyond the lower limits of quantification but above the lower limit of detection (extrapolated %) and below the lower limits of detection/out of range (OOR %) by assay, and C) Spearman's Rho correlation matrices comparing analyte levels between assays.

At the sample level, there were no HS measurements extrapolated off the standard curve and one of 38 (3%) measurements that were OOR for two analytes, IL-6 and IL-13 (Figure 1). The SS assay yielded measurements that were extrapolated below the standard curve for seven of ten (70%) analytes. The proportion of samples with extrapolated measurements ranged from one of 38 (3%) for IL-8, IL-10, and IFN-γ to 23 of 38 (62%) for IL-1β. The SS assay yielded measurements that were OOR for six of ten (60%) analytes, ranging from two of 38 (5%) for IL-10 to 17 of 38 (45%) for IL-13. The MSD assay yielded measurements that were extrapolated below the standard curve for six of 10 analytes (60%). The proportion of samples with extrapolated measurements was one of 21 (5%) for IL-1β to 19 of 21 (90%) for IL-2. The MSD assay yielded measurements that were OOR for seven of ten (70%) of analytes, ranging from two of 21 values (10%) for IL-12p70 and TNF-α to 20 of 21 values (95%) for IL-1β.

For the six analytes common to the HS and SS assays, there was high recovery in fractalkine, IL-7, and MIP1-β on both assays (Figure 2). The HS assay resulted in values extrapolated below the standard curve for three of six (50%) analytes, ranging from two of 38 (5%) for IL-5 to three of 38 (8%) for fractalkine. HS levels were OOR for two of six (33%) analytes including one of 38 (3%) for IL-7 and two of 38 (5%) of values for fractalkine. The SS assay resulted in values extrapolated below the standard curve for three of six (50%) analytes ranging from one of 38 (3%) for IL-7 to 35 of 38 (92%) for IL-5. SS values were OOR for three of six (50%) analytes ranging from one of 38 (3%) for fractalkine and IL-5 to ten of 38 (26%) for GM-CSF.

Figure 2

Among six cytokines and chemokines evaluated using the Luminex® xMAP high-sensitivity assay (HS) and standard sensitivity assay (SS): A) scatterplots of analyte levels plotted on a logarithmic scale, B) percentage of values extrapolated beyond the lower limits of quantification but above the lower limit of detection (extrapolated %) and below the lower limits of detection/out of range (OOR %) by assay, and C) Spearman's Rho correlation matrices comparing analyte levels between assays.

Correlation and agreement of circulating cytokines and chemokines values between assays

Correlation matrices of analyte levels between assays, after exclusion of levels OOR, are presented in Figure 1. Sensitivity analyses with inclusion of OOR values substituted at half the lower limit of detection did not affect results. TNF-α had the highest correlation of values between assays with a Spearman Rho for HS/SS of 0.71 (95% confidence interval (CI): 0.50–0.84, p < 0.001), HS/MSD of 0.74 (CI: 0.41–0.90, p = 0.004) and SS/MSD of 0.89 (CI: 0.72–0.96, p < 0.001). Significant correlations were also found for IL-8 HS/SS of 0.48 (CI: 0.18–0.70, p = 0.02) and IL/10 HS-SS of 0.49 (CI: 0.19–0.71, p = 0.02). Among the six analytes common to the HS and SS assays only (Figure 2), there were significant correlations between assays for MIP1-α with a Spearman Rho of 0.44 (CI: 0.13–0.67, p = 0.03) and MIP1-β of 0.53 (CI: 0.25–0.73, p = 0.005).

Bland-Altman analyses quantify the agreement of measurements between two assays. For all but one analyte with significant correlation of measurements between assays, the 95% confidence interval for LOA between platforms was larger than 100%, indicating poor agreement (Figure 3). Sensitivity analysis with removal of outliers (>3 standard deviations) did not alter the results. For TNF-α, there was moderately good agreement between SS and MSD assays with a LOA interval of 46%.

Figure 3

Bland-Altman comparing plasma levels of representative circulating cytokines and chemokines between assays. The y-axis represents the percent different between paired measurements and the x-axis represents the average of the analyte levels between assays. The dashed horizontal lines indicate the 95% confidence intervals of the percent difference. A 100% difference would indicate a doubling or halving of the mean.

Impact of a single freeze-thaw cycle on cytokine and chemokine recovery

Using the HS assay, we repeated measurements of 13 analytes in 22 samples after a single freeze thaw cycle and storage at -80 °C of 14 months, with high recovery for all analytes (Table 2). All cytokines and chemokines levels decreased after a single freeze-thaw cycle, with up to an 83% decrease in GM-CSF levels (IQR: -87 to -76%) (Table 3). Nine (69%) of 13 analytes had good correlation of analyte measurements between freeze-thaw cycles. Bland-Altman analyses demonstrated that five of nine analytes (56%) had moderately acceptable LOA after a single freeze-thaw cycle, though with variable systematic differences (Figure 4).

Table 2

Evaluation of cytokines and chemokines levels below the lower limits of quantification before and after a freeze-thaw cycle using three different assays.

	HS (n = 22)N (%)		SS (n = 23)N (%)		MSD (n = 21)N (%)
	Pre F/T OOR	Post F/T OOR	Pre F/T OOR	Post F/T OOR	Pre F/T OOR	Post F/T OOR
IL-1β	0	0	-	-	19 (90)	19 (90)
IL-4	0	0	-	-	4 (19)	3 (14)
IL-6	1 (5)	0	-	-	2 (10)	0
IL-8	0	0	-	-	0	0
IL-10	0	0	-	-	0	6 (29)
IL-12p70	0	0	-	-	2 (10)	0
IL-13	1 (5)	1 (5)	-	-	8 (38)	13 (62)
TNF-α	0	0	-	-	2 (10)	8 (38)
IL-17A	0	0	-	-	-	-
Fractalkine	0	0	-	-	-	-
GM-CSF	0	0	-	-	-	-
MIP-1α	0	2 (9)	-	-	-	-
MIP-1β	0	0	-	-	-	-
IL-2	-	-	-	-	0	0
IFN-g	-	-	-	-	2 (10)	7 (33)
IL-1Ra	-	-	0	3 (13)	-	-
IL-9	-	-	4 (17)	15 (65)	-	-
IP-10	-	-	0	0	-	-
Eotaxin	-	-	0	0	-	-
Gro	-	-	0	0	-	-
MCP-1	-	-	0	0	-	-
MCP-3	-	-	0	4 (17)	-	-
RANTES	-	-	0	0	-	-
sCD40L	-	-	0	0	-	-

Abbreviations: F/T = Freeze-thaw cycle; HS = MilliporeSigma Human High Sensitivity T-cell Panel; IQR = interquartile range; MSD = MesoScale Discovery Proinflammatory Panel 1 Kit; OOR = out-of-range (below the lower limit of detection); SS = MilliporeSigma Human Cytokine/chemokine Magnetic Bead Panel.

Table 3

Changes in cytokine and chemokine levels after a single freeze-thaw cycle.

	Median Percent Change in Levels (IQR)			Spearman Correlation (95% CI)
	HS (n = 22)	SS (n = 23)	MSD (n = 21)	HS (n = 22)	SS (n = 23)	MSD (n = 21)
IL-1β	-65.3 (-68 to -60)	-	‡	0.74 (0.45–0.89) ∗∗∗	-	‡
IL-4	-53.4 (-62 to -49)	-	‡	0.59 (0.22–0.82) ∗∗	-	‡
IL-6	+1.7 (-21 to +52)†	-	+17.5 (0 to +18)†	0.15 (-0.31–0.56)	-	0.32 (-0.27–0.75)
IL-8	-24.6 (-48 to -1)	-	+370.8 (-20 to +450)	0.19 (-0.27–0.57)	-	-0.022 (-0.46–0.42)
IL-10	-57.6 (-64 to -46)	-	-3.7 (-74 to -2)†	0.82 (0.61–0.93) ∗∗∗	-	0.26 (-0.31–0.69)
IL-12p70	-58.3 (-62 to -50)	-	-48.3 (-56 to -39)	0.87 (0.69–0.94) ∗∗∗	-	0.17 (-0.29–0.57)
IL-13	-65.5 (-81 to -29)†	-	‡	0.48 (0.29–0.77)	-	‡
TNF-α	-19.5 (-40 to -9)	-	+22 (+13 to +33)†	0.74 (0.45–0.89) ∗∗∗	-	0.64 (0.25–0.85) ∗∗
IL-17A	-18.5 (-27 to -7)	-	-	0.92 (0.82–0.97) ∗∗∗	-	-
Fractalkine	-24.7 (-32 to -14)	-	-	0.84 (0.64–0.93) ∗∗∗	-	-
GM-CSF	-82.6 (-87 to -76)	-	-	0.92 (0.81–0.97) ∗∗∗	-	-
MIP-1α	-24.4 (-49 to -19)†	-	-	0.44 (-0.02–0.74)	-	-
MIP-1β	-5.3 (-17 to +16)	-	-	0.58 (0.20–0.81) ∗∗	-	-
IL-2	-	-	-40.0 (-79 to -30)†	-	-	0.60 (-0.36–0.56)
IFN-g	-	-	+10.8 (-11 to +14)†	-	-	0.27 (-0.32–0.71)
IL-1Ra	-	+18.5 (-21 to +112)†	-	-	0.64 (0.25–0.85) ∗∗	-
IL-9	-	‡	-	-	‡	-
IP-10	-	+3.3 (-7 to +12)	-	-	0.75 (0.48–0.89) ∗∗∗	-
Eotaxin	-	-12.7 (-15 to -7)	-	-	0.83 (0.62–0.93) ∗∗∗	-
Gro	-	-29.5 (-39 to -19)	-	-	0.76 (0.49–0.89) ∗∗∗	-
MCP-1	-	-30.8 (-61 to +17)	-	-	0.79 (0.55–0.91) ∗∗∗	-
MCP-3	-	-46.8 (-77 to 12)†	-	-	0.64 (0.24–0.85) ∗∗	-
RANTES	-	+92.2 (+70 to +103)	-	-	0.017 (-0.41–0.44)	-
sCD40L	-	-43.7 (-57 to -27)	-	-	0.47 (0.06–0.74) ∗	-

Abbreviations: CI = confidence interval; HS = MilliporeSigma Human High Sensitivity T-cell Panel; IQR = interquartile range; MSD = MesoScale Discovery Proinflammatory Panel 1 Kit; SS = MilliporeSigma Human Cytokine/chemokine Magnetic Bead Panel.

∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001 for Spearman correlations rho with false discovery rate correction.

= 1 or more values OOR.

= > 50% of values OOR.

Figure 4

Bland-Altman comparing plasma levels of representative circulating cytokines and chemokines between freeze-thaw cycles. The y-axis represents the percent different between paired measurements and the x-axis represents the average of the analyte levels between freeze-thaw cycles. The dashed horizontal lines indicate the 95% confidence intervals of the percent difference. A 100% difference would indicate a doubling or halving of the mean.

Using the SS assay, we repeated measurements of nine analytes in 23 samples after one-freeze thaw cycle and storage at -80 °C of 14 months. Seven of nine (77%) analytes had a high recovery (Table 2). Changes in cytokine/chemokine measurements ranged from a 46.8% decrease in MCP-3 levels (IQR: -77 to +12%) to a 92% increase in RANTES levels (IQR: +70 to +103%) (Table 3). Three of nine (33%) analytes had measurements that became OOR after a single freeze-thaw cycle. Assay of IL-9 showed the greatest degradation, with >50% of values OOR. Assays of seven of nine analytes (77%) had good correlation of levels between freeze-thaw cycles. Bland-Altman analyses demonstrated that only MCP-1 had a moderately acceptable LOA (Figure 4).

Using the MSD assay, we repeated measurements of ten analytes in 21 samples after one-freeze thaw cycle and storage at -80 °C of 3 months. Five of ten (50%) analytes had high recovery (Table 2). Changes in cytokine/chemokine measurements ranged from a 48% decrease in IL-12p70 levels (IQR: -56 to -39) to a 371% increase in IL-8 levels (IQR: -20 to +450) to (Table 3). Six (60%) of 10 analytes had measurements that became OOR after a single freeze-thaw cycle. Assay of IL-1β, IL-4 and IL-13 showed the greatest degradation, with >50% of values OOR. TNF-α only had good correlation of levels between freeze-thaw cycles. Bland-Altman analyses did not demonstrate an acceptable LOA, indicating lack of interchangeability between levels from one freeze-thaw cycle to another.

Discussion

Age- and disease-specific cross comparisons between different multiplex formats are needed to select the optimal platform for cytokine and chemokine profiling. In this study of pediatric patients with epilepsy, we evaluated the recovery of plasma cytokines and chemokines using liquid- and solid phase multiplex assays. Recovery varied by analyte and assay, with IL-8, TNF-α and IFN-γ having the best recovery across all three platforms. Levels of IL-1β and IL-2 were most variable with values below the lower limit of quantification requiring extrapolation or OOR for all but the high-sensitivity assay, and we hypothesize this resulted in the observed differences in values and distributions between assays. We found the HS had high recovery for the ten analytes common to the three assays, the SS had high recovery for five of ten (50%) analytes, and MSD for four of ten (40%). Correlation of analyte levels between assays varied by cytokine/chemokine, with HS and SS assays having a higher number of analytes with good correlation when compared with the MSD assay. However, Bland-Altman plots demonstrate that despite correlation of levels between assays, levels were not interchangeable, stressing that comparison of results between investigations using different types of assays must be interpreted with caution.

Comparisons of cytokine recovery across assays has been investigated in adult cohorts with infection and inflammatory diseases (Breen et al., 2011; Malekzadeh et al., 2012, 2017). To our knowledge, this is the first study comparing assays of inflammatory cytokines and chemokines in the plasma of a pediatric or epilepsy cohort, with our results sharing similarities to prior investigations. Across studies particular cytokine concentrations may vary due to differences in recovery between assays (i.e. IL-4) or may be consistently variable (i.e. IL-1b). The IL-1 axis, including IL-1β, IL-1RA, IL-6 and TNF-α are of particular interest in epilepsy due to their potential role in epileptogenesis, disease modification and treatment (Kenney-Jung et al., 2016; Kwon et al., 2013; Numis et al., 2019). Similar to prior investigations, we find that IL-1β and IL-6 recovery was highest with the HS assay, with the SS and MSD assays providing levels that were largely extrapolated below the lower limit of quantification (Belzeaux et al., 2017; Dabitao et al., 2011; McKay et al., 2017). The HS assay was able to detect IL-1β levels within the assays dynamic range for all subjects, even after a single freeze-thaw. Other differences in analyte recovery, perhaps specific to a cytokine and chemokine levels in a (pediatric) epilepsy cohort, included enhanced recovery of IFN-γ and IL-12p70 compared to a cohort of HIV + men (using the same manufacturer's assay and analyzed in the same lab) as well as enhanced recovery of TNF-α compared to a group of patients with MS (Breen et al., 2011; Malekzadeh et al., 2017).

As biomarker investigations are leveraged against larger clinical trials in epilepsy, samples may undergo multiple freeze-thaw cycles (de Jager et al., 2009). Overall, we found that nearly all analytes saw a decrease in levels after a single freeze-thaw cycle, with HS handling a single freeze-thaw cycle better than MSD, but dependent on the particular cytokine/chemokine. Levels within an assay tended to correlate between a single freeze-thaw cycle, however Bland-Altman plots demonstrated concentrations are largely not interchangeable. The few, previous reports on the role of freeze-thaw cycles suggest that the effects are both protein- and population-dependent, varying from negligible changes in the analytes we evaluated in an adult cohort enrolled in a prostate, lung, colorectal and ovarian cancer screening trial, to significant variability in these analytes in adults with hand and wrist fractures (Huang et al., 2017; Parkitny et al., 2013; Simpson et al., 2020). We demonstrate in our cohort that even when the average differences in analyte concentrations after a freeze-thaw cycle are small, agreement may be poor and care must be taken in comparing results across any studies that have used a freeze-thaw cycle(s).

Limitations of our findings include analysis at a single site precluding the ability to assess inter-lab variability, which can be a significant factor in analyte detection. While absolute analyte values can vary between labs, we anticipate the trends in cytokine and chemokine changes would be consistent across testing sites (Breen et al., 2011). Cytokine concentrations may also vary with differing lots, or batches of assay kits (Breen et al., 2011). Our freeze-thaw cycle analysis relied on differing lots given the time between freeze-thaw cycles, which acts as a confounder on our results. We hypothesize that our results are more likely to reflect true changes in analyte levels due to the freeze-thaw cycling as opposed to lot variability due to our effect sizes and our finding that cytokine levels with significant correlations before/after a freeze-thaw cycle clustered by assay as opposed to individual analytes. Based on our groups' published experience with Luminex® xMAP assays and our epilepsy-specific analytes of interest, we restricted bead-based assays to kits from MilliporeSigma (Breen et al., 2011). We cannot compare our results to those generated from assays of other manufacturers (Berthoud et al., 2011). Since our analysis, MSD has released ultra-sensitive singleplex assay kits with potential for multiplexing and assay sensitivity to the fg/mL range and larger dynamic range compared to standard sensitivity MSD assays in key analytes (Meso Scale Diagnostics, 2020). It is possible that this assay would perform as well or better than the bead-based HS assays used here. In our freeze-thaw samples A limitation to external generalizability is our lack of a control group. While our results are comparable to similar investigations in adult cohorts, there may be nuances in cytokine and chemokine concentrations unique to a pediatric epilepsy cohort using these assays that will require further confirmation (Breen et al., 2011; Malekzadeh et al., 2012, 2017).

Conclusions

In epilepsy, circulating inflammatory cytokines and chemokines may serve as biomarkers for diagnosis, prognosis, and risk stratification for those at risk (Ravizza et al., 2017; Vezzani et al., 2019). Our results support the need for a standardized approach to biomarker assessment in epilepsy to allow for comparison between investigations. Our results show that assay choice with particular emphasis on disease-relevant proteins may also be critical to study results, as different assays have different yields, with many analytes extrapolated beyond the standard curve and limiting precision.

Declarations

Author contribution statement

Adam Numis: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Christine Fox and Daniel Lowenstein: Conceived and designed the experiments; Wrote the paper.

Philip Norris and Clara Di Germanio: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by the American Academy of Neurology, Clinical Research Training Fellowship in Epilepsy (A130002) and the American Epilepsy Society Research and Training Fellowships for Clinicians (A129351).

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare the following conflict of interests: Dr. Numis reports Grant Support from the (K23NS105918). Dr. Fox reports Grant Support from the (U54NS065705) and the Pediatric Epilepsy Research Foundation.

Additional information

No additional information is available for this paper.