Evaluation and validation of a novel 10-color flow cytometer.

BACKGROUND: Flow cytometry is a powerful technique that provides information regarding cell properties. In this study, we evaluated the analytical performance of a new flow cytometer, the 10-color BD FACSLyricTM , which could help doctors obtain reliable test results prior to clinical research.
METHODS: We used SpheroTM Rainbow Calibration Particles and the SpheroTM Nano Fluorescent Particle Size Standard Kit to validate the fluorescence sensitivity and linearity. The Beckman Coulter IMMUNO-TROL Cell was used as the quality control to evaluate the accuracy and reproducibility of surface markers detected by the flow cytometer. Furthermore, BD Calibrate APC Beads and CS&T Research Beads were applied to calculate the carry-over contamination rate and assess the instrument stability.
RESULTS: A linear regression equation between the molecules of equivalent soluble fluorochrome and fluorescence detection limit showed a good linear fit (R2  > 0.99). The minimum bead size detected by side scatter was 0.22 μm. The coefficient of variation percentage of each fluorescence channel was below 2%, and the carry-over contamination rate of the cytometer was under 0.2%. After running the BD FACSLyricTM cytometer continuously for 8 h, the median fluorescence index of particles remained close to that at the time of cytometer startup.
CONCLUSIONS: The 10-color BD FACSLyricTM cytometer showed good performance in the evaluation performed in this study and may be trusted to provide accurate results for clinical research.

INTRODUCTION

Flow cytometry (from the Greek words cyto = cell and metry = measure) (FCM) is a powerful technique that detects, characterizes, and analyses assayed cells through their fluorescence and light scattering responses and provides information about the physical and chemical properties of cells, including their morphology, cell granularity, and genetic identity, among others. Its advantage lies in its ability to analyze uniform cell populations and highlight non‐uniformity in samples without averaging, which distinguishes it from the Western blot technique. Moreover, FCM can also analyze complex cell populations according to user‐defined cell characteristics, including the cell number and size, macromolecular content, and genetic identity that can be determined through labels, stains, and probes, with an analysis rate of about 10,000 cells per second. With the continuous development of FCM during the past two decades, the results of scientific research have gradually diffused to clinical practice and FCM is playing a crucial role in the diagnosis, treatment, and monitoring of tumors, infections, primary immunodeficiencies, and hematological diseases. , In addition, FCM is also used extensively in lymphocyte subset classification, immunophenotyping, CD34 stem cell absolute count evaluation, and human leukocyte antigen B27 (HLA‐B27) and paroxysmal nocturnal hemoglobinuria (PNH) determination. Therefore, FCM performance evaluation is crucial for guaranteeing the accuracy of laboratory test results. In 2017, Cossarizza et al. and Veldhoen reported guidelines for flow cytometers, which focused on the instrument settings and laser adjustment. However, there is not much research on flow cytometers themselves, which is critical for the validation of FCM in a clinical setting. The novel BD FACSLyricTM flow cytometer has the capability of configuring up to 3 lasers—blue, red, and violet—12 fluorescence channels, and 14 parameters. Its maximum acquisition rate is 35,000 events per second, and there is no limit on the number of events acquired. We evaluated the analytical performance of the novel 10‐color BD FACSLyricTM flow cytometer in our laboratory to ensure its performance efficiency and result accuracy for clinical and research applications.

MATERIAL AND METHODS

This section describes the methods used for obtaining the performance characteristics of the novel 10‐color flow cytometer (BD FACSLyricTM).

Working conditions of the flow cytometer

The 10‐color flow cytometer (BD FACSLyricTM) was placed in an environment with room temperature (20°C), humidity between 40% and 70%, a power supply voltage of 220 ± 22 V and 50 ± 1 Hz and standard atmospheric pressure. Direct exposure to sunlight and other heat sources was eliminated.

Fluorescence sensitivity and linearity

The fluorescence performance was measured using SpheroTM Rainbow Calibration Particles (8 peaks, catalog number RCP‐30‐20A) with a nominal size of 3.0–3.4 μm. 500 μl of phosphate‐buffered saline (PBS) was mixed with a drop of the solution of calibration particles. The linear equation and R‐square of the molecules of equivalent soluble fluorochrome (MESF), MESF(Y), and mean fluorescence intensity (MFI) were calculated. It should be noted that the fluorescence detection limit should not be higher than 200 for fluorescein isothiocyanate (FITC) and 100 for phycoerythrin (PE). The fluorescence detection limit of fluorochrome in the corresponding fluorescence laser channels should meet the manufacturer's requirements (r ≥ .98).

Forward scatter (FSC) sensitivity

The SpheroTM Nano Fluorescent Particle Size Standard Kit (catalog number NFPPS‐52–4 K) was used, which was composed of four bead sizes with a diameter of 1.35, 0.88, 0.45, and 0.22 μm. According to the SSC and FITC signal, the FCM histograms displayed the peak signal and diameter of standard microbeads, which can be employed to determine the limit of FSC detection. The FSC detection limit is required to be ≤1 μm.

Signal resolution

The resolution was measured using BD CS&T Research Beads (catalog number 349523) using the FITC and PE channels, and the coefficient of variation (CV) of each channel was calculated. CS&T research beads consist of equal quantities of 3 μm bright, 3 μm mid, and 2 μm dim polystyrene beads in PBS with bovine serum albumin (BSA) and 0.1% sodium azide. The CV was monitored to guarantee that it remained at 3.00% or less.

FSC and Side scatter (SSC) resolution

We added 5 μl of peripheral blood into 1 ml of sheath fluid and collected at least 10,000 events in the flow cytometer to evaluate the FSC and SSC resolution. Then, we determined the FSC vs. SSC plots for erythrocytes and platelets. The FSC vs. SSC plots show the size overlap of erythrocyte (right‐top) and platelet (left‐bottom) populations.

DNA polity linearity

BD DNA QC particles (catalog number 349523) containing chicken erythrocyte nuclei (CEN) were stained by propidium iodide (PI). The DDM enables cell doublets in G0/G1 phase, as well as other aggregates, to be distinguished from single cells in G2+M phase, thus allowing better cell‐cycle estimates of cell percentages in G2+M phase. The linearity was calculated using the mean channel number, located in the Histogram Statistics box, for G2+M and the mean channel number for G0/G1 using the following formula. The linearity should be between 1.95 and 2.05.

Accuracy of surface marker determination

Quality controls (Beckman coulter IMMUNO‐TROL Cell, lot number 6607077) were determined via a BD Multitest™ 6‐color TBNK Reagent kit (BD Lymphocyte subset kit, lot number 644611). The means of CD3, CD4, CD8, CD19, CD16/56, and CD45 cell percentages were calculated (n = 5). The results should be within the reference range of the target value given in the Beckman coulter control cell specification.

Reproducibility of surface marker determination

Reproducibility was estimated as the CV of CD3, CD4, CD8, CD19, CD16/56, and CD45 determinations (n = 10) under identical conditions, which means that the operator, the instrument, and the laboratory conditions were kept constant within a short time interval. For this purpose, we used the Beckman coulter IMMUNO‐TROL Cell (catalog number 6607077). When cell percentage is ≥30%, the CV should not be higher than 8%; otherwise, CV should not be higher than 15%.

Carry‐over contamination

BD Calibrate APC Beads (catalog number 340487) in BD Trucount™ Tubes (lot number 340487) were counted three times. At least 100,000 standard particles were collected each time. Beads counts were determined and recorded as H –1, H –2, and H –3, respectively. Then, blanks (dd water) were measured and marked L –1, L –2, and L –3. The carry‐over contamination rate (C ) was calculated using equation (1). Ci should not be higher than 0.5%. Ci is the carry‐over contamination rate of the i th cycle, i = 1–3.

Stability of the BD FACSLyricTM flow cytometer

The BD CS&T Research Beads (catalog number 349523) were used for measurements of the supported BD digital flow cytometer (BD FACSLyricTM), and a histogram was generated and analyzed. The MFI of the beads at 0 h (FL1) was recorded. After performing the experiment continuously for 8 h, the experiment was repeated under the same parameter settings, and the MFI of standard particles (FL2) was calculated. When the ambient temperature does not exceed 5% of the setting temperature, the fluctuation range of the FSC and the peak fluorescence from all fluorescent channels should not exceed 10% within 8 h of starting up. where (FL1)is the MFI immediately after starting up and FL2 is the MFI 8 h after starting up.

Statistical methods

For each characteristic that was studied, continuous variables are presented as mean ± standard deviation (SD) or median (interquartile range). For cases with less than 30 values for the tested parameter, the normal distribution was verified by the Kolmogorov‐Smirnov normality test before performing the t test. Differences in categorical variables were assessed by the chi‐square test or the Fisher exact test. In addition, the concordance correlation coefficient was calculated. The precision is represented by the upper and lower limits of agreement (if the differences are normally distributed, 95% of them will lie between these limits). Statistical analyses were performed using SPSS 22.0 (IBM Corp). A two‐tailed p value of less than .05 was taken as the condition for statistical significance.

RESULTS

SpheroTM Rainbow Calibration Particles were used to detect the molecules of equivalent fluorochrome (MEF) value. A standard curve was generated with a fluorimeter using fluorochrome solutions of various concentrations. We collected at least 10,000 events and then calculated the MEF value dividing the equivalent fluorochrome concentration by the number of cells or particles used. The results for the FITC and PE channels are shown in Figure 1.

FIGURE 1

Fluorescence sensitivity and linearity. X‐axis indicates MESF value. Y‐axis shows the detection limit of fluorescence sensitivity. (A) FITC channel results and (B) PE channel results

FSC sensitivity

The mixed solution that contained the SpheroTM Nano Fluorescent Particle Size Standard Kit was processed. The different sized subsets of calibration beads are shown in Figure 2. The histogram results show the peak signal.

FIGURE 2

FSC sensitivity. SpheroTM Nano Fluorescent Particle Size Standard Kit, microbeads with a diameter of 1.35, 0.88, 0.45, and 0.22 μm. (A) Shows that SSC combined with FITC can distinguish four different sized beads, (B and C) illustrate beads count percentages and absolute counts according to SSC, respectively

Resolution signal of BD FACSLyricTM

BD CS&T Research Beads (lot number 349523) were mixed with 500 μl of PBS. We tested the solution and calculated the CV of the beads’ peak width in each channel, as shown in Figure 3

FIGURE 3

Resolution signal of the flow cytometer. (A) Shows collected events in SSC/FSC; P1 represents beads. (B, C, D, and E) Show the resolution signals of the FITC, PE, APC, and V500‐C channels, respectively. CVs are shown on the top left of each figure

FSC and SSC resolution

We used FSC and SSC to separate the platelets and red blood cells (Figure 4A). Moreover, we added 1 ml of lysing solution into a tube with 100 μl of peripheral blood. After the red blood cells were lysed, we collected at least 10,000 events to evaluate the separation of lymphocyte, monocyte, and granulocyte (Figure 4B).

FIGURE 4

FSC and SSC resolution. (A) Shows separation of platelets and erythrocytes and (B) shows separation of lymphocyte, monocyte, and granulocyte

CEN stained by PI were used to evaluate DNA polity linearity. The G2/M and G0/G1 of MFI were obtained, and the ratio was calculated (Figure 5).

FIGURE 5

DNA polity linearity. The peak of P1 and P2 separately shows the MFI of G2/M and G0/G1. The result was 50038 and 100139, respectively. The P1/P2 is 2.0012, which was close to the target value of 2

Accuracy of surface marker determinations

We tested five tubes for each level according to the directions of the Beckman coulter IMMUNO‐TROL Cell. The percentage and absolute cells of each lymphocyte subset were calculated to compare them with the target value of the QC samples. Table 1 shows that the results of every item were close to the target value, demonstrating the accuracy of the flow cytometer evaluated in this study.

TABLE 1

Accuracy of surface markers determinations

Item	Quality control 1			Quality control 2
	Average value	Target	Reference interval	Average value	Target	Reference interval
CD3%Lymphs	58.97	58.50	48.50–68.50	73.66	73.80	63.80–83.80
CD3+ Abs Cnt	738.80	719.00	560.80–877.10	1143.20	1106.30	885.00–1327.50
CD3+CD4+ %Lymphs	10.60	10.40	6.40–14.40	45.14	46.60	40.10–53.10
CD3+CD4+ Abs Cnt	127.60	127.80	76.70–178.90	700.80	698.50	558.80–838.20
CD3+CD8+ %Lymphs	39.07	41.70	34.70–48.70	23.77	24.20	17.20–31.20
CD3+CD8+ Abs Cnt	489.40	512.50	375.70–649.30	368.8	362.8	261.5–464.00
CD19+ %Lymphs	21.39	21.40	15.40–27.40	14.08	13.60	9.60–17.60
CD19+ Abs Cnt	267.80	263.00	164.90–361.10	218.80	203.90	141.10–266.7
CD3−CD16+CD56+ %Lymphs	17.80	18.80	13.00–23.00	10.69	11.10	4.00–18.00
CD3−CD16+CD56+ Abs Cnt	223.20	221.20	163.90–278.50	165.80	164.90	87.40–242.40

Reproducibility of surface marker determinations

We used the Beckman coulter IMMUNO‐TROL Cell to repeat the precision experiments ten times. The CVs of CD3, CD4, CD8, CD19, CD16/56, and CD45 determinations (n = 10) were calculated according to the previous results, as shown in Figure 6. The CV percentages of items in low‐level controls were all below 8%, while those of items in normal‐level controls were all under 6%. These results show excellent reproducibility.

FIGURE 6

Reproducibility of surface markers determinations. Lymphocyte subsets results are shown. (A and B) Correspond to quality control 1, (C and D) correspond to quality control 2. (A and C) Show the percentages of each lymphocyte subset; (B and D) show absolute counts

BD Calibrate APC Beads mixed with PBS in BD Trucount™ Tubes were collected three times containing at least 100,000 standard particles each time. The carry‐over contamination rates of the cycles were calculated as 0.17% (blanks result: 302, 183, and 133; beads result: 100019, 100020, and 100175), 0.13% (blanks result: 204, 320, and 74; beads result: 100188, 100166, and 100169), and 0.14% (blanks result: 227, 91, and 83; beads result: 100145, 100186, and 100175).

BD CS&T Research Beads (catalog number 349523) were mixed with PBS and measured after the startup of the cytometer. The beads’ MFI at 0 h (FL1) was recorded. After performing the experiment continuously for 8 h, the experiment was repeated under the same parameter settings, and the MFI of standard particles (FL2) was calculated. According to the calculated FL1 and FL2 values, the biases were measured and are shown in Figure 7.

FIGURE 7

Stability of the flow cytometer. Black histograms present MFI of beads of ten different laser channels at startup (0 h, FL1); white histograms represent those after running continuously for 8 h (8 h, FL2). The biases of these two different MFI were calculated, which were all below 6.3%

DISCUSSION

Validation is the process used to confirm the accuracy and precision of a given analytical method or instrument. After installing a new flow cytometer, the excitation light sources and optical system should first be evaluated before detecting clinical samples. According to the U.S. Clinical Laboratory Improvement Amendment (CLIA), for any given existing test, if significant changes have occurred in an analyzing system, such as the introduction of a new instrument into the test procedure, the performance evaluation of the new instrument should be verified under laboratory conditions before the system can be used in other applications. However, many clinical laboratories have ignored this. EuroFlow develops and standardizes fast, accurate, and highly sensitive flow cytometric tests for the diagnosis and prognostic (sub)classification of hematological malignancies as well as for the evaluation of treatment effectiveness during follow‐up. There are EuroFlow standard operating procedures for instrument setup and compensation for the 8‐color flow cytometer systems BD FACS Canto II and Navios and the 10‐ or 12‐color system BD FACS Lyric, and the instrument setup standard operating procedure (SOP) must be followed when applying EuroFlow guidelines. Note that the application of EuroFlow guidelines is based on the assumption of the appropriate performance of the flow cytometer system. If there is any problem with the laser itself, a direct compensation adjustment plan will produce bias and hinder the differential diagnosis.

Our performance evaluation was based on Clinical and Laboratory Standards Institute (CLSI) Guideline H62: Validation of Assays Performed by Flow Cytometry. H62 is, for the first time in the flow cytometry field, a validation guideline for the international community. In this study, we evaluated the analytical performance of a novel BD FACSLyricTM cytometer via lymphocyte subsets analysis. FACSLyricTM is the latest flow cytometer from BD Bioscience. Therefore, BD Bioscience conducted a series of performance evaluation tests and presented the results on their official website. As high‐speed detection is one of the focuses of FACSLyricTM, BD Bioscience has placed emphasis on run rates and flow rates. The carry‐over contamination obtained by BD Bioscience and that obtained in our study is substantially different. The sample carryover of ≤0.05% shown by BD Bioscience for FACSLyricTM is lower than the results obtained in our study, respectively, 0.17%, 0.14%, and 0.13%. Further, the stability evaluation conducted by BD Bioscience only entails one‐point stability, namely, the stability after startup. By contrast, in our study, we performed a two‐point stability evaluation, where one point is after startup and the other is after continuously running the experiment for 8 h. Moreover, we also performed different performance evaluations compared to those reported by BD Bioscience. Specifically, we focused on clinical sensitivity and accuracy. Our performance evaluation serves as a supplement for the evaluation performed by BD Bioscience, especially for clinical laboratory users. Based on the CLSI guidelines and previous studies by Selliah et al. and Piccoli et al., we evaluated the BD FACSLyricTsystem based on (1) fluorescence sensitivity and linearity, (2) slide scatter sensitivity, (3) resolution signal of the FACSLyricTM system, (4) FSC and SSC resolution, and (5) accuracy of surface markers determinations. Compared with the results presented by Gossez et al., which also used Beckman Coulter IMMUNO‐TROL Cell control samples, the reproducibility of lymphocyte surface markers (CD4) in BD FACSLyricTM is much better than that in SPT AQUIOS CL. The CVs of our BD FACSLyricTM are 1.94% and 4.52%, while those of SPT AQUIOS CL are 3.26% and 9.2%. Coetzee et al. also evaluated the CD4+ lymphocyte absolute count and percentage detection in two AquiosTM flow cytometers; it is determined that the performance of BD FACSLyricTM is nearly equal to that of those instruments. The CVs of the absolute count of CD4+ lymphocytes in AquiosTM flow cytometers are 4.29% and 7.62%, while those in the BD FACSLyricTM system are 3.76% and 6.96%. The CVs of the CD4+ lymphocyte percentage in AquiosTM flow cytometers are 2.41 and 3.82, while those in the BD FACSLyricTM system are 1.94% and 4.52%. These results show that the reproducibility in the BD FACSLyricTM system is close to that in AquiosTM flow cytometers. The results for signal resolution in different BD FACSLyricTM lasers meet professional standards when comparing the results corresponding to the BD FACSCanto II and the Beckman Coulter Navios. The MFIs of FITC are 13,740.04, 15,484.07, and 12068.03 in Canto, Navios, and BD FACSLyricTM, respectively, while those of PE are 13,362.45, 15,379.33, and 14257.14, respectively. These results indicate that the signal resolutions of these three flow cytometers are equivalent to each other. Compared with the six‐color lymphocyte subsets kit used in BD FACSCanto II, the six‐color TBNK kit used in BD FACSLyricTM significantly reduced the operating procedures and the on‐board detection time and optimized the lymphocyte subsets analysis process in clinical applications. For clinical research, BD FACSLyricTM shows potential as a high‐efficiency and high‐throughput platform to detect specimens, which could greatly help clinical technicians scale‐up flow cytometry detection. Moreover, BD FACSLyricTM provided strong support for clinicians ordering customized detection items.

Presently, with the extensive use of FCM for both clinical and research purposes, the performance evaluation criteria for detection systems are expected to become stricter. Clinical technicians need to grasp the protocols of performance evaluation for routine work in clinical laboratories. With the continued development of FCM in the future, its technical standards and applications will be further expanded. In addition, to enhance the results of the FCM system, it is suggested to focus on the performance and working conditions of the instrument, which would be based on the future developments in the innovation and fabrication aspects of this technology.

Despite its many advantages, this study had several limitations. This was a single‐center BD FACSLyricTM performance evaluation. Because of this, we did not estimate the batch to batch variations among different clinical laboratories, and therefore, the precision experiments are not complete. A multi‐center evaluation will obtain more information to make the criteria more demanding. In future work, a multi‐laboratory reproducibility assessment will be conducted following technical standards. Additionally, we conducted our validation experiments using only microbeads. In the future, we hope to add clinical samples, such as blood and bone marrow cells, to the evaluation to further improve our validation results. Nonetheless, the performance of the BD FACSLyricTM cytometer met the requirements of not only the new version of professional standards but also those of clinical applications. It was determined that BD FACSLyricTM is a reliable system for providing accurate results to clinicians.

CONFLICT OF INTEREST

Declaration or none declared.

AUTHOR CONTRIBUTION

Wei Guo contributed to the conception and design of the study. Lin Sun and Hui Wu involved in clinical evaluation Baishen Pan and Beili Wang interpreted the results. Lin Sun performed the statistical analysis and drafted the manuscript. Wei Guo supervised the study. All authors read and approved the final manuscript.