Efficiency and Health Economic Evaluations of BD OneFlow™ Flow Cytometry Reagents for Diagnosing Chronic Lymphoid Leukemia.

REASON FOR THE STUDY: To standardize the use of flow cytometry for classifying hematological malignancies and make the results reliable and reproducible across laboratories, the EuroFlow™ Consortium published a comprehensive specification of antibody-fluorochrome conjugates, standard protocols, and algorithms for analysis. The BD OneFlow™ system builds on, and further standardizes, the EuroFlow protocols. We aimed to assess the effects on safety, efficiency, and costs for laboratories of adopting the BD OneFlow reagent tubes (LST and B-CLPD T1) for diagnosing chronic lymphocytic leukemia.
METHODS: We compared in-house laboratory processes and results with those using the LST and B-CLPD T1 reagent tubes with, and without, blood film morphology. Outcome measures included concordance in classification results, and efficiency within the laboratory, that is, resource usage, staff time, unwanted events, and cost-consequences.
RESULTS: There was 100% concordance between the classifications made with in-house flow cytometry and those with the BD OneFlow reagent tubes. Using BD OneFlow tubes required 13 hours less staff time per month (i.e. for 100 samples) than the in-house process. Sensitivity analyses explored the effects of uncertainties in the price of the BD OneFlow tubes and the prevalence of CLL and identified the thresholds at which laboratories might expect cost-savings from adopting the BD OneFlow system. Laboratory and clinical staff considered the BD OneFlow system to be safe and effective.
CONCLUSIONS: Laboratories adopting the BD OneFlow system for classifying patients with suspected CLL can expect safe, efficient processes that can be cost saving if the discount on the list price, and prevalence of CLL (which will both vary between sites and countries), is within the thresholds suggested by the health economics sensitivity analysis.

Recent advances in flow cytometric immunophenotyping technologies allow suspected hematological malignancies to be rapidly and accurately classified. Many laboratories providing this service devise their own antibody‐fluorochrome combinations, develop, and validate the testing panel in‐house, and then analyze the diagnostic data accordingly. One result of this nonstandardized approach is that, while a final diagnosis made by two different laboratories on the same sample might not differ, there will be different workflows and different data generated for analysis. In 2004, the EuroFlow Consortium initiated a project to standardize the use of flow cytometry for the classification of hematological malignancies and to make the results reliable and reproducible across laboratories. The outcome was a comprehensive specification of the antibody‐fluorochrome conjugates, standard protocols, and algorithms for analysis 1.

BD Biosciences have developed a system, BD OneFlow™, which builds on, and further standardizes, the EuroFlow specifications and protocols by providing a set of reagent tubes with premixed dried‐down antibodies, new standard operating procedures for instrument setup and compensation (calibration), acquisition, and analysis templates. New software for multivariate analysis, Infinicyt™ (Cytognos SL, Salamanca, Spain), is available for EuroFlow databases 1, 2, 3. The initial two reagent sets available from the planned full BD OneFlow product line were the lymphoid screening tube (LST) and the first of four reagent tubes for classifying B cell lymphoproliferative disorders, the B‐cell Chronic Lymphoproliferative Diseases Tube 1 (B‐CLPD T1). The reagents in these two tubes have been validated by EuroFlow for the immunophenotyping of chronic lymphocytic leukemia (CLL). CLL was selected as the condition of interest for this study because it is the most common lymphoproliferative disorder and constitutes the bulk of the work in a clinical flow cytometry laboratory 4.

We hypothesized that adoption of the BD OneFlow system would improve the efficiency of sample processing in the laboratory (particularly in terms of staff time) and reduce the occurrence of near misses (events such as reagent mix‐ups and sample labeling errors that were detected before misclassification could occur). This would reduce the risk of error, without any negative impact on the diagnostic service offered in terms of diagnostic accuracy. The study therefore focused on laboratory outcomes, without considering the impact of diagnostic results on patients.

We compared classification with current in‐house practice to classification using the BD OneFlow LST and B‐CLPD T1 reagent tubes with, and without, peripheral blood film morphology. We assessed the contribution of blood film morphology to classification with OneFlow, because centralized laboratories may not have access to this.

METHODS AND MATERIALS

Methods for Workflow Evaluation

Setting

The study was carried out by the flow cytometry service that is housed within the Blood Sciences Laboratory of the Royal Victoria Infirmary (RVI), Newcastle upon Tyne Hospitals NHS Foundation Trust (NUTH). The laboratory is home to the Northern England Haemato‐Oncology Diagnostic Service (NEHODS).

Patient population

Samples were from adults with a lymphocytosis. Patient would have been clinically assessed to exclude infectious causes of lymphocytosis, so the prevalence of CLL was expected to be high, around 50%. Referrals came from general practitioners (GPs), NUTH clinics and hospitals, and other laboratories and hospitals.

Study design

This was a prospective observational cohort study in which three testing and classification strategies were followed for each sample:

1. Current practice: Flow cytometry using in‐house antibody/fluorochrome cocktails and full blood count data, other routinely used diagnostic information, and peripheral blood film morphology. This information was used to make the final diagnosis and guide clinical decisions.

2. BD OneFlow with morphology: Flow cytometry using the BD OneFlow LST and B‐CLPD T1 reagent tubes, full blood count data, other routinely used diagnostic information, and blood film morphology. This is potentially an immediate efficiency improvement possible within current NHS practice.

3. BD OneFlow without morphology: Flow cytometry using the BD OneFlow tubes, full blood count data, and other routinely used diagnostic information, but excluding morphology. This is potentially a novel (in the NHS) and even more efficient practice that might be suitable for regional/national service laboratories if it can be shown to be accurate.

No information from the BD OneFlow flow cytometry was made available to the patient or to their clinicians.

Classification of flow cytometry results

We chose to study CLL as it is the most common B cell malignancy seen in our laboratory. The study used simplified versions of the clinical classification 4 and the long‐standing flow cytometry CLL “Matutes score” 5 so that we could compare the efficiency of the BD OneFlow LST tube with the in‐house process. The efficiency analyses used both a minimal and a more fine‐grained classification as follows:

Samples were first classified as CLL; or Another condition; or Indeterminate—following the Matutes score. Indeterminate is a clonal B cell condition with some features of CLL, but not all.

Then the non‐CLL samples were subclassified as Malignant (i.e. any other clonal lymphoid disorder) or nonmalignant. Samples with CLL phenotype (clonal B‐lymphocyte count <5 × 109/L) were included with nonmalignant cases for the concordance and health economics analyses.

Concordance

Results were classified as concordant or discordant according to whether classification with the BD OneFlow reagent tubes (Strategies 2 and 3) was the same as, or different from, classification with the in‐house laboratory procedure (Strategy 1).

We took account of three causes for discordance: design, interpretation, and inaccuracy.

Discordant by design

While the in‐house process and the process with the BD OneFlow LST and B‐CLPD T1 reagent tubes are both designed to classify samples as CLL or non‐CLL, their designs differ in the further classification that is possible, in particular the ability to classify T cell disorders as likely malignant or not—our in house strategy cannot classify T cell disorders. For the purposes of this study, immunophenotyping that differed between strategies because of differences in further classification were reviewed by an expert panel, where appropriate regarded as “discordant by design,” counted as concordant in terms of diagnosis of CLL and are reported separately.

Discordant by interpretation

The results of the BD OneFlow immunophenotyping for classification with and without peripheral blood film morphology were interpreted by different hematologists. When interpretations differed, this was regarded as “discordant by interpretation” and the difference resolved (if possible) by expert panel review.

Discordant by inaccuracy

If immunophenotyping differed for reasons other than design or interpretation, this was regarded as discordance by inaccuracy.

Efficiency outcome measures

The key value proposition for the BD OneFlow system, increased efficiency within the laboratory, was assessed in terms of (1) resource usage (e.g. consumables such as antibodies and other reagents; validation of new batches of reagents; maintenance contracts); (2) staff time (e.g. instrument setup and calibration; stock management, steps in preparation for and processing of samples; analysis and reporting of flow cytometry results; training); (3) untoward events (e.g. errors in preparation of test cocktails; expiry of unused stock); and (4) health economics: cost‐consequences and budget impact.

Sample size and recruitment considerations

Approximately 100 flow cytometry tests for CLL are carried out routinely in the laboratory each month. It was estimated that, over a study period of 4 months, 100 samples could be enrolled and processed, and that generalizable conclusions on efficiency could be drawn from the data. For future studies on accuracy and utility, the data can be used to estimate optimal sample sizes.

Data management and statistical analysis

Data from the collection forms were entered into a spreadsheet, which was used to calculate the classification performance statistics.

Methods for Multicolor Flow Cytometric Immunophenotyping

Peripheral blood samples were processed by two different techniques: in‐house and BD OneFlow methods.

In‐house method

The in‐house method follows the process recommended by the British Committee for Standards in Hematology or validated against their guidelines 6. Whole blood was diluted, if required, to a white cell count of <20 × 109/L in phosphate buffered saline (PBS). Blood for the kappa/lambda tube only was washed three times in PBS. Blood was then stained for immunophenotyping using the in‐house four tube antibody panel, each tube containing four or five colors as detailed in Table 1. The concentrations of these antibodies were determined with standard titration studies. After 15 min incubation, the samples were lysed with BD FACS™ lysing solution (Becton Dickinson, BD). After a further 15 min incubation, tubes were centrifuged once and the cell pellet resuspended in PBS. In‐house panels were acquired within 1 h of staining using BD FACSDiva™ software in‐house templates, and 10,000 lymphocyte events, as defined by a forward‐side scatter gate, were collected. Experiment application settings were derived from BD™ CS&T beads and BD™ CompBeads (catalogue numbers 656,047 and 552,843). Beads are fluorescent particles, used to set photomultiplier voltages so that fluorescence levels are optimized. Analysis was carried out using in‐house BD FACSDiva software templates.

Table 1

Antibody Panels Used with In‐House Testing and with BD OneFlow LST and B‐CLPD T1 Tubes

Fluorochromes		FITC	PE	Per‐CP‐Cy 5.5	PE‐Cy 7	APC	APC‐H7	V450	V500
In‐house panels
Lymph one	Antibody	FMC7	CD23	CD19		CD5
	Clone	FMC7	EBV CS‐5	SJ2 5C1		L17F12
Lymph two	Antibody	CD103	CD22	CD20		CD10	CD45
	Clone	Ber‐ACT8	S‐HCL‐1	L27		HI10a	2D1
Lymph three	Antibody	CD79b	CD38	CD19		CD11c
	Clone	SN8	HB‐7	SJ2 5C1		S‐HCL‐3
Lymph four	Antibody	CD2	anti‐lambda	CD19		anti‐kappa
	Clone	S5.2	1‐155‐2	SJ2 5C1		TB 28–2
Fluorochromes		FITC	PE	PerCP‐Cy 5.5	PE‐Cy 7	APC	APC‐Cy7
6 –ColorTBNK	Antibody	CD3	CD16	CD45	CD4	CD19	CD8
	Clone	SK7	B73.1	2D1	SK3	SJ25C1	SK1
	Antibody		CD56
	Clone		NCAM16.2
BD OneFlow antibody panels
Fluorochromes		FITC	PE	PerCP‐Cy 5.5	PE‐Cy 7	APC	APC‐H7	V450	V500
BD OneFlow LST	Antibody	CD8	CD56	CD5	CD19	CD3	CD38	CD4	CD45
	Clone	SK1	MY31	L1F12	SJ25‐C1	SK7	HB7	SK3	2D1
	Antibody	anti‐lambda	anti‐kappa					CD20
	Clone	1–155‐2	TB28					L27
BD OneFlow B‐CLPD	Antibody	CD23	CD10	CD79b	CD19	CD200	CD43	CD20	CD45
	Clone	EBV CS‐5	MEM‐78	SN8	SJ25‐C1	MRC OX‐104	1G10	L27	2D1

Samples found not to have a clonal B cell population were set up with the BD Multitest™ 6‐color TBNK reagent, which detects T, B, and natural killer (NK) cells.

BD OneFlow method

The antibody panels for the LST and B‐CLPD T1 reagent tubes are detailed in Table 1. The tubes and specimens were processed according to the manufacturer's instructions 7, 8. BD OneFlow tubes were acquired immediately after specimen staining on a BD FACSCanto™ II, equipped with three lasers, using BD FACSDiva software and BD OneFlow acquisition templates, and 100,000 events were collected. Instrument settings were derived from the BD OneFlow Setup beads and the BD FC beads, 8‐color kit for BD OneFlow (catalogue number 658620). Analysis was carried out using BD FACSDiva analysis templates specifically designed for BD OneFlow application. Statistics were edited to allow for the mean fluorescence intensity for kappa, lambda, CD20, and CD79b to be accessed. The process was evaluated on a normal population before the project started.

Ethics and research governance

This study was approved by the HRA (IRAS project ID 196260, REC reference 16/NE/0369). The study sponsor was the Newcastle upon Tyne Hospitals NHS Foundation Trust, (R&D approval reference 8059).

Methods for Health Economic Evaluation

The health economics methods are described in the Supporting Information section.

RESULTS

Concordance Analysis

One hundred samples were obtained between March 13, 2017 and July 27, 2017; 56 patients had CLL; 44 patients had other causes of lymphocytosis, 27 of which were nonmalignant and 17 malignant. Nine samples were CLL phenotype and (for analysis only) are included with the nonmalignant group. Although the protocol specified that patients who had previously been diagnosed with CLL would be excluded, data from six such samples (inadvertently enrolled) were retained for the analyses, because their removal would have made no significant difference to the conclusions, and the full data set more precisely represents the usual flow cytometry workload for CLL investigation.

Classification with BD OneFlow immunophenotyping was as accurate as with the in‐house laboratory procedure. There were two instances of “discordant by design” in which BD OneFlow identified B cells that the in‐house laboratory process was not sensitive enough to detect, and these are shown in Figures 1 and 2. There was a single instance of initial “discordant by interpretation,” which on review, was regarded concordant for distinguishing between CLL and non‐CLL. Accurate classifications using the BD OneFlow LST and B‐CLPD T1 tubes were made without blood film morphology in all 100 cases. For details on concordance, see Table S3 in the online Supporting Information. Because the study did not aim to evaluate diagnostic performance and safety, it was not powered to do so, and no P‐values were calculated.

Figure 1

The difference in information provided by the in‐house and BD OneFlow processes (discordance by design) is illustrated by plots (a) and (b). Plot (a), for in‐house tube four, gated on FSC V SCC, shows that B cell events are low and the population in the north‐east quadrant is not distinct. Plot (b), for the LST reagent tube gated on B cells, shows the abnormal blue population much more clearly than is possible with the in‐house process.

Figure 2

The difference in information provided by the in‐house and BD OneFlow processes (discordance by design) is illustrated by the two pairs of plots. Plot (a) shows in‐house reagent tube 1 and Plot (b) in‐house reagent tube 4; both plots are gated on FSC V SSC. A large CD5+ kappa restricted B cell population is clearly visible. Plots (c) and (d) are for the BD OneFlow LST tube gated on B cells. A second, small, lambda restricted CD5+ B cell population is visible on plot (c) and in plot (d), but is not clear in the in‐house tubes.

Efficiency Outcomes: Staff Time and Laboratory Consumables

No unwanted events such as running out of stock or errors in processing were observed with either the BD OneFlow or the in‐house approaches. However, laboratory staff thought that the likelihood of such unwanted events would be lower with the BD OneFlow system, because there are fewer reagent tubes to manage, fewer reagents to process, no manual reagent pipetting, longer shelf‐life, and reagents can be stored at room temperature rather than in a refrigerator.

Resource usage and costs for the three testing strategies are detailed in Supporting Information Tables S2–S13. Of the 100 samples, 21 could be classified on the LST tube results alone, and 79 required the B‐CLPD T1 tube to complete the CLL/non‐CLL classifications; 25 samples that did not have a clonal B cell population were also tested with the TBNK tube to further classify the non‐CLL group.

Using BD OneFlow tubes required 13 hours less per month (i.e. for 100 samples) than the in‐house process, with the greatest time saving coming from pipetting and preparing tubes. Details are listed in the Supporting Information Table S2 and visualized in Figure 3. The staff estimated that further time savings of around 30 hours per month would have been possible if the Infinicyt software had been used. The health economics analyses took into account the pay band (salary level) of the different kinds of staff required for each task that was timed, and details are shown in the Supporting Information Tables S2, S3, and S4. With blood film morphology being done in all cases, there would be a saving of £36.63 per sample (£3663 per month), if all samples could be classified with the BD OneFlow LST tube alone. There would be an additional cost of £21.61 per sample (£2161 per month) were classification to also require the B‐CLPD T1 tube—see Supporting Information Table S7. Full details of the budget impacts are given in the Supporting Information Table S13.

Figure 3

Differences in staff time per specimen which would follow from adopting the BD OneFlow system (with blood film morphology). Processes that would take the same time are not shown. Time savings are associated with a cost saving. Further cost savings arise when lower salary grade staff can be employed. These potential cost savings are not shown in the figure but are detailed in the tables.

Exploration of the Effects of Uncertainties in Input Parameters

Sensitivity analyses explored the effects on the results due to uncertainties in the model's input parameters. Supporting Information Table S12 shows the sensitivity analysis that explores the impact that changes in the proportion of cases which require both the LST and B‐CLPD T1 reagent tubes would have on costs compared to current practice. When LST alone is sufficient for CLL classification in 40% of cases (with both LST and B‐CLPD T1 used in the remaining 60%), the expected cost of testing with BD OneFlow is less than the expected cost of testing with current practice (which assumes that the B‐cell panel is always required, with no TBNK). Cost savings of BD OneFlow increase further as this percentage is increased.

Figure 4, an additional sensitivity analysis, is an aid to applying these findings to other sites and countries. Savings depend on the new laboratory's costs for CLL classification (which include costs of staff time and use of resources), the laboratory's discount on BD OneFlow list prices, and their case‐mix, that is, the proportion of cases being investigated for CLL which require both the BD OneFlow LST and the B‐CLPD‐1 reagent tubes. The x‐axis crosses the y‐axis at £0, that is, changes in costs are zero if the new site has the same costs as our study site. To apply the plot in Figure 4 to another site, draw a line parallel to x‐axis so that it crosses the y‐axis at the difference between the cost for current methods at the study site and at the new site—in effect this is changing the reference or base cost to reflect that of the prospective new site. Identify the point on the new line where it crosses the line that represents the case‐mix at the new site. The X‐value of this point estimates the discount on the BD OneFlow list prices at which savings could be realized by the new laboratory.

Figure 4

This bivariate sensitivity analysis shows the effect on costs of (i) case‐mix (the prevalence of CLL in samples, i.e. the proportion of cases which require the B‐CLPD T1 reagent tube after testing with the LST tube) and (ii) the discount on the list price of LST and B‐CLPD T1 reagent tubes. The plots assume that the cost of blood film morphology is included. As this is both small and constant, lines for costs excluding blood film morphology are not shown.

DISCUSSION

The BD OneFlow system proved safe and capable of generating savings in the context of a regional laboratory in the UK NHS. With the use of the sensitivity analyses that explored the effects of uncertainties in the model's input parameters, the results can be applied to other sites in other countries. Efficiencies in terms of staff time saved by adopting the BD OneFlow system were assessed in terms of person‐hours and costs of person‐hours when taking pay scales into account. However, this undervalues the benefits of standardization with BD OneFlow, because this approach omits, for example, the impact on the operation of the laboratory when a flow cytometer is occupied for more than an hour with setup and validation processes that need to be carried out at regular intervals. The value of the additional information provided by the BD OneFlow system is illustrated by two patients who had abnormal B cell populations that were detected with the BD OneFlow LST tube but not with the in‐house method. Their immunophenotyping is shown in Figures 1 and 2.

Laboratory and clinical staff identified a range of pros and cons for the BD OneFlow system compared to the in‐house processes. Pros include standardization (which assures quality and safety, reduces the need for operator training, and supports the laboratory's accreditation) and the inclusion of anti‐CD200 in the antibody panel, which allows separation of mantle cell lymphoma from other B cell disorders 9. Possible cons (which would be site‐specific) include cost, the need for three lasers in the flow cytometer, and the need for training for clinicians interpreting the results.

Supplementary Table s1Concordance analyses for BD OneFlow results compared with the laboratory's practice

Supplementary Table s2. Staff timings for CLL testing

Supplementary Table s3. Costs of staff time

Supplementary Table s4. Costs of staff time (per specimen) — Additional panel for current practice

Supplementary Table s5. Differences in resource use: items used for CLL testing (per specimen) (all items listed but costs included for items which differed between strategies only)

Supplementary Table s6. Resource use items for BD Multitest™ 6‐color TBNK (per specimen)

Supplementary Table s7. Cost difference per sample tested with BD OneFlow LST tube alone or with both LST and B‐CLPD T1 tubes

Supplementary Table s8. Cost of BD Multitest™ 6‐color TBNK reagent (per specimen)

Supplementary Table s9. Cost estimation for test specific antibodies mix (per specimen) (Current practice)

Supplementary Table s10. Cost estimates for the BD OneFlow LST and B‐CLPD reagent tubes (per specimen) (UK list prices)

Supplementary Table s11. Cost estimation for setup and calibration‐specific reagents (Current practice and BD OneFlow)

Supplementary Table s12. Varying percentage in use of LST alone

Supplementary Table s13. Budget impact analysis (per year).

Click here for additional data file.