Evaluation of criteria of manual blood smear review following automated complete blood counts in a large university hospital.

BACKGROUND: There is great interest in reducing the number of automated complete blood counts requiring manual blood smear reviews without sacrificing the quality of patient care. This study was aimed at evaluating and establishing appropriate screening criteria for manual blood smear reviews to improve the performance in a hematology laboratory.
METHOD: A total of 1977 consecutive samples from the daily workload were used to evaluate four sets of screening criteria for manual blood smear reviews to identify samples with positive smear findings. Three sets of screening criteria were arbitrarily proposed in this study: Group 1 (narrow ranges), Group 2 (intermediate ranges), and Group 3 (wide limits) and one set (Group 4) was adapted from the International Society for Laboratory Hematology. All samples were run on Sysmex hematology analyzers and were investigated using manual blood smear reviews. Diagnostic accuracy and agreement were performed for each set of screening criteria, including an investigation of microscopic review rate and efficiency.
RESULTS: The microscopic review rates for Groups 1, 2, 3 and 4 were 73.85%, 54.52%, 46.33% and 46.38%, respectively; the false-negative rates were 0.50%, 1.97%, 2.73% and 3.95%, respectively. The efficiency and negative predictive values of Group 3 were 73.04% and 4.91%, respectively.
CONCLUSIONS: Group 3 had the best relationship between safety (false-negative rate: ≤3%) and efficiency to estimate the limits of automation in performing complete blood counts. This study strengthens the importance of establishing screening criteria for manual blood smear reviews, which account for the different contexts in which hematological determinations are performed. Each laboratory should optimize the screening criteria for manual blood smear reviews in order to maximize their efficiency and safety.

Introduction

A manual blood smear review (MBSR) is defined as the thorough and careful microscopic analysis of a well-prepared and stained smear of peripheral blood, with the objective of seeking morphological changes relevant to the diagnosis and monitoring of patients. It is also considered a tool of internal quality control for the evaluation of parameters provided by hematology analyzers. The process of MBSR is among the most time-consuming in hematology laboratories, and requires high technical competence to minimize errors inherent to the subjectivity of MBSR, including manual differential leukocyte counts (MDLC).1, 2, 3, 4, 5, 6

Over the last few years, the performance and abilities of automatic hematology analyzers have improved considerably. Although they still cannot identify all morphological abnormalities that may occur in peripheral blood, they can reliably decrease the MBSR without sacrificing quality.7, 8, 9, 10, 11, 12 The establishment of screening criteria (SC) for MBSR is critical and is based on the determination of screening limits for the major hematological parameters and occurrence of suspect flags. The morphological changes and cell percentages relevant for the diagnosis and monitoring of patients are defined as positive smear findings (PSF). The SC are formulated such that MBSR occurs only when needed for confirming the parameter values, or to provide relevant clinical information represented by the PSF in addition to that generated by hematology analyzers.13, 14, 15, 16, 17, 18

The main factors influencing the establishment of SC for MBSR can vary between institutions and include features such as, the type of population served, type of hematology analyzer employed, training and experience of the medical team, volume of work, number of professionals in the laboratory, medical specialties involved, complexity of the services offered, financial considerations, and regulatory policies of institutions.1, 4, 19 Although many SC for MBSR have been proposed, they are not completely applicable to all laboratories. Our previous work demonstrated that SC for MBSR adapted from the International Society for Laboratory Hematology (ISLH) were not adequate or safe for use in the Clinical hematology laboratory of Hospital de Clínicas at the Federal University of Paraná (HC-UFPR). Thus, as an improvement in searching for the ideal SC for MBSR with broader application in hematology laboratories, the objective of this study was to propose and evaluate new SC for MBSR, which serve as a model for adjustments and consider peculiarities in the profile of the populations served, maintaining safety and efficacy.

Methods

Study site

The investigation was conducted in the hematology laboratory of HC-UFPR, a general Class IV hospital and the largest provider of government healthcare services in the State of Paraná, southern Brazil, with 627 beds (574 in operation). Moderately to highly complex procedures are carried out in 59 departments including a hematologic malignancy unit, bone marrow transplant unit, emergency department and intensive care units. Approximately 30,000 outpatients are seen each month.

Samples, patients and hematology analyzers

The study design was approved by the local Ethics Review Board recognized by the National Research Ethics Committee (CONEP). Whole blood samples were collected in ethylenediaminetetraacetic acid (EDTA)-K2 (1.8 mg/mL) and results were obtained from the laboratory routinely, on seven consecutive days in the months of November and December (spring in the southern hemisphere) after their release into the hospital information system. Altogether 1977 (1100 females and 867 males) consecutive samples meeting the local specimen acceptance criteria were obtained from 1615 patients (946 females and 669 males), with an average age of 39.7 ± 22.7 years (range: 1 day to 96 years). Within these samples, 1320 (66.76%) were outpatients and 657 (33.24%) were being admitted or were hospitalized. Four hundred and ten samples were from the hematology unit, 232 from the emergency department, 154 from the intensive care unit (ICU), 94 from the adult ICU, 45 from the neurology unit, 37 from the bone marrow transplant unit, 27 from the pediatric and neonatal ICU, 27 from the infectious diseases unit, 21 from the liver transplantation unit, and two from the renal transplantation unit, among others. Three hundred and twenty-nine samples were from children aged ≤12 years and 36 were newborn babies. Furthermore, 1573 samples were analyzed in a Sysmex XE-2100D and 404 in a Sysmex XT-2000i hematology analyzer (both from Sysmex Corporation, Kobe, Japan), within 3 h after collection. The results of all 1977 samples provided by the hematology analyzers and MBSR were recorded in a spreadsheet. Of the 1615 patients, 1412 performed a single complete blood count (CBC) during the sample collection period while 203 underwent more than one CBC (117 patients underwent two, 30 patients underwent three; 40 patients underwent four; 15 patients underwent five and one patient underwent six). Both internal and external quality control procedures were followed to monitor performance of the hematology analyzers as well as reliability of the results. The adjustments and settings on the analyzers were performed by the manufacturer's scientific and technical support staff.

Manual blood smear review

For each sample, a blood smear was prepared and stained using the Sysmex SP-1000i automatic slide maker-stainer (Sysmex Corporation, Kobe, Japan). Samples that contained a low volume of whole blood were prepared manually by the wedge-spread film technique, using the May-Grünwald & Giemsa stains. MBSR and MDLC were performed in all samples in accordance with the recommendations of Barnes et al. regarding the step by step validation of criteria for MBSRs. In most cases, 100 leukocytes were counted in each smear by one of six independent observers with extensive experience (10–30 years) in MBSR. A count of 100 or 200 cells per sample on a single slide for only one of each observer was considered a suitable reference method to compare the findings.16, 21 All observers followed the same guidelines on classification and grading the morphological changes as well as description of specific cell types observed by microscopy such as, atypical lymphocytes, band neutrophils and immature granulocytes.

Study design

First, new PSF were locally designed and established (Table 1) to meet the consensus of the hematology laboratory at HC-UFPR in terms of morphological changes relevant to the diagnosis and follow-up of patients. Consequently, to identify samples with these PSF, four sets of SC as shown in Table 2 were evaluated: Group 1 (narrow cut-off limits), Group 2 (intermediary cut-off limits), Group 3 (wide cut-off limits) and cut-off limits adapted from the ISLH (Group 4). In the first three sets of SC established in this study, the selection of limits for each hematological parameter was performed arbitrarily and empirically. The SC adapted from the ISLH by Comar et al. were also evaluated in order to assess their performance in identifying samples with new PSF. The performance indicators for the sets of SC were: (i) rate of microscopic reviews lower than 50%. This was based on the study by Novis et al. on local limitations of the hospital laboratory information system with respect to the implementation of delta check rules that recognize discrepancies among the results of the current analysis and previous results of a pre-established period of time, also the amount of work and the number of professionals available to perform the smear reviews; (ii) although Barnes et al. recommend <5% false negative results as the safety threshold for patients, a rate of ≤3% was defined, as recommended by Cui et al. This study was not performed blinded, as the observers had access to the values of blood parameters and description of suspect flags in the equipment reports. Nevertheless, the observers had no prior knowledge of the cut-off limits in each set of SC and the new PSF proposed in this study.

Table 1

Positive smear findings in microscopic analysis.

Red blood cell series
Anisocytosis ≥2+	Microcytes ≥2+	Macrocytes ≥2+
Poikilocytosis ≥2+	Elliptocytes ≥2+	Stomatocytes ≥2+
Codocytes ≥2+	Dacryocytes ≥1+	Schistocytes ≥1+
Acanthocytes ≥2+	Drepanocytes: present	Spherocytes ≥1+
Howell–Jolly: present	Cabot ring: present	Basophilic stippling ≥1+
Rouleaux formation ≥2+	Hypochromia ≥2+	Polychromatophilia ≥2+
RBC agglutination: present	Echinocytes ≥2+	Any other RBC morphology ≥2+
	Hemoglobin C crystals: present	Hematozoa: present
Sum of the following alterations: (anisocytosis ≥1+) + (RBC shapes ≥1+) + (RBC color alteration ≥1+)
White blood cell series
Döhle bodies ≥1+	Toxic granulation ≥2+	Cytoplasmic vacuoles ≥1+
Polylobocytes ≥1+	Hyposegmented neutrophils: present	Neutrophil hypo/degranulation: present
Auer rod: present	Pseudo-Pelger-Huët: present	Dysplastic cells: present
Platelets
Giant platelets ≥2+	Microplatelets ≥2+	Platelet aggregates: ≥rare/occasional
Platelet anisocytosis ≥2+	Degranulated platelets: present	Gray platelets: present
Abnormal cell types
Blasts ≥1%	Promyelocytes ≥1%	Myelocytes ≥1%
Metamyelocytes ≥3%	Band neutrophils ≥15%	NRBC ≥1/100 leukocytes
Plasmocytes ≥1%	Prolymphocytes ≥1%	Atypical lymphocytes ≥5%
Other immature cells ≥1%

NRBC: nucleated red blood cells.

Note: Positive smear findings represent relevant clinical information in addition to that generated by hematology analyzers i.e. a minimum threshold of information, which must be described in complete blood counts as per local consensus indicating the morphological changes and cell percentages relevant for the diagnosis and monitoring of patients.

Table 2

Sets of screening criteria.

Parameter	Group 1Narrow limits	Group 2Intermediary limits	Group 3Wide limits	Group 4Adapted ISLH screening criteria16
Quantitative screening criteria
Hemoglobin	<9.5 g/dL or >18.0 g/dL	<8.0 g/dL or >19.0 g/dL	<7.5 g/dL or >20.0 g/dL	<7.0 g/dL or >18.5 g/dL
MCV	<77 fL (adults) and <74 fL (children aged ≤12 years) or >100 fL	<75 fL or >105 fL	<74 fL or >105 fL	<75 fL or >105 fL
MCHC	<31.5 g/dL	<31.0 g/dL	<30.0 g/dL	<30 g/dL
MCHC	>36.5 g/dL	>36.5 g/dL	>36.5 g/dL	>36.5 g/dL
RDW-CV	>17.0%	>18.0%	>19.0%	>22.0%
RDW-SD	>60 fL	>65 fL	>70 fL	–
Reticulocytes	>0.150 × 106/μL or >2.5%	>0.150 × 106/μL or >2.5%	>0.150 × 106/μL or >2.5%	>0.100 × 106/μL
WBC	<4.0 × 103/μL or >12.0 × 103/μL	<3.5 × 103/μL or >15.0 × 103/μL	<3.2 × 103/μL or >20.0 × 103/μL	<4.0 × 103/μL or >30.0 × 103/μL
Neutrophils #	<1.8 × 103/μL or >7.0 × 103/μL	<1.5 × 103/μL or >11.0 × 103/μL	<1.0 × 103/μL or >15.0 × 103/μL	<1.0 × 103/μL or >20.0 × 103/μL
Neutrophils %	<38% or >70%	<35% or >80%	<25% or >85%	–
Lymphocytes #	<0.8 × 103/μL or >5.0 × 103/μL (adults) and <0.8 × 103/μL or >7.0 × 103/μL (children aged ≤12 years)	<0.8 × 103/μL or >5.0 × 103/μL (adults) and <0.8 × 103/μL or >7.0 × 103/μL (children aged ≤12 years)	<0.5 × 103/μL or >5.0 × 103/μL (adults) and <0.5 × 103/μL or >7.0 × 103/μL (children aged ≤12 years)	>5.0 × 103/μL (adults) and >7.0 × 103/μL (children aged ≤12 years)
Lymphocytes %	<20% or >50%	<12% or >60%	<8% or >70%	–
Monocytes #	<0.1 × 103/μL or >1.3 × 103/μL (adults) and <0.1 × 103/μL or >2.0 × 103/μL (children aged ≤12 years)	<0.1 × 103/μL or >1.3 × 103/μL (adults) and <0.1 × 103/μL or >2.0 × 103/μL (children aged ≤12 years)	<0.1 × 103/μL or >1.5 × 103/μL (adults) and <0.1 × 103/μL>2.0 × 103/μL (children aged ≤12 years)	>1.5 × 103/μL (adults) and >3.0 × 103/μL (children aged ≤12 years)
Monocytes %	<1% or >12%	<1% or >16%	<1% or >20%	–
Eosinophils #	>2.0 × 103/μL	>2.5 × 103/μL	>2.5 × 103/μL	>2.0 × 103/μL
Eosinophils %	>20%	>25%	>30%	–
Basophils #	>0.3 × 103/μL	>0.3 × 103/μL	>0.5 × 103/μL	>0.5 × 103/μL
Basophils %	>2%	>2%	≥3%	–
Platelets	<100 × 103/μL or >600 × 103/μL	<100 × 103/μL or >800 × 103/μL	<90 × 103/μL or >1000 × 103/μL	<100 × 103/μL or >1000 × 103/μL
MPV	>12.0 fL	>12.5 fL	≥13.0 fL	≥12.5 fL
Qualitative screening criteria
Suspect flags	Immature granulocytes?
	Left shift?
	Atypical lymphocytes?
	Abnormal lymphocytes/blasts?
	Blasts?
	Nucleated red blood cells?
	Fragments? (schistocytes)
	Dimorphic RBC population (erythrocyte population heterogeneous in size)
	Lyse resistant RBC
	Platelet clumps? (generated by scatter gram)Platelet clumps (S)? (generated by the impedance channel)
	Turbidity/hemoglobin interference? Hemoglobin defect? Spurious hemoglobin measurement
ALL	MBSR if newborn
(*) and (----)	These symbols beside the counts on the readout indicate that automated counts are not reliable or not available for the sample in question, respectively

ISLH: International Society for Laboratory Hematology; MCV: mean corpuscular volume; MCHC: mean corpuscular hemoglobin concentration; RDW-CV: red blood cell distribution width coefficient of variation; RDW-SD: red blood cell distribution width standard deviation; RBC: red blood cells; WBC: white blood cells; MPV: mean platelet volume; MBSR: manual blood smear review.

Note: The terms narrow, intermediary and wide limits refer to their relative and absolute overall amplitudes in relation to the lower and upper limits of the complete blood count reference values.

Sample classification

The samples were classified by applying the rules of each set of SC and comparing them with the findings of the peripheral blood smear review. A sample was classified as true positive (TP) if it was selected to review by certain SC (Table 2) and the microscopic analysis produced some PSF (Table 1). A sample was classified as false positive (FP) if it was selected to review by SC with no PSF in microscopy. A sample was classified as false negative (FN) if it was not selected to review by any SC and contained some PSF by microscopic analysis. Finally, a sample was classified as true negative (TN) if it was not selected to review by any SC and the MBSR did not show any PSF.9, 16

Measures of diagnostic accuracy and agreement

Sensitivity, specificity, negative predictive value, positive predictive value, efficiency and confidence intervals for each set of SC were obtained with the Cálculos Estatísticos software for Windows v.1.8. These were calculated using the following equations:

Efficiency was defined as the capability of a set of SC to discriminate correctly the samples i.e., the true-positive results and true-negative results. The rate of microscopic reviews was computed as (%) = ([TP + FP]/[TP + FP + FN + TN]). Comparisons of receiver operating characteristic (ROC) curves were carried out to verify variations in the sensitivity and false positive fraction (1 − specificity) of different sets of SC using overall cut-off values. The Youden index (J) was calculated using the formula: Y = sensitivity + (specificity − 1). The Youden's J statistic, a function of both sensitivity and specificity, has values ranging from −1 to 1, and has a zero value when a diagnostic test gives the same proportion of positive results for groups with and without the disease, i.e. the test is useless. A value of 1 indicates that there are no false-positive results or false-negative results, i.e. the test is perfect. The index gives equal weight to false positive and false negative values, so all tests with the same value of the index have the same proportion of total misclassified results.26, 27

Statistical analysis

All statistical analyses were performed using the Statistica 8.0 (StatSoft Inc., Tulsa, OK, USA) and MEDCALC® version 7.3.0.1 (MedCalc Software, Mariakerke, Belgium) computer programs. The chi-square test (χ2) was performed to verify differences in each measure of diagnostic accuracy and agreement evaluated among all sets of SC. Additionally, Fisher's exact test, performed using 2 × 2 tables, was used to compare the sensitivity, specificity, efficiency, rate of microscopic reviews as well as the rates of false-negative results, false-positive results, true-negative results and true-positive results among pairs of sets of SC. A p-value of ≤0.05 was considered statistically significant.

Results

The analyzed samples displayed high amplitudes in the intervals of all hematologic parameters used as quantitative SC, reflecting heterogeneity of the sample at the hematology laboratory of HC-UFPR. Of the 1977 samples analyzed, 2.5% had hemoglobin levels between 2.5 g/dL and 7.1 g/dL and another 2.5% had hemoglobin levels between 16.3 g/dL and 20.0 g/dL. Regarding leukocyte counts, 2.5% of samples had counts of 0.01 × 103/μL to 1.44 × 103/μL and another 2.5% had counts between 22.4 × 103/μL and 107.8 × 103/μL. For the platelet counts, 2.5% of all samples had values between 1 × 103/μL and 22.0 × 103/μL and another 2.5% had counts of 608 × 103/μL to 2110 × 103/μL.

A total of 491 samples (24.84%) were considered positive for the PSF (Table 1) and 1486 (75.16%) were considered negative. Among the positive samples, morphological alterations were observed in 195 (39.71%) in the red blood cell series, 217 (44.19%) in the white blood cell series, seven (1.42%) in platelets, four (0.81%) in red blood cell series and platelets, 64 (13.03%) in the red and white blood cell series, two (0.41%) in white blood cell series and platelets and two (0.41%) had changes in all the three series. The automated count of reticulocytes was performed in 191 samples (only upon medical request).

The three most common, abnormal red blood cell series findings were anisocytosis ≥2+ (123 occurrences), the sum of changes [(anisocytosis ≥1+) + (poikilocytosis ≥1+) + (erythrocyte form ≥1+) + (change of color ≥1+)] (102 occurrences) and dacryocytes ≥1+ (66 occurrences). For the white blood cell series, the three most common findings were band neutrophils ≥15% (188 occurrences), myelocytes ≥1% (115 occurrences), and Döhle bodies (74 occurrences). It is noteworthy that blasts were found in 16 samples. Platelet aggregates (ten occurrences) were more frequent than giant platelets (four occurrences).

The quantitative SC which showed the highest rates of indication for MBSR in Groups 1, 2, 3 and 4 were lymphocytes <20% or >50% (38.89%), neutrophils <35% or >80% (19.92%), leukocytes <3.2 × 103/μL or >20.0 × 103/μL (11.68%) and leukocytes <4.0 × 103/μL or >30.0 × 103/μL (13.7%), respectively.

In Group 3, lower limits of screening for leukocytes (<3.2 × 103/μL) and platelets (<90.0 × 103/μL) were used. Unlike the SC of Group 4, the relative counts of leukocytes and the red blood cell distribution width standard deviation (RDW-SD) were used as SC in Groups 1–3. The cut-off points for platelets ranged from >600 × 103/μL (Group 1) to >1000 × 103/μL (Group 3 and Group 4). The evaluation results of SC among the four sets using the new PSF of HC-UFPR are shown in Table 3.

Table 3

Truth table of the sets of screening criteria.

Group 1 (narrow cut-off limits)	Total (n = 1977)
	%
True positive	24.33
False positive	49.52
True negative	25.65
False negative	0.50

95% CI: two-sided 95 percent confidence interval; ISLH: International Society for Laboratory Hematology.

First, the chi-square test was performed to verify differences in each measure of diagnostic accuracy and agreement evaluated among all the sets of SC, and all performance characteristics showed p-value ≤ 0.001. Consequently, the Fisher's exact test was performed in 2 × 2 tables to compare all the performance characteristics among the different sets of SC. The results of the comparison between the two hematology analyzers employed in this work were similar and for this reason, they are not shown separately. The efficiencies of SC of Groups 3 and 4 were 73.04% and 70.56%, respectively, with no significant differences between them (p-value = 0.09); however, they were statistically more efficient than the SC of Groups 1 and 2 (p-value < 0.001). The rates of false-negative results of Groups 3 and 4 were 2.73% and 3.95%, respectively (p-value = 0.0414), with the latter exceeding the safety limit of 3% of false-negative results established in this study. The rate of reviews in Group 3 (46.33%) was the lowest among the four sets of SC, including Group 4 (46.38%), although these two values are not statistically significant. Group 1 showed a sensitivity of 97.96% and the highest negative predictive value (98.06%) both of which were statistically higher than the values observed in the other sets of SC (p-value < 0.001). Group 3 presented the highest specificity (67.77%), classifying the true-negative samples more accurately. It also showed the highest positive predictive value (46.99%) by classifying the true-positive samples more accurately.

In Groups 1, 2, 3 and 4, the rates of false-negative results in inpatient samples (0.30%, 1.97%, 3.19% and 5.63%, respectively) were not different to the rates of false-negative results in all samples (0.50% – p-value = 0.7413; 1.97% – p-value = 1.0000; 2.73% – p-value = 0.5885 and 3.95% – p-value = 0.1011, respectively). Likewise, the rates of false-negative results in outpatient samples (0.60%; 1.96%; 2.50% and 3.18%, respectively) were not different from the rates of false-negative results in all samples (0.50% – p-value = 0.8104; 1.97% – p-value = 1.0000; 2.73% – p-value = 0.7402 and 3.95% – p-value = 0.2575, respectively). However, the rate of false-negative results in Group 4 (5.63%) was the only one among all the sets of SC that showed a significant increase in inpatients compared to outpatients (3.18% – p-value = 0.0150).

The rates of false-positive inpatient samples in Group 2 and Group 3 (38.05%; 30.13%, respectively) were higher than the total rates of false-positive results (31.66% – p-value = 0.0361 and 24.23% – p-value = 0.0308, respectively). With the exception of Group 1, the rates of false-positive results in inpatients (38.05%, 30.13% and 29.68%, for Groups 2, 3 and 4, respectively) were higher than those observed in the outpatient samples (28.33% – p-value = 0.0020; 21.66% – p-value = 0.0017 and 23.78% – p-value = 0.0320, respectively).

Importantly, the rates of microscopic reviews in inpatient samples of Groups 1, 2, 3 and 4 (90.25%, 73.97%, 64.84% and 61.94%, respectively) were significantly higher than those found in outpatient samples (63.86% – p-value < 0.0001; 44.46% – p-value < 0.0001; 37.27% – p-value < 0.0001 and 38.78% – p-value < 0.0001, respectively).

The rates of microscopic reviews decreased in Group 2 (54.52%) and Group 3 (46.33), and the rate of false-negative results was ≤3% in both. With the exception of Group 4, all the other groups achieved rates of false-negative results <3% and statistically different from the rate in the SC of Group 4 (p-value < 0.05). However, only the SC of Group 1 were capable of screening a sample with 2% plasma cells in a patient with multiple myeloma (confirmed in this patient's medical records).

Figure 1 shows the comparison of the ROC curves of Groups 1–4. Group 3 had the greatest area under the curve (0.784), while Group 1 had the smallest (0.659). Group 3 had the highest Youden index (0.5677), while Group 1 had the lowest (0.3207).

Figure 1

Comparison of receiver operating characteristic (ROC) curves of the screening criteria sets of Groups 1–4.

Note 1: 100 − specificity = false positive rate.

Note 2: Four ROC curves with different values of the area under de ROC curve are illustrated. A perfect test has an area under de ROC curve of 1. The chance level diagonal (black dashed line segment from 0 (x), 0 (y) to 100 (x), 100 (y)) has an area under de ROC curve of 0.5. The ROC curves of Groups 1–4 demonstrated different abilities to distinguish between samples with and samples without positive smear findings (colored dashed and solid lines) and lie between these two extremes. Group 3 with the greatest area under the ROC curve has the best overall diagnostic performance.

Note 3: Group 3 had the highest Youden index and gave the least proportion of misclassified results, whereas Group 1 had the lowest Youden index and gave the highest proportion of misclassified results.

Note 4: The area under the ROC curve for Groups 2 and 4 are almost identical. In the high false positive rate range (or high sensitivity range) Group 2 is better than Group 4, whereas in the low false positive rate range (or low sensitivity range) Group 4 is better than Group 2.

Note 5: Prevalence of positive samples: 24.84% (491 samples in 1977).

In the analysis of positive cases (true-positive results and false-positive results) (Table 4), the qualitative SC related to suspect flags displayed a good positive predictive value. The RDW-CV and the RDW-SD showed increased positive predictive values as the cut-off points were raised. The leukocyte count cut-off point of the SC in Group 1 showed the lowest positive predictive value while Group 2 showed the highest positive predictive value. The set of SC in Group 3 best identified the true-positive samples, with a positive predictive value of 47.7%.

Table 4

Analysis of positive cases observed in the screening criteria of Groups 1–4.

Parameter (values below and/or above the cut-off points of the screening criteria)	Group 1			Group 2			Group 3			Group 4
	TP (n)	FP (n)	PPV (%)	TP (n)	FP (n)	PPV (%)	TP (n)	FP (n)	PPV (%)	TP (n)	FP (n)	PPV (%)
Leukocytes (upper and lower limit)	228	345	38.8	153	199	43.46	98	133	42.42	96	174	35.5
Platelets (upper and lower limit)	130	164	44.21	113	137	45.2	111	134	45.3	111	134	45.3
Hemoglobin (upper and lower limit)	160	140	53.3	48	50	48.97	34	31	52.30	22	25	46.8
MCV (upper and lower limits, adults and children)	133	101	56.8	98	39	71.5	93	34	73.2	98	39	71.5
RDW-CV (upper limit)	188	115	62.0	148	53	73.6	120	24	83.3	47	2	95.9
RDW-SD (upper limit)	112	53	67.9	73	22	76.8	41	3	93.1	Not applicablea
MCHC (upper and lower limit)	108	82	56.8	75	43	63.5	41	17	70.7	41	17	70.7
Absolute neutrophil counts (upper and lower limit)	240	431	35.76	128	188	40.5	71	84	45.8	49	71	40.83
Absolute neutrophil counts + relative neutrophil count (upper limit)	132	227	36.8	58	50	53.7	23	12	65.7	Not applicablea
Absolute neutrophil counts + relative neutrophil count (lower limit)	35	96	26.7	31	71	30.4	20	42	32.3	Not applicablea
MPV (upper limit)	35	93	27.3	14	35	28.6	8	9	47.0	18	44	29
Suspect flags (all)	210	234	56.6	310	234	57.0	310	234	57.0	310	234	57.0
Total			32.95			41.93			47.7			45.03

TP: true positive; FP: false positive; PPV: positive predictive value; MCV: mean corpuscular volume; RDW-CV: red cell distribution width – coefficient of variation; RDW-SD: red blood cell distribution width standard deviation; MPV: mean platelet volume; MCHC: mean corpuscular hemoglobin concentration.

RDW-SD and relative neutrophil counts are not included in the screening criteria of Group 4.

Table 5 provides a breakdown of the false-negative results found in Groups 1–4. The morphological changes that led to the highest number of false-negative results among all the evaluated SC were band neutrophils ≥15%, and myelocytes ≥1%. In Group 3, the negative predictive value found for band neutrophils of ≥15% demonstrated that in 97.4% of the cases that did not indicate the presence of this morphological alteration; the samples actually contained no band neutrophils ≥15%.

Table 5

False-negative results and negative predictive values observed in Groups 1–4.

Criteria for positivity in smear	Group 1	NPV (%)	Group 2	NPV (%)	Group 3	NPV (%)	Group 4	NPV (%)
Band neutrophils ≥15%	6	98.8	18	97.9	27	97.4	35	96.6
(Anisocytosis ≥1+)+(poikilocytosis ≥1+) + (erythrocyte form ≥1+) + (change of color ≥1+)	–	100	5	99.4	6	99.4	8	99.4
Erythroblasts ≥1/100 leukocytes	–	100	1	99.9	1	99.9	1	99.9
Anisocytosis ≥2+	–	100	–	100	–	100	9	99.1
Poikilocytosis ≥2+	–	100	1	99.9	1	99.9	1	99.9
Polychromatophilia ≥2+	–	100	1	99.9	2	99.8	2	99.8
Schistocytes ≥1+	–	100	1	99.9	1	99.9	2	99.8
Dacryocytes ≥1+	2	99.6	7	99.2	9	99.1	10	99.0
Codocytes ≥2+	–	100	1	99.9	1	99.9	1	99.9
Basophilic stippling ≥1+	1	99.8	2	99.7	2	99.8	2	99.8
Howell–Jolly presence	–	100	1	99.9	1	99.9	2	99.8
Rouleaux formation ≥2+	1	99.8	1	99.9	1	99.9	1	99.9
Myelocytes ≥1%	2	99.6	9	98.8	10	99.0	13	98.7
Döhle bodies ≥1+	–	100	–	100	3	99.7	3	99.7
Polylobocytes ≥1+	–	100	–	100	–	100	1	99.9
Plasma cells ≥1%	–	100	1	99.9	1	99.9	1	99.9
Total (n)	12a	98.06	49a	95.66	66a	94.91	92a	92.64
% of false-negative samples	0.50		1.97		2.73		3.95

NPV: negative predictive value.

Some samples showed more than one positive smear finding.

In Group 3, of the 54 samples with false-negative results, there were 27 cases of left shift ≥15% of band neutrophils, ten with myelocytes ≥1%, 25 with changes in red blood cell series, three cases with the presence of Döhle bodies and one case with plasma cells ≥1%. There were 54 samples with erythroblasts ≥1/100 leukocytes within the 1977 samples analyzed, and only one sample was not flagged by the SC of Groups 2–4. There were no false-negative results for morphological changes in platelets. No cases of blasts were lost by the four SC.

Table 6 shows the top ten single SC that generated false-positive results in Groups 1–4.

Table 6

Top ten single screening criteria which generated false-positive results in Groups 1–4.

Criteria	Rate of false positives – n (%)
Group 1
Lymphocytes <20%	381 (19.27)
Neutrophils >70%	368 (18.61)
Neutrophils >7.0 × 103/μL	280 (14.16)
Unreliable differential leukocytes countsa	227 (11.48)
Leukocytes >12.0 × 103/μL	176 (8.9)
Leucocytes <4.0 × 103/μL	168 (8.49)
Neutrophils <38%	141 (7.13)
Platelets <100 × 103/μL	135 (6.82)
Lymphocytes >50%	126 (6.37)
RDW-CV >17%	117 (5.91)
Group 2
Unreliable differential leukocytes countsa	227 (11.48)
Platelets <100 × 103/μL	134 (6.77)
Leucocytes <3.5 × 103/μL	126 (6.37)
Lymphocytes <12%	121 (6.12)
Neutrophils >80%	120 (6.06)
Neutrophils <35%	115 (5.81)
Neutrophils <1.5 × 103/μL	113 (5.71)
Unreliable platelet countsa	95 (4.8)
Unreliable leukocyte countsa	94 (4.75)
Suspect flags of left shift	91 (4.6)
Group 3
Unreliable differential leukocytes countsa	227 (11.48)
Platelets <90 × 103/μL	117 (5.91)
Leucocytes <3.2 × 103/μL	110 (5.56)
Unreliable platelet countsa	95 (4.8)
Unreliable leukocyte countsa	94 (4.75)
Suspect flags of left shift	91 (4.6)
Neutrophils <25%	63 (3.18)
Suspect flags of platelet aggregates	63 (3.18)
Neutrophils >85%	57 (2.88)
Suspect flags of immature granulocytes	48 (2.42)
Group 4
Unreliable differential leukocytes countsa	227 (11.48)
Leucocytes <4.0 × 103/μL	169 (8.54)
Platelets <100 × 103/μL	131 (6.62)
Unreliable platelet countsa	95 (4.8)
Unreliable leukocyte countsa	94 (4.75)
Suspect flags of left shift	91 (4.6)
Neutrophils <1.0 × 103/μL	66 (3.33)
Suspect flags of platelet aggregates	63 (3.18)
Suspect flags of immature granulocytes	48 (2.42)
Unreliable MPVa	48 (2.42)

RDW-CV: red cell distribution width – coefficient of variation; MPV: mean platelet volume.

Screening criterion that caused most false-positive results.

The screening criterion that caused most false-positive results in Groups 2–4 was unreliable differential leukocyte counts (227 occurrences). The second major cause of false-positive results in Groups 2, 3 and 4 was platelet counts <100 × 103/μL, platelet counts <90 × 103/μL and leukocyte counts <4.0 × 103/μL, respectively.

Discussion

In studies that have proposed SC for MBSR, challenges exist in respect to standardization of the SC and PSF, since both are not completely applicable in all laboratories. Thus, each hematology laboratory should develop its own better-adapted SC for MBSR to capture samples with morphological findings of clinical significance (PSF), while accounting for the laboratory's peculiarities, as well as the knowledge and experience of the medical team.1, 2, 15, 16, 19, 28, 29, 30 As recommended by Froom et al., SC specifically for children aged under 15 months could also be established, as these authors observed a high rate of false-positive results due to samples with absolute lymphocyte counts above 7.0 × 103/μL. As demonstrated by Tseng et al., studies could be conducted through ROC curve analysis to define the best SC cut-off points for identifying the PSF. The SC for MBSR should be verified and improved continuously and, for this purpose, samples should be regularly and seasonally analyzed as a quality control procedure to determine the need for adjustments of the SC for MBSR used at that time; this would take into account changes in the patient's characteristics due to seasonal diseases. Galloway and Osgerby showed that variations can arise with the use of SC for MBSR, which are dependent on the type of hospital, the hematology analyzer used and geographical location of the laboratory. A study of the Q-Probe program, reported that the larger the hospital, the greater the percentage of MBSR will be. The PSF should encompass clinically significant morphological alterations, thus establishing a minimum threshold of information which must be described in CBCs as per local consensus. The PSF of HC-UFPR have their relevant threshold values either equal to or less than the PSF of ISLH.

Two basic principles should be followed when developing SC for MBSR: (1) reduction in the rate of false-negative results to an acceptable value. Groups 1–3 obtained false-negative rates below the limit of 3% specified in this study. Group 3 displayed a marginally higher efficiency compared to Group 4 and both were more efficient than Groups 1 and 2. However, the rate of false-negative results in Group 3 was significantly lower than that in Group 4 (p-value = 0.0414). Moreover, Kim et al. found a false-negative rate and specificity of 9.7% and 77.1%, respectively, using the SC of ISLH to capture samples with the PSF of ISLH.

In Groups 1–3, the rates of false-negative results in inpatients were similar to those observed in outpatients, suggesting that these studies can be used for both profiles, while those observed in Group 4 were significantly higher and thus, constitute a constraint on the safety of hospitalized patients. However, Pratumvinit et al. evaluated their own optimized SC to capture PSF of the ISLH and did not observe this difference in patient profile.

It is important that no false-negative result reflects the presence of previously undetected severe hematologic diseases, although false-negative rates of 3% to 5% are acceptable. To further expand on detecting the undiagnosed hematologic diseases, each institution should perform MBSR in patients within certain hospitalization units, such as the hematology unit, even at the cost of an increase in the rate of microscopic reviews.

The analysis of false-negative results in all groups also revealed a high incidence of neutrophil left shift. However, the interpretation of these results should consider the limitations of the MDLC: (i) observer misclassification and the high variation of leukocyte cell types between observers, (ii) slide cell distribution errors caused by the nonrandom distribution of leukocyte types on blood film prepared by the wedge-type technique, (iii) poor-quality blood film and inconsistent staining, (iv) poor statistical sampling validity due to variability in 100-cell proportional differential counts of given type of leukocyte, i.e., the fewer cells that are counted, the less precise the blood cell percentage is and (v) recording errors. For instance, unlike this study, Hyun et al. consider very high counts as clinically significant such as, promyelocytes ≥3%, neutrophils and band neutrophils ≥20%, atypical lymphocytes ≥7% and erythroblasts ≥2 or 3/100 leukocytes. In contrast, Jones et al. consider immature granulocytes ≥3%, atypical lymphocytes ≥10% and erythroblasts ≥2/100 leukocytes as significant, which shows the diversity of PSF in each laboratory.

Another important factor is to find SC that classify less samples as false positive in agreement with the second principle, which is to reduce the rate of microscopic reviews to an acceptable value. In all Groups, the total rates of microscopic reviews were directly influenced by limits established for the SC. Group 1 had the highest total rate of microscopic reviews and the lowest rate of false-negative results compared to other SC (0.5%, p-value < 0.001). Although a high MBSR rate is safe for the patients, it can be a problem in respect to the delay of results for laboratories that do not have a sufficient number of experienced professionals to perform MBSR. Group 1 is a good example of how the lack of SC optimization may result in an unnecessary labor-intensive workload due to the high rate of microscopic reviews.

The pre-set limit of <50% on the rate of microscopic reviews was observed in Group 3 and Group 4; however, the lowest rate of microscopic reviews as well as increased efficiency and safety observed in Group 3 made this set of SC more acceptable. Previous studies by Cui et al. and Pratumvinit et al. obtained lower rates of microscopic reviews than those found in Group 3. However, we chose not to make additional adjustments to the SC in Group 3 to further reduce the rates of microscopic reviews, since this could increase the rate of false-negative results above the safety limit. Therefore, we conclude that the SC of Group 3 constitute the limit of automation for the direct release of CBC results in an efficient and safe manner at the Hospital de Clínicas of UFPR.

Through the analysis of false-positive results, one can alter or remove some SC to reduce the rate of microscopic reviews, as high rates overload the professionals causing delays in the release of results. The rate of microscopic reviews of 46.33% obtained in Group 3 is within the capacity of the hematology laboratory at HC-UFPR.16, 17, 24, 34

Parameters such as MCV, RDW-CV, RDW-SD, MCHC, differential counts of neutrophils and suspect flags displayed the highest positive predictive values. Thus, appropriate adjustments to the cut-off limits of these parameters can increase the ability to detect abnormal samples. On the other hand, low leukocyte and platelet counts, as well as the MPV captured samples without morphological abnormalities producing the lowest positive predictive values. Notably, these results reflect the profiles of patients analyzed routinely in the hematology laboratory (quaternary hospital). Kim et al. demonstrated that SC with the best positive predictive values in a tertiary hospital in Korea were: hemoglobin, MCV, RDW and suspect flags. Likewise, Pratumvinit et al. demonstrated that the rate of false-positive results was higher in hospitalized patients than in outpatients.

In this study, for the first time, a comparison of ROC curves was used as an assessment tool for the evaluation of sets of SC for MBSR. Figure 1 shows the behavior of sets of SC with respect to pair sensitivity and fraction of false-positive results (100 – specificity). Group 3, relative to the others, did not display a higher sensitivity, but accurately classified the samples with an efficiency of 73.04% and a Youden index of 0.5677. The Youden index is appropriate for choosing the test that presents the least misclassified results (false-positive results + false-negative results); however, on certain occasions, it may be important to choose the test with fewer false-negative results, i.e., the most sensitive. A high sensitivity occurs due to the high number of microscopic reviews (Group 1), which may not be interesting, since it reduces the amount of correctly classified samples (efficiency). Importantly, the high efficiency of Group 3 was obtained at the expense of a 2.73% false-negative rate, which is still lower than the values suggested by Barnes et al. (<5%) and by Cui et al. (<3%); thus, conveying more reliability of the CBC results released directly after the application of this set of SC.

SC for MBSR with a high sensitivity may be beneficial in laboratories that mostly attend outpatients for screening purposes, while SC for MBSR with high specificity may be more useful in laboratories that attend inpatients, in order to provide faster and more efficient care. Group 3 displayed the highest specificity (67.77%), which can significantly reduce the delay of results for the patients treated at HC-UFPR, as it allows a better classification of true-negative samples (50.94%).

Importantly, for the performance evaluation of SC for MBSR, one should not only seek higher sensitivity specificity, and efficiency, but also maintain a balance between not overloading with unnecessary MBSR and failure to review microscopic samples containing clinically relevant morphological changes. Therefore, the rate of false-negative results must be maintained within the limits recommended by the laboratory.

In an attempt to reduce the microscopic reviews found in this study, some actions can be adopted for further studies such as, changing the PSF of band neutrophils from ≥15% to ≥20% or even removing it from the PSF, as the International Council for Standardization in Hematology (ICSH) recommends that band neutrophils should be segmented in MDLC due to wide inter-observer variations in their classification. This would allow elevation of the leukocyte and neutrophil values in SC. One could also deploy improvements in the laboratory information systems to enable deployment of delta check rules to increase the quality, safety and productivity of results.

The deployment of SC for MBSR is aimed to ensure that satisfactory performance in productivity brings benefits to patients, physicians and the laboratory itself by reducing costs and increasing the quality of exams. The application of Group 3 SC in our hematology laboratory enables approximately 120,000 CBC to be conducted annually, with approximately 64,400 to be released directly, allowing each professional to better utilize their time in the microscopic analysis of difficult cases.

The following limitations of the present study should be considered: (i) this study was not blinded; the observers had access to reports of hematology analyzers, which may have led to over-reports of certain morphological changes, especially those marked by suspect flags. (ii) The MBSR and the MDLC were performed by only one observer for each blood smear. On one hand, the review by two observers in a double-blind study could give more credibility to the results; however, it would negatively impact the practicality of implementation and periodic evaluations of SC for MBSR. Notably, Barnes et al. recommend that MBSR be performed by a single experienced observer in order to provide consistent results. In this study, the MBSR was performed by observers with extensive experience on morphological reviews in the hematology laboratory. (iii) The data presented were generated from a single population with a specific profile in a university hospital that uses a certain type of hematology analyzer, and at a certain time of the year (spring). Accounting for these local factors, SC for MBSR which were more efficient in the hematology laboratory of HC-UFPR may not be the most appropriate for other laboratories. (iv) The samples were collected consecutively for seven days, and thus, unusual changes that may constitute the PSF were not observed during this study. In order to assess these unusual PSF, further studies are needed with a longer sample collection period.

Automation in the hematology field has grown substantially in recent years, and more functions are available in the hematology analyzers such as, self-check of results, ease of maintenance and simplified interface protocols. These functions contribute to a reduction in the human interaction with analyzers to facilitate the workflow and decrease the delay of results. Hence, human efforts can be directed toward the verification of results of difficult MBSR cases, verification and analysis of the analytical performance of the hematology analyzers, and the evaluation and establishment of SC for MBSR through surveys and daily interactions between professionals in the lab, medical team and the hematology analyzers. All these factors aim to ensure appropriate, relevant, and high quality results released in a timely manner for decision-making and appropriate medical management.

Conclusions

The SC for MBSR of Groups 1–4 can be used in a hematology laboratory and the chosen set of SC depends on the need to increase efficiency, the safety of patients, as well as the availability of expertly trained personnel to perform the MBSR. Group 3 was the most appropriate for the hematology laboratory of UFPR, displaying the best relationship between safety (FN ≤3%) and efficacy for estimating the limits of the automation of CBCs. This study reinforces the importance of heterogeneity in the relevant population, such that the SC for MBSR meet the characteristics of different contexts in which hematological determinations are performed.

Conflicts of interest

The authors declare no conflicts of interest.