Impact of Interfacing Near Point of Care Clinical Chemistry and Hematology Analyzers at Urgent Care Clinics at an Academic Health System.

BACKGROUND: Point-of-care (POC) testing equipment is commonly utilized in outpatient clinics. Our institution recently interfaced POC chemistry and hematology devices at two outpatient clinics via middleware software to the central electronic health record (EHR), facilitating a comparison of manual transcription versus automatic reporting via interface. This allowed for estimation of serious/obvious error rates and manual time savings. Additional goals were to develop autoverification rules and analyze broad trends of results in response to common clinician complaints on the POC testing.
MATERIAL AND METHODS: Data were obtained from two satellite clinic sites providing both primary and urgent care within an academic health system. Interface of devices was accomplished via Instrument Manager middleware software and occurred approximately halfway through the 38 month retrospective timeframe. Laboratory results for three testing POC chemistry and hematology panels were extracted with EHR tools.
RESULTS: Nearly 100,000 lab values were analyzed and revealed that the rate of laboratory values outside reference range was essentially unchanged before and after interface of POC testing devices (2.0-2.1%). Serious/obvious errors, while rare overall, declined significantly, with none recorded after the interface with autoverified results and only three related to manual edits of results that failed autoverification. Fewer duplicated test results were identified after the interface, most notably with the hematology testing. Anion gap values of less than zero were observed more frequently in POC device tests when compared to central laboratory tests and are attributed to a higher proportion of Cl values greater than 110 mEq/L and CO2 values greater than 30 mEq/L with POC results. Time savings of eliminating manual data entry were calculated to be 21.6 employee hours per month.
CONCLUSIONS: In a switch from manual entry to automatic interface for POC chemistry and hematology, the most notable changes were reduction of serious/obvious errors and duplicate results. Significant time employee time savings highlight an additional benefit of instrument interfacing. Lastly, a difference between POC and central laboratory instruments is a higher rate of high Cl and CO2 values relative to the central laboratory.

Key Messages

Interface of POC testing reduces serious/obvious errors, lessens manual effort, and provides opportunities to standardize practices across clinics.

Introduction

Point-of-care (POC) testing equipment is commonly utilized in outpatient clinics, especially those remote from a central clinical laboratory.[,] POC equipment used in clinics can range from devices that provide one or a few results (e.g., glucometers) to more complex clinical chemistry and hematology analyzers that perform panels of testing. Result from POC instruments are often manually transcribed into the laboratory information system (LIS) or electronic health record (EHR), requiring manual effort and also an opportunity for transcription errors.[,] Relative to simpler POC devices such as glucometers, little investigation has been published regarding the impact of interfacing complex POC results to the EHR. Our institution interfaced chemistry and hematology POC devices at two outpatient sites with the central EHR, allowing for comparison of manual transcription versus interfaced reporting in terms of suspected error rates and workload impact.

The POC devices utilized in the present study are the Abaxis Piccolo Xpress and Sysmex pocH-100i Automated Hematology Analyzer. The Piccolo Xpress is a POC chemistry analyzer capable of running various panels of chemistry analytes including the 14-test comprehensive metabolic panel. The Piccolo’s portable size and lower device cost compared to larger chemistry analyzers make this an option for outpatient clinics, urgent care settings, and biosafety level (BSL) 2 and 3 areas (e.g., for patients with Ebola or other pathogens requiring containment procedures).[5, 6, 7, 8] The Piccolo generally performs similarly to larger chemistry analyzers in accuracy and precision, although biases have been noted for amylase, alkaline phosphatase, and total bilirubin.[,] The pocH-100i Analyzer is a compact POC device designed for low-volume complete blood count (CBC) panel testing. Similarly to the Piccolo, this device has a lower expense and smaller footprint when compared to traditional hematology instruments.[10, 11, 12, 13] The pocHi only performs a three-part white blood cell (WBC) differential. Immature cells such as blasts, myelocytes, or nucleated red blood cells may either not be differentiated or sometimes categorized as another cell type; presence of these type of cells may induce an error flag by the instrument.

While the accuracy of these two POC devices has been validated, there has been little investigation regarding the impact of interfacing these instruments into the EHR. There are multiple sources of error possible in reporting POC testing results including instrument error, operator error, and transcription error when recording results into the EHR or LIS. Previous work done by Mays and Mathias identified significant error rates associated with manual entry of POC glucometer data into the EHR, using a dataset where interfaced and manually entered results were inadvertently both entered into the system during a transition time from manual entry to electronic interface.[] Sowan et al identified and examined similar errors with manual recording of glucose values and concluded a resulting significant impact on administration of insulin.[] The results were consistent with previous work examining data entry errors, which described rates of 1–5%.[,] Both studies recommended interfacing of the glucometers with the EHR to reduce manual transcription errors. However, these studies were limited to a single POC device running a single test with a wide group of users. There is a significant gap in the literature examining the impact of interfacing POC instruments running panels of tests such as the Piccolo and pocHi.

This investigation sought to examine more complex POC devices and testing panels prior to and after the implementation of a POC device and EHR interface. We also estimate the manual work saved by interfacing of the instruments and describe autoverification rules developed in the process. Lastly, we examined patterns of electrolyte results on the Piccolo given that issues with occasional spurious sodium, chloride, and total CO2 results were the most common complaints from the two clinical sites running these analyzers.

Methods

Institutional Details

This retrospective study was conducted at an approximately 850-bed academic medical center with outpatient clinics throughout the local region. Data were obtained from two of these clinic sites, each providing urgent and primary care. The Urgent Care clinics are intended to treat non-life-threatening emergencies such as mild infections, cough, and sore throat. More critical issues such as chest pain and possible stroke are referred to the emergency department at the main medical center campus. The clinic sites employ both the Abaxis Piccolo Xpress (“Piccolo”) chemistry analyzer and Sysmex pocH-100i Automated Hematology Analyzer (“pocHi”). Both sites underwent interfacing of POC testing devices with the institutional EHR in June 2019 as described in detail below. Clinic testing oversight is done jointly by the clinics and the Pathology department. The Pathology department provides central quality assurance and laboratory directorship. Two employees travel between clinic sites throughout the health system and assist with training and competency. The data were collected as part of a retrospective study approved by the Institutional Review Board (protocol #202010420) covering the period from July 1, 2017 to October 22, 2020. This study was carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki).

Instrument Interface with EHR

The EHR for the institution is Epic Hyperspace (Epic, Inc., Madison, WI). The clinical laboratories use Epic Beaker Clinical Pathology as the LIS, with instrument interfaces to Beaker utilizing middleware software (Instrument Manager) from Data Innovations (Burlington, VT).[,] Our health system utilizes five separate Instrument Manager systems for different instrument groups: (1) Central medical center core clinical laboratory (mainly chemistry, coagulation, flow cytometry, and hematology); (2) Other central medical center laboratory areas such as Blood Bank and Microbiology; (3) Instrumentation at a large, offsite multispecialty outpatient/procedural building; (4) Non-pathology instruments used at the institution including those at the clinics analyzed in the present study; (5) “Community connect” hospitals that have contracted an arrangement to utilize the EHR of the health system. Separation of instrument manager systems increases complexity but allows for more targeted technical oversight and quality control as well as preventing an erroneous event from having an institution-wide impact. The first three categories described above have substantial informatics support from both Pathology and the central health system information technology (IT) known as Health Care Information Systems. The last two categories are mostly overseen by Health Care Information Systems with some collaboration from Department of Pathology informatics staff in helping to coordinate rules across the different middleware domains.

The Piccolo is interfaced with the institutional EHR through the use of Transmission Control Protocol/Internet Protocol (TCP/IP) to Data Innovations Instrument Manager (version 8.14.10), an instrument data aggregator. The aggregator then routes the information to the appropriate location in the EHR. Settings to permit this connection must be programmed on the instrument itself. The pocH-100i supports both a RS-232 serial and TCP/IP interface. The serial interface is used in combination with a Lantronix terminal server to convert the serial data stream going into and coming out of the instrument into TCP/IP format over ethernet back to Instrument Manager. This permits more extensive remote control of instrument connections, which is maintained centrally by Health Care Information Systems. Three of the IT specialists involved in this project have extensive experience with middleware software and associated hardware from prior roles within the Department of Pathology central medical center clinical laboratories.

Data Extraction and Analysis

Epic Reporting Workbench (RWB), an EHR data reporting tool,[] was used to retrieve laboratory results for three POC testing panels: basic metabolic panel (BMP; 8 tests), comprehensive metabolic panel (CMP; 14 tests), and complete blood count (CBC with three-part differential). The Piccolo can measure chemistry analytes in serum, whole blood in lithium heparin tube, or plasma using lithium heparin tube. Data were retrieved by individual component search in Epic RWB to allow for easier transcription of test components across software. BMP components included glucose, blood urea nitrogen, calcium, creatinine, sodium (Na), potassium (K), chloride (Cl), and CO2; anion gap (AG) was calculated in the laboratory information system by subtracting the sum of Cl and CO2 from Na. CMP components included alkaline phosphatase, alanine aminotransferase, aspartate aminotransferase, total bilirubin, albumin, and total protein in addition to the eight measured components of the BMP. AG can also be calculated from CMP results. CBC components included WBC count, red blood cell (RBC) count, hemoglobin concentration, hematocrit, mean corpuscular volume, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, platelet count, mean platelet volume, RBC distribution width, and standard deviation of RBC distribution width. WBC differentials were recorded but not analyzed in the dataset.

Data were extracted across all three testing panels. Unique laboratory values were defined as each component of a testing panel (e.g., a BMP for a single individual consists of eight unique measured values), as each component would either need to be manually entered into the LIS or EHR or transmitted by interface. Unique panel records represent a panel test (e.g., BMP, CMP, or CBC) for a patient at a unique specimen collection date and time. Data were recorded from the same patient if they had multiple panel tests at separate collection times in the timeframe of the study. Data for each panel were divided into three categories based on how test results were entered into the EHR: (1) Manual data entry before interface of POC devices with EHR; (2) Automatic entry after interface of POC devices with EHR; and (3) Manual entry or edits after interface of POC devices with EHR (most likely because result was blocked by autoverification rules or instrument error flag).

We examined multiple parameters before and after interfacing of results. An “abnormal value” is any value outside the normal reference range for a given component for a specific patient. Core laboratory references ranges for Na, Cl, and CO2 are 135–145 mEq/L, 95–107 mEq/L, and 18–29 mEq/L, respectively. The reference ranges for the same analytes on the Piccolo from package insert are 128–145 mEq/L, 98–108 mEq/L, and 18–33 mEq/L (Table 1). A test result was considered “out of valid checking range” if it fell outside of instrument specific ranges determined based on experience with central laboratory testing as well as validation experience and package insert information for the POC tests (Table 1). The valid checking ranges are outside autoverification limits and generally also outside critical value boundaries (if the analyte has critical values). The values outside of the valid checking range could be due to disease state but are also likely to be associated with an error. Serious/obvious errors are defined as values so far out of the reference range as to be physiologically unlikely or even implausible, such as a negative number input for an analyte concentration.

Table 1

Institutional laboratory reference ranges for point-of-care devices.

Piccolo ranges
Test	Units	Critical rangea	AV limitsa	Valid checkinga
Sodium	mEq/L	<120 or >160	125–150	115–160
Chloride	mEq/L	None	80–115	80–135
Potassium	mEq/L	<3.0 or >6.5	2.8–6.0	1.5–8.5
CO2	mEq/L	<10 or >50	15–40	5–40
Urea nitrogen	mg/dL	None	2–80	2–180
Creatinine	mg/dL	None	Up to 10.0	0.2–20.0
Glucose	mg/dL	<40 or >300	50–450	10–700
Calcium	mg/dL	<6.0 or >13.0	7.0–12.0	6.0–13.0
Total protein	g/dL	None	2.0–14.0	2.0–14.0
Albumin	g/dL	None	1.0–6.5	1.0–6.5
AST	U/L	None	5–2000	5–2000
ALP	U/L	None	5–2400	5–2400
Bilirubin, total	mg/dL	>10.0	0.1–30.0	0.1–30.0
ALT	U/L	None	5–2000	5–2000
pocH-100i rangesb
WBC	K/MM3	<=1.0 or >=50.0	1.0–50.0	1.0–99.0
RBC	M/MM3	None	0.30–7.00	0.30–7.00
Hemoglobin	g/dL	<=6.0 or >=22.0	7.0–18.0	4.0–21.0
Hematocrit	%	<=18 or >=55	18–50	15–55
Platelet	K/MM3	<=10 or >=1000	10–1000	10–1000

Abbreviations used: ALT, alanine aminotransferase; AST, aspartate aminotransferase; AV, autoverification; RBC, red blood cell; WBC, white blood cell.

AV limits describe the range at which values can release automatically via middleware to the electronic health record provided no other errors or flags intervene. For total protein and albumin, AV does not occur if albumin exceeds total protein concentration. Valid checking range define the analytical measurement range of the analyzer. Critical range are values on the extreme ends of clinical abnormality and have time-limited notification guidelines to the clinical team.

Some calculated parameters derived from the hematology measurements are not listed in the table as they were not assigned critical, AV, or valid checking ranges. These include mean corpuscular volume, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, mean platelet volume, red cell distribution width, and red cell distribution width standard deviation.

We separately quantified “missing values” (blank value for a component), as these can represent either an error in data entry (e.g., not inputting a valid result) or a value that cannot be reported due to an issue such as interference or error flag, which should be documented by comment by the test operator. Each testing panel containing a missing value was evaluated and explanations for the missing value, if present, were also recorded. Duplicates were defined as multiple test results appearing in the EHR for a single specimen and ordered test. Review of results attempted to determine reason for duplicate entries. Time spent in manual documentation was recorded over a period of two months at a clinic site that had not been interfaced. Time savings were calculated by comparing to interfaced reporting without manual entry (zero time spent).

Statistical Analysis

Analyses were performed using SPSS (PASW Statistics 18, Chicago, Illinois). To compare the impact of interfacing to error rates and result reporting, we used chi-square test with Yates’ correction. The measures we focused on were: results outside reference range, results outside valid checking range, missing values, serious/obvious errors, and duplicate results.

Results

Overall Demographics

The entire dataset consisted of 98,848 laboratory values from 8561 panels in 4489 unique patients (with some patients receiving multiple panels during the retrospective timeframe). This total consisted of the following with respect to panels: 10,523 non-blank values from 1317 panels in 1166 unique patients for the BMP; 40,602 non-blank values from 2902 panels in 2654 unique patients for the CMP; and 47,723 non-blank values from 4342 panels in 3862 unique patients for the CBC (Table 2).

Table 2

Patient demographics for each testing panel analyzed.

	Basic netabolic panel (BMP)	Comprehensive metabolic panel (CMP)	Complete blood count (CBC)
Unique patients	1167	2654	3862
Number female (%)	746 (63.9)	1634 (61.6)	2371 (61.4)
Average age (years)	47.2	40.9	39.3
Age standard deviation	19.4	19	20.7
Age median	48	38	36.4
Age range	1.0–89	1.0–89	0.03–89

Abnormal Results and Serious/Obvious Errors Before and After Interface

Across all three testing panels, unique value abnormal results remained consistent at around 2.0‑2.1% before and after interface of the POC devices (Table 3). This was mirrored when quantifying per testing panel (Table 4). Twelve (0.03%) serious/obvious errors were noted in total before interface. None were recorded after the interface for results that autoverified, and only three were evident after the interface for manual edits of results that failed autoverification. Examples of these serious/obvious errors included a mean platelet volume entered as 50 fL, Cl measurements recorded as 1 mEq/L, and multiple cases of negative numbers entered for measured analytes. Interfacing was associated with a significant decrease in serious/obvious errors for the discrete components for the Piccolo tests (P < 0.01) and for all tests (P < 0.005). There were significantly fewer missing values for the CBC testing after interfacing (P < 0.0001) (Table 3).

Table 3

Abnormal results and panel errors across all unique discrete values analyzed.

	Basic metabolic panel (BMP)a	Comprehensive metabolic panel (CMP)a	Complete blood count (CBC)b	Combinedc
	Per total (%)	Per total (%)	Per total (%)	Per total (%)
Results outside reference range
Before interface	55/5022 (1.1)	119/16583 (0.72)	624/17441 (3.6)	798/39046 (2.0)
After interface (Total)	70/6817 (1.0)	200/26919 (0.74)	1083/30282 (3.6)	1353/64018 (2.1)
Autoverified results only	61/6134 (1.0)	169/24405 (0.69)	981/27868 (3.5)	1211/58407 (2.1)
Manual entry after failed autoverification	9/683 (1.3)	31/2514 (1.2)	102/2414 (4.2)	142/5611 (2.5)
Missing values
Before interface	0/5022 (0.00)	7/16583 (0.04)	27/17441 (0.15)	34/39046 (0.09)
After interface (Total)	5//6817 (0.07)	21/26919 (0.08)	12/30282 (0.04)	38/64018 (0.06)
Autoverified results only	4/6134 (0.07)	14/24405 (0.06)	6/27868 (0.02)	24/58407 (0.04)
Manual entry after failed autoverification	1/683 (0.15)	7/2514 (0.28)	6/2414 (0.25)	14/5611 (0.25)
Results outside valid checking range
Before interface	2/5022 (0.04)	3/16583 (0.02)	7/17441 (0.04)	12/39046 (0.03)
After interface (Total)	2/6817 (0.03)	5/26919 (0.02)	10/30282 (0.03)	17/64018 (0.03)
Autoverified results only	1/6134 (0.02)	0/24405 (0.00)	7/27868 (0.03)	8/58407 (0.01)
Manual entry after failed autoverification	1/683 (0.15)	5/2514 (0.20)	3/2414 (0.12)	9/5611 (0.16)
Serious/Obvious errors
Before interface	3/5022 (0.06)	3/16583 (0.02)	6/17441 (0.03)	12/39046 (0.03)
After interface (Total)	0/6817 (0.00)	0/26919 (0.00)	3/30282 (0.01)	3/64018 (0.00)
Autoverified results only	0/6134 (0.00)	0/24405 (0.00)	0/27868 (0.00)	0/58407 (0.01)
Manual entry after failed autoverification	0/683 (0.00)	0/2514 (0.00)	3/2414 (0.12)	3/5611 (0.05)

Comparison of before and after interface for the two Piccolo panels (basic metabolic and comprehensive metabolic panel): results outside reference range, P = 0.99; missing values, P = 0.55; results outside valid checking range, P = 0.85; serious/obvious errors, P < 0.01. Analysis compares all results after interface to all results before interface.

Comparison of before and after interface for the complete blood count: results outside reference range, P = 0.99; missing values, P < 0.0001; results outside valid checking range, P = 0.89; serious/obvious errors, P = 0.12. Analysis compares all results after interface to all results before interface.

Comparison of before and after interface for the two Piccolo panels and the complete blood count results combined: results outside reference range, P = 0.60; missing values, P = 0.13; results outside valid checking range, P < 0.05; serious/obvious errors, P < 0.005. Analysis compares all results after interface to all results before interface.

Table 4

Abnormal results and panel errors per panel analyzed.

	Basic metabolic panel (BMP)	Comprehensive metabolic panel (CMP)	Complete blood count (CBC)	Combined
	Per panel (%)	Per panel (%)	Per panel (%)	Per panel (%)
Results outside reference range (1 or more in a panel)
Before interface	51/559 (9.1)	108/1106 (9.8)	426/1588 (26.8)	585/3253 (18.0)
After interface (Total)	66/758 (8.7)	190/1796 (10.6)	658/2754 (23.9)	914/5308 (17.2)
Autoverified results only	58/682 (8.5)	167/1628 (10.3)	584/2534 (23.1)	809/4844 (16.7)
Manual entry after failed autoverification	8/76 (10.5)	23/168 (13.7)	74/220 (33.6)	105/464 (22.6)
Missing values (1 or more in a panel)
Before interface	0/559 (0.00)	5/1106 (0.45)	14/1588 (0.88)	19/3253 (0.58)
After interface (Total)	5/758 (0.66)	11/1796 (0.61)	6/2754 (0.22)	22/5308 (0.41)
Autoverified results only	4/682 (0.59)	7/1628 (0.43)	2/2534 (0.08)	13/4844 (0.27)
Manual entry after failed autoverification	1/76 (1.3)	4/168 (2.4)	4/220 (1.8)	9/464 (1.9)
Results outside valid checking range (1 or more in a panel)
Before interface	2/559 (0.36)	3/1106 (0.27)	6/1588 (0.38)	11/3253 (0.34)
After interface (Total)	2/758 (0.26)	4/1796 (0.22)	5/2754 (0.18)	11/5308 (0.21)
Autoverified results only	1/682 (0.15)	0/1628 (0.00)	2/2534 (0.08)	3/4844 (0.06)
Manual entry after failed autoverification	1/76 (1.3)	4/168 (2.4)	3/220 (1.4)	8/464 (1.7)
Serious/Obvious errors (1 or more in a panel)
Before interface	2/559 (0.36)	2/1106 (0.18)	6/1588 (0.38)	10/3253 (0.31)
After interface (Total)	0/758 (0.00)	0/1796 (0.00)	3/2754 (0.11)	3/5308 (0.06)
Autoverified results only	0/682 (0.00)	0/1628 (0.00)	0/2534 (0.00)	0/4844 (0.00)
Manual entry after failed autoverification	0/76 (0.00)	0/168 (0.00)	3/220 (1.4)	3/464 (0.65)

A total of 54 duplicate entries of laboratory components were identified, with the most common outcome being identical but duplicated set of results (i.e., with no footnote/comment indicating a reason such as correction of value or documentation of event such as interference with an analyte). 0.71% of records had a duplicate without change before the interface compared to 0.06% for autoverified results and 0.17% overall after interface (Table 5). Interfacing was associated with a significant decrease in overall duplicates for the CBC testing (P < 0.0001) and for all the chemistry and CBC tests combined (P < 0.0001). 40 records across the dataset contained a missing value. No reason was provided for the missing value in 14 out of 17 (82.4%) missing values prior to interface. In contrast, 12 of 23 (52.2%) of the missing records after interface documented a reason such as test error(s) or other cause for cancellation (Table 6).

Table 5

Number of duplicates and reason for duplication across panels analyzed.

		Basic metabolic panel (BMP)	Comprehensive metabolic panel (CMP)	Complete blood count (CBC)	Combined
		Per total (%)	Per total (%)	Per total (%)	Per total (%)
All duplicates	Before interfaceAfter interface (Total)Autoverified results only	1/559 (0.18)a6/758 (0.79)a3/682 (0.79)	10/1106 (0.90)a4/1796 (0.22)a1/1628 (0.06)	24/1588 (1.5)b9/2754 (0.33)b1/2534 (0.04)	35/3253 (0.15)c19/5308 (0.15)c5/4844 (0.02)
	Manual entry after failed autoverification	3/76 (4.0)	3/168 (1.8)	8/220 (3.6)	14/464 (1.5)
Addition of missing value (1 or more values blank)	Before interfaceAfter interface (Total)	0/559 (0.00)2/758 (0.26)	1/1106 (0.09)3/1796 (0.17)	4/1588 (0.25)3/2754 (0.11)	5/3253 (0.15)8/5308 (0.15)
	Autoverified results onlyManual entry after failed autoverification	1/682 (0.15)1/76 (1.3)	0/1628 (0.00)3/168 (1.8)	0/2534 (0.00)3/220 (1.4)	1/4844 (0.02)7/464 (1.5)
Correction of abnormal value (1 or more in a panel)	Before interfaceAfter interface (Total)	0/559 (0.00)1/758 (0.13)	3/1106 (0.27)0/1796 (0.00)	4/1588 (0.25)1/2754 (0.04)	7/3253 (0.22)2/5308 (0.04)
	Autoverified results onlyManual entry after failed autoverification	0/682 (0.00)1/76 (1.3)	0/1628 (0.00)0/168 (0.00)	1/2534 (0.04)0/220 (0.00)	1/4844 (0.02)1/464 (0.22)
No Change (Duplicate Values Identical)	Before interfaceAfter interface (Total)	1/559 (0.18)3/758 (0.04)	6/1106 (0.54)1/1796 (0.06)	16/1588 (1.0)5/2754 (0.18)	23/3253 (0.71)9/5308 (0.17)
	Autoverified results onlyManual entry after failed autoverification	2/682 (0.29)1/76 (1.3)	1/1628 (0.06)0/168 (0.00)	0/2534 (0.00)5/220 (2.3)	3/4844 (0.06)6/464 (1.3)

Comparison of before and after interface for any duplicates for the two Piccolo panels (basic metabolic and comprehensive metabolic panel): P = 0.33. Analysis compares all results after interface to all results before interface.

Comparison of before and after interface for complete blood count: P < 0.0001. Analysis compares all results after interface to all results before interface.

Comparison of before and after interface for any duplicates for the chemistry and complete blood count results combined: P < 0.0001. Analysis compares all results after interface to all results before interface.

Table 6

Number of missing values and reason documented across all unique records analyzed.

		Basic metabolic panel (BMP)	Comprehensive metabolic panel (CMP)	Complete blood count (CBC)	Combined
		Per total (%)	Per total (%)	Per total (%)	Per total (%)
No reason documented	Before interfaceAfter interface (Total)Autoverified results onlyManual entry after failed autoverification	0/559 (0.00)2/758 (0.26)1/682 (0.15)1/76 (1.3)	2/1106 (0.18)5/1796 (0.28)3/1628 (0.18)2/168 (1.2)	12/1588 (0.76)4/2754 (0.15)0/2534 (0.00)4/220 (1.8)	14/3253 (0.43)11/5308 (0.21)4/4844 (0.08)7/464 (1.5)
Presence of test error(s)	Before interfaceAfter interface (Total)Autoverified results onlyManual entry after failed autoverification	0/559 (0.00)3/758 (0.40)3/682 (0.44)0/76 (0.00)	2/1106 (0.18)5/1796 (0.28)3/1628 (0.18)2/168 (1.2)	0/1588 (0.00)3/2754 (0.11)2/2534 (0.08)1/220 (0.45)	2/3253 (0.06)11/5308 (0.21)8/4844 (0.17)3/464 (0.65)
Test canceled for reason other than test error	Before interfaceAfter interface (Total)Autoverified results onlyManual entry after failed autoverification	0/559 (0.00)0/758 (0.00)0/682 (0.00)0/76 (0.00)	1/1106 (0.09)1/1796 (0.06)1/1628 (0.06)0/168 (0.00)	0/1588 (0.00)0/2754 (0.00)0/2534 (0.00)0/220 (0.00)	1/3253 (0.03)1/5308 (0.02)1/4844 (0.02)0/464 (0.00)

Estimated Autoverification Impact From Rules Based on Test Values Alone

From the parameters outlined in Table 1, we estimated impact on autoverification based solely on test values that were outside the autoverification limits. This analysis excludes other factors such as instrument error flags that were not reliably captured before instrument interface. This used the dataset of resulted values for the entire retrospective analysis timeframe. For the BMP, 3.8% of panels (21 of 559) had one or more values outside autoverification limits. The most common were high Na (n=4), low K (n=4), and high calcium (n=3). For the CMP, 15.3% of panels (161 of 1106) had one or more values outside autoverification range values. The most common were high Na (n=37), high glucose (n=22), low K (n=20), high BUN (n=19), and low calcium (n=18). For CBC, 3.8% of panels (48 of 1588) had one or more values outside autoverification range values. By far, the most common reason was hemoglobin and/or hematocrit above the upper autoverification range. This scenario accounted for 43 of the 48 (89.6%) of the CBC panels that would have failed autoverification.

Comparison of Electrolyte Values Between POC and Core Laboratories

The most common clinician complaints regarding the Piccolo testing that were directed to pathologist related to Na, Cl, and/or CO2 values that did not make sense in clinical context. Obvious trends were not evident on routine validation and between-laboratory comparisons, recognizing that these involve much smaller datasets compared to overall patient testing. Thus, we compared overall Na, Cl, CO2, and AG results between the clinics performing the Piccolo BMP and CMP compared to these same panels performed at the medical center core laboratory for the outpatient population on a Roche Diagnostics automated chemistry line. The most obvious difference was that the AG from these two chemistry panels for the Piccolo resulted in an increased number of AG values less than zero (negative values), a phenomenon very rare (<0.01%) with outpatient samples run in the core laboratory. Negative AG were seen in 1.4% and 3.8% of BMP and CMP panels, respectively, on the Piccolo. The other notable difference was a higher proportion of Cl values greater than 110 and CO2 values greater than 30, a phenomenon that was a common complaint from clinicians. Na results outside limits were comparable between the Piccolo and core laboratory (Table 7).

Table 7

Comparison across point-of-care and main hospital core clinical laboratory instruments of frequency of anion gap, sodium, chloride, and CO2 values outside of specific ranges.

	Percentage of values (%)
	BMP - POC (n=1585)	CMP - POC (n=11139)	BMP - core (n=185277)	CMP - core (n=163198)
Anion gap < 0	1.39	2.77	0.00	0.00
Anion gap > 18	1.01	0.87	3.03	2.62
Na < 130 mEq/L	0.82	0.93	1.98	1.66
Na > 150 mEq/L	0.25	0.33	0.09	0.05
Cl < 90 mEq/L	0.63	0.42	1.54	1.23
Cl > 110 mEq/L	2.78	3.21	0.96	0.57
CO2 < 18 mEq/L	0.69	0.13	2.22	1.58
CO2 > 30 mEq/L	11.55	15.49	2.79	2.34

Abbreviations: BMP, basic metabolic panel; CMP, comprehensive metabolic panel; POC, point-of-core.

Time Savings

We calculated time spent in manual documentation of results for CBC with differential (1 min. and 42 sec) and for CMP (1 min. and 14 sec). At one clinic site over the course of two months, an average of 468 CBCs and 405 CMPs were run per month. Together, interfacing of this clinic site would save approximately 21.6 hr of manual entry work per month (13.3 hr per month for CBCs and 8.3 hr per month for CMPs), recognizing that a small percentage of samples would have required manual intervention due to result components that fail autoverification. In addition to the two urgent care sites analyzed in the present study, our health system also has two additional sites (one pediatric clinic and one hematology/oncology specialty clinic) that use the Piccolo and pocHi devices with similar test volumes for CBCs, BMPs, and CMPs. Thus, time savings from autoverification of these devices across clinic sites becomes substantial.

Modified Autoverification Rules

After discussion with medical leadership of the clinics using the POC devices, updated autoverification rules were implemented. These rules built on the simplified autoverification schemes originally used. In particular, experience with the analyzers identified multiple opportunities for autoverification improvement including the following: Handling of results outside instrument measuring range (including possible absurd results), specimen interferences such as hemolysis, instrument error flags (e.g., unidentified cells on the CBC), and critical values (including consistency of reporting across instrument operators). The discussions with clinic leadership and personnel were especially valuable in working through possible scenarios and reaching a consensus when to prompt the instrument operator to perform actions such as remix specimen and repeat analysis (if feasible), draw a new specimen, notify the ordering clinician, and/or send specimen to the hospital core laboratory. These discussions also considered scenarios where the patient may be referred to the hospital emergency department. The group then decided on the verbiage for printouts that would direct the instrument operator what to do when autoverification failed.

The updated autoverification rules are summarized in Table 8 and include rules that go beyond numeric factors such as autoverification limits or measuring ranges. These changes have been generally well-received, with ongoing education when needed. An example where this workflow was helpful was a case involving a patient with beta-thalassemia who had markedly elevated nucleated RBCs that erroneously were categorized as WBCs on the pocHi, leading to total WBC count outside of autoverification range. The printout led the operator to discuss with clinician and then send to central laboratory, which measured a much lower WBC count and correctly identified the nucleated RBCs. We have encountered instances where confusion occurred with testing performed right near a shift change, with miscommunication between personnel leading to confusion (such as remixing and rerunning specimens). These have also represented opportunities for education and improvement.

Table 8

Autoverification rules to mitigate common errors of point-of-care devices and interfacing.

Result/Error on device	Explanation	Result suppressed	Manual partial entry permitted	Remix and repeat <1 hr	Other actions
Piccolo
~~~~	Sodium beyond measuring range	Yes	No	Yes	If repeat measurement also yields warning/error, notify provider and discuss whether to redraw, manually enter and finalize results, or send to core lab. If outside the 1 hour collection window, redraw or send to core lab.
!	CO2 resulted questionable values	Yes	No	Yes	If repeat measurement also yields warning/error, notify provider and discuss whether to redraw or send to core lab. If outside the 1 hour collection window, redraw or send to core lab.
Hemolyzed	Analyte resulted as hemolyzed or other value icteric/lipemic	Yes	Yes	No	Notify provider and discuss whether to redraw, manually file result (not preferred) or send to core lab.
Albumin > total protein concentration	Clear error in protein quantities	Yes	No	Yes	If repeat measurement also yields warning/error, notify provider and discuss whether to redraw or send to core lab. If outside the 1 hour collection window, redraw or send to core lab.
Beyond AV	Value is beyond autoverification limit for device	Yes	No	Yes	If repeat measurement also yields warning/error, notify provider and discuss whether to redraw, manually finalize results or send to core lab. If outside the 1 hour collection window, redraw or send to core lab.
Critical	Result value in critical range	No	No	No	Document in log
pocH-i100
Critical	Result value in critical range	No	No	No	Document in log
---.- or *	Contaminated or abnormal sample	Yes	No	Yes	If repeat measurement also yields warning/error, send to core lab.
Beyond AV	Value is beyond autoverification limit for device	Yes	No	Yes	If repeat measurement also yields warning/error, notify provider and discuss whether to redraw, manually finalize results or send to core lab.

Abbreviations: AV, autoverification.

Discussion

The present study assesses the impact of interfacing POC clinical chemistry and hematology testing to the central EHR via middleware software. The most visible change was a reduction of serious/obvious errors (including implausible values such as negative numbers) and a reduction in duplicated results. However, several serious/obvious errors were seen after the interface change with results that required manual entry due to result components failing autoverification. The serious/obvious error rates observed prior to interface were low when compared to previous work examining data entry, which is reassuring when considering the potential financial and emotional cost of an error. However, there is the caveat that more subtle errors likely occur and are not detected by broad comparisons.

Two previous studies have looked at glucometers, devices that produce a single result.[,] Glucometers are often widely dispersed throughout an institution (outpatient and inpatient) and thus utilized by many personnel of varying levels of experience and training. In the present study, the POC chemistry and hematology devices were located in a clinic laboratory area, with a relatively limited group of employees performing testing and recording the results, allowing for tighter quality control and training. A quality team from the Department of Pathology also provided oversight and guidance for laboratory testing across the offsite clinics. Exact estimation of POC error rates is challenging to do. One prior study took advantage of a fortuitous transition period where POC device operators manually entered results without realizing interface was active.[]

While reduction in manual entry errors is an important goal, another main benefit of interfacing POC devices with the EHR is to reduce the time and manpower spent on inputting values into the EHR. We have further modified our autoverification rules based on the experience with the change to interface and a simple rule set. This ongoing work provides more explicit directions to instrument operators in terms of rerunning specimens, drawing new specimens, seeking guidance from clinician who ordered the testing, and sending specimens to core laboratory. This has helped standardize practices across clinics and furthering collaboration between the clinics and pathology quality assurance, as described in other publications.[,,,]

We also investigated clinician complaints regarding issues with electrolytes that did not make sense in clinical context. Even though at a broad level the Piccolo measurements compared reasonably well with core laboratory automated instruments in validation and inter-instrument correlation studies, trends over a several year period revealed some key differences. Comparison of AG revealed one clue, which showed a much higher occurrence of negative values from results on the Piccolo compared to outpatient samples measured in the core laboratory. Analysis of the Piccolo revealed a propensity for higher Cl and CO2 values, thus resulting in more numerous low AG values. For this issue, the ability to extract and analyze a large dataset was invaluable.

Financial support and scholarship

Nil

Presentation at a meeting

N/A

Conflicting Interest

The authors all declare no conflicts of interest.