Real-world use of key performance indicators for point-of-Care Testing network accredited by ISO 22870.

OBJECTIVE: We aimed to evaluate the results of key performance indicators (KPIs) for a period of over three years, as well as their effectiveness as an improvement tool, to provide information about Point-of-Care Testing (POCT) management system performance and quality assurance. DESIGN AND METHODS: KPIs regarding the global POCT process, extra-analytical phase, quality assurance and staff training and competency were evaluated for blood gases, HbA1c, sweat test and non-connected and connected glucose in an ISO 22870 accredited network. We established the definition of every KPI and its corresponding target. The results of KPIs from all clinical settings were appraised every month during the study period, taking corrective actions when necessary.
RESULTS: Annual global results were generally acceptable. However, some clinical areas displayed deviations in specific months. The monitoring of these KPIs allowed us to detect the deviations immediately and identify their causes. These included errors in patient identification, consumables, strips, reagents, analyzers, calibration, internal and external quality control, sample management, connectivity, and operator identification strategy, among others.
CONCLUSIONS: The evaluation of these KPIs over time has shown their appropriateness. This set of quality indicators could be a useful tool for laboratory medicine leading POCT networks for better and safer patient care.

Introduction

La Paz University Hospital is a tertiary public hospital located in Madrid that is a referral center widely recognized for its range of specializations. It is one of the largest in Spain, with approximately 1300 beds, with several satellite facilities providing specialist services and 23 primary healthcare centers.

The Laboratory Medicine Department includes a specific Point-of-Care Testing (POCT) Unit that has led the hospital’s POCT network for the last 22 years. The core of the POCT Unit includes the POCT Director, the POCT Coordinator and the head of the Quality section. There is also a multidisciplinary POCT committee, which is composed of the hospital’s Board of Directors, representatives of the hospital administration, physicians and nursing personnel from medical and surgical services, staff from the IT department, the laboratory medicine team and the POCT Unit.

The hospital’s POCT network, led by laboratory medicine, has been growing since 1998. It currently includes 29 ABL90 Flex (Radiometer®, Brønshøj, Denmark) blood gas analyzers, 2 DCA Vantage (Siemens Healthineers®, PA, USA) HbA1c devices, 266 Accu-Chek Performa (Roche®, Basel, Switzerland) nonconnected glucometers, a Sweat Chek (Elitechgroup®, Logan, USA) sweat test device, and 5 recently installed Accu-Chek Inform II (Roche®, Basel, Switzerland) connected glucometers (Table 1). All blood gas analyzers, HbA1c devices, 4 nonconnected glucometers, and the sweat test are accredited by the International Organization for Standardization (ISO) 22,870 standard.

Table 1

La Paz University Hospital’s POCT network.

POCT (number of analyzers)	Clinical setting	Analyzers
Blood gases (29)	Emergency Laboratory	2
	Preanalytical Unit	2
	Delivery room	2
	Pediatric Emergency Department	1
	Neonatal ICU Department	3
	Pediatric ICU Department	1
	Pediatric Reanimation and Surgery Unit	1
	Pediatric Hemodynamics Unit	1
	Emergency Department	2
	Coronary Care Unit	1
	Pulmonology Department Doctor’s office	1
	Pulmonology Department	1
	Nephrology Department	1
	ICU Department	1
	Burn Unit	1
	Reanimation Unit 1	1
	Reanimation Unit 2	1
	Reanimation Unit 3	1
	Surgery Suite 1	1
	Surgery Suite 2	1
	Surgery Suite 3	1
	Cantoblanco Hospital	1
	Carlos III Hospital	1
HbA1c (2)	Adults Diabetes Unit	1
	Pediatric Diabetes Unit	1
Non-connected glucose (266)	84 different departments in the hospital	266
Sweat test (1)	Pediatric Pulmonology Unit	1
Connected glucometers (5)	Neonatal ICU Department	3
	Pediatric ICU Department	1
	Pediatric Diabetes Unit	1

According to the international standard for medical laboratories accreditation (ISO 15189:2012) and POCT accreditation (ISO 22870:2016), the laboratory establishes suitable key performance indicators (KPIs) to monitor and evaluate performance throughout critical aspects of analytical and extra-analytical (pre and post) processes.

The process of monitoring KPIs must be planned, including establishing the objectives, methodology, interpretation, limits, action plans for each objective, and duration of measurement [1,2]. The scope and number of the objectives and targets must be realistic and achievable. Otherwise, the success and continued commitment from POCT network participants will be compromised.

Despite the relevance of KPI monitoring, most publications are focused on laboratory medicine [[3], [4], [5]] and not specifically on POCT [[6], [7], [8]].

The creation and development of our homogenous POCT network includes a large number of operators in numerous clinical settings and follows the requirements of ISO 22870. The management of KPIs has been based on 3 key areas: quality assurance, staff training and competency, and continuous improvement. It is essential to select and periodically review the KPIs that could be useful to identifying opportunities for improvement as well as to assess the laboratory’s contribution to patient care.

This study aimed to evaluate the results of the KPIs over a period of more than 3 years and their effectiveness as an improvement tool to provide information about POCT management system performance and quality assurance.

Materials and methods

Taking advantage of the accreditation process, in January 2017 we reviewed and implemented KPIs in various areas of the POCT process for blood gases, HbA1c, and nonconnected glucose testing. When the sweat test and connected glucose were incorporated into the POCT network, they were also included in the KPI monitoring strategy. Data were collected from the laboratory information system (LIS) (Trak Care; Intersystems) and the POCT data managers (Aqure; Radiometer for blood gases and HbA1c and IT1000; Roche Diagnostics for connected glucose). In the case of the nonconnected glucose and sweat tests, we only considered the KPI evaluation for the analytical phase due to their lack of connectivity to a data management system or LIS.

The data for each clinical setting were collected monthly. All deviations were reviewed, and corrective actions were taken when necessary. The global average of all KPI results was also calculated and recorded in the improvement dashboard to give us an overview of the POCT network.

The definition of the selected KPIs and the corresponding target established for each by the laboratory are summarized in Table 2.

Table 2

Definition of the selected KPIs and the corresponding target established for each by the laboratory.

	Purpose of the evaluation	Key Performance Indicators		Target
1. Global POCT process	Adequate use of POCT in each clinical setting	1.1. Percentage of the tests reported in LIS over the tests performed in POCT analyzers	Blood gases reported/performed (%)	≥80
			HbA1c reported/performed (%)
			Glucose reported/performed (%)
	Duplicate test requests to the laboratory and POCT from the same clinical setting	1.2. Percentage of the tests reported in LIS by the laboratory over the tests reported in LIS by POCT	Blood gases reported in lab/reported in POCT (%)	≤5
			HbA1c reported in lab/reported in POCT (%)	100–200
			Glucose reported in lab/reported in POCT (blood gases and glucometers)	≤35
	Use of material resources	1.3. Difference between the number of tests considering the consumables used and the tests performed in POCT analyzers	Blood gases measurements considering the electrode cassette used - performed (n)	≤0
			Glucose measurements considering the strips used - performed (n)	≤600
2. Extra-analytical phase	Samples and analyzer management by POCT operators	2.1. Percentage of tests with pre-analytical errors (blood gases) or instrument alerts (glucometers) over the total tests performed in POCT analyzers	Blood gases with pre-analytical errors (non-homogeneus sample and insufficient volume)/performed (%)	≤10
			Instrument alerts in glucometers/glucose measurements performed (%)
	Patient identification by POCT operators	2.2. Percentage of tests with patient identification errors (electronic medical record ≤ 3 digits) over all the tests reported in LIS by POCT	Blood gases with patient identification errors/performed (%)	≤1
			HbA1c with patient identification errors/performed (%)
			Glucose with patient identification errors/performed (%)
3. Analytical phase	Fulfilment of analytical performance specifications established by laboratory	3.1. Percentage or number of magnitudes with coefficient of variation (CV) within analytical performance specifications over the total	Blood gases VC (%)	≥90
			HbA1c CV (n)	≥2
			Non-connected glucose CV (%)	≥90
			Sweat test CV (n)	≥2
			Connected glucose CV (%)	≥90
		3.2. Percentage or number of magnitudes with total error (TE) within analytical performance specifications over the total	Blood gases TE (%)	≥90
			Cooximetry TE (%)	≥90
			Bilirubin TE (n)	≥2
			HbA1c TE (n)	≥1
			Non-connected glucose TE (%)	≥90
			Sweat test TE (n)	≥1
			Connected glucose TE (%)	≥90
4. Staff training and competency	Personal identification strategy by POCT operators in order to assure that only trained operators use the POCT analyzers	4.1. Percentage of tests performed by the POCT operator with the highest activity over all tests performed in every clinical setting	Blood gases performed by the operator with the highest activity/all blood gases performed (%)	≤10

Results

The annual average and standard deviation for every KPI are shown in Table 3. The main causes of nonfulfillment found over the 3 year period, taking into account the targets established by the laboratory, are also included.

Table 3

Annual average and standard deviation for every KPI and the main causes of nonfulfillment found over the 3 year period.

Key performance indicators	Target	2017 mean (SD)	2018 mean (SD)	2019 mean (SD)	2020 mean (SD)	Main causes of error
Blood gases reported/performed (%)	≥80	93 (3)	91 (1)	91 (1)	90 (2)	Patient identification with unmatched numberErrors in sample management: pre-analytical errorsErrors in POCT analyzers: instrument alertsErrors in LIS, POCT data manager systems, connectivity
HbA1c reported/performed (%)		98 (3)	97 (3)	97 (2)	94 (5)
Glucose reported/performed (%)		No data	No data	93 (9)	98 (2)
Blood gases reported in lab/reported in POCT (%)	≤5	3 (5)	2 (3)	2 (2)	3 (3)	Recent incorporation of the blood gas analyzer into the POCT networkFrequent unavailability of the blood gas analyzer due to technical issuesInadequate blood gas analysis request to the laboratory from the clinical setting
HbA1c reported in lab/reported in POCT (%)	100–200	155 (24)	167 (23)	132 (28)	577 (633)	Changes in the number of patients assisted in doctor’s offices with POCT HbA1c measurement
Glucose reported in lab/reported in POCT (blood gases and glucometers)	≤35	No data	No data	3 (3)	13 (5)	Not detected
Blood gases measurements considering the electrode cassette used - performed (n)	≤0	0 (0)	0 (0)	0 (0)	0 (0)	Changes of the POCT activity in a particular clinical setting
Glucose measurements considering the strips used - performed (n)	≤600	No data	No data	0 (0)	830	High request for strips by clinical settings compared to the number of glucose measurements performed
Blood gases with pre-analytical errors/performed (%)	≤10	2 (0)	2 (0)	2 (0)	2 (0)	Need for more training for particular operatorsDifficulties in sample collection and managementTechnical incidents with analyzers
Instrument alerts in glucometers/glucose measurements performed (%)		No data	No data	0 (0)	5 (3)
Blood gases with patient identification errors/reported (%)	≤1	0 (0)	0 (0)	0 (0)	1 (1)a	Patient identification errors: “0”, random numbers, patient demographics identification, operator identification, etc.
HbA1c with patient identification errors/reported (%)		1 (1)	0 (0)	0 (0)	0 (0)
Glucose with patient identification errors/reported (%)		No data	No data	1 (2)	1 (0)
Blood gases CV (%)	≥90	96 (1)	99 (1)	98 (0)	99 (0)	Inadequate handling of internal quality control material by operatorErrors in consumables or reagentsTechnical errors in analyzers
HbA1c CV (n)	≥2	4 (0)	4 (0)	4 (0)	4 (1)
Non-connected glucose CV (%)	≥90	100 (0)	100 (1)	100 (0)	100 (0)
Sweat test CV (n)	≥2	No data	2 (0)	2 (0)	3 (1)
Connected glucose CV (%)	≥90	No data	No data	100 (0)	98 (4)
Blood gases TE (%)	≥90	92 (3)	98 (1)	98 (1)	99 (0)	Inadequate handling of external quality control material by operatorCalibration errorsSystematic and/or random errorsInterferencesErrors in consumables or reagentsTechnical errors in analyzers
Cooximetry TE (%)	≥90	98 (6)	92 (15)	97 (6)	97 (5)
Bilirubin TE (n)	≥3	No data	4 (1)	4 (0)	4 (1)
HbA1c TE (n)	≥1	2 (1)	2 (0)	2 (0)	2 (1)
Non-connected glucose TE (%)	≥90	100 (0)	100 (1)	100 (0)	100 (0)
Sweat test TE (n)	1	No data	1 (0)	1 (0)	1 (0)
Connected glucose TE (%)	≥90	No data	No data	No data	100 (0)
Blood gases performed by the operator with the highest activity/all blood gases performed (%)	≤10	12 (5)	5 (1)	5 (1)	8 (2)	Use of a POCT analyzer by untrained staff who use the personal identification of a qualified operator

2 (0) considering the modification of the KPI.

The main results of these indicators are detailed below.

Global POCT process

Adequate use of POCT in each clinical setting (percentage of the tests reported in LIS over the tests performed in POCT analyzers)

This KPI showed acceptable results every year. However, some clinical settings displayed deviations in specific months. In Fig. 1, the results of the KPI for each blood gas analyzer are shown, as an example of the entire monthly evaluation. Every deviation was immediately corrected after performing the corresponding action.

Fig. 1

KPI 1.1. Adequate use of POCT in each clinical setting. Monthly evaluation of the percentage of blood gas measurements reported in the LIS over all blood gas measurements performed in POCT analyzers. Each line corresponds to a specific clinical setting.

Target: ≥80% (red line).

∗Results above 100

% were due to sporadic errors in the information systems (the date of the test performed was different from the date of the test reported) that were corrected after their detection. . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Results below the target were generally related to errors in the pre-analytical phase, technical problems with POCT analyzers, or problems with information systems. This KPI detected these incidents, and the corrective actions implemented included more staff training, technical maintenance of the analyzers, and review of connectivity and information systems.

Duplicate test requests to the laboratory and POCT from the same clinical setting (percentage of tests reported in LIS by the laboratory over the tests reported in LIS by POCT)

Annual global results of this KPI were acceptable according to the target. Fig. 2 illustrates the results of every clinical setting during the study period. The high results observed for a particular clinical setting correspond to the time when a blood gas analyzer was incorporated. The figures gradually plummeted to values below the target until all the staff were trained and qualified to use the POCT analyzer.

Fig. 2

KPI 1.2. Duplicate test requests to the laboratory and POCT from the same clinical setting. Monthly evaluation of the percentage of blood gas measurements reported in the LIS by laboratory over blood gas measurements reported in the LIS by POCT analyzers from the same clinical setting. Each line corresponds to a specific clinical setting.

Target: ≤5% (red line).

∗Results out of the scale correspond to when a blood gas analyzer was incorporated into the POCT network. . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Other deviations were related to the unavailability of some blood gas analyzers in a particular month due to technical problems or inadequate requests sent to the laboratory. Corrective actions were taken in all these cases.

Regarding HbA1c and connected glucose, it is acceptable to request measurements from both the laboratory and POCT because their clinical use is different: laboratory (diagnosis and monitoring) and POCT (monitoring). The increase observed in HbA1c in 2020 was due to the extremely low number of patients assisted with POCT in doctors’ offices during the coronavirus disease 2019 (COVID-19) pandemic.

Use of material resources (difference between the number of tests considering the consumables used and the tests performed in POCT analyzers)

The annual results obtained for blood gases illustrate that the consumables selected were used efficiently in each clinical setting with an average of zero. A few departments showed some deviations in specific months with unusually low activity. In those cases, the corrective action was to change the capability of the electrode cassette. However, this indicator did not measure the actual use of every consumable. As a result, the definition of this KPI was modified to improve the results.

Regarding the connected glucometers, the results correspond only to the first quarter of 2020, when a high number of requests for strips was detected in a particular clinical setting. We contacted this particular department and, when results returned to below the target, continued monitoring the number of requests.

Extra-analytical phase

Sample and analyzer management by POCT operators (percentage of tests with pre-analytical errors [blood gases] or instrument alerts [glucometers] over the total tests performed)

Though the annual average for blood gases was acceptable, Fig. 3 shows the different results depending on the clinical setting. The only department with values above the target was the delivery room, which was mainly related to difficulties in the collection of fetal scalp and umbilical cord blood samples. As a corrective action, the laboratory has worked together with the Obstetrics and Gynecology department to reduce these errors since mid-2019.

Fig. 3

KPI 2.1. Sample and analyzer management by POCT operators. Monthly evaluation of the percentage of blood gas measurements with pre-analytical errors over blood gas measurements performed in POCT analyzers from the same clinical setting. Each line corresponds to a specific clinical setting.

Target: ≤10% (red line).

The line above the target represents the delivery room department. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

In the case of connected glucose, this indicator is associated with instrument alerts, which included technical incidents with strips or devices and the need for more staff training. The evaluation of this indicator detected these errors and led to corrective actions, including the recent replacement of one of the devices.

Patient identification by POCT operators (percentage of tests with patient identification errors (electronic medical record ≤3 digits) over all the tests reported in LIS by POCT)

This KPI is related to patient safety, and its annual results for blood gases were considered acceptable overall. However, some clinical settings showed higher values than the target. These errors did not have a direct impact on the electronic medical record (EMR) because the results were not incorporated into information systems.

The evaluation of these cases in detail showed other patient identification errors, considering all the tests performed instead of only the tests reported in the LIS. Moreover, the criteria of EMR ≤3 digits was insufficient to detect all possible errors. Thus, at the beginning of 2020, we started to simultaneously monitor a modification of this KPI. We collated all patient identifications that did not correspond to a true EMR over the total tests performed in each POCT analyzer. Tracking this new indicator in the first 5 months of 2020, we detected more deviations (mean [SD] 2 [1] %) than the 1 (1) % provided by the former KPI.

Fig. 4 illustrates the results of the former KPI and Fig. 5 those of the modified indicator. In both figures, an increase was observed in March and April 2020, when the COVID-19 pandemic showed a higher prevalence in Spain, due to the incorporation of numerous new operators at the same time.

Fig. 4

KPI 2.2. Patient identification by POCT operators. Monthly evaluation of the percentage of blood gas measurements with patient identification errors over blood gas measurements reported in the LIS. Each line corresponds to a specific clinical setting.

Target: ≤1% (red line). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Fig. 5

KPI 2.2. Patient identification by POCT operators. Monthly evaluation of the percentage of blood gas measurements with patient identification errors over blood gas measurements performed in POCT analyzers. Each line corresponds to a specific clinical setting.

Target: ≤1% (red line). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

These errors were not detected in the remaining tests included in the network.

Analytical phase

Fulfillment of analytical performance specifications (percentage or number of tests with coefficient of variation and total error within analytical performance specifications over the total)

The frequency of the internal quality control depends on the test, ranging from 8 ​h for blood gases to 24 ​h for HbA1c, glucose and sweat test. In the case of external quality assurance, the schedule is set by each organizer: monthly for blood gases, HbA1c and sweat test and every three months for glucose.

In general, the annual coefficient of variation and total error results were acceptable.

By means of internal quality control evaluation, we detected incidents with the handling of internal quality control material, consumables, strips or reagents, and technical errors in the analyzers, as well as defective control of material-specific batches or uncorrected assignments of the target value of control material.

This evaluation, together with the external quality assessment, identified occasional problems in specific clinical settings, such as a negative bias in HbA1c in June 2018 that led to the replacement of the POCT analyzer. Another finding was a positive bias in January 2018 in capillary glucose, which resulted in an alert by the Spanish Medicines Agency. The defective batches of strips were removed by the supplier.

Staff training and competency

Personal identification strategy by POCT operators (percentage of tests performed by the POCT operator with the highest activity over all tests performed in every clinical setting)

The initial results obtained in 2017 on this KPI revealed that the operator identification strategy was not being used properly in all cases, and untrained personnel were performing tests (Fig. 6). The most effective corrective action has been to contact the staff of the specific departments and discuss with them the relevance of this issue, providing training when necessary. We highlight the increase in training and competency issues observed in April 2020 in a new intensive care unit created for patients with COVID-19. Numerous professionals joined this clinical environment at the same time and, despite initial training in the use of POCT, the use of operator identification was not appropriate. This deviation was reduced the following month, and we will continue working on it.

Fig. 6

KPI 4.1. Personal identification strategy by POCT operators. Monthly evaluation of the percentage of blood gas measurements performed by the POCT operator with the highest activity over all tests performed in every clinical setting. Each line corresponds to a specific clinical setting.

Target: ≤10% (red line). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

This KPI is not useful in clinical settings with few POCT operators. The deviations observed in these settings lacks importance because they should not be considered as a real inappropriate use of the operator identification strategy.

Discussion

KPIs are used to monitor and evaluate critical areas that influence patient outcomes [2]. In POCT, it is relevant to include KPIs related to all the phases of the total testing process, including areas that could also be essential, such as operator training and competency [7].

The development of a consolidated POCT network over 22 years, with the implementation, monthly evaluation, and review of KPIs has allowed us to monitor the total POCT process, detecting deviations that could have an impact on patient care. These KPIs have periodically been evaluated in a dashboard as an overview of the entire POCT network, but also in every clinical setting. The global results for a KPI for all locations are often acceptable, as we have generally observed in this study. However, important deviations were detected in specific clinical settings in particular months. By means of this evaluation, corrective actions were taken immediately, preventing the problem from continuing over time. For us, the monthly evaluation of KPIs for each location in detail has been helpful.

Apart from the periodic evaluation of KPI results, it is important to assess the appropriateness of the KPI itself to appraise its suitability for the intended purpose. As a result, we could decide whether the KPI was useful, should be modified, or if its monitoring might no longer be justified [9]. As stated by Murphy et al. [2], our KPIs have been specific, measurable, achievable, relevant, and time time-phased. Only 2 of them needed to be modified.

For this purpose, connectivity and the use of laboratory information and POCT data manager systems are indispensable. They also allow us to evaluate the test request management, taking into account that both the laboratory and POCT are used for patient care. The test request rationalization is relevant for a correct interpretation of the results and their use in patient management.

With non-connected devices, such as our Accu-Chek Performa glucometers or the sweat test, there are numerous limitations for data collection. It is a tedious and time-consuming activity, and in many cases the information obtained is not complete [10]. Thus, many clinical laboratories adopt only a few conventional KPIs [10]. In our study, the KPI evaluations considered only the analytical phase (quality assurance) for these tests without connectivity.

Most of the published articles regarding KPIs are more related to conventional laboratory medicine and not specifically to POCT [[11], [12], [13]]. For example, the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) Working Group’s “Laboratory Errors and Patient Safety” developed a Model of Quality Indicators, available on www.ifcc-mqi.com [4,14]. Our results cannot be fully compared with those suggested by the IFCC because the definition of these indicators includes some aspects that are not common in POCT, such as unintelligible requests, unsuitable samples for transportation, and storage problems during the extra-analytical phase [3,4,15].

Regarding the 3 KPIs implemented to monitor the global POCT process, we highlight the fact that the number of duplicate test requests sent to the laboratory and POCT from the same clinical setting is especially relevant for blood gases. This indicator should be close to zero, given all the magnitudes measurable on the POCT analyzer ought to generally be performed exclusively as POCT and not also in the laboratory.

Concerning the adequate use of material resources, it is important to determine the need for consumables in every clinical setting according to its activity, and monitor their use with a specific KPI over time to fit the needs to the activity rates.

Among the indicators related to the extra-analytical phase, patient identification is generally mandatory in connected POCT analyzers to report results. Thus, among the most common and most concerning errors are patient misidentification caused by transcription errors and the use of nonstandard identification [16]. Studies performed in conventional laboratories have shown misidentification errors ranging from 0.040% to 0.083% [1]. These results are better than the 1% observed in our study in certain years and much better than the results obtained with the modified KPI measured simultaneously in 2020. This type of error is usually related to the participation of the many and varied POCT operators. When a deviation is detected, it is crucial to provide feedback to the staff of the relevant clinical environment focusing on the consequences of patient misidentification that can be dangerous and harmful. Developing new procedures to prevent further errors or additional training on this topic has also been useful. In our study, a reduction in this error was observed after these corrective actions were taken.

In regard to the analytical phase, the KPIs associated with quality assurance according to analytical performance specifications have been established. These include unacceptable performance in internal quality control, external quality assurance, and proficiency testing schemes [4]. These KPIs are essential to assessing the quality of patient results and identify errors that could have a significant impact on patients, such as the errors described in this study. In addition, the comparison of these KPIs between laboratory and POCT methods for the same measurand over time could also be useful for an overview of the analytical performance of all the methods used with patients [8].

Finally, staff training and competency are especially relevant in POCT because a large number of operators use the analyzers in various clinical settings. In addition, staff turnover is high, and consequently, initial training planning must be rapid and flexible to ensure that only qualified personnel are using the analyzers. In general, the KPIs on this issue include the number of training events completed by staff [4]. This KPI might be insufficient in POCT because it does not completely ensure the training and competency of all staff who are working in every clinical setting. For blood gases alone, there are currently more than 2000 trained operators. All of them must enter a personal identification code in the analyzer before any action. However, a new staff member could use the personal identification of another already qualified staff person. The KPI evaluated in this study was effective for detecting this type of error. Every time we identified an increase of this indicator, we contacted the clinical setting to assess whether it was due to actual misuse of operator identification. The impact of this corrective action observed on the KPI the following months was evident.

The harmonization of KPIs is also a key step in POCT. It could allow an inter-laboratory comparison (benchmarking) of data and performance, as was developed by the IFCC Working Group [4]. Preliminary experiences with comparing interlaboratory KPIs have found that few laboratories were regularly performing comprehensive data collection for various reasons. These include the complexity of monitoring KPIs over time, difficulties in defining and implementing procedures to identify and monitor KPIs regularly, and problems in data collection [10]. Therefore, this type of study, which reports the experience of monitoring KPIs over a period of time, could be useful for selecting effective indicators with a patient-centered approach. Efforts should be made to improve this issue specifically in POCT, in accordance with ISO 22870 requirements and focused on patient care.

Conclusions

Improving POCT quality requires the development, implementation, and validation of performance using meaningful and reliable KPIs. A multidisciplinary effort is required to achieve the desired performance goals and improve outcomes. The evaluation of these KPIs over time could determine a set of quality indicators and the implementation of improvement actions with POCT led by laboratory medicine, with the aim of achieving safer and better patient care.

Author statement

Paloma Oliver and Pilar Fernandez-Calle conceived of the presented idea, developed the theory, performed the computations, assessed the results and wrote the manuscript.

Roberto Mora, Jorge Diaz-Garzon, Daniel Prieto, Marta Manzano, Inmaculada Domínguez and Antonio Buño supervised the project and reviewed the research and manuscript.

All authors discussed the results and commented on the manuscript.

The authors declare that there is no conflict of interest regarding the publication of this article.