Identifying sources of error and selecting quality indicators for point of care testing.

OBJECTIVES: Point of Care Testing (POCT) is a rapidly expanding area of clinical laboratory testing and quality assurance is an important area of focus. Quality indicators (QIs) are a quality management system tool that monitors aspects of the testing process to help meet the challenges associated with maintaining high quality patient safety given the growth in POCT. Alberta aims to formalize the development and use of QIs for POCT.
DESIGN: and
Methods: Potential QIs were identified by reviewing both the current standards and guidelines for QIs in POCT, and the research regarding quality and sources of error in POCT. Quality practices and potential sources of error in POCT were identified by: 1) a Canadian national survey on POCT, and 2) direct observation in two local POCT programs.
RESULTS: A proposed selection of QIs in POCT were identified by incorporating the results from these investigations, while considering the unique characteristics of POCT. These QIs monitor the preanalytical, analytical, and post-analytical phases of testing, and support processes.
CONCLUSIONS: As POCT volumes and test menu expands, QIs will be a vital tool in monitoring error and maintaining high quality of results. Adoption of formal QIs will support continuous quality improvement and improved patient care.

Introduction

Quality indicators (QIs) are part of quality assurance (QA) in clinical laboratory medicine. They are an important tool in quality management systems to monitor the testing process both within the core laboratory, and for Point of Care Testing (POCT). QIs are measures of performance to determine if a process is meeting required performance characteristics [1], and they are a valuable tool for monitoring error and improving quality of patient results [2,3]. They allow for processes to be measured objectively, so that areas for improvement can be appropriately identified, and subsequent improvement measures can be monitored.

The International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) established a working group to identify and evaluate QIs to assess all stages of the testing process. This working group identified 57 QIs to evaluate key processes, covering the preanalytical, analytical, and postanalytical phases [4,5]. These QIs were selected to be applicable to, and address steps in, the testing process in clinical laboratories. The testing process differs in POCT from the clinical laboratory, and within POCT the process can be highly variable depending on the test, technology, and setting [6].

There are no established criteria for the selection of QIs in POCT. The Clinical and Laboratory Standards Institute (CLSI) identifies QIs as a potential assessment tool in POCT, but does not provide a comprehensive set of required, recommended or suggested QIs for POCT programs [7]. A recent guidance document on POCT by the Canadian Society of Clinical Chemists [8] identifies the lack of clinical end user adherence to policies and procedures as the biggest risk to quality in POCT, and provides examples of QIs that could be used to monitor this deficiency. However, the selection of additional QIs that focus on other aspects of POCT are not discussed. Another guidance document on POCT from the American Association for Clinical Chemistry (AACC) provides a partial list of useful POCT performance indicators, but leaves it up to each individual institution to decide which specific indicators to include in its QA program [9].

Identification and selection of QIs in a laboratory will be informed by the vision, values, and goals of the organization [10]. QIs should be selected based on risk so that they can focus on processes that are error prone and/or that impact patient safety. The determination of the best QIs require the identification of key areas of error across the total testing process [2]. There are recommendations on the selection of QIs for laboratory organizations as a whole, including POCT examples, and include QIs to assess: patient misidentification, instrument lockouts at the time of testing, quality control (QC) failures, result reporting in the wrong medical record, operator competency and training, and equipment maintenance [3,7]. Selection of QIs in POCT also needs to take into consideration factors that impact use by non-laboratory trained staff. This is because they result in unique reasons that result in patient care errors. Major sources of POCT error include issues surrounding: 1) patient identification, 2) operator competence, including non-adherence to procedure, 3) uncontrolled reagents and equipment, 4) appropriate use of QC, 5) performance of external quality assurance/proficiency testing (EQA/PT), 6) proper follow-up of results, and 7) appropriate documentation [[11], [12], [13], [14]].

It is important to balance all the potential QIs with the capacity to respond and engage with the information QIs provide. This is because each QI that is unsuccessful will require root cause analysis. Selecting too many QIs can result in information overload, such that response to and improvement of processes becomes overwhelmed. This could result in many unintended consequences that decrease the quality of the POCT programs over time, as potentially meaningful and impactful outcomes become lost in the data. To this end, a limited number of QIs can be more valuable for monitoring error, and to inform and guide continuous improvement. Selecting five to eight QIs across the different phases of the testing process, including targeting sources of error identified above, will facilitate continuous quality improvement, not overwhelm POCT staff or end users, improve patient safety, and support accreditation requirements.

Alberta Health Services (AHS) is a province-wide health system responsible for delivering healthcare to people living in Alberta, Canada. AHS has a clinical policy that dictates that POCT testing is the responsibility of laboratory services, and this includes governance, oversite and quality (AHS policy PS-90). Laboratory Services is predominantly carried out by Alberta Precision Laboratories (APL) in Alberta, and the POCT department of APL oversees all POCT within AHS. The setting of POCT programs varies across Alberta, including operating theatres, intensive care units, general medicine hospital units, outpatient clinics, long term care facilities, patient homes, and onboard ambulances and aircrafts. Accordingly, the staff performing POCT are also diverse, and include respiratory therapists, nursing staff, and paramedics. A major challenge in POCT is ensuring continued high quality testing across each of the POCT programs and amongst all the POCT end-users to maintain patient safety [14,15]. APL is seeking to build formal QIs for all the POCT programs in Alberta to better monitor and improve quality for each of the current and future programs.

This study involved three phases (Fig. 1). The process for the identification of potential QIs was completed following a review and subsequent synthesis of: 1) existing international, national, and local standards and guidance documents governing POCT, and 2) current literature regarding QIs for POCT. A survey was also completed on quality in POCT programs across Canada, and direct observation was carried out with local operators using two POCT programs in Edmonton, Alberta to identify current sources of POCT error. Potential QIs for use in POCT to monitor the total testing process were selected for further development. Such QIs will be valuable to all areas seeking to build the quality of their POCT programs.

Fig. 1

Process for selection of potential quality indicators (QIs). Flowchart of steps in the identification of QIs to monitor quality in point of care testing (POCT).

Materials and methods

Selection and review of accreditation and guidance documents

Accreditation standards and guidance documents at the international, national and regional level were identified. The international documents identified for review included ISO 15189 and ISO 22870, and the IFCC’s Model of QIs created by the IFCC working group “Laboratory Errors and Patient Safety”. In addition, recent guidance documents for POCT from national clinical chemistry societies in Canada (CSCC) and the United States (AACC) were reviewed. Local accreditation standards applicable to quality and QIs for POCT were also included. In Alberta, diagnostic laboratories are accredited by the College of Physicians and Surgeons of Alberta (CPSA).

Literature review of QI practices in POCT

Literature published between 2000 and 2019 regarding the application of quality monitoring practices for POCT, and QIs for POCT was reviewed. Key search terms for POCT included: point of care testing, POCT, and point-of-care. Key search terms for quality indicator included: quality indicator, quality assurance, performance indicator, inspection, and checklist. Searches were performed in Pubmed by combining key search terms for POCT and key search terms for quality indicators. Identified articles with combined search strategy were reviewed.

National survey of quality practices in POCT

A survey was developed to understand quality practices in POCT based on considerations of guidance documents, accreditation documents, published literature, and direct experience with POCT. Topics of focus included general quality practices (6 questions), quality practices for glucose meters (13 questions), and quality practices for the Abbott i-STAT device (16 questions). The full survey is provided in Supplemental Data 1. The survey was distributed via SurveyMonkey [16] to 34 clinical/medical biochemists in 10 Canadian provinces that oversee POCT programs in their local institution and/or a large healthcare network. There were 18 individuals from 7 provinces that completed the survey over a span of two weeks. One additional individual from an eighth province responded by email, but did not complete the survey itself. Responses were automatically recorded and tallied by SurveyMonkey, and subsequently analyzed in Microsoft Excel (version 2013).

Direct observation of POCT programs

In total, eighteen direct observations of clinical staff using the Roche ACCU-CHEK Inform II glucose meter were completed across three different urban centers in Edmonton, Alberta. Observations took place between the hours of 8:00 a.m. and 5:00 p.m. in the following clinical areas: adult emergency, pediatric emergency, post-operative recovery, adult intensive care unit, dialysis, family medicine inpatient unit, and a regional diabetes outpatient program.

There was a total of six observations that were completed for the i-STAT devices from three centers and two mobile units in Edmonton, Alberta. Observations took place between the hours of 8:00 a.m. and 5:00 p.m. in the following clinical areas and mobile units: pediatric extracorporeal membrane oxygenation unit, community respiratory therapy, diagnostic imaging, cardiovascular operating room, stroke ambulance (mobile unit), and pediatric transport team (mobile unit).

A checklist (Supplemental Data 2) was used to assess and record the results of each direct observation, and was developed for each device type. The checklist was based on the POCT program requirements specified by the APL POCT department for Inform II glucose meters or for i-STAT devices, as applicable. Each checklist also contained a “Comments” section where the observer could note any issues/observations that were not covered by the checklist questions.

Ethics review

An ethics review from the University of Albert Human Research Ethics Board was not required for this project, since it was a clinical laboratory quality improvement initiative.

Results

Accreditation standards and guidance documents for quality and QIs in POCT

The international standard ISO 22870:2016 “Point-of-Care Testing (POCT) - Requirements for Quality and Competence” outlines POCT requirements to be used in conjunction with ISO 15189 (International Organization for Standardization, 2012; International Organization for Standardization, 2016). ISO 22870:2016 states that POCT quality objectives need to be established and measurable, and that the organization should identify, collect and analyze data to evaluate where continual quality improvements can be made. The standard specifies that analysis of data should provide information about: a) healthcare provider/patient/customer satisfaction, b) conformity to POCT requirements, and c) characteristics and trends of POCT. POCT operator performance should also be monitored as part of a QA program [18].

The IFCC working group “Laboratory Errors and Patient Safety” Model of Quality Indicators (MQIs), updated in 2016 ​at the Consensus Conference “Harmonization of Quality Indicators in Laboratory Medicine: Two years later” has identified 57 QIs, separated into three categories: i) key processes, ii) outcome measures, and iii) support processes [5]. Cantero et al. sought to compare the error rate between POCT and the clinical laboratory for these QIs [19]. This group identified five QIs from the IFCC working group that were easily adaptable to POCT: i) errors in patient identification, ii) POCT samples without results, iii) samples with insufficient volume, iv) unacceptable results in EQA/PT, and v) %CV above target for IQC per year. The IFCC QIs examined for POCT use in this study were restricted to the preanalytical and analytical phase.

A recent CSCC position statement provided recommendations on POCT, and identified that POCT should have oversight from an accredited clinical laboratory [8]. It also provides guidance on quality management, identifying that there should be an appropriate QA program that, at minimum, includes: regular assessment of QC material, an EQA scheme, and recurring comparison of results to an accredited laboratory [8]. The AACC has also provided a guidance document on management of POCT that includes a list of potential QIs that may be monitored [9]. This document states that each program should track specific POCT indicators, and may include QIs for operator compliance, staff training and competency, and common sources of error. This document stipulates that QIs selected will depend on the institution and change over time.

In Alberta, diagnostic laboratories are accredited by the CPSA. The Laboratory Accreditation Program of the CPSA is based on relevant ISO standards, accepted best practices, CLSI guidelines, and provincial and national legislation [20]. The relevant ISO standards for POCT are ISO 22870:2016 and ISO 15189:2012 [17,18]. CPSA has comprehensive POCT standards, such as the requirement for a POCT quality management system to be in place, clear documentation and processes supporting that the laboratory is responsible for monitoring POCT accuracy and quality, and processes to be in place for comprehensive continual improvement programs. The formal use of QIs within the POCT programs in Alberta would support meeting multiple CPSA requirements.

The breadth of current standards and guidance documents provide direction to develop and ensure quality practices in POCT. A summary of pertinent sections of these documents is summarized in Table 1. As the scope and use of POCT flourishes, the need for quality management becomes ever more imperative. Quality management requirements include quality assurance plans, and the monitoring of processes. Strong QA plans and processes are challenging due to the variation in POCT devices, uses, settings and users. The quality management system needs to incorporate these factors.

Table 1

Summary of key quality and QI information from accreditation standards and guidance documents.

Document/Resource	Specific direction relevant to QIs for POCT
ISO 22870: Point-of-care testing (POCT) - Requirements for quality and competence [18]	• 4.2.4 a: POCT quality objectives are established and measurable • 4.9.3: The organization shall determine, collect, and analyze appropriate data to evaluate where continual improvement of the effectiveness of the QMS system can be made • 4.9.4: Analysis of data shall provide info about: a) healthcare provider/patient/customer satisfaction, b) conformity to POCT requirements, c) characteristics and trends of POCT, e) suppliers • 5.1.4 e: POCT operator performance shall be monitored as part of the quality assurance program
IFCC Model of Quality Indicators [21]	• Key Processeso Preanalytical: Misidentification errors, test transcription errors, incorrect sample type o Analytical: Unacceptable performances in IQC, unacceptable performances in EAQ-PT schemes, data transcription errors o Post-analytical: incorrect laboratory reports • Outcome Measureso Sample recollection, amended results, safety • Support Processeso Employee competence, client relationships
Point-of-care testing: A position statement from the Canadian Society of Clinical Chemists [8]	• Oversight of POCT program by an accredited laboratory, with quality assurance requirements established • Quality assurance program including QC and EQA-PT • Initial and ongoing competency programs • Compliance with accreditation standards, which may be derived from ISO 22870
AACC Guidance Document on Management of Point-of-Care Testing [9]	• POCT programs maintain ongoing-training and monitoring of test-performance • Each program should track specific indicators (QIs) to identify areas of improvement • Common POCT QIs include patient identification, hemolysis rates, device maintenance, trouble shooting QC failures, error flags with results, documentation of results • Laboratories participate in EQA-PT for POCT
CPSA Laboratory Accreditation Program [20]	• Appropriate measures are in place to monitor accuracy and quality of POCT • Requirement for comprehensive QMS for POCT and includes:o Monitoring of effectiveness of processes o Criteria for monitoring processes o Monitoring, measurement, evaluation, and improvement processes to demonstrate conformity and effectiveness • Comprehensive continuous improvement program exists

Key Processes

Outcome Measures

Support Processes

Requirement for comprehensive QMS for POCT and includes:

The literature review identified twenty six articles discussing quality assurance practices, encompassing QI practices for POCT programs. Identified articles included publications discussing sources of error in POCT, and practices to monitor this error, even if QIs were not directly identified. The need for of QIs or monitoring of specified measures (for example checklists) in POCT was specifically discussed in eight articles, with discussion of implementation or use of QI or quality assurance monitoring limited to four articles.. Publications with a focus on a particular device type were not reviewed. There is limited evidence of widespread use of QIs in POCT programs that target aspects across the total testing process. However, many sites have identified sources of error in POCT from audits [14,22,23], and a few specific sites have implemented QIs in their POCT programs as part of a quality management system. For example, a formal continuous quality improvement program was started in 1999 ​at Mount Sinai Medical Center for 23 POCT programs [23]. This program was instituted by a multidisciplinary committee that had laboratory leadership, and included a monthly review of general and program-specific QIs through review of a site visit checklist. General indicators included the number of unidentifiable samples, QC materials with correct label, lot and expiration use, reagents with correct expiration use, documentation of QC results, QC statistics, and performance and documentation of routine maintenance [23]. Of note, these QIs were well received by the POCT operators.

The Massachusetts General Hospital also introduced a site inspection checklist in 2010, and it demonstrated significant improvements in regulatory compliance [24]. This checklist was rotated between sites approximately once per year, and addressed regulatory requirements and safety issues, such as environment, training/competencies, QC/proficiency testing, specimen collection/identification, and result documentation. Following inspections, sites received a report that identified deficiencies and outlined remedial action. Over the course of 8 years, mean citations per site decreased from 3.17 to 0.27, and the number of sites without citations increased from year to year [22]. Deficiencies in documentation of operator competency, patient test results, procedure review by site director, result review for transcription error, and QC documentation accounted for over 40% of citations [22,24].

Widespread establishment and use of QIs in POCT is limited compared to adoption and use of QIs in the central laboratory. The work by the IFCC [21] on establishing the breadth of important QIs for the laboratory is very important and has laid the ground work for POCT. Although interest has begun in incorporating QIs for POCT programs, a more definitive approach to defining, establishing and incorporating QIs remains.

An increase in understanding of POCT program practices across Canada is required in order to help improve local, regional and national quality practices. Out of 34 clinical/medical biochemists that received the survey, a total of 19 individuals from 8 Canadian provinces provided responses (i.e. response rate of 56%). This included respondents from British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, New Brunswick, Nova Scotia, and Newfoundland. Respondents identified as clinical biochemists (15), POCT coordinator (1), and positions of clinical leadership (3). Twelve out of 19 respondents indicated that they oversee POCT in a large health care network that encompasses part, if not all, of their respective province and includes both urban and rural areas. The remaining 7 respondents indicated that they oversee POCT at a single centre or a multi-centre healthcare network in an urban setting. The general section and glucose meter section of the survey were completed by 18 respondents, while 13 of the 18 respondents also completed the i-STAT section of the survey (remaining 5 had no i-STAT program within their geographical oversight). One respondent indicated that laboratory services does not oversee POCT in their province, and was thus not considered further in the survey result analysis.

Most survey respondents felt that end-user training was effective for each of the two devices (72% for glucose meter and 75% for i-STAT). Training is an important component of quality in POCT, since training and operator competency have been identified as the most challenging issues in POCT [11,13,25,26]. Significant resources are directed towards end-user training and competency, and this survey supports that it has been a focus of POCT departments across Canada. However, the survey outlines that there remain challenges in training, including how to administer training, design of training programs, and maintaining and evaluating competency.

Survey results demonstrated that performance evaluation of glucose meters and i-STATs were similar across provinces, with liquid QC, EQA/PT schemes and results comparison to the central laboratory commonly used (Fig. 2A and 2B). Of note, frequency of evaluation was not elicited. Sources of error in POCT were identified in the preanalytical, analytical and post-analytical phases for both the glucose meter and i-STAT programs (Fig. 2C and 2D). Patient identification (44% of respondents) and sample collection (50% of respondents) were identified as the most common sources of preanalytical error for glucose meters. Sample collection was identified as the most common preanalytical error for the i-STAT (31% of respondents). The most common analytical error identified for glucose meter use was categorized as sample issues (33% of respondents). The most common analytical error identified by respondents for the i-STAT was sample loading (38%). In regards to post-analytical errors for glucose meter use, 56% of respondents reported that result reporting or documentation was the most common source of error. The most common post-analytical error for the i-STAT was result reporting (46% of respondents). When asked the most common overall source of error, 56% of respondents indicated that preanalytical errors predominated for glucose meters, while 46% of respondents identified analytical errors for the i-STAT meters.

Fig. 2

Canadian National Survey on POCT. Methods for evaluation of the performance of the glucose meter (A) and i-STAT (B) in POCT programs. Percentage of responses from national survey shown. Sources of errors identified by survey respondents (percentage of responses) in the preanalytic, analytic, and post-analytic phases of the total testing process for glucose meter (C) and i-STAT (D). Total number of respondents for glucose meter ​= ​18, i-STAT ​= ​13.

This survey provides insight into POCT programs in locations across Canada, and identifies errors most encountered with two common POCT programs. Incorporation of this information into local POCT departments would be valuable as areas begin to develop QIs to improve the quality of their current programs.

Direct observation of POCT

Direct observations of clinical staff using the Inform II glucose meter across a variety of clinical areas and healthcare centers revealed that standard operating procedures are generally followed, including proper storage of test strips and QC material, correct entry of operator and patient information into the meter, and appropriate use of personal protective equipment (PPE). Errors did occur though, and deviation from procedures were separated into preanalytical, analytical and post-analytical errors. Preanalytical errors identified included: improper labeling of QC material (56% of observations), failure to follow patient identification procedures (72% of observations), and failure to perform proper hand hygiene (39% of observations). Improper sample collection was also observed, with 44% of operators failing to allow the sampling site to dry after disinfecting and 33% of operators failing to wipe away the first drop of blood. In the analytical phase, the only error identified was not testing the second drop of blood (33% of observations), which is linked to failing to wipe away the first blood drop. Post-analytical procedure errors included cleaning of the meter after each patient use (neglected in 56% of observations), and failure to place the glucose meter on the base after testing to allow transmission of results to the electronic medical record (33% of observations).

Similarly, direct observations of clinical staff using the i-STAT device showed that several procedure steps were completed appropriately, including: appropriate daily electronic QC with an external simulator, proper dating and use of cartridges, and reporting and follow up of results. Specific concerns observed in more than location included failure to label sample containers (50% of observations), failure to perform hand hygiene (33% of observations), and failure to mix samples (33% of observations).

Direct observation of these two separate POCT programs identified potential sources of error that are occurring in day-to-day use of the devices. This information will aid in the selection of QIs to improve how local POCT programs are implemented, monitored and supported.

Selection of quality indicators

It is important to consider if QIs will be POCT program specific or applicable across programs. This is particularly important for organizations overseeing multiple programs across multiple healthcare sites, especially if they span significant geography that can impact the frequency for monitoring and follow up. In Alberta, there are over 20 different POCT devices that are supported, and many devices have regional/site-specific programs. These devices range from large analyzers with connectivity to the electronic medical record (e.g. blood gas analyzers), to hand held analyzers that use individual strips or cartridges (e.g. glucose meters and i-STATs), to manual testing (e.g. single use lateral flow urine hCG). As demonstrated in direct observations outlined above (Section 3.4), along with results from local site inspection checklists and monthly review of QIs in other jurisdictions [22,23], many sources of error are not specific to a single POCT device or program.

The identification of potential QIs is multifactorial. They can be developed from the insight obtained on sources of error in POCT by combining the outcomes described above, including the information from the previous quality-focused POCT literature, the survey results of general and device-specific POCT quality practices across Canada, and direct observation of POCT operators for two POCT programs in a healthcare region. Potential QIs in this study were evaluated, as outlined in Table 2, based on details described in Sections 1, 2.4: inclusion in guidelines, meeting accreditation requirements, identification in existing literature, being reported by survey respondents, and flagging in direct observations. The QIs that fulfilled each of these evaluation areas could be considered to be more relevant QIs. Selected QIs from this list were subsequently further classified by testing phase and associated processes, as outlined in Table 3, and details on proposed methods of data collection, calculation/evaluation processes, and frequency of analysis are provided.

Table 2

Evaluation of prospective QIs.

Quality indicator
	Inclusion in guidelinesa (IFCC/AACC/CLSI) [7,9,21]	Meet accreditation requirements	Identified/discussed in literature as a QI	Reported in study survey	Flagged in direct observation
PATIENT MISIDENTIFICATION AT TIME OF POCT TESTING	AACC	Yes	[27]	Yes	Yesd
	CLSI POCT04
REQUESTS WITH ERRONEOUS DATA ENTERED BY POCT USER	IFCC	Yes	None	Yes	Yes
NUMBER OF INSTRUMENT LOCKOUTS AT TIME OF TESTING	CLSI POCT04	Yes	[12]	No	No
SAMPLE COLLECTION ERRORS	IFCC	Yes	[12,27]	Yes	Yes
REJECTED SAMPLES	IFCCb	Yes		No
UNIDENTIFIABLE SAMPLES		Yes	[23]	No	Yes
HEMOLYSED SAMPLES	IFCC	Yes	[12]	No	No
	AACC
UNACCEPTABLE PERFORMANCE: QC RESULTS OUTSIDE OF DEFINED LIMITS	IFCC	Yes	[12,23]	Yesc	No
	CLSI POCT04
UNACCEPTABLE PERFORMANCE ON EQA-PT SCHEME	IFCC	Yes	[23,24]	Yesc	No
	AACC
INCORRECT LABORATORY REPORTS: PATIENT RESULTS POSTED TO THE WRONG MEDICAL RECORD	CLSI POCT04	Yes	None	Yes	No
DATA TRANSCRIPTION ERRORS	IFCC	Yes	[27]	Yesa	No
	AACC
REPORTS REQUIRING AMENDMENT	IFCC	Yes	None	No	No
	AACC
DOCUMENTATION OF RESULTS IN MEDICAL RECORD	AACC	Yes	None	Yes	No
TEST TURN AROUND TIME	IFCC	Yes	[12]	Yes	No
	AACC
AUDIT PERFORMANCE/SITE INSPECTION CHECKLIST PERFORMANCE	CLSI POCT04	Yes	[22,24,28]	Yesc	No
PERSONNEL LACKING DOCUMENTATION OF TRAINING OR COMPETENCY ASSESSMENT	IFCCbCLSI POCT04	Yes	[12,26,29]	No	No
EQUIPMENT RECORDS LACKING DOCUMENTATION OF APPROPRIATE MAINTENANCE	CLSI POCT04	Yes	None	Yes	Yes
	AACC
INSTRUMENT ROBUSTNESS: NUMBER OF CALLS FOR INSTRUMENT ERROR, INSTRUMENTS REQUIRING REPLACEMENT		Yes	[29]	No	No
INCIDENT/ADVERSE EVENTS CONCERNING SAFETY OF STAFF	IFCC	Yes	None	No	No
USER SATISFACTION	IFCC	Yes	[30]	No	No
	AACC

Specific wording of QI in guidelines may vary, and guideline listed if comparable QI identified in guideline.

Separate QIs listed by IFCC for specific reasons for rejected sample, including microbiological contamination and hemolysis.

QC, EQA/PT, and audits identified as ways to monitor quality in survey.

Failure to follow proper patient identification procedures.

Table 3

Potential QIs that can monitor POCT error.

Testing Phase/Process	QI	Example of Data Collection Methodsa	Examples of Calculations and Data Evaluation Processesb	Frequency of Analysis
Preanalytical	Patient misidentification at the time of POCT	Report with missing or incorrect patient information from POCT middleware	a) Count instances of missing or incorrect patient information per location	Quarterly
			b) Count total number of tests per location
			c) Calculate percentage
Preanalytical	Instrument lockouts at the time POCT	Report with lockout flag from POCT middleware	a) Count number of lockouts per location	Quarterly
			b) Count total number of tests per location
			c) Calculate percentage
Analytical	Unacceptable performance by internal quality control (IQC)	Report with IQC data and charts from POCT middleware	a) Count number of IQC results outside defined limits per device	Quarterly
			b) Count total number of IQC results per device
			c) Calculate percentage
Analytical	Unacceptable performance in EQA-PT schemes	Review of EQA-PT results by POCT staff	a) Count number of unacceptable performances in EQA schemes per deviceb) Count total number of performances in EQA schemes per devicec) Calculate percentage	Yearly
Post-analytical	Corrected reports	Track corrections by POCT staff within worksheet	a) Count number of corrected reports after release per site	Quarterly
			b) Count total number of released reports per site
			c) Calculate percentage
Total testing process	Audit performance	Track audit performance within worksheet	a) Count number of audit check failures per site	Yearly, and year over year comparison
			b) Count total number of audit checks per site
			c) Calculate percentage
Support process	Instrument robustness	Review call log for instrument error	a) Count number of calls for instrument error per device	Quarterly (instrument error)
			b) Count total number of tests per device
			c) Calculate percentage
		Track instruments put out of service within worksheet	a) Count number of devices replaced	Yearly (device replacement)
			b) Count total number of devices in use
			c) Calculate percentage
Support process	User satisfaction	Survey designed to assess satisfaction of POCT users	Complete per POCT program	Yearly

Data reviewed by POCT staff.

Calculations adapted from IFCC Quality Indicators Project [21].

Patient identification would be a valuable preanalytical QI, as it was identified by previous studies, the survey and direct observation to be an important source of error in POCT. Another potential QI in the preanalytical phase to consider is the number of instrument lockouts. This will only be applicable to POCT programs with this function, thus it excludes manual tests. This function could prevent device usage if QC testing protocols are not followed or if a user has not maintained competency.

In the analytical phase, QC and EQA/PT performance may be monitored. If there is an increase in the number of QC failures on a specific unit, it may signal that there is a problem with the instrument/method at the location in question, or that there is an operational problem by a user(s). It is important to know that even for a specific QI, there may be more than one cause and more than one reason for the unsatisfactory performance. A post-analytical QI could monitor frequency of corrected reports for a particular device, such as a blood gas analyzers. This has a direct impact on patient safety and was noted to be a common issue on the Canadian survey, often due to errors in entering patient demographics or sample type.

Additional QIs that are recommended are in the total testing process and as support processes to the POCT department. A proposed QI for the total testing process is to monitor audit performance. It would be a value-add to perform yearly audits, as well as to compare performance from year to year and discuss with the POCT end user lead for the unit. This has been previously reported at one medical center to improve audit performance (Khanna et al., 2018). QI support processes are those that do not fall directly into preanalytical, analytical, or post-analytical phases of testing. This could include: 1) a process to objectively measure causes for instrument replacement, and 2) evaluation of user satisfaction. Instrument downtime is regularly assessed in central laboratories, and it is known to impact workflow and turn-around-time. In POCT, downtime from instrument error can also significantly impact end user workflows and patient care, especially if POCT users frequently encounter errors that lead to a decrease in confidence of the results by the end users. Data from this type of QI could be combined within and across units to increase awareness on overall instrument performance. User satisfaction is an important feature of quality management, and an end user survey can be used to provide feedback of POCT programs [1]. As POCT is overseen by the laboratory, but performed by non-laboratory staff, concerns and issues are not always easily known. An end user survey supports communication and a process to address areas that are deficient.

Discussion

QIs are a powerful tool to improve current and forthcoming POCT programs, as they can identify areas prone to error and program gaps. Formal QIs to monitor and improve the quality of each of the POCT programs in Alberta are currently being explored. Identification, development, and implementation of potential QIs is a multi-phase process, and includes review of existing standards and guidelines, in concert with current quality practices in POCT. The national survey that was completed assessed current POCT practices across Canada, which complimented the results from the direct observation of two local POCT programs to aid in the identification of potential sources of error, and ultimately the required QIs within and between POCT programs.

Selecting QIs that can be applied across a range of POCT programs may be useful in a large healthcare organization. As identified through literature, the national survey, and direct observation, specific errors may be unique to individual POCT programs, but many sources of error are similar across POCT programs. Potential QIs could monitor processes in the: 1) preanalytical phase, including patient misidentification, 2) analytical phase, including performance in QC and EQA/PT schemes, and 3) support processes, including user satisfaction. Development of these potential QIs will include a specific definition of the QI, identification of how data will be collected and how the QI will be calculated, and how the QI will be evaluated. Adoption of formal QIs will support quality management requirements for POCT, promote continuous quality improvement, promote end-user satisfaction, and improve patient care.

This study aimed to identify QIs that can be applied to a range of POCT programs. Accordingly, selected QIs may not identify errors that only affect a specific POCT program. For example, there is no lockout feature on manual tests, and a QI to monitor user lockout will not be applicable to these tests. Alternatively, when testing with blood gas analyzers, the amount of time a sample sits before testing can affect results; none of the selected QIs monitor time from collection to testing, thus this step would not be captured. In addition, this study aimed to investigate QIs to monitor quality in POCT for a large healthcare organization. However, adoption of formal QIs for POCT is important for any organization or centre that has POCT, whether that be a single POCT program in a single unit of a hospital, or a large organization that oversees multiple programs across multiple institutions.

Once the selection of appropriate QIs have been identified, systematic development of the full QI can proceed. This process should be driven by the POCT department, and include consultation with POCT stakeholders to promote uptake and compliance. Development of a QI includes: 1) establishment of a clear definition of the QI, 2) specification of the objective and rationale of the QI, and 3) determination of the methodology to be employed to measure the QI. It is also important to consider the limitations of the QI, and to specifically acknowledge what the QI may not be able to measure. This may include what errors may be missed due to the method of data collection. Before implementation, the POCT team must also characterize what acceptable performance means, frequency of data collection, how data will be presented, and what action will be taken if a QI is outside of acceptable limits. Finally, QIs are not static and require regular review, revision, and potentially replacement. When performance of a QI remains stable and within limits, it may be retired and replaced with a new QI to provide opportunity for continual improvement.

Conclusion

In the clinical laboratory, the use of QIs is well established. This is illustrated by the focus on developing harmonized QIs to monitor quality in laboratory testing by the IFCC working group “Laboratory Errors and Patient Safety”. This working group aims to develop a harmonized QI program to monitor clinical laboratory processes and decrease error [21]. By contrast, the use of QIs in POCT is not well established. Here, a selection of potential QIs to monitor quality in POCT for a large healthcare organization are identified. This is the first step in development of a formal QI program for POCT. The use of QIs in the POCT setting is becoming more important, as test menus, volumes and device availability expands, alongside the number of POCT settings and users. Adoption of formal QIs will enhance the quality of POCT as it continues to advance.

Author statement

MB conducted data analysis and interpreted the data, wrote the manuscript, and revised the manuscript. AF conceived the study, assisted with data analysis and interpretation, and revised the manuscript. BH and KR developed and administered the national survey, designed audit checklists and completed the direct observations, and collected and analyzed data for both the survey and the direct observations. AAV conceived the study and the manuscript, assisted with data analysis and interpretation, and revised the manuscript. All authors approved the submitted manuscript and provided critical feedback.

This quality improvement initiative did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Funding

This quality improvement initiative did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors have no conflict of interest to declare for this manuscript.