Economic Evaluation of Total Laboratory Automation in the Clinical Laboratory of a Tertiary Care Hospital.

BACKGROUND: Total laboratory automation (TLA) is an innovation in laboratory technology; however, the high up-front costs restrict its widespread adoption. To examine whether the capital investment for TLA is worthwhile, we analyzed its clinical- and cost-effectiveness for the expected payback period.
METHODS: Clinical chemistry tests and immunoassays performed in the clinical laboratory of a tertiary care hospital were divided into a post-TLA group, including 1,182,419 tests performed during December 2019, and a pre-TLA group, including 1,151,501 tests performed during December 2018. Laboratory information system data were used to measure clinical effectiveness, and depreciation data were used to calculate TLA costs.
RESULTS: Laboratory performance improved after TLA adoption in all four key performance indicators: mean turn-around time (TAT), representing the timeliness of result reporting, decreased by 6.1%; the 99th percentile of TAT, representing the outlier rate, decreased by 13.3%; the TAT CV, representing predictability, decreased by 70.0%; and weighted tube touch moment (wTTM), representing staff safety, improved by 77.6%. Based on these effectiveness results, economic evaluation was performed using two approaches. First, the incremental cost-effectiveness ratio and wTTM were used as the most cost-effective performance indicators. Second, the expected payback period was calculated. Considering only staff cost reduction, it was anticipated that 4.75 yrs would be needed to payback the initial investment.
CONCLUSIONS: TLA can significantly enhance laboratory performance, has a relatively quick payback period, and can reduce total hospital expenses in the long term. Therefore, the capital investment for TLA adoption is considered to be worthwhile.

INTRODUCTION

Laboratory automation systems were developed to improve laboratory performance by reducing the number of repetitive tasks that could lead to human errors and standardizing the total testing process. There are a variety of laboratory automation systems, including pre-analytical modules, task-targeted automation (TTA), and total laboratory automation (TLA) systems [1]. The TLA system, which was first introduced in the early 1980s at Kochi Medical School, Kochi, Japan, performs both pre-analytic and post-analytic functions using analyzers that are directly interfaced to the automation system via a conveyor belt line [2, 3]. Although many laboratories have some level of automation (with approximately 4,000 laboratory automation systems installed worldwide as of 2016), few have entirely incorporated TLA systems [4].

Despite its advantages, the greatest restriction to the adoption of TLA systems is their high up-front costs [5]. Some economic studies have been conducted on the return on investment (ROI) and payback period of the up-front costs of installing TLA systems, their effect on operational costs as a result of contract consolidation and staff cost reduction, and the consumer price index-reflected ROI based on cash flow [6-9]. However, these studies generally did not involve fully automated TLA systems and were largely based on general cost analyses rather than cost-effectiveness analyses (CEA).

The clinical laboratory of Severance Hospital, Yonsei University, Seoul, Korea recently implemented a 78 m long TLA system, which is one of the largest systems currently used in Asia. Prior to adoption of the TLA system, the laboratory was operated using subtotal automation. Therefore, our hospital represents an adequate forum to clearly identify the clinical effectiveness and conduct an economic evaluation for TLA. The aim of this study was to verify whether the capital investment for TLA adoption is worthwhile from both clinical and operational perspectives. We compared the clinical effectiveness of TLA with that of subtotal automation and conducted an economic evaluation by determining the incremental cost-effectiveness ratio (ICER) and expected payback period.

MATERIALS AND METHODS

Study design

A retrospective study was conducted in the Department of Laboratory Medicine of Severance Hospital with 2,500 beds. This clinical laboratory generally performs over 1 million routine and emergency clinical chemistry tests and immunoassays per month. At the beginning of the study period, the laboratory processes were operated using the Hitachi Pre-Analytical Process Automation System (Hitachi Ltd., Tokyo, Japan), a subtotal automation system. This set-up included two islands of unidirectional automation systems with the same configurations, excluding post-analytic modules. Subsequently, the laboratory implemented the fully automated Aptio Automation (Siemens Healthcare Diagnostics Inc., Tarrytown, NY, USA) system, resulting in stabilization by November 2019 (Supplemental Data Table S1 and Figure S1).

Only data from clinical chemistry tests and immunoassays performed during the study period were included in this analysis. The data were compared between the post-TLA samples collected in December 2019 and pre-TLA samples collected in December 2018 to avoid confounding variables related to seasonal variation. The study procedures were reviewed and approved by the Institutional Review Board at Severance Hospital (Y-2019-0196).

Study variables

Effectiveness was determined by four key performance indicators (KPI), including three turnaround time (TAT)-related variables. TAT variables were considered to be important KPIs since TAT is a traditional quantitative outcome and is one of the most frequently used indicators for evaluating laboratory performance [10]. TAT was defined from the time of sample registration with barcode reading to the time of reporting the test results to the laboratory information system (LIS). To analyze the change in TAT after the TLA adoption, we evaluated the LIS data by measuring three variables: mean TAT, 99th percentile TAT, and TAT CV, corresponding to measures of “timeliness,” “outlier rate,” and “predictability,” respectively. To identify TLA-specific benefits, we considered the tube touch moment (TTM) indicator, which is defined as the average number of manual operations needed to process a sample from registration to archiving [11]. We used weighted TTM (wTTM) as the fourth KPI and weighted ratios for each manual procedure to precisely evaluate the impact of each step according to the three criteria: duration, risk, and clinical importance. Thus, wTTM is a measure of “staff safety.” The four KPIs were used to measure the incremental effectiveness (ΔE), which is defined as the difference in effectiveness between two interventions (pre- and post-TLA). For the cost variable, the annual incremental cost (ΔC) was determined, which is defined as the cost difference between pre- and post-TLA.

Data sources

Effectiveness data

LIS data from the pre- and post-TLA periods were collected to examine the effectiveness of TLA systems. The collected data included the test code, test name, and completion time of each test. The pre-TLA group included data from 1,151,501 tests and the post-TLA group included data from 1,182,419 tests (Supplemental Data Table S2). LIS data were then compared for the TAT-related variables.

Predictability is an important indicator for laboratory automation, and its reduction is typically studied according to variance [12]. The CV describes the distribution of results in terms of how many results are close to the mean.

To analyze wTTM, workflow analysis was additionally performed. Unlike the LIS data, workflow analysis was conducted from sample registration to archiving to create a sample processing flow chart. Following this, the weighted ratio for each processing step (1–4) was measured through qualitative data collection in the form of staff interviews, including one experienced laboratory staff member and two TLA specialists (Table 1).

Table 1

Criteria of weighted ratios

Weighted ratio	Descriptions	Criteria*
		Duration[†]	Risk[‡]	Clinical importance[§]
1	Low-level risk for all criteria (e.g., bulk input)	L	L	L
2	Same level of risk and importance as Level 1, but longer processing time (e.g., decapping, transporting)	M	L	L
3	Higher risk or importance than Level 2 (e.g., manual centrifugation)	M	M	M
4	If any of the three criteria are considered “high” (e.g., result verification, decapped sample handling)	H	H	H

*Represented as H (High), M (Moderate), L (Low), criterion levels were determined from in-depth staff interviews in the laboratory; †Duration: the average time for completion of manual actions; ‡Risk: manual actions that affect staff safety, such as exposure to samples during processing; §Clinical importance: manual actions considered critical for producing accurate clinical results.

Cost data

Economic evaluation represents the comparative analysis of alternative choices of action in terms of both costs and consequences [13]. CEA is the most frequently used economic evaluation method in healthcare [14]. The annual ΔC data were collected from the hospital to measure TLA depreciation. The cost of analyzers was excluded from the analysis, whereas the costs of pre-analytic, post-analytic, and track modules were included to minimize the cost intervention. Since the TLA cost data were bundled, it was not possible to separately analyze the costs associated with each module. Therefore, the TLA cost was allocated according to different analytic phases and analytic types based on the percentage of manual activity (Supplemental Data Table S3). The monetary unit for cost data was converted to 1 US dollar (USD) from 1,186.14 Korean Won (KRW), according to the average currency exchange rate in 2019 [7, 15].

We also calculated the expected payback period for the TLA system based on additional data collected and aggregated from the health data system. The average salaries (for 2013) of medical technicians working at the five largest hospitals in Korea were collected, and annual wage growth rates were reflected to calculate the average salary for the following years. In addition, the average numbers of patients in tertiary care hospitals in Korea from 2014 to 2018 were used to predict the number of tests expected per day for the upcoming five years from 2020 to 2024. The average productivity of TLA systems was obtained from previous studies [6, 9]. The number of staff members required was estimated by dividing the number of tests into the productivity level.

Statistical analysis

CEA method was used by calculating the ICER, which is the ratio of the difference in costs and the difference in effectiveness outcomes based on the following formula: (Incremental Cost)/(Incremental Effectiveness)=ΔC/ΔE. By comparing the ICER values among more than two groups, a more cost-effective alternative can be determined. To minimize the uncertainty of results, we calculated the expected payback period under three different scenarios (conservative, normal, aggressive) as a sensitivity analysis.

RESULTS

TLA reduced the mean TAT of immunoassays by 41.2 min, representing a greater reduction than that of the clinical chemistry tests (1.8 min) (Table 2). TLA decreased the 99th percentile TAT of immunoassays by 200.6 min, which was also greater than the decrease of clinical chemistry tests (26.0 min). Reduction in the 99th percentile TAT indicates the consistency of sample processing speed, which is crucial for managing most samples in a timely manner by reducing the proportion of outliers. This result showed that TLA significantly reduced outlier rates, indicating an improvement in overall laboratory performance.

Table 2

Cost-effectiveness analyses using four KPIs

KPIs	Pre-TLA	Post-TLA	∣∆E∣		∆C (1,000 USD)	ICER
			NU	%		NU	%
TAT-Mean (min)	73.5	69.0	4.5	6.1	356.2	-	58.4
CC	61.4	59.6	1.8	2.9	200.5	111.4	-
IM	212.4	171.2	41.2	19.4	155.7	3.8	-
99th percentile TAT (min)	262.6	227.7	34.9	13.3	356.2	-	26.8
CC	155.4	129.4	26.0	16.8	200.5	7.7	-
IM	1,493.1	1,292.5	200.6	13.4	155.7	0.8	-
TAT-CV	5.5	1.7	3.9	70.0	356.2	-	5.1
CC	8.2	1.8	6.4	78.1	200.5	31.3	-
IM	1.3	0.7	0.6	46.3	155.7	259.5	-
wTTM (NMT)	58.1	13.0	45.1	77.6	356.2	-	4.6
Pre-Analysis	21.0	8.0	13.0	61.9	128.7	9.9	-
Analysis	13.1	4.0	9.1	69.5	80.3	8.8	-
Post-Analysis	24.0	1.0	23.0	95.8	147.1	6.4	-

Abbreviations: KPI, key performance indicator; ΔE, incremental effectiveness; ΔC, incremental cost; NU, natural units; TAT, turnaround time; wTTM, weighted tube touch moment; NMT, number of manual touches; ICER, incremental cost-effectiveness ratio; TLA, total laboratory automation; CC, clinical chemistry; IM, immunoassay; USD, US dollar.

The TAT CV for clinical chemistry tests decreased by 78.1%, representing greater improvement than that for the immunoassays (46.3%). TLA reduced the wTTM from 58.1 to 13.0, representing a 77.6% reduction. Given that a previous study showed an 86% reduction in process steps with TLA [16], our result further supports its effectiveness. Specifically, the wTTM of the post-analysis phase decreased the most, by 95.8%.

The results of ICERs are described in natural units and percentages (Table 2). The ICERs in the mean TAT and 99th percentile TAT were more cost-effective for immunoassays than for clinical chemistry tests, whereas the ICERs in the TAT CV for clinical chemistry tests were more cost-effective than those for immunoassays. In wTTM, the post-analysis phase was the most cost-effective, with an ICER of USD 6.4k. Unlike natural units, ICERs expressed as a percentage can be used to compare the relative cost-effectiveness among the four KPIs. With 77.6% improvement in effectiveness, the wTTM was the most cost-effective KPI, with an ICER of USD 4.6k, followed by the TAT CV, 99th percentile TAT, and mean TAT. This result implies that TLA adoption is cost-effective with respect to both staff safety and predictability.

The expected payback period of TLA was also calculated by comparing two cases: without and with TLA implemented. Without TLA, more staff would be needed to cover the increasing number of tests. However, with TLA, the productivity of the laboratory is enhanced, leading to a reduction in staff. Table 3 shows that the assumed productivity would increase by 16.2% for the following two years. On average, the productivity would increase to 4,747 until 2021, thereby decreasing the number of staff members required. The break-even point for TLA is visualized in Fig. 1, wherein the TLA depreciation line represents the annual cost for TLA adoption, and the cost savings line can be interpreted as revenue. With an expected cost savings of USD 1,880 k at the end of 2025, it is anticipated that with on staff cost savings, it would take 4.75 years to payback the initial investment cost of TLA. We additionally performed the analysis under two different scenarios, aggressive and conservative anticipation, with 21.2% and 11.2% increases in productivity, which were expected to take 3.54 and 6.24 years to payback the initial cost, respectively. The expected payback periods were calculated considering that the actual number of full-time equivalents in 2020 has been reduced by 2.5–3.0.

Table 3

Total staff and cost savings

Year	Tests*	Without TLA			With TLA			Cost savings[‡]
		Productivity	FTE[†]	Staff cost[‡]	Productivity	FTE[†]	Staff cost[‡]
2020	58,333	3,519	16.6	1,068	4,087	14.3	919	148
2021	60,433	3,519	17.2	1,149	4,747	12.7	852	297
2022	62,609	3,519	17.8	1,237	4,747	13.2	917	320
2023	64,863	3,519	18.4	1,331	4,747	13.7	987	344
2024	67,198	3,519	19.1	1,433	4,747	14.2	1,062	371
2025	69,617	3,519	19.8	1,542	4,747	14.7	1,143	399
Total				7,759			5,879	1,880

*Tests refer to the number of tests per day; †FTE is calculated by tests/productivity; ‡Monetary unit: 1,000 USD.

Abbreviations: FTE, full time equivalent; TLA, total laboratory automation; USD, US dollar.

Fig. 1

Expected payback period.

Abbreviations: TLA, total laboratory automation; USD, US dollar.

DISCUSSION

An economic evaluation (including CEA) is rarely conducted for the adoption of medical devices, especially in Korea, because such evaluations are not required by regulators [13]. This is mainly due to the difficulty in measuring clinical effectiveness in terms of quality-adjusted life year, particularly in the in vitro diagnostics area. We conducted CEA as an initial economic evaluation of TLA, which was approached from the perspective of ICER calculation. ICERs were calculated for four KPIs related to system effectiveness in terms of both natural units and percentages. Although these calculations did not determine whether TLA itself was cost-effective but rather compared the relative cost-effectiveness between sub-elements and KPIs, this analysis represents a meaningful first step to generate the ICER threshold for TLA.

Some studies have indicated that one of the main benefits of implementing TLA is the increased staff safety level [5, 17]; the present study also showed that this was the most cost-effective KPI. Given that the post-analysis phase involves the highest exposure of laboratory staff to decapped samples, the high level of improvement offered by wTTM at this phase would likely significantly increase staff safety. The importance of staff safety has been particularly emphasized recently given the impact of the COVID-19 pandemic on substantially increasing the number of high-risk, infectious samples to be tested, thereby increasing the risk to laboratory staff; keeping staff away from these samples as much as possible increases their safety. Work safety can also fundamentally reduce the likelihood of mass infections in hospitals. TLA has increased staff safety by automating the entire laboratory workflow, especially in the post-analytic phase, and thus eliminating all manual actions that would potentially increase biohazard exposure and risk of injury. However, this study has a limitation since we focused only on staff safety and did not consider risk management in terms of safety related to the absence of error in the diagnostic testing [18-22].

Robotic sampling processes have standardized the entire workflow of a laboratory, allowing for relatively untrained personnel to adapt to the remaining laboratory jobs [23]. Accordingly, hiring standards for clinical personnel would be reduced, because clinical personnel would no longer require significant experience. By reducing the number of repetitive manual tasks, skilled staff can concentrate more on tasks such as result verification, which require higher levels of knowledge and experience. Notably, adopting TLA itself requires a different skillset to maximize efficiency, such as information technology expertise.

Other than clinical findings, there are some operational implications of the present results. One of the greatest barriers for introducing TLA is its high initial cost, similar to other innovative medical technologies [5, 24]. However, this study indicates that TLA can save money in the long run. Although not adopting TLA seems to cost nothing, with an expected increase in the number of patients, more staff will ultimately be required to handle the corresponding increasing number of tests, thereby increasing the staff cost. TLA should be evaluated to reinforce laboratory competencies as robotic processes require less staff and time to run the same number of tests. From a long-term perspective, TLA is an appropriate alternative for reducing laboratory operational costs.

Although this study focused on reducing laboratory expenses only, TLA would improve laboratory productivity, which will ultimately enhance revenue [25]. This will in turn contribute to the hospital’s ability to serve more patients, because it will not be limited by its testing capacity. With the ability to handle a higher volume of samples, more patients can be served in a timely manner. This may lead to an increase in hospital revenue and thus an increase in hospital scale. The potential growth of hospital revenue may justify the high initial cost of TLA conversion.

Finally, while the wTTM metric developed in this study is related to laboratory staff safety, the other three TAT-related KPIs (mean TAT, 99th percentile TAT, and TAT CV) are related to patient and clinician satisfaction [26-28]. The improvement of each of these KPIs can result in more timely diagnoses, thereby reducing patient waiting times. The reduced waiting times would in turn likely increase patient satisfaction, ultimately creating value for both patients and the hospital.

In conclusion, our effectiveness analyses showed that TLA significantly improved laboratory performance for the following KPIs: timeliness, outlier rate, predictability, and staff safety. TLA also has a relatively short payback period and can reduce the total expenses of the hospital in the long term. Therefore, the capital investment for TLA adoption would be worthwhile.