Evaluation of Biomedical Laboratory Performance Optimisation using the DEA Method.

INTRODUCTION: The Slovenian Resolution on the National Healthcare Plan notes that the country's medical laboratory activities are fragmented, which may result in cost-inefficiency and a reduction in the quality of the services provided. Defining the efficiency of laboratory service providers can therefore help us to pursue the objectives of the Resolution, i.e. to consolidate and integrate laboratory activities.
METHODS: Using the DEA method, we conducted an analysis of the efficiency of 20 biomedical laboratories in Slovenia, and made a comparison with a "virtual" laboratory, i.e. a merger of laboratories within a selected organisational unit. By testing different DEA models, we sought to determine whether the use of different input variables caused significant differences in the laboratories' efficiency scores.
RESULTS: The research results show that inefficiency resulting from the size of the units is 1.5 times greater than process inefficiency. Using a non-parametric Wilcoxon Signed Rank test, we determined, at a risk level of 0.05, that there was no difference between the efficiency results when using two different technical efficiency DEA models. When evaluating the virtually merged laboratory, we determined that, under all three models, the virtual laboratory achieved 100% VRS efficiency. However, when the CRS methodology was used, the laboratory showed a certain degree of scale inefficiency.
CONCLUSIONS: When evaluating merger of medical laboratories we note that the DEA method is methodologically suitable for evaluating the effects of health policy implementation, and is an appropriate tool for identifying where the field of laboratory medicine might be further developed and improved.

Introduction

For centuries, laboratory medicine has been one of the most important factors in providing proper patient care and promoting technological development in the field of medicine. The development of automation and information technology has led to the development of sensitive and specific laboratory tests that are invaluable to doctors when diagnosing and confirming diseases (1, 2). As a part of the public healthcare network, medical laboratories provides services that directly affect patients; furthermore, it provide services to other stakeholders, i.e. healthcare professionals, healthcare payers and health policy-makers (3). While the tendency to improve quality and productivity in the field of laboratory medicine is a constant one, it is important to bear in mind the constraints, which include staff shortages and limited funding. It is these constraints that compel laboratory managers to constantly consider introducing work processes and technologies that could optimise laboratory organisation and performance (4).

The issue of achieving efficiency is considered to be one of the most important intermediate objectives of health system effectiveness (5). Achieving efficiency in healthcare is often equated with the introduction of a free market economy, which can affect equal access to services, fairness in resource allocation and the achievement of quality standards (6). As a result of this dilemma, laboratory activities are treated differently in different systems. When measures to improve efficiency are adopted, a great deal of attention is devoted to laboratory concentration, which is essential if capacity is to be better utilised.

The efficiency of public service providers can be determined by means of different methods. However, data envelopment analysis (DEA) is one of the most commonly used methods in the field of healthcare (7, 8). The introduction of management systems, management processes and resource management planning aimed at improving efficiency in the healthcare sector is an important element in reducing costs and increasing productivity in healthcare organisations (9). One further aspect can be considered when our aim is to achieve greater efficiency, namely the use of technology in business processes. Technological development means that there is an increased need for greater specialisation of work, which leads to shorter turnaround times in laboratory medicine. In such cases, innovative ideas can significantly boost the efficiency and effectiveness of biomedical information systems (10). The laboratory work process is divided into several important and interconnected phases. By using the ABC process diagram, we can describe the individual phases in laboratory workflow (11, 12). One of the most important aspects of the renewal of work processes in the field of laboratory medicine is the introduction of functional automation. The merging of laboratories into a central hospital laboratory reduces overall costs because the number of biomedical analyzers (and therefore the associated labour costs) can be reduced (13). Our conclusion is that work process improvements resulting from the consolidation of laboratories and the determination of surpluses in the inputs used can only be identified through a thorough knowledge of the work process in biomedical laboratories.

The aim of our study is to determine the efficiency of biomedical laboratories by testing different specifications of DEA models. Our research question is whether the consolidation of laboratory activity can increase laboratory efficiency and, at the same time, reduce the costs allocated to laboratory services. The study also aims to evaluate the effects of a potential merged biomedical laboratory, and offer an optimally constructed biomedical laboratory at the tertiary healthcare level. The study is based on data from 20 biomedical laboratories in Slovenia.

Definition of Laboratory Activities and their Necessary Development in Slovenia

Slovenian health policy is determined by the Resolution on the National Healthcare Plan 2016–2025 (“Together for a Healthy Society”), the Health Services Act, and the Healthcare and Health Insurance Act. The Resolution identifies laboratory medicine as part of the health network and as an important factor in ensuring proper, high-quality patient management. It also notes that laboratory activities in Slovenia are fragmented and that they fail to provide sufficient test concentration; this results in incomparability of laboratory test results between individual laboratories and may lead to a reduction in the quality of the services provided and to cost inefficiency. One of the objectives of the Resolution is therefore to highlight the need to integrate and consolidate laboratory activities (14).

Slovenian experts in the field of laboratory medicine have identified the non-consolidation of laboratories and indirect payment for laboratory services as the main problems currently being faced. The consolidation and coordination of laboratories and the introduction of direct payment for laboratory services were identified as possible solutions to the current situation (15).

According to the records of the Ministry of Health, Ljubljana University Medical Centre (UKC) has eight separate biochemical laboratories, each under its own organisational unit. The Merged Laboratories Project, which aims to bring together the four largest UKC medical laboratories under one organisational unit and one location, is currently under way at the hospital.

Literature Review

Among the most widely used methods for efficiency analysis are ratio analysis, total productivity factor, the least squares method (COLS), stochastic frontier analysis (SFA) and data envelope analysis (DEA). The DEA method offers some advantages – specifically, an empirically determined frontier of production possibilities without a predetermined production function, and the possibility of creating goals for less efficient units (16).

A few healthcare DEA efficiency studies in the field of primary medical care (17, 18, 19) do exist, and they all come to a similar conclusion: that smaller primary healthcare centres are more scale inefficient. DEA efficiency studies of hospitals at the secondary healthcare level (5, 20, 21, 22) are also quite common. The results of the studies suggest that an understanding of hospital functioning, together with the results obtained from efficiency studies, can help us determine optimal hospital size.

While efficiency research has not been conducted to any great extent in the field of laboratory medicine, there are studies (23, 24) that identify laboratory efficiency using the DEA methodology and that provide a basis for the further development of efficiency research in laboratory activities.

Research on the impact of hospital mergers on determining hospital efficiency and potential surpluses in the inputs used is also important for our study. Efficiency improvements can usually only be achieved by improving scale efficiency (25, 26, 27, 28). Several studies use virtual hospital mergers as a tool to show the differences between unconsolidated and combined hospital activity (25, 27, 29). However, (30) identifies, in theoretical and methodological terms, the contributions of the virtually merged unit model used. This indicates the possibility of using the method of virtual merger of DMUs as a tool for identifying potential improvements and defining surpluses in the inputs used at potentially merged biomedical laboratories.

METHODOLOGY

Historically, the development of methods for assessing efficiency began with Debreu (1951), Koopmans (1951) and Farrell (1957), who laid the foundations for assessing the relative efficiency of DMUs (decision making units). A technically efficient input-oriented manufacturer therefore produces a constant output by minimising production factors. The appropriate ratio of inputs in terms of their price further defines allocative efficiency. Economic efficiency is a product of both technical and allocative efficiency (7, 26, 31).

In 1978 Charnes developed a linear program based on a non-linear program, i.e. a DEA method for analysing the efficiency of DMUs. The method has become particularly well-established in the public sector, since the frontier of linearly linked segments is determined on the basis of empirical data without a pre-specified production function. The CRS (constant return to scale) or VRS (variable return to scale) model can be used in the analysis to predict the technology used (7, 8, 32).

The input-oriented CRS model can be mathematically presented in the form of a linear program derived from a fundamental fractional program. The CRS model (Eq, 1), also named the CCR model, is a constant return model and is for n DMUs, of which each DMUj uses m inputs x and s outputs y, written in linear form (33). Values y, and x are the given values of outputs and inputs, indicating past operating results. Values µ and v are variable weights enabling each unit to be weighted for its optimal benefit function within the given limitations, determined by the values of variable y and x with all units. Efficient units are those that reach the ratio 1. The definition is written in the input form; therefore, the proportionally inefficient units reach the ratio <1.

The difference between the CRS and VRS models is present in the free variable, which is a dual variable associated with the constraint. The CRS therefore represents technical (cost) efficiency, while the VRS result represents pure technical (cost) efficiency. Scale efficiency (SE) is a ratio between CRS and VRS efficiency and enables us to define whether the cause of inefficiency lies in the non optimal size of the observed unit. We can also define cost-based scale efficiency (CSE).

DEA research on efficiency in healthcare is mostly input-oriented; this is because healthcare managers only control used resources, which is also supported by systematic reviews of DEA research (16, 34).

The selection of input and output variables was based on the research (23, 24, 35, 36). Laboratory output is produced by means of capital, labour and the use of consumables. In all the models presented, we define output as the number of basic, special and reference tests. Therefore, in our case, Model 1 (M1) represents the number of working hours, the number of biomedical analyzers, and the cost of laboratory material and reagents. Model 2 (M2) represents the number of working hours, the total purchase price of laboratory equipment, and the cost of laboratory materials and reagents. The cost model (M3) is defined by labour costs and the cost of laboratory material and reagents. In the technical efficiency model, we tried to determine whether there was a difference in determining technical efficiency by using a different capital input.

Because laboratories have more control over the inputs they consume than the outputs they produce, we used the input-oriented CRS and VRS DEA method to determine the efficiency of medical laboratories with an additional calculation of SE. By determining the surplus in the inputs used, we quantified the redundant resources used.

We constructed a virtual laboratory, based on the expected merger of four UKC laboratories at a single location, which consists of the sum of the actual values of the inputs and outputs of the four UKC laboratories. Using an ABC laboratory work process chart, we identified individual phases within which the work process could be optimised.

Empirical Data and Analysis

In our research, we determined the efficiency of laboratory service providers in Slovenia at the primary, secondary and tertiary healthcare levels. Eleven hospital biochemical laboratories (B), the laboratories of three tertiary institutions (T) and three major laboratories at the primary level (Z) were included in the study. All four UKC laboratories slated for merger were included in the analysis. The data for our research is not publicly available, and we obtained it from public healthcare institutions solely for the purposes of our research. We analysed data from 20 Slovenian medical laboratories for 2017. Due to the restricted set of laboratories analysed and the use of multiple inputs and outputs, our estimation of laboratory efficiency could potentially be higher than it actually is.

Data for our study was analysed using the Frontier program and the SPSS statistical program.

The smallest B1 laboratory has six employees who carry out a total of 7,458 effective working hours. The largest T1 laboratory has 107 laboratory workers who carry out the largest number of basic laboratory tests annually, i.e. 5,616,624. Its labour costs are EUR 2,290,144. The laboratories have 21 biomedical analyzers on average and spend an average of EUR 942,724.43 on reagents and laboratory material. A certain degree of expected heterogeneity is present on the side of output, since SDs may be 50% higher than average values. As is apparent from the value of the outputs, heterogeneity is present because of the specialised services of individual health institutions that go beyond the scope and sphere of influence of laboratory activity, which adjusts the range of laboratory tests it offers and performs.

Results

Table 2 shows the results of efficiency analysis using the CRS and VRS methodologies. As the aim of our study is to determine the optimal size of the merged UKC laboratory, we will primarily focus on presenting the results of the UKC laboratories in defining the results of the analysis.

Table 2

Efficiency results.

	Model 1				Model 2				Model 3
DMU	CRS (%)	VRS (%)	RTS	SE (%)	CRS (%)	VRS (%)	RTS	SE (%)	CRS (%)	VRS (%)	RTS	SE (%)
B1	34.93	100	1	34.93	39.86	100	1	39.86	37.26	100	1	37.26
B2	38.78	68.93	1	56.26	45.25	71.79	1	63.03	40.36	66.74	1	60.47
B3	100	100	-	100	100	100	-	100	100	100	-	100
B4	100	100	-	100	100	100	-	100	100	100	-	100
B5	69.81	88.59	1	78.80	70.54	80.64	1	87.48	64.05	73.40	1	87.26
B6	100	100	-	100	100	100	-	100	100	100	-	100
B7	100	100	-	100	78.08	86.62	1	90.14	55.75	60.32	1	92.42
B8	75.14	86.14	1	87.23	80.15	90.16	1	88.90	71.74	74.01	1	96.93
B9	64.53	77.56	1	83.20	66.66	79.06	1	84.32	63.83	75.18	1	84.90
B10	88.82	97.37	1	91.22	99.94	100	1	99.94	89.61	100	1	89.61
B11	100	100	-	100	81.03	96.73	1	83.77	97.54	100	1	97.54
Z1	100	100	-	100	100	100	-	100	87.36	100	−1	87.36
Z2	76.90	77.56	−1	99.15	86.97	87.34	1	99.58	76.9	77.56	−1	99.15
Z3	66.40	96.40	1	68.88	75.73	100	1	75.73	66.4	100	1	66.40
T1	100	100	-	100	100	100	-	100	100	100	-	100
T2	72.28	100	1	72.28	92.33	100	1	92.33	77.62	100	1	77.62
T3	100	100	-	100	100	100	-	100	100	100	-	100
T4	80.87	87.49	1	92.43	77.59	79.57	1	97.51	46.14	52.87	1	87.27
T5	60.04	72.64	1	82.65	60.04	72.64	1	82.65	58.44	67.17	1	87
T6	100	100	-	100	100	100	-	100	100	100	-	100
x(%)	81.43	92.63		87.35	82.71	92.23		89.26	76.65	87.36		87.56
SD	21.01	10.60		17.79	18.87	10.29		15.42	21.78	16.68		16.38

Source: own.

As Table 2 shows, the average technical efficiency of laboratories under Model 1 is 81.43%. As expected, average pure technical efficiency is higher (92.63%). Under Model 2, we observe higher average technical efficiency than under Model 1 (82.71%). However, the average pure technical efficiency is lower (92.23%). A comparison between laboratories when using the two models shows that SD is the highest in CRS technology prediction, i.e. for laboratories B11 and T2.

Table 2 shows that, when using cost Model 3, the average cost-efficiency score is, as predicted, lower (76.65%) than the technical efficiency score. Similarly, we observe a pure cost-efficiency score (87.36%) that is lower than the pure technical efficiency score.

We can establish that the laboratories observed show scale inefficiency that is 1.5 times higher than process inefficiency. This suggests that in the laboratories examined in our study, a greater share of inefficiencies result from suboptimal laboratory size than from the way laboratories work. They could therefore improve their technical efficiency mainly by adjusting their process size. The B3, B4, B6, T1, T3, Z1 and T6 laboratories are both CRS- and VRS-efficient, which means that they are of optimal size.

They form the frontier of production possibilities in the use of both models and, furthermore, the B7 and B11 laboratories are scale efficient under Model 1. The lowest efficiency was observed in connection with the B1 laboratory under all three efficiency models.

Our simulation was carried out in response to the planned merger of laboratories T1, T2, T3 and T4. The merging of these laboratories is therefore easier to carry out because the laboratories already operate within a single institution. The potential consolidation of other laboratories in our analysis could be based on their geographical proximity. The T1 and T3 laboratories are efficient when using both the CRS and VRS methodologies, which means they are also of optimal size. The T2 laboratory does not show process inefficiency when using Model 1. All inefficiency results from incorrect size. As the RTS of laboratories T2 and T4 is increasing, they could only achieve scale efficiency by increasing their size. This could be achieved through an increase in the resources used, as well as through a proportional increase in the number of laboratory tests provided. Model 2 produces a similar result.

In terms of determining technical efficiency, the potential merger of the T1, T2, T3 and T4 laboratories would have a positive impact, mainly due to the more optimal size of the T2 and T4 laboratories, which would now be part of a larger laboratory.

A comparison between the technical efficiency and cost-efficiency of the laboratories shows that the T4 laboratory has the biggest difference (34%). Further analysis should be carried out to determine whether the cause of the inefficiency may lie in the sub-specialisation of the T4 laboratory.

We used the Wilcoxon Signed Rank non-parametric test to determine statistical difference in the efficiency results. Using a significance level of 0.05, we established that the efficiency results for the two technical efficiency models in the case of our sample were not different (W 25>W critical 14).

As Table 3 shows, a virtually merged laboratory (Pseudo) would achieve 100% VRS efficiency under all three models. However, the laboratory shows a certain degree of scale inefficiency when the CRS methodology is used. The size of the process in the consolidated laboratory would be too large from the standpoint of decreasing returns to scale (RTS). The laboratory should reduce the amount of inputs it consumes and maintain the level of outputs produced. Despite the relatively low level of inefficiency identified, the surpluses in inputs used would be high, mainly due to the size of the merged laboratory.

Table 3

Efficiency of the virtually merged laboratory.

	DMU	CRS (%)	VRS (%)	RTS	SE (%)
MODEL 1	Pseudo 1	93.33	100	−1	93.33
MODEL 2	Pseudo 2	95.89	100	−1	95.89
MODEL 3	Pseudo 3	89.60	100	−1	89.60
x¯		92.94	100		92.94
SD		3.16	0		3.16

Taking the economic efficiency theory into account, we further defined the surpluses in inputs and deficits in outputs produced at the virtual laboratory. The values of the surpluses were defined using the DEA method, i.e. by predicting the optimal size of the laboratory. However, one should be aware that our set of analysed laboratories shows a certain degree of heterogeneity, which can make it more difficult to accurately estimate the target values of the input variables. A virtual laboratory could reduce the number of working hours by 16,686.01, labour costs by EUR 374,852.16, the number of analyzers by 11, the purchase price of laboratory equipment by EUR 205,256.91, and the cost of laboratory materials and reagents by EUR 1,265,576.49. On the output side, the laboratory could carry out 860,520 more basic laboratory tests and 112,344 more specialised laboratory tests. A laboratory constructed in such way would achieve and create a new production possibility frontier through the use of both CRS and VRS methodologies.

We used the ABC flowchart to show the work process in the laboratory. The essential stages of the process are: test ordering, admission of sample, sample triage, analysis, result validation, and the issuing of accurate and high-quality laboratory results.

Based on knowledge of the work process in medical laboratories, we suggested the process improvements in individual phases of laboratory work that could possibly result from the consolidation of laboratories Organisational chart 1). Irrespective of the number of collected samples, laboratories must ensure continued presence of trained laboratory personnel at the sample collection and admission point. A reduction in the workforce could be accomplished by unifying sample collection as well as sample admission points. Different biomedical analyzers are required to perform different diagnostic laboratory tests. In the case of scale inefficient laboratories, this fails to achieve a sufficiently high level of analyzers utilisation. Despite lower analyzers utilisation, daily, monthly and annual maintenance of devices must be carried out, along with the daily testing of control material. This presents additional costs resulting from the consumption of laboratory materials and reagents. In our study, we observe that automation can improve laboratory efficiency only if biomedical analyzers are fully utilised, something that can be achieved if laboratories are of the appropriate size. The merging of laboratories within a single health organisation would therefore lead to lower purchase prices through the joint procurement of laboratory reagents and materials, and biomedical analyzers. The uniform use of diagnostic equipment ensures greater comparability of results, and thus directly affects the quality and reliability of the laboratory findings issued. The envisaged merger of laboratories would result in greater specialisation of work tasks. Added value also represents the implementation of special tests and research activities.

Organisational chart 1

Source: own.

Work process in the laboratory.

In the case of separate laboratories, only larger laboratories have their own support staff, i.e. administration and service personnel. Administrative tasks in smaller laboratories are carried out by laboratory workers. Rationalisation of the number of support staff, i.e. service personnel and administration, would reduce labour costs in the consolidation process.

Discussion

Under the Slovenian Resolution on the National Healthcare Plan, laboratory activity is unconsolidated. This adversely affects the concentration of the laboratory tests conducted, the quality of work and, not least, the technical and cost-efficiency of laboratories. The DEA method enables us to identify best practices and to quantify the surpluses in the inputs used. By calculating scale efficiency, we can further determine whether the cause of the inefficiency lies in the size of the unit analysed or the results of the process inefficiencies.

While a constructed, virtually merged laboratory does not show process inefficiency, it does show some degree of scale inefficiency. Similarly, studies on hospital mergers show the importance of determining the optimal size of hospitals in their merger (26, 27). In light of the findings of these studies, we also observe that, due to the decreasing RTS, merged laboratories should reduce the resources used. The biggest advantage of merged laboratories is the reductions they bring in the cost of laboratory reagents and materials. Work process optimisation resulting from a more appropriate laboratory size would have a positive impact on achieving economies of scale. In this part, therefore, our study concurs with previous findings (13). Similarly, the results of our study show that labour costs, the costs of laboratory reagents and materials and the number of biomedical analyzers used would be reduced. The merged laboratory could thus reduce labour costs by 10% and the costs of laboratory reagents and materials by 17%. Labour force optimisation could be accomplished by unifying sample collection points, as well as admission and management staff at the laboratory. The harmonisation of technology used at the same location would consequently lead to greater utilisation of laboratory equipment and improve the comparability of the laboratory tests provided.

Conclusions

An evaluation of the efficiency of medical laboratories should be in the interest of health service payers and health policy-makers alike. We can conclude that efficiency evaluations are indispensable when planning a network of laboratory services, i.e. consolidation of laboratory services.

We note, like other researchers who have used the DEA method to evaluate health policy and reform (37, 38), that the method is methodologically suitable for evaluating the effects of health policy implementation, and is an appropriate tool for identifying further development and improvements in the field of laboratory medicine. The results of the analysis can help public healthcare institution managers to identify surpluses in the inputs used and the resulting process optimisation.

If decision-makers want easily accessible laboratory services, they must accept that this has a negative impact on the optimal performance of smaller medical laboratories (because their size is non-optimal). In conclusion, we must ask ourselves how much efficiency we are willing to sacrifice to ensure that health services are accessible to the greatest possible extent.

The main gap in the study is its inability to identify cost-efficiency using several models because public healthcare institutions’ records are incomplete, particularly the aspect that defines capital cost. An international comparison of the models used to determine efficiency would provide additional benefits to this field of study.

Table 1

Statistical analysis.

		Min.	Max.	Average	SD
INPUTS	Number of working hours	7,458	157,746	42,604.85	35,947.78
	Number of biomedical analyzers	6	68	21	16
	Cost of laboratory equipment	49,616	2,437,057	601,112.50	590,995.57
	Labour costs	97,332.54	2,290,144	657,896.07	591,933.26
	Cost of laboratory reagents and material	134,683	4,520,071	942,724.43	945,991.67
OUTPUTS	Number of basic tests	49,985	5,616,624	941,786.90	1,277,335.30
	Number of special tests	2,651	784,245	121,054.15	170,071.16
	Number of reference tests	24	111,940	18,066.60	26,016.38

Source: own.