The evaluation of hospital laboratory information management systems based on the standards of the American National Standard Institute.

INTRODUCTION: Nowadays, modern laboratories are faced with a huge volume of information. One of the goals of the Laboratory Information Management System (LIMS) is to assist in the management of the information generated in the laboratory. This study intends to evaluate the LIMS based on the standards of the American National Standard Institute (ANSI).
MATERIALS AND METHODS: This research is a descriptive-analytical study, which had been conducted in 2011, on the LIMSs in use, in the teaching and private hospitals in Isfahan. The data collecting instrument was a checklist, which was made by evaluating three groups of information components namely: 'System capabilities', 'work list functions,' and 'reporting' based on LIS8-A. Data were analyzed using the SPSS 20. Data were analyzed using (relative) frequency, percentage. To compare the data the following statistical tests were used: Leven test, t-test, and Analysis of Variance (ANOVA).
RESULTS: The results of the study indicated that the LIMS had a low conformity (30%) with LIS8-A (P = 0.001), with no difference between teaching and private hospitals (P = 0.806). The ANOVA revealed that in terms of conformity with the LIS8-A standard, there was a significant difference between the systems produced by different vendors (P = 0.023). According to the results, a Kowsar system with more than %57 conformity in the three groups of information components had a better conformity to the standard, compared to the other systems.
CONCLUSIONS: This study indicated that none of the LIMSs had a good conformity to the standard. It seems that system providers did not pay sufficient attention to many of the information components required by the standards when designing and developing their systems. It was suggested that standards from certified organizations and institutions be followed in the design and development process of health information systems.

INTRODUCTION

Nowadays, modern laboratories have to operate in a context wherein there is a vast volume of data. The advent of new technologies has led to an increasing growth in both the quality and quantity of information. Such growth may entail some challenges, which make adopting appropriate management methods inevitable.[1] Information management, as a systematic and cost-effective process, can play an efficient role in maintaining, utilizing, and disseminating data.

Collecting sufficient data for identifying the patients and their related samples is regarded as central to the laboratory performance. The collected data must be clear, accessible, complete, timely, and relevant, so that the healthcare professionals can use them to provide quality service.[23] A laboratory information management system (LIMS) can be especially helpful in integrating the laboratory with its working operations in such a way that it finally results in the acceleration of all involved processes and sub-processes.[1]

The Laboratory Information Management System is a computer application that is used for storing and managing data provided by healthcare professionals during their daily healthcare routines. This system is also applicable for controlling and managing the samples, reporting test results, and automation of all laboratory processes. It also enables Laboratory Managers to manage their resources, including laboratory staff and tools. Integrating LIMS with information systems of a healthcare facility allows and accelerates the mutual exchange of healthcare date between these two systems. LIMS mainly intends to facilitate the management of a vast amount of information available in the laboratory. This system is very advantageous for Quality Control and Quality Assurance Programs, as it utilizes quality management tools for process monitoring and optimization.[45]

On the basis of research findings, accessibility, reliability, and validity of data have a significant effect on the performance of a health information system and contribute to the quality of healthcare services.[6] According to the World Health Organization (WHO), the goal of healthcare information systems is to develop mechanisms to promote the efficient retrieval of patient information to be used for patient care, statistics, and for educational and research purposes.[7] Accessibility of standard data in an organized format is necessary for providing proper and timely medical services by the healthcare providers.[8] Many studies on the use of data in manual or automated formats, emphasize on the importance of appropriate information content for satisfying the healthcare goals.[91011] On the other hand, the type of information presented by the system to the users has a substantial effect on the user-system interaction and subsequently on the efficiency and effectiveness of the provided services. An effective way to enhance the information content of healthcare information systems is standardization of the presented information.[12] A comprehensive set of standards with respect to laboratory information systems has been developed by the ANSI.[13] This study intends to evaluate the information content of hospital LIMSs in teaching and private hospitals, in Isfahan, based on the standards of ANSI. In this study we have specifically used the following standard: Standard Guide for Functional Requirements of Clinical Laboratory Information Management Systems (LIS8-A),

Divergence of LIMSs from the standards can be considered as a main drawback for these systems, which can be opposed to the mission of these systems, that is, increasing accuracy of information, efficiency, effectiveness of work processes, and finally improving patient safety. Evaluation of these systems based on the established standards can shed light on the weaknesses of LIMSs, which are used in many hospitals and healthcare centers. Identification and detection of their problems is the primary step for optimization of these systems.

MATERIALS AND METHODS

This applied research is a descriptive–analytical study that was conducted cross- sectionally from June to May 2011. The research population consisted of all LIMSs in use in the 18 teaching and private hospitals in Isfahan (13 teaching and five private hospitals). The LIMSs used by these hospitals were provided by six vendors (Saya_Raya_Ekbatan-e-Hamedan, Lohan, Pouya Samane-ye Diva, Rayavarane Toseah, Kowsar, University Statistical Administration of Isfahan University). Each system was used by one to six hospitals. On account of the small size of the research population, sampling was not performed and the whole research population was studied. Evaluation of the laboratory information system was done using a researcher-designed checklist based on one of the standards of the ANSI and Clinical Laboratory Information Standardization Institute, that is, the LIS8-A standard. LIS8-A covered all the information components related to the performance of the system regarding storing, maintaining, and processing the information, for managerial decision-making.

This study intends to evaluate the functionalities of the LIMSs by reviewing the information components of the systems and their accordance to LIS8-A. These components were classified into three groups: ‘System capabilities’, ‘work list functions,’ and ‘reporting ’ and were reported in terms of both the hospitals and the software provider companies (vendors). The checklist included 63 items. When designing the checklist, the researcher excluded some items of the LIS8-A standard which based on the study objective, were not applicable to the health information systems in Iran. Some extra applicable items were added to the checklist. The content validity of the checklist was affirmed by six experts: A Healthcare Information Management Specialist, a Medical Informatics Specialist, two Software Engineers, a Laboratory Specialist, and a Pathologist. To collect the data, the researcher observed the LIMSs in use in the hospitals and interviewed the users. To analyze the data, if an item was in accordance with the standard it was given score of 1, and if not, it was given a score of 0. Irrelevant items were given no scores. Data were analyzed in SSPS 20 using descriptive–analytical statistics, including, frequency, relative frequency, and percentage. The Leven Test was used to compare the variances and two parametric tests, the t-test and analysis of variance (ANOVA) were used to compare the means. Statistical analysis was performed by using a significance level of α =0.05.

RESULTS

The results of this study on the LIMSs in use in the hospitals showed that the overall mean of accordance with the standard in the three categories was %30.33. According to one-sampled T test, the extent of accordance with the standard of the LIMSs was low (t = 8.91, P = 0.001).

Table 1 presents the extent of LIMS accordance with the LIS8-A standard in the hospitals. There was no significant difference in terms of the accordance of their LIMSs to the standard (t = 0.25, P = 0.806).

Table 1

The extent of conformity of the information content of the laboratory information system used in the selected hospitals with LIS-8 standard

Table 2 shows the extent to which the information content of LIMSs in hospitals conforms to LIS8-A. Based on these results, the conformity of LIMSs in teaching hospitals, in terms of capabilities of the system and work list functions, is better than in private hospitals.

Table 2

The extent of information content conformity to LIS8-A in the hospitals’ laboratory management information systems

The mean scores of the LIMSs conformity to the LIS8-A standard are represented in Table 3, based on the system vendors. The ANOVA statistical analysis revealed a significant difference, in terms of conformity to the standard among the systems provided by different vendors (P = 0.023). Based on this study, the extent of conformity of the LIMSs in five hospitals, which used the Kowsar System, was higher, and in those that used the Pouya Samane-ye Diva, it was lower than that in other hospitals

Table 3

The mean scores for conformity of the information content of the laboratory information management system with the LIS8-A standard based on their vendors

Based on the groups of information components [Table 4], the results of the descriptive statistics showed that the Kowsar system with 72% for ‘system capabilities’ and ‘reporting’ and with 57% for ‘work list functions,’ had a better conformity with the LIS8-A standard than the other systems. The conformity to the standard – except for the ‘reporting’ group in the Lohan System (51%) – was lower than 50% for different groups of information components in other systems.

Table 4

The conformity of different groups of information components to LIS-A standard based on the system vendors

DISCUSSION

The results of this study revealed that, in total, the extent of conformity of the studied LIMSs to the standard was low, with no significant difference between the systems in teaching and private hospitals. However, conformity to the standard was significantly different based on the vendors of the systems. Low scores of hospital information systems when compared to the standards indicated their poor quality. Based on ISO/IEC9126, the quality of the system in the user's perspective depends on its functionality, reliability, efficiency, effectiveness, as well as its capability for storage and exchange of the data. This standard has been driven from the ISO8402 Standard.[14]

Farzandipour et al.,[15] in their study entitled, ‘Hospital information systems user needs analysis: A vendor surveys,’ compared five major suppliers of hospital information systems in eight teaching hospitals. They found that the vendors of laboratory and pharmacy information systems considered approximately 40% of the users’ needs.

In our study the highest conformity with the standard concerning ‘system capabilities’ was observed in the teaching hospitals with 51% and among vendors in the Kowsar system with 72%, which are still far from the standard. Usability and efficiency are two significant attributes of the system influencing the physicians’ motivation toward using information systems.[16] Both these attributes are directly related to the presentation of the information component on the systems interfaces. According to the results of many studies,[171819] insufficient information components and poor presentation of these components can lead to poor usability, which in turn can result in a higher number of errors made by users, In another study, Ahmadi and Habibi koolaee[20] studied the viewpoints of the nursing staff, clinical users, and the department secretaries concerning functionality of the hospital information systems. This study showed that compared to the clinical users, the nursing staff considered the systems more useful and functional. Paying low attention to or ignoring standard functionalities of an information system in the design process can lower user motivation to use the system and have a negative effect on their adoption and acceptance.[1620]

This study indicated that none of the LIMSs had a good conformity to the standard. Azizi et al.,[21] in a study, evaluated to what extent the criteria developed by the American College of Physicians (ACP), with regard to the hospital information systems, were followed by the hospitals affiliated with the Iran, Tehran, and Shahid Beheshti Universities. In this study they reported that the conformity of the laboratory information systems in these hospitals with the standards of ACP was medium. The conformity of hospitals in the three universities with the ACP criteria was %29.8, %37.2, and %38.3, respectively. These scores reflect a low conformity of the existing systems with the standards. These results are consistent with the results of our study.

Besides our study, a research has so far been conducted on the compliance of the Iranian Health Information Systems, with users requirements or the existing standards, and it has revealed the weakness and strengths of such a system being used in healthcare institutions.[101520]

The results of this study have provided some evidence concerning the weaknesses and flaws of all evaluated LIMS systems. The flaws found in the LIMSs reveal the fact that system vendors have ignored a number of information components required by standards.

Considering the standards in the design process of laboratory information systems, and evaluation of these systems based on the standards, is central to improvement of the systems’ quality. This study shows that the status of the evaluated information systems compared to the standards is unsatisfactory. Many of the information components used by the standard have been neglected in the design of these systems. It seems that lack of a national standard leads to using poor quality systems in hospitals, as also disparity among systems provided by different vendors.

Taking the difference among various software products with regard to conformity with standards and constriction of the financial resources of the healthcare institutions, it is recommended that conformity with the required standards be given a high priority when buying or developing information systems. Moreover, considering the potential disconformities with the standards, the hospitals should contemplate about a mechanism that makes the post-implementation evaluation and promotion of the systems possible. In addition, prior to designing and implementing the system, the standards proposed by the certified institutions and organizations must be taken into account. Furthermore, prior to and during the design and implementation processes of the system, requirement analyses from the stockholders and potential users of the system should be carried out. Finally, it is recommended to provide required training to all the users of the system and to conduct periodical evaluation of the system, to prevent latent problems.

CONCLUSION

Given the results of the study, it can be claimed that compared to the standards, none of the LIMSs used by the hospitals of the city of Isfahan possess a good position. Based on the results, it seems that the software provider companies did not pay due attention to the appropriate informational components suggested in the standards. Hence, it is suggested that when designing and creating the system, the standards presented by the valid organizations and institutes must come into focus, too.