Validation of a laboratory and hospital information system in a medical laboratory accredited according to ISO 15189.

INTRODUCTION: The aim of the study was to present a protocol for laboratory information system (LIS) and hospital information system (HIS) validation at the Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital, Zagreb, Croatia.
MATERIALS AND METHODS: Validity of data traceability was checked by entering all test requests for virtual patient into HIS/LIS and printing corresponding barcoded labels that provided laboratory analyzers with the information on requested tests. The original printouts of the test results from laboratory analyzer(s) were compared with the data obtained from LIS and entered into the provided template. Transfer of data from LIS to HIS was examined by requesting all tests in HIS and creating real data in a finding generated in LIS. Data obtained from LIS and HIS were entered into a corresponding template. The main outcome measure was the accuracy of transfer obtained from laboratory analyzers and results transferred from LIS and HIS expressed as percentage (%).
RESULTS: The accuracy of data transfer from laboratory analyzers to LIS was 99.5% and of that from LIS to HIS 100%.
CONCLUSION: We presented our established validation protocol for laboratory information system and demonstrated that a system meets its intended purpose.

Introduction

Medical laboratories are the key partners in patient safety. Laboratory results influence 70% of medical diagnoses (1). The quality of laboratory service is the major factor that directly affects the quality of health care (2). Accreditation according to an internationally recognized standard such as ISO 15189 „Medical laboratories – Particular requirements for quality and competence“ is a recognition of a laboratory’s competence and compliance with the requirements of the standard (3). It is acknowledged as the single most effective route to comprehensive quality assurance (4).

Improving quality of the work process is one of the main goals of accredited laboratories, and development of a laboratory information system is a precondition for quality improvement in this phase of laboratory process (5,6). The Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital, Zagreb, Croatia, has successfully installed the Laboratory Information System (LIS) (7,8), and the Department of Laboratory Medicine, a part of the Institute, is additionally integrated into the Hospital Information System (HIS). The ISO 15189 standard requires that the computer software be addressed in the same way as the instruments (9). One of the newest accreditation requirements includes recommendations for the protection of LIS – Annex B. Amongst numerous demands, the laboratory management must ensure maximum security of data entry and reports (B5.) (10). To respond to the particular requirement of the accreditation standard for medical laboratories, the accuracy of the entire process of data transfer between laboratory equipment and laboratory (LIS) and hospital information (HIS) systems needs to be validated and appropriately documented.

The aim of this study was to establish a protocol for retrospective validation of laboratory and hospital information systems in the Institute of Clinical Chemistry and Laboratory Medicine in response to ISO 15189 requirements, as described in Annex B, section B5.

Considering the fact that only four out of more than 200 medical-biochemistry laboratories in Croatia have been accredited to ISO 15189 in the field of medical biochemistry (11) and that, to the best of our knowledge, validation process in Croatian medical laboratories has not been previously described, this paper could be of great assistance to other laboratory management in meeting accreditation requirements in the future.

Materials and methods:

The Institute of Clinical Chemistry and Laboratory Medicine comprises two departments, the Department of Clinical Chemistry consisting of four units, and the Department of Laboratory Medicine consisting of three units. The Institute performs testing in the areas of clinical chemistry, laboratory hematology and coagulation, cell immunology-and immunophenotyping, and molecular diagnostics. The accuracy of data transfer from laboratory equipment to LIS was validated in the entire Institute of Clinical Chemistry and Laboratory Medicine, both in the Department of Clinical Chemistry and the Department of Laboratory Medicine. The applications used were LIS – BioNet (Labnet software engineering, Josipovac, Croatia) and HIS – BIS (Grad d.o.o. computer engineering, Pula, Croatia).

The accuracy of data transfer from LIS to HIS was validated in the Department of Laboratory Medicine according to the reference documents (12–14). The main outcome measure was transfer accuracy expressed as percentage (%). Our acceptance criteria were a complete match between numeric data obtained from laboratory analyzers and results transferred into LIS, and between data entered into LIS and then into HIS. An additional acceptance criterion regarding data transfer from LIS to HIS was a complete match of comments entered by laboratory personnel.

Validation protocol was designed in cooperation with vendors of applications used according to the available literature (10,12–14) described in detail below. Validation of data transfer into LIS and HIS was carried out from 31. 03. 2009 till 13. 05. 2009 in the Department of Clinical Chemistry and from 20. 12. 2010 till 23. 02. 2011 in the Department of Laboratory Medicine. All data were processed in Microsoft Excel provided templates.

Transfer of data from LIS to laboratory equipment and back to LIS

Transfer of data from laboratory equipment into LIS was checked separately at each department. The analyzers used are presented in Table 1. The whole process was performed by entering virtual demographic data with all test requests into LIS and printing corresponding barcoded labels that provided laboratory analyzers with the information on requested tests. A total of about 200 tests at the Department of Clinical Chemistry and 180 tests at the Department of Laboratory Medicine in the areas of clinical chemistry, laboratory hematology and coagulation, cell immunology and immunophenotyping, and molecular diagnostics were included in the validation study. Results of the following tests were calculated: HbA1c (both percentage and SI units), HDL-3 cholesterol, LDL-cholesterol, VLDL-cholesterol, functional tests (albumin to creatinine ratio, creatinine clearance), and quantitative analysis of 24-h urine samples (albumin, total protein; electrolytes: potassium, sodium, chloride, calcium, inorganic phosphorous; urea, uric acid; cortisole). Barcoded labels were printed in duplicate for further purposes. Data from several virtual patients with different demographic data to cover a broad field of gender- and age-specific reference intervals were entered. Entries were made for virtual patients of both genders: 14 patients (aged 21–105 yrs) at the Department of Clinical Chemistry and 12 patients at the Department of Laboratory Medicine (aged 24–84 yrs). Labeled samples were placed on laboratory analyzers. All tests were performed from a pool of non-infective human samples. After the analysis, data obtained by transfer to LIS were verified and validated by authorised personnel (medical biochemist) and printed in the form of a laboratory report. The original printouts of the test results from laboratory analyzer(s) were compared with the LIS-derived report. All patient demographic data as well as their test results and comments from laboratory analyses were entered into the template provided by the information application vendor. Examples of the comments were as follows: “H” for high or result above the corresponding reference interval; “L” for low or result below the corresponding reference interval. Barcoded labels were also stuck on provided templates. The accuracy of all entered data, both demographics and test results, and their matches, was verified by signatures of the validator (medical biochemist) and the manager of the company which created and installed the application. The whole procedure was also performed for laboratory analyzers that were not connected to LIS with a remark that the data were manually imported into LIS. Calculated test results from LIS were also validated using only a calculator and formulas required to calculate the results. Both data from the thus verified results and those from the analyzer outside the LIS network were entered into similar templates and their accuracy confirmed by signatures of the validator (medical biochemist) and the vendor manager. The whole validation process was finalized by a thorough description of the application protocol, servers and specifications of users’ computers with installed laboratory information system.

Table 1.

List of analyzers included in validation.

Equipment	Manufacturer	Connected to LIS
	Department of clinical chemistry
AU400	Olympus Optimal Co. (Tokio, Japan)	YES
AU600	Olympus Optimal Co. (Tokio, Japan)	YES
Fractoscan Junior	Olympus Diagnostica (Hamburg, Germany)	YES
XS1000i	Sysmex Corporation (Kobe, Japan)	YES
XE-2100	Sysmex Corporation (Kobe, Japan)	YES
CA560	Sysmex Corporation (Kobe, Japan)	YES
CA1500	Sysmex Corporation (Kobe, Japan)	YES
AxSYM	Abbott Laboratories (Illinois,USA)	NO
CLINITEK 500	Bayer Corporation (Pittsburg, USA)	YES
VIVA-E	Vital Scientific N.V. (Dieren, Netherlands)	YES
EPICS XL	Coulter (Miami, USA)	YES
ELECSYS 2010	Hitachi High-Technologies Corporation(Tokio, Japan)	YES
STAT PROFILE pHOx1	Nova Biomedical (Waltham, Massachusetts, USA)	YES
STAT PROFILE pHOx2	Nova Biomedical (Waltham, Massachusetts, USA)	YES
STAT PROFILE CCX	Nova Biomedical (Waltham, Massachusetts, USA)	YES
	Department of laboratory medicine
AU400	Olympus Optimal Co. (Tokio, Japan)	YES
AU600	Olympus Optimal Co. (Tokio, Japan)	YES
ADVIA 120	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	YES
ADVIA 60	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	YES
Advia Centaur XP	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	YES
Rapidlab 348	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	YES
Osmometer 3250	Advanced Instruments (Massachusetts, USA)	NO
CLINITEK 500	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	YES
StatStrip	Nova Biomedical (Waltham, USA)	YES
ImageQuant TL	GE Healthcare Bio-Sciences AB (Uppsala, Sweden)	NO
Immulite	Diagnostics Products Corporation (Los Angeles, California, USA)	NO
Delfia 1234	LKB Wallac (Turku, Finland)	NO
STAT FAX 2100	Awarenes Technology Inc. (USA)	NO
BFT II	Siemens Healthcare Diagnostics Inc. (Tarrytown, New York, USA)	NO
Kmotto 20 (keyboard)	provided by LIS vendors	YES

LIS – laboratory information system

Transfer of data from HIS to LIS and back to HIS

The transfer of data was examined in the Department of Laboratory Medicine, the only department integrated into HIS. HIS and LIS are connected by a Health Level 7 (HL7)-protocol. HL7 specifies the number of flexible standards, guidelines and methodologies by which various healthcare systems can communicate with each other. Such guidelines or data standards are a set of rules that allow information to be shared and processed in a uniform and consistent manner (15). Validation was performed by entering data on four virtual patients of various ages (10–54 yrs) and both genders into HIS, requesting all tests in HIS and creating real data in a finding generated in LIS. At this point there was no real sample testing because data transfer from analyzer to LIS had been previously validated. Considering that HIS environment enables screen view of the laboratory test data, a print screen from HIS with completed laboratory test results was made upon validating virtually created data in LIS. A hard copy of the laboratory report from LIS and a print screen from HIS were compared. Demographic data and test results obtained from both LIS and HIS, together with previously described comments, were entered into a corresponding template and the accuracy of their match verified by signatures of the validator (medical biochemist) and the software company manager. The entire validation process was finalized by a description of a HL7 protocol for connecting LIS and HIS.

Results

The validation process was performed in the entire Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital. The personnel authorized to validate data obtained were medical biochemists employed at the Institute. All obtained data were validated by authorised medical biochemists without employing auto-validation. The entire validation documentation is archived at the Institute’s secretariat.

There were some discrepancies observed during the process. As software of some of the analyzers cannot include all reference intervals used in every day routine work, results on such analyzers were not adequately labelled when being out of reference intervals. Such labels differed between analyzers (arrows, H or L, asterix), while in LIS they were marked as H or L (in the Department of Clinical Chemistry), aligned to the left if the result was below reference interval or to the right if the result was above reference interval (in the Department of Laboratory Medicine). All these remarks were entered into provided templates with additional explanation if necessary. Four analyzers in the Institute (two in each department) did not have printers integrated or connected to the interface, for which an additional comment was added.

Nevertheless, the described discrepancies were not among our acceptance criteria because they reflected instrumentation characteristics rather than data transfer accuracy. Discrepancies in numeric data transfer were found for two tests, in which printouts of analyzer-derived data analyzer had different number of decimal places from those of data transferred to LIS, the accuracy of data transfer thus being about 99.5 %. However, a careful analysis of these two cases showed that LIS and the analyzer expressed results differently (as percentage or a whole number, respectively), for which an additional explanation was supplied in templates. Considering the fact that calculated test results from LIS also showed no discrepancies, it was concluded that complete match of data between laboratory analyser and LIS was achieved.

No discrepancies were observed in the transfer of data from LIS to HIS either in the numeric data transfer or in the written comment transfer. The accuracy of the transfer from LIS to HIS was 100%. All data with corresponding comments entered into provided templates were checked and verified by an authorised medical biochemist. There was no need for corrective actions. The whole documentation including completed templates with laboratory findings and original printouts from analyzers or print screens from HIS were sent to our LIS and HIS vendors. The owners of the company, after carefully examining the data, confirmed the accuracy of transferred data by signing each completed template.

Discussion

The Institute of Clinical Chemistry and Laboratory Medicine, Merkur University Hospital, Zagreb, Croatia, implemented accreditation standards in every day routine work. New accreditation requirements include Annex B with eight requirements in the B5. section regarding safety of data entry and reports. A protocol for validation was defined and the validation process performed at the Departments of Clinical Chemistry and Laboratory Medicine of the Institute of Clinical Chemistry and Laboratory Medicine. The most important problem we encountered was the difficulty in performing retrospective validation of a computer software, since this kind of validation has not been previously described in detail (16,17). There are some “what-to” but not “how-to” articles (18), and the requirements of the ISO 155189 standard are too general, leaving it to laboratories to establish a protocol and standards for the validation process and associated documentation (19). Although initial system validations were performed by vendors of commercial computer software, a documented system validation demonstrates to all concerned that the laboratory information system manages information well, with the expected accuracy and reliability, file integrity, auditability and management control (20,21). An important issue in the process is the massiveness of documentation acquired during the process, as almost 400 different tests were used. Supporting documentation such as screen prints or laboratory findings serve to document the success or failure of the test case. Some discrepancies observed in the process, already described in the previous section, were carefully examined by the authorised medical biochemist and software vendors, and were concluded not to interfere with daily workflow nor the accuracy of final laboratory finding. Although this procedure is very comprehensive and time-consuming, it is mandatory in achieving accreditation requirements as set in the Annex B of the ISO15189 Accreditation Standard.

Limitations to this study were that there among the eight sections in Annex B of the ISO 15189 Accreditation Standard our study focused only on B5. section that is related to safety of data entry and reports, whereas there are other features of LIS that require validation. In addition, data for young children reference intervals were not taken into account. However, this protocol for validation process, developed at the Institute of Clinical Chemistry and Laboratory Medicine, ensures traceability of data transferred within and between LIS and HIS, and ensures that the equipment and laboratory information systems used meets their intended purpose, minimizing some types of medical errors in laboratory diagnostics.