An Evaluation Protocol for Picture Archiving and Communication System: a Systematic Review.

INTRODUCTION: Picture archiving and communication system (PACS) serves to store, transmit, communicate and manage medical images. A logical evaluation protocol assists to determine whether the system is technically, structurally and operationally fit. The purpose of this systematic review was to propose a logical evaluation protocol for PACS, particularly useful for new hospitals and other healthcare institutions in developing countries. METHODS AND MATERIALS: We systematically reviewed 25 out of 267 full-length articles, published between 2000 and 2017, retrieved from four sources: Science Direct, Scopus, PubMed and Google Scholar. The extracted data were tabulated and reviewed successively by three independent panels of experts that oversaw the design of this study and the process by which the PACS evaluation protocol was systematically developed.
RESULTS: The outcome data were ranked by expert panels and analyzed statistically, with the reliability established at 0.82 based on the Pearson's correlation coefficient. The essential components and the best options to establish an optimal PACS were organized under nine main sections: system configuration;system network;data storage; datacompression;image input; image characteristics; image presentation; communication link; and system security, with a total of 20 components, each of which capable of working optimally with one or more program options.
CONCLUSIONS: This systematic review presents an objective protocol that is an ideal tool for the evaluation of new or existing PACS at healthcare institutions, particularly in developing countries. Despite the significant advantages, the protocol may face minor limitations, largely due to lack of appropriate technical resources in various clinical settings and the host countries.

1. INTRODUCTION

Picture archiving and communication system (PACS) is a complex unit, which is used for capturing, transmitting, storing, distributing, displaying and interpreting medical images. In recent years, PACS has been increasingly utilized by radiology and other hospital departments due to its important advantages, such as transmission speed, productivity, simultaneous access to medical images from distant clinics settings, rapid image examination and archiving, and for being cost-effective (1, 2). Further, PACS reduces doctor’s dependence on technicians, and facilitates rapid consultation among physicians and surgeons from multiple clinical settings. This system omits repetition for taking digital images and resolves such problems as the loss of videos, the need for recording them multiple times, and the associated high costs (3, 4).

In many hospitals, however, storing medical images is linked to the existing technologies at the radiology or medical records departments, which can affect the quality and integrity of the data (5). Due to its wide acceptance, establishing PACS has been considered as a logical strategy by the executives of many hospitals worldwide (6). However, unless it is installed and operated properly, this system can lead to problems with significant consequences and repeated maintenance costs (2). For instance, PACS may lack the required efficiency and flexibility if installed and operated improperly, largely due to the lack of a logical evaluation protocol (7). Using a systematic and objective evaluation protocol significantly helps identify and remove the technical and logistic deficiencies, and ensures that the system has the best technical, operational and clinical quality and efficiency, that are based on standard guidelines (7).

The purpose of this study was to provide a state-of-the-art evaluation protocol for PACS, based on a systematic review of the published literature since year 2000. The evaluation protocol provided by this study should greatly assist hospital executives and decision makers, particularly in developing countries, to acquire and utilize the most efficient and cost-effective PACS package for their healthcare institutions.

2. MATERIALS AND METHODS

Guided by expert radiologists and academicians in digital field, we planned and conducted this systematic review to develop a protocol for the evaluation of PACS. We used Delphi method (8, 9) to validate the initial questionnaire and the final PACS evaluation protocol. Delphi is a structured, evaluation technique, relying on the answers and guidance received from one or more expert panels. Our study design consisted of the following order and steps:

Literature Search: Using the essential key terms, such as digital imaging and communications in medicine (DICOM), PACS, and the evaluation protocols, we searched for English articles that had elaborated on the conceptual, logical and physical perspectives for the development of PACS. Additional selection criteria for the articles were as follows: being available in full-length manuscripts and published in reputable databases, such as Science Direct, Scopus, PubMed and Google Scholar since year 2000. We found 25 out of 267 full-length articles that were consistent with the scope of the study. The 25 articles were approved for the systematic review by our first expert panel, consisting of the following four individuals: a) a senior radiologist with experience in working with PACS; b) two individuals with expertise in medical informatics and image processing; and, c) an academician, specialized in health information management.

Data Collection & Ranking: The first expert panel carefully reviewed the 25 selected articles and extracted the essential components and required options for a successful PACS package. The extracted information were tabulated under various categories in a preliminary questionnaire that was sent to a group of 42 expert faculty and clinicians from the schools of medicine at Amir Kabir, Iran, Shahid Beheshti and Tehran universities in Tehran, Iran. These individuals, with equally advanced expertise, were asked to rank the importance of each of the necessary PACS components on a scale of one to 100.

Data Validation: The completed questionnaires that were returned by each member of the second panel of experts were compiled and the ranked data were analyzed by descriptive statistical methods. The results were reviewed by the third panel of experts, consisting of five individuals, with the similar expertise as those in the first and second expert panels. Lastly, the selection of the final items assigned for inclusion in the proposed PACS evaluation protocol was approved by the third expert panel as follows:

Items ranked below 50% for importance and relevance, were omitted.

Items ranked at 51-74% for importance, were reviewed by the 3rd expert panel for a second time and ranked again. If the second round ranking for any item was below 50%, it was omitted. All items ranked at 51-74% in the second round as well as those ranked at 75-100% for their importance and relevance, were included in the protocol.

The final items in the proposed protocol (Table 1) were reviewed individually again and were approved for release to the medical and professional community.

Table 1

Essential Components and Applications for PACS. Abbreviations: WAN = Wide Area Network; MS-SQL= Microsoft Structured Query Language;

ATM = Asynchronous Transfer Mode; TCP/IP = Transmission Control Protocol/Internet Protocol; ISDN = Integrated Services Digital Network; DICOM = Digital Imaging and Communications in Medicine; LCD = liquid-crystal display; HL7 = Health Level Seven.

3. RESULTS

The data derived from the questionnaire were compiled, ranked and reviewed by the three expert panels. The data were analyzed statistically, with the reliability determined at a Pearson’s correlation coefficient of 0.82. The essential contents of the protocol, as verified by the expert panels, were tabulated in Table 1 under nine main sections: system configuration; system network; archiving and compression; image input; image characteristics; communicative link; software properties; and system security. Each section required multiple components and options for proper PACS operation, as shown in Table 1.

System Configuration: There were five essential components required for the system configuration to operate properly as follow: i) The basic processing was essentially Intel-based.

ii) The operating system required Windows, Linux or Mac program option. iii) The program language was either C++, Java or Visual Basics. iv) Similarly, the system database needed either Oracle, MS SQL or MySQL option. v) The user interface had a total of five options to work with the system input and output: using a mouse, a keyboard or a touchpad, or the system could use the three options simultaneously. The output could be printed on laser or inkjet printers, or be displayed on digital monitors.

System Network: The essential components of the system network were the architecture, communication protocols, transmission cables and wide area network (WAN). The system architecture was either client server-based or web-based. The communication protocol employed ATM, Ethernet or TCP/IP standard, with the transmission means being fiber optic, twisted pair, or CAT series cables. The best option for networking (WAN) was either ISDN, ATM, or Internet.

Data Storage and Compression: The data storage means was accomplished on either optical disks or hard disk drives (HDD), with the compression being readily retrievable by the user or stored permanently (lossless), which is not easily retrieved.

Image Input & Characteristics: The capturing of digital images was achieved either directly or by mosaic method, although most images could be digitized, compatible with DICOM. The image characteristics involved various matrix sizes, such as: 1024 x 1024, 1024 x 1280 or 2560 x 2048. The images could be displayed on 21-inch or larger monitors in color or black and white at a resolution of 2048 x 2048, 2560 x 2048 or 4096 x 4096, and presented at multiple work stations. The image access time was dependent on image size and network capability.

Communication Link, Security & Software: The communication utilized HL7 or DICOM linkage, and the identity was maintained through assigned username and secret password for individual users. The system software had the capability of image processing, statistical analyses and other database management.

4. DISCUSSION

Previous studies have examined the optimal components for the PACS configuration (10, 11). Also, the programming language (11-15) and database management have been evaluated (15, 16). The protocol presented by this systematic review is consistent with the solid recommendations of these and other studies for the evaluation of the PACS configuration. Other requirements such as the best operating systems, basic processing, program language and the output options were also included in the evaluation model (17). Therefore, our protocol is not only consistent with the recommendations of previous studies on PACS, but also provided the state-of-the-art steps necessary for the smooth operation of modern PACS at hospitals and other healthcare institutions.

Other studies (12,18, 19), have strongly suggested the use of such system network as client servers or web-based architecture, using ATM, Ethernet and/or TCP/IP protocols. Hence the inclusion of these network elements in our PACS evaluation protocol among the essential prerequisites. It is evident that communication protocols are of a great importance for the optimal operation besides the architecture issue (12). One study has suggested that cloud computing might be considered as a network architecture (20); however, ample evidence in support of this idea, especially from developing nations, were lacking at the time this review was being conducted.

Data storage and image retrieval, using optical disk or HDD, are the key components in an optimal PACS, hence the reason we included them as the vital constituents in the evaluation protocol. Consistently, these elements have been recommended by well-founded previous research (11,21-24).

As elaborated by previous researchers (12, 13), data compression can impact PACS, both in short-term and long-term data, and it can reduce the file size for exchange among users. Although this could negatively impact the quality of images, the rate and method of image compression were addressed in this review and included in the PACS evaluation protocol. Having the data compression feature may influence the costs of initial equipment purchasing and the maintenance; however, this feature provides a major advantage to the data communication process.

It has been suggested that the best mode of capturing images is the direct reception from video recording devices (11, 13). In the present review, digital capturing of images and the digitizers’ properties were carefully examined with our main focus on resolution, as reflected in the evaluation protocol. Also, the property of data and image viewers necessitates that the image resolution is matched with various applications in PACS (23). In this study, other factors such as matrix size, viewing device, work stations and number of monitors were all examined, then included in the evaluation protocol. As recommended by another study (10), work stations should be classified based on their image resolution so that users can choose them based on their needs. This study (10) suggests that CRT monitors should be replaced by LCD ones so that color images can also be viewed.

Studies have emphasized the proper use of communication links for the optimal operation of user-equipment interface in PACS (20, 25). Other studies (12, 21, 22, 26) recommend that data security to be observed according to HIPPA regulations. This issue was addressed heavily in the present review and was included in the evaluation protocol.

Evidently, there are different capabilities and divisions in any system. Among those considered in the present systematic review, are image processing capability, management and statistical capabilities, which are reflected in the evaluation protocol, consistent the recommendation of earlier studies (10, 11, 16, 20, 27). Although the software components may vary, depending on the institutional needs, the PACS capabilities are largely similar, if not identical, in different nations. This is because the nature of collecting, storing and utilizing the data has been standardized in recent years and in various healthcare institutions.

5. CONCLUSIONS

Modern hospitals spend a significant amount of funding to establish PACS, which is not only a vital component to their radiology departments but also serves the entire institution. This system holds a large amount of medical data and patient information, and therefore, must be operated optimally, maintained systematically, and protected from improper components and technical options. Equally importantly, PACS must be readily able to communicate the large amount of medical information and images it holds with other users, such as physicians and professionals at other hospitals and healthcare institutions within and outside the city and country. This can only be materialized if PACS is made up of appropriate components, options and applications that are capable of providing for optimal operations for many years.

Based on validated information searched from 25 publications, and the oversight and careful inputs received from three expert panels, this systematic review has developed a state-of-the-art protocol for the evaluation of the nine components that function best with multiple program options in a logically installed PACS. Despite the significant advantages of the proposed evaluation protocol, it may face minor limitations, largely due to lack of appropriate technical resources in various clinical settings and the host countries.