A Survey on the Users' Satisfaction with the Hospital Information Systems (HISs) based on DeLone and McLean's Model in the Medical-Teaching Hospitals in Isfahan City.

BACKGROUND AND
PURPOSE: The user's satisfaction with information system in fact denotes the extent the user is satisfied with the system's achievement in fulfilling his/her information requirements. This study tries to explore the users' satisfaction with hospital information systems (HISs) based on DeLone and McLean's model focusing on the medical-teaching hospitals of Isfahan city.
METHODOLOGY: This study which was applied and descriptive-analytical in nature was carried out in the medical-teaching hospitals of Isfahan city in 2009. Research population consisted of the system users from which a sample was selected using random sampling method. The size of the sample was 228. Data collection instrument was a self-developed questionnaire produced based on the satisfaction criterion in the DeLone and McLean's model. Its content validity was assessed based on the opinions given by the computer sciences professionals with its estimated Cronbach's alpha found to be 92.2%. The data were analyzed using SPSS software.
FINDINGS: As the findings of the study showed, the differences among the mean scores obtained for the satisfaction with different kinds of HISs in use in the hospitals were statistically significant (p value≤0.05). Generally, Kowsar System (old version) and Pouya Samaneh Diva system gained the highest and lowest mean scores for the criterion in question, respectively. The overall mean score for the satisfaction was 54.6% for different types of systems and 55.6% among the hospitals.
CONCLUSION: Given the findings of the study, it can be argued that based on the used model, the level of users' satisfaction with the systems in question was relatively good. However, to achieve the total optimum condition, when designing the system, the factors affecting the enhancement of the users' satisfaction and the type of hospital activity and specialty must be given special consideration.

1. INTRODUCTION

As one of the essential constituents of the medical and healthcare systems, hospital information system (HIS) integrates health data collecting, processing, analyzing and reporting and provides us with appropriate indicators for checking and assessing the health system performance (1, 2). This system is an comprehensive software which integrates the patient-related data to be exchanged among different departments and medical centers in a way that it can speed up the care and treatment process, enhance the satisfaction, improve the services quality and decrease the costs (3). It automatically manages the data related to the clinical, financial, nursing, laboratory, pharmacy as well as radiology and pathology departments (4). The HIS includes 8 sub-systems (clinical information system, financial information system, laboratory information system, nursing information system, pharmacy information system, the picture archiving and communication system and radiology information system) (5).

According to the World Health Organization, HIS’s primary function is globally development of patient data mechanized service which leads to the better efficient retrieval of the data required for treatment, statistics, teaching and research purposes. HISs are designed for the integrated collection of data, their processing, reporting as well as using essential data for improving the efficacy and effectiveness of health services through a better management across all the levels (6).

Despite their numerous benefits, these systems suffer from some weaknesses (7) including designing the system without considering the fundamental and well-known standards (8) and inflexibility of the software (9). Furthermore, the results of the previous evaluations on HISs all revealed that they have some defects and limitations. Accordingly, to identify and resolve these problems, the need to continuous evaluation before, during and after system implementation is strongly felt (8). Continuous evaluation is considered as one of four main steps followed in creating and developing information systems (10) and its aim is determining the performance of the HISs’ sub-systems and identifying their problems (11). As Geisler et al (1998) argue, the main function of evaluation is to determine to what extent the performed action meets the stipulated standard, to improve the efficiency and effectiveness, to provide a basis for programming and a communicative tool for relating other activities in the medical and healthcare information systems (12).

Evaluation is an endless continuous process (7). Without analyzing the users’ perception about an information system, its evaluation is impossible. The users’ satisfaction is assumed as a performance guarantee for an information system (13). A system which fails to satisfy the users’ requirements and is not consumer-based is judged as a weak system (14). In fact, one of the biggest reasons why an information system fails to achieve some of its planned goals is either ignoring or paying insufficient attention to the human factors. This, in turn, would lead to a failure in establishing a proper interface between the system and its users and creating a sense of ownership in the users towards the system (15). Numerous models have been proposed for evaluating HISs one of which is the DeLone and McLean’s model of information systems success (16). After an all-encompassing survey on the criteria efficient in evaluating the information systems, they proposed their model which consisted of six main criteria and one comprehensive model for evaluating the information systems. In this model, six main dimensions covering the overall performance of the system are emphasized (17) one of which is the user’s satisfaction. User’s satisfaction signifies the response to the use of the information system output (18). In this study, the indicators relating to the system’s success are mostly discussed. They include user’s satisfaction, satisfaction with the hardware and software, satisfaction with the system development project, the user’s complains regarding the information system center, the users’ satisfaction with the intermediaries (16, 19).

Ribiere in his study entitled “Hospital information systems quality: a customer satisfaction assessment tool” concluded that the best approach for maintaining the user’s satisfaction with HIS is to design the system based on the system users’ and not the system developers’ ideas and needs (15).

Vassilios in his article “Methods for evaluating hospital information systems: a literature review” has enumerated the user’s satisfaction as one of the necessary approaches for evaluating the HISs. He finally concludes that compared to other methods, evaluating the user’s satisfaction with the information systems may be the most effective evaluation method (14).

Evaluating the HISs based on user’s satisfaction criterion is of paramount importance, since system users are in fact the consumers of the system and its information and services so that by recognizing the factors accounting for the users’ dissatisfaction and analyzing them, the system quality may be enhanced resulting in increased medical care quality (15).

The present study intended to explore the rate of users’ satisfaction with HISs based on DeLone and McLean’s model in the medical-teaching hospitals in the city of Isfahan.

2. METHODOLOGY

This research which is applied- descriptive in nature using analytical method tried to evaluate the HISs in use in the medical-teaching hospitals situated in Isfahan city, Iran. This evaluation was carried out based on the user’s satisfaction criterion expressed in the DeLone and McLean’s model. To do so, 11 medical-teaching hospitals (Shahid Beheshti, Shahid Chamran, Noor & Ali Asqar, Imam Musa Kazem, Is Ibn Maryam, Al-zahra, Ayatollah Kashani, Feiz, Seyyed Al-shohada, Farabi and Amin) located in Isfahan city were involved. The study period was from June to February in 2010. Research population included the HIS’s developers, hospital’s IT authorities and HIS’s users in the medical-teaching hospitals in the Isfahan city. It is noteworthy that Najaf Abad’s Modares hospital which belongs to the medical-teaching hospitals of Isfahan possessed no HIS while the Al-Zahra as a public hospital was just at early stages of this system commissioning and its pilot implementation. As a result, they were omitted from the research population.)

As for the HIS’s users, first, the total number of the users in each hospital was calculated. Then, the size of the sample was estimated based on the minimum and maximum number of the users in the hospitals in question using the following equation:

The sample size was found to be at least 20 for each hospital which in sum, for all the hospitals, at least 220 people were involved (random sampling method was used for selecting the users’ group).

The study data were collected using a self-designed questionnaire developed based on the users’ satisfaction criterion following DeLone and McLean’s model. To distribute the system users’ questionnaire among the hospitals, the researcher visited them in person. The content validity of the questionnaire was assessed referring to the ideas and viewpoints given by the department’s professors as well as the computer sciences field’s experts, HIS’s authorities and information management and health informatics personnel in the field offices.

To assess its reliability, Cronbach alpha coefficient calculated by SPSS software was used which was found to be 92.2%.

After final control, the collected data were put into the SPSS software 18. To analyze the data gathered by the measuring instruments and transform the qualitative responses into the quantitative ones, the item- weighing method was used. Evaluating the respondents’ views was done using 5–point Likert scale. Finally, the mean scores gained for the criteria in question were compared using one-way variance analysis test.

3. RESULTS

The demographic characteristics investigated included gender, age, level of education and field of study. The greatest number of the users aged 20-30, 70.6% of them were female and 49.6% were graduate student.

In Table 1, the results of the comparison of the mean scores obtained for different types of HISs in the research population have been summarized. For this criterion, the highest and lowest mean scores were found to be for Kowsar System (old version) (62.9 %) and Pouya Samaneh Diva System(47.5%), respectively.

Table 1

The mean scores gained for the attributes of satisfaction with different types of HISs in the research population

Based on the one-way variance analysis, the differences among the mean scores for the satisfaction with different types of HISs were statistically significant not being the same (p value≤0.05).

As it can be seen from table 2, on the satisfaction criterion, among different hospitals in question, Beheshti hospital gained the highest mean score (0.63%) while the lowest mean score found to be for Amin hospital (47.5%).

Table 2

The comparison of mean scores gained for satisfaction among different hospitals

4. DISCUSSION

Considering the results, it can be inferred that among the attributes of satisfaction with different HISs, the attribute “satisfaction with the intermediaries” and “satisfaction with the software and hardware” gained the highest and lowest rank, respectively. Therefore, it can be claimed that the systems under research suffer from some limitations and drawbacks as far as the attribute “satisfaction with the software and hardware” is concerned.

Lee et al in one research entitled “Implementation of physician order entry: user satisfaction and self-reported usage patterns” found that the users generally were satisfied with this system (20) which is relatively inconsistent with the results of the present study which revealed a relatively desirable level of satisfaction.

As for the satisfaction components across different hospitals, the results were similar to those obtained for the components of satisfaction with different types of HISs.

As for satisfaction criterion among different hospitals, the 11 hospitals were at a relatively favorable level (54.6%).

This claim that hospital’s type of activity (medical-teaching) and type of specialty (general/specialty hospital) certainly play a direct and remarkable part in the performance of HIS may justify the difference observed in the results obtained for satisfaction criterion across different hospitals. As it is evident, the mean scores for the satisfaction criterion among different hospitals were statistically different. This can be attributed to the probable effect of hospital’s activity and specialty type. When designing the HIS, the foregoing parameters especially the type of specialty must be taken into account.

5. CONCLUSION

It seems to be an inevitable necessity to develop and evaluate HISs based on standard and well-known models. In this way, we can ensure the designed system is applicable fulfilling the users’ satisfaction. In sum, the findings of this study showed that the six HISs in question were significantly different in the level of users’ satisfaction with Kowsar System (old version) enjoying the highest rank. As for comparison among different hospitals, Beheshti hospital gained the highest mean score. This can be explained by the possible effect the type of activity and the type of specialty have on the system performance. On the contrary, the lowest mean score belonged to Pouya Samaneh Diva System.

Although Kowsar System (old version) gained the highest mean score on the satisfaction criterion, it has some limitations and defects which need to be addressed. Some of its limitations are as follows:

The availability of suitable software and hardware infrastructures and a computer network required for implementing the system;

Daily virus scanning using the most advanced virus detection software;

Using up-to-date software and hardware in the development of the system;

Using appropriate data banks for the system.