Evaluation of the Integrated Health Information System (IHIS) in Public Hospitals in Cyprus Utilizing the DIPSA Framework.

INTRODUCTION: The Ministry of Health of the Republic of Cyprus has implemented an Integrated Health Information System (IHIS) in two hospitals. However, no evaluation of IHIS has been conducted to assess its safety, efficiency and effectiveness. The proper utilization of IHIS is essential for the provision of quality healthcare services. AIM: The purpose of this study was to evaluate the current IHIS in public hospitals in Cyprus utilizing the DIPSA evaluation framework.
METHODS: A total of 309 subjects, including doctors, nurses and other healthcare professionals, participated in the study. The DIPSA evaluation framework assessed the users' perception in five categories namely, satisfaction, collaboration, system quality, safety and procedures, using Likert scale and 3 open questions. Correlation between the categories was assessed using the Pearson correlation coefficient, and multiple regression analysis was used to examine the relationship between the demographic characteristics and categories. Data analysis was done using SPSS v24.
RESULTS: All five categories were rated moderately, between 2.5 and 3, by the participants. All categories were correlated (P < 0.01). Multiple regression analysis indicated the need for improvement between the professionals (mainly doctors and nurses) and the categories. The open questions pointed out the need for improvement in all 3 factors examined (Technology, Human Factor, Organization). DISCUSSION: The moderately rated categories, in the Cyprus IHIS, suggest that there is a lot of room for improvement. Some interventions are suggested that could positively and simultaneously affect one or more categories.

INTRODUCTION

The use of technology and information in healthcare can help collect, process and share information within an organization (1-5). With proper utilization, IHIS can increase the effectiveness and quality of healthcare services (6, 7), improve the provision of services, clinical procedures and their effectiveness (8, 9), reduce errors, provide support to healthcare professionals and improve management and sharing of information (2, 3, 10).

If IHIS is not used correctly, it can adversely impact healthcare and the absence or provision of incorrect information could lead to erroneous decisions concerning the patient’s health and even cause harm (11). The same outcome or delays in decision-making can also occur due to inherent technological problems, such as bugs, crashes or a non-user friendly environment (12, 13). Taking all this into consideration, it is important for an IHIS to be continuously assessed (14, 16).

Even though the Ministry of Health of the Republic of Cyprus implemented an IHIS in two public hospitals in 2007 (19), 10 years later, it has never been assessed.

AIM

The purpose of this study was to evaluate the current IHIS in public hospitals in Cyprus. The utilized DIPSA framework evaluated the following categories: satisfaction, collaboration, system quality, safety and procedures.

METHODS

Sample

The research was conducted in 2017 in Cyprus. Approximately 3.200 staff utilized the IHIS in the Nicosia General Hospital (NGH) and the Amochostos General Hospital (AGH), and 1503 were healthcare professionals, including doctors, nurses and allied healthcare professionals (23). In total, 309 subjects participated in the study, a sampling that is representative of the general population with a confidence interval of 95% and a margin of error of 5% (24). A stratified random sampling was used based on the profession and the hospital of each participant, for the selection of the sample.

DIPSA evaluation framework

The IHIS in the public hospitals in Cyprus was assessed by the DIPSA evaluation framework, as previously described in Stylianides et al, 2018 (14). Briefly, it consisted of a questionnaire with demographic characteristics, 42 questions in the Likert scale and 3 open questions. It measured different categories, namely, satisfaction, collaboration, system quality, safety and procedures within the three main factors identified as Human Factor (collaboration and satisfaction), Technology (system quality and safety) and Organization (procedures). Within the categories, satisfaction measured user’s satisfaction with the IHIS in relation to the effort spent, the quality of information provided and the performance. Quality of the system was evaluated in relation to its availability, reliability, and access and quality of information provided. Users were also asked if the IHIS supported collaboration between the healthcare professionals. Daily procedures were evaluated under the category Procedures, and if the system was beneficial and ensured safety of the patients.

Statistical analysis

Descriptive and inferential statistics were used. Demographic characteristics included gender, age, working experience (in years), profession and the hospital. The 42 questions were grouped into categories utilizing factor analysis. The 3 open questions were used to allow the participants to suggest ways of improving the system within the different categories.

The average score of the responses from each category was calculated on a scale of 1-5 (Likert scale). Every positive response increased the value of the category, whereas every negative response decreased it. Afterwards, the average was calculated between the values 1-5, with 5 being the highest. The reliability of each category was confirmed with the Cronbach’s alpha coefficient. The average score of the responses were calculated.

Correlation between the categories was assessed by the Pearson coefficient correlation method. Comparison of means was also done between demographic characteristics and categories. Independent samples t-test was used for gender and hospital, and One-way Anova and Bonferroni test “Post-hoc” were used for age, experience and profession. Results from the comparison of means with P -value ≤ 2 (25) were used in multiple regression analysis to examine the relationship between the demographic characteristics and categories. Multiple regression analysis indicated the results with values between 0 and 1, the highest value being 1 it means the result is more important, the lowest the value being 0 it means that it is less important. Data analysis was done using SPSS v24.

RESULTS

Descriptive statistics

The majority (64.1%) of the responders were female, and approximately 65% belonged to the groups up to 39 years old, while about 35% were 40 years old or more. Most professionals were nurses 217 (70.2%), followed by doctors 67 (21.7%) and other healthcare professionals 25 (8.1%). Regarding the working experience in the specific hospital, 213 (68.9%) worked for more than 5 years and 93 (30.1%) were working for up to 5 years. Most of the participants (261–84.5%) worked at the NGH and 48 (15.5%) at AGH (Table 1).

Table 1.

Demographic Characteristics

Gender	Male	109 (35.3%)
	Female	198 (64.1%)
Age	< 30	82 (26.5%)
	30 – 39	118 (38.2%)
	40 – 49	61 (19.7%)
	> 50	46 (14.9%)
Experience to the specific hospital (in years)	< 1	22 (7.1%)
	1 – 5	71 (23%)
	6 - 10	91 (29.4%)
	> 10	122 (39.5%)
Profession	Doctors	67 (21.7%)
	Nurses	217 (70.2%)
	Other	25 (8.1%)
Hospital	NGH	261 (84.5%)
	AGH	48 (15.5%)

Inferential statistics

The average score of the responses in the Likert scale for the categories satisfaction, collaboration, system quality, safety and procedures was calculated. All categories were rated moderately between 2.5 and 2.9 (Satisfaction = 2.53, Collaboration = 2.75, System quality 2.77, Safety 2.83 and Procedures 2.93), indicating that the health professionals were neither completely satisfied nor completely dissatisfied with the different categories. The Cronbach’s alpha of all categories was found to be ≥ 0.837 (Procedures = 0.887, Collaboration = 0.916, Safety = 0.837, Satisfaction = 0.923, System Quality = 0.940). The correlation between the categories was statistically significant (P < 0.01). Table 2 shows the Pearson correlation coefficient (r) for all categories and the highest coefficient was observed between the categories satisfaction and system quality.

Table 2.

Correlation between factors

Correlation		Human factor		Technology		Organization
		Satisfaction	Collaboration	System Quality	Safety	Procedures
Human factor	Satisfaction	1	r = 0.595	r = 0.884	r = 0.662	r = 0.541
	Collaboration		1	r = 0.579	r = 0.667	r = 0.484
Technology	System Quality			1	r = 0.621	r = 0.638
	Safety				1	r = 0.521
Organization	Procedures					1

Table 3 presents the comparison of means between demographic characteristics and categories. P -value ≤ 0.2 were used for multiple regression analysis.

Table 3.

Comparison of means between demographic characteristics and categories for multiple regression analysis (P -value). * The category “other” refers to other healthcare professionals.

	Demographic characteristics	Categories
Type of test		Procedures	Collaboration	Safety	Satisfaction	System Quality
Independent samples t-test	Gender	0.091	0.996	0.243	0.062	0.048
	Hospital	0.275	0.477	0.228	0.031	< 0.001
One-Way ANOVA	Age	0.564	0.352	0.666	0.416	0.839
	Experience	0.192	0.200	0.440	0.830	0.901
	Profession	0.002	< 0.001	0.029	0.003	< 0.001
Bonferroni test	Nurses/ other*	0.005	0.001		0.002	< 0.001
	Doctors/ other		0.001
	Doctors/ other	0.001			0.012	0.002

The relationship between the demographic characteristics and categories assessed by the multiple regression analysis was statistically significant, with the exception of the category safety. Professions had a high impact on all categories. The category procedures had the highest impact on doctors with 0.832 units followed by nurses with 0.661 units, followed by healthcare professionals with experience more than 10 years with 0.485 units. System quality had a high impact on nurses with 0.803 units, followed by doctors with 0.744 units, followed by healthcare professionals in AGH 0.456 units, followed by male gender with 0.221 units. Then, the category collaboration for nurses (0.686 units), followed by healthcare professionals with experience > than 10 years with 0.333 units. Finally, the category satisfaction for nurses with 0.670 units, followed by doctors with 0.619 units. The 3 open questions allowed the participants to suggest ways of improving the system within the different categories. The 5 most common responses were: “Better training to healthcare professionals” 128 (28.9%) “System upgrade” 104 (23.5%) “Keep a log of data/procedures” 98 (22.1%) “Better cooperation between healthcare professionals” 66 (14.9%) and “Better access to the system by all healthcare professionals” 55 (12.4%).

DISCUSSION

The present study presents the first assessment of the IHIS in public hospitals in Cyprus, since its implementation in 2007. Using the DIPSA evaluation framework, healthcare professionals evaluated the IHIS at the Nicosia General Hospital and Ammochostos General Hospital within 5 categories: satisfaction, collaboration, system quality, safety and procedures. Overall, all categories were evaluated as average, indicating that the healthcare professionals were neither satisfied nor dissatisfied with the system, suggesting the need for further improvement.

The category system quality was moderately rated. Technological barriers, such as compatibility issues and being non user-friendly, can limit the use and access to relevant and immediate information (26). The need for upgrading the existing IHIS system in terms of software and hardware, and making the system easier to use was expressed by the participants. Software and hardware upgrade can improve the system quality (27), can also result a better outcome of healthcare services (28-30). Therefore, such measures would improve system quality within the current IHIS, and lead to increased utilization (28).

The highest correlation was observed between system quality and satisfaction, similar to results found in Greece (31) and Indonesia (32). In addition, in Malaysia, Salleh et al. (30) found out that system quality affects not only satisfaction but also effectiveness, that is in agreement with other researchers (33-35). Based on our results, system quality had a high impact on doctors and nurses, and as such, technological upgrade would be beneficial.

While our results indicated that the category satisfaction was rated moderately by the participants, research done in Singapore (36) and in Iran (27) showed that users were satisfied with their IHIS. In these studies, users received training on how to use the systems and were actively collaborating and providing feedback for system improvement. Therefore, an active involvement and knowledge of the system led to high satisfaction (27). Participants in the present study, primarily expressed their need for training on the use of the IHIS. This is reinforced by several studies that show that knowledge on the IHIS (29) and training of healthcare professionals can result in more successful use of technology and improved user satisfaction (2,37-39). Training of the users seem to be an important way of covering the daily needs of healthcare professionals, and would most specifically impact on the satisfaction of doctors and nurses. In addition, as mentioned above, improvement in system quality, such as technological upgrade, can positively impact user satisfaction.

Poor collaboration between healthcare professionals can result in mistakes/errors (40), and therefore collaboration ideally should be rated high. Collaboration between healthcare professionals at both hospitals in Cyprus scored moderately, which was further emphasized by their expressing the need for better collaboration, and impacted mostly nurses and all healthcare professionals working for more than 10 years within the hospital. Improving communication and respect between the different groups of healthcare professionals, leading to increased collaboration, could be achieved through joint training, workshops and seminars clearly identifying the duties and responsibilities of each group (41).

Procedures can affect positively or negatively the outcome in healthcare (39). Implementation of policies and guidelines have been shown to increase the use of IHIS, and improve the outcomes and procedures, and provide a holistic approach towards the patient (42). In addition, provision of detailed instructions on the daily tasks within the clinical environment, could also result in a better outcome. These are in agreement with the declared needs of the participants in our study that requested the maintenance of a data log and guidelines for the existing processes, and should have an impact on doctors and nurses, and healthcare professionals with more than 10 years of experience.

Safety of patients depends on multivariate causes. Having as a central point the user, correct use of technology (43) or issues pertinent to the user such as fatigue, shortcuts and even reduced communication and cooperation between health professionals can have a negative impact (44). The category safety could be improved through simulation of patient scenarios allowing for measurements of the safety and quality of the system, before its actual use (45); training of the users to increase their interaction with the IHIS; increased or improved cooperation between professionals, all of which lead to increased satisfaction and the provision of safe healthcare (40). Up-to-date technology can also increase communication in order to improve better healthcare quality and access (46, 47). In line with the needs of the participants in the two public hospitals in Cyprus, the implementation of “smart” technology could provide quick access to information, as well as, faster response in healthcare emergencies (48, 49). Moreover, better access to the information can result in better decision making regarding the patient’s health and better collaboration (49).

While we tried to individually separate the categories and their impact on the evaluation of the IHIS in Cyprus, the fact that all categories are correlated, and that any single one of them affects the other, all the suggested actions are expected to have an overall and simultaneous effect on the system. For example, training of the user affects the implementation of procedures/data logs in the system, but also seems to affects positively the category satisfaction (50), and could improve collaboration between healthcare professionals (41) and the safe use of the system (40). Software and hardware upgrade in the category technology could improve satisfaction (33), as well as, the system quality. “Smart devices” will impact on the collaboration (51) and access (48, 49) of healthcare professionals to the IHIS in a positive way.

The main limitation of the study was that the only stakeholders included were healthcare professionals.

CONCLUSION

The IHIS in public hospitals in Cyprus was evaluated for the first time since its implementation in 2007. The results suggest that all areas assessed could be improved, with more emphasis in relation to system quality and the satisfaction of the users. Moreover, special attention should be paid to the different health professions (e.g. doctors and nurses), and those professionals with more than 10 years of work experience. There is a need for the evaluation and follow-up of the IHIS on a regular basis for the provision of effective and quality healthcare.