Implementation Status of Health Management Information System in Hospitals of South West Shoa Zone, Oromia, Central Ethiopia.

BACKGROUND: Health Management Information Systems (HMIS) is an information system for health management and planning at district, regional, and national levels. Problems of data collection and processing, poor information use, lack of identifying information required at specific levels, and inadequacy of resources are hindering the implementation of the program.
OBJECTIVE: To assess the implementation status of Health Management Information System (HMIS) in hospitals of South West Shoa Zone, Oromia, Central Ethiopia.
METHODS: The assessment was conducted in South West Shoa Zone hospitals using a cross-sectional study design. Using three dimensions, namely, availability of resources, compliance to guideline, and utilization of information were studied. All hospitals in the zone were included and 210 health workers were included in the study. Quantitative data were entered into Epi-data and analyzed using SPSS.v.23. Descriptive summery was done using the proportion and frequencies and qualitative results were analyzed thematically. The study was conducted from February 20, 2020, to March 20, 2020.
RESULTS: All hospitals have at least one computer specifically assigned for HMIS, while none of the hospitals have an allocated budget for HMIS. Regarding service training, 82% of the respondents had received HMIS training. The overall Implementation status of HMIS was 58%, which showed as a judgment parameter of "poor" implementation. Particularly, availability (58%) and compliance (55%) dimensions are categorized as "poor" implementation, whereas the information utilization dimension is categorized as a judgment parameter of "fair" implementation.
CONCLUSION: The overall implementation status of HMIS in the study area was poorly implemented according to the preset criteria with stakeholders. Therefore, allocation of budget specifically for HMIS, strengthening capacity building activities such as training and supportive supervision with written feedbacks are recommended for effective implementation of the Health Management Information System.

Introduction

Health Management Information Systems (HMIS) refers to information systems for health management and planning at district, regional, and national levels.1 They are a process whereby health data are recorded, stored, saved, and processed for decision-making.2 Management Information System is a set of components and procedures organized with the objective of generating information that will improve healthcare management at all levels.1 The Health Management Information System generates information about the quantity, quality, distribution, and reach of health and service provision, resources needed to provide those services.3 In public health programs it is primarily concerned with issues like healthcare delivery, disease control programs, and administrative issue.4

Worldwide HMIS have been found to be important tools for health reforms. As stated by the World Health Organization (WHO) the proper collection, management, and use of information within healthcare system determines the systems’ effectiveness. As a result, assessing the systems’ own environment and organization must be conducted for strengthening countries’ health information systems.5,6 In Ethiopia, HMIS was launched in 2008 with the goal of creating comprehensive and standardized national HMIS for evidence-based decision-making at all levels. It focuses on information use, data quality, human resource, information communication technology (ICT), and financial resource and is believed to improve health system efficiency and effectiveness.7,8

Health Management Information Systems (HMIS) are essential constituents of effective health programs in developing countries.9,10 Today there is a growing interest for quality information by different level managers for evidence-based decision-making. Quality information is the most important product of HMIS, and it is the base for monitoring and evaluation (M&E). However, this quality information is not delivered from HMIS and M&E become weak due to a lack of reliable data supply from HMIS.11,12 The weaknesses of an HMIS are reported in the areas of incomplete institutionalization, non-standardized data collection and reporting, weak information use, and limited resources.11 The government of Ethiopia and Federal Ministry of Health (FMOH) direct resources and responsibilities for delivery of public services towards health institutions like Woredas, hospitals, health centers, and health posts.12,13

According to an implementation evaluation conducted in Ethiopia in 2009, the identified problems in implementing HMIS were non-fulfillment of required inputs such as human resources in card rooms, health information technicians as per standards, and deficiencies of some forms and registers were the major ones. According to the health sector transformation plan (HSTP) in Ethiopia the major problems of HMIS implementation status are data quality and information use at all levels, which is due to many factors, like the lack of attention given to HMIS, the shortage of resources, and lack of a strategic plan as well as lack of integration, absence of standards and guidelines, inadequate staffing, and poor ownership.19 Other problems identified by the study as hindering the HMIS implementation were parallel reporting, shortage of supportive supervision, and review meeting as per the standard.11,18

As a result, the assessment of HMIS implementation status as well as identifying challenges to implementation of HMIS is very important. Therefore, this study identifies problems of availability, compliance, and utilization of data faced by hospitals during the implementation of the HMIS and assesses the contribution of follow-up to strengthen the HMIS system at hospitals.

Methods

Study Area and Period

Southwest Shewa Zone is one among the zonal administrations located in Oromia Regional State, 114 km from the capital city of Ethiopia, Addis Ababa. The total population of the zone was 1,101,129. Southwest Shewa zone is comprised of four government hospitals, one other government organization (OGO) hospital, 54 health centers and 264 health posts. The hospitals were named as St. Luke, Tulu Bolo, Ameya, Leman, and Bantu hospitals, respectively. There are also 72 private clinics and 28 drug stores in the zone. The study was conducted from February 20 to March 20, 2020 in hospitals of South West Shoa Zone.

Study Design

Cross-Sectional Study Design

Sample Size

According to the WHO guideline for district health systems for assessing its functionality17 all hospitals in the zones were included in the study. A total of 210 study participants were included in the study. The study subjects are HMIS focal persons, hospital managers, care providers, and case team heads of hospitals. For the availability dimension 10 HMIS officers were interviewed because the responsible person for the resources are controlled and availed by the HMIS officers. For the compliance dimension a total of 150 care providers who work in the hospitals were interviewed. For the information utilization dimension a total of 50 case team and hospitals heads were interviewed. For qualitative data 15 key informant in-depth interviews were conducted. The participants of key informant in-depth interview were five HMIS focal persons of hospitals, five maternal and child health head of hospitals, and five medical directors.

Data Collection Tool

For quantitative data an interviewer-administered structured questionnaire which contains interviewee’s socio-demographic background and position of the respondent in the organization that assesses availability of resources, compliance to standard and implementation reliability as well as utilization of HMIS information was developed. Questionnaires were adapted from the Health Matrix Network (HMN) and PRISM frame work.15,16 For qualitative data an interview guide was used to assess the availability of resources, compliance to standard, and utilization of HMIS information. An in-depth interview was conducted with key informants. Twelve months document review templates, resource inventory and physical observation checklist were also developed and used for reviewing documents and observation, respectively.

Data Collectors

Six BSc health professionals who know the local language and are trained on HMIS were recruited for data collection. Two supervisors with a master’s degree in health informatics and trained on HMIS were recruited. One day training on data collection procedures was given for data collectors and supervisors at the same time 2 days prior to data collection. Trained data collectors and supervisors participated in pre-testing to adapt themselves to the tools for the next work.

Data Analysis

For quantitative data the interviewee questionnaire was entered into SPSS version 23.0 and Microsoft office Excel 2010 by the principal investigator and the analysis and descriptive summary was done using proportion and frequencies. Finally, the dimension of the implementation status of HMIS was judged based on the pre-set judgment matrix. For qualitative data the notes of field books were transcribed, and the transcription was coded, the codes were categorized and finally the categories were grouped to the emerging themes. Then, qualitative data were analyzed based on the emerging themes. Finally, the result of quantitative data was presented and supplemented by the result of qualitative data.

HMIS Implementation Dimensions

Availability dimension: presence of inputs (resources) for the implementation of an HMIS program which include human, financial, and material resource in the hospitals.

Compliance dimension: is the level at which hospitals and health workers compile to the national guideline as well as delivering the proposed activities.

Utilization dimension: is using information of HMIS at all levels. It was assessed by using information for decision-making to take immediate action, presentations, data displays recommended by reformed HMIS to be displayed, quarterly minute performance review meetings, receipt of quarterly supportive supervision using HMIS information, and discuss it on the performance review meeting.

Judgment Criteria

The weight of dimensions and the respective indicators was given depending on their level of relevance to the program and discussion with the stakeholders. In each implementation dimension detailed indicators to decide the performance of the data quality were used. The value of dimensions is the sum of respective indicators, then the sum of all dimensions was attributed to the implementation status of HMIS (Table 1).

Table 1

Matrix with Its Judgment Criteria of Implementation Status of HMIS in Hospitals of SWS 2020

Dimension	Recommended	Weight Given	Observed	Percentage Achieved (%)
Availability	100	30
Compliance	100	40
Utilization	100	30
Overall Implementation	100	100

Notes: Judgment criteria: Effectively implemented if weighted value is ≥80; Fairly implemented if weighted value is 79–60; Poorly implemented if weighted value is <59.

Ethical Consideration

Appropriate research ethical clearance was obtained from the ethical review committee of South West Shewa Zonal health office (reference number: WEFG/542/2019, July 2019). This study was conducted in accordance with the Declaration of Helsinki: each study participant was well informed about the aim of the study, benefits, and risks; informed written consent was secured from study participants; study participants’ confidentiality was maintained; no personal identifiers were used in the data collection questionnaire, and codes were used in place of them; the recorded data were not accessed by a third person, except the researcher; and data sharing will be enacted based on the consent and permission of research participants and the ethical and legal rules of data sharing.

Results

Socio Demographic Characteristics of the Respondents

From 210 interviewed respondents, about 61.9% were males. Regarding educational status, 53.3% of the respondents were degree holders followed by 34.8% who were diploma holders, while only 11.9% were master level professionals (Table 2).

Table 2

Socio Demographic Characteristics of Respondents for Assessment of HMIS Implementation in Hospitals of SWS, 2020

Variables	Number	Percent
Sex of the respondents
Male	130	61.9
Female	80	38.1
Working Experience
Less than 2 years	50	24
2–5 years	30	14
5–10 years	80	38
Greater than 10 years	50	24
Educational status of the respondents
Diploma	73	34.8
Degree	112	53.3
Masters	25	11.9
Position of the respondents in institution
Manager	11	5.2
Health expert	156	74.3
HMIS workers	10	4.8
HR, budgeting and planning	33	15.7

Availability of Resources

A result from resource inventory and respondent interview for availability of inputs revealed that all hospitals have electricity, computer specifically assigned for HMIS, and HMIS minute book for performance monitoring on HMIS information. All hospitals do not allocate a budget for HMIS from their total health budget and only 20% had a written regulatory and planning framework regarding the use of health information systems (Table 3).

Table 3

Availability of Resources Implementation of HMIS in Hospitals of SWS, 2020

Resources (Inputs)	Number of Hospitals	Percent (%)
Written regulatory and planning frameworks regarding the use of HIS
Yes	1	20
No	4	80
HMIS unit (office) specifically assigned for HMIS in the organization
Yes	2	40
No	3	60
Health information technician (HITs)
Yes	5	100
Budget specifically allocated for HMIS from health budget
No	5	100
Hospitals have HMIS training manuals
Yes	3	60
No	2	40
HMIS implementation guidelines
Yes	5	100
HMIS minute book
Yes	4	80
No	1	20
At least one computer specifically assigned for HMIS
Yes	5	100
Working computer with internet service
Yes	4	80
No	1	20
Land line telephone service
Yes	3	60
No	2	40
Have Electricity
Yes	5	100

An interviewee said,

… all activities are performed in HMIS. There is no health program independent of HMIS … we assign budget for the whole programs together not for each program and activities … since the activities of all programs are performed through HMIS, we do not specifically allocate budget for HMIS … On what code of the financial rule can we assign budget for HMIS independently? … I believe, HMIS can share budget with other programs and activities.

Compliance Dimension

Out of all hospitals, 82% of respondents revealed that healthcare workers received HMIS training at least once in service training. Also, 60.7% of respondents have received supportive supervision as per the standard and only 41% have received written feedback of supervision as per the standard. Eighty-one (54%) healthcare workers analyze and interpret HMIS data for decision making and only 26.7% of respondents incorporated HMIS tasks in their employees’ performance appraisal (Table 4).

Table 4

Compliance Dimensions for Implementation Status of HMIS in Hospitals of SWS, 2020

Compliance Indicators	Number	Percent (%)
Healthcare workers received HMIS training at least once in service training
Yes	123	82
No	27	18
Healthcare workers analyze their data in its service unit
Yes	95	63.3
No	55	36.7
Healthcare workers interpreted and summarize HMIS data
Yes	81	54.0
No	69	46.0
Healthcare workers received supportive supervision as per the standard
Yes	91	60.7
No	59	39.3
Healthcare workers received quarterly supportive supervision with written feedback as per the standard
Yes	62	41.3
No	88	58.7
Healthcare workers conduct self-assessment review meetings as per the standard
Yes	95	63.3
No	55	36.7
Healthcare workers participated in hospitals review meeting
Yes	32	21.3
No	118	78.7
Healthcare workers incorporated HMIS tasks in their employee’s performance appraisal
Yes	40	26.7
No	120	80.0

A 28-year old key informant said,

… there is problem of data accuracy at facility level. The main problem is difference of data on the registration book and on the tally sheet as well as the reported one. Most health professionals working at facility level do not tally what they have working … There is no use of Lot quality Assurance Sampling (LQAS) in hospital. Even though higher data accuracy problem is seen at hospital level, administrative level is not free of this problem …

Data Quality

Report timeliness, completeness, and accuracy were seen by reviewing documents of 12 months (July 2018 to June 2019). Three out of five (60%) hospitals fulfilled report timeliness, three out of five (60%) fulfilled data completeness and one out of five (20%) fulfilled data accuracy criteria (Table 5).

Table 5

Report Timeliness, Completeness and Accuracy in Hospitals of SWS, 2020

Compliance Indicator	Number Hospitals	Percent (%)
Health institutions fulfill report timeliness criteria
Yes	3	60
No	2	40
Health institutions fulfill data accuracy criteria
Yes	1	20
No	4	80
Health institutions fulfill data completeness criteria
Yes	3	60
No	2	40

Data completeness and accuracy was assessed using four data elements which were Immediate postpartum family planning (IPPFP), Measles, Early postnatal care (e-PNC), and bacteriologically confirmed TB cases enrolled in a cohort were selected purposively with zonal health departments since these were their concerns. For data completeness records from the registers for these data elements were recounted and cross-matched with the figures in the corresponding monthly reports of the health facilities at the respective administrative health office. Data elements of these registers were not filled out properly according to the national guidelines and the registration column box of reportable data elements at the end of the registration page also not filled. All hospitals the accuracy of data elements were found to be immediate postpartum family (55%), early postnatal care (e-PNC) (61.3%), measles (66.7%), and total number of bacteriologically confirmed TB cases enrolled in cohort (64%), which were found which is less data accuracy.

HMIS focal person key informant says

… there is problem of reporting and data quality at lower level … Most health professionals do not tally what they have done, and they report only from the registration book and the registration column box of reportable data not filled … the data on the tally sheet, registration book and reported are not the same … But the problem of HMIS implementation at administrative level is reporting problem and not shortage of materials resource as that of facility level …

Utilization Dimension

The values of utilization dimension were obtained by key informant interview, reviewing the minute book of the performance monitoring team, receipt of quarterly supportive supervision visits, and observation of data displays on achievement of target, map of catchment area, total population profile of the area and staffing. Out of 50 respondents 28 (56%) use decision based on HMIS data at performance review meeting in the last 3 months. Also, only 32 (64%) case team heads responded that the collected data were analyzed on monthly or quarterly bases. Regarding data display, 62% of the respondents preset minimum requirement of data display (Table 6).

Table 6

Information Utilization Dimensions for Implementation Status of HMIS in Hospitals of SWS, 2020

Utilization Indicator	Number	Percent (%)
Healthcare workers that have performance review meetings monthly and analyses of their data monthly
Yes	32	64
No	18	36
Healthcare workers who use HMIS data for decision-making
Yes	28	56
No	22	44
Healthcare workers who display their achievement of target, population profile, and staffing by table or graph or chart in the service delivery unit
Yes	31	62
No	19	38

A 32-year old key informant said,

… Most of health professional collect data for the purpose of report. They did not analysis and interpret the collected data.…health professional challenges for utilization of information were poor infrastructure, resource shortage, absence/poor internet connection, poor computer access, and workload at unit service delivery. To improve utilization of information continuous supportive supervision and performance review meeting were very important.

Overall Implementation of HMIS

The overall evaluation of HMIS was 58%, which was shown as a judgment parameter of poor implementation. Whereas, availability dimension was 58%, compliance dimension was 55%, and information utilization dimension was 60%. Availability and compliance dimensions are categorized as judgment parameters of poor implementation and information utilization for decision-making was categorized as fair implementation (Table 7).

Table 7

An Overall Implementation Status of HMIS in Hospitals of SWS 2020

Dimension	Weight Given/Expected	Weighted Value/Observed	Percentage Achieved (%)	Judgment
Availability	30	17.5	58	Poor implementation
Compliance	40	22	55	Poor implementation
Utilization	30	18	60	Fair implementation
Overall Implementation	100	57.5	58	Poor implementation

Notes: Judgment criteria: Effectively implemented if weighted value is ≥80; Fairly implemented if weighted value is 79–60; Poorly implemented if weighted value is <59.

Discussion

The study tried to assess the availability of inputs, and how data processing, quality of data, and information generated at hospital levels are utilized. Regarding the availability of standards and inputs of HMIS resources in the hospitals such as the HMIS unit office (40%) and HMIS focal person (100%). The result of this study was high compared to the study conducted in pioneering regions of Ethiopia in 2009 which indicated that 75% of the health facilities have a HMIS focal person.11 This may be due to that the regional Health bureau and zonal health office have given a special focus on recording and reporting of health data using standard tools of HMIS. Four out of five hospitals (80%) reported that they had internet services in their facilities for sending and receiving activities and 60% of hospitals have a land telephone service.

Regarding compliance out of total respondents, 82% received at least one-time training on HMIS. The finding of this study is lower compared to the study conducted in pioneer regions in which 90% of the staff were trained on HMIS.11 This difference may be due to the pioneer regions where pilot testing areas during the initiation of HMIS implementation and being pilot testing area may create an opportunity for training of many staff. The study also revealed that 60.7% of the participants have received supportive supervision from their subsidy health office as per the standard and received supervision; only 41.3% had written feedback. This finding was somewhat less than the finding from pioneer regions, which stated that 75% of respondents have received supportive supervision as per the standard.11 This slight difference could be due to the context in which the supervision was conducted; in the former finding the study areas were pilot testing areas and there could be continuous follow-up for expanding the program to other places.

The finding of the study pointed that 63.3% of the respondents conducted review meetings as per the standard. The finding of this study is greater as compared to the finding of the pioneer regions which was 50%.11 An increment in the current study could be due to the time of the study; in the former study the HMIS was at an initiation stage and it was not completely institutionalized while the current study is conducted after scaling up of HMIS throughout the country, and this could familiarize the institutions with conducting the review meeting.

On the side of report timeliness, the study revealed that 60% reported within the standardized schedule. This finding is greater than the finding of Jimma zone which was 38.12%,19 but less than the finding of pioneer regions, which was 91.7%,11 and SNNP region, which was 77%.15 This difference may be due to the lack of continuous supervision and feedback in the study area.

The findings of this study also revealed that 41.3% of respondents have the quarterly receipt of supportive supervision and discuss it at performance review meetings. The finding of the study showed an increment when compared to the study of the pioneer regions11 but lower as compared to the study conducted in the SNNP region.15 Regarding decision-making based on HMIS data, 56% made decisions based on HMIS data. This finding is greater studies conducted in Jimma zone14 and SNNP region.15 This difference may be due to the demand of government and stakeholders on HMIS data and focus given for informed decision-making at all levels.

Conclusions

Overall, the three dimensions availability of resources, compliance to guideline, and utilization of information were below the standard agreed upon with the stakeholders for the judgment of implementation status. As a result, the implementation status of an HMIS program in the study area was “poor”. This implies that it needs improvement, especially health professionals were not recording the data according to the national HMIS guideline due to not being well trained about the utilization of the HMIS tools. Similarly, the information they collected was not utilized by respective bodies.

Data quality such as report timeliness, completeness, and accuracy did not met the minimum expected level of the national standard. Local utilization of information for decision-making was found to be less and, in most hospitals, data generated through HMIS were not used for local decision-making. Though HMIS is one of the four agendas in HSTP “information revolution” and the national health policy this needs the due attention by the ministry, there was no policy at the country level for a national management information system which guides to generate quality information in a timely manner and ensure the use of information in planning and management of health services.