Quality of the Mental Health Information System in a Specialized Mental Hospital in Bangladesh.

INTRODUCTION: A good quality Mental Health Information System (MHIS) is essential for the efficient planning and delivery of appropriate care services. Accurate, valid, reliable, complete, legible, timely and accessible data is a vital element to achieve this goal. AIM: This study aims to identify the limitations of the MHIS of the National Institute of Mental Health (NIMH), Bangladesh, by reviewing available data and make recommendations for improvement.
MATERIALS AND METHODS: A facility-based cross-sectional study was conducted at NIMH from July to September 2016. Secondary data was collected from various hospital records and reports and was quantitatively analyzed to assess data quality and the MHIS. National quantitative and survey reports were also reviewed.
RESULTS: Routine data collected at the NIMH is of poor quality in terms of accuracy and validity, reliability, and completeness. Access to data and the data's timeliness were also found to be sub-optimal. Various data collection formats were used for monthly and yearly reports at various time points, which has limited the scope of data analysis for further use.
CONCLUSION: The quality of NIMH data is inadequate, thus limiting its usefulness and requiring effective measures for improvement.

INTRODUCTION

According to World Health Organization (WHO) “a mental health information system (MHIS) is a system for collecting, processing, analyzing, disseminating and using information about a mental health service and the mental health needs of the population it serves.” (1). WHO has identified five stages of a MHIS including: i) collection or data gathering; ii) processing or data collation and preparation for analysis; iii) analysis or data checking and review; iv) dissemination or communication of analysis results; and v) use or application of the results to improve services, planning, evaluation and development (1). The quality of a MHIS cannot be ensured without accurate data. According to WHO, essential components of data quality are accuracy and validity, reliability, completeness, legibility, timeliness and accessibility (2).

Adequate information, evidence and research are required for an appropriate formulation of mental health policy, planning and evaluation (3). The collected information has to be sound and reliable to assist in evidence-based decision making (4). After action is taken, a timely and relevant information system should be in place for monitoring and evaluating interventions and services (3). However, in most low- and middle-income countries, a routine MHIS is rudimentary or absent (3). Under such circumstances, understanding the needs of the local population and subsequent rational planning are not possible (3). Globally, a mental health information monitoring system exists in 69% of low-income countries, 79% of lower-middle-income countries and 80% of upper-middle-income countries (5). Only 55% of countries with a mental health information monitoring system publish their data (5). If data is not disseminated, there is little possibility that the information is being used for decision making and improving service development (5).

In Bangladesh, health service providers and the health-care delivery system are supposed to report routine data to administrative levels above on a monthly basis (6). The reported data reaches the Ministry of Health and Family Welfare (MoHFW) through various levels of the administrative hierarchy, and is disseminated through Health Bulletins, Newsletters, Yearbook on Health, Digital Health Guideline and Voice of MIS-Health (6). Unfortunately, a large number of private and nongovernmental organizational health facilities are not yet under the umbrella of the formal Management Information Systems (MIS) of the MoHFW (6). The MoHFW has recently established a web-based data collection system named District Health Information System version 2 (DHIS2) in order to collect routine health data from public health facilities (7). For this database, data can be entered at the source and summary tables, charts and geographic information system maps can be prepared for all levels of the hierarchy (7).

Around the globe, mental ill-health represents a large part of the disease burden of non-communicable diseases, but often it has not gotten enough importance by the health systems (8). In Bangladesh, the prevalence of mental health problems among adults is 16.1% (9). But only 0.44% of the total health budget is allocated for mental health and only 35.59% of those funds are directed to mental hospitals (10).

To our knowledge, no study has yet been conducted to assess the quality of the MHIS of NIMH or any other Bangladeshi mental health institution. Therefore, this study aims to assess the quality of the MHIS of NIMH by reviewing WHO’s five stages of a MHIS- collection, processing, analysis, dissemination and use. We also assess the data quality based on WHO’s essential components- accuracy and validity, reliability, completeness, legibility, timeliness and accessibility. We hope that the findings will fill the knowledge gap of the quality of the MHIS in Bangladesh and our recommendations will support interventions for improvement.

MATERIALS AND METHODS

Study setting, design and sample

A facility-based cross-sectional study with quantitative methods was conducted at NIMH, which is the only mental health institute of Bangladesh with academic functionalities (11). NIMH is in the capital city, and has a 200-bed specialized mental hospital attached to it (12) serving the whole country with a population of 161 million (13). NIMH was established in April 2001 (14) with only outpatient department (OPD); however, in May 2002 an inpatient department was created.

NIMH patients are randomly distributed to four outpatient consultation rooms by purchasing consultation tickets. Each room has an outpatient register book to record individual patient information, which is completed by the doctor while consulting with each patient. There is one record keeper who prepares all cumulative monthly and yearly reports.

For this study, outpatient aggregate data from April 2001-June 2016 monthly and yearly reports were collected. Individual OPD patient data was gathered from the January-June 2016 OPD registers and inpatient aggregate data was collected from May 2002-June 2016 monthly and yearly reports.

All OPD and inpatient extracted data were assessed for data quality using the aforementioned WHO criteria for essential components of data quality. Besides NIMH data, national quantitative and survey reports were also analyzed to confirm the aforementioned WHO essential stages of MHIS. Additional documents reviewed included: i) Local Health Bulletins- 2013, 2014, 2015, 2016 (12, 15-17) and ii) Bangladesh Health System Review 2015 (6).

Data analysis

Stata version 14 was used for data entry and analysis.

Ethical consideration: The Ethical Commission of the Medical Faculty at Heidelberg University, Germany, and the Ethical Review Committee of the James P. Grant School of Public Health at BRAC University, Bangladesh, provided their approval (approval numbers S271/2016 and 80 respectively) for this study. Written permission was also received from the Director of the NIMH for the data collection.

RESULTS

Components of data quality

Accuracy and validity

A total of 71% of the 102 monthly and yearly OPD and inpatient reports reviewed had errors including incomplete or misspelled disease terms, or inconsistent code selection from the 10th revision of the International Classification of Diseases (ICD 10). Examples of erroneous recordings included “personality” instead of “personality disorder”, “retardation” instead of “mental retardation” and “stupar” instead of stupor. Depressive disorders were sometimes grouped with anxiety disorders under ICD 10 code F40-44 and at other times under ICD 10 code F41.

A total of 0.08% of 19429 individual OPD patient data records between January and June 2016 had invalid information. For example, “came for report” was written in the diagnosis column.

Reliability

Only 50 of 78 monthly inpatient reports were dated and 20% of the dates were unreliable. For example, January 5, 2014 was the date for all four monthly reports from January 2014 to April 2014. The majority (71%) of the 81 monthly and yearly inpatient reports reviewed contained unreliable information. For example, ICD 10 code F-45 was written for both somatoform disorder and obsessive-compulsive disorder. Whereas in other monthly inpatient reports, obsessive-compulsive disorder was recorded as ICD 10 code F-42. ICD 10 code R40.1 was recorded for stupor, delusional and personality disorder in various monthly inpatient reports. For dementia, two distinct ICD 10 codes, F00-03 and G30.0+, were recorded in separate monthly inpatient reports. Personality disorder was recorded with three ICD 10 codes, F52, F07 and F60-69 in separate monthly inpatient reports.

Completeness

Data for 19429 individual patients were recorded in OPD registers from January-June 2016. OPD registers have nine columns including serial and registration numbers, name, sex, age group, weight (kg), diagnosis, treatment/advice/services provided and admission/referral. Generally, only the registration number, name, sex and age were documented (other columns left empty). The number of patients per data item and percentage of data completeness is presented in Table 1.

When patients returned for a re-visit, the register instructions declared that their previous registration number should be noted in the ‘name’ column. However, this was not done for a single patient in the register samples reviewed. Bodyweight was not recorded for a single patient and on observation no weight machine was seen in any of the OPD consultation rooms.

All 21 monthly and yearly OPD reports reviewed were complete. However, in 55% of the total 81 monthly and yearly inpatient reports, ICD 10 codes were absent or incomplete.

Legibility

In one of the four OPD registers, hand-written data was very difficult to read. Some entries extended into a neighboring column resulting in poor readability. All 102 monthly and yearly reports of both OPD and inpatient departments were computer-typed and legible.

Timeliness

The OPD register is completed by doctors while consulting with their patients. The majority (59%) of 102 monthly and yearly OPD and inpatient reports were marked with the date of preparation. The range of time required to prepare the monthly and yearly reports was 1-5 months. For reports with the date of preparation, 42% were prepared within one month, 16% within two months and 22% more than two months after end of a particular month.

Accessibility

All relevant personnel have access to the data and reports at this hospital. But, since the data recording system is still paper-based, retrieval of information (e.g., a previous patient/visit) is inconvenient, time consuming, and sometimes impossible.

Stages of MHIS

WHO’s five stages of a functioning MHIS- collecting, processing, analyzing, disseminating and using data exist at NIMH. However, analysis and utilization of individual patient data is limited. For example, monthly and yearly OPD reports contain information only for age group, sex and total number of care recipients. The inpatient reports contain data on disease code, bed occupancy rate and mortality in addition to information presented in the OPD monthly and yearly reports. The remaining individual patient data is unused.

One particular concern for data analysis is the changes in the reporting format for monthly and yearly reports of both the OPD and inpatient departments. From the start of the OPD in April 2001 and the inpatient department from May 2002-December 2007, only the monthly total number of patients was documented in the reports. From January 2008, the number of male and female patients was noted with an overall total. In January 2012, a new column was added to that format, and included the total number of children. However, the children’s sex in specific numbers was not recorded.

For the inpatient department, an additional reporting format was introduced in January 2010 whereby the number of patients in various disease categories and age groups were presented in both monthly and yearly reports. The format was also changed several times for disease codes and age groups. These changes are presented in Tables 2 and 3.

The MoHFW publishes a health bulletin each year for information dissemination. Important findings or concerns and subsequent recommendations are summarized at the end of these reports which is proof of data analysis and utilization by the institute. Policy makers also address some issues mentioned in the reports from time to time which indicates use of the information generated. For example, to fulfill the demand of a rising number of patients, the NIMH’s bed count was increased from 150 to 200 along with an increase in associated resources.

DISCUSSION

This study reveals that there is a functional MHIS at the NIMH since it was established in 2001. However, the quality of routinely collected data is sub-optimal in accuracy and validity, reliability, and completeness. Timeliness and access to data were found to be concerns as well. WHO’s recommended five stages of a MHIS mentioned earlier are present at NIMH.

Although no evaluation of the MHIS in Bangladesh has been conducted, an assessment of the status of the overall Health Information System (HIS) in Bangladesh was done in 2009. The “Health Metrics Network Assessment Tool” was used and the adequacy of six HIS elements were found to be as follows: resources 41%; indicators 73%; data sources 52%; data management 0%; information products 76%; and dissemination and use 50% (18). A WHO report stated that the data collected by the Bangladeshi MIS is of poor quality from the perspective of timeliness, correctness and accuracy (6).

The reasons behind poor quality of data might be the doctor’s and record keepers’ lack of skill or understanding about the importance of data. Health workers often do not have standardized instructions on data collection methods, nor proper training (1). Information collected should be made interesting to the health workers by showing how it links to the outcomes of their activities (1). The doctors’ limited time to fill in the patient registers due to extreme workloads might result in the incompleteness of OPD registers. Similar findings have been reported in a study conducted in the Philippines (19). In many countries, HIS staff time and allocated resources are inadequate (1).

NIMH has a paper-based data recording system, whereby patients’ previous treatment records are in piles in the medical record room. Finding an individual patient record is tedious, time consuming, and sometimes impossible. Documents are lost or badly damaged over the years. If data is not accessible, then even accurate documentation has no value (2). A computer-based information system is recommended as the best collection method (18) because it is user-friendly and sustainable. Most mental health problems are chronic (20), therefore, the MHIS should be oriented to a continuous care approach. A computer based data recording system would not only provide an easier access but also would create the possibility to incorporate clinical MIS in it, which would serve the purpose of evaluating the relationship of a clinical process with the outcome of care and to develop best practices (21).

From all data collected at NIMH, only a fraction is routinely analyzed and reported to the MoHFW. Individual patient data is not extensively analyzed by the institute. Missing OPD register data is identical to the unanalyzed and unutilized data. In less-resourced countries, much of the collected health data is not used to provide reasonable feedback to planners and managers or to influence program implementation (22, 23). Since the NIMH uses multiple monthly and yearly reports created at sporadic time intervals, it is not possible to analyze the available data for continuous time periods. Although periodic evaluation and modification is a routine process for improving a MHIS (18), one of the MHIS principles is establishing a minimum data set (1). Therefore, before implementation of any data collection-, compilation- and reporting format proper research and consultation with all stakeholders are necessary to make the data fruitful. If the information is not available in a format that meets users’ needs, then the data is of little value (4).

Information reported to the MoHFW is published each year in an online health bulletin (24), with a summary on vital findings, concerns and relevant recommendations (12). This is definitely an advancement since in the past no report was published with comments on the data (11). However, according to WHO, the data collected through the Bangladeshi MIS is rarely used for health planning. Managers and policy planners in Bangladesh still generally rely on survey data and not routine MIS data (6).

CONCLUSIONS

Our study findings provide an understanding about the current quality of the MHIS of NIMH, which serves as a baseline for comparison for future research and interventions. The findings indicate the current poor quality of the NIMH data due to limitations in accuracy, validity, reliability, completeness, timeliness and access. Aspects of data analysis also need to be improved.

NIMH should strive to improve data quality to generate sufficient and accurate knowledge that can eventually be used for informed decision making. A standard reporting format should be created and separate personnel should be assigned for individual OPD-patient data entry, in particular. NIMH should also perform more extensive analysis of individual patient data and use those findings for both internal and external development. All associated NIMH personnel should receive periodic training on their MHIS-related tasks. Moreover, NIMH should upgrade the information-recording system from paper-based to computer-based. The quality of MHIS at NIMH should be evaluated every five years and relevant recommendations for improvement should be implemented accordingly.

Above all, to enable an optimal usefulness of the data, findings should be used by the decision makers at every stage of the management process. Information generated should be used by all stakeholders in the mental health care system, and not only by policymakers and managers.

Table 1.

Completeness of each data column for individual entries in OPD patient registers from January to June 2016 in Bangladesh’s NIMH

Column Headings	Column completed in patient records (total number of patients =19429)	Column completed in patient records (%)
Serial number	19429	100
Registration number	19429	100
Name	16284	83.8
Sex	13346	68.7
Age group	19311	99.4
Weight in kg	0	0
Diagnosis	1280	6.6
Admission/referral	62	0.3
Treatment/advice/services provided	95	0.5

Table 2.

Various disease codes used for NIMH inpatient monthly reporting during specific time periods from January 2010–June 2016. 1) bold-italic and shaded boxes represent changes; 2) diagnostic terms and ICD 10 codes written in the monthly in-patient reports recorded here as written (no corrections)

Jan 2010 -Feb 2011	Mar 2011 -Aug 2012	Sep 2012 -Mar 2013	Apr 2013 -Dec 2013	Jan 2014–Feb 2014	Mar 2014 -Jun 2014	Jul 2014	Aug 2014 -Nov 2014	Dec 2014 -May 2015	Jun 2015 -May 2016	Jun 2016
Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)	Schizophrenia (F20)
Manic Depressive Psychosis (F30-39)	Manic Depressive Psychosis (F30-39)	Manic Depressive Psychosis (F30-39)	Manic Bipolar/ Psychosis (F30-39)	Schizophrenia (F20-29)	Manic Bipolar/ Psychosis (F30-39)	Manic Depressive Psychosis (F30-39)	Manic Bipolar/ Psychosis (F30-39)	Manic Bipolar/ Psychosis (F30-39)	Manic Bipolar/ Psychosis (F30-39)	Manic Depressive Psychosis (F30-39)
Anxiety disorder (F40-44)	Anxiety disorder (F40-44)	Anxiety disorder (F40-44)	Anxiety/depressive disorders (F40-44)	Manic Bipolar/ Psychosis (F31)	Anxiety/ depressive disorders (F41)	Anxiety disorder (F41)	Anxiety/ depressive disorders (F41)	Anxiety/ depressive disorders (F41)	Anxiety/ depressive disorders (F41)	Anxiety disorders (F41)
Somatoform disease (F45)	Somatoform disorder (F45)	Obsessive compulsive disorder (F45)	Obsessive compulsive disorder (F45)	Manic Bipolar/ Psychosis (F30-39)	Obsessive compulsive disorder (F42)	Obsessive compulsive disorder (F42)	Obsessive compulsive disorder (F42)	Obsessive compulsive disorder (F42)	Obsessive compulsive disorder (F42)	Obsessive compulsive disorder (F42)
Sexual dysfunction (F52)	Sexual dysfunction (F52)	Personality (F52)	Personality (F52)	Anxiety/ depressive disorders (F41)	Personality (F60-69)	Personality (F60-69)	Personality (F60-69)	Personality (F60-69)	Personality (F60-69)	Personality (R40.1)
Dementia (F00-03)	Dementia (F00-03)	Dementia (F00-03)	Dementia (F00-03)	Obsessive compulsive disorder (F42)	Dementia (G30.0+)	Dementia (G30.0+)	Dementia (G30.0+)	Dementia (G30.0+)	Dementia (G30.0+)	Dementia (G30.0+)
Epilepsy	Epilepsy	Epilepsy	Epilepsy (G40)	Personality (F07)	Epilepsy (G40)	Epilepsy (G40)	Epilepsy (G40)	Epilepsy (G40)	Epilepsy (G40)	Epilepsy (G40)
Mental retardation (F70-79)	Mental retardation (F70-79)	Mental retardation (F70-79)	Mental retardation (F70)	Dementia (F00-03)	Mental retardation (F79)	Mental retardation (F79)	Mental retardation (F79)	Mental retardation (F79)	Mental retardation (F79)	Mental retardation (F79)
Substance abuse (F10-19)	Substance abuse (F10-19)	Substance abuse (F10-19)	Substance abuse (F10)	Epilepsy (G40)	Substance abuse (F19)	Substance abuse (F19)	Substance abuse (F19)	Substance abuse (F19)	Substance abuse (F19)	Substance abuse (F19)
	Other	Stupar	Stupar (R40.1)	Retardation (F79)	Stupar (R40.1)	Stupar (R40.1)	Stupor (R40.1)	Stupar Delusional (R40.1)	Delusional (R40.1)	Delusional
		Other	Other	Substance abuse (F19)
				Stupar (R40.1)

Table 3.

Various age group formats used for NIMH inpatient reporting during specific time periods from January 2010-June 2016. * months

Period	Age groups for inpatient reporting in years
Jan 2010- Dec 2013	0-11*	1-4	5-9	10-14	15-19	20-29	30-39	40-49	50-59	60-69	70
Jan 2014- Dec 2015	0-4	5-14	15-24	25-49	>50
Jan- Jun 2016	0-4	5-14	15-30	31-50	>50