Evaluation of surveillance system for pneumonia in children below five years, Tema Metropolis, Ghana, 2012 - 2016.

BACKGROUND: We evaluated the pneumonia surveillance system in Tema Metropolis to determine whether it is meeting its objectives and to assess its attributes.
DESIGN: Descriptive primary and secondary data analysis. DATA SOURCE: We interviewed health staff on the system's operation and resources. We also extracted 2012-2016 surveillance dataset for under-five pneumonia cases and deaths from the District Health Information Management System for review. PARTICIPANTS: Health staff. INTERVENTION: The Centers for Disease Control (CDC) updated guidelines for evaluating surveillance systems was used to assess system attributes. MAIN OUTCOME MEASURE: state of the pneumonia surveillance system in Tema.
RESULTS: A suspected case was defined as fast breathing in any child < 5 years old. The case definition was easy to apply, even at the community level. From 2012 to 2016, a total of 3,337 cases and 54 deaths (case fatality rate 1.6%) was recorded from 13 (23.6%) of 55 health facilities. Two epidemics were missed by the district because data were not being analysed. There were no laboratory data on antimicrobial resistance. Although reporting timeliness increased from 28.1% in 2012 to 83% in 2016, data inconsistencies existed between reporting levels.
CONCLUSION: The surveillance system for under-five pneumonia in Tema Metropolis is simple, stable, flexible, timely, but of low sensitivity and acceptability, and only partly meeting its objectives. Major shortcomings are lack of laboratory data, non-use of data and low representativeness. FUNDING: The study was supported by a grant to author DB by the President's Malaria Initiative (PMI) -CDC CoAg 6NU2GGH001876.

Introduction

Childhood pneumonia is an important cause of morbidity and mortality.1 Current estimates put its incidence at 0.05 episodes per child-year in developed countries and 0.29 episodes per child-year in developing countries.2 The World Health Organization reports that in 2015, pneumonia was responsible for the deaths of over 922,000 children, contributing 16% of under-five deaths worldwide.3 More than 90% of these deaths occurred in Sub-Sahara Africa and Southeast Asia.3 In Ghana, the estimated incidence of childhood pneumonia is 0.22 episodes per child-year.4 Pneumonia remains the third leading cause of death in children under five years – after malaria and prematurity.5

Despite a reported 18% reduction in under-five pneumonia mortality over the last fifteen years,6 a retrospective review of mortality data in a children's hospital in Accra showed an increasing trend in proportionate mortality.7 A surveillance system exists for pneumonia in children below five years in Ghana which seeks to detect cases and epidemics in a timely manner, monitor antimicrobial resistance, and reduce the proportion of severe pneumonia.8 Information generated by this system is used to make policy decisions and monitor interventions.8 Therefore it is necessary that data entered accurately reflects the disease occurrence in the population. Periodic evaluation of the system will not only assess the quality of information generated, but also identify the weak areas along the surveillance processes for improvement.9

Since its implementation in 1998, the under-five pneumonia surveillance system in Tema Metropolis has not been evaluated. The system was therefore evaluated for the period 2012–2016 to determine whether it was meeting its set objectives, and to assess its usefulness and attributes.

Methods

Study setting

The evaluation was conducted in January 2017 in Tema Metropolis – one of sixteen districts in Greater Accra Region of Ghana. Located on longitude 5.7348N, latitude 0.0302E, Tema has three sub-districts (Tema Central, Tema West and Tema East) and a population of 351,616. Children under five comprise one-fifth (70,323) of the population.10

The healthcare structure comprises health facilities at the community level (CHPS compound, community clinics), sub-metropolitan level (polyclinic, health centre), and metropolitan level. The Tema Metropolitan hospital is the largest health facility and serves as both a primary and referral facility for residents within the district and two neighbouring districts. There are five government-owned health facilities and over fifty private health facilities.

Case definitions, operation and resources of surveillance system for under-five pneumonia

The under-five pneumonia surveillance system is facility-based and targets children less than five years old. A suspected case is defined as a child presenting with cough or difficult breathing, and 50 or more breaths per minute for infant age 2 months up to 1 year or 40 or more breaths per minute for young child 1 year up to 5 years. Severe pneumonia is defined as a child presenting with cough or difficult breathing and any general danger sign, or chest indrawing or stridor in a calm child. A confirmed case is defined as radiographic or laboratory confirmation of pneumonia.

Cases are reported through DHIMS2 (District Health Information Management System), a web-based centralized system, using standardized monthly Out-patient department (OPD), morbidity and Integrated Disease Surveillance and Response (IDSR) reporting forms. There are five reporting levels with clearly defined tasks as shown in Figure 1. At the health facilities, clients from the community report to the consulting rooms or emergency rooms where their demographic details (name, age, sex, place of residence) and diagnoses are recorded in registers. The registers have columns for provisional diagnosis, referral for confirmation and final diagnosis. Suspected cases of pneumonia are referred for radiographic or laboratory confirmation. After pneumonia is confirmed, it is classified as either severe or non-severe pneumonia and recorded in the final diagnosis column. At the end of the month, validated aggregates of cases and deaths are entered into the electronic monthly OPD morbidity form in the DHIMS2 as well as paper-based IDSR forms.

Figure 1

Information flow chart for under-five pneumonia surveillance system in Ghana

The IDSR forms are submitted to the metropolitan health directorate. Once data are entered into the DHIMS, they are accessible in real time to anyone with authorized access. Data can be edited until ninety days after submission after which they become permanent record. Facilities are expected to have submitted their data by the fifth day of ensuing month. At the district level, the health information officer and disease control officer aggregate surveillance data from reporting facilities and review them for timeliness and completeness.

They perform data analysis at the district level and are responsible for outbreak detection and response. The regional and national levels usually play supervisory and supportive roles. Data analysis is expected at all levels of data flow. DHIMS2 provides tools for tabular and graphic presentation of data. The pivot tables and data visualizer tools can be used to generate trends by sex and age at the facility level through to the national level.

At the district level, performance reviews are expected to be held annually during which data are shared, although facilities are encouraged to conduct regular reviews throughout the year. The system is funded by the government. The materials and equipment required for operation are registers, tally sheets, materials for microbial testing and susceptibility, X-rays, microscope, computers and internet access.

Study design

The surveillance system evaluation was carried out from December 2016 to January, 2017 using the Centers for Disease Control (CDC) guidelines for evaluating public health surveillance systems.

Data collection

The Greater Accra regional and Tema metropolitan health directorates, and directors at the health facilities were engaged prior to start of evaluation to ensure their participation and readiness to use findings. Key health staff involved in the operation of the surveillance system in the various health facilities were identified. Data were gathered through interviews, observation and records review.

Disease control officers, health information officers, community health nurses, physicians, laboratory scientists and record personnel were purposively sampled from health facilities and interviewed. The district disease control officer and district health information officer were also interviewed.

Qualitative data were collected through semi-structured interviews and observation. An interview guide and an observation checklist were designed along pre-determined themes adapted from Centre for Disease Control's (CDC) updated guidelines for evaluating surveillance systems.9 Data were captured on the following; respondents' knowledge on case definitions used and reporting flow, data collection methods, data analysis and dissemination, uses of surveillance data, system integration, and availability of resources required for operation. The activities of surveillance personnel were observed and logged daily. Facility laboratories were also inspected for availability of functional incubators, culture media, and antimicrobial susceptibility testing disks.

Quantitative data were collected through records review. Electronic surveillance dataset on under-five pneumonia from 2012 to 2016, with its performance indicators, was exported to MS Excel spreadsheets from the DHIMS2. Laboratory registers and medical cause of death certificates were also reviewed for microbiologic data and pneumonia-related deaths respectively.

Electronic monthly data were checked for correctness by comparing with data from facilities' consulting room registers and tally sheets.

Data analysis

Performance of system in meeting its objectives

Participants' responses to questions on surveillance activities were presented as frequencies and proportions. The age and sex frequencies and proportions of pneumonia cases, and case fatality rate were also calculated.

Simplicity, flexibility and stability

Simplicity, flexibility and stability were assessed by direct content analysis 11 of the qualitative data. Simplicity was assessed using the following metrics; time spent on surveillance activities and minimum skill and data required for operation. The system was said to be simple if it required little time, fewer minimum data, and readily available resources and skills for operation.

Respondents were asked about changes in the system's operation since implementation, and its response to the changes. The system was flexible if it responded to changes in surveillance procedure with little additional resources. Stability was assessed by the availability of resources needed for operation. The system was stable if all needed resources and skills were readily available. Sensitivity, timeliness, representativeness and acceptability Were also assessed.

Sensitivity was assessed at two levels; case detection and outbreak detection levels. Based on a model for estimating childhood pneumonia incidence,4 about 15,471 pneumonia cases per year would be expected for a population of 70,323 children below five years old. The number of observed cases was compared with the expected to assess underestimation.12 Sensitivity at case detection level was estimated by dividing the number of observed cases by 15,471, expressed as a percentage. At outbreak detection level, health staff were asked about outbreaks during the period under evaluation. Additionally, cumulative sum (C2) threshold values were calculated and displayed against monthly case frequencies to detect epidemics.

Timeliness was assessed by calculating two parameters; time elapsed between time provisional diagnosis was made and confirmation, and the timeliness of reporting.

A report was said to be timely if it reached the district or was entered into the DHIMS2 by the 5th day of ensuing month. The percentage of timely reports was calculated for each year.

The system was said to be representative in place and person if it captured pneumonia cases across all the sub-districts, and across different age-groups below five years. Acceptability was assessed by the facility participation rate, which was defined as the proportion of health facilities reporting on under-five pneumonia to the district level.

Data quality

The quality of data was evaluated by completeness of reporting, including zero reporting, and data correctness. The proportion of entries in DHIMS2 out of the expected (12 entries per facility per year) was calculated to determine the reporting rate. Data correctness was assessed using data entry error rate in three major health facilities in the district. Data entered into DHIMS2 from January 2013 through December 2016 were compared with facility data as the gold standard. Allowing 10% error rate, the proportion of data points with entered data being 10% more or less than the gold standard was calculated.

Ethical approval

The Director of the Diseases Surveillance Department of the Ghana Health Service granted approval for the access and use of the data for this review. Permission was also officially sought from the Regional Director of Health Services for the use of the data. Oral consent was obtained from health workers before interviews were conducted. To ensure confidentiality, coded patient identification numbers were used in place of names and aggregate analyses done on data. Data held on computers were encrypted with a password which was made available only on a need to know basis.

Results

Twenty health staff were interviewed. Four (20%) were diseases control officers, four (20%), health information officers, three (15%), physicians, three (15%), biomedical scientists, three (15%), record personnel, and three (15%) community health nurses. About 65% (13/20) of respondents were males. Except two record personnel, all respondents had been at current post for at least one year.

Surveillance system performance

All respondents knew the case definitions for pneumonia. Ninety percent (18/20) knew the flow of reporting. A sample of 45 cases recorded in the consulting room registers had demographic characteristics indicated. The final diagnosis was recorded in only 53% (24/45) of cases sampled.

No case was classified as severe or non-severe. In all facilities visited, cases were tallied daily or weekly, and aggregated by record personnel with support from the health information officer.

At the end of the month, data validation teams stationed in all facilities and comprising health information officer, disease control officer, and heads of departments, ensured that data were accurate before entry into the DHIMS. At the district health directorate, piles of hard copy IDSR forms from reporting health facilities were kept in a cabinet. None of the respondent had ever analysed under-five pneumonia surveillance data. Forty-five percent (9/20) of respondents responded that they could confidently analyse data. Of these, none had analysed data because they perceived that under-five pneumonia was not of primary concern to the district due to its associated low mortality.

From 2012 through 2016, the system recorded 3,337 cases and 54 deaths with an overall case fatality rate of 1.6%. Males were 51% (1711/3337). Majority of cases were aged 12 – 59 months (1945/3337, 58 %%). Tema Central (2645/3337, 79%) recorded the highest proportion of cases and Tema East (248/3337, 7%), the lowest proportion of cases. The number of cases was highest in 2012 (928/3337, 28%) and lowest in 2016 (304/3337, 9%) (Table 1). There were no laboratory records on antimicrobial resistance. One out of three reporting health facilities visited had a designated microbiology lab with three assigned biomedical scientists, functional microscopes, incubator, and materials for microbial and drug susceptibility testing.

Table 1

Reported under-five pneumonia cases and deaths, Tema Metropolis

Variable	Number of case-patients (%)
Year	2012	2013	2014	2015	2016	Total
Number	928	726	545	834	304	3337
Sex
Male	479 (52)	379 (52)	247 (45)	441 (53)	165 (54)	1711 (51)
Female	449 (48)	347 (48)	298 (55)	393 (47)	139 (46)	1626 (49)
Age group
<28days	0	0	0	50 (6)	2(1)	52 (2)
1–11 months	456 (49)	385 (53)	99 (18)	275 (33)	125 (41)	1340 (40)
12–59 months	472 (51)	341 (47)	446 (82)	509 (61)	177 (58)	1945 (58)
Sub-district
Tema Central	654 (70)	612 (84)	463 (85)	699 (84)	217 (71)	2645 (79)
'Tema West	153 (17)	52 (7)	54 (10)	118 (14)	67 (22)	444 (13)
Tema East	121 (13)	62 (9)	28 (5)	17(2)	20(7)	248 (8)
Deaths (CFR)	14 (1.5%)	9 (1.2%)	8 (1.5%)	19(2.3%)	4 (1.3%)	54 (1.6%)

Usefulness of the surveillance system

All respondents agreed that data from the system should be used for decision making. About 55% (11/20) responded that data should be used to monitor trends, and 10% (2/20), to evaluate the impact of immunization. None of the respondents, nor their facilities had performed any activity using data from the under-five surveillance system as basis. At the district level, there was no documentation that surveillance data had been used for any action.

Simplicity

Almost all respondents (19/20) agreed that the system was easy and quick to operate. All respondents could use the case definitions to detect cases, even at the community level. No additional information, other than demographic characteristics were needed, and no special forms i.e. case investigation forms needed to be filled.

The time spent on surveillance usually resulted from counting cases as data entry was usually a straightforward procedure. The time spent on surveillance usually resulted from counting cases as data entry was straightforward.

Flexibility

The under-five pneumonia surveillance system is easily integrated in the Integrated Disease Surveillance and Response without any reported challenges. All respondents noted that there had been no changes in case definition nor reporting flow since its implementation. The addition of new reporting facilities over the years bore no extra cost to the system.

All the disease control officers (4) and health information officers (4) had received training on the DHIMS2 and were familiar with reporting. The remaining respondents (12/20, 60%) had not been formally trained on DHIMS2. The system was regularly operational with minimal interruption during periods of system upgrade. Respondents reported data as part of their routine work, and did not receive extra monetary benefit. Additionally, the items required for the daily operations of the surveillance system were existing resources used by health facilities, thus bore no additional cost to the system.

All respondents noted that, although laboratories might not have the full capacity to conduct laboratory confirmation and antimicrobial susceptibility testing, X-rays were available in many facilities for radiological confirmation. Internet access and connectivity were restrictive factors but the ‘offline mode’ feature of the DHIMS permitted data entry during periods of internet unavailability. When internet became available later, entered data would be automatically uploaded to the central database.

Sensitivity

The system detected 6% (928/15,471), 5% (726/15,471), 4% (545/15,741), 5% (834/15,471) and 2% (304/15,471) of the expected estimated 15,741 cases in 2012, 2013, 2014, 2015 and 2016 respectively. The system also failed to detect epidemics.

A comparison of reported monthly cases with computed cumulative sum (C2) threshold values showed that observed cases exceeded expected threshold in November 2014 and January 2015 (Figure 2). A follow up interview revealed that the district health staff had not picked up these excesses.

Figure 2

Reported under-five pneumonia cases and cumulative sum thresholds, Tema, 2012–2016

Timeliness

In all facilities visited, the average estimated time that elapsed between the time a case was suspected, and radiographic confirmation averaged one hour. The proportion of entries made into the DHIMS2 before the stipulated deadline was 28% (19/67) in 2012, 58% (25/43) in 2013, 94% (34/36) in 2014, 91% (43/47) in 2015, and 83% (38/46) in 2016.

Representativeness and Acceptability

Under-five pneumonia cases were recorded in all three sub-districts and all age-groups below five years, including neonates. Only 13 (23.6%) out of the 55 health facilities in the district had ever reported on under-five pneumonia cases from 2012 through 2016. Three out of five (60%) government-owned health facilities submitted reports throughout the period under evaluation. Only 20% (10/50) of private health facilities had ever reported on under-five pneumonia to the next level (Table 2).

Table 2

Health facility participation rate in under-five pneumonia surveillance system, Tema, 2012–2016

Period	Participating facility			Monthly reports submitted (n=660)
	Government (n-5)	Private (n-50)	All facilities (n-55)
2012	3(60%)	10(20%)	13(24%)	67(10%)
2013	3(60%)	5(10%)	8(15%)	43(7%)
2014	3(60%)	4(8%)	7(13%)	36(5%)
2015	3(60%)	5(10%)	8(15%)	47(7%)
2016	3(60%)	9(16%)	12(22%)	46(7%)

Among participating facilities, reporting was infrequent. For the five-year period, the proportions of expected monthly reports submitted by all health facilities to the district level was highest in 2012 (67/660, 10%) and lowest in 2014 (36/660, 5%) (Table 2). There were no ‘zero’ reports. Out of the 124 data points from three selected facilities, 81 (65%) had incorrect data entries in DHIMS2 for the number of under-five pneumonia cases.

Discussion

We evaluated the surveillance system for pneumonia in children below five years in the Tema Metropolis. The system is partially meeting its objectives. Although the system records cases and deaths, it does not monitor antimicrobial resistance nor the proportion of severe cases. The finding of a weak laboratory component in a system that relies on laboratory confirmation is comparable to results of similar evaluations in other low resource countries,13–15 and also reflective of the poor collaboration between laboratory surveillance and disease surveillance in these countries.

Pneumonia has multiple aetiologies that include dangerous pathogens with potential for natural as well as manmade outbreaks. Owing to lack of laboratory confirmation, the system will not be able to detect such pathogens nor their resistance pattern to commonly used antimicrobials should they occur. Furthermore, opportunities to monitor aetiology-specific pneumonia in order to modify case management guidelines, and interventions such as routine immunization will be missed. The district management team and the health facility could explore options of should work with donors and government to improve their laboratory infrastructure and human capacity. Again, training of health and surveillance staff on microbiological testing could also help improve the laboratory component.

Respondents expressed that data from the system should be analysed although they seemed to consider only data collection as their principal responsibility. Further interaction uncovered their belief that under-five pneumonia was not a priority disease in their facilities due to its low associated fatality. The decision to continue or discontinue a surveillance system to a large extent depends on the perceived usefulness of the system in addressing the health issue at hand. The mere collection of data, without transforming into practical information for action undermines the entire purpose of disease surveillance systems. Also, if the capacity for data transformation at the lower levels is deficient, local health problems will be missed, despite available data, as demonstrated by the two outbreaks that were not detected in the Metropolis.

Timeliness and sensitivity are conceivably the most fundamental attributes for a system that seeks to detect cases and epidemics early. There is no standard measure of timeliness as it takes into account delay at every point in the surveillance process, from recognition of symptoms, through reporting to health facility to case management and reporting. For this evaluation, timeliness was assessed by the mean reporting timeliness and time taken for radiographic confirmation. Based on our results, the system can be said to be timely. This is in accordance with the assertion by several studies that electronic reporting improves the timeliness of surveillance systems.16–18 The rapid increase in facility reporting timeliness over the five-year period may be attributed to surveillance personnel becoming more accustomed to the system with time. Consistent with the model developed by Rudan et al, 2 about 15, 471 under-five pneumonia cases per year should be expected in Tema Metropolis. The recorded annual cases (304 - 926) falls way too short of the expected. The reasons for the low case recording are many. Firstly, the system is facility-based therefore influenced by appropriate care-seeking behaviour which is reported to be low. 6

As such, mild cases and those accessing healthcare from places other than the health facilities in the district will be missed. Additionally, with 76% of health facilities, mostly privately-owned, not reporting cases, the system is prone to underreporting. The low coverage of private health facilities is similar to what pertains in other evaluated systems. 13

The integration with other systems is advantageous to the under-five pneumonia surveillance. Pneumonia, unlike malaria and Human Immunodeficiency Virus infection, receives little national attention and budgetary allocation. However, lessons learned from past workshops organized by the Malaria Control Programme on DHIMS are applicable to the pneumonia surveillance system since the same individuals are responsible for reporting all the diseases that have been integrated. The evaluation further demonstrates that the system is simple and stable although the data generated are insufficient to reliably represent the public health importance of pneumonia in children less than five years in the Tema Metropolis.

Limitations

The study participants are key personnel responsible for the operation of the surveillance system and may have reported what is ideal other than their actual practice. This was minimized by combining interviews with daily observations, record review, and interviews with surveillance staff at the different levels to increase credibility. Also, it would have been helpful to assess timeliness at every point in the surveillance processes. However, we were only able to assess timeliness by two metrics due to the unavailability of individual level data.

Conclusion

The surveillance system for pneumonia in children below five years in the Tema Metropolis partly met its objectives for the period 2012–2016. The system was found to be timely, stable, simple and flexible but has a weak laboratory component.