Implementing the routine immunisation data module and dashboard of DHIS2 in Nigeria, 2014-2019.

In 2010, Nigeria adopted the use of web-based software District Health Information System, V.2 (DHIS2) as the platform for the National Health Management Information System. The platform supports real-time data reporting and promotes government ownership and accountability. To strengthen its routine immunisation (RI) component, the US Centers for Disease Control and Prevention (CDC) through its implementing partner, the African Field Epidemiology Network-National Stop Transmission of Polio, in collaboration with the Government of Nigeria, developed the RI module and dashboard and piloted it in Kano state in 2014. The module was scaled up nationally over the next 4 years with funding from the Bill & Melinda Gates Foundation and CDC. One implementation officer was deployed per state for 2 years to support operations. Over 60 000 RI healthcare workers were trained on data collection, entry and interpretation and each local immunisation officer in the 774 local government areas (LGAs) received a laptop and stock of RI paper data tools. Templates for national-level and state-level RI bulletins and LGA quarterly performance tools were developed to promote real-time data use for feedback and decision making, and enhance the performance of RI services. By December 2017, the DHIS2 RI module had been rolled out in all 36 states and the Federal Capital Territory, and all states now report their RI data through the RI Module. All states identified at least one government DHIS2 focal person for oversight of the system's reporting and management operations. Government officials routinely collect RI data and use them to improve RI vaccination coverage. This article describes the implementation process-including planning and implementation activities, achievements, lessons learnt, challenges and innovative solutions-and reports the achievements in improving timeliness and completeness rates. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Until 2014, the Government of Nigeria used an Excel-based tool to collect information on vaccine administration and management from the health facilities of the national Expanded Programme on Immunsation.

Users of this tool reported data quality issues and no tracking mechanism for vaccine accountability or cold chain functionality.

The National Primary Health Care Development Agency (NPHCDA) and Federal Ministry of Health requested that partners develop a routine immunization (RI) module for the District Health Information System, version V.2 (DHIS2) platform.

One implementation officer (IO) was deployed per state for 2 years to support operations. And over 60, 000 RI health workers were trained on data collection, entry and interpretation, and each Local Government Area Immunisation Officer (LIO) in the 774 Local Government Areas (LGAs) received a laptop and stock of RI paper data tools.

While numerous challenges still exist, including lack of reliable government funding for the initiative, by December 2017, the DHIS2 RI module had been rolled out in all 36 states and the Federal Capital Territory, and all states now report their RI data through the RI Module.

Background

The National Health Management Information System (NHMIS) was activated in Nigeria in 1999. In 2010, NHMIS was configured on the national platform of the District Health Information System, V.2 (DHIS2). This platform is used to report primary healthcare data, including those for the national Expanded Programme on Immunisation (EPI).1–3 DHIS2 is an open source software platform for reporting, analysis and dissemination of data for all health programmes. It captures data at the local government area (LGA) level, and displays information for all administrative levels on its dashboard, which can be customised to meet the specific needs of the user.4

Until 2014, routine immunisation (RI) data and vaccine management information were collected exclusively through WHO’s District Vaccines and Devices Monitoring Tool (DVD-MT).5 Government users of the Excel-based tool reported: lack of access to historical data; inadequate data management skills to operate the tool; data entry errors; 1 month delay between data collection and analysis; inconsistencies between reports from health facilities (HFs) and LGAs for the same month; lack of data cross-checks to assess data accuracy, absence of timely corrective actions, and no online system to track vaccine accountability or cold chain functionality.6 7

In 2014, The National Primary Health Care Development Agency (NPHCDA) and Federal Ministry of Health (FMoH) requested that partners develop an RI Module on the DHIS2 platform to collect information on vaccine administration and management from the HFs of the national EPI. This module was designed to minimise some of the challenges associated with the use of DVD-MT. It calculated the indicators of the Accountability Framework for RI in Nigeria (AFRIN) in real time and posted the results on its dashboard.6 The goal was to ensure that government officials could access high-quality RI data to monitor performance and improve RI vaccination coverage.8 This article describes the implementation process—including planning and implementation activities, achievements, lessons learnt, challenges and innovative solutions—and reports the achievements in improving timeliness and completeness rates.

Approach to implementation

Partnerships

In 2014, the Bill & Melinda Gates Foundation (BMGF) provided funds over the course of the 5-year grant to support the development and implementation of the RI Module, while the government of Nigeria provided leadership on the NHMIS through FMoH, and on the RI Module through NPHCDA. These two agencies also facilitated advocacy visits at the national and state levels. The US Centers for Disease Control and Prevention (CDC) provided technical leadership as well as financial support. The African Field Epidemiology Network (AFENET) and its National Stop Transmission of Polio (NSTOP) programme were responsible for the implementation of the RI Module in all 36 states and the Federal Capital Territory, as well as technical support to states and the national level for RI data analysis and use. Other technical partners included the WHO Nigeria Country Office, the UNICEF Country Office, the Clinton Health Access Initiative and the Health Information System Programme of Nigeria.

DHIS2 RI Module and dashboard

The DHIS2 RI Module is an online platform to enter, store and analyse RI data from all HFs in Nigeria that offer these services. The RI dashboard incorporates 20 AFRIN indicators to monitor the EPI performance and process (figure 1). Its purpose is to strengthen the infrastructure for RI data collection, visualise RI data for action, develop workforce capacity in health information systems at all levels, improve data quality and use at all levels and support innovation and learning on health information systems.9–13 Data inputted into the DHIS2 RI module originate at HF level, and are collected on four paper-based collection forms. These forms were introduced in 2014, and include:

Figure 1

Visualisation of four AFRIN indicators on the DHIS2 RI dashboard (September 2019). AFRIN, Accountability Framework for RI in Nigeria; DHIS2, District Health Information System, V.2; LGAs, Local Government Areas; RI, routine immunisation.

NHMIS summary form (2013 version), which collects information on number and type of antigen doses administered, as recorded in the tally sheets and the HF monthly vaccination summary.

NHMIS supplementary form, which collects information on the number of RI sessions conducted, supportive supervision visits conducted, RI funding availability and information on newly introduced vaccines after 2013 (eg, inactivated poliovirus, rotavirus).

Vaccine management summary form, which collects information on the utilisation of antigens, diluents, devices and other vaccination materials.

Reaching every ward microplan summary form, which collects information on planned RI sessions and estimations of target population size.

The LGA receives the paper-based forms each month from each HF that offer RI services. The local immunisation officer (LIO) enters the information from the previous month on the DHIS2 data entry interface by the 15th of the following month. HFs do not enter their data on the platform because most do not have access to internet or computers, or the technical capacity to perform this task. Once entered, the raw data and the updated indicators immediately appear on the module’s dashboard.

There are 21 indicators on the RI dashboard, each reported using a graph, table or map. The indicators are divided in the following seven categories:

Data reporting

Reporting rates (public and private HFs)

Reporting rates (all HFs)

Proportion of HFs with timely reporting

Proportion of HFs reporting on time (all HFs)

Coverage and drop-out rates

Monthly and annual coverage rate, by antigen

Monthly coverage rate of selected antigens (aggrega ted)

Annualized coverage rate for selected antigens

Monthly and annual drop-out rates for Pentavalent 1-3 and BCG-Measles

Monthly dropout rate (Penta1/Penta 3) (aggregated)

Status of monthly Penta drop-out rate

OPV 3 vaccination coverage by LGA

RI vaccination sessions

Proportion of planned fixed and outreach sessions that were implemented in the past one month

LGAs holding at least 90% of fixed sessions in the past one month

LGAs holding at least 90% of outreach sessions in the past one month

Supervision

Proportion HFs that received at least one supervisory visit

Vaccine logistics

Proportion of HFs with vaccine supply stock-out reported in the past one month

Commodity stock balance

Vaccine stock balance

Cold chain functionality

Proportion of vaccine refrigerators functional

RI funding disbursed to health facilities

Proportion of HFs receiving adequate RI funding

HF allocation: Expected over actual for the last 6 months

The national-level officers review these data each month to provide feedback to the lower levels and design corrective actions. The health officials at the state and LGA levels—who include LIOs, cold chain officers, monitoring and evaluation officers, partners and political authorities—are required to review and analyse RI data on a monthly basis to identify gaps in RI services as well as emerging trends in RI utilisation (see figures 1 and 2). This information is intended to be immediately used to correct and improve RI performance in HFs, LGAs and states. HF staff receive information about their performance during these visits from the levels above. Most of these operations depend on the technical and financial support of partners in the field.

Pilot

The implementation of the RI module and dashboard on Nigeria’s DHIS2 platform was piloted in November 2014 in Kano state. The aim was to assess feasibility of implementing the RI Module at the state level, test the newly developed infrastructure and tools for data collection and analysis, and document challenges that could affect the implementation to other states. The first activity was to conduct advocacy visits to state-level authorities to promote how the introduction of the DHIS2 system could improve the completeness and accuracy of RI records. AFENET/NSTOP staff developed the training curriculum and instructed select state-level and LGA-level healthcare officers on data aggregation and entry into the DHIS2 interface, and on data analysis, interpretation and use for decision making. Between November 2014 and January 2015, the LGAs were scheduled in phases to receive the 2-week training. At the end of the pilot, NSTOP staff documented two lessons learnt: (1) deploy an AFENET/NSTOP consultant to each state for 2 years to support DHIS2 operations and train government staff and (2) include a computer literacy session in the curriculum. These lessons were applied in all subsequent state-level trainings and roll-out programmes.

Scale-up

The RI module/dashboard was rolled out to the remaining 35 states of Nigeria and the Federal Capital Territory. States had different implementation start dates—beginning in November 2014 in Kano and finishing in Jigawa in December 2017. The launch in each state took 2 years and progressed across four phases: preimplementation, technical launch, active phase and graduation (see figure 2). The goal was to have all states graduated and running the DHIS2 RI Module/dashboard independently by November 2019.

Figure 2

Implementation phases of the DHIS2 RI module and dashboard in each state over a 2-year period. DHIS2, District Health Information System, V.2; RI, routine immunisation.

Preimplementation: NPHCDA national-level staff, with CDC and AFENET, conducted advocacy visits to state-level authorities. The purpose of these visits was to ensure that the RI Module would receive political, financial and logistics support throughout the life of the project. The LIO was the officer designated for the monthly RI data entry process. Each state received a tailored RI dashboard, as well as training session for state-level and LGA-level officers.

Technical launch: The technical launch took place during the first 6 months of the module’s implementation at the state level. Updated RI data collection tools and one laptop were distributed to each LGA. NSTOP recruited one implementation officer (IO) per state, and deployed him/her at the start of the state’s implementation process to support RI module operations. The IOs worked within the State Primary Health Care Development Agency (SPHCDA) or the State Ministry of Health (SMoH) to implement supportive supervisory visits to improve RI data accuracy and completeness. Also, the IO trained the DHIS2 Focal Person (housed in the SPHCDA or SMoH) to take over all operations related to the RI Module and dashboard.

Active implementation phase: The active phase took place between months 6 and 18 of the implementation period. The IO conducted supportive supervisory visits to monitor the implementation of the RI module and dashboard, and provide on-the-job mentoring. Data validation rules, enhanced data visualisation options and an updated list of HFs were programmed on the DHIS2 platform to minimise the number of data entry errors. States used the RI dashboard to develop standardised monthly RI reports and interpret its results. A national-level monthly RI report was presented to NPHCDA and the National Emergency Routine Immunisation Coordination Centre (NERICC) for follow-up action.

Graduation: These activities took place during the last 6 months of the state-level implementation period. The national-level team visited the state twice to ensure a smooth transition. At the end of the 2-year period, the IO was withdrawn from the state and the DHIS2 Focal Person started operating independently.

Postgraduation: Activities during this phase ensured that state-level gains were maintained after SPHCDA took over all financial support and human resources for management of the RI module. A key component was NSTOP’s advocacy to government authorities (especially the states’ Executive Secretaries) to include DHIS2 funding in the state’s budget. NPHCDA’s Department of Planning, Research and Statistics (PRS) continues to provide technical support to the DHIS2 Focal Persons.

State-level support

Authorities in most states supported DHIS2 operations during and after the implementation period. For example, 22 states provide monthly internet bundles of 3.5 GB for RI data transmission, and 33 states conduct supportive supervision and on-the-job training during their monthly visits to HFs and LGAs.14 Seven states consistently provide funds to DHIS2 operations in their yearly budget. These funds are provided through a budget line for DHIS2 activities, the use of Save One Million Lives, and the financial support of partners. These funds ensure laptops replacement and maintenance, and supportive supervisory visits at the LGA and HF levels. Also, 30 states provide funding for printing NHMIS forms, while the remaining seven states rely on forms provided by the national level.

Patient and public involvement

Patients were not involved in any component of this project. The information collected through the DHIS2 RI module were the administrative data routinely recorded by the government of Nigeria as part of the national EPI programme. No additional information was sought or recorded.

Outcomes

National level

Monthly completeness reporting rates improved dramatically over 4 years, from November 2014 to July 2019 (see figure 3). More than 80% of LGAs reported on all four RI data sources. The overall completeness measure is stable at 80%–90% since July 2018.

Figure 3

Completeness rate of the monthly reporting for the four sources of RI data in Nigeria, from the launch of the DHIS2 RI module project in November 2014 to July 2019*. *The data do not represent the 774 LGAs at one time. Rather, information from the LGAs was added as each state moved into the active phase. The graph reports information on all LGAs starting in December 2017. DHIS2, District Health Information System, V.2; HF, health facility; LGAs, local government areas; NHMIS, National Health Management Information System; RI, routine immunisation.

State and LGA levels

All states report RI data through the Module, identified one DHIS2 Focal Person, and review the RI dashboard each month. Online supplementary appendix 1 reports the completeness rate of reporting for each state, both during the IO’s deployment and after his/her withdrawal. No state reported a steep drop in completeness rate after the IO left, and some states continue to improve.

Lessons learnt

Before the implementation of this module, Nigeria RI data were not fully available to authorities because the previous system was offline, poorly maintained and hosted by partners. Now government has full ownership of the platform. This engagement, coupled with the financial investments that some states made, suggests that the DHIS2 RI module and dashboard are successfully incorporated into the Nigerian primary healthcare system.

Implementation challenges and lessons learnt

NPHCDA and AFENET/NSTOP encountered numerous challenges to the implementation process of the RI module. Below is a list of issues and the corresponding lessons learnt:

Seven states have yet to budget funds for the maintenance of the RI module, and no state fully funds all aspects of the RI module. Activities like laptop maintenance, NHMIS form printing and LGA-level RI review meetings cannot be carried out consistently because of this deficit.

AFENET/NSTOP could not maintain the rigorous implementation schedule without the support of government authorities. However, once the IO withdrew from a state, some HFs and LGAs stopped reporting their monthly RI data regularly. Data managers from both NPHCDA and NERICC communicate regularly with the states to encourage timely reporting.

High staff turnover, leading to poor DHIS2 management skills at the LGA level. AFENET/NSTOP organises supervisory visits for data quality since 2017. These visits are opportunities for refresher trainings. NPHCDA and partners are considering expanding their field visits to all LGAs.

Subpar accuracy of RI data reporting on both paper-based and electronic tools. Errors are due to transcription or summation mistakes at the HF level.14 Another serious concern is the lack of paper tools for data collection. NPHCDA and AFENET/NSTOP programmed multiple validity checks and compulsory data elements in the RI module, developed indicators to identify implausible values and identify outliers. WHO introduced additional data quality indicators on the DHIS2 platform in January 2019. These tools reduced the number of errors for most states (see figure 4). The customisation of the RI dashboard, along with the introduction of a user-friendly interface, enhanced the module’s relevance to states, and promoted the use of RI data to improve the quality of RI services.

Figure 4

Example of RI data quality improvement on the RI module. Three has been a steady reduction in the number of data entry errors* since January 2019. *There should be no data entry for antigens hepatitis B, doses 1 and 2 because they are not included in the Nigerian immunisation schedule. RI, routine immunisation.

Lack of reliable internet connectivity. Many LGA officers use internet cafes or work in the evening. NPHCDA introduced a daily SMS reporting system in the 18 states to ensure that RI data could be transmitted to the national level. Soon, Gavi will sponsor the introduction of electronic data collection at the HF level across Nigeria. Data transmission will rely on internet connectivity, SMS texting or a combination, depending on the location of the HF.

Management of HF lists that offer RI services, which are seldom updated but are essential to calculate accurate vaccination coverage rates. FMoH is working to establish standard operating procedures for updating and managing the HF list that is uploaded onto the DHIS2. The list of HFs has already been updated for 18 states, and the Global Fund has provided financing to complete the work in the remaining 19 states.

Finally, for countries who wish to replicate this implementation model, we recommend that the introduction plan for a module-based public health data collection project include the same elements as our introduction process:

Advocacy and support to the development of government policy and HIS infrastructure.

Gap analysis and planning to guide implementation.

Key stakeholders’ involvement at every step of the implementation process.

Continuous improvement of the plan through data review.

Utilisation of multiple funding sources.

Limitations

The main limitation was the lack of reliable Government funding for DHIS2. AFENET/NSTOP provided technical and financial support in the field, but its participation in this project ended in February 2020. WHO already started to reduce its footprint in the LGAs as part of its Polio Endgame strategy. To mitigate this withdrawal, AFENET/NSTOP used the last year of the project to transfer technical knowledge and skills to the Department of Planning, Research and Statistics (PRS) of NPHCDA. As of April 2020, tts data managers can manage the RI module independently. Gavi and the Global Fund continue to provide financial support for the next milestone of data quality improvement, and other partners continue to provide technical support.

Next steps

Training programmes are shifting daily RI module operations from AFENET to PRS-NPHCDA. NERICC continues to receive the monthly RI data and reports for its own operations and planning. PRS-NPHCDA, NERICC, AFENET/NSTOP, CDC and other stakeholders will address RI data quality issues on the platform.

Conclusions

FMoH, NPHCDA and NSTOP promoted the sustainability of the RI data management system by introducing the DHIS2 RI Module and dashboard to all states of Nigeria. To date, all states are using this module to report their RI data to the national level, and to propose targeted activities and corrective actions for underperforming LGAs and HFs based on information on the DHIS2 platform. While numerous challenges still exist, the implementation of the RI module and dashboard in Nigeria was successfully accomplished within the given time frame. Operations continue to strengthen the accuracy and completeness of the RI information.