Evaluation of the adverse events following immunization surveillance system, Ghana, 2019.

BACKGROUND: With over 80% of children worldwide vaccinated, concerns about vaccine safety continues to be a public health issue. Ghana's Adverse Events Following Immunization surveillance started in 1978 with the objective to promptly detect and manage AEFI cases either real or perceived. Periodic evaluation of the surveillance system is critical for optimal performance; hence we evaluated the system to assess its attributes, usefulness and system's performance in meeting its objectives.
METHODS: A case of AEFI was defined as any untoward medical event occurring within 28 days after vaccination and may not necessarily have causal relationship with the vaccine use. We reviewed surveillance data and procedures for the period 2014 to 2018 and interviewed key stakeholders. Adapting the CDC's Updated Guidelines for Evaluating Public Health Surveillance Systems, we assessed the system's attributes and usefulness. We performed summary descriptive statistics on quantitative data and directed content analysis on information gathered from interviews.
RESULTS: In all, 2,282 AEFI cases including 476 (21%) serious cases (life threatening events) were reported for the period. The highest case detection rates of 61.45 AEFIs per 100,000 surviving infants was recorded in 2018. Reporting forms were modified to accommodate new indicators without any disruption in the function of the system. At the national level, completeness of 100 randomly sampled reporting forms (100%) and was higher than the region (27%) but timeliness (50%) was lower than the region (83%). All (16/16) Community Health Nurses interviewed indicated "fear of being victimized" as the reason for underreporting, nonetheless, the system was useful as it made them cautious when vaccinating children to prevent reactions. Data on AEFI surveillance was also useful in guiding training needs and provision of vaccination logistics.
CONCLUSION: The AEFI surveillance system is useful at all levels but partially meeting its objective due to underreporting. We recommend training and supportive supervision to improve timeliness of reporting, data completeness and acceptability.

Introduction

Vaccination is an important public health tool that prevents about 2 to 3 million deaths worldwide yearly [1]. Generally, vaccines are safe and effective and undergo extensive safety monitoring prior to its usage in immunization programs. However, no vaccine is entirely without risk and Adverse Events Following Immunization (AEFI) may occur [1] which can be detected through Post market surveillance.

Adverse Events Following Immunization (AEFI) is any untoward medical occurrence that follows immunization and does not necessarily have a causal relationship with usage of the vaccine [2]. The adverse event may be any unfavorable or unintended sign, abnormal laboratory finding, symptom or disease. AEFIs may range from mild reactions such as fever or rash to severe reactions such as convulsions, coma and even death [3]. Based on the outcome of the event, the World Health Organization (WHO) has classified AEFIs as Serious and Non-serious which is used for regulatory classification. AEFIs are further classified based on the cause of the event. These include, vaccine product-related, vaccine quality defect-related, immunization error-related, immunization anxiety-related, coincidental and unknown events [4]. Hence, AEFIs may be true adverse reactions (ARs) to the vaccine, the vaccination process or may be unrelated.

Though over 80% of children worldwide are vaccinated, vaccine hesitancy continues to be a public health issue [5]. Surveillance on immunization safety is hence crucial to build trust and to reassure the public that AEFIs are being monitored and actions are being taken to reduce risks. This will reduce vaccine-hesitancy and sustain efforts by immunization programs [6]. Reporting rates of AEFIs are usually low, especially in developing countries [7]. This is due to over reliance on the routine system of reporting which is subject to underreporting and poor pharmacovigilance infrastructure in developing countries [8]. To address the issue of underreporting the WHO therefore instituted the Global Vaccine Safety Initiative, which set out indicators primarily for monitoring case reporting [9]. A case reporting target of 10 AEFI per 100,000 surviving infants per year was set in the Global Vaccine Action Plan (GVAP) to monitor performance of AEFI surveillance systems [10].

In 2015, the average reporting rate was 549 AEFI per 100,000 surviving infants globally. The number of countries that reported rates greater than 10 per 100,000 surviving infants also increased from 8 (4%) in 2000 to 81(42%) in 2015 [11]. Meanwhile, only half (51%) of the countries in the WHO African Region, reported AEFI cases in 2015, and the region recorded one of the lowest case reporting rates of 74 AEFIs per 100,000 surviving infants [11]. Studies done in some African countries have also reported incidence of AEFI ranging from 13 to 34% [3, 12].

In Ghana, routine reporting of AEFI began with the initiation of Expanded Program on Immunization in 1978 and also by the Food and Drugs Authority (FDA) in 2001 upon establishment of the National Pharmacovigilance Centre [4]. The surveillance system was established with an overall aim of promptly detecting and managing AEFI, real or perceived. This would contribute to the credibility of immunization program and prevent inappropriate responses to reports of AEFI that could lead to crises in the absence of a surveillance system.

We evaluated Ghana’s AEFI surveillance system to assess its attributes, its usefulness and to determine whether the system is meeting its objectives.

Methods

Evaluation design

The evaluation was a cross-sectional study that involved collection of data through reviewing of surveillance data, observation of surveillance procedures and interviewing of key stakeholders. The evaluation was conducted in January 2019 and covered the period from 2014 to 2018.

Study area

The evaluation was done in Ghana using Greater Accra Region as the focal region and Ga East Municipality, as the focal district. Interviews were done among selected stakeholders from the district, regional, and national levels, and surveillance records were sampled at all these level for the evaluation. In 2018, Ghana’s population of children under 5 years of age targeted for routine and supplementary childhood immunization was 5,922,867 with 912,194 fully immunized. The Greater Accra Region also had an under 5 years population of 579,805 with 136,150 fully immunized, while Ga East District which is one of the 26 districts in Greater Accra Region had under 5 years population of 22,431 with 3,455 children fully immunized. The district has 16 health facilities, one health centre, one polyclinic and one Quasi Government. It also has 14 private health centres and 15 Community-based Health Planning and Services (CHPS) zones of which one has a compound. There is one health care worker to every 132 children under five who offer immunization and related services. Health worker training on AEFI is done prior to every supplemental immunization activity.

Data collection tools and methods

We adapted the Centre for Disease Control and Prevention Updated Guidelines for Evaluating Public Health Surveillance Systems [13]. This tool provides indicators to measure the qualities of a surveillance system. Using these indicators, we developed an interview guide and checklist to suit the objectives of the evaluation and describe system’s operations. Thirty key informants from health facilities (19) district (6), regional (3) and national (2) levels were purposively selected and interviewed based on their role in AEFI surveillance. Questions were asked regarding operations of the surveillance system. We observed surveillance processes and reviewed data (both paper and electronic-based) from 2014 to 2018 at all levels. Data was collected on the number and type AEFI, timeliness and completeness in reporting among others. The sources of data were; a) case-based forms, b) weekly and monthly Integrated Disease Surveillance c) Response (IDSR) reports, d) DHIMS 2 and e) Technical Advisory Committee for Vaccines and Biological Products (TAC-VBP) reports.

A case of AEFI was defined as any untoward medical event occurring within 28 days after vaccination which does not necessarily have causal relationship with the vaccine use [14]. The adverse reaction was classified as Serious or Non-serious or Clustered [15].

Serious AEFI

An AEFI was considered serious if; It is life threatening, results in inpatient hospitalization or prolongation of existing hospitalization, results in persistent significant disability/incapacity, results in death, or results in a birth defect/congenital anomaly.

Non-serious AEFI

These are events that do not meet the definition of a serious AEFI and usually include mild to moderate symptoms that are temporarily lived. Though common, these must be carefully monitored as they may signal a potentially larger problem with the vaccine or immunization as well as have negative impact on the public acceptance of the vaccine.

AEFI clusters

Two or more AEFIs related in time and/or place or vaccine administered.

Assessment of performance of the AEFI surveillance system in meeting its objectives

We reviewed the objectives of the surveillance system to substantiate its fulfilment or otherwise. Its objectives are to; detect, correct and prevent immunization error-related AEFIs, identify problems with vaccine lots or brands leading to vaccine reactions, prevent false blame arising from coincidental adverse events, maintain confidence by properly responding to community concerns, generate new hypotheses about vaccine reactions and estimate rates of occurrence of AEFIs. This was assessed through records review and interviews. Summary descriptive statistics, was used to determine number of AEFI cases detected and proportion investigated to determine its cause. Directed content analysis was used to summarize information on how AEFIs were managed at the facilities, ability of the investigation process to refute false claims of vaccine-related reactions and use of data for signal event detection and corrective action such as withdrawal of an implicated vaccine.

Assessment of usefulness of the AEFI surveillance system

We interviewed stakeholders on; the system’s effect on policy decisions and use of data for action, its contribution to the detection and prevention of AEFIs, and its effect on clinical practices. We also assessed ability of the system to achieve its set objectives through records review and interviews. Directed content analysis was used to summarize information gathered.

Assessment of attributes of the AEFI surveillance system

We described and analyzed each attribute of the system as shown in .

Ethical considerations

Permission was sought from the Ghana Health Service, Greater Accra Regional Health Directorate and Ga East Municipal Health Directorate. Permission was also sought from Food and Drugs Authority and Expanded Program on Immunization for the use of data. Informed written consent was sought from interviewees without any form of duress. Data obtained was put into Microsoft excel which was password protected. This evaluation was done as part of routine assessment by the Ghana Health Service and Ghana Field Epidemiology and Laboratory Training Program (GFELTP).

Results

Background characteristics of study participants

Thirty health staff were interviewed of which 21 (70%) were females. The mean (SD) age and years of service among respondents was 38.7 (4.1) years and 13.1(2.3) years respectively. At the facility level, sixteen (53.3%) community health nurses, two (6.7%) medical officers, one (3.3%) pharmacist were interviewed while at the district level, four (13%) disease control/surveillance officers, two (6.7%) Health Information Officers were interviewed. At the regional level three (10%) key informants; the Regional DCO, DSO and EPI focal person were interviewed and at the national level two (6.7%) key informants; from EPI Program Manager and Head of Pharmacovigilance of FDA were interviewed.

Out of the 30 respondents, 26(87%) understood the purpose of the AEFI surveillance system and 28(93%) could define AEFI. All respondents knew at least two presenting signs/symptoms of AEFI, while 24(80%) knew the correct timelines for submission of forms. Six (20%) did not know how to fill a reporting form all of which were community health nurses.

Operations of the surveillance system

The operation of the AEFI surveillance system is a collaborative effort between the Expanded Program on Immunization of the Ghana Health Service and the Food and Drugs Authority (FDA). The system involves collection and collation of routine data using the health structures of the Ghana Health Service. Case reporting is passive where caregivers/vaccinees report adverse events to health facilities. The health facilities record reported events using a standard case reporting form and submits the report to the District Health Directorate. The District focal person enters the data on the forms into District Health Information Management System II (DHIMS 2) and transmits the form to the National level through an intervening Regional Focal Point. Reports must be backed by case reporting forms and validated before entry into the DHIMS. Data are further aggregated at the regional and national levels. Also, some notifications are sent directly from the community or the health facility to the FDA through an electronic reporting system. Once such notification is received, focal points at the FDA complete the standard AEFI form (Fig 1).

Fig 1

Information pathway, AEFI surveillance system, Ghana.

Source: Guidelines for Surveillance of AEFI in Ghana [15].

Investigation of an AEFI is initiated within 24 to 48 hours of receipt of report at the lowest level with the capacity to carry out that investigation. Causality assessment is done on serious AEFI cases and AEFI clusters by a Technical Advisory Committee for Vaccines and Biological Products (TAC-VBP) constituted by the FDA. Meetings of the Committee are held at interval of at most two months. Feedback back on receipt of report is given within 48 hours to the reporting facility/level. However, feedback on serious AEFI is given bimonthly when the TAC-VBP meets for causality assessment.

Sustainability of the system depends on the availability of adequate resources at each level of the surveillance system. The Gavi Alliance is a major financier of the immunization Program and AEFI surveillance in Ghana. Other Partners include WHO and UNICEF. Tools for AEFI surveillance include AEFI reporting form, investigation form, guidelines for completing AEFI reporting form, electronic line listing form, clinical laboratory form and the guideline for AEFI Epidemiological investigation.

Assessment of performance of the surveillance system in meeting objectives

Objective 1: Detect, correct and prevent immunization error-related AEFIs

The surveillance system detected a total of 2,282 AEFI cases including 21% (476/2282) serious cases during the 5-year period (2014–2018) of which 15% (72/476) were immunization error related. All (16/16) CHNs interviewed acknowledged that the presence of AEFI surveillance makes them more cautions when vaccinating in order to prevent immunization error-related AEFIs. Health workers had also taken measures to prevent immunization error-related AEFIs which include stopping student nurses on attachment from administering vaccines.

Objective 2: Identify problems with vaccine lots or brands leading to vaccine reactions

The AEFI case-based form captures the lot/batch number of vaccines that the vaccinee took prior to the reaction. This aids to the identify any problems vaccine lot during investigation. About 97% (463/2282) of serious cases were investigated using the lot/batch number of vaccines.

Objective 3: Prevent false blame arising from coincidental adverse events

By investigating deaths that followed vaccination and performing causality assessments the system has been able to prevent misinformation speculating that the vaccine attributed to the death.

Objective 4: Maintain confidence by properly responding to parent/community concerns

The awareness campaigns and health education given on AEFIs during immunization sessions increased public confidence. Also, all reported AEFIs were managed at no cost to the affected individuals as this cost was absorbed by the Expanded Program on Immunization. This made caregivers more comfortable to vaccinate their children and report any events.

Objective 5: Generate new hypotheses about vaccine reactions and estimate rates of occurrence of AEFIs

The system was able to monitor trends of AEFI cases and detect unusual trends in number of cases, however there was no documentation on incidence of AEFIs in Ghana.

The reporting form captures demographic features of the vaccinee as well the characteristics of the vaccine taken. This informs the age groups at risk and the antigens causing the most adverse reaction for public health action to be taken. The EPI coordinators at the regional and national level revealed during interviews that data on AEFI surveillance was useful in guiding training needs and provision of vaccination logistics. Lastly, the system led to change of clinical practices as all CHNs indicated that the system made them extra careful during injection in order to prevent abscess and ensured that vaccines are stored properly and used within required time.

Simplicity

Sequential relay of information in the system was clear and well structured. Ninety-three percent (28/30) of stakeholders interviewed knew and understood the AEFI case definition. Identifying a case involved observing the individual and could be captured through easily detectable symptoms requiring basic tools like a thermometer. The case reporting form was a one-page document, with 25 core variables which during interviews, 80% (24/30) stakeholders said it takes less than 15 mins to fill. On the other hand, serious cases required laboratory confirmation and causality assessment which had longer turn-around time especially for a coroner’s case. The case investigation form is a four-page document with many variables that required some level of expertise to fill. Additionally, the perfect characterization of AEFI is elusive due to absence of pathognomonic signs or available laboratory tests that would allow confirmation of diagnosis.

Acceptability

Stakeholder participation at the national level was high, proven by periodic meetings to review data, analyze and discuss ways to improve the system. Support from WHO, UNICEF and GAVI also demonstrated high acceptance of the system by partners. Stakeholders interviewed at regional level revealed that the system is essential and easy to run. Meanwhile, at the district, all 16 community health nurses interviewed addressed concerns of being victimized for reported events (

Sensitivity

The case definition was sensitive, capturing any untoward medical event after vaccination whether caused by the immunization or not. At the national level, the GVAP case detection target of 10 AEFI/100,000 surviving infants /year was met only in 2017 and 2018 with 35.69 and 61.45 AEFIs per 100,000 surviving infants respectively ( In 2018, 80% (8/10) of regions in Ghana achieved this target based on routine vaccination reports (. At the district level, the annual case detection rate for 2014, 2017 and 2018 was 4.54, 5.83 and 4.54 AEFIs per 10,000 surviving infants respectively. No case was detected in 2015 and 2016 (

Positive Predictive Value (PPV)

PPV is the proportion of AEFIs that are actually caused by the vaccine. This was assessed at the national level since causality assessment is only done at this level. For 2014 and 2015, PPV could not be calculated due unavailability of data, however in 2016, 2017 and 2018 the country recorded PPVs of 66.7%, 50% and 63.6% respectively.

Timeliness

Seven case reporting forms were found at the district of which two were submitted to the region on time. At the region, 83(83%) of 100 forms randomly selected were submitted by districts on time. At the national level, 50% (50/100) of reports were timely submitted (within 28 days from onset of symptom) in 2017 and 2018 and 80% of forms were timely submitted in 2016. Upon interviewing surveillance staff at the district and sub-district levels, 88% (22/25) stated that they did not get feedback when they sent reports to higher levels.

Stability

The AEFI surveillance system is run on the Ghana Health Service (GHS) structures and health workers report data as part of their routine work requiring no extra financial benefit. Cased-based forms were readily available either at the facility or online to be downloaded and printed when needed. The AEFI surveillance guidelines was available at all levels. All levels also had surveillance focal persons for EPI who served as the AEFI focal person. Surveillance data was stored electronically at all levels in the DHIMS 2 which can be assessed once authorized. This makes it easy to integrate the data into other systems. However, GAVI is the sole financier of immunization and vaccine safety in the country.

Data quality

All case investigation forms randomly sampled at the national level had all the 25 core variables completely filled. At the region, only 27(27%) of the 100 forms sampled were completely filled while at the district none of the seven forms available was completely filled. Data imputed into the DHMS were inconsistent with data available at each level of the system. For instance, Ga East recorded 92 AEFI cases in the DHMS in 2018, which proved to be a data entry error.

Representativeness

Case reports were from both males and females, however, distribution of cases was mainly among children under five, excluding pregnant women and travelers who took tetanus and yellow fever vaccines respectively. All regions submitted case reports to national level for all years assessed. However, reporting rates among districts to the region was low as only 3 out of 26 districts in Greater Accra submitted AEFI case reports in 2014 and 2015, while 5 districts submitted in 2016 and 2017. At the district level, case reports were mainly from health facilities in 2 out of 5 subdistricts for all years assessed, but included both private and public facilities.

Flexibility

The monthly vaccination report and case-based form were modified to accommodate new indicators without any disruption in the function of the system. The AEFI case definition was also altered from a definition that implicated the vaccine use to a definition that the AEFI may or may not be caused by the vaccine. It required a day training for the disease control and surveillance officers at the district levels and on-job coaching for the stakeholder at the facility level. Dissemination of reports is made flexible by inclusion of phone calls and through WhatsApp platforms for instances where paper report submission will delay response. However, the case-based form must be forwarded within 48 hours or be filled by the recipient so as not to compromise the completeness of reports. The reporting form also had a space for additional information to be added.

Discussion

The surveillance system was found to be functional and well-structured due to a collaborative effort among the Ghana Health Service, the FDA and Partners. Health workers had high knowledge of the system which could be attributed to their longer years of work experience in service (mean years in health service was 13.1). Yet high knowledge among health workers did not reflect reporting rates. This is because the system is heavily reliant on voluntary reporting by caregivers and most of these caregivers were not reporting to the facilities. This may be because caregivers were not educated enough to know that they have to report these events, as found in a study by Constantine et al. [3]. However, Chimusoro et al., had contrary findings where though caregivers had good knowledge on the surveillance system, they still did not report [16]. It could also be because caregivers may feel coming back to the health facility to report is burdensome. To curb this, an active, participant-centered system can be adapted as an adjunct to traditional passive AEFI surveillance. This is an innovation that allows caregivers use communication technologies to report an AEFI to a facility or a central body such as the FDA and has proven to increase reporting rates in other countries [7, 17].

Another reason for low reporting rates was the fear of victimization among the vaccinators. Though most of these health workers knew the purpose of the surveillance system, they still perceived that reporting AEFI will question their work proficiency. This makes the system less acceptable to the vaccinators. Zvanaka et al. had similar findings where health workers had high knowledge on AEFIs yet refused to report due to fear of victimization and fear of exposing work incompetency [18]. This shows that knowledge and perception of health workers towards a surveillance system has great influence on its success [19], hence training must be geared towards debunking this perception.

We found the AEFI case definition to be sensitive as it removes the need to ascertain causality. This also confers a level of simplicity on the surveillance system. However, efforts made to make the system simple and sensitive, did not reflect on case reporting rates especially for 2015 and 2016. High reporting rates in 2018 however could be attributed to the frequent sensitization and training on AEFI during a Yellow fever and Measles-Rubella campaigns held in that year. This goes to prove that routine reporting of AEFI will receive such boost if given similar attention as during campaigns.

Case reporting was simple but case investigation was cumbersome, partly due to the lack of pathognomonic signs and laboratory tests for perfect characterization of an AEFI [20]. As a result of these complexities in the investigation process, assessment of causality is limited to serious AEFIs [4]. Thus, for AEFIs classified as non-serious, its cause may not be identified for corrective actions to be taken.

Representativeness of the system was threatened as the system did not adequately capture AEFIs on Tetanus and yellow fever vaccines received by the pregnant women and travelers respectively. Thus, the ante-natal clinic and the port health clinics must be engaged to report AEFIs. Also, the region had many silent districts which may mean these districts are not sensitized enough to report. A study found representative of the AEFI surveillance system in Kwekwe District in Zimbabwe to be threatened by failure of private facilities to report. This was attributed to low knowledge among health workers in the private sector [16]. This contradicts with our findings where both private and public facilities reported because, immunization services provided at private facilities are mostly carried out by government staff from the various subdistricts as part of their routine work.

Errors in reporting into the DHIMS and inconsistencies in the number of case reports observed at all levels suggest a weak data validation system, Meanwhile, accurate reporting is crucial in informing decision making.

The system was stable with both human and material resources available for operation. However, if GAVI withdraws its funding, the system may collapse. Hence the GHS/EPI and FDA must institute strategies to take over funding of the system if support from GAVI is halted.

The system’s flexibility in the use of phone and WhatsApp platforms for reporting reduces the delays in reporting. Nonetheless, human negligence could lead to failure to fill a form and hence gaps in reporting.

The AEFI surveillance system is very useful based on indicators assessed. However, lack of timely feedback from higher levels can eclipse its usefulness. Surveillance staff could be discouraged from sending case-based forms to higher levels, which could impede the use of data for decision making. In Kwekwe District in Zimbabwe, 71.7% of health workers in a study stated that they did not receive feedback from the upper levels and hence were discouraged to forward reports to these levels [16]. Zvanaka et al. also indicated that though the majority of health workers reported the system to be useful, its usefulness is not evidenced by reported feedbacks especially at the district and subdistricts [18]. The usefulness of the system can therefore be improved if national and regional levels give timely and precise feedback on AEFI reports.

Unlike drugs, vaccines are given to seemingly healthy individuals and in vast numbers especially among infants. Thus, issues arising from vaccination are less tolerated by the general public. With this, a more robust surveillance system has to be achieved to maintain public confidence in immunization services to sustain efforts by stakeholders.

This evaluation had a few limitations. There was the possibility of recall bias during interviews as study was conducted in 2019, but required investigation from year 2014. As such, records had to be reviewed to confirm some of the facts obtained during the interviews. Also, most interview questions were asked again among other stakeholders to review the validity of the previous information collected. Again, the researcher relied on AEFI case-based reports and secondary data which could be subject to reporting and data entry errors.

Conclusion

The AEFI surveillance system is well structured with standardized data collection tools. The system is useful in estimating magnitude of morbidity of AEFIs and improving clinical practices. The system was flexible and stable, with a sensitive case definition however, reporting rates were low due to underreporting. With this the system was partially meeting its objective to promptly detect and manage AEFIs. Acceptability and simplicity were fair, while representativeness, data quality and timeliness were inadequate. We recommended training of vaccinators on AEFI reporting by the AEFI focal persons to counter the fear of being victimized for reported events. We also recommend periodic supportive supervision by Health Management Teams to reporting facilities to ensure improvement system’s attributes and use of data for action.

The West African Health Organisation (WAHO)

The Ghana Field Epidemiology and Laboratory Training Programme

The Disease Surveillance Department of the Ghana Health Service.

The Food and Drugs Authority, Ghana.

The Expanded Program on Immunization, Ghana.

Dr. Basil Benduri Kaburi

Dr Selorm Botwe (Ga East Municipal Director of Health Services)

Mr. Bright Amoako (Health Information Officer, Ga East Municipal)

Manuscript dataset.

(XLSX)

Click here for additional data file.

19 Aug 2021

PONE-D-21-21870

Evaluation of the Adverse Events Following Immunization Surveillance System, Ghana, 2019

PLOS ONE

Dear Dr. Frimpong,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The authors present an evaluation of Ghana’s AEFI surveillance system (i.e. does the system meet the programme’s objectives) using an adaptation of a CDC tool. Vaccine safety is an important area of research and underserved in sub-Saharan Africa and the study has the potential to add to the knowledge base. Unfortunately, in its current form, the manuscript does not meet the criteria for publication.

The overall methodology is very unclear, although some study procedures are described. At present it is difficult to relate the methods to the objectives and results.

Reviewer 2 has offered a very thorough summary with constructive suggestions.

Please also review my more detailed comments in the attached document.

Please submit your revised manuscript by Oct 03 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Emma K. Kalk

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

3. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

4. Please upload a copy of Figure 1 and 2, to which you refer in your text on page 6 and 12. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The present manuscript (Evaluation of the Adverse Events Following Immunization Surveillance System, Ghana, 2019) seeks to evaluate Ghana’s AEFI surveillance system. Although the subject regarding AEFI in children is certainly of general interest, the manuscript is not uniform and easy to read, and has several English inaccuracies. Given the low quality of the methods and the poor originality of the findings, the manuscript in its current version is not acceptable for publication.

Reviewer #2: This study is a good assessment of the AEFI reporting system in Ghana and is a valuable contribution to the world of vaccine safety surveillance and system strengthening. A good deal of information has been gathered. However, it is a little difficult to connect the methodology to the results. There would be benefit in revising the layout so that objective and measurable are clearer.

Specific commets:

Abstract:

Line 40: The three updates within last decade appears not to be included in the main body of text

Line 43: ‘All (16/16) Community Health Nurses interviewed indicated “fear of being victimized”’ doesn’t connect to the second part of the sentence; needs to be made more clearer

Introduction:

Line 54: “Vaccine safe and effective; however adverse events following immunization”: this is contradictory; although I agree vaccines are generally safe; perhaps add the word, Generally vaccines are safe and effective;

Also it would be beneficial to add something that vaccines undergo extensive safety testing during clinical trials but rare events can be detected through post marketing surveillance

Line 60 ; serious and non-serious should not be confused with severity; based on the outcome of the event (e.g. hospitalization, death); seriousness is not based on severity; this sentence should be corrected.

Line 66: vaccine hesitancy is a public health issue; I would suggest replacing the term “concerns” with vaccine hesitancy:

67: There should be an explanation on why surveillance on immunization is essential; I.e.: to build trust; to reassure the public that AEFIs are being monitored and actions to reduce risks are being taken

Line 69:AEFI to AEFIs

Line 69: Under reporting is a well-known with routine reporting systems

Line70: the term routine is preferable to passive

Line 70: HICs also rely on routine systems; under reporting is a general limitation of passive/routine systems; but is there any other reason that LMICs have additional underreporting?

Line 70 -72: perhaps rephrase to emphasize the link: underreporting was addressed via the GVSI. One of the targets of… was set.

Materials and methods

Line 93: is there a reference for CDC guidelines for evaluating ?

Line 94: how were these guidelines adapted? What were the quantitative and qualitative data that was collected?

Line 98: Study area paragraph: The evaluation was made through interviewing stakeholders in this region? And reviewing reports in this district? Perhaps clarify what resources in this area was used to do the evaluation.

Line 125: I don’t think the term detect can be applied here; perhaps just state that caregivers/ vaccinees report to health facilities. How do they report? Using a reporting form?

Line 126: What is meant by manage reports? They receive and code them? And they transfer the reports?

Line 153: title says data collection and analysis, but no analysis was described: perhaps expand on the analysis or remove analysis from title.

Also please make data sources clearer :by stating five different sources of data was collected data included: a) interviews .. b) case-based forms; c) IDSR reports; d) DHIMS2; e) TAC.VBP reports

Line 159: explanation of what TAC-VBP reports are?

Line 165: explain what is meant by hypothesis generation; is this signal detection?;

Line 199; prefer to use the word understood rather than knew

Line 205 section on performance of the surveillance system in meeting objectives: objective were; 1) detect, correct and prevent immunization error-related AEFIs; 2)

identify problems with vaccine lots or brands leading to vaccine reactions, 3) prevent false

blame arising from coincidental adverse events, 4) maintain confidence by properly responding

to community concerns, 5) generate new hypotheses about vaccine reactions and estimate rates

of occurrence of AEFIs.

Use subtitles to indicate the results of assessment of each of these objectives;

Line 208: would it be better to say 463/476 (97%) of all serious cases were investigated?

Line 209: suggest to rephrase replace the word exonerate; By investigating deaths that followed vaccination and performing causality assessments the system has been able to prevent misinformation speculating that the vaccine attributed to the deaths? Also link tis to the objective of prevent false blame arising from coincidental adverse events

Line 211: link lot/batch number to objective: identify problems with vaccine lots or brands leading to vaccine reactions

Line 212: I would replace the word victim; perhaps individual;

Line 212: who paid for the management of the AEFIs?

Line 214: it is difficult to get an incidence rate with spontaneous reporting as there is always under reporting. Even if exposure information is known, the actual number of AEFIs collected through spontaneous reporting is an under estimation of incidence. Active surveillance methods are needed to estimate the incidence

Line 212-213: does this refer to signal ( hypothesis generated)? This needs to be made more clearer

Line 215: how was the usefulness assessed? Was this through interviews? Where did this information come from?

Line 223-224: case definition of AEDI reports?,

Line 224: please do not use the word victim; an individual is suspected to have AEFI; causality is not determined; the word victim gives a negative connotation, when most of the time benefits of the vaccine outweigh the risk

Line 225-230: was 15 minutes to fill a report and investigations into serious cases considered to be simple?

Line 280-281: was this a human error or a problem with the tools?

Line 308: in Ghana is this the case, do caregivers have to go back to the facility to report? Or are there tools in place to allow patient/caregiver reporting?

Line 344: full stop (.) after system

Line 349: does Whatsapp platforms compromise the completeness of reports?

Another limitation is recall bias in interviews; as study was conducted in 2019 , investigating from year 2014.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Reviewarticle.docx

Click here for additional data file.

Submitted filename: PONE-D-21-21870.docx

Click here for additional data file.

15 Oct 2021

The Academic Editor

PLOS ONE

Dear Sir

Evaluation of the Adverse Events Following Immunization Surveillance System, Ghana, 2019 (PONE-D-21-21870)

Thank you for the review and providing comments to improve our manuscript. The table below are answers or responses to specific comments made by the reviewers.

COMMENT COORECTIONS MADE

ABSTRACT

Line 40: The three updates within last decade appears not to be included in the main body of text

The monthly vaccination report and case-based form were modified to accommodate new indicators. The AEFI case definition was also altered from a definition that implicated the vaccine use to a definition that the AEFI may or may not be caused by the vaccine. This has been included in lines 493 to 496 at the results section.

‘All (16/16) Community Health Nurses interviewed indicated “fear of being victimized”’ doesn’t connect to the second part of the sentence; needs to be made more clearer

The sentence has been revised to build the connection between the two phrases to make the meaning of sentence clearer. This can be seen in line 45 to 48.

INTRODUCTION

“Vaccine safe and effective; however adverse events following immunization”: this is contradictory; although I agree vaccines are generally safe; perhaps add the word, Generally vaccines are safe and effective;

The word “Generally” was introduced to the sentence in line 60.

Also, it would be beneficial to add something that vaccines undergo extensive safety testing during clinical trials but rare events can be detected through post marketing surveillance

The sentence was improved by adding that vaccines undergo extensive safety monitoring prior to its usage in immunization programs. However, no vaccine is entirely without risk. This can be found in lines 60 to 63.

Serious and non-serious should not be confused with severity; based on the outcome of the event (e.g. hospitalisation, death); seriousness is not based on severity; this sentence should be corrected.

This classification is based on WHO guidelines. The sentence was altered to “Based on the outcome of the event, the World Health Organization (WHO) has classified AEFIs as Serious and Non-serious which is used for regulatory classification”. This can be seen in lines 68 to 70.

Vaccine hesitancy is a public health issue; I would suggest replacing the term “concerns” with vaccine hesitancy:

The term “concerns” was replaced with “vaccine hesitancy” shown in line 75.

There should be an explanation on why surveillance on immunization is essential; I.e.: to build trust; to reassure the public that AEFIs are being monitored and actions to reduce risks are being taken

An explanation to why surveillance on immunization safety is crucial has been added to the sentence. This can be seen in line 76 to line 85

AEFI to AEFIs

AEFI was changed to AEFIs in line 86.

The term routine is preferable to passive

The term passive surveillance was changed to routine surveillance in line 87.

HICs also rely on routine systems; under reporting is a general limitation of passive/routine systems; but is there any other reason that LMICs have additional underreporting?

Poor pharmacovigilance infrastructure was stated as another reason that LMICs have additional underreporting in line 88.

Perhaps rephrase to emphasize the link: underreporting was addressed via the GVSI. One of the targets of… was set.

The sentence was rephrased as “To address the issue of underreporting the WHO therefore instituted the Global Vaccine Safety Initiative, which set out indicators primarily for monitoring case reporting” for emphasis as seen in lines 88 to 90.

METHODS

Is there a reference for CDC guidelines for evaluating?

The reference for the CDC guideline has been added to line 136.

How were these guidelines adapted? What were the quantitative and qualitative data that was collected?

How the CDC guidelines was adapted has been explained in lines 136 to 138.

Study area paragraph: The evaluation was made through interviewing stakeholders in this region? And reviewing reports in this district? Perhaps clarify what resources in this area was used to do the evaluation.

The resources in the area used to do the evaluation has been explained in lines 122 to 124.

I don’t think the term detect can be applied here; perhaps just state that caregivers/ vaccinees report to health facilities. How do they report? Using a reporting form?

The phrase caregivers/vaccinees detect was changed caregivers/vaccinees report adverse events to health facilities. The health facilities record reported events using a standard case reporting form and submits the report to the District Health Directorate. This has been stated in lines 306 to 308

Line 126: What is meant by manage reports? They receive and code them? And they transfer the reports?

How reports are managed has been explained in lines 308 to 312.

Line 153: title says data collection and analysis, but no analysis was described: perhaps expand on the analysis or remove analysis from title.

The title “data collection and analysis,” has been changed to “data collection tools and methods”. Nonetheless, Table 1 shows how data collected on attributes was analysed.

Also please make data sources clearer: by stating five different sources of data was collected data included: a) interviews .. b) case-based forms; c) IDSR reports; d) DHIMS2; e) TAC.VBP reports

From lines 232 to 235 the sources of data were listed as suggested.

Explanation of what TAC-VBP report is?

Line 165: explain what is meant by hypothesis generation; is this signal detection?;

TAC-VBP reports explained as Technical Advisory Committee for Vaccines and Biological Products.

RESULTS

Prefer to use the word understood rather than knew

The word knew has been changed to understood in line 282.

Section on performance of the surveillance system in meeting objectives: objective were; 1) detect, correct and prevent immunization error-related AEFIs; 2)

identify problems with vaccine lots or brands leading to vaccine reactions, 3) prevent false

blame arising from coincidental adverse events, 4) maintain confidence by properly responding

to community concerns, 5) generate new hypotheses about vaccine reactions and estimate rates

of occurrence of AEFIs.

Use subtitles to indicate the results of assessment of each of these objectives;

For the section on performance of the surveillance system in meeting objectives, all findings have been grouped under the subtitles which are the 5 objectives of the system. This can be seen in lines 336 to 362.

Would it be better to say 463/476 (97%) of all serious cases were investigated?

This has been corrected in line 347.

Suggest to rephrase replace the word exonerate; By investigating deaths that followed vaccination and performing causality assessments the system has been able to prevent misinformation speculating that the vaccine attributed to the deaths? Also link this to the objective of prevent false blame arising from coincidental adverse events

the phrase the system has been able to exonerate EPI has been revised to the system has been able to prevent misinformation speculating that the vaccine attributed to the death. Seen in line 351.

I would replace the word victim; perhaps individual;

The word victim was replaced with individual in line 357

Who paid for the management of the AEFIs?

Management of the AEFIs is paid for by the EPI, stated in line 357

Does this refer to signal (hypothesis generated)? This needs to be made more clearer

Hypothesis generation here means to assign possible causes to the reaction.

how was the usefulness assessed? Was this through interviews? Where did this information come from?

How usefulness was assessed and the sources of information for this assessment has been explained from lines 364 to 368.

Please do not use the word victim; an individual is suspected to have AEFI; causality is not determined; the word victim gives a negative connotation, when most of the time benefits of the vaccine outweigh the risk

The word victim was changed to individual in line

In Ghana is this the case, do caregivers have to go back to the facility to report? Or are there tools in place to allow patient/caregiver reporting?

As at the time this evaluation was done in January 2019, caregivers have to go back to the facility to report AEFI. Later in 2019, a MedSafety mobile application was developed. This is still being piloted in-country and its implementation is not countrywide.

Does Whatsapp platforms compromise the completeness of reports?

To prevent issues of incomplete reports, all reports passed through WhatsApp platforms must be accompanied by a case-based form. Nonetheless this gives room for human negligence which can still lead to incomplete report. This was explained in lines 556 and 557

Another limitation is recall bias in interviews; as study was conducted in 2019, investigating from year 2014.

Study limitation was added in lines 576 to 581.

Proper introduction to and explanation (and referencing) of the tool and how this was applied (e.g. stakeholder interviews and record reviews with reference to the table). I sense that there are both quantitative and qualitative elements in the tool.

An introduction to and explanation (and referencing) of the tool and how this was applied has been included in lines 135 to 138.

Certain attributes are assessed (presumably part of the evaluation tool) but these are not explicitly related to the objectives of the AEFI surveillance system.

Though some attributes are not directly related to the study objectives, they were assessed to understand operations of the system and to explain or discuss findings seen in the attributes of the surveillance system. For instance, knowledge was assessed to understand its relation to underreporting or incomplete reporting,

It may be useful to define the outcome terms in the text when you report the results e.g. PPV of the system detecting the number of vaccine-related AEFI etc.

Explanations to the outcome terms are in the CDC guidelines referenced in this manuscript. However, PVP has been explained in line 461 as its application is different from that of infectious disease surveillance systems.

Are these results presented? Did you limit the study to AEFI related to EPI only?

AEFI surveillance in Ghana is limited to EPI. However there were recommendations to the EPI after this evaluation for integration of other units that use vaccinations like the Port Health (for Yellow fever vaccination) and Maternal health Unit (for tetanus vaccination)

General Comments:

Generally, Objectives and measurables need to be made clear; difficult to connect methodology to results: layout needs revision.

Correction

The objective of the study was to assess the attributes of Ghana’s AEFI surveillance system, its usefulness and to determine whether the system is meeting its objectives. This was stated in the last sentence of the introduction. However, to connect these objectives to the methods and results, sub-headings (for each objective) have been introduced in the method and results section to improve consistency.

Also, the layout of the report has been revised for clarity. The operations of the system has been moved to the results section. For the assessment of system meeting its objectives, all findings have been grouped under the subtitles which are the 5 objectives of the system.

Additional requirements:

1.) Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response

The format of the manuscript is revised according to the PLOSE ONE style

Thank you for your comments and inputs

Sincerely,

Joseph Asamoah Frimpong

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

22 Dec 2021

7 Feb 2022

The AEFI case definitions, have been put under data collection within the Methods Section. This can be seen in lines 132 to 142.

Submitted filename: Response to Reviewers.pdf

Click here for additional data file.

16 Feb 2022

Evaluation of the Adverse Events Following Immunization Surveillance System, Ghana, 2019

PONE-D-21-21870R2

Dear Dr. Frimpong,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Emma K. Kalk

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

21 Feb 2022

PONE-D-21-21870R2

Evaluation of the Adverse Events Following Immunization Surveillance System, Ghana, 2019.

Dear Dr. Frimpong:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Emma K. Kalk

Academic Editor

PLOS ONE

Table 1

Assessment of attributes of the AEFI surveillance system.

Attributes	Data collection	Data analysis
Simplicity	Interviews	Directed content analysis to summarize health workers’ knowledge on case definition, ease of; confirming a case, completing the case-based forms, flow of information through the levels of the system and case investigation.
Acceptability	Interviews	Directed content analysis to describe evidence on stakeholders’ participation in the surveillance system at all levels
	Records review
Sensitivity	Records review	Summary descriptive statistics to determine proportion of regions/districts (or the years) that achieved the GVAP case detection target of 10 AEFI/100,000 surviving infants/year
Positive Predictive Value	Records review	The number of AEFIs related to the vaccine or the vaccination process out of all the serious AEFIs for which causality was assessed.
		*PPV=NumberofAEFIsvaccine−relatedNumberofAEFIsthatcausalitywasassessed×100
Timeliness	Records review	Summary descriptive statistics to determine proportion of AEFIs that are reported to the next level on time and feedbacks received on time as well as turnaround time for AEFI investigation.
	Interviews
Stability	Interviews	Directed content analysis to summarize availability and sustainability of funds, logistics and human resources needed to run the system as well as level of integration of the system into the IDSR.
Data quality	Records review	Summary descriptive statistics to determine proportion of case-based forms, that are completely filled. Directed content analysis to describe data validation.
Representativeness	Records review	Directed content analysis to describe reported events by age, sex, time of report, and place (sub-district/district/region) of reporting
Flexibility	Records review	Directed content analysis to summarize how the system had adapted to changing information needs and operation conditions with little increase in cost, time and funds.
	Interviews

Table 2

Annual performance on AEFI reporting in Ghana, 2014–2018.

Year	AEFI cases	#Surviving infants	AEFIs/100,000 surviving infants/year
2014	10	1024577	0.98
2015	20	1048147	1.91
2016	41	1071855	3.83
2017	391	1095677	35.69
2018	688	1119618	61.45

Table 3

Regional performance on AEFI reporting in Ghana, 2018.

Region	AEFI cases	Total pop. (million)	#Surviving infants	AEFI/100,000 surviving infants/year
Ashanti	49	5.66	215808	22
Brong Ahafo	47	2.78	105640	44
Central	128	2.52	95760	13
Eastern	34	3.17	120460	28
Northern	75	2.99	113620	66
Greater Accra	169	4.83	183540	92
Upper West	23	0.82	31160	7
Upper East	80	1.24	47120	16
Volta	69	2.54	96520	7
Western	15	3.02	114760	13

Table 4

Annual performance on AEFI reporting in Ga East Municipality, 2014–2018.

Year	AEFI cases	#Surviving infants/year	AEFI/10,000 surviving infants/year
2014	3	6610	4.54
2015	0	6571	0.00
2016	0	6688	0.00
2017	4	6858	5.83
2018	9	7010	12.83