Evaluating the functionality and effectiveness of ZITAG and related expanded programme on immunisation technical committees in Zambia.

The Zambia Immunisation Technical Advisory Group (ZITAG) was established in 2016 as an advisory body to provide evidence-based recommendations on vaccine policy. As part of the Gave Full Country Evaluation, we evaluated the functionality and effectiveness of ZITAG and related EPI committees through an online stakeholder survey of Interagnecy Coordinating Committee (ICC), ZITAG and Extended programme on Immunisation Technical Working Group (EPI-TWG) members, document review and key informant interviews. The survey was sent out via SurveyMonkey between May and July 2020 to 69 members of ZITAG, ICC or the EPI TWG. A total of 52 individuals responded (75%). Eight key informant interviews were also carried out at the national and global level in September 2020 to elaborate further on some of the quantitative findings and for triangulation. Findingsrevealed that the EPI committees were reasonably functional and effective entities, each with its unique role, though some overlaps occurred. Functionality was shown by having a broad membership with wide expertise and long-serving members; sub-committees existed and meetings were occurring regularly. Leadership and coordination structures also existed and were largely felt to be working well. Funding challenges however persisted, in particular for ZITAG operations and functionality of its subcommittees. Effectiveness and value addition fo the committees to the EPI was illustrated through decision-making processes and evidence use as well as relatively good country ownership in terms of commitment, legitimacy, capacity and accountability. Full independence and ownership may however be compromised by funding challenges. Recent changes to ICC terms of reference and focus beyond immunisation side-lined the EPI and weakened the linkage between ICC and ZITAG with many ZITAG recommendations not having been followed through by ICC as the ultimate endorsing entity.

Expanded Programme on Immunisation

Expanded Programme on Immunisation Technical Working Group

Expanded Programme on Immunisation Costing

Full Country Evaluation

Inter-Agency Coordinating Committee

Key Informant Interviews

Ministry of Health

Permanent Secretary

Reproductive, Maternal, New-born, Child, Adolescent Health & Nutrition

Strategic Advisory Group of Experts

Standard Operating Procedure

Terms of Reference

Technical Working Group

World Health Organisation

United Nations International Children’s Emergency Fund

Zambia Immunisation Technical Advisory Group

Background

Zambia has been receiving significant support from Gavi since 2001 towards new vaccine introductions (NUVI), immunisation services support (ISS), and health system strengthening (HSS). Since 2013, several new vaccines have been introduced and from 2018, Zambia started the implementation of the Gavi-funded Health Systems Strengthening grant. Gavi contribution to the Zambian immunisation programme over the period 2001 to 2019 amounted to approximately $164,000,000 [1].

Taking advantage of various Gavi support streams, the national immunisation programme in Zambia has expanded the profile of the programme, introducing several life-saving vaccines and implementing other initiatives to strengthen the country’s immunisation programme. National immunisation coverage has increased during the period 2013 – 2019 [2]. Importantly, childhood mortality from vaccine-preventable diseases have also declined [3]. However, progress remains uneven across the country and policy questions about sustainability have emerged [2].

The WHO and its Strategic Advisory Group of Experts (WHO-SAGE 2017) recommended that countries establish National Immunisation Technical Advisory Groups (NITAGs) in order to advise policymakers on immunisation related matters, including new vaccine introductions [4]. This resulted in endorsement by all countries at the 65th World Health Assembly of the 2012 Global Vaccine Action Plan (GVAP) objective of “all countries having a functional NITAG by 2020” as a means to improve quality and ownership of national immunisation programmes [4]. The GVAP Monitoring and Accountability Framework has the existence of a NITAG as a key element and this is aimed at assisting countries with the formulation of national immunization policies and strategies and advising on technical issues related to national immunization programmes [5]. Consequently, the ZITAG was established in Zambia in 2016 as an advisory body to provide evidence-based recommendations on vaccine policy [6]. Part of ZITAG mandate was to provide evidence on the programmatic and economic implications of vaccine introductions. ZITAG was thus integrated into the national health system as an advisory body established to provide independent and evidence-based advice to MOH by offering a credible, transparent and independent decision-making process regarding vaccination and to propose recommendations in order to strengthen the national immunisation programme [6].

The other related committees discussed in this paper are the Expanded Programme on Immunisation Technical Working Group (EPI-TWG) and the Interagency Coordination Committee for Reproductive, Maternal, Neonatal, Child and Adolescent Health and Nutrition (ICC RMNCAH&N or simply ICC). EPI-TWG was established to have advisory, coordination and monitoring functions on matters related to EPI by providing technical guidance for priority setting and for deployment of resources immunisation activities [7]. The ICC was formed to advise the government and other private sector/NGOs implementing agencies on issues related to RMNCAH&N, particularly around resource-mobilisation, advocacy for sustained political commitment, and to monitor and evaluate RMNCAH&N activities [8]. Table 1 provides a comprehensive summary and comparison of these three EPI committees.

Table 1

Organisation of the EPI Committees in Zambia.

Element	ZITAG(Zambia Immunisation Technical Advisory Group)	ICC RMNCAHN(Interagency Coordinating Committee for Reproductive, Maternal Neonatal, Child, Adolescent Health and Nutrition)	EPI TWG(Expanded Programme On Immunisation Technical Working Group)
Formation	2016	1999	Exact year not available but longest standing of the three committees
Main role	Provide independent and evidence-based advice to MOH by offering a credible, transparent and independent decision-making process regarding vaccination and propose recommendations in order to strengthen the immunisation programme	Advise the government and other private sector/NGOs implementing agencies on issues related to maternal health, child health and nutrition, particularly around resource-mobilisation, advocacy for sustained political commitment, and to monitor and evaluate RMNCHN activities	Advisory, coordination and monitoring functions on matters related to EPI by providing technical guidance for priority setting and deployment of resources
Membership	Members serve in personal capacity and selected based on broad range of expertise required	Agencies, partners and the key MOH policy staff, technical units	Various partners supporting immunization activities
Meeting frequency	Biannual	Quarterly	Monthly
Funding	MOH and cooperating partners	Mainly MOH	MOH and Cooperating partners
Chair	One of the members, appointed by the Permanent Secretary of the Ministry of Health	Permanent Secretary at the Ministry of Health	Child Health Specialist at Ministry of Health
Length of membership	Three years, renewable	Indefinite	Indefinite
Main areas of expertise of members	Mainly public health, child health, health systems and delivery, research, and vaccinology	Mainly public health, health systems and delivery child health, evaluation, and partnerships	Mainly public health, child health, health systems and delivery, and evaluation
Subcommittees	• National Certification Committee for Polio Eradication (NCC), • National Polio Expert Committee (NPEC), • National Task Force (NTF), • National Epidemics Committee	None	• Service Delivery, • Advocacy, Communication and Social Mobilisation, Logistics & Cold Chain, • Monitoring and Evaluation
Coordination/Secretariat	Child Health Unit at Ministry of Health	Director of Public Health at Ministry of Health	Child Health Unit at Ministry of Health
Conflict of interest procedures	In place and followed	None	None

Gavi requires that NITAG recommendation be submitted with some applications for funding. So far, ZITAG has been involved in decisions around HPV vaccine national introduction, replacement of PCV 10 with PCV 13 and the introduction of Covid-19 vaccines. The Gavi FCE report of 2017 lauded the country for establishing ZITAG but recommended making it functional in order to, amongst other things, aid sustainability of the EPI [2]. However, the functionality and effectiveness of the ZITAG in relation to other related committees such as the EPI-TWG and the ICC was not well-established, including as far as it relates to the sustainability of EPI. For example, there were questions on independence, coordination, leadership, and capacity in these technical committees.

The overall goal of this evaluation, which was a subcomponent of the Gavi Full Country Evaluation (Gavi FCE) for 2020–2021, was to identify opportunities and obstacles to sustainability, financing, and programmatic performance of the Zambian immunisation programme. In this paper, we focus on one of the four evaluation questions of our work, which was: “What is the functionality and effectiveness of ZITAG and related committees that have been set up to support decision-making processes around new vaccine introduction from a financial and programmatic sustainability perspective?”.

Methods, data collection and analysis

We developed a framework for evaluating the functionality and effectiveness of the committees based on literature review. The framework is presented in Table 2, detailing elements assessed for functionality and those for effectiveness. We defined functionality based on the six indicators in the WHO/UNICEF Joint reporting form with addition of others based on further literature review [5], [9], [10], [11], [12], [13]. Elements of effectiveness were also derived from literature review, including country ownership [11], [14].

Table 2

Elements of functionality and effectiveness of ZITAG and related technical committees.

Functionality

Effectiveness

1. Composition (skills mix and size) 2. Leadership (who is the chair and level of influence) 3. Existence of ToRs 4. Clarity of ToRs 5. Coordination (who coordinates and how) 6. Independence (who funds them and evidence generation) 7. Adequacy of funding 8. Frequency of meetings 9. Availability of meeting minutes (evidence of meetings held) 10. Existence and work arrangements of sub-committees

1. Interactivity outside formal meetings 2. Influence of decision making (adoption of suggested recommendations) 3. Adequacy of time to review and make recommendations 4. Independence (decision making processes within ZITAG) 5. Added value of ZITAG (efficiency, timeliness, responsiveness, avoidance of duplication) 6. Country ownership of ZITAG (autonomy, participation in decision making among members, transparency among members, independence from Gavi, good governance and legitimacy)

The framework used to assess ownership looked at several subcomponents which included power and legitimacy (country governments have the power and legitimacy (right) to set priorities and make decisions that are respected by the donors), commitment and responsibility (political stakeholders commit to take responsibility for aid funded programs that address an identified need), capacity (the capacity to sustain initiatives and programs) and accountability (recipients and donors are accountable to each other and to their citizens for programs, systems, and strategies) [14].

Based on these elements of functionality and effectiveness a questionnaire was developed and a survey was sent out via SurveyMonkey between May and July 2020 to 69 members of ZITAG, ICC or the EPI-TWG using a list that was provided by Child Health Unit (CHU) of the Ministry of Health, which is the coordinating unit for all child health programmes in the Ministry of Health, including immunisation. A total of 52 individuals responded to the survey (75% response rate). The respondents comprised of 22 members of ZITAG, 25 members of EPI-TWG and 33 members of ICC. Note that membership to ZITAG, ICC and EPI-TWG were non-exclusive and thus one could be a member of all three. The analysis showed that those with such overlapping membership were nine and were from CHU, World Health Organisation (WHO) and the United Nations International Children’s Emergency Fund (UNICEF), who were key players in the EPI programme and cut across all three organs. Most of the non-respondents were members of the ICC (which is the largest of the three committees) who indicated that they had not been regularly active over the recent years or had moved onto other roles or other organisations.

In addition to the survey, eight (8) key informant interviews (KIIs) were also carried out at the national level in September 2020 with some key members of ZITAG, EPI-TWG and ICC in order to elaborate further on some of the quantitative findings that needed follow up and in-depth interrogation. This was also used to triangulate the findings of the survey.

Analysis was carried out using Stata and simple descriptive statistics were utilised. A substantial number of questions utilised a five-point Likert scale ranging from strongly disagree to strongly agree (strongly disagree, disagree, neither agree nor disagree, agree and strongly agree). In the final data output a weighted score was utilised with strongly disagree being 1 and strongly agree being 5. Thus, these Likert scale findings are reported on a scale of one to five with five being the best and most desirable score.

Findings and discussion

The findings and discussion are in two sections, the first on functionality and the second on effectiveness of the EPI committees. Each subheading provides a brief summary of the finding, followed by a detailed elaboration.

Functionality of EPI committees

Membership of committees: Membership to the committees (ZITAG, ICC & EPI-TWG) was drawn from a diverse group of stakeholders, many of whom were long-servicing members with a broad mix of skills.

Composition:Fig. 1 shows organisational affiliation of respondents and government was the major contributor of members. The “other” category consisted of statutory or regulatory agencies. This shows the diverse representation of different groups on the committees. However, a number of respondents indicated that some other organisations could be considered for inclusion (Annex 1), which included additional academic institutions, government agencies, professional associations, regulatory agencies and international partners that were not already members. While it is neither practical nor reasonable to include all the suggested organisations, it is worth scrutinizing which ones would add the most value to the work of the EPI programme and including them in future.

Fig. 1

Organisational affiliation of respondents.

Length of membership: ZITAG has a three-year membership period, which is renewable. Most of the 22 surveyed members had been members for two, three or four years (27.3% each). A further 9.1% had been members for less than a year and five years, respectively. In terms of membership to EPI-TWG and ICC, 6–10 years was commonest duration of membership followed by 3–5 years as illustrated in the Fig. 2. ZITAG had the shortest duration of membership since it was only formed in 2016. It is interesting to note that many members on EPI-TWG and ICC had been there for a substantial period. This illustrates the commitment of members and is likely invaluable in preserving organisational memory as well as in being good for organisational learning and capacity in the long run.

Fig. 2

Length of membership to ICC and EPI-TWG.

Member skills mix: Areas of expertise of individuals on all the three committees were wide-ranging as illustrated in Fig. 3 which provides the expertise by committee. The most common areas of expertise across all three committees were public health (66.0%), child health (51.2%), health systems and delivery (46.8%) and research (36.2%). It should be noted that members were multi-skilled, and thus, these categories were not mutually exclusive.

Fig. 3

Expertise of members in committees.

Having “at least five areas of expertise represented among its membership” is considered essential to the functionality of a National Immunisation Technical Advisory Group (NITAG) [13]. An analysis of ZITAG members only revealed that the most common areas of expertise were still public health (57.1%), child health (57.1%), health systems and delivery (47.6%), research (42.9%) and vaccinology (38%). Other expertise were also represented, showing diversity and meeting of this minimum requirement for functionality. Incorporating a broad set of skills is crucial for ensuring that these organs are functional and can bring a well-balanced and broader set of perspectives on major policy decisions.

According to WHO guidance with regard to structural viability “NITAGs should have core- and non-core members, with distinct roles, at least five expertise areas represented, with potentially external experts co-opted in working groups.” [9] This is fulfilled in Zambia as the membership covered more than five areas of expertise, and there were core and non-core members who were included in meetings as need arose, and both groups were captured in the 22 members who responded to the survey. However, one recommended area that was missing was expertise in adult/geriatric medicine. Economic evaluation, which seemed to have been a gap as noted in previous FCE reports, was catered for with two members indicating expertise in this area, although the development of clear policy and guidelines for the economic evaluation of new vaccines had not been done yet as was recommended [2]. The two experts in Economics could advise on matters related to economics and financial sustainability. It must be noted, however, that no formal local studies have been done to date on cost-effectiveness or budget impact analysis to do with the introduction of new vaccines and thus external data is still relied upon.

Having diverse membership in terms of expertise, length of membership and organisational background is clearly a positive factor when it comes to promoting sustainability. It shows longstanding commitment on the part of the members and their organisations and is a plus for institutional memory. It can also imply that they are more likely to feel accountable for the decisions made and the outcomes and impact of such decisions.

Conflict of interest procedures: Conflict of interest declaration and confidentiality agreements for ZITAG members as well as keeping of appropriate records was being done but can be improved upon.

Fourteen of the 22 members of ZITAG (63.6%) indicated that they had been asked to declare any conflict of interest at the time of joining ZITAG. It is a requirement that all members declare any conflict of interest prior to confirmation of the appointment by MOH and that members sign a confidentiality agreement form [6]. Further, the Ministry is supposed to maintain a register of members’ interests and signed confidentiality agreements. Key informant interviews indicated that this is being done and that members actually sign a conflict-of-interest declaration at each deliberation, given that meeting composition may vary from time to time depending on the topic being discussed.

This is important as ZITAG deliberations are confidential and may not always be publicly disclosed and further, there is need to ensure that members do not have any conflict of interest such as links to pharmaceutical companies, vaccine manufacturers or distributors or anyone who may benefit financially from a ZITAG vaccine recommendation [6]. The nature of the decisions tabled before ZITAG have huge implications for affecting costs/budgets and thus the sustainability of EPI in Zambia. It is therefore important that members act in the best interests of the country and not external parties’ interests. Thus, it is normal practice in many countries that a NITAG member with a conflict of interest related to a matter being deliberated on must leave the room and cannot vote on issues related to the specific matter [15]. ZITAG terms of reference (TORs) indicate that membership may be terminated if there is a change in affiliation or other changes resulting in a conflict of interest, as well as lack of professionalism, involving, for example, breach of confidentiality [6].

However, it must be noted that public disclosure of deliberations of NITAGs and communicating to the public as well as health care professionals have been recommended as a way of promoting transparency alongside a decision-framework [16]. This currently does not happen in Zambia and is something the country should consider in the future.

Awareness of functions of own and other committees: While all the members of three committees were mostly aware of their own functions, ZITAG and ICC members were less aware about the functions of one another’s committees.

Members of the committees were asked if they have copies of various documents such as terms of reference, membership lists, contact information for members and minutes of meetings. For ZITAG members, the overall average score for having all such documentation was 72.4%. For EPI-TWG the average score was 71.0% and for ICC it was 57.6%. Documentation can thus generally be improved upon across all three organs and in particular for ICC where it was weakest. It was not clear what the challenges were here but KIIs indicated that this could partly be linked to the secretariats for the three organs, that of ZITAG and EPI-TWG being CHU and that for ICC having been recently changed from CHU to the office of the Director Public Health at the Ministry of Health. ICC also had wider and more fluid membership, which may have made it harder to coordinate and provide the necessary documentation to all members.

In terms of awareness of the functions of the various committees, most members were aware of who chairs their committee, the TORs, its coordination, funding, full membership, and functions. The average score across committees for such awareness by members was 90.5% for ZITAG, 92% for EPI-TWG and 89.2% for ICC. Of concern, 27% of both ICC and ZITAG members, respectively, were unaware of the functions of the other committee. Nine percent (9%) of ZITAG members were unaware of the functions of EPI-TWG whereas 15% of ICC members were unaware of EPI-TWG functions. Twelve percent of EPI-TWG members were unaware of the functions of ZITAG and 8% were unaware of the functions of ICC. However, it must be remembered that some members were cross-cutting across the three committees and thus this may influence levels of awareness.

This situation needs to be remedied as it is important that EPI committee members have a full understanding of their own TORs as well as those of other related organs in order to aid efficiency and avoid duplication and overstepping of boundaries. Further, in view of limited time and heavy workloads for most members, there is a need to optimally make use of them in their relevant areas. From a sustainability perspective, this would aid efficiency, effectiveness, respect for one another’s roles, and an appreciation of the importance of the recommendations and decisions being made at the various levels. A case in point, illustrated later in this section, is the lack of consideration and deliberation by ICC on recommendations made by ZITAG in recent years, which hinders the effectiveness of the latter.

Leadership and coordination: Generally, there was clear and good coordination and effective leadership structures for ZITAG, ICC and EPI-TWG, but ZITAG lacking independent secretariat for coordination hampers its functionality.

Most aspects of leadership and coordination of ZITAG, EPI-TWG and ICC were rated well by members. Out of a possible total score of five, ZITAG and EPI-TWG both scored 4.4 whereas ICC scored 4.2. The details of the various elements are shown in Fig. 4. Most members felt the coordination structures and leadership structures were clear and that the leadership was effective. The TORs were also felt to have been clear. This shows the functionality of these organs in supporting decision-making and providing leadership to the EPI programme, though there were some challenges as noted in other sections of this report. As far as the sustainability of EPI is concerned, having these strong leadership and coordination structures was a reassuring finding.

Fig. 4

Leadership and coordination of EPI committees.

It must be noted, however, based on KII findings, that the intention from design was to have an independent secretariat for ZITAG with dedicated staff who would work closely with the chair to organise ZITAG meetings, prepare reports, finalise recommendations, prepare annual plans, and coordinate the technical working groups [6]. This had not occurred due mainly to a lack of funds to establish such a secretariat as well as allow ZITAG to have its own offices. Key informants were not hopeful about this situation changing soon due to lack of funding and also the perceived lack of will to establish this secretariat. These factors bring into question the full independence of ZITAG, particularly as it relates to avoiding potential conflicts of interest. The Global Network of NITAGS (GNN) notes the need for NITAGs to be independent if they are to execute their functions well, but that many countries are far from being at this level [15].

Existence and functionality of sub-committees of ZITAG and EPI-TWG: EPI-TWG subcommittees were highly active and contributed immensely to the work of both ICC and ZITAG, but ZITAG subcommittees were hampered by lack of an independent secretariat to coordinate meetings and were thus less functional and effective.

Half or ZITAG members were not part of any committee, and six were from the secretariat (27.3%). Two were members of the National Certification Committee for Polio Eradication, two on the National Polio Expert Committee and three on the National Task Force. The ‘’other’’ category consisted of Hepatitis, Rotavirus and liaison but these were not official ZITAG committees according to the TORs [6]. Given this scenario, it was doubtful that these subcommittees were functioning as they were intended to since membership was minimal. This was attributed to the lack of dedicated funding for ZITAG as well as the independent secretariat not being formed yet. Most times, ZITAG meetings were called to address specific matters relating to new vaccine introductions, mainly driven by Gavi’s push, and thus these subcommittees did not meet as required. Key informant interviews confirmed these findings:

“I think they [ZITAG subcommittees] can work better, but they are working well enough. We have two in-person meetings in a year scheduled for them and in between we ask that they meet remotely. If we had a better, well-disposed secretariat with a bit more time than the current team that are overloaded, we are the ones who should be coordinating these remote operations for them. Now we are at their mercy, you know one person remembering that they need to meet and discuss. So, we actually see that they tend to do more at those in-person meetings as opposed to subcommittees. Cos ideally the first meeting of ZITAG is when they meet to do the planning for the year, the second meeting is a subcommittee meeting and the third meeting is also a subcommittee meeting to conclude and prepare to present to the final meeting of ZITAG where everyone presents what their discussions were in the subcommittees. But I think the planning meeting goes well because three days is often enough for them to plan, the last meeting goes well because it is long enough to hear the arguments of the various subcommittees. It is the operations of the subcommittees that are a bit restricted even if they are working remotely.”

Meeting minutes for ZITAG also pointed to irregularity of meetings and sometimes rushed meetings as multiple issues were deliberated on in a short space of time, brining into question the level of interrogation and discussion. Often these meetings were ad-hoc and called to approve urgent Gavi applications [17], [18], [19], [20], [21], [22], [23], [24], [25]. It was thus pointed out that a dedicated secretariat would be helpful in ensuring ZITAG members, who were quite busy in their day-to-day work, were reminded of meetings and prompted to meet as subcommittees regularly.

Most EPI-TWG members were part of the service delivery (60%) or M&E (44%) sub-committees. Logistics and cold chain each accounted for 16% of members. Twelve percent were not members of any sub-committee. Two individuals indicated being members of the “other groups” which they indicated as new vaccine introduction sub-committee, although this was not a defined sub-committee of the EPI-TWG according to the TORs.

EPI-TWG sub-committees were highly active, as evidenced by meeting minutes and KIIs. They played a key role in all aspects of the EPI programme and were critical in providing required evidence, reports and documentation to both ZITAG and ICC. As noted in FCE reports and confirmed by KIIs, they depended on the same very committed individuals to drive many EPI activities and thus faced a huge workload in delivering on EPI workplans [2]. This sometimes affected efficiency and timely implementation due to having conflicting demands, especially in the current scenario of many vaccines having been brought on board and more still planned for introduction.

Funding for ZITAG, ICC and EPI-TWG: Minimal funding required for operations of ICC and EPI-TWG, but ZITAG requires more funding to function as planned, which is currently lacking, making it dependent on EPI partners for its operations.

In relation to the adequacy of funding and independence of the committees, opinions were divided but average scores were as detailed in the Fig. 5. Although ZITAG scored highest on independence it scored lowest on adequacy of funding. Other than affecting the establishment of an independent office and secretariat, some felt it was also affecting applications for membership to ZITAG as some people may shun volunteering for such bodies due to their costs not being covered at a minimum:

Fig. 5

Funding and independence of EPI committees.

“We ask people to come for meetings and we are at the mercy of these people because we cannot reimburse their time and other resources they are putting in. Yes, even if you ask people to volunteer…you cannot ask someone to volunteer and then they drive to the ZITAG meeting using their own fuel and their time. Yes, let them volunteer but take care of the means.”

Generally, ZITAG required more funding to operate than EPI-TWG and the ICC. A big part of this is the need to cater for the costs of meetings for ZITAG as the TORs indicate that ZITAG meetings should be held in a neutral place for the sake of independence. Until a secretariat and offices are established for ZITAG this is likely to continue to be a challenge. This situation is not unique to Zambia as a 2019 study found that unreliable funding was one of the major challenges faced by countries in operationalising NITAGs [11]. This is further complicated by the current lack of both global and country systems to support NITAGs generally [11].

ZITAG depended a lot on support from cooperating partners such as WHO, UNICEF and CIDRZ for its operations in terms of having meetings. A single ZITAG meeting could cost over $6,000 to organise, according to budgets availed. In terms of local currency units, this was a significant amount. For example, the $6000 in 2019 was ZMW 65,000 but was ZMW 120,000 in 2020.

“ZITAG, where there are technical people, we are paying sitting allowances, the conference room that they are going to, we have people who are travelling from different provinces to come here, so its DSA [daily subsistence allowance], it costs more money to bring people together for these meetings. For a TWG, once in a while we buy tea, but usually not, so it is not as costly as ZITAG meetings.”

Regularity and frequency of meetings: Regular meetings, with mostly timely sharing of agenda and minutes, were held for ZITAG, EPI-TWG and ICC, but ZITAG meeting frequency was less ideal than the others. Further, recent changes to ICC TORs have meant immunisation has become less prominent and not given adequate time at such meetings.

Overall, members agreed or strongly agreed on the adequacy of time for ZITAG meetings, that the agenda and minutes were circulated at least a week before meetings, that minutes for meetings were available and that there was some interaction by members outside formal meetings. However, about a third felt the meeting frequency was inadequate. ZITAG meetings were scheduled twice a year at a neutral venue, with ad-hoc meetings called, as necessary. These findings suggest that generally, members of ZITAG, ICC and EPI-TWG were informed about decisions, plans, evaluations, and other matters pertaining to the immunisation programmes. Further, regular meetings ensured that members had opportunities to make their contributions to foster a sustainable immunisation programme. ZITAG TORs indicated three criteria for removal from membership, and one of them was the failure to attend two consecutive meetings. KIIs indicated that this was enforced, as recently three members were removed from membership and replaced for not being available for meetings.

Ordinary meetings of the ICC were planned to be held on a quarterly basis, but extra-ordinary meetings could be called whenever the need arose to address urgent matters and the Mother, and Child Health Units were responsible for the coordination of these meetings. ICC meetings were generally well-rated [6].

Effectiveness of EPI committees

Independence and Country Ownership: Country ownership was perceived to be strong for all EPI partners in terms of local power and legitimacy, local commitment and responsibility, accountability and local capacity, but funding challenges continue to be a potential stumbling block and barrier to full independence.

In terms of country ownership, most of members felt that the committees had local power and legitimacy, had local commitment and responsibility, and had adequate accountability and adequate local capacity. The biggest challenge to country ownership and thus sustainability was funding for the EPI programme and for organs such as ZITAG, in particular. Adequately funding these bodies was one element of ownership that needed strengthening so that they could be well-established and function independently and sustainably [13].

It was noted earlier that some partners cut across all three committees and these were mainly funders who supported the functioning of these committees. Although their roles on ZITAG may not be decision-making roles but simply facilitators of the functioning of ZITAG, it is possible that they may have had an influence on the members in terms of financing, calling of meetings and provision of necessary documentation and evidence. This brings into question the full independence of ZITAG versus a scenario in which it would be fully self-sufficient and coordinated independently as designed.

Effectiveness of roles and linkages between ZITAG, ICC and EPI TWG: EPI-TWG was a strong backbone for EPI in Zambia and had strong linkages to both ZITAG and ICC, but ZITAG and ICC linkages were weaker and recent broadening of the mandate of ICC hampered the recommendations of ZITAG as well as the focus on immunisation in general.

Participants were asked to rate how well they understood the linkages between the three immunisation organs and how they worked together, on a scale of 0 to 10 with “0″ being do not understand at all and ”10″ being very well understood. The mean score for this was 7.6 (SD 2.7), indicating that while most have a good idea, there are still some gaps in understanding the roles of the various organs.

On the same rating scale as above, regarding how well the ICC and ZITAG work together in Zambia, the mean score was 6.5 (SD 2.8) out of a possible 10, and how well the ICC and EPI-TWG work together was 7.3 (SD 2.7). Lastly, in terms of how well the EPI-TWG and ZITAG work together, the mean score was 7.0 (SD 2.3). This illustrates that the EPI-TWG has better working relationships with both ICC and ZITAG whereas the linkages between ICC and ZITAG were weaker. The fact that nearly a third of members in each of these two organs did not know the function of the other as illustrated previously is further supported by this finding. However, all three organs can improve on how well they work together, according to these scores, and understanding their various roles is one critical step in aiding this.

Based on these findings and key informant interviews, it is clear that the EPI-TWG is the most active of the three committees and is involved in all the processes and decisions of ICC and ZITAG. This is so in spite of challenges such as limited participation by some members as pointed out in several FCE reports [2], [26]. However, the active members have shown great commitment over the years and have worked tirelessly with large workloads in order to keep the EPI programme on track. Is it thus not amiss to consider it the key organ of the EPI or the glue that binds everything together.

Key informants were concerned that in the past two years or so, the role of ICC in immunisation-related decision-making had been greatly diminished and less effective. This follows a recent change to ICC terms of reference in which several things were changed, amongst them the chairmanship was moved from the Minister of Health to the Permanent Secretary at MOH, the secretariat was shifted from the Child Health Unit to the office of the Director Public Health and the mandate was broadened beyond immunisation to include RMNCAH&N activities broadly. This shift was strongly supported by Gavi, Swedish International Development Agency (SIDA) and the UN agencies.

Some key informants indicated that as a result of these changes, immunisation has become less prominent on the agenda for ICC and in the recent past, almost two years, there had been no ZITAG matter tackled at all during ICC meetings, which had become more like steering committee meetings, in their view. This was further compounded, some felt, by the relatively obscure positioning of the Child Health Unit within MOH as it was felt a programme of such importance as immunisation should be more strategically placed. Examples were given of the prominent placement of malaria programmes (National Malaria Elimination Centre), the Zambia Public Health Institute (ZNPHI) and Blood Bank in comparison to the EPI.

“We raised the issue as civil society of the capacity of the EPI and simply in terms of the establishment. The ICC has not taken this very seriously in my opinion. Because the ICC has got the capacity to make a difference in terms of where we place the Child Heath Unit and EPI as a programme and the prominence we give it even within the health sector and structurally within in the ministry of health as a programme. There is insufficient discussion and focus on this…Having to have the EPI programme report through someone who reports through someone in order to be on the decision-making table is…no! Immunisation is a foundational issue for the whole country and therefore the current placement is not ideal.” Key informant

This challenge is not unique to Zambia as in one recent multi-regional study, it was found that many countries faced the challenge of lack of integration of NITAGs with decision making processes, which hampered action on their recommendations [11]. In a comparative case study of Armenia, Ghana, Indonesia, Nigeria, Senegal and Uganda the key recommendation was need for sustainable technical and financial support as most of the countries reported challenges in this area [13]. Wiyeh et al. (2018) point out that research from other countries has shown that poor channels of communication and weak coordination between NITAGs and the Ministries of Health is one of the major challenges faced as NITAGs face challenges in communicating their recommendations to the ministries so that they can be considered and acted upon [13]. This has major implications for efficiency and sustainability as the mandate of ZITAG remains unfulfilled if its recommendations are not considered. For example, the switch from 10 dose to 5 dose measles vial and the recommendation to introduce a typhoid vaccine have been pending to be heard by ICC for nearly two years now. This brings into question the effectiveness of both ICC and ZITAG if ZITAG recommendations are not deliberated on and approved or otherwise by ICC as required.

“Between the [EPI] TWG and ZITAG, I think an interdependent relationship [exists] between the two. Because there are times when you are discussing issues in the TWG and people just say just send that one to ZITAG. That direct line is well appreciated by the TWG between the TWG and ZITAG. The relationship between the TWG and ICC is also quite clear. Things that need to go to ICC people say take that one to ICC… The issue is the relationship between ICC and ZITAG which relationship is facilitated by the secretariat. The secretariat for the ZITAG is a component of the secretariat for the TWG, but the CHU is no longer a secretariat for ICC. The regularity, the governance, everything really has changed. So, for instance once ZITAG concludes discussions things that now have to go to ICC can’t go because the regularity of ICC meetings has been affected, the governance there has been affected, and the way the agenda is done is a bit different. In fact, we have had instances where someone says we will take these ZITAG matters and discuss them at the next meeting, now let’s focus on these and those issues. We are nearly two years that ZITAG issues have not passed through ICC. So, there is a bit of a disturbance in the governance there…The recommendations of ZITAG cannot go further without passing through ICC. If it was a case where the ministry was self-reliant, we could easily pass them through MOH senior management and government policy now and go in this direction, but we inevitably need the input of partners. For example, one of the recommendations of ZITAG that has not been heard by ICC is the switch from 10 dose to 5 dose measles vial and there is quite a good argument, but the country cannot switch until ICC says yes or no, so we are stuck with the report. Another one is the recommendation to introduce a typhoid vaccine. We can’t have that done because we do not have the input of the ICC. So that is where I would say the bond is not as strong as it was and desired.” Key informant

Decision analysis framework and evidence use: Availability of evidence for decision making by ZITAG and ICC was considered adequate, and a decision-analysis framework is in place to guide recommendations by ZITAG that is evidence-based, structured, transparent, reproducible, and reliable and follows an immunisation process, but a few members were unaware of it. There was some doubt on the adequacy of ICC oversight in decision-making, with a perception that it may be a “rubber stamp” for endorsing recommendations.

Most (81.8%) of ZITAG members were aware of the existence of a decision analysis framework to guide their decisions as ZITAG. The ratings by these members on the adequacy of the various elements necessary is illustrated in Fig. 6. In key informant interviews, there was uncertainty on whether budget impact analysis and cost-effectiveness were considered in decision-making. This concern arose from recent financing challenges the programme had faced in securing both government and partner financial support to fund new initiatives. It was an important aspect of the work of the ZITAG, EPI-TWG and ICC to guide on the short term and long-term impacts of decisions on the budget. So many programmes pass the cost-effectiveness test but impose a burden on the budget, which if not well considered can undermine sustainability. These aspects of the decision-making framework thus needed strengthening.

Fig. 6

Decision framework characteristics for ZITAG.

In this regard, ZITAG was said to be useful in “…bringing independent decisions based on evidence particularly for the introduction of new vaccines; although this is limited on the economic evaluations and impact/cost implications of vaccines to be introduced.” Survey Respondent

Evidence use and decision making by ICC was mostly rated well by members who largely felt there was adequate use of local evidence for decision-making, that there was transparency and credibility in the decision-making process regarding vaccines and that ICC adopted recommendations suited to the Zambian setting. However, nearly a third were unsure of the use of local evidence and a quarter were unsure about whether ICC was effective. Fifteen percent were also unsure about credibility and transparency in the decision-making process regarding vaccines.

Key informant interviews also raised some doubts as to how effectively decisions were actually made by ICC in terms of the time they dedicate to deliberating on tabled EPI matters at meetings and the time members have to review provided information before meetings, particularly since the expansion of the mandate to RMNCAH&N matters.

“There was a bit of a concern on our side that the ICC started to function a bit like the CCM [Country Coordinating Mechanism] for Global Fund and that was not really what we wanted in principle. The ICC was really meant to be establishing the country and it is meant to just be the oversight body around child health basically broadly. I think RMNCAH&N is quite broad now and when you broaden it that widely I can imagine that the immunisation space gets significantly smaller. So, they went kind of to the extreme compared to other countries that have stayed focused on the child and adolescent health agenda. So, a two- or three-hour meeting now looking at RMNCAH&N, you can imagine its starts to become very superficial in nature now because you cannot do justice to those topics in a two-to-three-hour meeting, it is not possible.” Key Informant

There was also concern of the technical capacity given that some members of ICC were more of diplomats in their organisations rather than technocrats on the matters being discussed.

“You should also have very high-level people attending these meetings especially from bilateral donors, but what you see is that you do not necessarily have health people at the meetings especially from bilateral donors, some of them are really more diplomats, so it is a good advocacy platform but your need to use it effectively even though it need not be an overly technical meeting as that is why you have a ZITAG. It is not ideally formulated yet in the Zambian context.” Key Informant

Thus, there was a perception that ICC had become more of a “rubber-stumping” body to approve recommendations made by EPI-TWG or ZITAG without giving them the necessary due diligence.

“In 2020 there has not been an ICC meeting yet. Even the support that we get as EPI from ICC is now minimal and we have not benefitted much as EPI from ICC. It is used as we have an obligation to pass this through ICC before we submit. But ICC is supposed to be an advocacy platform for us to raise funds for EPI. They just endorse decisions and sign. I wish it would really be used as resource immunisation or advocacy meeting [for EPI].” Key Informant

“For me, ICC is kind of a rubber stamp in that…they do not really take time to understand [matters].” Key Informant

ZITAG promotion of generation of local data through surveillance and other studies was not studied specifically but was one area that most NITAGs lag behind in [10].

Limitations

Generally, the evaluation was executed according to the evaluation methodology. Despite initial fears related to COVID-19, the team was able to execute the interviews, surveys (using online platforms) and gather all relevant documents. Some interviews were done via Zoom while the majority were done face to face. In order to enhance response rates for the online survey, the field team made multiple follow-ups to obtain a response rate of at least 70% or more. All major EPI partners were interviewed. Desirability bias is another limitation to be aware of, although the authors believed this was minimised due to the trust they have built in evaluating the EPI team over the years.

Conclusion

EPI-TWG was a strong foundation for the immunisation programme in Zambia and was the driving force for much that happens in this area, supporting the ZITAG and ICC. ICC provided a platform for high-level coordination and collaboration of stakeholders in EPI and resource mobilisation while endorsing recommendations made by the other EPI committees. ZITAG had contributed to strengthening evidence use across the EPI and in having an independent multidisciplinary team to guide vaccine decisions in the country, leading to improved transparency.

Thus, in conclusion and in terms of financial and programmatic sustainability, which was one of the foci of this study, ZITAG, EPI-TWG and ICC were key in fostering such sustainability and were strong to varying degrees in terms of both functionality and effectiveness. However, they need strengthening in certain areas in order to add further value to the work, resource mobilisation and sustainability of the EPI programme. In particular, ICC seems to be the weaker link between the three organs in terms of promoting the work of EPI with regard to regularity of meetings, prominence of EPI in the meetings, effectiveness of aiding decision-making and acting as an advocacy and fundraising platform for EPI. These matters need to be urgently addressed as it has the core mandate for helping with raising funds for EPI.

Recommendations

There was generally diverse and longstanding membership of the various committees, but consider identifying and adding some additional key organisations to committees. There was good progress in terms of adding members with economic evaluation expertise to ZITAG, but a framework for economic evaluation (e.g., budget impact analysis and cost-effectiveness) should be developed. The government and Gavi should consider providing funding directly to ZITAG in the short term to establish the independent secretariat and improve functionality and effectiveness. The relevant secretariats should ensure all members across EPI-TWG, ZITAG and ICC understand the functions of other organs and thus their placement and specific roles.

ZITAG should ensure that all members are oriented on the decision-analysis framework. ICC secretariat should put mechanisms in place to promote parity of focus areas discussed during ICC meetings and ensure that backlog of ZITAG recommendations is cleared and that EPI is given due attention during such meetings. There is need for improved functionality of ICC in terms of prioritizing and providing adequate time to review EPI matters.

CRediT authorship contribution statement

Moses C. Simuyemba: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing – original draft, Visualization, Supervision. Chitalu Chama-Chiliba: Conceptualization, Writing – review & editing. Abson Chompola: Conceptualization, Writing – review & editing. Aaaron Sinyangwe: Conceptualization, Project administration, Writing – review & editing. Abdallah Bchir: Conceptualization, Funding acquisition, Writing – review & editing, Validation. Gilbert Asiimwe: Conceptualization, Funding acquisition, Writing – review & editing. Felix Masiye: Conceptualization, Writing – review & editing, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

ZITAG	ICC	EPI-TWG
• Zambia Medical Association • Civil Society Organisations • Ministry of Finance • Midwives Association of Zambia • Association of Paediatric Nurses in Zambia	• Zambia Medical Association • Office of the Vice President, • Ministry of Finance • Zambia National Public Health Institute • Rotary Club	• Zambia Paediatricians Association • Disaster management mitigation unit (DMMU) • Ministry of Finance • Ministries of Higher Education, General Education, Community Development and Social welfare, Chiefs and Traditional Affairs, National Guidance and Religious Affairs, Information and broadcasting • University of Maryland • Rotary • Business houses such as banks