Dual indices for prioritizing investment in decentralized HIV services at Nigerian primary health care facilities.

Decentralizing health services, including those for HIV prevention and treatment, is one strategy for maximizing the use of limited resources and expanding treatment options; yet few methods exist for systematically identifying where investments for service expansion might be most effective, in terms of meeting needs and rapid availability of improved services. The Nigerian Government, the United States Government under the President's Emergency Plan for AIDS Relief (PEPFAR) program and other donors are expanding services for prevention of mother-to-child transmission (PMTCT) of HIV to primary health care facilities in Nigeria. Nigerian primary care facilities vary greatly in their readiness to deliver HIV/AIDS services. In 2012, MEASURE Evaluation assessed 268 PEPFAR-supported primary health care facilities in Nigeria and developed a systematic method for prioritizing these facilities for expansion of PMTCT services. Each assessed facility was scored based on two indices with multiple, weighted variables: one measured facility readiness to provide PMTCT services, the other measured local need for the services and feasibility of expansion. These two scores were compiled and the summary score used as the basis for prioritizing facilities for PMTCT service expansion. The rationale was that using need and readiness to identify where to expand PMTCT services would result in more efficient allocation of resources. A review of the results showed that the indices achieved the desired effect-that is prioritizing facilities with high need even when readiness was problematic and also prioritizing facilities where rapid scale-up was feasible. This article describes the development of the two-part index and discusses advantages of using this approach when planning service expansion. The authors' objective is to contribute to development of methodologies for prioritizing investments in HIV, as well as other public health arenas, that should improve cost-effectiveness and strengthen services and systems in resource-limited countries.

Key Messages

In an era of limited health care budgets, a system of dual indices, summarized to indicate facility readiness to provide services and local need, can help decision makers to identify the most eligible facilities for investment in PMTCT or other services.

The use of multiple, weighted indicators not only identifies facilities with stronger services, but also provides evidence of specific areas of service weaknesses and facilitates measurement of changes over time.

In the current economic climate, this method gives policy makers a new, and needed, tool for evaluating multiple facilities and prioritizing the most eligible for investments in new or expanded services.

The dual index system, which balances the complex interactions of readiness, need and feasibility, can be used to assess facilities for investment in PMTCT, other HIV services, and other public health areas of concern.

Introduction

Access to HIV services is part of Goal 6 of the Millennium Development Goals (UN 2013), endorsed internationally in 2000, with a new target of universal access by 2015 accepted in 2010 (WHO, UNICEF and UNAIDS 2011; UNAIDS 2012). Access to prevention of mother-to-child transmission (PMTCT) of HIV and preventive services for infants is prioritized in the Global Plan Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping Their Mothers Alive (UNAIDS 2011). Yet many resource-limited countries—especially those in sub-Saharan Africa, where 68% of people with HIV infection live (WHO, UNICEF and UNAIDS 2011)—face enormous challenges in delivering health care services, including those for HIV (UK NGO AIDS Consortium 2011). In high HIV incidence countries, meeting national and international goals in mother and child health may not be attainable without effective scale-up of HIV services. This scale-up will require effective, efficient planning.

The Government of Nigeria is a case in point. With a prevalence rate of more than 3% and over 3 million people living with HIV, Nigeria has Africa’s second-largest number of HIV infected persons (WHO 2013). Currently, mother-to-child transmission accounts for an estimated 10% of transmission, and Nigeria has one of the world’s highest burdens for vertical HIV transmission due to poor PMTCT coverage (Nkwo 2012). The context of the global HIV burden, combined with developments within the Nigerian health care system, form the larger background for the development of the priority index described in this article. The Nigerian health system is organized along three tiers of health care—primary, secondary and tertiary. Each tier provides a different level of services; referral systems are expected to link the three tiers (Akande 2004). Primary health facilities are most numerous and are located closer to communities than higher-level facilities. They provide the most basic level of health services and are intended to be the first point of contact for patients (FMoH 1990; Osibogun 1996). Nigeria remains underserved for HIV services (NACA 2012), with most PMTCT services offered only at the secondary and tertiary levels. There has been a concerted effort by the Nigerian Government, the United States Government (USG) under the President’s Emergency Plan for AIDS Relief (PEPFAR) program and other donors to bring HIV services, and particularly PMTCT services, to the primary health care facility level. This should decongest overburdened secondary and tertiary health care facilities while increasing the geographic coverage of services.

In 2012, MEASURE Evaluation under phase III assessed 268 Nigerian health facilities that had been targeted for USG/PEPFAR resources to expand access to PMTCT services, either by introducing new services where USG/PEPFAR-supported implementing partners were not yet offered or by increasing the scope and quality of existing services. As a part of the assessment, MEASURE Evaluation provided an approach for identifying which facilities should be prioritized for PMTCT investment. The objective was to support efficient utilization of resources by prioritizing facilities where expansion could be achieved most rapidly, or where the need for the services was highest.

Justification

There is a body of experience and tested methods for collecting facility-level information about readiness to provide quality health care services, including PMTCT. The Service Availability and Readiness Assessment (SARA), developed by the World Health Organization (WHO), and the Service Provision Assessment (SPA), developed by ICF Macro, are two well-known examples with tested tools and written methodologies. These two methods were harmonized so that for items the two surveys have in common, information collected using either tool is comparable (MEASURE DHS 2010; WHO 2012).

There are a few examples of using indices and summary scores comprised of multiple indicators for monitoring change. These include a summary score to measure the quality of family planning services (CCP 1998; Hong ); a summary score based on a “balanced scorecard” to measure facility-level changes in primary health care services and quality (Edward ); and a summary score to compare the status of facilities with regards to readiness to provide services (WHO 2012). However, there is limited literature on experience within the health sector in developing countries with aggregating multiple indicators into a single measure and using this for planning and to prioritize resource allocation.

This article describes the development of a two-part index for prioritizing investments at facility level for PMTCT service delivery or expansion; lays out the rationale for the domains and indicators included in each index; and discusses the methodology’s strengths as well as ways of improving the indices and scoring measures. It then discusses the implications for using this methodology for planning and investment purposes. The article used both the quantitative and qualitative data in the development of the index.

Methodology

Sample and data collection

The data used in this article were collected, through interviews and observations, from 268 USG-supported primary health care facilities in 17 states and the Federal Capital Territory of Nigeria. These facilities were not representative of facilities in each state, but rather were purposively selected for PMTCT service expansion as part of the PEPFAR’s HIV service decentralization initiative. A majority of the assessed facilities are government owned (95%), and they are almost evenly distributed between urban and rural areas (52 vs 48%).

Data were collected in each facility by trained research assistants using structured tools largely adapted from existing, tested tools, including the SARA and SPA tools. Additional questions to measure governance, leadership and linkages among services at each facility were adapted from the 2007 Tool to Assess Site Readiness for Initiating Antiretroviral Therapy (ART) or Capacity for Existing ART Sites, produced by John Snow, Inc. (Hirschhorn ). Indicators for the quality of services were largely adapted from the Nigerian National Guidelines for Prevention of Mother-to-Child Transmission of HIV (PMTCT) (FMoH 2010).

The respondents for the service and facility information were identified within the facility as key informants most knowledgeable about the issues being examined. In addition, providers for all relevant services were interviewed to ascertain training related to the services they delivered. Research Assistants validated informants’ responses about facility resources and documents through observation.

Data quality was ensured by field supervision and random validation of responses, as well as targeted validation where responses were inconsistent or questionable. The data were entered into Census and Survey Processing System databases and analysed using SPSS.

Findings from the initial questionnaires were used to develop the readiness and needs indices. Indicators ultimately included in the indices were agreed upon in collaboration with USAID experts and other stakeholders, including the FMoH, which helped to tailor the indices to the local context.

Developing the readiness and needs indices

The objective was to develop a methodological approach that would result in improved decisions when prioritizing facilities for PMTCT service expansion. Stakeholders, including the USG PEPFAR team, indicated specifically the requirement to prioritize facilities where HIV prevalence was highest and those with higher utilization in order to reach larger numbers of potential HIV positive women, but also needed to be realistic about the feasibility of expanding the services in terms of speed and resources. MEASURE Evaluation developed a dual index—one comprised indicators that showed the “readiness” for each facility to expand PMTCT services, with the assumption that where components of relevant health services and systems already existed, services could be expanded more rapidly. The other index of “need” assumed that the least prepared facilities—those that would require extensive resources prior to service introduction—might also be located in higher-risk and underserved communities or localities.

Readiness refers to the presence of infrastructure, systems and resources necessary for providing quality services. Five domains, or categories, comprised of 20 indicators were selected to represent readiness. The domains and indicators were identified from a review of current PEPFAR program priorities, guidelines for PMTCT services, the SARA readiness indicators for PMTCT services, additional indicators for quality services from the SPA survey and discussions with stakeholders and technical advisors about the needs and priorities for the survey. PMTCT service guidelines include many different components that all contribute to quality PMTCT services but are not necessarily directly linked to PMTCT (e.g. infection control measures). It was decided that including all major items considered critical for general as well as PMTCT specific service quality would provide a measure of the status of the facility in the context of being able to provide a full package of PMTCT services with good quality. A full description of the indicators and domains for readiness and the variables within the quality indicators for PMTCT and HIV Counselling and Testing (HCT) services is provided in the publication National HIV/AIDS Division, Federal Ministry of Health (FMoH) [Nigeria] and MEASURE Evaluation, 2014. Assessment of Primary Health Care Facilities for Decentralization of HIV/AIDS Services in Nigeria 2012.

Table 1, panel 1, provides a summary of the objectives behind the selection of indicators that comprise the readiness index, and the rationale for inclusion in the index. The index includes several community resource indicators. Community services may include socioeconomic services, or non-HIV-related services. Linkages with any community services was identified as important because it provided an entry for expanding HIV-related services in the community—important for addressing stigma issues that impact service utilization and follow-up. A facility with links to HIV-related community services would receive credit for both indicators—providing internally weighting for existing links with community HIV services.

Table 1.

Conceptual framework for selection of readiness indicators, needs and feasibility indicators

Objective	Indicators	Rationale for inclusion in index
Panel 1: Readiness Indicators
Facilitate rapid scale-up	Related services in the facility (ANC, delivery, HIV C&T)	Staff trained in related services and familiar with clients already exist and can be used for service expansion
	Geographic access to related services (ART)	Referral resources for ARV treatment to supplement basic PMTCT services exist and can be used for maternal treatment
	Planning for PMTCT has started (staff have been trained, managers have been identified, physical infrastructure is present)1	Key factors for service expansion are in process of being addressed. Management support should be strong
Facilitate quality	HIV testing at service point (HCT and PMTCT)	Improved follow through on testing
	Links with community HIV/AIDS related services	Improved client identification, follow-up, and adherence
	Quality components present for PMTCT and HCT services (based on indicators from SARA and SPA surveys for readiness and systems in place)2	Facilities with existing quality indicators relevant to, but not necessarily linked to PMTCT may be expected to continue to maintain quality—this reduces the pressure for newly addressing all PMTCT service quality items
Provide synergies to support quality and rapid scale up	Links with other HIV/AIDS related services	Additional resources to support facility-specific needs
	External TA, funds, goods in-kind
	Links with any (HIV or non-HIV) community services3	Systems to build on exist for client identification and follow-up to improve compliance
		Additional services (socioeconomic) to address client socioeconomic support needs may be available
		Improved communication and trust for addressing this, still stigmatizing illness
Systems to support quality services	Drug storage and management practices4	Drug supplies maintained
		Drug wastage and loss minimized
Panel 2: needs and feasibility indicators
Prioritize populations with highest prevalence	HIV test positivity from HCT and PMTCT services, where available	Positivity was categorized, with higher rates of positivity scored higher. Both test results were considered important to provide a more complete picture of the population served by the facility. PMTCT and HCT services capture different populations so HCT results might reflect higher-risk, and PMTCT results might reflect lower-risk populations
Prioritize those with less access to alternative service sites	Distance to ART service site	Distances were categorized, with longer travel distances scored higher.
Prioritize facilities where larger numbers of eligible clients are likely to be available	ANC caseload	ANC caseload was categorized, with higher caseloads scored higher. Higher caseloads would improve rapid coverage for PMTCT
Prioritize facilities offering partial PMTCT services	PMTCT services include HIV testing but referral for preventive ART	These facilities should already have staff trained for HIV test counselling so expansion should be more rapid
		Compliance with PMTCT preventive ARV should be strengthened
Ensure that staff of the cadre allowed to provide PMTCT services are present	Number of staff of the eligible cadre who work in the facility	Number of staff was categorized with higher numbers scored higher. It was not considered realistic to prioritize a facility assuming that staff eligible to provide the service would be available in a reasonable time frame if they were not already present

1Planning was not assessed for facilities that already had PMTCT services that included HIV testing and preventive ART. Planning was assessed for facilities reported PMTCT services, but that only provided testing and referral

2NBS Tanzania and Macro International Inc. 2007; MOH [Uganda] and Macro International Inc. 2008) and the Nigeria National Guidelines for PMTCT (FMOH 2010)

3Indicators for this domain were identified by reviewing program strategies for PMTCT services and related patient care in the National PMTCT Guidelines, and from discussions with the FMoH and USG/PEPFAR (Katabarwa ; Okeibunor )

4These indicators were drawn from SPA survey documents

The needs index included six indicators to identify not only where need was the greatest (regardless of readiness), but also where the start-up of PMTCT services was feasible in practical terms. Table 1, panel 2, provides a summary of the objectives behind the selection of indicators that comprise the needs index, and the rationale for inclusion in the index. Staffing is assessed in both the readiness and the needs index, but each measures different aspects. The readiness index captures staff trained in issues relevant to PMTCT services, while the needs index measures total numbers within the cadre eligible for training—an indicator for feasibility of expanding services. While training of staff can be addressed through programming priorities and funding, the actual availability of staff at the cadre the FMoH defines as eligible for providing PMTCT services would be a key factor in the feasibility of start-up or expansion of services. It was considered unlikely that additional professional staff would feasibly be made available in a time frame sufficient for service scale up, as the roots of the Nigerian staffing problems are complex.

Scoring and weighting for the summary index

Readiness index

Rather than arbitrarily assigning priority to the variables in the readiness index, it was decided that in most cases the argument could be made that the variables were of equal importance, so each indicator was given the value of ‘1,’ and where the indicator had multiple variables (such as service quality for HCT and PMTCT services), the percent of variables present within the indicator would be the score, with 100% giving the maximum score of 1. HCT and PMTCT quality scores were weighted by a factor of 2, to provide a small advantage for facilities with higher values for quality of existing services because of its causal link with patient safety and wellbeing.

Need index

In response to the stakeholder objectives and concerns—wanting to prioritize facilities serving high need populations, with high utilization, but also where rapid scale up of services was feasible, it was decided that although this index used only six variables to describe need and feasibility, its scoring should contribute at least half of the summary priority score. Having no similar scales to draw upon, each need indicator was weighted based on discussions and consensus among the stakeholders (MEASURE Evaluation, USAID/Nigeria and PMTCT specialists at HIV/AIDS division of the FMoH) (see Box 1).

Box 1. Determining weights for need indicators

We determined the weights for need indicators based on factors affecting the need for PMTCT services and the feasibility of instituting or strengthening these services.

HIV test positivity for each HCT and PMTCT service received was categorized using UNAIDS prevalence categories (GARPR 2012) and weighted by 1.5, with the resultant possible scores ranging from 0 to 6. The rationale was that high positivity indicated a need for PMTCT services, and increased the need to ensure that existing PMTCT services were of good quality. Facilities without HCT or PMTCT (18 and 46% of facilities, respectively) did not receive any addition to their need score based on HIV test positivity.

Distance to ART services was categorized with possible scores ranging from 0 (service available in the facility) to 5 (for the furthest distance from the facility to ART services) with the rationale that facilities without practical alternatives for PMTCT should be prioritized.

ANC caseload was categorized with possible scores ranging from 0 (<49 new clients in the past three months) to 3 (≥100 new clients in the past 3 months). The researchers' consensus was that a maximum score of 3 provided a reasonable balance for the weight within the index—ANC caseload could have a slightly lower priority than other feasibility indicators given the importance of the other indicators, and the increased subjectivity that would arise in trying to divide low or high ANC client caseloads further.

Offering HIV testing for PMTCT but referring positive women for ART was given a score of 4 to bring this variable's priority close to that of the other indicators, with the rationale that expansion of the service was a priority to influence client follow-through on preventive ART for the positive mother and her newborn.

Available staff included possible scores ranging from 0 (no staff in the “trained” cadre) to 4 (four or more trained staff) to align this with the other feasibility indicators.

Relative contributions of the readiness and need indices

If a facility had the highest score for each indicator, the 20 readiness indicators could potentially contribute 44% (22 points) to the summary score, while the six need indicators could provide 56% (28 points) of the total 50-point score. Highest contributors to the score were evidence of planning for PMTCT service expansion (16%) and facility measures for HIV prevalence (12% each for HCT and PMTCT test positivity). Figure 1 shows the relative contribution from the readiness and needs indicators to the final summary score.

Figure 1.

Percent of contributions of each element to the readiness and needs index.

Data analysis

Descriptive analyses using cross-tabulations were carried out to provide a profile of facilities’ infrastructure and resources, by facility and by region, so that findings for specific service items and components could be more useful for program managers. The results from the scored indices that included the score for readiness, the score for need and the total score were then provided, by state and facility, ordered from highest total score to lowest. The highest scoring facilities were recommended for prioritization of PMTCT service expansion.

After the data were analysed the scoring for the indices was reviewed critically to identify where changes might improve the effectiveness of the methodology for prioritizing facilities for investment. The prioritization results were analysed by grouping facilities into four quartiles, based on the summary scores, with quartile 1 representing the lowest-scoring facilities, and quartile 4 the highest and looking at the percentage of each variable within facilities in each quartile (Table 2).

Table 2.

Indicator findings by priority score quartiles and scoring for each indicator (n = 268)

	Indicator	Percentage of facilities with the indicated indicator by quartile (n = 268)				Maximum score for indicator
		1st Quartile (%)	2nd Quartile (%)	3rd Quartile (%)	4th Quartile (%)
1	HIV rapid test at HCT service point***	31.8	56.1	70.6	79.4	1
2	HIV rapid test at PMTCT service point***	9.1	25.8	45.6	58.8	1
3	Delivery services in facility***	62.1	89.4	92.6	94.1	1
4	Links with community workers***	40.9	57.6	67.6	79.4	1
5	Links with community HIV/AIDS-related services***	34.8	50.0	63.2	77.9	1
6	Links with HIV/AIDS-related services***	19.7	50.0	47.1	77.9	1
7	HIV related technical support from external sources***	21.2	56.1	76.5	75.0	1
8	External funds from non-government sources*	0	7.6	7.4	4.4	1
9	Drug management practices quality score***1	4.5	4.5	10.3	23.5	1
10	Drug storage practices all good*2	42.4	43.9	50.0	63.2	1
Percent in the category reporting PMTCT HIV testing and preventive ARV so questions 13–19 were not asked		25.8	37.9	72.1	88.2
11	Facility reports plans for expanding PMTCT services*	9.1	25.8	22.1	10.3	1
12	At least one staff trained for PMTCT services in past 2 years	4.5	18.2	14.7	8.8	1
13	Current staffing sufficient for PMTCT**	7.6	6.1	1.5	2.9	1
14	A manager has been designated for PMTCT	0	10.6	8.8	7.4	1
15	Plans for additional staff (or current staffing sufficient)**	27.3	50.0	47.1	66.2	1
16	New staffing approved (or current staffing sufficient)	13.6	22.7	14.7	27.9	1
17	Site for visual and auditory privacy for PMTCT exists***	48.5	47.0	23.5	10.3	1
18	Sufficient conditions for storage of ARVs***	33.3	33.3	13.2	8.8	1
Mean weighted score within the quartile (2 × percent of all quality items present)
19	HCT Service quality	0.45	0.98	1.32	1.42	2
20	PMTCT service quality	0.22	0.54	1.00	1.22	2
	Maximum possible score for readiness:					22
	PMTCT service need and feasibility for service expansion
21	Prevalence for HIV testing with HCT services***
	No information	62.1	25.8	2.9	4.4	0
	<0.041	30.3	56.1	45.6	16.2	1.5
	0.041 to <.068	6.1	15.2	26.5	20.6	3
	0.068 to <0.08	0	1.5	5.9	5.9	4.5
	Great than or equal to 0.08	1.5	1.5	19.1	52.9	6
22	Prevalence for HIV testing with PMTCT services***
	No information	9.1	4.5	7.4	0	0
	<0.041	90.0	92.4	77.9	60.3	1.5
	0.041 to <0.068	0	3.0	8.8	11.8	3
	0.068 to <0.08	0	0	4.4	4.4	4.5
	≥0.08	0	0	1.5	23.5	6
23	Average monthly new ANC clients (based on 12 months data)***
	1–49 clients	93.9	87.9	63.2	57.4	1
	50–99 clients	4.5	7.6	19.1	14.7	2
	>99 clients	1.5	4.5	17.6	27.9	3
24	HIV test for PMTCT but refer outside for preventive ARV***	4.5	7.6	10.3	29.4	4
25	Estimated minutes of travel to the nearest location where client can receive ART services**
	This facility	1.5	3	7.4	7.4	1
	<15 min	31.8	25.8	19.1	10.3	2
	15–30 min	47	37.9	29.4	35.3	3
	31–60 min	10.6	18.2	23.5	25	4
	>60 min (or do not know)	9.1	15.2	20.6	22.1	5
26	Total number of staff assigned to the facility who are defined within the Nigeria MoH system as trained***
	0	1.5	0	0	0	0
	1	13.6	4.5	0	1.5	1
	2	21.2	13.6	5.9	2.9	2
	3	16.7	6.1	1.5	4.4	3
	≥4	47	75.8	92.6	91.2	4
	Maximum possible score for need					28

*P < 0.05, **P < 0.01, ***P < 0.001. Notes: All values are percentages unless otherwise noted.

1Sample drugs examined have no stock out past 6 months, drug management records are up-to-date, no expired drugs found, among all drugs no expired observed, and medicines are stored first-in-first-out.

2Storage practices compliance with the principles of drugs off floor, away from sun and dampness, no evidence of rodents or pests, storage area well ventilated and drugs are stored in either a dedicated room or cabinet.

Results for each indicator by quartile were also reviewed to identify any patterns that might point towards revising the methodology for constructing the indices.

Results

Among facilities in quartiles 3 and 4, the highest priority quartiles, the proportion of comprehensive facilities and hospitals, was higher than in the lower two quartiles, and health centres comprised a comparatively larger proportion of those in quartiles 1 and 2, or lowest priority quartiles. The proportion of the lowest level of facilities (health post or primary health care centres) was similar across the quartiles (Figure 2).

Figure 2.

Type of facility by priority score quartile. Note Quartile 4 CHC/Hospital/other should be 25: numbers not exactly 100% due to rounding (it’s 51.5 and 23.5) for the other categories.

Almost all facilities in the top two priority score quartiles (99 and 97%, respectively) provided some HIV testing (Figure 3) and provided PMTCT services with HIV testing and preventive ART (56 and 59%, respectively) (Figure 4), and those in quartile 4 were also three or more times more likely than other quartiles to be offering HIV testing in ANC, but referring patients for preventive ART. HIV testing was captured in multiple variables within the domains, potentially contributing up to 14 (28%) of the maximum possible score of 50.

Figure 3.

HIV testing availability by priority score quartile.

Figure 4.

PMTCT service availability by priority score quartile.

A review of the readiness indicators by quartile shows that for most indicators, a linear pattern exists (i.e. the proportion of facilities with the indicator increases from the lowest to the highest quartile) although there were cases where the relationships between facility scores from the lowest to highest quartiles were non-linear—that is, they showed no consistent pattern relative to the increasing score. The average scores for facilities selected for prioritization, by quartile, ranged from 14.1 to 28.1 out of a possible total of 50 (Figure 5).

Figure 5.

Prioritization scores for readiness and need by summary score quartile. Note Maximum possible score for Need Index = 28 and Readiness Index = 22.

While the “readier” facilities—those already offering HIV testing and PMTCT services—were more likely to be prioritized, it was need indicators that contributed the most (and disproportionally) to the priority scores, across all quartiles. If facilities achieved the maximum score for all indicators in the need and readiness indices, the scoring mechanism allowed need to contribute 56% to the final score, but on average, need contributed up to 61% of the final score and up to 65% in the primary health centres that fell in quartile 4 (Table 3).

Table 3

Percent of total score contributed by need indicators

Type of facility	1st Quartile	4th Quartile	Total (all facilities)
Primary health centre/dispensary/health post	64% (n = 2)	65% (n = 17)	62% (n = 67)
Health centre	65% (n = 46)	62% (n = 35)	61% (n = 162)
Comprehensive HC/hospital	76% (n = 16)	59% (n = 17)	59% (n = 39)
All facilities	65% (n = 66)	62% (n = 68)
Quartile
1			65% (n = 66)
2			57% (n = 66)
3			59% (n = 68)
4			62% (n = 68)
Total			61% (n = 268)

The higher proportion of comprehensive health centres and hospitals within quartile 4 was not surprising, since it was expected that these higher level facilities would have better developed services, and thus would score higher on the indicators than primary health care centres, health posts or dispensaries. It was surprising, however, that the lowest level facilities were represented fairly similarly across the quartiles.

The priority scoring was reviewed within each quartile by facility type to assess the extent to which facility type influenced the results (Figures 6 and 7). Within quartiles, there were no major differences between facility types in the scores for readiness or need. Need consistently constituted a higher proportion of the score than would be possible if all indicators for both need and readiness were present. Facilities within the lowest quartile showed both lower need and lower readiness than facilities in the highest quartile; however, both need and readiness remained in similar proportions across all quartiles.

Figure 6.

Priority score for facilities in the highest (4th) quartile.

Figure 7.

Priority score for facilities in the lowest (1st) quartile.

Comparisons of findings between facility types within quartile 4 showed that high-scoring low-level facilities (primary health care/dispensary/health post) had more community links than the higher-level facilities (Table 4). They also had high ANC caseloads, were far from ART services and more often reported they provided HIV testing for PMTCT but referred for preventive ARV (all indicators of need).

Table 4.

Indicator findings by facility type for facilities in quartile 4 (n = 68)

		Dispensary/PHC(n = 16)	Health centre (n = 35)	Comprehensive/hospital (n = 17)
1	HIV rapid test at HCT service point***	68.8%	82.9	82.4	1
2	HIV rapid test at PMTCT service point ***	50.0%	62.9	58.8	1
3	Delivery services in facility***	100%	88.6	100	1
6	Links with community workers***	87.5%	77.1	76.5	1
7	Links with community HIV/AIDS-related services***	87.5%	74.3	76.5	1
8	Links with HIV/AIDS-related services***	87.5%	68.6	88.2	1
9	HIV-related technical support from external sources***	50	80	88.2	1
10	External funds from non-government sources*	6.3	2.9	5.9	1
11	Drug management practices quality score***1	18.8	25.7	23.5	1
12	Drug storage practices all good*2	50.0	65.7	70.6	1
	Percent in the category reporting PMTCT HIV testing and preventive ARV so questions 13–19 were not asked	6.3	17.1	5.9
13	Facility reports plans for expanding PMTCT services*	6.3	14.3	5.9	1
14	At least one staff trained for PMTCT services in past 2 years	0	14.3	5.9	1
15	Current staffing sufficient for PMTCT**	6.3	0	5.9	1
16	A manager has been designated for PMTCT	0	11.4	5.9	1
17	Plans for additional staff (or current staffing sufficient)**	62.5	60	82.5	1
18	New staffing approved (or current staffing sufficient)	25	20	47.1	1
19	Site for visual and auditory privacy for PMTCT exists***	0	17.1	5.9	1
20	Sufficient conditions for storage of ARVs***	0	17	0	1
Mean weighted score within the quartile (2 × percent of all quality items present)
21	HCT Service quality	1.26	1.42	1.53	2
22	PMTCT service quality	1.14	1.19	1.33	2
Maximum possible score for readiness					22
PMTCT service need and feasibility for service expansion
23	Prevalence for HIV testing with HCT services***
	No information	12.5	2.9	0	0
	<0.041	6.3	11.4	35.3	1.5
	0.041 to <0.068	18.8	28.6	5.9	3
	0.068 to <0.08	12.5	2.9	5.9	4.5
	≥0.08	50	54.3	52.9	6
24	Prevalence for HIV testing with PMTCT services***
	No information	0	0	0	0
	<0.041	62.5	57.10	64.7	1.5
	0.041 to <0.068	18.8	14.3	0	3
	0.068 to <0.08	0	2.9	11.8	4.5
	≥0.08	18.8	25.7	23.5	6
25	Average monthly new ANC clients (based on 12 months data)***
	1–49 clients	43.8	65.7	52.9	1
	50–99 clients	18.8	8.6	23.5	2
	>99 clients	37.5	25.7	23.5	3
26	HIV test for PMTCT but refer outside for preventive ARV***	50	25.7	17.6	4
27	Estimated minutes of travel to the nearest location where client can receive ART services**
	This facility	0	5.7	17.6	1
	<15 min	12.5	11.4	5.9	2
	15–30 min	31.3	31.4	47.1	3
	31–60 min	25	28.6	17.6	4
	>60 min (or do not know)	31.3	22.9	11.8	5
28	Total number of staff assigned to the facility who are defined within the Nigeria MoH system as trained***
	0	0	0	0	0
	1	0	2.9	0	1
	2	6.3	2.9	0	2
	3	0	8.6	0	3
	≥4	93.8	85.7	100	4
	Maximum possible score for need				28

*P < 0.05, **P < 0.01, ***P < 0.001.

1Sample drugs examined have no stockout past 6 months, drug management records are up-to-date, no expired drugs found, among all drugs no expired observed and medicines are stored first-in-first-out.

2Storage practices compliance with the principles of drugs off floor, away from sun and dampness, no evidence of rodents or pests, storage area well ventilated and drugs are stored in either a dedicated room or cabinet.

Discussion

In a novel approach for planning resource investment in a decentralized setting, generally accepted indicators were used to construct two indices (for readiness and need) and then summarized the scores for the two to prioritize facilities for PMTCT service initiation or expansion. This method represents an innovative use of generally accepted readiness, quality, and need measures to prioritize investments for service expansion in a decentralized health care setting.

Overall, the dual index method was found to function as hoped in determining which facilities to prioritize for PMTCT services. By combining and weighting 28 indicators of readiness and need, facilities were identified as most likely to serve populations having most need of the services and those where expansion could feasibly be rapidly achieved. Patterns across all four scoring quartiles revealed that the prioritized facilities tended to be hospitals and comprehensive health centres, though some lower-level facilities with strong community links and high need (among other characteristics) also were prioritized. The following addresses specific aspects of the findings.

Scoring

Although scores for most indicators increased when looking from lowest to highest quartile, scoring for some indicators did not show consistent linear trends. This can be interpreted as supporting the concept of using an inclusive index in which essentially all key variables defined as important for quality services are measured and then a summary score calculated, rather than prioritizing a few select variables. Including many indicators, each of which is considered important for quality services, not only prioritizes those facilities with more components of quality services, but also identifies specific areas for quality services that are weak. This also maintains a focus on the full range of components that are important for quality services, rather than focusing on a subset.

A review of the scoring, and the effect of the weighting showed that the indices achieved the desired effect—prioritizing facilities with high need even when readiness was weaker. Need, however, did not completely overshadow readiness; readiness scores increased from quartiles 1 to 4. The facilities in quartile 1 had not only low readiness scores, but also lower need scores, compared with facilities in the higher quartiles. It is possible that some facilities in quartile 1 that offered no HIV testing (51%) had high need that was not captured since they did not have HIV prevalence rates to go into the index. However, the quartile 1 facilities also were low on other indicators of need—for example, they had low ANC case loads, and were not further from ART services than facilities in the other quartiles. Thus, the indices were felt to give a fair representation of quartile 1 facilities as weak facilities for investments for service initiation or expansion.

Utilization of the indices

Although the overall utilization of the prioritization index for planning and managing PMTCT interventions scale-up was weak, the individual facility profiles that graphically present the strengths and weaknesses for each facility have been used to develop strategies to improve upon the weak areas for individual facilities by the implementing partners. Based on our research experience, several issues were identified that may contribute to weak utilization of summary scores for planning purposes.

Practical experience at all levels lies with addressing specific items for program improvement and planning, and this was evident when, on follow-up, stakeholders requested individual item information for each facility. There seems to be discomfort and lack of understanding for how to use the summary scores along with individual indicators for global prioritization, but then proceeding to identify specific issues to address for improving service availability and quality.

Stakeholders recognized problems with trying to plan using many indicators, since different indicators do not consistently show improvement or weakness across facilities. The authors experience when working with groups identifying indicators for program monitoring have found that it is difficult to gain agreement on a few key indicators when there are many service components related to quality. So many indicators are frequently monitored for M&E for a service. The utility of a compiled score that pulls together all items of programmatic interest, as a picture of where services are in the context of the ideal, needs to be better communicatedStakeholder/planner question “are we better or worse?” can only be answered when the full range of program priorities is considered as an aggregate measure.

Stakeholders for this study were not familiar with the content of the assessment report. On follow-up, information that was included in state reports that presented individual facility information was requested, and the researchers had to point out where the information was in the reports.

While stakeholders were appreciative of the scoring (and reported it will be used for future prioritization of resources), there may have been other non-service and need related issues that influenced decisions about initial investments for service expansion. It is not uncommon to feel a need for geographic even-handedness, and sometimes there is external pressure to focus resources to an area. The time before reports were completed was also a factor as the stakeholders felt they needed to move forward before results were available.

Limitations

A critical review of the variables, scoring and weighting used in the indices pointed to some limitations. Although no evidence was found that these limitations erroneously skewed prioritization, the results would be more self-evident with clearer criteria for variables for which information is not uniformly available. Future prioritization indices should consider the following issues:

Scoring for HIV positivity for the need index should have been limited to HCT services, rather both HCT and PMTCT. This would present a more focused variable for population risk, since HCT test results likely come from the general population or those at higher risk. PMTCT clients could well represent a lower-risk population (usually married, more motivated to seek health care—results in this survey showed higher prevalence for HCT populations over PMTCT populations). Where HIV testing was not conducted, positivity could be imputed using state-level prevalence. This approach would have provided a more accurate picture of need, even where HIV test results at facility level were not available.

The variables on planning for PMTCT services were only assessed at facilities reporting that they did not offer PMTCT services that include both testing and preventive ARV. The original objective was to prioritize facilities without current services that were in the process of planning to add PMTCT; however, this resulted in underscoring for facilities that offer these services but are planning to expand them. The low scores for quality indicators for PMTCT services support the notion that plans for improving services should have been assessed for all facilities, so that facilities offering weak quality PMTCT services would receive additional prioritization for investment.

Broader implications

Though MEASURE Evaluation developed this dual set of indices specifically to prioritize facilities for PMTCT services, the concept of using summary scores on readiness and need, to prioritize facilities for a planned investment, is relevant to other HIV services as well as other health facility systems and services. The comprehensiveness of the readiness/need indices suggests broader uses.

A key strength of a summary score for indices composed of multiple elements is its ability to provide a rigorous and transparent method for presenting a multifaceted picture of a facility, incorporating many diverse aspects of a system or service. The summary scoring system pulls together the results across multiple indicators, that are not consistently strong or weak across facilities, to provide a single measure. This provides a more complete understanding of the overall status of the services being measured. The authors believe that this approach is superior to the approach of selecting a few key indicators from many important components, which commonly results in prioritizing a few easily quantifiable items. It ensures that all aspects of quality that programs invest in are included in the measure, reducing the risk of equally important but non-prioritized items being de-emphasized.

Using a rigorous approach to prioritize facilities can improve the cost-effective allocation of resources, and may support planners when external pressures for prioritizing investments in services are exerted. Although, realistically, there will always be compromise, hard facts that can be shown to be based on an objective and comprehensive picture may be more convincing when justifying decisions taken.

An additional potential use for this methodology is to monitor changes in facilities over time. The initial score can serve as a baseline, and repeat measures recorded, facilitating measurement of overall change over time. This easily understood transparent method for evaluating facilities can also be used to justify additional investments in improving systems and programs.

The researchers who collaborated on this novel approach for prioritizing resources invite comment from, and replication by, other researchers, including researchers in related health fields.

Conclusions

In a resource constraint environment, budgets for public health programs, policy-makers, donors, program planners and managers face the task of ensuring that scarce funds go as far as possible to achieve desired health outcomes and meet global health goals, including eliminating mother-to-child transmission of HIV. Decentralizing HIV and AIDS services to primary health care levels is a key strategy for improving the coverage of these services, but this may require efficient allocation of resources.

The objective of this article is to share a methodology used to prioritize facilities for PMTCT service expansion. In two indices—one for facility readiness and one for local need—multiple variables are measured and then weighted in accordance with their relative importance to readiness, quality of care, supportive environment, need and feasibility of service expansion. Summarizing findings for multiple important indicators into one score, provides a simplified way to prioritize facilities for investments and for justifying decisions if needed. If the data are periodically collected, this also provides a measure for evaluating change over time in a comprehensive context.

This particular summary score specifically needed to take into account both need for services and the preparedness of each facility to offer a given service. The indices were constructed with a weighting system designed to balance readiness and need. A review of the results showed that the objectives of the prioritization exercise seemed to have been achieved. Other researchers could potentially revise the scoring mechanism depending on the objective of the score to be used.

A review of the scoring methods (after the fact) identified areas for future improvement. However, the concept of developing summary scores for an inclusive set of variables representing the complex range of elements that influence service quality, rather than focusing on a limited set of variables, appears to be a sound approach for identifying the most eligible of a group of facilities for investment. The authors believe that this methodology would be adaptable in uses beyond PMTCT and HIV, and encourage other researchers to replicate it.

The limited use of the information (which was presented in assessment reports to address specific information requests by the stakeholders) thus far reinforces the need to include not only dissemination of results, but the use of participatory approach in the interpretation of results with engagement of program decision-makers and planners in order to help incorporate it into planning. Developing more user-friendly presentations of information, so that the results at the summary level and how these link with specific service components are more self-evident, may improve the understanding, and therefore utilization of summary scores for planning (and monitoring change over time).

The consensus of stakeholders was that when considering the selection of facilities either to commence or expand existing health services, the needs environment of the facility is as important as the readiness conditions of the facility. The dual indices of needs and readiness is a potentially pragmatic tool in health programs to help policy makers and manager, select facilities that are ready to scale-up or commence PMTCT of HIV service delivery to maximize returns on investment as well as for a given population. The conceptual structure of the indices has broad applicability to health service planning problems beyond HIV and AIDS services.