Evaluating the effect of ward-based outreach teams on primary healthcare performance in North West Province, South Africa: A plausibility design using routine data.

BACKGROUND: North West Province (NWP), South Africa, was an early adopter of the primary healthcare (PHC) ward-based outreach team (WBOT) strategy and has made considerable progress in implementing it. Given the interest in and expectations of greater investment in WBOTs, assessing their impact on and contribution to PHC outputs and health outcomes is becoming increasingly important.
OBJECTIVES: To describe the application of a plausibility evaluation design for assessing the contribution of WBOTs to PHC performance in NWP, comparing changes in coverage, utilisation and outcome indicators in facilities with and without WBOTs.
METHODS: Routine data from the District Health Information System on both WBOTs and PHC facilities for the period 2011/12 (prior to implementation) to 2014/15 (3 years after implementation began) were extracted. Analysis involved the following three steps: (i) selection of indicators sensitive to community-based action; (ii) data cleaning; and (iii) comparison of the degree of change in median indicator values between 2011/12 and 2014/15 in facilities with and without WBOTs (a difference-in-differences analysis).
RESULTS: Changes in indicator values in facilities were grouped into four categories: (i) indicators where there was greater (statistically significant) improvement in facilities with WBOTs (couple year protection rate, measles immunisation coverage in children aged <1 year, incidence of children aged <5 years with severe diarrhoea with dehydration); (ii) indicators that declined or worsened, but less so in facilities with WBOTs at statistically significant levels (antenatal first visits as a percentage of children born in that year, PHC utilisation rate of children aged <5 years); (iii) indicators that improved in all facilities with no significant difference between facilities with and without WBOTs (antenatal attendance before 20 weeks, prophylactic vitamin A coverage to children aged 12 - 59 months); and (iv) indicators that remained unchanged in all facilities (immunisation coverage in children aged <1 year, postnatal mother visits at 6 days, cervical cancer screening coverage in women aged ≥30 years, PHC utilisation rate of children aged ≥5 years).
CONCLUSION: Notwithstanding the limitations of routine data and the need to approach the findings with caution, this analysis suggests that WBOTs plausibly had some positive effects on the overall performance of the PHC system. We propose a methodology to monitor the performance of WBOTs using routine PHC indicators that programme managers could apply elsewhere.