Primary health care performance: a scoping review of the current state of measurement in Africa.

INTRODUCTION: Countries with strong primary healthcare (PHC) report better health outcomes, fewer hospital admissions and lower expenditure. People-centred care that delivers essential elements of primary care (PC) leads to improved health outcomes and reduced costs and disparities. Such outcomes underscore the need for validated instruments that measure the extent to which essential, evidence-based features of PC are available and applied to users; and to ensure quality care and provider accountability.
METHODS: A systematic scoping review method was used to identify peer-reviewed African studies and grey literature on PC performance measurement. The service delivery dimension in the Primary Healthcare Performance Initiative conceptual framework was used to identify key measurable components of PC.
RESULTS: The review identified 19 African studies and reports that address measuring elements of PC performance. 13 studies included eight nationally validated performance measuring instruments. Of the eight, the South African and Malawian versions of Primary Care Assessment Tool measured service delivery comprehensively and involved PC user, provider and manager stakeholders.
CONCLUSION: 40 years after Alma Ata and despite strong evidence for people-centred care, significant gaps remain regarding use of validated instruments to measure PC performance in Africa; few validated instruments have been used. Agreement on indicators, fit-for-purpose validated instruments and harmonising existing instruments is needed. Rigorous performance-based research is necessary to inform policy, resource allocation, practice and health worker training; and to ensure access to high quality care in a universal health coverage (UHC) system-research with potential to promote socially responsive, accountable PHC in the true spirit of the Alma Ata and Astana Declarations. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Measurement of primary care (PC) performance globally informs and guides primary healthcare (PHC) systems strengthening.

There is however scanty information about PC performance measurement in Africa despite the commitment to strengthen PHC and universal health coverage.

Little is being done to measure PC service delivery in Africa using validated instruments.

Few validated tools have been used to date and no research has been conducted to determine which instruments are best suited to the African context.

Validated PC performance measures are needed in Africa to guide, support and evaluate efforts to improve access to quality PC.

Research is needed in Africa to evaluate instruments used and/or develop new measures to provide evidence necessary to strengthen PC and ensure universal health coverage.

Introduction

The primary goal of a health system is to optimise health.1 Primary healthcare (PHC) is the backbone of a healthcare system and central to improving accountability in health service delivery.2 Countries with strong PHC report lower health expenditure, better outcomes and fewer hospital admissions.3 4 In 2008 the WHO call for PHC reform recognised the need to reorganise primary care (PC) around people’s needs and expectations with potential to reduce the occurrence of common illnesses by up to 70%.5 The emphasis was underscored in 2017.6 As a core dimension of PHC, comprehensive people-centred PC can reduce the burden of disease by improving opportunistic screening, disease prevention and health promotion.7 8 Comprehensive PC is in line with the Alma Ata and Astana Declarations; with universally accepted definitions of PC; and with universal principles of family medicine and their practice in Africa.9–13 Given high-level support for PC and growing evidence of its contribution to improving health outcomes, evaluating PC performance using validated instruments should be a key activity of PHC strengthening and reform.

Kidd et al 13 note the substantial evidence for PC as central to strengthening health systems. When practiced as a multidisciplinary team and population-based approach, person-centred PC as the entry point to healthcare meets most healthcare needs including health promotion and disease prevention. Evidence-based PC delivery involves key elements shown to be essential for cost-effective PC–namely access to PC as first contact; continuity; comprehensive; coordinated; family-orientated and community-orientated care.13–15 When these elements are available to PC users and applied in a person-centred approach to care, outcomes include improved health and satisfaction, and reduced costs and health inequities.3 Evidence in favour of such outcomes underscores the need to measure PC using appropriately validated instruments.

This scoping review aimed to determine the current state of PC performance measurement in Africa and identify knowledge gaps in PC service delivery that can be addressed by research.

Methods

Fifteen members of the Primafamed PHC African Network,16 faculty from university departments of family medicine in 11 sub-Saharan African countries and Belgium met over 3 days in response to a call for scoping reviews identifying gaps in PC research in Africa. Participants self-selected themselves to three teams, each tasked with identifying topics to review. Our five-member team used the nominal group technique method17 to generate and prioritise a list of PC research challenges in Africa. The need for validated, contextualised instruments to measure PC performance obtained the highest score by group consensus. This review therefore addresses the question: ‘What is already known about valid tools or measures or instruments to measure primary care performance in Africa?’ We used the staged scoping review method recommended by Peters et al 18 to identify, extract and categorise relevant data from peer-reviewed and grey literature on instruments used to measure PC performance in Africa. We used the Preferred ReportingItems for Systematic Reviews and Meta-Analyses-Extensionfor Scoping Reviews checklist to guide reporting the results.19

Definitions

The focus of the review is PC measurement. The terms ‘primary care’ and ‘primary healthcare’ and their definitions are frequently used interchangeably in the literature.20 21 This presents a challenge when searching the literature for studies focusing specifically on PC performance measurement that is, on essential elements of personal PC as a dimension of the wider PHC system including population-based approaches to healthcare.22 Definitions used to distinguish PC and PHC are available in Appendix A. We used the five-dimension Primary Healthcare Performance Initiative (PHCPI) conceptual framework (Appendix B) to assist making the distinction by locating the study in the service delivery dimension.23 Service delivery domains–namely population health management, PC facility organisation and management, access, availability of effective PHC services and high quality PHC–and their respective subdomains are in line with the essential elements of PC noted above.2 4 15

Literature search

Step 1

Given the breadth of PC, we conducted an initial search of electronic databases using iterations of a basic search strategy comprising PC MeSH terms and keywords to identify literature that addressed the review question. The initial results were then used to design a stronger search strategy.

Step 2

We elicited the assistance of experienced reference librarians to design a more focused, unified PC search strategy using the Pubmed search builder (Appendix C). Additional MeSH terms and keywords from the initial search results meeting our criteria were included. The Pubmed strategy was adapted to search other databases. We searched in Pubmed; Web of Science; Africa-wide (EBSCOhost); Scopus; Hinari and CINAHL. Each member of the team was assigned a database(s) and applied the inclusion and exclusion criteria (table 1) to the titles and abstracts of the search results. The full texts were examined for more information and to assist with inclusion/exclusion if the abstracts were insufficient. We sought to identify grey literature by searching in Google Scholar, Sabinet and Google. All results selected were submitted to the team leader (GB) who removed duplicates and captured key information in a spreadsheet including titles and abstracts.

Table 1

Inclusion and exclusion criteria

		Inclusion	Exclusion
1	Sources	Published or unpublished peer-reviewed studies and reports; full text available online	Full text not available online or not available through university subscribed databases
2	Focus	Studies or reports of instruments or measures of primary care performance that is, performance on PHCPI service delivery items. Search conducted for period 01/01/2003–30/06/2019	Performance measures or studies limited to care of specific conditions, functional health status, age, gender, or levels of care other than PHC
3	Context	African context	Non-African
4	Language	English	Non-English

PHC, Primary healthcare; PHCPI, Primary healthcare performance initiative.

Step 3

Each team member examined and applied the selection criteria to the results in the spreadsheet compiled in Step 2, adding her/his comments to justify inclusion/exclusion. Reference lists and reports were examined for any African studies not identified in Step 2. The unified search notwithstanding, results included studies that focused only on PHC dimensions other than service delivery or other PHC dimensions and selected service delivery domains. Identifying research focusing specifically on PC—especially those using validated performance measuring instruments—was problematical. We therefore developed a rating scale for relevance to determine which studies to include or exclude at this stage: High Relevance Grade 1 (HR-1)—African studies using validated instruments to measure PC performance comprehensively that is measuring performance on most PHCPI service delivery domains (C1–C5); and High Relevance Grade 2 (HR-2)—African studies measuring performance on a single or limited number of service delivery domains that is, not comprehensively, and/or focusing largely on other PHCPI dimensions. The final rating for each study was decided on by consensus after each team member examined the full texts of studies in the spreadsheet along with the others’ ratings. In this way studies with full texts available were rated for their relevance to the review question. Studies that did not achieve team consensus were discussed in conference calls. Full texts of the final selection were examined further to extract additional data for entry in a standardised template (table 2).

Table 2

Summary of 19 studies that met the criteria (ordered alphabetically and publication date)

Reference	Country	Type of study	Study aim/ purpose	Methods	Instrument(s)	Scope: what does it measure?*	Key findings
Bresick et al. Evaluating performance of South African primary care24	South Africa	PC performance measurement	Cross-sectional descriptive survey of South African PC performance in seven provinces	PC performance determined by user, provider & manager questionnaires	South African Primary Care Assessment Tool (ZA PCAT)	PC performance as determined by users; providers & managers; performance on eight core domains; 11 sub-domains; 115 items (user PCAT) (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e)	Users rated accessibility, continuity and community orientation as poor; coordination of information and family-centredness inadequate; and comprehensiveness, coordination of care, cultural competency and PHC team availability adequate. Managers and providers significantly more optimistic about performance than users. Strengthening PC and improving users’ experience should focus on access, continuity and community-orientated PC; and reduce gaps between user experience and provider assessment of performance
Renggli et al Toward improved health service quality in Tanzania25	Tanzania	Instrument analysis and performance measurement	Analyse an electronic tool developed to assess and monitor PHC quality in Tanzania	Quantitative and qualitative methods used to investigate tool appropriateness.	Tanzanian electronic Tool to Improve Quality of Healthcare (e-TIQH)	A range of quality dimensions; assessment focused on processes and structural adequacy of healthcare (B2;C2b; C4b,c)	Robustness checks confirmed e-TIQH’s ability to score facilities and group indicators into different dimensions. Findings show accuracy measuring and monitoring quality and steering improvement measures. A feasible option for routine measures of quality and a foundation to improve services
Dullie et al. Develop-ment and validation of Malawian PCAT26	Malawi	Validation of PCAT for use in Malawi	Develop the PCAT-Mw; evaluate reliability and validity; measure PC performance for adult users	Forward and backward translation of ZA-PCAT before cross-sectional survey to test validity and intra-rater and inter-rater reliability	PCAT-Mw	Measures five PC domains (first contact access, continuity, comprehensiveness, coordination and community orientation); 7 subdomains and 29 items. (C1a, b; C3c, C4d, C5a, b, c, d, e; E1)	Comprehensive metric analyses showed PCAT-Mw reliability and validity in assessing core PC domains based on adult users’ experience. PCAT-Mw useful for Malawian health service research
Dullie et al. Quality of primary care from patients' perspective27	Malawi	Baseline audit of PC in a Malawian rural health district	Evaluate PC performance based on users experience of services and association with socio-demographic characteristics	Cross sectional survey to determine mean scores for total PC and PC domains; linear regression to determine socio-demographic associations	PCAT-Mw	Measures PC organisation and performance on access; continuity; comprehensiveness of services available and provided; and community orientation (C1a, b; C3c, C4d, C5.a, b, c, d, e; E1)	Partial functioning of Malawian PC. Performance is weak for access, relational continuity and comprehensiveness of services available. Sex, geographical location, self-rated health status, duration of association with facility and facility affiliation associated with user assessment of PC
Macarayan et al. Assessment of quality of primary care with facility surveys38	Ethiopia, Kenya, Malawi, Namibia, Nepal, Rwanda, Senegal, Tanzania, Uganda	Performance assessment	Assess whether existing facility surveys capture relevant aspects of PC performance; report on PC data quality	Selected data analysed from Service Provision Assessment (SPA) surveys	Data analysis using survey data. Three indicators selected from SPA surveys: competent systems; evidence-based care; and user experience	Measures aspects of user experience, health outcomes and processes such as timely action, choice of provider, affordability, ease of use, dignity, privacy, non-discrimination, autonomy and confidentiality (C3a,c; C4b,d;C5e; E1,2)	Gaps in measuring user experience, health outcomes and processes—including timely action, choice of provider, affordability, ease of use, dignity, privacy, autonomy and confidentiality. No information available on care competence outside maternal and child health. PC quality scored low
von Pressentin et al. Influence of Family Physicians in a district health system28	South Africa	Impact study	Determine the influence of family physicians on PC performance to assist human resource planning	Cross-sectional study comparing district hospitals & PC centres with and without family physicians in South Africa	ZA PCAT	Evaluate impact of family physicians on South African district health system. (C1;B; C5a,b,c,d,e; E2)	District hospitals with family physicians scored better on indicators of health system performance and clinical processes. In contrast PC centres with family physicians had significantly poorer mean scores for continuity and coordination of care
Wisniewski et al. Comparison of objective measures and patients’ perceptions of quality of services39	Democratic Republic of Congo	Cross-sectional; visit-level data from household & government facility survey	Compare objective measures with user perceptions of health service quality	Household & government facility surveys	Data analysis using survey data	Measures user perception of quality related to availability of drugs and equipment; patient-centeredness; and safety compared with objective measures of quality (B1;C4e; C5e)	User assessment of quality inaccurate; expectations and prior experience impacted perceptions. Future research should examine whether improving user knowledge of what to expect and transparency of facility quality data improves accuracy of user assessments
Hunter et al. The Ideal Clinic in South Africa29	South Africa	Report	Report progress on implement-tation research model	211-item ICRM checklist of performance indicators administered to PC facilities to measure compliance	Standardised checklist (ICRM) comprising five domains; 10 components; 32 sub-components each comprising a number of items	Measures performance on range of dimensions—mainly operational and infrastructural components that do not map easily onto the PHCPI framework for example, administration includes signage, staff identity and dress codes; record archival & retrieval. (A1c, A3a; B1,2,3 4; C3b, C4b,c,e)	Progress shown; SA Health Department staff described the initiative’s progress and ongoing challenges against a 2013 baseline audit
Mukiapini et al. Baseline measure of PHC team function and overall PHC performance30	South Africa	Baseline audit of PC and team function	Measure baseline PHC team effectiveness and overall PHC performance	Cross-sectional study using the Nominal Group Technique and ZA PCAT to assess team function and baseline performance	ZA PCAT user, provider & manager instruments	PC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a, b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b, c,d,e	PHC team members rated team as well-functioning (70% agreement on 7-item PHC team function sub-domain) yet NGT method revealed communication and leadership as key challenges to effective function; baseline results similar to Western Cape PCAT study32
Ogaji DS et al. Development and validation of the PES for PHC31	Nigeria	Validation of a patient measure of PHC	Develop and validate a patient evaluation scale (PES) for use in Nigerian PHC	Iterative process to develop and validate questionnaire with users at PC centres used literature reviews; user and expert interviews; cognitive testing; cross-sectional surveys	Full version of patient evaluation scale (PES)	Items measure facility, organisation and health services provided. (B2;C1c; C3a,c E2)	The PES and PES-SF (full and shortened versions; 27 and 18 items respectively) may be useful in practice and research aimed at patient evaluation, comparing performance, understanding trends and testing patient-focused improvements in PHC in Nigeria
Ogaji. Questionnaires for Patient Evaluation of PHC: a systematic review41	Nigeria	Systematic Review	Systematic search for patient questionnaires evaluating PC and implications for practice	Systematic Review	Patient questionnaires evaluating PC	Questionnaires include patient evaluation of dimension C domains and other dimensions	Review of instruments needed for patient evaluation of PC and implications for practice in Nigeria
Bresick et al. Western Cape PCAT study—PC organisation and performance32	South Africa	PC performance measurement	Baseline measure of comprehensive PC centres’ performance & organisation	ZA PCAT administered to 1432 users; 100 clinicians; 64 managers; 13 PC facilities; 10 sub-districts	ZA PCAT user, provider & manager instruments	PC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b,c,d,e)	Managers scored access, family centeredness and cultural competence significantly lower than practitioners; users scored comprehensive services available, services provided, and community orientation significantly lower than practitioners and managers
Kress et al. Assessment of PHC performance using the PHCPI framework40	Nigeria	PHC system performance assessment	Assess PHC performance and causes of under-performance	Analysis of facility data from World Bank Service Delivery Indicators Survey	PHCPI Conceptual Framework used as a tool	Examined all PHCPI dimensions. Multiple data collection methods and tools used. (A;B;C;D;E)	PHC performance hindered by segmented supply chains; lack of financial access, infra-structure, drugs, equipment and vaccines at facility level; poor health worker performance. Financing and governance identified as root causes of dysfunction
Alhassan et al. Comparison of perceived and technical healthcare quality in PHC facilities33	Ghana	Survey involving users & staff (household & PHC facility)	Compare perceived & technical healthcare quality in PHC facilities; determine if perceptions are associated with technical quality proxies in health facilities	Cross-sectional surveys examine differences in user and staff perceptions on quality healthcare; and determine technical quality of care	SafeCare Essentials tool (Household & staff question-naires)	(A1b, c; B2; C4b, e; D1 a, b, c, d, e, f)	Negative association between technical quality & user-perceived quality care. Significant staff-client differences in all healthcare quality proxies suggesting information imbalance between clients & service providers. Improving technical quality alone will not lead to better user-perceived quality care and willingness to use accredited health facilities
Bresick et al. Adaptation and cross-cultural validation of the United States PCAT34	South Africa	Validation of measurement tool	Adapt and cross-culturally validate original adult expanded USA PCAT (AE) for use in South Africa	Evaluation of item phrasing; modified Delphi & Nominal Group Technique methods with expert panellists; user focus group to obtain consensus on content relevant to South African users	Validated ZA PCAT—PC adult user, provider & manager versions	Measures PC performance on access, service availability, continuity, coordination, comprehensiveness, community and family orientated care; cultural sensitivity; and team function (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e)	Consensus on inclusion of all original PCAT domains. PHC team availability and function added as a new domain. Three of 95 original items achieved <70% agreement and excluded as not relevant to SA; 19 new items added. A few items needed rephrasing for local use. ZA PCAT isi-Xhosa and Afrikaans translations produced
Babatunde et al. PHC consumers' perception of quality of care and its determinants42	Nigeria	Cross-sectional descriptive study	Measure PHC user satisfaction and assess user perception of quality of care and determinants	Semi-structured interviewer- administered questionnaire adapted from QUOTE (Quality of care from patient's eyes) (CMAI 2004)	Adapted semistructured questionnaire	User perception of quality of care including greeting patients; respect for patients’ opinion; whether satisfied with time spent with doctor, with treatment & care received; and uninterrupted consultations (C3.a-c)	Waiting time had inverse relationship (not statistically significant) with level of satisfaction. Perception of care was lowest in domains of privacy and respect for patients' opinion
Phaswana et al. Patient satisfaction with PHC services in a selected district35	South Africa	Cross-sectional descriptive study	Measure patient satisfaction with PHC services in a rural district	Patient survey using adaptation of a validated user satisfaction questionnaire	Adapted Health System Trust (2004) Client Satisfaction Survey Questionnaire	Measured PC user satisfaction on eight domains: access, empathy, referral, general satisfaction, service standards, reliability, health promotion, tangibles, assurance. (elements of C2–C5)	Regarding access to services: 3/4 women felt they could reach clinic by phone, get appointments, language not a barrier, treated the same day without being turned away. for 7/10 journey takes longer than 1 hour and cost more than ZAR 10
Peltzer et al. Use of the European Task Force for Patient Evaluation of General Practice (EUROPEP)36	South Africa	EUROPEP psychometric property assessment for SA use	To examine psychometric properties of South African (isiXhosa) version of the EUROPEP	Validation process described factor analysis done on data collected from large patient sample using the EUROPEP tool	EUROPEP	Patient evaluation of provider competence; patient-provider communication; and practice organisation (C2b; C4b;C5e)	Exploratory factor analysis identified three subdimensions: medical technical content of care, patient–provider communication, and practice organisation. Reliability of items and sub-scales was high. The EUROPEP can be a valuable instrument to assess patients' evaluations of PHC in South Africa
Chukwuani et al. A baseline survey of the PHC system43	Nigeria	Baseline survey of PHC	Audit PC operations; measure community perceptions and expectations of services; develop performance indicators; obtain baseline data for policy and planning	Rapid health facility operations audit and community survey	Structured questionnaires to collect qualitative and quantitative data	Evaluate stewardship; service provision; administrative and financial management; measure user utilisation of health services, perception of service delivery and healthcare financing (A1.b, A2.b, C2a, C2b,C4b, C4c)	Lack of operational efficiency in most facilities; poor maintenance; services needed not provided; not enough skilled health workers; poor service provision. Access to essential drugs a key indicator for community evaluation of performance. Poor funding, management and infrastructure. Operations lack a budget, financial management system and policy on resource generation. Low utilisation precludes efficient PHC

*See table 3: PHCPI dimensions

ICRM, Ideal Clinic Realisation and Maintenance; PC, primary care; PCAT-Mw, Malawian South African Primary Care Assessment Tool; PHC, primary healthcare; PHCPI, primary healthcare performance initiative.

Table 3

Validated instruments: PHCPI dimensions covered

PHCPI dimensions	Validated instruments
	ZA PCAT	PCAT-Mw	PES	SafeCare Essentials tool	ICRM Components	EUROPEP	e-TIQH	Client Satisfaction Survey
A—System	(A1b,c)*(A3a,b)			A1b, c	A1c, A3a
B—Inputs	(B4)		B2	B2	B1, B2, B3, B4, B5		B2
C—Service delivery	C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e†	C1a, b; C3c, C4d, C5.a, b, c, d, e	C1c, C2b, C3a, C3c	C4b, e; D1 a, b, c, d, e, f	C3b, C4a, C4b, C4c, C4e	C2b; C4b; C5e	C2b; C4b,c	elements of C2–C5
D—Outputs	D1a,b,(c),(d),(e),f;	D1a,b,(c),(d),(e),f
E—Outcomes	E1,2	E2	E2

*ZA PCAT bracketed domains and subdomains indicate some coverage at the level of personal primary care; not at a wider community or district health system level - for example, B4 where the ZA PCAT assesses PHC Team member (workforce) availability at primary care clinic level.

†See appendix B or https://improvingphc.org/phcpi-conceptual-framework for domain names A1-3; B1-5; C1-5; D1; E1-5.

e-TIQH, electronic Tool to Improve Quality of Healthcare; EUROPEP, European Task Force for Patient Evaluation of General Practice; ICRM, Ideal Clinic Realisation and Maintenance; PCAT-Mw, Malawian South African Primary Care Assessment Tool; PES, Patient Evaluation Scale; PHPCI, Primary Healthcare Performance Initiative; ZA PCAT, South African Primary Care Assessment Tool.

Patient and Public Involvement

No patients nor members of the public were involved in the review.

Results

The databases search yielded 5215 peer-reviewed studies (figure 1). At this stage a newly published study24 not yet indexed in Medline was added; it met the HR-1 rating. All titles were screened; abstracts of titles that screened positive or uncertain were examined. After removing duplicates from the results submitted by each team member, 86 results remained in the spreadsheet. Sixteen16 grey literature results were added giving a total of 102 results identified as potentially eligible for further examination. Two results were excluded because the full texts were unavailable online. The full texts of the remaining 100 results were examined by team members as described in Step 3. HR-1 and HR-2 rated results were combined. Team consensus was achieved on 19 results as meeting the inclusion criteria (table 2)—18 published, peer-reviewed studies and one published, peer-reviewed report.

Figure 1

Flow diagram.

Thirteen studies24–36 addressed eight nationally validated PC performance instruments—namely the South African and Malawian versions of the Primary Care Assessment Tool (ZA PCAT and PCAT-Mw respectively)26 34; the electronic Tool to Improve Quality of Healthcare (e-TIQH) in Tanzania25; the Nigerian Patient Evaluation Scale (PES)31; the South African Ideal Clinic Realisation and Maintenance (ICRM) instrument29; SafeCare Essentials tool33; the European Task Force for Patient Evaluation of General Practice (EUROPEP)37; and the Client Satisfaction Survey Questionnaire.35 (Information on instruments is summarised in Appendix D uploaded as a supplementary file.) Seven of the 13 validated tools studies were PCAT studies—five conducted in South Africa and two in Malawi. The ZA PCAT, PCAT-Mw and PES studies included the theoretical basis and methods used to validate them in the African context. Table 3 compares the validated instruments against the PHCPI dimensions and domains covered and shows the dominant dimensions in each case. As is evident in table 2, most of the studies assessed performance on more than one PHCPI dimension. The remaining six of the 19 results focused largely on measuring one or more dimensions in the PHCPI framework in addition to service delivery. Three3 of the six studies used data analysis methods to analyse existing data from household surveys, WHO health accounts and other sources.38–40 The remaining three studies41–43 concerned patient questionnaires and qualitative methods such as focus groups to determine users’ evaluation or satisfaction with PHC services or their perceptions and expectations of services. They include a systematic review of patient questionnaires evaluating PC in Nigeria.41 These studies are included because they mention known tools which were adapted although they fail to describe how these were adapted and whether they were validated. Five studies entered in the spreadsheet in Step 2 were excluded because information on the origin or local development of their user survey questionnaires (for use in local communities) was not provided.

Discussion

The review aimed to document the current state of PC performance measurement in Africa and to identify gaps in knowledge and research. In order to manage the challenge of the interchangeable use of ‘primary care’ and ‘primary healthcare’ and their definitions in the literature, we chose the service delivery dimension of the PHCPI conceptual framework to define and situate PC within the PHC system. Notwithstanding efforts to limit the search to PC (results therefore do not reflect the range of PHC performance research in Africa), it continued to yield many studies addressing other PHC dimensions. Each selected study measured at least one service delivery subdomain. Few studies measured performance on most service delivery domains and subdomains (table 3) despite global calls and strong evidence supporting PC.

Eight nationally validated instruments (table 3) measuring PC performance in Africa were identified; five—namely the PES, e-TIQH, SafeCare Essentials, ICRM and Client Satisfaction Survey Questionnaire—were developed in Africa. The ZA-PCAT, PCAT-Mw and EUROPEP—developed in the USA and Europe respectively—were cross-culturally adapted and validated. Only the e-TIQH, ZA PCAT, Malawian PCAT, Nigerian PES and EUROPEP studies described the validation processes followed.

Regarding coverage of service delivery domains, the PCAT measured PC performance on more domains and subdomains than other instruments (table 3)—that is, measured PC performance comprehensively—and involved user, provider and manager stakeholders with potential therefore to address demand and supply performance features. Only the PCAT and EUROPEP are among the six validated instruments selected in a review by Haggerty et al.44 The PCAT also features in a literature review on PC assessment tools45 and in a paper emphasising the need for internationally validated instruments and indicators that ‘measure safety and quality of PC (and) integration with the rest of health system and workforce development’.13 Seven of the 13 studies on validated instruments focused on the PCAT compared with one paper each for the other seven validated instruments.

The Tanzanian-developed e-TIQH25 measures performance on 6 PHC components (Appendix D) addressing processes and structural adequacy of healthcare—but configured differently to PHCPI dimensions. Of note is the use of the e-TIQH to provide service providers and managers with baseline performance data and to measure impact afterward. The study illustrates how a performance measuring instrument can be used as an intervention to influence practice positively after feedback is given directly to providers and managers. Other comprehensive instruments listed in table 3 could be used to similar effect.

The PES (full version), developed and validated for use in Nigeria, and the EUROPEP, developed in Europe and validated for use in South Africa, enable user evaluation of PC. The PES enables patients to evaluate elements of facility organisation and health services provided (table 2); the EUROPEP enables patient evaluation of provider competence, patient–provider communication and practice organisation. The EUROPEP study37 assessed the psychometric properties of the South African (isiXhosa) version.

The Client Satisfaction Survey study35 illustrates the need for agreement on indicators, definitions and instruments to assess PC performance13 46—in this case from the user perspective. It measures patient satisfaction on ‘eight domains’ of performance namely access, empathy, referral, general satisfaction, service standards, reliability, health promotion, tangibles and assurance. As with other studies, it is difficult to map these domains onto the PHCPI service delivery dimension. For example, ‘tangibles’ comprises 10 questions covering service infrastructure and management that is, PHCPI B and C dimensions; ‘assurance’ comprises questions regarding provider competence (C4b), coordination (C5c) and information on prevention and promotion (C5d). The South African ICRM instrument is a standardised checklist-based audit tool focusing mainly on PC facility compliance with operational and structural components rather than PC process.

Three studies applied data analysis methods38–40 to assess performance using existing survey data from various sources. A Nigerian study40 analysed data from health and household surveys, WHO health accounts and workforce data. Different data analysis methods were used to assess performance on at least one domain of each of the five PHCPI dimensions. These methods contrast with instruments noted above that can potentially be used by PC and district health services stakeholders—including users, providers and managers. On the other hand the five-dimension coverage in the Nigerian study points to shortcomings of other instruments in table 3. Despite the ZA PCAT’s relatively comprehensive measure of service delivery, coverage of C1 and C2 domains is minimal.

Data synthesis

What do these findings mean? Thematic categorisation of the results is problematical and perhaps premature given the paucity of studies meeting the review criteria. At this stage of African PC performance research there seems little value categorising instruments according to whether nationally or not nationally developed; whether nationally validated or not; measuring performance comprehensively or selectively; user-survey questionnaires only or user-survey instruments and/or collecting data from other sources. Similarly, categorising results by method for example, desktop methods analysing existing data or using instruments to collect data de novo (with potential to involve key PC stakeholders). Attempting categorisation may however be useful as an exercise to identify gaps in PC performance research for example, regarding instruments required for relevant and reliable data needed to ensure high quality PHC in Africa.

The need for collaboration and agreement on indicators and instruments: The review findings reflect a lack of collaboration and agreement among researchers regarding the most appropriate indicators and instruments needed to evaluate African PC performance—highlighting a need for a unified approach in order to address knowledge gaps comprehensively. Studies failed to address the need for consensus on performance indicators and instruments most suited to African PC performance measurement—reflecting a global gap in health systems research.13 46 The difficulty mapping performance indicators used in most studies onto a common framework such as the PHCPI or comparing study findings, underscore the need for agreement on indicators and their definitions within a unified approach. Few studies focused on service delivery and none covered the C-domains sufficiently comprehensively—further gaps in African PC performance research. Knowledge gaps and research questions include which existing methods and instruments are fit-for-purpose in Africa and which need validating; whether there is a need to develop new instruments specifically for the African context and why; and whether instruments validated nationally (in Africa) can be cross-culturally adapted to measure PC performance effectively in other African states and enable comparison of local districts and African countries.

The fragmentation of instruments and research efforts to measure performance is not limited to Africa. Studies in high-income countries46–48 note that many validated instruments available to measure PHC do not all address important attributes nor the same range. Such fragmentation is emphasised by several studies (excluded in Methods Step 3) that sought to measure performance using locally developed questionnaires to survey patient satisfaction or perceptions of services. They failed to provide the necessary information on methods and instruments used; to describe how or where the instruments were developed; or how they were ‘pretested’. These findings highlight a need in Africa to improve the rigour of instrument development and use, and to reduce research fragmentation—especially in a resource-scarce context. The studies identified do not inform decision-making regarding evidence required to achieve agreement on instruments most suited to measure PC performance; nor help determine what constitutes cost-effective PC in the African context; nor do they discuss how the absence of agreement on indicators, instruments and methods should be addressed. These findings suggest an urgent need for a unified, African PC performance research framework.

The need for comprehensive evaluation of performance: The role of user surveys is supported by evidence that patient-centred care and patient feedback improve health outcomes. However, the studies using the e-TIQH, SafeCare Essentials and ZA PCAT instruments show the value of sampling more than one source. Their results (Key findings in table 2) show significant discrepancies between staff and user evaluations of service performance—in effect questioning the value of results from one source only and suggesting a need for technically based as well as user experience-based assessments. Results obtained using the SafeCare Essentials tool show a negative association between technical quality and user-perceived quality of care and significant user-provider differences on all indicators measured. The ZA PCAT studies similarly show significant differences between users, service providers and managers on key elements of PC performance including community-orientated PC and comprehensiveness of services provided. The SafeCare Essentials study nevertheless emphasises user evaluation—‘Improving technical quality alone will not lead to better user-perceived quality care and willingness to utilise accredited health facilities’—important to note when considering methods and instruments to use.

The need for a skilled workforce: The Nigerian baseline study43 refers to a key component of performance when noting the impact of insufficient skilled health workers on the operational efficiency of PC facilities and their ability to provide services required. Studies primarily addressing performance on features of workforce provision at the upstream PHCPI input dimension level will have been largely missed by a search strategy focused on service delivery. The importance of a sufficient and adequately trained workforce for PC performance in all domains is important to note and highlights a shortcoming of a review limited to one PHC dimension. Cost-effective PC is dependent on upstream dimensions A and B. In similar vein such resources may be wasted, and their intended outcomes not realised if essential features of PC are not available to users and applied in their care. At the service delivery (C) level, the ICRM and PCAT instruments (table 3) address availability of sufficient (C4a) and appropriately skilled (C4b) providers.

In addition to the absence of a comprehensive discussion on indicators and instruments, the studies also failed to address the potential to link validated instruments and performance results with development and implementation of interventions to strengthen PC—and ultimately PHC. Instruments and methods such as the e-TIQH, SafeCare Essentials and PCAT that involve users, providers and managers have added value given the potential to initiate user-provider-manager stakeholder collaboration during data collection as well as intervention development and implementation. Such instruments could help identify common ground among key stakeholders despite different roles and responsibilities, thereby strengthening PHC.

Addressing the knowledge gaps and research questions highlighted should be integral to PHC strengthening and health sector reform in Africa. It is important to note however that change driven by political agendas and narrow, group-based interests may ignore evidence advocating genuine reform45—particularly when evidence challenges a biomedical, diseased-based approach to care at entry to the health system. Involving key stakeholders who collaborate when measuring PC performance and applying the findings may provide checks and balances needed to reduce the impact of group-based agendas.

Africa is a resource-scarce continent challenged by a quadruple disease burden and recalcitrantly poor health indices. A bottom-up approach to performance assessment may generate innovative, inexpensive and context-appropriate interventions required. Realising such goals is more likely if instruments measuring PC performance in the field are placed in the hands (following training) of district-level stakeholders including managers, providers and users. Other stakeholders include postgraduate-trained PC physicians (family physicians) specifically trained to deliver PC. It is here in the PHC system that PC physicians have their biggest impact—applying their clinical, public health and behavioural science expertise to ensure essential PC elements are available to users and applied in their care; a role within a multidisciplinary PHC team approach being promoted by the Primafamed PHC African Network promotes.16 28 PC physicians, nurse practitioners and other members of the PHC team should be trained (jointly) to use validated instruments measuring their performance.

Although this review focused on the PHCPI service delivery dimension, research aimed at developing a compendium of cross-culturally adapted and validated instruments should be considered—to select from as needed—to rigorously and comprehensively measure PHC performance in Africa across the five PHCPI dimensions. Wide stakeholder commitment and collaboration on performance evaluation should also consider developing instruments that are country-specific where necessary and yet enable comparison of findings and experiences13 within and between African countries. Such developments could stimulate cross-border collaboration to include development and implementation of interventions and strategies that strengthen PHC, thereby reducing wasteful duplication, fragmentation and cost. Such benefits may extend beyond Africa. Development of an Africa PCAT was first proposed at a Primafamed workshop in Nairobi in 2016 (Summary report, unpublished, of PCAT workshop by G. Bresick and K. von Pressentin). The Ibero-America (IA) PCAT is an example of a regional PCAT—the result of a collaborative research project involving Ibero-American countries.49 International PCAT researchers including researchers from Africa presented their research at the 2018 World National Colleges and Academies of Family Medicine/Primary Care conference in Seoul. Areas for collaboration identified included the need to adapt the original PCAT for universal use and to establish an international PCAT research forum. This review contributes to Africa’s role in research to improve and promote PC performance research and measurement globally.

Study limitations

Notwithstanding the case made for a specific PC focus, the review addresses a single dimension of the multi-dimensional PHC system. It was confounded by the interchangeable use of terms in the literature—hence the use of the PHCPI conceptual framework to reduce the impact of consequent challenges. The framework is however a recent development not yet widely accepted—as indicated by its use in only one result.40 Mapping indicators onto service delivery domains and subdomains (table 2 Scope column; and table 3) was loosely done and needs closer examination. Most studies failed to define performance indicators measured, confounding alignment of results with service delivery domains and underscoring the need for collaboration among PC researchers, service providers, managers and other key stakeholders. These and other factors—such as extracting reported data from English language papers only—add to the limitations of the review. We believe nevertheless that the results reflect a dearth of studies addressing PC performance measurement in Africa and highlight key gaps in an important area of health systems research.

Conclusion

Health systems research conducted in Africa to date has not addressed PC performance measurement adequately despite global calls for PHC reform; the pressing need for rigorous performance measurement to guide reform; and strong evidence that comprehensive, people-centred PC improves health outcomes at reduced cost. Insufficient attention is being given to use of validated instruments. Our review findings show low use of validated instruments and limited evidence to support validation of measurement properties (reliability, validity and acceptability) of most instruments used. Excluded studies included those addressing patient perceptions and experiences of PC using locally developed instruments that were poorly described and unvalidated—arguably a waste of scarce resources. Using validated instruments to measure performance is necessary for reliable data to inform PHC quality improvement23—for example, to guide development and implementation of appropriate interventions. The paucity of studies using such instruments to measure PC comprehensively should be addressed. The knowledge and research gaps highlighted are more glaring 40 years after the Alma Ata Declaration and 10 years after the WHO call for ‘PHC: now more than ever’ and similar initiatives. The global call for UHC further underscores the need to improve health service quality—fundamental to achieving UHC and its goal of ensuring access to quality healthcare for all.25 50 51 High quality research is required for evidenced-based data to justify directing scarce resources to strengthening PHC systems—particularly in Africa; and to support decision-making generally. Health services that cannot show evidence of valid performance appraisal will be increasingly difficult to justify.52–54 Research addressing PC performance may have potential to promote more socially responsive and accountable PHC in the true spirit of the Alma Ata and recent Astana Declarations.9 10