Abortion metrics: a scoping review of abortion measures and indicators.

Consensus is lacking on the most appropriate indicators to document progress in safe abortion at programmatic and country level. We conducted a scoping review to provide an extensive summary of abortion indicators used over 10 years (2008-2018) to inform the debate on how progress in the provision and access to abortion care can be best captured. Documents were identified in PubMed and Popline and supplemented by materials identified on major non-governmental organisation websites. We screened 1999 abstracts and seven additional relevant documents. Ultimately, we extracted information on 792 indicators from 142 documents. Using a conceptual framework developed inductively, we grouped indicators into seven domains (social and policy context, abortion access and availability, abortion prevalence and incidence, abortion care, abortion outcomes, abortion impact and characteristics of women) and 40 subdomains. Indicators of access and availability and of the provision of abortion care were the most common. Indicators of outcomes were fewer and focused on physical health, with few measures of psychological well-being and no measures of quality of life or functioning. Similarly, there were few indicators attempting to measure the context, including beliefs and social attitudes at the population level. Most indicators used special studies either in facilities or at population level. The list of indicators (in online supplemental appendix) is an extensive resource for the design of monitoring and evaluation plans of abortion programmes. The large number indicators, many specific to one source only and with similar concepts measured in a multitude of ways, suggest the need for standardisation. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Recent decades have seen progress in the delivery of safe abortion services.

However, consensus is lacking on the most appropriate measures or indicators to document progress at programmatic and country level.

A recent systematic review has identified 75 indicators of abortion quality subdivided into structure, process and outcomes indicator.

We extracted information on 792 indicators, many specific to one source only.

Indicators of access and availability and of the provision of abortion care were the most common.

Indicators of outcomes focused on physical health, with few measures of psychological well-being and no measures of quality of life or functioning.

There were few indicators attempting to measure the context, including beliefs and social attitudes at the population level.

The majority of indicators used special studies either in facilities or at population level.

The large number indicators, with similar concepts measured in a multitude of ways, suggest the need for standardisation.

Areas of further development include population-based indicators of effective coverage and standardised indicators of well-being and respectful care.

Introduction

Improving women’s reproductive health and rights is essential to achieving the sustainable development goals. Recent decades have seen progress in the delivery of safe abortion services. The development of medical abortion, for example, is a relatively recent technology which can allow women to access abortion at home or in primary healthcare services. However, many social, economic, health service and policy factors continue to affect women’s ease of access to comprehensive abortion services. Modelled estimates suggest that 56 million women had an abortion annually between 2010 and 2014, and 25 million of these were classified as less or least safe.1 Abortion complications therefore continue to be an important cause of maternal death globally, with 22 800 women dying from these every year.2

Researchers and evaluators support public health efforts to improve women’s reproductive health and rights by monitoring progress,3 documenting women’s reproductive health needs,4 conducting implementation research5 and testing novel approaches to improve access to quality abortion care services in experimental or quasi-experimental studies.6 7 Measuring accurately the interventions or care provided, how and in which context women in need access them, and their impact on health and other outputs or outcomes is essential to improve the evidence base, to enhance the implementation of safe abortion programmes and to track progress in reducing unsafe abortions.

Measures and indicators are key ingredients for research and evaluation. The terminology measures refers to numbers or quantities, whereas indicators are more specific time-bound measures or statements that are used to establish targets or demonstrate changes following programmatic or public health interventions. While there have been efforts to refine and standardise the methodology around capturing the incidence of abortion,8 9 and measuring access to, delivery of, and quality of abortion care,10 consensus on the most appropriate measures or indicators is lacking and there are gaps in ‘practical’ implementation metrics.11 For example, different sets of abortion indicators are used or have been proposed by international non-governmental organisations (NGOs), and other international actors such as MEASURE Evaluation. As part of its coordinating role in establishing standards, the WHO published a technical document on national-level monitoring for achievement of universal access to reproductive health, proposing related indicators of safe abortion care that countries are encouraged to report within the context of monitoring national reproductive and maternal health programmes.9 12 Consequently, the WHO safe abortion guidelines included 11 indicators for monitoring access and outcomes and noted 21 other issues to consider for periodic assessment grouped into access to safe abortion services, availability of safe abortion and quality of care.9 However, these guidelines do not provide further details on indicators that sexual and reproductive health programmes might use to evaluate the success of their interventions in these 21 areas.

A recent systematic review of three databases and grey literature has since identified 75 indicators of ‘abortion quality’ subdivided into structure, process and outcomes indicators.10 Following on this review, the objective of this paper is to provide a broader and more extensive summary of relevant abortion measures and indicators used over 10 years in the published scientific literature. Our aim is to inform and enlarge the debate on how progress in the provision and access to abortion care can be best captured.

Methods

Study design

We undertook a scoping review rather than a systematic review as we anticipated that the primary studies reporting abortion measures and indicators were likely to be very broad in their scope. The methods for the review were informed by a framework developed for scoping reviews.13 This paper follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for reporting on systematic reviews.

Patient and public involvement statement

Abortion patients and the public were not involved in the design, or conduct, or reporting or dissemination plans of our scoping review.

Search strategy

We searched two databases (PubMed and Popline) on 10 August 2018. The full search strategy is provided in online supplemental appendix 1, covering free-text and MESH terms for “abortion”, combined with free-text and MESH terms for “measurements” and “indicator”. We consulted websites of international NGOs and associations working on women’s reproductive rights for additional data. We included information that was available on the websites at the time of our search, regardless of when it was originally published.

Document selection and data extraction

We included studies and NGO documents reporting abortion-related measures or indicators that captured various domains relevant to reproductive health programmes such as abortion policies, access or availability, prevalence or incidence, quality of care, or health outcomes. In the rest of this paper, we will use the terminology ‘indicators’ throughout, unless otherwise necessary. Studies were eligible for inclusion if they were published within a 10-year period, from the beginning of 2008, and were conducted within health facilities or at the population level (ie, regionally or nationally representative), with the aim of measuring changes, success or progress or assessing programmatic needs. We focus on the 10 recent years only, as there have been major changes in the way abortion is perceived, and in the abortion methods and techniques. Study designs included cross-sectional studies, intervention studies, time series and cohort studies. Policy studies, NGOs’ monitoring and evaluation documents and methodological papers that reported on new indicators were also included. We did not exclude any articles, documents or websites based on language during title and abstract screening; however, full review was only conducted for papers written in English or French. We did not assess study quality.

Titles and abstracts were screened by one reviewer (MD and VF), with 10% double screened to assess concordance. Potentially relevant papers were read in full by one reviewer (MD) and were checked again by VF and CC to verify that they met the inclusion criteria. Data were extracted into Excel if the publication was deemed relevant. The type of data extracted depended on whether the paper identified a new indicator, which had not yet been identified through the review of full texts, or whether we had already captured the indicator from another study. For new indicators, we extracted the name of the indicator, what it measures (eg, women’s satisfaction with abortion care), a description of the numerator and the denominator (where relevant) and data sources. For a few indicators, which were qualitative in nature (such as description of regulation availability), we included the descriptive statement in place of the numerator. If the indicator had already been extracted from another full-text, the details already extracted were checked, and the new reference was added to the list of studies using that indicator. Where a description of an indicator was provided, but the indicator appeared aspirational and untested, with no source of empirical data provided, we left the description of the data sources empty.14

Data synthesis

Data synthesis was conducted using the steps suggested by Arskey and O’Malley, covering preparation of an analytical framework, numerical analysis on the extent and nature of studies, and thematic analysis.15 Once data extraction was completed, we grouped indicators according to a conceptual framework, which was developed inductively by bringing indicators into thematic groups. Several existing frameworks were also consulted, including one of which was proposed by Taylor and colleagues16 and one by Bryce and colleagues.17 According to our framework, indicators were classified into one of seven mutually exclusive groups, ranging from abortion access and availability through to the health impact of abortion (eg, death). Although family planning is a crucial intervention to prevent unwanted pregnancy, family planning indicators were only included in the context of post-abortion care and number of abortions averted.

After grouping the indicators, all authors met and reviewed the list of indicators and their groupings, agreeing the most appropriate group for certain indicators when there was uncertainty, and standardising the terminology across indicators. We then counted the number of indicators in each category of the conceptual framework as well as the number of indicators using each data source. We also counted the number of times an indicator was reported in more than one document.

Results

Figure 1 shows the process of selecting studies. A total of 1999 abstracts were identified from the search of the two databases. Most of these were excluded after reviewing the title and abstract (N=1782); no additional articles were identified for inclusion based on double screening a 10% sample by a second reviewer. All included documents in this paper were in English. There are nine scientific papers for which we did not conduct a full review because they were not in English or French. Of the 199 articles for which the full text was obtained and reviewed, 135 were identified as relevant for this review. Additionally, through the search of websites, seven relevant documents were added. Ultimately, we extracted information on 792 indicators from 142 documents (add online weblink/see online supplemental appendix 2).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram for the scoping review.

The conceptual framework, developed as part of the review, alongside the number of indicators extracted for each domain of the conceptual framework is shown in figure 2. Indicators could be grouped in seven different domains as follows: (1) social and policy context (including attitudes and beliefs among women and the population, stigma and laws and regulations); (2) abortion access and availability (including signal functions); (3) abortion prevalence and incidence; (4) safe abortion and post-abortion care (ie, measures of quality of care, including clinical and respectful care); (5) abortion outcomes (including adverse events, near-miss and other physical or psychological complications for women using services); (6) abortion impact on health (ie, measures of disabilities and mortality at the population level; indicators of financial impact appear in access and availability) and (7) characteristics of women who have abortion and with abortion outcomes.

Figure 2

Conceptual framework for scoping review, with numbers of indicators for seven domains and forty subdomains (NB: domain 3 has two sub-domains for which results have been combined in this figure).

Using a logical model design, the conceptual model represents how access to and availability of services, and care provided, can lead to health outcomes in individual women or impact on mortality and morbidity at the population level, considering the influences of context and women’s characteristics. In this framework, the arrow between ‘abortion access and availability’ and ‘abortion prevalence and incidence’ is used to capture the complex relationship between these domains, whereby in some contexts the frequency of abortion will drive the availability of services, while in other contexts the availability of services may lead to an increase in the frequency of abortion. Each of these seven domains is then further divided into subcategories as presented in figure 2. For example, abortion care has the largest number of subcategories (N=13), followed by abortion access and availability (N=8) and abortion care (N=7), reflecting perhaps the large number of indicators in these domains.

There were 225, 206 and 150 indicators related to abortion access and availability, abortion care and abortion outcomes (adverse events/morbidity/psychological well-being), respectively. There were far fewer indicators looking at the social and policy context or measuring abortion incidence/prevalence or impact (67, 28 and 18, respectively). In addition, 98 indicators covered the characteristics of women who used abortion care or for whom abortion outcomes were documented (53 for demographic and reproductive characteristics; 11 for reasons for abortion; 20 for socioeconomic characteristics and 14 for women’s knowledge and perception). Most of the indicators in this group were not population based. There were no indicators of the consequences of being denied a termination of pregnancy, such as household poverty.

Details of each of the 792 indicators, including the definition of the numerator and denominator and data sources can be found in the online weblink (see online supplemental appendix 2). Table 1 provides a summary of the number of indicators within each domain captured by each different type of data source. For example, the majority of indicators on abortion access and availability were calculated using health provider surveys (N=114, 56% of indicators in this domain). In contrast, facility records/ Health Management Information System (HMIS) data (N=19, 70%) and population survey/census (N=13, 48%) were the predominant sources of data for abortion incidence and prevalence. Facility-based records and health information management systems were a moderately common data source, used for 211 indicators overall. However, all the remaining indicators relied on data sources requiring special studies either in facilities or at population level. Of these, client exit interviews were the most common data source (N=180) and, despite their potential to provide representative data, population-based surveys and censuses were among the least used data sources (N=99).

Table 1

Number of abortion indicators obtained according to data sources, 2008–2018 (N=760*)

	Facility records, HMIS, service statistics	Prospective extraction of individual medical record for morbidity study	Client exit interviews	Special health facility assessment survey	Health provider survey	Population survey or census	Other data source	Total number of indicators with information on data source for each domain
Abortion access and availability	6	0	16	69	114	10	12	204
Abortion incidence and prevalence	19	3	1	0	0	13	1	27
Abortion care	78	63	77	7	2	19	11	198
Abortion outcomes	58	72	40	2	1	19	3	149
Abortion impact	8	5	0	0	0	11	7	18
Social and policy context	4	0	0	1	7	15	43	66
Characteristics	38	21	46	0	0	12	19	98
Total	211	164	180	79	124	99	96	760†

Some indicators are measured with more than one sources.

*32 aspirational indicators, for which a data source was not proposed, are excluded from this table.

†Total in column, not row which includes double counting (953 count).

HMIS, Health Management Information System.

Finally, many indicators were specific to a particular data collection approach (eg, attitudes of health providers obtained in provider surveys),18 a type of intervention (indicators of clinical management obtained in clinical audits or the evaluation of a clinical intervention)19 or a context (measures of emotional or psychological well-being only found in studies conducted in high-income country contexts).20 Some appear limited to specific authors or institutions, as they developed methodologies suited to their own purpose. Less than a quarter of indicators were proposed by multiple sources, as shown in our weblink (N=175; 22%). These included most indicators in the prevalence and incidence category (19 out of 28).

Discussion

In this scoping review of 10 years of published literature, we found a very large number of abortion-related indicators, with less than a quarter proposed or used by more than one source. Indicators of access and availability and of the provision of abortion care were by far the most common. Indicators of outcomes were fewer and less comprehensive as they very much focused on physical health, with few measures of psychological well-being and no measures of quality of life or functioning. Similarly, there were few indicators attempting to measure the context, including beliefs and social attitudes at the population level. These indicators of knowledge and ‘demand’ for reproductive health services are however important to ensure speedy access to safe abortion services when needed.21

We conducted a comprehensive scoping review of published papers. Our search and screening strategies were designed to be sensitive. We included all research papers that focus on documenting changes including assessing the success or progress of abortion programmes or interventions, supplemented by programme documents that were specific to monitoring and evaluation and could be found online on the website of key NGOs and some governmental agencies active in sexual and reproductive health. Our scoping review is broader compared with the review conducted by Dennis and colleagues,10 since we went beyond quality of care and monitoring and evaluation, and consequently we identified over ten times as many indicators (792 vs 75). Our online list of indicators provides a rich resource for abortion programmes and research projects, enabling the selection of indicators that can be tailored to their own goals and objectives. Our list also includes examples of country level indicators that can measure the performance of health sector programmes in ensuring sexual and reproductive health, including for abortion services.11 We have conducted this review as the first step in a multi-stage process for deriving a set of standardised indicators which can be used across reproductive health programmes in diverse settings.

Our review has some limitations. It is possible that screening criteria about what counts as a measure or an indicator of success or progress were different between the screeners. We did attempt to minimise this risk, by double screening a sample of studies and discussing the selection of indicators in detail during the standardisation process. The inductive process for building up the indicator conceptual framework was a subjective process, and another group of investigators may have classified the indicators differently. Although family planning is an important intervention to reduce deaths from unwanted pregnancies, our exclusion of papers that focused exclusively on family planning constitutes arguably another possible limitation. The MEASURE Evaluation website can be consulted for its extensive list of family planning and reproductive health indicators.14 As we only included two databases and the grey literature available online from well-known international NGOs active in reproductive health and only two country reports from high-income countries (UK, USA), relevant documents may have been missed. In particular, we did not systematically search websites of national health management information systems for abortion indicators. Finally, we categorise indicators into only one of the domains in the conceptual framework; however, in many cases, indicators could have been categorised into several domains.

Due to the extensive number of indicators identified, it was beyond the scope of this project to provide an assessment of their suitability and robustness. Good indicators have several characteristics including validity, reliability, relevance, ease of measurement, actionability, comparability potential and responsiveness to change22 as well as being based on evidence and acceptability.23 Indeed, we found in our review that similar constructs are defined and measured in a myriad of ways, increasing the number of indicators without ensuring the use of validated and standardised measures. Moving forward, a more consistent understanding of validity can help guide prioritisation of indicators and development of new indicators where there is a validity gap identified.24 It also allows standardisation enabling comparison across time and settings where these indicators are used.

Validation and standardisation have some uniquely important features for sexual and reproductive health indicators. Even though many such indicators are established, there are measurement difficulties specific to induced abortion related to the stigmatisation and restrictive nature of the procedure in some contexts, which will affect the data sources, validity, reliability and utilisation of the indicators depending on the setting, and therefore comparisons. For example, reporting bias can severely under-estimate the measurement of abortion incidence, abortion outcomes and abortion care utilisation with population survey approaches, explaining the under-utilisation of this data source.8 It is difficult to distinguish when post-abortion care is provided to women with spontaneous abortion vs those with induced abortion in legally restrictive settings as information on induced abortion is often omitted or inaccurate in medical records and facility registers leading to misclassification.25 Severe abortion complications are not always found in the gynaecological wards,26 requiring investigators to screen records in emergency rooms and general intensive care units. Abortion-related deaths may not be reported. Women and their partners buy misoprostol and mifepristone ‘under the radar’ from licensed and unlicensed drugs sellers, community networks and increasingly online. This can limit the comparability of information over time and across settings for all types of indicators. Recent efforts to improve the usefulness of indicators or the utilisation of available data are multiple, and include for example, the development of approaches to measure the coverage of signal functions for abortion care at the population level through the use of multiple sources of data27 28 and advanced statistical modelling to fill the gaps in information on unsafe abortion incidence.1

Our scoping review results could be used by different stakeholders, including researchers and programme managers, to inform their own development of data collection tools for monitoring and evaluation of sexual and reproductive health programmes. Each indicator in our list was provided with one or several references to illustrate the use of the indicators. These results also aim to inform a future research agenda on abortion indicators measurement. While our list of indicators is extensive, there remains a need in some areas to develop or test new indicators: for example, to broaden the type of health outcome indicators for abortion to quality of life and functioning; to respond to the challenge of collecting valid data in the community as the provision of abortion care is shifting to community-based distribution of medical abortion; to standardise indicators, in particular those that are morbidity related29; to measure performance of the health sector11 and to align with other initiatives on effective coverage30 and health system strengthening monitoring.31 Comprehensive measurement of patient-centred indicators, such as women’s experience of care is also required.32

Conclusion

The list of indicators identified in this paper provides a good ‘starting point’ and an extensive resource for the design of monitoring and evaluation plans of abortion programmes.33 The large number of indicators, with similar concepts measured in a multitude of overlapping ways, suggest a need for further investigation on their validity and for standardisation. This is especially needed for when the measurement intention is a comparison over time or between settings. In particular, a smaller set of standardised and prioritised indicators may be required for tracking global trends, monitoring the implementation of guidelines at the country level or comparing success across programmes and countries. Areas of further development include population-based indicators of effective coverage and standardised indicators of well-being and respectful care.