Methods for evaluating the benefits and harms of antenatal and newborn screening programmes adopted by health economic assessments: protocol for a systematic review.

INTRODUCTION: Complex organisational arrangements are required to deliver antenatal and newborn screening programmes. Decision-makers consider the benefits and harms of screening when reviewing the evidence about these programmes. Economic evaluations contribute one important part of this assessment process. However, it is not fully understood what approaches health economic assessments have adopted to measure and value benefits and harms. This study aims to systematically review and critique the published and grey literature on methods for identifying, measuring and valuing the benefits and harms of antenatal and newborn screening adopted by health economic assessments. METHODS AND ANALYSIS: Nine bibliographic databases will be searched from 2000 onwards. These search strategies will be supplemented by manual reference searching of bibliographies, forward citation searching, contacts with experts, author searching and web searching for grey literature. Studies will be selected for review if they report health economic assessments of an antenatal or newborn screening programme. Assessments of title and abstracts and full reports will be undertaken independently with disagreements resolved through discussion. Data extraction will include fields to assess the reporting quality of the studies using the Consolidated Health Economic Evaluation Reporting Standards statement and a bespoke ancillary form to assess how benefits and harms have been accounted for. ETHICS AND DISSEMINATION: This is an evidence synthesis review from already published materials and hence ethics committee approval or written informed consent will not be required. Our results will be disseminated by publishing in high-impact peer-review journals and presenting at relevant conferences. PROSPERO REGISTRATION NUMBER: CRD42020165236. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This protocol allows for transparency to the methods and processes that will be followed in reducing possible biases and issues with interpretation.

This systematic review will offer the highest level of evidence about benefits and harms of antenatal and newborn screening programmes adopted by health economic assessments.

Key limitation is the absence of a meta-analysis as it will not be possible to combine the information that will be extracted from these economic evaluations using a summary metric.

Introduction

Antenatal and newborn screening allows earlier diagnosis of certain high-risk conditions and leads to timely decisions around the care and treatment of pregnant women and infants. National population screening programmes are therefore recommended for implementation by screening agencies in various countries. One such example is the UK National Screening Committee (UK NSC), which makes independent, evidence-based recommendations to ministers in the four countries of the UK. Antenatal and newborn screening is covered by 6 of the 11 National Health Service (NHS) screening programmes in the UK, namely fetal anomaly screening, infectious diseases in pregnancy screening, newborn and infant physical examination, newborn blood spot screening, newborn hearing screening and sickle cell and thalassaemia screening. They represent mainstays of national screening strategies with far-reaching implications for population health and well-being.1 The number of conditions considered for inclusion within the antenatal and newborn screening programmes is likely to increase as a result of technological developments, such as next generation sequencing.2 Screening organisations such as the UK NSC consider the benefits and harms of screening, of which cost-effectiveness assessments are one component of the evidence generation.3 The organisation’s recommendations are grounded in up-to-date evidence but stakeholders’ opinions are carefully listened to and their views reflected in the final decision.

Cost-effectiveness assessments of many forms of screening considered by the UK NSC (eg, bowel cancer screening, abdominal aortic aneurysm screening) are conducted within a cost–utility framework and expressed in terms of incremental cost per quality-adjusted life-year (QALY) gained, where the QALY combines preference-based health-related quality-of-life weights (health utilities) with data on length of time in the health states of interest.4 This approach to cost-effectiveness assessment mirrors those recommended more broadly by health technology assessment agencies in the UK, such as the National Institute for Health and Care Excellence in England and the Scottish Medicines Consortium in Scotland.5 6 It also mirrors the preferred form of cost-effectiveness assessment adopted by health technology assessment, pricing and reimbursement authorities in several other industrialised countries.7–9

A number of methodological factors have constrained capacity to evaluate antenatal and newborn screening programmes using the standard incremental cost per QALY gained metric. These include challenges surrounding the valuation of prenatal life when decisions following antenatal screening and diagnostic testing result in the termination of the fetus or unborn child,10 11 the absence of a multiattribute-utility measure validated for use in infancy and through early childhood,12 and the challenges surrounding QALY aggregation across the mother, child and potentially other family members.13 14 Furthermore, attributes of relevance to parents, such as the utility derived from information per se or reassurance following a screen-negative test result, and the disutility associated with a false positive test result or overdiagnosis of disease, are likely to be missed, or at least inadequately covered, by standard approaches to health utility measurement, such as available multiattribute utility measures (eg, EuroQo (EQ-5D), Short-Form 6D (SF-6D), Health Utility Index (HUI) Mark 3).15–17 Moreover, a number of ethical challenges compound the technical complexities surrounding economic assessments of antenatal and newborn screening programmes. These emanate from differences in moral perspectives on the status of the fetus or unborn child11 and how society should value disability,18 and differing perspectives on the ownership of genetic information19 and the potential harms of inadequately informed consent processes on parental autonomy.20

Screening organisations have recognised the limitations of the methods generally adopted by economic assessments of antenatal and newborn screening programmes. Assessments that have expressed cost-effectiveness in terms of incremental cost per QALY gained have tended to overlook relevant aspects of benefits and harms and have been constrained by the tools available for the measurement, valuation and aggregation of those benefits and harms. The consequence is that recommendations about antenatal and newborn screening are made on the basis of differing approaches to the construction and valuation of outcomes and without a common cost-effectiveness threshold that reflects opportunity cost in the health system. Invariably, this results in suboptimal levels of population health and well-being.

Objectives

The main objective of this study is to systematically review and critique the methods for identifying, measuring and valuing the benefits and harms of antenatal and newborn screening adopted by economic assessments in the published and grey literature.

Any amendments made to this protocol when conducting the review will be outlined in the manuscript reporting the results of the systematic review.

Methods and analysis

We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols checklist when writing our report.21

Eligibility criteria

The Population, Intervention, Comparator, Outcome and Study design framework was used to develop our study eligibility criteria (table 1), which will be applied to our literature searches. The searches will be limited to studies published after 1 January 2000. Studies reporting health economic assessments, such as economic evaluations and studies that use economic frameworks of cost-effectiveness evidence or economic notions of value (eg, multicriteria decision analyses, programme budgeting and marginal analyses) of antenatal or newborn screening programmes, will be included. We will include studies published in all languages, but limit studies to those conducted in developed countries (defined, for the purposes of this review, as a member of the Organisation for Economic Co-operation and Development (OECD)22).

Table 1

Inclusion and exclusion criteria for identification of relevant studies

Characteristics	Inclusion criteria	Exclusion criteria
Population	Pregnant womenNewborns	Anyone other than pregnant women or newbornsStudies on animalsNot conducted in a developed country (ie, not a member of OECD)*
Intervention	Antenatal or newborn screening programme†	Preconception screeningScreening mechanism
Comparator	No screening or specific form(s) of screening other than experimental intervention(s), as defined by specific conditions
Outcome	Benefits and harms of antenatal or newborn screening that have been identified, measured and valued by economic assessments
Study design	Full economic evaluation: Cost-effectiveness analysis Cost–utility analysis Cost–benefit analysis Cost–consequences analysis Cost–minimisation analysis Economic framework that incorporates cost-effectiveness evidence or economic notion of value (eg, multicriteria decision analysis, programme budgeting and marginal analysis)	Descriptive cost analysisBudget impact analysisNot a full economic evaluationStudy without primary evidence: Editorial Letter Methodological research without applied evidence Perspective, opinion or commentary Protocol Review

*Studies from countries that become OECD members after the title/abstract screening process was completed will not be included in our review (last OECD member included was Colombia).22

†This includes actual and proposed, for example, hypothetical screening programmes as well as any aspect of a screening programme (defined as a whole system of activities needed to deliver high-quality screening), for example, the performance of screening test.

OECD, Organisation for Economic Co-operation and Development.

Information sources

We will conduct systematic searches of both published and grey literature, including peer-reviewed journal articles controlled by commercial publishers and documents produced by all levels of government, academia, business and industry. The following electronic bibliographic databases will be searched: Medline (OvidSP), (1946–present) Embase (OvidSP) (1974–present), NHS Economic Evaluation Database (via CRDWeb https://www.crd.york.ac.uk/CRDWeb/), EconLit (Proquest), Science Citation Index, Social Science Citation Index and Conference Proceedings Citation Index-Science (Web of Science Core Collection) (1945–present), CINAHL (EBSCOHost) (1982–present) and PsycINFO (OvidSP) (1806–present). SCOPUS will be used to run forward and backward citation searches once relevant studies are identified. The academic electronic database searches will be supplemented by manual reference searching of bibliographies, contacts with experts in the field and author searching.

The list of grey literature to be searched will include but not restricted to a pool of relevant websites, which will initially be informed by a recent systematic review of national policy recommendations on newborn screening that identified 30 websites of national and regional screening organisations with documentation about antenatal and/or newborn screening recommendations.23 This will be widened to cover websites reported by the Health Grey Matters checklist and those for national and regional screening organisations, Health Technology Assessment (HTA) agencies, paediatrics organisations and obstetrics and gynaecology societies in OECD countries, as well as international decision-making bodies, such as the WHO, the European Council, European Commission and the European Observer.23 24 We will also contact committee members from national screening organisations across OECD countries and reach more widely through the International Society for Prenatal Diagnosis and the International Society for Neonatal Screening in order to further capture unpublished economic models that have been used to influence national policy. The development of the contact directory will be informed by previous research.25 26 We will email the objectives of our systematic review and our search criteria to each organisation and corresponding authors enquiring about further documentation not identified through the main grey literature sources. Contact details for national screening organisations are not reported as we do not have permission to share them with third parties. A customised web scraping tool that uses the Google search engine will be built using Python to query directly the stated websites and automate the data extraction processes.

Search strategy

The search strategies applied to the published literature will be developed using a combination of Medical Subject Headings and free-text keywords related to health economic assessments of antenatal and newborn screening programmes. We will develop the search strategies with one of the coauthors (NR) who is a local information specialist with expertise in conducting systematic literature reviews in the health sciences. The draft search strategy applied to MEDLINE reported in online supplemental table 1 will be adapted to other databases.

A simplified search strategy derived on the basis of Cochrane guidelines will be applied to our searches of the grey literature.27

Data management

The results of the literature searches will be uploaded to Endnote to remove duplicates. Unique records will be imported into Covidence,28 an online software program that facilitates collaboration among reviewers during the screening and data extraction stages. The software will allow us to import references and files to be screened and enter information into a precreated data extraction form after removing duplicates. The files used for the full-text screening will be downloaded using Endnote software package V.X9.2 (Clarivate, Philadelphia, USA, 2019), a reference management system specifically designed for managing bibliographies and citations. The review team will develop and test screening criteria based on the inclusion and exclusion criteria specified in table 1. A calibration exercise will be undertaken to pilot and refine the screening criteria before the formal screening process starts. For non-English language papers, Google Translate (Google, Mountain View, California, USA) will be used to translate relevant documents at the initial screening stage and if the article/report is regarded as relevant, translation into English will be conducted by professional translator(s).

Selection process

Two reviewers (MEP and MY) will independently screen the titles and abstracts of all retrieved articles and document the reasons for study exclusion according to the criteria specified in table 1. Potentially relevant articles will be read in full text independently by the same two reviewers (MEP and MY), and study eligibility based on the inclusion and exclusion criteria will be assessed. At each stage of the selection process, any disagreement will be resolved by discussion and consensus between the two reviewers. When consensus cannot be reached, input from the rest of the review team (OR-A and SP) will be obtained.

Data collection process

A data extraction sheet, which will be piloted and refined using five to ten randomly selected studies identified in either the academic electronic databases or the grey literature, will be created following recommendations from the Cochrane Handbook for Systematic Reviews of Interventions.27 As we had anticipated a large number of articles to data extract, after consulting our Independent Oversight Committee members and Information Specialist, a selection of 10% of the papers/reports will be extracted independently by two health economists (MEP and MY), followed by a reconciliation process. The rest of the papers will be shared between the two reviewers. Furthermore, any uncertainties related to data extracted by the two independent reviewers (MEP and MY) will be discussed with the two senior investigators (OR-A and SP) at weekly meetings. The list of variables that will be extracted from each report included at the final stage of the review process will be finalised following the piloting and refinement of the data extraction sheet.

The data extraction form will consist of two parts: (1) a main proforma, which will contain items from the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist,29 modified where applicable to align with our research focus (ie, benefits and harms within economic assessments); bibliographic details; condition(s) screened; aggregation of benefits and harms; whether the authors made any policy recommendation based on their economic evaluation evidence; and whether the authors might have any vested interest in promoting their screening programme or mechanism; and (2) an ancillary form that will identify methods of measurement and valuation of benefits and harms by screening test outcomes (ie, true positives, false positives, true negatives and false negatives). The ancillary form will also record the stage of the disease pathway at which the screening test was administered; phase(s) of the screening programme (ie, screening, diagnostic and/or treatment), whether a decision-analytical model had been graphically or textually depicted; and the consequences associated with treatment where applicable.

Data items

In order to reduce bias from including data from multiple reports of the same study, we will treat multiple articles published by the same authors with similar titles and abstracts as linked companion studies (ie, multiple reports from a single study) and only include the most detailed publication in our final outputs. Similarly, if conference abstract(s) and a journal article by a similar group of authors have been published on the same topic, only the journal article will be included at the full-text screening stage. Since we are interested in the methodological approaches to the measurement and valuation of benefits and harms and how the results were reported, if the article title suggests that an economic evaluation was conducted but neither the methods nor the results are presented in the abstract or full-text, the article will be excluded at the screening stage. Articles that do not focus specifically on pregnant women or newborns but report separate results of screening of pregnant women or newborns within broader populations will still be included. In addition, we will not contact authors for missing data on individual data items included in our data extraction sheet and instead will record them as ‘not reported’.

Assessment of reporting quality of individual studies

The reporting quality of each included economic study will be assessed using the CHEERS checklist.29 The items will be scored as one if reported in full or zero if not reported or partially reported. The total score may not sum to 24 points (the maximum score for the CHEERS checklist) as some contributing items might not always be applicable (eg, model assumptions would not be applicable if a decision model did not form the basis of the economic assessment).

Ethics and dissemination

This review is based on data available from secondary sources and published materials and hence ethics committee approval or written informed consent will not be required for this study as primary data will not be collected. Our results will be disseminated by publishing in high-impact peer-review journals and presenting at relevant conferences. A National Institute for Health Research (NIHR) HTA monograph will also be published. We will also present our findings to key stakeholders including the UK NSC committees that make decisions on antenatal and newborn screening and our patient and public involvement (PPI) network established to contribute to the design, conduct, analysis, reporting and dissemination of all aspects of this study.

Patient and public involvement

This study is a systematic review of secondary evidence, and therefore, no direct patient recruitment is involved. However, the topic had been identified and prioritised for research to meet the needs of the UK NHS and, in particular, the gaps identified by the UK NSC on methods used to assess the benefits and harms of antenatal and newborn screening programmes to patients and their families. We also have the director of the organisation Antenatal Results and Choices as one of our coinvestigators to provide valuable insights into the understanding of the complexity of PPI in the context of antenatal and newborn screening. As aforementioned, we will produce plain English (lay) summary versions of our findings aimed at PPI audiences.

Final remarks

The findings of this systematic review will be used to inform methodological recommendations and build the foundation for future economic assessments of antenatal and newborn screening. The primary vehicle for achieving this will be the VALuing the bENefits and harms of anTenatal and newborn screenIng programmes in heAlth economic assessments (VALENTIA) study, a recently funded project by the UK NIHR HTA Programme (https://www.npeu.ox.ac.uk/valentia). The evidence generated by VALENTIA will inform a stakeholder workshop leading to a set of final recommendations for outcomes measurement and valuation within future economic assessments in this clinical area. Ultimately, this should improve the conduct and reporting of health economic assessments of antenatal and newborn screening.

A caveat to the proposed study methods is the lack of meta-analysis; it will not be possible to combine the results of the economic evaluations in pooled metrics. Nevertheless, this should not necessarily be viewed as a study limitation as our aim is not to derived pooled cost-effectiveness estimates for particular types of screening, but rather to generate new knowledge about the methods adopted by economic assessments in this clinical area.