Effectiveness of decision aids on cancer-screening decision-making: an umbrella review protocol.

INTRODUCTION: Although systematic reviews have shown how decision aids about cancer-related clinical decisions improve selection of key options and shared decision-making, whether or not particular decision aids, defined by their specific presentation formats, delivery methods and other attributes, can perform better than others in the context of cancer-screening decisions is uncertain. Therefore, we planned an overview to address this issue by using standard umbrella review methods to repurpose existing systematic reviews and their component comparative studies. METHODS AND ANALYSIS: We will search PubMed, Embase, the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects from inception through 31 December 2021 with no language restriction and perform full-text evaluation of potentially relevant articles. We will include systematic reviews of randomised controlled trials or non-randomised studies of interventions that assessed a decision aid about cancer-screening decisions and compared it with an alternative tool or conventional management in healthy average-risk adults. Two reviewers will extract data and rate the study validity according to standard quality assessment measures. Our primary outcome will be intended and actual choice and adherence to selected options. The secondary outcomes will include attributes of the option-selection process, achieving shared decision-making and preference-linked psychosocial outcomes. We will qualitatively assess study, patient and intervention characteristics and outcomes. We will also take special care to investigate the presentation format, delivery methods and quality of the included decision aids and assess the degree to which the decision aid was delivered and used as intended. If appropriate, we will perform random-effects model meta-analyses to quantitatively synthesise the results. ETHICS AND DISSEMINATION: Ethics approval is not applicable as this is a secondary analysis of publicly available data. The review results will be submitted for publication in a peer-reviewed journal. PROSPERO REGISTRATION NUMBER: CRD42021235957. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This will be a first of its kind umbrella review to formally assess the empirically documented effectiveness of decision aids about cancer-screening decisions by using standard methods.

This review will investigate what specific components and/or attributes of the decision aids are more important than others to help improve particular screening decision outcomes.

Heterogeneity in outcome measures is expected, which could preclude formal quantitative syntheses.

Introduction

Shared decision-making about healthcare options involves a communication process. This process between healthcare providers and recipients, typically in the context of clinical encounters, involves sharing and understanding existing scientific evidence on the possible options, availability, and costs thereof, and the expected consequences of each option, together with the recipients’ preferences to work together to make an optimal decision.1 However, expanding medical information and the resultant option complexity has made this process challenging for both groups concerned.

To facilitate this communication process, thereby improving the recipients’ health outcomes and ultimately reducing the overall costs ideally, decision aids as a complementary tool have been introduced.2 Decision aids can be provided through diverse presentation formats and delivery methods (eg, written materials, including infographics, videos, group sessions and online interactive materials). A new simplified visual format that graphically presents evidence-based data on the benefit and harms of healthcare options has also been proposed recently.3

Similar to other medical interventions, cancer screening is a double-edged sword and holds potential benefits (eg, reduction in cancer development or cancer-specific mortality) and harms (eg, increase in unnecessary additional interventions and procedures).4 Therefore, a number of decision aids to facilitate provider–recipient communication in the context of cancer-screening decision-making have been developed, the effectiveness of which has then been assessed and systematically reviewed.5 Randomised evidence has shown that, compared with usual care with no decision aid, use of cancer-related decision aids improved key attributes of the option-selection process and achieving shared decision-making, such as improved knowledge, risk perception, and value-choice agreement in cancer screening, prevention and treatment decisions in general.6 7

However, uncertainty remains as to whether or not specific presentation formats and delivery methods and/or other specific attributes of decision aids for cancer-screening decision-making are better than the alternatives. The purpose of this study will be to compare the effectiveness of alternative decision aids for cancer screening. To achieve this goal, we will comprehensively re-examine the existing comparative evidence from systematic reviews and primary studies that compared alternative decision aids or from studies that compared a decision aid with a conventional management as the ‘common’ control in general clinical encounters between healthcare providers and healthy, asymptomatic, average-risk adults.

Methods and analysis

This umbrella review protocol follows and addresses the items proposed by the Preferred Reporting Items for Overviews of Systematic Reviews including harms checklist.8 A brief version of this umbrella review protocol has been registered (PROSPERO: CRD42021235957), and in the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and rationale.

Eligibility criteria and outcomes of interest

Table 1 summarises our inclusion criteria, which follow a generally accepted framework to formulate systematic review questions comprising six key components: populations, interventions, comparator interventions, outcomes, timings and settings listed under the patient, intervention, comparator, outcome, timing and setting (ie, the so-called PICOTS) framework.9

Table 1

Inclusion criteria and clinical outcomes of interest based on the patient, intervention, comparator, outcome, timing and setting (PICOTS) framework

PICOTS item	Specific details
Population	Healthy adults with average risk in consultation with healthcare providers
Interventions	A decision aid of any format
Comparators	Another decision aid(s) or conventional management with no decision aid
Outcomes
O1	Intended and actual choice, adherence to chosen option
O2	Attributes of the option-selection process and achieving shared decision-making, preference-linked psychosocial outcomes and other outcomes
Timing	Short term: defined as <12 months of the postintervention follow-up durationLong-term: defined as ≥12 months the postintervention follow-up duration
Setting	All settings (eg, face-to-face or online sessions, clinical encounters in primary care or self-learning at home)

We will include systematic reviews with or without a meta-analysis of randomised controlled trials (RCTs) or non-randomised studies of interventions (NRSIs) of any size that assessed a decision aid of any presentation format compared with an alternative decision aid or conventional management with no decision aid as control to facilitate provider–recipient communications in cancer-screening decision-making in healthy average-risk adult, prospective, cancer-screening recipients.

Our definition of a systematic review is as proposed by Krnic Martinic et al and requires a review that has (1) an explicit research question, (2) sources that were searched, a reproducible search strategy, (3) explicit inclusion and exclusion criteria, (4) explicit screening and selection methods, (5) critical appraisal and reports on the quality/risk of bias (ROB) of the included studies and (6) information about data analysis and synthesis methods that show the reproducibility of the results.10

We will consider NRSIs to be comparative studies that adopt a cohort design, a case-control design, a controlled before and after design, an interrupted-time-series design and a ‘quasi-’ randomised design as proposed previously.11 We will include studies of decision aids for screening tests for all cancer subtypes. We will focus on specific cancer subcategories, specifically, breast, cervix, colorectal, lung and prostate cancer.

We will exclude systematic reviews of studies of asymptomatic but high-risk populations (eg, patients suspected to have a hereditary cancer syndrome who will take genetic testing, such as the BRCA gene test), populations with previous screening abnormalities that require repeat, follow-up, or confirmatory tests, or diseased populations. We also will exclude systematic reviews of studies of a pertinent population but with a non-comparative design, typically, single-group studies without comparison groups.

We will only include peer-reviewed, full-text publications or health technology assessment reports irrespective of languages. We will exclude narrative reviews and systematic reviews only published as conference abstracts or letters.

The classification of our outcomes of interest follows the methods adopted in the previously conducted similar broad umbrella reviews of decision aids in different clinical contexts.12 13 Our primary outcome of interest will be ‘program-level measures,’ that is, intended and actual choice (ie, participation and selection of an option) and adherence to a selected option. We selected these measures as the primary outcomes as they could evaluate material outcomes of a programme incorporating a decision aid acknowledging that they may not necessarily be the goal of comparative effectiveness in shared decision-making. The secondary outcomes will be ‘attribute-level measures,’ including attributes of the option-selection process and of achieving shared decision-making and preference-linked psychosocial outcomes (ie, anxiety, worry, caregiver burden, depression, self-efficacy, decisional regret) and other health (service)-related outcomes, including consultation time. The attributes of the option-selection process will include knowledge, accurate risk perceptions and informed choice, whereas the attributes of achieving shared decision-making will include recognition of decision-making, decisional conflict, provider–recipient communication, participation in decision-making and satisfaction. Psychosocial outcomes will include psychological distress as an adverse outcome of interest.

Information sources and search strategies

We will search the PubMed, Embase, Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effects databases from the inception of data collection through 31 December 2021, using free-text terms, such as “cancer,” “screening,” and “decision aids” as well as their synonyms. We will not employ filters for identifying systematic reviews. The complete search strategy and full list of databases are available in online supplemental file 1. As additional searches, we will peruse the reference lists of eligible systematic reviews reports. We also will search PROSPERO for identifying any ongoing systematic review projects.

Data management and selection

The electronic search results will be imported into EndNote V.X9 (Clarivate Analytics, Philadelphia, USA), and duplicate results will be removed manually. Two independent reviewers will double-screen abstracts by using Abstrackr, a web-based software for citation screening (Center for Evidence Synthesis in Health, Brown University, Province, USA).14 We will then peruse all potentially eligible full-text reports that more than one reviewer screens and accept for eligibility. All non-English publications will be translated into English before full-text assessment. Any discrepant results will be resolved by consensus. We will employ adjudication by a third reviewer in case of unresolved discrepancies. In case of multiple systematic review reports on the same topic published by the same group of investigators, we will include the most updated one.

Additional search for primary studies

Fact box is a newly introduced evidence-based data presentation tool with a simplified visual format.3 Therefore, we will additionally search PubMed, Embase and the Cochrane Central Register of Controlled Trials from 2015 through the present by using terms, such as “cancer,” “screening,” and “fact box” along with their synonyms to identify RCTs or NRSIs that specifically focused on fact box. We will not employ filters for identifying studies with specific research designs.

Data collection

We will perform data extraction on two levels—at the first level from the included systematic review and the second level from the primary studies assessed within the included systematic review—as performed in a previous umbrella review of a different topic.15 We will develop a standardised data extraction form for the two levels to extract pertinent data.

One primary reviewer (MH) will extract data, and one or more other reviewer (CH and/or TT) will verify all extracted data. Disagreements will be resolved by consensus in face-to-face research group meetings, and a third reviewer will adjudicate any unresolved discrepancies. We will contact the study authors for missing or unresolved (1) key descriptive data and (2) all numerical data by email. We will send two additional email correspondences if no response is received by 2 weeks after the previous correspondence attempt.

For the first-level extractions, we will extract the first author’s name, publication year, review objectives, number and design of primary studies, number and source of participants, length of follow-up and funding sources as review characteristics as well as demographic (eg, average age, percentage of men) and cancer risk profiles of the study participants, cancer types for which cancer-screening tests were performed, details on the decision aids, definitions of and/or tools to measure the outcomes of interest as characteristics of PICOTS. Then, we will extract aggregate-level numerical data on the intended and actual choice and adherence to a selected option (eg, intention to participate, actual participation and change of selection), attributes of the option-selection process and achieving shared decision-making (eg, knowledge, risk perception, clarity of value, discussion, interest, acceptability, attitude), preference-linked psychosocial outcomes (eg, decisional anxiety, conflict, distress, uncertainty, confidence) and other outcomes (eg, satisfaction with decision, decision-making role and informed choice). We will extract the absolute difference in or relative risk of proportions of events (eg, success or failure) or aggregate scores or ratings estimated by using, for example, established patient-reported outcome measures for each outcome.

For the second-level extraction, we will extract the first author’s name, study design and study-level descriptive and numerical data. Descriptive data on the assessed decision aid(s) will include the name and other identifiable information (eg, development date and/or version), presentation format (eg, written materials, audios, videos, hybrid or computer-based or web-based materials), delivery methods (eg, one-on-one session vs group session; face-to-face onsite meeting vs teleconference unidirectional delivery vs interactive delivery), assigned aggregate scores based on the International Patient Decision Aid Standards instrumental criteria V.4.0 (ie, 6 items for qualification criteria; 10 items for certification criteria and 28 items for quality criteria),16 and the degree to which the decision aid was delivered and used as intended per the Standards for UNiversal reporting of patient Decision Aid Evaluation studies checklist.17 Numerical data will include the number of participants and outcome measures listed in the first-level extraction items separately from the primary studies. For studies reporting event-type binary outcomes, we will extract 2×2-cross-classified count data. For continuous outcomes, we will extract and standardise the results; for example, by using the standardised mean difference following the standard methods.18

Assessment of methodological quality and quality of evidence

Two reviewers will independently assess the included systematic reviews by using the revised version of A MeaSurement Tool to Assess systematic Reviews tool.19 We will first assess each of the 16 items (ie, (1) use of the Participant, Intervention, Comparison, Outcome framework; (2) availability of review protocol; (3) rationales of study design selection; (4) comprehensive search; (5) duplicate study selection; (6) duplicate data extraction; (7) provision of excluded study list; (8) detailed presentation of included studies; (9) use of established ROB assessment tools; (10) funding sources for included studies; (11) use of appropriate meta-analytic methods; (12) assessment of the impact of ROB on the synthesised results; (13) accounting for ROB in interpreting the results; (14) assessment of heterogeneity; (15) investigation of publication bias; (16) disclosure of conflict of interest), which are rated as ‘yes’ or ‘no’, and then assign an overall confidence in the results of each systematic review as ‘high’, ‘moderate’, ‘low’ or ‘critically low.’19

Two reviewers also will independently assess primary studies individually. For RCTs, we will use the revised tool to assess ROB in randomised trials (RoB 2 tool).20 We will assess five domains of RCT study validity (ie, randomisation process, deviations from intended interventions, missing outcome data, measurement of outcomes, selective reporting) and then assign an overall ROB for each trial. For NRSIs, we will use the Risk of Bias In Non-randomized Studies of Interventions tool for cohort studies,11 and the Cochrane Risk of Bias Assessment Tool for Non-Randomized Studies of Interventions for case-control studies.21 We will assess seven domains of study validity (ie, confounding, participant selection, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes and selective reporting) and then assign an overall ROB for each study.

Meta-bias

We will assess funnel-plot asymmetry for assessing small-study effects and non-reporting biases (including publication bias) if ≥10 studies per each specific outcome assessed in a specific design (ie, for RCTs and NRSIs separately) are included and data are deemed amenable to quantitative synthesis.22

Data synthesis

First, we will qualitatively synthesise the results by using tables and graphs. Specifically, we will first depict overlap of the primary studies included in the eligible systematic review applying methods on the basis of the number of included reviews and primary studies.23 Then, we will qualitatively compare the characteristics of the PICOTS items extracted from the non-overlapping primary studies by each design (ie, RCTs and NRSIs separately) and assess the presence or absence of clinically important between-study heterogeneity. We also will visually assess statistical between-study heterogeneity by using forest plots if appropriate.

If data are determined to be amenable to quantitative synthesis according to the proposed guidelines,24 25 we will then perform study-level pairwise hierarchical random-effects model meta-analysis by using a fully Bayesian framework26 with evidence-based informative priors for the contrast heterogeneity variance, tau.27 For studies reporting binary-count data, we will perform a random-effects meta-analysis to obtain summary relative risk estimates by using the binomial likelihood with logit link in a generalised linear modelling framework.26 For studies reporting continuous outcome data, we will use the standardised effect estimates and their variances and perform a random-effects meta-analysis by using the conventional, approximate ‘normal–normal’ model.26

If appropriate, we will consider extending the pairwise meta-analytical model to a network meta-analysis to combine both direct and indirect comparative data via common comparator groups to synthesise all available comparisons in a single analysis.26 In this case, we will use a contrast-based network meta-analysis model with fixed study intercepts proposed by Lu et al,28 which is a recently recommended standard network meta-analysis model.29

Additional analyses

We will formally assess between-study heterogeneity quantitatively by estimating the between-study heterogeneity parameter, tau and I2 statistics and their corresponding 95% credible intervals. An I2>50% will indicate intermediate heterogeneity, whereas an I2>70% will indicate high heterogeneity.30 If high heterogeneity is suspected, we will explore influential and/or outlier studies that might explain the heterogeneity by constructing Graphical Display of Study Heterogeneity plots.31

We will perform subgroup analyses and study-level univariable random-effect meta-regressions if there are ≥10 studies per specific outcome of interest.26 Preplanned candidate factors will include cancer categories (ie, colorectal cancer, breast cancer, prostate cancer, lung cancer and cervical cancer), study-level overall ROB assigned (ie, high, moderate, low or critically low), specific presentation forms (written materials, videos or computer-based or web-based interactive materials) and delivery methods (eg, one-on-one meeting vs educational session; face-to-face meeting vs teleconference).

Statistical analysis

We will conduct all analyses by using Stata V.16.1 SE (Stata Corp.) and WinBUGS V.1.4.3 (MRC Biostatistics Unit, Cambridge, UK). All tests will be two sided, and statistical significance will be accepted for p<0.05.

Patient and public involvement

We did not involve patients or the public in the preparation of this umbrella review protocol.

Ethics and dissemination

Ethics approval is not applicable as this is a secondary analysis of publicly available data. The findings from the review will be disseminated through publications in peer-reviewed journals and presentations at conferences.