Methods for the health technology assessment of complex interventions: a protocol for a scoping review.

INTRODUCTION: In healthcare policy and economic literature, research on the health technology assessment (HTA) of complex interventions (CIs) is becoming increasingly important. In many developed countries, HTA guides decision-making to help achieve greater value for money when funding health care. However, research has yet to identify the forms of evidence and evaluation criteria that should be used in the HTA of CIs. Previous research has established that the HTA of CIs requires multiple factors to be evaluated but there is no agreement on which factors ought always to be considered. There is equally little agreement on which forms of evidence ought to be collected or synthesised and how. We plan to perform a systematic scoping review in order to identify the range of evaluation criteria and types of evidence currently used in the HTA of CIs. METHOD AND ANALYSIS: This protocol was developed to guide the methodological framework for the conduct of a scoping review on health technology assessment (HTA) of complex interventions (CIs), using the Joanna Briggs Institute guidelines and the six-stage framework proposed by Arksey and O'Malley, in addition to more recent innovations in scoping review methodology. A grey literature search will supplement the primary searches of seven electronic databases for studies available in English between January 2000 and August 2020. Two reviewers will independently screen all search results for inclusion and data will be extracted using a customised data extraction or charting form. Any dispute will be resolved by consensus or through arbitration by a third author. The mnemonic Population, Concept and Context will be adopted to establish criteria for selecting relevant literature, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses: Extension for Scoping Review will be used for reporting the results. Several explanatory-descriptive methods will be used for analysing the extracted data including frequency and trend analyses as well as reflexive thematic coding and analysis.Mapping evidence on the HTA of CIs will allow us to gain a better understanding of both established and emerging practices, including the information types, requirements, values and parameters that are incorporated in the HTA of CIs. We also expect the findings of the scoping review to help identify research gaps that will guide future studies. As healthcare becomes more complex in its delivery, it is timely to determine how these complex interventions should be assessed so that policy decisions can be made about whether implementation and public funding is warranted. ETHICS AND DISSEMINATION: This scoping review will involve secondary analysis of already collected data, and thus, does not require ethics approval. The research findings will be submitted to peer-reviewed journals for publication and will also be disseminated at conferences and seminars. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This protocol outlines the methodological framework for conducting a scoping review of health technology assessment methods to evaluate complex interventions.

The scoping review aims to systematically search, identify and map different understandings of complex interventions. It also proposes to map and synthesise the approaches used in health technology assessment to evaluate complex interventions.

The scoping review will be driven by an improved analytical method grounded in previous research.

The review is limited to English-language studies published between January 2000 and August 2020.

This review may not include all health technology assessment methodologies for complex interventions as some may not have been publicly documented.

Introduction

Recent improvements in health outcomes for patients have largely been attributed to the development of innovative approaches to delivering healthcare as well as the increasing adoption, diffusion and use of health technologies or models of service delivery.1–3 However, previous research has emphasised that new and emerging health technologies or interventions are not always accessible, available or affordable to all potential patients.4–9 This is particularly a problem when the medical condition is chronic and the intervention is complex, for example, caring for patients with dementia.10–13

Health technology assessment (HTA) was introduced in the 1980s to guide the development of evidence-based health policy and funding decisions.12 14 It has already facilitated patient access to cost-effective technologies that improve health outcomes, and contributes to value for money investment in the context of scarce health resources.15 16 Complex interventions (CIs) are being increasingly studied using HTA methods but clarification and guidance is needed on how CIs should be assessed to properly inform public funding decisions in healthcare.

The HTA of CIs is an emergent area of research17–19 yet it is somewhat complex and fragmented because it has not yet been systematically studied and there appears to be no consensus on what constitutes a CI. We are, therefore, proposing to conduct a scoping review of the approaches used by HTA practitioners when assessing CIs to help guide decisions on the funding of these interventions. This protocol is for such a systematic scoping review of the literature.

Health policy-makers and professionals increasingly insist on good evidence to support decisions about the development, adoption, acquisition, appropriateness, utilisation, pricing, procurement and reimbursement of health technologies.20 The development and expanding use of HTAs in the public and private sectors in developed economies and recently in low to middle-income countries21–27 reflect this insistence. In particular, the study of the HTA of CIs has gained prominence in recent years.17 28 29

According to the Australian Department of Health16 an HTA includes a variety of processes and mechanisms that use scientific evidence to evaluate the safety, efficacy, clinical effectiveness and cost-effectiveness of health services. Without concrete evidence, however, the uptake and diffusion of health technologies is likely to be influenced by a range of social, financial and institutional factors, resulting in suboptimal health outcomes and inefficient use of resources.30

The International HTA Definition Joint Task Group31 defined HTA as:

A multidisciplinary process that uses explicit methods to assess the value of a health technology at different points in its lifecycle. The purpose is to inform decision-making to promote an equitable, efficient, and high-quality health system.

In turn, a health technology comprises any intervention—being a test, device, medicine, vaccine, procedure, programme or system—developed to assess, prevent, diagnose or treat medical conditions; promote health; provide rehabilitation; organise healthcare delivery; or support physical, psychological or social functional and behavioural changes.32 33 A health intervention has also been broadly defined as ‘an act performed for, with or on behalf of a person or population whose purpose is to assess, improve, maintain, promote or modify health, functioning or health conditions’.32

By contrast, there is no agreed definition of a ‘CI’ in health. A broad, inclusive definition of a CI includes the following components.34–38

A CI contains multiple elements or components that adapt to systemic and environmental changes (there is intervention complexity).

A CI has simple, complicated or multiple causal pathways with feedback loops, and it has mediators and moderators with synergistic effects (there is pathway complexity).

A CI emphasises non-linear relationships, interactions and interconnections between components, for example, communication and control (there is interaction complexity).

A CI targets multiple actors, for example, individuals, groups and/or institutional levels (there is population complexity).

A CI is responsive or sensitive to initial conditions. Changes in initial conditions may present emerging behaviour and variable responses or outcomes whose details are not predictable (commonly referred to as ‘the butterfly effect’).

A CI requires multifaceted adoption, uptake or integration methods, perspectives or strategies (there is implementation complexity).

A CI exists within systems and operates in a dynamic multidimensional environment (there is contextual complexity).

A CI causes practical and methodological difficulties for assessors (there is evaluation complexity).

In a nutshell, a CI is a heath technology with multiple components that interact synergistically in non-linear, dynamic pathways that depend on their history (initial conditions) and a context in which patterns, reactions and outcomes are not easily predicted.

Interventions can be understood as being simple, complicated, complex or chaotic. Formal rule-based interventions with anticipated impacts fall into the simple (known) domain. An example of a simple intervention can be vaccination, medication or injection of insulin in accordance with protocols in place. In this domain, clinical practitioners and patients can easily standardise the procedure and it is known what the effects and consequences of the mechanisms are.39 40 Complicated interventions correspond to the knowable domain, meaning that the relationship between cause and effect may not be fully known by all decision-makers but can be known through reductionist ways of thinking and objectivity, meaning that human behaviour or complex phenomenon can only be explained by breaking it down into smaller/simpler component parts.41 An example of a complicated intervention could be a surgical operation where expertise, detailed protocol and latest scientific evidence are all required in order to do the procedure.18 40 42 Simple interventions call for ‘best practices’ and complicated ones apply ‘good practices’.43

CIs, on the other hand, sit in the domain of ‘emergent practice’ that cannot be solved with ‘best’ or ‘good’ practices alone. For CIs, a fixed or static evaluation is unlikely to be sufficient; rather, experimentation is required to probe the components of the intervention and determine which factors are critical to its success. In this domain, the relationship between cause and effect can only be perceived in retrospect, not in advance.44 45 Integrated care, behavioural interventions for medication compliance and outpatient palliative care services are examples of CIs, where the success in managing the problems of the first patient is not an indication that others will be treated effectively since no fixed rules, guidelines or formulae exist to warrant success.37 39 46 47 Cognitive–behavioural therapy (CBT) for depression, for instance, consists of multiple components that interact to improve the condition, even though we may not be sure how these components interact, nor what they do to each other. Such an intervention may involve multiple treatment approaches, aimed at providing support on multiple levels (eg, individuals, groups, systems and community) by various professionals and different means and in a non-linear manner. Changes in results are not proportional to any changes in input, and the technique may require multifaceted training or adoption, and require special consideration of the environment or the context in which CBT is developed, implemented and assessed.

HTA methods have been developed to assess the quality, safety, efficacy, effectiveness and cost-effectiveness of health interventions and programmes, but CIs are often complex in ways that pose challenges to the use of conventional HTA methods. Current HTA methods and practices have largely been tailored towards simple and complicated interventions where specific plans, procedures, techniques, guidelines and protocols are developed to direct action and achieve outcomes. HTA traditionally occurred at the point of a new technology’s entry into the health system, rather than once the technology was established. HTAs to inform a public funding decision were uncommon when it related to changes in the organisation and delivery of established healthcare. With recent moves in some health systems to integrate HTA practices into implementation science and health technology management,28 48 49 the types of interventions being evaluated have changed. While HTA methods work quite well in simple and complicated situations, there is a growing need to improve how HTA handles CIs in practice.

Previous research has recognised that HTA is multifactorial. Healthcare decision-makers rely heavily on HTAs to achieve greater value for money, an overarching concept that encompasses comparative safety, relative effectiveness, cost-effectiveness, social consequences, organisational implications and legal and ethical aspects of a technology.50–53 However, there is a view that it does not fully consider the real-world context, complexities and contingencies relating to health technologies, and relies more on isolating single factors.18 44 54 Recent studies on HTAs of CIs have identified many relevant factors such as context (eg, setting and socioeconomic or sociocultural aspects), implementation issues, patient-related factors and preferences, safety profile and innovation level.55 56 However, the relevance of these studies has been constrained by the applicability of HTA reports on CIs across different jurisdictions and various levels of government or decision making.57 Recognition that CIs are contingent on their context not only has implications for the effect of the CIs but also for their acceptability; feasibility of development, evaluation and implementation; and sustainability.58

There is no apparent consensus on what constitutes a CI and how different stakeholders, experts or researchers define it, nor what evidence and evaluation criteria should be used in the evaluation of CIs (and hence HTA-based decision making). Thereby, it is unclear what dimensions of context foremost determine if a technology should be funded or not. It is also unclear how HTA is being conducted at the international level to formally evaluate CIs and to what extent the proposed or used evaluation criteria and types of evidence differ according to the definitions used.

In view of these research gaps, we will conduct a systematic scoping review to map how CIs are defined in HTA and also determine the range of methods used to evaluate CIs in HTA. This will help to identify any knowledge gaps. More importantly, it will help HTA experts and health policy-makers to understand the types of information, criteria, requirements and processes that currently feature in HTAs of CIs, and thus better equip them to decide between options for conducting such HTAs.

We describe the protocol proposed for undertaking and reporting this scoping review.

Methods and analysis

This scoping review protocol was designed to guide the formulation of review aim(s) and questions, eligibility criteria and conduct characteristics of the review.

Previous studies have proposed a variety of review types to synthesise the available evidence, of which the ‘scoping review’ or ‘scoping study’ approach has recently gained considerable popularity and recognition as a transparent and systematic methodology for examining a broadly covered but unclear topic. The aim is to provide a comprehensive map of the available literature and to identify key concepts, types of evidence, theories or research gaps.59 This allows research scholars and policy-makers to explain the working concepts and theoretical limits of a particular topic, and to make informed decisions within a shorter time frame.60 61 The first scoping review framework was most likely published in 2005,62 and as such, this form of analysis is a relatively new approach63 for which there is still no universal definition, consistent application or definitive procedure for performing or reporting in all research fields.63 64 The following features, however, are specific to most definitions:

Scoping reviews (or scoping studies), also known as ‘mapping’ reviews, represent a method of evidence synthesis to explore research concerns,61 65 identify research gaps,62 update empirical evidence on new research areas,65 summarise and disseminate findings from a body of knowledge that is heterogeneous in methodology or discipline,66 map or chart the extent (ie, volume, range and type) as well as the nature and characteristics of evidence underpinning a research area,62 67 and may be used where the research area is broad, complicated or not comprehensively reviewed.61 67

Scoping reviews are not necessarily a quick alternative to systematic reviews.62 They represent a type of synthesis rather than a type of systematic review, but many of the steps and processes taken in systematic reviews are reflected in scoping reviews. The differences are slight and relate to the research aims and aspects of method. Scoping reviews, for instance, typically do not include critical appraisal, nor are the findings used to produce policy or practice recommendations since they are reviews that describe rather than evaluate and report.60

This scoping review will be part of a large-scale study on the ‘HTA of CIs’.

The rationale for the choice of scoping review methodology

For this study, we determined that a scoping review would be the most appropriate method of collating the research evidence on how HTA is undertaken for CIs. The HTA of CIs is an emerging research area that is still fragmented, complex, broad, poorly understood, understudied or not researched in detail.68 69 The scoping review represents the most suitable method for responding to the questions of the proposed study, as it aims to outline, map, synthesise and disseminate different concepts and types of evidence that exist around the conduct of HTA in relation to CIs, and identify the gaps for further research.

Scoping reviews are susceptible to bias and must be rigorously and transparently designed and prepared to ensure the findings and reports are accurate and relevant for end users.61 62 65 66 70 71 The (The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR)66 will form the basis for reporting findings in the present scoping review. The PRISMA-ScR checklist is mainly used to ensure the findings are adequately reported and that the review is replicable and therefore credible.72

Registration of the review protocol

In line with systematic review methodology73 and scoping review methodology,74 75 the current study begins with a protocol developed before undertaking the actual scoping review in order to minimise bias in the review process. Developing such a protocol is critical as it predefines the research aim(s), questions and methods to be addressed, with criteria for the inclusion and exclusion of literature that relates to the scoping review’s aim(s) and question(s). The protocol provides a systematic approach to the conduct and reporting of the review, allows for transparency of the process, and enables readers to understand how the findings of the scoping review have been arrived at. Unlike a typical systematic review, which seeks to answer a specific or series of question(s)—based on very precise inclusion criteria, for example, on the basis of the Population, Intervention, Comparator and Outcome (PICO) elements—the scoping review has a wider ‘scope’ with correspondingly less restrictive inclusion criteria. It is intended to respond to a series of broad/open questions based on the elements of the (Population, Concept and Context (PCC) inclusion criteria.75 Since the purpose of the proposed scoping review is to identify, explore and map international efforts related to the conduct of HTA of CI—including the processes and/or methodologies used in the HTAs of CIs—we used the latter.

This protocol was prospectively registered with the Open Science Framework (https://osf.io/kv9hu/) since the PROSPERO database does not accept systematic scoping review protocols. An official record of this protocol promotes transparency and assists in reducing duplication of the work, as will this protocol publication.

Patient and public involvement

No patient involved.

Stages of the proposed scoping review

The scoping review outlined in this protocol will be driven by an improved analytical method grounded in previous research. This will conform to Arksey and O’Malley’s six-stage methodological framework62 but feature further refinements made by Levac, et al61 and the Joanna Briggs Institute (JBI).75 We will also draw on insights from more recent innovations in Arksey and O’Malley’s framework made by Colquhoun et al65 and Tricco et al66 in setting out the plan and reporting of the review.

We will rely on the following six stages in this scoping review: (1) developing the research question, (2) identifying relevant studies, (3) selection of eligible studies, with the establishment of a set of inclusion and exclusion criteria; (4) charting the data, that is, sifting, charting and sorting information based on the key issues and themes; (5) collating, summarising and reporting the results, and (6) consulting key stakeholders to inform and validate study findings.

Stage 1: developing the research question(s)

The first stage in undertaking a scoping review is research question formulation and linking the question(s) to the study aims/objectives.61 62 Research questions will direct the subsequent stages of this review. Whereas systematic reviews address specific and focused research questions,73 scoping studies feature research questions that are broad and the focus is on mapping and summarising the extent and nature of the evidence.61 65

The main research question for this scoping review is ‘how is HTA being done by HTA evaluation agencies to formally evaluate CIs at the international level?’. The research subquestions are:

What different definitions of a CI exist in the literature, and how do these definitions differ?

What evaluation criteria and domains are assessed in the HTA of CIs?

To what extent do the evaluation criteria and evidence types differ according to the CI definitions used?

Does the approach differ from country to country?

What is current practice in the HTA of CIs in Australia?

We have a particular interest in what is being done to evaluate CIs in Australia, so we are prespecifying Australia for subgroup analysis as part of our review.

The proposed scoping review will use the PCC framework to align the study selection with the research question. The JBI75 recommends PCC as a less restrictive alternative to the PICO (recommended for systematic reviews) for the construction and interpretation of scoping review questions and to describe elements of the inclusion criteria.

Stage 2: identifying relevant studies

The identification of relevant studies and information sources involves the creation of a search strategy, underpinned by inclusion and exclusion criteria. These criteria are categorised under the broad PCC mnemonic recommended for scoping reviews, as set out below.75 We sought the advice of a senior medical librarian to apply the search strategy to the widest possible degree, since the main attribute of a scoping review is full coverage of the literature on the subject. As a consequence, the search strategy will follow the three-phased JBI process,75 and extend to three more additional phases as follows:

The first phase entails the identification of indexing terms and text words. To inform these, a scoping search of two relevant online databases—that is, Medline (PubMed) and Embase—will be pilot tested to compile and preselect a list of keywords from titles, abstracts and index terms used in publications most relevant to the topic. Citations will be downloaded into an Endnote database and keywords and indexing terms will be identified. Use of the PubReMiner (a free version data mining tool: http://hgserver2.amc.nl/cgi-bin/miner/miner2.cgi) will also retrieve the most frequently occurring keywords and indexing terms (Medical Subject Headings, MeSH) that will be incorporated into the final search query. The keywords and Mesh terms retrieved will provide the foundation for creating a definitive search strategy through Medline (PubMed interface). Using an adapted version of the PRESS Evidence-Based Checklist for Peer Review of Electronic Search Strategies, this preliminary search strategy will be peer-reviewed independently by external HTA evaluators with information specialist expertise76 and readjusted following the feedback.

The review will focus on English language evidence published worldwide between January 2000 and August 2020, and also a purposive sample of selected countries (and their respective HTA bodies) for their HTA reports and guidance documents. This consists of a diverse mix of countries including members of International Network of Agencies for Health Technology Assessment (INAHTA) (www.inahta.org) as indicated in box 1.

INAHTA Members

ACE-Agency for Care Effectiveness, Singapore

AETS-Agencia de Evaluación de Tecnologias Sanitarias, Spain

AETSA-Andalusian Agency for Health Technology Assessment, Spain

Agenas-The Agency for Regional Healthcare, Italy

AHRQ-Agency for Healthcare Research and Quality, United States of America

AHTA-Adelaide Health Technology Assessment, Australia

AIHTA -Austrian Institute for Health Technology Assessment, Austria

AOTMiT-Agency for Health Technology Assessment and Tariff System, Poland

AQuAS-Agència de Qualitat i Avaluació Sanitàries de Catalunya, Spain

ASERNIP-S-Australian Safety and Efficacy Register of New Interventional Procedures -Surgical, Australia

ASSR-Agenzia Sanitaria e Sociale Regionale (Regional Agency for Health and Social Care), Italy

AVALIA-T-Galician Agency for Health Technology Assessment, Spain

CADTH-Canadian Agency for Drugs and Technologies in Health, Canada

CDE -Center for Drug Evaluation, Taiwan, Republic Of China

CEDIT-Comité d’Evaluation et de Diffusion des Innovations Technologiques, France

CHQAC-Center for Healthcare Quality Assessment and Control, Russian Federation

CONITEC-National Committee for Technology Incorporation, Brazil

DEFACTUM-Social & Health Services and Labour Market, Denmark

FinCCHTA-Finnish Coordinating Center for Health Technology Assessment, Finland

G-BA-The Federal Joint Committee (Gemeinsamer Bundesausschuss), Germany

GOeG-Gesunheit Österreich, Austria

HAD-Uruguay-Health Assessment Division, Ministry of Public Health, Uruguay

HAS-Haute Autorité de Santé, France

HTRG-Health Technology Reference Group, Australia & NEW Zealand

HIQA-Health Information and Quality Authority, Ireland

HIS-Healthcare Improvement Scotland, United Kingdom

HQO-Evidence Development and Standards Branch, Canada

HTW-Health Technology Wales, United Kingdom

IACS-Health Sciences Institute in Aragon, Spain

IECS-Institute for Clinical Effectiveness and Health Policy, Argentina

IETS-Instituto de Evaluación Tecnológica en Salud, Colombia

IETSI-Institute of Health Technology Assessment and Research, Peru

IHE-Institute of Health Economics, Canada

INEAS-National Authority for Assessment and Accreditation in Healthcare, Tunisia

INESSS-Institut national d’excellence en santé et en services, Canada

IQWiG-Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, Germany

KCE-Belgian Health Care Knowledge Centre, Belgium

MaHTAS-Health Technology Assessment Section, Ministry of Health Malaysia, Malaysia

NECA-National Evidence-based healthcare Collaborating Agency, Korea

NICE- National Institute for Health and Care Excellence, UK

NIHR-National Institute for Health Research, UK

NIPH-Norwegian Institute of Public Health, Norway

OSTEBA-Basque Office for Health Technology Assessment, Spain

RCHD-Ministry of Public Health of the Republic of Kazakhstan, Republican Centre for Health Development, Kazakhstan

SBU-Swedish Agency for Health Technology Assessment and Assessment of Social Services, Sweden

SEC-Department of HTA at the State Expert Centre of the Ministry of Health, Ukraine

SFOPH-Swiss Federal Office of Public Health, Switzerland

UVT-HTA Unit in A. Gemelli Teaching Hospital, Italy

ZIN-Zorginstituut Nederland, The Netherlands

ZonMw-The Netherlands Organisation for Health Research and Development, The Netherlands

Non-INAHTA Members

The Department of Health Technology Assessment, Iran

HIRA-Health Insurance Review and Assessment Service) South Korea

Institute of Healthcare Technology Assessment, Shomachi and Department of Technology Assessment and Biostatistics, Japan

HiTAP-Health Intervention and Technology Assessment Program, Thailand

CMTP-Centre for Medical Technology Policy, United States of America

KDTD Turkish Evidence Based Medicine Association, Turkey

CMeRC-HTA Unit Charlotte Maxeke Research Cluster-Health technology Assessment Unit, South Africa

In choosing target countries, the review aims to include countries that have a diverse set of HTA mechanisms in place (eg, clinical benefit assessment plus economic evaluation verses clinical benefit assessment alone) along with those with different levels of economic development from high-income, middle-income and low-income countries. The rationale for selection of these counties would also be the variation in their health system financing (eg, tax-supported, social security-based or private insurance corporations and out-of-pocket), the organisation of the healthcare delivery system (centralised verses decentralised structure), and the perspective used in HTA (health system verses societal perspective), so that the sample can be representative of the major types of health systems and HTA approaches across different nations.

The rational for choosing the INAHTA HTA agencies is that these agencies

Assess technology in healthcare.

Are non-profit organisations.

Relate to a regional or national government.

Are funded at least 50% by public sources.

Provide free access to publicly available reports to other INAHTA members on request. (www.inahta.org).

Data are limited to evidence published in English language because the research team is unable to obtain and translate non-English literature and remain within the expected project timeline. However, we will retain the abstracts/summary for non-English studies with English abstracts that seem to correspond to our inclusion criteria (aside from the language) for the purposes of reporting.

Keywords and MeSH terms selected for the scoping search (based on the PubMed platform) include: ((“complex intervention”[tiab] OR “complex technology”[tiab] OR “complex technologies”[tiab] OR “complex program”[tiab] OR “complex programme”[tiab] OR “multi-component intervention”[tiab] OR “multi-component program”[tiab] OR “multi-component programme”[tiab] OR “multi-part intervention”[tiab] OR “multi-part program”[tiab] OR “multi-part programme”[tiab] OR “multifactorial intervention”[tiab] OR “bundled intervention”[tiab] OR “bundled program”[tiab] OR “bundled programme”[tiab]) AND ((HTA OR “health technology assessment” OR “Technology Assessment, Biomedical”[Mesh] OR “Comparative Effectiveness Research”[Mesh] OR “Evidence-Based Medicine”[Mesh] OR evaluat*[tiab] OR assess*[tiab] OR reimburs*[tiab] OR fund*[tiab] OR insur*[tiab] OR “health technology management”) OR (“Health”[Mesh] AND (“Decision Making”[Mesh] OR “Decision-mak*”[tiab] OR “value assessment” OR Technology Assessment [tiab] OR policy [tiab] OR “Policy Making”[MAJR]))). The yield from the electronic database search will be recorded in a table.

The second phase includes developing and implementing unique database queries for each database included in the protocol, based on the list of keywords and index terms retrieved from the previous phase. Apart from Medline/PubMed, these databases will include the Cochrane Library [Cochrane Database of Systematic Reviews and Cochrane Central Register of Controlled Trials ], PsycINFO, CINAHL, Embase, Epistemonikos and the INAHTA HTA database.

The third step builds on the data analysis of the preliminary searches and involves a manual search for potential publications not found through the online search of databases by reviewing the archives of three leading journals with the highest number of published papers in HTA of CIs.

In the fourth phase, a search will be performed to identify grey literature on the topic. This requires manual searches of any non-indexed and unpublished literature of relevance to this review including research in progress, theses, in-press articles, technical HTA reports, guidelines and procedure documentation concerning the evaluation of CIs. Searches will be conducted through Google, websites of HTA international (https://www.htai.org/), the International Society of Pharmacoeconomic Outcomes Research (https://www.ispor.org/), as well as and the websites of HTA agencies that are members of INAHTA (http://www.inahta.org/).77 The HTA websites and repositories outlined above will be reviewed for guidance documents and actual reports of HTAs of CIs. The search strategy in this phase will help avoid publication bias and identify any other relevant material(s) for inclusion, so the study will not be narrowed to only peer-reviewed papers, noting that HTA is a policy science and much of the documentation does not make it into peer reviewed journals. If necessary, authors of original guidance documents and HTA reports will be contacted for further information or missing data.

On the successful completion of Stage 2 that is, the search strategy, search results (final studies identified) will be collated and imported into the most recent EndNote software (Clarivate Analytics), with duplicate entries removed.

AB will develop the searches and these will be peer reviewed by TM and independently by an information specialist. On finalisation of the search strategy, AB will conduct the electronic searches. The search strategy will be limited to the English language, and those studies published between January 2000 and August 2020. Year 2000 was chosen as the starting point for this review as many HTA and CI guidelines were developed from this date.78 79 The search will be updated just prior to submitting the results for publication.

Stage 3: screening and selection process (selection of eligible studies)

Eligibility criteria

The relevant documents to address the research question will be selected using the PCC study selection criteria (table 1).

Table 1

PCC framework for selection of eligible studies

PCC element	Definition/determinants: (per JBI reviewer’s Manual Ch.11)	Inclusion criteria
P-Population*(Who?)	Documents produced by HTA agencies or HTA networks, HTA evaluators or HTA methodologists	Published studies and grey literature in the form of HTA reports or HTA guidance or methods documents HTA reports include secondary research studies that involve systematic reviews, realist reviews, meta-analyses, meta-syntheses, mixed-methods reviews, qualitative reviews, rapid reviews with/without economic evaluations, budget impact analyses and ethical, social, legal and organisational analyses undertaken to specifically inform a health policy decision. Studies showing that their full-texts are available in English through academic journals, institutional repositories, archives or other collections of scientific and other articles
C-Concept†(What?)	How health technology assessment* of CIs† is undertaken? How health technology assessment of CIs ought to be conducted? * HTA is defined as outlined on page 1† CI is defined as outlined on page 1	Must have a specific focus on the HTA of CIs in healthcare
C-Context‡(With What Qualifiers?)	All settings are considered. HTA must be conducted for an access or funding decision whether at the national, regional or hospital level.	Global search for all published studies and grey literature Search of a purposive sample of selected countries (and their respective HTA bodies) for their HTA reports and guidance documents from the INAHTA member list (www.inahta.org) English language studies available between January 2000 and August 2020

*Population/participant entails important characteristics of the study population, setting or participants such as age, gender and other qualifying criteria.

†Concept includes details that relate to elements that would be detailed in the scoping review such as the ‘CIs’, HTA, outcomes or other ‘phenomena of interest’. The concept should be clearly articulated to guide the scope and breadth of the research.

‡Context is defined as the conditions and circumstances that are relevant to the application of an intervention, for example, setting (eg, hospital) and sociocultural aspects (knowledge, beliefs, conceptions, customs, institutions and any other capabilities and habits acquired by a group that may influence uptake).56 Cultural factors, socioeconomic issues, geographical location, politics, specific racial preferences, gender interests may be included in context.

CI, complex intervention; HTA, health technology assessment; INAHTA, International Network of Agencies for Health Technology Assessment; JBI, Joanna Briggs Institute.

The screening and selection of eligible studies will be carried out in two phases, beginning with the initial screening for relevant studies based on the title and/or abstract, followed by full-text review of those studies selected at the screening stage. Both phases will involve independent duplicate screening of a random 10% of the records to ensure the screening approach taken is reliable. A kappa score will be calculated to determine inter-rater reliability. Reviewers will confer to determine the reason for any differences in study selection and the approach will be amended accordingly. Any paper over which there is a lack of consensus for inclusion in the review will be adjudicated by a third independent reviewer.63 80 81 A single reviewer will screen the remaining database.

In the first screening stage, the citation or study record will be screened as ‘included, excluded or uncertain’. In the second stage, the full-text article will be retrieved of those categorised as included or uncertain and the study selection criteria applied again until there is a definitive decision regarding study inclusion. The third screening stage consists of checking the reference lists of included studies—that is, all included studies will be scanned and cross-checked for more relevant studies (backward citation searching).

We will document the reasons for the inclusion of uncertain studies and exclusion of other resources in a PRISMA-ScR flow diagram.66 Details of the literature screening and results obtained will be outlined narratively and presented in a graphical diagram developed from the PRISMA-ScR flow chart66 or in accordance with the JBI guideline.75

Exclusion criteria

We will exclude studies that specifically do not meet the above inclusion criteria. Studies without evidence of HTA of CIs, those conducted or published in languages other than English and those where the full-text cannot be accessed will be excluded. Exclusions will also apply to other forms of publications including letters, commentaries or editorials, narrative reviews, study protocols, as well as conference abstracts and presentations.

Stage 4: extracting and charting the data

In this stage, the data from all the studies included will be collected and charted using an adapted or structured descriptive-analytical approach. The process by which data is extracted in scoping reviews is termed ‘charting’ where a logical and descriptive summary of the study results must be recorded corresponding to the aims and questions of the scoping review.63 72 82 The data extraction template (known as data charting form or data abstraction tool) will reflect all the variables that are consistent with the research questions and aims of the scoping review.66 83 It is designed in detail to capture the relevant information about the study’s key characteristics and detailed information about all the terms and metrics used to describe and summarise the phenomena of interest, that is, the HTA of CIs. The supplementary file contains the ‘data extraction and charting form’ and outlines a detailed list of these components (online supplemental table 1). The data extraction form presented in the supplementary file will be piloted with not more than 10% of the included studies by the key reviewer (AB), and further examined within the research team to ensure accuracy and consistency of the extracted data.84

Stage 5: collating, summarising and reporting the findings

The distinctive purpose of conducting a scoping review is to compile the results and to provide an overview of narrowly defined matters rather than a meta-synthesis of results.62 This stage aims to summarise the findings collected in order to explore the known and unknown aspects of the given topic and to identify areas that need to be studied further.82 83

Stage 5 will primarily draw on recent innovations in reporting scoping review results including the PRISMA-ScR.66 It will also make use of an interpretative methodology for presenting a narrative or descriptive account. A narrative synthesis will interpret and summarise the extracted findings in relation to the proposed scoping review question and in the light of the main research aims.84–86 Some researchers argue that the inherent challenges to undertaking a scoping review is to establish a framework for presenting the narrative account.82 85 This will be achieved through an inductive data analysis approach such as reflexive thematic analysis.87 In doing so, we will import the extracted data from the charting tool into the NVivo software V.1288 for coding and analysis. Themes will then be created and their relationship to the research question and aims will be examined. The collection, abstraction and reporting process will also draw on previous works, where a visual representation or numerical overview of amount, distribution and evolution (ie, frequency and trend analyses), and nature of evidence on HTA of CIs will be presented through figures or tabular formats.61 72 86 89 The mapping approach suggested by Miake-Lye et al will be specifically used to enhance the mapping results from the literature screening stage.90 The process of mapping the qualitative data, if any, will follow an initial coding, focused coding (categorisation), data displaying through diagrams or tables, pattern and themes identification, summarising and synthesising. Such ‘evidence mapping’ is pivotal to performing a scoping review and displays the research context or environment graphically. These proof maps provide better identification of research patterns and themes and help to postulate explanations for summarising and synthesising the results.

Finally, the meanings and implications of the findings in relation to the aims of the study will be discussed and reported for future research, policy, and practice.

Methodological quality appraisal

Competing arguments exist regarding whether scoping reviews ought to include the methodological quality appraisal of primary studies. While some support inclusion, many others object. According to Peters et al,63 a formal quality assessment of the primary studies is not generally performed within a scoping review because unlike systematic reviews, scoping reviews are mainly designed to provide an overview or map of the existing evidence base regardless of quality. As this research is aimed at mapping the available research64 on HTAs of CIs, this scoping review will not undertake any formal assessment of the methodological quality of the included studies.

Stage 6: consulting key stakeholders

The present scoping review constitutes the first study of a five-stage research project. There will not be any consultation exercise or stakeholder involvement during the scoping review. However, stakeholder consultation will occur in the later stages of the major research project. A web-based survey and a two-round Delphi study of decision-makers, HTA experts and other stakeholders will be conducted to obtain more information and perspectives regarding the definition of CI and the HTA methods for evaluating CIs.62

Discussion

HTA has been developed as a form of policy science that aims to inform policy and practice in the delivery and financing of healthcare interventions and technologies.91 New health technology, however, is not always available to all potential patients, accessible, or affordable.6–8 This is a problem for CIs.10 11 Despite many advances in the application, implementation, methods and practices of HTA in Australia and other international settings, there remains a need to improve the transparency and consistency in regulatory and reimbursement decisions related to CIs. The HTA of CIs is an emergent area of research yet understanding of it is somewhat complex and fragmented because it has not yet been systematically studied.

To the best of our knowledge, there is no comprehensive scoping review of the HTA of CIs. Mapping evidence on the HTA of CIs will enable us to better understand both established and emerging practices, including the information types, criteria and values that are used in the assessment of these interventions. We developed an a priori protocol to undertake a systematic scoping review study on the HTA of CI across the world. The scoping review methodology proposed by Arksey and O'Malley62 will primarily guide the review, along with insights representing more recent innovations in the field.61 65 66 75 85

We propose to adopt a step-by-step procedure that will adequately address the requirements for our scoping review, especially in terms of data extraction, data analysis and reporting of the results. This helps us to boost our approach’s feasibility, rigour, reproducibility, reliability and credibility, including by minimising reporting bias.92

The review, however, is subject to a number of limitations. The main limitation of the current scoping review is that some HTA practices may remain undocumented or not publicly accessible, so even searching HTA agencies’ websites may not provide the full picture of evaluation criteria used to assess CIs. The review is also limited in that we can only include evidence that is in English because of resource constraints. This is not an uncommon reason and may lead to some relevant research being excluded,93 although in the majority of cases the exclusion of non-English studies will have minimal impact on the overall conclusions of a review.94

We estimate the timeline for completion of this study to be 6 months. Any protocol amendments and non-compliance will be documented and reported transparently in the actual scoping review.

Ethics and dissemination

This scoping review will involve secondary analysis of already collected data and thus does not require ethics approval. The research findings will be submitted to peer-reviewed journals for publication and will also be disseminated at conferences and seminars.