Identifying and Prioritizing Gaps in Neuroendocrine Tumor Research: A Modified Delphi Process With Patients and Health Care Providers to Set the Research Action Plan for the Newly Formed Commonwealth Neuroendocrine Tumor Collaboration.

PURPOSE: Neuroendocrine tumors (NETs) are a diverse group of malignancies that pose challenges common to all rare tumors. The Commonwealth Neuroendocrine Tumor Collaboration (CommNETS) was established in 2015 to enhance outcomes for patients with NETs in Canada, Australia, and New Zealand. A modified Delphi process was undertaken involving patients, clinicians, and researchers to identify gaps in NETs research to produce a comprehensive and defensible research action plan.
METHODS: A three-round modified Delphi process was undertaken with larger representation than usual for medical consensus processes. Patient/advocate and health care provider/researcher expert panels undertook Round 1, which canvassed 17 research priorities and 42 potential topics; in Round 2, these priorities were ranked. Round 3 comprised a face-to-face meeting to generate final consensus rankings and formulate the research action plan.
RESULTS: The Delphi groups consisted of 203 participants in Round 1 (64% health care providers/researchers, 36% patient/advocates; 52% Canadian, 32% Australian, and 17% New Zealander), of whom 132 participated in Round 2. The top eight priorities were biomarker development; peptide receptor radionuclide therapy optimization; trials of new agents in advanced NETs; functional imaging; sequencing therapies for metastatic NETs, including development of validated surrogate end points for studies; pathologic classification; early diagnosis; interventional therapeutics; and curative surgery. Two major areas were ranked significantly higher by patients/advocates: early diagnosis and curative surgery. Six CommNETS working parties were established.
CONCLUSION: This modified Delphi process resulted in a well-founded set of research priorities for the newly formed CommNETS collaboration by involving a large, diverse group of stakeholders. This approach to setting a research agenda for a new collaborative group should be adopted to ensure that research plans reflect unmet needs and priorities in the field.

INTRODUCTION

Neuroendocrine tumors (NETs) are a diverse, uncommon group of malignancies important because of their increasing incidence and prevalence, partly as a result of increased detection with advances in imaging technology.[1] NETs are extremely heterogeneous, with a prognosis ranging from months in aggressive disease to decades for indolent disease. Multiple new diagnostic and treatment modalities have become available in the past 10 years.

The treatment of patients with NETs poses all of the challenges common to other rare tumors for both patients and clinicians, including delays in diagnosis; low levels of nonexpert clinician understanding of the disease; limited evidence base for treatment options, including a paucity of clinical trials[2-4]; difficulty in obtaining research funding; small patient numbers over large geographical areas; and limited advocacy for improved patient care.[5,6] International research consortia have been identified as a key strategy for integrating multidisciplinary expertise, enhancing the evidence base, and improving management of rare cancers.[3,7]

The Commonwealth Neuroendocrine Tumor Group (CommNETS) is a new collaboration among patients with NETs, clinicians, and researchers in Canada, Australia, and New Zealand. The group formed in 2015, with the mission of improving outcomes for patients with NETs, recognizing similarities of care provision in the three countries: high-performing universal health care with centralized processes for introduction of new medical procedures and treatments.[6,8] In addition, the three countries confronted the challenge of geographically dispersed populations.

CommNETS was established explicitly to facilitate collaboration in areas of need in NET research and care common to member countries, including greater power to recruit to clinical trials, shared registry information, and interchange of clinical expertise. Although it shares similarities with other international NET societies, including the European Neuroendocrine Tumor Society and the North American Neuroendocrine Tumor Society, a distinguishing feature is the deliberate embedding of patients and advocates as full members of the group, with the shared vision of CommNETS as the actual vehicle to conduct high-quality clinical trial and translational research. This maximizes opportunity and productivity over what can be achieved as individual centers or countries. Development of a research agenda that was meaningful to patients and health care providers was an essential first step.

To establish an a priori robust and defensible research agenda, an evaluation of gaps in NET research from the perspectives of patients/advocates and health care providers/researchers was undertaken using a modified Delphi process. The Delphi methodology of developing consensus about future planning and identifying emerging trends and issues is well established in fields as diverse as health care, infrastructure planning, and defense.[9-11] It is based on the principle that structured decision processes drawing on a wide range of expertise result in better decisions and prioritization than unstructured ones.[12] Delphi processes are most relevant in circumstances where there is insufficient empirical evidence to guide clinical practice and decision making alone.[13,14] This method usually involves a panel of 15 to 20 topic experts who rate agreement with a series of statements in an iterative fashion, typically with three rounds of ranking.[15-18]

One shortfall of an expert group, however, is that it may represent a homogeneous and incomplete view of the topic.[12,19] Modification of the Delphi process is well accepted for both the priority question development process and the panel composition, the latter to achieve inclusion of a more heterogeneous population, which may be split into multiple panels to broaden demographic and other desired representations.[15-17] The modified Delphi process has been used successfully to set various health care priorities over the last 15 years.[15,20-23] The benefit of an increased number of participants needs to be weighed against the challenge of ensuring continuity of participation.

It has been established that exposing health care professionals to the views of patients may influence the prioritization of outcomes.[20] This is particularly relevant in the field of NETs because of its heterogeneity with a variety of clinical interactions, treatments, and outcomes experienced by patients; the involvement of a broad range of multidisciplinary health care professionals; and the lack of well-defined outcome end points to measure outcomes in clinical studies. In setting priorities for the new CommNETS collaboration, we selected a modified Delphi methodology to ensure a scientifically valid and rigorous set of strategic priorities that harnessed the views of all stakeholders.

METHODS

A three-round modified Delphi process was undertaken over a 6-month period, collecting the views of two expert panels to identify gaps in NET research and then developing a consensus ranking of research priorities.[12,24] The Patient/Advocate Panel included patients, caregivers, patient advocates, support societies, and health care consumer representatives, whereas the Health Care Provider/Researcher Panel consisted of multidisciplinary health care professionals, including medical, nursing, and allied health practitioners and basic and translational researchers. The panel participants were recruited through e-mail invitations circulated through NET patient advocacy groups and support societies and clinical and research groups and networks in Canada, Australia, and New Zealand. The rationale for separating the panels was to allow the questions to be presented using appropriate descriptors (lay v medical terminology) or targeted when relevant to one panel only.

Round 1

The purpose of the Round 1 online survey was to identify gaps in current research and propose relevant NET research priorities or topics. A systematic review of current NET clinical trials was undertaken to identify areas under active investigation to underpin the survey topics. A 70-question survey, framed as a series of statements, was devised by a multidisciplinary project steering committee to be deliberately overinclusive of potential research topics and priorities. This was presented as a two-part electronic survey (Data Supplement) during an 18-day response window. The first part required both panels to rate 17 research priorities (proposed areas of research) on a five-point scale from high (1) to low (5), presented in four categories: the conduct of trials and research, trials/research that did not involve investigating efficacy of therapies, trials of systemic therapies, and trials of local therapies. The second part, sent only to the Health Care Provider/Researcher Panel, rated agreement with 52 specific research topics (statements outlining the specific methodology for the research question). Agreement was set at > 80% of participants, consistent with the consensus threshold set for other Delphi processes.[15,25,26]

Round 2

This online survey was open to all Round 1 participants. Comparative ranking of the Round 1 research priorities was undertaken by both panels within the conserved four categories. The Health Care Provider/Researcher Panel also ranked the specific and detailed research topics that had reached 80% agreement in Round 1.

Round 3

This comprised a face–to-face workshop conducted among a subset of panel members, all of whom had participated in the previous rounds. Participant numbers were limited because of logistics and cost; however, all who applied to attend were accommodated. Detailed deliberation on the 10 top research priorities and topics emerging from Round 2 was undertaken through a full day of group work, exploring the following aspects of each topic: importance and significance, feasibility, and relevance for the CommNETS tri-nation context and the specific consumer perspectives. The consumer perspective was specifically discussed in detail as representation of this panel at the workshop was proportionally reduced. Each participant then assigned three votes among the nine research priorities (two had been merged during workshop deliberations) and three votes within each of the four categories of research topics, for five voting rounds. Votes could be distributed across separate research priorities/topics or all assigned to a single entity; this approach is based on voting methods used for multicriteria decision analysis.[27,28]

Statistical Analysis

Descriptive statistics were used to analyze responses in Rounds 1 and 2 (means and standard deviations). The ranking of research statements and topics was based on the mean score across the two panels and was compared using the Mann-Whitney U test.

RESULTS

Two hundred three participants undertook Round 1 (Patient/Advocate and Health Care Provider/Researcher Panels, 36% and 64%, respectively), of whom 132 participated in Round 2 (32% and 68%, respectively) and 49 in Round 3 (6% and 49%, respectively). There were 17 research priorities and 42 research topics presented in Round 1, which was reduced to 10 priorities and 26 topics for Round 2.

Panel Composition

Representation by country was similar across all rounds (Canada, Australia, and New Zealand average, 45%, 34%, and 21%, respectively). Three patients (one from the Carcinoid-Neuroendocrine Tumour Society Canada; one each from the Unicorn Foundation Australia and Unicorn Foundation New Zealand) represented the Patient/Advocate Panel in Round 3; in addition, one member of the Health Care Provider/Researcher Panel had a personal history of NET and another specifically represented indigenous cultural aspects of care and research. The following specialties were represented on the Health Care Provider/Researcher Panel: nursing, medical physics, medical oncology, endocrinology, anatomic pathology, nuclear medicine, endocrine surgery, upper GI surgery, diagnostic radiology, and interventional radiology. Research expertise covered the fields of translational, laboratory, health services, and psychosocial research, as well as the conduct of clinical trials.

Rounds 1 and 2

All 17 initial research priorities were confirmed to be important in Round 1; however, 16 of 52 proposed research topics did not reach the consensus threshold (Data Supplement). In Round 2, all but two research priorities received broadly similar rankings between the panels. Early diagnosis of NETs was ranked top priority by the Patient/Advocate Panel but only ninth by the Health Care Provider/Researcher Panel (mean ranking, 4.0 v 9.6; P < .001; Data Supplement). Curative surgery for NETs was also rated significantly higher by the Patient/Advocate Panel (mean ranking, 7.2 v 9.9; P = .008). Other statistically significant differences were noted in the ranking of sequencing of therapies for metastatic disease (8.9 v 6.3; P = .01) and peptide receptor radionuclide therapy (8.8 v 6.6; P = .045). The Health Care Provider/Researcher Panel ranked 26 topics to be taken forward into Round 3 (Data Supplement).

Round 3

After extensive discussion, workshop participants voted on the final research priorities (Table 1) and topics (Table 2). Seven working groups were then formed to develop action plans (Table 3).

Table 1

Final Research Priorities

Table 2

Final Research Topics by Category

Table 3

Planned Activities on the Basis of Research Priorities

DISCUSSION

As a new research collaboration for neuroendocrine cancer, defining research priorities in a structured way from the outset was considered vital. Traditionally, many health-related societies have been established without formal processes to interrogate and document capabilities and constituent priorities. Furthermore, stakeholder engagement with patients and advocates, although often sought, is frequently subsequent to the development process and limited to feedback on an advanced set of proposals. The challenge in establishing CommNETS as a tri-nation multidisciplinary collaboration with the mission of improving outcomes for patients with NETs was to define from the outset a robust and defensible set of streamlined research topics prioritized through broad consultation. Recognition of current research in the field was required to avoid duplication. Understanding the differences in health systems, funding models, and patterns of care among the three countries was also important to achieve a workable model for collaborative activities and to recognize issues that may be restricted to particular sites or settings.

The modified Delphi methodology allowed inclusion of a much larger number of patients and clinicians/researchers than many health-related Delphi consensus activities, including those recently performed in the NET field.[29,30] These have typically involved a small number of experts with broadly similar experience, which is ideal for a complex discussion of specific treatment guidelines or other expert statements. Because our aims were different, we facilitated a blue-sky approach while adopting a formal process to gather a wide range of perspectives. Consequently, the primary strength of this study is that the rankings of research priorities and topics robustly reflects the needs of patients/advocates and clinicians/researchers across our member countries.

From a diverse and comprehensive base, 73% of the research questions floated in Round 1 were taken forward. These included the study of the utility of various old and new end points in trials, selection of patient populations for trials, and the need for trials of both systemic and nonsystemic therapies. Those not prioritized centered on the feasibility of including different subtypes of NETs in the same trial; imaging for trials (likely related to the different availability of imaging modalities across the three countries); and development of clinical trials, particularly those related to surgery. The lack of consensus regarding surgical trials was in part due to concerns about the difficulty of balancing robust trial design with adequate accrual in this rare cancer.

The modified Delphi process identified two main areas of difference in priorities between the two panels: early diagnosis of NET and curative resection. This was explored in Round 3, in which deliberations revealed that the Health Care Provider/Researcher Panel had ranked these lower not because they were considered unimportant but, rather, they were considered to be too difficult to study. This was attributed to the lack of an obvious research path to approach these topics, as well as the lack of obvious craft group ownership. This was acknowledged as a salient lesson and stimulated discussion regarding the process of harnessing research expertise and familiarity to devise projects with novel approaches to tackle difficult but important areas.

One limitation of this study is the relatively small number of patients and advocates present for Round 3. We used an online methodology for Rounds 1 and 2 to specifically allow for increased patient input; however, for Round 3, the logistic and financial constraints of attending an overseas meeting affected more participants on the Patient/Advocate Panel. Although there is broad and enthusiastic acceptance of consumers in disease-related societies across the health community, funding remains a major challenge. The strategy of specifically considering the consumer perspective seems to have been a successful surrogate.

In conclusion, CommNETS, as a new international NET organization, has developed a comprehensive set of prioritized consensus research topics through broad stakeholder engagement that uniquely included the patient voice from inception. The modified Delphi project demonstrates that formal processes can bring patient and advocate voices together with clinicians and researchers to formulate a plan for collaborative research on the basis of priorities addressing unmet needs. CommNETS will now undertake research activities on the basis of the ranked priorities and topics matched to perceived gaps in NET research and care across Canada, Australia, and New Zealand. Research prioritization and goal setting by collaborative groups should be undertaken through formal processes to ensure that subsequent resource utilization has a comprehensive and strategic basis.