Barriers to clinical adoption of next generation sequencing: Perspectives of a policy Delphi panel.

This research aims to inform policymakers by engaging expert stakeholders to identify, prioritize, and deliberate the most important and tractable policy barriers to the clinical adoption of next generation sequencing (NGS). A 4-round Delphi policy study was done with a multi-stakeholder panel of 48 experts. The first 2 rounds of online questionnaires (reported here) assessed the importance and tractability of 28 potential barriers to clinical adoption of NGS across 3 major policy domains: intellectual property, coverage and reimbursement, and FDA regulation. We found that: 1) proprietary variant databases are seen as a key challenge, and a potentially intractable one; 2) payer policies were seen as a frequent barrier, especially a perceived inconsistency in standards for coverage; 3) relative to other challenges considered, FDA regulation was not strongly perceived as a barrier to clinical use of NGS. Overall the results indicate a perceived need for policies to promote data-sharing, and a desire for consistent payer coverage policies that maintain reasonably high standards of evidence for clinical utility, limit testing to that needed for clinical care decisions, and yet also flexibly allow for clinician discretion to use genomic testing in uncertain circumstances of high medical need.

Introduction

Initial optimism over the sequencing of the human genome has given way to more sober discussion of challenges posed by the complexity of genomic information (Arnedos et al., 2014), the inadequacy of many clinical studies (Holmes et al., 2009), the growing complexity of bioinformatics (Fernald et al., 2011), the economics of personalized medicine (Davis et al., 2009), and other factors (Chan and Ginsburg, 2011). The development of high-throughput sequencing techniques, or next generation sequencing (NGS) (and now third- and fourth-generation sequencing) (Kulkarni and Pfeifer, 2015), could lay the foundation for clinical integration of genomic data. However, NGS worsens all the existing challenges for identifying relevant genomic discoveries, studying benefits and harms, and getting beneficial discoveries from the laboratory to the clinic for rapid patient access.

As a result, policymakers are increasingly focusing on ways to facilitate the development and application of genomic medicine. Most notably, President Obama's Precision Medicine Initiative focuses on expanding cancer clinical trials and creating a voluntary database of information on a million-person cohort for a “big data” approach to sorting out the meaning of human genetic variation (FACT SHEET). Similarly, the 21st Century Cures initiative of the U.S. House Energy and Commerce Committee focuses on the acceleration of innovative cures, including the development of precision medicine (21st Century Cures).

Policy challenges must be addressed for such initiatives to succeed. Legal commentators have speculated about what the policy challenges will be, particularly as they relate to regulatory oversight of NGS, coverage and reimbursement of clinical NGS tests, and intellectual property and data sharing (Kaufman et al., 2014, Javitt and Carner, 2014, Deverka and Dreyfus, 2014, Cook-Deegan and Chankrasekharan, 2014, Evans, 2014). To effectively respond to these challenges, policymakers need to tackle the most important and tractable issues first, taking account of the perspectives of multiple stakeholders.

To assess the relative importance and tractability of issues arising in the clinical integration of DNA sequence data, we carried out a four-round policy modified Delphi study using a multi-stakeholder panel of experts to identify and rank the most important policy challenges, and to deliberate about solutions for the top four challenges. This paper describes the results of the first two rounds of the Delphi process, which were designed to identify the most important and most tractable barriers to clinical adoption of NGS testing.

Materials and methods

We conducted a modified policy Delphi, an iterative survey technique conducted in rounds with a select group of experts to assess a specific policy question (Linstone and Turoff, 1975, Turoff, 1970, Adler and Ziglio, 1996). The Delphi method as originally conceived in the 1950s by RAND was designed as a homogeneous panel of experts responding to questionnaires and developing consensus opinions asynchronously and anonymously (Linstone and Turoff, 1975). Users of the technique subsequently modified the approach for a wider range of applications. The policy Delphi approach uses a heterogeneous group of experts to explore differences of opinion and produce a range of policy options (Turoff, 1970, Adler and Ziglio, 1996). We used a “modified” Delphi approach, which includes a round of well-facilitated direct interaction among Delphi panelists (Fitch et al., 2001).

This project was designed to answer two questions in succession – one question for each two rounds. The first two rounds of this study asked: what is the group's collective judgement regarding the most important and tractable barriers to the clinical adoption of NGS? The second two rounds asked: for the most important barriers identified, what are proposed policy solutions? The answer to the first question is meaningful and worth detailed consideration in itself. In addition, since the method design is modular, reporting on the first module has no effect on the conduct of the second module. Hence we have elected to report this result now, while we continue to explore the answer to the second question.

For the purposes of this study, we defined ‘expert’ as a person with professional expertise or significant lay knowledge in the field of genetics, government, health policy, patient advocacy, or law. A history of NGS-related publications, presentations, or service on task forces or public committees was evidence of expertise. Experts were contacted by email and offered an honorarium of $100 per round of completed Delphi. Out of 93 invitations sent, 48 experts agreed to participate. The final composition of the participants is displayed in Table 1.

Table 1

Expert stakeholder composition of Delphi panel.

Primary profession	N (%)
Genomic researcher	8 (17%)
Clinician or health care provider	3 (6%)
Payer	3 (6%)
Research funder	2 (4%)
Regulator or policy maker	3 (6%)
Lawyer or legal scholar	5 (10%)
Informatician	1 (2%)
Health economist	2 (4%)
Product developer	4 (8%)
Patient advocate	4 (8%)
Social scientist	2 (4%)
Industry funder	2 (4%)
Other	9 (19%)
TOTAL	48 (100%)

An initial list of challenges was developed based on preliminary research (Holmes et al., 2009, Curnutte et al., 2014), including work published by the project team in a series of papers (Kaufman et al., 2014, Javitt and Carner, 2014, Deverka and Dreyfus, 2014, Cook-Deegan and Chankrasekharan, 2014, Evans, 2014), and on project team member subject matter expertise. This initial series of 19 potential policy challenges potentially hampering clinical NGS was assessed by the Delphi panel in the first round. The challenges spanned three topic areas: intellectual property (3 challenges), regulation (6 challenges), and coverage and reimbursement (10 challenges).

At the end of Round 1, the challenges deemed most important by the participants were analyzed using a weighted scoring system. Points were assigned as: “strongly agree” = 2, “agree” = 1, “disagree” = − 1, “strongly disagree” = − 2. The scale was normalized to run from − 100 to 100: − 100 thus represented strong disagreement of all panelists; 100 represented strong agreement of all panelists. A cutoff score of 60 was used to select challenges that would advance to Round 2 for reassessment (i.e., challenges generally agreed to be important moved to the next round for further assessment).

Challenges from Round 1 exceeding the threshold score of 60 were included in Round 2 for re-assessment by the group. We also included a new series of panel-suggested additional challenges for assessment (suggested through open text boxes). The project team also revised and re-presented four challenges from Round 1 that related to FDA and CLIA (results from these questions were ambiguous, perhaps in part because in the interim between rounds the FDA released a proposed new framework for regulating in vitro diagnostics), and asked a series of follow-up questions to confirm and clarify the Round 1 results.

In both rounds, panelists assessed individual challenges using Likert scales and performed ranking exercises to evaluate the relative importance of challenges (or categories of challenges) and the perceived feasibility of addressing the challenges. Panelists had opportunities to provide open-ended comments and suggestions throughout the questionnaire. Between Rounds 1 and 2, all participants were provided with the results of the first round: a 22-page packet displaying aggregate scores (as calculated above) for each challenge (e.g., the data presented in Table 1, Table 2 below in graphical form) and an anonymized summary of all the open-text comments organized by theme.

Table 2

Round 1 individual policy challenge assessment scores for importance.

Challenge description	Scorea
Diagnostic companies are able to maintain proprietary databases on the substantial variety of clinically meaningful mutations found in patients. Refusal to share this type of information could impede the development of clinical useful NGS tests.	77
The traditional framework many payers use for assessing diagnostic tests for coverage cannot keep pace with the rate of NGS-based genomic discovery.	73
Different payers have different evidentiary standards for assessing clinical utility, leading to inconsistent policies on coverage and reimbursement for NGS-based testing.	68
Some payers refuse to cover NGS because the specific information needed for patient management is unclear when NGS-based testing is ordered.	67
Some payers refuse to cover NGS because the technology itself is considered experimental, investigational, or unproven.	67
The traditional framework many payers use for assessing diagnostic tests for coverage does not account for the future utility of heritable risk prediction or associated prevention strategies.	59
The traditional framework many payers use for assessing diagnostic tests for coverage cannot accommodate the discovery and use of rare variants in precision medicine.	56
The submission of a claim to a payer for confirmatory testing of incidental findings is typically not covered by payers, since in this case, there is either no diagnosis of the diagnosis does not appear clinically relevant to the test being done.	55
Currently payers do not reimburse separately for the sequencing and interpretative components of NGS testing.	55
cDNA and the short DNA sequences used as primers and probes in genetic testing are still potentially protected, leaving open an avenue for companies like Myriad to challenge would-be competitors offering genetic testing and potentially hindering NGS companies from developing clinically optimal diagnostic testing products and services.	55
The performance characteristics for analytic validity specified by CLIA do not readily apply to NGS platforms due to the complexity of the technology/bioinformatics needed for analysis and interpretation.	54
A company's ability to offer interpretive services may be limited by State laws limiting the practice of medicine (including the clinical interpretation of laboratory results for patients) to licensed professionals.	50
An ongoing legal debate exists over whether genomic interpretation should be deemed to be the practice of medicine.	50
It is unclear who, if anyone, would pay for periodic re-analyses of stored NGS data given that under current policies, a re-analysis not prompted by a specific clinical question or diagnosis would likely be challenged.	47
If sequencing and interpretive services bifurcate into separate services offered by different companies, it is unclear which federal or state agency would have the authority to regulate freestanding interpretive services.	44
Without FDA review of LDTs for safety and efficacy for clinical use, patients may be put at risk.	43
While recent Supreme Court rulings have reduced some uncertainty about what kind of diagnostic molecules and methods can be patented, there remains lingering uncertainty about patent eligibility that may dampen incentives to develop genomic diagnostics products and services.	40
In the absence of clinical diagnostic claims, the FDA's authority to regulate NGS platforms as laboratory-developed tests (LDTs) is unclear.	39
Reimbursement policy may drive whether NGS raw data output will become part of a patient's clinical record or whether only the clinical interpretation of the results will be retained, since data retention will likely be associated with additional costs.	29

Scoring scale: − 100 (all strongly disagree challenge is important) to 100 (all strongly agree).

Questionnaires for Rounds 1 and 2 were administered online using Qualtrics®. Individual survey links were sent to each participant to allow the project team to track individual responses. Forty-three experts participated in the first round and 35 participated in the second round, although not all participants answered all questions. This type of attrition is a well-known challenge of using the Delphi method (Adler and Ziglio, 1996). However, for the purposes of the policy Delphi survey, so long as attrition does not lead to homogenization of the group composition (it did not in this case), we expect the essential underlying logic of the method to remain intact.

Note also that Delphi surveys should not be construed as statistical population sampling. The group participating in this survey was purposively sampled (Bryman, 2012) according to our criteria for expertise in specified stakeholder categories. Our claim is not that this group is necessarily representative of some population, but that the perspectives and deliberations of this group are worth consideration due to the expertise and composition of the group.

Respondents in Rounds 1 and 2 were blind to each other's identities. They were sent private, individualized email links to electronic surveys. Email instructions were provided such that all participants were blind copied. Only the project team was aware of the identities of the respondents. The study was reviewed and approved by institutional review boards (IRBs) at the Johns Hopkins University, Baylor University, and the Chesapeake IRB.

The Delphi process underlying this report is summarized in Fig. 1.

Fig. 1

Project approach: first module.

Results

In Round 1, 19 challenges were assessed. Table 2 shows the weighted score (described above) for each challenge assessed in Round 1. The scores are all positive (not negative), indicating that as a group, overall, the panelists ascribed some importance to each issue. However, lower scores tend to indicate more disagreement and weaker support for the importance of the challenge. Five challenges had a score of at least 60, meaning there was overall agreement that these issues are important barriers to clinical adoption of NGS. By this measure, the panel felt that the most important policy challenge for NGS was the ability of diagnostic companies to maintain proprietary databases. Notably, this challenge was also ranked as one of the least politically feasible to address.

The other high-scoring challenges in Round 1 were related to coverage and reimbursement of NGS testing: 1) the inability of coverage and reimbursement frameworks to keep pace with innovation; 2) differing payer standards for clinical utility leading to inconsistent policies for coverage and reimbursement of NGS-based clinical testing; 3) payer refusals to cover NGS testing when uncertainty existed around the specific information needed for patient management; 4) and payer classification of NGS as investigational (by definition) and therefore not covered. Challenges related to FDA and the Clinical Laboratories Improvement Amendments (CLIA) were relatively low-scoring in Round 1.

The importance scores for the challenges assessed in Round 2 are provided in Table 3. Again, the ability of diagnostic companies to maintain proprietary databases emerged as the most important barrier to clinical adoption of NGS. In Round 2 follow-up and confirmatory questions, 74% respondents also opposed allowing companies to maintain private variant databases as protected trade secrets while 66% opposed the patenting of complementary DNA strands or DNA primers.

Table 3

Round 2 individual challenge assessment scores for importance.

Challenge description	Scorea
Diagnostic companies are able to maintain proprietary databases on the substantial variety of clinically meaningful mutations found in patients. Refusal to share this type of information could impede the development of clinically useful NGS tests.	76
There is a lack of standardization for reanalysis and reporting updates to variants calls.	63
There is a lack of standardization for reporting NGS test results (e.g., determining which results to report, how to effectively communicate findings, and to whom those findings should be communicated).	60
Different payers have different evidentiary standards for assessing clinical utility, leading to inconsistent policies on coverage and reimbursement for NGS-based testing.	58
Some payers refuse to cover NGS because the specific patient management decision to be informed by testing is unclear when NGS-based testing is ordered.	56
Some payers refuse to cover NGS because the technology itself is considered experimental, investigational, or unproven.	52
The clinical integration of NGS requires new infrastructure that includes, for example, better EHR systems to store and cross-reference genomic information with other health-related information to facilitate clinical decision making.	50
CLIA cannot ensure accurate and valid NGS-based test results because the Centers for Medicare and Medicaid Services (CMS) have not promulgated specific standards for laboratories performing genetic testing.	48
The inclusion of genetic counseling and communication of NGS test results is not sufficiently standardized as part of clinical practice.	45
The lack of education and training of health care professional in the areas of genetics and genomics is impeding the realization of clinical NGS.	45
The traditional framework many payers use for assessing diagnostic tests for coverage cannot keep pace with the rate of NGS-based genomic discovery.	38
The clinical integration of NGS raises questions about data ownership and access, including the entitlements of patients and physicians.	37
FDA's authority to regulate tests and products depends on their intended use. In the absence of specific clinical claims (predictive or diagnostic), FDA may be limited in its ability to regulate NGS-based tests.	36
Under new HIPAA privacy rule amendments, patients may have unrestricted access to protected health information (PHI), including raw data (BAM, FASTQ, VCF) stored by laboratories covered by HIPAA.	21
FDA's newly proposed framework for regulating LDTs creates significant barriers to achieving the benefits related to clinical integration of NGS.	20
There is no clear legal guidance on the scope of potential legal liability for laboratories and clinicians in connection with the interpretation and reporting of NGS results.	18
Patients and their caregivers are not sufficiently versed in genetics and genomics to participate in decision making related to NGS testing.	17
Existing protections are not adequate to protect the privacy of patient data in the context of clinical NGS.	− 2
FDA's newly proposed framework for regulating LDTs fails to address significant risks related to clinical integration of NGS (e.g., incorrect diagnosis, treatment, or prevention of disease).	− 6

Scoring scale: − 100 (all strongly disagree challenge is important) to 100 (all strongly agree).

In addition, 2 new challenges suggested by panelists in Round 1 had scores of at least 60 in this assessment: the lack of standardization for re-analysis and reporting updates to variant calls; and the lack of standardization for reporting NGS test results.

Although coverage and reimbursement issues dominated the challenges scoring highly important in Round 1, in Round 2 supplemental questions, respondents were divided when asked if payers are “too restrictive” in covering NGS: 37% disagreed, 34% agreed, and 29% expressed no opinion. When asked what types of testing payers should cover:

77% of the panel favored coverage of NGS testing for tumor profiling.

70% of the panel favored coverage of whole genome and exome sequencing for rare diseases.

66% of the panel favored coverage for prenatal testing.

52% of the panel favored coverage for pharmacogenetic testing.

49% of the panel favored coverage for newborn screening.

29% favored coverage for disease risk prediction.

Most panelists believed payers should vary their standards of evidence depending on clinical need and context (out of 32 panelists responding, 63% were in favor, 16% were not in favor, and the remainder took no position).

Individual FDA-related challenges did not score highly for importance in Round 1. When asked in Round 2 which elements of the NGS clinical pipeline should be regulated, 54% favored FDA regulation of consumables and arrays, 57% favored regulation of sequencing instrument manufacturers, and 54% favored regulation of interpretation and reporting of results. This latter result stands in contrast to a separate set of questions in which only 40% agreed that interpretive services should be subject to FDA regulation, while 63% indicated that interpretive services constitute the practice of medicine. A minority endorsed regulating other aspects of the NGS pipeline: 44% favored regulation of performance of sequencing and 43% favored regulation of alignment and annotation. Only 21% supported FDA regulation of data storage. A majority (57%) indicated that mechanisms other than FDA oversight could be used to address clinical risks to patients from the use of NGS.

Support for FDA's 2014 proposal to regulate laboratory-developed tests (LDTs) was also mixed. Paradoxically, while the challenge “FDA's newly proposed framework for regulating LDTs fails to address significant risks” made it into the top five results of the final ranking exercise (discussed below), it was the lowest-scoring item in individual challenge assessments (Table 3). The group was split on this question, with somewhat more panelists disagreeing than agreeing that this is an important barrier to clinical adoption of NGS.

The Delphi panel was generally optimistic about the future usefulness of clinical NGS, with 71% (24/34) agreeing or strongly agreeing that “within the next five years, NGS will be a valuable tool across many areas of clinical practice.” When asked in an open-ended question to identify the areas of clinical practice in which NGS will be most valuable, the overwhelming answer was oncology (19 responses, as compared to 4 or fewer for all other disease categories). In addition, 74% of the panel viewed clinical risks to patients from the use of NGS-based testing as moderate to low.

As a final step in Round 2, panelists were asked to select what they considered to be the 3 most important issues from an overall list of the 19 challenges presented. For the 35 participating panelists, the 5 challenges most often picked for the top 3 were:

Tied for first place: “Diagnostics companies are able to maintain proprietary databases on the substantial variety of clinically meaningful mutations found in patients. Refusal to share this type of information could impede the development of clinically useful NGS tests”.

Tied for first place: “Different payers have different evidentiary standards for assessing clinical utility, leading to inconsistent policies on coverage and reimbursement for NGS-based testing”.

Second place: “There is a lack of standardization for reporting NGS test results (e.g., determining which results to report, how to effectively communicate findings, and to whom those findings should be communicated)” (panelist-suggested challenge).

Tied for third place: “Some payers refuse to cover NGS because the specific patient management decision to be informed by testing is unclear when NGS-based testing is ordered”.

Tied for third place: “FDA's newly proposed framework for regulating LDTs fails to address significant risks related to clinical integration of NGS (e.g., incorrect diagnosis, treatment, or prevention of disease)”.

Discussion

The exercise described in this article has potential limitations in that, as noted above, the panel cannot necessarily be seen as representative of a larger population of experts. Moreover, a somewhat different composition of expert stakeholders participated in each round, and both of these groups differed in composition somewhat from that of the larger group of experts who originally agreed to participate. The effect that these variations may have had on the exercise cannot be predicted, since the opinions of the “missing” stakeholders are unknown. Nevertheless, the policy Delphi is not designed for precise reproducibility, but as a structured exercise to bring out the values and opinions of expert stakeholders whose thoughts are deemed relevant to the underlying questions. Our key concern therefore was to preserve a heterogeneous group composed of key stakeholder categories.

The results of this exercise consistently spoke to a majority of the Delphi panel's discomfort with proprietary mutation databases as a significant barrier to effective clinical adoption of NGS. Promoting and encouraging data-sharing is one clear antidote to assuring that important mutation data is available to clinicians and patients when needed. Numerous repositories are in various stages of development with content that may be overlapping, complementary or redundant (Rubenstein et al., 2015). The existence of so many repositories and efforts to curate data about genomic variation could splinter or Balkanize data when data-pooling can be more effective for assessing the meaning of genomic variation in populations. The curation projects and databases are developing protocols to enable pooling of data, e.g., through the Global Alliance for Genomics and Health and many other collaborations. Establishing and maintaining the data repositories, norms and practices for sharing data and samples, and interfaces for clinical use will be challenges through the foreseeable future (Ray). For this reason, FDA's proposal to use a public repository like ClinVar as a “regulatory grade” database that can form the basis for in vitro diagnostic device clearances is a potentially positive development, since it would promote contributions to a specific public resource and increase the likelihood that this resource would continue to receive funding (US Food and Drug Administration, 2015). This prospect is made more promising by the Precision Medicine Initiative's provision to provide $10 M in funding to the FDA (FACT SHEET). Vice President Biden's recently announced “cancer moonshot” may also direct resources to encourage data-sharing, but some observers worry it will disrupt existing initiatives (Hayden, 2016). In any event, companies desirous of maintaining proprietary databases are unlikely to participate in public-private partnerships for data-sharing without compelling incentives (Cook-Deegan et al., 2013).

The results additionally point overall to a perception that payers are a bottleneck to patient access to promising genomic testing. The coverage challenge ranked as most important reveals a perceived inconsistency in the standards payers use to assess clinical utility of NGS, leading to inconsistent coverage policies. Yet a 2012 review of 10 major commercial health plan policies found approximately 90% consistency in coverage decisions for genetic and pharmacogenetic tests, even though the decisions were sometimes based on somewhat different bodies of evidence (Hresko and Haga, 2012). If payers have substantively different evidence standards for utility, it does not seem to translate into significantly different coverage decisions for genetic and pharmacogenetic testing. With respect to NGS testing specifically, our panel may see growing variability in the way different payers approach testing, not because of differing standards for utility, but because of payer uncertainty over the technology, the proliferation of new clinical uses, and challenges in managing the information NGS produces.

However, the overall perception of payers as a bottleneck has some empirical support. Relatively few genetic and pharmacogenomic tests on the market have even undergone evidence reviews for coverage because of an overall lack of clinical utility studies available to review (Hresko and Haga, 2012). Thus, a bottleneck exists that is related to the way payers define utility (generally in terms of improved health outcomes for patients when test information is used) and the lack of supporting evidence for this kind of clinical utility, rather than between-payer inconsistency in the definition or standards. Groups such as the American College of Medical Genetics (ACMG) have taken positions on how clinical utility should be defined (Anon., 2015). These statements reflect a tension between stakeholders who would broaden the concept of utility to include patient benefits not directly measured as health outcomes, and more restrictive definitions that payers tend to favor, such as that of the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group (Teutsch et al., 2009). This disconnect is likely to be exacerbated by NGS, given an expanding universe of potential clinical applications with expanding capabilities for genomic analysis.

Despite health plans' ability to help or hinder patient access through coverage policy, the importance of payers as stakeholders to be engaged in policy is not fully appreciated. The President's Precision Medicine Initiative puts an emphasis on cancer as an initial priority, devoting $70 M of the proposed annual increment of NIH's budget to work at the National Cancer Institute (FACT SHEET). Yet, the Presidential Initiative's call for public-private partnerships to build an enhanced cancer research enterprise seeks to engage “academic medical centers, researchers, foundations, privacy experts, medical ethicists, and medical product innovators”—not payers. Presumably payers were excluded because they are not traditionally conceived as research organizations. Yet some major health plans are making major investments in data infrastructure and data mining, such as the UnitedHealth-owned Optum Labs' partnership with the Mayo Clinic (Wallace et al., 2014). Moreover, excluding payers could fail to address one of the foremost chokepoints in translating NGS technology into clinical use. Without health plan assent to the products of research (i.e., agreement that clinical utility of new genomic tests has been established via research findings), patient access to new valuable discoveries may be hindered.

The Delphi panel also identified oncology as a key area of growth for the clinical use of NGS—and by extension a clinical domain in which the policy challenges of clinically integrating NGS data into clinical practice will first be confronted. Payer engagement is therefore essential for constructive policy solutions in this arena. Several current initiatives do reach out to payers as partners in discussing coverage of clinical NGS for oncology. These include the Targeted Agent and Molecular Profiling Registry (TAPUR), the Molecular Evidence Development Consortium (MED-C), (both of which ask payers to cover testing as part of genomic evidence generation in oncology), the Tapestry Network's SPOT/Dx working group (not specifically focused on payer coverage of clinical NGS, but payers are involved in discussions of real world evidence for genomics) (SPOT/Dx Working Group), and the Center for Medical Technology Policy's Green Park Collaborative initiative on NGS coverage standards and policy (CMTP-Funded Effort to Develop Evidence Standards for NGS Ca, Center for Medical Technology Policy, 2015).

While the panel expressed misgivings about the role of health plans as a hindrance to patient access, many panelists approved maintaining relatively high standards of clinical utility, prioritizing coverage of genomics for care of serious disease. Overall, the group seemed to desire a middle ground where reasonable standards of utility are applied consistently, but not in a way that hinders clinician discretion to gather genomic information in complex clinical scenarios, especially in important clinical areas like oncology.

For the project team, the most surprising finding from this exercise thus far has been the lack of relative importance assigned to individual challenges addressing FDA regulation as a potential barrier to clinical NGS testing. The new FDA proposed framework for regulating LDTs had just been released when the Round 1 survey was underway. The team therefore used Round 2 to probe the new developments. We found that our participants were eager to provide us with open-ended comments expressing opinions on the ongoing controversy over FDA's authority to regulate LDTs (Evans and Watson, 2015, Evans et al., 2015, Lander, 2015, Litwack et al., 2015, Sharfstein, 2015). Nevertheless, FDA regulation of NGS continued to poll as less important than other challenges for clinical NGS. Even though an FDA policy challenge tied for third place in the overall ranking at the conclusion of Round 2, as noted, this specific policy challenge had the weakest response when evaluated on an individual basis (with a weighted score of − 6; only 37% of respondents said it was an important challenge). This attitude might be attributed either as skepticism that FDA reforms will ever be implemented, or as acceptance of the FDA's presence in this arena as relatively unobtrusive compared to other challenges.

“Lack of standardization for reporting NGS test results” was the only participant-suggested challenge that ranked highly important by participants in Round 2. However, the specific nature of the challenge was unclear to panelists; i.e., is the challenge “determining which results to report,” or “how to effectively communicate findings,” or “to whom should findings be communicated?” These questions are being further evaluated in the last two Delphi rounds.

Advanced sequencing technologies are capable of rapidly generating enormous amounts of genomic information with increasing cost effectiveness. These advances hold great promise for bringing genomics into the clinic for use in precision medicine. However, they also strain the traditional policy structures for regulation, coverage and reimbursement, and intellectual property. We identified key areas of concern in a mapping exercise with an expert Delphi panel to identify the most important policy challenges to clinical adoption of NGS. The panel's future assessment of potential solutions to key challenges and the pros and cons associated with possible policy remedies will be explored in further investigation.