A survey of current practices for genomic sequencing test interpretation and reporting processes in US laboratories.

PURPOSE: While the diagnostic success of genomic sequencing expands, the complexity of this testing should not be overlooked. Numerous laboratory processes are required to support the identification, interpretation, and reporting of clinically significant variants. This study aimed to examine the workflow and reporting procedures among US laboratories to highlight shared practices and identify areas in need of standardization.
METHODS: Surveys and follow-up interviews were conducted with laboratories offering exome and/or genome sequencing to support a research program or for routine clinical services. The 73-item survey elicited multiple choice and free-text responses that were later clarified with phone interviews.
RESULTS: Twenty-one laboratories participated. Practices highly concordant across all groups included consent documentation, multiperson case review, and enabling patient opt-out of incidental or secondary findings analysis. Noted divergence included use of phenotypic data to inform case analysis and interpretation and reporting of case-specific quality metrics and methods. Few laboratory policies detailed procedures for data reanalysis, data sharing, or patient access to data.
CONCLUSION: This study provides an overview of practices and policies of experienced exome and genome sequencing laboratories. The results enable broader consideration of which practices are becoming standard approaches, where divergence remains, and areas of development in best practice guidelines that may be helpful.Genet Med advance online publication 03 Novemeber 2016.

INTRODUCTION

Exome sequencing is rapidly gaining acceptance as a useful diagnostic test in clinical medicine[1,2,3]. While a less frequently utilized option, genome sequencing enables more comprehensive genome analysis with expanded data and uniform depth of coverage, albeit at higher cost and lower average depth of coverage[4,5]. With each of these tests, collectively referred to here as “genomic sequencing,” there are numerous laboratory processes required to support the identification, interpretation and reporting of variants that may be clinically significant for the patient.

A typical genome has about 3.5 million differences when compared to the reference genome, of which 0.6 million are rare or novel[6]. Although exome sequencing focuses on the subset of variants within or near coding sequences, hundreds to thousands of variants are identified for analysis per patient for clinical relevance[7]. A major challenge is determining which, if any, of the identified variants may be relevant to the indication for testing and thereby warrant inclusion in the final test report. Beyond variants of possible diagnostic or therapeutic relevance, laboratories may also choose to identify incidental or secondary variants unrelated to the testing indication (referred to below as “secondary findings”), but potentially relevant to the patient and their family’s health. Unquestionably, the inclusion or exclusion of variants in test reports may have substantial impact on patient care. It is therefore of critical importance to explore how these decisions are made.

Laboratories and professional societies have begun developing and recommending approaches to support this complex and labor-intensive process, which combines the practice of medicine with burgeoning next generation sequencing (NGS) laboratory procedures[8,9,10,11,12]. Here we present the results from a survey of laboratories experienced in genomic sequencing to explore protocols supporting the testing, interpretation and reporting processes. The goal of this study was to identify workflow and reporting practices that are shared and/or discordant among laboratories in an effort to highlight practices that are becoming standard and to determine areas in need of standardization or best practice recommendations.

METHODS

Recruitment

US laboratories that are Clinical Laboratory Improvement Amendments (CLIA)-certified and offer exome and/or genome sequencing, either to support the National Human Genome Research Institute (NHGRI) and National Cancer Institute (NCI) funded Clinical Sequencing Exploratory Research (CSER) program[13] or for routine clinical services, were invited to participate in the survey. Clinical services outside of CSER were primarily identified through a search of NCBI’s Genetic Test Registry (http://www.ncbi.nlm.nih.gov/gtr/). A total of 27 laboratory groups were identified. An invitation letter describing the purpose of the survey was sent by email to each laboratory. One follow-up invitation was sent to non-responding laboratories. Of the 27 laboratories contacted, 21 responded and completed participation (78% response rate) (Table 1).

Table 1

List of participating laboratories

	CSER (n=9)	Clinical (n=12)
Exome Only	BASIC3 (Baylor)	Ambry Genetics
	CanSeq (DFCI)	Baylor Miraca Genetics
	NCGENES (UNC)	The Children’s Hospital of Philadelphia (CHOP)
	NEXT Medicine (UW)	Columbia University Medical Center
	PediSeq (CHOP)	Emory
		GeneDx
		University of California Los Angeles
		University of Chicago
Genome Only	Kaiser Permanente	Illumina
	MedSeq
Exome & Genome	HudsonAlpha	Medical College of Wisconsin
	MI-ONCOSEQ (U. Michigan)*	Partners Laboratory for Molecular Medicine
		Personalis

Genome performed for only a subset

Participation was completely voluntary and laboratories did not receive compensation for completion of the study. This study was submitted to the Partners HealthCare IRB for review on October 12th 2014 and determined to be exempt. Consent was implied by agreement to participate.

Surveys

A 73-item survey was developed by the study team with input from the CSER Actionability and Return of Results Working Group (see Supplementary Materials and Methods). Items were formatted as multiple-choice with the option to provide free text and were grouped topically including the following elements of genomic sequencing laboratory processes:

The survey was emailed to each laboratory group who identified staff that were key informants regarding the laboratory’s sequencing, interpretation and reporting processes. Surveys were completed between December 2014 and February 2015. Study leads (HMM, JMO and HLR) reviewed the completed survey through a follow-up phone call with each laboratory’s primary respondents, enabling clarification of questions and answers. Each follow-up call lasted approximately 1 hour and was audio-recorded.

Data Analysis

Due to the small sample size, only frequency counts and descriptive results were utilized to demonstrate trends of agreement or divergence among the CSER program and clinical service laboratories. Counts are combined for clinical and CSER laboratories when responses do not appear to differ between the settings.

RESULTS

Participants

Responses were collected from CLIA-certified laboratories offering exome and genome sequencing, including nine of the ten laboratories supporting CSER research programs as well as twelve laboratories offering routine clinical genomic services, for a total of 21 laboratories. All responses represented practices at the time of survey completion. Some laboratories supported both a routine clinical service and a CSER program. However, these laboratories usually had separate protocols (and in some cases different personnel) specific to each purpose. For the purposes of this study, these approaches were treated as two separate laboratories. Hereafter the laboratories offering routine clinical services will be referred to as “clinical” laboratories (N=12) and the laboratories supporting research programs will be referred to as “CSER” laboratories (N=9) (Table 1).

Overall, 16 laboratories (out of 21) reported a history of completing >50 exome or genome sequencing tests, and a high proportion of clinical laboratories (8 of the 12) had completed >200 tests at the time of the survey.

Sample and Consent

All laboratories required written documentation of consent prior to performing the test, with some variance in whether it was signed by the patient only (12 of 21), or the patient and physician (9 of 21).

The majority of clinical laboratories (8 of 12) performed genome- or exome-scale testing on trio samples in >50% of cases, whereas the opposite was true of CSER laboratories, with 6 of 9 sequencing probands only.

Phenotype Collection and Target Gene List Development

Phenotypic information was collected from the referring clinician through several means, the most frequent of which were: free text fields on the requisition form (14 of 21), attached medical records and clinic notes (14 of 21) and via written responses to targeted questions (13 of 21). Only one CSER laboratory did not collect any phenotypic information due to their primary project aim of carrier status rather than diagnostic analysis.

There was notable variance in how frequently laboratories reported using indication-specific gene lists for targeted analysis of individual cases: 10 of the 21 laboratories used them in all or almost every case (>90–100%); 4 in the majority (50–90%) of cases; 1 in 10–50% of cases; and 6 rarely (<10%) or never used them. For three of the laboratories, the same gene list was applied for all cases due to the test indication: carrier status analysis or oncology (somatic and germline) testing. Most laboratories that used indication-based gene lists did not have a policy governing how lists were developed or updated, though two clinical laboratories reported gene lists were assembled on a case-by-case basis.

Case Review

Variant interpretation and case review processes include several steps, and laboratories identified individuals with various roles in these processes (Figure 1). For the purpose of this survey, we defined the initial analysis as the non-automated primary collection of variant-level evidence for case-specific clinical interpretation. This initial analysis was most frequently performed by PhD-level analysts or fellows (Medical Genetics or Pathology), and genetic counselors were also frequently involved. Although some laboratories involved American Board of Medical Genetics and Genomics (ABMGG) board-certified medical geneticists (PhD trained) or pathologists in the initial analysis steps, these individuals were most often utilized in secondary review and report sign-out.

Figure 1

Type of individual who performs initial interpretive analysis of variants after bioinformatics filtering

Group discussion was frequently employed within the process of variant review for potential case-specific relevance and all but one laboratory had regularly scheduled meetings for this purpose. This clinical laboratory instead conducted ad hoc meetings for an estimated 10–50% of cases. Approximately 60% of laboratories (12 of 21) reported group discussion is part of all or almost every case (90–100%) (6 clinical; 6 CSER), whereas 3 clinical laboratories utilized group discussion for only a small portion (<10%) of cases. The make-up of professionals in the group discussions nearly always involved board-certified medical and clinical geneticists, genetic counselors, and bioinformaticians, but also commonly included basic science researchers, non-genetics physicians, trainees and ethicists.

Exclusive of group discussions, more than one individual was involved in the interpretation and reporting of each case: 7 laboratories (4 clinical; 3 CSER) typically utilized two individuals, 9 laboratories (4 clinical; 5 CSER) utilized three individuals, and 5 laboratories (4 clinical; 1 CSER) reported ≥ four individuals were involved in each case. No laboratories reported using only one individual per case.

The time required for full case review was predominantly based on estimates rather than timed work and varied widely among groups (see Supplemental Figure S1 and S2).

Sanger Confirmation

Sanger sequencing was used for orthogonal confirmation by the majority of laboratories (17 of 21) for all reported variants or for variants considered to represent possible diagnostic findings (Figure 2). Only one clinical lab indicated they did not confirm any reported variants. Most laboratories had protocols specifying which types of reported variants were or were not Sanger confirmed. For example, in one laboratory, only the germline variants were confirmed when paired somatic/germline sequencing was performed. Another laboratory chose not to confirm reported variants related to pharmacogenetics, carrier status or low penetrant genetic risk associations. Yet another confirmed only those variants that clearly matched the clinical indication question, whereas, if a particular variant(s) explained only one of several clinical features, it would not be confirmed. Notably, the survey did not elicit analytical quality thresholds used in the decisions of which variants to review, focusing instead on policies regarding how different types of variants are handled.

Figure 2

Types of variants for which Sanger confirmation routinely occurs

Sequencing Test Report

Variant classes included in the report were influenced by whether the variant was relevant to manifesting symptoms or considered a secondary finding. In regards to diagnostic indications, 19 of 20 laboratories reported variants deemed pathogenic or likely pathogenic as well as variants of uncertain significance (VUSs) if related to manifesting symptoms. None of the clinical laboratories returned VUSs for secondary findings, though, two CSER laboratories did, consistent with their research goals. Three laboratories included likely benign or benign variants on the test report for diagnostic findings.

Case-specific filtering strategies, including gene lists, were used by most laboratories during analysis for at least a portion of cases to facilitate identification of potentially relevant genes/variants. In addition, case-specific data quality and exon coverage for phenotypically relevant genes may be considered during case analyses. Interestingly, 4 of 15 laboratories that reported use of phenotypically derived gene lists in at least a portion of cases indicated that such methods were not regularly communicated in the test report, though one laboratory would make them available by request. Further, case-specific coverage of genes, regardless of whether a gene list was used, was not universally conveyed. Ten of 21 laboratories routinely reported case-specific coverage and an additional 4 laboratories made it available separately or by request.

Secondary Findings

All laboratories indicated that they report secondary findings; however, there was considerable variability in the types of secondary findings reported and whether patients were allowed to “opt-out” of receiving them (Table 2). Eight clinical laboratories allowed patients to opt-out of receiving these results, whereas four of the clinical laboratories required “opt-in” for secondary findings reflecting deliberate patient choice. Of the CSER laboratories, one targeted to healthy participants did not allow opt-out of any findings, seven allowed opt-out of all secondary findings, and one required disclosure for a predefined set of genes while requiring opt-in to learn additional subsets. It is noted that the specific aims of each study influenced CSER laboratory approaches towards return of secondary findings and that the CSER protocols were likely written prior to the publication of the ACMG guidelines[12,14].

Table 2

Secondary findings reported

	Types of Secondary Findings Reported							Opt-out Available		No Opt-out
	ACMG 56* Genes	Medically Actionable**Identified DuringDiagnostic Analysis	Monogenic Disease(Adult-onset)	Monogenic Disease(Childhood-onset)	Carrier Status***	PharmacogeneticVariants	Complex Traits	Opt-Out Available for allSecondary Findings	Opt-in Required forSecondary Findings	No Opt-Out forACMG 56 Genes	Medically ActionableIdentified DuringDiagnostic Analysis
CSER	X		X	X	X	X		X
CSER	X		X	X	X			X
CSER	X		X	X	X			X
CSER	X		X	X	X	X	Xa			Xb
CSER	Xc		Xc	Xc				X
CSER	Xd		X	X	X	X	X		Xe	Xd
CSER	Xf		X		X	X		X
CSER	Xg	X	X		Xh				X
CSER	X	X						X			X
Clinical	X		X	X	X			X
Clinical	X	X						X
Clinical	X	X						X			X
Clinical	X							X
Clinical	X		X	X	X	X			X
Clinical	X	X						X
Clinical	X	X				X		X
Clinical	X		X	X	X			X
Clinical		X	X	X	X	X			X		X
Clinical	X								X
Clinical	X				X			X
Clinical	Xi	Xi							X

ACMG 56 genes was created to target adult-onset, monogenic disease considered to be highly medically actionable [12]

This category is indicated when the process for identification and reporting of these variants was considered a standard laboratory procedure. “Medically Actionable” or “Medically Important” was defined by the individual laboratories.

The number of genes analyzed for carrier status varied by laboratory from 3 genes to all OMIM recessive conditions.

Complex genetic risks if OR >2. Additional categories include blood group antigens and lipid profile associated variants.

Opt-out is not an option for any findings.

All secondary findings are limited to cancer-related germline findings in 1600 genes selected for somatic cancer purposes. Some of the ACMG 56 are contained within this targeted panel.

Use a list of approximately 160 “medically actionable” genes that includes several of the ACMG 56 list. No opt-out available for findings from this list.

Opt-in required for “non-medically actionable” secondary findings.

ACMG 56 except for 4 genes (RB1, WT1, APOB, PCSK9) causing childhood onset conditions, as study only enrolls adults

Use a list of 117 genes which include the ACMG 56.

The primary finding for this program was carrier status. It is included here to represent return of non-diagnostic information. However, compound heterozygous or homozygous variants identified through carrier analysis will also be returned.

Use ACMG 56 gene list as a guide; however, only report incidental findings discovered via diagnostic analysis

Five clinical laboratories limited secondary finding variant analysis and reporting principally to the 56 genes recommended for return by the American College of Medical Genetics and Genomics (ACMG)[12], though three of these reported additional “medically actionable findings” incidentally discovered in the course of the primary diagnostic analysis. Another clinical lab only reported variants within the ACMG 56 genes if they were discovered in the course of the primary diagnostic analysis. The number of genes included for analysis and return ranged from the ACMG 56 to “any human disease gene” (>4500 genes) and included categories such as monogenic disease, carrier status, pharmacogenomic variants, complex traits and blood antigen prediction. Again, research aims influenced the number and categories for CSER laboratories as exemplified by one that did not analyze four of the ACMG genes that caused childhood-onset conditions (RB1, WT1, APOB and PCSK9) because their study enrolled only adults, but added 50 additional adult-onset medically actionable genes to their list.

Reanalysis

The majority of laboratories indicated that they had reanalyzed case-specific data to provide an updated report at least once (11 of 12 clinical and 4 of 9 CSER). The instances were rare, however, with 7 of 12 clinical and 6 of 9 CSER laboratories indicating that reanalysis rarely or never occurred. Only one clinical laboratory routinely reanalyzed every case. When reanalysis was performed, roughly half used the existing variant call format (VCF) file and half performed new alignment and variant calling. Of the clinical laboratories, six indicated reanalysis would be free of charge, five charged a fee, and one was still developing its policy.

Raw Data Return

The US Department of Health and Human Services issued a rule change that went into effect in April 2014 specifying that clinical laboratories must provide copies of completed test reports to the patient upon request[15]. At the time of survey collection, there was some question as to what types of data this encompassed[16]. When queried about this new rule, three clinical laboratories indicated they would return uninterpreted sequencing data to patients and/or physicians upon request, whereas eight clinical laboratories would return it only to the physician. The frequency of actually doing this was low, however; one clinical laboratory had returned such data to patients (< 10% of cases). Return to physicians was more frequent, with four returning it in 10–50% of cases and four returning it for less than 10% of cases. Laboratories indicated a fee may be charged to offset costs associated with storage devices, shipping, and other expenses.

DISCUSSION

The results of our exploratory survey revealed numerous areas of convergence across laboratories, suggesting practices that may represent a developing standard for exome and genome sequencing. Examples include consent documentation, default inclusion of some secondary findings as well as allowing some choice, and case review by more than one individual. There were also some notable areas of divergence that warrant further discussion and possible clarification through practice guideline development.

Clinical NGS guidelines recommend detailed phenotypic information to aid the laboratory analysis and interpretation process[10]. Consistent with this recommendation, phenotypic data about the clinical indication(s) for testing were collected by all laboratories in various formats. The creation of phenotype-guided gene lists for defining high-priority genes was not uniformly used during the bioinformatic filtering steps. While many cases may be solved without such gene lists, the prioritized review of variants in genes most likely to be implicated enables further in-depth consideration of variants in higher likelihood genes to ensure they are not overlooked. The absence of these approaches in some laboratories may reflect the diversity of bioinformatics pipelines employed or other strategies to incorporate phenotype. It is also worth noting that if phenotypic assumptions are incorrect, over-reliance on gene lists could cause a clinically relevant variant to be missed. Continued standardization of analysis pipelines, robust methods and standardized ontologies for phenotypic collection, and the development of high quality curated gene-phenotype datasets are important to the assurance of comprehensive, consistent case analysis.

In conflict with sequencing guidelines, case-specific analysis approaches (e.g. phenotypically guided gene lists) were not routinely communicated through the test report[10]. When these analysis details are omitted from the report, it may be unclear to the clinician which genes were/were not analyzed for their patient or how well any particular gene was covered. This information would be of highest importance in the case of a negative result. Failing to provide a clear depiction of gene-centric analysis and coverage could lead to false assumptions by the ordering clinician that key genes had been ruled out.

Another area of discordance is revealed in laboratories’ use of group discussion in the case review process. Although all laboratories reported that more than one individual was involved in the analysis and interpretation of a given case, little more than half routinely utilized group review for most cases. Benefits of group discussion or solicitation of ad hoc expertise include the insight from multiple perspectives and expertise including bioinformatics[8], basic science, and clinical domain knowledge[17]. This ‘peer-review’ may improve accuracy and confidence in decision-making regarding the potential clinical relevance of variants[18]. Case volume, time demands and costs are obvious constraining factors for implementing group case review, as is solicitation of disease area expertise. It may be practical for experienced laboratories to consider this approach only in particularly challenging cases.

Most laboratories were in agreement about which variant classification categories warranted inclusion on the clinical report. However, policies governing the confirmation of reported variants, typically through Sanger sequencing, differed somewhat from lab to lab. Notably, one clinical lab indicated that they do not confirm any variants, and several others had certain categories of reported variants that were not confirmed. While it has been generally recommended that laboratories Sanger confirm reported germline variants[10], as laboratories gain experience with the analytic performance of NGS, there is increasing movement towards defining thresholds for quality (and perhaps clinical significance) for which such confirmation is unnecessary[19,20,21].

Survey results indicated that reanalysis of data remains predominantly an ad hoc service performed on request, rather than an integrated process. Several factors may play a role in this absence of common practice, including the rapid pace at which new genetic knowledge is generated as well as the relatively new addition of exome and genome testing services (many clinical exome sequencing services are less than three years old). Another important driver is likely the lack of the billing reimbursement infrastructure needed to support data reanalysis, interpretation and reporting[22].

In 2014, new Federal rules issued by the US Department of Health and Human Services (DHHS) specified that clinical laboratories are required to provide copies of completed test reports directly to the patient upon request[15]. January 2016 guidance from DHHS clarified that for genomic tests, the access right includes “a copy of the completed test report, the full gene variant information generated by the test, as well as any other information in the designated record set concerning the test”[23]. A major aim of the rule and guidance is to grant individuals access to their protected health information maintained by providers. Likely due in part to the uncertainty regarding the rule, it is interesting to note that the majority of laboratories reported they would release uninterpreted or unvalidated sequence data to the ordering physician only. Despite the recent 2016 clarification, guidance may be welcomed by the laboratory community on how best to effectuate this right of access. The customary practice in medicine has been to convey medical data through a patient’s clinician, who can interpret its significance in the appropriate clinical context as uninterpreted and unvalidated data would likely be inaccessible to direct interpretation by most patients. Laboratory practices are likely to continue to evolve as patients are taking an increasingly active role in their health and exercising their right to advance medicine through data sharing. Laboratories and clinicians may perceive some risks both professional (if subsequent analyses were to conflict with the original assessment) and resource related due to this type of data return. It is, however, critical that we as professionals maintain transparency and support peer review of our practices to ensure the highest quality care be delivered to patients.

Limitations

Given the rapid evolution of genomic sequencing practices, the testing market, and legal rules, as well as < 100% response from all laboratories at the time of this survey, these results may not reflect current exome and genome sequencing practices. All respondents reviewed the data for accuracy and major changes immediately prior to submission, however. Further, these results do not reflect practices outside the US and the majority of laboratories represented were primarily academic institutions, many of which may have had practices influenced by specific grant funding for clinical sequencing. Furthermore, because the survey review was conducted by respondents’ professional peers, it is possible some responses may have been impacted by perceived best practices. Lastly, this study provides observational data about current practices. There remains a need, however, for systemic evidence collection to define the clinical validity and utility of genomic sequencing such that recommended best practices can be better informed by underlying evidence.

CONCLUSIONS

This study provides an overview of general practices and policies of experienced exome and genome sequencing laboratories. The results enable broader consideration of which practices are becoming standard approaches and where development of best practice guidelines may be helpful. Notable areas for improvement include:

Transparency and clarity regarding test methods and limitations. We recommend that the scope of analysis (including the use of gene lists), case-specific coverage metrics, and analytical limitations, should be communicated with the report. Information might include:

List of genes targeted for analysis and the phenotype elements used to select them;

Stated threshold for minimum coverage and notation when coverage of a targeted gene falls below that threshold; and/or

Known pathogenic variation relevant to the indication but not detectable by the test.

Utilization of clinical domain expertise in case review. We suggest that laboratories consider implementing group case review with inclusion of varied expertise including clinical domain expertise. While not necessary for all cases, laboratories may wish to define circumstances in which group review is critical for improved case-specific determination of clinically relevant variants to report, as well as to provide a rich learning environment for all staff.

Confirmation of reported variants. We recommend that all variants reported to have potential diagnostic significance must reach a defined threshold for data quality or require confirmation by an orthogonal method. This should apply to both indication-specific and secondary finding variants likely to be used in clinical care. This would not apply to the return of uninterpreted sequence data addressed below under data access.

Data access guidelines. Federal rules and guidance now establish a patient’s right of access to “the completed test report, the full gene variant information generated by the test, as well as any other information in the designated record set concerning the test.” We suggest that the professional community establish guidelines surrounding the return of sequencing data directly to patients to guide laboratories in honoring patients’ right of access while minimizing potential harm from misunderstood or incorrectly interpreted results.

Data reanalysis. We recommend laboratories develop internal genomic sequencing data reanalysis guidelines. These guidelines are best informed by the professional community and should address both laboratory-initiated reanalysis and clinician-initiated reanalysis as well as the appropriate data to be reanalyzed (existing VCF vs. raw data requiring new alignment and base calling vs. new sequencing run) and when re-testing would be recommended over reanalysis.