The need for polygenic score reporting standards in evidence-based practice: lipid genetics use case.

PURPOSE OF REVIEW: Polygenic scores (PGS) are used to quantify the genetic predisposition for heritable traits, with hypothesized utility for personalized risk assessments. Lipid PGS are primed for clinical translation, but evidence-based practice changes will require rigorous PGS standards to ensure reproducibility and generalizability. Here we review applicable reporting and technical standards for dyslipidemia PGS translation along phases of the ACCE (Analytical validity, Clinical validity, Clinical utility, Ethical considerations) framework for evaluating genetic tests. RECENT
FINDINGS: New guidance suggests existing standards for study designs incorporating the ACCE framework are applicable to PGS and should be adopted. One recent example is the Clinical Genomics Resource (ClinGen) and Polygenic Score Catalog's PRS reporting standards, which define minimal requirements for describing rationale for score development, study population definitions and data parameters, risk model development and application, risk model evaluation, and translational considerations, such as generalizability beyond the target population studied.
SUMMARY: Lipid PGS are likely to be integrated into clinical practice in the future. Clinicians will need to be prepared to determine if and when lipid PGS is useful and valid. This decision-making will depend on the quality of evidence for the clinical use of PGS. Establishing reporting standards for PGS will help facilitate data sharing and transparency for critical evaluation, ultimately benefiting the efficiency of evidence-based practice.

INTRODUCTION

Atherosclerotic cardiovascular disease (ASCVD) is the leading cause of death in most nations [1,2]. Guidelines for the prevention of ASCVD center on screening of risk factors (e.g. age, sex, smoking, cholesterol, diabetes, blood pressure) to identify and treat individuals who are at increased risk [3]. However, traditional risk factors may not fully capture heritable risk, and assessment of genetic risk has emerged as a method for improving risk prediction [4]. Although genetic risk itself is not modifiable, individuals who carry elevated genetic risk may benefit the most from early identification and treatment [5,6].

A polygenic score (PGS) can quantify an individual's genetic risk for an outcome, such as coronary artery disease (CAD), or it may quantify the genetic influences on a measurable trait, such as cholesterol level or blood pressure. When measuring risk for a disease, the term polygenic risk score (PRS) is often used. PGS are calculated by summing the estimated effects of multiple genetic variants for a given trait, and they may consist of any number of variants from tens to millions. The effect estimates of the variants on the trait are derived from genome-wide association studies (GWAS). Unlike Mendelian genetics, in which single rare large-effect variants are often sufficient to cause disease, PGS describe the much more common phenomenon of inherited risk through the additive burden of many small-effect risk alleles. PGS can be conceptualized as the quantification of genetic factors that help explain disease heritability.

PGS for lipid traits are currently used as research tools for the study of dyslipidemias, and lipid PGS are already being incorporated into clinical practice in selected settings [7]. A full discussion of the potential clinical applications of lipid PGS is beyond the scope of this article (see Trinder and Brunham in same issue), but three broad areas of interest include: diagnosis of severe dyslipidemias, risk stratification for cardiovascular disease, and personalizing therapeutic approaches. Whatever the eventual clinical application, rigorous standards will be needed to establish best practices and to promote reproducibility and generalizability [8,9]. One approach to evaluating genetic tests is the ACCE framework, which refers to demonstrating Analytic validity, Clinical validity, and Clinical utility and Ethical considerations [10]. Here, we use the ACCE framework to highlight the needs for lipid PGS reporting standards to facilitate translational research and clinical applications. A detailed description of the translational agenda for genetic epidemiology is described elsewhere [11].

no caption available

REPORTING STANDARDS FOR POLYGENIC SCORES

Reporting standards refer to the minimal requirements for ensuring sufficient and transparent reporting on a study's design, methods, results, interpretation, and generalizability, specific to the particular study type and purpose [12]. They include both ‘what’ items should be reported, and ‘how’ they should be described. Standards are usually developed by a team of multidisciplinary experts to maximize relevance to downstream stakeholders [13]. As a secondary benefit, reporting standards naturally structure study meta-data, facilitating the systematic reviews and meta-analyses needed to synthesize a literature for practice guidelines [14,15].

Recently, the Clinical Genomics Resource (ClinGen) consortium and European Bioinformatics Institute's Polygenic Score Catalog have created a joint statement (termed ‘PRS-RS’) on recommended reporting standards for PGS [16▪▪]. This effort builds on foundational reporting standards in genetic risk prediction [17] and multivariable prediction models for individual prognosis or diagnosis [18].

The key components of these standards include reporting on the background rationale for the development of the score, study population definitions and data parameters, risk model development and application details, risk model evaluation metrics, and translational considerations, such as generalizability beyond the target population studied. These reporting standards, along with other technical standards, can help to facilitate the rigorous evaluation of new PGS in terms of analytic validity, clinical validity, clinical utility, and ethical considerations (Fig. 1). Here we discuss the application of standards, such as PRS-RS, to lipid PGS.

FIGURE 1

Standardizing along the ACCE framework for polygenic score translation. Translational considerations for polygenic score reporting standards along the ACCE framework using the vignette of translating LDL-C PGS into clinical care for ASCVD risk prediction for the general population. Row 2 describes research types and goals along the ACCE framework (Analytical validity, Clinical validity, Clinical utility, Ethical legal social implications) of genetic testing evaluation. Row 3 highlights the need for transparent reporting using ‘phenotyping’ as an illustrative example, with attention to aligning definitions across study types. Concepts apply to other reporting items, such as reporting on ancestry or population demographics, and comprehensive reporting on all aspects of the study are needed for critical evaluation of a score. ACCE, Analytical validity, Clinical validity, Clinical utility, Ethical considerations; ASCVD, atherosclerotic cardiovascular disease; LDL-C, LDL-cholesterol; PGS, polygenic scores.

Analytical validity

The analytical validity of a genetic test is a function of its ability to accurately and reliably measure the genotype(s) of interest [10]. It, therefore, depends on the sensitivity, specificity, and reproducibility of the genetic assay and the downstream computational analyses used to determine genotypes. Applying a PGS to a given individual requires assessing that individual for each genetic variant that is included in the score. Thus, analytic validity reflects how the score is assessed rather than the score itself. For example, a PGS for LDL-cholesterol (LDL-C) constructed from 12 genetic variants has been used to show that polygenic hypercholesterolemia may contribute towards a familial hypercholesterolemia phenotype [19]. In terms of analytic validity, the application of this score to an individual or cohort depends on accurately assessing for those specific genetic variants. The authors did so using a custom genotyping array constituted of 200 000 variants [20], including the 12 variants of the score. For simple scores consisting of a small number of variants, direct assay of each variant is typically used (e.g. by genotype array), and the analytical validity is a function of the assay itself.

In contrast, analytical validity becomes more complex when considering expansive scores that cannot be measured solely by genotype array. For example, Rippati et al. constructed a PGS for LDL-C that consisted of ∼6 million variants. This LDL-C PGS strongly associated with risk for CAD in the study population [21]. Assessing for several million genetic variants cannot be done with standard genotype arrays. One option is to use whole-genome sequencing. However, the more common approach is to use imputation. Imputation is a method of inferring the presence of unassayed genetic variants based on the presence of assayed genetic variants [22]. These inferences are based on correlations between genetic variants (i.e. linkage disequilibrium) observed in a reference population. When imputation is used to calculate a PGS, the quality of the imputation impacts the analytical validity. In the case of the Rippati et al. study, both the study cohort and the imputation reference cohort were sampled from the relatively homogeneous Finnish population. Such congruence contributes towards more accurate imputation and thus better analytical validity.

These examples help to illustrate the PGS reporting standards that are needed when considering analytical validity. Details of the study population and score development are particularly important in this context. The genome build used to construct the score, the method of genetic assay (e.g. the genotype array), the approach to imputation, and the quality-control filters used throughout the computational analysis all must be reported in order to allow researchers to gauge analytical validity and to replicated findings.

When PGS are eventually implemented in clinical settings, regulatory bodies will likely establish clinical standards based on assessments of analytical validity. In United States, clinical genetic tests are regulated by the Food and Drug Administration (FDA) and testing laboratories must meet standards put in place by the Clinical Laboratory Improvement Amendments (CLIA) and College of American Pathologists (CAP) [23,24]. Although many of the existing clinical standards for genetic testing may extend to PGS, it can be expected that clinical implementation of PGS will require additional regulatory oversight.

Clinical validity

The clinical validity of a genetic test refers to its ability to detect or predict the phenotype or disease of interest, and it is commonly established through observational or longitudinal studies that demonstrate the strength of association between the assayed variant(s) and the phenotype. For Mendelian diseases, clinical validity is a function of the pathogenicity of the variants under consideration. Guidelines from the American College of Medical Genetics and Genomics and the Association for Molecular Pathology have helped to establish standards for evaluating the available evidence to determine if a given variant is likely to be pathogenic, benign, or of uncertain significance [25]. These guidelines cannot be directly applied to PGS, as the framework of pathogenic versus benign does not translate to the context of a score. Nonetheless, key principles behind these guidelines remain relevant. Specifically, clinical validation of a PGS requires that the study cohort is a representative sample of the population for whom the PGS is intended. Further, it is necessary to validate each intended use of a PGS.

The statistical measures of clinical validity are clearly important to report, as described by the reporting standards for score evaluation in PRS-RS [16▪▪]. However, all categories described in the reporting standard should be addressed to provide a full understanding of the clinical validity of a PGS. Score background and study population are particularly important for lipid PGS. The score background encompasses the clinical measure or outcome the score is intended to predict. A lipid PGS may be developed for multiple purposes. For example, a PGS for LDL-C is methodologically constructed to predict LDL-C levels, but it may be used in order to predict other outcomes, such as response to medication [26] or development of atherosclerotic cardiovascular disease [21]. Each purpose requires distinct clinical validation. Characteristics of the study population used for validation will dictate the contexts that a score maintains its clinical validity. Lipid values vary by age, sex, and genetic ancestry group, and the performance of a PGS in a study cohort may depend heavily on these factors. It is also important to report how the validation population was sampled (e.g. biobank, clinical cohort, case--control study, etc.). For example, in a study of participants with familial hypercholesterolemia presenting to a lipid clinic, those who had both a monogenic familial hypercholesterolemia variant and a high PGS for LDL-C had the highest risk for premature ASCVD [27]. The studied LDL-C PGS may have clinical validity for predicting ASCVD in patients with familial hypercholesterolemia referred to specialty clinics but the clinical validity of this score for the general population cannot be assumed. Conversely, a lipid PGS that improves ASCVD prediction in a general population may not necessarily hold the same clinical validity when applied to individuals with familial hypercholesterolemia. Finally, there may be special considerations for evaluating the clinical validity of integrated risk models that combine PGS with conventional risk factors.

Clinical utility

Clinical utility refers to the benefit--risk assessment of a new clinical test in practice, typically focused on health outcomes [10]. As analytical validity and clinical validity are prerequisites for establishing clinical utility, the PGS reporting standards described in the previous sections will be essential for future studies to establish clinical utility for lipid PGS. Once validated, a PGS can be used as an intervention in clinical utility study designs, such as randomized control trials [28]. Unified definitions across clinical validity and clinical utility studies can help with translation [29,30]. For example, the predicted outcome phenotype defined in a clinical validity study should match the clinical definition of related clinical utility studies, as informed by current clinical practice definitions. Figure 1 provides an illustrative vignette of the translational value of unified reporting standards using ‘phenotyping’ as an example.

Reporting standards are available for typical clinical utility study types, including randomized control trials [31], pragmatic randomized control trials [32], or quality improvement research [33]. There is also recent attention to hybrid effectiveness study designs, which bridge efficacy metrics from randomized control trials (e.g. PGS ability to improve clinical outcomes) and effectiveness outcomes of implementation research (e.g. uptake of PGS in target populations) [30]. Elements of the aforementioned reporting standards will need adapted to PGS-specific study design and outcome considerations for clinical utility. Downstream perspectives should be engaged early on efforts to create new PGS reporting standards to ensure interests are aligned prior to implementing a new tool at scale [30,34]. For example, clinicians can help define ideal target populations, risk thresholds, and other pragmatic considerations.

Ethical considerations

Analytic validity, clinical validity, and clinical utility are often the primary focus of research into new biomedical tools but ethical considerations are equally, if not more, important for determining if widespread adoption of the tool is appropriate. It is still unclear how to deliver PGS without exacerbating current ethical considerations for clinical genetic testing including concerns related to discrimination, stigmatization, privacy, psychological harms, and implications for family members [35-39]. Further, polygenic traits/diseases are common and may require greater attention to addressing questions about the public health utility of genetic testing, such as cost-effectiveness [40] and equitable access to care [41,42].

Already, the most pressing ethical consideration for PGS is that of health disparities. Although PGS offers a promising step forward for more precise diagnosis and risk stratification, PGS research is at risk of propagating and amplifying existing health disparities [43]. GWAS, which serves as the basis for PGS development, has a strong historical bias for focusing on populations of European ancestry [44]. This bias has now led to the development of PGS that are optimized for individuals of European ancestry [45,46], and this bias in turn leads to validation studies focused on populations of European ancestry.

Addressing health disparities will require a multifaceted approach but clear reporting on the genetic ancestry of study populations and the limitations and generalizability of new PGS will be a key component of the solution. Depending on the methods used, the ancestry of several populations may need to be reported, including: the GWAS population used to construct the score, the reference population used for imputation, the training population used to optimize the score, and the testing population used to validate the score. Consistent reporting of this information with the publication of new scores will hopefully serve to draw attention to the disparities that currently exist and to help the research community to address these disparities in a directed fashion. Similarly, the adoption of reporting standards regarding the study of public health utility metrics, such as economic evaluation [47], patient/public involvement [48], and implementation effectiveness [15,49], will help to systematically quantify other pain points in the healthcare delivery of PGS for policy-makers.

CONCLUSION

Methods for PGS development and application are rapidly evolving, and lipid PGS are likely to be integrated into clinical practice in the future. Clinicians, healthcare systems, and policy-makers will need to be prepared to determine if and when assessment for lipid PGS is useful and valid. This decision-making will depend on the quality of evidence for the clinical use of PGS. Establishing reporting standards for PGS will help facilitate data sharing and the continued advancement of PGS research in order to ultimately benefit patients.

Many of the advancements in PGS research are thanks to data-sharing standards already established by the genomics research community. The GWAS catalog [50] has played a pivotal role, allowing researchers to easily access the results of prior GWAS in order to construct PGS. Recently, the PGS Catalog [51▪▪] was established to further this purpose and includes much of the metadata relevant to the PGS reporting standard described in this perspective. Currently, the PGS hosts more than 300 scores across more than 100 traits, including 15 scores for lipid traits. However, many scores have yet to be deposited. In addition to community resources, such as the PGS Catalog, additional strategies and incentives may be necessary. Journals can help promote widespread use of reporting standards through publication policies, and funding agencies may feel compelled to enforce reporting standards in order to improve translational efficiency.

Finally, it is important to acknowledge that commercial clinical testing of PGS has already begun [16▪▪]. Lipid PGS and ASCVD prediction are key first areas for commercialization, given the public health significance and anticipated actionability. Standardization should be done now to proactively guide future study development and not in retrospective response to the accumulation of imperfect study outcomes. Standards alone will not be sufficient in standardizing care. Equally important considerations include provider training, public education, and sufficient planning to ensure equity, access, and protections [52]. The scientific and medical communities may lead these efforts, but polygenic risk score usage is primed for public engagement around precision medicine, and more diverse teams are needed to tackle these issues.

Limitations

In this perspective, we have outlined recommendations for reporting standards for PGS, focusing on lipid PGS to provide context. This narrow scope on single PGS reporting ignores a key anticipated benefit to PGS testing – the ability to use one assay to perform multiplexed analysis with several PGS [53]. With the same set of genotype data, it is possible to assess an individual for multiple scores. Moreover, the validity and utility of available scores is expected to improve over time, making re-analysis of the same genotype data a likely scenario. Although many of the concepts presented here are general, the settings of multiplexed and repeated analysis will likely require unique considerations and reporting standards. Other multiplexed genetic screening efforts may offer guidance, including newborn screening and return of incidental findings from exome sequencing.

Acknowledgements

None.

Financial support and sponsorship

This work is funded by the National Institutes of Health (NIH) National Human Genome Research Institute (NHGRI) Clinical Genome Resource (ClinGen) grant U41HG009649

J.W.K. is supported by the NIH through grants: P30DK116074 (to the Stanford Diabetes Research Center), R01 DK116750, R01 DK120565, R01 DK106236; and by the American Diabetes Association (grant #1-19-JDF-108).

Conflicts of interest

There are no conflicts of interest.