Development and validation of a brief screening instrument for psychosocial risk associated with genetic testing: a pan-Canadian cohort study.

OBJECTIVES: To develop a brief, reliable and valid instrument to screen psychosocial risk among those who are undergoing genetic testing for Adult-Onset Hereditary Disease (AOHD).
DESIGN: A prospective two-phase cohort study.
SETTING: 5 genetic testing centres for AOHD, such as cancer, Huntington's disease or haemochromatosis, in ambulatory clinics of tertiary hospitals across Canada. PARTICIPANTS: 141 individuals undergoing genetic testing were approached and consented to the instrument development phase of the study (Phase I). The Genetic Psychosocial Risk Instrument (GPRI) developed in Phase I was tested in Phase II for item refinement and validation. A separate cohort of 722 individuals consented to the study, 712 completed the baseline package and 463 completed all follow-up assessments. Most participants were female, at the mid-life stage. Individuals in advanced stages of the illness or with cognitive impairment or a language barrier were excluded.
INTERVENTIONS: Phase I: GPRI items were generated from (1) a review of the literature, (2) input from genetic counsellors and (3) phase I participants. Phase II: further item refinement and validation were conducted with a second cohort of participants who completed the GPRI at baseline and were followed for psychological distress 1-month postgenetic testing results. PRIMARY AND SECONDARY OUTCOME MEASURES: GPRI, Hamilton Depression Rating Scale (HAM-D), Hamilton Anxiety Rating Scale (HAM-A), Brief Symptom Inventory (BSI) and Impact of Event Scale (IES).
RESULTS: The final 20-item GPRI had a high reliability-Cronbach's α at 0.81. The construct validity was supported by high correlations between GPRI and BSI and IES. The predictive value was demonstrated by a receiver operating characteristic curve of 0.78 plotting GPRI against follow-up assessments using HAM-D and HAM-A.
CONCLUSIONS: With a cut-off score of 50, GPRI identified 84% of participants who displayed distress postgenetic testing results, supporting its potential usefulness in a clinical setting.

A significant group of individuals undergoing genetic testing for adult onset disease experience distress or challenges in adaptation, whereas some might develop depression or anxiety.

Existing psychological screening tools do not take into consideration of ‘risk factors’ associated with heritable illness or genetic-related stressors.

A screening tool designed for genetic testing services is a useful tool to guide clinicians in relation to which patients would benefit from added psychosocial support during the genetic testing process.

A subgroup of patients undergoing genetic testing required added psychosocial support to facilitate adaptation to genetic/risk information. Busy genetic service providers can face challenges to identify these individuals and provide timely interventions or referrals.

A new brief instrument was designed and validated to identify those individuals at risk for psychological distress such as depression or anxiety who are undergoing genetic testing for adult-onset hereditary diseases.

This is the first study to develop and validate a psychological screening instrument for the genetic testing field.

This newly developed tool, Genetic Psychosocial Risk Instrument (GPRI), is the first reported psychosocial screening instrument for use across Adult- Onset Hereditary Diseases (AOHD).

The GPRI demonstrates promising psychometric properties as a tool designed to help genetics healthcare providers determine which of their patients undergoing genetic testing for AOHD is at increased psychological risk and who will benefit from added psychosocial support.

The study findings are limited by the characteristics of the sample; most participants were women and undergoing testing for BRCA1/2. Future studies could further address the validity of GPRI in male populations and in the rare AOHDs, such as Huntington's disease.

Introduction

Genetic predisposition is an important determinant of chronic disease and disability. Despite the benefits of genetic testing, such as increased screening or prophylactic interventions, individuals at high risk for serious illness may become increasingly fearful or distressed about the future. In fact, a consistent finding is that the majority of individuals do adjust to genetic test results; however, a subset of individuals undergoing genetic testing for Adult-Onset Hereditary Disease (AOHD) experience psychological distress, such as anxiety or depressive symptoms. A screening tool, designed for the genetic testing context, would be helpful in assisting geneticists, genetic counsellors or primary care providers to identify this particular group for the implementation of appropriate preventive or follow-up interventions. Herein, we present a newly developed psychological risk screening instrument that can be readily used within a genetic service for AOHD.

Risk factors and psychological impact of genetic testing: the evidence

The knowledge of genetic risk is lifelong and individuals and families often find themselves confronted with an ongoing need to face issues and make decisions. Examples include decision-making around prevention and treatment options (eg, increased surveillance, prophylactic surgery, chemoprevention), the need to notify family members about a mutation in the family and personal decision-making, for example, decisions involving childbearing.1 2 Studies utilising standardised measures of distress (eg, global measures of anxiety or depression symptoms) have demonstrated that 8–25% of individuals undergoing genetic testing experience distress, the level of which falls within the clinical ranges for depression and anxiety.2–5 Studies that have utilised standardised measures of disease-specific distress (ie, instruments measuring breast/ovarian cancer worry) have demonstrated higher prevalence levels.6 7

The risk factors for psychological symptoms among individuals undergoing genetic testing have been delineated in several studies.4 8 9 While there is generally elevated distress in using global measures for depression or anxiety among those who receive positive test results,9–11 individuals testing negative or receiving uninformative results may also have adjustment difficulties 12 following testing. For example, individuals may feel guilt or continue to worry about their disease risk even when testing negative.2 7 12 These findings highlight the importance of considering risk factors in addition to the type of the test result itself. Individuals who have elevated psychological symptoms at the pretest stage and those with a previous psychiatric history (ie, depression) are particularly at risk for an adverse psychological outcome after testing.2 8 9

Additional risk factors for distress are more specific to the genetics context and include the level of penetrance of the gene mutation or degree of certainty of developing the disease.4 The perception of control over the disease (including the number of prevention/treatment options) and perception of the immediacy of risk (proximity in age to perceived disease onset) are important predictors.4 13 The expectation of a negative test result can play a role in adjustment, as can the context of test results of other family members.9 14 As in other medical areas, specific coping styles can affect adjustment.15 The prior experiences with loss of family members to disease, as well as the developmental level (ie, young age) of the individual at the time of the loss,2 3 16 are significant factors affecting potential adjustment. In addition, the prior experience of giving care to a family member with the disease and lower levels of social support have been associated with poorer adjustment following a positive test result.2–4 8 16

It is clear that there is not just one predominant factor, but rather a series of variables that can be assessed prior to receiving a test result that may contribute to elevated levels of psychological distress following genetic testing.2 17 Emotional reactions may impede the assimilation of risk information and the adoption of preventive measures recommended following notification of a mutation.2 18 Psychological distress occurs along a continuum19 20 and can be difficult to identify by health professionals.21 Distress may not become manifest to the healthcare team until the patient reaches an observable crisis level, that is, the onset of severe depression or anxiety, or in case of significant conflicts with the family. An early screening instrument would enable healthcare providers to identify patients being at higher psychological risk in order that appropriate support can be given at the right time. In fact, there is now a general consensus that genetic testing should be accompanied by psychological support to promote optimal adjustment.2 22

Screening for psychological risk factors: why is it necessary?

The gold standard for identifying psychologically distressed individuals involves structured clinical interviews administered by a clinical psychologist or psychiatrist.21 However, it is too costly and often not feasible in genetic clinics. Standardised measures of psychological functioning (eg, global scales of depression or anxiety) can also be used as a method for identifying distress. However, few clinics use these measures in practice because of personnel and time requirements for scoring and their interpretation. Furthermore, items on these measures typically focus on symptoms of anxiety or depression, rather than on variables associated with heritable disease or genetic testing or risk, which may pose barriers for use by genetics health service providers who may prefer instruments that, at face value, appear to them and their patients as being clinically more relevant to the genetic testing context.

More recently, new outcome measures designed to assess the psychological impact of receiving genetic information have been developed. For example, the Multidimensional Impact of Cancer Risk Assessment is designed to assess concerns and impacts associated with genetic testing for BRCA1/2,19 and another tool, the Psychological Adaptation to Genetic Information Scale, is now available.23 While these measures will require further validation they provide more clinically relevant approaches to capturing specific impacts of genetic information, such as the increased sense of vulnerability often experienced following genetic testing.19 23

Measures of global psychological functioning and the evolving outcome measurement tools for the genetics field are not designed to ‘predict’ vulnerability for future distress, but rather, they measure current distress levels. Screening, the aim of the tool developed in this study in contrast, is a rapid, cost-effective alternative21 to prospectively identify individuals who may experience significant difficulty in their attempts to adapt to their genetic information.17 A screening tool enables providers to offer timely and focused educational and psychosocial interventions to prevent future distress.

The primary objective of this study was to develop a brief, reliable and valid psychological risk screening instrument for use in the genetic testing context. The new instrument aimed to incorporate empirically-based risk factors for psychological symptoms and would need to show a high sensitivity, specificity and predictive validity indicating risk for future distress postgenetic testing results. A cut-off point would need to be determined to guide clinical decisions as to whether or not to refer, further assess or intervene to reduce an individual's expressed concern.

Methods and materials

The study was carried out from September 2005 to July 2010, with research ethics board approval from participating genetics clinics: Toronto (Mount Sinai Hospital, North York General Hospital, Princess Margaret Hospital); Ottawa (Children's Hospital of Eastern Ontario) and Vancouver (British Columbia Cancer Agency). Individuals beginning the genetic testing process for AOHD at each site were approached by genetic counsellors on the project team for their permission to be contacted about the study. Those who expressed interest were mailed the baseline package that included the informed consent. The informed consent included all components of the study, including questionnaires, follow-up phone calls, telephone interviews, as well as the release of their genetic testing information to the research team.

A two-phase approach was used for this study: Phase I: Item generation and refinement, and Phase II: Validation. The multistage method24 takes validation into consideration at each stage of scale development and has been used successfully in previous studies.25

Phase I: Item generation and refinement

Item generation

To generate items for the Genetic Psychosocial Risk Instrument (GPRI), a literature search was performed for the following AOHDs: Cancer (Hereditary Breast-Ovarian Cancer Syndrome/Lynch Syndrome), Huntington's disease (HD) and haemochromatosis. These diseases were selected as they represented the majority of patients attending genetic clinics and had an associated available psychosocial literature for review. Databases including Cinahl (1982–2006), Medline (1966–2006), PsychInfo (1985–2006) and Pubmed (1985–2006) were searched, apet from a hand search of references from major publications. Keywords included: genetic screening, genetic testing, psychological, psychological well-being, psychological adjustment, stress, adaptation, cancer worry, disease worry and distress. Selection criteria for the literature review included studies with a follow-up design or review articles. Each selected study was reviewed by two reviewers on its quality of evidence and generalisability using a standardised template. A total of 73 relevant studies were identified among the disease groups: 49 on cancer, 20 on HD, 2 on haemochromatosis and 2 that described mixed conditions.

Risk factors for psychological distress identified by the literature review provided the basis for item generation. Items were written in a mixed format where respondents were asked for their endorsement of each statement ranging from a Yes/No for risk factors of binary nature, to a 5-point Likert-type scale for risk factors with stages in frequency and/or intensity. The instrument items were further refined by 10 genetic service providers (3 geneticists, 4 genetic counsellors, 2 oncologists and 1 genetics nurse) rating items on comprehension, readability and perceived clinical relevance using a 10-point scale with 0 being ‘excellent/definitely relevant’ and 10 being ‘very poor/definitely not relevant’. Risk factor items were removed if rated above 5 by more than three providers. Providers were also asked to suggest additional risk factor items. These suggestions were checked against the literature for empirical evidence. Following this step, seven volunteers undergoing genetic testing for AOHDs were recruited to try out the scale for clarity, succinctness and relevance from the clients’ perspectives. At this stage, the proposed instrument consisted of 56 items: demographics (4 items); perceived risk (8 items); life events and family history of the disease (8 items); perceived impact of carrying a mutation (9 items); family communication (6 items); disease-specific concerns (5 items); optimism (3 items); social support (3 items), premorbid functioning and previous psychiatric history (10 items).

Item refinement

Subjects: Following informed consent, a convenient sample of 141 participants who had given blood for genetic tests at the Toronto and Ottawa sites completed the GPRI (using a three patients per item ratio) to select the best items for the candidate scale. The participants were middle aged (48.67±13.29), mostly women (77%) testing for hereditary breast cancer, and many (65%) had already suffered the onset of the illness.

Scoring: To ensure that binary items carry an equal weight as the 5-point Likert-type items, a score of 5 was assigned to Yes and 1 to No. A score of 3 or mean-substitute was assigned to Not Applicable to allow it to be counted in the total score. Reliability analysis was carried out and a Cronbach's α was set for 0.75 or higher for the scale to move to the next phase.26 Any item with an item-total correlation less than 0.20 was identified for potential removal. Using team consensus, a total of 19 items were removed, combined or substituted, resulting in a 37 item GPRI candidate scale at the end of phase I.

Phase II: Scale validation

Subjects: Individuals undergoing genetic testing for one of the AOHDs in each of the five study sites were invited to participate: (1) age 18 or above undergoing genetic testing for cancer, HD or haemochromatosis; (2) fluent in English and (3) residing within 1.5-h driving distance from the study site. Although the onset of an AOHD was not an exclusion criterion, individuals in advanced stages of the illness and/or those who were unable to consent due to cognitive impairment were excluded. At baseline, participants were asked to complete a set of self-report questionnaires (eg, Brief Symptom Inventory (BSI), etc) described below within a 1-month period following the provision of a blood sample. For those who received a genetic test result, questionnaires were mailed within 2 weeks to 1 month of the disclosure of the test result. These participants were also telephoned to complete the Hamilton Depression and Hamilton Anxiety Scales to further assess depressive and anxiety symptoms.

Materials: At baseline, three psychosocial measures were used: GPRI Candidate Scale from Phase I. To facilitate scoring of the scale by genetic providers, scores for response to each item on the GPRI were imbedded in the questionnaire, where clinicians could calculate a total score in less than 5 min. BSI: The BSI is a 53-item measure of psychological distress that contains three global scales (1) depression, (2) anxiety and (3) somatisation.27 While it has some limitations due to its being a self-report measure, it has been well validated and widely used in medical and psychiatric populations to assess psychological functioning; Impact of Event Scale (IES): the IES is a 15-item, Likert-style scale used to assess the experience of a specific stress response and is designed to be easily anchored in relation to a specific stressor or life event. It has been extensively utilised in the genetics literature to assess genetic testing-related distress; we similarly anchored the IES items in relation to the anticipation of the genetic test result at baseline and in relation to the actual genetic test at follow-up. The IES has two sub-scales: (1) intrusive thoughts and feelings associated with the stressful life event, and (2) items associated with patterns of avoidance of certain thoughts, feelings or situations.28

Measures at 1-month postgenetic testing results included: the self report scales of the BSI, IES and each participant received a telephone call for the telephone-based Hamilton Depression Rating Scale (HAM-D) and Hamilton Anxiety Rating Scale (HAM-A). The HAM-D evaluates a depressed mood, as well as vegetative and cognitive symptoms of depression and comorbid anxiety symptoms.29 The HAM-A quantifies the severity of anxiety symptomatology and consists of 14 items. The HAM-D and HAM-A have demonstrated validity in a clinical interview, in person or by telephone.30 These two instruments were selected as main outcome measures based on the literature that the standardised interview-based rating scales should be used over subjective report scales as the principal outcome criterion in psychological distress both in general practice and in research trials.31 Cases would be defined by established cut-offs from the literature for HAM-D ≥1232 or HAM-A ≥10.33 These cut-off points were established for populations in general practice, which was our study population.

The 1-month follow-up time point was selected as it is when elevated distress might occur.34 In addition, the 2-week duration criterion for depression defined by the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition is met by this time frame.

Assessing psychometric property of the scale

As a first step, items were required to have at least an 80% response rate. Second, each item was examined to determine its contribution to the internal consistency of the total 37-item scale. The minimum item-total correlation was set at 0.20.35

A principal components factor analysis with varimax rotation was performed on the candidate scale to examine the factor structure and the loading of the items. To assess the convergent validity of the candidate scale, the correlations between baseline GPRI, IES and BSI were calculated. To assess the sensitivity, specificity and predictive value of the GPRI, the follow-up HAM-D and HAM-A were used to identify ‘cases’ who met cut-offs for either the depression or anxiety symptomatology. For example, participants with a high GPRI at baseline would be classified as ‘at risk’ for future onset of adjustment difficulties. This would be confirmed by a high HAM-D or HAM-A score or ‘case’ during 1-month follow-up. Similarly, those with a low GPRI score should receive a low score in HAM-D or HAM-A as ‘non-cases’. The predictive value of the GPRI, describing the proportion of test-positives (in our case, high GPRI) who truly have the psychological condition (ie, cases identified by HAM-D or HAM-A), was tested by a receiver operating characteristic (ROC) curve which visually plotted the true-positive rate (sensitivity) over the false-positive rate (1 − specificity). We included cases to be identified by either anxiety and/or depressive symptomatology as both have been reported in the literature.8 9

To address the issue of missing follow-up data in a cohort study, as suggested in the literature,36 we tested the assumption that the subsample with missing data had a similar baseline exposure (similar GPRI) as the non-missing subsample by comparing baseline GPRI between the participants and dropouts. This step assesses if there was systematic bias resulting from the loss of information in the follow-up period.

Results

Participant characteristics

Study packages were mailed to 1129 individuals interested in hearing more about the study. Of these individuals, 722 of them consented and 712 (98%) completed the GPRI. Most participants were tested for the inheritable cancers, while a small percentage of participants were tested for haemochromatosis and HD. Similar to phase I, phase II participants were mostly women, at the mid-life stage, and more than half had a past diagnosis of the disease (see table 1).

Table 1

Description of Phase II participants’ characteristics (N=712)

Variables in GPRI*
Age in years: mean (SD)		49.80 (±12.53), range 18–80, median 50.00
Gender: n (%)	Male	85 (12%)
	Female	627 (88%)
Type of AOHD being tested: n (%)	Cancer (BRCA)	580 (82%)
	Cancer (other, ie, Colon)	90 (13%)
	Huntington disease	31 (4%)
	Haemochromatosis	5 (1%)
Personal history of disease being tested: n (%)		441 (62%)
Recent significant event (diagnosis of or loss of significant others to the disease being tested): n (%)		333 (47%)
Disease worries affect daily mood (strongly agree or somewhat agree): n (%)		189 (27%)
Sad in the past month (often or almost all the time): n (%)		121 (17%)
Anxious in the past month (often or almost all the time) n (%)		121 (17%)

*Note: there are missing data for some GPRI variables. The total count for each variable does not necessarily add up to 712.

AOHD, Adult-Onset Hereditary Disease; GPRI, Genetic Psychosocial Risk Instrument.

Of the 712 participants, 85 (12%) did not receive genetic testing results at the scheduled follow-up time and were not eligible for follow-up measures on psychological symptoms in response to a genetic testing result. Of the remaining 627 participants, 152 (24%) did not return the self-administered follow-up questionnaires and 12 (2%) submitted the follow-up questionnaire package but did not complete a standardised telephone interview using HAM-D and HAM-A (up to four telephone calls were made to reach each participant). Therefore, the final number of participants with complete follow-up data is 463 (74%). The age and baseline GPRI score were compared between individuals who did not receive genetic testing results (age 51.4±12.7, GPRI 49.3±12.7), those who did not return the follow-up questionnaires (age 48.1±11.6, GPRI 50.2±14.4) and those who completed follow-up measures (age 50.1±12.8, GPRI 49.1±13.5) were compared. There was no statistically significant group difference (analysis of variance and all post hoc comparisons p>0.05).

Owing to the similarity between the dropouts and the completers, we proceeded with reliability and validity analysis of the tool using the subsample that provided outcome data.

We carried out the calculations for distress level, for example, for depression and anxiety symptoms using the BSI data, for specific distress associated with a genetic test result using the IES. Approximately 13.0–20.1% of participants reached the threshold of moderate to severe distress, respectively, (see table 2).

Table 2

Psychological symptom of distress 1-month postgenetic testing results by disease type (N=475)

	Overall N (%)	Huntington	BRCA	Other cancer
IES intrusion >=17*	60 (13.0%)	5 (23.8%)	51 (12.5%)	4 (9.5%)
IES avoidance >=17*	65 (13.7%)	5 (23.8%)	57 (14.0%)	3 (7.1%)
BSI-18 total >=1343	95 (20.1%)	6 (28.6%)	86 (21.1%)	3 (7.1%)

*Shemesh E et al. Post-traumatic stress, non-adherence and adverse outcome in survivors of a myocardial infarction. Psychosomatic Med 2004;66: 521–26.

BSI, Brief Symptom Inventory; IES, Impact of Event Scale.

HAM-D and HAM-A interview data from 463 participants were used as a further validation tool to measure psychological symptoms postgenetic testing results. Defined by cut-offs for HAM-D ≥1232 or HAM-A ≥10 in the literature,33 the rate for psychological distress of either depression or anxiety was 13.7% (N=63). The rate was 13% for HD, 15% for breast cancer and 7% for Lynch Syndrome.

Reliability and factor analysis

A reliability analysis was performed on 37 items. Twenty items belonging to 18 questions were selected based on the criteria for item selection described in the methods section. The Cronbach's α of the 20-item GPRI was 0.81, suggesting a good level of internal consistency.

The factor analysis resulted in a psychometrically sound three-factor solution, with subscales representing the dimensions of: (1) perceived impact and personal adjustment to genetic testing (12 items); (2) history of mental health concerns (5 items) and (3) personal history/family history/loss to cancer (3 items). All three factors met the minimum eigenvalue criteria of 1.

The first 12-item factor (α = 0.85), accounting for 22% of the variance, includes items associated with the anticipated or experienced impact of being at high risk for AOHD. Example items included: ‘My worries about the disease affect my daily mood’; ‘The disease for which I am at risk is currently causing a significant disruption in my family life’.

The second five-item factor (α=0.76) accounted for an additional 14% of the total variance, and reflected a sense of a person's history or vulnerability in the area of mental health, for example, ‘I have had emotional problems in the past’. These items have been used in other medical health areas37 38 and tend to be predictive of maladjustment20 following a life event.

The third three-item factor (α=0.08) accounted for 8% of the total variance and pertained to personal or family-related experiences associated with the heritable disorder for which the participant is undergoing testing. Examples include: ‘I have a personal diagnosis of the disease for which I am receiving counselling’; ‘I lost a close family member to the disease for which I am receiving counselling’ and ‘I have taken care of a very ill parent or another close family member’. These three final items had low item total correlation because they were different from the rest of the items in that they focused on direct experiences related to the illness, rather than psychosocial-related items. These items were kept in the scale as they contributed significantly to the overall variance, and correlated highly with HAM-D and HAM-A. To determine the relationships between the three factors/subscales, correlations were computed. Factor1 and factor2 had moderate correlations with each other (factor1/factor2 r=0.30, p<0.01). The correlation of the first two factors with factor3 was much lower as expected (factor1/factor3 r=0.06, and factor2/factor3 r=0.01, not statistically significant). These results support the multidimensional character of the GPRI scale (see table 3).

Table 3

Genetic Psychosocial Risk Instrument factor solutions and factor loadings

	Factor loadings	Communalities	Item-total	Item mean
▸ My worries about the disease affect my daily mood	0.759	0.652	0.582	2.22
▸ I worry often about my risk of getting the disease	0.742	0.551	0.529	2.67
▸ I am concerned about my risk of getting the disease	0.656	0.484	0.472	3.28
▸ I have generally felt nervous and anxious in the past month	0.652	0.538	0.600	2.54
▸ I have generally felt sad in the past month	0.627	0.524	0.572	2.58
▸ If I learn that I have a genetic mutation,
… I will have more problems in my life	0.617	0.406	0.399	2.79
…I will have difficulties with my family relationships	0.513	0.324	0.424	1.62
… I will change plans for my career	0.451	0.228	0.262	2.08
▸ The disease is currently causing a significant disruption in my family life	0.568	0.408	0.463	2.42
▸ I am worried that my test result will impact on my relationship with my significant other	0.546	0.308	0.383	2.54
▸ I am worried about talking to my children about the heritable nature of the disease for which I am being tested	0.522	0.326	0.453	2.04
▸ I feel guilty that I might pass on the disease risk to my children	0.508	0.276	0.414	3.11
Factor 1: Anticipated or experienced impact of having a disease risk or genetic mutation: 12 statements, Cronbach's α=0.85, interitem correlation=0.32, variance explained=22%
▸ I have had emotional problems in the past	0.796	0.655	0.423	2.66
▸ I have been diagnosed with a depressive or anxiety disorder in the past	0.769	0.596	0.349	2.01
▸ I have had counselling with a mental health professional in the past	0.762	0.593	0.433	2.85
▸ I have had emotional problems that led me to thoughts about suicide	0.623	0.389	0.262	1.45
▸ I am now seeing a counsellor for one or more of these emotional concerns	0.509	0.272	0.274	1.35
Factor 2: Personal history or vulnerability to mental health issues or symptoms: 5 items, Cronbach's α=0.76, interitem correlation=0.39, variance explained=14%
▸ I have taken care of a very ill parent or another close family member	0.687	0.493	0.116	2.36
▸ I lost a close family member (eg, parent/sibling) to the disease for which I am receiving counselling/testing	0.667	0.445	–0.002	2.87
▸ I have/had a personal diagnosis of the disease for which I am receiving counselling/testing	–0.642	0.413	–0.073	3.47
Factor 3: Personal or family history of the genetic disease being tested in the clinic: 3 items, Cronbach's α=0.08, interitem correlation=0.03, variance explained=8%

One additional statement ‘I am interested in talking to a counsellor about one or more of these concerns’ was added to the tool at the end as suggested by the participants and providers to remind them of the option of seeing a counsellor if required. This statement is not part of the items examined during the instrument development and therefore does not carry a score.

The total score for the 20-item GPRI ranged from 20 to 100, with a sample mean of 49.36±13.23. The total was calculated by the sum of the raw scores for each of the statements. Females had a significantly higher score for the GPRI than males (50.37±13.14 vs 41.91±11.47, p<0.01), and participants testing for HD had a higher but non-significant score than participants testing for cancer (52.24±13.24 vs 49.37±13.22, n.s.).

Validity

Construct validity—correlations: the GPRI was assessed for its correlation with other standardised self-report measures of psychological functioning collected at baseline. Convergent validity was demonstrated by the correlation between the GPRI and the following measures: a positive correlation with the IES total score at r=0.51, p<0.001 and with BSI at r=0.58, p<0.001.

Sensitivity, specificity and the predictive value of GPRI for future distress: the telephone interview-based HAM-D and HAM-A were used to identify participants who presented specific psychological symptoms of distress such as depression and/or anxiety during the 1-month postgenetic testing follow-up. A total of 63 ‘cases’ (13.6% of the 463 completers) were identified as having psychological distress levels above the specified thresholds defined in the methods section for either anxiety or depression symptoms or both. About 23% among participants testing positive met the distress threshold, as did 10% among those with negative results and 20% among the uninformative participants. Participants scoring above the HAM-D (N=55) threshold had significantly higher GPRI scores than participants below the threshold (N=408; 61.12±13.27 vs 47.91±12.27, p<0.01). The same patterns were observed for HAM-A high (N=40) versus low (N=423; 62.53±12.92 vs 48.25±12.43, p<0.01).

Other demographic characteristics of these 63 participants include: most were women and undergoing testing for BRCA1/2, which was similar to the whole sample of 712 participants (table 1). Compared with the whole sample, these 63 participants had a slightly higher percentage of personal history of cancer (65% vs 62%), higher rate of recent significant event of loss (56% vs 47%), greater percentage reporting disease worries affecting mood (54.8% vs 27%), having a feeling of sadness in the past month (46% vs 17%) and anxiousness in the past month (33% vs 17%). Our instrument captured all of these characteristics of this subsample.

The predictive value of a test describes how many of the test-positives (in this case, a high score on GPRI) truly have a psychological condition. An ROC curve was used to plot the true-positive rate (sensitivity) over the false-positive rate (1 − specificity). A good ROC curve rises sharply, indicating a high proportion in true positives and a low proportion of false positives. The ROC curve for the GPRI was 0.78, which is considered as an indicator of an adequate screening instrument.39

An important purpose of the GPRI in our study was to identify individuals at risk for postgenetic testing psychological distress. Therefore, the cut-off value was set to maximise sensitivity—in another word, not to miss detecting a ‘case’. Using a GPRI cut-off score of 50, the instrument was able to predict 84% of the ‘cases’ identified by HAM-D or HAM-A conducted postgenetic testing results, with a specificity value of 60% (figure 1).

Figure 1

Receiver operating characteristic curve.

Discussion

The aim of this study was to develop a brief, easy-to-use psychosocial screening instrument specific to the genetic testing context and to examine its reliability and validity (Appendix A). To our knowledge, this is the first report of a psychosocial screening instrument for use across AOHD. Unlike current psychological instruments used mainly in research studies in genetics clinics to identify existing global symptoms of depression and anxiety, or impacts, the GPRI assesses psychological risk factors, such as the specific anticipated impacts of a genetic testing result and the perception of the disease. The GPRI demonstrates promising psychometric properties as a tool designed to assist genetics healthcare providers determine which of their patients undergoing genetic testing for AOHD is at increased psychological risk and should quite likely be considered for additional psychosocial support to facilitate adjustment to a test result.

A high reliability was demonstrated by a Cronbach’s α at 0.81, moderate to high item-total correlation and inter-item correlation of the whole scale. The construct validity of the scale was supported by high correlations between the GPRI and standardised psychological measures (BSI, IES). The clinical utility and predictive value of the GPRI was supported as well. A GPRI score above the cut-off of 50 at baseline was able to predict 84% of ‘distress’ cases identified by HAM-D or HAM-A, a strong indicator of its potential usefulness in a clinical setting.

A brief self-administered screening tool will be easy and quite likely highly acceptable for incorporation into genetics clinics. The GPRI can be completed and scored quickly during clinical visits and without additional burden to patients and health providers. In addition, by focusing specifically on known risk factors associated with inheritable illness, the instrument will be perceived as being more clinically relevant and acceptable to patients. Patients with higher GPRI scores can be flagged and either receive telephone follow-up to further assess concerns or potential distress or be invited back for an appointment for further assessment and required psychological treatment.

Alternatively, genetic clinics with available psychosocial personnel could utilise the tool to guide referrals for a formal psychosocial assessment that can further explore and address specific, self-reported psychological factors. For example, in the case where an individual is particularly fearful of developing an illness or is concerned about specific impacts, such as expecting relationship or family communications difficulties, information on communication strategies, personal coaching or family-based interventions could be employed to support the individual. For an individual who reports a history of psychological illness, a mental health professional could further assess current psychological functioning and implement specific approaches, as well as offering cognitive-behavioural strategies or psychotropic medication to assist in the management of anxiety or depressive symptoms.40 Several items incorporate variables related to heritable disease experiences and associated perceptions which can be used to guide educational interventions to correct any myths or beliefs.

The scale appeared highly acceptable to patients. A high face validity will contribute to better scale uptake being perceived as ‘user friendly’ and clinically relevant, compared, for example, with a standardised psychological instrument on depression, which has demonstrated some barriers to clinic uptake.19 The GPRI, by contrast, might be considered as a ‘communimetric measure’, that is, the items themselves are useful for the clinician in communicating concerns about specific areas of functioning directly with the patient.41

Left untreated, significant levels of psychological symptoms may lead to a lower quality of life40 and lower satisfaction with genetics services21 A psychological screening approach allows both for careful monitoring during a known stressful period—that of awaiting test results42—and provides an opportunity for any planned follow-up care. Flagging those individuals who might benefit most from psychosocial care also best utilises the often limited psychological resources in genetic clinics.2 20 21

Our study findings are limited by the characteristics of the sample, in that most participants were women and undergoing testing for BRCA1/2. This pattern is similar to that observed in the literature on genetic testing for AOHD, which is predominantly focused on the Hereditary Breast-Ovarian Cancer Syndrome. We attempted to obtain a larger sample of individuals undergoing genetic testing for HD or Lynch Syndrome, which would presumably provide a greater sample of males. However, these sample pools were much smaller. However, this study and the resulting GPRI represent an attempt to begin the development of a general tool that addresses concerns that are relevant across genetic samples. Our belief, stemming from clinical practice and the associated literature, suggests that the identified mental health issues or adjustment risk factors are not disease specific. We suggest that future studies further address the validity of GPRI in male populations and in the rare adult onset hereditary diseases, such as HD. Future studies should also include randomised controlled trials to assess the effectiveness of the GPRI in predicting distress, its impact on referral patterns, patient and provider satisfaction, as well as on cost-effectiveness. The GPRI could also be evaluated in primary care settings where genetics services might be offered more frequently to meet the demand.

Conclusions

This is the first study to develop a screening tool specifically to help identify individuals undergoing genetic testing for AOHD who are at increased psychological risk. The study resulted in an easy-to-use, 20-item scale consisting of three factors with promising psychometric properties. The GPRI has the potential to be used as a clinical screening tool and as a validated measure for future studies. Future work can examine its impact on clinical referral patterns within the field of genetics, and on its acceptability, reliability and validity with larger samples of individuals undergoing genetic testing for HD, Lynch Syndrome and potentially for emerging new genetic tests, such as for cardiac or psychiatric disorders.