Validation of the multidimensional impact of Cancer Risk Assessment Questionnaire to assess impact of waiting for genome sequencing results.

OBJECTIVE: To determine whether the existing Multidimensional Impact of Cancer Risk Assessment (MICRA) scale, which assesses impact of receiving genetic test results on individuals being assessed for cancer risk, can be successfully adapted to cancer patients experiencing prolonged waiting for results of germline genome sequencing (GS).
METHODS: Patients previously diagnosed with likely hereditary cancer (n = 250) who were waiting for germline GS results completed questionnaires 3 months after baseline. We adapted the MICRA to measure anxiety associated with waiting for results, and assessed factor structure, internal consistency, test-retest reliability and construct validation.
RESULTS: Factor analysis revealed four factors: distress, positive experience, family support and uncertainty. Internal consistency for each sub-scale was high with the values of Cronbach's alpha for the distress, positive experiences, family support and uncertainty sub-scales 0.92, 0.88, 0.92 and 0.87, respectively. Test-retest reliability was poor, with intra-class correlations of 0.53, 0.13, 0.33 and 0.52 for the four factors, respectively. Construct validation showed large correlations between the MICRA distress and uncertainty sub-scale scores and the Impact of Events score intrusion (0.42 and 0.62, respectively) and IES avoidant thinking sub-scales (0.40 and 0.58, respectively) but not the Hospital Anxiety and Depression Scale sub-scales.
CONCLUSIONS: The adapted MICRA identified test-related anxiety and uncertainty in a population of cancer patients waiting for germline GS results. Results suggest that the distress and uncertainty sub-scales of the adapted measure are most useful in this context.

BACKGROUND

Human germline genome sequencing (GS) promises to benefit public health through improving the specificity of population surveillance for cancer risk. Individual testing allows for personalisation of risk assessment and surveillance protocols. Stratification of surveillance may create opportunities to achieve high rates of detection and effective early treatment, while advising those identified at average risk to undergo surveillance less frequently or not at all. However, these opportunities will only be realised if patients cope with and act on their test results.

GS differs from most standard medical tests in that it generates an unprecedented volume of data to process, which may have implications for blood relatives. GS for cancer risk may result in information that is (a) relevant to indication for test, (b) relevant to other cancers or diseases (secondary) or (c) of unknown significance. Secondary findings confront both patients and families with risks they had not been seeking to identify nor were prepared to face. The high incidence of findings of uncertain significance, whose meaning may or may not become clearer over time, can further increase uncertainty, and be confusing and worrying for patients. ,

Han and colleagues have identified three triggers of uncertainty in health care: indeterminate outcomes, imprecise risk estimates, and complexity in results. These are all true of genomic output. Previous studies have shown that, following receipt of genomic information in the research setting, only around one‐third of healthy individuals appeared to make positive lifestyle changes. , There is evidence that patients low in tolerance for uncertainty may have lower intention to even receive some GS results.

In view of the potential benefits of receiving and acting on GS results, understanding how patients experience the testing procedure is needed to support cancer patients undergoing GS. Findings on psychological and behavioural impacts of genetic testing vary widely , , , and identification of patients who find GS distressing is critical given that this is a potentially modifiable factor , as opposed to demographic or disease‐related variables.

Anxiety and uncertainty after GS may be exacerbated by the length of time between sample collection and provision of results, which is longer than routine blood assessments. At present, GS is not commonly used in the clinical oncology context, but turnaround for testing is approximately 18–24 months in a research context. This prolonged wait may be a trigger for increased patient discomfort, frustration, uncertainty and ongoing worry. This response has previously been identified in qualitative studies. None of the few previous quantitative studies investigating psychosocial outcomes while waiting for GS results have identified this response, perhaps because the standard measures used in these studies, such as the 15‐item Impact of Events Scale (IES), were aimed at detecting post‐traumatic stress symptoms. Such symptoms may be more intense than those typically experienced after GS and tend to be reported only in patients with existing psychological vulnerabilities.

Most psychosocial genomic studies currently underway internationally are using the 25‐item Multidimensional Impact of Cancer Risk Assessment (MICRA) to assess impact of result disclosure after genetic testing including measuring distress and uncertainty. The MICRA was designed to identify sub‐clinical symptoms experienced by people specifically in response to receiving genetic test results. It has robust psychometric characteristics and has been shown to discriminate between subgroups of vulnerable genetic testing participants. We therefore considered it a possible candidate to measure the psychological impact reported in qualitative interviews.

The Psychosocial Issues in Genomic Oncology (PIGeOn) sub‐study followed 1000 probands with a cancer of likely hereditary origin for 12 months after GS (prior to receipt of results), to determine the impact of waiting for results. We adapted the MICRA to measure distress associated with waiting for results. We hypothesized that the MICRA would sensitively capture participants' psychological responses and enable us to assess how populations tolerate the ongoing distress and uncertainty involved in this type of testing. Because the MICRA has not yet been administered in the period before test results are given, we conducted psychometric testing to ensure the validity of our adaptation in this setting. Here we report on our validation of the adapted MICRA. Our primary objective was to finalize its scale structure via psychometric testing. Secondary objectives were to assess its reliability, validity, acceptability and interpretability in patients with a previous diagnosis of cancer of likely genetic origin waiting for GS results.

METHODS

Participants

Participants were recruited to the Genetic Cancer Risk in the Young study (RisC) which is being conducted at the Garvan Institute of Medical Research in Sydney, Australia and aims to investigate heritable causes of cancer. The target population consists of adults with histologically confirmed malignancy under 40 years at diagnosis; or having >1 primary cancer diagnosed <50 years; or having >2 primary cancers at any age. All participants undergo GS and are offered the choice to be informed if they have a pathogenic variant that increases cancer risk, and/or secondary findings that may be important to their health. RisC participants who receive an actionable result following GS are offered genetic counselling and tailored risk management in a subsequent study.

The PIGeOn sub‐study aims to examine the psychosocial, behavioural and ethical impact of GS. Patients consent to PiGeOn when they consent to RisC. Both studies were approved by the St Vincent's Hospital Human Research Ethics Committee (HREC/16/SVH/24).

Data collection

PiGeOn participants completed a questionnaire including the adapted MICRA measure 3 months after testing. Consecutive participants who completed and returned the questionnaire were asked to complete the adapted MICRA again, about 2 weeks later, to assess test–retest reliability. Validity was established through comparison with other measures completed in the first questionnaire.

Measures

. The original MICRA was developed to assess psychological responses after receipt of genetic results and is comprised of three sub‐scales: distress, positive experiences and uncertainty. High scores indicate greater distress, positive thinking or uncertainty, respectively. The original 25 MICRA items were reviewed by a multi‐disciplinary team. As the purpose of the modified MICRA was to assess psychological responses while waiting for results, items relating to behaviour which could occur only after receiving results were excluded. Items excluded from the original MICRA were: ‘Feeling guilty about my test result’ (item 4), ‘Understanding clearly my choices for cancer risk reduction’ (item 13) and ‘Feeling regret about getting my test results’ (item 21). Other items that related to result receipt were reworded to ensure focus on waiting for results. For example, item 2 was reworded from ‘Feeling sad about my test result’ to ‘Feeling sad about not receiving my test result’.

The is a 14‐item questionnaire which comprises two 7‐item sub‐scales measuring anxiety and depression. Higher scores reflect greater morbidity.

The is a 15‐item questionnaire assessing the experience of intrusive and avoidant thinking related to a specified stressful event. Here the stressful event is the ongoing uncertainty associated with waiting for genomic test results. The IES was used to assess construct validity in the original MICRA validation study. Higher scores indicate greater cancer‐related anxiety.

Sample size and statistical analysis

The proposed total of 250 participants was based on the ‘rule of thumb’ recommendation for sample size in psychometric analyses. This amounts to 5–10 subjects per item, to reduce the effect of chance. , As 22 items were included in the amended scale, at least 220 patients were required to ensure that all psychometric and statistical criteria would be met. For test–retest reliability assessment, a sample size of 120 was used to achieve 95% power to detect a Pearson correlation of 0.30 or greater at the 0.05 significance threshold. All analyses were conducted in the statistical software R v3.6.1.

As this was a revision of the original questionnaire, we conducted exploratory factor analysis using the statistical software R, estimating factor loadings using weight least squares to account for the ordinal nature of the item responses. We used a promax rotation to allow factors to be correlated. We first examined the Kaiser–Myer–Olkin (KMO >0.8) and Bartlett's test (p < 0.01) to determine if the data were appropriate for exploratory factor analysis. The number of factors were determined using the scree plot, parallel analysis (using the psych package) and the minimum averaged partial method (using the nFactors package).

of each of the MICRA sub‐scales was determined by calculating Cronbach's alphas for summary scores.

was established for the subset of 120 participants who completed the MICRA on two occasions, via a Pearson correlation and 95% confidence intervals.

was assessed by comparing the relationship between the adapted MICRA and the HADS and IES. We computed sub‐scales score as the sum of responses for each sub‐scale in the adapted MICRA, and calculated the Pearson correlation between these scores and four other scores: IES intrusion, IES avoidance, HADS anxiety and HADS depression. The correlations were assessed using Cohen's guidelines. We hypothesised that the correlations between the MICRA sub‐scale scores and the IES would be large, as both measures are specific to the testing context. We expected correlations with the HADS would be moderate since general anxiety and depression is likely to be impacted less by test‐specific issues.

was assessed through missing data analysis and user feedback. Data is presented as descriptive statistics.

RESULTS

Sample

A total of 252 participants completed the adapted MICRA within the T1 questionnaire, with 120 participants completing the test–retest (100% response rate). Average age of participants was 43.7 years (range 20–80 years) and 69% were female (173/252). All had a previous diagnosis of cancer, 30% (75/252) had more than one primary cancer previously diagnosed and 71% (179/252) had a diagnosis of rare cancer. Participant demographics are summarised in Table 1.

TABLE 1

Demographic characteristics

	Total sample (N = 252)	Completed test–retest (N = 75)
	N, %	N, %
Demographics
Sex
Female	173 (69)	50 (67)
Male	79 (31)	25 (33)
Age (years)
Mean (SD)	43.7 (14.0)	46.0 (13.5)
Median (IQR)	40 (19)	43 (21)
Range	20–80	22–77
Education
High school or less	44 (17)	14 (19)
Vocational training	39 (15)	15 (20)
University	168 (67)	46 (61)
Missing	1 (0.4)	0
Medical/science occupation
Yes	21 (8.3)	6 (8.0)
Non‐English‐speaking background
Yes	43 (17)	8 (11)
Socio‐economic status
Mean (SD)	7 (2.6)	7 (2.6)
Range	1–10	2–10
Accessibility and Remoteness Index of Australia
Urban	233 (93)	71 95)
Marital status
Married	150 (60)	52 (69)
Biological children
Yes	154 (61)	49 (65)
Cancer history
Visited a family cancer clinic
Yes	70 (28)	23 (31)
Family history of cancer
Yes	210 (83)	65 (87)
Multiple primary cancers
Yes	75 (30)	27 (36)
Time since cancer diagnosis (months)
Mean (SD)	8.9 (10)	9.9 (11)
Median	4.9	5.2
Range	0–52	0.3–52
Cancer incidence
Common (>12 incidences/100,000 population)	61 (24)	14 (19)
Less common (6–12 incidences/100,000 population)	12 (4.8)	2 (2.7)
Rare (<6 incidences/100,000 population)	179 (71)	59 (79)

Factor structure

The KMO (0.83) and Bartlett's (p < 0.01) scores both indicated suitability of the data for exploratory factor analysis. The scree plot, parallel analysis and minimum averaged partial methods all supported the extraction of four factors. Factor loadings are shown in Table 2. Two items had weak loadings (<0.4), and the remaining items formed the following factors : Distress sub‐scale, comprising items 1–3 (three items), three of the five items from the original MICRA distress sub‐scale ; Positive experiences sub‐scale, comprising items 4 and 5 (two items), two of the four items from the original MICRA positive experiences sub‐scale; and an additional item from the original uncertainty sub‐scale which also loaded on this factor. The magnitude of loading (0.43) on this last item was lower than the other two items' loadings (−0.67 and −0.69) ; Uncertainty sub‐scale, comprising items 6–13 (eight items) broadly corresponding to the original uncertainty sub‐scale, but including two items previously found to load with distress ; Family support sub‐scale, comprising item 16 and 17 (two items), the other two items representing the original positive experiences sub‐scale.

TABLE 2

Loadings from exploratory factor analysis

	1	2	3	4
Feeling upset about not receiving my whole genome sequencing test result	0.96	0.01	−0.03	−0.02
Feeling sad about not receiving my test result	0.97	−0.02	−0.02	−0.04
Feeling anxious or nervous about not receiving my test result	0.68	0.07	0.01	0.2
Feeling relieved about not receiving my test result	−0.02	−0.67	0.08	0.02
Feeling happy about not receiving my test result	0.01	−0.69	0.07	0.02
Feeling a loss of control	0.16	0.14	0.06	0.44
Having problems enjoying my life while waiting for my test result	0.17	0.18	0.19	0.52
Worrying about my risk of cancer developing	−0.08	−0.07	−0.02	0.78
Being uncertain about what my test result will mean about my cancer risk	−0.01	−0.01	−0.1	0.76
Being uncertain about what my test result will mean for my child(ren) and/or family's cancer risk	−0.06	0.01	−0.13	0.69
Having difficulty making decisions about my cancer risk	0.05	0.12	0.01	0.68
Feeling frustrated that there are no definite cancer risk reduction options for me yet	0.21	−0.13	−0.07	0.58
Thinking about not receiving my test result has affected my work or family life	0.24	0.24	0.16	0.46
Feeling concerned about how my test result will affect my insurance status	−0.12	0.16	−0.05	0.2
Having difficulty talking about my cancer risk with family members	−0.07	0.29	0.07	0.37
Feeling that my family has been supportive during the whole genome sequencing process	−0.03	−0.03	0.94	−0.01
Feeling satisfied with family communication while waiting for my test result	0.00	−0.01	0.95	−0.03
Worrying that the whole genome sequencing process has brought about conflict within my family	−0.01	0.43	0.07	0.1

Note: Loadings >0.4 in magnitude are in bold text.

Internal consistency

The values of Cronbach's alpha for the distress, positive experiences, family support and uncertainty sub‐scales were, respectively, 0.92, 0.88, 0.92 and 0.87, respectively, which are all high.

Test–retest reliability

The sub‐scale means are shown in Table 3, as are the intraclass correlations, which all indicate poor test–retest reliability.

TABLE 3

Sub‐scale score means, internal consistency and test‐retest reliability

	Mean	SD	Cronbach alpha	ICC (95% CI)
Distress	2.13	3.57	0.92	0.53 (0.25, 0.86)
Positive experiences	8.66	2.43	0.88	0.13 (−0.03, 0.62)
Family support	2.98	3.76	0.92	0.33 (0.12, 0.72)
Uncertainty	8.59	7.99	0.87	0.52 (0.29, 0.73)

Construct validation

Correlations between the MICRA sub‐scale scores (calculated as the sum of items for each factor identified in exploratory factor analysis) and the four comparison instruments are shown in Table 4. Using Cohen's guidelines, the correlations between the distress sub‐scale and the IES sub‐scales were moderate–large, and small–moderate for the HADS sub‐scales. We observed a similar pattern for the positive experiences sub‐scale. The family support sub‐scale weakly correlated with all four HADS and IES sub‐scales. The uncertainty sub‐scale was strongly correlated with all four HADS and IES sub‐scales.

TABLE 4

Correlations between MICRA sub‐scales and comparison instrument scores

	HADS anxiety	HADS depression	IES intrusion	IES avoidance
Distress	0.19	0.20	0.42	0.40
Positive experiences	−0.10	−0.14	−0.34	−0.41
Family support	0.02	0.03	−0.12	−0.14
Uncertainty	0.48	0.45	0.62	0.58

Abbreviations: HADS, Hospital Anxiety and Depression Scale; IES, Impact of Events score; MICRA, Multidimensional Impact of Cancer Risk Assessment.

Acceptability and Interpretability

Completion rates for all questions were 100% at both Time 1 and Time 2. Participant feedback confirmed that questions were perceived as easy and not distressing to answer.

DISCUSSION

The MICRA is a brief questionnaire that assesses concerns that are specific to high‐risk cancer patients receiving genetic test results. We adapted the MICRA to assess the impact of prolonged waiting for GS results in a cancer population. Factor analysis of the adapted MICRA found four factors: Distress (questions 1–3), Positive experience (questions 4–5), Uncertainty (questions 6–13) and Family support (questions 16 and 17), resulting in a 15‐item measure of distress experienced during prolonged waiting for GS results.

We found that the adapted MICRA is acceptable to the majority of participants and easy to complete.

The adapted MICRA showed poor test–retest reliability. One explanation, particularly in the context of the positive experiences subscale, is that the standard deviation for positive experiences is small with respect to the mean. This indicates that members of this cohort had similar levels of ‘positive experience’ to each other. If all participants have a similar level of ‘positive experience’ it is difficult to get a good measure of test–retest reliability as there is insufficient diversity of responses to demonstrate that those with low positive experience stayed low, and those with high positive experience stayed high. This is also shown in the confidence intervals with upper bounds that are quite high. The same may be true for the other measures, but is strongest in positive experiences (which correspondingly has the lowest ICC value). A more diverse cohort is needed to accurately establish test–rest reliability.

The wording of some questions in the adapted MICRA could also have contributed to the poor test–retest reliability. ‘Feeling relieved’ and ‘feeling happy’ about not receiving results yet, could be conceptually challenging for participants and not a likely common reaction during this waiting period. However, some participants who expect a positive result and view this as a negative outcome (conferring more risk on themselves and their family), may prefer to defer knowledge, and find not yet knowing the result yet, a relief. Thus further work may be needed before a decision is made whether to retain or omit these items from the measure in future use.

It is also possible that the initial MICRA completion, which was presented in the context of a long routine questionnaire, was not done carefully, and that the retest version, presented in isolation 2 weeks later, was completed more mindfully. A meta‐analysis, which identified 20 papers exploring response rates in shorter versus longer questionnaires, found an association between response rate and questionnaire length (p ≤ 0.0001); response rates were lower for longer questionnaires, although the authors noted this association should be interpreted with caution because of the difficulty separating the impact of content from length of questionnaires. Further work is needed to establish test–retest reliability of the adapted MICRA in this population.

With regards to construct validation, the adapted MICRA distress and positive experience sub‐scales correlated moderately (positive experiences) to highly (distress) with the IES, although correlations were only small to moderate with the HADS sub‐scales. This was to be expected, since MICRA aims to assess test‐specific rather than general anxiety and depression; these results support the specificity of the questionnaire.

The uncertainty sub‐scale had the strongest correlations with both IES and HADS sub‐scales. Since our sample were still awaiting their results, it makes sense that uncertainty, rather than distress, was the most prominent issue for this group. Indeed, it has been noted that clinical levels of distress may be rare in this population. Qualitative findings from this cohort revealed that many participants were not focussing on the test, and many had forgotten whether or which results they had requested.

The family support sub‐scale had weak correlation with both IES and HADS. Previous research has suggested that cancer patients undergoing GS may defer consideration of family implications for GS results until a positive result is received, which may explain these weak correlations. It is also possible that, since this cohort had already decided to participate in GS at the time of test completion, that consultation with family regarding whether to undergo testing was no longer relevant.

The correlations detected between the MICRA distress and uncertainty sub‐scale scores and the IES sub‐scales indicates that the adapted MICRA detected a specific test‐related anxiety in this cohort of cancer‐affected participants. We therefore propose that the family and positive feelings sub‐scales, as less highly correlated and less relevant for the experience of waiting for results, should be omitted from this adapted form in future. The adapted MICRA would therefore have 11 items measuring distress and uncertainty, suitable for administration to populations awaiting GS results. Such a questionnaire could be used to identify patients struggling to cope with prolonged waiting, who may need additional support and intervention.

Study limitations

Limitations of this study include the fact that the sample were participating in a research study and may have been less invested in GS than those proactively seeking knowledge of cancer risk through GS in a clinical setting. Test‐retest reliability in this sample was poor, and further work is needed to establish whether this would be improved when assessed within a shorter questionnaire battery. Validation was not assessed in terms of known‐group comparisons or responsiveness to change over time, and other scales may be useful in assessing construct validity.

Clinical implications

This study found that the adapted MICRA detected a specific test‐related anxiety in this cohort of cancer‐affected participants. This is the first validated measure identified to capture this data. Use of the 11‐item adapted MICRA will enable identification of patients undergoing GS who require psychological support while awaiting results. We believe that by screening cancer patients waiting for results of germline GS for cancer risk assessment, those with raised scores for distress and uncertainty would be identified early, allowing early intervention to prevent escalation of morbidity to clinical levels.

CONCLUSIONS

The adapted MICRA may be useful to assess test‐related impact of waiting for GS results in a population of cancer patients. Results suggest that the distress and uncertainty sub‐scales of the adapted measure are most useful in this context, resulting in an 11‐item scale. Use of this scale could identify which patients need increased support while waiting for the results of GS in a clinical context.

CONFLICT OF INTEREST

The authors have no known conflict of interest to disclose.