Screening for distress and needs: Findings from a multinational validation of the Adolescent and Young Adult Psycho-Oncology Screening Tool with newly diagnosed patients.

OBJECTIVE: Adolescents and young adults (AYAs) diagnosed with cancer commonly experience elevated psychological distress and need appropriate detection and management of the psychosocial impact of their illness and treatment. This paper describes the multinational validation of the Distress Thermometer (DT) for AYAs recently diagnosed with cancer and the relationship between distress and patient concerns on the AYA-Needs Assessment (AYA-NA).
METHODS: AYA patients (N = 288; 15-29 years, Mage  = 21.5 years, SDage  = 3.8) from Australia (n = 111), Canada (n = 67), the UK (n = 85) and the USA (n = 25) completed the DT, AYA-NA, Hospital Anxiety Depression Scale (HADS) and demographic measures within 3 months of diagnosis. Using the HADS as a criterion, receiver operating characteristics analysis was used to determine the optimal cut-off score and meet the acceptable level of 0.70 for sensitivity and specificity. Correlations between the DT and HADS scores, prevalence of distress and AYA-NA scores were reported.
RESULTS: The DT correlated strongly with the HADS-Total, providing construct validity evidence (r = 0.65, p < 0.001). A score of 5 resulted in the best clinical screening cut-off on the DT (sensitivity = 82%, specificity = 75%, Youden Index = 0.57). Forty-two percent of AYAs scored at or above 5. 'Loss of meaning or purpose' was the AYA-NA item most likely to differentiate distressed AYAs.
CONCLUSIONS: The DT is a valid distress screening instrument for AYAs with cancer. The AYA-POST (DT and AYA-NA) provides clinicians with a critical tool to assess the psychosocial well-being of this group, allowing for the provision of personalised support and care responsive to individuals' specific needs and concerns.

BACKGROUND

Adolescent and young adult (AYA ) cancer patients have been recognised as a distinct group with specific needs and concerns since the mid‐1990s. Internationally, it is estimated there are over a million new cases of cancer amongst AYAs each year, and due to their relative high survival rates and long life expectancy, the burden of cancer for this age group is greater for both the individual and society than any other. As a consequence of this need, services have been established to provide holistic cancer care specifically to AYA patients. , , , ,

The considerable psychological impact of cancer on the diagnosed individual and their family and friends has been well documented. , , Thus, routine distress screening was recommended by the National Comprehensive Cancer Network (NCCN) in 2007, along with the development of a distress screening tool for adults with cancer; the NCCN Distress Thermometer (DT) and Problem Checklist (PCL). The DT is a single‐item, self‐report measure of distress with an 11‐point scale (0–10). The PCL is a checklist of potential causes of distress grouped within domains, and patients endorse their specific concerns.

AYAs have been found to have high levels of distress at diagnosis, during treatment and for many years post treatment. , In 2011, in response to an identified need for an age appropriate screening tool, the AYA Oncology Psychosocial Care Manual was developed, describing best practice in psychosocial screening, assessment and care planning for AYA cancer patients. No distress screening tool had been validated across the AYA age range and the NCCN DT and PCL was chosen as the preferred tool following modification for the AYA population. Palmer et al. with input from AYA patients and survivors, and AYA cancer healthcare professionals, adapted the PCL to be more developmentally appropriate for AYA. This adapted PCL was called the AYA Needs Assessment (AYA‐NA). Adaptations included both the addition and removal of items and domains. For example, in the family domain, items of ‘Mum and/or dad’ and ‘Brother(s) and/or Sister(s)’ were added. Items were also added to the emotional domain to reflect how AYAs often express distress (e.g., Boredom, Moodiness, Confusion) and in the physical domain ‘Hair loss’ was added, and ‘Indigestion’ was removed. Furthermore, two new domains (social and information) were added to account for the AYA experience, for example ‘Isolated from friends', ‘Missing important events’ and ‘Feeling involved in decision making’. This AYA distress screening tool (comprising the DT and AYA‐NA) became known as the Adolescent and Young Adult Psycho‐Oncology Screening Tool (AYA‐POST ) and was identified as a promising measure of distress in a 2014 review of psychosocial measures available for AYAs with cancer.

The NCCN DT and PCL version has been extensively validated with adult cancer populations. Further, Recklitis et al. examined the use of the DT with adults 18–40 year olds who had completed treatment (mean age: 29 years) and Chan et al. reported using the DT in Asian AYAs, finding a cut‐off of 4 was significantly associated with worry, depressed mood and nervousness. Chan et al. also reported testing the original adult PCL with AYAs, finding that relationships existed between endorsement of checklist items and distress. However, no investigation of the optimal cut‐off on the DT for recently diagnosed AYAs or of the appropriateness of the AYA‐NA has been undertaken to date.

A meta‐analysis by Ma et al. found that approximately three‐quarters of validation studies of the DT used the Hospital Anxiety and Depression Scale (HADS) as a criterion, and, concluded that a score of 4 or higher is the optimal screening cut‐off for further assessment. Of the 42 included studies in this analysis, 30 (71.4%) compare the DT to the HADS. Of these 30 studies, 13 (43.3%) identify 4 as being the optimal cut‐off score and 7 (23.3%) identify 5 as being the optimal cut‐off score.

A large study was developed to evaluate the clinical utility of the AYA‐POST, validate the DT, report the endorsement of the AYA‐NA and measure the prevalence and predictors of distress amongst AYAs. The protocol for this study has been reported previously. Given the significant international interest in the AYA‐POST, certain aspects of the larger study were completed collaboratively across countries with participants from the USA, Canada and UK. This paper reports on this multinational work. Given the variation in ages considered to be AYA internationally, the authors selected an age range of 15–29 years as it encompasses the whole age range typically used in the UK (15–24 years) and Australia (15–25 years), and much of the age range typically used in North America (15–39 years).

This paper reports on aspects of the validation of the AYA‐POST. The main aim of this paper is to report the optimal cut‐off score on the DT for AYAs aged 15–29 years that balances sensitivity and specificity, using the HADS as a criterion. As mentioned above, the HADS is the most common criterion in validations of the DT, and is validated with both adults and adolescents. , Secondarily, we estimate the prevalence of distress amongst AYAs, examine the relationship between distress scores and AYA‐NA responses, and describe the endorsement of items in the AYA‐NA including determining whether any items are redundant and the usefulness of items added to reflect the concerns of AYAs.

METHOD

Participants

A total of 288 AYA (aged 15–29 years) cancer patients within 3 months of diagnosis were recruited from 28 cancer centres in Australia, Canada, UK and the USA (M age = 21.5 years; SD age = 3.8; range: 15–29; 25%: 15–19 years, 64%: 20–25 years, 11%: 26–29 years). The demographics and characteristics of the sample are presented in Table 1.

TABLE 1

The demographics and cancer‐related characteristics of the AYA cancer patients from Australia, Canada, UK and USA

		Australia (n = 111)	Canada (n = 67)	UK (n = 85)	USA (n = 25)	Total (n = 288)
		M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Age at diagnosis		20.2 (3.1)	22.6 (4.2)	19.8 (2.6)	24.4 (3.0)	21.5 (3.8)

Abbreviation: NP, not provided.

Ethics

Ethics was obtained at the following lead sites within Australia: the Prince of Wales Hospital (HREC/14/POWH/261), Northern Territory Department of Health and Menzies School of Health Research (HREC‐2014‐2295), Children's Health Queensland Hospital and Health Service (HREC/14/QRCH/374), Women's and Children's Hospital (HREC/14/WCHN/113), Peter MacCallum Cancer Centre (14/178), Sir Charles Gairdner (2015‐048) and ACT Health (ETH.11.14.331) Human Research Ethics Committees. Within Canada, approval was obtained from the Research Ethics Boards at each participating institution including Princess Margaret Cancer Centre (PMCC) (Toronto, Ontario), McMaster Children's Hospital (Hamilton, Ontario), Alberta Children's Hospital (Edmonton, Alberta) and British Columbia (BC) Women's and Children's Hospital (Vancouver, BC). For the UK, the West of Scotland Research ethics service gave approval (14/WS/1009) and in the USA, the USC Health Sciences Institutional Review Board; Office for the Protection of Research Subjects; University of Southern California gave approval for protocol number HS‐15‐00651.

Measures

Adolescent & Young Adult Psychosocial Oncology Screening Tool

The AYA‐POST consists of the DT and the AYA‐NA:

The DT is a ‘thermometer’ presenting numbers vertically from 0 to 10. People rate their distress over the last week with 0 indicating ‘no distress’ and 10 indicating ‘high distress’. It is consistent with the original NCCN version.

The AYA‐NA is a list of specific concerns that could contribute to distress for AYA cancer patients and may provide healthcare professionals with specific biopsychosocial issues to address. The list was developed with experienced AYA clinicians and AYAs with a cancer experience, providing good content validity. Table 2 provides a complete list of items as presented in the tool, grouped into six domains: practical, family, emotional, physical, social and information.

TABLE 2

For participants from Australia, UK and the USA: The proportion endorsing at least 1 need per AYA‐NA domain, the average proportion of items identified as a problem per domain) and the percentage of participants, endorsing each AYA‐NA item

AYA‐NA domain	Proportion of AYAs endorsing 1+ need	Proportion of items endorsedM (SD)Range	AYA‐NA item (as presented in the tool)	% Yes
Practical (5 items) (N = 219)	67.6%	24.1% (23.2)0%–100%	Housing or living arrangements	14.6
			Education c	34.7
			Work or career c	28.8
			Transport or parking	13.7
			Bills or finances	28.8
Family (5 items) (N = 219)	49.8%	17.4% (21.7)0%–100%	Mum and/or dad c	32.4
			Brother(s) and/or sister(s) c	23.3
			Partner, boyfriend or girlfriend c	16.0
			Child (ren)	4.1
			Other family members c	11.4
Emotional (11 items) (N = 219)	78.5%	20.1% (19.2)0%–91%	Sadness	28.3
			Feeling alone or isolated c	16.4
			Anxiety or fear	33.3
			Guilt c	11.0
			Boredom c	46.1
			Anger or frustration c	25.1
			Extreme moodiness c	12.3
			Feeling hopeless or helpless c	16.4
			Feeling confused c	18.7
			Loss of meaning or purpose c	10.0
			Loss of faith or spirituality	3.2
Social (5 items) (N = 134 a )	69.4%	25.2% (22.3)0%–100%	Isolated from friends c	17.9
			Missing important events c	32.1
			Friends don't understand c	6.0
			Worry about boy/girlfriend c	21.1
			Missing doing the ‘normal stuff’ with friends c	49.3
Physical (19 items b ) (N = 219)	84.9%	19.8% (17.0)0%–89%	General appearance	26.9
			Hair loss c	38.4
			Breathing difficulty	12.3
			Fitness or sporting ability c	29.2
			Sleeping difficulty	33.3
			Constipation or diarrhoea	22.8
			Sexual concerns	10.0
			Loss of libido b , c	9.7
			Pain when having sex b , c	4.5
			Fertility c	19.6
			Eating or appetite	29.2
			Extreme exhaustion or tiredness c	28.8
			Memory or concentration	18.7
			Tingling in hands or feet	11.9
			Pain	21.9
			Nausea or vomiting	26.9
			High temperature or fever	11.4
			Use of alcohol and/or drugs c	4.6
			Other medical worry c	6.9
Information (5 items) (N = 219)	28.3%	8.3% (15.6)0%–100%	Understanding of information c	20.5
			Feeling involved in decision making c	6.8
			Feeling listened to c	7.8
			Rights to confidentiality c	1.8
			Rights to privacy c	4.6

Participants from the UK were not included in the social domain, as they used a different version of this subscale.

Participants from the UK completed the 17‐item, paediatric version of the AYA‐NA which excludes 2 items on sexual functioning in the physical domain.

Items that are new or modified (cf. the adult version).

Hospital Anxiety and Depression Scale

The HADS is a widely used 14‐item measure, to estimate anxiety, depression and general distress. It was originally validated for use with people aged 16–65 years and the most widely used HADS‐Total cut‐off score in DT validation studies is 15 indicating clinical levels of distress. Since then it has been validated in adolescents aged 12–17 years, identifying cut‐off scores of 9 and 7 to indicate clinical levels on the Anxiety and Depression subscales respectively.

Statistical analysis

Receiver operator characteristic (ROC) analysis was used to determine a suitable cut‐off for the DT Score. The aim was to maximise sensitivity (true positive rate) and specificity (true negative rate) to detect individuals with a HADS total score of 15 or higher; a minimum of 0.70 for sensitivity and specificity was considered to be indicative of a valid psychological screening tool. , Sensitivity is primary for a screening tool, as specificity can be compensated for in care provision following screening. Youden's Index was also calculated to integrate sensitivity and specificity. The area under the curve (AUC) was calculated as an additional index of test accuracy.

Analyses were run using three different criteria: a score of 7 on the HADS‐Depression ; a score of 9 on the HADS‐Anxiety ; and a score of 15 on the HADS‐Total.

Analyses on the AYA‐NA included data from all countries except Canada, who used a different approach in their checklist. Frequencies of endorsement for individual items and domains were described. Chi square tests were performed to identify whether there were significant differences in how distressed and non‐distressed participants endorsed individual items and domains. A Bonferroni correction for multiple analyses was applied to the chi‐square tests to maintain a familywise error rate of ≤0.05 (α = 0.05/50 = 0.001).

Procedure

Participants were eligible if they were within the AYA age range at their hospital, had been diagnosed with cancer within the last 3 months and were judged as able to complete the survey (e.g., had adequate English and were not in intensive palliative care). Most participants completed the survey whilst at the relevant hospital, however some were given the option of completing it at home. The survey was completed on paper everywhere except in Canada where it was completed online. A participant information sheet was provided, and informed consent obtained.

RESULTS

DT and HADS scores

The mean DT score was 3.93 (SD = 2.36), the mean HADS scores were 11.04 (SD = 6.61) for the HADS‐Total, 6.24 (SD = 4.20) for the HADS‐Anxiety and 4.80 (SD = 3.53) for the HADS‐Depression. The Pearson correlation between DT score and HADS‐Total scores was large and positive, (r = 0.65, p < 0.001), as were the correlations between DT score and HADS‐Anxiety score (r = 0.65, p < 0.001) and HADS‐Depression score (r = 0.48, p < 0.001).

Determining the DT cut‐off score

ROC analysis for HADS‐Total, HADS‐Anxiety and HADS‐Depression are depicted in Figure 1.

FIGURE 1

ROC curve for the HADS‐Total, HADS‐Anxiety and HADS‐Depression scores, using a DT cut‐off score of 5

Using the HADS‐Total criterion, optimal sensitivity (0.82) and specificity (0.75) were achieved for a DT score of 5 (Youden's J = 0.57, AUC = 0.84, 95% CI = 0.79–0.89). The DT score of 5 was also optimal for the HADS‐Anxiety subscale clinical cut‐off (sensitivity = 0.71, specificity = 0.81, Youden's J = 0.52, AUC = 0.81, 95% CI = 0.76–0.87; see Figure 1) and the HADS‐Depression subscale clinical cut‐off (sensitivity = 0.84, specificity = 0.68, Youden's J = 0.52, AUC = 0.81, 95% CI = 0.75–0.88; see Figure 1). Results for all DT scores are provided in supporting materials.

Rates of distress

Using the cut‐off of 5 on the DT, the overall rate of distress was 41.5%. There were higher rates of distress in females (53.8%) than males (31.2%; χ2 = 15.11, df = 1, p < 0.001), and in older than younger AYAs (20–29 years: 46.4%; 15–19 years: 29.7%; χ 2 = 5.673, df = 1, p = 0.017). Rates of distress differed between countries (χ 2 = 9.012, df = 3, p = 0.029). The proportion of participants with clinically elevated distress by DT score of 5 or more was 25.4% in Canada, 42.4% in the UK, 47.2% in Australia and 48.0% in the USA.

Almost every AYA (95.9% of participants) reported at least one concern on the AYA‐NA. Additionally, 41.1% reported 10 or more concerns: this included 49.5% of Australian AYAs (N = 109); 36.0% of USA participants (N = 25) and 31.8% of UK participants (N = 85). Endorsement of domains and individual items are in Table 2. Participants were most likely to endorse at least one physical concern (84.9%), emotional concern (78.5%), social concern (69.4%) or practical concern (67.6%). The most frequently endorsed items were ‘Missing doing the “normal stuff” with friends’ (49.3%), ‘Boredom’ (46.1%), ‘Hair loss’ (38.4%), ‘Education’ (34.7%), ‘Anxiety or fear’ (33.3%), ‘Sleeping difficulty’ (33.3%) and ‘Mum and/or dad’ (32.4%).

Associations between AYA‐NA items and distress

Distressed participants tended to endorse AYA‐NA items to a greater extent than non‐distressed participants, overall: 67% of distressed participants reported 10 or more concerns, compared to 20% of non‐distressed participants. Chi‐square analyses indicated that distressed participants endorsed 12 of the AYA‐NA items significantly more frequently than non‐distressed participants. These items were from the emotional, social and physical domains (see Table 3 for details).

TABLE 3

Prevalence of AYA‐NA item endorsement amongst distressed and non‐distressed AYAs

Domain	AYA‐NA item	Proportion of group endorsing item		Odds ratio a	χ 2
		Distressed	Non‐distressed
Emotional	Sadness	47%	13%	6.09	31.27
	Feeling alone or isolated	29%	6%	7.62	20.89
	Anxiety or fear	53%	16%	5.88	33.32
	Guilt	18%	4%	4.96	10.86
	Boredom	38%	14%	1.42	15.97
	Extreme moodiness	22%	4%	6.37	15.74
	Feeling confused	29%	9%	3.97	13.99
	Loss of meaning or purpose	19%	3%	8.99	16.16
Social	Isolation from friends	29%	7%	5.40	11.307
Physical	Sleeping difficulty	48%	21%	3.43	17.473
	Extreme exhaustion or tiredness	42%	17%	3.55	16.690
	Memory or concentration	28%	10%	3.44	11.486

Note: Only items with a significant (p < 0.001) χ 2 tests are shown.

Odds ratio shows the odds that a patient will be classified as distressed on the DT if they endorse that AYA‐NA item.

DISCUSSION

This paper provides the first validation of the DT as a distress screening tool with AYAs recently diagnosed with cancer, supporting its use with this cohort. Since its development, requests to use the AYA‐POST have come from 20 countries in five continents demonstrating the need for a specific distress screening tool for this age group. This multinational study supports clinicians and researchers using the AYA‐POST, with a distinct age‐appropriate cut‐off on the DT to identify AYAs for further assessment. Additionally, our study presents striking information on the high distress rates and many concerns of AYAs with cancer while also observing some variation between participants from different countries.

We identified firm evidence for the DT's convergent validity by correlation with all HADs scales, and AYAs who were identified as distressed reported more concerns on the AYA‐NA. The ROC analyses consistently indicated that a DT cut‐off score of 5 and above provided acceptable specificity and sensitivity scores to constitute a positive screen and undertake further detailed assessment. As previously mentioned, a meta‐analysis showed seven (23.3%) DT validation studies with adult populations identified an optimal cut‐off of 5. Focusing on young adults, Recklitis et al. examined the validity of the DT using a standardised clinical interview as a reference criterion. They found that a DT cut‐off of 5 had a sensitivity of 68.2% and a specificity of 78.3% to detect a psychiatric diagnosis, though it should be noted that their sample comprised cancer survivors who were at least 2 years off treatment with an age range of 18–40 years.

Using the DT cut‐off of 5, 41.5% of AYAs in our study reported elevated levels of distress. We found that older AYAs were more likely to reach criterion for distress than younger AYAs, as did Chan et al. As has been reported in adults, more females than males met the criterion for distress in this study. The mean score of 3.93 on the DT in this study is consistent with Chan et al. in 24–39 year olds (3.7–3.8), but higher than they reported for 15–24 year olds (2.7–2.8).

Rates of distress varied between countries, with Canadian participants reporting lower levels of distress. While it is possible that there are baseline differences in distress between the countries, previous research has found that rates of anxiety and depression amongst adolescents in Western countries is similar. Exploratory secondary analysis (not reported) suggests that the differences found here may be partially explained by demographic differences between the samples such as the Canadian sample having a lower proportion of females and a higher proportion living with their parents.

Many concerns were reported by the study participants and all items were endorsed by at least some participants. Given this, no AYA‐NA items were considered redundant. Over 40% of AYAs endorsed at least 10 of the items, with 2 in every 3 who were distressed reporting 10 concerns or more. Across all the study participants, 96% endorsed at least 1 item. We observed support for the modifications made to the PCL in the development of the AYA‐NA, as the most endorsed items (both overall and by distressed participants) were not present in the PCL. The most endorsed items included those added to the AYA‐NA: ‘Missing doing the “normal stuff” with friends’, ‘Boredom’ and ‘Hair loss’. As these are not in the PCL, it is unknown how many adults would endorse them, however it is noteworthy that they were items that AYAs and clinicians recommended adding when modifying the adult checklist. The social domain was added for AYAs and highly endorsed, adults are more likely to endorse items from the financial domain, whereas ‘Bills and finance’ had low endorsement from AYAs. ‘Loss of meaning or purpose’ and ‘Feeling alone or isolated’ were the two items most likely to differentiate distressed from non‐distressed AYAs, reflecting perhaps the extraordinary life disruption a cancer diagnosis brings, including the loss of connection with friends and lack of opportunity for connection with others going through a similar experience.

Limitations

Whilst the multinational element of this study may minimise biases of location and context, the countries included are all Western and English speaking so there are limitations in the generalisability of the findings. Further validation of this tool in language—appropriate formats should be sought in non‐English speaking countries and cultures. Conducting a multinational study brought its own challenges such as differences in demographic variables and how the AYA‐NA was administered, which resulted in data being excluded from some analyses. We were also unable to conduct a sub‐group analysis to compare cut‐offs between adolescents and young adults due to insufficient numbers of participants who were under eighteen.

While the HADS is widely accepted as a screening tool for depression and anxiety in medical settings and has been the most commonly used reference criterion for validating the DT, its suitability for use with cancer patients and adolescents has been questioned. ,

Finally, the study focused on AYAs within 3 months of diagnosis and so we cannot comment on distress across the cancer trajectory. This homogeneity is also a strength of the study as a meta‐analysis in adults reported variation in DT cut‐off scores at different points in the cancer trajectory.

Clinical implications

Our findings indicate that a higher cut‐off of 5 (cf. the standard cut‐off of 4) is more appropriate with this age group, necessitating a small and significant change in practice for institutions currently using the adult cut‐off of 4 with AYAs. A cut‐off of 4 would result in specificity falling below 0.7, which is unacceptably low. , A cut‐off of 5, which has a higher Youden's Index, will reduce the number of false positives with relatively little impact on the test's sensitivity (sensitivity = 0.82 for a cut‐off of 5, and 0.90 for a cut‐off of 4, based on the HADS Total score).

This study provides confidence in the validity and clinical usefulness of both elements of the AYA‐POST as a sufficiently accurate, brief and broad psychosocial screening tool for AYAs recently diagnosed cancer. The validation data supports its on‐going use in practice, to support individual care, guide appropriate resource allocation within the health sector and facilitate epidemiological population‐based research in future.

Understanding which items on the AYA‐NA are strongly endorsed generally, and for those who are identified as distressed, is clinically helpful in identifying which services to resource, the appropriate mix of healthcare professional skills and timely intervention. For example, ensuring that AYAs stay connected with others during their treatment and are supported in renegotiating their life purpose, is essential. Likewise, older AYAs and females are more likely to indicate distress and may require additional, tailored support.

CONCLUSIONS

This study establishes a clinically appropriate cut‐off score of 5 on the DT for recently diagnosed AYAs, characterises the levels of distress and identifies the concerns of these AYAs, in a multi‐institutional, multinational study in primarily English‐speaking countries. The availability of a validated tool for use with this vulnerable population will enable health professionals to confidently screen for distress and unmet needs, thus leading to better support provision and guidance on service development for AYAs diagnosed with cancer.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

Supplementary Material

Click here for additional data file.