Clinical assessment of emotions in patients with cancer: Diagnostic accuracy compared with two reference standards.

BACKGROUND: Previous research has suggested that clinical assessment of emotions in patients with cancer is suboptimal. However, it is a possibility that well-trained and experienced doctors and nurses do recognize emotions but that they do not evaluate all emotions as necessitating professional mental health care. This implies that the sensitivity of clinical assessment should be tested against the need for professional mental health care as reference standard, instead of emotional distress. We hypothesized that the observed sensitivity of clinical assessment of emotions would be higher when tested against need for professional mental health care as reference standard, compared with emotional distress as reference standard. PATIENTS AND METHODS: A consecutive series of patients starting with chemotherapy were recruited during their routine clinical care, at a department of medical oncology. Clinical assessment of emotions by medical oncologists and nurses was derived from the patient file. Emotional distress and need for professional mental health care were assessed using the Distress Thermometer and Problem List.
RESULTS: Clinical assessment resulted in notes on emotions in 42.2% of the patient files with 36.2% of patients experiencing emotional distress and 10.8% expressing a need for professional mental health care (N = 185). As expected, the sensitivity of clinical assessment of emotions was higher with the reference standard "need for professional mental health care" compared with "emotional distress" (P < .001). For specificity, equivalent results were obtained with the two reference standards (P = .63).
CONCLUSIONS: Clinical assessment of emotions in patients with cancer may be more accurate than previously concluded.

INTRODUCTION

Previous research has suggested that clinical assessment of emotions in patients with cancer by doctors and nurses is suboptimal. Empirical studies repeatedly reported a rather low sensitivity—a substantial risk of thinking that patients do not have emotional problems while they in fact do experience problems—and a rather high specificity—a small risk of identifying patients as experiencing emotional problems when in fact they are not.1, 2, 3 These findings suggest that doctors and nurses may not be very sensitive to the psychological state of patients with cancer. However, it is a possibility that well‐trained and experienced doctors and nurses do recognize emotions, but that they do not evaluate all emotions as necessitating professional mental health care. This would imply that the sensitivity of clinical assessment should be tested against the need for professional mental health care as reference standard, instead of emotional distress.

In the face of a life‐threatening disease and a burdensome treatment, emotions are natural and potentially even adaptive.4, 5, 6, 7 For example, fear causes cognitive shifts, prioritizing efforts to cope with the threatening event; physiological changes associated with fear prepare and support these behavioral responses.8 Sadness turns our attention inwards, promoting resignation and acceptance; the expression of sadness may elicit sympathy and support from other people.9 This implies that emotion per se does not necessitate mental health intervention. Indeed, patients with cancer have been frequently observed to decline professional mental health care, although they did experience strong cancer‐related emotions.10, 11, 12, 13, 14, 15 Patients may not be aware of all aspects of professional mental health care, or they may not be willing to participate in a professional mental health care program. However, patients emphasized that they want to deal with emotions themselves or with support from relatives, friends, doctors, and nurses.16, 17, 18, 19 Only when patients experience emotions that they cannot cope with or that interfere with daily functioning may professional intervention be required.20, 21 Thus, it has been suggested that a distinction needs to be made between adaptive and maladaptive emotions; only maladaptive emotions necessitate professional mental health care.7

Research on clinical assessment of emotions by doctors and nurses has relied on instruments such as the Distress Thermometer as a reference standard against which the detection of emotional problems was tested.2, 22, 23 These instruments generate a score of the intensity of emotional distress; patients scoring above a specific cut off score are deemed to be in need of professional mental health care. These instruments do not differentiate between adaptive and maladaptive emotions, with only maladaptive emotions leading to the need for professional mental health care. This may result in an underestimation of the diagnostic accuracy of clinical assessment of emotions, as the distinction between no need (in case of adaptive emotions) and need for professional mental health care (in case of maladaptive emotions) is not taken into account. We hypothesized that the observed sensitivity of clinical assessment of emotions would be higher when tested against need for professional mental health care as reference standard, compared with emotional distress as reference standard. Further, we hypothesized that specificity is not dependent on the reference standard: We expected oncologists and nurses to prioritize efforts to include those patients in need of professional mental health care over excluding patients not in need. The aim of this study was to evaluate the diagnostic accuracy of clinical assessment of emotions by medical oncologists and nurses in patients with cancer, compared with two reference standards: (a) emotional distress as experienced by patients and (b) need for professional mental health care as expressed by patients.

METHODS

Design

This was a retrospective study at a department of medical oncology. The clinical assessment of emotions was derived from the patient file: The file was screened for notes by medical oncologists or nurses on emotions and associated referrals. As a reference standard to define cases, we used (a) emotional distress as assessed with the Distress Thermometer and Problem List, or (b) need for professional mental health care, as expressed by the patient on the Problem List. Sensitivity, specificity, and the diagnostic odds ratio (DOR) of clinical assessment were calculated, with (a) emotional distress as reference standard and with (b) need for professional mental health care as reference standard. DOR is a measure of the effectiveness of a diagnostic test. It is defined as the ratio of the odds of the test being positive if the subject has a disease relative to the odds of the test being positive if the subject does not have the disease.

Patients and procedure

A consecutive series of patients were recruited at the Department of Medical Oncology at the Amsterdam University Medical Centers (location VUmc), Amsterdam, the Netherlands. In this tertiary referral hospital, professional health care, including psycho‐oncology and supportive oncology services, is available at the site. The psychologists, social workers, and psychiatrists are specialized in working with patients with cancer. At the time of the study, there was no psychological screening policy in place. The following inclusion criteria were applied: patients with cancer, starting with systemic antitumor treatment (chemotherapy), life expectancy of at least 3 months established by doctors, and age between 18 and 85 years. Exclusion criteria were second opinion, head‐neck cancer (due to interference with other research), participation in the TES‐trial24 (due to overlap of the studies), insufficient command of the Dutch language, receiving other systemic anticancer treatment, or no informed consent. During treatment with systemic chemotherapy, patients were seen by their doctor approximately once every 3 weeks. Three months after starting treatment, patients were asked to participate by telephone. If a patient agreed, questionnaires were sent by mail, and their file was examined. The study protocol was approved by the METC of the Amsterdam UMC, location VUmc (2015.072/NTR5208), and all patients provided informed consent.

Measures

Clinical assessment of emotions was rated positive if there was a note in the patient file describing fear, anxiety, stress, low mood or depression, or a (possible) referral for psychosocial or mental health care because of these symptoms. Using a structured form, the patient file was fully screened by one trained psychologist for notes or a referral related to emotional symptoms, covering the 3 months after the patient started treatment.24 At the time of the study, on average 13 medical doctors and 15 nurses were employed at the Department of Medical Oncology. As part of their normal work routine, they kept record on treatment progress in the patient file. This could involve notes on the patient's psychological status or a referral for psychosocial or mental health care. No instructions on the recording of psychological status or referrals were given to doctors or nurses.

We used the Distress Thermometer and Problem List to calculate reference standards.25 These questionnaires have been recommended in the guideline for distress management.26 Patients completed these questionnaires 3 months after starting treatment; the time frame for answering the questionnaires was the last 3 months. The Distress Thermometer asks patients to assess their general level of distress, caused by physical, emotional, social, and practical problems, on a scale from 0 to 10. The cutoff score on the Distress Thermometer is 5.25 The accompanying Problem List identifies problems in five domains: the practical, social, cognitive‐emotional, spiritual/religious, and physical. Patients indicated whether they experienced a problem in these domains using a 5‐point scale (ranging from 1 “no problem at all” to 5 “very much a problem”). For every domain in which patients reported at least one problem, we asked whether they (a) received a referral to an expert for the problem, (b) did not receive a referral but would have preferred a referral for the problem, (c) did not want or receive a referral for the problem but would be willing to consider a referral in the future, or (d) did not want or receive a referral, neither in the present nor in the future.17

Emotional distress as reference standard was rated present if the patient scored 5 or higher on the Distress Thermometer and had at least 1 problem with score 3 or higher in the category “emotional problems” on the Problem List.17 The need for professional mental health care as reference standard was rated present if the patient had received a referral or wished to have received a referral for emotional problems on the Problem List.17

Analysis

Descriptive statistics (means, standard deviations, and frequencies as percentages) were used to describe patient characteristics, clinical assessment, emotional distress, and need for professional mental health care. Cross tabulations were employed, and sensitivity, specificity, and DOR were calculated to evaluate the diagnostic accuracy of clinical assessment against each of the two reference standards. A bootstrap analysis was used to compare the diagnostic accuracy of clinical assessment with emotional distress as reference standard vs need for professional mental health care as reference standard. A logarithmic transformation was applied to DOR. Two‐tailed tests of significance were performed, using alpha = .05. Data were analyzed using IBM SPSS Statistics version 20.0 (IBM SPSS Statistics for Windows, Armonk, NY) and Stata version 14.1 (Stata Data Analysis and Statistical Software, StataCorp LP, Texas).

RESULTS

Patients

Two hundred and forty‐one patients were asked to participate in the study, of whom 231 consented (96%). Of the patients included in the study, 80% completed the questionnaire (N = 185). In these 185 patients, there were no missing data. Patients were predominantly male (60.5%), with a mean age of 63.4 years. The majority (70.3%) had a spouse or partner. Almost all patients (91.9%) were born in the Netherlands. Baseline characteristics of the study sample are presented in Table 1.

Table 1

Sample characteristics

Gender, female, n (%)	73 (39.5)
Age in years, mean ± SD	63.4 ± 12.2
Living with a partner, yes, n (%)	130 (70.3)
Dutch nationality, yes, n (%)	170 (91.9)
Type of cancer, n (%)
Breast	26 (14.1)
Prostate	11 (5.9)
Colon/Rectum	37 (20.0)
Female reproductive organs	6 (3.2)
Male reproductive organs	6 (3.2)
Stomach/esophagus	20 (10.8)
Kidney/urethra	7 (3.8)
Bladder	7 (3.8)
Pancreatic	8 (4.3)
Skin	43 (23.2)
Sarcoma	2 (1.1)
Other	10 (5.4)
Unknown	2 (1.1)

Note: N = 185.

Clinical assessment of emotions

For 42.2% of patients (78 out of 185 patients), notes were found in the patient file describing fear, anxiety, stress, low mood or depression, or a referral because of these symptoms (Table 2).

Table 2

Clinical assessment, emotional distress and need for professional mental health care

Notes in the patient file, n (%)		78 (42.2)
Topic of notes, n (%) *
‐ Fear or anxiety	19 (10.3)
‐ Stress	21 (11.4)
‐ Low mood or depression	47 (25.4)
‐ (Possible) referral for psychosocial or mental health care	27 (14.6)
Emotional distress (defined as Distress Thermometer score ≥ 5, and ≥ 1 problem with score ≥3 in the category ‘emotional problems’), n (%)		67 (36.2)
Need for professional mental health care, n (%)		20 (10.8)
‐ Referral, n (%)	15 (8.1)
‐ Wish for referral, n (%)	5 (2.7)

N = 185

Total is more than 78 patients, because of multiple notes per patient

Level of emotional distress

The mean score on the Distress Thermometer was 5.2 (SD = 2.6). In terms of emotional problems, 81.6% of patients reported one or more problems, and 36.2% had experienced emotional distress (defined as Distress Thermometer score ≥5, and ≥1 problem with score ≥3 in the category “emotional problems”) (Table 2).

Need for professional mental health care

Need for professional mental health care was operationalized as either “having received a referral” or “wished to have received a referral” for emotional problems. In this case, 10.8% of the 185 patients expressed the need for professional mental health care, of which 8.1% received a referral and 2.7% would have liked referral for an emotional problem (Table 2). In patients with one or more emotional problems (N = 151), 13.2% expressed a need for professional mental health care, of whom 9.9% received a referral and 3.3% would have liked a referral for an emotional problem.

Diagnostic accuracy of clinical assessment

In Table 3, clinical assessment of emotions is set out against the two reference standards. The sensitivity of clinical assessment was higher when need for professional mental health care was used as a reference standard (0.85), compared with emotional distress as the reference standard (0.54). The specificity was equivalent for the two reference standards (0.63 vs 0.64). The DOR was 9.66 using need for professional mental health care as reference standard, and 2.10 with emotional distress as reference standard. A bootstrap analysis showed that the difference in sensitivity (z = 3.85; P < .001) and in DOR (z = 2.33; P = .020) between the two reference standards was statistically significant, whereas the difference in specificity was not statistically significant (z = .48, P = .629).

Table 3

Diagnostic accuracy of clinical assessment of emotions, compared with two reference standards

	Clinical Assessment		Sensitivity (95%CI)	Specificity (95%CI)	DOR (95%CI)
	Positive	Negative
Emotional distress
Present Absent	36 (19.5%) 42 (22.7%)	31 (16.8%) 76 (41.1%)	0.54 (0.42‐0.65)	0.64 (0.55‐0.72)	2.10 (1.14‐3.87)
Need for professional mental health care
Present Absent	17 (9.2%) 61 (33.0%)	3 (1.6%) 104 (56.2%)	0.85 (0.64‐0.95)	0.63 (0.55‐0.70)	9.66 (2.72‐34.31)

Abbreviation: DOR, diagnostic odds ratio.

DISCUSSION

This study is a first step towards a better understanding of the clinical assessment by medical oncologists and nurses of emotions in patients with cancer. We posited that a distinction should be made between emotions that do or do not necessitate professional mental health care, and we assumed that well‐trained and experienced medical oncologists and nurses make this distinction. As expected, we found that the sensitivity of clinical assessment was higher with patients' need for professional mental health care as reference standard compared with emotional distress as reference standard, while specificity remained constant. This result provides support for the idea that medical oncologists and nurses differentiate between emotions that do or do not necessitate professional mental health care.

Clinical implications

Our results show that while 36.2% of patients experienced emotional distress, a much smaller percentage (10.8%) of patients had a need for professional mental health care. These outcomes are in line with previous findings.10, 11, 15, 27 There are several explanations for the low need for professional care, such as not being aware of the full spectrum of professional health care, stigma related to mental health care, a preference to self‐manage their problems, or a preference to rely on other sources of support.16, 20 Our interpretation is that the majority of patients prefer to deal with their emotions themselves, with support from doctors, nurses, partner, family, and friends.28

In previous studies, the sensitivity of the detection of distress was reported as 0.09 to 0.12 (comparing patient and doctor ratings of individual symptoms such as depressed mood3), as 0.36 (comparing patient ratings on the General Health Questionnaire to doctor ratings of patients' distress29), as 0.21 to 0.74 (comparing patient and doctor ratings on the items of the Beck Depression Inventory1), and as 0.64 (comparing patient and doctor ratings on the Distress Thermometer1). The observed differences in sensitivity values may reflect differences in patient characteristics, doctor characteristics, medical setting, or study design. In the present study, with distress as the reference standard, we found a sensitivity of 0.54. Although at the higher end of the spectrum, this falls well within the range of previously reported values. However, when using “need for professional mental health care” as the reference standard, we found a sensitivity of 0.85. This value is clearly superior to sensitivity values with emotional distress as the reference standard, both in our own study (P < .001) and in the other studies cited above.1, 3, 29 Need for professional mental health care—as expressed by patients themselves—appears to be a better reference standard for clinical assessment than emotional distress. We hypothesize that this is because there is a need to differentiate between emotions that do or do not necessitate professional mental health care.

Systematic screening for psychological distress has been advocated, based on the assumption that clinical assessment of emotions is suboptimal.26 So far, there is no evidence that screening for psychological distress results in a reduction of distress.15 Case finding is the alternative to systematic screening.28 Our study provides evidence supporting case finding, as doctors and nurses were found to be sensitive to emotional problems in their patients. Resources could be invested in further strengthening the communication skills of doctors and nurses, and in securing sufficient time for discussion of emotional concerns with patients, instead of investing in systematic screening.

Study limitations

Several methodological considerations should be noted. First, a strength of the current study was the study design, which provided data covering the first 3 months of treatment instead of a single moment in time. The retrospective rating of distress and need for mental health care may have introduced some bias. Even if such a bias has occurred, it is unlikely that this could explain the differential sensitivity of clinical assessment of emotions compared with the two reference standards. Nevertheless, replication of the present study using a prospective design is desirable. Second, we used patient‐expressed need for care as a first, preliminary measure for emotions that necessitate professional mental health care. This approach resulted in a remarkable sensitivity of clinical assessment of emotions. However, not all patients may be aware of their need for professional care. We contend that it is important to develop explicit criteria for what constitutes an emotion that necessitates professional care. These criteria could be derived from recent research on features distinguishing between adaptive and maladaptive emotions (see, for example, references herein7, 21, 30). Third, a limitation of the present study is the reliance on notes in the patient file produced by oncologists and nurses. We assumed that they made notes if they considered the emotions to constitute a problem necessitating professional mental health care, now or in the near future. In contrast, we assumed that they would not make a note if they considered the emotions to be a normal response to a life‐threatening disease and a burdensome treatment. In our view, it is remarkable that in about half of the patients' (42.2%) notes on emotions were made: this seems to indicate that oncologists and nurses do pay attention to emotions in their patients. In a future study, an explicit rating of whether or not emotions necessitate professional mental health care could be used. Fourth, the study was performed in one single department of medical oncology. This tertiary oncology center offers psychosocial services to patients in need, provided primarily by medical social workers who participate in weekly multidisciplinary team meetings. If indicated, psychological or psychiatric care is offered. Generalization of our results to other departments of medical oncology cannot be taken for granted. Similarly, generalization of our results to other medical specialties cannot be taken for granted. In medical oncology, patients are often treated over a longer period of time, which may facilitate clinical assessment of emotions. Fifth, patients and medical oncologists/nurses were not aware of each other's assessment of emotions. Thus, in that respect, clinical assessment and reference standards were independently assessed. However, medical oncologists and nurses did interact with patients during treatment, resulting in the high sensitivity of clinical assessment of emotions. This interaction constitutes the very essence of clinical assessment of emotions. We consider clinical assessment of emotions based on the interaction between the patient and doctors/nurses to constitute a major strength of this study. Sixth, patients' files were reviewed by one trained psychologist: There is no information on interrater reliability.

In conclusion, the sensitivity of clinical assessment of emotions by medical oncologists and nurses seems more accurate than previously concluded. In future research, explicit criteria should be developed to differentiate between emotions that do or do not necessitate professional mental health care.

AUTHOR CONTRIBUTIONS

Design: Dekker, Braamse, van Linde, and Verheul. Formal analysis: Hoogendoorn. Manuscript preparation: Dekker, van Linde, and Braamse. Manuscript review and editing: All authors.