Proposing a new stroke-specific screening tool for depression: Examination of construct validity and reliability.

OBJECTIVES: The optimal tool for identifying depression after stroke is yet to be identified. In the present study, we propose a new context-specific screening tool for PSD and examined its construct validity and reliability within existing data on recent stroke survivors.
METHODS: We assessed baseline data being collected as part of an intervention to improve one-year blood pressure control among recent (≤one month) stroke survivors. Depression was measured using the Hospital Anxiety and Depression Scale (HADS-D). We also independently administered the 26-items Health Related Quality of Life in Stroke Patients (HRQOLISP-26), a stroke-specific measure developed from a large cross-cultural sample. Using standard protocol, we identified 6 psychoemotional-domain items of the HRQOLISP-26 fitting a single dimensional model with phenomenological and conceptual overlap with the depression framework in the 10th revision of the International Classification of Diseases (ICD-10). We examined construct validity by comparing HRQOLISP-E with the HADS-D, and known group validity by comparing with age, gender, and stroke severity using both the Pearson product moment coefficient and multivariate regression analyses. Internal consistency and split-half reliability were also determined.
RESULTS: Each HRQOLISP-E item (r = -0.40 to -0.53, all p < 0.001), as well as the total HRQOLISP-E score (-0.53, p < 0.001) showed significant correlation with the HADS-D. The HRQOLISP-E scores also correlated significantly with age and stroke severity. Depression assessed using the HRQOLIPS-E was independently associated with older age and stroke severity. All HRQOLISP-E items scale correlations were> 0.8 (0.81-0.93) compared with 0.56-0.68 for the HADS-D (Cronbach's alpha =0.939, versus 0.742 for the HADS-D, Split-half reliability = 0.899, versus 0.739 for HADS-D).
CONCLUSION: These results provides preliminary support for further development of the HRQOLISP-E as a context-specific screening tool for PSD through an investigation comparing the proposed measure against a referent-standard clinical diagnostic criteria such as the ICD 10 or Diagnostic and Statistical Manual of Fourth Edition of the Diagnostic and Statistical Manual of Mental Disorders.

Introduction

Stroke-burden is increasing rapidly [4] due to the rising morbidity and disability [12]. Most of poststroke morbidity is due to depression [9], which though treatable [7], is mostly unidentified and untreated [6]. Consequently, improvement in recognition and access to effective treatment has become a priority, and routine screening for poststroke depression (PSD) is now increasingly recommended [20].

The standard recommendation for a diagnosis of PSD suggests that depression diagnoses should most appropriately be based on a semi-structured mental state examination and clinical criteria such as the Diagnostic and Statistical Manual of Mental Disorders (DSM IV/V) or the 10th Revision of the International Classification of Diseases (ICD 10) for depression due to stroke with major depressive-like episode or depressive features [16]. Currently, it remains very difficult to identify the most appropriate screening tool for PSD out of the wide variety available [3].

The experience of depression may vary across socio-economic and clinical circumstances [5]. As such, it is desirable for assessment tools to reflect the user-context. However, available screening tools for PSD are generic, and originally designed for use in general psychiatric populations [8]. In this way, they lack content validity for stroke.

In the two systematic reviews of the psychometric properties of all tools (N = 27) that were used for PSD screening up to February 2014 [3], [8], none of the measures identified was sufficient in ‘ruling-in’ depression in stroke survivors because of many false-positives [8]. Tools meeting psychometric and clinical utility criteria in the reviews [3], [8]were either no longer in contemporary use or require specialist training and/or are time consuming to administer. The authors noted that the Hospital Anxiety and Depression rating Scale (HADS) which was psychometrically adequate in the stroke population was limited by the enormous cost of its acquisition. Therefore, the optimal tool for identifying depression after stroke is yet to be identified. Such measure should ideally be easy to interpret and acceptable to both stroke patients and healthcare professionals.

In the present study we propose a new context-specific screening tool for PSD, the ‘HRQOLISP-E’, which is empirically designed from the 26-items version of the Health Related Quality of Life in Stroke patients (HRQOLISP-26) [10]. The HRQOLISP-26 is a stroke-specific measure developed from a large cross-cultural, transnational, patient-controlled sample, and based on a comprehensive model [11]. We aim to examine construct validity and reliability of the HRQOLISP-E within existing data on recent stroke survivors.

Methods

Sites

We evaluated a dataset comprising baseline information collected as part of a ongoing study of an intervention to improve one-year blood pressure control among recent (< 1 month) stroke survivors who were discharged from four hospitals in Nigeria. Ethical approval was obtained from the institutional review boards covering the four hospital sites; the University of Ibadan/University College Hospital joint ethics committees (which cover the World Federation of Neurorehabilitation-Blossom Specialist Medical Center), Federal Medical Center, Abeokuta, and Sacred Heart Hospital. Participants provided written, informed consent before interviews were conducted.

Subjects

Inclusion criteria

1); age 18 years or older, and 2), recent (≤ one month) survivors of ischaemic or haemorrhagic stroke survivors. The diagnosis of stroke was confirmed based on neuro-imaging and clinical examination criteria [17].

Included patients were informed about the study, and the procedure was explained to them in their home language.

Exclusion criteria

1), patients with severe communication difficulties (N = 34) or aphasia (N = 42); 2) patients with severe cognitive impairments or dementia [(Modified Community Screening Instruments for Dementia (CSID) ≤ 20)]; 3), Modified Rankin Scale (MRS) ≥ 3; and 4), significant comorbid medical illnesses (e.g., chronic kidney disease) [13].

Measures

Stroke survivors meeting study criteria underwent baseline assessments within the first month of stroke.

PSD was ascertained using the Depression subscale of the Hospital Anxiety and Depression Scale (HADS-D) [22]. The HADS is one of the most widely used screening tool for PSD, and as reported in a recent systematic review of all such instruments [3], it is one of two tools with superior psychometric properties and clinical utility indices in stroke populations. As such, it could be considered a useful referent tool for the development. It includes a total of 14 items each with a score of between 0 and 3. One half of the items are related to anxiety while the other half is specific for depression. The developers of the scale recommend a cut off ≥ 8 for the ascertainment of depression in clinical settings. The HADS has been previously validated in Nigeria [2] where the HADS-D was found to have a sensitivity ranging 89.5–92.1% and a specificity of 86.6–91.1%. Given the acclaimed properties of the HADS-D, we used depression ascertained using the measure as a referent standard for the purpose of the present study.

The HRQOLISP-26 was also independently administered within 15–20 min of the HADS. The HRQOLISP-26 is a flexible and valid shortened version of the comprehensive HRQOLISP suitable for regular assessment of all domains of health related quality of life. It has been found to demonstrate excellent psychometric properties and is valid for routine use in stroke survivors. The instrument has been tested and validated for use in Ibadan and is applicable multiculturally. The HRQOLISP-26 is comprised of four therapeutically relevant domains: Physical, psycho-emotional, cognitive, and eco-social. Apart from the cognitive domain with 5 items, the other domains are composed of 7 items. The items in the psycho-emotional domain are designed to assess stroke-specific emotional wellbeing.

Other data collection

The following information was obtained from all participants using a standardized questionnaire: demographic data, personal history of smoking, alcohol consumption, and physical activities, medical history of hypertension, diabetes, hyperlipidaemia and heart disease, the use of medications for these conditions, and family history. Information on dietary patterns was obtained using the food frequency questionnaire. The severity of stroke was ascertained using the National Institute of Health Stroke Scale and Stroke Levity Scale [14].The average of two blood pressure (B·P) measurements was recorded. Each B·P measurement was obtained using an Omron HEM-907 XL 26 blood pressure monitor and the readings recorded according to standardized protocol provided by the manufacturers. Along with the blood pressure and pulse rates, anthropometric measurements of weight, height, waist and hip circumferences were also undertaking. Records of other relevant risk factors for stroke were also made. This includes fasting blood sugar, lipid profile, electrocardiogram, carotid Doppler and echocardiography.

Statistical analyses

Descriptive statistics such as means and standard deviations were used to summarize quantitative variables, while frequencies and proportions were used for discrete variables. Stroke survivors with clinical depression where those who had a score ≥ 8 on the HADS-D.

We explored dimensionality of the Psycho-emotional items of HRQOLISP-26 by examining whether the domain could best serve as a single independent scale. For this, we conducted exploratory factor analysis (EFA) on all 7 items in the Psycho-emotional domain of HRQOLISP-26. Factors obtained following initial maximum likelihood exploration were further rotated using the varimax procedure. Factors were recorded when they have eigenvalues greater than unity. For the factor extraction, loadings of ≥ 0.5 were considered meaningful.

We then examined content validity. For this, a consultant psychiatrist independently examined each of the Psycho-emotional items and determined whether the scale contained the minimum number of items required for the diagnostic assessment of depression specified in the ICD 10.

Next, we investigated the phenomena of screening positive for depression in the HADS-D versus the empirically-determined depression scale (HRQOLISP-E). Further exploration of criterion validity was then conducted by examining the correlation of the scores of each items of HRQOLISP-E with the total HADS-D score using the Pearson product moment correlation (r) [15]. The same method was used for the total HRQOLISP-E scores, and for the investigation of additional evidence of construct validity, by examining the correlation of the HRQOLISP-E total scores with age, gender (male = 1, Female = 2) and stroke severity. The independent association of HRQOLISP-E with known predictors of depression such as age, gender, cognitive impairment and stroke severity were investigated in both univariate and multivariate regression analyses.

Finally, we examined HRQOLISP-E item scale correlations, coefficient alpha and split-half reliability.

All analyses were conducted using Stata MP version 14.0 [18].

Results

Subject characteristics

There were 249 males and 142 females in the study sample (Table 1). Their mean age was 57.3 (± 11.7) years. There was no significant difference in the mean age for men (57.4 ± 12.2) and women (57.0 ± 10.8). Over 90% of the subject had at least 6 years of formal education. Nearly all participants in the present study had either mild or moderate stroke (Table 1).

Table 1

Characteristics of the study sample by HADS-D status.

Variables	Overall sample (n = 387)	No depression (n = 125)	Depression (n = 262)	p-Value
Mean age	57.4 (11.6)	55.4 (10.8)	58.3 (11.9)	0.0208⁎
Age (years)
≤ 45	58 (15.0)	25 (20.0)	33 (12.6)
45–65	235 (60.7)	79 (63.2)	156 (59.5)
> 65	94 (24.3)	21 (16.8)	73 (27.9)	0.023⁎
Gender
Male	248 (64.1)	84 (67.2)	164 (62.6)
Female	139 (35.9)	41 (32.8)	98 (37.4)	0.377
Education
None	32 (8.3)	12 (9.6)	20 (7.6)
Primary	72 (18.6)	17 (13.6)	55 (21.0)
Secondary	107 (27.7)	35 (28.0)	72 (27.5)
Higher	176 (45.5)	61 (48.8)	115 (43.9)	0.342
Occupation
Skilled/professional	117 (30.2)	43 (34.4)	74 (28.2)
Semi-skilled	52 (13.4)	13 (10.4)	39 (14.9)
Manual	108 (27.9)	36 (28.8)	72 (27.5)
Retired	88 (22.7)	25 (20.0)	63 (24.1)
Others	22 (5.7)	8 (6.4)	14 (5.3)	0.532
Stroke severity: NIHSS
Mild (≤ 15)	385 (99.5)	124 (99.2)	261 (99.6)
Moderate (16–20)	2 (0.5)	1 (0.8)	1 (0.4)	0.542f
Severe (21–42)	–	–	–
Stroke severity: SLS
Mild (11–15)	312 (81.5)	112 (91.1)	200 (76.9)
Moderate (6–10)	65 (17.0)	9 (7.3)	56 (21.5)
Severe (0–5)	6 (1.5)	2 (1.6)	4 (1.5)	0.001f
Total	100	32.3	67.7

f - Fisher's exact test.

(p < 0.05).

Construct validity

In EFA, a one factor model containing 6 of the 7 psychoemotional domain items explained 67.9% of the total variance (Table 2). These items (conceptually referred to as HRQOLISP-E) overlapped with depression items described in the conceptual framework of depression in the ICD 10 (Table 2). In the same table, the 6 empirically derived items satisfied the minimum number of items recommended for the clinical diagnostic ascertainment of depression in both the ICD 10 framework and the HADS-D. However unlike the ICD 10 criteria, the HRQOLISP-E and HADS-D do not include somatic symptoms.

Table 2

Phenomenology: HRQOLISP-E items compared with HADS-D and the ICD 10 conceptual framework for depression.

ICD 10 (model) of depression	HRQOLISP-Ea	HADS-D
Core symptoms of depression
1. Depressed mood (sadness, emptiness, hopelessness, tearfulness, Irritability) 2. Loss of interest or pleasure	1. Often/always have negative feelings (Blue mood, anger, despair, anxiety, depression, fear) 2. Seldom or never able to laugh 3. Seldom or never enjoy work 4. Seldom or never Enjoy pleasure	1. Not quite able to laugh and see the funny side of things as much as I always could 2. Not often/at all feel cheerful 3. I still hardly/only a little enjoy the things I used to enjoy 4. I hardly/definitely less than before look forward to enjoyment 5. I can very seldom/not often enjoy a book, radio or TV
3. Decreased energy or easily fatigued	5. Seldom or never Have enough energy for everyday life	–
Other symptoms of depression
4. Reduced self-esteem and self-confidence	6. Seldom or never Able to accept bodily appearance	6. I have lost interest in my appearance
5. Ideas of guilt and unworthiness
6. Bleak pessimistic view of the future	–	–
Many somatic/biological symptoms of depression also listed	N/A	N/A
Two core symptoms + two others = major depression	Any four = depression	Any four = depression

Total variance explained = 67.9, Factor loadings (0.82–0.92), the question ‘How often do you have negative feelings?’ in the original HRQOLISP-E is excluded from the HRQOLISP-E because of low factor loading = 0.31.

Eigenvalue = 4.8,

In table 3, each HRQOLISP-E item (r = − 0.40 to − 0.53, all p < 0.001) and the total score (− 0.53, p < 0.001) showed significant correlation with the HADS-D. In the same table, the HRQOLISP-E scores correlated significantly with age, gender and stroke severity. Depression assessed using the HRQOLIPS-E was independently associated with older age and stroke severity (Table 4).

Table 3

Comparison of HRQOLISP-E and HADS-D scores for All, depressed and non-depressed participants.

	All patients	Depresseda	Non-depressed
HRQOLISP-Eb	r(p-value)	r(p-value)	r(p-value)
Reduced energy	− 0.5 (< 0.001)	− 0.1 (0.026)	− 0.4(< 0.001)
Unable to accept own bodily appearance	− 0.5 (< 0.001)	− 0.2 (0.004)	− 0.4 (< 0.001)
Unable to enjoy work	− 0.4 (< 0.001)	− 0.0 (0.544)	− 0.4 (< 0.001)
Unable to laugh	− 0.5 (< 0.001)	− 0.4 (< 0.001)	− 0.3 (0.003)
Unable to enjoy leisure	− 0.5 (< 0.001)	− 0.3 (< 0.001)	− 0.5 (< 0.001)
Dissatisfied with feelings	− 0.5 (< 0.001)	− 0.3 (< 0.001)	− 0.4 (< 0.001)
Total	− 0.5 (< 0.001)	− 0.2 (< 0.001)	− 0.4 (< 0.001)
Demographics
Age	− 0.2 (0.001)	− 0.1 (0.248)	− 0.2 (0.012)
Gender	− 0.1 (0.051)	− 0.1 (0.236)	− 0.1 (0.327)
Stroke severity: NIHSS	− 0.5 (< 0.001)	− 0.4 (< 0.001)	− 0.6 (< 0.001)
Stroke severity: slsb	0.8 (< 0.001)	0.7 (< 0.001)	0.6 (< 0.001)

Depression status is according to HADS-D where higher scores represents greater severity.

Lower scores represents greater severity.

Table 4

Relationship between HRQOLISP-E and known predictors of poststroke depression.

Variables	Depression (HRQOLISPE)
	Univariate OR (95% CI)	Adjusted OR (95% CI)
Age (years)
≤ 45	1.00	1.00
45–65	1.67 (0.92–3.01)	1.32 (0.71–2.43)
> 65	1.85 (0.94–3.62)	2.38 (1.11–5.09)⁎
Gender
Male	1.00	1.00
Female	1.75 (1.14–2.69)⁎	1.33 (0.83–2.16)
Education
None	1.00	1.00
Primary	2.09 (0.89–4.89)	2.56 (0.99–6.65)
Secondary	1.21 (0.55–2.67)	1.69 (0.70–4.07)
Higher	1.23 (0.58–2.62)	1.59 (0.68–3.71)
Cognitive impairment	2.09 (0.80–5.49)	2.35 (0.66–8.39)
Stroke severity: SLS
Mild (11–15)	1.00	1.00
Moderate (6–10)	21.77 (6.69–70.84)⁎	3.17 (1.49–6.74)⁎
Severe (0–5)	2.11 (0.38–11.67)	1.38 (0.24–7.86)

p < 0.05.

Reliability

In Table 5, all HRQOLISP-E items scale correlations were > 0.8 (0.81–0.93) compared with 0.56–0.68 for the HADS-D. The Cronbach's alpha and split half reliability indices of the new measure were also better than those of the HADS-D (Cronbach's alpha = 0.939 vs 0.742 for the HADS-D, Split-half reliability = 0.899 vs 0.739 for HADS-D).

Table 5

Split half and internal consistency reliability.

HRQOLISP-E	Item-scale correlations
Reduced energy	0.899
Unable to accept own bodily appearance	0.928
Unable to enjoy work	0.894
Unable to laugh	0.805
Unable to enjoy leisure	0.897
Dissatisfied with feelings	0.870
Overall Cronbach's alpha	0.939
Split-half reliability	0.899
HADS-D
Enjoyed things as before	0.627
Able to laugh	0.646
Feel cheerful	0.677
Feel slowed down	0.665
Lost interest in own appearance	0.672
Hopeful of enjoyment	0.624
Enjoy leisure	0.564
Overall Cronbach's alpha	0.742
Split-half reliability	0.739

Discussion

We found in the present study that 6 of the 7-items psycho-emotional domain of HRQOLIP-26, a stroke-specific measure, fitted a one factor structure, thus demonstrating that the component items (N = 6) measures a conceptual model in a single dimension (HRQOLISP-E). These empirically-determined items overlapped phenomenologically with depression items in both the ICD 10 and HADS-D. The items also satisfy the minimum number required for a diagnosis of depression in both diagnostic criteria, thus demonstrating content validity for depression.

The conceptual framework of depression in the ICD 10 emphasizes both number and quality of symptoms. For example, symptoms such as low mood, loss of interest and reduced energy, which are described as the major symptoms of depression, are given higher diagnostic weights compared with other symptoms in the diagnostic code. Five of the 6 HRQOLISP-E items overlap conceptually with these higher-ranked depression symptoms. These five symptoms are also represented in the HADS-D.

Apart from being derived from the HRQOLISP-26, a stroke-specific measure developed from a large cross-cultural, transnational, patient-controlled sample, and which was based on a comprehensive model [11], the exclusion of somatic and cognitive symptoms of depression in the HRQOLISP-E is a further demonstration of its stroke-specificity. Somatic and cognitive symptoms of depression such as reduced concentration, memory functions and altered sleep patterns overlap with symptoms that are commonly encountered in stroke survivors independent of emotional disturbances [3]. Therefore, their inclusion in depression screening tools is likely to confound scores and lead to inaccurate clinical decisions or research findings in the context of stroke.

The correlation and independent association of the HRQOLISP-E total score with known predictors of PSD such as age, gender, and stroke severity are evidence of construct validity. Construct validity of the HRQOLISP-E was also demonstrated by moderate items correlation with the HADS-D and a high percentage (67.9%) explained variance. We think that, the mostly moderate correlation coefficient found was due to the HADS-D, the comparator measure for these analyses, not being the gold standard for depression ascertainment and diagnoses; and also not being stroke-specific. In all studies in the literature where the HADS-D has been compared with the referent standard DSM (III-R and IV) criteria [1], [19], [21], as an example, the measure demonstrated a positive predictive value (PPV) of between 29% and 44% for depression within one week of stroke.

The finding in the present study that the HRQOLISP-E appear to demonstrate better internal consistency and split half reliability is additional evidence that the HADS-D is unlikely to be the best criterion to compare the new stroke-specific measure of depression. We are mindful of the effect this particular limitation on the results of the present study. It is feasible that HRQOLISP-E may perform differently against a stronger depression criterion than the HADS-D. However, as we have not carried out clinical diagnostic assessments as part of the present study, we have chosen the HADS-D as the next best criterion to compare HRQOLISP-E based on evidence from systematic reviews and accompanying metaanalysis [3], [8].

We note that stroke survivors in the present study were identified as part of a randomized controlled trial (RCT). Persons who were too ill to provide subsequent follow-up information were excluded, thus, suggesting that the sample for the present study may not be typical of the full spectrum of stroke survivor population in our study setting. For example, nearly all participants in the present study had stroke of mild or moderate severity. It is reasonable to expect that persons with more severe stroke may likely have more physical and mental health morbidity.

For now, the results of the present study provides preliminary support for further development of the HRQOLISP-E as a context-specific screening tool for PSD through an investigation comparing the proposed measure against standard clinical diagnostic criteria such as the DSM IV/V and ICD 10.Such investigation will also generate additional reliability data, responsiveness and cut-off scores providing the best balance between sensitivity and specificity. The clinical utility of screening tools for PSD will be improved if such measures reflect the user-context since the experience of depression may vary across socio-economic and clinical circumstances [5]. The ‘HRQOLISP-E’ is empirically designed from a stroke-specific measure and appears to demonstrate good construct validity. Initial reliability information shows that the new instrument may be potentially more reliable for depression screening in acute stroke compared with some of the best available tools for this purpose. This finding requires confirmation from studies using a more generalizable sample of stroke survivors and comparing the proposed measure against referent standard clinical diagnostic criteria such as the DSM IV/V and ICD 10.