Performance of the 3-item screener, the Edinburgh Postnatal Depression Scale, the Hopkins Symptoms Checklist-15 and the Self-Reporting Questionnaire and Pregnancy Risk Questionnaire, in screening of depression in antenatal clinics in the Blantyre district of Malawi.

Background: Screening instruments for antenatal depression vary in performance. This study aimed at assessing the performance of a range of screening instruments in detecting depressive symptoms in antenatal clinics in Blantyre district, Malawi.
Methods: A cross-sectional study was conducted to screen for depression among women attending 8 selected antenatal clinics in Blantyre district using 3-item screener, Edinburgh Postnatal Depression Scale (EPDS), Hopkins Symptoms Checklist-15 (HSCL-15), Self-Reporting Questionnaire (SRQ) and Pregnancy Risk Questionnaire (PRQ). The instruments were administered to a random sample of 480 pregnant women. Data were analysed using SPSS 22.0 testing for performance differences in proportions of screen positives and how screen positive results might differ by particular variables.
Results: The prevalence estimates yielded by screening instruments ranged from 12.9% (SRQ) to 42.1% (3-item screener). There were no significant differences in prevalence estimates for EPDS, HSCL-15, PRQ and SRQ. There were performance differences in the proportions of screen positives with significant systematic differences between proportions of screen positives of PRQ and SRQ (p<.001), EPDS and HSCL-15 (p=.001), HSCL and PRQ (p<.001), and EPDS and SRQ (p<.001). Screen positive results on HSCL-15, PRQ, 3-item screener and EPDS were found to differ by variables such as "not being supported by partner" which resulted in respondents having ≥3 times chances to screen positive on these four instruments. The screen positive results on SRQ were found not to differ by age, education, employment status, marital status, setting, gestation and number of pregnancies. Conclusions: There were minimal variations in the performance of the EPDS, SRQ and HSCL-15 as standard public health screening instruments. However, systematic differences between proportions of screen positives exist and screen positive results from these instruments differed by demographics. It is important to validate screening instruments against a gold standard to ensure relevant clinical outcomes for pregnant women with depression.

Introduction

Screening for depression and risk factors during pregnancy is important for the management of the mental health and well-being of pregnant women and unborn babies1. In different resource level settings, effective screening of antenatal depression is dependent on instruments that are validated in these contexts. Though numerous instruments for screening of depression in antenatal clinics in low resource settings exist2, some of these instruments are not specifically designed for use during pregnancy. Some instruments have been designed for post-natal depression and few validation studies have been conducted in antenatal settings.

Performance of these instruments in detecting depression during pregnancy may vary with population, setting and structure of screening instruments themselves3–9. Inclusion of somatic items in a screening instrument may also affect the validity of the instrument as these may occur as part of the normal pregnancy10. Furthermore, the structure and format of these screening instruments which requires an individual to choose a response out of multiple options for each question rather than ‘yes’ or ‘no’ might not be easily understood by individuals with low literacy levels10. Due to concerns about variations of performance of screening instruments in different contexts11, the validity of screening instruments currently being used in antenatal clinics in low resource settings is of concern. In these settings, many women have low literacy levels, and midwives have high workloads with limited time to screen the emotional status of pregnant women12. This study aimed to assess the performance of the Edinburgh Postnatal Depression Scale (EPDS), Hopkins Symptoms Checklist-15 (HSCL-15), Self-Reporting Questionnaire (SRQ) and Pregnancy Risk Questionnaire (PRQ) in detecting depression in antenatal clinics in Blantyre district, Malawi. In addition, the 3-item screener for depression was included as it has been recommended that valid ultra-brief instruments for screening of depression which are short, easy to administer, clinically acceptable, and are minimally affected by literacy, may be more suitable in detecting possible cases of depression in primary care13,14. The PRQ was included because apart from screening depression it also assesses psychosocial risk factors for depression during pregnancy15.

Methods

This study used a cross-sectional quantitative survey design to screen for depression amongst a population of pregnant women (N=1593) attending 8 selected antenatal clinics in Blantyre district in February 2015. Sample size was calculated using the following parameters: estimated sensitivity (Se) of 96%, estimated specificity (Sp) of 57%, estimated prevalence (p) of 21%,16 and the Confidence Interval (CI) of .05. The calculated sample of 480 provided adequate caseness for screening for depression in pregnant women.16 Sample inclusion criteria were: attended antenatal care, 18 years old and above, written consent before joining the study and ability to speak and understand Chichewa (a local language). Exclusion criteria were: complications of pregnancy or known mental or medical conditions. A total of 496 pregnant women were invited to participate in this study of which 16 declined resulting in 480 who participated.

Screening instruments

Five instruments were included, namely: EPDS, HSCL-15, SRQ, PRQ and the 3-item screener for depression.

EPDS

The EPDS is the most commonly used instrument in pregnancy and has previously been reported as a valid (Se= 68.8%, Sp= 79.5%) and reliable (Cronbach's α=.9) instrument for screening antenatal depression in Malawi10. The EPDS is a 10-item self-reporting questionnaire which was originally designed to measure postnatal depression17 but has also been validated for screening antenatal depression2. The instrument measures depressive symptoms experienced by an individual in the past seven days18. The EPDS has a maximum total score of 30 with a standard cut off score of ≥10 for depression caseness19.

HSCL-15

The HSCL-15 was found to be valid (Se= 89%, Sp= 80%) and reliable (Cronbach's α =.9) in screening depression among pregnant women in Tanzania9. The HSCL-15 consisted of a fifteen items self-reporting inventory for assessing depressive symptoms which have been disturbing an individual in the past seven days20. The 15 items are measured on a Likert scale (1 to 4). The depression score is the calculated average of the 15 items. The HSCL-15 has a maximum average score of 4 with standard cut off of average depression score ≥1.7521.

SRQ

The SRQ has previously been used in Malawi and was found to be valid (Se= 76.3%, Sp= 81.3%) and reliable (Cronbach's α = .83) in detecting possible depression cases among pregnant women10. The SRQ has 20 questions which are used to assess for psychiatric symptoms that an individual has experienced in the past month22. The instrument has a maximum total score of 20 with a standard cut off ≥10 for depression caseness23.

PRQ

The PRQ is a valid instrument (Se=44% and Sp= 92% in an Australian population)15 designed to assess psychosocial risk and protective factors for depression during pregnancy and used to predict antenatal or postnatal depression15. The instrument has a maximum total score of 90 with a cut off ≥46 for depression caseness. The PRQ has 18 items which assess for psychosocial risk and protective factors for depression from childhood to the present.

The 3-item Screener

The instrument has two screening questions (Se= 96%)24 and a question asking “ are you depressed?” (Se= 94%)25 which have been found to be effective in screening depression. The screening questions rate depressive symptoms a person has had in the past month. The one-item screening question asks if a person is feeling depressed. The maximum total score for the 3-item screener was 3 and cut off was set as ≥1 for depression caseness.

Translation of screening instruments

Previously validated Chichewa language versions of the EPDS and the SRQ existed and were used in this study10. The 3-item screener, HSCL-15 and PRQ were translated into Chichewa by the first author and a social worker based on the minimum standards (back translation and monolingual testing) for applying an instrument that was developed in another language26.

Data collection procedure

Data collection was done by the first author and two registered midwives as research assistants, from January to May 2016. The assistants received two days training to familiarise them with the study, the data collection instruments and the data collection process. One research assistant was assigned to randomly select pregnant women from queue at antenatal clinics and invite them to participate in the study. The research assistant systematically picked every other third pregnant woman on the queue after randomly picking the first. Due to the low literacy levels, the first author and a second research assistant administered the 3-item screener, HSCL-15, SRQ, EPDS and PRQ by reading the questions and recording the answers on behalf of respondents.

Data analysis

The IBM Statistical Package for Social Sciences (SPSS) version 22.0 was used to analyse data. Significance level was set at 95%. Caseness (screening positive for probable antenatal depression) was determined using the following cut off scores: the 3-item screener (cut off ≥1), the HSCL-15 (cut off ≥1.75),21 the SRQ (cut off ≥10),23 the EPDS (cut off ≥10)19 and the PRQ (cut off ≥46).15 Descriptive statistics were used to analyse and summarise demographic characteristics in relation to probable antenatal depression cases identified by each screening instrument. Instruments' reliability were tested using Cronbach's α. Pearson Chi square test was used to compare differences between screen positives and negatives and different demographic variables. To test the agreement among instruments in detecting proportions of same individuals as screen positives, the McNemar test was used, with a statistically significant test confirming the presence of systematic differences between proportions of positive responses between any two instruments27. In addition, the possible differences in screen positive results by demographics and pregnancy factors were examined using binomial regression models.

Ethics approval

The study was granted ethics approval by the research committee of the University of the Western Cape and the College of Medicine Research and Ethics Committee (COMREC), University of Malawi.

Results

Demographics

A total of 480 respondents completed questionnaires (response rate of 96.8%). The age of respondents ranged from 18 to 43 years (mean 25.2 ±5.5). The mean number of pregnancies per respondent was 2.4 ±1.3 (range =1 to 6 pregnancies), with a current mean gestation period of 26.7 weeks ±7.4 (range= 5 to 40 weeks). More than half of the respondents were unemployed (52.5%, n=252), had more than primary school education (53.8%, n=256) and were from an urban area (65.2%, n=313). Nearly all the respondents were supported by a partner (92.9%, n=446).

Prevalence of depression (screen positives for depression caseness) by different instruments

The SRQ, HSCL-15, EPDS, PRQ and 3-item screener were reliable instruments in the setting (Cronbach's α= .86, .85, .80, .70 and .70 respectively). The prevalence (respondents who screened positive for depression) ranged from 12.9% (95% CI 9.9%–15.9%) (SRQ) to 42.1% (95% CI 37.7%–46.5%) (3-item screener) (Figure 1). There were no significant differences in the proportions of screen positives identified by the PRQ [20.2% (95% CI 16.6%–23.8%)], EPDS [19% (95% CI 15.5%–22.5%)], HSCL-15 [13.5% (95% CI 10.4%–16.6%)] and, SRQ [12.9% (95% CI 9.9%–15.9%)], though the 3-item screener detecting a significantly higher number of screen positives [42.1% (95% CI 37.7%–46.5%)] (Figure 1).

Figure 1

Screen positive prevalence for depression with confidence levels for all instruments

Agreement of instruments in detecting screen positives

Though there were insignificant variations among the prevalence estimate identified by the instruments, excluding the 3-item screener, there were performance differences in the proportions of screen positives, indicating that these estimates do not include exactly the same individuals. The McNemar's tests revealed significant systematic differences between proportions of screen positives from the following instruments: PRQ and SRQ (p<.001), EPDS and HSCL-15 (p=.001), HSCL and PRQ (p<.001), and EPDS and SRQ (p<.001). No significant systematic differences were found between proportions of screen positives from HSCL-15 and SRQ (p=.77), and EPDS and PRQ (p=.58).

Differences in performance of instruments by demographics and pregnancy factors

Possible differences in screening positive results by other variables such as demographics and pregnancy factors were examined using a binomial regression models for all the five screening instruments used in this study. With the odds (the chance of an individual without depression being a screen positive) as the effect measure, the SRQ at cutoff≥10, was the only instrument with screen positive results which did not differ by age (Odds Ratios [OR]=1.02, p=.63), education (OR=.72, p=.27), employment status (OR=.71, p=.25), marital status (OR=1.69, p=.34), setting (OR=.96, p=.90), gestation (OR=.99, p=.72) and number of pregnancies (OR=1.07, p=.71). This is consistent with the finding that there were no significant demographic differences between screen positives and negatives on the SRQ (Table 1).

Table 1

Demographics, pregnancy factors and the performance of screening instruments

Demographic and Pregnancy factors	EPDS≥10		HSCL-15 ≥1.75		SRQ≥10		PRQ≥46		3-item screener ≥1
	Positive, 91(19)	Negative, 389(81)	Positive, 65(13.5)	Negative, 415(86.4)	Positive, 62(12.9)	Negative, 418(87.1)	Positive, 97(20.2)	Negative, 383(79.8)	Positive, 202(42.1)	Negative, 278(57.9)
Occupation
Unemployed	53(58.2)	199(51.2)	45(69.2)	207(49.9)	36(58.1)	216(51.7)	58(59.8)	194(50.7)	109(54)	143(51.4)
Employed	38(41.8)	190(48.8)	20(30.8)	208(50.1)	26(41.9)	202(48.3)	39(40.2)	189(49.3)	93 (46)	135(48.6)
	(χ2=1.5, p=.22)		(χ2=8.4, p=.004)*		(χ2=.88, p=.34)		(χ2=2.6, p=.11)		(χ2=.29, p=.59)
Education level
Primary or none	46(50.5)	176(45.2)	34(52.3)	188(45.3)	34(54.8)	188(45)	54(55.7)	168(43.9)	86 (42.6)	136(48.9)
Secondary or above	45(49.5)	213(54.8)	31(47.7)	227(54.7)	28(45.2)	230(55)	43(44.3)	215(56.1)	116(57.4)	142(51.1)
	(χ2=.84, p=.36)		(χ2=1.1, p=.29)		(χ2=2.1, p=.15)		(χ2=4.3, p=.04)*		(χ2=1.9, p=.17)
Marital status
Supported by partner	80(87.9)	366(94.1)	56(86.2)	390(94)	57(91.9)	389(93.1)	87(89.7)	359(93.7)	182(90.1)	264(95)
Not being supported by partner	11(12.1)	23(5.9)	9(13.8 )	25(6)	5(8.1)	29(6.9)	10(10.3)	24(6.3)	20 (9.9)	14(5)
	(χ2=4.3, p=.04)*		(χ2=5.2, p=.02)*		(χ2=.1, p=.75)		(χ2=1.9, p=.17)		(χ2=4.2, p=.04)*
Setting
Urban	57(62.6)	256(65.8)	40(61.5)	273(65.8)	40(64.5)	273(65.3)	60(61.9)	253(66.1)	135(66.8)	178(64)
Rural	34(37.4)	133(34.2)	25(38.5)	142(34.2)	22(35.5)	145(34.7)	37(38.1)	130(33.9)	67 (33.2)	100(36)
	(χ2=.33, p=.57)		(χ2=.45, p=.5)		(χ2=.02, p=.9)		(χ2=.6, p=.44)		(χ2=.41, p=.52)
Age in years	25.2±4.9	25.1±5.6	25.9±4.9	25±5.6	25.7±5.3	25.1±5.5	26.5±5.7	24.8±5.4	25.8±5.74	24.7±5.5
	(χ2=2.7, p=.43)		(χ2=5.1, p=.17)		(χ2=5.3, p=.15)		(χ2=7.4, p=.06)#		(χ2=7.4, p=.06)#
Gestation in weeks	27.7±7.4	26.5±5.7	27±7.3	26.7±7.4	26.5±7.7	26.8±7.3	26.4±7.4	26.8±7.4	26.7±7.6	26.8±7.2
	(χ2=6.4, p=.09)		(χ2=1.5, p=.68)		(χ2=7.3, p=.06)#		(χ2=6.2, p=.1)		(χ2=4.2, p=.24)
Pregnancies	2.3±1.2	2.4±1.3	2.7±1.2	2.3±1.3	2.5±1.2	2.3±1.3	2.6±1.3	2.3±1.3	2.5±1.3	2.2±1.3
	(χ2=5.7, p=.13)		(χ2=13.7, p=.003)*		(χ2=7.5, p=.06) #		(χ2=9, p=.03)*		(χ2=7.1, p=.07) #

Data= n(%) or mean ± standard deviation, EPDS=Edinburgh Postnatal Depression Scale, HSCL-15=Hopkins Symptoms Checklist-15, SRQ=Self Reporting Questionnaire, PRQ=Pregnancy Risk Questionnaire, p=p value,

significance set at ≤.05,

approaching significance set at ≤.05

“Not being supported by partner” was significantly associated with being screen positive for depression in three out of the five screening instruments, with the SRQ and PRQ being the exceptions (Table 1). However, screen positive results on all instruments differed by a variable, “not being supported by partner” [HSCL-15 (OR=7.75, p=.<001), PRQ (OR=3.69, p=.004), the 3-item screener (OR=3.27, p=.003) and EPDS (OR=3.23, p=.01)], except the SRQ (OR=1.69, p=.34) with respondents “not being supported by partner” having 3 or more chances to screen positive on EPDS, HSCL-15, PRQ and the 3-item screener. Being older was also associated with a single chance of screening positive on PRQ (OR=1.07, p=.04), and approaching significance in the 3-item screener (Table 1).

Though a significant association between unemployment and screening positives for depression were found on the HSCL-15, respondents who were employed were less likely to screen positive on EPDS, HSCL-15, PRQ and the 3-item screener with the ratios of the probability of screening positive on the 3-item screener (OR=.66, p=.04), EPDS (OR=.62, p=.06), and PRQ (OR=.51, p=.01) and HSCL-15 (OR=.26, p<.001) were less than 1.

Significant associations were only found for education level with screen positives for depression on the PRQ with most of the screen positives having low education (primary education or none) (Table 1). However, screen positive results for the 3-item screener only were found to differ with education (OR=1.5, p=.05).

All instruments showed pregnancy factor differences between screen positives and screen negatives with the number of pregnancies being a significant factor in the HSCL-15 and PRQ, with the SRQ and 3-item screener approaching significance (with screen positive women reporting higher number of pregnancies) (Table 1). For the HSCL-15, the differences in screen positive results by number of pregnancies approached significance (OR=1.38, p=.07) with the other instruments being not significant. Depression was also associated with higher gestation ages, with the SRQ being significantly higher and the EPDS approaching significance (Table 1). However, screen positive results for all five instruments did not differ by gestational ages in this study.

Discussion

This study confirmed that the performance of the screening instruments in detecting depression during pregnancy may vary in different populations or settings2,13,28, and by the types of instrument used for screening antenatal depression29.

Performance in identifying screen positives by instruments

Excluding the 3-item screener, there were no significant variations in the sample depression prevalence estimates between the instruments based on screen positives as identified by the standard cutoffs for each instrument. From a public health screening perspective, this confirms the validity of these instruments for screening for depression in this setting30,31, though further studies need to be done to assess the validity of these cutoff scores for these settings. In addition, there were performance differences in the proportions of screen positives with significant systematic differences between proportions of screen positives of the PRQ and SRQ (p<.001), EPDS and HSCL-15 (p=.001), HSCL and PRQ (p<.001), and EPDS and SRQ (p<.001), indicating that the proportions of screen positives do not include the same respondents. The differences in time frames of the screening instruments might have contributed to variations in performance of these instruments as it ranged from lifetime (PRQ) and last week (EPDS). Pregnant women may have memory lapse to recall remote information asked by instruments with longer time frames (the 3-item screener, PRQ and SRQ) compared to those with shorter ones (EPDS and HSCL-15). This has implications for using these instruments routinely for clinical screening for depression of pregnant women in antenatal clinics in Malawi. The variations in performance of instruments indicate the importance of validating screening instruments for clinical settings in the actual context prior to clinical use and comparing the results generated by a screening instrument against a gold standard to establish the instrument's level of accuracy 32,33 in detecting depression.

Demographics, pregnancy factors and the performance of screening instruments

Differences in screening results by other variables was found. Demographics such as age, education, employment status34 and marital status35–38 are associated with the chances of individuals screening positive on various instruments.

Single status is associated with antenatal depression16,39,40 and pregnant women who lack support from their partners are likely to suffer from depression.37,41 Our study is consistent with other studies which found that “not being supported by partner” was associated with screening positive on EPDS37,38 and a 2-question screener38, and screening positive results for four instruments (EPDS, HSCL-15, PRQ and the 3-item screener) were found to differ by “not being supported by partner” with respondents who were not being supported by partner having 3–8 times chances of screening positive on these four instruments. Despite that “not being supported by partner” impacted performance of all screening instruments, it remains a risk factor for depression in the local context where nearly all the respondents were supported by a partner (92.9%, n=446). A systematic review found that unemployment was not significantly associated with antenatal depression.35 Consistent with this study, there were no significant differences in employment status among pregnant women who screened positive on EPDS, PRQ, SRQ and the 3-item screener. However, contrary evidence indicates that employment status is significantly associated with positive screen on EPDS among South Korean pregnant women 42 and antenatal depression is more prevalent amongst unemployed pregnant women43–46. In this study, respondents who were employed had high chances to screen negative on EPDS, HSCL-15, PRQ and the 3-item screener while screen positive results on SRQ did not differ with employment. These inconsistent findings may be attributed to the effect of employment status on the performance of screening instruments which is not unidirectional.

Pregnancy factors and depression has had mixed results with some studies showing no significant association between number of pregnancies and antenatal depression rated on the EPDS, 37 SRQ47 and other screening instruments48, and other studies showing that women with multiple pregnancies are likely to have depression during pregnancy49–52. Our study confirms this with a significant association between number of pregnancies per woman and screening outcomes on HSCL-15, PRQ, SRQ and the 3-item screener. Limitations of the study

The screening instruments were administered sequentially, and it is possible that performance of subsequent instruments might have been influenced by respondents' knowledge of similar questions already covered by the preceding instrument/s. The differences in rating time frames and structures of the screening instruments may also be a further limitation.

Conclusion

There appears to be minimal variations in the performance of the EPDS, SRQ and HSCL-15 as standard public health screening instruments. However, systematic differences between proportions of screen positives exist and screen positive results from these instruments differ by demographics. Therefore, it is important to validate screening instruments in local settings against a gold standard to ensure relevant clinical outcomes for pregnant women with depression attending antenatal care.