Correspondence Between the Neuropsychiatric Interview M.I.N.I. and the BDI-II and MADRS-S Self-Rating Instruments as Diagnostic Tools in Primary Care Patients with Depression.

OBJECTIVE: To investigate the correspondence between the diagnoses received by patients with symptoms of common mental disorder attending primary care, based on the diagnostic instrument International Neuropsychiatric Interview (M.I.N.I.) and the self-assessment instruments such as Beck Depression Inventory (BDI-II) and Montgomery-Asberg Depression Rating Scale - self-rating version (MADRS-S), respectively.
DESIGN: Data were collected from a prospective observational study, ADAS, between 2014 and 2015.
SETTING: Twenty-eight primary care centers in Region Västra Gotaland, Sweden. PATIENTS: A total of 192 patients, 18-60 years of age, on sick leave ≥14 days, with mild/moderate depression, anxiety syndrome, and stress-related mental illness were included. MAIN OUTCOME MEASURES: Scores of the assessment instruments (BDI-II and MADRS-S) on inclusion, sensitivity, specificity, positive (PPV) and negative predictive value (NPV) for BDI-II and MADRS-S, respectively, with M.I.N.I used as diagnostic instrument.
RESULTS: Using M.I.N.I. as gold standard, the BDI-II and MADRS-S showed almost the same sensitivity (86.9% and 87.4%, respectively), but specificity for MADRS-S was doubled compared to BDI-II (36% and 18%, respectively). There was a significant association between MADRS-S and M.I.N.I. (p=0.027). However, the same analysis between BDI and M.I.N.I. was not statistically significant (p= 0.635). NPV and PPV were calculated from assumed prevalences (10% and 75%) and were higher for MADRS-S compared to BDI-II. The PPV differences were between 2% and 7% and NPV differences were between 3% and 19%.
CONCLUSION: With M.I.N.I. as gold standard, MADRS-S performs better than BDI-II as a self-assessment tool in the primary care context for depression diagnostics.

Introduction

In Sweden, 10–15% of the adult population suffers from depression and anxiety disorders and stress-related mental illness, usually referred to as common mental disorders (CMD).1,2 Epidemiological studies show that around 70% of all patients with depression and anxiety are treated in primary care.3 The need for a validated screening instrument for high-risk groups with depression has been the subject of much debate in primary care.4 In the medical consultation, a variety of assessment instruments and rating scales can be used to classify degree and type of symptoms but there is no explicit recommendation concerning the assessment scales to be used.1 Validation and reliability of the International Neuropsychiatric Interview (M.I.N.I.) compared to the Structured Diagnostic Interview for DSM-III-R (SCID-P) and compared to a structured interview developed by WHO for ICD-10 (CIDI)5,6 has shown acceptable validity and reliability for M.I.N.I., and M.I.N.I. can be a useful tool for identifying individuals with increased risk of mental illness in the primary care setting.7 Using M.I.N.I. for diagnostic purposes is, however, time-consuming and requires about 25–30 minutes by trained assessors.8 Therefore, regular use of M.I.N.I. as a diagnostic instrument can be difficult in the time-pressured primary care. Further, M.I.N.I. is not recommended for diagnosis of stress-related mental illness (adjustment disorder), an increasingly prevalent CMD diagnosis in Swedish primary care.9

There are a variety of self-assessment instruments and rating scales in use in primary care such as the Beck Depression Inventory (BDI-II)10 and the Montgomery–Asberg Depression Rating Scale-Self (MADRS-S).11 Although studies comparing BDI-II and MADRS-S have been performed,12–14 few such studies have been conducted in primary care. The time required for completing MADRS-S or BDI-II amounts to a few to ten minutes, and the patients manage the instruments themselves.

The purpose of the study was to investigate the correspondence between M.I.N.I. and BDI-II and MADRS-S, respectively, for patients with depression, anxiety and stress-related mental illness in the primary care setting.

Patients and Methods

Data were collected from the prospective observational ADAS-study in the Region Västra Götaland, Sweden, between 2014 and 2015.15 All patients from 28 primary care centers (PCCs) who were aged 18 to 60 years and had been on sick-leave for ≥14 days and diagnosed with mild/moderate depression (MADRS-S<35), anxiety syndrome and/or stress-related mental illness were asked to participate in the study by the health center’s rehabilitation coordinator. A total of 192 patients were included and filled in BDI-II and MADRS-S at baseline. M.I.N.I. (version 6.0.0d) and the fatigue syndrome scale (UMS)16 were used as diagnostic instruments by specially educated research nurses.

The exclusion criteria were substance dependence, alcohol abuse, generalized anxiety disorder, bipolar disorder, psychotic disorders or any severe psychiatric disorder, including severe depression (MADRS-S>35), high risk of suicide, or insufficient knowledge of the Swedish language.

Statistical Methods

Power analysis, the assumption was based on an approximate measure obtained in a previous primary care study. 15 With a power of 0.80 and a significance level of 5% (alpha = 0.05 and beta = 0.20), a total of 57 individuals tested by each instrument was judged required for the analysis. The statistical analyses were made using statistical software SPSS, version 24. Statistical significance was set at p<0.05.

Standard statistical methods were used for descriptive statistics. The continuous variables were analyzed by independent-samples t-test and categorical variables or frequencies by Pearson chi-square test. The consistency between the different instruments, M.I.N.I. and BDI-II and M.I.N.I. and MADRS-S, respectively, was analyzed by crosstabs with chi-square tests. In order to analyze the relationship between the BDI-II and M.I.N.I. and between the MADRS-S and M.I.N.I., the BDI-II and MADRS-S scores were dichotomized using a cut-off point of 13 and 12, respectively, according to the guidelines for minor and major depression.10,11,17,18 The ROC curve was used to determine these cut-off limits.19 Results for the M.I.N.I., BDI-II, and MADRS-S were compared between men and women.

To determine the sensitivity and specificity between BDI-II and M.I.N.I. and MADRS-S and M.I.N.I., respectively, cross tables were analyzed for these variables.

Several positive predictive values (PPV) and negative predictive values (NPV) were calculated for different assumed prevalences between 10% and 75% for both variables BDI-II and MADRS-S. From these values, linear diagrams to see how they were related to each other were created.

Results

A total of 147/151 women and 39/41 men filled out all of the questions in MADRS-S and BDI-II respectively, as well as received M.I.N.I diagnoses. In total, 186 of 192 patients had complete responses for the MADRS-S and BDI-II and all 192 had received M.I.N.I. diagnoses. No significant differences between men and women regarding the variables BDI-II, MADRS-S, and M.I.N.I. were found. See Figure 1 and Table 1.

Figure 1

Flow-chart over the inclusion, where 33 persons were excluded from the study and further six persons missed one or more questions of MADRS-S and/or BDI-II.

Table 1

Age, Socioeconomic Characteristics, Lifestyle, M.I.N.I, BDI-II and the MADRS-S Scores of the Participants Included in the Study

Characteristics	Men	Women	Total	p-value
	n=41	n=151	n=192
Age				0.49
Mean (SD)	43.7 (9.4)	42.4 (10.3)	42.7 (10.1)
Occupation Class, n (%)				0.78
Senior officials	15 (42.9)	62 (44.3)	77 (44.0)
Lower officials	6 (17.1)	30 (21.4)	36 (20.6)
Manual/students	14 (40.0)	48 (34.3)	62 (35.4)
Employment Level, n (%)				0.01
Full-time	31(91.2)	99 (70.7)	130 (74.7)
Other (25–75%)	3 (8.8)	41 (29.3)	44 (25.3)
Marital Status, n (%)				0.90
Cohabiting	30 (73.2)	112 (74.2)	142 (74.0)
Single	11 (26.8)	39 (25.8)	50 (26.0)
Born				0.02
Nordic country, n (%)	31 (75.6)	135 (90.0)	166 (86.9)
Outside Nordic country, n (%)	10 (24.4)	15 (10.0)	25 (13.1)
Educational Level, n (%)				0.94
Primary education	3 (7.3)	9 (6.0)	12 (6.3)
Secondary education	20 (48.8)	77 (51.0)	97 (50.5)
University or college	18 (43.9)	65 (43.0)	83 (43.2)
Physical activity leisure time, n (%)				<0.001
At least 4 hrs/week	27 (65.9)	134 (89.3)	161 (84.3)
Never	14 (34.1)	16 (10.7)	30 (15.7)
Smoking, n (%)				0.18
Yes	6 (14.6)	37 (24.5)	43 (22.4)
No	35 (85.4)	114 (75.5)	149 (77.6)
Alcohol, n (%)				0.96
>Once a week	6 (15.0)	22 (14.7)	28 (14.7)
Never/once a month	34 (85.0)	128 (85.3)	162 (85.3)
Current sick leave, n (%)				0.48
Yes	35 (85.4)	134 (89.3)	169 (88.5)
No	6 (14.6)	16 (10.7)	22 (11.5)
Depression M.I.N.I., n (%)				0.79
Yes	39 (95.1)	142 (94.0)	181 (94.3)
No	2 (4.9)	9 (6.0)	11 (5.7)
MADRS-S mean (SD)	21.9 (8.9)	20.3 (7.9)	20.6 (8.1)	0.28
BDI-II mean (SD)	23.9 (10.0)	23.2 (9.7)	23.3 (9.7)	0.70

Note: P-value tests the difference between men and women.

There were statistically significant differences between men and women concerning employment level, physical activity, and ethnicity. Women had higher frequency of physical activity (p < 0.01) and full-time working (p= 0.014) compared to men. See Table 1.

The analysis showed that153/186 patients were identified by MADRS-S and M.I.N.I. and 152/186 patients were identified by BDI-II and M.I.N.I. as having depression. There was a significant statistical association between MADRS-S and M.I.N.I. (p= 0.027). The same analysis between BDI-II and M.I.N.I. was not statistically significant (p= 0.635). The results of ROC analyses show that MADRS-S total score: AUC= 0.679 with 95% CI (0.521, 0.836), p=0.047 < 0.05 but for BDI-II total score: AUC= 0.663 with 95% CI (0.504, 0.823), p=0.069 >0.05 ie not statistically significant at the 5% level. According to ROC analysis for cutoff limit 12 for MADRS-S, sensitivity = 0.88 and specificity = 0.36, while for BDI-II with cutoff = 13, sensitivity = 0.82 and specificity = 0.18.

Using M.I.N.I. as gold standard showed that about 81.7% of the patients were identified as depressed when using BDI-II, while 82.3% were identified as depressed when using MADRS-S. See Table 2.When using M.I.N.I. as gold standard, almost the same sensitivity was obtained for BDI-II and MADRS-S, but twice as high specificity was obtained for MADRS-S compared to BDI-II. See Table 3. Table 4, Figure 2 and 3 show the NPV and PPV values that were calculated from different prevalences ranging between 10% and 75% for both BDI-II and MADRS-S variables. The lines for both PPV and NPV values were higher for MADRS-S compared to BDI-II.

Table 2

Number and Percentage of Patients Identified with Depression by BDI-II and MADRS-S When Using M.I.N.I. As Gold Standard

M.I.N.I.	BDI-II ≥ 13	BDI-II < 13
	n (%)	n (%)
Yes	152 (81.7)	23 (12.4)
No	9 (4.8)	2 (1.1)
Total		186 (100.0)
M.I.N.I.	MADRS-S ≥ 12	MADRS-S < 12
	n (%)	n (%)
Yes	153 (82.3)	22 (11.8)
No	7 (3.8)	4 (2.2)
Total		186 (100.0)

Note: Depression according to BDI-II >13, MADRS-S >12.

Table 3

Sensitivity and Specificity for BDI-II and MADRS-S, Respectively, with M.I.N.I. Used as Gold Standard

	Sensitivity	Specificity
BDI-II	86.9%	18.2%
MADRS-S	87.4%	36.4%

Table 4

NPV and PPV Values Calculated from Different Prevalences Ranging Between 10% and 75% for BDI-II and MADRS-S Variables

Prevalence % (95% CI)	PPV MADRS-S % (95% CI)	PPV BDI II % (95% CI)	NPV MADRS-S % (95% CI)	NPV BDI II % (95% CI)
10 (6.3, 15.5)	13 (8.7, 18.9)	11 (7.1, 16.6)	96 (91.8, 98.2)	93 (88.1, 96.1)
20 (14.7, 26.6)	26 (20.0, 33.0)	21 (15.5, 27.7)	92 (86.9, 95.3)	85 (78.7, 89.7)
30 (23.6, 37.2)	37 (30.1, 44.4)	31 (24.6, 38.3)	88 (82.2, 92.1)	76 (69.1, 81.8)
40 (33.0, 47.5)	48 (40.7, 55.4)	41 (33.9, 48.5)	82 (75.6, 87.1)	68 (60.7, 74.5)
50 (42.6, 57.4)	58 (50.6, 65.1)	51 (43.6, 58.4)	75 (68.0, 80.9)	58 (50.6, 65.1)
60 (52.6, 67.1)	67 (59.7, 73.6)	61 (53.6, 68.0)	67 (59.7, 73.6)	48 (40.7, 55.4)
70 (62.8, 76.4)	76 (69.1, 81.8)	71 (63.8, 77.3)	56 (48.6, 63.2)	37 (30.1, 44.4)
75 (68.1, 80.9)	80 (73.4, 85.4)	76 (69.1, 81.8)	50 (42.6, 57.4)	32 (25.5, 39.3)

Figure 2

Linear diagram for PPV values calculated from different prevalences ranging between 10% and 75% for BDI-II and MADRS-S variables.

Figure 3

Linear diagram for NPV values calculated from different prevalences ranging between 10% and 75% for BDI-II and MADRS-S variables.

Discussion

Using M.I.N.I. as the gold standard, the MADRS-S and BDI-II instruments were shown to have acceptable sensitivity to identify most patients with depression in the primary care context. The specificity for MADRS-S was higher and showed a significant relationship to the M.I.N.I. However, the specificity for BDI.II was too low and too many individuals without depression were identified as having depression.

Previous research has shown that there was a good correlation between MADRS-S and BDI-II in the primary care context.12 Further, MADRS-S has been shown to have high NPV (95%), and thus is a relevant instrument for excluding depression diagnosis, which is also an important issue in primary care.20

By using M.I.N.I., the most validated and accepted diagnostic instrument for depression and anxiety syndromes in Swedish care, we could compare the self-assessment instruments most used in Swedish primary care: BDI-II and MADRS-S.1 In primary care, a high sensitivity, as well as high specificity of an instrument, is important, but due to the fact that the prevalence of even the most common mental disorders in primary care is not higher than around 5–15%,21 the PPV and NPV of an instrument are more important measures for primary care clinicians. All patients who have depression should be identified, but at the same time, it is equally important to exclude patients who do not have depression. Both BDI-II and MADRS-S showed too low PPV for being useful as diagnostic instruments in primary care, but MADRS-S could be regarded as an instrument useful for excluding depression based on its high NPV.

Also Patient Health Questionnaire-9 (PHQ-9), more commonly used in Great Britain, the Netherlands and Sweden, has recently been shown to be useful for ruling out depression with an NPV of 97% regarding a prevalence of 12% in the primary care setting, which is equivalent to the results obtained for MADRS-S in our study.22

Using the concepts “sensitivity” and “specificity” to assess the reliability of the scales, we found that MADRS-S had higher specificity. By using different prevalences, we tested the usefulness of the scales in primary care. According to this procedure, MADRS-S was performing better, no matter what the prevalence was. Thus, MADRS-S could be the preferred simple self-assessment instrument in primary care. In an earlier study, we have investigated how patients with depression perceived the use of MADRS-S. It was perceived as an instrument that showed the patients, that they really had depression and changes in symptoms over time as well as a help to know that the GP had taken them seriously.23

Methodological Considerations

The study has several strengths. Firstly, our results are consistent with Council on Technology Assessment in Health care which is that MADRS-S is useful for ruling out depression.7,22 MADRS-S is a validated self-assessment instrument that is simple, short, free of charge, and easily understandable, BDI-II is expensive and can only be ordered by doctors and psychologists.

An additional strength of the study is that it was performed within primary care, where the majority (70%) of all patients with depression are taken care of.1 Another strength is that the data were collected by research staff and did not require additional work on the part of the health center staff, which is an important factor in being able to combine research and clinical work.24

There were also limitations. MADRS-S does not catch patients with CMD such as anxiety, stress-related mental illness and fatigue syndrome. According to exclusion criteria, we did not include patients who had difficulty filling in and understanding the forms. Such difficulties could be due to various reasons, such as language problems, cognitive problems, or serious health problems. However, we may have missed a large group of patients purely on the basis of language difficulties, and we cannot assume that the results would be valid for the group of patients with a high prevalence of CMD because of immigrant background.

Conclusion

All in all, with M.I.N.I. as gold standard, MADRS-S performs better than BDI-II, as a self-assessment tool in the primary care context for depression assessment. MADRS-S has a better specificity, a higher PPV and NPV than BDI-II. We, therefore, suggest that MADRS-S can be used in the clinical situation.