The diagnostic validity of depression scales and clinical judgement in the Kurdistan region of Iraq.

We aimed to find the depression rating scale with the greatest accuracy when applied by psychiatrists in Iraqi Kurdistan. We recruited 200 patients with primary depression and 200 controls living in the Kurdistan region of Iraq. The Mini International Neuropsychiatry Inventory (MINI) was used as a gold standard for DSM-IV depression. We also used: the two-item and the nine-item versions of the Patient Health Questionnaire (PHQ2, PHQ9), the Hospital Anxiety and Depression Scale (HADS), the Calgary Depression Scale for Schizophrenia (CDSS) and the Centre for Epidemiological Studies Depression (CES-D) scale. Interviews were performed by psychiatrists who also rated their clinical judgement using the Clinical Global Impression (CGI) scale and other mental health practitioners. All scales and tools performed with high accuracy and reliability. The least accurate tool was the PHQ2; however, with only two items it was efficient. Sensitivity and specificity for all tools were above 90%. Clinicians using the CGI were accurate in their clinical judgement. The CDSS appeared to be the most accurate scale for DSM-IV major depression and the PHQ2 the most efficient. However, only the CDSS appeared to offer an advantage over psychiatrists' judgement.

Üstün et al (2004) estimated that depression is the fourth leading cause of global disease burden. The burden of depression on the healthcare system is equally significant, with an estimated US national annual medical cost of approximately $26 billion in 1990 (Broadhead et al, 1990; Greenberg et al, 1993). The National Comorbidity Survey Replication (NCS-R), conducted with people aged at least 18 years, found a 12-month prevalence of 9.5% for any DSM-IV mood disorder, with 6.7% for major depression and 1.5% for dysthymia (Kessler et al, 2005). A mental health survey in Iraq which was conducted in collaboration with World Health Organization in 2007 showed that ‘anxiety’ was the most common class of disorders (13.8%) and major depressive disorder (MDD) the most common disorder (7.2%) (Alhasnawi et al, 2009; World Health Organization, 2009).

The extensive literature on screening and case-finding for depression has been reviewed elsewhere. Screening for depression has been supported by recommendations from the US Preventive Services Task Force (Agency for Healthcare Research and Quality, 2002), the UK National Institute for Health and Clinical Excellence (2004) (NICE) and the Canadian Task Force on Preventive Health Care (MacMillan et al, 2005). Our aim was to find the tool with the highest accuracy relative to a robust gold standard.

Methods

We recruited 200 patients with primary depression and 200 people without depression living in the Kurdistan region of Iraq. The Mini International Neuropsychiatry Inventory (MINI) was used as a gold standard to define DSM-IV major depression. Ethical approval was granted by the relevant ethical committee for research in Erbil.

Recruitment was undertaken by trained psychiatrists and mental health practitioners working in both out-patient clinics and in the only psychiatric unit of the largest teaching hospital, as well as two health centres which provide out-patient psychiatric services in Erbil. Three trained mental health practitioners administered all the scales after three trained psychiatrists used the Clinical Global Impression (CGI) scale (severity of illness subscale) to evaluate their own clinical judgement based on a full standard psychiatric assessment. The psychiatrists then administered the MINI.

The controls were recruited via random sampling by dividing the city of Erbil into ten regions. The data were collected between April 2009 and March 2010.

A power calculation suggested that, in order to have an 80% chance to detect a 10% difference in sensitivity or specificity, 197.5 patients would be required in each sample. All those who consented were successfully followed up. We excluded patients who were severely unwell. We did not recruit those unable or unwilling to consent. We also excluded those with current substance misuse.

Tools

We used the following scales: the two-item and nine-item versions of the Patient Health Questionnaire (PHQ2, PHQ9), the Hospital Anxiety and Depression Scale (HADS), the Calgary Depression Scale for Schizophrenia (CDSS) and the Centre for Epidemiological Studies Depression (CES-D) scale. We also used the CGI in order to quantify clinical opinion. Scales were administered after the CGI and MINI. We collected reliability data for this approach using Cronbach alpha scores. The CDSS was developed at the University of Calgary and its use has been evaluated for both relapsed and remitted patients with schizophrenia (Addington et al, 1992).

Analysis

We used an ROC analysis for each scale against an interview standard diagnosis of depression based on the MINI. In addition we calculated the optimal sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).

Results

All scales and tools performed with high accuracy compared with the MINI for DSM-IV major depression. The least accurate tool was the PHQ2, but with only two items it was nevertheless very efficient. Sensitivity and specificity for all tools was above 90%. Judging by the area under the curve (Fig. 1), the most accurate scale was the CDSS. All patients who scored positive with the CDSS were correctly classified as having depression.

Fig. 1

ROC plot

Clinicians used the CGI to rate their opinion, blind to the results of the scales. Using the CGI their sensitivity was 97.0% and specificity 99.0%. Only in the case of the CDSS was accuracy better with than without the scale, and then only moderately so and short of statistical significance.

Discussion

We found that all five scales performed well in the hands of trained mental health practitioners. However, we also found that psychiatrists without assistance were accurate when evaluated against the MINI. This study may suggest that diagnostic tools are of limited value in specialist settings, when compared with clinical routine judgement.

Only a handful of studies have previously examined the accuracy of psychiatrists’ clinical judgement. Taiminen et al (2001) compared routine discharge diagnoses based on DSM-IV and best-estimate diagnoses and results from the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) in 116 first-admission patients with psychosis and severe affective disorder. Diagnostic agreement was moderate (kappa 0.51), suggesting frequent errors in routine diagnosis, even when using DSM-IV criteria.

Our results show a high accuracy for psychiatrists in Iraq and also high accuracy of all tested tools. The CDSS appeared to be the optimal scale.

We wish to acknowledge several limitations to this study. First, the CGI and MINI were administered by the same researchers. Also, there was no formal matching of cases and controls. Although we intended to administer all the questions by self-report alone, in practice issues with literacy meant we administered them to some of the patients verbally, with the assistance of a trained interviewer.

In conclusion, we found that the CDSS was the optimal method to diagnose depression; however, we also found the psychiatrists’ opinion alone was very accurate and therefore it is unclear from our sample whether questionnaires would appreciably help clinicians in their diagnoses.