Anthony L Vaccarino1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Amir H Kalali1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Pierre Blier1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Susan Gilbert Evans1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Nina Engelhardt1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Jane A Foster1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Benicio N Frey1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, John H Greist1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Kenneth A Kobak1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Raymond W Lam1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Glenda MacQueen1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Roumen Milev1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Daniel J Müller1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Sagar V Parikh1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Franca M Placenza1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Sakina J Rizvi1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Susan Rotzinger1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, David V Sheehan1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Terrence Sills1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Claudio N Soares1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Gustavo Turecki1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Rudolph Uher1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Janet B W Williams1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Sidney H Kennedy1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, Kenneth R Evans1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22. 1. Drs. Vaccarino, Evans and Gilbert Evans are with Indoc Research in Toronto, Ontario, Canada. 2. Dr Kalali is with the International Society for CNS Drug Development in San Diego, California. 3. Dr. Blier is with the Departments of Psychiatry and Cellular & Molecular Medicine at the University of Ottawa Institute of Mental Health Research in Ottawa, Ontario, Canada. 4. Dr. Engelhardt is with Intra-Cellular Therapies in New York, New York. 5. Dr. Foster is with Department of Psychiatry & Behavioural Neurosciences at McMaster University in Hamilton, Ontario, Canada. 6. Dr. Frey is with the Department of Psychiatry and Behavioural Neurosciences at McMaster University and Mood Disorders Program and Women's Health Concerns Clinic, St. Joseph's Healthcare in Hamilton, Ontario, Canada. 7. Dr. Greist is with the Department of Psychiatry at the University of Wisconsin School of Medicine and Public Health in Madison, Wisconsin. 8. Dr. Kobak is with the Center for Telepsychology in Madison, Wisconsin. 9. Dr. Lam is with the Department of Psychiatry at the University of British Columbia in Vancouver, British Columbia, Canada. 10. Dr. MacQueen is with Mathison Centre for Mental Health Research and Education at the Department of Psychiatry, University of Calgary in Calgary, Alberta, Canada. 11. Dr. Milev is with the Department of Psychiatry and the Department of Psychology, and Centre for Neuroscience Studies at Queen's University in Kingston, Ontario, Canada. 12. Dr. Müller is with the Centre for Addiction and Mental Health and the Department of Psychiatry at the University of Toronto in Toronto, Ontario, Canada. Parikh is with the Department of Psychiatry at the University of Michigan in Ann Arbor, Michigan. 13. Dr. Placenza is with the Department of Psychiatry at the Krembil Research Centre, University Health Network, University of Toronto in Toronto, Ontario, Canada. 14. Dr. Rizvi is with Arthur Sommer Rotenberg Program for Suicide and Depression Studies at St. Michael's Hospital in Toronto, Ontario, Canada. 15. Dr. Rotzinger is with St. Michael's Hospital (Unity Health Toronto) and Department of Psychiatry at the University of Toronto in Toronto, Ontario, Canada. 16. Dr. Sheehan is with the University of South Florida College of Medicine in Tampa, Florida. 17. Dr. Sills was with OCBN in Toronto, Ontario, Canada. 18. Dr. Soares is with the Department of Psychiatry at Queen's University and Providence Care Hospital in Kingston, Ontario, Canada. 19. Dr. Turecki is with the Department of Psychiatry at McGill University in Montreal, Quebec, Canada. 20. Dr. Uher is with the Department of Psychiatry at Dalhousie University in Halifax, Nova Scotia, Canada. 21. Dr. Williams is with the Departments of Psychiatry and Neurology at Columbia University in New York, New York. 22. Dr. Kennedy is with the Department of Psychiatry at the University of Toronto and Centre for Depression and Suicide, St. Michael's Hospital in Toronto, Ontario, Canada.
Abstract
Objective: The goal of the Depression Inventory Development (DID) project is to develop a comprehensive and psychometrically sound rating scale for major depressive disorder (MDD) that reflects current diagnostic criteria and conceptualizations of depression. We report here the evaluation of the current DID item bank using Classical Test Theory (CTT), Item Response Theory (IRT) and Rasch Measurement Theory (RMT). Methods: The present study was part of a larger multisite, open-label study conducted by the Canadian Biomarker Integration Network in Depression (ClinicalTrials.gov: NCT01655706). Trained raters administered the 32 DID items at each of two visits (MDD: baseline, n=211 and Week 8, n=177; healthy participants: baseline, n=112 and Week 8, n=104). The DID's "grid" structure operationalizes intensity and frequency of each item, with clear symptom definitions and a structured interview guide, with the current iteration assessing symptoms related to anhedonia, cognition, fatigue, general malaise, motivation, anxiety, negative thinking, pain, and appetite. Participants were also administered the Montgomery- Åsberg Depression Rating Scale (MADRS) and Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR) that allowed DID items to be evaluated against existing "benchmark" items. CTT was used to assess data quality/reliability (i.e., missing data, skewness, scoring frequency, internal consistency), IRT to assess individual item performance by modelling an item's ability to discriminate levels of depressive severity (as assessed by the MADRS), and RMT to assess how the items perform together as a scale to capture a range of depressive severity (item targeting). These analyses together provided empirical evidence to base decisions on which DID items to remove, modify, or advance. Results: Of the 32 DID items evaluated, eight items were identified by CTT as problematic, displaying low variability in the range of responses, floor effects, and/or skewness; and four items were identified by IRT to show poor discriminative properties that would limit their clinical utility. Five additional items were deemed to be redundant. The remaining 15 DID items all fit the Rasch model, with person and item difficulty estimates indicating satisfactory item targeting, with lower precision in participants with mild levels of depression. These 15 DID items also showed good internal consistency (alpha=0.95 and inter-item correlations ranging from r=0.49 to r=0.84) and all items were sensitive to change following antidepressant treatment (baseline vs. Week 8). RMT revealed problematic item targeting for the MADRS and QIDSSR, including an absence of MADRS items targeting participants with mild/moderate depression and an absence of QIDS-SR items targeting participants with mild or severe depression. Conclusion: The present study applied CTT, IRT, and RMT to assess the measurement properties of the DID items and identify those that should be advanced, modified, or removed. Of the 32 items evaluated, 15 items showed good measurement properties. These items (along with previously evaluated items) will provide the basis for validation of a penultimate DID scale assessing anhedonia, cognitive slowing, concentration, executive function, recent memory, drive, emotional fatigue, guilt, self-esteem, hopelessness, tension, rumination, irritability, reduced appetite, insomnia, sadness, worry, suicidality, and depressed mood. The strategies adopted by the DID process provide a framework for rating scale development and validation.
Objective: The goal of the Depression Inventory Development (DID) project is to develop a comprehensive and psychometrically sound rating scale for major depressive disorder (MDD) that reflects current diagnostic criteria and conceptualizations of depression. We report here the evaluation of the current DID item bank using Classical Test Theory (CTT), Item Response Theory (IRT) and Rasch Measurement Theory (RMT). Methods: The present study was part of a larger multisite, open-label study conducted by the Canadian Biomarker Integration Network in Depression (ClinicalTrials.gov: NCT01655706). Trained raters administered the 32 DID items at each of two visits (MDD: baseline, n=211 and Week 8, n=177; healthy participants: baseline, n=112 and Week 8, n=104). The DID's "grid" structure operationalizes intensity and frequency of each item, with clear symptom definitions and a structured interview guide, with the current iteration assessing symptoms related to anhedonia, cognition, fatigue, general malaise, motivation, anxiety, negative thinking, pain, and appetite. Participants were also administered the Montgomery- Åsberg Depression Rating Scale (MADRS) and Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR) that allowed DID items to be evaluated against existing "benchmark" items. CTT was used to assess data quality/reliability (i.e., missing data, skewness, scoring frequency, internal consistency), IRT to assess individual item performance by modelling an item's ability to discriminate levels of depressive severity (as assessed by the MADRS), and RMT to assess how the items perform together as a scale to capture a range of depressive severity (item targeting). These analyses together provided empirical evidence to base decisions on which DID items to remove, modify, or advance. Results: Of the 32 DID items evaluated, eight items were identified by CTT as problematic, displaying low variability in the range of responses, floor effects, and/or skewness; and four items were identified by IRT to show poor discriminative properties that would limit their clinical utility. Five additional items were deemed to be redundant. The remaining 15 DID items all fit the Rasch model, with person and item difficulty estimates indicating satisfactory item targeting, with lower precision in participants with mild levels of depression. These 15 DID items also showed good internal consistency (alpha=0.95 and inter-item correlations ranging from r=0.49 to r=0.84) and all items were sensitive to change following antidepressant treatment (baseline vs. Week 8). RMT revealed problematic item targeting for the MADRS and QIDSSR, including an absence of MADRS items targeting participants with mild/moderate depression and an absence of QIDS-SR items targeting participants with mild or severe depression. Conclusion: The present study applied CTT, IRT, and RMT to assess the measurement properties of the DID items and identify those that should be advanced, modified, or removed. Of the 32 items evaluated, 15 items showed good measurement properties. These items (along with previously evaluated items) will provide the basis for validation of a penultimate DID scale assessing anhedonia, cognitive slowing, concentration, executive function, recent memory, drive, emotional fatigue, guilt, self-esteem, hopelessness, tension, rumination, irritability, reduced appetite, insomnia, sadness, worry, suicidality, and depressed mood. The strategies adopted by the DID process provide a framework for rating scale development and validation.
Authors: R Uher; A Farmer; W Maier; M Rietschel; J Hauser; A Marusic; O Mors; A Elkin; R J Williamson; C Schmael; N Henigsberg; J Perez; J Mendlewicz; J G E Janzing; A Zobel; M Skibinska; D Kozel; A S Stamp; M Bajs; A Placentino; M Barreto; P McGuffin; K J Aitchison Journal: Psychol Med Date: 2007-10-09 Impact factor: 7.723
Authors: Janet B W Williams; Kenneth A Kobak; Per Bech; Nina Engelhardt; Ken Evans; Joshua Lipsitz; Jason Olin; Jay Pearson; Amir Kalali Journal: Int Clin Psychopharmacol Date: 2008-05 Impact factor: 1.659
Authors: Anthony L Vaccarino; Derek Beaton; Sandra E Black; Pierre Blier; Farnak Farzan; Elizabeth Finger; Jane A Foster; Morris Freedman; Benicio N Frey; Susan Gilbert Evans; Keith Ho; Mojib Javadi; Sidney H Kennedy; Raymond W Lam; Anthony E Lang; Bianca Lasalandra; Sara Latour; Mario Masellis; Roumen V Milev; Daniel J Müller; Douglas P Munoz; Sagar V Parikh; Franca Placenza; Susan Rotzinger; Claudio N Soares; Alana Sparks; Stephen C Strother; Richard H Swartz; Brian Tan; Maria Carmela Tartaglia; Valerie H Taylor; Elizabeth Theriault; Gustavo Turecki; Rudolf Uher; Lorne Zinman; Kenneth R Evans Journal: Front Psychiatry Date: 2022-02-07 Impact factor: 4.157