G Miguel Esponda1,2, G K Ryan3, J Eaton3,4, M De Silva5, G Lockwood Estrin6, S Usmani7, L Lee8, J Murphy9, O Qureshi3, T Endale10, M Regan11. 1. Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King's College London, 16 De Crespigny Park, Camberwell, London, SE5 8AB, UK. georgina.miguel_esponda@kcl.ac.uk. 2. ESRC Centre for Society and Mental Health, King's College London, London, UK. georgina.miguel_esponda@kcl.ac.uk. 3. Department of Population Health, London School of Hygiene and Tropical Medicine, London, UK. 4. CBM Global, Cambridge, UK. 5. Department of Population Health, Wellcome Trust, London, UK. 6. Centre for Brain and Cognitive Development, Department of Psychological Sciences, Birkbeck College, University of London, London, UK. 7. Independent Researcher, Minneapolis, MN, USA. 8. Independent Researcher, London, UK. 9. Department of Psychiatry, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada. 10. Department of Counselling and Clinical Psychology, Teachers College, Columbia University, New York, NY, USA. 11. Health Improvement Directorate, Public Health England, London, UK.
Abstract
BACKGROUND: Given the underinvestment in global mental health to-date, it is important to consider how best to maximize the impact of existing investments. Theory of Change (ToC) is increasingly attracting the interest of funders seeking to evaluate their own impact. This is one of four papers investigating Grand Challenges Canada's (GCC's) first global mental health research funding portfolio (2012-2016) using a ToC-driven approach. METHODS: A portfolio-level ToC map was developed through a collaborative process involving GCC grantees and other key stakeholders. Proposed ToC indicators were harmonised with GCC's pre-existing Results-based Management and Accountability Framework to produce a "Core Metrics Framework" of 23 indicators linked to 17 outcomes of the ToC map. For each indicator relevant to their project, the grantee was asked to set a target prior to the start of implementation, then report results at six-month intervals. We used the latest available dataset from all 56 projects in GCC's global mental health funding portfolio to produce a descriptive analysis of projects' characteristics and outcomes related to delivery. RESULTS: 12,999 people were trained to provide services, the majority of whom were lay or other non-specialist health workers. Most projects exceeded their training targets for capacity-building, except for those training lay health workers. Of the 321,933 people screened by GCC-funded projects, 162,915 received treatment. Most projects focused on more than one disorder and exceeded all their targets for screening, diagnosis and treatment. Fewer people than intended were screened for common mental disorders and epilepsy (60% and 54%, respectively), but many more were diagnosed and treated than originally proposed (148% and 174%, respectively). In contrast, the three projects that focused on perinatal depression exceeded screening and diagnosis targets, but only treated 43% of their intended target. CONCLUSIONS: Under- or over-achievement of targets may reflect operational challenges such as high staff turnover, or challenges in setting appropriate targets, for example due to insufficient epidemiological evidence. Differences in delivery outcomes when disaggregated by disorder suggest that these challenges are not universal. We caution implementers, funders and evaluators from taking a one-size-fits all approach and make several recommendations for how to facilitate more in-depth, multi-method evaluation of impact using portfolio-level ToC.
BACKGROUND: Given the underinvestment in global mental health to-date, it is important to consider how best to maximize the impact of existing investments. Theory of Change (ToC) is increasingly attracting the interest of funders seeking to evaluate their own impact. This is one of four papers investigating Grand Challenges Canada's (GCC's) first global mental health research funding portfolio (2012-2016) using a ToC-driven approach. METHODS: A portfolio-level ToC map was developed through a collaborative process involving GCC grantees and other key stakeholders. Proposed ToC indicators were harmonised with GCC's pre-existing Results-based Management and Accountability Framework to produce a "Core Metrics Framework" of 23 indicators linked to 17 outcomes of the ToC map. For each indicator relevant to their project, the grantee was asked to set a target prior to the start of implementation, then report results at six-month intervals. We used the latest available dataset from all 56 projects in GCC's global mental health funding portfolio to produce a descriptive analysis of projects' characteristics and outcomes related to delivery. RESULTS: 12,999 people were trained to provide services, the majority of whom were lay or other non-specialist health workers. Most projects exceeded their training targets for capacity-building, except for those training lay health workers. Of the 321,933 people screened by GCC-funded projects, 162,915 received treatment. Most projects focused on more than one disorder and exceeded all their targets for screening, diagnosis and treatment. Fewer people than intended were screened for common mental disorders and epilepsy (60% and 54%, respectively), but many more were diagnosed and treated than originally proposed (148% and 174%, respectively). In contrast, the three projects that focused on perinatal depression exceeded screening and diagnosis targets, but only treated 43% of their intended target. CONCLUSIONS: Under- or over-achievement of targets may reflect operational challenges such as high staff turnover, or challenges in setting appropriate targets, for example due to insufficient epidemiological evidence. Differences in delivery outcomes when disaggregated by disorder suggest that these challenges are not universal. We caution implementers, funders and evaluators from taking a one-size-fits all approach and make several recommendations for how to facilitate more in-depth, multi-method evaluation of impact using portfolio-level ToC.
Entities:
Keywords:
Global mental health; Implementation; Theory of change
Authors: Vikram Patel; Shekhar Saxena; Crick Lund; Graham Thornicroft; Florence Baingana; Paul Bolton; Dan Chisholm; Pamela Y Collins; Janice L Cooper; Julian Eaton; Helen Herrman; Mohammad M Herzallah; Yueqin Huang; Mark J D Jordans; Arthur Kleinman; Maria Elena Medina-Mora; Ellen Morgan; Unaiza Niaz; Olayinka Omigbodun; Martin Prince; Atif Rahman; Benedetto Saraceno; Bidyut K Sarkar; Mary De Silva; Ilina Singh; Dan J Stein; Charlene Sunkel; JÜrgen UnÜtzer Journal: Lancet Date: 2018-10-09 Impact factor: 79.321
Authors: Arthur Kleinman; Georgia Lockwood Estrin; Shamaila Usmani; Dan Chisholm; Patricio V Marquez; Tim G Evans; Shekhar Saxena Journal: Lancet Date: 2016-06-04 Impact factor: 79.321
Authors: Crick Lund; Mark Tomlinson; Mary De Silva; Abebaw Fekadu; Rahul Shidhaye; Mark Jordans; Inge Petersen; Arvin Bhana; Fred Kigozi; Martin Prince; Graham Thornicroft; Charlotte Hanlon; Ritsuko Kakuma; David McDaid; Shekhar Saxena; Dan Chisholm; Shoba Raja; Sarah Kippen-Wood; Simone Honikman; Lara Fairall; Vikram Patel Journal: PLoS Med Date: 2012-12-27 Impact factor: 11.069
Authors: Erica Breuer; Prasansa Subba; Nagendra Luitel; Mark Jordans; Mary De Silva; Bruno Marchal; Crick Lund Journal: BMJ Glob Health Date: 2018-12-30
Authors: June Larrieta; Georgina Miguel Esponda; Yashi Gandhi; Nikita Simpson; Maurice Biriotti; Anna Kydd; Julian Eaton; Grace Kathryn Ryan Journal: BMJ Glob Health Date: 2022-05
Authors: Jill K Murphy; Pallab K Maulik; Keith Dobson; Ishtar Govia; Raymond W Lam; Candelaria I Mahlke; Annabel S Müller-Stierlin; Ionela Petrea; Carla Aparecida Arena Ventura; Melissa Pearson Journal: Can J Psychiatry Date: 2022-02-18 Impact factor: 5.321
Authors: Gbotemi B Babatunde; André Janse van Rensburg; Arvin Bhana; Inge Petersen Journal: Child Adolesc Psychiatry Ment Health Date: 2022-06-18 Impact factor: 7.494