Grace Yeeun Lee1, Ian Bernard Hickie1, Jo-An Occhipinti1,2, Yun Ju Christine Song1, Adam Skinner1, Salvador Camacho3,4, Kenny Lawson1, Adriane Martin Hilber3,4, Louise Freebairn1,2,5. 1. Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia. 2. Computer Simulation & Advanced Research Technologies (CSART), Sydney, NSW, Australia. 3. Swiss Centre for International Health, Swiss Tropical and Public Health Institute, Basel, Switzerland. 4. University of Basel, Basel, Switzerland. 5. Research School of Population Health, The Australian National University, Canberra, ACT, Australia.
Abstract
INTRODUCTION: Systems modelling and simulation can improve understanding of complex systems to support decision making, better managing system challenges. Advances in technology have facilitated accessibility of modelling by diverse stakeholders, allowing them to engage with and contribute to the development of systems models (participatory modelling). However, despite its increasing applications across a range of disciplines, there is a growing need to improve evaluation efforts to effectively report on the quality, importance, and value of participatory modelling. This paper aims to identify and assess evaluation frameworks, criteria, and/or processes, as well as to synthesize the findings into a comprehensive multi-scale framework for participatory modelling programs. MATERIALS AND METHODS: A scoping review approach was utilized, which involved a systematic literature search via Scopus in consultation with experts to identify and appraise records that described an evaluation framework, criteria, and/or process in the context of participatory modelling. This scoping review is registered with the Open Science Framework. RESULTS: The review identified 11 studies, which varied in evaluation purposes, terminologies, levels of examination, and time points. The review of studies highlighted areas of overlap and opportunities for further development, which prompted the development of a comprehensive multi-scale evaluation framework to assess participatory modelling programs across disciplines and systems modelling methods. The framework consists of four categories (Feasibility, Value, Change/Action, Sustainability) with 30 evaluation criteria, broken down across project-, individual-, group- and system-level impacts. DISCUSSION & CONCLUSION: The presented novel framework brings together a significant knowledge base into a flexible, cross-sectoral evaluation effort that considers the whole participatory modelling process. Developed through the rigorous synthesis of multidisciplinary expertise from existing studies, the application of the framework can provide the opportunity to understand practical future implications such as which aspects are particularly important for policy decisions, community learning, and the ongoing improvement of participatory modelling methods.
INTRODUCTION: Systems modelling and simulation can improve understanding of complex systems to support decision making, better managing system challenges. Advances in technology have facilitated accessibility of modelling by diverse stakeholders, allowing them to engage with and contribute to the development of systems models (participatory modelling). However, despite its increasing applications across a range of disciplines, there is a growing need to improve evaluation efforts to effectively report on the quality, importance, and value of participatory modelling. This paper aims to identify and assess evaluation frameworks, criteria, and/or processes, as well as to synthesize the findings into a comprehensive multi-scale framework for participatory modelling programs. MATERIALS AND METHODS: A scoping review approach was utilized, which involved a systematic literature search via Scopus in consultation with experts to identify and appraise records that described an evaluation framework, criteria, and/or process in the context of participatory modelling. This scoping review is registered with the Open Science Framework. RESULTS: The review identified 11 studies, which varied in evaluation purposes, terminologies, levels of examination, and time points. The review of studies highlighted areas of overlap and opportunities for further development, which prompted the development of a comprehensive multi-scale evaluation framework to assess participatory modelling programs across disciplines and systems modelling methods. The framework consists of four categories (Feasibility, Value, Change/Action, Sustainability) with 30 evaluation criteria, broken down across project-, individual-, group- and system-level impacts. DISCUSSION & CONCLUSION: The presented novel framework brings together a significant knowledge base into a flexible, cross-sectoral evaluation effort that considers the whole participatory modelling process. Developed through the rigorous synthesis of multidisciplinary expertise from existing studies, the application of the framework can provide the opportunity to understand practical future implications such as which aspects are particularly important for policy decisions, community learning, and the ongoing improvement of participatory modelling methods.
Authors: Andrew Page; Jo-An Atkinson; William Campos; Mark Heffernan; Shahana Ferdousi; Adrian Power; Geoff McDonnell; Nereus Maranan; Ian Hickie Journal: Aust N Z J Psychiatry Date: 2018-04-19 Impact factor: 5.744
Authors: Petrônio C L Silva; Paulo V C Batista; Hélder S Lima; Marcos A Alves; Frederico G Guimarães; Rodrigo C P Silva Journal: Chaos Solitons Fractals Date: 2020-07-07 Impact factor: 9.922
Authors: Wilma E Waterlander; Angie Luna Pinzon; Arnoud Verhoeff; Karen den Hertog; Teatske Altenburg; Coosje Dijkstra; Jutka Halberstadt; Roel Hermans; Carry Renders; Jacob Seidell; Amika Singh; Manou Anselma; Vincent Busch; Helga Emke; Emma van den Eynde; Lieke van Houtum; Wilma J Nusselder; Meredith Overman; Susan van de Vlasakker; Tanja Vrijkotte; Erica van den Akker; Stef P J Kremers; Mai Chinapaw; Karien Stronks Journal: Int J Environ Res Public Health Date: 2020-07-08 Impact factor: 3.390