Ba' Pham1, Ebrahim Bagheri2, Patricia Rios1, Asef Pourmasoumi2, Reid C Robson1, Jeremiah Hwee3, Wanrudee Isaranuwatchai4, Nazia Darvesh1, Matthew J Page5, Andrea C Tricco6. 1. Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1W8, Canada. 2. Department of Electrical and Computer Engineering, Ryerson University, 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada. 3. Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th floor, Toronto, Ontario M5T 3M7, Canada. 4. Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1W8, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, ON M5T 3M6, Canada. 5. School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria 3004, Australia. 6. Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1W8, Canada; Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th floor, Toronto, Ontario M5T 3M7, Canada. Electronic address: triccoa@smh.ca.
Abstract
OBJECTIVES: To illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process. STUDY DESIGN AND SETTING: We simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to "discover" process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions. RESULTS: The 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565-6,368) and 20 studies (6-42). The average review completion time was 463 days (range: 289-629) (881 person-hours [range: 243-1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion. CONCLUSION: Event log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.
OBJECTIVES: To illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process. STUDY DESIGN AND SETTING: We simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to "discover" process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions. RESULTS: The 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565-6,368) and 20 studies (6-42). The average review completion time was 463 days (range: 289-629) (881 person-hours [range: 243-1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion. CONCLUSION: Event log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.
Authors: Zhengyi Deng; Kanhua Yin; Yujia Bao; Victor Diego Armengol; Cathy Wang; Ankur Tiwari; Regina Barzilay; Giovanni Parmigiani; Danielle Braun; Kevin S Hughes Journal: JCO Clin Cancer Inform Date: 2019-08
Authors: Ba' Pham; Jelena Jovanovic; Ebrahim Bagheri; Jesmin Antony; Huda Ashoor; Tam T Nguyen; Patricia Rios; Reid Robson; Sonia M Thomas; Jennifer Watt; Sharon E Straus; Andrea C Tricco Journal: Syst Rev Date: 2021-05-26
Authors: Ingrid Arevalo-Rodriguez; Andrea C Tricco; Karen R Steingart; Barbara Nussbaumer-Streit; David Kaunelis; Pablo Alonso-Coello; Susan Baxter; Patrick M Bossuyt; Javier Zamora Journal: Diagn Progn Res Date: 2019-04-04
Authors: Allison Gates; Michelle Gates; Daniel DaRosa; Sarah A Elliott; Jennifer Pillay; Sholeh Rahman; Ben Vandermeer; Lisa Hartling Journal: Syst Rev Date: 2020-11-27