Gina Wildeboer1, Saskia M Kelders2, Julia E W C van Gemert-Pijnen1. 1. University of Twente, Department of Psychology, Health and Technology, Center for eHealth Research, Enschede, The Netherlands. 2. University of Twente, Department of Psychology, Health and Technology, Center for eHealth Research, Enschede, The Netherlands. Electronic address: s.m.kelders@utwente.nl.
Abstract
BACKGROUND: Research has shown that web-based interventions concerning mental health can be effective, although there is a broad range in effect sizes. Why some interventions are more effective than others is not clear. Persuasive technology is one of the aspects which has a positive influence on changing attitude and/or behavior, and can contribute to better outcomes. According to the Persuasive Systems Design Model there are various principles that can be deployed. It is unknown whether the number and combinations of principles used in a web-based intervention affect the effectiveness. Another issue in web-based interventions is adherence. Little is known about the relationship of adherence on the effectiveness of web-based interventions. OBJECTIVE: This study examines whether there is a relationship between the number and combinations of persuasive technology principles used in web-based interventions and the effectiveness. Also the influence of adherence on effectiveness of web-based interventions is investigated. METHODS: This study elaborates on the systematic review by [37] and therefore the articles were derived from that study. Only web-based interventions were included that were intended to be used on more than one occasion and studies were excluded when no information on adherence was provided. 48 interventions targeted at mental health were selected for the current study. A within-group (WG) and between-group (BG) meta-analysis were performed and subsequently subgroup analyses regarding the relationship between the number and combinations of persuasive technology principles and effectiveness. The influence of adherence on the effectiveness was examined through a meta-regression analysis. RESULTS: For the WG meta-analysis 40 treatment groups were included. The BG meta-analysis included 19 studies. The mean pooled effect size in the WG meta-analysis was large and significant (Hedges' g=0.94), while for the BG meta-analysis this was moderate to large and significant (Hedges' g=0.78) in favor of the web-based interventions. With regard to the number of persuasive technology principles, the differences between the effect sizes in the subgroups were significant in the WG subgroup analyses for the total number of principles and for the number of principles in the three categories Primary Task Support, Dialogue Support, and Social Support. In the BG subgroup analyses only the difference in Primary Task Support was significant. An increase in the total number of principles and Dialogue Support principles yielded larger effect sizes in the WG subgroup analysis, indicating that more principles lead to better outcomes. The number of principles in the Primary Task Support (WG and BG) and Social Support (WG) did not show an upward trend but had varying effect sizes. We identified a number of combinations of principles that were more effective, but only in the WG analyses. The association between adherence and effectiveness was not significant. CONCLUSIONS: There is a relationship between the number of persuasive technology principles and the effectiveness of web-based interventions concerning mental health, however this does not always mean that implementing more principles leads to better outcomes. Regarding the combinations of principles, specific principles seemed to work well together (e.g. tunneling and tailoring; reminders and similarity; social learning and comparison), but adding another principle can diminish the effectiveness (e.g. tunneling, tailoring and reduction). In this study, an increase in adherence was not associated with larger effect sizes. The findings of this study can help developers to decide which persuasive principles to include to make web-based interventions more persuasive.
BACKGROUND: Research has shown that web-based interventions concerning mental health can be effective, although there is a broad range in effect sizes. Why some interventions are more effective than others is not clear. Persuasive technology is one of the aspects which has a positive influence on changing attitude and/or behavior, and can contribute to better outcomes. According to the Persuasive Systems Design Model there are various principles that can be deployed. It is unknown whether the number and combinations of principles used in a web-based intervention affect the effectiveness. Another issue in web-based interventions is adherence. Little is known about the relationship of adherence on the effectiveness of web-based interventions. OBJECTIVE: This study examines whether there is a relationship between the number and combinations of persuasive technology principles used in web-based interventions and the effectiveness. Also the influence of adherence on effectiveness of web-based interventions is investigated. METHODS: This study elaborates on the systematic review by [37] and therefore the articles were derived from that study. Only web-based interventions were included that were intended to be used on more than one occasion and studies were excluded when no information on adherence was provided. 48 interventions targeted at mental health were selected for the current study. A within-group (WG) and between-group (BG) meta-analysis were performed and subsequently subgroup analyses regarding the relationship between the number and combinations of persuasive technology principles and effectiveness. The influence of adherence on the effectiveness was examined through a meta-regression analysis. RESULTS: For the WG meta-analysis 40 treatment groups were included. The BG meta-analysis included 19 studies. The mean pooled effect size in the WG meta-analysis was large and significant (Hedges' g=0.94), while for the BG meta-analysis this was moderate to large and significant (Hedges' g=0.78) in favor of the web-based interventions. With regard to the number of persuasive technology principles, the differences between the effect sizes in the subgroups were significant in the WG subgroup analyses for the total number of principles and for the number of principles in the three categories Primary Task Support, Dialogue Support, and Social Support. In the BG subgroup analyses only the difference in Primary Task Support was significant. An increase in the total number of principles and Dialogue Support principles yielded larger effect sizes in the WG subgroup analysis, indicating that more principles lead to better outcomes. The number of principles in the Primary Task Support (WG and BG) and Social Support (WG) did not show an upward trend but had varying effect sizes. We identified a number of combinations of principles that were more effective, but only in the WG analyses. The association between adherence and effectiveness was not significant. CONCLUSIONS: There is a relationship between the number of persuasive technology principles and the effectiveness of web-based interventions concerning mental health, however this does not always mean that implementing more principles leads to better outcomes. Regarding the combinations of principles, specific principles seemed to work well together (e.g. tunneling and tailoring; reminders and similarity; social learning and comparison), but adding another principle can diminish the effectiveness (e.g. tunneling, tailoring and reduction). In this study, an increase in adherence was not associated with larger effect sizes. The findings of this study can help developers to decide which persuasive principles to include to make web-based interventions more persuasive.
Authors: Suzanne Bakken; Sue Marden; S Sonia Arteaga; Lisa Grossman; Alla Keselman; Phuong-Tu Le; Ruth Masterson Creber; Tiffany M Powell-Wiley; Rebecca Schnall; Derrick Tabor; Rina Das; Tilda Farhat Journal: Am J Public Health Date: 2019-01 Impact factor: 9.308
Authors: Marie Bh Yap; Katherine A Lawrence; Ronald M Rapee; Mairead C Cardamone-Breen; Jacqueline Green; Anthony F Jorm Journal: JMIR Ment Health Date: 2017-12-19
Authors: Lori Wozney; Anna Huguet; Kathryn Bennett; Ashley D Radomski; Lisa Hartling; Michele Dyson; Amanda S Newton; Patrick J McGrath Journal: J Med Internet Res Date: 2017-08-09 Impact factor: 5.428
Authors: Aniek J Lentferink; Hilbrand Ke Oldenhuis; Martijn de Groot; Louis Polstra; Hugo Velthuijsen; Julia Ewc van Gemert-Pijnen Journal: J Med Internet Res Date: 2017-08-01 Impact factor: 5.428
Authors: Melinda R Achilles; Melissa Anderson; Sophie H Li; Mirjana Subotic-Kerry; Belinda Parker; Bridianne O'Dea Journal: Digit Health Date: 2020-05-21