Thérèse Shaw1, Donna Cross2, Stephen R Zubrick3. 1. Telethon Kids Institute, The University of Western Australia, Perth, Australia therese.shaw@telethonkids.org.au. 2. Telethon Kids Institute, The University of Western Australia, Perth, Australia. 3. Telethon Kids Institute, Faculty of Education, The University of Western Australia, Perth, Australia.
Abstract
BACKGROUND: Involvement in bullying at school is detrimental to students' mental and physical health; however, school antibullying programs have not been found to be uniformly successful. Self-reported frequency of involvement in bullying victimization and perpetration, often used as outcome measures in intervention impact evaluation studies, may be subject to response shift, particularly in intervention conditions. Such differential shifts could lead to biased estimates of intervention effects. OBJECTIVES: This study investigated the presence of reconceptualization, reprioritization, and recalibration response shift, resulting from intervention implementation. METHOD: The study subjects were Grade 8 students (n = 3,382, 53% female) in the 35 schools participating in the Cyber Friendly Schools Project, a longitudinal group-randomized intervention trial. Response shift was assessed by comparing traditional and retrospective pretest measures of bullying involvement, as well as testing for measurement invariance over time in the Forms of Bullying Scale (FBS) using confirmatory factor analyses. RESULTS: No evidence of response shift was found, indicating students' understandings of bullying behavior remained stable over time. These findings also demonstrate the applicability of the FBS in longitudinal studies involving adolescents. CONCLUSION: While response shift was not present in our study, researchers conducting program evaluations in other contexts are advised to consider testing for this potential source of bias in their studies.
RCT Entities:
BACKGROUND: Involvement in bullying at school is detrimental to students' mental and physical health; however, school antibullying programs have not been found to be uniformly successful. Self-reported frequency of involvement in bullying victimization and perpetration, often used as outcome measures in intervention impact evaluation studies, may be subject to response shift, particularly in intervention conditions. Such differential shifts could lead to biased estimates of intervention effects. OBJECTIVES: This study investigated the presence of reconceptualization, reprioritization, and recalibration response shift, resulting from intervention implementation. METHOD: The study subjects were Grade 8 students (n = 3,382, 53% female) in the 35 schools participating in the Cyber Friendly Schools Project, a longitudinal group-randomized intervention trial. Response shift was assessed by comparing traditional and retrospective pretest measures of bullying involvement, as well as testing for measurement invariance over time in the Forms of Bullying Scale (FBS) using confirmatory factor analyses. RESULTS: No evidence of response shift was found, indicating students' understandings of bullying behavior remained stable over time. These findings also demonstrate the applicability of the FBS in longitudinal studies involving adolescents. CONCLUSION: While response shift was not present in our study, researchers conducting program evaluations in other contexts are advised to consider testing for this potential source of bias in their studies.
Authors: Ingrid V E Carlier; Wessel A van Eeden; Kim de Jong; Erik J Giltay; Martijn S van Noorden; Christina van der Feltz-Cornelis; Frans G Zitman; Henk Kelderman; Albert M van Hemert Journal: Int J Methods Psychiatr Res Date: 2019-06-17 Impact factor: 4.035