Zoltan Dienes1, Simon Coulton2, Nick Heather3. 1. School of Psychology, University of Sussex, Brighton, UK. 2. Centre for Health Services Studies, University of Kent, Canterbury, UK. 3. Department of Psychology, Northumbria University, Newcastle upon Tyne, UK.
Abstract
AIMS: To illustrate how Bayes factors are important for determining the effectiveness of interventions. METHOD: We consider a case where inappropriate conclusions were drawn publicly based on significance testing, namely the SIPS project (Screening and Intervention Programme for Sensible drinking), a pragmatic, cluster-randomized controlled trial in each of two health-care settings and in the criminal justice system. We show how Bayes factors can disambiguate the non-significant findings from the SIPS project and thus determine whether the findings represent evidence of absence or absence of evidence. We show how to model the sort of effects that could be expected, and how to check the robustness of the Bayes factors. RESULTS: The findings from the three SIPS trials taken individually are largely uninformative but, when data from these trials are combined, there is moderate evidence for a null hypothesis (H0) and thus for a lack of effect of brief intervention compared with simple clinical feedback and an alcohol information leaflet (B = 0.24, P = 0.43). CONCLUSION: Scientists who find non-significant results should suspend judgement-unless they calculate a Bayes factor to indicate either that there is evidence for a null hypothesis (H0) over a (well-justified) alternative hypothesis (H1), or that more data are needed.
RCT Entities:
AIMS: To illustrate how Bayes factors are important for determining the effectiveness of interventions. METHOD: We consider a case where inappropriate conclusions were drawn publicly based on significance testing, namely the SIPS project (Screening and Intervention Programme for Sensible drinking), a pragmatic, cluster-randomized controlled trial in each of two health-care settings and in the criminal justice system. We show how Bayes factors can disambiguate the non-significant findings from the SIPS project and thus determine whether the findings represent evidence of absence or absence of evidence. We show how to model the sort of effects that could be expected, and how to check the robustness of the Bayes factors. RESULTS: The findings from the three SIPS trials taken individually are largely uninformative but, when data from these trials are combined, there is moderate evidence for a null hypothesis (H0) and thus for a lack of effect of brief intervention compared with simple clinical feedback and an alcohol information leaflet (B = 0.24, P = 0.43). CONCLUSION: Scientists who find non-significant results should suspend judgement-unless they calculate a Bayes factor to indicate either that there is evidence for a null hypothesis (H0) over a (well-justified) alternative hypothesis (H1), or that more data are needed.
Authors: James E A Hughes; Jamie Ward; Elin Gruffydd; Simon Baron-Cohen; Paula Smith; Carrie Allison; Julia Simner Journal: Mol Autism Date: 2018-10-12 Impact factor: 7.509