Nicholas J Parr1,2, Christopher M Loan3,2, Emily E Tanner-Smith4,2. 1. U.S. Department of Veterans Affairs Evidence Synthesis Program Coordinating Center, VA Portland Health Care System, 3710 SW U.S. Veterans Hospital Rd., Portland, OR 97239, USA. 2. Prevention Science Institute, University of Oregon, 6217 University of Oregon, Eugene, OR 97403, USA. 3. Department of Educational Methodology, Policy, and Leadership, University of Oregon, 5267 University of Oregon, Eugene, OR 97403, USA. 4. Department of Counseling Psychology and Human Services, University of Oregon, 5251 University of Oregon, Eugene, OR 97403, USA.
Abstract
AIMS: To illustrate a machine learning-based approach for identifying and investigating moderators of alcohol use intervention effects in aggregate-data meta-analysis. METHODS: We illustrated the machine learning technique of random forest modeling using data from an ongoing meta-analysis of brief substance use interventions implemented in general healthcare settings. A subset of 40 trials testing brief alcohol interventions (BAIs) was used; these trials provided 344 estimates of post-intervention effects on participants' alcohol use as well as data on 20 potential moderators of intervention effects. These candidate moderators included characteristics of trial methodology and implementation, intervention design and participant samples. RESULTS: The best-fitting random forest model identified 10 important moderators from the pool of 20 candidate moderators. Meta-regression utilizing the selected moderators found that inclusion of prescriptive advice in a BAI session significantly moderated BAI effects on alcohol use. Observed effects were also significantly moderated by several methodological characteristics of trials, including the type of comparison group used, the overall level of attrition and the strategy used to address missing data. In a meta-regression model that included all candidate moderators, fewer coefficients were found to be significant, indicating that the use of a preliminary data reduction technique to identify only important moderators for inclusion in final analyses may have yielded improved statistical power to detect moderation. CONCLUSIONS: Machine learning methods can be valuable tools for clarifying the influence of trial, intervention and sample characteristics on alcohol use intervention effects, in particular when numerous candidate moderators are available.
AIMS: To illustrate a machine learning-based approach for identifying and investigating moderators of alcohol use intervention effects in aggregate-data meta-analysis. METHODS: We illustrated the machine learning technique of random forest modeling using data from an ongoing meta-analysis of brief substance use interventions implemented in general healthcare settings. A subset of 40 trials testing brief alcohol interventions (BAIs) was used; these trials provided 344 estimates of post-intervention effects on participants' alcohol use as well as data on 20 potential moderators of intervention effects. These candidate moderators included characteristics of trial methodology and implementation, intervention design and participant samples. RESULTS: The best-fitting random forest model identified 10 important moderators from the pool of 20 candidate moderators. Meta-regression utilizing the selected moderators found that inclusion of prescriptive advice in a BAI session significantly moderated BAI effects on alcohol use. Observed effects were also significantly moderated by several methodological characteristics of trials, including the type of comparison group used, the overall level of attrition and the strategy used to address missing data. In a meta-regression model that included all candidate moderators, fewer coefficients were found to be significant, indicating that the use of a preliminary data reduction technique to identify only important moderators for inclusion in final analyses may have yielded improved statistical power to detect moderation. CONCLUSIONS: Machine learning methods can be valuable tools for clarifying the influence of trial, intervention and sample characteristics on alcohol use intervention effects, in particular when numerous candidate moderators are available.