J W Finney1, S C Monahan. 1. Program Evaluation and Resource Center, Veterans Affairs Palo Alto Health Care System, California 94025, USA.
Abstract
OBJECTIVE: This review builds on the innovative research synthesis of Holder and his colleagues, addresses some of the limitations of the box-score approach to assessing treatment effectiveness that they used and provides a second approximation of the cost-effectiveness of treatment for alcoholism. METHOD: For each of 141 comparative treatment studies, we determined whether or not it found at least one statistically significant positive effect on a drinking-related outcome variable for each of the modalities examined in a paired contrast with one other condition. We next calculated the predicted probability of each study yielding at least one statistically significant treatment effect, based on the number of tests for treatment effects conducted. Following that, for each study of a particular treatment modality, the strength of the "weakest competitor" against which the modality had been compared was determined. For each modality, we used the average predicted probability of the relevant studies finding a significant effect and the average effectiveness of the weakest competitor to predict the modality's effectiveness. RESULTS: We calculated an Adjusted Effectiveness Index (AEIn) for each modality, which was the difference between its predicted and actual effectiveness score. Our AEIn results were consistent with those of Holder et al. in suggesting that some of the same modalities appear to be effective or ineffective. Our results differed from their findings with respect to other modalities, however. Using data presented by Holder and his colleagues on the minimum estimated cost of providing different modalities, we offer a second approximation of the modalities' cost-effectiveness. CONCLUSIONS: Overall, we found a smaller range of effectiveness across modalities than did Holder and his colleagues and a nonsignificant relationship between cost and effectiveness. Like Holder et al., we do not believe major treatment provision or funding decisions should be based solely on this type of review.
OBJECTIVE: This review builds on the innovative research synthesis of Holder and his colleagues, addresses some of the limitations of the box-score approach to assessing treatment effectiveness that they used and provides a second approximation of the cost-effectiveness of treatment for alcoholism. METHOD: For each of 141 comparative treatment studies, we determined whether or not it found at least one statistically significant positive effect on a drinking-related outcome variable for each of the modalities examined in a paired contrast with one other condition. We next calculated the predicted probability of each study yielding at least one statistically significant treatment effect, based on the number of tests for treatment effects conducted. Following that, for each study of a particular treatment modality, the strength of the "weakest competitor" against which the modality had been compared was determined. For each modality, we used the average predicted probability of the relevant studies finding a significant effect and the average effectiveness of the weakest competitor to predict the modality's effectiveness. RESULTS: We calculated an Adjusted Effectiveness Index (AEIn) for each modality, which was the difference between its predicted and actual effectiveness score. Our AEIn results were consistent with those of Holder et al. in suggesting that some of the same modalities appear to be effective or ineffective. Our results differed from their findings with respect to other modalities, however. Using data presented by Holder and his colleagues on the minimum estimated cost of providing different modalities, we offer a second approximation of the modalities' cost-effectiveness. CONCLUSIONS: Overall, we found a smaller range of effectiveness across modalities than did Holder and his colleagues and a nonsignificant relationship between cost and effectiveness. Like Holder et al., we do not believe major treatment provision or funding decisions should be based solely on this type of review.