Philip Burgess1, Jane Pirkis, Tim Coombs. 1. School of Population Health, University of Queensland, Sumner Park BC, Qld, Australia. p.burgess@uq.edu.au
Abstract
OBJECTIVE: Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing. METHODS: Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006-2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM). RESULTS: The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission-discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively. CONCLUSIONS: In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention.
OBJECTIVE: Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing. METHODS: Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006-2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM). RESULTS: The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission-discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively. CONCLUSIONS: In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention.
Authors: Rivka M de Vries; Rob R Meijer; Vincent van Bruggen; Richard D Morey Journal: Int J Methods Psychiatr Res Date: 2015-10-08 Impact factor: 4.035
Authors: Hans Kortrijk; Barbara Schaefer; Jaap van Weeghel; Cornelis L Mulder; Astrid Kamperman Journal: PLoS One Date: 2019-01-09 Impact factor: 3.240
Authors: Edwin de Beurs; Matthijs Blankers; Philippe Delespaul; Erik van Duijn; Niels Mulder; Annet Nugter; Wilma Swildens; Bea G Tiemens; Jan Theunissen; Arno F A van Voorst; Jaap van Weeghel Journal: BMC Psychiatry Date: 2018-07-13 Impact factor: 3.630