Evaluation of a complex, population-based injury claims management intervention for improving injury outcomes: study protocol.

INTRODUCTION: Injuries resulting from road traffic crashes are a substantial cause of disability and death worldwide. Injured persons receiving compensation have poorer recovery and return to work than those with non-compensable injury. Case or claims management is a critical component of injury compensation systems, and there is now evidence that claims management can have powerful positive impacts on recovery, but can also impede recovery or exacerbate mental health concerns in some injured people. This study seeks to evaluate the impact of a population-based injury claims management intervention in the State of Victoria, Australia, on the health of those injured in motor vehicle crashes, their experience of the compensation process, and the financial viability of the compensation system. METHODS AND ANALYSIS: Evaluation of this complex intervention involves a series of linked but stand-alone research projects to assess the anticipated process changes, impacts and outcomes of the intervention over a 5-year time frame. Linkage and analysis of routine administrative and health system data is supplemented with a series of primary studies collecting new information. Additionally, a series of 'action' research projects will be undertaken to inform the implementation of the intervention. A program logic model designed by the state government Transport Accident Commission in conjunction with the research team provides the evaluation framework. ETHICS AND DISSEMINATION: Relatively few studies have comprehensively examined the impact of compensation system processes on the health of injured persons, their satisfaction with systems processes, and impacts on the financial performance of the compensation scheme itself. The wholesale, population-based transformation of an injury claims management model is a rare opportunity to document impacts of system-level policy change on outcomes of injured persons. Findings will contribute to the evidence base of information on the public health effects of injury claims management policy and practice. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Evaluation of a population-based policy and practice change that affects all people injured in a motor vehicle crash in the State of Victoria, Australia.

Use of a co-designed program logic model as the framework for the evaluation methodology.

Inability to randomise participants to ‘treatment’ and comparison conditions may limit the ability to attribute causation to specific findings.

Use of multiple data sources including population-based registries, qualitative data and prospectively collected data provides rich detail on the process, impact and outcomes of the policy and practice change.

Introduction

Burden of transport injury

Injuries resulting from road traffic crashes are a substantial cause of disability and death worldwide.1 2 Approximately 50–60% of major trauma hospital admissions are the result of transport crashes in western countries,3 and road transport-related injuries account for more than 10% of all hospitalisations due to injury,4 with head, neck and thoracic injuries most frequently reported.5 Transport injuries commonly result in significant disability, including physical disability,6 changes in mental state and reduced quality of life1 and delayed return to work.7 8

Although the injured person is normally the person most directly affected, injuries can also have long-term impacts on family members, coworkers, healthcare providers, employers and a wide variety of other individuals and groups. Aside from the direct physical impact of injury, individuals often lose financial independence through loss of earnings and dependence on others,9 develop or exacerbate mental health issues,10 and place greater strain on personal relationships.11 Families are also affected, with disruptions to home life a frequent outcome following injury.12 At a community level, injury manifests in a change to society's productivity and competitiveness, greater use of social services, and increased demand on public and private resources.13

Injury claims management

In many countries, treatment and rehabilitation following transport accidents is provided by government compensation or insurance systems. In such systems, claims managers (also called case managers or adjudicators) have a critical role. Claims managers are the primary interface between the compensation system, the injured person and others involved in the rehabilitation of the injured person (eg, healthcare providers, employers). Claims managers have a critical decision-making role regarding payments for treatment, income replacement, and provision of healthcare and other services to the injured person.

A recent qualitative study in Australia demonstrated that claims managers experience their role as highly stressful, with multiple competing priorities from within their own organisations and from external parties (personal communication, S Newnam, 2015). These findings are consistent with anecdotal reports of a high level of staff turnover in the industry (up to 25% per annum), and challenges in embedding good practice and appropriate training and education for those at the ‘front line’.

There is now evidence that claims and case management can have powerful positive impacts on recovery,14 but that it can also impede recovery or lead to exacerbation of mental health concerns in some injured people.15 16 Effective claims management is also considered to be critical to maintaining the financial viability of injury compensation systems, and ensuring a positive experience for injured persons and employers engaging with the system. Despite the potential importance of claims management to the injury recovery process, there is relatively little published research evidence in the field. Very few studies have examined the impact of changes in claims handling processes on injury recovery, and to the best of our knowledge, there has been only one prior published study involving transport accidents.17

Aims

The primary aim of this study is to evaluate the impact of a population-based injury claims management intervention in the State of Victoria, Australia, on the health outcomes of those injured in motor vehicle crashes, their experience of the compensation process, and the financial viability of the compensation system. A secondary aim is to provide research evidence to support the development and refinement of the intervention during its implementation.

Methods and analysis

Context

In Victoria, those injured in land-based transport crashes involving a car, motorcycle, tram, bus or train are eligible to claim compensation for treatment, income replacement, rehabilitation and long-term support services via the Transport Accident Commission (TAC), regardless of fault. Additionally, the TAC provides compensation for injury and death for individuals travelling in a Victorian-registered motor vehicle in other Australian states. Injuries and death occurring on the road but not involving a motorised vehicle (eg, a collision between a pedal cyclist and a pedestrian) are not eligible for compensation. The State of Victoria had a population of approximately 5.6 million residents as at December 2010.

The TAC pays for healthcare, disability and other services on behalf of its clients, but is not a provider of such services. In the 2009/2010 financial year, the TAC spent $A903 million on services provided to clients. The organisation accepts approximately 16 000 new claims per annum and at any one time has approximately 40 000 claims under management.18 Since its introduction in 1987, the TAC scheme has accepted approximately 480 000 compensation claims. Clients of the TAC (ie, injured persons) have a statutory right to choose the provider of their healthcare and other services, and the TAC reimburses the provider on the client's behalf. There are limits imposed on some services as described in the TAC treatment payment and other policy (available via http://www.tac.vic.gov.au). Those severely injured and who are ‘not at fault’ can make a claim for common law lump sum payments for damages beyond those provided under the no-fault scheme.

Approximately one-third of TAC clients are admitted to hospital within 28 days of their transport accident.19 The remaining two-thirds either attend an emergency department but are not admitted, or visit their general practitioner. Those admitted to hospital utilise a median of 19 medical and paramedical (allied health) services in the 12-month period postdischarge19

Previous research has suggested that TAC clients have poorer outcomes than those with matched but non-compensable injuries.20 Specifically, this study identified that TAC clients with severe orthopaedic injury were less likely to return to work, reported higher levels of pain and poorer quality of life than their non-compensable peers. This finding provided an important impetus for the review of the TAC claims management model and, in part, led to the development of the intervention that is the focus of this evaluation.

Injury claims management intervention

The TAC injury claims management system has evolved over the 25 years that the organisation has been in operation. A review of claims operations conducted in 2009 identified numerous opportunities for improvement including

Better aligning of case load with case complexity and claims manager experience.

Implementing a data-driven segmentation system to reduce manual claims handling, and assisting in the identification of complex cases.

Minimising the movement of cases between claims managers.

Moving to a model of person-centred care for clients with complex and long-term care needs.

It was proposed that realisation of these opportunities would result in improved client satisfaction, improved client health and well-being, and reductions in the costs of claims management. This review recommended a major redesign of the claims management model. The logic model underpinning the proposed intervention is described in figure 1.

Figure 1

Logic model of expected process changes, impacts and outcomes associated with case management intervention.

The model has two major elements, with different management activities being undertaken for very severe injury (‘Independence’ branch), and very mild to moderate injury (‘Recovery’ branch). Approximately 95% of all clients are enrolled via the Recovery branch, and these incur approximately 50% of annual claims costs. The injuries managed by this division range from minor musculoskeletal symptoms to relatively serious injuries such as orthopaedic trauma and mild and moderate traumatic brain injury. The remaining 5% of cases are enrolled in the Independence branch, and account for the remaining 50% of claims costs. These cases include severe traumatic brain injury, paraplegia, quadriplegia, multiple amputations and some severe cases of burns.

The intervention is primarily internally focussed, with changes to processes including introduction of data-based algorithms to aid more accurate segmentation of cases according to their complexity, to ensure that the case is matched to the most appropriate claims management team. Other changes included substantial training and education for frontline staff, and in the Independence branch, the introduction of person-centred case planning to align the injured person's goals with service provision. In both Recovery and Independence, new positions were created and teams completely restructured. Underpinning the intervention was a change in philosophy that the TAC move from a ‘passive’ payer of services to an ‘active’ participant in the postinjury recovery process.

These process changes are expected to lead to a range of impacts in the short to medium term, including earlier and more appropriate healthcare and disability service delivery, improvement in engagement of healthcare and disability service providers with the compensation organisation and with clients, improvements in claim manager capability, earlier and more sustainable return to work among clients without serious life-long injuries, and a consequent reduction in lodgement of common law claims.

In turn, these impacts are expected to lead to meaningful changes in the organisation’s key performance indicators, including improvements in health and well-being of clients in the Recovery branch, improvements in the independence and social inclusion of clients in the Independence branch, and improvements in the client's experience of the claims management process. Changes in health, well-being and independence are anticipated to lead to a reduction in need for service provision, and a consequent reduction in case costs and long-term liabilities.

The intervention is being implemented in a staged approach. Stage 1 of the Recovery model was implemented in October 2010, and involved reallocating all existing clients to an appropriate team based on case complexity, and reallocation of claims managers to new functional roles. Stage 2 of Recovery occurred in October 2011, and involved providing claims managers with education, training and practical tools to assist with management of the claim. Education and training programmes were designed to increase the skills of case managers in their engagement with injured people, healthcare and disability providers. This included, for instance, training in motivational interviewing techniques. A range of practical tools were introduced including data-based claims triage tools, changes to information technology systems to promote information capture and usage, and information. The research team was involved in developing and/or evaluating some of these via the action research projects (described below).

Stage 1 of the Independence model was implemented in April 2011 and involved development of client-centred case plans for all new claims, and development of a new role focussed on case plan development. Stage 2 was implemented in April 2012 involved reallocation of existing claims (the majority of long-term severely injured cases) to the new model. This is outlined in figure 2.

Figure 2

Timeline of case management intervention components.

Evaluation framework and strategy

The evaluation study focuses on assessing changes to process occurring as a consequence of the intervention implementation, the short-to–medium term impacts of those process changes, and the long-term outcomes. The process evaluation will occur predominantly over years 1–3 (2010–2012), during the staged implementation of the major elements of the intervention. The impact evaluation will occur over years 2–4 (2011–2014), and the outcome evaluation will occur in years 4 and 5 (2014 and 2015). This latter point is premised on a sufficient number of new clients being managed under the new model and, moreover, that an appropriate length of time is available to observe the claim activity and health outcomes of these clients.

Routinely collected data

Several existing data sources will be utilised in the course of this evaluation. A summary of these data sources and their design is provided in table 1.

Table 1

Routinely collected data

Source	Population, data collected and design
TAC claims database	Population: all TAC clients. Approximately 16 000 new clients per annum
	Data collected: demographic, injury, accident details, health and disability service payments, claims management data
	Design: population-based registry
TAC client surveys	Population: stratified random sample of approximately 1500 TAC clients per annum
	Data collected: client self-rated mental and physical health, satisfaction, pain, return-to-work outcomes
	Design: telephone-administered semiannual cross-sectional survey
Victorian State Trauma Registry	Population: all cases of hospitalised major trauma (Injury Severity Score >12) in the State
	Data collected: prehospital (ambulance), clinical, surgical, demographic information at baseline, plus self-reported health-related quality of life, pain, return to work and functional status at 6, 12 and 24 months postinjury
	Design: population-based registry
Victorian Orthopaedic Trauma Registry	Population: all cases of orthopaedic trauma admitted for >24 h to four hospitals in Victoria
	Data collected: clinical, surgical, demographic information at baseline plus self-reported health-related quality of life, pain, return to work and functional status at 6, 12 and 24 months postinjury
	Design: sentinel site registry

TAC, Transport Accident Commission.

The TAC maintains a claims management database that contains case-level data for every claim received by the organisation since its inception in 1987. Each record contains information necessary for the management of the compensation claim, including accident, demographic, injury details, and detailed records of payments for health and other services. A de-identified data set will be obtained for analysis. We have previously published information on health service utilisation following transport injury using this data set.19 21 22

Additionally, the TAC conducts annual surveys of large cross-sections of its clients. These include telephone-based client experience surveys which have been conducted since 2000 on a semiannual basis. These surveys ask the clients to rate their experiences of interacting with the TAC, and rate their level of satisfaction with service provided. Since 2008 (2 years prior to the implementation of the claims management intervention), the TAC has also collected self-rated health outcome data from its clients using tools such as the SF-12, numerical pain rating scales, the Orebro Musculoskeletal Pain Questionnaire, and return-to-work outcomes also via telephone administration. We have recently examined self-rated patterns of return to work following transport injury using this data.23

The State of Victoria also has a population-based trauma registry established in 2000. The Victorian State Trauma Registry (VSTR) collects data about all major trauma patients in the State with major trauma defined as any of the following:1 death after injury;2 an Injury Severity Score >12;3 an intensive care unit stay >24 h, requiring mechanical ventilation; and4 urgent surgery. Self-reported functional, quality of life and return-to-work outcomes are collected up to 2 years postinjury using validated measures with over 85% follow-up routinely achieved. Approximately 45–50% of cases on the VSTR result from transport crashes, and are thus TAC compensable clients. The methods for this registry have been described in detail elsewhere.24

Additionally, a sentinel site registry for orthopaedic trauma also exists in the State. The Victorian Orthopaedic Trauma Outcomes Registry (VOTOR) was established in 2003, and collects data about all orthopaedic trauma requiring emergency admission (>24 h) to the two major adult trauma centres, one regional trauma centre and one metropolitan trauma centre for an orthopaedic (bone and/or soft tissue) injury. Approximately 35% of major trauma centre cases on VOTOR result from transport crashes and are thus TAC compensable.20 The methods for this registry have also been described elsewhere, and the VSTR and VOTOR share a common follow-up methodology.24

In addition to separate analyses, the claims and health system data sets will be linked, and analyses of the linked data set undertaken to achieve the aims of the evaluation. Institutional ethics approval has been granted for use of the data sets and their linkage (Monash University Human Research Ethics Committee approvals # CF09/3150–2009001727 and # CF11/1604: 2011000895; Department of Health Human Research Ethics Committee approval #11/14).

Process, impact and outcome evaluation

Five studies will be undertaken to address specific research questions related to the process, impact and outcomes of the claims management intervention. A summary of these studies, their design and aims is provided in table 2.

Table 2

Process, impact, outcome studies

Study	Aim, design and data collection
Staff surveys	Aims: to examine case managers experiences of the case intervention. To determine changes in case managers skills and capability
	Design: repeat cross-sectional internet-based survey of sample of case managers
	Data collection: baseline April 2010, follow-up survey May 2011 and August 2012
Common law lodgement study	Aim: to determine factors (including injured person characteristics) associated with filing a common law claim
	Design: (1) repeat cross-sectional analysis of claims management database before and after claims management changes; (2) content analysis of claims files
	Data collection: (1) all accepted claims lodged between Jan 2006 and Dec 2012 extracted for analysis; (2) 20 claim files from period prior to intervention and 20 from period after intervention extracted for analysis
Claims management intervention data linkage study	Aim: to determine the impact of the intervention on the self-rated health, return to work of TAC clients with major trauma and orthopaedic trauma. To determine the impact of the intervention on claim costs
	Design: analysis of linked outcomes registry and claims management databases using a segmented regression approach
	Data collection: all accepted claims lodged between 2007 and 2014 extracted for analysis
Qualitative client study	Aim: to examine the injured persons experience engaging with the compensation system before and after the intervention
	Design: qualitative study of seriously injured clients injured before and after intervention
	Data collection: semistructured interviews focused on experiences of dealing with TAC
Case planning study	Aims: to examine health and disability provider experiences of the case planning process. To understand the similarities and differences between TAC planning processes and those of other ‘like’ organisations
	Design: qualitative study of health and disability providers involved in case planning. Series of case studies of case planning in lifetime care focussed insurance, health and disability organisations
	Data collection: semistructured interviews focussed on experiences of engaging in the TAC planning processes. Interviews, questionnaires, and document analysis with ‘like’ organisations

TAC, Transport Accident Commission.

Implementation of the injury claims management model represents a substantial organisational change, with claims staff required to take on new roles and responsibilities, undertake new training, and change the intensity and content of their interactions with clients. A series of surveys of claims managers will assess the case managers’ experience of the intervention process and self-rated changes in skills and capability. Surveys of Independence branch staff will examine their ability to align health and disability service provision with an injured person's goals, and their perception of the engagement of health and disability providers with the case-planning process.

In the Victorian transport accident system, injured persons can lodge a claim for award of additional damages under common law (beyond the no-fault benefits paid routinely by the system) if the accident was not due to their fault, and their injury meets a severity threshold. One objective of the intervention is to reduce the number of common law lodgements. Analysis of the case management database will be undertaken to determine the impact of the Recovery model on lodgement of common law claims, compared with persons injured before the introduction of the model. A concurrent content analysis of a stratified random sample of case files will provide insight into factors leading to common law lodgement before and after the intervention.

In the Independence branch, the engagement of the injured person, family/carers, health and disability providers in the case planning process will be assessed via a series of qualitative interviews and focus groups. An additional component will document case planning processes in a small number of Australian and international organisations that fund lifetime healthcare and disability support services for seriously injured people, and compare these processes to those of the TAC.

The outcomes of the intervention will be assessed by linking the claims management database to the VSTR and VOTOR. Analysis of health-related quality of life, pain scores, functional and return to work outcomes up to 24 months postinjury will be undertaken in cohorts injured before and after the introduction of the claims management intervention.

Injured person's experiences of claims management will be assessed in two ways. First, by analysis of routinely collected TAC client satisfaction survey data collected before and after the intervention. Second, by conducting in-depth qualitative study of cohorts of seriously injured clients whose injury occurred before and after the intervention.

Action research projects

A further four projects will be undertaken during implementation of the intervention to assist with refinement of the intervention, and provide feedback to the organisation that can be incorporated into the models and lead to improved practice. These studies will assume an action-research methodology where elements of the intervention are defined or refined through a process of planning, action and evaluation.25 These projects are participative in nature, as both the research team and representatives of the organisation are involved in their definition, conduct, interpretation and translation into practice or policy. The intervention is dynamic in that it is expected to evolve throughout its implementation. There are many factors influencing this evolution including the internal and external operating environment of the organisation. The action research projects are another source of information that will influence implementation, and thus there is the potential for these studies to have an impact on the results of the primary studies described above. These studies address the second aim of the project, and are summarised in table 3.

Table 3

Action research projects

Study	Aim, design and data collection
Remote mental health interventions	Aim: to determine if remotely delivered mental health interventions are feasible within a personal injury case management environment
	Design: systematic literature review
	Data collection: English language studies published between January 2001 and December 2011
Client conversational tool	Aim: to determine the predictive validity of a claims management screening tool for mental health, persistent pain and return to work outcomes
	Design: multivariate regression analysis of prospectively collected data from a cohort of TAC clients extracted from claims management database
	Data collection: all accepted claims lodged between April 2010 and June 2011
Return-to-work predictive model	Aim: to develop an algorithm that predicts return to work status at 6 months postinjury using routinely collected claims data
	Design: multivariate regression analysis of retrospective cohort of TAC clients extracted from case management database
	Data collection: all accepted claims lodged between January 2005 and December 2009
Case planning in a compensation setting	Aim: to develop best practice principles of case management within a personal injury compensation environment
	Design: narrative literature review
	Data collection: English language studies published between 1985 and 2012

TAC, Transport Accident Commission.

A narrative literature review will be conducted to document best practice principles of case management and to identify how these might be applied/modified in a personal injury compensation setting. This review will be used to inform the design of education and training for case managers.

As described in the logic model, a feature of the Recovery model is earlier and more accurate identification of clients with complex needs. One stage of this segmentation process involves administration by case managers of a screening questionnaire early in the life of the claim, in order to identify clients who may need further assistance with return to work, pain, or mental health issues. The ability of the prediction tool, developed using logistic regression modelling, to predict claim destination will be assessed against prespecified performance criteria, as per the approach described by Wolfe et al.26 This analysis will also provide feedback on the delivery format and administration of the tool. A separate analysis will develop a predictive model for use on claim acceptance, using data available at claim lodgement, to identify clients who are less likely to return to work (the Return to Work Predictive Model).

It is envisaged that the second stage of the Recovery intervention will involve the implementation of a remote (telephone or internet based) intervention to prevent the onset of mental health conditions such as depression, anxiety and post-traumatic stress in clients with complex needs. A systematic review will examine the effectiveness of remote interventions in preventing mental health conditions following traumatic injury. This review will be used to guide development of a remote intervention that will be trialled as part of a separate research initiative.

Ethics and dissemination

Effective and efficient dissemination of research findings, and adoption of the research by the compensation organisation to affect policy and practice change is a critical component of this evaluation. This is particularly the case for the action-research projects which aim to provide timely feedback to the organisation so that processes may be examined and, if necessary, altered during the implementation phase. The project will develop a number of mechanisms for promoting knowledge translation and engagement between the evaluation team and the organisation. These will occur through collaborative project planning, governance and reporting activities. A project steering committee has been established comprising the chief investigators and key personnel responsible for intervention design and implementation from within the TAC. This committee will meet on a quarterly basis throughout the project and will be the critical liaison point. The committee provides a mechanism for two-way information flow (data and information regarding the claims models from the TAC to the research team and research output and reports from the research team to the TAC). Additionally, senior managers within the TAC have been appointed as ‘business owners’ for components of the project. These individuals act as primary points of contact for the research team. Finally, a series of interactive results presentations have been organised to engage the organisation more broadly regarding the output of the evaluation and implications for policy and practice.

There are relatively few studies that have comprehensively examined the impact of compensation system processes on the health of injured persons, their satisfaction and impacts on the financial performance of the compensation scheme itself. The wholesale transformation of the claims management model undertaken by the TAC is a rare opportunity to document impacts of comprehensive system-level policy change on the outcomes of injured persons.

In evaluating the impact and outcomes of a population-based personal injury claims management intervention, the evaluation utilises a methodology that makes substantial use of existing system and public health data, in addition to undertaking numerous stand-alone research projects that will assess individual components of the intervention. The evaluation is designed to assess the critical components of an intervention logical model developed in conjunction with the injury compensation organisation. The findings of the evaluation have the potential to fill substantial knowledge gaps in the impact and performance of compensation systems.

As noted, injury claims management policy and practice has potentially powerful impacts on the recovery of injured persons, their experience of compensation system processes, and the financial viability of compensation systems. This project will establish new knowledge in an area with a relatively limited evidence base. We anticipate that the project will lead to changes in policy and practice within the State of Victoria, and will influence policy and practice in other jurisdictions.