Development of a minimum dataset for subacute rehabilitation: a three-round e-Delphi consensus study.

OBJECTIVES: To develop a minimum dataset to be routinely collected across a heterogenous population within a subacute rehabilitation service to guide best care and outcomes for patients, and value for the health service.
DESIGN: Three-round e-Delphi exercise, followed by consensus meetings.
SETTING: Multicentre study in Brisbane, Australia. PARTICIPANTS: Rehabilitation decision-makers, researchers and clinicians were invited to participate in the e-Delphi exercise. A multidisciplinary project steering committee (rehabilitation decision makers, researchers, clinicians and consumers) participated in consensus meetings.
METHODS: In round 1 of the e-Delphi, participants responded to an open-ended question, generating data and outcomes that should be routinely collected in rehabilitation. In rounds 2 and 3, participants rated the importance of collecting each item on a nine-point scale. Consensus was defined a priori, as items rated as 'essential' by at least 70%, and of 'limited importance' by less than 15%, of respondents. Consensus meetings were held to further refine and define the dataset for implementation.
RESULTS: In total, 38 participants completed round 1 of the e-Delphi. Qualitative content analysis of their responses generated 1072 codes, which were condensed into 39 categories and 209 subcategories. Following two rounds of rating (round 2: n=32 participants; round 3: n=28 participants), consensus was reached for 124 items. Four consensus meetings (n=14 participants) resulted in the final dataset which included 42 items across six domains: (1) patient demographics, (2) premorbid health and psychosocial information, (3) admission information, (4) service delivery and interventions, (5) outcomes and (6) caregiver information and outcomes.
CONCLUSIONS: We identified 42 items that reflect the values and experiences of rehabilitation stakeholders. Items unique to this dataset include caregiver information and outcomes, and detailed service delivery and intervention data. Future research will establish the feasibility of collection in practice. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A strength of this study was the inclusion of multiple stakeholder groups, producing a consensus-derived minimum dataset that accounts for the diverse perspectives and priorities across rehabilitation settings (as opposed to specific subdivisions, professions and conditions).

Response rates decreased across Delphi rounds in the current study, but representation was maintained across all stakeholder groups, discipline backgrounds and clinical specialty.

The sample was limited to universities and/or publicly funded rehabilitation services in Queensland, Australia, potentially limiting generalisability to international and/or private rehabilitation settings.

The final dataset was refined by a multistakeholder steering committee with local implementation in mind, meaning that further refinement may be needed for implementation in other rehabilitation settings.

Introduction

In recent years, there has been an increase in the prevalence of health conditions that result in severe disability, resulting in greater demand for rehabilitation services globally.1–3 This demand is expected to continue to increase in a climate of higher survival rates, longer life expectancies and the increasing prevalence of chronic disease.1–3 Rehabilitation is defined as a series of interventions required when everyday function is limited due to either ageing or a health condition, including chronic diseases or disorders, injuries or traumas.4 An estimated 74% of all years lived with disability are caused by health conditions for which rehabilitation may be beneficial.1 In view of a rising need for rehabilitation, WHO global disability action plan calls for efficient models of care that can strengthen and extend rehabilitation.5

A need to rethink rehabilitation has resulted in a shift towards value-based models of care, which account for consumer perspectives on high-quality outcomes6–9 and are defined by the ratio between these outcomes and their respective costs.10 Core to achieving value-based care is the capture of data that extends beyond measures of cost and efficiency, to measures of effectiveness, value and experience.6 11 Data collection is vital for quality improvement, as it enables institutions to evaluate their existing practice, and to identify areas for ongoing improvement and innovation.12 However, there is currently a wide variation in the data and outcomes collected in rehabilitation, and current data collection initiatives may not fully encompass the principles of patient-centred and value-based care. Minimum datasets (MDS) within healthcare settings have been used to reduce such variation in data collection and have been defined as a coherent set of data elements which should be collected for specific categories or domains of healthcare.13 Ultimately, the collection of MDSs via registries can facilitate standardised care and quality improvement in addition to enabling a means of data collection for research.14 However, an MDS for rehabilitation which captures the principles of value-based care is yet to be defined.

While there are many datasets that collect condition-specific data and outcomes,15 there are few that measure care, global outcomes and value-based indicators across a heterogeneous rehabilitation population. Such a dataset has the potential to evaluate outcomes of rehabilitative care, measure alignment with best evidence recommendations, identify opportunities for service improvement and research, and benchmark care over time and between rehabilitation services.14 Therefore, the aim of this project was to develop an MDS, or a single, consolidated set of data items and core outcomes, for use in rehabilitation services to guide best care and outcomes for patients, and value for the health service.

Methods

Study design

A three-round e-Delphi exercise16 was employed in this study. The Delphi technique is widely used to gather opinion and generate consensus through multiple rounds of questionnaires.17 It is an iterative decision-making process where responses from each round are summarised and communicated back to participants to allow them to review their responses in reference to the median group response in the subsequent rounds.18 The initial round often begins with an exploratory open-ended question where responses generated in that round are then rated by participants in subsequent rounds.19 Consensus meetings were held with the project steering committee to further refine and define the data items and outcomes nominated in the e-Delphi in readiness for implementation.

Participants

An expert panel was established for the e-Delphi using purposive sampling. The following participant groups were eligible to participate: clinicians currently working in an inpatient or community-based rehabilitation service (specialists, doctors, nurses and allied health); researchers currently undertaking clinical or health services research in an area related to inpatient, outpatient or community-based rehabilitation services; decision-makers who have a current management role in a rehabilitation service. Participants could identify as more than one of these participant groups (eg, clinician–researchers).

In Delphi studies, it is recommended that explicit criteria are used to select an expert panel that has knowledge about the subject being investigated.20 Participants were selected to achieve maximum variation in health discipline, rehabilitation setting, and clinical specialty. While there is no agreed minimum sample size that ensures the reliability of Delphi studies,16 previous research has found that reliable Delphi outcomes can be obtained with small expert panels that are selected using strict inclusion criteria.21

A project steering committee was convened to oversee local implementation of the rehabilitation dataset. This committee included consumers (n=3; past rehabilitation client, caregiver, consumer advocate) and rehabilitation clinicians (n=5), managers (n=2) and researchers (n=4) working at the new specialist treatment and rehabilitation service, which is a 100-bed general and specialist rehabilitation service in Brisbane, Australia. The steering committee included representatives from nursing, allied health and medicine, as well as inpatient, outpatient and community-based rehabilitation settings.

Recruitment

Expressions of interest were sought from participants, with information disseminated via existing local rehabilitation-relevant health service and university email lists. All potential participants who expressed interest in participating were screened for eligibility. The final participant panel was purposively sampled to achieve maximum variation in the following variables: stakeholder group, health discipline, rehabilitation setting and clinical specialty.

Data collection and analysis

The questionnaires were administered using the online survey platform Qualtrics. All three rounds of the e-Delphi were piloted by the project team prior to survey deployment; minor changes were made such as formatting, restructuring of questions, and addition of question prompts. A link to each questionnaire was distributed to each participant via email. Data collection occurred between July and October 2020.

Round 1

Participants responded to an open-ended question: ‘What data should be routinely collected for all rehabilitation patients in order to describe care and outcomes?’ Open-ended questions are recommended to reduce the risk of bias through guiding or directing participant response.22 Participants were prompted to consider the following domains: provider demographics, patient and caregiver information, service delivery and outcomes (online supplemental file 1). Responses were analysed using qualitative content analysis,23 a method frequently used in Delphi studies to analyse open-ended responses.24 25 All participant responses were coded and allocated into categories within each of the four domains. To ensure consistency of data interpretation and coding, the coding generated by the primary coder (HC) was reviewed by additional authors (SJW and AMY).

Round 2

Items from round 1 were combined with additional data from a scoping review of rehabilitation datasets26 and multistakeholder focus groups defining successful rehabilitation (Wallace et al, What is ‘successful rehabilitation’? A multistakeholder nominal group technique study to inform rehabilitation outcome measurement; in preparation), and randomly presented to participants in round 2 to prevent an order effect. Participants were asked to rate the importance of routinely collecting each item using a modified version of the Grading of Recommendations Assessment, Development and Evaluation rating scale, whereby 1–3 indicates ‘limited importance’, 4–6 is ‘important but not essential’ and 7–9 is ‘essential’.27 The number and percentage of respondents rating the importance of items as ‘limited’, ‘important but not essential’ and ‘essential’ were calculated for each item.

Round 3

Items rated as ‘essential’ by at least 50% of participants in round 2 were randomly presented to participants in Round 3, alongside the median rating. Participants were asked to consider their ratings in reference to the wider group.28 Criteria for final consensus was defined a priori as a rating of 7–9 by at least 70% of the respondents and 1–3 by less than 15% of the respondents.28 An overview of the e-Delphi exercise and participant response and attrition rates is presented in figure 1.

Figure 1

Summary of e-Delphi process.

Consensus meetings

The steering committee convened for four meetings between December 2020 and June 2021. The committee agreed on guiding principles for inclusion of data items in the final dataset (ie, feasible and acceptable, patient and family-centred, interprofessional, considers the continuum of care, goal oriented and meaningful). Guiding principles were drawn from a related qualitative study which explored stakeholder perspectives of ‘successful rehabilitation’ (Wallace et al, in preparation). The first meeting involved reviewing the final list of items from the e-Delphi, and systematically including or removing items, as informed by these guiding principles. Prior to each subsequent meeting, members received an options paper drafted by the research team (AMY and AC) outlining proposed options for the definition, source, codeset values and/or measurement instrument for each data items. Online surveys using Microsoft Forms and individual meetings with subject matter experts were used to obtain feedback from members between meetings to inform the development of these options papers. During meetings, each data item was discussed in turn until group consensus was achieved.

Patient and public involvement

Consumer representatives were consulted on the study design from conception, and involved in reviewing and selecting the final dataset items as part of the steering committee.

Results

e-Delphi round 1

A total of 45 participants commenced the survey, with 38 completing at least one question in round 1 (table 1). All participants worked within a Queensland university and/or publicly funded rehabilitation service located in south-east Queensland. Participants who did not attempt at least one question were removed from the dataset. Of the 20 participants who identified as clinicians, 13 worked in a hospital inpatient rehabilitation setting, 5 worked in community rehabilitation and 2 worked in a hospital acute setting. The healthcare decision makers represented within this survey (n=12) included directors and executive directors, managers and programme managers. Eighteen researchers participated, representing a range of career levels.

Table 1

e-Delphi participant demographics for round 1

Variable		Round 1 (n=38)
Age, years	18–24	0 (0%)
	25–34	5 (13.2%)
	35–44	13 (34.2%)
	45–54	12 (31.6%)
	55–64	8 (21.1%)
	65+	0 (0%)
Gender	Female	32 (84.2%)
	Male	5 (13.1%)
	Other	1 (2.6%)
Stakeholder group*	Clinician	20 (52.6%)
	Researcher	18 (47.4%)
	Decision-maker	12 (31.6%)
Discipline background*	Nursing	5 (12.5%)
	Occupational therapy	5 (12.5%)
	Physiotherapy	5 (12.5%)
	Medicine	4 (10%)
	Speech pathology	3 (7.5%)
	Psychology	3 (7.5%)
	Dietetics	3 (7.5%)
	Pharmacy	3 (7.5%)
	Audiology	3 (7.5%)
	Social work	2 (5%)
	Other†	4 (10%)
Area of specialty (clinicians and researchers)	Multiple clinical specialties	9 (15.5%)
	Geriatrics	13 (22.4)
	Stroke	10 (17.2%)
	Acquired brain injury	7 (12.1%)
	Progressive neurological conditions	7 (12.1%)
	Spinal cord injury	2 (3.4%)
	Other‡	10 (17.2%)
Years of experience (clinicians)	1–3	2 (10%)
	3–5	1 (5%)
	5+	17 (85%)
Career stage (researchers)	Predoctoral	7 (38.9%)
	Postdoctoral: Early-mid career researcher (≤15 years post-PhD)	7 (39.9%)
	Postdoctoral: Experienced researcher (>16 years post-PhD)	4 (22.2%)
Years of managerial experience (decision makers)	1–3	1 (8.3%)
	3–5	0 (0%)
	5+	11 (91.7%)

*Participants were able to select more than one response.

†Includes ‘health services research’, ‘biomedical science’, ‘rehabilitation engineering’ and ‘sociology’.

‡Includes ‘major trauma/burns’, ‘amputation’, ‘deconditioning’, ‘scope of practice’, ‘hearing impairment and rehabilitation’, ‘mental health’, ‘ankle fracture’, ‘trauma data analysis’, ‘health economics’ and ‘orthopaedic conditions’.

§

§

Analysis of participant responses produced 1072 codes in total, which were condensed into 39 categories and 209 subcategories (online supplemental file 1). These items were cross-checked with a scoping review26 and focus groups (Wallace et al, in preparation) that form part of this project. An additional seven items were added and presented for rating across rounds 2 and 3.

e-Delphi rounds 2 and 3

A total of 32 participants completed round 2, and 28 participants completed round 3 (online supplemental file 2). Out of 216 items presented in round 2, 192 items were rated as essential by more than 50% of participants (online supplemental file 3) and were subsequently presented in round 3. In round 3, 124 items were rated as essential by at least 70% of the participants and of limited importance by less than 15% of the respondents in round 3 (table 2).

Table 2

Final list of items (n=124) reaching consensus in round 3, with percentage of respondents rating item as ‘essential’

Provider demographics
Unit/facility description	Setting: Inpatient/outpatient, day therapy (86%)
Treating team	Treating disciplines (93%) Type of service provided (89%)	Specialist outpatient therapies (86%) Occasions of service by each discipline for each client (75%)
Patient and caregivers information: patient information
Demographics and identification	Age/birthdate (96%) Indigenous status (89%)	Language (89%) Sex/gender (86%)
Premorbid health status	Sensory impairments (93%) Medical/health history (82%) Medications (79%)	Drug and alcohol history (79%) Psychosocial history (75%)
Premorbid social, functional and community participation	Preadmission living situation (96%) Premorbid function/mobility/ADLs (93%) Existing family/carer supports available (89%) Existing community supports/social connections (82%)	Premorbid level of community participation (82%) Premorbid communication (79%) Intent of return to work (79%) Referrals/assessments for aged care home services (75%)
Client premorbid health rating/quality of life	Well-being/worry/anxiety (93%)	Health-related quality of life (71%)
Admission information	Date/time of admission or follow-up (93%)	AN-SNAP classification of rehabilitation admission (71%)
Current diagnoses and impairments	Primary diagnosis/referral reason (100%) Mobility, function and ADLs (100%) Condition/Impairment severity (96%) Cognitive impairment (96%) Communication impairment (96%) Date of onset/time since diagnosis (93%)	Dysphagia and modified diet (93%) Decision-making capacity (89%) Risk of clinical incidents (82%) Frailty (75%) Rehabilitation complexity (71%) Behaviour challenges (71%)
Participation	Participation (86%) Rehabilitation goals (82%)	Client expectations (82%) Motivation (71%)
Discharge/post care-related information	Need for carer (100%) Discharge setting/location (96%) Date/time of discharge (96%) Need for services postdischarge (96%) Referral to other services/services accessed when no longer receiving active rehabilitation (93%)	Ongoing postdischarge care requirements (89%) New residential aged care facility admission (82%) Discharge barriers (79%) Driving instruction on discharge (79%) Discharge summary completion on time (75%) Advanced care planning (75%)
Patient and caregivers information: caregiver information
Demographic and health information	Level of involvement (86%) Living arrangement (79%)	Health issues that impact care (79%) Relationship to client (71%)
Others	Capacity to provide care (82%) Barriers to care (79%)	Access to support (71%) Home living situation (71%)
Service delivery
Intervention details	Type of intervention/treatment details (96%) Mode of delivery (86%) Service delivery model (86%) Frequency of intervention (86%) Occasions of service (82%) Duration and length of intervention (82%) Adherence to treatment (79%)	Date of intervention (75%) Dose/intensity of treatment (75%) Total therapy sessions (75%) Medication and dose (75%) Whether intervention match patient goals (71%) Barriers to intervention (71%)
Disciplines/ clinicians, family and patient involvement	Patient goal (89%) Care plan provided to the patient (89%) Education provided to patient and caregiver (86%) Discipline that performs intervention (86%)	Medication assessment/recommendations prior to discharge (82%) Family and/or advocate involvement and support (82%) Hours/sessions per day in therapy (across disciplines) (75%) Interdisciplinary care (71%)
Care plan, resources and postdischarge activities	Postdischarge follow-up (93%) Management plan (including discharge plan) (89%) Home visits (82%) Future goals at discharge from rehabilitation service (79%)	Psychosocial support (79%) Level of resources required (79%) Instructions for ongoing care (71%)
Outcomes
Impairment	Change in function (100%) Change in impairment (93%)	Change in diagnosis (79%) Length of time to return to function; change in function (75%)
Activity and participation	Activities of daily living (100%) Activity and participation (93%) Ability to return to previous roles/functions/activities (89%)	Community participation (86%) Ability to return to meaningful roles/functions/activities (82%)
Psychosocial and emotional well-being	Quality of life (100%) Psychological well-being (100%)	Social well-being (86%) Emotional response to treatment (79%)
Goal attainment	Goal attainment (96%)	Unmet needs/goals (79%)
Health outcomes	Mortality (93%) Cognition (89%) Dysphagia/specific diet (89%) Pain (82%)	Frailty (79%) Communication access (71%) Physical access (71%)
Satisfaction and experience	Satisfaction with care (82%) Reasons why people ceased treatment (82%)	Patient and family service experience (71%) Compliments (71%)
Admission and readmission	Length of stay in acute/ rehabilitation (93%) Readmission (93%)	Changes in episode of care (79%) Waiting times for review and admission (75%)
Carer outcomes	Carer stress (79%) Carer confidence (79%)	Carer quality of life (71%)
Others	Level of independence (93%) Complications/adverse events and impact on rehabilitation outcomes (93%)

ADLs, activities of daily living; AN-SNAP, Australian National Sub-acute and Non-Acute Patient32.

Consensus meetings

The outcome of the four steering committee meetings was a final dataset of 42 data items, with items across the six domains including patient demographics (six items), premorbid health and psychosocial status (seven items), admission information (four items), service delivery and interventions (12 items), patient outcomes (nine items) and caregiver information and outcomes (four items) (table 3; online supplemental file 4) provides a draft data dictionary for these data items). The committee made the decision to remove items from the round 3 list that were considered to be too focused on physiological or condition-specific outcomes (eg, dysphagia, communication impairment), were already captured and reported in hospital-wide datasets (eg, mortality, discharge destination), were specific to the inpatient setting only (eg, need for post-discharge services) and where consensus could not be reached about its definition (eg, specialist outpatient therapies).

Table 3

Final list of items (n=42) for inclusion in the rehabilitation dataset

Patient information
Demographics	Person identifier Age Sex	Indigenous status Language Need for interpreter
Premorbid health and psychosocial status	Medical/health history Sensory impairments Drug and alcohol history Psychosocial history	Mood Carer supports Community supports
Admission information	Total length of stay in hospital Length of stay in rehabilitation	Suspension of rehabilitation Reason for rehabilitation
Service delivery and interventions	Rehabilitation setting Collaborative goal setting Intervention date Intervention type Intervention duration Barriers to intervention	Treating disciplines Occasions of service Mode of delivery Intervention—individual/group Intervention—interprofessional Medication interventions
Outcomes	Goal attainment Impairment Change in functional status Activity and participation Health-related quality of life	Patient experience Patient satisfaction Complications and adverse events Readmissions
Caregiver information
Caregiver information and outcomes	Current caregiver status Capacity and willingness to care	Preparedness to care Carer experience

Discussion

The primary aim of this study was to develop a consolidated set of data items and core outcomes for use in rehabilitation services to guide best care and outcomes for patients and value for the health service. Following the completion of a three-round e-Delphi exercise and consensus meetings, a final dataset including 42 items was produced. These included items to describe the patients who are receiving rehabilitation (demographics, premorbid health status), the rehabilitation admission and interventions provided, and the patients’ rehabilitation outcomes.

Unsurprisingly, there are items included in this dataset that are commonly collected in other national and international rehabilitation datasets. A recent scoping review aimed to identify the common data items included in rehabilitation MDS and core outcome sets.26 The dataset developed in this study shares 18 data items with the list generated in this review, with data commonly collected in other datasets on patient demographics (patient identifier, age, sex, ethnicity/race), premorbid health status (medical/health history, psychosocial history, community supports), admission (length of stay, reason for rehabilitation), service delivery and interventions (intervention provided) and patient outcomes (impairment, change in functional status, activity and participation, health-related quality of life and readmissions). However, the dataset developed in the current study includes 24 items not commonly included in other rehabilitation datasets. These additional items provide further description of rehabilitation service delivery and interventions (eg, whether interventions were provided in a group or interdisciplinary setting; whether collaborative goals were set with patients and their family) and person-centred outcomes (eg, goal attainment, patient experience). Importantly, this dataset includes outcomes of caregivers, specifically whether they were adequately prepared for the caregiver role and their experience of the rehabilitation process. These additional items reflect the core focus of rehabilitation, particularly in terms of being patient and family-centred and goal orientated,29 and may bridge the gaps in current rehabilitation datasets to allow evaluation of rehabilitation through a value-based healthcare lens.

The recruitment of a variety of different stakeholders, including healthcare decision-makers, researchers and clinicians, was a key strength of this study as it allowed the data items to capture the broader rehabilitation priorities, as opposed to those relevant only to specific subgroups or professions. Overall, the items in this dataset were derived from a consensus between 14 different health disciplines from a range of rehabilitation settings and clinical specialties. Previous rehabilitation datasets have primarily focused on specific conditions (eg, stroke, spinal cord injury), with only 13 of 27 datasets involving consumers in their development.26 Inclusion of consumers and carers in the foundation work which informed the dataset development (Wallace et al, in preparation) and in the project steering committee may explain the inclusion of data items and outcomes specific to caregivers as well as additional outcomes at the life impact level. Additionally, participation by health service managers and clinicians may explain the increased focus on describing service delivery and interventions to allow refinement and improvement of the service provided to patients based on the outcomes achieved.

This study presents a methodologically valid and consolidated set of data items and outcomes that is relevant for implementation within rehabilitative care settings. Given the shift toward value-based healthcare models,10 these findings may enable rehabilitation services to measure outcomes that are likely meaningful for patients and families. This dataset may encourage the collection of holistic and clinically relevant data that capture the priorities of rehabilitation, by collecting data relating to provider demographics, patient and caregiver information, service delivery and outcomes. As the expert panel was limited to participants working in universities or public rehabilitation services in Queensland, Australia, validation of the dataset to ensure its relevance in international and/or privately funded rehabilitation settings is required before implementation in other settings.

As this study has generated a list of items and outcomes that would be useful to collect within rehabilitation services, it is crucial that further work is completed to identify and select the most suitable outcome measurement tools to collect this data, with instruments need to be reliable, valid and responsive to detect changes over time,30 while also being feasible in clinical practice and low burden for both patients and clinicians. There is also a need to evaluate the implementation process given the challenges with implementing collection of patient-reported outcome measures that have been reported to date.31 Additional evaluation of the use and outcomes of this dataset in practice would also contribute knowledge to further support the benefits of MDS and clinical registries, such as improving clinical outcomes and care processes and facilitating research.12

Conclusion

This study presents a consolidated dataset that captures the priorities and perspectives of key stakeholders of rehabilitation services. Consensus was reached for a total of 124 items, with a strong distribution towards items relevant to patient and caregiver information. This was refined by a multistakeholder committee to a dataset containing 42 items ready for implementation in a local rehabilitation service. The MDS developed in this study may bridge the gaps in current data collection initiatives for rehabilitation services and improve the capacity of services to provide patient-centred care. Ultimately, this study provides a preliminary MDS which may facilitate quality improvement and value-based care. Future research is needed to evaluate the implementation process and outcomes of the MDS, to determine whether it meets its objective to improve the provision of value-based rehabilitation care.