A multicentre randomised feasibility STUdy evaluating the impact of a prognostic model for Management of BLunt chest wall trauma patients: STUMBL Trial.

OBJECTIVE: A new prognostic model has been developed and externally validated, the aim of which is to assist in the management of the blunt chest wall trauma patient in the emergency department (ED). The aim of this trial is to assess the feasibility and acceptability of a definitive impact trial investigating the clinical and cost-effectiveness of a new prognostic model for the management of patients with blunt chest wall trauma in the ED.
DESIGN: Stepped wedge feasibility trial.
SETTING: Four EDs in England and Wales. PARTICIPANTS: Adult blunt chest wall trauma patients presenting to the ED with no concurrent, life-threatening injuries. INTERVENTION: A prognostic model (the STUMBL score) to guide clinical decision-making. OUTCOME MEASURES: Primary: participant recruitment rate and clinicians' use of the STUMBL score. Secondary: composite outcome measure (mortality, pulmonary complications, delayed upgrade in care, unplanned representations to the ED), physical and mental components of quality of life, clinician feedback and health economic data gathering methodology for healthcare resource utilisation.
RESULTS: Quantitative data were analysed using the intention-to-treat principle. 176 patients were recruited; recruitment targets were achieved at all sites. Clinicians used the model in 96% of intervention cases. All feasibility criteria were fully or partially met. After adjusting for predefined covariates, there were no statistically significant differences between the control and intervention periods. Qualitative analysis highlighted that STUMBL was well-received and clinicians would support a definitive trial. Collecting data on intervention costs, health-related quality of life and healthcare resource use was feasible. DISCUSSION: We have demonstrated that a fully powered randomised clinical trial of the STUMBL score is feasible and desirable to clinicians. Minor methodological modifications will be made for the full trial. TRIAL REGISTRATION NUMBER: ISRCTN95571506; Post-results. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The main strength of this feasibility trial is that it has tested all of the methodological components of the future definitive trial, including the trial infrastructure, randomisation process, data management system, both qualitative and quantitative analysis, follow-up procedures and a health economic analysis.

The health economic analysis undertaken will ensure accurate future funding applications for the full definitive trial.

The qualitative analysis has informed the researcher teams’ understanding of the use of the risk score by clinicians in daily clinical practice.

The cluster randomised trial design that needed to be tested in this feasibility trial was limited by the small number of clusters (emergency departments participating). In the future definitive trial, more clusters will be possible, due to the greater number of participating sites.

Introduction

Blunt chest wall trauma accounts for over 15% of all trauma admissions to emergency departments (EDs) worldwide, with reported mortality ranging between 4% and 60%.1–3 Difficulties in the management of blunt chest wall trauma patients in the ED are becoming increasingly well recognised in the literature.4 5 The patient commonly presents to the ED with no respiratory difficulties, but can develop complications up to approximately 72 hours after initial presentation.1–4 Clinical symptoms are not considered by most clinicians to be an accurate predictor of outcome following non-life-threatening blunt chest wall trauma.1–7 Decisions regarding the appropriate level of care required by the patient following discharge from the ED are therefore difficult and are further compounded by the lack of current national guidelines. Improvements in identification of high-risk patients on initial assessment in the ED could lead to a reduction in these poor outcomes.8 9

A new prognostic model (STUMBL Score) to assist the management of blunt chest wall trauma patients in the ED has been developed and externally validated in a large multicentre study.1 5–7 Guidelines recommend that prognostic models should not be used in clinical practice until an impact trial has been completed, in which the clinical application of the model has been tested.10 11 The aim of this trial was to establish the feasibility and acceptability of a future definitive trial, which will determine whether the prognostic model can be used safely and effectively in clinical practice in the UK.

Methods

Trial design and randomisation

This was a multicentre, prospective, cluster randomised (stepped wedge) trial12 with a qualitative component. The unit of randomisation was the ED, rather than the patient, since the intervention was used by ED clinicians with the aim of studying effects on patient outcomes.13 All EDs began as controls testing conventional management without the model for a period of 1 month. Every month, one hospital was randomly assigned to the intervention period (using the prognostic model), until all hospitals were intervention sites. Online supplementary file 1 outlines the trial procedure, and the protocol describes the trial in full.14

Population

Patients presenting to the ED with isolated blunt chest wall trauma were included in the trial if they were aged 18 years or over and capable of giving consent to participation. Exclusion criteria were under the age of 18 years, lacked capacity to give informed consent, presented with immediately life-threatening injuries or any concurrent injury that would determine the patient’s management.

Patient and public involvement

Two ex-trauma patients were members of the STUMBL Trial Development Group that developed the protocol and designed the study. They continued to sit on the Trial Management Group, attending all research meetings and contributing to the overall running of the trial. A further two ex-trauma patients sat on the Trial Steering Committee (TSC). Trial participants were instructed to contact the research team if they were interested in the trial results.

Setting and recruitment

This feasibility trial ran in four EDs in the UK: Royal Gwent Hospital, Newport; Musgrove Park Hospital, Taunton (both non-major trauma centres); Salford Royal Hospital and Manchester Royal Infirmary (both major trauma centres). ED clinicians and research nurses were responsible for screening, recruiting and consenting eligible patients to the trial.

Sample size

The trial had a 5-month recruitment period, and each hospital aimed to recruit 30–80 patients. This was the minimum number considered necessary to test data collection processes based on existing recommendations for meaningful estimates of parameters of interest.15 This recruitment period also allowed for loss to follow-up, low response rates with follow-up surveys and difficulties with recruiting at weekends if research nurses only worked Monday to Friday.

Intervention

The intervention was a prognostic model to guide clinicians' decision-making. The model was used during initial patient assessment and provided a suggestion of the appropriate management (whether the patient could be safely discharged home or needed admission to either a ward or a critical care). Participants were asked to complete one survey (SF-12v1)16 on initial presentation and two more surveys at 6 weeks (SF-12v1 and a Client Services Receipt Inventory (CSRI)). Online supplementary file 2 illustrates the patients’ journey.

Outcome measures

Primary outcome measures

1. Patient recruitment rate.

2. Clinicians’ use of the model in the intervention period.

Secondary outcome measures

Patient outcomes: (1) composite outcome measure: which includes in-hospital mortality, pulmonary complications (chest infection, pneumonia, pneumothorax, haemothorax, pleural effusion or empyema) diagnosed during hospital admission, delayed upgrade in care (defined as a deterioration in a patient’s condition on the ward, requiring admission to critical care) and unplanned representations to the ED, (2) overall mean quality of life (as reported by SF-12v1 survey).

Organisational outcomes: (1) clinician recruitment rate, (2) participant questionnaire response rate and (3) clinicians training attendance rate.

Qualitative outcomes: (1) clinician feedback regarding training quality and use of the prognostic model in clinical practice, (2) clinician and research nurse feedback regarding trial methods.

Health economic outcomes: (1) trainer/trainee hourly rates/training time/travelling time, (2) equipment costs and (3) resource usage by patients during the trial period.

Criteria for establishing feasibility

In order to evaluate the feasibility of a full definitive trial, the TSC assessed the findings against predetermined success criteria (box 1) using a traffic light system.17

80% or more of eligible patients consent to data collection and follow-up.

90% or more use of the model by clinicians during intervention period.

95% or more of clinicians working within the participating emergency department agree to take part in the trial.

Follow-up data for patient secondary outcomes can be collected for 80% or more of patients.

All clinicians involved in the trial receive formal training in the use of the model.

Patient secondary outcome measures reported in the intervention period are equal to, or better than, those reported during the conventional management period.

Mean quality of life reported in intervention arm is not less than 80% of that reported in control arm.

Traffic light assessment: Green means the target was achieved, amber means the target was not achieved but progression is possible with some minor protocol modifications and red means progression to a full trial is not possible.

Data collection and management

Research Electronic Data Capture18 was used for data capture, management and completion of the electronic case report forms.

Data analysis

Quantitative analyses were performed on Stata V.15.1 SE using the intention-to-treat principle and were not powered to detect clinically important effects. Results are presented as numbers (percentages), means (SD) and medians (interquartile ranges) where non-normally distributed. Baseline characteristics between groups were completed using the Wilcoxon rank-sum test and Fisher’s exact test. The risk score was also calculated retrospectively for control cases for baseline comparisons purposes only.

A binary composite outcome measure was calculated, indicating if the participant had experienced at least one of the contributing factors (mortality, unplanned representation to the ED, delayed upgrade in care or onset of pulmonary complications). Intervention/control groups were compared using logistic regression analysis (using exact logistic regression where there were zero-count cells). Covariates included age at the time of presentation, the number of clinically suspected rib fractures, preinjury use of anticoagulants, the presence of pre-existing chronic lung disease and oxygen saturation on initial assessment in the ED. Analysis of individual components of the composite outcome measure was completed using the same method.

Sensitivity analyses were undertaken following clinical review of the data. The following factors were considered: the type of logistic regression analysis used, inclusion of additional covariates, representations leading to immediate discharge and unrelated representations/deaths. Each was analysed individually, on the composite outcome measure, with all covariates included.

A purposive sample of the clinicians based on seniority, location and trial participation were invited to take part in a short interview in which they were asked to discuss the model in relation to their clinical practice. Two research nurses (where available, the Royal Gwent Hospital only had one research nurse) from each site were asked to attend a focus group in which the trial’s methodology was discussed. One site was unable to attend. Focus group and interviews were audio-recorded and transcribed verbatim by a professional transcription service. The qualitative researcher (CO’N) monitored this process; transcripts were checked for accuracy against the original recording and were anonymised in accordance with best practice standards. The cleaned and anonymised transcripts were uploaded into NVIVO V.11.19

Thematic analysis was used to analyse the transcripts using a code book, which developed initially from the background literature and feedback on the training as well as issues that emerged through the process of constant comparison, which underpins qualitative data analysis. Due to the small numbers of transcripts, the coded transcripts were checked by a second qualitative researcher, using the coder comparison query tool.

Health economic analysis focused on establishing the most appropriate framework for a future health economic analysis. The feasibility of collecting data on quality of life and resource use was assessed, along with a provisional assessment of cost categories associated with the intervention (eg, staff training time). To capture resource use by patients from an NHS/PSS (National Health Service/Personal Social Service) perspective, an adapted resource usage questionnaire was used. Data were assessed to examine the completeness of data captured, such as response rate and missing items.

Results

All sites were able to recruit over the 5-month period, and there were no difficulties implementing the stepped wedge design (online supplementary file 1). There were no obvious differences between numbers recruited at each site when considering major trauma centre status, with the Royal Gwent Hospital recruiting 31 patients, Musgrove Park Hospital recruiting 47 patients (plus one withdrawal of consent), Salford Royal Hospital recruiting 38 patients and Manchester Royal Infirmary recruiting 60 patients (figure 1). There was some inconsistency around screening/consenting between sites. For example, research nurse availability affected the recruitment and consent process in smaller sites. Another inconsistency occurred as one site did not keep a screening log throughout the trial.

Figure 1

STUMBL CONSORT diagram. CONSORT, Consolidated Standards of Reporting Trials; CSRI, Client Services Receipt Inventory.

Feasibility criteria

All feasibility criteria were green or amber (table 1). The primary outcomes were both green. It proved impractical to obtain precise figures for the number of clinicians working in EDs over the trial period, but the information available suggests that no one dissented from using the tool and all were appropriately trained. There were initial difficulties in obtaining follow-up questionnaire data; however, response rates improved once telephone and email follow-ups were introduced, justifying the amber status. Clinical follow-up data were only from hospital records (ie, participants who were admitted or subsequently represented to hospital).

Table 1

Feasibility outcomes

Feasibility criteria	Result	Feasibility assessment
Primary outcomes
80% or more of eligible patients consent to data collection and follow-up (patients approached to participate in trial: n=186)	176/186 (94.6%)	Green
90% use of the model during intervention period (n=76 in intervention period)	Model used in 73 cases (96.1%)	Green
Secondary outcomes
95% or more of clinicians working within the participating ED agree to take part in the trial	100%	Green
Follow-up data for patient secondary outcomes can be collected for 80% or more of patients (n=176)	Clinical data: 176 (100%) SF-12: 128 (72.7%) CSRI data available: 137 (78.8%)	Green Amber Amber
All clinicians involved in the trial receive formal training in the use of the model	100%	Green
Patient secondary outcome measures reported in the intervention period are equal to, or better than, those reported during the conventional management period	See table 3 below	Amber
Mean quality of life reported in intervention arm is not less than 80% of that reported in control arm	See table 4 below	Green

Number of cases (percentages). Feasibility criteria traffic light system: green, feasibility criteria achieved; amber, feasibility criteria not achieved but progression is possible with some minor protocol modifications.

CSRI, Client Services Receipt Inventory; ED, emergency department.

Baseline characteristics

The intervention group was significantly older, with more severe injuries and more likely to sustain a low velocity fall, when compared with the control group. There were a significantly higher number of road traffic accidents in the control group (table 2). To understand the differences between groups, we retrospectively calculated the risk score for participants in the control group using routinely collected data. We found that the intervention group scored significantly higher on the prognostic model than the control group.

Table 2

Baseline subject characteristics

Characteristic	Categories	Control period	Intervention period	Overall
Gender		n=100	n=76	n=176
	Male	65 (65.0%)	53 (69.7%)	118 (67.1%)
	Female	35 (35.0%)	23 (30.3%)	58 (32.9%)
Age		n=100	n=76	n=176
	Years	48.9 (17.8)	55.0 (17.8)	51.5 (18.0)*
Number of clinically suspected rib fractures	Number	n=100 0 (0, 1)	n=76 1 (0, 2)	n=176 0.5 (0, 1)*
Preinjury anticoagulant use		n=99	n=73	n=172
	Yes	3 (3.0%)	6 (8.2%)	9 (5.2%)
	No	96 (97.0%)	67 (91.8%)	163 (94.8%)
Pre-existing lung disease		n=100	n=76	n=176
	Yes	10 (10.0%)	7 (9.2%)	17 (9.7%)
	No	90 (90.0%)	69 (90.8%)	159 (90.3%)
Oxygen saturation at initial assessment		n=92	n=70	n=162
	95%–100%	80 (87.0%)	67 (95.7%)	147 (90.7%)
	94%–90%	11 (12.0%)	3 (4.3%)	14 (8.6%)
	85%–89%	1 (1.09%)	0 (0.0%)	1 (0.6%)
	80%–84%	0 (0.0%)	0 (0.0%)	0 (0.0%)
	75%–79%	0 (0.0%)	0 (0.0%)	0 (0.0%)
	70%–74%	0 (0.0%)	0 (0.0%)	0 (0.0%)
Prognostic model	Total score	n=91 5 (3–9)	n=67 9 (5–13)	n=158 7 (4–11)*
Mechanism of injury		n=100	n=76	n=176
	Low velocity fall	27 (27.0%)	44 (57.9%)	71 (40.3%)†
	High velocity fall	14 (14.0%)	9 (11.8%)	23 (13.1%)
	RTA	20 (20.0%)	5 (6.6%)	25 (14.2%)†
	Contact sport	4 (4.0%)	3 (3.9%)	7 (4.0%)
	Alleged assault	7 (7.0%)	5 (6.6%)	12 (6.8%)
	Other	28 (28.0%)	10 (13.2%)	38 (21.6%)
Quality of life score (measured by SF-12)	Physical Component Score	n=100 41.0 (30.7–49.9)	n=76 43.6 (31.8–52.1)	n=176 41.5 (31.5–50.2)
	Mental Component Score	n=100 54.0 (44.5–59.4)	n=76 55.8 (50.2–58.0)	n=176 55.3 (46.4–59.0)
Presenting with pulmonary complications		n=100	n=76	n=176
	Yes	1 (1.0%)	5 (6.6%)	6 (3.4%)
	No	99 (99.0%)	71 (93.4%)	170 (96.6%)
Preinjury antiplatelet use		n=99	n=73	n=172
	Yes	5 (5.1%)	6 (8.2%)	11 (6.4%)
	No	94 (94.9%)	67 (91.8%)	161 (93.6%)
Frailty score		n=92	n=60	n=152
	Very fit	50 (54.3.%)	32 (53.3%)	82 (53.9%)
	Well	24 (26.1%)	19 (31.7%)	43 (28.3%)
	Managing well	8 (8.70)	5 (8.3%)	13 (8.6%)
	Vulnerable	9 (9.8%)	1 (1.7%)	10 (6.6%)
	Moderately frail	1 (1.1%)	2 (3.3%)	3 (2.0%)
	Severely frail	0 (0.0%)	1 (1.7%)	1 (0.7%)
Presented at major trauma centre		n=100	n=76	n=176
	Yes	66 (66.0%)	32 (42.1%)	98 (55.7%)†
	No	34 (34.0%)	44 (57.9%)	78 (44.3%)
Time site in trial stage at presentation	Time in days	36.5 (18–57)	23.5 (11–46)	29.5 (15–54)*

Mean (SD); median (IQR); n, non-missing sample size; number (%).

*Significant p<0.05 using Wilcoxon rank sum test.

†Significant p<0.05 using Fisher’s exact test.

RTA, road traffic accident.

Clinical outcomes

There was a complication rate of 8%, when considering the composite outcome measure (table 3). The ORs for mortality and delayed upgrade in care were calculated using exact logistic regression with no covariates due to 0-count cells.

Table 3

Secondary outcome measure (composite measure)

Criteria	Total n=176	Control period n=100	Intervention period n=76	Adjusted OR (95%)
Composite outcome measure	14 (8.0%)	4 (4.0%)	10 (13.2%)	3.89 (1.01, 14.95)
Secondary outcome measures
Mortality	2 (1.1%)	0 (0.0%)	2 (2.6%)	3.24 (0.25, +infinity)
Unplanned presentations to ED	10 (5.7%)	3 (3.0%)	7 (9.2%)	4.89 (0.97, 24.80)
Delayed upgrade in care	1 (0.6%)	0 (0.0%)	1 (1.3%)	1.32 (0.03, +infinity)
Onset of pulmonary complications	3 (1.7%)	1 (1.0%)	2 (2.6%)	2.41 (0.09, 66.78)

Number (%); adjusted ORs calculated using logistic command. The composite outcome measure was a binary variable indicating if the participant had at least one of the contributing secondary outcome measures (mortality, unplanned representation to the ED, delayed upgrade in care or onset of pulmonary complications).

ED, emergency department.

Results of the quality of life analysis (SF-12v1) found no statistically significant difference between the intervention and control groups, in either the physical or mental components (table 4).

Table 4

Health-related quality of life

	Control group	Intervention group
Physical Component Score at baseline	41.0 (30.7 to 49.9); n=100	43.6 (31.8 to 52.1); n=76
Physical Component Score at follow-up	50.1 (38.5 to 54.5); n=75	43.8 (34.3 to 52.1); n=51
Change in Physical Component Score	4.45 (−1.1 to 15.7); n=75	0 (−7.8 to 12.1); n=51
Mental Component Score at baseline	54.0 (44.5 to 59.4); n=100	55.8 (50.2 to 58.0); n=76
Mental Component Score at follow-up	53.1 (45.4 to 58.7); n=75	55.2 (42.2 to 58.7); n=51
Change in Mental Component Score	−0.1 (−8.1 to 4); n=75	−2.4 (−10.6 to 2.3); n=51

Data are presented as median (IQR). Only complete forms were included in this analysis; no imputation was undertaken.

Sensitivity analysis—composite outcome measure only

The type of regression analysis undertaken had the largest effect on the results. Due to the low number of events seen, we considered the impact of using an exact logistic regression (using the Stata exlogistic command) and a Penalised Maximum Likelihood Estimation (the ‘Firth method’, using the Stata firthlogit command). Exact logistic regression produced results consistent with our initial calculation; the Firth method produced results that were more favourable to the intervention. Further analysis that considered other covariates made little difference to the overall findings, nor did whether the prognostic model’s management guidance was followed, accounting for the unrelated deaths/representations, or our assumption that all pulmonary complications diagnosed on the day of randomisation were present at admission. Excluding representations leading to a discharge had a large impact on the size of the 95% CI, but this may be due to very low number of events remaining for analysis (online supplementary file 3).

Missing data

Some data did not appear to be missing entirely at random, with one site missing considerably more data than the others. Exploration suggested that this was due to lower research nurse levels and unforeseeable circumstances. There was difficulty in obtaining completed CSRI forms compared with SF-12s. Some CSRIs were completed by research nurses using available NHS records and contacting General Practitioner (GPs), particularly in one of the early intervention sites, leading to a higher proportion completed than would otherwise be the case.

Serious adverse events

Two deaths were reported. One was unrelated to the chest injury, the other patient was placed on a palliative care pathway following admission to hospital.

Qualitative results

Four research nurses attended the focus group and nine clinician interviews were completed. Thirty-six training feedback questionnaires were completed. Following coding, three main themes covering issues of process, impact and mainstreaming were identified.

Trial processes refer to informants’ perspectives on the running of the trial from planning and design issues through to data collection, publicity and training. Staff were positive about the trial, they liked the stepped wedge design and data collection was largely unproblematic. The use of social media was commended and staff were keen for this to be retained. Using Twitter to drive patient recruitment was highlighted by everyone as a great motivator and a way to encourage team involvement and to drive recruitment.

The consent model used in this trial, and the alternative of presumed consent, was not considered contentious by the teams. The training was well received and could be offered in different formats. Data collection tools and improvements made to them during the trial were considered appropriate. Some improvements such as telephone follow-ups worked well. Otherwise, there were no other suggested changes regarding trial processes. It was clear that staff appreciated the value of the STUMBL Score and would support a definitive trial to ensure its wider implementation.

Health economic evaluation

There were no significant differences in quality of life between control and intervention groups at either baseline or follow-up. However, analysis only included complete cases (n=126 of 176), with no imputation undertaken, and should be viewed with caution. The mean training time was 31 min (SD: 24 min), and it usually took place at a trainee’s place of work. As the pay grades for clinicians varied, training costs varied substantially between £6.45 and £108 per session (mean = £31.82, SD = £20.6). When training could be carried out in a trainee’s place of work, the cost per person was minimised.

The CSRI and SF-12 captured the data required for a full economic evaluation in the main trial. Measures can be taken to adapt the instruments to capture key data that are likely to have a greater impact on costs.

Discussion

This is the first trial to test the feasibility of conducting a cluster-randomised controlled trial into the effectiveness of a prognostic model for the management of blunt chest wall trauma patients presenting to the ED. All feasibility criteria were met, suggesting that progression to a full impact trial is possible.

This trial has highlighted some minor modifications that need to be made to the methods prior to moving forward to the full trial. Research nurse availability varied substantially between sites. In one of the smaller (trauma unit) sites, difficulties in patient recruitment were noted due to the fact that there was only one research nurse working on the trial, so out-of-hours recruitment was not possible. This suggests that capacity is something to consider carefully when recruiting sites for any future definitive trial. Reliance on research nurses to recruit participants, rather clinicians recruiting as part of patient management, will need consideration when planning the future trial. A retrospective consent model was discussed in some clinician interviews, which may need consideration as one strategy to improve recruitment numbers.

Participant outcomes were compared to ensure that there were no large differences between groups and that we remain in equipoise regarding use of the model in EDs. Although analysis of the composite outcome measure suggests a difference between groups, this may be because the two major trauma centres were the last sites to transition to the intervention (and therefore had longer in the control phases) and collectively, they recruited most patients to the trial. The intervention group was significantly older, more severely injured and scored higher on the prognostic model than the control group (using retrospective scores for the control group). A larger number of sites will be used in the future trial, which should reduce baseline variation. Overall, the trial population was younger than previous research1 6 by the group which may be due to the inclusion of a higher proportion of major trauma centres. Due to a low number of adverse events compared with previous research, the regression analysis produced wide CIs. This trial is not powered to detect differences in outcomes, and therefore any differences between groups should be considered with caution.

The complication rate was lower than that reported in our development and validation work.1 6 This could be explained by the proportion of major trauma centres included in this trial, when compared with the previous work, although this would need further investigation. This information will prove useful when calculating the sample for a full trial.

There was difficulty in obtaining completed CSRI surveys. As a result, some CSRIs were completed by research nurses using available NHS records and contacting GPs. The most costly data gathered from the CSRI questionnaire would normally be inpatient stays and outpatient visits, especially when there are multiple stays/visits following the initial visit to the ED. Unfortunately, an oversight of the timing of the CSRI meant questions only asked about the 6 weeks following discharge, resulting in a failure to capture costs involved in the initial ED treatment. This is a key cost driver, and in a full trial, the CSRI questions would be amended to capture recall since arrival at ED.

In conclusion, this work has demonstrated that progression to the full definitive impact trial is feasible, with some minor modifications.