Promoting early treatment for mild traumatic brain injury in primary care with a guideline implementation tool: a pilot cluster randomised trial.

OBJECTIVES: New clinical practice guidelines for the management of mild traumatic brain injury (mTBI) emphasise that family physicians should proactively screen and initiate treatment for depression/anxiety, insomnia and headaches. This study aimed to evaluate the feasibility of delivering an implementation intervention to family physicians.
DESIGN: Pilot cluster randomised controlled trial.
SETTING: Specialty outpatient clinic (recruitment) and primary care (implementation). PARTICIPANTS: 114 primary care clinics were randomised. These clinics were associated with 137 unique family physicians caring for 148 adult patients who sustained an mTBI within the previous 3 months and were seeking care for persistent symptoms.
INTERVENTIONS: Patients completed self-report screening measures for depression/anxiety, insomnia and headaches. A tailored letter that incorporates the patient's screening test results and associated treatment algorithms was sent to their family physician (or walk-in clinic). Physicians at clinics assigned to the control condition received a generic letter, without the screening test results. PRIMARY OUTCOME MEASURES: Feasibility outcomes included the frequency of primary care follow-up, retention rates and reliability of patient recall of their physicians' actions (primary mechanistic outcome). The primary efficacy outcome was the Rivermead Post-Concussion Symptom Questionnaire (RPQ).
RESULTS: Most patients (97.8%; 128 of 131) followed up at the primary care clinic they planned to. Retention rates were 88% (131 of 148) and 78% (116 of 148) at the 1-month and 3-month assessments, respectively. Agreement between patient recall of their physicians' actions and medical chart audits was moderate (intraclass correlation coefficient=0.48-0.65). Patients in the experimental group reported fewer symptoms on the RPQ compared with those in the control group, whose physician received a general letter (B=-4.0, 95% CI: -7.3 to -0.7).
CONCLUSIONS: A larger trial will need to address minor feasibility challenges to evaluate the effectiveness of this guideline implementation tool for improving mTBI clinical outcomes and confirm the mechanism(s) of intervention benefit. TRIAL REGISTRATION NUMBER: NCT03221218. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

This pilot study included randomisation and blinding of physicians who were the target of a behaviour change intervention.

We relied on patient recall to assess physician behaviour (referrals and prescriptions).

Feasibility indicators included the proportion of patients who followed up with the physician or clinic they planned to, agreement between patient recall and medical chart audit regarding physician behaviour, and patient retention.

Patients were recruited from outpatient specialty clinics, likely resulting in selection bias.

Introduction

Mild traumatic brain injuries (mTBIs) are very common, affecting 600–1200 per 100 000 people each year.1 2 In adults, falls and motor vehicle accidents are the most common mechanisms of injury.3 The clinical presentation following mTBI typically includes non-specific ‘postconcussion’ symptoms, such as headache, dizziness, sleep disturbance, poor concentration and emotional lability. These symptoms often resolve over the following days to weeks.4–6 However, at least 15% of patients continue to report multiple symptoms well beyond a month post injury.7 Persistent postconcussion symptoms are associated with reduced work productivity8 9 and quality of life,10 as well as high healthcare utilisation.11 12

Optimal management of persistent postconcussion symptoms was ill-defined until the Ontario Neurotrauma Foundation (ONF) sponsored an interdisciplinary team to synthesise the best available evidence and develop practice guidelines for primary care practitioners.7 A core principle of these guidelines is early, proactive management, starting with ‘primary’ symptoms—depression/anxiety, insomnia and headache. Primary symptoms are designated as such because they are most likely to respond to treatment (effective interventions are available), which can improve other (secondary) symptoms, such as difficulty concentrating and forgetfulness.7

Family physicians are uniquely positioned to implement this core component of the ONF guidelines. They are increasingly the first medical professionals to assess patients with mTBI.13 They also tend to carry the burden of follow-up care for patients who first present to an emergency department.14 Patients whose postconcussion symptoms do not promptly resolve after mTBI typically return to see their family physician on average 4–5 times over the 12 weeks after injury,15 16 providing an ideal window for early management to prevent chronicity.

Active implementation will be necessary for the ONF guidelines to promote uptake in primary care.17 18 Barriers to clinical practice guideline uptake can arise from the nature of the behaviour demanded by the guidelines, the adopters of the guidelines (in this case family physicians) and the practice environment. Of the many barriers identified in the literature,19 20 a few appear most relevant to managing mTBI in primary care. First, time constraints are a ubiquitous challenge to guideline adherence in primary care,20 21 but may be especially challenging to mTBI management because patients tend to present with numerous, diverse symptoms. Suggesting which symptoms to prioritise and having patients complete screening tools for priority symptoms before the consultation can help streamline care. Second, the ONF guidelines are comprehensive, intended to cover the full continuum of care, across practice settings. Family physicians prefer concise, tailored recommendations.19–21 Highlighting the most impactful actions, a family physician can take (ie, initiating treatment for depression, anxiety, insomnia and headache) in the initial clinic visits may enhance guideline use. Evidence from other health conditions suggest that guideline implementation tools tailored to the user are more effective.22 23 Another potential barrier to implementation of the ONF guidelines in primary care is that, although virtually all family physicians encounter patients with mTBI, few (10%–12%) see more than 10 mTBI cases per year, which may limit their confidence in their knowledge or skills and recollection of relevant practice guidelines.24 25 Given their relatively infrequent need to reference the ONF guidelines, delivering a guideline tool at or just prior to the clinical encounter may enhance guideline adherence.26 Systematic reviews support that reminders are among the most effective strategies for guideline implementation.26–29

In the present study, we report a pilot cluster randomised controlled trial of a guideline implementation tool that aims to distill the ONF guidelines into a small number of actionable messages for family physicians, tailored to their patient, and provides point-of-care reminders to implement those actionable messages. This pilot study had three objectives. The first was to document evidence of undertreatment, that is, that patients who screened positive for one or more primary symptoms were not consistently receiving evidence-based treatment for those symptoms. This would confirm an implementation intervention target. A second objective was to evaluate the feasibility of delivering an implementation intervention to family physicians and measuring its impact, without relying on family physician’s active participation as research subjects, which would introduce selection bias and observation (Hawthorne) effects. The feasibility outcomes included the (a) proportion of patients who identified a primary care clinic where they intended to seek follow-up care and attended that clinic at least once post intervention, (b) proportion of patients who could be reached for outcome assessment and (c) agreement between the patient recall and chart review indicators of family physician behaviour. A third objective was to estimate parameters necessary to guide planning of a future larger trial, specifically, the intervention effect size (on postconcussion symptom severity), cluster size and cluster effects (intracluster correlation coefficient). We hypothesised that family physicians who receive the guideline implementation tool will be more likely to promptly initiate treatment for primary symptoms compared with family physicians who receive a generic letter that provides instructions for downloading the full guidelines (ie, passive dissemination control, which does not address the implementation barriers reviewed above). We further hypothesised that patients whose family physicians received the tailored follow-up letters would report fewer postconcussion symptoms at follow-up, in other words, that family physician behaviour change would translate into better clinical outcomes.

Methods

Patients were recruited from two programmes (in Vancouver and Coquitlam, British Columbia, Canada) that provide early education for patients with mTBI. Both sites are Medical Services Plan (publicly) funded and have similar referral patterns. About 50%–60% of referrals come from family physicians, with the remainder from emergency departments and other sources. Referred patients are offered a single group education session about mTBI recovery, return to activity advice and self-management strategies, delivered by an occupational therapist. A minority (<10%) is referred internally for other specialty rehabilitation services.

Between July 2017 and October 2018, group education session attendees were informed about the study and invited to complete the consent form and baseline questionnaires before leaving the session, or afterwards (within the following week) via a secure web-based survey platform (REDCap). As part of the questionnaire package, participants were asked to designate where they plan to seek follow-up care (a specific family physician or walk-in clinic) and to sign an authorisation for the research team to access medical records from that provider/clinic.

Patients were considered eligible if they were 18–60 years old, sustained an mTBI less than 3 months ago (per the referring physician), self-reported having sufficient English reading comprehension for standardised questionnaires and designated where they planned to receive follow-up primary care (name and address of a family physician identifiable in the College of Physicians and Surgeons of British Columbia online database or a specific walk-in clinic).

In a cluster randomised design, primary care clinics were randomised to the guideline implementation tool (tailored follow-up letters) or generic follow-up letters. A simple 1:1 randomisation sequence was kept concealed from research team members who made the allocations. Once a participant was deemed eligible and completed the baseline assessment (described below), the primary care clinic they planned to access was randomised by obtaining the next allocation in the sequence. There were two constraints. If the clinic had already been randomised (when they were named by a previously enrolled patient participant), they retained their group assignment. If a participant was under the care of a family physician associated with a clinic that had already been randomised, the clinic (and linked physician–participant dyad) received that predetermined group assignment.

Patient participants completed follow-up assessments online (REDCap; secure cloud-based data collection software), or if they preferred, by telephone, 1 and 3 months later. Participants flow through the study is illustrated in figure 1.

Figure 1

Consolidated Standards of Reporting Trials diagram. mTBI, mild traumatic brain injury.

Patient and public involvement

Patients were not involved in the design or conduct of this pilot trial. Patient partners joined our research team in the dissemination phase to help identify limitations of the study and directions for future research (eg, addition of a patient engagement intervention component).

Interventions

The implementation intervention involved systematically screening patients for depression, anxiety, insomnia, and headaches and sending a follow-up letter to the family physician (guideline implementation tool) that provides the screening test results and highlights associated management recommendations (with treatment algorithms) from the ONF guidelines. Patient participants completed a battery of questionnaires (<20 min of administration time) that have been validated as screening tools for the ‘primary symptoms’ prioritised for treatment in the ONF mTBI guidelines, listed below.

Depression. The Patient Health Questionnaire (PHQ) is a 9-item symptom inventory developed to screen for major depressive disorder. It has repeatedly demonstrated strong case-finding properties (eg, sensitivity=0.77–0.80, specificity=0.92–0.94),30 31 including in traumatic brain injury.32 The standard cut-off, >10, has similar diagnostic efficiency for depression after TBI,32–34 though a more conservative approach that also requires at least one of the cardinal depression symptoms (sadness and/or anhedonia) to be endorsed (ie, item rating of 2+) is more appropriate for mTBI,32 35 and so was used for this study. There is symptom overlap with mTBI and depression, but strong evidence that a symptom-inclusive diagnostic approach (ie, counting all possible depressive symptoms towards a diagnosis regardless of etiological attribution) most accurately identifies depression after mTBI.33 36

Anxiety. The Generalized Anxiety Disorder (GAD-7) scale is a 7-item instrument that has been validated a primary care screening disorders for GAD and other anxiety disorders.37 38 A score of 10 or higher is considered a positive screen.

Insomnia. The Insomnia Severity Index (ISI) is a 7-item self-report questionnaire that has been used in sleep disorder research and practice for over 20 years.39 40 In a recent primary care validation study, the ISI had 82% sensitivity and 82% specificity.41 A score of 14 or higher is considered a positive screen.

Headache. The Headache-Attributed Restriction, Disability, Social Handicap and Impaired Participation (HARDSHIP) tool42 was published in 2014 as a WHO collaborative endeavour. It uses an algorithm based on patients’ responses to structured questions to discriminate between the major post-traumatic headache types—migraine, tension-type, and medication overuse headache, based on the International Classification of Headache Disorders. This feature is essential because the ONF guidelines for mTBI advise different interventions for each headache type. Diagnostic accuracy is somewhat higher for migraine-type headaches (sensitivity 0.63–0.83, specificity 0.82–0.99) than for tension-type headaches (sensitivity=0.51–0.64, specificity=0.81–0.99).42 We modified the HARDSHIP for self-report (vs lay interviewer) administration.

As part of the initial assessment, patients completed a health record release authorisation form, giving permission for their family physician/clinic to send their medical chart (for the dates covering the study period) to the research team. Patient participants were encouraged to make an appointment to see their family physician 1–4 weeks after the initial assessment (allowing at least 1 week for the letter to arrive). Patients were blinded to their group assignment.

After patient participants completed the baseline assessment, including the above measures, the research team generated and faxed a letter to their family physician or designated walk-in clinic. In the experimental (guideline implementation tool) condition, the letter introduced the ONF guidelines and the principle that ‘family physicians take a proactive symptom-based management approach to managing concussions’. The main body of the letter reported the patient’s screening test results (scores and interpretations) and the suggestion to ‘consider scheduling a routine clinic visit for this patient to review their positive screening tests’. Appended to the letter were copies of the treatment algorithm from the ONF guidelines for any primary symptom that the patient participant screened positive for. In the control condition, the letter simply directed family physicians where to access the full ONF guidelines online. Their patient’s screening test results were not reported. Neither the experimental or control letter mentioned the research study hypotheses or there were different types of letters or the research objectives to measure physician behaviour change. In this way, physicians were not only blinded to their group assignment but also to the fact that their behaviour was a research study outcome.

At the conclusion of the data collection period, we aimed to audit the medical records for 10% of the sample and examine agreement with patient recall of their physicians’ actions. To meet this goal, assuming at least a 75% response rate, we submitted requests for medical records for 20 patient participants from their family physicians. The requests included a debriefing letter, explaining the research study purpose and procedures, and a copy of the health record release authorisation form signed by the patient.

Outcome measures

We measured physician behaviour via patient recall (at 1-month and 3-month follow-up) regarding what interventions their physicians performed or prescribed since their injury. The questions, patients responded to, are shown in table 1. If a patient reported that their family physician took an action (advice/counselling, referral, or medication prescription) for a particular symptom, they were queried about specific interventions in the ONF guideline treatment algorithm for that symptom. Note that physician actions (eg, medication renewal or change) for a pre-existing symptom that persisted or worsened following the injury would have been captured by this methodology. To assess the validity of this patient recall method, a blinded audit of medical records from family physicians was performed for 10% of the sample. Physician actions were coded using a standardised chart extraction form.

Table 1

Patient recall of their physician’s treatment recommendations

Symptom	Intervention	Control1 month	Exper1 month	Control3 month	Exper3 month
Headache	Not applicable—no headaches.	3/66 (5%)	7/58 (12%)	3/62 (5%)	5/47 (11%)
	Home treatments, such as heat packs and self-massage.	16/63 (25%)	22/51 (43%)	19/59 (32%)	18/42 (43%)
	Advice about lifestyle changes, such as eating regular healthy meals, staying hydrated and getting exercise.	30/63 (48%)	23/51 (45%)	28/59 (48%)	18/42 (43%)
	Non-drug therapy, such as physiotherapy or massage therapy.	34/63 (54%)	33/51 (66%)	27/59 (46%)	24/42 (57%)
	Medications to take regularly each day, whether or not you have a headache.	13/63 (21%)	10/51 (20%)	18/59 (31%)	7/42 (17%)
	Medications to take when you get a headache (‘prn’ or as-needed).	34/63 (54%)	29/51 (57%)	33/59 (56%)	22/42 (52%)
	At least one headache intervention.	43/63 (68%)	39/58 (67%)	43/62 (69%)	29/47 (62%)
Depression /anxiety	Not applicable—no mental health problems.	12/68 (18%)	7/58 (12%)	8/62 (13%)	7/47 (15%)
	General advice about lifestyle changes to improve your mood, such as regular social and physical activity.	17/56 (30%)	15/51 (29%)	19/54 (35%)	17/40 (43%)
	Medication to improve your mood (depression, stress or anxiety).	9/56 (16%)	6/51 (12%)	13/54 (24%)	9/40 (23%)
	Referral to a psychologist or counsellor.	7/56 (13%)	14/51 (28%)	15/54 (28%)	16/40 (40%)
	Referral to a psychiatrist.	1/56 (2%)	4/51 (8%)	3/54 (6%)	4/40 (10%)
	At least one mental health intervention.	22/56 (39%)	26/51 (51%)	28/54 (52%)	24/40 (60%)
	At least one mental health intervention excluding general advice.	14/56 (25%)	17/51 (33%)	21/54 (39%)	19/40 (48%)
Insomnia	No applicable—no sleep problems.	12/68 (18%)	8/58 (14%)	8/62 (13%)	7/47 (15%)
	Ordering tests to check if you have treatment medical conditions that can cause sleep problems, such as blood tests for hormone imbalances or polysomnography for sleep apnea.	2/56 (4%)	4/50 (8%)	5/54 (9%)	5/40 (13%)
	Advice on healthy sleeping habits (sleep hygiene), such as using your bed only for sleep, maintaining a regular sleep schedule and avoiding naps.	20/56 (36%)	12/50 (24%)	15/54 (28%)	14/40 (35%)
	Advice on lifestyle factors that promote good sleep, such as getting light and exercise during the day, and avoiding alcohol, caffeine and heavy meals close to bedtime	21/56 (38%)	13/50 (26%)	16/54 (30%)	14/40 (35%)
	Medications to improve your sleep.	14/56 (25%)	9/50 (18%)	15/54 (28%)	11/40 (29%)
	At least one insomnia intervention.	26/56 (46%)	17/50 (34%)	28/54 (52%)	24/40 (60%)

The denominator for the ‘not applicable’ ratings represents the total number of patients who provided a response to each symptom, at each timepoint. The numerator of that proportion (ie, number of patients who reported ‘not applicable’, meaning that they did not experience that symptom) was subtracted from the denominator for all of the treatment variables. This way, the proportions for the treatment variables represent the number of patients who were provided, referred or prescribed a particular treatment for that particular symptom relative to the number of patients who endorsed having that symptom.

In the 1-month and 3-month follow-up assessments, patient participants also completed a battery of self-reported outcomes. The primary efficacy outcome measure was symptom severity, as measured by the Rivermead Post-Concussion Symptom Questionnaire (RPQ)43. The RPQ prompts patients to rate the current severity with which they are experiencing 16 of the most common symptoms after mTBI. Symptoms are rated on a scale from 0 (not experienced) to 4 (severe problem), and summed to created a total score that ranges from 0 to 64. Note that symptom ratings of 1 are not counted towards the total score (ie, recoded as 0) because they represent ‘no more of a problem’ compared with before the injury.43 Higher total scores indicate greater overall symptom burden. The RPQ is the most widely used outcome measure in mTBI research.5 44 The 1-month and 3-month follow-up assessments also including the PHQ-9, GAD-7, ISI, as well as secondary outcome measures of global disability (WHO Disability Assessment Schedule V.2.0, 12-item self-report45 46) and quality of life (Quality of Life After Brain Injury—Overall Scale47).

Statistical analyses

Patient recall of their physician’s actions to initiate treatment for primary symptoms using interventions recommending by the ONF guidelines was coded as present or absent, and then aggregated for each group (experimental vs control), separately for depression/anxiety, insomnia and headaches. The result is a percentage of patients who received a guideline-compliant intervention, where patients who denied having a particular symptom were excluded from the dominator. These results are presented descriptively. Intraclass correlation coefficients (ICCs) with a two-way random model characterised the agreement between patient recall of physician action and chart audit evidence of physician action.

Linear mixed modelling incorporated all repeated measures of the RPQ to estimate the effect of the intervention on symptom recovery trajectories, accounting for clustering at the level of clinic, physician and patient. The model included available data points for participants who did not complete all outcome assessments. Time was coded categorically (1 vs 3 month follow-up). Baseline RPQ was included as a covariate. ICCs were derived from the model using the formula: (between-cluster variance)/(between-cluster variance+within-cluster variance). Other patient-reported outcomes are reported descriptively.

Results

The 148 enrolled patients were treated by 137 unique family physicians at 114 different primary care clinics. The attrition rate was 11.5% (131 of 148 retained) at the 1-month follow-up and 21.6% (116 of 148 retained) at 3-month follow-up, as illustrated in figure 1. Half (50.0%; 74 of 148) of participants completed the baseline assessment online versus in-person, and 54.2% (71 of 131) and 56.9% (66 of 116) completed the 1-month and 3-month follow-up assessment, respectively, online versus by telephone. Participant characteristics by group, including rates of positive screening, are shown in table 2.

Table 2

Participant characteristics

	Control (N=81)	Experimental (N=67)
Age (M; SD years)	38.9 (12.4)	42.5 (11.6)
Sex (n; % female)	49 (61%)	47 (70%)
Ethnicity (n; %)
Caucasian	51 (63%)	46 (69%)
Asian Canadian	21 (26%)	16 (24%)
Indigenous	9 (11%)	5 (8%)
Education (n, %)
College/university	53 (65%)	51 (76%)
High school diploma	27 (33%)	13 (19%)
Less than high school	1 (1%)	3 (5%)
Pre-existing conditions (n, %)
Headaches	21 (26%)	21 (31%)
Sleep problems	21 (26%)	18 (27%)
Depression	22 (27%)	26 (39%)
Anxiety	25 (31%)	23 (34%)
Previous mild traumatic brain injury/s	28 (35%)	28 (42%)
Injury mechanism (n, %)
Motor vehicle accident	35 (43%)	35 (52%)
Fall	17 (21%)	7 (10%)
Sport/recreation	16 (20%)	8 (12%)
Other	13 (16%)	17 (25%)
Loss of consciousness
Present	13 (16%)	10 (15%)
Absent	57 (70%)	43 (64%)
Unknown	11 (14%)	14 (21%)
Receiving or seeking compensation (n, %)
Yes	48 (59%)	47 (70%)
No	26 (32%)	13 (19%)
Unknown	7 (9%)	7 (11%)
Time from injury to baseline assessment (M, SD days)	40.5 (19.8)	41.1 (19.1)
Screened positive (n, %)
Headaches	56 (69%)	41 (61%)
Migraine	33 (41%)	20 (30%)
Tension type	15 (19%)	13 (19%)
Medication overuse	8 (10%)	8 (13%)
Depression or anxiety	54 (67%)	41 (61%)
Depression	42 (52%)	30 (45%)
Anxiety	42 (52%)	33 (49%)
Insomnia	44 (54%)	34 (51%)
Number of visits to family physician from injury to 1-month follow-up assessment (M, SD)	4.9 (2.8)	5.1 (3.5)
Number of visits to family physician from injury to 3-month follow-up assessment (M, SD)	7.3 (4.6)	6.6 (4.9)

Three patient participants (two in experimental and one in control group) did not receive the intervention as planned; one did not return to see their family physician after the education session, one sought care from a different physician than they initially designated in the baseline assessment and a third designated a chiropractor instead of a family physician. Based on the intention-to-treat principle, these participants were included in all analyses, unless otherwise noted.

Patient recall of physician actions

The rates at which family physicians took specific guideline-compliant actions for specific symptoms, according to patient recall, are reported in table 1. Rates of guideline-compliant treatment (ie, at least one guideline-compliant action was taken for a given symptom) were 8–12 percentage points higher in the experimental (vs control) group for depression/anxiety but not consistently higher in the experimental group for headaches and sleep problems. Across groups, the most commonly prescribed medications for headache, by far, were acetaminophen and/or ibuprofen (n=42), with naproxen (n=9) being the only other medication to be prescribed for more than five patients. Of medications taken for depression or anxiety (n=15), most were selective serotonin reuptake inhibitors (n=12). For sleep problems, the most commonly prescribed medications were melatonin (n=5), amitriptyline (n=3), zopiclone (n=2) and cyclobenzaprine (n=2).

Blinded chart audits

We received 90% (18 of 20) of medical charts requested from family physicians. Several records did not include the full date range requested (ie, did not include the date of the 3-month follow-up assessment), so we analysed the 1-month follow-up patient-reported data only. At the 1-month follow-up, patients self-reported seeing their family physicians M=4.59 (SD=2.18) times since their mTBI and chart audits provided evidence of M=4.12 (SD=2.20) visits over the same period. The agreement between these sources was strong (ICC=0.82, p=0.001) and the disagreements did not appear systematic (the distribution of difference score was approximately symmetric around 0).

In examining the concordance between patient report and chart audit regarding physician action, we considered referrals and prescriptions, and not advice/counselling, because such actions would be expected to be documented less consistently. For mood, the absolute agreement rate was 81% (ICC=0.65, p=0.024). For sleep, the absolute agreement rate was 81% (ICC=0.48, p=0.101). For headaches, the absolute agreement rate was 75% (ICC=0.53, p=0.072).

Patient-reported clinical outcomes

The RPQ total score was the response variable in linear mixed modelling. Several participants were missing a single item on the RPQ. In these cases, missing item scores were imputed with the participant’s mean item score (rounded to nearest whole number) for the 15 symptoms that they provided a rating for on the RPQ. This imputation was performed for 10 participants at the baseline assessment, 3 at the 1-month follow-up, and 2 at the 3-month follow-up.

An initial linear mixed model included random effects for the clinic, physician and participant, but could not reach convergence, likely because there were few clinics with more than one physician and few physicians associated with more than one patient. We prioritised including a random effect for clinic, the unit of randomisation. The group by time interaction term was non-significant (p=0.802), and so was removed to simplify the model. The final model included group (experimental vs control treatment arm) and time (1-month vs 3-month follow-up) as main effects, baseline RPQ scores as a covariate, clinic identification number as a random effect (intercept) and postintervention RPQ scores as the response variable. RPQ measurements at baseline were included as a covariate to adjust the estimates of the main effects. The two main effects, group and time, were significant. The passage of time from month 1 to month 3 (B=−5.6, 95% CI: −7.9 to −3.3), and being in the experimental group versus in the control group (B=−4.0, 95% CI: −7.3 to −0.7) were associated with decreasing RPQ scores. The RPQ recovery trajectories, stratified by group, are shown in figure 2. Q–Q plots of the scaled residuals and dot whisker plots of the random effects indicated that they were approximately normally distributed. A residual versus fitted plot indicated that the model was linear and variance was constant. Few potentially influential outliers were identified using the Cooks distance for participants. The test for statistical significance, which re-evaluates the model by excluding one participant at the time, revealed no changes in the significance of the group or time effect. The mean cluster size for the final model was 1.35 and the ICC was 0.37.

Figure 2

Plot of recovery trajectories by group.

Excluding cases (n=3) that did not receive the intervention as intended had minimal impact on the linear mixed modelling findings. The main effects for time (B=−5.7, 95% CI: −8.0 to −3.4) and group (B=−4.1, 95% CI: −7.5 to −0.7) were marginally stronger. Table 3 provides descriptive statistics for the RPQ and other patient-reported outcome measures.

Table 3

Descriptive statistics for patient-reported outcome measures

	Total sample M (SD)						Control group M (SD)						Experimental group M (SD)
	N	Intake	N	1 month	N	3 month	n	Intake	n	1 month	n	3 month	n	Intake	n	1 month	n	3 month
Rivermead Post-Concussion Symptoms Questionnaire	130	36.7 (13.7)	127	30.7 (15.2)	112	24.5 (14.6)	71	37.1 (13.9)	69	32.7 (14.0)	62	27.3 (14.3)	59	36.4 (13.6)	58	28.2 (16.2)	50	21.1 (14.4)
Personal Health Questionnaire-9	146	13.4 (5.6)	128	11.1 (5.7)	114	8.1 (6.0)	79	13.7 (5.6)	69	11.6 (5.7)	63	8.9 (6.6)	67	13.0 (5.5)	59	10.4 (5.7)	51	7.1 (5.0)
Generalized Anxiety Disorder-7	145	9.8 (5.5)	128	8.6 (5.3)	113	7.1 (5.5)	78	10.1 (5.4)	70	9.1 (5.2)	62	8.2 (5.6)	67	9.6 (5.7)	58	8.0 (5.3)	51	5.7 (5.1)
Insomnia Severity Index	147	14.3 (7.3)	127	12.2 (6.7)	113	10.4 (6.8)	80	14.2 (7.6)	69	12.0 (6.6)	63	10.6 (7.1)	67	14.4 (7.1)	58	12.5 (6.9)	50	10.2 (6.6)
WHO Disability Assessment Schedule	–	–	126	30.3 (10.2)	110	24.8 (9.8)	–	–	68	31.0 (9.9)	60	26.3 (10.6)	–	–	58	29.6 (10.5)	50	22.9 (8.5)
Quality of Life After Brain Injury—Overall Scale	–	–	128	49.6 (18.9)	114	56.5 (19.9)	–	–	70	47.5 (17.4)	63	52.4 (18.9)	–	–	58	52.2 (20.3)	51	61.6 (20.0)

Discussion

Evidence to guide the clinical management of mTBI has grown exponentially and expert consensus on best practices is emerging. The ONF guidelines7 highlight proactive screening and treatment for primary symptoms (depression, anxiety, insomnia and headache) by family physicians as a core principle. Widespread penetration of this best practice will require active implementation with evidence-based strategies. The present pilot cluster randomised trial evaluated the feasibility of a guideline implementation tool tailored to the family physician user, with actionable patient-specific screening test results to support the clinical encounter. The findings supported the feasibility of recruiting patients with mTBI directly and having them identify where they access primary care follow-up. Most patients (97.0%; 511 of 527 screened) identified a specific family physician or walk-in clinic where they intended to access follow-up care. Of those patients who were eligible and enrolled, 97.8% (128 of 131) actually visited the physician/walk-in clinic they planned to at least once over the month following enrolment. Retention rates at 3-month follow-up were reasonable, and importantly, comparable in the experimental (78%; 63 of 81) and control (79%; 53 of 67) groups. However, additional retention strategies may be necessary to achieve a more desirable (<15%) attrition rate. There was moderate agreement on family physicians’ actions (referrals and prescriptions) between patient recall and chart audit. Rather than providing a compelling validation of patient recall as a proxy measure for physician behaviour, the results of this pilot study suggest that caution is warranted in interpreting patient recall and that using family physicians chart to obtain implementation outcome data may be feasible (and not dependent on retaining patients in the study).

In addition to evaluating these feasibility indicators, the present study aimed to estimate parameters necessary to calculate sample size requirements for a definitive cluster randomised trial, that is, the intervention effect size and clustering metrics. In intention-to-treat analyses, the guideline implementation tool was associated with a 4-point reduction in global symptom burden on the RPQ. The meaningfulness of a 4-point difference between the experimental and control group is unclear, as a minimally clinically important difference has not been established for this measure. For reference, 2 months of recovery time (from the 1-month to 3-month follow-up) were associated with a 5.6-point reduction on the RPQ. As shown in table 3, a similar trend favouring the experimental group is apparent across a number of secondary outcome measures.

Clustering was estimated to be high (ICC=0.37) compared with prior cluster randomised trials in primary care,48 likely because the cluster sizes in the present study were very small (mean=1.35, median=1, n=90;79% had only one enrolled patient) and it was not possible to simultaneously account for clustering at the physician or patient level. Rather than recruiting patients directly (eg, through an outpatient specialty clinic), future studies might consider recruiting primary care clinics to obtain greater within-cluster variance.

We hypothesised that the guideline implementation tool would be associated with higher rates of ONF guideline-compliant intervention, supporting this variable as a mechanistic outcome (how the implementation tool reduces patients’ symptom burden). The guideline implementation tool did not appear to increase headache or insomnia treatment. The majority of patients were treated for headaches, primarily with over-the-counter analgesics. In the ONF guidelines, this is a reasonable first step. It may be that family physicians are already familiar and comfortable with first-line headache management strategies. A longer observation period may be necessary to detect intervention-related differences as family physicians move to second-line and third-line treatment options for headaches after mTBI. Timely treatment for insomnia was actually somewhat higher in the control group. The ONF guidelines advise against prescribing benzodiazepines for sleep. However, greater compliance with this advice cannot account for the lower rate of treatment (including prescription of medications) in the experimental group because only one patient participant in the whole sample reported being prescribed a benzodiazepine (lorazepam) for sleep.

Rates of receiving at least one mental health treatment were 8–12 percentage points higher in the experimental group. This effect size is in the range of what has been reported in prior guideline implementation trials.23 28 The present study was likely not sufficiently powered to detect a statistically significant difference of this arguably important magnitude. The overall difference in mental health treatment was largely driven by a higher rate of referral to psychological therapy/counselling in the experimental group (28% at 1 month and 40% at 3 months) compared with the control group (13% at 1 month and 28% at 3 months), which is consistent with the ONF recommendation for non-pharmacological treatment in patients with mild–moderate severity mental health disorders, which represent the vast majority. Differences favouring the experimental group were smaller at the 3-month follow-up. It is possible that the guideline implementation tool improves the timeliness of mental health treatment access, but not the overall access rate.

The relatively low rates of treatment for mental health and sleep problems in comparison to the number of patients screening positive in these areas suggest possible undertreatment. Some positive screening test findings could have been false positives. For example, a family physician may have judged that a patient with a high Insomnia Severity Inventory Score does not, in fact, have a sleep disorder that requires treatment. It is also possible that some patients were already being treated for pre-existing mental health and/or sleep problems at the time of their injury, and so no new action on the part of the family physician was required. As seen in table 2, a history of mental health and/or sleep problems prior to the injury was not uncommon. A future trial should distinguish between stable pre-existing conditions, pre-existing conditions exacerbated by mTBI and new mTBI-related conditions.

This pilot study has not only strengths, including randomisation and double blinding, but also important limitations. We relied on patient recall of what their family physicians did, which did not correspond perfectly to what treatment recommendations and referrals were documented in the medical chart. It is unclear whether physicians took actions that were not consistently documented and/or patients failed to recall accurately some actions their doctors took. Prior research suggests that structured inquiry (as used in the present study) facilitates patient recall of treatment decisions, but even cued recall may be incomplete or inaccurate 15% of the time.49 It is also possible that the implementation tool prompted clinically helpful family physician behaviour change in ways not captured by our measurement of guideline-compliant actions. The sample size for the present study was pragmatic, as the focus of this pilot phase trial was to assess feasibility and estimate the intervention effect to inform planning of a larger trial. A larger trial should be powered to conduct subgroup analyses to clarify whether the implementation intervention is most effective for patients with mildly elevated screening test scores (eg, bringing relatively subtle but clinically significant primary symptoms to family physicians’ attention) versus patients with moderately to severely elevated scores (eg, encouraging physician action in the smaller number of patients who can most benefit from aggressive management). Participants were recruited from outpatient concussion clinics, and so are likely representative of the subpopulation of patients with mTBI who seek care for persistent symptoms, but not generalisable to the mTBI population. The high rates of positive screening for primary symptoms are evidence of selection bias. Another limitation is that the intervention and study design did not allow us to confirm whether the treating physician actually reviewed the follow-up letter. We are aware that in some group practices, especially walk-in clinics, the physician assigned to review faxed lab tests and other incoming documents may not be the same physician who sees the patient in follow-up. The implementation tool in the present study was based on the second edition of the ONF guidelines.7 An updated version was published in June 2018,7 but retained the core recommendation to prioritise management of headache, anxiety/depression and insomnia in primary care.

It may be possible to enhance the effectiveness of our implementation intervention, such as by adding a patient-mediated component50 51 to facilitate their engagement in the clinical encounter (eg, discussion of treatment options) and follow through on treatment recommendations might enhance the intervention effect. A narrower focus, such as depression/anxiety, could simplify the guideline implementation tool and remove the onus on prioritising recommended actions, thereby increasing its use. It is also possible that identifying and screening patients earlier would enhance the impact of the guideline implementation tool. In the present study, the implementation intervention occurred 6 weeks post injury, on average. Additional input from qualitative interviews with knowledge user’s regarding perceived barriers and facilitators could help refinement of the guideline tool.52

In conclusion, this pilot study generally supported the feasibility of our intervention approach and trial design, but highlighted challenges that will need to be addressed in a definitive efficacy trial. Further research is needed to determine whether a guideline implementation tool to support the core ONF guideline to proactively screen and initiate treatment for primary symptoms (depression, anxiety, insomnia and headache) after mTBI in primary care improves patient outcomes and why (ie, clarify the mechanisms underlying the intervention effect).