Low intensity pulsed ultrasound for bone healing: systematic review of randomized controlled trials.

Objective To determine the efficacy of low intensity pulsed ultrasound (LIPUS) for healing of fracture or osteotomy.Design Systematic review and meta-analysis.Data sources Medline, Embase, CINAHL, Cochrane Central Register of Controlled Trials, and trial registries up to November 2016.Study selection Randomized controlled trials of LIPUS compared with sham device or no device in patients with any kind of fracture or osteotomy.Review methods Two independent reviewers identified studies, extracted data, and assessed risk of bias. A parallel guideline committee (BMJ Rapid Recommendation) provided input on the design and interpretation of the systematic review, including selection of outcomes important to patients. The GRADE system was used to assess the quality of evidence.Results 26 randomized controlled trials with a median sample size of 30 (range 8-501) were included. The most trustworthy evidence came from four trials at low risk of bias that included patients with tibia or clavicle fractures. Compared with control, LIPUS did not reduce time to return to work (percentage difference: 2.7% later with LIPUS, 95% confidence interval 7.7% earlier to 14.3% later; moderate certainty) or the number of subsequent operations (risk ratio 0.80, 95% confidence interval 0.55 to 1.16; moderate certainty). For pain, days to weight bearing, and radiographic healing, effects varied substantially among studies. For all three outcomes, trials at low risk of bias failed to show a benefit with LIPUS, while trials at high risk of bias suggested a benefit (interaction P<0.001). When only trials at low risk of bias trials were considered, LIPUS did not reduce days to weight bearing (4.8% later, 4.0% earlier to 14.4% later; high certainty), pain at four to six weeks (mean difference on 0-100 visual analogue scale: 0.93 lower, 2.51 lower to 0.64 higher; high certainty), and days to radiographic healing (1.7% earlier, 11.2% earlier to 8.8% later; moderate certainty).Conclusions Based on moderate to high quality evidence from studies in patients with fresh fracture, LIPUS does not improve outcomes important to patients and probably has no effect on radiographic bone healing. The applicability to other types of fracture or osteotomy is open to debate.Systematic review registration PROSPERO CRD42016050965. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

For over 20 years, patients have used low intensity pulsed ultrasound (LIPUS) as an adjunct treatment to improve bone healing. Based on radiographic outcomes, the US Food and Drug Administration and the UK National Institute for Health and Care Excellence (NICE) have approved LIPUS for fracture healing.1 2 Depending on country and device, LIPUS devices currently cost around £1000-4000 (€1170-4700; $1250-5000). In 2008, 45% of trauma surgeons in Canada prescribed bone stimulators to manage tibia fractures, equally split between LIPUS and electrical stimulation (21% each).3 In 2006 sales from LIPUS amounted to about $250m in the US alone.3 4

Within the past seven years, 10 systematic reviews have assessed the effectiveness of LIPUS for bone healing.5 6 7 8 9 10 11 12 13 14 Because existing randomized controlled trials were limited by small sample size, risk of bias, inconsistent results, and failure to cover outcomes important to patients, no review offered definitive conclusions. All reviews identified the need for additional randomized controlled trials. In addition, recent reviews used suboptimal strategies for outcome selection, data synthesis, analysis, and interpretation, leading to potentially misleading conclusions. For instance, the most recent systematic review,11 published in the top specialty journal for orthopedic surgeons, considered radiographic union a “critically important outcome” and did not assess the effect of LIPUS on the outcomes of pain relief or re-operation, which are important to patients. Their conclusion that “LIPUS treatment effectively reduces the time to radiographic fracture union” is questionable because it is based on the pooled absolute difference in days to healing, which does not account for the large variation in healing time, showed high unexplained heterogeneity (I2=94%), and was driven by studies at high risk of bias. This positive conclusion has the potential to expand the already considerable use of a potentially ineffective treatment.

This systematic review is part of the BMJ Rapid Recommendations project, a collaborative effort from the MAGIC research and innovation program (www.magicproject.org) and BMJ. The aim of the project is to respond to new potentially practice changing evidence and provide a trustworthy practice guideline in a timely manner.15 In this case, the publication of the TRUST trial,16 a multicentre trial that randomized 501 patients with tibia fractures and has cast doubt on the effectiveness of LIPUS, initiated the process. This systematic review informed a parallel guideline published in a multi-layered electronic format on bmj.com 17 and MAGICapp (https://www.magicapp.org/public/guideline/mL6yYj).

We assessed whether LIPUS, compared with sham device or no device, improved outcomes important to patients and radiographic healing in those with any kind of fracture or osteotomy.

Methods

Guideline panel and patient involvement

According to the BMJ Rapid Recommendations process,15 a guideline panel provided critical oversight to the review and identified populations, subgroups, and outcomes of interest. The panel included six content experts (five orthopedic or trauma surgeons and one physiotherapist), six methodologists (four of whom are also front line clinicians), and four patients with personal experience of fractures (one of whom had used LIPUS). All patients received personal training and support to optimize contributions throughout the guideline development process. The members of the patient panel led the interpretation of the results based on what they expected the typical values and preferences of patients to be, as well as the variation between patients.

Information sources

We searched Medline, PubMed, Embase, CINAHL, and the Cochrane Central Register of Controlled Trials up to 16 November 2016, using a combination of keywords and MeSH terms for fracture, orthopedic surgical procedures, and ultrasound. Additional searches included trials registries clinicaltrials.gov and isrctn.com. An experienced research librarian designed the search strategies (appendix 1). Two independent reviewers scanned the references from eligible studies, related systematic reviews, and all studies citing eligible randomized controlled trials on Google Scholar.

Study selection

We included randomized controlled trials that compared LIPUS with a sham device or no device in patients with any type of fracture regardless of location (long bone or other bone), type (fresh fracture, delayed union, non-union, or stress fracture), or clinical management (operative or non-operative). We included any type of osteotomy, including distraction osteogenesis. We excluded trials published only as protocols or abstracts if we were unable to get the final results from investigators.

Two reviewers, independently and in duplicate, screened the titles and abstracts of identified articles and acquired the full text of any article that either reviewer judged to be potentially eligible. They independently applied the eligibility criteria to the full texts and, when consensus could not be reached, resolved disagreements through discussion or adjudication by a third reviewer.

Data collection

Two reviewers used standardized forms to independently abstract data; they resolved disagreements by discussion or involved a third reviewer when required. Extracted data included characteristics of patients and fractures, clinical management, risk of bias, intervention details, statements about compliance with treatment, and outcomes.

Risk of bias assessment

Two reviewers independently assessed risk of bias using a modified Cochrane risk of bias instrument that includes response options of “definitely or probably yes” (assigned a low risk of bias) or “definitely or probably no” (assigned a high risk of bias), an approach we have previously validated.18 On the study level, we assessed generation of randomization sequence, concealment of allocation, blinding of patients, caregivers, and outcome reporting (by comparing each publication with its corresponding published protocol, when available). For each outcome within studies, we assessed blinding of outcome assessors, loss to follow-up, and additional limitations. We considered ≥20% loss to follow-up to represent a high risk of bias, unless the investigators performed appropriate sensitivity analyses showing the robustness of the results. We categorized a trial as being at low risk of bias for a particular outcome if we identified no limitation for any risk of bias item. As a post hoc sensitivity analysis, we alternatively considered a more conservative threshold of ≥10% loss to follow-up because the categorization of three trials depended on this criterion (19%,16 28%,19 and 31%20 loss to follow-up for the outcome radiographic healing).

Outcomes

Patients identified functional recovery (time to return to work and time to full weight bearing), pain reduction, and number of operations for subsequent fracture or related to osteotomy (re-operation for operatively managed fracture and osteotomy) as the most important outcomes for patients considering LIPUS for bone healing. Because many clinicians currently base their management on time to radiographic healing, a surrogate outcome important only insofar as it influences patient experience, the panel requested its inclusion in our review. We extracted all outcomes that fell into these categories as well as adverse effects related to the ultrasound device.

Synthesis of results

We pooled treatment effects of LIPUS on similar outcomes across eligible trials, regardless of clinical subgroups, focusing on complete case analysis. We calculated pooled estimates and associated 95% confidence intervals using random effects models for meta-analysis with three or more studies and fixed effects models for meta-analysis with two studies. We examined heterogeneity associated with all pooled analyses using both the χ2 test and I2 statistic. SAS version 9.4, R version 3.1, and Review Manager 5.3 provided software for the statistical analysis.

For time to event outcomes, we pooled hazard ratios. For studies that did not apply methods of survival analysis, we considered time to event reported as a continuous variable (for example, days to return to work) at the longest follow-up time. We used the relative effect measure ratio of means (mean LIPUS/mean control) to account for the baseline difference in fracture healing depending on type of bone and (such as scaphoid, clavicle, tibia, etc) and fracture or procedure (such as stress fracture or distraction osteogenesis). We pooled the natural logarithm of the ratio of means and presented the results as percentage difference (relative change). For studies that reported the proportion of patients who achieved the event at a specific time point, we calculated risk ratios.

When studies used different instruments to measure the same construct on a continuous scale, we converted all instruments to the most commonly used instrument among studies and then pooled results using the weighted mean difference.21

For the outcomes number of subsequent operations and adverse events related to the device, we calculated both risk ratios, which are preferable in case of varying baseline risks, and risk differences, which allow inclusion of studies with zero events in both groups.

In consultation with the expert and patient guideline panel, we prespecified three subgroup hypotheses to explain heterogeneity of effects between studies: LIPUS will show larger effects in studies at high risk of bias; effects will differ based on clinical subgroups; and LIPUS will show larger in studies effects with greater patient compliance. In consultation with the six clinical experts on the parallel guideline panel, we classified eligible randomized controlled trials according to the following five clinical subgroups: operatively managed fresh fractures, non-operatively managed fresh fractures, stress fractures, non-union, and osteotomy (including distraction osteogenesis). Because compliance was inconsistently reported, two reviewers independently categorized trials using response options of “definitely or probably high compliance” or “definitely or probably moderate compliance,” using as a guide a definition of high compliance as at least 80% of patients who applied LIPUS for at least 80% of the total time prescribed. We conducted univariable tests of interaction to establish if the effect size from the subgroups differed significantly from each other and, to test independence of subgroup effects, performed multivariable meta-regression in which we included risk of bias (high versus low), compliance with LIPUS treatment (high versus moderate), and clinical subgroups (as above) as independent variables in a single model.

Only one outcome, days to radiographic healing, included enough studies to perform all planned subgroup analysis. We had prespecified in our protocol at least three studies per group. We assessed the credibility of significant subgroup effects using the criteria suggested by Sun and colleagues.22 Based on the finding that risk of bias seemed to independently explain the high heterogeneity in the outcome days to radiographic healing, we performed subgroup analysis by risk of bias for all outcomes.

The authors and the guideline panel achieved consensus in categorizing the quality of evidence for all reported outcomes as high, moderate, low, or very low using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. In the GRADE approach, randomized controlled trials begin as high quality but can be rated down because of risk of bias, inconsistency, indirectness, imprecision, or publication bias.23 We considered rating down for inconsistency if the magnitude and direction of effects were dissimilar, the confidence intervals had minimal overlap, the test of heterogeneity was significant, or the I2 was high.24 For outcomes with 10 or more studies, we inspected symmetry of funnel plots and performed Egger’s statistical test for publication bias.25

To calculate absolute effects, we applied the effect estimate from the meta-analysis to the control arm of the TRUST trial, which enrolled patients with tibia fractures and had the largest sample size of any eligible study that was at low risk of bias. The approach to rating certainty of individual outcomes was fully contextualized—that is, in rating quality about any individual outcome, we took into account the findings on the other outcomes.

Patient involvement

Four patient representatives were full members of the guideline and contributed to the selection and prioritization of outcomes, values and preferences assessments, and critical feedback to the protocol for the systematic review and the BMJ Rapid Recommendations manuscript.

Results

Search results

We identified 3849 potentially eligible abstracts, retrieved 42 studies in full text, and found 26 eligible randomized controlled trials (fig 1).16 19 20 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Two trials, Handolin and colleagues32 33 and Emami and colleagues,29 30 provided two publications reporting on the same group of patients. There were no shared patients between the TRUST pilot26 and the definitive trial.16 Our registry search yielded four protocols of potentially eligible trials; one was discontinued because of slow recruitment (ISRCTN90844675, personal communication, outcome data not available yet), one manuscript is under peer review (NCT00744861, personal communication: “no difference between the control group and the ultrasound group”), one is completed but unpublished (JPRN-UMIN000002005, no response from investigators), and the last is still ongoing (NCT02383160). Attempts to acquire the full text of another potentially eligible study,51 reported in a recent systematic review,11 were unsuccessful.

Fig 1 Studies included in review of low intensity pulsed ultrasound compared with control (sham device or no device) for patients with fracture or osteotomy

Study characteristics

Eligible trials enrolled patients with operatively managed fresh fractures (n=7); non-operatively managed fresh fractures (n=6); stress fractures (n=2); non-unions (n=3); and osteotomies (n=8), of which five were distraction osteogenesis (table 1). Most trials enrolled patients with tibia fractures or osteotomies (n=14). All but two trials applied LIPUS for 20 minutes every day, either for a fixed period or until radiographic healing. Otherwise, one trial applied LIPUS for 15 minutes a day,36 and another trial for five minutes every second day.39 Fifteen trials (60%) provided their control group with an inactive device that was indistinguishable from the active LIPUS. Only three trials (12%) were explicitly free from industry funding.28 42 48

Table 1

Characteristics of studies included in review of low intensity pulsed ultrasound for bone healing after fracture

	Bone	Type of fracture/surgery	Open fracture (%)	Management	Women (%)	Mean age (years)	No of randomized patients		Sham device	Dose and duration of LIPUS	Maximum follow-up	Explicit free of industry funding
							LIPUS	No ultrasound
Busse, 201426	Tibia	Fresh fracture	27	Operative	24	40	23	28	Yes	20 min/day to healing*	1 year	No
Busse, 201616	Tibia	Fresh fracture	23	Operative	31	40	250	251	Yes	20 min/day to healing*	1 year	No
Dudda, 201127	Tibia	Distraction osteogenesis	NA	Operative	11	39	16	20	No	20 min/day to healing*	35 weeks	No
El-Mowafi, 200528	Tibia	Distraction osteogenesis	NA	Operative	0	35	10	10	No	20 min/day to healing*	12 months	Yes
Emami 199929 30	Tibia	Fresh fracture	13	Operative	25	37	15	17	Yes	20 min/day to healing*	20 weeks	No
Gan, 201431	Tibia, fibula, metatarsal	Stress fracture	0	Non-operative	83	30	15	15	Yes	20 min/day for 28 days	12 weeks	No
Handolin, 2005a32 33	Lateral malleolus	Fresh fracture	0	Operative	47	42	11	11	Yes	20 min/day for 42 days	12 weeks	No
Handolin, 2005b34	Lateral malleolus	Fresh fracture	0	Operative	56	40	15	15	Yes	20 min/day for 42 days	18 months	No
Heckman 199420	Tibia	Fresh fracture	4	Non-operative	19	33	48	49	Yes	20 min/day to healing*	140 days	No
Kamath, 201545	Tibia and femur	Fresh fracture	0	Operative	NR	36	33	27	No	20 min/day for 1 month	16 weeks	No
Kristiansen 199719	Distal radius	Fresh fracture	0	Non-operative	84	56	40	45	Yes	20 min/day for 70 days	140 days	No
Leung, 200435	Tibia	Fresh fracture	47	Operative	11	35	16	14	Yes	20 min/day for 4 months	5 months	No
Liu, 201436	Distal radius	Fresh fracture	NR	Non-operative	36	67	41	40	No	15 min/day for ≥12 weeks	At least 12 weeks	No
Lubbert, 200837	Clavicle	Fresh fracture	0	Non-operative	16	38	61	59	Yes	20 min/day for 28 days	8 weeks	No
Mayr, 200038	Scaphoid	Fresh fracture	0	Non-operative	17	37	15	15	No	20 min/day to healing*	120 days	No
Patel, 201439	Mandible	Fresh fracture	NR	Non-operative	25	15-35	14	14	No	5 min qad for 24 days	5 weeks	No
Ricardo, 200640	Scaphoid	Non-union	NA	Operative	0	27	10	11	Yes	20 min/day to healing*	4 years	No
Rue, 200442	Tibia	Stress fracture	0	Non-operative	50	19	Probably 20	Probably 20	Yes	20 min/day to healing*	NR	Yes
Rutten, 201241	Tibia	Non-union	0	Operative	70	41-63	10	10	Yes	20 min/day for 5 months	5 years	No
Salem, 201443	Tibia	Distraction osteogenesis	NA	Operative	14	30	12	9	No	20 min/day to healing*	NR	No
Schofer, 201044	Tibia	Non-union	NA	Operative	24	44	51	50	Yes	20 min/day for 16 weeks	16 weeks	No
Schortinghuis, 200546	Mandible	Distraction osteogenesis	NA	Operative	75	65	4	4	Yes	20 min/day for 4 weeks	30 months	No
Schortinghuis, 200847	Mandible	Distraction osteogenesis	NA	Operative	NR	56	5	4	Yes	20 min/day for 6 weeks	44 months	No
Tsumaki, 200448	Tibia	Distraction osteogenesis	NA	Operative	81	68	21 knees	21 knees	No	20 min/day to healing*	NR	Yes
Urita, 201349	Ulna and radius	Osteotomy (shortening)	NA	Operative	63	48	14	13	No	20 min/day to healing* or 12 weeks	24 weeks	No
Zacherl, 200950	Hallux valgus	Osteotomy (deformity correction)	NA	Operative	85	53	26 toes	26 toes	Yes	20 min/day for 42 days	1 year	No

NA=not applicable; NR=not reported; qad=every other day.

*Until radiographic healing.

Risk of bias

We contacted authors to resolve areas of uncertainty and successfully clarified details in five randomized controlled trials.32 33 35 37 40 We considered six trials to be at low risk of bias,16 26 29 37 46 47 and the remaining 20 studies to be at high risk of bias (table 2). The main limitations were failure to report method for allocation concealment (15 trials), unblinded patients (10 trials), unblinded caregivers or outcome assessors (10 trials), and high or unclear numbers of patients excluded from the analysis (13 trials; table 2).

Table 2

Risk of bias in studies included in review of low intensity pulsed ultrasound for bone healing after fracture

	Sequence generation adequate	Concealment of treatment allocation	Patients blinded	Caregivers blinded	Outcome assessors blinded	Outcomes reported as planned (link to protocol)*	No other bias detected	Loss to follow-up (%) for outcome radiographic healing unless specified otherwise
Busse, 201426	Yes	Yes	Yes	Yes	Yes	Yes	Yes	2%
Busse, 201616	Yes	Yes	Yes	Yes	Yes	Yes	Yes	19% for radiographic healing, 11% for return to work, 9% for weight bearing
Dudda, 201127	Yes	No	No	No	No	Unclear	Yes	Unclear, assumed to be 0
El-Mowafi, 200528	Yes	No	No	No	No	Unclear	Yes	5%
Emami 199929 30	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	3%
Gan, 201431	Yes	No	Yes	Yes	Yes	Unclear	Yes	23% (pain)
Handolin, 2005a32 33	Yes	No	Yes	Yes	Yes	Unclear	Yes	5%
Handolin, 2005b 34	Yes	No	Yes	Yes	Yes	Unclear	Yes	No eligible outcome reported
Heckman 199420	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	31%
Kamath, 201545	Yes	No	No	No	Yes	Unclear	Yes	No eligible outcome reported
Kristiansen 199719	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	28%
Leung, 200435	No†	No† ‡	No‡	No‡	No‡	Unclear	No§	Unclear, assumed to be 0
Liu, 201436	Yes	No	No	No	Yes	Unclear	No¶	Unclear, assumed to be 0
Lubbert, 200837	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	16% (days to return to work)
Mayr, 200038	Yes	No	No	No	Yes	Unclear	Yes	0
Patel, 201439	Yes	No	No	No	No	Unclear	Yes	Unclear, assumed to be 0
Ricardo, 200640	Yes	No	Yes	Yes	Yes	Unclear	Yes	Unclear, assumed to be 0
Rue, 200442	Yes	No	Yes	Yes	Yes	Unclear	Yes	Unclear, probably 35%
Rutten, 201241	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	45%
Salem, 201443	Yes	No	No	No	No	Unclear	Yes	Unclear, assumed to be 0
Schofer, 201044	Yes	Yes	Yes	Yes	Yes	Unclear	No**	Unclear, assumed to be 0
Schortinghuis, 200546	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	0 for subsequent operation
Schortinghuis, 200847	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	0 for subsequent operation
Tsumaki, 200448	Yes	Yes	No	No	No	Unclear	No††	Unclear, assumed to be 0
Urita, 201349	No‡‡	No	No	No	Yes	Unclear	Yes	Unclear, assumed to be 0
Zacherl, 200950	Yes	No	Yes	Yes	Yes	Unclear	No§§	Not included in meta-analysis, insufficient reporting§§

*Studies by Busse et al16 26 were protocol NCT00667849; for all other studies no protocol published and trial not registered.

†Quasi-randomized based on sequence of admission.

‡Inactive device distinguishable from active device.

§Unadjusted clustering, 30 fractures of 28 patients were randomized.

¶Implausibly narrow confidence intervals.

**Prognostic imbalance: patients with non-union fractures in LIPUS arm were considerably older.

††Bilateral surgery: one tibia randomized to LIPUS and one to no treatment. Correlation of 0.5 assumed in our analysis of days to radiographic healing.

‡‡Used odd-even system for treatment allocation.

§§Randomized 44 patients but analyzed 52 toes, clustering unclear, SDs not reported.

Table 3 summarizes findings of all outcomes. Interactive tables summarizing findings are available online at https://www.magicapp.org/public/guideline/mL6yYj.

Table 3

GRADE summary of findings in review of low intensity pulsed ultrasound for bone healing after fracture

Outcome	Study results (95% CI) and measurements	Absolute effect estimates			Quality of evidence	Narrative summary
		No ultrasound	LIPUS	Difference (95% CI)
Days to return to work	% difference: 2.7% (−7.7% to 14.3%) in days, lower better. Based on data from 392 patients in 3 studies	Mean 200 days	Mean 205 days	5 days later (15 earlier to 20 later)	Moderate*	LIPUS probably has little or no impact on time to return to work
Days to full weight bearing	% Difference: 4.8% (−4.0% to 14.4%) in days, lower better. Based on data from 483 patients in 2 trials at low risk of bias	Mean 70 days	Mean 73 days	3 days earlier (3 earlier to 10 later)	High	LIPUS has no impact on time to full weight bearing
Pain reduction. Follow-up 4-6 weeks	Mean difference: −0.93 (−2.51 to 0.64) 0 to 100 visual analogue scale, lower better, minimal important difference: 10-15. Based on data from 626 patients in 3 trials at low risk of bias	Mean 40	Mean 39	1 lower (3 lower to 1 higher)	High	LIPUS has no impact on pain reduction
Subsequent operations. Follow-up 8 weeks-44 months	Risk ratio: 0.80 (0.55 to 1.16). Based on data from 740 patients in 7 studies	160/1000	128/1000	32 fewer (72 fewer to 26 more)	Moderate*	LIPUS probably has little or no impact on subsequent operation
Days to radiographic healing	% Difference: −1.7% (−11.2% to 8.8%) in days, lower better. Based on data from 483 patients in 3 trials at low risk of bias	Mean 150 days	Mean 147 days	3 days earlier (17 earlier to 13 later)	Moderate*	LIPUS probably has little or no impact on time to radiographic healing
Adverse effects related to device. Follow-up 5-52 weeks	Risk difference: 0% (−1% to 1%). Based on data from 839 patients in 9 studies	0/1000	0/1000	0 fewer (10 fewer to 10 more)	High	LIPUS has no impact on adverse effects related to device

*Because of serious imprecision.

Functional recovery

Only the TRUST trial assessed time to return to work with a time to event analysis and found no significant effect (hazard ratio 1.11 favoring control, 95% confidence interval 0.82 to 1.50; 343 patients).16 Three trials assessed the number of days to return to work; the pooled effect was not significant (2.7% later return with LIPUS, 95% confidence interval 7.7% earlier to 14.3% later; I2=0%; 392 patients) (fig 2). We found no significant interaction with risk of bias (P=0.86). If we consider an alternative threshold of ≥10% loss to follow-up for assessing risk of attrition bias, all three studies would fall into the category of high risk of bias. Given the consist absence of effects, however, this would not lower our confidence in the result. A fourth trial in patients with delayed union of tibia fracture provided insufficient data for inclusion in meta-analysis (table 2) but reported no significant difference in days to return to work.50

Fig 2 Difference in days to return to work after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

Only the TRUST trial assessed time to full weight bearing with a time to event analysis and found no significant effect (hazard ratio 0.87 favoring LIPUS, 95% confidence interval 0.70 to 1.08; 451 patients). Three trials assessed the number of days to full weight bearing. Overall results suggested no significant effect on full weight bearing with LIPUS but high heterogeneity (I2=95%). The effect of the one trial at high risk of bias (40.0% earlier, 48.4% to 30.3 earlier)35 differed significantly from the consistent results from the two trials at low risk of bias (4.8% later, 4.0% earlier to 14.4% later; 483 patients; interaction P<0.001, subgroup effect not effected by alternative threshold for missing data) (fig 3).

Fig 3 Difference of days to full weight bearing after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device), by risk of bias. Interaction P<0.001

Appendix 2 presents results of other functional outcomes including return to leisure activities, return to household activities, return to level of function before injury, and physical function measured with a multidimensional questionnaire. None of these was significantly affected by use of LIPUS, nor did they show substantial inconsistency.

Pain reduction

Four trials assessed pain, two with a 100 mm visual analogue scale37 49 and two with the “bodily pain” subdomain of the SF-36 instrument.16 26 After we transformed all results to a 100 mm visual analogue scale, findings at three to six weeks’ follow-up showed no significant effect of LIPUS on pain reduction but high heterogeneity (I2=97%). The effect of the one trial at high risk of bias (28.12 mm lower, 95% confidence interval 37.05 to 19.19 lower)39 differed significantly from the consistent results from the three trials at low risk of bias (0.93 mm lower, 2.51 lower to 0.64 higher; 626 patients; I2=0%; interaction P<0.001; fig 4). The subgroup effect was no longer significant when we used a threshold of ≥10% missing data to designate a trial at high risk of attrition bias (P=0.35, fig A in appendix 3). Two other small studies assessed pain intensity at five months but could not be included in the meta-analysis. One reported pain outcomes only narratively (no effect),41 and another used a modified instrument with unclear scale and variance (no effect).49

Fig 4 Mean difference of pain reduction after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device) by risk of bias. All instruments transformed to 0-100 visual analogue scale. Interaction P<0.001

Other outcomes for pain included pain intensity assessed at multiple time points and number of painful days (appendix 4). None showed a significant effect of LIPUS nor substantial inconsistency.

Number of subsequent operations

Ten trials reported the number of subsequent operations, including three trials reporting zero events in both arms. Neither the pooled risk ratio (0.80 in favor of LIPUS, 95% confidence interval 0.55 to 1.16; I2=0%; seven trials, 693 patients; fig 5) nor the pooled risk difference (3% reduction with LIPUS, 7% reduction to 2% increase; I2=0%; 10 trials, 740 patients) showed a significant effect. There was no significant interaction with risk of bias on either scale (risk ratio: P=0.75; risk difference: P=0.64). The results did not depend on the threshold for missing data.

Fig 5 Risk ratio of number of subsequent operations related to fracture after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

Time to radiographic healing

Two trials used time to event analysis methods to assess time to radiographic healing16 26 and showed no significant effect of LIPUS (hazard ratio 1.06 in favor of control, 95% confidence interval 0.86 to 1.32; I2=0%; 532 patients). Fifteen trials reported the number of days to radiographic healing. Overall results suggested accelerated radiographic healing with LIPUS (26% earlier, 33.6% to 17.8% earlier; I2=85%). The effect differed significantly between the 12 trials at high risk of bias (32.8% earlier, 39.5% to 25.3% days earlier; I2=78%; 446 patients) and the three trials at low risk of bias (1.7% earlier, 11.2% earlier to 8.8% later, I2=10%; 483 patients; interaction P<0.001; fig 6). This subgroup effect fulfilled eight of nine credibility criteria relevant to risk of bias as an explanation of heterogeneity (table 4). In addition, the subgroup effect was robust to our sensitivity analysis using a more conservative threshold for defining risk of attrition bias (interaction P=0.004, fig B in appendix 3). The effect of LIPUS on days to radiographic healing did not differ significantly across clinical subgroups (interaction P=0.13, fig C in appendix 3) or between high and moderate compliance with treatment (interaction P=0.99, fig D in appendix 3). In our multivariable meta-regression, which included risk of bias, clinical subgroups, and compliance with treatment, the only significant effect modifier was the risk of bias (P=0.005).

Fig 6 Percentage difference in days to radiographic healing after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device), by risk of bias. Interaction P<0.001

Table 4

Credibility of subgroup effects for risk of bias for outcome days to radiographic healing in studies of low intensity pulsed ultrasound for bone healing

Criteria22	Rating (yes means higher credibility)
Is subgroup variable a characteristic measured at baseline or after randomization?	Not applicable for risk of bias
Is effect suggested by comparisons within rather than between studies?	No, between studies
Was subgroup effect specified a priori?	Yes, specified in our protocol
Was direction of subgroup effect specified a priori?	Yes, we expected larger effects for studies at high risk of bias
Is there indirect evidence that supports hypothesized interaction (biological rationale)?	Not applicable for risk of bias
Was subgroup effect one of a small number of hypothesized effects tested?	Yes, one of three
Does interaction test suggest low likelihood that chance explains the apparent subgroup effect?	Yes, significant in univariable subgroup analysis (P<0.001)
Is significant subgroup effect independent?	Yes, significant in multivariable meta-regression (P<0.01)
Is size of subgroup effect large?	Yes, 31.8% acceleration in high risk of bias trials versus 1.7% acceleration in low risk of bias trials
Is interaction consistent across closely related outcomes within study?	Yes, risk of bias explained heterogeneity in outcomes weight bearing and pain
Is interaction consistent across studies?	Yes, high risk of bias studies consistently showed large effects, low risk of bias studies small effects

Another randomized controlled trial in patients with delayed union of tibia fracture reported only the proportion of healed fractures at 16 weeks and did not find a significant difference (65% in the LIPUS and 46% in the control arm, P=0.07; high risk of bias towards LIPUS because of serious imbalance in age of fracture at baseline).44

The funnel plot based on time to radiographic healing was not clearly asymmetrical, and Egger’s test for publication bias was not significant (P=0.25, fig E in appendix 3).

Adverse effects related to device

Seven studies explicitly reported the absence of any adverse effects related to the device; two other studies reported mild transient skin irritations in six patients. The pooled risk ratio based on these two studies (2.65 in favor of control, 95% confidence interval 0.32 to 22.21; 129 patients) was not significant, nor was the pooled risk difference based on all nine trials (0%, 1% reduction to 1% increase; I2=0%; 839 patients; fig 7). We found no significant interaction with risk of bias on the risk difference scale (P=0.75).

Fig 7 Risk difference in adverse effects related to ultrasound device after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

Discussion

Main findings

Our systematic review provides moderate quality evidence that LIPUS applied to patients with fractures or osteotomies has no effect on time to return to work or the number of subsequent operations (table 3). Overall results suggested a possible reduction of days to full weight bearing, pain, and radiographic healing, but with large variability between studies strongly associated with risk of bias as an effect modifier: only trials with high risk of bias showed benefit. Based on randomized controlled trials at low risk of bias, we found high quality evidence that LIPUS has no effect on pain reduction, days to full weight bearing, or adverse effects related to the device, and moderate quality evidence that LIPUS has no effect on days to radiographic healing (table 3).

Comparison with other systematic reviews

Our results are consistent with other systematic reviews in concluding that most randomized controlled trials on LIPUS treatment are poorly reported, lack outcomes important to patients, and are at high risk of bias.5 6 7 8 9 10 11 12 13 14 Our systematic review, however, differs from previous systematic reviews in several important aspects. Firstly, we include the recently published TRUST trial,16 by far the largest trial on LIPUS treatment for bone healing, which reported a number of outcomes important to patients. Secondly, our choice of outcomes and interpretation of findings were informed by a guideline panel including patients with personal experience of fractures in the context of BMJ Rapid Recommendations. Patients considered functional recovery, pain reduction, and operations as critical outcomes, while expressing little interest in the commonly reported surrogate outcome of radiographic healing. Thirdly, we used optimal statistical approaches and, in particular, the ratio of means to combine days to radiographic healing, return to work, or full weight bearing across studies. This relative effect measure is most appropriate in the context of LIPUS, where the average time to recovery differs substantially between clinical subgroups. For instance, a lower grade stress fracture is likely to heal much faster than a complicated tibia fracture. It is not surprising, therefore, that previous meta-analyses found high heterogeneity when they used absolute mean differences to pool across studies.8 11 12

Finally, we used the GRADE approach to assess the quality of evidence, taking into account the results of subgroup analysis based on risk of bias: when effects differed significantly between high and low quality trials, we based our conclusions on trials at low risk of bias. Our approach of limiting conclusions to trials at low risk of bias depends on our judgment of risk; however, our ratings of risk of bias were consistent with those of a previous Cochrane systematic review.5 Furthermore, most trials judged to be at high risk of bias had limitations in more than one domain, and some had additional sources of bias including baseline imbalance or unclear clustering when patients had more than one fracture or operation. Application of our risk of bias judgments as an effect modifier met eight of nine relevant criteria for a credible subgroup analysis (table 4). A post hoc sensitivity analysis exploring a more conservative threshold for attrition bias (≥10% loss to follow-up) yielded, for all outcomes, results essentially consistent with the primary analyses

Limitations

The primary limitation of our review is the failure of most trials to measure or report outcomes important to patients. Of the 26 eligible trials, 11 reported, in sufficient detail for inclusion in meta-analysis, outcomes that patients consider critical for decision making.16 26 27 29 31 32 33 35 37 39 46 47 Of these, the only four trials that contributed substantial data included either patients with operatively managed fresh tibia fracture16 26 29 or conservatively managed clavicle fracture.37 One could question the extent to which our results apply to patients not included at all (such as children) or under-represented (stress fractures, non-union, and osteotomies) in the eligible trials. Qualitative subgroup effects (such as no benefit in one subgroup and important benefit in another) are, however, unusual. In the absence of evidence to the contrary, it might therefore be reasonable to apply our results to these populations. Our subgroup analysis and meta-regression for radiographic healing found no effect modification based on clinical subgroups. Certainly, the burden of proof regarding the effect of LIPUS in children and under-represented populations rests with those who might postulate a benefit.

LIPUS compared with electrical stimulation

Our findings are similar to those of a 2016 systematic review of 15 small trials that explored electrical stimulation versus sham treatment for fracture healing, only four of which were at low risk of bias.52 This review found moderate quality evidence for a 35% reduction (95% confidence interval 19% to 47%; I2=46%) in the rate of radiographic non-union. The authors found no evidence of a subgroup effect based on clinical presentation (that is, fresh fractures, delayed union or non-union, spinal fusion, or surgical osteotomy; interaction P=0.41). They did not explore whether risk of bias explained heterogeneity, but all four trials at low risk of bias showed no significant effect on radiographic union.53 54 55 56 They found a small reduction in pain (mean difference of −7.7 mm on a 100 mm visual analogue scale for pain, −13.92 to −1.43) and low quality evidence for no difference in functional outcome (mean difference of −0.88 points on 100 point SF-36 physical component summary score, −6.63 to 4.87).

Conclusions

In conclusion, moderate to high quality evidence shows that LIPUS fails to accelerate return to work, return to full weight bearing, and pain reduction or reduce the need for subsequent operation. If one gives highest credibility to combined effects from all available randomized controlled trials, low quality evidence would suggest a large reduction in time to radiographic healing. If, however, one gives higher credence to trials at low risk of bias, moderate to high quality evidence suggests that LIPUS not only has no effect on outcomes important to patients but also fails to accelerate radiographic healing. The evidence applies directly to patients with fresh fractures. The applicability to children and other under-represented populations, particularly those with non-union, for which no trustworthy direct evidence exists, is open to debate.

• Poolman RW, Agoritsas T, Siemieniuk RAC, et al. Low intensity pulsed ultrasound (LIPUS) for bone healing: a clinical practice guideline. BMJ 2017;356:j576. doi:10.1136/bmj.j576

summary of the results from the Rapid Recommendation process

Magic App (www.magicapp.org)

expanded version of the results with multilayered recommendations, evidence summaries, and decision aids for use on all devices

Low intensity pulsed ultrasound (LIPUS) devices are marketed worldwide to accelerate recovery from a fracture or osteotomy

Previous systematic reviews provided no definite conclusions about the effect of LIPUS on outcomes important to patients and radiographic healing

A guideline panel including patients and clinical experts informed outcome selection, importance of outcomes, subgroup analyses, and interpretation of results

Subgroup analyses suggested that beneficial effects of LIPUS are restricted to trials at high risk of bias

With inclusion of the recently published TRUST trial, sufficient high quality data for patients with fresh fractures has accumulated to conclude that LIPUS fails to improve outcomes important to patients and radiographic healing