A consensus exercise identifying priorities for research into clinical effectiveness among children's orthopaedic surgeons in the United Kingdom.

Aims: High-quality clinical research in children's orthopaedic surgery has lagged behind other surgical subspecialties. This study used a consensus-based approach to identify research priorities for clinical trials in children's orthopaedics.
Methods: A modified Delphi technique was used, which involved an initial scoping survey, a two-round Delphi process and an expert panel formed of members of the British Society of Children's Orthopaedic Surgery. The survey was conducted amongst orthopaedic surgeons treating children in the United Kingdom and Ireland.
Results: A total of 86 clinicians contributed to both rounds of the Delphi process, scoring priorities from one (low priority) to five (high priority). Elective topics were ranked higher than those relating to trauma, with the top ten elective research questions scoring higher than the top question for trauma. Ten elective, and five trauma research priorities were identified, with the three highest ranked questions relating to the treatment of slipped capital femoral epiphysis (mean score 4.6/ 5), Perthes' disease (4.5) and bone infection (4.5).
Conclusion: This consensus-based research agenda will guide surgeons, academics and funders to improve the evidence in children's orthopaedic surgery and encourage the development of multicentre clinical trials. Cite this article: Bone Joint J 2018;100-B:680-4.

The treatment offered to children undergoing both elective and trauma surgery in orthopedics varies widely.[1-5] This is often due to differences in opinion, which surgeons frequently accept because much of the literature underpinning practice is contradictory.[6] Another determinant is the poor quality of the evidence, which until recently almost universally involved observational case series. In recent years, there has been a cultural shift in orthopedic surgery with a growing number of successful randomized clinical trials that have reduced uncertainty and influenced both practice and policy.[7-12] However, this has predominately involved adult orthopaedic surgery and is yet to be realized within children’s orthopaedics.

One of the challenges faced by surgeons wishing to undertake clinical research is access to funding for high-quality studies. The challenge faced by funding institutions is how to identify questions in need of funding, particularly in research naïve specialties with little prior engagement with funding bodies. These groups are often sufficiently disparate that important clinical research ideas may never permeate from clinicians to funders.

The British Society for Children’s Orthopedic Surgery (BSCOS) promotes Paediatric Orthopaedic Surgery in the United Kingdom, and is affiliated to the British Orthopaedic Association (BOA). One of its key aims is to facilitate, instigate and encourage engagement in multi-centre national and international clinical research. BSCOS formed a research committee to help achieve this ambition. An important task was to determine the research priorities of its members in order to set the agenda for studies into clinical effectiveness in children’s orthopaedic surgery in the United Kingdom and Ireland. This paper describes the methodology used and reports the priorities that were identified.

Patients and Methods

The study used a modified Delphi process to achieve consensus on research priorities amongst surgeons.

Stage 1A: Scoping survey to identify important research questions

An online survey, composed using ‘Googleforms’, asked the question “Thinking about your clinical practice in the field of children’s orthopaedic surgery, what are the most important clinical research questions that need addressing?”. Responses were encouraged in a Population, Intervention, Comparator and Outcome (PICO) format. The survey was distributed to all members of BSCOS, and responses were collected anonymously. Participants could submit an unlimited number of research ideas. The survey was open for two weeks before and one week after the BSCOS annual scientific meeting in Glasgow, in 2017. A paper survey was also produced using the same format; this was distributed to all attendees at the meeting.

Stage 1B: Refinement of research questions

A list of the research questions was compiled, and grouped according to topic areas. Questions were reviewed by an expert panel from members of the BSCOS research committee to ensure that the question related to research involving clinical effectiveness. Questions pertaining to basic science research were not considered. Similar or related research questions were merged into a single question. Questions were grouped into the themes of ‘Trauma Care’ and ‘Elective Care’.

Stage 2A: Delphi Round 1 – Ranking and refining of research questions

A survey, also using Googleforms, was sent to all members of BSCOS by e-mail irrespective of whether they had submitted a response to the stage 1 survey. BSCOS has 206 practicing members, six affiliate members, 21 associate trainee members and 179 consultant full members. Members were asked to review each of the questions and rate them on a five-point Likert scale (‘low priority’ to ‘high priority’), based on the importance of each question to their clinical practice. The survey was available for completion for three weeks. Reminders were sent by e-mail after two weeks and 24 hours before it closed. Respondents were also invited to submit additional questions and to suggest any refinements to the existing questions.

The research committee considered all suggested refinements and additions to ensure that suggestions were not duplicates of existing questions and that questions related to orthopaedic clinical effectiveness research in children and to add clarity to any areas of ambiguity.

Stage 2B: Delphi Round 2 - Ranking of research questions, with prior response outcomes

The survey was sent again to respondents to the first round of the consensus process (stage 2A). They were given a graphic display (bar chart) indicating the responses of participants in the first survey with the instruction “We will now present the research questions from the previous round, and ask you to re-score the questions. We will also show you the scores from participants in round one (stage 2A), which will demonstrate the current state of collective opinion which may help to inform your choices”. Participants were then invited to score questions again but with knowledge of the group responses.

One question was changed in response to comments from the first round of the process, by splitting it into two distinct questions. This was clearly indicated to the participants, and the prior question and responses to that question were shown. Newly added questions were indicated clearly.

Stage 2C: Analysis and ranking

The research questions scored in stage 2 were ranked based on the overall mean score per question. The research committee reviewed the scores, and formulated a list of the questions in an order of priority.

Results

Stage 1

A total of 30 questions were submitted, from which 22 unique topics were identified; 14 related to elective care and eight to trauma. Seven were duplicates and one was excluded because it did not relate to clinical effectiveness. The questions submitted and details of how they were used are shown in supplementary material.

Stage 2A

A total of 87 respondents completed this round; 25 suggestions were made to add to or revise the list. Three suggestions related to elective care; two were added to the next round and one was excluded as it related to basic science. A total of 21 suggestions related to trauma, from which six were added, one revised and the remainder were excluded being either duplicates, related to basic science or did not pertain to children’s orthopedic clinical effectiveness research (supplementary material). One trauma question was revised by separating it into two, thus by splitting the ‘population’ by type of distal radial fracture but maintaining the same intervention, comparator and outcome.

Bar charts were produced to show the scores for each question (supplementary material, which is a copy of the questionnaire sent out in stage 2B of the prioritization exercise).

Stage 2B

A total of 86 respondents completed this stage (one who completed stage 2A did not respond to any requests to participate further). The number of responses to each question varied between 81 and 86 respondents. The mean score for the ‘relative degree of importance’ of the questions posed was 3.4 (1.6 to 4.7). The full scored and ranked list of research priorities is available in supplementary material.

Stage 2C

Uncertainties involving elective care were ranked higher than those involving trauma with the top ten elective research questions scoring higher than the top question for trauma. Given the range of scores awarded, it was decided to prioritise ten elective and five trauma research questions (Table I). Whilst the question relating to the fixation of fractures of the medial epicondyle was the highest-ranked within trauma, the committee were aware that this question had already been commissioned by the United Kingdom National Institute of Health Research (NIHR) and was therefore not included in the ranked list. The question that was refined in stage 2A by dividing the population of interest was again reconstituted, as dividing it seemed to have little effect on how either part was prioritized.

Discussion

This is the first study to establish clinical research priorities for children’s orthopaedic surgery. Given the large number of participants, the results are likely to be broadly representative of children’s orthopaedic surgeons throughout the United Kingdom. The top priorities include questions relating to slipped capital femoral epiphysis, Perthes’ disease, infection, cerebral palsy, hip dysplasia and a range of fractures. These results reflect clinical dilemmas frequently confronting children’s orthopedic surgeons where evidence is lacking.

Setting research priorities was a primary task of the BSCOS research committee in order to co-ordinate collaborative research in children’s orthopaedic surgery in the United Kingdom. Establishing the agenda was important to focus researchers in their efforts toward the long-term goals of the wider surgical community and to assist funding bodies when prioritizing where research funding may be best used. The BSCOS research committee is aware that the number and complexity of studies must remain achievable by their members consisting of practicing surgeons. By focusing on the high-priority research areas identified in this study, the collaborative network can ensure that the key questions are effectively addressed.

The Delphi process is an iterative process that is an efficient method of aggregating informed judgments in order to achieve consensus through a large group of participants.[13] Such methods are increasingly used to identify research priorities in medicine.[14-16] The current study is better described as a ‘modified Delphi’, as it combined the iterative Delphi process to generate a ranked list with an expert panel to guide the development of the questions and produce consensus through structured interactions.[17]

Ongoing work is already consistent with the proposed research agenda. A nationwide observational study of Perthes’ disease and slipped capital femoral epiphysis, funded by the United Kingdom National Institute of Health Research (NIHR), the British Orthopaedic Surgery Surveillance (BOSS) study is ongoing. This study is successfully recruiting in 143 hospitals in the United Kingdom and is designed as a IIb project within the IDEAL framework for surgical innovation, to define uncertainty and inform future intervention studies.[18] This large collaborative of children’s surgeons has also embarked on other work to enhance the feasibility of clinical trials, such as the development of core outcome sets and the use of routinely collected data to inform the development of trials.[19]

The study has limitations. Invitations to participate in the survey were sent to members of BSCOS, yet surgeons do not have to be members to treat children; this is particularly the case in trauma. We were pleased that the invitation to the Delphi study was tweeted to members of the United Kingdom Orthopaedic Trauma Society (OTS), as this may have broadened the views beyond simply the members of BSCOS. Although there were limited responses to the scoping survey, many more responded to the Delphi process with the opportunity of adding priorities. Whilst there were only 86 respondents in all, this broad representation is likely to have reached data saturation to identify the key research priorities. In a study such as this, saturation of the data is more important than the rate of response. Few questions were added to the priorities, particularly amongst elective conditions during the first round of the Delphi, which together with questions being in keeping with known gaps in the literature expected by the committee, suggests that consensus was reached. However, whilst we canvassed the opinion of many surgeons, we did not include other key stakeholders such as physiotherapists, nurses, patients and their families. In order to broaden the scope of the prioritization exercise, BSCOS has embarked on formal involvement with James Lind Alliance Priority Setting Partnerships (JLA PSP),[20] which will allow a broader array of participants. Together these two processes should provide a wide agenda for research in children’s orthopaedic surgery in the United Kingdom.

The outcomes of this exercise will be formally shared with key funders in the United Kingdom including the Prioritization Group within the National Institute of Health Research Health Technology Appraisal Programme, Arthritis Research UK and Action Medical Research Children’s Charity. Dissemination to professionals will be made via the British Orthopaedic Association newsletter and website, and BSCOS at their annual meeting and through their website. By publishing these findings, we will further broaden their reach, enabling international surgeons and funders to focus on these questions, which are likely to be similar in all developed healthcare systems. This forms a framework, around which surgeons, academics and funders can together improve evidence in children’s orthopaedic surgery. Better evidence should reduce uncertainty and reduce unnecessary variations in care. Given the rarity of many conditions in children’s orthopaedic surgery, we hope that the results of this study will serve as an impetus to the development of international multi-centre randomized clinical trials.

Take home message:

- The study captured the views of surgeons in the United Kingdom in order to identify the key research questions in need of multi-centre clinical trials.

Twitter

Follow D. C. Perry @MrDanPerry

Table I

Top ten ranked clinical effectiveness research questions in children’s orthopaedic elective care and the top five in children’s orthopaedic trauma care

Rank	Question	Mean score (max 5)
Elective care
1	Is ‘acute correction of deformity’ more clinically effective than ‘a strategy of pinning in situ with later correction if required’ for the treatment of severe stable Slipped Capital Femoral Epiphysis?	4.6
2	Is ‘surgical containment’ more clinically effective than ‘non-surgical care’ in the treatmentof Perthes’ disease of the hip?	4.5
3	Is ‘a short-course of antibiotics’ as clinically effective as ‘an extended course’ in the treatment of childhood bone and joint infections?	4.5
4	Is ‘prophylactic fixation’ more clinically and cost effective than ‘a strategy of active monitoring’ in the treatment of the opposite hip in stable Slipped Capital Femoral Epiphysis (SCFE)?	4.3
5	What are the current approaches to the surgical management of children with ambulant Cerebral Palsy (CP), and how may differences in treatment choices influence outcomes?	4.3
6	Is ‘nationwide selective screening’ more clinically and cost-effective than ‘universal ultrasound screening’ in the detection of hip dysplasia in the newborn?	4.3
7	What are the current approaches used in the management of late presenting hip Dysplasia in infants, and how may differences in treatment choices influence outcomes?	4.2
8	Is ‘active monitoring’ as clinically effective as ‘treatment with a neonatal hip splint’ in the treatment of babies with centered, but sonographically immature hips?	4.2
9	What are the current approaches used in the surgical management of hip disease in children with Cerebral Palsy, and how may differences in treatment choices influence outcomes?	4.0
10	Is ‘operative intervention’ more clinically effective than ‘best non-operative care’ in the management of children with patella instability.	4.0
Trauma care
Not prioritized (already prioritized by a funding body).	Is ‘surgical fixation’ more clinically effective than ‘non-operative care’ in the managementof children with medial epicondyle fractures of the elbow?	4.0
1	What is the clinical and cost-effectiveness of intramedullary nails versus plate fixation for the treatment of adolescent femoral fractures?	3.5
2	What is the clinical and cost-effectiveness of intramedullary nails versus plate fixation versus external fixation for the management of tibial shaft fractures in children?	3.5
3	Is ‘anatomical reduction +/- fixation’ more clinically effective than ‘outpatient functional reduction’ in the management of younger children with displaced distal radius fractures (either at the metaphysis or the physis)?	3.5
4	Is ‘surgical fixation’ more clinically effective than ‘non-operative care’ in the management of children with minimally displaced fractures of the distal tibial physis?	3.3
5	Is ‘a strategy of buried wire fixation and removal as required’ more clinically effective than ‘a strategy of percutaneous fixation and early removal’ for the treatment of children with lateral condyle fractures of the elbow?	3.3