The BOSS Study. Determining the incidence and clinical outcomes of uncommon conditions and events in orthopaedic surgery.

INTRODUCTION: There is widespread variation in the management of rare orthopaedic disease, in a large part owing to uncertainty. No individual surgeon or hospital is typically equipped to amass sufficient numbers of cases to draw robust conclusions from the information available to them. The programme of research will establish the British Orthopaedic Surgery Surveillance (BOSS) Study; a nationwide reporting structure for rare disease in orthopaedic surgery.
METHODS: The BOSS Study is a series of nationwide observational cohort studies of pre-specified orthopaedic disease. All relevant hospitals treating the disease are invited to contribute anonymised case details. Data will be collected digitally through REDCap, with an additional bespoke software solution used to regularly confirm case ascertainment, prompt follow-up reminders and identify potential missing cases from external sources of information (i.e. national administrative data). With their consent, patients will be invited to enrich the data collected by supplementing anonymised case data with patient reported outcomes.The study will primarily seek to calculate the incidence of the rare diseases under investigation, with 95% confidence intervals. Descriptive statistics will be used to describe the case mix, treatment variations and outcomes. Inferential statistical analysis may be used to analyze associations between presentation factors and outcomes. Types of analyses will be contingent on the disease under investigation. DISCUSSION: This study builds upon other national rare disease supporting structures, particularly those in obstetrics and paediatric surgery. It is particularly focused on addressing the evidence base for quality and safety of surgery, and the design is influenced by the specifications of the IDEAL collaboration for the development of surgical research.

This cohort seeks to define the incidence of disease, with automated mechanisms to identify cases that may be missing compared to external sources of data.

This study will develop efficient data capture mechanism, capable of being readily adapted to a range of rare diseases.

This study will build infrastructure within orthopaedic surgery to perform prospective research.

A bespoke software solution will enable routine prompts for follow-up from both patients, and clinicians.

Background

The evidence-base in orthopaedic surgery has rapidly grown with the introduction of patient registries. Common procedures or conditions are now frequently recorded in national registries, with recent developments particularly pertaining to treatment with hip arthroplasties[1,2] and hip fractures.[3] Registries seek to improve quality and safety by identifying unexpected variation in outcomes based on patient risk factors, surgical implants, surgical technique, hospital, and surgeon. These registries have the ability to detect variation for rare outcomes in high-volume procedures; however, there are many conditions and procedures within orthopaedic surgery that are rare. Procedural registries are unable to drive quality improvement in these areas, because low case-volume makes differentiating between natural variation and special-cause variation difficult or impossible. Furthermore, rare conditions are often associated with a variety of treatment interventions, and a variety of ways to assess a ‘successful outcome’. In this context, ‘variety’ is almost certainly a surrogate for ‘uncertainty’.

There is a need to extend the evidence base regarding the quality and safety of care for uncommon orthopaedic diseases. Routine data sources are not sufficiently detailed or comprehensive to investigate the outcomes of these uncommon conditions, or able to control for potential confounders. High-quality evidence to inform clinical practice is lacking. Studies based in individual units require retrospective review of many years of data, may be compromised by changes in practice over time, may fail to collect confounders, and may not be generalisable to other units because of differences in the patient populations.

Other branches of medicine have had similar challenges. In obstetrics, rare events have been investigated through a mechanism known as the UK Obstetric Surveillance System (UKOSS).[4] UKOSS covers all consultant-led obstetric units in the UK and allows for the identification and study of uncommon events and rare disorders. Studies carried out using UKOSS have provided robust, promptly-reported evidence on incidence, risk factors, clinical practice, management, and outcome resulting in a wealth of peer-reviewed publications to influence policy and care. The UKOSS methodology was subsequently rolled-out to paediatric surgery through the British Association of Paediatric Surgery Congenital Anomaly Surveillance System.[5]

This protocol describes a programme of work to investigate uncommon conditions and events in orthopaedic surgery: The British Orthopaedic Surgery Surveillance (BOSS) Study. The BOSS Study collects data on rare orthopaedic conditions using a similar framework to UKOSS, though automates much of the process with a bespoke online reporting system. The BOSS Study has been designed with two children’s hip diseases as the basis for development: slipped capital femoral epiphysis and Perthes’ disease. The BOSS Study platform can readily be adapted to encompass other disease cohorts, and developed to become a platform for pragmatic clinical trials of rare disease.

Aims

To generate evidence to enhance care and outcomes for patients with uncommon orthopaedic injury/disease by developing a nationwide reporting system (BOSS), and conduct a series of studies using this system.

What is the incidence of the condition of interest in England, Scotland, and Wales?

How does incidence vary by region?

What is the case-mix variation (i.e. patient factors, disease factors)?

What is the variation in treatment, and is this related to patient, disease, or surgeon factors (i.e. surgeon volume)?

What influence do patient, disease, and surgical decision-making have on surgeon-reported outcomes?

What influence do patient, disease, and surgical decision-making have on patient reported outcomes?

Methods

The study will identify nationwide prospectively collected disease cohorts. This involves the identification of all cases of the relevant disease in England, Scotland and Wales over a given time-period. This will be facilitated by a network of clinicians representing all orthopaedic units that treat the condition under investigation, reporting through the bespoke BOSS online platform (Table I, Figure. 1). The cohorts will be formed in two parts:

Table I.

The key components of BOSS:

A network of reporting surgeons (at least one in every British orthopaedic unit treating the disease(s) under investigation). Collection of anonymised data about cases via an online clinical research platform. Every participating hospital engages in a monthly process to confirm completeness of case identification via an automated monitoring tool. If the condition of interest has specific ICD-10 codes, potential ‘missing cases’ are identified using pseudo-anonymised reports from national routine hospital administrative data, with automated alerts to clinicians in the relevant treating hospital. Potential ‘missing cases’ are also identified using pseudo-anonymised reports from a national network of reporting trainee-surgeons, with automated alerts to clinicians in the relevant treating hospital. Collection of anonymised surgeon-reported follow-up from routine clinical data at fixed time points, with automated prompts. Collection of individual patient consent to enable the collection of patient reported outcome measures (PROMs) and future data-linkage to other healthcare databases. Consent may be collected: – Using paper-based consent by recruiting hospitals. – Online via the study website, with a unique access code shared with the participant by the treating hospital.

Figure. 1

Overview of the study mechanism.

Anonymized surveillance cohort: A consecutive comprehensive surveillance cohort encompassing all cases from all hospitals that may treat the condition. Case details will be anonymized. The consecutive surveillance cohort is important to minimize selection bias, in accordance with recommendations of the IDEAL framework.[6]

Nested-consented cohort: An identified cohort formed from a subset of individuals within the surveillance cohort who consent to the collection of patient-reported outcome measures, and future data linkage.

BOSS Study clinical network

As a comprehensive surveillance study, service evaluation data will be collected from all British orthopaedic units treating children and adolescents, with sites acting as data collection centres. All orthopaedic units treating children have agreed to participate as part of their audit and risk management activities, with encouragement from the British Orthopaedic Association, and the British Society for Children’s Orthopaedic Surgery.

Each hospital will nominate up to four clinician(s) responsible for reporting to the BOSS Study. This may include the orthopaedic audit lead, paediatric orthopaedic lead, a nurse specialist (i.e. nurse specialist/ trauma nurse or similar) and another relevant individual to fit with local protocols, such as an orthopaedic surgery trainee.

BOSS bespoke software

The study will use a system of bespoke software that automates many aspects of the follow-up and communication with sites. The software has been built around the REDCap Electronic Data Capture platform[7,8] hosted at the University of Liverpool, UK, with bespoke elements adding to the functionality to facilitate the study. The study data collection software has also been optimized for access via a web browser or mobile web browser, and has an application for use on mobile devices.

Case ascertainment

Case identification and data collection

Reporting clinicians will be able to log on to the BOSS online reporting platform (https://www.BOSS.surgery) at any time to record a new case of disease using an electronic case report form (eCRF) within REDCap. Validation checks during data entry will ensure that the case is eligible for inclusion. Minimal patient identifiers will be collected, including month and year of birth, sex, and date of admission. Each reported case will be allocated a unique study ID. The clinician in each hospital will be asked to keep a record of the unique study ID, linking it to identifying information in order to avoid duplication of reporting, and to enable long-term follow-up of the case. As per good clinical practice and information governance, the data collected will be those detailing the condition and treatments used as required for each specific condition. All information collected will be routinely available from the medical records, including images from the picture archiving and communication system (PACS).

Monthly case confirmation

An automated monthly email will be sent to clinicians and other nominated representatives at the reporting hospitals (Table II). This email will document the cases uploaded in the last month, and ask the teams to confirm the authenticity and completeness of the information that they have provided, by clicking a unique URL. Clinicians will be asked to confirm the authenticity of uploads even if ‘no cases’ have been seen. If the confirmation link is not clicked after one week the system will automatically generate a second email prompting a response. If a response is not received after two weeks, sites will be contacted by email or telephone. Responders and non-responders for the previous month will be published monthly in a newsletter.

Table II.

Contents of the monthly case confirmation. Unique automated email sent to reporting clinicians in each hospital.

Local summary of cases entered in the past month. Unique link to click to confirm monthly case ascertainment complete (reminder email after seven days sent to non-responders). List of potential cases identified by other sources (i.e. national Administrative data) which appear relevant to that hospital, but not already evident within database. A unique link alongside each case can be clicked to identify cases as erroneous. Cases due follow-up. Study newsletter.

Missing cases

1)National routine administrative data: Where appropriate, routine administrative databases will be acquired on a monthly basis to identify potential missing cases, using either disease or procedural codes of relevance. These databases are routinely used for administrative purposes (i.e. billing), and include the Hospital Episode Statistics (England), Scottish Morbidity Record (Scotland) and Patient Episode Data (Wales). It is acknowledged there is a lead-time of up to three months to achieve completeness within these databases. The treating hospital, date of surgery, sex, and age of the patient is obtained from each database – no identifiable data will be collected. These details will be uploaded to the BOSS system, whereby an automated process to match cases within the BOSS database to cases within administrative data using the minimal identifiers available will be carried out. The assumption is made that no hospital will treat more than one case of the rare disease with the same minimal identifiers in a given day.

If administrative data identifies a case, though no such case is identified within the BOSS Study database, a line will automatically be written into the ‘click and confirm’ email which provides the minimal identifiers available to the clinician, and an ‘incorrect case hyperlink’ alongside the case. The minimal identifiers will enable the treating clinical team to verify the authenticity of the case (i.e. by operation date), yet there is insufficient detail to enable anyone outside the direct care team to identify the case. If a case is truly missing, then the treating team will be asked to upload details via an eCRF. If the case is erroneously coded within national hospital administrative data (for example, national administrative data collected details for a follow-up case of disease rather than a new case of disease) then the hyperlink adjacent to the case can be clicked, which identifies that the case is erroneous, and prevents this case being flagged in future. Details of each missing case will continue to appear in the monthly ‘click and confirm’ email until each case is resolved, either by completion of a relevant eCRF, or confirmation that the case is erroneous.

2)Trainee data matching: Within every hospital is a network of surgical trainees. These are junior doctors at various stages of training, who work alongside consultant colleagues. A national requirement of the orthopaedic surgical training curriculum in the UK is to participate in research. These junior surgeons are an excellent group, around whom to create a further mechanism to maximize case ascertainment. Each trainee registered within the study will be given an individual login, and is encouraged to log data each month (even if only to say ‘nothing to report’). If a trainee observes a case in question, then they indicate this on the ‘trainee reporting eCRF’, indicating the hospital, and minimal details as described above. Matching trainee-reported cases to the BOSS database will occur as for routine administrative data, and a line will be written into the monthly ‘click and confirm’ e-mail, including details of the case (including minimal identifiers), and an ‘incorrect case hyperlink’. Again, reporting clinicians are expected to either confirm the authenticity by completing a surgeon eCRF, or identifying if the case is reported erroneously.

Routine case record follow-up

Follow-up will be requested at predefined intervals according to the study protocol for the disease under consideration. Follow-up details will be collected from the routine medical record, with clinicians expected to complete follow-up from the available medical record up to the defined date of follow-up. As the study uses routine records, no specific follow-up is mandated for the purpose of the study.

Prompts for surgeon follow-up will be automatically written into the monthly ‘click and confirm’ email. A line of text indicating the study case number, disease, and follow-up time point will be added each time a case becomes eligible for follow-up.

Follow-up is completed in REDCap, to which clinicians are directed from the www.BOSS.surgery website. Follow-up prompts will remain in the ‘click and confirm’ email until follow-up of each eligible patient has been carried out.

Individual patient consent and follow-up (nested consented cohort)

Patients and their families may be approached to enrich the study with additional information. This information is in the form of patient/proxy-reported outcomes, and their permission to enable linkage of their unique patient details (via their unique healthcare number (NHS or CHI number)) to healthcare databases in future – i.e. future linkage to the National Joint Register. Consent for this, and the acquisition of study data, will be captured in one of two ways:

Paper: This uses a traditional model of study enrolment using information leaflets, face-to-face consent conversation, paper consent/assent forms, and paper patient reported outcome measures (PROMs). This may occur in outpatient clinics, or by postal questionnaire. Participants identifiable information and non-identifiable data is linked using the unique study ID attributed to the participant in the surveillance cohort.

Online: All hospitals are issued with a unique access ID which is linked to the unique study ID. Hospitals are able to share this access ID with participants after enrolment into the surveillance cohort. Participants may then enrol into the study online, by reading the multimedia patient information materials (an animation and other web-based information). Participants are offered access to discuss the study with a member of the research team via telephone. Participants are then able to complete consent/assent online, with triggers for the completion of PROMs sent automatically by e-mail.

Statistics and analysis

The study is primarily designed to establish the incidence of the disease under consideration over the reporting period. Incidence rates will be estimated using reported new cases, and denominators from census data. Rates will be stratified by country, region, age and sex; 95% confidence intervals will be calculated based on an assumed Poisson distribution for counts. Variations in case-mix will be reported using descriptive statistics. Variations in treatment will be reported using descriptive statistics. Surgical management decisions are likely to vary by hospital, patient, and disease. In order to better understand this process, decision trees will be constructed to characterize overall management practices and highlight variation by centre. Other analyses will be undertaken depending upon the nature of the disease being studied, for example modelling of outcomes in relation to patient, disease, and treatment factors.

Research management and governance

The BOSS Study management group will meet monthly to ensure the smooth day-to-day running of the study.

The BOSS Study steering committee will have responsibility for overall strategic direction and governance of the programme and will meet via at least six-monthly teleconferences. This group will be made up of consultants/ academics and patient representatives with an interest in the patient population, the BOSS chief investigator and members of the study management group.

Ethical considerations and approval

The principal ethical consideration raised by this programme of work concerns patient consent. As described, data collection for the surveillance cohort will be limited to information from medical records only, will not involve the active participation of individuals, and will not include identifying information, such as names, addresses, postcodes, dates of birth, NHS or hospital numbers. The responsibility for care of patients included in studies carried out using BOSS will remain with the usual clinical team. Data will be collected from the clinical team after the event or outcome of interest has occurred; all participants will receive standard care.

The generalizability of the studies to be carried out relies on being able to calculate accurate and unbiased estimates of incidence, management practices and outcomes. For this reason, it is essential that data are collected on all cases occurring in the population; the process of seeking individual consent from participants in these circumstances would mean that this would not be possible and would be likely to introduce substantial bias to frequency and effect size estimates. Collecting anonymized data as proposed in the absence of consent is unlikely to cause significant harm for the individuals whose data are included in BOSS studies. In keeping with the national data guardian approach to ‘no surprises’, the patient section of the BOSS Study website details the data collection activity taking place and how to opt out of this activity. The BOSS Study is publicized through relevant national disease support groups as an additional route to inform potential participants of this work.

This programme of work complies with the Helsinki Declaration and has been reviewed and approved by the National Research Ethics Service Committee London – City and East (REC ref. 15/LO/2202) and the Health Research Authority (190754). The release of national administrative data for this purpose is subject to further approvals relevant to the specific disease and locality (England/Wales/Scotland). This protocol is based on the research ethics committee approved protocol.

Patient and public involvement

Representatives of orthopaedic user groups were consulted in the development of the funding application for this programme of work. Members of this group will also sit on both the steering committee and the management group.

Discussion

The BOSS Study is building on the success of other national surveillance study platforms to identify the incidence and outcomes of rare disease in orthopaedic surgery. Growing research infrastructure within orthopaedic surgery is particularly important, where a culture of research is not yet embedded in routine practice.

The design is particularly influenced by the recommendations of the IDEAL framework, which is an organization making methodological recommendations for research in surgery.[6] Much of orthopaedic surgical research falls within ‘stage 2b’ of the framework – necessitating ‘co-operative prospective evaluation of the techniques’. IDEAL 2b designs focus on identifying important patient characteristics (the case mix), technical intervention variables including co-interventions, and clinical outcomes of interest. This BOSS Study will therefore describe the disease incidence, case mix, risk factors, and variations in surgical interventions, in order to determine the safety and efficacy of different surgical strategies.

In the longer-term the infrastructure and bespoke software developed for the BOSS Study will enable this to be readily adapted to facilitate pragmatic randomized clinical trials in rare orthopaedic disease and thus provide the gold standard evidence to guide practice and improve outcomes.