What are the essential features of a successful surgical registry? a systematic review.

OBJECTIVE: The regulation of surgical implants is vital to patient safety, and there is an international drive to establish registries for all implants. Hearing loss is an area of unmet need, and industry is targeting this field with a growing range of surgically implanted hearing devices. Currently, there is no comprehensive UK registry capturing data on these devices; in its absence, it is difficult to monitor safety, practices and effectiveness. A solution is developing a national registry of all auditory implants. However, developing and maintaining a registry faces considerable challenges. In this systematic review, we aimed to identify the essential features of a successful surgical registry.
METHODS: A systematic literature review was performed adhering to Preferred Reporting Items for Systematic Review and Meta-Analysis recommendations. A comprehensive search of the Medline and Embase databases was conducted in November 2016 using the Ovid Portal. Inclusion criteria were: publications describing the design, development, critical analysis or current status of a national surgical registry. All registry names identified in the screening process were noted and searched in the grey literature. Available national registry reports were reviewed from registry websites. Data were extracted using a data extraction table developed by thematic analysis. Extracted data were synthesised into a structured narrative.
RESULTS: Sixty-nine publications were included. The fundamentals to successful registry development include: steering committee to lead and oversee the registry; clear registry objectives; planning for initial and long-term funding; strategic national collaborations among key stakeholders; dedicated registry management team; consensus meetings to agree registry dataset; established data processing systems; anticipating challenges; and implementing strategies to increase data completion. Patient involvement and awareness of legal factors should occur throughout the development process.
CONCLUSIONS: This systematic review provides robust knowledge that can be used to inform the successful development of any UK surgical registry. It also provides a methodological framework for international surgical registry development. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This review provides a systematic and evidence-based foundation for the development of any surgical registry.

We adopted a rigorous approach searching both the scientific and grey literature and used thematic analysis to develop our data extraction table.

Data analyses at all stages were cross-checked by a second judge and discussed at consensus meetings.

We did not perform quality assessment of the publications included in this review, owing to the non-empirical nature of included publications and the considerable heterogeneity among types of included publications.

By excluding non-surgical registries, we may have failed to capture important information on registry development. Our decision was based on surgical registries having specific attributes that we wanted to learn from including: datasets, strategies to increase surgeon ‘buy in’, funding sources, key challenges and others.

Introduction

The effective regulation of surgical implants is vital to patient safety. The Poly Implant Prothese (PIP) breast implant and metal-on-metal hip implant scandals have identified the risks of not gathering long-term data on implants and surgical outcomes systematically.1 2 As such, there is a UK and European-wide drive to establish surgical registries.3 In the UK, there are a number of well-known surgical registry initiatives including: the National Joint Registry (NJR), the National Hip Fracture Database (NHFD), the National Bariatric Surgery Registry (NBSR) and others. There are currently few registry initiatives in Ear, Nose and Throat surgery, particularly within the field of hearing.

Hearing loss is an area of unmet need,4–7 and industry is targeting this field with a growing range of surgically implanted hearing devices.8–11 Currently, there is no comprehensive UK registry capturing data on these devices10 12; in its absence, it is difficult to monitor safety, practices and effectiveness.5 13 A solution to this is developing a national registry of all auditory implants. However, developing and maintaining a surgical registry faces considerable challenges, with the majority of registries having poor rates of data completion and short life spans.14 15 In order to develop a successful surgical registry, it is important to learn from the experiences of previous and existing registries. In this systematic review, we aimed to identify the essential features of a successful surgical registry.

Materials and methods

Registration

This systematic review was registered on the PROSPERO database. Registration number: CRD42016039793.

Design

Systematic review and narrative synthesis.

Search strategy and selection criteria

A systematic review was performed adhering to Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) recommendations.16 With expert librarian support, we designed and conducted a comprehensive search of the Medline and Embase databases from inception to November 2015 using the Ovid Portal. An updated search was performed in November 2016. The search string used was ((surgery or surgical) AND (register or registers or registry or registries)) AND (britain$ or ‘united kingdom$’ or uk or england$ or northern ireland$ or wales$ or scotland$). The full search strategy is provided in (online supplementary appendix 1). All registry names identified in the screening process were noted and searched in the grey literature. Available national registry reports were reviewed from registry websites. We also visually scanned reference lists and searched relevant citations in the grey literature. Two authors (RM and JP) searched the literature independently and compared results at each stage of the PRISMA flow chart (figure 1). A third author (AS) arbitrated disagreements.

Figure 1

PRISMA diagram. PRISMA, Preferred Reporting Items for Systematic Review and Meta-Analysis.

Criteria for publications to be included were: publications describing the design, development, critical analysis or current status of a national surgical registry. Exclusion criteria were: non-English language, publications over 10 years old and publications describing non-surgical or non UK-registries.

Data extraction and synthesis

A data extraction table was produced in Microsoft Excel, containing 20 column headings developed by the first author (RM) (see table 1). These headings were developed following immersion in the dataset and using thematic analysis to identify the key themes for data extraction. RM extracted the data, allocating relevant information from each included publication to each of the data columns described in table 1. A second author (JP) cross-checked the development of the data extraction table and the data extraction, and this process was discussed at two interim consensus meetings. Data were then synthesised by summarising the data under each column heading into a structured narrative, following the principles outlined by Popay et al.17

Table 1

Data column headings and their descriptions

Dataset column headings	Description
Author(s)	Author of article
Title	Title of article
Year	Year of publication
Name of registry	Name of registry
Type of surgery	Operation(s) captured by the registry
Collaborations	Collaborations developed for the registry
Registry leadership and management	How the registry was managed and/or lead
Objective(s)	The objective(s) of the registry
Registry development and/or design	How the registry was developed and/or designed
Funding	How the registry was funded
Rationale behind dataset	The rationale behind selecting the registry dataset
Dataset	The dataset of the registry
Data processing	How the registry data were processed
Strategies to increase data completion	Strategies used/found by the registry to increase data completion
Data reporting	How the registry reported/disseminated their results
Patient involvement	How patients were involved in the registry and viewpoints on patient involvement in registries
Difficulties encountered/challenges	Difficulties and challenges encountered by the registry
Benefits of registries	The benefits of the registry
Measures of a successful registry	Factors that determine a successful registry
Legal factors, ethics and data access	Legal factors, ethics and data access for the registry

Results

After duplicates were removed, titles and abstracts of 1389 publications were screened. Thirty-five additional records were identified from other sources. Fifty-nine publications fulfilled the criteria for analysis. After conducting our updated search, 10 additional publications were included, resulting in 69 publications for analysis. See figure 1 for the PRISMA flow chart.

Included publications consisted of annual registry reports and analyses, registry overview documents, editorials, commentaries, registry proposal documents and registry review articles and covered a range of surgical specialties (see box 1). (Online supplementary appendix 2) shows the full data extraction table, identifying the relevant information from each included publication.

Surgical specialty

Orthopaedics

Renal surgery

Neurosurgery

Cardiac surgery

Upper gastrointestinal surgery

Urology

Plastic surgery

Breast surgery

Colorectal surgery

Cardiothoracic surgery

Vascular surgery

Endocrine surgery

Ear, Nose and Throat surgery

Below is a narrative synthesis of the full data extraction table. The numerical and alphabetical digits below correspond to the data extraction columns in (online supplementary appendix 2).

Registry planning

Registry leadership and management (1.G)

Registries are typically led by steering committees comprising professional and clinical stakeholders as well as patient representatives.18–22 Steering committees should have overall responsibility for registry design, data monitoring, data analysis23 as well as strategic direction, oversight and allocation of registry resources.19 21 24 25

It is important for registry management to receive input from both clinical and data management experts.26 27 Local registry managers help maximise data completion and accuracy21; and private companies have been employed to successfully manage several UK national registries.25 28–30

The objective(s) of a surgical registry (1.H)

Registries should have a clear set of objectives from the outset; these often include: improving patient care, providing comparisons of standards, monitoring current practice, monitoring device durability and intervention performance, identifying variations in service provisioning as well as guiding commissioning and guideline development.12 19 20 22 30–32 Other aims include gaining a better understanding of disease epidemiology19 21 33 and promoting future research, innovation, efficiency, transparency and patient decision making.28 34–38 The addition of objectives at a later stage, after the registry is established, will likely lead to challenges.12 14 15 32 For instance, a registry developed to improve patient care will unlikely be successful in driving research, due to the registry not being developed to collect and report on data relevant to researchers.12 20 22 23 Registries including the NHFD, NJR and NBSR have demonstrated that by setting clear objectives from the outset and by involving key stakeholders including clinicians, patients and researchers during registry development, a registry can successfully deliver on multiple objectives, including improving patient care and driving research.20 25 27

Funding (1.J)

Registries require considerable resources for initial set-up and ongoing maintenance.26 Owing to implant life span, implant registries in particular should plan for long-term funding. Central funding sources include the Healthcare Quality Improvement Partnership, National Health Service England, the Department of Health (DOH) and national commissioners.22 26 39 Industry can also contribute to funding, although it is important to consider governance around industry access to registry data.21 29 40 41 Other sources of funding include participating hospitals,21 charities,42 professional societies,43 annual capitation fees36 and charging for data requests.26 Registry costs can also be incorporated into the price of each implant.27 Funding often comes from multiple sources.20 21 26 27 44

Establishing collaborations (1.F)

It is important to form strategic national collaborations among stakeholders including: patient groups, clinicians, specialist societies, industry, commissioners, funding bodies, hospitals, academic groups and those involved in data collection and management.19 26 27 32 Working with and learning from existing regional registries was a successful strategy adopted by the National Vascular Registry.45 International collaborations can help align the registry with global surgical initiatives27 38 46 and links with the implant industry can facilitate implant tracking.47 Collaborations with national institutes including the National Institute for Health and Care Excellence (NICE) and the Royal Colleges can align registry data with national guidelines development and revalidation.19 Collaborations with geriatrics societies and charities can help data collection on elderly patients.20

Registry development and design (1.I)

Reaching stakeholder consensus on registry objectives, dataset and activities is essential.20 36 48 The registry can be developed from existing smaller registries,45 and piloting the registry is important in obtaining user feedback.21 40 49–51 Web-based electronic platforms facilitate quick and accurate data collection, and tailored IT systems can be developed to provide a secure, interactive and easy-to-use registry platform.20 29 30 50 52 NICE advises that registries should be recorded on a national database of registers.23

Dataset and data management

Rationale behind a registry dataset (1.K)

It is advisable for datasets to be developed through stakeholder and patient consensus meetings,48 53 54 with a balance between comprehensibility and feasibility: comprehensive datasets are unlikely to achieve data completion, while limited datasets may be less useful.24 29 38 Flexible datasets built with the ability to evolve can help promote registry longevity, but an initial period of consistency helps embed the registry.26 49 It can also be useful to build on existing registry datasets from the same specialty.28 46 51 54

While collecting quality of life (QoL) and patient-reported outcomes (PRO) data is vital for evaluation of treatments and services,55 56 collecting such data in the context of a national registry is resource intensive and may affect data completion.55 Deciding which PROs to choose can also be an area of controversy and disagreement.55 If PROs are introduced, it is advisable to keep the number of questions short and for these data to be collected directly from patients at regular, planned time points, rather than relying on clinic follow-ups.30 55

The design of registry datasets can accommodate national guideline recommendations23 45 57 58; for example, the NHFD dataset is designed to facilitate easy comparison to NICE guidance,20 and the National Vascular Registry adapted datasets to capture key issues highlighted by National Confidential Enquiry into Patient Outcomes and Deaths.45

Dataset (1.L)

While specific registry data items vary between surgical specialties, the majority of UK surgical registries collect the preoperative, operative and postoperative data items summarised in table 2. A free text box can also be included to capture additional relevant information.30

Table 2

The data items collected by the majority of UK surgical registries

Preoperative	Operative	Postoperative
Name of centre	Name of operation	Outcome data specific to operation
Patient identifier	Time to surgery from first appointment	QoL/PRO outcome measure
Patient demographics	Type of anaesthetic (local or general)	Date of discharge
Patient comorbidities	ASA grade	Length of stay
Whether discussed at MDT meeting	Thromboprophylaxis regimen	Complications
Indication for surgery	Primary or revision case	Morbidity
Date of diagnosis	Elective or emergency surgery	Mortality (and cause)
Preoperative investigations and results	Date of surgery	Dates of follow-up
Date of admission	In or out of regular hospital hours	Follow-up outcomes
GP information	Site/side of surgery	Need for further treatment
	Surgical technique/approach	Need for further surgery
	Difficulty of procedure	ITU admission (planned/unplanned)
	Intraoperative problems	Destination of discharge
	Date of consent
	Grade of surgeon
	Surgical time
	Funding for operation (NHS/private)
	Use of antibiotics
	Type of implant and implant serial number

ASA, American Society of Anaesthesiologists; GP, general practitioner; ITU, intensive therapy unit; MDT, multidisciplinary team; NHS, National Health Service; PRO patient-reported outcome; QoL, quality of life.

Data processing (1.M)

To improve data quality and accuracy, data from participating centres should be internally validated by local registry managers and clinicians before being cleaned.21 59 60 Data cleaning can take place locally or centrally and involves detecting and resolving data problems.26 28 32 Prior to central analysis, data can be returned to each contributing centre to take any necessary remedial actions.26 53 59 61 On site data verification by auditors is considered good practice.40 60 62 Although these visits focus on completeness and accuracy of data, they also provide an important opportunity for education of clinicians and local registry managers adding to ongoing data quality40 48 60 62 and for discussion with administrators about appropriate resources for information management.60 Feedback through reports evaluating quality of local data collection can be sent to contributing centres to stimulate improvements, and independent validation of data including data completeness, mortality, readmission and revision can be achieved by linking registry patient records to the Office of National Statistics and Hospital Episode Statistics (HES).18 35 36 58 60 62 63 NICE recommends that the process for data collection, storage and analysis should be independent of any particular company or commercial interest.23

Data reporting (1.P)

Registries usually publish information via annual online comprehensive reports,21 26 32 36 62–64 research publications and presentations.27 39 62 65 There is controversy surrounding the publication of surgeon-specific data. Evidence suggests that publishing these data is associated with improvements in mortality62 as well as increased transparency, patient trust and improved supervision of junior surgeons,25 66 with no evidence of ‘risk-adverse’ surgical behaviour.26 62 66 When publishing surgeon-specific outcomes, it is important to statistically adjust for case mix, to take into account complex, high-risk cases.63 66 It is recommended that team level data are published to reflect that outcomes are dependent on the entire surgical team, not solely the consultant surgeon.66 Minimising the time between the surgical event and the release of data is also important for the identification of faulty implants or unsafe practices.63

Challenges and data completion

Difficulties encountered/challenges (1.R)

Registries relying on voluntary data submission are dependent on user motivation and are unlikely to achieve complete data capture.35 56 67 Voluntary data submission can also result in reporting bias with under-reported complications and a non-consecutive, non-representative patient group.35 44 64 Insufficient financial resources for registry development and maintenance is a frequent challenge56 68 69 as is lack of stakeholder and patient ‘buy-in’, resulting in poor data quality and completeness.22 31 43 Registries can be perceived to worsen documentation pressures, which may compromise data recording and limit participation.22 51 Reaching stakeholder consensus on the registry dataset is challenging,22 70 and datasets with unclear definitions as well as those unable to adapt to changes in practice can result in difficulties in drawing national comparisons and tracking surgical activity.28 31 43 50 62 Collecting long-term follow-up data can also be challenging, particularly when patients are under the care of multiple hospitals and clinicians.25 44 51 55 70

Strategies to increase data completion (1.N)

Data completion can be optimised by careful registry design and by involving stakeholders throughout its development promoting ‘buy-in’.25 26 An online registry that is user-friendly, multibrowser compatible, simple, quick-to-use and has clear data definitions will increase data input.24 26 30 Other optimisation strategies include real-time data input, reminders for mandatory fields, hover-tip prompts, on-screen data validation checks, numeric limits, auto-calculations, drop-down menus, calendar support and limiting free-text fields.19 25 40 48 50 51 71 It is critical that data input is supported by allocation of dedicated time and resources, regional training sessions, succinct user guides, real-time ‘chat’ support, as well as email and telephone support.19 22 40 43 Mobile ‘apps’ allow easy remote registry access and can also help increase data completion.22 24 30 47

Registries that are of clear value to clinicians and institutions are more likely to achieve data completion.25 26 30 46 For example, registry systems producing automated clinic letters or operation notes or that help record data for self-audit and revalidation are more likely to be used.18 25 35 37 A research friendly registry can also help increase participation, particularly if registry contributors can be listed coauthors.41 65

Regular performance feedback can help maintain local interest in the registry.18 19 55 The NHFD produces online graphs with live data on performance, time-to-surgery, mortality, length of stay (LOS), best practice and patient safety.20 The NJR has increased registry participation through a programme of local audits and by issuing data quality certificates that provide incentive to submit high-quality data and highlight hospitals not complying with mandatory requirements. Another measure employed by the NJR is sharing cost-saving information on best implant prices, on the proviso that hospital trusts submit data to the NJR.27

Regular published reports and journal articles have been found to raise the profile of the registry, highlight non-participating units and increase data completeness and accuracy.60 Advertising can increase awareness and participation via press coverage, emails, society bulletins, letters to eligible members, conferences, regional meetings, word-of-mouth and through journal advertisements.20 35 44 51 58 60 72

Making data input compulsory for revalidation or commissioning, or both, appears to be the most successful method of increasing data completion.19 22 25 27 51 60 62 67 70

Patient involvement and legal factors

Patient involvement (1.Q)

Patient involvement in registry leadership, design, development and reporting increases the relevance of the registry to patients, commissioners and policy makers.18 27 31 36 54Patients entering their own data via electronic patient portals can be particularly useful in collecting QoL and long-term follow-up data.22 24 30 47 55 To help increase registry patient participation, it is important to acquire consent early, have a registry coordinator for patient follow-up and have multiple language options.55 Facilitating patient access to data promotes transparency, patient choice and involvement.27 62 63

Legal factors, ethics and data access (1.U)

UK surgical registries must comply with DOH data protection and information governance legislation for secure processing of patient healthcare data.21 36 53 This process can be guided by the Data Protection Act, General Medical Council guidance, the Caldecott Confidentiality Principles and information found in the Information Governance Toolkit of the Health and Social Care Information Centre.36 39 73 The registry should be implemented and reported in accordance with Declaration of Helsinki ethical principles.40 Patient informed consent should be obtained for data submission, and data should be anonymised in all cases.30 40 53 60 70 Failure to function within a legal framework can result in legal termination with potential criminal repercussions.26

While easy access to the registry is essential,24 data privacy should be maintained and data should be stored securely and not shared without appropriate permissions.22 26 32 36 63 70 It is important for data release to be governed under a defined data-sharing agreement, where the security and uses of the data are clearly defined.19 21 36 Registries can have subcommittees or data managing groups that are responsible for reviewing formal access requests and ethical assessment.19 29 36 40

Registry success

Benefits of registries (1.S)

Surgical registries can help underpin research including randomised controlled trials, assess and improve cost-effectiveness as well as inform risk prediction models.26 36 47 74 75 Other benefits include improved patient decision making, treatment development and identification of trends in practice.25 28 56 Registries can facilitate inter(national) comparisons between centres as well as personal audit and revalidation.30 35 46 55 67 75 Publicly accessible registries can increase public trust and promote transparency and patient choice.61 With the growing number of surgical implants, registries can help identify both the highest performing and faulty implants.47 71 76 The collection, feedback and publication of registry data is now a recognised way of informing clinical practice, driving quality improvement and improving patient care and safety.40 61 63 71 Since the National Audit Cardiac Surgery registry was introduced, risk-adjusted in-hospital mortality for cardiac surgery in the UK has fallen by over 50% despite more elderly and high-risk patients having surgery each year.26 Following the start of the NHFD, rates of early surgery increased from 54.5% to 71.3% and 30-day mortality fell from 10.9% to 8.5%.20

Registry data can support agencies to monitor and evaluate the quality of healthcare delivered.20 They can also help identify national variations in service provisioning, map and evaluate patient pathways as well as inform health service commissioning and policy.37 45 56 58 71 74 77 Regulatory organisations including NICE recognise the value of registries in technology assessment particularly in the absence of formal trials.23 44 70 When compared with trials, registries require fewer resources and often collect data from a broader population base so their findings have strong external validity.41 78 They also frequently provide data on long-term outcomes that exceed the study window of a trial.65 They can be of particular value when investigating patient groups that are usually excluded from clinical trials such as the elderly.79

Measures of a successful registry (1.T)

A successful registry is one that is easily accessible, has a high degree of data completion and participation and helps promote inter(national) collaboration.22 26 63 68 69 They provide timely feedback to their users, identify trends in practice, improve standards of care and identify failures at the earliest opportunity.20 48 63 Successful registries are useful to their stakeholders and contain validated data that are accurate and easy to analyse.22 39 55 71 79

Discussion

In this systematic review, we have identified the fundamentals for developing a successful UK surgical registry. While we highlight the need for a registry of auditory implants, our findings have implications to the wider surgical community since we provide information that can be used to inform the development of any UK surgical registry.

Summary of findings

The fundamentals to successful registry development identified by this synthesis are summarised in figure 2 and include: steering committee to lead and oversee the registry; clear registry objectives; planning for initial and long-term funding; strategic national collaborations among key stakeholders; dedicated registry management team; consensus meetings to agree registry dataset; established data processing systems; anticipating challenges; and implementing strategies to increase data completion. Patient involvement and awareness of legal factors should occur throughout the developmental process.

Figure 2

Overview of the key steps required for the development of a successful UK surgical registry.

Relevance to existing research

There is a clear need for surgical registry data to improve patient safety and help regulate surgical practices. Concerns over the evidence base for surgical implants in general has been raised by the IDEAL (Idea, Development, Exploration, Assessment, Long-term monitoring) collaborative and the House of Commons Science and Technology committee.3 80 Across the UK and European Union, implants can enter surgical practice on the basis of equivalence data, meaning that an implant can be used on the basis of similarity to another implant rather than evidence of its own safety and effectiveness.3 80 Transparency and postmarket surveillance are additional concerns with data on safety and performance of implants not being fully published.3 The recall of the PIP breast implants and metal-on-metal hip implants identify the dangers of relying on equivalence data for the evaluation of safety and efficacy.1 2

Owing to these concerns, the IDEAL collaborative, DOH, NICE, policymakers and commissioning groups have called for surgical registries that can collect prospective outcome and safety data, promote transparency as well as provide patients and the public with information on their care.3 8 11 80 81 It has also been recognised that registry data can serve as a valuable alternative to randomised trials, which can be unfeasible and of limited scientific use, particularly at the development stage of a surgical innovation.41 65 When compared with trials, registries require fewer resources, have stronger external validity and tend to provide longer term outcome data.41 65

Implications

This review provides evidence-based knowledge on registry development that can be used by existing and developing UK surgical registries to increase their chance of success. Successful registries provide essential clinical and cost-effectiveness data for policy and guidelines development.26 47 74 75 They also help develop (inter)national research collaborations as well as promote patient choice, trust and transparency.25 28 56 61 Other implications include facilitating inter(national) benchmarking and personal audit.35 46 55 67 75 Successful registries help drive healthcare quality improvement, improve patient safety and allow commissioners and service providers to monitor quality, detect faulty implants early, monitor patient usage, identify variations in practice and allocate payments fairly.45 47 56 71 74 76 From an international perspective, this review provides a methodological framework that can be adopted by other countries to promote successful national surgical registry development.

Strengths and limitations

We acknowledge that the quality and reliability of included publications likely varied due to their heterogeneity; publications included: annual registry reports and analyses, registry overview documents, editorials, commentaries, registry proposal documents and registry review articles. In addition, owing to the nature of included publications, much of the data collected were from non-empirical, opinion-based articles. This heterogeneous and non-empirical nature of included publications also precluded formal quality assessment. We recognise that the development of the data extraction table and the data extraction may have been influenced by researcher bias. However, to mitigate this, both stages were cross-checked by a second researcher and discussed at two interim consensus meetings. We also acknowledge that by excluding non-surgical registries, we may have failed to capture important information on registry development. Our decision was based on surgical registries having specific attributes that we wanted to learn from including: datasets, strategies to increase surgeon ‘buy in’, funding sources, key challenges and others.

A key strength of this review is that it provides an evidence-based foundation for the development of any surgical registry. We adopted a rigorous approach searching both the scientific and grey literature and used thematic analysis to develop our data extraction table. Moreover, data analyses at all stages were cross-checked by a second judge and discussed at consensus meetings.

Conclusion

This systematic review provides robust knowledge that can be used to inform the successful development of any UK surgical registry. It also provides a methodological framework for international surgical registry development.