Can multicentre urodynamic studies provide high quality evidence for the clinical effectiveness of urodynamics? ICI-RS 2019.

AIMS: Lower urinary tract (LUT) function can be investigated by urodynamic studies (UDS) to establish underlying functional abnormalities in the LUT. A multicentre registry could present an opportunity to improve the scientific evidence base for UDS. During the International Consultation on Incontinence Research Society (ICI-RS) meeting in Bristol, United Kingdom 2019, an expert panel discussed the potential of a multicentre urodynamic registry to improve the quality of urodynamic output.
METHODS: the potential importance of a multicentre urodynamic registry, parameter inclusion, quality control, and pitfalls during a registry roll-out were reviewed and discussed. RESULTS AND
CONCLUSIONS: The clinical utility, evaluation, and effectiveness of UDS remain poorly defined due to a lack of high quality evidence and large study populations. Therefore, the ICI-RS proposes formation of a urodynamic panel for future roll-out of a registry. The inclusion of basic parameters was discussed and the essential parameters were defined as well as the potential pitfalls of a registry roll-out. The discussion and recommendations in this paper form the base for future urodynamic registry development.

INTRODUCTION

The concept of “big data” is generally understood to be “the information asset characterized by a high volume, velocity and variety to acquire specific technology and analytical methods for its transformation into value.” In terms of healthcare the European Commission study on big data in public health, telemedicine, and healthcare, refers to “routinely or automatically collected datasets, which are electronically captured and stored” for multiple purposes. This may refer to sources traditionally categorized as real‐world data, including administrative databases (eg, on hospital discharge), electronic health records, and disease registries, but can also refer to digital data collected automatically by machines or by patients, and wearable technology.

Within uro‐oncology big data collection projects already exist, such as the PIONEER consortium supported by the European Association of Urology. The PIONEER project is designed to answer several questions with regard to prostate cancer by using the potential of big data. For example, there is still a lack of information on risk factors for prostate cancer and patient characteristics. This means prediction of treatment outcome remains poor.

The potential benefit of clinical registries to improve healthcare quality has been explored to some extent within urology. Clinical registries may add granularity for risk adjustment, expand the focus of quality measurement by inclusion of endpoints meaningful to urologists. In addition, clinical registries may have an indefinite follow‐up period and establish a framework for local interventions to address quality gaps. The possibilities of big data have not yet been fully explored in functional urology, specifically the urodynamic community, and may be extremely valuable for the future. Therefore, this topic was discussed during the International Consultation on Incontinence Research Society (ICI‐RS) meeting 2019. During this session various subtopics have been presented to set the scene, followed by discussion amongst the participants and led by the chairman of the session.

WHY IS A URODYNAMIC REGISTRY OF IMPORTANCE?

Lower urinary tract symptoms (LUTS) affect a substantial proportion of the population (including children) and are associated with significantly impaired quality of life and co‐morbidities. , Lower urinary tract (LUT) function can be investigated by urodynamic studies (UDS), commonly performed as filling cystometry and a pressure‐flow study (PFS), to establish underlying functional abnormalities in the LUT. However, the clinical utility, evaluation, and effectiveness of UDS remain poorly defined due to a lack of high quality evidence. Clinical decisions based on urodynamics are currently without a strong evidence base and may be affected by subjective interpretation by the healthcare professional. This interpretation is made by pattern recognition and depends on the expertize and clinical experience of the healthcare professional. However, the interobserver agreement of urodynamic findings is generally low. In some cases, clinical guidelines for UDS have been based purely on low grade scientific evidence or expert opinion in the absence of high level evidence such as well‐conducted randomized control studies or exceptionally strong observational studies. Conversely, some studies with limited scope have had their findings generalized to the detriment of evidence‐based healthcare. By using predictive algorithms with machine learning, which needs a large dataset to perform, the interpretation of urodynamic data could become more objective and accurate. This might lead to a more accurate diagnosis resulting in more appropriate treatment.

Clinical registries could present an opportunity to improve the scientific evidence base currently lacking in UDS by the collection and sharing of UDS data both nationally and internationally. Currently, no such registry exists for UDS. Although some centers have their own local clinical registries, the data collected and software utilized vary from center to center, making it difficult to share data. A global registry would allow the collection of standardized datasets from multiple centers, thus providing health professionals and researchers with relevant and good quality data on a larger scale.

Clinical registries already play an important role in medical and scientific research in other areas of healthcare. Data collected within registries are often utilized to monitor disease, assess treatment outcomes and improve diagnostic techniques and healthcare delivery. Increasingly, clinical registries in healthcare are being utilized to develop cost saving strategies, improve quality management processes and ensure clinical guidelines and standards are being met. Registries have been shown to be valuable because the information they contain can be analyzed and used to make data‐driven (ie, evidence‐based) decisions and improvements to patient care. Consequently, clinical registries bring together clinical practice interventional, outcomes and potential areas of improvement to inform best practice. In other areas automated interpretation of examinations based on large datasets has been the standard for many years. For example, in ECG machines, which have facilitated diagnostic conclusions and enhanced accuracy. Today, as a result of big data, the failure rates of the proposed diagnoses of ECG machines are lower than those of a physician.

By collecting a significant amount of urodynamic data into one large database, information on the evolution of the disease or regarding treatment response could be effectively and reliably measured. Comparing the data of thousands of patients would enable new trends on the pathophysiology of, for example, overactive or underactive bladder to be explored. This could in turn lead to new strategies in disease management. Moreover, urodynamic responses to treatment (eg, medication, botulinum toxin, neuromodulation) could be more thoroughly evaluated, which would better identify predictive factors for success or failure. The database could further reveal potential weaknesses of urodynamic investigations guiding the society to improve teaching courses and advising the industry for further improvement of their products.

WHAT TO LEARN FROM EXISTING MULTICENTRE DATABASES

Following the publication of one letter commenting the trial published by Nager et al, also known as VALUE trial, members of the Italian Society of Urodynamics (Società Italiana di Urodinamica [SIUD]) put together the information contained in their individual databases, with the aim of understanding: (a) how many patients could be considered “uncomplicated stress urinary incontinence (SUI) patients,” using roughly the same criteria as the VALUE trial; (b) How the use of invasive urodynamics could impact on the final diagnosis in the uncomplicated and complicated patients. After few months of work, the authors of the paper published in 2016, were able to collect 2053 patients studied with invasive urodynamics in six centers in Italy. However, the data collection was relatively easy for several reasons: (a) the centers already had a good collaboration based on previous mutual respect and trust; (b) the databases were sufficiently homogeneous and comprehensive to provide robust data, despite the retrospective design of the study. The multicentre database study concluded that urodynamics are still mandatory in “complicated” patients with SUI.

With this experience in mind, the SIUD, together with the research foundation of the Italian Society of Urology started a registry of patients with overactive bladder who had been evaluated in different centers in Italy. The project was presented at the International Continence Society (ICS) meeting in Florence in 2017 and is ongoing. Despite the good collaboration amongst the centers, this project is proceeding slowly and has proved to be more difficultly than expected for several reasons: (a) data collection is time consuming (on an electronic centralized support); (b) No funding has been provided for regular face to face meetings of the investigators, thus making the exchange of information more difficult; (c) the European Union General Data Protection Regulation (https://gdpr.eu) changed the rules for sensitive data collection, thus making the investigators work in this project more difficult.

WHICH PARAMETERS TO INCLUDE IN A REGISTRY

The urodynamic community is in a favorable position when it comes to the collection and collation of data. The major reason for this has been the establishment of reference documents from the ICS that standardize terminology and define good practice within the topic area of lower urinary tract symptoms and urodynamic testing. , The standardization of terminology document details the classification of LUTS into storage, voiding, and post‐micturition symptoms and provides precise definitions for each individual symptom. This document has been instrumental to the formation of a universal language for communication between healthcare professionals in this field and eliminated any ambiguity surrounding the specific meaning of common symptoms. Using this pre‐defined set of urinary symptoms, validated questionnaires for urinary tract symptoms have been developed and are in widespread use. Questionnaires, such as those produced by the International Consultation on Incontinence Questionnaire project are freely available in many languages but a wide range of other questionnaires are also in existence. This presents a problem for the collection of data from multiple sources and makes the concept of generation of “big data” potentially difficult to achieve especially if different tools are used for clinical assessment in different units. With regard to the generation of large multicentre databases it is recommended that participating units standardize their clinical assessment by using the same validated questionnaire so that data‐pooling and meta‐analysis are made easier

The establishment of large databases of urodynamic studies is however more straight‐forward especially given that a single document detailing “good urodynamic practices” is in existence. This document details the requirements for an “ICS standard urodynamic test” which is defined as “Uroflowmetry and post‐void residual plus transurethral cystometry and PFS: all tests are performed in the patients preferred or most usual position: comfortably seated and/or standing.” This document and the standardization of terminology document previously referred to describes the measurements that should be made during urodynamics. In addition in 2004 the ICS standard for digital exchange of urodynamic study data has been revised.

These measurements are already standardized and therefore combination of multiple databases would be feasible. The advantage of large—scale urodynamic data collation would be the ability to pool large numbers of datasets and reduce the risk of selection or operator bias.

An approach additional to inclusion of separate data points could be the inclusion of the raw data and traces. This enables the option to use predictive analytics, such as machine learning. With machine learning, certain patterns in urodynamic data could be detected that are not visible to the human eye. A technique is already used in imaging and referred to as radiomics. If these patterns are correlated with clinically relevant outcomes, they can be used as important tools in diagnosis and treatment (predictive analytics). For example, a certain combination of urodynamic features could predict the chance of an underlying neurological disease, leading to an earlier referral to a neurologist. Also, in patients with neurogenic dysfunction of the lower urinary tract computer algorithms could predict the occurrence of future renal impairment, necessitating early intervention to protect renal function.

KEY FEATURES IN QUALITY CONTROL

Measurements have little value unless their quality can be assured. If the level of quality varies between different data sources, it is questionable how those data can be credibly combined. It is thus essential, when considering the construction of a UDS data registry, that there are assurances that the data submitted have an acceptable level of quality. In urodynamics, the quality of a pressure signal is shown by normal, stable resting pressures, continuous live signal, good response to coughs (or other pressure rises), and assurance of zero to atmospheric pressure. For flow signals, artefacts should be screened out, so that values of maximum flow and time duration are correct. In addition, all values should be within credible limits, to guard against machine recording or data input errors. For descriptive data, for example, symptoms, only standard terms should be used and free text avoided.

To manage this quality across multiple sites submitting data, it will be necessary to ensure that levels of quality are attained not just on commencement of the program, but also throughout its life. Strategies to manage this could be:

Submission of example traces from sites to demonstrate technical capacity

Regular audit of quality to demonstrate that performance survives changes of staff

Setting credible limits for data values as an automatic screen for anomalies

Investigation of automated systems of gauging UDS trace quality

Identification of outlier data to prompt original data checking

Random periodic checks of sample traces to ensure data transfer integrity

Use of restricted number of parameters to limit data noise and management load

These strategies imply some commitment of resources, but without them the quality of the data could not be assured and the purpose of the registry be placed in jeopardy. To mitigate against unforeseen problems in data collection and data quality, a trial phase of the registry is proposed.

ICI‐RS PANEL DEFINED CHALLENGES FOR A REGISTRY ROLL‐OUT

Table 1 sums up the most important challenges to face when designing and distributing a multinational and multicentre urodynamic data collection.

Table 1

Challenges in the design of a multinational and multicentre urodynamic data collection

Data agreements	Considerations on which existing registries to include (how to approach them) (obtain contracts and permissions)
Patient consent	This may have been given for the registry to hold the data; consent should also be checked if the data can be used for research by third parties.
Data anonymisation	Minimize risk of reidentification.
IT infrastructure and data procurement	Defining storage location of the registry data and secure safe transport of the data.
Data access and storage	There are generally two types of data access; (a) a centralized model where the registries provide the data and which can be stored on individual servers, or (b) a federated model where there is access to data in the server and which can only extract aggregated statistics. The servers would need to be multi node cpu, with large memory capacity, and dedicated data management software such as Hadoop, and statistical software, such as R. Depending on where the data are coming from, there may be restrictions on where it can be stored (for example, EU data will have an EU storage range).
Common data model	Securing the data to enter the registry in a similar format and unit with identical variable names/labels. This will also allow programmers to navigate between datasets with efficiency.
Data protection	The need for support on GDPR EU ruling on data privacy and protection, to ensure data are handled compliantly and legally.

FUTURE RESEARCH

Agreements and recommendations of the ICI‐RS panel are as follows:

Need for “big data” in the urodynamic society

The panel agrees on the fact there is a lack of robust evidence to support the utility and value of urodynamic studies. The interobserver agreement of urodynamic findings is generally low and in some cases clinical guidelines for UDS have been based exclusively on low grade scientific evidence or expert opinion in the absence of high level evidence. Clinical registries could present an opportunity to improve the scientific evidence base currently lacking in urodynamic studies by the collection and sharing of urodynamic study data both nationally and internationally.

Formation of an international urodynamics panel supported by an international urological society

To design and build a multicentre/multinational urodynamics database it starts with collaboration between the key (European) urodynamic centers. Trust and a common motivation to answer questions related to pressure‐flow studies and eventually to improve measurement techniques are essential in this respect. The urodynamics research panel should look into projects that have already started (such as the PIONEER project for prostate cancer) to define and understand the process involved in such a project. The ICS Urodynamics Committee should be approached for participation and guidance.

Defining the pitfalls of a multicentre urodynamics registry roll‐out.

Table 1 represents the basic pitfalls involved in multicentre and multinational roll‐out of a urodynamics registry. The table represents basic data management issues involved in a large study. In addition, quality control of the urodynamic traces and data entries remains an important feature during a future roll‐out. These features are essential in the framework moving forward with a urodynamics registry. Questions to be answered with regard to:

IT infrastructure and data storage: How will the data be transported and will the data be stored at one server?

Data harmonization: Will each dataset be analyzed separately and then the estimates pooled? Or is it possible to create one large combined data?

Data protection: How can we secure protection of the data?

Financial structure

To maintain a long lasting successful multicentre registry financial support is indispensable. Industry and international urological society should join forces as we may need a urodynamics registry in line with the new (2020) Medical Devices Regulations.

CONCLUSION

At present the value of urodynamics as a diagnostic tool is poorly defined due to a lack of high quality evidence, the lack of large study populations and clinical embedding. The 2019 ICI‐RS panel defined a distinct need for a multicentre approach to determine current weaknesses of urodynamic measurements and to address its role in treatment response. This document indicates the potential hurdles that need to be taken for a successful multicentre urodynamics registry. In addition, the panel has defined recommendations for future research and project development.