International Validation of the Population Based Malta Vascular Registry: A Vascunet Report.

INTRODUCTION: Quality registries can be used to study treatment patterns and changes in trends of complications related to procedures and devices. To ensure that such study is possible in registries, there is a need to use standardised variables and the registry must have high internal and external validity. This study was an international external and internal validation of the newly initiated Maltese Vascular Registry. REPORT: Two international vascular registry consultants visited the Maltese Vascular Registry (MVR), and conducted an external validation on all carotid and aortic aneurysm repairs performed in 2017-18. The external validation was conducted by comparing hospital administration lists with the MVR list. Using randomly chosen numbers, an internal validation was conducted of 20 random cases of carotid and 20 aortic aneurysms, 10 from 2017 and 10 from 2018, to validate date of operation and procedure against the patient's medical record. The registry is built as a database using national personal identifier codes, with variables for date and type of procedure, and anaesthetic method used, to which a note is attached describing the indication, intervention, and follow up. Between the hospital registry and the MVR, 111 of 115 cases could be identified correctly, corresponding to 97% external validity. Between the patient case records and the MVR, the dates and procedures of 20/20 carotid and 20/20 aortic aneurysm procedure were identical, indicating 100% internal validity. DISCUSSION: The MVR showed an external validity of 97% and internal validity of 100%. Future work should incorporate specific variables for comorbidity, procedures, and outcomes, with the registry aiming to incorporate international recommended variables for comorbidity, procedure, and outcome.

Introduction

Registries are an important tool that can be used to improve quality of care and harmonise medical treatment for patients, confirm the results from evidence (i.e. RCTs) in broader populations, and safeguard that guideline recommended treatments reach patients.

Vascunet is an international collaboration set up to collate the international comparison of outcomes following vascular procedures on a national and regional basis. To achieve this, use of standardised variables for comorbidity and outcome is necessary. Recently, recommendations for a minimum dataset of vascular registries have been published for peripheral arterial disease,, and templates for aortic aneurysm and carotid treatment variables are available (www.vascunet.org). Standardisation of reporting alone does not make comparisons possible. To create reliable clinical quality registries, these must be validated to confirm that data on all relevant procedures they are set to monitor are captured (external validity), but also to validate that the variables collected are correct and can be validated in the patient journal (internal validity). To increase external validity of participating registries, some national registries have been audited by independent specialists within the Vascunet group.

The purpose of the present report was to conduct external and internal validation of the Maltese Vascular Registry (MVR).

Report

The MVR was initiated in 2007 by Professor Kevin Cassar, head of the Department for Vascular Surgery, the Mater Dei Hospital at the University of Malta. From 2015 on it gained full national coverage of all vascular and endovascular procedures conducted in Malta. The entire data management and maintenance of the registry are performed by C. Kassar and his team, without any additional funding from third parties. Malta consists of a group of islands and in 2018 the population was estimated to be 475 000. Healthcare is provided through one public university hospital, the Mater Dei Hospital, and eight local healthcare centres. The Maltese vascular service is provided through the Mater Dei Hospital. Until June 2019 the registry contained data on more than 20 000 procedures, and every year approximately 700 arterial, 350 venous, and 150 vascular access procedures are registered.

The MVR uses the national Maltese personal identification code as the key variable. Attached to the key variables for the conducted procedure (up to a total of three procedures in the same operation), details of procedure, side, date of admission, date of procedure, date of discharge, indication, seriousness (minor, intermediate, or major, defined according to the risk of complications inherent in the procedure, so for example procedures under local anaesthetic such as toe amputation are minor, whereas peripheral bypass is major), consultant, type of anaesthesia (general or local/locoregional) are captured. A free text file is linked each file containing details on demographics, comorbidities, pre-operative, post-operative, and long term courses (lifetime vascular history), and whether the patient was re-admitted with complications. The hospital administration system is set up so that a patient cannot be admitted to a department without being registered. Additionally, procedures cannot be initiated if the patient is not booked and registered as admitted in the administration system. With use of these “hard stops”, the administrative system ensures 100% capture of all procedures.

As recommended by Vascunet,, the MVR was examined for external and internal validity. Only aortic and carotid procedures were validated because not all registries capture procedures other than these.

For external validation, cross references were made for 2017 and 2018 between the hospital administration system and the MVR, investigating the similarity between the two registries on identity, date, and type of procedure of all aortic aneurysm and carotid artery procedures.

For internal validation, 20 aortic aneurysm and 20 carotid cases (half in 2017 and half in 2018) were compared between the MVR and the medical journal. The fields validated were date of operation and procedure type (open or endovascular). The fields indication, type of anaesthesia, and seriousness were not validated because, at present, they are the only procedure related variables, containing limited quality related outcomes.

The present validation was performed by the authors on 28 May 2019. Validators are national representatives of their national registries and have previously participated in validations. NE had previously validated the Danish Vascular registry in 2018.

External validation

Aortic aneurysm repair

All cases in the MVR were in the administrative system. However, three cases in the administrative system could not be found in the MVR (Table 1).

Table 1

External validation, number of procedures in 2017 and 18.

	Administrative system	Maltese Vascular Registry
Aortic aneurysm procedures
2017	21	20
2018	30	28
Carotid procedures
2017	26	25
2018	38	38

One case in 2017 missing from the MVR, described development of claudication after an elective EVAR. The patient underwent a groin revision without complication and no further treatment was necessary. The two cases in 2018 missing from the MVR, were an open AAA repair in which the patient had no complication, and a ruptured AAA that embolised peri-operatively to both legs causing the patient to die before discharge.

Carotid repair

One case was missing from the MVR compared with the administrative system. After searching for the ID in MVR the case was found, but the date has been entered incorrectly as 2016. There were no complications during the admission.

This resulted overall in a 97% (111/115) external validity.

Internal validation

Details for all 40 cases were identical between the MVR and the medical journal for procedure date and type of intervention (aortic open repair, EVAR, or carotid endarterectomy), resulting in 100% internal validity.

Discussion

The MVR and the independent vascular unit are still under development, which impacted the observation period during which the Maltese vascular service underwent a transfer of care from both general and vascular surgeons to become an independent vascular service. This change in vascular practice included opportunistic screening for aortic aneurysm and carotid disease, explaining an almost 30% increase in activity over the two year period.

Despite this transition, the external validation showed that it is possible with very few resources and a high degree of personal commitment from the responsible physicians to build and maintain a basic, but high quality vascular registry.

At present there are no variables in the registry on comorbidity, operative/endovascular methods and devices used, or complications and outcomes, because this information is kept in attached text files without connection to the administrative system. This means that new individual variables would be retrospectively created from the attached journal records or by going back through the medical journal to when the studies were conducted.

To increase the usability and validity, the registry needs to introduce/develop variables containing information about comorbidity for risk adjustment, operative/intervention parameters important for outcome as well as outcome variables, so that data are reported in standardised ways and analyses can be conducted without the need for re-entry of data from text files (depending on the analysis undertaken). This task can be demanding in countries and regions with limited resources and IT infrastructure. Simply allocating the resources needed to develop a full vascular registry containing the proposed variables for collecting data on aortic aneurysm, carotid and peripheral arterial procedures is expensive and time consuming. The MVR is in the unique situation of full coverage of all vascular procedures undertaken in Malta, offering population based evidence. This is achieved via centralisation of the entire vascular service to one centre. Obtaining full national coverage is rarely achievable in other countries. Only registries with full national coverage data have the power to reflect vascular treatment of an entire country. Although challenging because of limited resources and manpower to further improve and maintain, the unique benefit lies in such complete coverage. For this reason, the MVR should be encouraged to develop its database further, to make it useable for outcome measures in long term follow up. The Vascunet collaboration has demonstrated its ability to compare data from different vascular registries.6, 7, 8, 9 However, the weakness and challenge of including and comparing different registries with data of varying quality, remains a reason for debate; it is of importance to seek high quality data and achieve the best possible alignment.

It would be useful to align databases and preferably even the data platform with international standards, complying with European data protection rules (EU-GDPR), to maximise the value of collected work to increase patient safety and harmonise patient care on an international basis.

For a country requiring such a vascular database, transfer of a fully developed and validated database from a country in which it is already established seems like an obvious solution. However, the inherited cost of translation, hosting, and continuous updating is often so high as to exceed the price of developing a local, nationally or regionally developed database, in which local data protection rules are fulfilled, and maintenance is likely to be possible. Another strategy to achieve standardisation, including the EU-GDPR, and device specific data, would be to build one shared common data platform, covering all necessary items to allow true head to head comparisons between countries. Creation of such a database would be a challenge but would be an achievement of utmost value for all, including patients. Such a shared vascular data platform would allow additional nations, regions, or hospitals to join the collaboration easily, gaining information about their quality and, by contributing to international collaborations, receiving benchmark data to improve patient care. The MVR has demonstrated its eagerness and should continue to improve its database further in order to make future outcome measures possible.

Funding

Travel expenses were funded by the European Society of Vascular Surgery to Vascunet for international validation of member vascular registries.

Conflicts of interest

None.