Survival, reintervention and surveillance reports: long-term, centre-level evaluation and feedback of vascular interventions.

The combination of registry and administrative claims data have facilitated research and quality improvement efforts. Using Vascular Quality Initiative (VQI) registry data and Medicare claims we have generated centre-specific survival, reintervention and surveillance reports which benchmark participating centres' performance to the VQI as a whole and to published guidelines. In 2021, we distributed these reports to 303 participating centres. These reports offer an opportunity for centres to evaluate their performance and identify focus areas for quality improvement work. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background

Resources such as registry data and administrative claims data offer insights that have been used to advance clinical knowledge and practice in many ways.1–7 The combination of registry data with detailed clinical information and administrative claims data with more complete long-term follow-up, provide valuable information that neither source could offer independently. We have previously demonstrated the usefulness of the combined data sources in facilitating the evaluation of long-term vascular device outcomes.8 We sought to leverage this data infrastructure to create the survival, reintervention and surveillance (SRS) reports, which provide centre-specific feedback on select vascular procedures and outcomes. More specifically, the feedback provided allows each centre to compare their performance to the VQI overall and examine compliance to published guidelines.

Purpose of the registry

The Society for Vascular Surgery’s Vascular Quality Initiative (VQI) is a multidisciplinary, quality improvement and patient safety collaboration with registries collecting information such as preoperative risk factors, intraprocedural variables, postprocedural outcomes and 1-year follow-up data on select vascular procedures from over 800 centres across the USA. Beginning in 2010, regional quality improvement groups were aggregated into a national registry to facilitate multicentre data collection, research efforts and quality improvement projects. Subsequently, the Vascular Implant Surveillance and Interventional Outcomes Network (VISION) was founded in 2015 as a collaborative effort between the VQI and the Medical Device Epidemiology Network (MDEpiNet). The goal of VISION is to establish a national repository of linked VQI registry data and Medicare claims data to allow for more robust research on the long-term safety and effectiveness of vascular procedures.

Aims of the report

This report aims to describe the development of the VQI SRS reports including:

Registry information collection and application to centre-specific reports.

SRS report contents and report usage.

Possible report applications and future improvements.

General methods

VQI registry data collection

The VQI represents a multidisciplinary, multicentre effort to collect key information needed to evaluate the quality and safety of vascular surgery procedures. Data collected by the VQI includes: preoperative risk factors, intraoperative variables, postoperative outcomes and 1-year follow-up data for 13 major vascular procedures as outlined in table 1. Data are entered into a secure cloud-based platform called Pathways designed for the VQI (and operated by FIVOS). Each participating centre is required to enter data for all eligible cases. There is generally a designated manager at each centre that enters the data. The VQI then aggregates the data and generates blinded (deidentified) datasets for QI and research purposes. To date, the registry is composed of data pertaining to over 900 000 cases collected by more than 800 participating centres.

Table 1

Procedures collected by SVS VQI

Procedures collected by SVS VQI
Carotid artery stenting*	Suprainguinal bypass*
Carotid endarterectomy*	Lower extremity amputation
Endovascular AAA repair*	Peripheral vascular intervention*
Haemodialysis access*	Thoracic and complex EVAR*
Inferior vena cava filter	Varicose vein procedures*
Infrainguinal bypass*	Venous stenting

*Available within VISION database.

EVAR, endovascular aneurysm repair; SVS, Society for Vascular Surgery; VISION, Vascular Implant Surveillance and Interventional Outcomes Network; VQI, Vascular Quality Initiative.

VISION-Medicare data linkage

Taking the utility of VQI data a step further is the VISION linkage of the VQI registry data with Medicare data. Records of Medicare part A and part B beneficiaries within the VQI registry are linked to corresponding Medicare claims data.9 VQI records are linked to Medicare claims data using a combination of direct and indirect methods. The direct linkage is done by CMS. For VQI patients not matched to Medicare beneficiaries through direct linkage with social security numbers, a previously validated, indirect linkage algorithm is employed. The indirect linkage relies on: facility ID, zip-code, date of birth, sex and procedure date. This indirect linkage method has demonstrated >99% specificity in previous works.9 Among patients captured in the VQI, 72%–88% (procedure dependent) are eligible for Medicare linkage, and among them 88%–94% (again, procedure dependent) are successfully linked.

The completeness of Medicare data for use in research, especially mortality and demographic data, is greater than VQI alone and has been found to be reliable in prior works.10 11 Our colleagues have shown that Medicare data captures over 30% more deaths than VQI alone, as deaths in claims are validated by multiple external sources.12 As far as missingness of data is concerned, the VQI data does present occasional challenges relating to clinical variables such as urgency (occasionally missing) and diameter entries (occasionally missing or invalid). In the generation of the SRS reports, however, all data included in the report is required to be valid and non-missing for the observation to be included.

After linking procedures in the VQI registry to Medicare claims, longitudinal outcomes such as death, reintervention, postprocedure imaging and procedure-specific adverse events are identified. Figure 1 demonstrates the flow of information ultimately leading to the SRS reports. To choose procedures and outcomes to focus on, a subgroup of the VISION Steering Committee held a series of meetings to review and discuss outcome variables and definitions. Final selection of variables was made based on evidence-based guidelines and data availability. Initial work undertaken by the VQI VISION team includes the creation of Medicare-matched data sets for: endovascular aneurysm repair (EVAR), open abdominal aortic aneurysm repair, peripheral vascular interventions, thoracic EVAR, carotid artery stenting, carotid endarterectomy, infrainguinal bypass, supra-inguinal bypass, haemodialysis access and varicose vein procedures.

Figure 1

Flow chart of information movement and processing leading to SRS reports. SRS, survival, reintervention and surveillance.

Patient and public involvement statement

Given the nature of this clinician focused end-product and the sensitive information involved in the registry itself, there was no patient/public involvement in the creation of this work.

Main findings with discussion

SRS report generation and dissemination

With centre-level feedback serving as a cornerstone of the VQI mission, the VISION team distributed their first centre-specific SRS reports in 2019 using the matched data sets for EVAR.1 13 In 2021, this effort was expanded to include SRS reports for EVAR procedures, asymptomatic elective AAA repairs (EVAR or open repair), and carotid interventions.

SRS report metrics

For EVAR, the SRS reports include 5-year Kaplan-Meier estimates for: freedom from reintervention, freedom from surveillance failure and freedom from death (figure 2). As long-term benefit is known in the literature, these parameters are included to allow benchmarking to other participating centres as well as to published outcomes and guidelines.14–18 Surveillance metrics were included given consensus recommendations for lifelong annual imaging and because late aneurysm rupture may be related to surveillance failure.14 19 20 EVAR SRS reports also include median AAA diameter, median age, per cent male and per cent urgent/emergent (figure 2). These additional variables were included for centres to better interpret their performance relative to the rest of the VQI; for example, a centre with an above average aneurysm diameter, or performing a greater percentage of emergent cases than the VQI average, might expect to see greater reintervention rates.

Figure 2

Sample SRS report on EVAR performance. EVAR, endovascular aneurysm repair; SRS, survival, reintervention and surveillance.

The asymptomatic elective AAA reports include data on: 5-year survival, percentage of procedures performed for aneurysms less than 5.5 cm for males (or less than 5 cm for females), per cent of EVAR procedures with imaging within 15 months, and percentage of repairs on patients ≥80 years (figure 3). These metrics were chosen to help evaluate centre-specific morbidity, mortality and adherence to treatment size and imaging guidelines.

Figure 3

Sample of SRS metrics for elective AAA surgeries. AAA, abdominal aortic aneurysm; SRS, survival, reintervention and surveillance.

SRS reports for carotid interventions (endarterectomy and stenting) similarly provide information on 5-year survival and per cent of patients over 80 years old. Additional metrics provided for carotid interventions include per cent of procedures performed for asymptomatic disease, and per cent of patients prescribed statins and antiplatelets at discharge. Similar to the above procedures, these metrics were also chosen to allow for comparison to other participating centres and to evaluate adherence to established standards in the management of carotid atherosclerotic disease.21 22

SRS report dissemination and use patterns

In 2021, SRS reports were distributed to 303 participating centres. To gain a preliminary understanding of how these reports were used, we examined how frequently each centre accessed their most recent SRS report and studied how this was related to other VQI quality measures, especially centre-level long-term patient follow-up rates and centre-level aneurysm sac diameter reporting. Sac diameter reporting rates were chosen to serve as crude indicators of registry input engagement, and administrative (non-technical) centre performance.

Among these 303 centres, 180 (59.4%) had at least one access to their report. Long-term follow-up rates across all report receiving centres ranged from 0% to 100% with a mean follow-up rate of 64%. In contrast to sac rate reporting, mean long-term follow-up rates were significantly higher in centres with at least one access compared with those with no access (69% vs 55%, p<0.0001). Sac diameter reporting rates across all report receiving centres ranged from 0% to 100%, with a mean sac diameter reporting rate of 54%. Mean sac diameter reporting rates were similar in centres with at least one SRS report access compared with centres with no access (57% vs 49%, p=0.07).

Figure 4 demonstrates the distribution of long-term follow-up rates, where the trend towards greater follow-up among report accessing centres can be seen. Scatterplot analyses of number of accesses vs sac diameter reporting rates and number of accesses versus long-term follow-up rates, however, did not reveal a linear relationship (R2=0.01 and R2=0.04, respectively). Distribution plotting revealed a pattern towards higher long-term follow-up rates and sac reporting rates among centres in or above the 95th percentile (>5 accesses) for report access. However, among centres with no access, the inverse relationship between the number of report accesses and sac diameter reporting rate was not apparent with sac long-term follow-up rates and sac reporting rates being more widely and evenly distributed (figure 5).

Figure 4

Distribution of long-term Follow-up rates by presence or absence of report access.

Figure 5

Distribution plots of sac diameter reporting rates and long-term follow-up rates by level of report access.

Discussion

Tracking patient outcomes and quality of care is a long-standing effort that many registries have sought to address. These efforts most often use a single data source (eg, patient chart review, billing data). While such efforts are commendable, these methods—when used in isolation—possess inherent limitations in either the granularity of clinical information available, or the ability to follow patients across space and over larger swaths of time. Here, we outline a hybrid-data approach used to generate reports on centre-specific outcomes.

VQI SRS reports are a tool leveraging hybrid data sources to facilitate quality improvement within vascular surgery. The long-term follow-up offered by these reports represents a key strength of this approach. For example, patients being lost to follow-up (relocation, death, etc) has traditionally limited the collection of any further data from a given patient. VISION’s linkage to Medicare claims, however, allows for ongoing capture of pertinent clinical events (reintervention, amputation, death, etc) across multiple centres. In addition, the SRS reports can serve as a tool for a centre to compare its performance to other VQI participants. By easily visualising where one’s centre lies among peers on key performance indicators, it becomes easier to highlight specific areas for improvement. The quality metrics reviewed also offer insights into patient demographics which can be related to how well centres are conforming to various guidelines. For example, AAA metrics such as that seen in figure 3 demonstrate what per cent of elective AAA surgeries at a given centre are being performed below consensus size thresholds, as well as how well a centre meets recommended postoperative imaging surveillance guidelines.

The SRS reports are not without limitations. Due to CMS suppression requirements, reports are only available to centres that perform a sufficient number of cases with Medicare patients. In addition, the specificity could be improved with more granular data on patient demographics and device data, however, further disaggregation at the centre level would reveal smaller numbers which is not permitted by CMS. Another limitation of the SRS reports is the time lag of the necessary data due to the time needed for claims data cleaning, application, and processing; For example, only data up to 2018 was fully available for inclusion in the 2021 SRS reports.

Looking ahead, the SRS reports could be improved by focusing on centre adoption and additional reporting metrics. Regarding centre use of the SRS reports, our preliminary study of report access revealed that ~59% receiving centres actually opened the report. We hope with improved communications and ease of access to these reports we might be able to improve the uptake of these SRS reports. From a content standpoint we hope to improve the SRS reports by incorporating implant data. As a specialty heavily reliant on various devices such as grafts and stents, device-level outcomes could offer insights to not only vascular surgery centres, but to additional stakeholders such as device manufacturers, the Food and Drug Administration (FDA), and even patients themselves. For example, similar reports could be created to monitor device performance rather than centre-specific performance. If a specific device reveals a pattern of early failure, that could have implications for surgeon device selection, manufacturer recalls and FDA review. In the future, we also hope to further refine the reports with more granular additions such as sex-specific metrics.

Looking ahead, new or existing registries could adopt similar methods to deliver centre-specific reports. Importantly, the methods outlined for creating the SRS reports are not specific to vascular surgery and could be employed in other specialties, surgical or otherwise. Some potential uses for these methods could include assessing compliance with various evidence based guidelines or disparities in treatment based on sex, race/ethnicity, geography, etc. In addition, within the surgical community, registries have often focused on shorter-term outcomes such as in-hospital complications or 30-day mortality. Our methods show how leveraging Medicare claims data facilitates tracking patient outcomes over longer periods of time, and we believe that better understanding the longevity of our interventions and implantable devices is, and will continue to be, of growing importance.