Development and validation of a risk score to prioritize patients for evaluation of access stenosis.

BACKGROUND: Access flow dysfunction, often associated with stenosis, is a common problem in hemodialysis access and may result in progression to thrombosis. Timely identification of accesses in need of evaluation is critical to preserving a functioning access. We hypothesized that a risk score using measurements obtained from the Vasc-Alert surveillance device could be used to predict subsequent interventions.
METHODS: Measurement of five factors over the preceding 28 days from 1.46 million hemodialysis treatments (6163 patients) were used to develop a score associated with interventions over the subsequent 60 days. The score was validated in a separate dataset of 298,620 treatments (2641 patients).
RESULTS: Interventions in arteriovenous fistulae (AVF; n = 4125) were much more common in those with the highest score (36.2%) than in those with the lowest score (11.0). The score also was strongly associated with interventions in patients with an arteriovenous graft (AVG; n = 2,038; 43.2% vs. 21.1%). There was excellent agreement in the Validation datasets for AVF (OR = 2.67 comparing the highest to lowest score) and good agreement for AVG (OR = 1.92).
CONCLUSIONS: This simple risk score based on surveillance data may be useful for prioritizing patients for physical examination and potentially early referral for intervention.

INTRODUCTION

Maintaining a functioning arteriovenous (AV) access is critical to providing adequate dialysis clearance, reducing catheter exposure, and lowering the costs of care in patients on chronic hemodialysis. Indeed, vascular accesses (VAs) are considered to be the “Achilles' heel” of hemodialysis. Vascular access‐related expenses and complications account for approximately 20% of hospitalizations, $2.8 billion in healthcare costs in the US, and substantial morbidity and mortality. As surgical procedures and hospitalizations associated with attempts to maintain a functioning AV access are major contributors to these costs, methods to avoid these adverse outcomes are needed.

Inadequate access blood flow is a common problem in mature AV fistulas (AVF) and grafts (AVG), resulting in inefficient dialysis and an inability to attain prescribed Kt/V as an indicator of adequate dialysis. Hemodynamically significant stenoses may occur in the inflow, main body, or outflow of the vascular access, reducing blood flow in the access. Stenosis can occur at a single site or develop at multiple sites; these may develop concurrently or more commonly sequentially.

Stenosis invariably leads to access dysfunction. Inadequate access flow most often is associated with one or more underlying stenoses, which can cause recirculation of blood within the access site during dialysis and eventually culminate as thrombosis. Thrombectomy procedures, however, fail to salvage between 10% and 30% of accesses, necessitating the insertion of a central venous catheter (CVC) and potentially the creation of a new AV access. Even a successful thrombectomy may reduce the useful life of the access. Since there are a limited number of sites available for the placement of an AV access, protecting and extending the life of a functional AV access should be a major goal of quality vascular access care. Given that approximately 20% of chronic hemodialysis (HD) patients using a CVC have exhausted all of their viable access sites, it is critical to maintain each functioning AV access as long as possible.

Interventions, including angioplasty and access revision, may be used to treat access flow dysfunction in an attempt to preclude thrombosis and extend the useful life of the access. Early identification of patients with a functionally restrictive access lesions is critical so that they can be referred for intervention in a timely manner. Clinical indicators, such as persistent swelling of the arm, excessive post‐dialysis bleeding, inability to maintain prescribed pump speed, or a drop in dialysis adequacy suggest the presence of stenosis. However, these clinical indicators are considered to be late indicators of dysfunction as the degree of stenosis has often increased to more than 50% of the vessel diameter (typically averaging >70%) by the time they are discovered. Furthermore, these clinical indicators have no objective standard that can be used as a reference to trigger a referral. An adequate physical examination can detect significant stenosis, but such exams require a trained clinician as well as the time to perform them, which can be limited in busy dialysis units. Methods to help identify and prioritize patients who are at risk of complications for clinical examination and potential referral for intervention are needed to improve access outcomes.

The Vasc‐Alert vascular access surveillance technology uses treatment data collected during each dialysis session (blood pump flow to the dialyzer (Qb), venous and negative arterial pressure, and the mean arterial pressure (MAP)) recorded every 30 min in the medical record to derive the inta‐access pressure (IAP) at the tip of the venous needle. Simply put, the venous IAP is calculated by subtracting the back pressure caused by the resistance to the blood flow through the needle and tubing from the venous drip chamber recorded pressure, and adjusted for the dialyzer blood flow, type of needle and tubing, and hematocrit. To normalize these calculations between dialysis sessions, the IAP is then divided by the MAP and is called the venous access pressure ratio (VAPR). The Vasc‐Alert technology also provides a method for monitoring for issues on the arterial side by taking the absolute value of the pre‐pump arterial pressure (PPAP) and dividing it by the blood pump flow rate (Qb), which is referred to as the arterial access ratio (AAPR).

High VAPR values in the access are considered to be a marker for outflow stenosis, whereas high AAPR values reflect inflow stenosis. Because Vasc‐Alert calculates VAPR and AAPR for each treatment, trends in the access can be discerned. For example, an increase in VAPR alone over time can reflect an increase in access pressure due to a growing “downstream” venous occlusion. Similarly, high AAPR values are indicative of an inflow occlusion. Alerts are issued to staff indicating an access at risk for complication when there are three consecutive pressure readings above a predefined threshold. The efficacy of this device in identifying dysfunctional accesses is well established, with sensitivity of 74% and specificity of 92%. , Measurement of IAP is also helpful during angiography to assure all lesions have been detected and corrected.

While the issuance of a VAPR or AAPR alert indicates an access complication risk, there are a number of other metrics that are derived from the Vasc‐Alert results that are also indicative the potential for an access complication. For example, a patient on alert every week for the past month is probably more at risk than a patient with only one alert. Likewise, a patient with very high VAPR or AAPR pressures is probably more at risk than a patient whose pressures are just at the threshold. Another risk indication is how rapidly pressures are increasing or in some cases decreasing over time. An access with VAPR pressures increasing very slowly may not be at the same level of risk as an access with pressures increasing more rapidly. Clinical staff are trained to interpret these other more subtle indications of risk when reviewing the Vasc‐Alert reports and triaging patients for interventional referral.

We hypothesized that a simple access risk score that summarizes these various risk indicators from the surveillance device results, would be associated with subsequent interventions and could be used to more easily by staff (more user friendly) to prioritize patients for evaluation and possible intervention.

METHODS

Data Sources

Data were compiled separately for use as a Development dataset and a Validation dataset. For the Development dataset, electronic data records on interventions were obtained from three vascular access centers from 2008–2018. These intervention records were linked to the patients' Vasc‐Alert results from 86 dialysis facilities that used Vasc‐Alert and sent their patients to these access centers. Patient data were included only for periods during which intervention data were available for at least 6 months with no gaps. Because some patients at a given dialysis facility may go to an access center other than the one from which we received data, we limited our analyses to patients with at least one recorded intervention at the one of the three selected access centers. As patients were not necessarily enrolled at their first treatment or prior to their first intervention, the first intervention during the study period does not necessarily represent that specific patient's or specific access's first intervention. Rather, the study data are a snapshot of time during a given patient's course. A separate Validation dataset was obtained from four different access centers supporting patients treated at a separate set of HD centers. Treatments from 2017 through 2020 were included in this dataset, following the same inclusion and exclusion criteria as for the Development dataset.

Measurements

Vasc‐Alert calculates the VAPR and AAPR in the access with each treatment and presents these results graphically so that trends over time can be evident. From these results, summary metrics can be derived such as the number of alerts over time (density), the magnitude of venous outflow or pre‐pump pressure (severity) and their rate of increase or decrease (slope), and whether adequate blood flow is being attained (see Figure 1). The researchers identified five metrics, each of which provides a different perspective on the health of the AV access. These metrics were assessed as potential indicators of risk. They include the following:

FIGURE 1

Graphic is displaying 6 months of venous access pressure ratio results, which can be summarized mathematically by breaking the graph down into its core components, that is, amount of pressure (severity, height of the bars), number of alerts (density, number of red bars), and the rate change over time (slope) [Color figure can be viewed at wileyonlinelibrary.com]

The mean VAPR as a measure of venous pressure in the access. Higher pressures are markers for increasing stenosis in the access. Increasing occlusion in the access increases the risk of thrombosis.

The slope of VAPR as an indication of how the pressure is changing over time. The steeper the slope, the faster the occlusion is growing.

The number of alerts in the past 28 days. An alert requires three consecutive VAPR or AAPR readings above the threshold and is a well‐accepted indication of access risk. The more alerts, the more the access is at risk.

The mean AAPR as a measure of arterial inflow adequacy. The more negative the pre‐pump arterial needle pressure at an achieved dialyzer blood pump flow, the more likely is the blood flow in the access compromised, typically due to an inflow occlusion.

The number of treatments in the past 28 days in which the average blood flow rate did not achieve at least 90% of the prescribed blood flow rate (“low BFR”). Inability to achieve prescribed blood flow is known to be a clinical indication of access dysfunction.

The mean monthly VAPR and AAPR metrics are calculated by averaging the results from the previous 28 days. The VAPR slope is calculated over the previous 28 days. Patients had to have at least 4 HD treatments with complete measurements during the 28‐day time‐period to be included in analyses.

Interventions

For each HD treatment, we assessed whether an intervention was performed during the subsequent 60 days. This period was chosen to provide staff adequate time to confirm the indication of risk by reviewing clinical records and performing a physical examination. It also provides enough lead time to make a referral. Interventions considered included angioplasty, thrombectomy, atherectomy, stent placement, surgical revision, and catheter insertion. Diagnostic angiograms or other procedures without treatment of stenosis were not included.

Statistical analyses

Each HD treatment was treated as a separate observation with or without an intervention in the subsequent 60 days. To reduce the potential impact of autocorrelation due to repeated interventions within a single patient, only treatments prior to the first intervention during the study period for an individual patient were included in the analyses.

We assessed the association of each individual metric with the incidence of an intervention during the subsequent 60 days using logistic regression models. Continuous metrics (i.e., VAPR, slope of VAPR, mean AAPR) were modeled using restricted cubic splines with 5 knots at the 5th, 27.5th, 50th, 72.5th, and 95th percentiles. Categorical variables (i.e., number of alerts, number of treatments with low BFR) were modeled in grouped categories. Metrics found to be significantly (p < 0.01) associated with subsequent interventions were included in the development of the score.

Score Development

We developed the score using sequential logistic regression models. Based on expert opinion, the metrics were included in sequential logistic regression models in the following order: mean VAPR, VAPR slope, the number of VAPR alerts, mean AAPR, and the number of treatments where BFR achieved was less than 90% of the prescribed rate.

We first modeled VAPR versus intervention using restricted cubic splines, as described above. We then categorized the distribution of mean VAPR into bins based on the shape of this curve and assigned a number of “points” based on the strength of the association of each bin with the odds of intervention. We next included a similar cubic spline for VAPR slope into a model containing the categorized mean VAPR and then categorized the distribution of VAPR slope and assigned “points” based on the observed adjusted association. We then included the number of alerts into a model including categories of both mean VAPR and VAPR slope and categorized the number of alerts based on the observed adjusted association. We next added mean AAPR as a continuous variable and then low pump BFR as a categorical variable. The saturated model included all five metrics. The sum of the “points” were then categorized into a score of 1–10 based on the distribution of points, with approximately 2.5% of participants with a score of 10, 5% with a score of 9, 7.5% with a score of 8, 10% with a score of 7, and an equal distribution of score ≤6. Separate sequential models were developed to estimate associations between these metric and interventions for upper arm fistulae, lower arm fistulae, and grafts.

Validation Process

The scoring system described above was then applied to the Validation dataset. The same inclusion and exclusion criteria were applied, the scores were calculated, and these were compared to the observed proportion of treatments with each score that was associated with an intervention in the subsequent 60 days. Odds ratios were calculated from bivariate logistic regression models. Confidence intervals and p‐values accounted for the repeated measurements within patients through clustered analyses. All analyses were performed with Stata, 16.1 SE (www.stata.com).

RESULTS

Development data

The development dataset consisted of data for 6,163 patients from 86 HD centers and a total of 1,457,987 dialysis treatments. This included 980,062 treatments in 4125 patients with an AVF and 477,925 treatments in 2,038 patients with an AVG. A total of 141,041 (14.4%) of the treatments in patients with an AVF and 126,295 (26.4%) of the treatments in patients with an AVG were associated with any intervention in the subsequent 60 days.

Each of the individual metrics were strongly associated with the incidence of an intervention in the subsequent 60 days (p < 0.001) (Figures 2 and 3). Curvilinear associations with interventions were observed for VAPR, VAPR slope, and AAPR.

FIGURE 2

Association of individual metrics with probability of a subsequent intervention in the next 60 days among those with an arteriovenous graft. Gray columns represent the distribution of measurements across all observations. (A) Mean venous access pressure ratio (VAPR). (B) Slope of VAPR. (C) arterial access pressure ratio (AAPR). (D) Number of VAPR alerts in the previous 28 days; (E) number of treatments with low blood flow rate in the previous 28 days

FIGURE 3

Association of individual metrics with probability of a subsequent intervention in the next 60 days among those with an arteriovenous fistula. Gray columns represent the distribution of measurements across all observations. (A) Mean venous access pressure ratio (VAPR). (B) Slope of VAPR; (C) arterial access pressure ratio (AAPR); (D) number of VAPR alerts in the previous 28 days; (E) number of treatments with low blood flow rate in the previous 28 days

The proportion of treatments with an AVF associated with an intervention in the subsequent 60 days varied from a low of 11.0% for those with a risk score of 1 to 36.2% for those with a risk score of 10 (Figure 4). This represents an odds ratio of 4.60 (95% confidence interval [CI]: 4.03, 5.25; p < 0.001) comparing the highest to the lowest risk score. As expected, the incidence of intervention in the development data increased monotonically with higher risk scores. The association of a risk score with the incidence of intervention was similar in AVG, though the incidence of interventions was higher in all categories of the score. Those treatments in the highest category had 2.81‐fold higher odds of an intervention as compared to those in the lowest category (odds ratio = 2.81 [95% CI: 2.42, 3.26; p < 0.001]).

FIGURE 4

The chart is displaying the proportion of dialysis treatments by their access risk score (1 to 10) associated with an intervention (angioplasty or thrombectomy) in the subsequent 60 days for patients with an arteriovenous (AV) fistula or graft [Color figure can be viewed at wileyonlinelibrary.com]

Validation data

The Validation dataset consisted of data for 2641 patients from 84 HD centers and a total of 298,610 dialysis treatments. This included 215,154 treatments in 1,814 patients with an AVF and 83,456 treatments in 827 patients with an AVG. A total of 36,790 (17.1%) of the treatments in patients with an AVF and 16,598 (19.9%) of the treatments in patients with an AVG were associated with an intervention in the subsequent 60 days.

There was excellent agreement between the results in the Validation and Development datasets for AVF. The proportion of treatments with an AVF associated with an intervention in the subsequent 60 days varied from a low of 12.8% for those with a risk score of 1% to 28.2% for those with a risk score of 10 (Figure 5). This represents an odds ratio of 2.67 (95% CI: 2.11, 3.39; p < 0.001) comparing the highest to the lowest risk score.

FIGURE 5

This chart indicates the agreement between the results in the Development and Validation datasets as measured by the association of each treatment by access risk score with an intervention in the subsequent 60 days for patients using an arteriovenous fistulae (AVF) [Color figure can be viewed at wileyonlinelibrary.com]

The association of the risk score with incidence of intervention showed less agreement in AVG (Figure 6). At all levels of the score, the incidence of an intervention was lower in validation than tin developmental data The overall incidence of interventions was significantly lower in the Validation data than in the Development data (19.9% versus 26.5%, respectively). Nonetheless, as in the developmental data base, a higher risk score was monotonically associated with a higher incidence of intervention, ranging from 16.9% in the lowest category to 26.2% in the highest category (OR = 1.92 [95% CI: 1.41, 2.62]).

FIGURE 6

This chart indicates the agreement between the results in the Development and Validation datasets as measured by the association of each treatment by access risk score with an intervention in the subsequent 60 days for patients using an arteriovenous fistulae (AVF) [Color figure can be viewed at wileyonlinelibrary.com]

Because a score is calculated for each patient's treatment, each score represents one data point on a continuum of access complication risk that can vary treatment to treatment. Based on only a single treatment a cut‐point of ≥7, for example, results in a sensitivity and specificity of 38% and 78% for AVF and 31% and 79% for AVG.

DISCUSSION AND CONCLUSIONS

Using the results from five pressure and flow‐based factors that can be derived for each dialysis treatment, we developed and validated a scoring algorithm that was strongly associated with the incidence of an intervention in the subsequent 60 days. While a complication may arise at any time for an AV access, the derived access risk score effectively places each access on a continuum scale of risk, and while an AV access with a score of 1 may not be at high risk, it still could develop complications. The algorithm was able to categorize treatments with AVF into 10 risk groups, with the highest risk group having a nearly 3‐fold higher odds of an intervention compared to the lowest risk group in the validation sample. The association between the risk score and incidence of a future intervention was somewhat less strong in AVG, but still associated with 2‐fold higher odds in the validation sample. The access risk score could be useful for staff in busy dialysis facilities to prioritize patients for physical examination and potential referral for intervention.

The predicted incidence of intervention in the Validation dataset was in excellent agreement with the Development dataset for AVF. In AVG, however, the agreement was lower. The overall incidence of treatments within 60 days of an intervention for AVF also was similar (14.4% vs. 17.1% for the Validation and Development datasets, respectively), whereas the Validation dataset for AVG had a significantly lower overall incidence than the Development dataset (26.4 vs. 19.9%, respectively). This may be due to the fact that the Development and Validation datasets were sourced from different facilities and access centers, as well as from different time periods. Different protocols and technical developments might have also influenced the data cohorts.

This version of the score utilized five calculated factors derived from measurements of pressures and blood flows related to an AV access, with each factor measuring a different aspect of the access. Taken individually, each measures progression over time of an access risk factor, that is, high pressures, increasing slope, and blood flow deficiency. In a sense each factor is a different window into the current health of an access. By combining and balancing these various risk factors into a simple and easily understood composite access risk score, it not only makes it easier for clinical staff to utilize the reports but should also increase the efficacy of the medical device. This conclusion seems to be supported by clinical use of the access risk score, but further studies are needed to confirm these observations.

Vascular access surveillance is defined as the use of a medical device to test the access for the presence of stenosis. There are two types of vascular access surveillance devices available for use in HD facilities to identify patients at risk of access dysfunction. They either measure the blood flow through the access or the pressure at the AV access needle site. Flow and pressure are closely related characteristics of the access. When stenosis increases to a significant degree, it restricts the amount of blood flowing through the access for the venous (outflow) side, while at the same time increasing or decreasing the pressure for an inflow lesion (arterial). Devices that measure blood flow (Transonic and On‐Line Flow from Fresenius) use a reduction in blood flow as a marker for stenosis, whereas the Vasc‐Alert methodology measures changes in pressures at the two dialysis needles as well a blood pump flow as an indicator of risk from stenosis. Both flow and pressure react to the change in the patency of the access due to the increasing stenotic occlusion. In a sense, they are measuring the same phenomenon but using two different indications of risk, that is, when the stenosis restricts outflow, the flow decreases and the pressure increases. A secondary concomitant inflow stenosis would mask this increase but flow through the dialysis circuit would still be restricted.

Flow‐based devices require clinical staff to conduct a test directly on the patient's access. Both marketed devices require that the blood flow through the needles be reversed in order to perform the test. Because this process requires between 15 and 30 min to complete, flow‐based surveillance is usually performed relatively infrequently, usually only once a month, which may not be frequent enough, particularly in grafts [19}. Direct Doppler ultrasound can also be performed and does correlate with ultrasound dilution measurements of access total blood flow (Qa), but direct Doppler is not readily available and is costly. Decreasing access blood flow measured by either dilution or Doppler has been shown to be a reliable indicator of subsequent thrombosis during a 12‐week follow‐up period. Preemptive repair of subclinical stenoses (angioplasty and/or open surgery) detected by surveillance has been shown to increase the longevity of AVF accesses when compared to clinical examination alone ‐ so called “monitoring.” However, access flow (Qa) varies with systolic blood pressure, age, location of the access, obesity, and diabetes. Use of a single critical value of Qa as a threshold for referral in all patients may be simplistic, and the optimal threshold might vary by patient subgroup. This may explain the lack of widespread uptake of Qa for the surveillance of vascular access. As for AVF, observational studies using pressure measurements show a reduction in thrombosis rate in both AVF and AVG (12,23),

The importance of PPAP, a measure of the amount of negative pressure needed to attain the desire Qb through the arterial needle, has only recently been “rediscovered” by others. , , Excessive negative PPAP can lead to a decrease in the delivery of blood flow, inadequate dialysis, and hemolysis and in most settings is restricted to a value of −250 mm Hg. Unfortunately, these recommendations are often disregarded in clinical practice with pressure sensors being “removed” from the dialysis circuit. The absolute PPAP to blood pump speed (Qb) ratio which we term AAPR may reflect dysfunction of the vascular access. In a retrospective analysis, of 490 hemodialysis patients with an AVF, this parameter alone had a sensitivity and specificity of 61% and 73% respectively using a cut off value of 0.5 in predicting AVF dysfunction. Over one year of observation, AVFs with AAPR > 0.5 had lower survival and a 3.26‐fold greater risk of failure.

Referring hemodialysis patients for elective access angiography and percutaneous transluminal angioplasty (PTA) is commonly done to prevent access failure. Chan et al performed an observational matched cohort analysis among 40,132 Medicare beneficiaries receiving hemodialysis with a fistula or graft. PTA was found to be frequent at a rate of 20.9 procedures per 100 access years. Angiography and PTA significantly increased access survival when compared with nonintervention. , , The greatest benefit occurs in patients whose accesses were “new” or those with low blood flow. Since Qb is a function of Qa and pressure changes can detect single or multiple stenosis, the current approach of an access risk score that determines relative risk for needing an intervention within 2 months would be useful to dialysis staff.

Because Vasc‐Alert surveillance uses both pressure and Qb treatment data that is captured automatically for every treatment, staff time is not required to test the access. In addition, the availability of treatment data for each dialysis session means that the access is effectively being tested every dialysis session. Because Vasc‐Alert is a screening tool and not a diagnostic test, HD facility staff are instructed to clinically examine patients that are on alert or have a high score for additional clinical indicators before making a referral for a preventive intervention. While a pressure‐based surveillance device does not require staff to conduct a test, the reports must be reviewed especially for patients that are on alert with the goal of identifying patients who are at‐risk for an access complication. Because changes in the access can occur very quickly, Vasc‐Alert issues new reports every week, which can make the review of individual patient reports a time‐consuming task especially for larger dialysis facilities. Hence, the utility of a 1 to 10 access risk score to help prioritize the patients at most risk.

As an aid for prioritizing patients, Vasc‐Alert has traditionally presented the listing of patients on alert ranked by order of the highest number of alerts in the prior 30 days. However, a ranking by the number of alerts is not necessarily the best metric for prioritization. For example, a patient may have three or four alerts because they are constantly on alert, but when the patient's report is reviewed, the average VAPR pressure may be barely above the threshold and holding steady, (no upward slope showing increasing trend of pressure). While the access may have a lot of alerts, these additional metrics indicate that the stenosis is stable and may not be at the same level of risk as a patient with only two alerts but a very high VAPR average or a patient with a very steep VAPR slope. While experienced users are able to review the graphic reports and discern relatively quickly the patients who are at the highest risk, this becomes more difficult for a less experienced user, especially when a large number of patients are on alert.

Limitations of this study include the lack of inclusion of other variables that may be associated with risk of complications, including patient demographics, comorbidities, and prior interventions. Inclusion of such factors would most likely increase the ability to predict future complications. An additional limitation is the inclusion only of patients with at least one intervention during the study period. This was considered necessary to ensure we did not include any patients where interventions would not be documented (i.e., immortal time bias). This could have inflated the overall incidence of interventions and limited the ability to discriminate patients at relatively low risk of interventions. However, the analysis did examine periods of the patients in the study during time periods of relatively low risk that were not followed by a documented intervention. We also included only the first intervention during the study periods for any specific patient to decrease any carry over bias due to re‐stenosis of the access. Excluding such follow‐on interventions may have, conversely, limited the ability to discriminate very high‐risk patients that have repeated access issues. Strengths of the study include the large datasets, the inclusion/exclusion criteria designed to minimize bias, and the inclusion of a validation cohort.

In summary, the algorithms developed from five pressure‐based metrics automatically derived from data captured during HD treatments are strongly associated with risk of subsequent interventions. The 1 to 10 score rankings provided may be useful to clinical staff to prioritize patients for clinical evaluation and potential referral for interventions. It also presents an objective method for categorizing the relative risk of an access for complications, which could be quite useful for access‐related research.