Adaptation and validation of an adverse drug reaction preventability score for bleeding due to vitamin K antagonists.

Although drug therapy is inherently associated with the risk of adverse drug reactions (ADRs), some of these events are preventable. The estimated proportion of preventable ADRs varies from one study or clinical context to another. Bleeding caused by antithrombotic agents (and particularly vitamin K antagonists, VKAs) constitutes one of the most frequent causes of ADR-related hospitalization.Hence, the objective of the present study was to adapt and validate an ADR preventability score for bleeding due to VKAs and evaluate the preventability of bleeding in 906 consecutive hospitalized, VKA-treated adult patients with a risk of major bleeding (defined as an international normalized ratio ≥5) over a 2-year period. A specific preventability scale for VKA-associated bleeding was developed by adapting a published tool.Overall, 241 of the 906 patients in the study experienced at least 1 VKA-associated bleeding event. The scale's reliability was tested by 2 different evaluators. The inter-rater reliability (evaluated by calculation of Cohen's kappa) ranged from "good" to "excellent." Lastly, the validated scale was used to assess the preventability of the VKA-associated bleeding. We estimated that bleeding was preventable or potentially preventable in 109 of the 241 affected patients (45.2%).We have developed a useful, reliable tool for evaluating the preventability of VKA-associated bleeding. Application of the scale in a prospective study revealed that a high proportion of VKA-associated bleeding events in hospitalized, at-risk adult patients were preventable or potentially preventable.

Introduction

Drug therapy is inherently associated with the risk of adverse drug reactions (ADRs), which is modulated by several factors. These ADRs have significant economic and clinical costs, as they often lead to emergency department visits, admission to hospital, or the prolongation of hospitalization.[ The estimated proportion of preventable ADR varies considerably (between 1.4% and 90%, depending on the study).[ These disparities may be due to the absence of a uniform method for assessing preventability. Indeed, methods for assessing the preventability of ADRs range from implicit evaluations to explicit algorithms. Likewise, the reliability of the tools used to assess preventability varies greatly and is rarely optimal.[ Due to the specific features of each drug class, the development of class-specific preventability scales may constitute a valuable approach for improving the quality of data in this field.

Vitamin K antagonists (VKAs) and direct oral anticoagulants (DOAs) are used in clinical practice for the prevention and treatment of thromboembolic complications. Given that anticoagulants reduce the blood's ability to clot, unwanted bleeding is an inevitable risk. In a French national survey of a representative sample of medical wards in public hospitals, adverse drug reaction- (ADR-) related hospitalizations were very frequent. Hemorrhage caused by antithrombotic agents (and particularly VKAs) was the main cause of ADR-related hospitalizations.[

In 906 consecutive hospitalized, VKA-treated adult patients with a risk of major bleeding, we recently determined that the main factors associated with a serious bleeding risk were an international normalized ratio (INR) ≥8.5, a history of recent gastrointestinal lesions, a history of recent trauma, and prior noncompliance known to the medical staff.[ In the same line, the HAS-BLED bleeding risk score (an abbreviation of “Hypertension, Abnormal Renal/Liver Function, Stroke, Bleeding History or Predisposition, Labile INR, Elderly, Drugs/Alcohol Concomitantly”) was first described in 2010. It is recommended by the European and Canadian guidelines for estimating the risk of major bleeding. In 2011, the Anticoagulation and Risk Factors in Atrial Fibrillation (ATRIA) study group described a new bleeding risk scheme for AF, which includes 5 weighted risk factors: anemia, severe renal disease, age ≥75 years, previous bleeding, and diagnosed hypertension.[ Although these bleeding scores are designed to estimate the bleeding risk, they provide no information on the preventability of this frequent adverse event once it has occurred. Most of these factors are preventable in as much as they are known or can be measured prior to the administration of antithrombotic agents. Hence, the objective of the present study was to adapt and validate an ADR preventability score for VKA-associated bleeding and evaluate the preventability of bleeding in 906 hospitalized, VKA-treated adult patients with an INR ≥5.

Patients and methods

The present study was based on a post hoc analysis of a 2-year prospective study performed in Amiens University Hospital (Amiens, France).[ The latter study was designed to identify all VKA-treated adults presenting with an INR ≥ 5 at admission and to detect the most relevant risk factors for bleeding. All patients gave their written, informed consent. The study was approved by the local independent ethics committee (Comité de Protection des Personnes Nord Ouest II, Amiens, France) and performed in accordance with the ethical principles of the Declaration of Helsinki.

Study population

We included all consecutive VKA-treated adults with a major bleeding risk (defined as an INR≥ 5 on admission) admitted to Amiens University Hospital between January 1, 2006, and December 31, 2007. Bleeding status was evaluated for each patient at the time of inclusion.

Data collection

The patients were selected prospectively on the basis of the INR measured by the hematology laboratory at Amiens University Hospital. Patients with INR ≥ 5 were included in the study if they had also been treated with VKAs prior to or during hospitalization. Each patient could be included only once.

For each patient, the characteristics of bleeding events having occurred during hospitalization (type, site, date of onset, severity, treatment, and outcome) were recorded. Furthermore, the following characteristics were recorded prospectively by questioning the physician, the medical staff, and the patient and consulting the patient's hospital records:

Demographic characteristics (age and gender) and medical history, including treated hypertension, diabetes, hypercholesterolemia, cancer, gastrointestinal lesions in the preceding 3 months, chronic kidney disease, alcoholism, surgery in the preceding 3 months, stroke in the preceding 3 months, trauma in the preceding 2 weeks, and infection in the preceding 2 weeks. For alcoholism, only the patient's physician and medical records were consulted.

Characteristics of drug treatments: compound, dose, treatment start date, indication for VKAs, any associated medications, the person administering the treatment, adherence to treatment, regular use of an anticoagulation booklet, patient education (the patient was asked if he/she could remember being given an explanation about treatment with VKAs), recent changes in the VKA dosage and/or regimen, and INR values (the value on admission, the latest of any prior laboratory tests, and the second value evaluated during the current hospital stay). Treatment outside the scope of the current French guidelines was defined as off-label prescription (i.e., outside the indications given in the French summary of product characteristics [SmPC]), inappropriate treatment with regard to a previous INR value (e.g., the absence of VKA dose adaptation for an INR outside the therapeutic range), and (iii) previous noncompliance known to medical staff. Appropriate medical care associated with the following INR was defined as a change in the VKA prescription (a dose decrease or withdrawal).

All data were entered into a computer database (Access 2003, Microsoft, Redmond, WA).

Development of a preventability scale for VKA-associated bleeding

We adapted Olivier et al's[ assessment scale with a view to evaluating the preventability of VKA-associated bleeding. To this end, we had to make the initial algorithm specific for VKAs, overcome the inaccuracies of the initial method, and improve reproducibility.

A specific working group (comprising a cardiologist, a neurologist, a general practitioner, a pharmacist, and a pharmacovigilance specialist) was charged with validating the adapted scale. The group defined an explicit, VKA-specific algorithm with 4 items: Compliance with recommendations for the drug, other bleeding risk factors identified for the patient, suitability of prescription to patient's living conditions and environment and prescription probably unavoidable for the patient.

Successive drafts were tested until a consensus was reached and then adopted for use in the present study (Table 1 ).

Table 1

Evaluation of the preventability of a VKA-associated hemorrhage (adapted from Olivier et al).

Preventability was evaluated for each patient (using Table 1 ) by 2 independent members of the working group. The 2 members worked separately. If all the requisite information was available, it took an average of 3 minutes to perform this evaluation. After Table 1  had been completed for each patient, an overall score was calculated (as with Olivier et al's scale) by summing the 4 component items. Lastly, the overall score was classified as follows: –13 to –8: “preventable”; –7 to –3: “potentially preventable”; –2 to +2: “unevaluable”; +3 to +8: “not preventable.”

Data analysis

Two members of the working group applied the new scale to all patients with 1 or more bleeding ADRs. The scale's reliability was then assessed by calculating 2 measures of inter-rater agreement:

Cohen's kappa (κ) for the qualitative assessment of inter-rater agreement. The nonparametric κ test is used to estimate the magnitude of the real component of agreement between matched, qualitative judgments. By convention, the degree of inter-rater agreement is considered to be “excellent” for κ ≥0.8, “good” for 0.61< κ < 0.8, “moderate” for 0.41< κ <0.6, “poor” for 0.21< κ <0.4 and “bad” or “very bad” for κ ≤ 0.2.[

Spearman's intra-class correlation coefficient for the quantitative assessment of inter-rater agreement (on each item and on the overall score). The closer the coefficient is to 1, the greater the degree of inter-rater agreement.

Lastly, 241 bleeding cases were evaluated and classified according to the new, validated scale as “preventable,” “potentially preventable,” “not preventable,” and “unevaluable.”

Statistical analyses were performed with SPSS software (version 18.0, SPSS Inc., Chicago, IL) for Windows (Microsoft Corp, Redmond, WA). In all tests, the threshold for statistical significance was set to P < 0.05.

Results

Over a 2-year inclusion period, 906 hospitalized patients were included in the study. The patients’ characteristics are summarized in Table 2. The mean ± SD (range) age of the study population was 76.6 ± 12.4 (19–100), with a median of 77. In total, 439 patients (48.5%) were male. Some of patients had comorbidities, such as diabetes (28.6%) and hypertension (58.9%).[ The main indication for treatment with VKAs was atrial fibrillation (Table 3). Overall, 241 patients (26.6%) experienced at least 1 clinically apparent bleeding event and 665 did not. Eighty percent of these 241 patients were classified as having experienced a serious ADR. Table 4 summarized the data on the types of bleeding; the most frequent type was subcutaneous bleeding.

Table 1 (Continued)

Evaluation of the preventability of a VKA-associated hemorrhage (adapted from Olivier et al).

Table 2

Demographic, clinical, and biochemical characteristics of the study population.

Table 3

Characteristics of the vitamin K antagonist treatments.

Presentation of the validated bleeding scale

Table 1  highlights in gray items that differ from those in Olivier et al's original scale.[ Some items were removed (“known adverse reactions” and “recommendations accessible at date of last prescription or last administration”) and a number of new items were introduced.

Each item has been subdivided into various criteria (related to the French Drug Agency's official recommendations): 10 for item A, 9 for item B, 3 for item C, and 2 for item D.

Only the most relevant criteria for VKA were selected. For example, for item A, the respect of interactions “contra-indicated” or “not recommended” rather simple “precautions.” We also removed duplications.

Reliability of the preventability scale

The inter-rater agreement (evaluated qualitatively by κ) ranged from “good” to “excellent” (κ: 0.68–0.92), and the degree of overall agreement ranged from 85.1% to 98.3% (Table 5). The degree of inter-rater agreement was greatest for adverse effects classified as “preventable” or “not preventable.”

Table 4

Distribution of the patients’ bleeding events, by seriousness.

Table 6 shows the correlation coefficients. The value for the overall score was 0.84, and items C and D were associated with the greatest degree of inter-rater agreement.

Table 5

Values of κ for each class of ADR and for serious ADRs.

Preventability of the bleeding events

The 241 patients with bleeding cases were evaluated according to the validated preventability scale. In the event of disagreement, the 2 experts had to re-evaluate the case and reach a consensus. As shown in Table 7, we found that VKA-associated bleeding events were preventable or potentially preventable in 45.2% of the affected patients.

Table 6

Spearman's correlation coefficient for each item and for the overall score.

Discussion

In a large study of 906 hospitalized, high-INR, VKA-treated patients, the incidence of bleeding was 26.6% (21.4%, if only serious bleeding was taken into account). In fact, VKA-induced bleeding is a major important cause of ADR-related hospitalizations.[ The various scores developed with regard to VKA bleeding[ have focused on estimating the bleeding risk. However, the bleeding risk scores do not address the preventability of events that have occurred; this is why we decided to develop and then validate a specific preventability scale for VKA-associated bleeding in a large patient population. The final scale was found to be reliable. We estimated that bleeding was preventable or potentially preventable in 109 of the 241 affected patients (45.2%).

As emphasized by Hakkarainen et al, instruments for assessing the preventability of ADR range from implicit approaches (in which preventability is loosely defined) to explicit algorithms (in which criteria are clearly defined).[ In the first group, the evaluator make a subjective judgment of whether or not the ADR is preventable on the basis of a case summary[ or a confidence scale from 0 to 5 or 6.[ The second group comprises specific criteria for each preventability category[ or an explicit algorithm.[ At present, there is no “gold standard” for evaluating the preventability of ADRs. For the present study, we decided not to use a subjective method of assessment that would be overly dependent on the evaluator's training and experience.[ In fact, a preliminary assessment of our data using a subjective method yielded “very bad” agreement (κ = 0.12).

We identified 2 published French-language, objective methods for estimating the preventability of ADRs in general: one developed by Imbs et al[ and another developed by Olivier et al.[ However, the reliability of the published tests was poor and, thus, required improvement. Indeed, Olivier et al's scale performed very badly (κ = 0.11) in the evaluation of 49 VKA-associated bleeding manifestations.[ In the present study, the reliability of the overall preventability score for VKA-associated bleeding was good (κ = 0.73). Hence, moving from a general scale to a class-specific scale was associated with greater qualitative and quantitative reliability.

Of the 906 hospitalized patients with a high INR, 241 (26.6%) experienced clinically apparent bleeding and 665 did not. About 12% of cases of bleeding were preventable (including 2 deaths caused by bleeding) and 33.2% were potentially preventable (including 8 deaths caused by bleeding). In the subgroup of patients with severe bleeding, the percentage of potentially preventable cases was higher (36%); the 70 cases included 8 deaths caused by bleeding and 8 deaths in which bleeding may have contributed. Overall, almost half of the cases with serious bleeding were preventable or potentially preventable. Our findings agree with literature reports that a significant proportion of VKA-associated bleeding events could have been prevented by more appropriate treatment.[ These results are in agreement with those reported in the Hakkarainen et al meta-analysis, with 52% of the ADRs considered to be preventable among patients admitted to hospital, and 45% among those hospitalized.[ Our results enhance the importance to develop corrective action to prevent ADRs.

The main strength of the present study was its use of a validated ADR preventability scale in a large sample (241 patients with reported bleeding). Furthermore, our scale was focused on the most frequent bleeding ADRs (those linked to VKAs).[ The present score could be very useful for estimating the quality of health care and evaluating the preventability of bleeding events, rather than predicting the occurrence of bleeding events in patients treated with VKAs (a number of predictive scores have already been published).

The study had some limitations. In view of the present study's design, it was not possible to obtain data on each patient's time in therapeutic range; we only obtained the INR measured in the previous month. The present scale is specific for VKAs and should not be used with for other drug classes. However, it is noteworthy that DOAs are associated with a similar bleeding risk; it might be interesting to adapt the present preventability scale for application to DOA-related bleeding. Fluindione is the most frequently prescribed VKA used in France. It will be necessary to confirm the present scale's validity in another country where other VKAs are used. However, the preventability factors are probably the same.

In conclusion, 241 out of 906 hospitalized VKA-treated, at-risk patients (INR > 5) patients (26.6%) experienced at least 1 clinically apparent bleeding event. In total, 194 patients experienced serious bleeding. Of the 241 cases of bleeding ADRs, 45% were preventable or potentially preventable. This proportion was higher still (49%) for the patients with serious bleeding. We have developed a useful, reliable tool for evaluating the preventability of VKA-associated bleeding.

Table 7

Assessment of preventability for the 241 patients with at least 1 bleeding event.