Quality evaluation of case series describing four-factor prothrombin complex concentrate in oral factor Xa inhibitor-associated bleeding: a systematic review.

INTRODUCTION: As oral factor Xa (oFXa) inhibitor use has increased, so has publication of case series describing related bleeding managed with four-factor prothrombin complex concentrate (4F-PCC).
OBJECTIVE: This review aimed to identify case series describing 4F-PCC management of oFXa inhibitor-related bleeding and appraise their methodological and reporting quality.
DESIGN: We searched Medline and EMBASE (1 January 2011 to 31 May 2020) to identify series of ≥10 patients with oFXa inhibitor-related major bleeding given off-label 4F-PCC. Case series were evaluated using a validated tool adapted for this topic. The tool addressed patient selection, bleed/outcome ascertainment, causal/temporal association and reporting.
RESULTS: We identified 14 case series. None had ≥100 patients (range=13-84), three were prospective, two detailed appropriate inclusion criteria and four noted consecutive inclusion. While 12 series provided clear/appropriate methods for diagnosis of intracranial haemorrhage (ICH); none did so for extracranial bleeds and it was not clear whether bleeding was adjudicated in any. Haemostatic effectiveness, thrombosis and mortality were together evaluated in 12 series, but only seven used validated methods to evaluate/diagnosis haemostasis in ICH, six in gastrointestinal bleeds, five in other bleeds and three in thrombosis. Independent adjudication of haemostasis (n=1) and thrombosis (n=2) was infrequent. Thirty-day follow-up for mortality and thrombosis was noted in five and seven series. Anticoagulation measurement/levels in at least some patients were conveyed in three series. Few series provided data on anticoagulant agent/dose (n=4), time from anticoagulant (n=4), time-to-reversal (n=7), baseline (n=7) or change (n=0) in neurologic function.
CONCLUSIONS: Although many case series describe off-label use of 4F-PCC for oFXa inhibitor-related bleeding, methodological flaws and/or poor reporting necessitates caution in interpretation. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study compiles all available literature meeting inclusion criteria regarding the off-label use of four-factor prothrombin complex concentrate to manage oral factor Xa related major bleeding.

This study brings attention to the methodology and reporting flaws of this literature which gives perspective when considering effectiveness and safety.

The disease-specific tool used in this study is derived from a previously validated tool, however, our disease-specific tool has not been peer reviewed.

Introduction

Randomised controlled trials have demonstrated oral factor Xa (oFXa) inhibitors to be at least non-inferior to warfarin for preventing stroke and systemic embolism in patients with non-valvular atrial fibrillation (NVAF)1–3 and reducing recurrent thrombosis in patients with venous thromboembolism (VTE).4–6 Moreover, data suggest that oFXa inhibitors have a similar or reduced risk of overall major bleeding compared with warfarin, with a reduction in fatal bleeding including intracranial haemorrhage (ICH).1–6 Consequently, the proportion of NVAF and acute VTE patients treated with oFXa inhibitors has increased in lieu of warfarin.7 8

Despite the short duration of pharmacological action (anticoagulation effect) of oFXa inhibitors (apixaban, edoxaban and rivaroxaban), reversal agents are often needed to manage patients with severe or life-threatening bleeds.9 10 In May 2018, the US Food and Drug Administration approved coagulation factor Xa (recombinant), inactivated –zhzo (USAN: andexanet alfa), the first specific reversal agent to manage oFXa inhibitor-related bleeding.11 Shortly after, in April 2019 the European Medicines Agency also approved andexanet alfa for this indication.12 Prior to regulatory approval of andexanet alfa, various non-specific reversal agents were supported by guidelines13–15 as an off-label approach to manage oFXa inhibitor-related severe or life-threatening bleeds, most notably, four-factor prothrombin complex concentrate (4F-PCC). Evidence, primarily in the form of small case series, has suggested that 4F-PCC is safe and efficacious in the management of oFXa inhibitor bleeding, but variation in reporting, sample size, bleed definition and severity, haemostasis endpoint definitions and hospital practices, including various types and doses of 4F-PCC, make it difficult to assess their generalisability. While all case series have innate limitations, there may still be substantial variation in their clinical usefulness based on the quality of methods used and extent of reporting of methods and results. Therefore, we sought to systematically identify existing case series describing 4F-PCC use for the reversal of oFXa inhibitor-related bleeding and to evaluate their methodological and reporting quality.

Methods

Preparation of this report was in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.16

Search strategy

We performed a bibliographic literature search of Medline and EMBASE from 1 January 2011 (year of first oFXa inhibitor availability) through 31 May 2020. Our search strategy is available in online supplemental appendix 1. Bibliographic searches were augmented with backwards citation tracking and review of conference proceedings of major cardiology, neurology and thrombosis and haemostasis meetings over the past 2 years (the latter were searched to identify case series available only in abstract form for inclusion into a pre-specified sensitivity analysis only).

Study selection

Two investigators screened citations and assessed eligible reports for inclusion with disagreements reconciled through discussion or by a third investigator. To be included in this review, case series had to describe the use of 4F-PCC in ≥10 patients for management of major, severe or life-threatening bleeding while taking an oFXa inhibitor. Reports describing the use of andexanet alfa, three-factor PCC, activated PCC, unspecified PCC or recombinant factor VIIa as the primary reversal agent were excluded; as were those assessing the reversal of dabigatran or warfarin, reversal of non-bleeding surgical patients, non-major bleeds or healthy volunteers.

Data abstraction

Two investigators independently extracted all data with disagreements resolved by discussion or a third investigator. The following data were sought from each study: first author’s last name; year of publication; journal and its impact factor; specific inclusion and exclusion criteria; enrollment timeframe; number of patients included and outcomes reported on; renal function at presentation; location of bleed; method of diagnosis/ascertainment of bleeding and any thrombotic events; measurement of neurologic function; anticoagulant characteristics (agent, dose, indication, time last taken, drug concentration level, anti-factor Xa activity level); reversal agent information (agent, dose, time to administration); concomitant methods of achieving haemostasis used (surgeries or procedures, transfusions, additional reversal agents or medications); reporting of haemostatic effectiveness, thrombotic events and mortality; definition of haemostatic effectiveness applied; adjudication of bleeding events, haemostatic effectiveness and/or thrombotic events; duration of follow-up for haemostatic effectiveness, change in neurologic status, thrombotic events and mortality; and description of treatment site(s) (ie, geographic region/country, comprehensive stroke centre, level one trauma centre).

Methodological and reporting quality assessment

We performed critical appraisal of the methodological and reporting quality of each included case series. We modified a tool originally developed by Murad et al17 for use in our disease/indication-specific literature review. Our tool uses exploratory questions/items to assess a case series’ methodological and reporting quality in respect to its selection, exposure and outcome (ie, alternative causes, dose–response and sufficient duration of follow-up) and whether cases were reported with sufficient detail to allow for generalisability to patients in other practices. We included questions evaluating the domains of selection (n=5 items), ascertainment (n=12 items), causal and temporal association (n=6 items) and reporting (n=15 items). Items for the selection, ascertainment, causal and temporal association domains were answered/assessed as ‘yes’, ‘no’, ‘unclear’ (or ‘not applicable’). Items for reporting were assessed as ‘yes’ or ‘no’. The specific criteria used to assess each item are provided in online supplemental appendix 2. Evaluation of methodological and reporting quality was performed by two investigators with all disagreements resolved by discussion or a third investigator.

Descriptive statistics were used to summarise assessment of each item, with the proportion of case series assessed as ‘yes’ (+), ‘no’ (−) and ‘unclear’ (?) divided by the number of applicable case series (excluded studies deemed not applicable). Continuous data (eg, journal impact factor and sample size) were reported as medians with 25%, 75% ranges.

Case series available as abstracts only would likely accentuate/inflate the number of ‘unclear’ or ‘no’ designations due to their limited word count and the lack of detailed peer review; therefore, abstracts were not included in our primary analysis. We did perform sensitivity analysis whereby both full-text and abstract-only case series were included.

Patient and public involvement

No patient involvement.

Results

Literature search

The literature search identified 500 non-duplicate citations with four additional citations identified through other sources, resulting in 504 total citations (figure 1). After title and abstract review, 464 citations were excluded, leaving 40 for full-text review. On the full-text review, 14 case series met inclusion criteria for this systematic review without exclusions.18–31 An additional nine case series available as abstracts only were included in the sensitivity analysis only.32–40

Figure 1

Summary of case series search and selection. 3F, three-factor; oFXa, oral factor Xa; PCC, prothrombin complex concentrate.

Characteristics of case series

The impact factor of journals in which case series were published ranged from 0.0420 to 16.562 (median, 2.873) (online supplemental eTable 1). The number of patients in identified case series ranged from 13 to 84 (median, 32) (table 1). Most studies included apixaban (n=13) and/or rivaroxaban (n=13). Atrial fibrillation was the most common indication for anticoagulation across all 14 case series. ICH was included in all case series, with nine series including GI and eight other types of extracranial bleeds.

Table 1

Full-text case series, number of patients, anticoagulant and indication for anticoagulation

Case series	N	Anticoagulant, n (%)			Indication, n (%)			Bleed location, n (%)
		A	Ed	R	AF	DVT/PE	Other	ICH	GI	Other
Barra et al20	11	3 (27)	0 (0)	8 (73)	8 (73)	3 (27)	NR	11 (100)	0 (0)	0 (0)
Korobey et al25	59	40 (68)	0 (0)	19 (32)	49 (83)	16 (27)	NR	59 (100)	0 (0)	0 (0)
Reynolds et al27	31	14 (45)	0 (0)	17 (55)	22 (71)	6 (19)	3 (10)	17 (55)	7 (23)	7 (23)
Arachchillage et al19	80	40 (50)	0 (0)	40 (50)	68 (85)	13 (16)	0 (0)	46 (58)	24 (30)	10 (13)
Dybdahl et al21	35	17 (49)	0 (0)	18 (51)	31 (89)	5 (14)	0 (0)	35 (100)	0 (0)	0 (0)
Frontera et al22	46	31 (67)	0 (0)	15 (33)	44 (96)	3 (7)	NR	35 (76) *	11 (24)	0 (0)
Smith et al31	31	17 (55)	0 (0)	14 (45)	28 (90)	3 (10)	NR	18 (58)	1 (3)	12 (39)
Allison et al18	33	6 (18.	0 (0)	27 (82)	24 (73)	6 (18)	3 (9)	30 (91)	1 (3)	2 (6)
Harrison et al24	14	NR	NR	NR	12 (86)	3 (21)	2 (14)	14 (100)	0 (0)	0 (0)
Schenk et al28	13	0 (0)	0 (0)	13 (100)	NR	NR	NR	10 (77)	1 (8)	2 (15)
Schulman et al29	66	29 (44)	0 (0)	37 (56)	56 (85)	10 (15)	1 (2)	36 (55)	16 (24)	15 (21)
Sheikh-Taha30 2018	29	13 (45)	0 (0)	16 (55)	23 (79)	5 (17)	1 (3)	21 (72)	4 (14)	4 (14)
Majeed et al26	84	39 (46)	0 (0)	45 (54)	67 (80)	21 (25)	21 (25)	59 (70)	13 (16)	12 (14)
Grandhi et al23	18	2 (11)	0 (0)	16 (89)	16 (89)	1 (6)	3 (17)	18 (100)	0 (0)	0 (0)

*Study pooled intracranial haemorrhage and intraspinal bleed.

A, apixaban; AF, atrial fibrillation; DVT, deep vein thromboembolism; Ed, edoxaban; GI, gastrointestinal; ICH, intracranial haemorrhage; NR, not recorded; PE, pulmonary embolism; R, rivaroxaban.

Methodological and reporting quality

Selection

Two of identified case series specified all three key inclusion criteria (specific notation of a major bleed, anticoagulant(s) used and time since last anticoagulant dose) (figures 2 and 3). Eight case series did not provide timing since the last anticoagulant dose and four did not provide data regarding both time since last anticoagulant dose and the specific anticoagulant(s) used (figure 4). Four case series noted they enrolled consecutive patients. Ten case series had no patients lost to follow-up, with the remaining reporting anywhere from 6% to 9.7% of patients lost to follow-up. Three case series described prospective collection of data.

Figure 4

Key inclusion criteria components in full-text case series. Figure expands on the findings of online supplemental figure 2, S1.

Percentage of full-text case series that received a ‘Yes’, ‘no’ or ‘unclear’ for selection quality items. Number of case series with each assessment is labelled within the bar. Percentages are based on case series in which the item’s assessment was deemed applicable. Refer to online supplemental appendix 2 for specific definitions used to assess quality.

Individual full-text case series assessment of selection, ascertainment, casual and temporal association items. Refer to online supplemental appendix 2 for specific definitions used to assess quality. GI, gastrointestinal; ICH, intracranial haemorrhage; NA, not applicable.

Ascertainment of qualifying bleeding event

The methods used for ascertainment of ICH diagnosis were specified and deemed appropriate in 12 case series, though the diagnosis of gastrointestinal (n=9) or other extracranial bleeds (n=8) were not described in any case series (figure 5). Further, no case series noted the use of an independent committee or process for adjudication of the diagnosis of the qualifying bleed.

Figure 5

Percentage of full-text case series that received a ‘Yes’, ‘no’ or ‘unclear’ for bleeding event ascertainment items. Number of case series with each assessment is labelled within the bar. Percentages are based on case series in which the item’s assessment was deemed applicable. Refer to online supplemental appendix 2 for specific definitions used to assess quality. GI, gastrointestinal; ICH, intracranial haemorrhage.

Ascertainment of outcomes

Twelve case series assessed each of the three pre-specified key outcomes including haemostatic effectiveness, mortality and thrombosis (figure 6). Of those that assessed haemostatic effectiveness, five (other bleeds) to seven (ICH) reported the use of a validated set of diagnostic criteria (ie, those of the International Society on Thrombosis and Haemostasis or previous used in trials by Sarode and colleagues).41 42 Three case series described and reported thrombotic events using an accepted clinical definition/diagnostic criteria. Neurologic function was ascertained using a validated tool four case series involving ICHs. For haemostatic effectiveness adjudication, one case series described using an independent party (and one explicitly stated not adjudicating events). Two case series explicitly noted they adjudicated thrombotic events, while the remainder did not make their methodology clear.

Figure 6

Percentage of full-text case series that received a ‘Yes’, ‘no’ or ‘unclear’ for outcomes ascertainment items. Number of case series with each assessment is labelled within the bar. Percentages are based on case series in which the item’s assessment was deemed applicable. Refer to online supplemental appendix 2 for specific definitions used to assess quality. GI, gastrointestinal; ICH, intracranial haemorrhage.

Causal and temporal associations

The duration of follow-up for haemostatic effectiveness was defined as between 3 and 24 hours for ICH and 36–60 hours for extracranial bleeds in eight case series (figure 7). Follow-up was ≥30 days for mortality and thrombotic events in five and seven case series, respectively; ≤30 days in six and seven case series, respectively. For neurologic changes, follow-up duration was within 12–36 hours in three series and unclear in the remainder. Seven case series clearly stated that no other reversal agent(s) were used prior to the 4F-PCC. Anticoagulant levels or anti-factor Xa activity levels were measured in three case series (all using a calibrated machine), not measured in two case series and unclear in the remaining nine.

Figure 7

Percentage of full-text case series that received a ‘Yes’, ‘no’ or ‘unclear’ for causal and temporal association items. Number of studies with each assessment is labelled within bar. Note that ‘not applicable’ designations are not incorporated. Refer to online supplemental appendix 2 for specific definitions used to assess quality.

Reporting of characteristics at presentation

A summary of reporting of characteristics at presentation across all case series is depicted in figures 8 and 9. Four case series provided both the anticoagulant used and the dose. All but one case series provided information regarding the reversal agent and dose. Time since last anticoagulant dose to presentation and time to administering the reversal agent from diagnosis was reported in four and seven case series, respectively. Use of concomitant antiplatelets and renal function at presentation was reported in thirteen and nine case series. Neurologic function at presentation was reported in seven case series. A description (ie, comprehensive stroke centre, level I trauma centre and so on) and geographical region of the investigation site was reported in seven case series.

Percentage of full-text case series that received a ‘Yes’ or ‘no’ for reporting of characteristics at presentation items. Number of studies with each assessment is labelled within bar. Refer to online supplemental appendix 2 for specific definitions used to assess quality.

Individual full-text case series assessment for reporting items. Refer to online supplemental appendix 2 for specific definitions used to assess quality.

Reporting of outcomes

The reporting of outcomes across all case series is depicted in figure 10. Most case series provided data on haemostatic effectiveness (n=13), thromboembolic events (n=14) and mortality (n=13). Other measures to manage bleeds including surgeries and/or procedures, transfusions and other haemostatic medications were reported in nine, eleven and nine of case series, respectively. Change in neurologic function was not reported as an outcome in any case series.

Figure 10

Percentage of full-text case series that received a ‘Yes’ or ‘no’ for reporting of outcomes. Number of studies with each assessment is labelled within bar. Refer to online supplemental appendix 2 for specific definitions used to assess quality.

Sensitivity analysis

The addition of abstracts to full-text series resulted in a decreased median sample size of 31 (online supplemental eTable 2). No case series available as an abstract only adequately reported inclusion criteria (online supplemental eFigure 1a, eFigure 2a), detailed how thrombotic events were ascertained (online supplemental e Figure 1b) or reported on anticoagulant agent and dose, time since last anticoagulant dose to arrival and renal function at presentation (online supplemental eFigure 1c, eFigure 2b). The remainder of assessed quality items were generally similar between the sensitivity and primary analyses (online supplemental eFigure 1d, eFigure 1e, eFigure 1f).

Discussion

Our systematic review identified 14 modestly sized full-text case series published in journals of varying impact factor (and an additional nine abstracts presented at international/national conferences). Using an adapted version of a tool17 specifically designed to assesses methodological and reporting quality of case series, we identified the presence of several common methodological flaws and reporting deficiencies that limit these case series’ internal and external validity and consequently necessitate clinicians/readers to use caution when interpreting their results.

One key methodological concern noted in the identified case series were unclear definitions, and lack of adjudication of, the index bleed (especially extracranial), haemostatic effectiveness and thrombosis. Despite accepted definitions of haemostasis that have been endorsed by the International Society of Thrombosis and Haemostasis or previously used in clinical trials,41 42 valid ascertainment of haemostatic effectiveness was only performed in 54% of case series including ICH, 74% including GI bleeds and 63% of other bleeds. Frequently, investigators relied on clinical judgement to assess haemostatic effectiveness. Similarly, only three case series clearly described and used the requirement for a validated measure (ie, ultrasound) to objectively confirm and report the diagnosis of a thrombotic event.20 27 28 43 Less than one-quarter of case series performed (independent or secondary) adjudication of outcomes.44 More frequent use of a prospective study design (only 21% of identified case series reported being prospective) would allow for many of these concerns to be addressed.

Another common methodological flaw was case series’ failure to impose and/or describe a maximum time since last anticoagulation dose (part of inclusion in 14%, reported in 29%) and/or the need for sufficiently elevated anticoagulation activity/levels for inclusion (measured in 21%). Guidelines state that a reversal agent should only be considered when a patient is expected to have clinically relevant levels of anticoagulant.13 Given the relatively short half-life (8–15 hours for apixaban; 7–13 hours for rivaroxaban) and duration of pharmacological activity seen with oFXa inhibitors, it is estimated that <25% of the drug would be present 14 hours after the last dose and <10% after 24 hours in most patients.45 46 Inclusion of patients presenting with bleeds more than a day after the last dose or without verification of anticoagulation activity in case series could result in an overestimation of 4F-PCCs effectiveness.

Identified case series often failed to follow patients for sufficient duration of time to assess important outcomes including mortality (which can be seen as early as 48–72 hours after presentation in 20% of patients with ICH, but up to 40% by 30 days47) and thrombosis (which occurs in up to 15% of 4F-PCC users at 30 days).28 Moreover, the factor II in 4F-PCC has a half-life of ~60 hours48 and requires ~12 days to fully clear from the body post-infusion.46 Only 36% and 50% of case series follow patients for ≥30 days for mortality and thrombotic events, respectively. Due to the short duration of follow-up used in these case series, the risk of mortality and thrombotic events could have been underestimated.

Insufficient reporting was also present in identified case series. Few of the included case series provided detailed data on anticoagulant agents used, dosage, time from last anticoagulant administration, time from presentation for bleeding to 4F-PCC administration or baseline neurologic function (in ICH patients). The dose of 4F-PCC was reported in the majority of case series; however, the dosage was inconsistent between studies ranging from 25 to 50 U/kg. Beyond the methodological concerns noted above, incomplete or lack of reporting of such detail makes it more difficult for clinicians to understand how these case series apply to their patients (generalisability) and how they might change their clinical practice.

Many of the case series limitations discussed above are known challenges when performing a study with this design.17 49 While case series are often mistakenly interpreted as reporting on treatment efficacy, that is not their objective. Rather, case series are typically descriptive and intended to be hypothesis generating only. Even conscientious investigators are limited by the data available to them (contained within their electronic health record), particularly when data is collected retrospectively. The flaws discussed previously and the inherent limitations of case series may explain much of the substantial variance in haemostatic effectiveness (ranging from 60%20 to 94%23) reported with 4F-PCC in identified series,18–40 and further underscores the importance of reporting quality metrics for case series when evaluating medical literature.

Based primarily on case series such as those identified in our review (as well as clinical opinion), guidelines and position statements have been published detailing the role of 4F-PCC as a reversal agent in the management of oFXa inhibitor-related bleeding.13–15 European Stroke Organisation recommends andexanet alfa first line and with second line option of 4F-PCC use if andexanet alfa not available for managing oFXa inhibitor-related ICH, but the strength of evidence supporting this recommendation is graded as ‘very low’.13 Updates to AHA/ACC/HRS atrial fibrillation guidelines also provide guidance on oFXa inhibitor reversal, making a class IIa/B (moderate) recommendation for andexanet alfa use in life-threatening bleeding, without mentioning 4F-PCC.50 Position statements from both the North American Anticoagulation Forum and the Emergency Medicine Cardiac Research and Education Group recommend 4F-PCC use as an alternative to andexanet alfa when it is unavailable (no strengths of recommendation provided).14 15 Although these recommendations may mention the use of 4F-PCC in oFXa inhibitor-related bleeding, clinicians should understand the strength of these recommendations is low based on the poor quality of evidence available.

We believe the tool we adapted for use in this systematic review provides a comprehensive framework that clinicians and other peer-reviewers can use to aid when critically appraising and developing case series of reversal agents (eg, 4F-PCC) for oFXa inhibitor-associated bleeding. This tool may be especially useful in the absence of study designs with greater internal validity in order to evaluate the relative quality among case series. It is important to note, however, that our tool has some limitations. Although we based our disease-specific tool on a previously validated generic case series assessment,17 ours has not undergone extensive peer evaluation and its reliability/validity is unclear. In its present form, our tool uses 38 items to assess methodological and reporting quality. We acknowledge that the number of items and time needed to appraise a case series may be burdensome to clinicians (and limit its use). Lastly, it is often difficult to assess the true methodological quality of a case series because of incomplete or unclear reporting. ‘Unclear’ designations for items does not imply proper or improper use of methods (ie, a case series may have used valid methods, but simply did not describe it in their report). For the abovementioned reason, case series published as abstracts only were excluded from our base analysis as they are more likely to have incomplete reporting due to strictly imposed word/character limits and the lack of back-and-forth peer-review.

Conclusion

Although many case series describing 4F-PCC for managing oFXa inhibitor-related bleeding have been published, the presence of common methodological flaws and/or poor reporting necessitates caution in interpretation. Any data from these case series, are at best, hypothesis generating for future prospective, controlled studies. Major flaws of case series identified included unclear definitions, and lack of adjudication of, the index bleeding, effectiveness and thrombosis, failure to validly ascertain effectiveness in many cases and overall under-reporting of relevant clinical or methodological information. The tool adapted for this systematic review may be useful to clinicians and peer-reviewers who need to critically appraise case series of reversal agents for oFXa inhibitor-associated bleeding. To best support patients with oFXa inhibitor-related bleeds, it is crucial to assess the safety and efficacy of reversal agents using rigorous frameworks and across larger samples with enhanced generalisability.