Ryan Pelletier1, Jeff Nagge2, John-Michael Gamble3. 1. Hospital pharmacist in Parry Sound, Ont. 2. Clinical Associate Professor in the School of Pharmacy at the University of Waterloo in Ontario. 3. Clinical Associate Professor in the School of Pharmacy at the University of Waterloo. jm.gamble@uwaterloo.ca.
Abstract
OBJECTIVE: To assess the variation in bleeding risk estimates and risk stratification among Web and mobile applications for patients with atrial fibrillation. DESIGN: Cross-sectional study. SETTING: Simulated patient population. PARTICIPANTS: Hypothetical patient cohorts that encompassed all possible binary risk factor combinations for each clinical prediction model. INTERVENTIONS: Twenty-five bleeding risk calculators (18 Web and 7 mobile apps), each of which used 1 of 4 clinical prediction models to predict an individual's 12-month bleed risk: ATRIA (Anticoagulation and Risk Factors in Atrial Fibrillation), HAS-BLED (hypertension [systolic blood pressure >160 mm Hg], abnormal renal or liver function, stroke [caused by bleeding], bleeding, labile international normalized ratio, elderly [age >65 years], drugs [acetylsalicylic acid or nonsteroidal anti-inflammatory drugs] or alcohol [≥8 drinks per week]), HEMORR2HAGES (hepatic or renal disease, ethanol abuse, malignancy, older [age >75 years], reduced platelet count or function, rebleeding risk [history of past bleeding], hypertension [uncontrolled], anemia, genetic factors, excessive fall risk, and stroke), and mOBRI (modified Outpatient Bleeding Risk Index). MAIN OUTCOME MEASURES: Four simulated cohorts were constructed. The coefficient of variation, relative difference (RD), and 95% CI for annual bleeding risk estimates were calculated for all hypothetical patient cohorts. Additionally, pairwise agreement between calculators across low- (<10%), moderate- (10% to 20%), and high-risk (>20%) categories of patients was determined. RESULTS: The risk estimates the calculators generated were imprecise, with coefficients of variation ranging from 14% for HEMORR2HAGES to 64% for mOBRI. Wide variation was observed in annual risk estimates for calculators using the mOBRI (maximum RD=4.3) and HAS-BLED (maximum RD=3.1) models. The 95% CI of mean annual bleeding risk varied among models; 1 calculator using the HAS-BLED model had a 95% CI of mean annual risk estimates of 5.4% to 6.2%, while another HAS-BLED calculator reported a 95% CI of 17.7% to 18.5%. Concordance for risk category stratification among calculators was high for those based on mOBRI and ATRIA (=1 for both). Poor agreement was observed in 1 calculator using HEMORR2HAGES (=0.54) and another using HAS-BLED ( range=-0.11 to 0.35). CONCLUSION: Inconsistencies and a lack of precision were observed in annual risk estimates and risk stratification produced by Web and mobile bleeding risk calculators for patients with atrial fibrillation. Clinicians should refer to annual bleeding risks observed in major randomized controlled trials to inform risk estimates communicated to patients.
OBJECTIVE: To assess the variation in bleeding risk estimates and risk stratification among Web and mobile applications for patients with atrial fibrillation. DESIGN: Cross-sectional study. SETTING: Simulated patient population. PARTICIPANTS: Hypothetical patient cohorts that encompassed all possible binary risk factor combinations for each clinical prediction model. INTERVENTIONS: Twenty-five bleeding risk calculators (18 Web and 7 mobile apps), each of which used 1 of 4 clinical prediction models to predict an individual's 12-month bleed risk: ATRIA (Anticoagulation and Risk Factors in Atrial Fibrillation), HAS-BLED (hypertension [systolic blood pressure >160 mm Hg], abnormal renal or liver function, stroke [caused by bleeding], bleeding, labile international normalized ratio, elderly [age >65 years], drugs [acetylsalicylic acid or nonsteroidal anti-inflammatory drugs] or alcohol [≥8 drinks per week]), HEMORR2HAGES (hepatic or renal disease, ethanol abuse, malignancy, older [age >75 years], reduced platelet count or function, rebleeding risk [history of past bleeding], hypertension [uncontrolled], anemia, genetic factors, excessive fall risk, and stroke), and mOBRI (modified Outpatient Bleeding Risk Index). MAIN OUTCOME MEASURES: Four simulated cohorts were constructed. The coefficient of variation, relative difference (RD), and 95% CI for annual bleeding risk estimates were calculated for all hypothetical patient cohorts. Additionally, pairwise agreement between calculators across low- (<10%), moderate- (10% to 20%), and high-risk (>20%) categories of patients was determined. RESULTS: The risk estimates the calculators generated were imprecise, with coefficients of variation ranging from 14% for HEMORR2HAGES to 64% for mOBRI. Wide variation was observed in annual risk estimates for calculators using the mOBRI (maximum RD=4.3) and HAS-BLED (maximum RD=3.1) models. The 95% CI of mean annual bleeding risk varied among models; 1 calculator using the HAS-BLED model had a 95% CI of mean annual risk estimates of 5.4% to 6.2%, while another HAS-BLED calculator reported a 95% CI of 17.7% to 18.5%. Concordance for risk category stratification among calculators was high for those based on mOBRI and ATRIA (=1 for both). Poor agreement was observed in 1 calculator using HEMORR2HAGES (=0.54) and another using HAS-BLED ( range=-0.11 to 0.35). CONCLUSION: Inconsistencies and a lack of precision were observed in annual risk estimates and risk stratification produced by Web and mobile bleeding risk calculators for patients with atrial fibrillation. Clinicians should refer to annual bleeding risks observed in major randomized controlled trials to inform risk estimates communicated to patients.
Authors: Jason G Andrade; Atul Verma; L Brent Mitchell; Ratika Parkash; Kori Leblanc; Clare Atzema; Jeff S Healey; Alan Bell; John Cairns; Stuart Connolly; Jafna Cox; Paul Dorian; David Gladstone; M Sean McMurtry; Girish M Nair; Louise Pilote; Jean-Francois Sarrazin; Mike Sharma; Allan Skanes; Mario Talajic; Teresa Tsang; Subodh Verma; D George Wyse; Stanley Nattel; Laurent Macle Journal: Can J Cardiol Date: 2018-11 Impact factor: 5.223
Authors: Craig T January; L Samuel Wann; Hugh Calkins; Lin Y Chen; Joaquin E Cigarroa; Joseph C Cleveland; Patrick T Ellinor; Michael D Ezekowitz; Michael E Field; Karen L Furie; Paul A Heidenreich; Katherine T Murray; Julie B Shea; Cynthia M Tracy; Clyde W Yancy Journal: Heart Rhythm Date: 2019-01-28 Impact factor: 6.343
Authors: Jacques Donzé; Nicolas Rodondi; Gérard Waeber; Pierre Monney; Jacques Cornuz; Drahomir Aujesky Journal: Am J Med Date: 2012-08-30 Impact factor: 4.965
Authors: Paulus Kirchhof; Stefano Benussi; Dipak Kotecha; Anders Ahlsson; Dan Atar; Barbara Casadei; Manuel Castella; Hans-Christoph Diener; Hein Heidbuchel; Jeroen Hendriks; Gerhard Hindricks; Antonis S Manolis; Jonas Oldgren; Bogdan Alexandru Popescu; Ulrich Schotten; Bart Van Putte; Panagiotis Vardas Journal: Eur Heart J Date: 2016-08-27 Impact factor: 29.983
Authors: John A Kanis; Nicholas C Harvey; Helena Johansson; Anders Odén; Eugene V McCloskey; William D Leslie Journal: J Clin Densitom Date: 2017-07-14 Impact factor: 2.617
Authors: Ethan D Borre; Adam Goode; Giselle Raitz; Bimal Shah; Angela Lowenstern; Ranee Chatterjee; Lauren Sharan; Nancy M Allen LaPointe; Roshini Yapa; J Kelly Davis; Kathryn Lallinger; Robyn Schmidt; Andrzej Kosinski; Sana M Al-Khatib; Gillian D Sanders Journal: Thromb Haemost Date: 2018-10-30 Impact factor: 6.681
Authors: Johanna A A G Damen; Lotty Hooft; Ewoud Schuit; Thomas P A Debray; Gary S Collins; Ioanna Tzoulaki; Camille M Lassale; George C M Siontis; Virginia Chiocchia; Corran Roberts; Michael Maia Schlüssel; Stephen Gerry; James A Black; Pauline Heus; Yvonne T van der Schouw; Linda M Peelen; Karel G M Moons Journal: BMJ Date: 2016-05-16