Juntong Zeng1,2,3, Jianzhun Shao4, Shen Lin1,2,3,5, Hongchang Zhang4, Xiaoting Su1,2,3, Xiaocong Lian4,6, Yan Zhao1,2, Xiangyang Ji4,6, Zhe Zheng1,2,3,5,7. 1. National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, Beijing, People's Republic of China. 2. State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, Beijing, People's Republic of China. 3. Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic of China. 4. Department of Automation, Tsinghua University, Beijing, People's Republic of China. 5. Department of Cardiovascular Surgery, Fuwai Hospital, National Center for Cardiovascular Diseases, Beijing, People's Republic of China. 6. Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, People's Republic of China. 7. National Health Commission Key Laboratory of Cardiovascular Regenerative Medicine, Fuwai Central-China Hospital, Central-China Branch of National Center for Cardiovascular Diseases, Zhengzhou, People's Republic of China.
Abstract
OBJECTIVE: Warfarin anticoagulation management requires sequential decision-making to adjust dosages based on patients' evolving states continuously. We aimed to leverage reinforcement learning (RL) to optimize the dynamic in-hospital warfarin dosing in patients after surgical valve replacement (SVR). MATERIALS AND METHODS: 10 408 SVR cases with warfarin dosage-response data were retrospectively collected to develop and test an RL algorithm that can continuously recommend daily warfarin doses based on patients' evolving multidimensional states. The RL algorithm was compared with clinicians' actual practice and other machine learning and clinical decision rule-based algorithms. The primary outcome was the ratio of patients without in-hospital INRs >3.0 and the INR at discharge within the target range (1.8-2.5) (excellent responders). The secondary outcomes were the safety responder ratio (no INRs >3.0) and the target responder ratio (the discharge INR within 1.8-2.5). RESULTS: In the test set (n = 1260), the excellent responder ratio under clinicians' guidance was significantly lower than the RL algorithm: 41.6% versus 80.8% (relative risk [RR], 0.51; 95% confidence interval [CI], 0.48-0.55), also the safety responder ratio: 83.1% versus 99.5% (RR, 0.83; 95% CI, 0.81-0.86), and the target responder ratio: 49.7% versus 81.1% (RR, 0.61; 95% CI, 0.58-0.65). The RL algorithms performed significantly better than all the other algorithms. Compared with clinicians' actual practice, the RL-optimized INR trajectory reached and maintained within the target range significantly faster and longer. DISCUSSION: RL could offer interactive, practical clinical decision support for sequential decision-making tasks and is potentially adaptable for varied clinical scenarios. Prospective validation is needed. CONCLUSION: An RL algorithm significantly optimized the post-operation warfarin anticoagulation quality compared with clinicians' actual practice, suggesting its potential for challenging sequential decision-making tasks.
OBJECTIVE: Warfarin anticoagulation management requires sequential decision-making to adjust dosages based on patients' evolving states continuously. We aimed to leverage reinforcement learning (RL) to optimize the dynamic in-hospital warfarin dosing in patients after surgical valve replacement (SVR). MATERIALS AND METHODS: 10 408 SVR cases with warfarin dosage-response data were retrospectively collected to develop and test an RL algorithm that can continuously recommend daily warfarin doses based on patients' evolving multidimensional states. The RL algorithm was compared with clinicians' actual practice and other machine learning and clinical decision rule-based algorithms. The primary outcome was the ratio of patients without in-hospital INRs >3.0 and the INR at discharge within the target range (1.8-2.5) (excellent responders). The secondary outcomes were the safety responder ratio (no INRs >3.0) and the target responder ratio (the discharge INR within 1.8-2.5). RESULTS: In the test set (n = 1260), the excellent responder ratio under clinicians' guidance was significantly lower than the RL algorithm: 41.6% versus 80.8% (relative risk [RR], 0.51; 95% confidence interval [CI], 0.48-0.55), also the safety responder ratio: 83.1% versus 99.5% (RR, 0.83; 95% CI, 0.81-0.86), and the target responder ratio: 49.7% versus 81.1% (RR, 0.61; 95% CI, 0.58-0.65). The RL algorithms performed significantly better than all the other algorithms. Compared with clinicians' actual practice, the RL-optimized INR trajectory reached and maintained within the target range significantly faster and longer. DISCUSSION: RL could offer interactive, practical clinical decision support for sequential decision-making tasks and is potentially adaptable for varied clinical scenarios. Prospective validation is needed. CONCLUSION: An RL algorithm significantly optimized the post-operation warfarin anticoagulation quality compared with clinicians' actual practice, suggesting its potential for challenging sequential decision-making tasks.
Authors: A-K Hamberg; M-L Dahl; M Barban; M G Scordo; M Wadelius; V Pengo; R Padrini; E N Jonsson Journal: Clin Pharmacol Ther Date: 2007-02-14 Impact factor: 6.875
Authors: Stephen E Kimmel; Benjamin French; Scott E Kasner; Julie A Johnson; Jeffrey L Anderson; Brian F Gage; Yves D Rosenberg; Charles S Eby; Rosemary A Madigan; Robert B McBane; Sherif Z Abdel-Rahman; Scott M Stevens; Steven Yale; Emile R Mohler; Margaret C Fang; Vinay Shah; Richard B Horenstein; Nita A Limdi; James A S Muldowney; Jaspal Gujral; Patrice Delafontaine; Robert J Desnick; Thomas L Ortel; Henny H Billett; Robert C Pendleton; Nancy L Geller; Jonathan L Halperin; Samuel Z Goldhaber; Michael D Caldwell; Robert M Califf; Jonas H Ellenberg Journal: N Engl J Med Date: 2013-11-19 Impact factor: 91.245
Authors: Jeffrey L Anderson; Benjamin D Horne; Scott M Stevens; Amanda S Grove; Stephanie Barton; Zachery P Nicholas; Samera F S Kahn; Heidi T May; Kent M Samuelson; Joseph B Muhlestein; John F Carlquist Journal: Circulation Date: 2007-11-07 Impact factor: 29.690