Artificial Intelligence for Detection of Cardiovascular-Related Diseases from Wearable Devices: A Systematic Review and Meta-Analysis.

PURPOSE: Several artificial intelligence (AI) models for the detection and prediction of cardiovascular-related diseases, including arrhythmias, diabetes, and sleep apnea, have been reported. This systematic review and meta-analysis aimed to identify AI models developed for or applicable to wearable and mobile devices for diverse cardiovascular-related diseases.
MATERIALS AND METHODS: The searched databases included Medline, Embase, and Cochrane Library. For AI models for atrial fibrillation (AF) detection, a meta-analysis of diagnostic accuracy was performed to summarize sensitivity and specificity.
RESULTS: A total of 102 studies were included in the qualitative review. There were AI models for the detection of arrythmia (n=62), followed by sleep apnea (n=11), peripheral vascular diseases (n=6), diabetes mellitus (n=5), hyper/hypotension (n=5), valvular heart disease (n=4), heart failure (n=3), myocardial infarction and cardiac arrest (n=2), and others (n=4). For quantitative analysis of 26 studies reporting AI models for AF detection, meta-analyzed sensitivity was 94.80% and specificity was 96.96%. Deep neural networks showed superior performance [meta-analyzed area under receiver operating characteristics curve (AUROC) of 0.981] compared to conventional machine learning algorithms (meta-analyzed AUROC of 0.961). However, AI models tested with proprietary dataset (meta-analyzed AUROC of 0.972) or data acquired from wearable devices (meta-analyzed AUROC of 0.977) showed inferior performance than those with public dataset (meta-analyzed AUROC of 0.986) or data from in-hospital devices (meta-analyzed AUROC of 0.983).
CONCLUSION: This review found that AI models for diverse cardiovascular-related diseases are being developed, and that they are gradually developing into a form that is suitable for wearable and mobile devices. © Copyright: Yonsei University College of Medicine 2022.

INTRODUCTION

Digital healthcare is a field that deals with personal health and medical information, devices, systems, and platforms, and encompasses the convergence of comprehensive medical services.1 Although digital healthcare is an umbrella term used for various purposes, in essence, it refers to offering healthcare services for the prevention, diagnosis, treatment, and follow-up management of diseases regardless of time and location by connecting healthcare with information and communication technology.

Traditionally, the field of healthcare has been centered on healthcare institutions and physicians, and the medical information they generated was exclusively handled by healthcare workers within the hospital. However, over time, the focus shifted to the prediction and prevention of chronic diseases, rather than treatment-oriented medicine, with the growing interest in healthcare outside the hospital.2 Against this background, digital healthcare is also developing at a rapid pace due to the remarkable advances in computing power and technologies related to data acquisition and analysis. For instance, the development of various sensors, communication networks, and portable and wearable devices has enabled data acquisition during the day-to-day life of a patient.3 Based on the acquired data as such, the development of algorithms for diagnosing or predicting specific diseases outside the hospital, so-called medical artificial intelligence (AI), has emerged as a major area of research.

Cardiovascular disease is the most common cause of death in the United States,4 and it incurs annual medical costs of up to $363 billion.5 Since most cardiovascular diseases follow a chronic course, lifestyle modifications and the periodic monitoring of diseases are very important even after acute treatment in hospital. Therefore, it is difficult to cope with the burden and death of cardiovascular disease by treatment in hospital alone.

Meanwhile, several AI models for the detection and prediction of arrhythmias, diabetes, and sleep apnea using biosignals, such as those used for electrocardiogram (ECG) or photoplethysmography (PPG), have been reported. A systematic literature review revealed that deep learning models for analyzing ECG showed better performance compared to existing methods, such as the hardware or rule-based algorithms used for the detection or prediction of cardiovascular disease.6 However, it is difficult to apply any of them to a wearable device. A conventional supine 12-lead ECG or arterial pressure waveform, for instance, requires complicated or invasive procedures.

We postulated that a separate review of AI algorithms which could be applied to wearable devices was necessary to evaluate the applicability of AI for cardiovascular-related diseases in digital healthcare settings. This systematic literature review and meta-analysis aimed to identify the AI models developed for or applicable to wearable devices for cardiovascular-related diseases.

MATERIALS AND METHODS

Search strategy

This systematic review and meta-analysis was performed in accordance with the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020.7 The searched databases included Medline, Embase, and Cochrane Library. The purpose of the search was to identify the studies related to medical AI used to detect or predict cardiovascular-related disease, which have been developed or applicable to portable and wearable devices. A wide range of keywords were set by referring to previous related reviews89 to include the AI models for various cardiovascular-related diseases (cardi*, heart*, coronary*, angina*, ventric*, myocard*, pericard*, isch(a)em*, arrhythm*, atrial fibrillat*, tachycardi*, endocardi*, stroke, cerebro*, hypertensi*, blood pressure, hyperlipid*, hyperchole*, hyperlipo*, hypertrigly*, arterio*, arthero*, metabolic, deep learning, neural network, artificial intelligence, smart, wearable, portable, 1-lead, single lead, photoplethysmog*, and PPG). The search was performed on September 5, 2021, by one author (S.L), and all publications searched from January 1, 1970 to the present were searched without any limitation on the publication date.

Study selection

The abstracts and titles of the retrieved studies were first screened by two main reviewers (S.L and Y.C). The criteria for eligible studies were as follows: 1) studies whose main tasks related to the detection or prediction of cardiovascular-related disease; 2) studies that developed AI models by utilizing either deep learning or conventional machine learning algorithms; 3) studies that developed AI models for smartphones or using data acquired from mobile and wearable devices; and 4) studies that used biosignals that are considered relatively easy to obtain outside of healthcare institutions, such as 1-lead or 2-lead ECG and PPG, even if those studies did not directly present the keywords associated with wearable device. The exclusion criteria were as follows: 1) studies that simply measured or estimated cardiovascular parameters (e.g., heart rate and blood pressure), regardless of disease status; 2) studies that used invasive signals (e.g., arterial pressure waveform); 3) studies that reported only the models with biosignals that are difficult to apply to wearable devices, such as conventional supine 12-lead ECG; and 4) studies that did not report the quantitative performance of the model.

All studies whose eligibility could not be determined based on the abstract and title alone were included in the full-text evaluation. Studies written in languages other than English or Korean were excluded due to the language proficiency of the authors. Studies with only abstracts in which details on the study settings or design, the nature of the data used were unknown, and the details of the model were unknown were also excluded. In the case of disagreement between the two main reviewers, eligibility was finally decided through discussion with two additional reviewers (J.S.R and S.Y).

Data extraction and quality assessment of study

We extracted important variables using data extraction sheets from the final selected studies. The data sheets were defined prior to conducting the literature searches and study selection. The items extracted from each study included the publication year, author, target disease, task, data availability (public or proprietary), data source (in-hospital device or wearable device), input data domain (ECG, PPG, combined or others), algorithm type (conventional machine learning or deep neural network), and model performance metrics.

So far, there has been no optimal assessment tool that can evaluate the applicability and risk of bias for AI studies. Therefore, we evaluated the risk of bias of individual studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool.10 However, since the validity of using the scale has not been verified for AI studies, it was not used to determine whether to include or exclude a particular study.

Data synthesis

We intended to synthesize the performance of AI models that could be mounted on wearable devices or mobile devices, and that could be used to predict various cardiovascular-related diseases and present them as summary statistics. However, according to our pilot study, most of the identified models were developed for the detection of atrial fibrillation (AF). Therefore, the quantitative meta-analysis was limited to studies on AF. For those studies, the number of true positives, false positives, true negatives, and false negatives were extracted. If those parameters were not directly presented, they were estimated from the number of subjects (AF and non-AF) and the performance metrics, such as sensitivity and specificity. In order to resolve heterogeneity among studies in the meta-analysis, subgroup analysis was performed according to the algorithm type (conventional machine learning vs. deep neural network), data availability (public vs. proprietary), and data source (in-hospital device vs. wearable device). Only qualitative analysis was performed for studies on diseases other than AF.

Statistical analysis

The number of true positives, false positives, true negatives, and false negatives were used to calculate meta-analyzed sensitivity and specificity. As high heterogeneity between studies was suspected, a random-effects model was used for synthesis. Forest plots for sensitivity and specificity for AF detection was created by using R package meta.11 Hierarchical summary receiver operating characteristics curves and 95% confidence interval (CI) were estimated by the Reitsma bivariate model12 using R package mada.13 All statistical analyses were performed by using R version 4.1.0 (R Statistical Computing).

RESULTS

The PRISMA 2020 flow diagram is shown in Fig. 1. After a full text review, 102 studies in total were included in the qualitative review.1415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115 Table 1 presents the summary statistics for the included studies. Table 2 and Supplementary Table 1 (only online) shows the description and findings of the individual studies. Supplementary Fig. 1 (only online) shows the quality assessment for the included studies using the QUADAS-2 tool. As described above, as the validity of the tool used to evaluate the quality of research for development and validation of AI models has not yet been verified, all individual studies were included in the final analysis regardless of the quality assessment results.

Fig. 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 flow diagram for study selection.

Table 1

Summary of the Included Studies

Characteristics		No. of studies
Target disease
	Arrhythmia	62
	Sleep apnea	11
	Peripheral vascular disease	6
	Diabetes mellitus	5
	Hyper/hypotension	5
	Valvular heart disease	4
	Heart failure	3
	Critical care	2
	Others	4
Algorithm type
	Conventional machine learning alone	34
	Deep learning network (alone or combined)	68
Input domain
	Electrocardiography	70
	Photoplethysmography	17
	Multimodal data	10
	Others	5
Data availability
	Public dataset	59
	Proprietary dataset (alone or combined)	43
Data source
	In-hospital device alone	55
	Wearable device (alone or combined)	47

Table 2

Artificial Intelligence Models for Cardiovascular-Related Diseases Other Than Arrhythmia

Study	Task	Dataset	Method	Results
Yu, et al., 202176	Sleep apnea (Screening for sleep apnea)	Apnea-ECG DB	LSTM	Accuracy, 87.09%; Sensitivity, 77.96%; Specificity, 91.74%; F1 score: 0.8161
Chang, et al., 202077	Sleep apnea (Detection of sleep apnea)	Apnea-ECG DB	CNN	Accuracy, 97.1%; Sensitivity, 95.7%; Specificity, 100%
Iwasaki, et al., 202178	Sleep apnea (Screening of sleep apnea)	Proprietary (1-lead ECG of 24 patients)	LSTM	Sensitivity, 100%; Specificity, 100%
Papini, et al., 202079	Sleep apnea (Estimation of the apnea-hypopnea index)	SOMNIA and HealthBed DB	CNN	AUC, 0.80; Accuracy, 85%; Sensitivity, 38%; Specificity, 94%
Wang, et al., 201980	Sleep apnea (Detection of sleep apnea)	Apnea-ECG DB	Time window with a neural network	Per segment: AUC, 0.945; Accuracy, 87.3%; Sensitivity, 85.1%; Specificity, 88.7% Per recording: AUC, 1.000; Accuracy, 97.1%; Sensitivity, 100.0%; Specificity, 91.7%
Bozkurt, et al., 201981	Sleep apnea (Determination of respiratory arrests)	Proprietary DB (2358 PPG)	SVM	Accuracy, 87.36%; Sensitivity, 86%; Specificity, 88%
Wang, et al., 201982	Sleep apnea (Detection of sleep apnea)	Apnea-ECG DB and UCDODB	CNN	AUC, 0.950; Accuracy, 87.6%; Sensitivity, 83.1%; Specificity, 90.3%
Lin, et al., 201883	Sleep apnea (Detection of obstructive sleep apnea)	Apnea-ECG DB	ANN	Accuracy, 79%; Sensitivity, 90%; Specificity, 73%
Urtnasan et al., 201884	Sleep apnea (Detection of obstructive sleep apnea)	Proprietary DB (1-lead ECG of 82 persons)	CNN	Sensitivity, 96%; F1 score: 0.96
Sharma, et al., 201685	Sleep apnea (Detection of sleep apnea)	Apnea-ECG DB	SVM	AUC, 0.978; Accuracy, 97.14%; Sensitivity 95.8%, Specificity, 100%
Babaeizadeh, et al., 201086	Sleep apnea (Detection of sleep apnea)	Apnea-ECG DB	Quadratic classifier	Accuracy, 84.7%; Sensitivity, 76.7%; Specificity, 89.6%
Allen, et al., 202187	Peripheral vascular disease (Detection of peripheral arterial disease)	Proprietary DB (214 PPG)	CNN	Accuracy, 88.9%; Sensitivity, 86.6%; Specificity, 90.2%
Lee, et al., 202088	Peripheral vascular disease (Prediction of ankle brachial index)	MIMIC III	LSTM	Accuracy, 98.34%; Sensitivity, 97.14%; F1-score: 0.9743
Dall’Olio et al., 202089	Peripheral vascular disease (Prediction of vascular aging)	Heart for Heart	CNN	AUC, 0.953
Alty, et al., 200790	Peripheral vascular disease (Prediction of arterial stiffness)	Proprietary DB (461 PPG)	SVM	Accuracy, 86.1%; Sensitivity, 86.7%; Specificity, 85.3%
Allen and Murray 199691	Peripheral vascular disease (Arterial pulse waveform classification	Proprietary DB (366 PPG)	ANN	Accuracy, 80%; Sensitivity, 92%; Specificity, 63%
Allen and Murray 199392	Peripheral vascular disease (Classification of peripeheral vascular disease of the lower limb artieries)	Proprietary DB (150 PPG)	ANN	Accuracy, 90%; Sensitivity, 93%; Specificity, 85%
Baig, et al., 202193	Diabetes mellitus (Early detection of prediabetes and type 2 diabetes mellitus)	Proprietary DB (Demographics, vital signs, activity data, ECG, and others)	Fuzzy inference system	Accuracy, 91%; Sensitivity, 94%; Specificity, 90%
Avram, et al., 202094	Diabetes mellitus (Detection of diabetes)	Proprietary DB (2589448 PPG)	CNN	Primary cohort: AUC, 0.766 (95% CI, 0.750–0.782); Sensitivity, 75%; Specificity, 65%, Contemporary cohort: AUC, 0.740 (95% CI, 0.723–0.758); Sensitivity, 81%; Specificity, 54%
Porumb, et al., 202095	Diabetes mellitus (Detection of nocturnal low glucose)	Proprietary DB (1-lead ECG of 25 persons)	CNN	AUC, 0.907; Accuracy, 92.8%; Sensitivity, 91.6%; Specificity, 89.9%
Porumb, et al., 202096	Diabetes mellitus (Detection of hypoglycemic events)	Proprietary DB (ECG of 4 persons)	CNN and RNN	5-min prediction: Accuracy 87.7%; Sensitivity, 88.3%; Specificity, 88.5% 10-min prediction: Accuracy, 90.0%; Sensitivity, 87.4%; Specificity, 92.2%
Faruqui, et al., 201997	Diabetes mellitus (Forecasting daily glucose levels)	Proprietary DB (Daily monitoring of diet, physical activity, weight, and blood glucose over 6 months of 10 patients)	LSTM	Accuracy of 64.837% for ±10% range of the actual glucose level value
Lee, et al., 202198	Hyper/hypotension (Prediction of intraoperative hypotension)	The VitalDB	CNN	AUC, 0.931 (95% CI, 0.929–0.934); Sensitivity, 85.6% (95% CI, 85.3%–86.0%); Specificity, 85.6% (95% CI, 85.3%–85.9%)
Kwon, et al., 202099	Hyper/hypotension (Detection of pulmonary hypertension)	Proprietary DB (70709 1-lead ECG)	CNN	Internal validation: AUC, 0.859 (95% CI, 0.855–0.863); Accuracy, 76.4% (95% CI, 76.1%–76.8%); Sensitivity, 80.0% (95% CI, 79.6%–80.3%); Specificity, 74.7% (95% CI, 74.4%–75.0%) External validation: AUC, 0.902 (95% CI, 0.900–0.905); Accuracy, 84.0% (95% CI, 83.7%–84.3%); Sensitivity, 80.0% (95% CI, 79.7%–80.2%); Specificity, 84.3% (95% CI, 84.0%–84.6%)
Devaki, et al., 2020100	Hyper/hypotension (Diagnosis of hypertension)	Proprietary (PPG of 140 subjects)	CNN	Accuracy, 83.3%; Sensitivity, 100%; Specificity, 75%
Naifisi, et al., 2018101	Hyper/hypotension (Identification of hypotension-related episodes)	Proprietary DB (781 PPG of 10 patients)	AdaBoost	Accuracy, 94.5%; Sensitivity, 91.7%; Specificity, 95.8%
Liang, et al., 2018102	Hyper/hypotension (Hypertension risk stratification)	MIMIC II and MIMIC III	CNN	F1 score of Normal vs. prehypertension: 0.8052; F1 score of Normal vs. hypertension: 0.9255; F1 score of Normal+prehypertension vs. hypertension: 0.8295
Kwon, et al., 2020103	Valvular heart disease (Detection of mitral regurgitation)	Proprietary DB (70529 1-lead ECG)	CNN	Internal validation: AUC, 0.758 (95% CI, 0.753–0.762); Accuracy, 52.6% (95% CI, 51.2%–53.7%); Sensitivity, 90.0% (95% CI, 89.6%–90.3%); Specificity, 40.8% (95% CI, 39.6%–41.9%) External validation: AUC, 0.850 (95% CI, 0.842–0.857); Accuracy, 57.3% (95% CI, 56.1%–59.2%); Sensitivity, 90.1% (95% CI, 89.5%–90.5%); Specificity, 56.0% (95% CI, 54.9%–57.2%)
Yang, et al., 2020104	Valvular heart disease (Detection of aortic stenosis)	Proprietary DB (Seismocardiogram and gyrocardiogram of 21 patients)	CNN	Accuracy, 95%; Sensitivity, 94%
Yang, et al., 2020105	Valvular heart disease (Detection of aortic stenosis)	Proprietary DB (Seismocardiogram and gyrocardiogram of 21 patients)	Random forest	Accuracy, 98.96%; Sensitivity, 98.33%; Specificity, 99.58%
Kwon, et al., 2020106	Valvular heart disease (Detection of aortic stenosis)	Proprietary DB (56689 1-lead ECG)	CNN	Interval validation: AUC, 0.845 (95% CI, 0.841–0.848) External validation: AUC, 0.821 (95% CI, 0.816–0.825)
Cho, et al., 2021109	Heart failure (Detection of heart failure with reduced ejection fraction)	Proprietary DB (47203 1-lead ECG)	CNN	Internal validation: AUC, 0.874 (95% CI, 0.859–0.890); Accuracy, 67.1% (95% CI, 65.5%–68.6%); Sensitivity, 93.2% (95% CI, 90.9%–95.6%); Specificity, 63.2% (95% CI, 61.5%–65.0%) External validation: AUC, 0.929 (95% CI, 0.911–0.946); Accuracy, 82.5% (95% CI, 81.4%–83.7%); Sensitivity, 92.1% (95% CI, 88.0%–95.8%); Specificity, 82.1% (95% CI, 80.9%–83.2%)
Ahmedov, and Amirjanov, 2021107	Heart failure (Measurement of a cardiac stroke volume)	Proprietary DB (Blood pressure, heart performance measured by ballistocardiographic sensor, skin warming time of 92 persons)	Fuzzy model	Correlation r: 0.803; Mean square error: 8.185
Wang and Zhou, 2019108	Heart failure (Detection of congestive heart failure)	BIDMC-CHF, CHF-RR, MITNSRDB, FD, and NSR-RR	LSTM	Accuracy, 82.51%–99.22%
Rashid and Al Faruque, 2020 110	Critical care (Detection of myocardial infarction)	PTB diagnostic ECG DB	Binarized neural network	Accuracy, 90.29%; Sensitivity, 90.41%; Specificity, 90.16%
Kwon, et al. 2020111	Critical care (Detection of cardiac arrest)	Proprietary DB (47505 1-lead ECG)	CNN	Internal validation: AUC, 0.887 (95% CI, 0.846–0.929); Sensitivity, 85.7% (95% CI, 75.9%–92.6%); Specificity, 78.1% (95% CI, 76.9%–79.4%) External validation: AUC, 0.921 (95% CI, 0.899–0.998); Sensitivity, 82.2% (95% CI, 81.5%–83.0%); Specificity, 82.2% (95% CI, 81.5%–83.0%)
Grogan, et al., 2021112	Others (Detection of cardiac amyloidosis)	Proprietary DB (4995 1-lead ECG)	CNN	AUC, 0.86
Kwon, et al., 2020113	Others (Detection of anemia)	Proprietary DB (70074 1-lead ECG)	CNN	Internal validation: AUC, 0.870 (95% CI, 0.853–0.887); Sensitivity, 87.8% (95% CI, 84.1%–90.8%); Specificity 68.0% (95% CI, 67.0%–69.1%) External validation: AUC 0.841 (95% CI, 0.815–0.866); Sensitivity, 88.7% (95% CI, 83.3%–92.8%); Specificity 65.4% (95% CI, 64.0%–66.8%)
Chiarelli, et al., 2019114	Others (Prediction of cardiovascular age)	Proprietary DB (2400 1-lead ECG + PPG)	CNN	Correlation r, 0.92; Mean square error, 7 years
Fan, et al., 2019115	Others (Prediction of 1-day-forward self-reported wellness)	Proprietary DB (1-lead ECG of 11 persons)	Bidirectional LSTM	Accuracy, 93.21%; Sensitivity, 92.51%; F1 score: 91.98%

ECG, electrocardiography; PPG, photoplethysmography; ANN, artificial neural network; AUC, area under the curve; CNN, convolutional neural network; LSTM, long short-term memory; SVM, support vector machine; RNN, recurrent neural network.

Refer to Supplementary Table 1 (only online) for artificial intelligence models for arrhythmia detection. Refer to Supplementary Table 2 (only online) for dataset abbreviations and description.

Model and dataset characteristics

In 68 studies, deep learning methods, such as neural networks, were used alone or in combination with other machine learning methods (Table 1). Unlike conventional methods that extracted several handcrafted features from ECG or PPG and used them for training and validation of algorithms for disease prediction, methods that used deep learning tended to utilize the signal itself as an input (Fig. 2). For input domain, 1-lead and 2-lead ECG were the most commonly used biosignals, followed by PPG. They were often used as a single input source, but some studies used multimodal data.232898

Fig. 2

Schematic illustration for wearable device-based artificial intelligence for cardiovascular-related diseases. ECG, electrocardiography; PPG, photoplethysmography; CNN, convolutional neural network; RNN, recurrent neural network; LSTM, long short-term memory.

A total of 59 studies used an open dataset, most of which were related to the detection of arrhythmias or sleep apnea. Supplementary Table 2 (only online) summarizes the representative public datasets containing 1-lead or 2-lead ECG or PPG data.116117118119120121122123124125126127128129130131132133134135136137 However, the models for diabetes and other diseases mostly used proprietary datasets that were newly collected or constructed for the study. There were 47 studies that utilized data collected through wearable devices, such as smart watches.

Arrhythmia

The studies related to arrhythmias accounted for the largest proportion, at 62 studies1415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475 (Supplementary Table 1, only online). Most studies had AF detection as the main task. The arrhythmias other than AF, such as premature ventricular contraction, were often grouped and treated as “non-AF” or “other rhythms.” A large number of models have been reported based on the rich public dataset, which allowed performance comparisons between the studies. Although many studies have utilized public datasets, such as the MIT-BIH database alone, there were also studies that performed external validation by constructing a proprietary dataset with multi-institutional data.17232627

In total, 26 studies included AF as a detection target, and thus were included in the quantitative meta-analysis. The diagnostic accuracy of AF detection models are presented in Figs. 3 and 4A. The results overall show very high performance with a meta-analyzed sensitivity of 94.80% (95% CI, 91.94%–96.68%) and specificity of 96.96% (95% CI, 94.99%–98.17%). In the subgroup analysis performed to resolve the heterogeneity between studies, however, there was a significant difference in the model performance between studies (Fig. 4B–D). The models developed with deep neural network alone or combined with other algorithms (AUROC of 0.981) showed a superior performance compared to the models developed with conventional machine learning alone (AUROC of 0.961). In contrast, the models tested on the proprietary data (AUROC of 0.972) showed an inferior performance compared to the models tested on the public data (AUROC of 0.986). In addition, the model tested on the data acquired from wearable devices (AUROC of 0.977) showed an inferior performance than the models tested on the data acquired from in-hospital devices (AUROC of 0.983).

Fig. 3

Meta-analyzed sensitivity and specificity of artificial intelligence for atrial fibrillation detection.

Fig. 4

Hierarchical summary of receiver operating characteristics curves of artificial intelligence for atrial fibrillation detection. (A) All studies. (B) Studies with conventional machine learning vs. studies with deep neural networks. (C) Studies tested with public dataset vs. studies tested with proprietary dataset. (D) Studies tested with data acquired from in-hospital devices vs. studies tested with data acquired from wearable devices. HSROC, hierarchical summary receiver operating characteristics.

With the advent of deep neural networks, the competition for predictive performance has virtually reached saturation, and the development of a lightweight model that can be mounted on a wearable device and operated in real time has recently become a major topic of research.1827 There have been studies to determine a critical point that does not compromise the diagnostic accuracy while reducing the data size through compression of the ECG signal itself, in addition to reducing the weight of the model.323843

Meanwhile, research using the ECG data collected through a ring-type or patch-type device, rather than data collected in a hospital, has been reported.4044 These studies have also shown satisfactory performance in general, but reported slightly lower performance for detecting arrhythmias compared to the models trained with resting ECG-based data obtained in the hospital. One study, in particular, reported difficulty for an algorithm trained on resting ECG data to show proper prediction performance when applied to ambulatory ECG data.64 In addition, a model that could detect AF in a large number of patients using a video of the facial region rather than ECG monitoring was reported, confirming that diverse types of input data could be utilized.36

Other conditions

There were few studies on the screening and detection of diseases such as sleep apnea,7677787980818283848586 peripheral vascular diseases,878889909192 diabetes,9394959697 hyper/hypotensive disease,9899100101102 valvular disease,103104105106 heart failure,107108109 myocardial infarction,110 cardiac arrest,111 and other conditions, including cardiac amyloidosis and anemia (Table 2).112113114115

The most common study following arrhythmia was the detection of sleep apnea. Sleep apnea itself is not a cardiovascular disease, but obstructive sleep apnea is a major detection target for prediction models for cardiovascular-related disease that use wearable devices, as it is associated with or increases the risk of major cardiovascular diseases. The public dataset PhysioNet Apnea-ECG116 has been used in numerous studies. There have been several studies that used raw ECG with deep learning algorithms, but machine learning models using handcrafted features, such as heart rate variability, have also been reported. In addition to ECG, there have also been studies that use PPG, many of which were studies that used proprietary datasets. There was a study to predict the apnea-associated index using wrist-worn reflective PPG obtained directly using a wearable device.79 This particular study investigated the changes in model performance due to limbic movement, and reported that false positive detection had an adverse effect on the specificity and positive predictive value of the model.

These studies mainly used proprietary datasets, as there were very few public datasets available. As expected, they showed very heterogeneous performance, depending on the task. For instance, the models showed very high predictive power for valvular diseases, such as aortic stenosis, which is known to accompany some changes in the ECG. However, the models did not show an equally high performance for the detection of diabetes or anemia that were not accompanied by substantial changes in the ECG. In a study that developed an AI model for heart failure, the algorithm was tested on several populations with a varying prevalence of heart failure, confirming a very large difference in the accuracy of the algorithm from 82.51% to 99.22%, depending on the prevalence of heart failure in the study population.108

For diseases other than arrhythmia, it was difficult to compare the performance of models due to the small number of studies for each disease, as well as the use of data from heterogeneous settings. Considering the low availability of public data and the characteristics of studies with individual datasets, external validation was rarely performed.

DISCUSSION

This systematic review and meta-analysis summarized the AI models that were developed for or that are potentially useful for wearable devices, and that could be used to detect or predict cardiovascular-related diseases. In addition to the models for arrhythmia detection, including AF, studies have reported on models to predict various cardiovascular-related diseases such as sleep apnea, diabetes, valvular disease, and anemia.

Fig. 2 summarizes the development and application of an AI model for the detection and prediction of cardiovascular-related diseases. The first step is data collection. The most commonly used biosignal for monitoring using a wearable device is the 1-lead or 2-lead ECG signal, followed by PPG. Owing to a large database, such as the MIT-BIH arrhythmia database,122123 various detection models for arrhythmias, including AF, have been reported. As these were hospital-based data, there was a limit to using the data in implementing an algorithm for wearable devices. Fortunately, the availability of biosignals that were collected using mobile devices has increased, as databases such as the PhysioNet Computing in Cardiology Challenge117 have been made public. However, there is a clear need for ambulatory biosignals, since the performance of an algorithm developed from the data of a resting setting is quite poor in an ambulatory setting.64 Recently, in addition to Holter monitoring, which has been traditionally used, the use of patch-type, watch-type, ring-type, and clothing-type monitoring systems have been approved by the FDA.138 Accordingly, the availability of ambulatory biosignals is expected to increase significantly in the future.

One thing to note in relation to the data is the difference in systematic performance between the models that use public datasets, proprietary datasets, data acquired from in-hospital devices, and data acquired from wearable devices. Data characteristics may vary significantly depending on the methods (resting vs. ambulatory) and settings (intensive care unit, inpatient, outpatient, and general population) of data acquisition. For example, the prevalence of cardiovascular diseases varies greatly depending on the population, and the class imbalance between normal data and disease data may have a significant impact on the model performance.108 Therefore, it is very important in model development to use the collected from a population group that has characteristics similar to the target group to which the developed AI model will be applied. In order to develop a more universal model from the perspective of digital healthcare, it is preferable to use data collected from large general populations rather than data collected from hospitals.

The collected data are transformed into a form more suitable for model learning through preprocessing, such as denoising or normalization. For example, the length of the input signal could be changed or converted to an average signal, depending on the type or design of the AI model. Various machine learning methods have been used to predict cardiovascular disease using handmade features such as heart rate variability, R-R interval, and QRS amplitude extracted from ECG or PPG as general input values. However, the popularization of deep neural networks has enabled raw signals themselves to be used as input data. Since biosignal data has a time-series characteristic and is not limited in the length of an input signal, recurrent neural network (RNN) or long short-term memory is logically the most preferred model. However, there have been a considerable number of models that used convolutional neural network (CNN) alone or CNN combined with RNN-based algorithms. In addition to signals, facial imaging videos have been used in a model for detecting arrhythmias.36 Those developed with deep neural networks generally perform better than traditional models, suggesting the possibility that AI can extract features from imaging as well as signal data that can be used for predicting disease states that humans cannot visualize. Although current studies have often used a single biosignal as an analysis domain, it would be possible to develop a multimodal model that utilizes various signals, imaging data, and other clinical information together.

Nevertheless, the most significant problem with deep learning is that it has a black box-like characteristic that makes understanding its operation and judgment principle difficult. Despite the high accuracy of deep learning models, the decisions cannot be accepted by healthcare workers without proper interpretation. While a small number of studies have adopted a strategy that uses salient maps to improve interpretability,98113 such attempts are still difficult to find compared to the models in the field of medical imaging. Extracting interpretable handcrafted features from clinical information or biosignals that are familiar to human experts and using them together in deep learning models could be a feasible alternative to improving the interpretability of AI.

Unlike medical imaging, AI models for cardiovascular-related disease are preferred to be used with wearable and portable devices carried by individuals in their day-to-day lives. It is difficult to operate a large-scale model on portable devices due to hardware limitations. For the analysis of lifelogs collected in real time, the operating speed is also a major consideration. Therefore, it is essential to develop a lightweight and fast model, even if there is a slight loss in performance.1827323843 Second, the algorithm is operated in an environment that is not well-controlled, unlike a hospital environment. As mentioned earlier, a model that is developed using resting biosignals as a source rarely works well for ambulatory biosignals.64 In addition, even resting ECG signals may be more prone to artifacts or noise if they are acquired in daily life. If too many false positive alarms for these abnormal signals occur, the user may prefer to stop using the wearable device. Therefore, proper data preprocessing and thresholding are required to detect a life-threatening condition without tiring the user.

Wearable device-enabled detection of cardiovascular-related diseases are likely to become more common as healthcare technology expands.139 It is estimated that more than 50 million people in the United States use wearable device to record their daily activities.140 These lifelogs could lead to the early detection of diverse cardiovascular-related diseases and potentially life-threatening conditions. However, although patient-activated daily monitoring has the potential benefit to improve the detection of subclinical or occult diseases, there are limitations of their widespread use. For example, poor signal quality and false alarms can lead to inappropriate interpretation, resulting in unnecessary medical referrals and testing.141 It is essential to understand the limitations of consumer-based technologies to avoid improper dependence on the diagnosis and treatment of cardiovascular diseases.142 Even while these devices may help diagnose cardiovascular-related diseases, it remains to be seen how they can be optimally incorporated in current healthcare practices to improve patient outcomes.139

One limitation of this systematic review and meta-analysis lies in the high heterogeneity of the studies included in the analysis. For instance, some studies used ECG data obtained from an intensive care unit, whereas some studies directly utilized data obtained with a smart watch. Although a subgroup analysis was performed to resolve the heterogeneity, nevertheless, inappropriate statistical estimates may have been produced in the process of quantitatively synthesizing studies that were conducted with data that were collected using different modalities from different population groups. In addition, for diseases and tasks other than AF, meta-analysis could not be performed, as there were too few studies to be quantitatively synthesized.

In conclusion, this systematic review and meta-analysis revealed that AI models for the diagnosis and prediction of various cardiovascular-related diseases as well as arrhythmias are being developed, and that they are gradually developing into a form that is suitable for wearable and mobile devices. Numerous studies have demonstrated that the deep learning algorithm shows very high performance compared to the existing analysis methods that use human visualization or the extraction of handmade features for biosignals, such as ECG or PPG signals. However, there must still be sufficient consideration of various aspects, such as the data acquisition process, characteristics of the acquired data, characteristics of the population to which the algorithm is applied, weight reduction of the algorithm, working principle, and interpretability of the model, to develop a practical medical AI model that can be used in the real world.