| Literature DB >> 35190470 |
Simone Calcagno1, Giuseppe BiondiZoccai2, Tatjana Stankovic3, Erzsebet Szabo4, Aniko Berta Szabo4, Istvan Kecskes5.
Abstract
PURPOSE: In a comparator study, designed with assistance from the Food and Drug Administration, a State-of-the-Art (SOTA) ECG device augmented with automated analysis, the comparator, was compared with a breakthrough technology, Cardio-HART (CHART).Entities:
Keywords: arrhythmias; cardiac; echocardiography; electrocardiography; heart failure; heart valve diseases
Mesh:
Year: 2022 PMID: 35190470 PMCID: PMC8862477 DOI: 10.1136/openhrt-2021-001852
Source DB: PubMed Journal: Open Heart ISSN: 2053-3624
Referral decision tree
| Send/don’t binary decision | Detailed referral decision |
| Not refer — don’t send |
No Action Watch 12 months Watch 6 months Watch 3 months Not sure (no evidence for decision) |
| Refer to cardiology — send |
Routine (60–90 days) Intermediate (15–30 days) Urgent (3–5 days) Not sure (no evidence for decision) |
| Not sure (no evidence for decision) |
2021 Cardio-Phoenix.
Performance of GP referral decision on ECG and CHART reports compared with by consensus ground truth
| Metric | ECG report | CHART report | CHART compare to ECG | ||||
| Lower conf. % | Observed | Upper conf. % | Lower conf. % | Observed | Upper conf. % | ||
| SE% | 42.00 | 48.52 | 55.08 |
|
|
| Significant increase |
| SP% | 73.63 | 78.59 | 83.01 |
|
|
| Significant increase |
| PPV% | 55.74 | 63.19 | 70.20 |
|
|
| Significant increase |
| NPV% | 61.78 | 66.85 | 71.64 |
|
|
| Significant increase |
| K% | 22.52 | 27.90 | 33.71 |
|
|
| Significant increase |
| AUC% | 66.02 | 70.80 | 75.02 |
|
|
| Significant increase |
| LR+ | 1.84 | 2.27 | 2.86 |
|
|
| Significant increase |
| LR− | 0.70 | 0.65 | 0.62 |
|
|
| Significant decrease |
| PR% | 29.17 | 33.09 | 37.20 | 32.69 | 36.73 | 40.91 | Less significant increase |
Better performance values are shown in bold.
Confusion matrices and performance equations available in online supplemental file 1.
AUC, area under the curve; CHART, Cardio-HART; GP, general practitioner; K, kappa statistics; LR+, positive Likelihood Ratio; LR−, Negative Likelihood Ratio; NPV, negative predictive value; PPV, positive predictive value; PR, positive rate; SE, sensitivity; SP, specificity.
Figure 1Performance of GP decision by consensus ground truth. (A) Binary confusion matrix on ECG report, (B) binary confusion matrix on CHART report, (C) performance metrics with confidence intervals, (D) estimated ROC curve based on the detailed referral decision. AUC, area under the curve; CHART, Cardio-HART; GT, ground truth; GP, general practitioner; LR+, positive likelihood ratio; LR−, negative likelihood ratio; NPV, negative per cent agreement; PPV, positive per cent agreement; ROC, receiver operating characteristic.
Diagnostic points used in CUUS at GP and RC diagnosis forms
| Diagnostic points | Description | Supported by | |
| ECG | CHART | ||
| Arrhythmia, blocks | Rhythm problem, | Yes | Yes |
| LV dysfunction | Left Ventricular Systolic and | No | HART LVSD |
| Ischaemia | ST/STT deviation | Yes | Yes |
| Structural | LV Hypertrophy | LVH (Low sensitivity) | HART LVH |
| Valve disease | Aortic Stenosis and Insufficiency | No | HART AS |
| PCG-based murmur | S1 splitting | No | Yes, CHART PCG |
| MCG/PCG based | EMAT, PEP, LVET interval | No | Yes, CHART MCG |
CHART, Cardio-HART; CUUS, Clinical Utility and Usability Study; EMAT, electro-mechanical activation time; GP, general practitioner; IVCT, isovolumic contraction time; IVRT, isovolumic relaxation time; LA, left atrial; LAE, left atrial enlargement; LV, left ventricular; LVDD, LV diastolic dysfunction; LVET, left ventricular ejection time; LVSD, LV systolic dysfunction; MCG, mechano-physiological signal; MPI, myocardial performance index; PCG, phonocardiogram; PEP, pre-ejection period; RA, right atrial; RAE, right atrial enlargement; RC, referral cardiologist; RV, right ventricular; RVH, right ventricular hypertrophy; SPI, systolic performance index; WMA, LV wall motion abnormality.
Figure 2Statistics of the medical findings diagnosis between the tree groups: GP on ECG, GP on chart and RC on chart. CHART, Cardio-HART; GP, general practitioner; RC, referral cardiologist.
Agreement performances calculated between GP and ORC decisions and GP and RC decisions
| Metric | ECG report | CHART report | CHART compare to ECG | |||||
| Lower conf. % | Observed | Upper conf. % | Lower conf. % | Observed | Upper conf. % | |||
| ORC versus | PPA% | 44.8 | 52.0 | 59.1 |
|
|
| Significant increase |
| NPA% | 72.9 | 77.7 | 81.9 |
|
|
| Not significant increase | |
| PPV% | 49.6 | 57.1 | 64.4 |
|
|
| Significant increase | |
| NPV% | 69.0 | 73.8 | 78.3 |
|
|
| Significant increase | |
| K% | 24.5 | 30.2 | 36.2 |
|
|
| Significant increase | |
| PR% | 29.2 | 33.2 | 37.3 | 32.8 | 36.8 | 41.0 | Not significant increase | |
| RC versus | PPA% | 48.0 | 55.7 | 63.2 |
|
|
| Significant increase |
| NPA% | 73.7 | 78.5 | 82.7 |
|
|
| Not significant increase | |
| PPV% | 49.5 | 57.3 | 64.8 |
|
|
| Significant increase | |
| NPV% | 72.5 | 77.3 | 81.6 |
|
|
| Not significant increase | |
| K% | 28.2 | 34.4 | 40.9 |
|
|
| Significan | |
| PR% | 29.2 | 33.2 | 37.5 | 33.1 | 37.3 | 41.6 | Not significant increase | |
Better performance values are shown in bold.
CHART, Cardio-HART; GP, general practitioner; K, kappa statistics; NPA, negative per cent agreement; NPV, negative predictive value; ORC, over-reading cardiologist; PPA, positive per cent agreement; PPV, positive predictive value; PR, positive rate; RC, referral cardiologist.
Figure 3Agreement performance analysis of (A) GP decision versus ORC decision, (B) GP decision versus RC decision. CHART, Cardio-HART; GP, general practitioner; NPA, negative per cent agreement; NPV, negative predictive value; ORC, over-reading cardiologist; PPA, positive per cent agreement; PPV, positive predictive value; PR, positive rate; RC, referral cardiologist.
Interview results for comparison ECG over CHART questions and CHART as a start point for ECG questions
| Interview question for | Answer | ||||
| Yes (Improve) | No (not-improve) | Summary | |||
| ORC | RC | ORC | RC | ||
| When comparing a patient’s CHART#1 report to the ECG-only report, your understanding of the patient’s cardiac status: | 70.4% | 64.4% | 29.6% | 35.6% | CHART improves the understanding of patient cardiac status in 64%–70% and increases the certainty of referral decision in 80%. |
| Did CHART#1 help you to better understand the cardiac status of the patient, compared with ECG-only? | 76.0% | 76.6 | 24.0% | 23.4% | |
| Were you more certain of your decision with CHART#1 then with the ECG-only report? | 79.9% | 79.9% | 20.1% | 20.1% | |
| Would you change any part of your initial ECG-only diagnosis after comparing it to CHART#1? | 20.3% | 18.8% | 79.7% | 81.2% | ORC and RC change their ECG-based referral based on the CHART report 19%–22% |
| Did you change the Send/Don’t decision in CHART#1 from ECG-only? | 22.4% | 19.5% | 77.6% | 80.5% | |
| Did the “Send Priority” change in CHART#1? | 49.1% | 48.7% | 50.9% | 51.3% | In case of consistent ‘Send’ decision from ECG to CHART, CHART change the referral priority in 49%. |
| Did the Treatment Options in CHART#1 change? | 14.7% | 9.3% | 85.3% | 90.7% | In case of consistent ‘Don’t’ decision from ECG to CHART, CHART changed the watch option priority 11%–16%. |
| Did the Watch option in CHART#1 change? | 16.1% | 11.7% | 83.9% | 88.3% | |
CHART, Cardio-HART; ECHO, echocardiography; ORC, over-reading cardiologist; RC, referral cardiologist.