Richard E Gregg1, Ting Yang2, Stephen W Smith3, Saeed Babaeizadeh2. 1. Advanced Algorithm Research Center, Philips Healthcare, Andover, MA, USA. Electronic address: rich.gregg@philips.com. 2. Advanced Algorithm Research Center, Philips Healthcare, Andover, MA, USA. 3. Hennepin County Medical Center, Minneapolis, MN, USA.
Abstract
Many studies that rely on manual ECG interpretation as a reference use multiple ECG expert interpreters and a method to resolve differences between interpreters, reflecting the fact that experts sometimes use different criteria. The aim of this study was to show the effect of manual ECG interpretation style on training automated ECG interpretation. METHODS: The effect of ECG interpretation style or differing ECG criteria on algorithm training was shown in this study by careful analysis of the changes in algorithm performance when the algorithm was trained on one database and tested on a different database. Morphology related ECG interpretation was summarized in eleven abnormalities such as left bundle branch block (LBBB) and old anterior myocardial infarction (MI). Each of the two databases used in the study had a reference interpretation mapped to those eleven abnormalities. F1 algorithm performance scores across abnormalities were compared for four cases. First, the algorithm was trained and tested on randomly split database A and then trained on the training set of database A and tested on randomly chosen test set of database B. The previous two test cases were repeated for opposite databases, train and test on database B and then train on database B and test on the test set of database A. RESULTS: F1 scores across abnormalities were generally higher when training and testing on the same database. F1 scores were high for bundle branch blocks (BBB) no matter the training and testing database combination. Old anterior MI F1 score dropped for one cross-database comparison and not the other suggesting a difference in manual interpretation. CONCLUSION: For some abnormalities, human experts appear to have used different criteria for ECG interpretation, as evident by the difference between cross-database and within-database performance. Bundle branch blocks appear to be interpreted in a consistent manner.
Many studies that rely on manual ECG interpretation as a reference use multiple ECG expert interpreters and a method to resolve differences between interpreters, reflecting the fact that experts sometimes use different criteria. The aim of this study was to show the effect of manual ECG interpretation style on training automated ECG interpretation. METHODS: The effect of ECG interpretation style or differing ECG criteria on algorithm training was shown in this study by careful analysis of the changes in algorithm performance when the algorithm was trained on one database and tested on a different database. Morphology related ECG interpretation was summarized in eleven abnormalities such as left bundle branch block (LBBB) and old anterior myocardial infarction (MI). Each of the two databases used in the study had a reference interpretation mapped to those eleven abnormalities. F1 algorithm performance scores across abnormalities were compared for four cases. First, the algorithm was trained and tested on randomly split database A and then trained on the training set of database A and tested on randomly chosen test set of database B. The previous two test cases were repeated for opposite databases, train and test on database B and then train on database B and test on the test set of database A. RESULTS: F1 scores across abnormalities were generally higher when training and testing on the same database. F1 scores were high for bundle branch blocks (BBB) no matter the training and testing database combination. Old anterior MI F1 score dropped for one cross-database comparison and not the other suggesting a difference in manual interpretation. CONCLUSION: For some abnormalities, human experts appear to have used different criteria for ECG interpretation, as evident by the difference between cross-database and within-database performance. Bundle branch blocks appear to be interpreted in a consistent manner.