Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory.

The early detection of acute myocardial infarction, which is caused by lifestyle-related risk factors, is essential because it can lead to chronic heart failure or sudden death. Echocardiography, among the most common methods used to detect acute myocardial infarction, is a noninvasive modality for the early diagnosis and assessment of abnormal wall motion. However, depending on disease range and severity, abnormal wall motion may be difficult to distinguish from normal myocardium. As abnormal wall motion can lead to fatal complications, high accuracy is required in its detection over time on echocardiography. This study aimed to develop an automatic detection method for acute myocardial infarction using convolutional neural networks (CNNs) and long short-term memory (LSTM) in echocardiography. The short-axis view (papillary muscle level) of one cardiac cycle and left ventricular long-axis view were input into VGG16, a CNN model, for feature extraction. Thereafter, LSTM was used to classify the cases as normal myocardium or acute myocardial infarction. The overall classification accuracy reached 85.1% for the left ventricular long-axis view and 83.2% for the short-axis view (papillary muscle level). These results suggest the usefulness of the proposed method for the detection of myocardial infarction using echocardiography.

Introduction

Acute myocardial infarction (AMI) is a disease in which myocardial cells become necrotic due to thrombus formation or blood vessel occlusion. AMI causes severe chest pain and requires immediate treatment, such as percutaneous transluminal coronary recanalization or coronary artery bypass grafting. It is important to diagnose AMI as early as possible because it can lead to heart failure, arrhythmia, or sudden death.

Echocardiography, a noninvasive imaging modality used to diagnose AMI, enables the real-time assessment of cardiac function and complications and evaluation of regional abnormal wall motion in patients with AMI. Therefore, it is widely used in cardiology. However, depending on disease range and severity, regional abnormal wall motion can be difficult to recognize. Moreover, the accuracy of its recognition depends on sonographer experience.

Deep learning, an artificial intelligence technique, was recently confirmed to have excellent processing power. Convolutional neural networks (CNNs), which are deep learning models, have been widely applied in various fields such as medical image analysis [1-12]. In the domain of echocardiography, Kusunose et al. proposed a method for detecting regional abnormal wall motion on echocardiography images and obtained a high detection accuracy with an area under the curve of approximately 0.9 [7]. Huang et al. proposed a technique for the visualization of AMI on echocardiography, and their results showed a Dice index of approximately 0.8 [9]. Thus, CNN is a highly accurate technique for AMI detection and classification.

Deep learning has also been widely applied for the multi-classification of video datasets. Recurrent neural networks (RNNs), which are also deep learning models, are particularly effective at predicting and classifying sequential data such as wave signals, natural language, and video [13-16]. RNNs have a recursive structure, meaning that they analyze and produce outputs based on previous time-series and sequential data. Long short-term memory (LSTM), which was recently developed, can be used to analyze even larger and longer-term datasets than RNNs can process [17, 18]. Methods based on LSTM have been widely applied, and Ullah et al. proposed a multi-classification method for video images in large video datasets that obtained a classification accuracy of > 90% [19]. Zhou et al. proposed a natural language classification method based on LSTM and obtained a classification accuracy of >80% [20]. In addition, there has been considerable research done on combining CNNs and RNNs [21].

Kusunose et al. conducted a study on a novel detection method using deep learning techniques for abnormal regional wall motion noted on echocardiography. Multiple deep learning models were utilized to classify echocardiography into routine normal cases and those with abnormal regional wall motion. They adopted the ROC curve as an evaluation method and compared the classification accuracy of the clinical technologists to that of the proposed deep learning model. This comparison resulted in the AUC of the proposed method (0.97) and the AUC of the clinical technologists (0.95) being virtually equivalent, confirming the effectiveness of the proposed method. However, the study had a few limitations. The first limitation was the use of short-axis echocardiogram views only at the level of the papillary muscle. Second, the deep learning input data were limited to only three images per case: end-diastolic, mid-systolic, and end-systolic. Therefore, we attempted to detect AMI by analyzing the wall motion temporal changes and inputting the echocardiograph of one cardiac cycle into deep learning, which can then analyze time series. In addition to the short-axis papillary muscle (PM) level view, we used another view that is easy to use, the left ventricular long axis view, to detect AMI. We believe this method could be integral to the development of detection support technology so that non-specialist physicians in their clinics can accurately detect an AMI. In this study, we focused on CNN and LSTM, which have been widely applied to medical image analysis and can be used for video processing and analysis. Specifically, we aimed to develop an automated detection scheme for AMI using CNN and LSTM in echocardiography.

Materials and methods

Overview of the proposed method

An overview of the proposed method is presented in Fig 1. Echocardiography images were loaded into VGG16 [22], a CNN model, to extract the features. The obtained features were then analyzed using LSTM for the classification of AMI and normal myocardium.

Fig 1

Overview of the proposed method.

Echocardiography

For this study, we collected short-axis PM level and left ventricular LX view images taken with ultrasound equipment (Vivid E9 and Vivid E95, GE Healthcare) at Fujita health university hospital. A total of 202 cines were collected as inputs: 99 diagnoses of acute anteroseptal infarction of the proximal left anterior descending artery in the American Heart Association Committee Report and 103 normal cases. Cardiologists and experienced sonographers usually estimate the culprit coronary artery in patients with myocardial infarction using left ventricular long-axis, left ventricular short-axis, apical four-chamber, apical two-chamber, and apical long-axis views. Here we employed the short-axis PM level and left ventricular long-axis views because it is relatively easy to detect anteroseptal infarction of the proximal left anterior descending artery, which can be used to evaluate abnormal wall motion in cases of anteroseptal infarction. Moreover, all cases of anteroseptal infarction used in this study underwent coronary angiography, and occlusion of #6 was observed by examination. In addition, patients who underwent percutaneous coronary intervention after coronary angiography and had abnormal wall motion on echocardiography were included. Fig 2 shows the views used. Table 1 and Fig 3 show the baseline clinical characteristics of this cohort and the selection of the study population, respectively. P values indicate differences between patients who have normal myocardium and AMI. P < 0.05 was considered statistically significant.

Fig 2

Left ventricular long-axis view and short-axis papillary muscle level view (left: views of anatomy; right: normal views).

Table 1

Baseline clinical characteristics of the study cohort.

Characteristic	Control (n = 103)	AMI (n = 99)	P value
Age, mean (SD)	66.7(12.1)	69.4 (11.6)	0.21
Female sex, no. (%)	38(38.4)	24 (24.2)	0.051
Body mass index (kg/m 2 ), mean (SD)	22.4(4.2)	21.3(7.9)	0.07
Systolic blood pressure (mmHg), mean (SD)	127.3(21.5)	149.9 (29.6)	<0.001
Diastolic blood pressure (mmHg), mean (SD)	75.23(13.9)	88.6 (20.8)	<0.001
Cardiac rhythm	76.7(13.5)	84.1 (19.9)	0.026
Sinus rhythm, mean (SD)		83.1 (21.6)
Atrial fibrillation, mean (SD)		85.1 (20.6)
Site of AMI
Anterior/lateral/inferior (%)		Anterior (100)
Coronary risk factors
Hypertension, no. (%)	4(4.04)	58 (58.6)	<0.001
Diabetes, no. (%)		30 (30.3)
Dyslipidemia, no. (%)		47 (47.5)
Smoking, no. (%)		57 (57.6)
Laboratory data
White blood cell count, mean (SD)	6.7(3.5)	10.1 (7.0)	<0.001
Hemoglobin (g/dL), mean (SD)	10.8(2.5)	13.7 (12.9)	<0.001
Creatinine (mg/dL), mean (SD)	1.2(1.7)	1.2(1.3)	<0.001
eGFR (mL/min/1.73 m 2 ), mean (SD)	78.4(32.1)	82.3 (36.6)	0.41
BNP (pg/mL), mean (SD)	85.8(144)	391.8 (738.6)	0.009
CK-MB (ng/ml), mean (SD)	5.8(5.9)	280.9 (452.1)	<0.001
TnI (ng/ml), mean (SD)	0.2(0.6)	89.3 (226.1)	<0.001
Medications
β-blocker, no. (%)	16(15.1)	63 (63.6)	<0.001
ARB /ACE-I, no. (%)	8(7.8)	65 (65.6)	<0.001

Fig 3

Selection of the study population.

The image preprocessing involved electrocardiogram (ECG) removal, cropping, and frame interpolation of the views. On echocardiography, a two-lead ECG is drawn to identify indicators during the cardiac cycle such as end-diastole and end-systole. Fig 4 shows the preprocessing of the input images. In this study, to recognize the cardiac wall motion on each view, the ECG was removed and trimmed to form a bounding rectangle. To consider differences in heart rate between patients, one cardiac cycle was extracted from each image and the number of frames was interpolated to 30. Interpolation means that if the number of frames in the video image for one cardiac cycle was 50–60, they were interpolated at equal intervals so that the number of frames was 30, whereas if the number of frames was 10–20, they were interpolated so that the total number of echocardiography images for all patients was 30. Linear interpolation was used as the complementation method, and each of the images for one cardiac cycle was extracted based on the length between the peaks (R-R interval) of the simultaneously recorded ECG. This study was approved by an institutional review board of Fujita Health University and informed consents were obtained from patients subject to the condition of data anonymization (No. HM19-345).

Fig 4

Preprocessing of the input data.

All frames of one cardiac cycle taken from the original echocardiography image were interpolated to 30 frames, and the ECG in the image was removed and trimmed.

Feature extraction

The features of the interpolated images were extracted for input into the classification model [23]. CNNs can extract features simultaneously as the final outputs or extract parameters as intermediate outputs from individual layers. Varshni et al. used CNNs to extract features from chest radiographs [24]. Hyeon et al. used CNNs to extract features from cytology images and used conventional machine learning methods to differentiate between benign and malignant cells [24, 25]. By using CNNs as feature extractors and other models as output layers, new inputs can be added and the accuracy further improved compared to CNN use alone. Therefore, we focused on this method and adopted VGG16 and global pooling [26], another CNN model, as feature extractors. Using these feature extraction methods, we extracted features from all frames of the interpolated echocardiography images.

VGG16 model

This study used VGG16, a CNN model developed by the Visual Geometry Group at the University of Oxford in 2014, for the feature extraction. The structure of VGG16 is shown in Fig 5. It consists of 13 convolutional layers, 5 pooling layers, and 3 fully connected layers [27]. We introduced the VGG16 pretraining network using the large natural ImageNet image dataset. From the second fully connected layer of VGG16, 4096 features were extracted and input into the classification model.

Fig 5

VGG16 structure.

Global pooling. Global average pooling (GAP) and global max pooling (GMP) are 2 compression methods for feature maps extracted from images by CNNs. These compression methods select only the maximum or average values from the last feature map of the CNN and pool all of the feature maps. Fig 6 shows a simplified diagram of the GAP and GMP methods. These compression methods were applied to the feature maps after completion of the convolutional and pooling processes until just before the fully connected layer of the CNN. GAP, shown in the upper part of Fig 6, extracts the average value from each feature map and outputs only the extracted value as an intermediate output. GMP, shown in the lower part of Fig 6, extracts the maximum value from each feature map and outputs it as an intermediate output. This process significantly reduces the number of dimensions from the original feature map parameters and prevents overfitting. In this study, we performed global pooling of the 7 × 7 × 512 feature maps extracted by VGG16 and output the average or maximum value from each feature map, which resulted in a total of 512 parameters.

Fig 6

Simplified diagram of the global average pooling and global max pooling compression methods.

Pooling is performed to extract the maximum or average values from the last feature map of the CNN.

LSTM networks

Because the detection of AMI on echocardiography requires the evaluation and analysis of wall motion over time, two-dimensional images with different time phases were input into the CNN. We also focused on RNNs, which are excellent tools for processing sequential data and effective for time-series information such as cine images and wave signals as well as text data and natural language. Therefore, this model is characterized by its ability to control sequential information. Fig 7 demonstrates the principle of RNN, where , , and are the input, output, and weight of the hidden layer, respectively.

Fig 7

Schematic diagram of a recurrent neural network.

In the diagram, is the input, is the hidden layer, and is the output. The RNN learns by passing the weights of the hidden layer to the next hidden layer in the forward direction.

The RNN connects the layer at time (t) with the previous layer (t−1) and calculates the parameters in the hidden layer ((t)) and the output according to the following equations: , , and denote the weights calculated during training, and f(z) and g(z) denote the sigmoid and softmax functions, respectively. The equations for the respective activation functions are as follows:

However, because RNNs theoretically store all past data during training, the vanishing gradient problem arises due to the divergence and disappearance of weights. Therefore, we focused on LSTM, an improved RNN model. The principle of the LSTM is shown in Fig 8. The difference between LSTM and RNN is that LSTM features a mechanism of information selection called “gate” and “cell.” There are 3 types of gates: “input,” “output,” and “forgetting.” Eq (i) shows the formula for the forgetting gate (): From the input information, the output of the LSTM layer at (t−1) and the cell, the information that is unnecessary in the learning at (t), is selected and “forgotten.” Eq (ii) is used to determine the input gate (): The output of the last LSTM layer and the value of the cell are used to determine the new value to be updated. The value of the updated cell () is then determined by Eq (iii): The value of the cell determined by the above equation is propagated to the next LSTM layer, and the output () of the LSTM layer at (t) is determined by Eqs (iv) and (v) in the output gate section: Using this gating mechanism, LSTM can analyze long-term series data and solve the vanishing gradient problem of conventional RNNs. We introduced these mechanisms to analyze the wall motion over time using echocardiography.

Fig 8

Diagram of the long short-term memory principle.

Input, forgetting, and output gates are used to determine the information to be input into, retained, and output from the cells, enabling the learning of sequential data over a long period of time.

Finally, as shown in Fig 9, the features extracted by the VGG16 method were input into the LSTM to classify the normal and AMI cases. For the hyperparameters, we set the learning rate to 1 × 10−5, the number of epochs to 50, the batch size to 30, and the input data size to 4096 × 30.

Fig 9

Classification of myocardial infarction and normal myocardium cases. LSTM, long short-term memory.

Features from 1 to 30 frames were input to each LSTM, and the classification was performed using the softmax function based on the LSTM output.

Evaluation

Cross-validation method

The cross-validation method was used to evaluate the classification accuracy of the constructed model. Fig 10 shows a simplified diagram of the cross-validation method. All datasets were initially divided into several groups, one of which was used as the test group for the evaluation, while the remaining data were used for training. Thereafter, the accuracy was comprehensively calculated by repeating the process such that all data are test data. In this study, we used a five-fold cross-validation method in which the 202 echocardiography images were divided into 142 cases for training, 20 cases for evaluation during training, and the remaining 40 cases for testing, with random sampling so that all cases were used as test data.

Fig 10

Simplified diagram of the cross-validation method (number of folds = 5).

Comparison with conventional artificial neural network

To demonstrate the effectiveness of our method, the classification accuracy was also evaluated using five-fold cross-validation on a normal artificial neural network (ANN), which does not feature a mechanism to handle time-series relations separately from LSTM [28, 29]. A schematic of the ANN is shown in Fig 11, in which all features extracted by the VGG16 method for each frame were combined and used as input to the neural network to classify the normal and AMI cases. For the hyperparameters, we set the learning rate to 1 × 10−5, the number of epochs to 50, the batch size to 30, and the input data size to 4096 × 30.

Fig 11

Schematic representation of the artificial neural network classification of myocardial infarction and normal cases.

Features from 1 to 30 frames were transformed into one-dimensional data and input into an ANN.

Results

Tables 2 and 3 show the confusion matrices and overall classification accuracies of LSTM and ANN for the LX images, while Tables 4 and 5 show the results for the PM images. Table 6 shows the classification accuracy for the LX and PM images for the given parameters and classifiers. Tables 7 and 8 show the sensitivities, specificities, and area under the curves (AUC) for the LX and PM images for the given parameters and classifiers. The accuracy of LSTM was the best when GAP was used for both the LX and PM images. The accuracies for the LX and PM images were 0.896 and 0.867, respectively.

Table 2

Overall classification accuracy of long short-term memory for long-axis view images.

		Estimated		Overall accuracy
		Normal myocardium	Myocardial infarction
Actual	Normal myocardium	89	14	0.851
	Myocardial infarction	16	83

Table 3

Overall classification accuracy of the artificial neural network for the long-axis view images.

		Estimated		Overall accuracy
		Normal myocardium	Myocardial infarction
Actual	Normal myocardium	78	25	0.772
	Myocardial infarction	21	78

Table 4

Overall classification accuracy of the long short-term memory for the short-axis papillary muscle level images.

		Estimated		Overall accuracy
		Normal myocardium	Myocardial infarction
Actual	Normal myocardium	89	14	0.832
	Myocardial infarction	20	79

Table 5

Overall classification accuracy of the artificial neural network for the short-axis papillary muscle level images.

		Estimated		Overall accuracy
		Normal myocardium	Myocardial infarction
Actual	Normal myocardium	85	18	0.802
	Myocardial infarction	22	77

Table 6

Overall classification accuracy with changing parameters.

View	Classifier	Input parameters for the classifier
		Output of FC2	GAP	GMP
LX	LSTM	0.851	0.896	0.896
	ANN	0.772	0.817	0.817
PM	LSTM	0.832	0.867	0.817
	ANN	0.802	0.797	0.837

ANN, artificial neural network; FC2, second fully connected layer of VGG16; GAP, global average pooling; GMP, global max pooling; LSTM, long short-term memory; LX, long-axis view; PM, short-axis papillary muscle level

Table 7

Results of long-axis view images.

View	Classifier	Input parameters for classifier	Sensitivity	Specificity	AUC
LX	LSTM	Output of FC2	0.838	0.864	0.870
		GAP	0.879	0.913	0.943
		GMP	0.889	0.901	0.871
	ANN	Output of FC2	0.788	0.757	0.737
		GAP	0.788	0.845	0.871
		GMP	0.768	0.864	0.830

Table 8

Results of short-axis papillary muscle view images.

View	Classifier	Input parameters for classifier	Sensitivity	Specificity	AUC
PM	LSTM	Output of FC2	0.798	0.864	0.883
		GAP	0.879	0.854	0.943
		GMP	0.808	0.825	0.891
	ANN	Output of FC2	0.777	0.825	0.796
		GAP	0.717	0.874	0.901
		GMP	0.818	0.854	0.869

Figs 12–15 compare the ROC curve. Figs 16–23 show representative correctly and incorrectly classified cases on echocardiography.

Fig 12

ROC curves of the LSTM versus ANN in long-axis view.

Fig 15

ROC curves of the GAP versus GMP in short-axis papillary muscle view.

Fig 16

False-positive cases on long-axis view images.

Fig 23

True-positive cases on short-axis view papillary muscle level images.

Discussion

In this study, we proposed an automated classification scheme for AMI and normal cases on echocardiography images using deep learning. The VGG16 method was used to extract features from the echocardiography images, while LSTM was used for the classification. The comparison of the classification models (Tables 2 and 3) shows that the results obtained with LSTM were better than those obtained using the ANN. The overall classification accuracy using LSTM was 0.852 for the LX images and 0.832 for the PM images. These results suggest that LSTM can classify AMI and normal cases with higher accuracy than ANN and analyze and classify useful features over time. In addition, the classification accuracy of LSTM suggests that the image information of one cardiac cycle (consisting of 30 frames) is useful for analyzing myocardial motion, thereby distinguishing AMI from normal myocardium. Unlike ordinary ANNs, LSTMs have a mechanism known as “gates,” optimizing them for time-series information analysis. The results showed that the LSTM can detect AMI on echocardiography with better accuracy than ANN by analyzing time series information; moreover, the results confirmed the superiority of the proposed LSTM method. Further, an AI-based study on laryngitis pathology classification showed that the LSTM classification accuracy was 15% higher than regular ANNs [30]. Other studies using AI-based solar radiation prediction have also shown that LSTM has superior accuracy in predicting solar radiation [31]. In this study by comparison, the classification accuracy was improved using LSTM, resulting in more effective use of time series data, once again confirming the effectiveness of this method.

Table 4 shows that the overall classification accuracy of LSTM was best on the LX and PM images when GAP was used. The classification accuracy on the LX images was 0.896, while that on the PM images was 0.867. These results suggest that GAP extracts more useful features for classification than GMP. In addition, the results of the comparison between the features extracted from the fully connected layer and the 512 features extracted by GAP showed that the classification accuracy of LSTM increased when GAP was used. This finding suggests that GAP reduces the number of unnecessary features for classification and achieves more efficient learning by reducing the number of parameters. The reason for the lack of change in classification accuracy between GAP and GMP when LSTM was used in the LX images may be that similar parameters were extracted from the feature maps by GAP and GMP during the pooling process.

Visual comparison of the incorrectly and correctly classified cases showed that those with low video contrast, high noise, or high brightness of the myocardium were misclassified. These results suggest that image quality, such as noise and contrast in the video image, is among the most important factors in the classification of AMI and normal cases on echocardiography. In addition, incorrect cases tended not to be shown adequately in the image: the left ventricle was blurred and a different short-axis level view was shown. In future work, accuracy should be improved by the analysis of echocardiography images and patient data from more facilities to create a robust network model.

Similar studies are listed in Table 9 for comparison with our study. Since there are very few studies with the same images and objectives, a simple comparison with this study may be difficult. However, our method was able to classify MI with an accuracy of more than 80% using 202 cases, confirming its validity.

Table 9

Comparison of related works.

Author	dataset	purpose	method	performance
Kusunose et al. [7]	Patients with old MIMI: 400Normal: 100	Detection of regional abnormal wall motion on echocardiography	Classification using CNN	AUC: 0.97
Zhang et al. [32]	Patients with MIMI: 169Control: 69	Detection of chronic MIMI on nonenhanced cardiac cine MRI	Classification using region extraction model and LSTM	Sensitivity: 0.90Specificity: 0.99AUC: 0.94
Baloglu et al. [33]	MIMI: 148Normal: 52	Detection of MIMI on 12-lead ECG	Classification using CNN	Overall accuracy: 0.9978
Huang et al. [9]	Regional wall motion abnormality: 947	Detection of regional abnormal wall motion on echocardiography	Segmentation using U-Net	Dice index: 0.756
Vece et al. [34]	Patients with takotsubo syndrome: 110Patient with a MIAMI: 110	Detection of Takotsubo cardiomyopathy on echocardiography	Classification using machine learning	AUC: 0.801 Overall accuracy: 0.745
Shimizu et al. [35]	Patients with takotsubo syndrome: 50Patient with MIAMI: 50	Detection of Takotsubo cardiomyopathy and AMI on echocardiography	Classification using CNN	Mean accuracy: 0.748
Shahin et al. [36]	8 of cardioviews: 432	Classification of echocardiographic view types	Classification using CNN and LSTM	Accuracy: 0.963

We then calculated the accuracy of the classification using left ventricular LX and short-axis PM level views, which were subsequently used to detect anteroseptal infarction. Acute anteroseptal infarction was evaluated by cardiologists and experienced sonographers using left ventricular LX and short-axis PM level views as well as apical four-chamber and apical LX views, allowing for observation of the apex. However, inexperienced clinicians, non-cardiologists, residents, and those otherwise unfamiliar with echocardiography may find it difficult to obtain apical four-chamber and apical LX view images with adequate quality. In addition, the detection of AMI is an emergent matter and requires accurate and rapid detection using left ventricular LX and short-axis PM level views, which are relatively easy to obtain. These results indicate that this method correctly identifies acute anteroseptal infarction with superior accuracy clearly distinguishing it from normal myocardium. Therefore, this method can greatly assist non-cardiologists and inexperienced clinicians alike to diagnose acute anteroseptal infarction during initial treatment.

Limitations and future works

This study has a few limitations. First, the echocardiograms were performed at the same institution. Second, this classification method is performed offline; therefore, it is necessary to apply it to real-time processing so that classification during echocardiography can be utilized. Third, we did not evaluate each segment of the heart individually; rather, we examined only the acute anterior wall septal infarction with occlusion of #6 in the American Heart Association Committee Report, which occurs the most frequently. Since this study focused only on acute anteroseptal infarction, the classification and evaluation of infarcts in each segment and in other coronary dominant regions should be performed in the future.

Conclusion

In this study, we developed an automatic detection scheme for AMI on echocardiography images using CNN and LSTM. The accuracy of the classification showed that our proposed method was able to classify AMI and normal cases with high accuracy, confirming its effectiveness as a supplemental tool for the detection of AMI on echocardiography. Here specialists and skilled doctors can easily detect an anteroseptal infarction. However, it may be difficult for residents and physicians who are unfamiliar with echocardiography at the time of the initial visit or physicians in non-cardiology clinics to detect it. Anteroseptal infarctions occur frequently and require accurate detection and diagnosis, regardless of the technician or physician’s experience, field, or situation. This method can contribute to the detection of AMI and is expected to lead to its appropriate treatment of and the prognosis of affected patients. Another technical novelty of this study is the use of LSTM, which enables a time-series analysis of wall motion on echocardiography. The results showed that LSTM can detect AMI more accurately than ANNs without a time-series analysis function, confirming its superiority using LSTM. In addition, here we used left ventricular long-axis and short-axis views (papillary muscle level), which are minimal and easy to depict for diagnosis, as input. Since the classifications were performed using highly accurate views, we found the possibility of applying this method using LSTM to other views, which are easy to take. In addition, although this study focused only on acute anteroseptal infarction, its methodology is expected to be extended to the detection of infarcts in other coronary artery dominant regions.

31 May 2021

PONE-D-21-16074

Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory

PLOS ONE

Dear Dr. Teramoto,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Based on the comments received by the reviewer and my own observations, I recommend major revisions for the paper. The authors have to address all the suggestions/comments from the reviewers carefully.

Please submit your revised manuscript by Jul 15 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Thippa Reddy Gadekallu

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In ethics statement in the manuscript and in the online submission form, please provide additional information about the patient records used in your retrospective study. Specifically, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Please include the data sources used in the Data availability statement and Methods section. We note that the origin of the testing dataset does not seem to be referenced.

4.  Thank you for including your ethics statement:  "This study was approved by an institutional review board and informed consents were obtained from patients subject to the condition of data anonymization (No. HM20).".

Please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: - Spell out each acronym the first time used in the Abstract as well as the body of the paper.

- Make sure all figures and tables are referred to in the text.

- There are many typos and grammatical mistakes in the entire paper.

- There must be column names in the tables with all borders.

- All the key terms of the equations must be mentioned

- Qualities of figures are not good.

- Relevant literature review of latest similar research studies on the topic at hand must be discussed

- The summary at the end of the literature review should be focused on the limitations of related work.

- The authors should clearly mention all parameters used to evaluate the performance of the classifiers.

- Authors can add below references:

1)  Rehman, A., Rehman, S. U., Khan, M., Alazab, M., & Reddy, T. (2021). CANintelliIDS: Detecting In-Vehicle Intrusion Attacks on a Controller Area Network using CNN and Attention-based GRU.  IEEE Transactions on Network Science and Engineering .

2)  Abbas, S., Jalil, Z., Javed, A. R., Batool, I., Khan, M. Z., Noorwali, A., ... & Akbar, A. (2021). BCD-WERT: a novel approach for breast cancer detection using whale optimization based efficient features and extremely randomized tree algorithm.  PeerJ Computer Science ,  7 , e390.

Reviewer #2: 1. What are the limitations of existing works that motivated the current research?

2. What is the novelty of the work?

3 List out the main contributions of the current work?

4. SUmmarize the related works in the form of a table.

5. Some of the recent works on LSTM and CNN applied in different domains such as the following can be discussed in the paper: "Hand gesture classification using a novel CNN-crow search algorithm, A multidirectional LSTM model for predicting the stability of a smart grid, Image-Based malware classification using ensemble of CNN architectures (IMCEC)".

6. COmpare the results obtained with recent state-of-the-art.

7. Discuss about the limitations and future scope of the current work.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

29 Aug 2021

Dear Editor and Reviewers:

We thank you and the reviewers for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. I look forward to working with you and the reviewers to move this manuscript closer to publication in PLOS ONE.

The manuscript has been rechecked and the necessary changes have been made in accordance with the reviewers’ suggestions. The responses to all the comments have been provided and the changes in the manuscript are indicated in red font.

In detail, please refer to the response document and revised manuscript.

Thank you for your consideration. I look forward to hearing from you.

Sincerely,

Atsushi Teramoto, Ph.D.

Submitted filename: Response_R1_20210717.docx

Click here for additional data file.

13 Oct 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Binh P. Nguyen, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: No

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: I believe there were a few good questions have been asked and authors have responded and addressed all previous comments.

Reviewer #4: Thank you for the opportunity to review ‘Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory” by Muraki et al. In this paper, the authors investigated the feasibility of automatic detection method for acute myocardial infarction using convolutional neural networks (CNNs) and long short-term memory (LSTM) in echocardiography. A total of 202 cines were obtained with 99 diagnoses of acute anteroseptal infarction 106 normal cases. The authors supported the usefulness of the CNNs and LSTM method for the detection of myocardial infarction using echocardiography. The following are my comments on the manuscript:

1. The main limitation of this study is only one segment (anteroseptal infarction) was included, which may not be clinically applicable. Further, the authors included the myocardial infarction with proximal left anterior descending (LAD) artery occlusion which normally results in multiple abnormal segments. I am wondering that the authors take other segments into the account.

2. Could the authors provide the criteria for myocardial infarction with proximal LAD occlusion in this study? Was coronary angiography performed for all patients to confirm the proximal LAD lesions?

3. It would be much better if the authors can provide the time of echocardiography captured: is it before or after revascularization? Noticeably, echocardiograms were obtained after revascularization, the regional wall motion abnormalities may return to normal.

3. Most importantly, the occlusion of proximal LAD will result in multiple abnormal segments which are easily detected by echocardiogram in clinical practice. I am wondering how significantly this study contributes to clinical practice.

5. As a clinical physician, I would like to know the sensitivity, specificity, and area under the curve of this technique. These metrics will provide physicians deeper insights into this technique.

6. Lastly, it would suggest that the authors provide Table 1: baseline clinical characteristics of this cohort which will illustrate the target population that this technique is applying on.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

22 Nov 2021

Please refer the response document.

Submitted filename: PLOSONE_response.docx

Click here for additional data file.

2 Feb 2022

Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory

PONE-D-21-16074R2

Dear Dr. Teramoto,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Binh P. Nguyen, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

10 Feb 2022

PONE-D-21-16074R2

Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory

Dear Dr. Teramoto:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Binh P. Nguyen

Academic Editor

PLOS ONE