A New Method for Pseudo-increasing Frame Rates of Echocardiography Images Using Manifold Learning.

Increasing frame rate is a challenging issue for better interpretation of medical images and diagnosis based on tracking the small transient motions of myocardium and valves in real time visualization. In this paper, manifold learning algorithm is applied to extract the nonlinear embedded information about echocardiography images from the consecutive images in two dimensional manifold spaces. In this method, we presume that the dimensionality of echocardiography images obtained from a patient is artificially high and the images can be described as functions of only a few underlying parameters such as periodic motion due to heartbeat. By this approach, each image is projected as a point on the reconstructed manifold; hence, the relationship between images in the new domain can be obtained according to periodicity of the heart cycle. To have a better tracking of the echocardiography, images during the fast motions of heart we have rearranged the similar frames of consecutive heart cycles in a sequence. This provides a full view slow motion of heart movement through increasing the frame rate to three times the traditional ultrasound systems.

INTRODUCTION

Echocardiography is one of the widely used diagnostic tools in the imaging world, because it is noninvasive, safe, and inexpensive; it also allows real-time visualization of the size and shape of the heart, its pumping capacity and the location and extent of any damage in its tissues.

An important issue in echocardiography is to track the rapid variations of myocardium and heart valves in real time. Thus, it is necessary to have a high frame rate imaging system to illustrate the fast moving structures.

A real time ultrasound image is made up of many information lines obtained in a very short period of time. Typical number of lines in the field of view ranges from about 100 to over 200 lines per frame. There exist a proper number of lines to have a good lateral resolution. In an ordinary cardiac imaging, by considering 128 lines per frame, 20 cm depth and 1500 m/s for the speed of sound, the maximum frame rate is around 30 (frames/sec). But for fast moving structures, such as heart valves, a frame rate up to 80 (frames/sec) may be required. For fetal echocardiography and small animal imaging, where typically high heart rates take place, high frame-rate and full-view ultrasound imaging is an emerging topic, but it is a challenging issue in echocardiography of adult.[1] Some simple methods for increasing the frame rate are expressed in literatures. Konofagou et al.[2] and D’hooge et al.,[3] increased the frame rate to above 200 Hz by reducing the view angle and decreasing the number of lines in a sector. Applying this method, the time needed to construct the whole image reduced and the frame rate increased. Kanai et al.[45] used a sparse sector-scan format. He achieved the peak frame rate of 450 Hz by decreasing the number of ultrasound beams to a value about 16. However, reducing the field of view was not suitable for full-view cardiac imaging. Also, the lateral resolution was decreased proportional to the number of lines.

Wang et al.[6] proposed a high frame rate imaging technique based on ECG gating and retrospective multi-sector composite method. Other developed methods for increasing the frame rate are coded-excitation ultrasound imaging,[7] parallel beam forming technique[8] and synthetic aperture (SA) ultrasound imaging.[9] However, these methods require complicated calculations.

In this paper, a new method based on considering cyclic movement of the heart using manifold learning algorithm for increasing frame rates of echocardiography images is proposed. Manifold learning is a recently introduced approach in nonlinear dimension reduction which can extract the relationship between frames of consecutive images in low-dimensional space.[1011] Image manifolds have been shown to be useful representation tools for magnetic resonance imaging (MRI) sequences,[12-15] facial images,[1617] and tracking.[18-20]

The paper is organized as follows. The ‘method’ section gives a brief background in manifold learning and followed by a description of LLE[11] (Locally Linear Embeddings) algorithm which is used. Embedding echocardiography images and increasing the frame rate using manifold learning is explained in sections four and five, respectively. Benefits and limitations of this research are summarized in the ‘conclusion’ section.

MATERIALS AND METHODS

Dimensionality reduction is important in machine learning for simplifying of analyzing large data sets in classification, compression, and visualization of high dimensional data.

Multidimensional scaling (MDS)[21] and principal component analysis (PCA)[22] are the traditional linear techniques for dimensionality reduction. These methods often fail to discover complex nonlinear data structures. Various studies have shown that nonlinear techniques outperform their linear counterparts in highly nonlinear artificial tasks.[15] Manifold learning is a recent approach for nonlinear dimensionality reduction. A lot of data sets often consist of a large number of samples, each of which involves many features. Manifold learning algorithms attempt to uncover intrinsic parameters in order to find a low-dimensional representation of the data. We can refer to Isomap,[10] LLE,[1123] Laplacian embeddings,[24] and Local Tangent Space Alignment (LTSA)[25] as examples of manifold learning algorithms.

In cardio-pulmonary imaging, the dimensionality of a frame in a video sequence – which usually considered as number of pixels in a frame – is normally a large value. But, in fact, frame variations of a particular patient can be small due to the deformation caused by the patient's heartbeat, imaging geometry, noise and time-varying effects of contrast agents. Accordingly, when there are only a few comprising causes of the variation, these images naturally have a low dimensional structure which can be extracted by manifold learning algorithms. As some applications, manifold embedding is applied to segmentation,[1213] interpolation,[14] and noise reduction[15] of cardiac MRI images.

In echocardiography, images which are taken with the same imaging geometry of the same patient have one principle degree of freedom and that is the phase of the heartbeat. Indeed, changes in the heartbeat phase lead to variations in the shape of the heart, but not to its position dominantly. Hence, manifold learning can depict each image as a point on the reconstructed manifold and the relationship between images in the new domain can be obtained according to periodicity of the heart cycle.[2627] The main trait of LLE algorithm is that similar frames in high-dimensional space retain their neighborhoods in low-dimension space too.[1123] Accordingly, we choose this algorithm as a proper one in manifold learning algorithms. In the next section, we briefly introduce the principles of this algorithm.

LLE Algorithm

LLE algorithm transforms N observations of X data set with dimensionality D into a new dataset Y which consists of N samples with dimensionality d (where d < D, and often d < < D). This process retains the geometry of the data as much as possible. In other words:

The embedding is optimized to preserve the local configurations of nearest neighbors. The algorithm is divided into three different steps and summarized in Figure 1.

Figure 1

Diagram of the LLE algorithm. The three main steps are: (1) define the neighborhood for each point, (2) solve for the reconstruction weights, and (3) learn embedding which preserves the reconstruction weights. Image obtained[23]

In the first step, the K nearest neighbors for each data point is described as measured by Euclidean distance. The outcomes of the LLE are normally steady in a range of neighborhood sizes.[23] In this research by practical computation, the stable range was 8 < k < 12 and we select k=10 nearest neighbors for each data point.

In the second step of the LLE, every data point reconstruct from its nearest neighbors. This can be expressed as the cost function by minimizing the reconstruction error:

Where x is the current data point, x denotes one of the nearest neighbors, and W is the corresponding reconstruction weight between x and x. To calculating the weights, the E(w) in (2) is minimized by considering two constraints: a sparseness constraint and an invariance constraint. Note that W=0 if x is not in the neighborhood of x, hence, any data point x is rebuilt exclusively from its neighbors. This fact is equal to the sparseness constraint. The invariance constraint is satisfied by putting (Σw=1). Accordingly, the LLE obtains the least squares solution to w by assuming these constraints.

In the final step of the algorithm, each low dimension output y is projected by high dimension input x with quoting global internal coordinates on the manifold. This is conceivable by choosing the d dimensional coordinates of each output y to minimize the cost function (3):

Equation (3), like the one in cost function (2) is based on locally linear reconstruction errors. The only dissimilarity here is that w are fixed and output y are optimized. In this step of the algorithm original inputs, x is not included. Hence, the geometric information in w describes the embedding completely. Therefore, the goal is calculating y with low dimension in the same way that high dimensional inputs x are reconstructed by w weights. By rewriting (3) as the quadratic form:

The sparse, symmetric, and semipositive definite matrix M has N×N dimensions and is equal to:

where I is the identity matrix. Finally, by solving eigenvalue problem of cost matrix M, the embedding cost function (4) can be minimized. The bottom d+1 non-zero eigenvectors of cost matrix prepare an ordered set of d embedding coordinates.[23]

Data acquisition

The apical two-dimensional gray scale sequences of three healthy volunteers were acquired using a Vivid 3 ultrasound machine with a 2.0 MHz probe and stored in AVI format with 200×200 resolution, including the ECG display. Four-chamber and two-chamber views were used. For conducting the present study, sequences of three successive cycles were stored with care being taken to ensure that there was no probe or respiratory movement during the data acquisition.

Embedding echocardiography images

LLE algorithm was implemented on three cases of normal echocardiography images. One case had N=80 frames of two heart cycles and the others contain three heart cycles with N=120 frames. Each frame was a matrix with resolution of 200×200 pixels which reshaped to 1×40000 array, so the dimension of first space was 40000, which was very high. We embed the images in two dimensional space by LLE with K=10 nearest neighbors. The plot of normalized eigenvalues shows that almost all the information in the cardiac cycle is captured in two dimensions; thus, the images have been embedded in a two-dimensional space by the LLE algorithm. Figures 2–4 show the image manifolds of these cases.

Figure 2

The two dimensional non-linear embedding of three cycles in normal hearts using LLE algorithm (using k=10 neighbors), first case

Figure 4

The two dimensional non-linear embedding of three cycles in normal hearts using LLE algorithm (using k=10 neighbors), third case

The two dimensional non-linear embedding of two cycles in normal hearts using LLE algorithm (using k=10 neighbors), second case

In these figures, each symbol is a projection of the original consecutive images connected to each other via a line. Three points are highlighted with their corresponding images for better understanding.

Because of the various shapes and dynamical nature of the heart in different subjects, the corresponding manifolds are not similar. But in a particular subject, the manifold cycles are similar. However, in all the cases, the cyclic nature of the cardiac motion leads to a close curve. It should be considered that because of variable heart rate of each case, the number of frames per cardiac cycle is variable too.

Pseudo increasing echocardiography frame rate

In echocardiography, high temporal resolution is required for the dynamic heart to observe rapid motion of the myocardium and valves for early detection of cardiac diseases. Thus, high frame rate is an essential factor in ultrasound cardiac imaging. Several methods for increasing ultrasound frame rate were mentioned in the ‘introduction’ section.

A novel method for increasing frame rate and providing slow motion analysis can be extracted from image manifold information. As mentioned before, by manifold embedding with LLE algorithm, the similar frames of echocardiography images remain neighbors to each other in two-dimension space and relationship between the frames is obtained in a close cycloid. In this method, consecutive cycles of the heart are used and their manifolds are drawn in one trajectory.

For pseudo increasing frame rate, we arrange similar frames of multiple heart beat cycles which are extracted from the manifold. Thus, the consecutive frames are sequenced. In the other words, instead of having 3 heart beat cycles each of which has 40 frames, we can have one cycle with 120 frames. Figure 5 shows an image manifold of three cycles of normal four-chamber view with different colors for each cycle. Because imaging procedure is independent of heart beat movement, the probability of registering frames on each other in obtained manifold is very low. In other words, the similar images stand nearby in manifold but not exactly in a same position.

Figure 5

Image manifold of three heart beat cycle. Each cycle has different color

Figure 6 shows four consecutive frames of first cycle and Figure 7 shows ten frames containing specified frames from two other cycles that embedded between these four frames. These Figures demonstrate advantage of proposed method in increasing frame rate and providing slow motion analysis. Slow motion of opening mitral valve in 10 continues frames can be tracked better.

Figure 6

Four continuous frame of one cycle. Opening mitral valve is distinctive in frames

Figure 7

Ten frames of continues sequence extracted from three cycles of image manifold of Figure 5. The numbered frames correspond to frames in Figure 6

By this approach, we can simply increase frame rate by the advantages of having full view demonstration and without any damage on lateral resolution. Also the computation time with matlab 2009a and 2 GHz CPU configuration was less than five seconds which is a very fast method.

The results obtained with our method were validated by the experienced echo cardiologist (golden standard) on three healthy volunteers. The cardiologist emphasized on accuracy and usefulness of the presented method.

CONCLUSION

Since echocardiography images have natural and non-linear deformations, linear dimension reduction methods cannot recognize the correct relationship between frames. LLE algorithm is one of the most popular manifold learning and non-linear dimension reduction methods. The main advantage of LLE algorithm is that similar frames in high-dimensional space retain their neighborhoods in low-dimension space too; this fact was the important reason for selecting the LLE algorithm as a desired method.

Results of using LLE algorithm in analyzing echocardiography images in other applications, such as de-noising[27] and extracting end-systole and -diastole frames,[26] emphasize on benefits and power of this algorithm on echocardiography images.

Because the heart movement is independent of imaging procedure and there is no synchronization between ECG and imaging, the probability of exact similarity of frames in different cycles is low. Hence, in dimension reduction, the neighborhood mapped points do not obvolute exactly. After recognizing the relations between frames, we achieved increasing frame rate by arranging consecutive frames of multiple heart beat cycle which are extracted from manifold. By this approach, we can pseudo increase frame rate two or three times more than the original frame rate. The most benefit of this method is illustrating fast movements of myocardium and heart valves in a full view image without decreasing lateral resolution.

An important issue in this method is fixing patient and probe while doing echocardiography. Because any sudden movement of probe or patient causes drastic differences in frames and the manifold trajectory in consecutive heart cycles would not be the same.

Although the results show that the proposed method is applicable for normal cases, we are going to develop the algorithm on a wide range of normal and abnormal cases in order to find a diagnosis protocol for different heart diseases.

We believe that manifold learning algorithm can make a new horizon in analyzing the medical images and particularly echocardiography images.

BIOGRAPHIES

Parisa Gifani received the B.S. degree in Electrical Engineering from Ferdowsi University of Mashhad, Mashhad, Iran, in 2007 and the M.S. degree in Medical Engineering from Iran University of Science and Technology, Tehran, Iran in 2010. Now she is a Ph.D. candidate in Medical Engineering from Iran University of Science and Technology, Tehran, Iran. Her research interests are Medical Image processing and Echocardiography.

Hamid Behnam received the B.S. degree in Electrical Engineering from Iran University of Science and Technology, Tehran, Iran, in 1988, the M.S. degree in Medical Engineering from Sharif University of Technology, Tehran, Iran, in 1992, the Ph. D. degree in Applied Electronics from Tokyo Institute of Technology, Tokyo, Japan in 1998. Since 1998 till 2004, he was a researcher at Iran Research Organization for Science and Technology and from 2004 he has been a faculty member at Iran University of Science and Technology, in Tehran, Iran. Currently, he is an Assistant Professor of Biomedical Engineering at the IUST. His research interests are Ultrasound in Medicine, Medical Image processing and Medical signal processing.

Zahra Alizadeh Sani earned a medical degree at Mashhad University of Medical Sciences, Mashhad, Iran in 1998. She was Resident of Cardio Vascular Diseases in Mashhad University of Medical Sciences, Mashhad, Iran in Oct. 2002-Sep. 2006 and completed fellowships in Echocardiography (Sep.2007-Mar.2009), at Iran University of Medical Sciences, and in Basic Cardiac MR (June.2009),St-Marien-Hospital Bonn Germany. Her research interests are Cardiovascular MRI, Physics, Pulse sequence, clinical application, Cardiovascular CT, Angiography, physics, technique, clinical application Echocardiography, physics, clinical application. Since 2009, she works as an assistant professor of Cardiovascular Imaging in Cardiovascular Imaging department, Shaheed Rajaei Cardiovascular and Medical center, Tehran University.