Medical Image Magnification Based on Original and Estimated Pixel Selection Models.

BACKGROUND: The issue of medial image resolution enhancement is one of the most important topics for medical imaging that helps improve the performance of many post-processing aspects like classification and segmentation towards medical diagnosis.
OBJECTIVE: Our aim in this paper is to evaluate different types of pixel selection models in terms of pixel originality in medical image reconstruction problems. A previous investigation showed that selecting far original pixels has highly better performance than using near unoriginal/estimated pixels while magnifying some benchmarks in digital image processing.
MATERIAL AND METHODS: In our technical study, we apply two classical interpolators, cubic convolution (CC) and bi-linear (BL), in order to reconstruct medical images in spatial domain. In addition to the interpolators, we use some geometrical image transforms for creating the reconstruction models.
RESULTS: The results clearly demonstrate that despite the absolute preference of the original pixel selection model in the first research, we cannot see this preference in medical dataset in which the results of BL interpolator for both tested models (original and estimated pixel selection models) are approximately the same as each other and for CC interpolator, we only see a relatively better preference for the original pixel selection model.
CONCLUSION: The current research reveals the fact that selection models are not a general factor in reconstruction problems, and the structure of the basic interpolators is also a main factor which affects the final results. In other words, some interpolators in medical dataset can be affected by the selection models, while, some cannot. Copyright: © Journal of Biomedical Physics and Engineering.

Introduction

Although spatial domain interpolation methods have been evaluated and compared, optimal performance evaluation for an interpolation algorithm is difficult to explain. Indeed, the suitability of a spatial domain interpolator is extremely associated with some specific factors considered. Furthermore, other items may also affect the choice of an interpolation method and the precision of its outputs. The definition of modelling is to create a series of frameworks in order to imitate a real system. A model will not be the same as the reality, and the model which imitates the reality. In many barriers, the processes performed are based on a model. In this article, the aim is to analyze modelling performed for the special problem of interpolationof medical images based on the directly original and indirectly estimated pixel selection models. Interpolation based on spatial estimation can be used for various purposes such as image de-noising and restoration [ 1 ], image zooming/magnification and reconstruction [ 1 ], image security [ 2 ], image fusion [ 3 - 4 ], image compression [ 5 ] and image de-mosaicking [ 5 - 6 ].Therefore, image interpolation approaches play a pivotal role in medical image processing.They are wanted for many feasible applications of medical imageprocessing such as computed tomography (CT),and magneticresonance imaging (MRI). In these images, the reconstruction could utilize interpolation. In new X-ray imagingsystems such as digital subtraction angiography (DSA),interpolation provides assistance to enable the computer-boosted regulation of the current radiograph and the mask image. In addition, interpolation for focusing or revolving medical images after their acquirement is usually utilized in diagnosis and cure, and interpolators are combined into systems for computer-aided diagnosis (CAD), computer-assisted surgery (CAS) and image transmission, communication systems [ 7 - 8 ] and surveillance systems [ 9 ]. For example, if under interpolation, the total size of an image is reduced to1/4 of its first size, it means that each dimension of the image has been reduced to a half size.

As follows, we firstly review the basic concepts of image resizing problems and digital resolution in digital images. In the third section, we introduce our reconstruction models based on two different pixel selection approaches. In fourth and fifth sections, evaluation scheme and numerical results are investigated. The final section is allocated to the conclusion.

Material and Methods

Image Fundamentals

This study is a technical research with focus on simulation and computer-assisted image modeling to investigate image reconstruction modeling.

The digitization of images can often constitute two simultaneous processes of sampling and quantization. These two procedures are briefly described in order for the consideration of image storage requirements. Sampling process is used to digitize the spatial information in an image. It is often reached by dividing an image into a square, rectangular and newly hexagonal array of sampling points. Each of the sampling spots is referred to as a photo elementcalled a pixel.Ademonstration of a digital image is provided when consisting of a central dark area with the light climbing towards the periphery, see Figure 1. The process may be summarized as the digitization of an analog image into an N×N array of pixels. A sample value for N is 128 for a nuclear medicine scan, 512 for CT and MRI scans, 1024 for a DSA image and 2048 for a computed radiography image and digital radiograph. Note that N has weights which are round number and powers of 2, because of the binary nature of modern computing sort. Naturally, the bigger number of pixelsdenotes higher spatial resolution of the digitized image.

Figure 1

A representation of a digital image

The number of bits, b, is essential to show an image in digital format. It is explicit that the bit number is given by b = N×N×M for a square array with N-pixel and the radiometric accuracy of M-bit. The following chart shows that the number of bits is vital to represent images that are digitized. It must be known that very huge values are required to attain the resolution, which is utilized in medical imaging. Spatial accuracy is not the only factor in determining the spatial resolution of an image, because, on the other hand, the dot per inch (DPI) value is another effective factor in the spatial resolution of an image. In Figure 2, although the dimensions of both images HR and HR’ are equal, HR’ image has less DPI than image HR, so this fact reveals the resolution of an image does not depend on its sole size. If no methods of spatial estimation (interpolation) are used for resizing the image, then, min{DPI, but the use of appropriate estimator (interpolator) causes 1/4 DPI. In the example shown in Figure 2, the rate of compression of data is 0.75, as Eq. (1), that is, 0.75% of the primary data has been omitted (we have assumed that any type of image coding has not been used except the resizing-based approach). Figure 3 shows the importance of image interpolation that prevents the blocking issue of the simple pixel replication (For more detail, see Table 1).

Figure 2

A workflow of compression process through the interpolation methods; HR, LR and HR’ denote high resolution, low resolution and reconstructed images, respectively.

Figure 3

Images magnified based on repetition and interpolation of pixels.

Table 1

The number of bits, which is essential to save non-coded images digitized to spatial resolution of N×N pixels and depth of M.

Spatial Accuracy (N×N)	Radiometric Accuracy (Depth)
	M=8	M=10	M=12
128×128	131,072	163,840	196,608
256×256	524,288	655,360	768,432
512×512	2,097,152	2,621,440	3,145,728
1024×1024	8,388,608	10,485,760	12,582,512
2048×2048	33,554,432	41,943,040	50,331,648

Compression Rate(C.R.)=1-(b'/b) (1)

If b'= 1/4b→C.R.=0.75

For a primary HR image with 700dpi resolution (dpi is the unit of DPI factor) which has the dimension of 512×512 (pixels) and the depth of 8-bit, the primary volume is 216.32 (bits), which in the end, the total volume of HR’ will be 216.32/4 bits, but with a DPI it is less than 300 dpi.

Image Reconstruction

In compression of the information, changing compressed information into an uncompressed state is called decompression. To do so, we use the methods that are called reconstruction methods. There are various methods of reconstruction used; this way, re-sampling is one of the most useful tools. Re-sampling can be the sum of two operations up-sampling for resizing and estimating methods like spatial domain interpolators. In many aspects of image processing, the spatial domain estimation means the same as interpolation method. The most important geometrical transform in order to do resizing in digital images is an affine linear transform as Eq. (2), where the output is consequently obtained by Eq. (3). According to Eq. (3), (X,Y) are the primary coordinates of pixels, (X',Y') and are the secondary coordinates of the pixels in resized (up-sampled) image. It is clear that after resizing, there is a possibility of producing new pixels, which should be estimated by an estimation method; the estimation is done based on the existing values of adjacent pixels.

(2)

(3)

Figure 4 shows the way of resizing (up-sampling) for the sample image with the size of 2×2 whose dimensions have been doubled in both of length and width c = c = 2. After performing the required rescaling, the next step is to estimate values for the new pixels through interpolation methods. To specify a better estimation method, a technical assessment is required by numerical standards of quality assessment (QA) which is based on similarity evaluation of outputs, different interpolators and the source HR image. In the science of computer graphics, it can be shown that when similarity of a reconstructed image is more then another one to the source HR image, then, its DPI value will be greater than the second case equivalently [ 1 ]. There are various estimation methods based on interpolations such as classic methods, cubic convolution (BL) and bi-linear (CC) [ 1 , 10 - 12 ] used in this study. We investigate pixel selection models under these two interpolation methods. More explanations of the interpolation methods can be found in [ 1 ].

Figure 4

Up-sampling based on standard linear affine transform.

Experiment Design

In order to investigate the effects of various estimators, we have to design a test. To this end, first, we have to change the primary image (HR) into LR version through down-sampling, then, through reconstruction, it is changed into HR’ which has equal dimensions to the source HR image. In order to have a numerical quality assessment (QA), we can use some metrics such as PNSR [ 13 ] or SSIM [ 3 ] which obtain the order of similarity between HR and HR’ images. PNSR is as Eq. (4) and is used here. Although it does not match human visual system (HVS) [ 4 ] well, it has been known as an acceptable standard metric, and is one of the most well-known standards of quality assessment in digital image processing (DIP).

(4)

The mapping used between two images under re-sampling and down-sampling processes can be clearly effective on outputs and some parts of modeling where error comes from these two stages; however, our objective here is not the error of this section. Thus, for all tests, an equal mapping must be used in the stage of re-sampling (a design like Figure 4) and down-sampling (a design like Figure 5). The design used in down-sampling stage is based on Eq. (5) or equivalently Figure 5 under the condition of c = 1/2.

Figure 5

Down-sampling model used.

(5)

Results

Modeling Results

Our goal is to do compression with the rate of C.R.=1-(1/16)=0.94, that is, we compress images with the dimensions of 512×512up to 128×128. To do this, we will simply use Eq. (5) under cx =cy = 4. The problem which should be discussed is the number of re-sampling steps. It is clear that it is possible to do the image reconstruction directly and produce the HR’ image (or HR’_direct). In this state, we use Eq. (3) under cx =cy = 4, but another way is to perform re-sampling in two steps, we again use Eq. (2) under cx =cy = 4 in order to produce HR’ (or HR’_indirect), whose dimensions are 256×256 in the first step, and 512×512 in the second step. The reason of using multi-step state is that in many scientific articles dealing with interpolation methods, the number of steps is not usually investigated most of the time, this work is done in only one-step, whereas many new methods for spatial domain estimation are not capable to do the reconstruction in a step like LMMSE [ 3 - 4 ]. Thus, there is the potential to change such algorithms to the direct mode of estimation (i.e. one-step); if the direct approach in an application for a wide range of scalable (free-size) interpolators has better performance. For instance, LMMSE method is one, which will do the mentioned problem in two steps. Therefore, it is evident that, in a case of comparing this method with other methods, it is desired to change other methods such as LMMSE to produce another version, which is capable of estimating in just one-step (or directly). Basically, two interpolation methods should be compared when all conditions of modeling are the same as each other. It is therefore clear that the first step is the equality of the dataset, and the metrics of QA that are the general requirements of simulation; nonetheless, these two are not all the conditions, the implementation of models is also very important. In Table 2, the results of PNSR metric for test images including benchmarks such as Lena, Baboon, etc. are seen [ 14 ]; where all images have been tested in terms of four models based on original/direct and estimated/indirect pixel selection models and also two interpolators of BL and CC (they are therefore named Direct BL, Indirect BL, Direct CC, and Indirect CC). In Table 3, PNSRs of all models for 8 medical test images (Figure 6) are shown.

Table 2

PNSRs of all models for the benchmarks.

Sample Image	Direct BL	Direct CC	Indirect BL	Indirect CC
Baboon	23.1811	23.5885	22.9007	23.5320
Barbara	23.3498	23.6039	23.1909	23.5833
Boat	25.0423	25.5370	24.7197	25.4792
Lena	29.2312	30.0148	28.7892	29.9396
Peppers	28.4763	29.2906	28.0100	29.2214
Plane	26.3751	27.2243	25.8854	27.1464
Sailboat	24.7903	25.5305	24.3578	25.4600
Tiffany	26.1302	26.5471	25.8921	26.5110

Table 3

PNSRs of all models for medical test images.

Sample Image	Direct BL	Direct CC	Indirect BL	Indirect CC
Image1	32.981	32.8670	32.2649	32.4277
Image2	32.651	32.8475	32.4645	32.7899
Image3	36.129	36.4860	35.5892	35.9414
Image4	34.087	35.6459	33.9790	34.0205
Image5	30.543	31.7586	30.0479	30.9286
Image6	35.451	36.2075	34.6418	35.9688
Image7	34.098	35.3335	34.0089	35.6231
Image8	33.283	33.8882	33.7415	34.0013

Figure 6

Medical test images contain various types of images, e.g. MRI, Ultrasound, X-ray images.

The results clearly show that despite the absolute preference of original pixel selection model in the first research [ 14 ], we cannot see this preference in medical dataset in which the results of BL interpolator for both tested models (original and estimated pixel selection models) are nearly the same as each other. For CC interpolator, we only see a relatively better preference for the original pixel selection model. Figures 7 and 8 also describe the data of Table 3 for BL and CC interpolators separately (under two different pixel selection models).

Figure 7

CC-based results for direct and indirect models in medical images.

Figure 8

BL-based results for direct and indirect models in medical images.

Eq. (6) is a simple metric for computing the distance or difference between two selected models. The results of using this metric are shown in Figure 9, and Table 4 describes its details. It is clear that “d13” has the least value among all labels, because Direct BL and Indirect BL models in medical dataset nearly have the same PSNRs.

Figure 9

It shows the distances of all models compared each other.

Table 4

Details for Figure 9

Distance	Interpolation Methods	Description
d12	Direct CC, Indirect CC	Difference in Fig. 7
d13	Direct BL, Indirect BL	Minimum difference, Difference in Fig. 8
d14	Direct BL, Indirect CC	-
d23	Direct CC, Indirect BL	Maximum difference
d24	Direct CC, Direct BL	-
d34	Indirect BL, Indirect CC	-

(6)

Figure 10 shows some processed images under the mentioned models visually. See more details in its capture. Visual comparison has usually been a main factor for quality assessment.

Figure 10

Columns “B”, “C”, “D”, and “E” show the different models of Direct BL, Direct CC, Indirect BL, and Indirect CC, respectively; and the column “A” shows the source images before any processing including down-sampling and reconstruction. Direct CC model has the best quality in practice.

Discussion

This paper gives a complementary viewpoint on findings in [ 14 ]. The results clearly demonstrate that despite the absolute preference of the original pixel selection model in the first research, we cannot see this preference in medical dataset in which the results of BL interpolator for both tested models (original and estimated pixel selection models) are approximately the same as each other and for CC interpolator, we only see a relatively better preference for the original pixel selection model. The current research reveals the fact that selection models are not a general factor in reconstruction problems, and the structure of the basic interpolators is also a main factor which affects the final results. Therefore, in designing new magnification algorithms for direct zoom of medical images, it is possible that selecting original pixels might not have a considerable impact on the algorithm performance. In other words, some interpolators in medical dataset can be affected by the selection models, while, some cannot. In addition, computational complexity [ 15 ] for direct and indirect models should be further discussed in terms of number of operations. For a future work, we can consider these results as a guideline for improving non-freesize interpolators to a more free-size state or generally other interpolators [ 16 - 22 ] and image processing applications such as [ 9 , 23 - 24 ].

Conclusion

The analyses of the outputs show that ignoring the manner of reconstruction models is clearly effective in numerical outputs. The current research exhibited that selection models are not a general factor in reconstruction problems, and the structure of the basic interpolators is also a main factor which affects the final results. Therefore, in designing new magnification algorithms for direct zoom of medical images, it is possible that selecting original pixels will not have a considerable impact on the algorithm performance. In other words, some interpolators in medical dataset can be affected by selection models, but some cannot.