Kui Liu1, Jieqing Tan2, Liefu Ai3. 1. School of Computer and Information, Hefei University of Technology, Tunxi Road, 23009 Hefei, China ; The Key Lab of Intelligent Perception and Computing of Anhui Province, Jixian Road, 246011 Anqing, China. 2. School of Computer and Information, Hefei University of Technology, Tunxi Road, 23009 Hefei, China. 3. The Key Lab of Intelligent Perception and Computing of Anhui Province, Jixian Road, 246011 Anqing, China.
Abstract
To eliminate the staircasing effect for total variation filter and synchronously avoid the edges blurring for fourth-order PDE filter, a hybrid regularizers-based adaptive anisotropic diffusion is proposed for image denoising. In the proposed model, the [Formula: see text]-norm is considered as the fidelity term and the regularization term is composed of a total variation regularization and a fourth-order filter. The two filters can be adaptively selected according to the diffusion function. When the pixels locate at the edges, the total variation filter is selected to filter the image, which can preserve the edges. When the pixels belong to the flat regions, the fourth-order filter is adopted to smooth the image, which can eliminate the staircase artifacts. In addition, the split Bregman and relaxation approach are employed in our numerical algorithm to speed up the computation. Experimental results demonstrate that our proposed model outperforms the state-of-the-art models cited in the paper in both the qualitative and quantitative evaluations.
To eliminate the staircasing effect for total variation filter and synchronously avoid the edges blurring for fourth-order PDE filter, a hybrid regularizers-based adaptive anisotropic diffusion is proposed for image denoising. In the proposed model, the [Formula: see text]-norm is considered as the fidelity term and the regularization term is composed of a total variation regularization and a fourth-order filter. The two filters can be adaptively selected according to the diffusion function. When the pixels locate at the edges, the total variation filter is selected to filter the image, which can preserve the edges. When the pixels belong to the flat regions, the fourth-order filter is adopted to smooth the image, which can eliminate the staircase artifacts. In addition, the split Bregman and relaxation approach are employed in our numerical algorithm to speed up the computation. Experimental results demonstrate that our proposed model outperforms the state-of-the-art models cited in the paper in both the qualitative and quantitative evaluations.
With the popularity of image sensor, digital images play a key role in people’s daily life. Unfortunately, images are ineluctably contaminated by noise during acquisition, transmission, and storage. Therefore, image denoising is still an open and complex problem in image processing and computer vision (Chatterjee and Milanfar 2010). Image denoising aims to recovering the original image u from the observed noisy image , where , and n is the zero-mean Gaussian white noise with standard deviation .During the past three decades, lots of approaches for removing noise have been developed from linear models to nonlinear models. Linear models perform well in the smooth area. However, they don’t preserve edges and corners. To overcome the disadvantages of the linear denoising models, nonlinear denoising models have been developed which have a good balance between noise removal and edge-preserving. Nonlinear models based on variation (Rudin et al. 1992) and partial differential equation (PDE) (Perona and Malik 1990) have been widely used for image denoising. The best known variational denoising model is the total variation (TV) model proposed by Rudin et al. (1992), which minimizes the following equation,where is a bounded open domain with Lipschitz boundary, denotes the gradient operator, is the TV regularization term, is the fidelity term, is the regularization parameter, which measures the trade off between the regularization term and the fidelity term.The classical TV model is efficient for removing noise and preserving the edges. However, it possesses some undesirable properties in the recovered image under some circumstances, such as the staircasing effect. To overcome the deficiency of the original TV model (Strong 1997), developed the adaptive TV regularization based variational model as,where g(x) is an adaptive edge-stopping function, which is defined in Strong (1997) as follow,where is a threshold parameter for balancing the noise removal and edge preservation, and is the Gaussian filter with standard deviation . Seen from (3), g(x) is smaller near the edges and larger away from the boundaries, so the model of (2) has the capability of preserving the edges while removing noise because the diffusion is stopped across edges.In addition, Nikolova replaced the -norm with the -norm in the fidelity term of TV model in Nikolova (2002). Osher et al. (2005) proposed an iterative regularization method of TV model. Chen et al. (2010) presented an adaptive total variation method based on the difference curvature. Wang et al. (2011) put forward a modified TV model.Numerical experiments demonstrate that the models mentioned above have good performance in the terms of the trade-off between removing noise and preserving the edges. Unfortunately, the staircasing effects appear in the recovered image owing to using the TV-norm as the regularization term. To overcome this shortcoming, high-order PDE filters have been proposed and applied for image denoising successfully (Lysaker et al. 2003; Liu et al. 2011). One of the most classical fourth-order PDEs (LLT) is introduced by Lysaker et al. (2003)where denotes the Laplacian operator. However, a major challenge is that higher-order PDEs blur the edges during image denoising.To make use of the advantages of both TV filter and high-order PDE filters, some hybrid regularization models are recently proposed, which combined the second-order partial differential equations and the fourth-order partial differential equations (Oh et al. 2013). Li et al. (2007) proposed the adaptive image denoising model based on hybrid regularizers combining the advantages of TV model and LLT model as follows,where g(x) also denotes the edge-stopping function defined as in (3). The results of experiments indicate that the model of (5) performs better than the pure second-order or hight-order models.In recent years, efficient computational algorithms for solving the denoising models have emerged in large numbers, for instance, fixed point iteration, gradient descent methods, primal-dual methods, relaxation methods, Bregman iteration and split Bregman method, and so on. These methods are efficient for image denoising while preserving the edges.Inspired by Li et al. (2007) and Liu (2015), we propose a novel adaptive anisotropic diffusion model, incorporating the advantages of the total variation filter and the fourth-order filter, and develop an efficient computational algorithm. The main contributions of our paper can be generalized as follows. First of all, the hybrid regularization term of the novel model is composed of total variation regularization and a fourth-order filter. The fidelity term uses the -norm as opposed to the more commonly used -norm or -norm. The two above-mentioned filters can be adaptively selected according to the diffusion function. When the pixels locate at the edges, the total variation filter is selected to filter the image, which can preserve the edges. When the pixels belong to the flat regions, the fourth-order filter is adopted to smooth the image, which can eliminate the staircase artifacts. Another main contribution is that the split Bregman and relaxation approach are successively employed in our numerical algorithm to speed up the computation. Experimental results demonstrate that our proposed model achieves higher quality in both the qualitative and quantitative aspects than that of the state-of-the-art models cited in the paper.The remainder of this paper is organized as follows. In “Preliminaries” section, we give some definitions. In “The new model and algorithms” section, we give the proposed model and numerical implementation in detail. The experimental results are given in “Experiments” section. Finally, this paper is concluded in the fifth section.
Preliminaries
In this section, we give a brief overview of some necessary notations and definitions for the proposed model, which will be used in the subsequent sections.
Definition 1
(Chen and Wunderli 2002). Let be an open bounded subset of with Lipschitz boundary. Given a function . Then the total variation of u in is defined as,where div is the divergence operator, is the subset of continuously differentiable vector functions of compact support contained in , and is the essential supremum norm.
Remark 1
Let the Sobolev space be . If , the space is a Banach space.
Definition 2
(Liu et al. 2007). Let be an open bounded subset of with Lipschitz boundary. Given a function . Then the seminorm of u is defined as,wherewith and , and .
Definition 3
(Liu 2015). Let be an open subset of with Lipschitz boundary. Given a function , and let be a continuous real function. Then the total Variation of u in is defined by,where the vector valued function . Moreover, the seminorm is characterized by .
Definition 4
(Liu 2015). Let be an open subset of with Lipschitz boundary. Given a function , and let be a continuous real function. Then the weighted seminorm of u in is defined as,and .
Definition 5
(Jia et al. 2011). For and , the soft thresholding operator is defined as,
The new model and algorithms
The proposed model
Meyer analyzed that there exists no oscillation function in the space and a weaker -norm is appropriate to represent textured or oscillatory patterns (Meyer 2001), so that we replace -norm of the fidelity term with -norm. Therefore, a novel adaptive image denoising model is proposed,where u and are the recovered image and the noisy image, respectively. Seen from Eq. (12), the -norm is considered as the fidelity term and the regularization term is composed of a total variation regularization and a fourth-order filter in the proposed model. , and is the inverse Laplace operator. The diffusivity function g(x) is defined as,where Gaussian filter pre-smooths the noisy image. The larger standard deviation of the noise is, the larger standard deviation of Gaussian filter is. We set , where C lies between 0 to 1. When , it means that the pixels locate at the edges. Then total variation filter is selected to filter the image, which can preserve the edges. When , it means that the pixels belong to the flat regions. Then the fourth-order filter is adopted to smooth the image, which can eliminate the staircase artifacts. We replace g(x) with g in the next part of this article. Figure 1 shows the results of image denoising by our proposed model and model from Li et al. (2007), which demonstrates that the model whose fidelity term uses the -norm yields better results in image denoising since -norm is appropriate to represent textured or oscillatory patterns.
Fig. 1
Results of image denoising by our model and model from Li et al. (2007). a Original image, b noisy image with , c result by our model, d noise, detecting by our model, e result by model from Li et al. (2007), f noise, detecting by model from Li et al. (2007)
Results of image denoising by our model and model from Li et al. (2007). a Original image, b noisy image with , c result by our model, d noise, detecting by our model, e result by model from Li et al. (2007), f noise, detecting by model from Li et al. (2007)
The numerical algorithm for the proposed model
We apply a split Bregman method (Cai et al. 2009) to solve Eq. (12). The idea of split Bregman method is to use splitting operator and Bregman iteration to solve various inverse problems (Goldstein and Osher 2009).We turn Eq. (12) into the following constrained minimization problem by introducing an auxiliary variable z,The method of solving the constrained minimization problem is that it may be transformed into the unconstrained minimization problem, so the constrained problem (14) can be turned into the following unconstrained problem by introducing an auxiliary variable b,By taking advantage of split Bregman method, Eq. (15) can be solved iteratively according to the following equations,where k is the number of iterations.
Solve the first subproblem in Eq. (16)
At present, the Euler–Lagrange equation method is usually used to solve the problem similarity to the first subproblem in Eq. (16). However, it works slowly. To accelerate the computation speed, the split Bregman algorithm and relaxation algorithm are adopted to solve the first subproblem in Eq. (16).First, we define , and , and then the first subproblem in Eq. (16) can be rewritten as follows,where , , and are the first-order difference operators and the second-order difference operators, respectively. All the difference operators are approximated using following formulas:where represents the image size.Second, we introduce four auxiliary variables and , and then Eq. (17) can be transformed into the following constrained optimization problem,with and .The above constrained problem (22) are turned into the unconstrained minimization problem,where the parameters and . Let and , and apply the split Bregman method, Eq. (23) can be solved by following equations,with the update equations,where . For , choose and .According to the relaxation algorithm (Jia et al. 2011), we may define,So, we havewhere , , and are respectively the adjoint operators of , , and . and have the following discrete forms,Definitely, and .
Solve the second subproblem in Eq. (16)
For the second subproblem in Eq. (16), we derive the Euler–Lagrange equation with respect to z, which is as follows,This is a linear equation, so additional operator split (AOS) iteration and Gauss–Seidel (GS) iteration can be used to solve Eq. (30). We use AOS iteration to solve this equation.In summary, the proposed algorithm for image denoising can be described as follows,
Experiments
In this section, we experimentally compare our proposed model with the state-of-the-art models. All experiments are performed under Matlab R2009a on a PC with an Intel CPU of 1.7 GHz and 4 GB memory. Six grayscale images viz. “Manmade”, “Lena”, “Peppers”, “Barbara”, “Cameraman”, and “House” are selected as testing examples for both qualitative and quantitative evaluations. The original test images are shown in Fig. 2. The performances of all methods are compared quantitatively by using the peak signal to noise ratio (PSNR), structural similarity index measure (SSIM) (Wang et al. 2004), multi-scale structural similarity index (MS-SSIM) (Wang et al. 2003), and feature-similarity index (FSIM) (Zhang et al. 2011). In addition, we also compare the computing time and iterations of six models. PSNR is defined as follows,withwhere u and are respectively the recovered image and the original image. Generally, the larger the value of the PSNR, the better the performance. However, PSNR is inconsistent with human visual judgments. SSIM, MS-SSIM, and FSIM are close to the human vision system, so we also use them to assess the noise removal quality. SSIM is defined by,where and are the mean and variance of u, respectively, is the covariance of u and , and and are two constants to avoid instability. MS-SSIM is defined by,where the luminance distortion , the contrast distortion and the structure distortion at scale i between images u and are defined as follows,where and represent the mean intensity of u and at scale i; (resp. ) is the standard deviation of u (and ) at scale i, and is the covariance between u and at scale i. , and are three small constants to avoid instability. In this paper, the values of the exponents , and are set as the same as those in Wang et al. (2003). FSIM is defined by,where at each location x is the similarity measure, which is defined as product of the similarity function on Phase Congruency (PC) and similarity function on Gradient Magnitude (GM). and are defined as follows,where and denote the PC maps extracted from u and , respectively, and and denote the GM maps extracted from u and , respectively; and are two small positive constants to avoid instability.
Fig. 2
Original test images. a Manmade image, b lena image, c peppers image, d barbara image, e cameraman image, f house image
The termination condition for all experiments is defined as follows,where and are respectively denoising results at nth and iteration, and is a given positive number. We set in the experiments.Original test images. a Manmade image, b lena image, c peppers image, d barbara image, e cameraman image, f house imageFigure 3 shows the results for the 8-bit gray-scale synthetic image with size pixels, which is corrupted by zero-mean Gaussian white noise with . In our experiments, we use the trial-and-error method for determining the optimal parameters. We set , , , , and in our algorithm, while all parameter values in TV model Rudin et al. (1992), LLT model Lysaker et al. (2003), non-local means (NLM) model Buades et al. (2005), BLS-GSM Portilla et al. (2003), and hybrid model Liu (2015) are chosen manually by trial-and-error method to ensure the best results. Figure 3a, b are the original and noisy images, respectively. Figure 3c–h show that the denoising results of TV model, LLT model, NLM model, BLS-GSM, hybrid model, and our proposed model, respectively. Figure 3c indicates that there exists the staircase effects in the recovered images by TV model. Although LLT model has the advantage on relieving the staircase effect, the edges are blurred and there are serious speckles in the recovered image. The computational efficiency of NLM model is very low. There exits the edges blurring in the result of BLS-GSM from Fig. 3f. We also find that there still exits a little stair-case effect by hybrid model from Fig. 3g. Our proposed model relieves the staircasing effects and avoids the edges blurring. Table 1 shows the PSNR, SSIM, MS-SSIM, and FSIM values corresponding to Fig. 3. From Table 1 and Fig. 3, it is shown that our proposed model produces the best result. At the same time, the proposed method takes less computational time than LLT model, NLM model and hybrid model, but the computational time of our proposed method is slightly inferior to that of TV model and BLS-GSM.
Fig. 3
Results of the synthetic image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Table 1
Performance comparison of the recovered results with different methods in Fig. 2
Method
PSNR
SSIM
MS-SSIM
FSIM
Iterations
Time(s)
TV model
31.97
0.91
0.97
0.94
246
11.20
LLT model
30.62
0.86
0.95
0.75
216
87.21
NLM model
32.15
0.89
0.97
0.89
1
275.42
BLS-GSM
31.94
0.92
0.96
0.91
1
4.68
Hybrid model
32.67
0.92
0.96
0.90
197
93.72
Proposed model
33.49
0.93
0.97
0.93
147
81.72
Results of the synthetic image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelPerformance comparison of the recovered results with different methods in Fig. 2In order to demonstrate that our model can also work well for natural images, the next experiments are conducted for different images corrupted by zero-mean Gaussian white noise with . The experimental results are shown in Figs. 4, 5, 6, 7 and 8, where the experimental results of TV model, LLT model, NLM model, BLS-GSM, hybrid model, and our proposed model are illustrated, respectively. The figures show that our model produces the visually most appealing results among the six models. The quantitative PSNR, SSIM, MS-SSIM, and FSIM values are presented in Fig. 9, which depicts that the performance of our proposed model are better than those of TV model, LLT model, NLM model, BLS-GSM, and hybrid model. In order to verify the better performance of our proposed method, Fig. 10 shows the enlarged regions cropped from Fig. 4.
Fig. 4
Results of Lena image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Fig. 5
Results of Peppers image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Fig. 6
Results of Barbara image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Fig. 7
Results of Cameraman image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Fig. 8
Results of House image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Fig. 9
Comparison results in a PSNR, b SSIM, c MS-SSIM and d FSIM for Gaussian noise with
Fig. 10
The enlarged detail regions cropped from Fig. 3. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our model
Results of Lena image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelResults of Peppers image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelResults of Barbara image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelResults of Cameraman image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelResults of House image by the six models. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelComparison results in a PSNR, b SSIM, c MS-SSIM and d FSIM for Gaussian noise withThe enlarged detail regions cropped from Fig. 3. a Original image, b noisy image, c result by TV model, d result by LLT model, e result by NLM model, f result by BLS-GSM model, g result by hybrid model, h result by our modelWe also use six images corrupted with different levels of Gaussian noise to examine the performance of the proposed model and the alternative models. Tables 2, 3, 4 and 5 give the PSNR, SSIM, MS-SSIM, and FSIM values obtained by the proposed model and the alternative models, respectively. From Tables 2, 3, 4 and 5, it can be observed that our model is greater than or equal to the other five models in PSNR, SSIM, MS-SSIM, and FSIM for the same standard deviation, which demonstrate that our model provides the best noise removal performance at different noise level.
Table 2
Comparison results in PSNR(dB) for different levels of Gaussian noise
Comparison results in PSNR(dB) for different levels of Gaussian noiseComparison results in SSIM for different levels of Gaussian noiseComparison results in MS-SSIM for different levels of Gaussian noiseComparison results in FSIM for different levels of Gaussian noise
Conclusions
To eliminate the so-called staircase effect in total variation filter and avoid the edges blurring for fourth-order PDE filter, we propose an adaptive anisotropic diffusion model for image denoising, which is composed of a hybrid regularization term combining a total variation filter and a fourth-order filter and the fidelity term using the -norm. We also develop an efficient algorithm to solve our proposed model. Numerical experiments show that our proposed model has the highest PSNR, SSIM, MS-SSIM, and FSIM values among the six methods, and can preserve important structures, such as edges and corners.