Fact-checked by Grok 2 weeks ago

Edge-preserving smoothing

Edge-preserving smoothing is a class of nonlinear image processing techniques designed to reduce noise and fine details in digital images while preserving significant edges and structural features, thereby avoiding the blurring artifacts commonly associated with linear low-pass filters like the Gaussian filter.^[1] Unlike traditional smoothing methods that apply uniform averaging across the image, edge-preserving approaches adaptively control the degree of smoothing based on local image gradients, ensuring that sharp discontinuities—such as object boundaries—are maintained to support accurate feature detection and visual fidelity.^[2] The foundational concepts of edge-preserving smoothing emerged in the late 1980s and early 1990s, with early work by Yaroslavsky in 1985 introducing robust diffusion-based methods for noise reduction in scale-space representations.^[2] A seminal advancement came in 1990 with the anisotropic diffusion model proposed by Perona and Malik, which formulates smoothing as a partial differential equation where diffusion is inhibited across high-gradient regions to sharpen edges while blurring homogeneous areas.^[3] This was followed in 1998 by the bilateral filter from Tomasi and Manduchi, a non-iterative, spatially localized method that weights pixel contributions by both spatial proximity and radiometric similarity, enabling efficient edge retention in gray-scale and color images.^[4] Subsequent developments have expanded the field with variants like the guided image filter (2010), which uses a guidance image for linear-time computation and applications in depth enhancement, and more recent techniques such as the fast M-estimation Gaussian filter with automatic basic width determination (2023), which automates parameter selection for improved denoising in recognition tasks.^[2]^[1] These methods are pivotal in computer vision and graphics, finding applications in noise suppression, tone mapping for high-dynamic-range imaging, super-resolution reconstruction, and preprocessing for segmentation or stereo matching, where preserving edges ensures downstream algorithms perform robustly without loss of perceptual quality.^[1]

Introduction

Definition and Principles

Edge-preserving smoothing refers to a class of nonlinear image processing techniques that reduce noise and suppress fine-scale details in homogeneous regions while maintaining the integrity of significant edges and boundaries. These methods achieve smoothing by selectively halting the diffusion process or modulating filter weights in areas exhibiting high image gradients, thereby preventing the unwanted blurring of structural features that define object outlines and textures. This approach is particularly valuable in scenarios where preserving perceptual sharpness is essential for subsequent analysis or visualization.^[5] The fundamental principles of edge-preserving smoothing revolve around the detection and respect for edges, which are typically identified by the magnitude of the local image gradient. Nonlinear operators adapt their behavior spatially, adjusting the influence of neighboring pixels based on both their spatial proximity (domain kernel) and photometric similarity (range kernel) to the central pixel. In uniform areas with low gradients, smoothing proceeds aggressively to eliminate noise; however, across high-gradient edges, the adaptation reduces the contribution from dissimilar pixels, effectively preserving boundary sharpness. An early illustration of this principle is anisotropic diffusion, where the diffusion coefficient diminishes at edges to halt smoothing.^[6]^[5] Unlike linear filters, such as the Gaussian blur, which convolve the image with a fixed kernel to apply uniform smoothing and consequently attenuate all high-frequency components—including edges—edge-preserving methods are spatially variant and leverage edge-aware metrics like local gradient magnitudes or variance to discriminate between noise and structure. This selectivity avoids the isotropic blurring inherent in linear approaches, enabling noise reduction without compromising perceptual fidelity.^[5] At a mathematical level, edge-preserving smoothing often takes the general form of a weighted average over a local neighborhood \Omega, where the weights combine domain and range components to enforce edge awareness:

I^{\text{smoothed}}(\mathbf{x}) = \frac{1}{W(\mathbf{x})} \sum_{\mathbf{y} \in \Omega} w_d(\|\mathbf{x} - \mathbf{y}\|) \, w_r(|I(\mathbf{x}) - I(\mathbf{y})|) \, I(\mathbf{y})

Here, w_d represents the domain weight (e.g., based on spatial distance), w_r the range weight (e.g., based on intensity difference), and W(\mathbf{x}) the normalization factor ensuring the weights sum to unity. This formulation adaptively downweights contributions from pixels separated by edges, promoting smoothing within similar regions.^[4] These techniques emerged in the late 1970s through the 1990s as responses to the shortcomings of earlier linear denoising methods, which struggled with artifacts like salt-and-pepper noise while inevitably degrading edge quality in images.^[7]

Applications

Edge-preserving smoothing plays a crucial role in image denoising, where it effectively reduces noise in photographs while maintaining sharp structural edges, thereby improving visual quality without introducing blurring artifacts. In medical imaging, such as MRI and CT scans, these techniques attenuate Gaussian noise common in acquisition processes, preserving anatomical boundaries essential for accurate diagnosis. For instance, anisotropic diffusion methods control smoothing based on local edge strength to denoise MRI images while retaining fine details like tissue interfaces. Similarly, in satellite imagery, edge-preserving filters address speckle noise in synthetic aperture radar (SAR) data, enabling clearer visualization of terrain features without distorting boundaries.^[8] As a preprocessing step for image segmentation, edge-preserving smoothing enhances object boundaries, facilitating tasks in computer vision such as tumor detection in medical scans and terrain mapping from aerial images. By sharpening edges prior to segmentation algorithms, it improves the delineation of regions of interest, reducing false positives in boundary detection for applications like brain tumor identification via MRI. This preprocessing step ensures that subsequent machine learning models, such as support vector machines or convolutional neural networks, operate on cleaner inputs with preserved structural integrity.^[9]^[10]^[11] In artistic rendering and photo editing software, edge-preserving smoothing enables stylization effects, such as cartoon-like abstractions or texture simplification, by smoothing intra-region variations while keeping inter-region edges intact. Tools employing these methods allow users to create non-photorealistic images with bold outlines and uniform color regions, as seen in hierarchical segmentation-based stylization pipelines. The bilateral filter, in particular, supports real-time applications in such software for efficient detail enhancement during editing workflows.^[12]^[13] For video processing, edge-preserving smoothing is applied frame-by-frame to reduce noise in low-light footage, maintaining temporal consistency and edge sharpness across sequences. Nonlinear spatio-temporal filters, such as rational operators, attenuate impulse noise in video communications while preserving motion edges, making them suitable for real-time denoising in broadcast or surveillance systems.^[14] Specific applications include noise removal in no-flash photography, where edge-preserving filters fuse low-light images with a guiding flash exposure to recover details without halo artifacts around edges. In high dynamic range (HDR) imaging pipelines, these methods decompose images into base and detail layers for tone mapping, ensuring smooth gradients in overexposed regions while retaining edge contrast for natural-looking results. Overall, edge-preserving smoothing improves the signal-to-noise ratio in noisy environments and offers tunable parameters—such as spatial and range sigma in bilateral variants—to balance smoothness against detail retention, adapting to diverse imaging conditions.^[15]

Classical Filters

Median Filter

The median filter is a non-linear, non-iterative technique for edge-preserving smoothing that replaces each pixel's intensity with the median value of intensities within a local neighborhood, typically a square window such as 3×3 or 5×5 pixels.^[16] This operation sorts the pixel values in the neighborhood and selects the middle value (for odd-sized windows), effectively reducing noise while avoiding the averaging that can blur sharp transitions.^[16] Mathematically, the output intensity at position \mathbf{x} is given by

I'(\mathbf{x}) = \median \{ I(\mathbf{y}) \mid \mathbf{y} \in N(\mathbf{x}) \},

where N(\mathbf{x}) denotes the neighborhood centered at \mathbf{x}, and \median is the median operator.^[17] The filter preserves edges because the median is robust to outliers; in regions near discontinuities, the sorted values cluster on either side of the edge, allowing the median to select a value from the dominant side without crossing the boundary, unlike linear filters that average across it.^[17] The primary parameter is the window size, which must be an odd integer to ensure a central pixel; common choices include 3 or 5, with larger windows providing stronger smoothing at the cost of potential minor blurring of fine edges.^[16] Computationally efficient implementations achieve constant time per pixel using histogram-based methods that update gray-level histograms as the window slides, making it suitable for real-time applications despite the sorting requirement.^[18] It excels at suppressing impulse noise, such as salt-and-pepper noise, where random high or low outliers are isolated and thus ignored in the median selection.^[16] However, the median filter can introduce blocky artifacts or streaking in textured regions with fine details, as repeated median operations may flatten subtle variations into uniform blocks. It is less effective for Gaussian noise, which affects all pixels additively rather than as outliers, often requiring larger windows that compromise detail preservation.^[17] Originally introduced in the 1970s for one-dimensional signal processing by Tukey to handle nonsuppressible noise, it was adapted to two-dimensional images in the late 1970s and early 1980s for noise reduction and smoothing.^[19]^[16]

Bilateral Filter

The bilateral filter, introduced by Tomasi and Manduchi in 1998, extends the traditional Gaussian filter by incorporating both spatial proximity and intensity similarity to achieve edge-preserving smoothing.^[20] It operates as a weighted average where contributions from neighboring pixels are modulated by Gaussian weights in both the spatial domain and the range domain, enabling non-iterative, local processing that reduces noise while maintaining sharp edges.^[21] This nonlinear approach replaces the intensity of each pixel with a blend of nearby values, weighted to favor similar intensities and nearby locations, thus avoiding the blurring of discontinuities typical in linear filters.^[4] The core formulation of the bilateral filter for an input image I at pixel \mathbf{x} is given by:

I'(\mathbf{x}) = \frac{1}{W(\mathbf{x})} \sum_{\mathbf{y} \in N(\mathbf{x})} G_s(\|\mathbf{x} - \mathbf{y}\|) \, G_r(|I(\mathbf{x}) - I(\mathbf{y})|) \, I(\mathbf{y}),

where G_s is the spatial Gaussian G_s(d) = \exp\left(-\frac{d^2}{2\sigma_d^2}\right), G_r is the range Gaussian G_r(i) = \exp\left(-\frac{i^2}{2\sigma_r^2}\right), N(\mathbf{x}) denotes the neighborhood around \mathbf{x}, and the normalization factor is W(\mathbf{x}) = \sum_{\mathbf{y} \in N(\mathbf{x})} G_s(\|\mathbf{x} - \mathbf{y}\|) \, G_r(|I(\mathbf{x}) - I(\mathbf{y})|).^[4]^[21] Edge preservation arises from the range term G_r, which downweights pixels with intensities far from I(\mathbf{x}), effectively preventing the averaging of values across intensity discontinuities such as edges.^[4] This mechanism ensures that smoothing occurs primarily within homogeneous regions, preserving boundaries without introducing halo effects in most cases.^[21] The filter is controlled by two key parameters: the spatial standard deviation \sigma_d, which governs the locality of the neighborhood (typically 2-5 pixels for fine-scale denoising), and the range standard deviation \sigma_r, which sets sensitivity to intensity differences (often 0.1-0.2 of the image's dynamic range, such as 25-50 for 8-bit grayscale images).^[4] Larger \sigma_d extends the smoothing kernel for broader effects, while higher \sigma_r allows more aggressive blending across minor intensity variations.^[21] Notable variants include the joint bilateral filter, which applies range weights derived from a separate guidance image rather than the input itself to enhance edge alignment, and the cross-bilateral filter, which leverages a flash image's edges to denoise a corresponding no-flash image in low-light scenarios.^[21]^[22] Among its advantages, the bilateral filter generates smooth intensity gradients within regions while preserving overall image structure, and approximations such as separable implementations enable real-time performance on modern hardware.^[21] However, it can introduce gradient reversal artifacts near edges, where local minima or maxima appear inverted, and its direct implementation incurs a high computational cost of O(N r^2) for an image of N pixels and filter radius r.^[23]^[24] The guided filter offers a faster linear alternative that mitigates some of these issues.

Diffusion-Based Methods

Anisotropic Diffusion

Anisotropic diffusion is a partial differential equation (PDE)-based method for edge-preserving smoothing that evolves the image intensity I over time by simulating a diffusion process where the diffusion coefficient varies according to the local image gradient. The general evolution equation is given by

\frac{\partial I}{\partial t} = \nabla \cdot \left( c(\|\nabla I\|) \nabla I \right),

where c(\|\nabla I\|) is the conductance function that decreases as the gradient magnitude \|\nabla I\| increases, thereby controlling the amount of smoothing applied.^[6]^[25] This approach preserves edges by setting a low conductance c at high-gradient locations, which halts diffusion across boundaries and prevents blurring of sharp transitions, while permitting strong intra-region smoothing in homogeneous areas with low gradients. As a result, the method effectively reduces noise and produces piecewise smooth images without diffusing significant structural features.^[6]^[25] To implement the PDE numerically, finite difference schemes are commonly used to discretize the spatial derivatives, with explicit schemes (e.g., forward Euler) requiring small time steps for stability and implicit schemes offering greater stability at the cost of solving linear systems per iteration. Key parameters include the total diffusion time t, which determines the extent of smoothing; the initial (maximum) conductance c_0 (often set to 1 for normalized diffusion); and the edge threshold \lambda (also denoted as K) in the conductance function, typically chosen between 10 and 30 for 8-bit images to balance edge preservation and noise reduction.^[25]^[26] The technique excels at handling Gaussian noise and yielding piecewise smooth outcomes, making it suitable for denoising while maintaining perceptual quality. However, it is sensitive to noise in the gradient estimation, which can lead to false edges, and may produce staircasing artifacts—blocky, stepwise approximations—in sloped regions.^[25] Anisotropic diffusion was introduced by Perona and Malik in 1990 as a scale-space framework for edge detection and smoothing, with subsequent refinements by Weickert in the late 1990s emphasizing tensor-based diffusion for structure preservation.^[6]^[25]

Perona-Malik Model

The Perona-Malik model, proposed in 1990, represents a seminal approach to anisotropic diffusion tailored for edge-preserving smoothing in images. It modifies the diffusion process by incorporating edge-sensitive conductance functions that reduce smoothing across high-gradient regions while allowing it within homogeneous areas. This model builds on the general framework of nonlinear diffusion but specifies explicit forms for the diffusivity to achieve selective smoothing.^[3] Central to the model are two conductance functions, denoted as c_1 and c_2, which depend on the image gradient magnitude \|\nabla I\| and a threshold parameter K > 0. The first, c_1(\|\nabla I\|) = \exp\left(-\left(\frac{\|\nabla I\|}{K}\right)^2\right), follows a Gaussian decay, promoting diffusion in wide, low-contrast regions by rapidly attenuating conductance near stronger edges. The second, c_2(\|\nabla I\|) = \frac{1}{1 + \left(\frac{\|\nabla I\|}{K}\right)^2}, exhibits an inverse gradient behavior, which favors preservation of isolated, sharp edges over broader transitions. The parameter K serves as an edge threshold, calibrated to the noise level or desired contrast sensitivity, enabling the model to distinguish noise from significant boundaries.^[3]^[25] The model's edge preservation arises from a forward-backward diffusion dynamic: in low-gradient areas (\|\nabla I\| < K), forward diffusion smooths noise effectively; in high-gradient regions (\|\nabla I\| > K), reduced or negative effective diffusivity halts blurring, potentially sharpening edges through backward diffusion. This mechanism promotes intra-region smoothing within homogeneous areas while enhancing inter-region separation at boundaries. Implementation typically involves explicit finite-difference schemes iterated over discrete timesteps, with stability requiring \Delta t < 0.25 h^2 / \max(c), where h is the grid spacing; larger steps risk instability due to the backward component.^[3]^[25] Advantages of the Perona-Malik model include its simplicity and effectiveness in generating scale-space representations for edge detection, outperforming isotropic methods by maintaining sharp discontinuities during noise reduction. However, it suffers from ill-posedness stemming from the unstable backward diffusion, leading to paradoxes where solutions may not exist or behave unexpectedly under perturbations. Additionally, the choice of K is sensitive to noise, often requiring empirical tuning.^[3]^[27]^[25] Extensions of the model in the 1990s addressed these limitations by coupling it with regularization, such as pre-smoothing the gradient with a Gaussian kernel to ensure well-posedness, and integrating it into scale-space analysis for multi-resolution processing. These refinements, including robust variants with statistical constraints, improved stability and edge continuity without altering the core conductance paradigm.^[28]^[29]

Optimization and Graph-Based Approaches

Guided Filter

The guided filter is an explicit edge-preserving smoothing method that leverages a guidance image to perform local linear filtering, introduced by He, Sun, and Tang in 2010 for applications such as flash/no-flash image processing and depth map upsampling.^[30] Unlike implicit filters, it derives an explicit kernel from a local linear model, enabling efficient computation while inheriting structural details from the guidance image, which can be the input image itself or an external reference.^[30] The algorithm assumes that within each local window \omega_k centered at pixel k, the filtering output q_i for input image p is a linear transform of the guidance image I: q_i = a_k I_i + b_k for all i \in \omega_k, where a_k and b_k are constant coefficients per window.^[30] These coefficients are determined by minimizing a cost function that balances fidelity to the input and regularization:

E(a_k, b_k) = \sum_{i \in \omega_k} \left( (a_k I_i + b_k - p_i)^2 + \epsilon a_k^2 \right),

where \epsilon is a regularization parameter to prevent degenerate solutions in low-variance regions.^[30] The closed-form solutions are

a_k = \frac{\text{cov}(I, p)^k}{\sigma_I^{k^2} + \epsilon}, \quad b_k = \bar{p}_k - a_k \mu_I^k,

with \text{cov}(I, p)^k = \frac{1}{|\omega|} \sum_{i \in \omega_k} I_i p_i - \mu_I^k \bar{p}_k, \sigma_I^{k^2} = \frac{1}{|\omega|} \sum_{i \in \omega_k} I_i^2 - (\mu_I^k)^2, \mu_I^k as the mean of I in \omega_k, and \bar{p}_k as the mean of p in \omega_k.^[30] The final output at each pixel i averages the coefficients over all overlapping windows containing i: q_i = \bar{a}_i I_i + \bar{b}_i.^[30] Edge preservation occurs because the output gradient approximates \nabla q \approx \bar{a} \nabla I near strong edges in the guidance image, directly transferring edge locations without reversal artifacts common in methods like the bilateral filter.^[30] In flat regions of the guidance, \sigma_I^{k^2} is small, so \epsilon dominates to enforce uniform smoothing, while in textured areas, the filter adapts to preserve details.^[30] Key parameters include the window radius r, typically set to 4–8 pixels for balancing locality and smoothness, and the regularization \epsilon, often in the range $10^{-3} to $10^{-2} (e.g., $0.01) to control the trade-off between edge preservation and noise reduction.^[30] The method achieves linear time complexity O(N) for an image of N pixels, using integral images for efficient mean and variance computation, making it faster and more exact than approximate bilateral filter implementations.^[30] It provides explicit edge awareness without the halo effects or gradient reversals seen in joint bilateral variants.^[30] For large r, its behavior asymptotically approaches that of the bilateral filter when using the input as guidance.^[30] Applications include denoising in flash/no-flash pairs, where the non-flash image guides smoothing of the flash image to reduce overexposure while preserving textures, and joint upsampling of low-resolution depth maps guided by high-resolution color images.^[30] Limitations arise from the local linearity assumption, which can introduce boxy artifacts in regions with small r or halos near sharp edges if the guidance lacks sufficient detail.^[30] Since its introduction, the guided filter has been extended to high-dimensional signals and integrated with deep learning models for enhanced performance, as reviewed in early 2025.^[31]

Total Variation Minimization

Total variation (TV) minimization is an optimization-based framework for edge-preserving smoothing that seeks to reduce noise while maintaining sharp discontinuities in images. The core model, known as the Rudin-Osher-Fatemi (ROF) formulation, minimizes the total variation of the image subject to a data fidelity constraint, formulated as:

\min_{I} \ \mathrm{TV}(I) + \lambda \|I - I_0\|_2^2,

where \mathrm{TV}(I) = \int_\Omega \|\nabla I\| \, dx measures the total length of edges in the image I, I_0 is the noisy input image, and \lambda > 0 is a regularization parameter balancing smoothness and fidelity to the original data.^[32] This L1-norm-based TV term promotes piecewise constant solutions by penalizing variations in the image gradient, effectively measuring the "edge length" rather than the magnitude of changes.^[32] The edge-preserving property arises from the L1 norm of the gradient, which favors abrupt jumps (discontinuities) over gradual transitions, in contrast to L2-based methods like Gaussian smoothing that smooth across edges.^[32] This sparsity-inducing effect in the gradient domain leads to solutions with flat regions separated by sharp boundaries, making TV minimization particularly effective for denoising while avoiding blurring of object outlines.^[33] The ROF model was originally proposed for grayscale image denoising.^[32] Variants include anisotropic and isotropic TV formulations; the anisotropic version discretizes TV as \sum_{i,j} (|\partial_x I_{i,j}| + |\partial_y I_{i,j}|), treating horizontal and vertical gradients separately to better preserve directional edges, while the isotropic form uses \sum_{i,j} \sqrt{(\partial_x I_{i,j})^2 + (\partial_y I_{i,j})^2} for rotationally invariant edge detection. Extensions in the 2000s adapted TV to color images by applying vector-valued norms, such as the coupled TV for RGB channels to handle chromatic discontinuities. Further developments introduced nonlocal TV, which incorporates patch similarities across the image for texture preservation beyond local edges.^[34] Solving the ROF model involves non-smooth convex optimization, typically via gradient descent methods as in the original formulation, or more efficient primal-dual algorithms that reformulate the problem in a saddle-point structure.^[32]^[35] Fast approximations include the Chambolle projection algorithm, which computes the exact minimizer through dual projections onto the TV ball in a fixed number of iterations.^[33] In discrete settings, the image is represented on a grid, with gradients approximated by finite differences, and the optimization proceeds iteratively.^[32] The parameter \lambda controls the trade-off and is often tuned based on noise level, with typical values ranging from 10 to 100 for standard grayscale images assuming unit-normalized intensities.^[33] Advantages of TV minimization include its strong theoretical foundation as a convex problem guaranteeing global optima and unique solutions under certain conditions, alongside its ability to produce sharp, oversmoothing-free edges in denoised outputs.^[32]^[33] However, it can introduce staircasing artifacts—unwanted stepwise approximations—in regions of smooth intensity ramps, and the optimization is computationally intensive for large images without accelerated solvers. The ROF model was introduced in 1992, marking a foundational shift toward variational methods in image processing.^[32] Its extensions to color and nonlocal variants emerged prominently in the 2000s, broadening applications to textured and multichannel data.^[34] More recent nonconvex variants, such as truncated \ell_p minimization introduced in 2025, further enhance edge preservation by selectively penalizing small gradients.^[36]

Advanced Techniques

Iterative Solvers

Iterative solvers are essential in edge-preserving smoothing when single-pass filters fail to adequately handle high levels of noise, as repeated applications allow for progressive refinement of the image while maintaining structural boundaries.^[37] In graph-based formulations, images are modeled as signals on undirected graphs where pixels correspond to nodes, enabling smoothing operations that respect edges by leveraging the graph's topology.^[38] The graph structure is defined by an adjacency matrix A, where A_{ij} represents the similarity weight between neighboring pixels i and j, often computed via Gaussian kernels on intensity differences.^[39] The degree matrix D is diagonal with D_{ii} as the sum of weights connected to node i, and the graph Laplacian is given by L = D - A.^[38] Edge-preserving smoothing can then be achieved iteratively through polynomial approximations of the heat diffusion equation on the graph, expressed as (I - \alpha L)^k y, where y is the input image signal, \alpha is a step size, and k denotes the number of iterations.^[40] Among iterative methods, Chebyshev iteration approximates functions of the Laplacian efficiently for large graphs, requiring only matrix-vector multiplications with L and preserving edges by attenuating high-frequency noise components.^[40] For quadratic approximations in smoothing energies, the conjugate gradient (CG) method solves the resulting linear systems arising from graph Laplacians, accelerating convergence in non-linear edge-preserving filters.^[37] In spectral filtering scenarios, the locally optimal block preconditioned conjugate gradient (LOBPCG) addresses eigenvalue problems of the Laplacian to extract low-frequency components for smoothing, offering robustness for partial eigenvector computations. LOBPCG was introduced in 2001.^[41]^[42] For non-quadratic problems like total variation minimization, proximal gradient methods iteratively apply soft-thresholding to gradients, while the alternating direction method of multipliers (ADMM) decomposes the problem into parallelizable subproblems for efficient convergence in image denoising. Recent advances include parallel proximal-gradient algorithms for large-scale TV minimization, enhancing efficiency for 3D imaging tasks as of 2023.^[43]^[44] Typical parameters include an iteration count k ranging from 10 to 50, sufficient for balancing noise reduction and detail preservation, and a step size \alpha < 2 / \lambda_{\max}(L), where \lambda_{\max}(L) is the largest eigenvalue of the Laplacian, ensuring numerical stability.^[40] These solvers offer advantages such as guaranteed convergence to global optima for convex formulations and inherent parallelizability on graph structures, making them suitable for large-scale imaging tasks.^[37] However, they can be computationally slow for ill-conditioned Laplacians common in high-resolution images, often necessitating preconditioning techniques to improve efficiency.^[45] The conjugate gradient method originated in the 1950s for solving linear systems and was adapted to imaging applications in the 2000s.^[42] In total variation denoising, these methods enhance convergence rates compared to direct solvers.^[43]

Edge-Enhancing Smoothing

Edge-enhancing smoothing extends traditional edge-preserving techniques by actively amplifying discontinuities in the image, thereby sharpening edges while reducing noise in homogeneous regions. This approach is particularly valuable for applications such as deblurring blurred images and extracting prominent features for further analysis like segmentation.^[46] One seminal technique involves shock filters, which employ nonlinear partial differential equations (PDEs) to propagate shocks that create step-like edges from blurred or noisy inputs. Introduced by Osher and Rudin in 1990, these filters model the evolution of the image intensity I using a hyperbolic PDE of the form \partial I / \partial t + \text{sign}(\nabla I \cdot \nabla (|\nabla I|^{-1} \Delta I)) |\nabla I| = 0, where the shock propagation enhances edge contrast by concentrating intensity gradients at discontinuities.^[46] Shock filters are effective for restoring sharp boundaries in medical imaging and computer vision tasks, but require careful discretization to maintain stability.^[46] Another method builds on anisotropic diffusion by incorporating negative conductance to promote edge sharpening rather than mere preservation. In variants of the Perona-Malik model, the diffusion coefficient g(|\nabla I|) is chosen such that it increases with the gradient magnitude, leading to forward diffusion in low-contrast areas and backward (sharpening) diffusion near edges, as described by the modified equation \partial I / \partial t = \div (g(|\nabla I|) \nabla I). This negative conductance amplifies high-frequency components at boundaries, improving edge definition in textured images. Graph-based approaches achieve edge enhancement by assigning negative weights to edges in the graph Laplacian, which concentrates the smoothing "heat" at boundaries and simulates wave-like propagation to sharpen features. Relaxing the positivity constraint on adjacency matrix weights allows the filter to infer sharper graph models from data, enhancing discontinuities without over-smoothing interiors. For instance, the graph Laplacian L = D - A with negative entries in A (the adjacency matrix) results in an indefinite operator that boosts edge signals during iterative smoothing. Such methods with negative weights emerged in the 2010s as computational graph methods advanced. Key parameters in these methods include shock strength, which controls the intensity of edge amplification in shock filters, and thresholds that balance enhancement against smoothing in diffusion-based models—typically set based on noise levels to avoid excessive sharpening.^[46] Recent developments as of 2025 include truncated \ell_p minimization models for edge-preserving and enhancing smoothing, which eliminate insignificant details while amplifying salient edges using nonconvex optimization.^[46]^[47] These techniques offer advantages such as improved edge contrast for downstream tasks like object segmentation and robustness to moderate blur.^[46] However, over-application can introduce oscillations or spurious edges, particularly in regions with complex textures.^[46] Historically, shock filters originated in 1990.^[46]

Extensions

Edge-Preserving Upsampling

Edge-preserving upsampling addresses the limitations of traditional interpolation methods, such as bicubic upsampling, which often introduce blurring and smoothing across edges during image resolution enhancement.^[48] These conventional approaches fail to maintain sharp boundaries, leading to loss of detail in high-frequency regions, whereas edge-preserving techniques leverage high-resolution prior images to guide the interpolation process and preserve structural integrity.^[48] Key techniques in edge-preserving upsampling include guided upsampling methods that apply bilateral or guided filters to a low-resolution input, using a co-registered high-resolution guidance image—such as a flash photograph or intensity map—to modulate the filtering weights and enforce edge adherence.^[48] The joint bilateral upsampling algorithm exemplifies this approach: it first performs a preliminary upsampling of the low-resolution input using simple interpolation, then refines the result by applying a bilateral filter where the range weights are derived from the intensity differences in the high-resolution guidance image to ensure that the output respects edges in the guidance.^[48] The core formulation of joint bilateral upsampling operates on the upsampled grid of the low-resolution image L, producing the output U(x) as follows:

U(\mathbf{x}) = \frac{1}{W(\mathbf{x})} \sum_{\mathbf{y}} G_{\sigma_s}(\|\mathbf{x} - \mathbf{y}\|) \, G_{\sigma_r}(|I_{\text{high}}(\mathbf{x}) - I_{\text{high}}(\mathbf{y})|) \, L_{\text{ups}}(\mathbf{y})

where G_{\sigma_s} is the spatial Gaussian kernel, G_{\sigma_r} is the range Gaussian kernel applied to the intensity difference in the high-resolution guidance I_{\text{high}}, L_{\text{ups}} is the preliminary upsampled low-resolution input, and W(\mathbf{x}) is the normalization factor \sum_{\mathbf{y}} G_{\sigma_s}(\|\mathbf{x} - \mathbf{y}\|) \, G_{\sigma_r}(|I_{\text{high}}(\mathbf{x}) - I_{\text{high}}(\mathbf{y})|).^[48] This formulation ensures that the upsampling process weights neighboring pixels based on both spatial proximity and photometric similarity in the guidance image, thereby avoiding smoothing across discontinuities.^[48] Applications of edge-preserving upsampling are prominent in super-resolution tasks involving depth maps, where low-resolution depth data is enhanced using aligned RGB images as guidance to recover sharp depth boundaries without introducing artifacts.^[49] Another significant use is in no-flash to flash image transfer, where a low-resolution no-flash image is upsampled and refined using a high-resolution flash counterpart to produce a detailed, noise-reduced output that preserves natural edges.^[48] Critical parameters in these methods include the guidance strength, which controls the influence of the high-resolution prior via the range kernel \sigma_r, and the upsampling factor, with 2x to 4x being common for balancing computational cost and quality in practical scenarios.^[48] The guided filter offers efficiency advantages in this context due to its linear-time complexity compared to the bilateral filter's higher cost.^[30] Advantages of edge-preserving upsampling include the ability to sharpen fine details in interpolated regions and reduce aliasing artifacts that plague standard methods, resulting in visually coherent high-resolution outputs.^[48] However, the technique's performance heavily depends on the quality and alignment of the guidance image, potentially introducing artifacts such as edge bleeding or misalignment if there is a mismatch between the low- and high-resolution inputs.^[48] Historically, edge-preserving upsampling gained popularity in the 2000s for consumer imaging applications, with joint bilateral upsampling introduced in 2007 as a foundational method.^[48] Post-2010 advancements have integrated deep learning hybrids, such as trainable guided filters, to further enhance adaptability and performance in guided super-resolution tasks.^[50] More recent developments as of 2025 include universal feature upsampling methods like AnyUp, which preserve edges in depth maps using guidance images, and edge-aware attention mechanisms for detail-preserving super-resolution.^[51]^[52]

Integration with Other Processes

Edge-preserving smoothing is commonly applied as a preprocessing or post-processing step in image segmentation pipelines to enhance boundary accuracy while reducing noise that could otherwise lead to over-segmentation or boundary blurring. In mean-shift segmentation, bilateral filtering or anisotropic diffusion is integrated to smooth intra-region variations without crossing edges, enabling more coherent mode detection and improved partitioning of homogeneous areas. For graph-cut methods, edge-preserving smoothing refines the energy minimization by providing a cleaner affinity graph, where smoothed images yield sharper cuts that align with true object boundaries, as demonstrated in retinal image segmentation workflows combining K-means clustering with graph cuts. This integration preserves structural details, resulting in more robust superpixel generation for subsequent partitioning tasks. In video denoising, edge-preserving smoothing extends to temporal domains through spatio-temporal bilateral filters, which incorporate motion compensation via optical flow to align frames and apply weights across space, intensity, and time. These filters use a temporal window of 3-5 frames to exploit redundancy while avoiding artifacts from occlusions or rapid motion, with optical flow estimation (e.g., TV-L1 or PWC-Net) warping adjacent frames for consistent denoising. For instance, joint learning frameworks fine-tune denoising networks alongside flow estimators, achieving improvements in PSNR over frame-independent methods on noisy video sequences with additive white Gaussian noise. Such temporal extensions maintain edge sharpness across frames, supporting applications in surveillance where dynamic content requires real-time processing. Recent advancements since 2017 have focused on hybrid integrations with deep learning, such as CNN-guided filters that leverage U-Net architectures to automate parameter selection in bilateral filtering, particularly for medical imaging like PET scans where respiratory gating demands case-specific edge preservation. These hybrids achieve significant noise reduction while preserving standardized uptake values with minimal bias compared to manual tuning, enabling seamless incorporation into motion correction pipelines without extensive hyperparameter adjustment. Additionally, robust M-estimation techniques, like the fast M-estimation Gaussian filter with automatic basic width determination, enhance texture removal in denoising by adaptively handling salt-and-pepper noise, outperforming traditional nonlocal means and adaptive bilateral filters in edge retention for super-resolution preprocessing. Nonlocal means methods benefit from edge-aware patch matching, where randomized search (e.g., PatchMatch) combined with guided or cross-bilateral filtering propagates labels across superpixels, improving preservation in textureless regions and reducing errors in stereo matching tasks. In hybrid setups, learning rates around 10^{-3} to 10^{-4} are tuned for CNN components to balance convergence and edge fidelity during training on diverse datasets. These integrations offer key advantages, including gains in segmentation Intersection over Union (IoU) metrics—for example, edge-preserving downsampling boosts IoU by 8.65% at 1/4 resolution and 11.89% at 1/8 resolution in CT datasets^[53]—by mitigating noise-induced boundary shifts in downstream tasks like object detection. However, they introduce limitations such as increased computational complexity from global optimizations or flow estimation, often requiring seconds per frame, which poses challenges for real-time video applications and necessitates approximations like superpixel-based acceleration.

References

[1]
Edge-preserving smoothing filter using fast M-estimation method ...
Apr 4, 2023 · Edge-preserving smoothing filter using fast M-estimation method with an automatic determination algorithm for basic width. Yudai Yamaguchi ...
[2]
https://arxiv.org/pdf/1503.07297.pdf
[3]
[PDF] Scale-space and edge detection using anisotropic diffusion
Abstract The scale-space technique introduced by Witkin involves generating coarser resolution images by convolving the original image with a Gaussian kernel.
[4]
[PDF] Bilateral Filtering for Gray and Color Images
Bilateral filtering smooths images while preserving edges by combining nearby image values based on geometric and photometric similarity, using a non-iterative ...
[5]
[PDF] Evaluation and Comparison of Edge-Preserving Filters - arXiv
Dec 26, 2020 · Also called edge-preserving smoothing operators, they form a pivotal step in many algorithms for image manipulation including tonemapping, image ...
[6]
Scale-space and edge detection using anisotropic diffusion
Abstract: A new definition of scale-space is suggested, and a class of algorithms used to realize a diffusion process is introduced.
[7]
https://www.semanticscholar.org/paper/Edge-preserving-smoothing-Nagao-Matsuyama/fa241ee45ef756606eda9aba25211a95f400ac34
[8]
Edge preserving smoothing | Semantic Scholar
Edge preserving smoothing · M. Nagao, T. Matsuyama; Published 1 April 1979 · Published 1 April 1979 · 1979; Computer Science · Computer Graphics and Image ...
[9]
[PDF] Enhanced Directional Smoothing Algorithm for Edge-Preserving ...
We demonstrate the effectiveness of this new filtering method by comparing it to established speckle noise removal techniques on SAR images. Keywords: ...
[10]
https://ieeexplore.ieee.org/document/9230635
[11]
https://arxiv.org/pdf/2402.16221
[12]
[PDF] Integrating Preprocessing Methods and Convolutional Neural ... - arXiv
Feb 25, 2024 · Bilateral Filtering: To improve edge preservation, bilateral filtering is applied, enhancing the distinction between the tumor and surrounding ...
[13]
Stylization and abstraction of photographs - ACM Digital Library
Our system transforms images into a line-drawing style using bold edges and large regions of constant color. To do this, it represents images as a hierarchical ...Abstract · Information & Contributors · Index Terms<|separator|>
[14]
Detail and color enhancement in photo stylization
In this paper, we propose an automatic system for photo manipulation that brightens an image and alters the detail levels. The process first applies an edge- ...Abstract · Information & Contributors · Published In
[15]
Design and real-time implementation of a 3-D rational filter for edge ...
A nonlinear spatio-temporal filter capable of significantly attenuating noise in image sequences without corrupting image details is presented.
[16]
[PDF] Generation of High Dynamic Range Illumination from a ... - arXiv
Aug 2, 2017 · The input image is first decom- posed into illumination and reflectance components by using an edge-preserving smoothing filter. Then the ...
[17]
https://ieeexplore.ieee.org/document/1163708
[18]
http://mesh.brown.edu/engn1610/refs/PerreaultHebert-tip2007.pdf
[19]
[PDF] Median Filtering in Constant Time
Abstract— The median filter is one of the basic building blocks in many image processing situations. However, its use has long been hampered.
[20]
Nonlinear (nonsuperposable) methods for smoothing data
The proposed filter, RM, which is based on exact radial basis function artificial neural networks achieves a superior performance to the other filters ...
[21]
Bilateral filtering for gray and color images - IEEE Xplore
Bilateral filtering smooths images while preserving edges using a nonlinear combination of nearby image values, combining geometric and photometric similarity.
[22]
[PDF] Bilateral Filtering: Theory and Applications - People | MIT CSAIL
Abstract. The bilateral filter is a non-linear technique that can blur an image while respecting strong edges. Its ability to decompose an image into.
[23]
[PDF] Digital Photography with Flash and No-Flash Image Pairs
The joint bilateral filter relies on the flash image as an estimator ... The results of denoising with the joint bilateral filter are shown in Figure 4(c).
[24]
[PDF] A Gentle Introduction to Bilateral Filtering and its Applications
The bilateral filter is technique to smooth images while preserving edges. It can be traced back to 1995 with the work of Aurich and Weule [1995] on nonlinear ...
[25]
[PDF] Fast Bilateral Filtering By Adapting Block Size
ABSTRACT. Direct implementations of bilateral filtering show O(r2) com- putational complexity per pixel, where r is the filter window radius.
[26]
[PDF] Anisotropic Diffusion in Image Processing
The second goal of this book is to present an in-depth treatment of an interest- ing class of parabolic equations which may bridge the gap between scale-space ...
[27]
An efficient anisotropic diffusion model for image denoising with ...
Jul 1, 2021 · For many image applications, Perona and Malik [15] introduced an anisotropic diffusion ... values of k = 15, 25, and 35 respectively.
[28]
The Perona--Malik Paradox | SIAM Journal on Applied Mathematics
The Perona--Malik equation is a formally ill-posed parabolic equation for which simple discretizations are nevertheless numerically found to be stable.
[29]
Image Selective Smoothing and Edge Detection by Nonlinear ...
Image Selective Smoothing and Edge Detection by Nonlinear Diffusion. Authors: Francine Catté, Pierre-Louis Lions, Jean-Michel Morel, and Tomeu CollAuthors Info ...
[30]
[PDF] Robust Anisotropic Diffusion - Center for Neural Science
Abstract—Relations between anisotropic diffusion and robust statistics are described in this paper. Specifically, we show that anisotropic diffusion can be ...<|separator|>
[31]
[PDF] Guided Image Filtering - People | MIT CSAIL
In this paper we propose a new type of explicit image filter, called guided filter. The filtering output is locally a linear transform of the guidance image.
[32]
Nonlinear total variation based noise removal algorithms
Abstract. A constrained optimization type of numerical algorithm for removing noise from images is presented. The total variation of the image is minimized ...Missing: paper | Show results with:paper
[33]
An Algorithm for Total Variation Minimization and Applications
We propose an algorithm for minimizing the total variation of an image, and provide a proof of convergence. We show applications to image denoising, zooming,
[34]
Nonlocal Operators with Applications to Image Processing
Nonlocal operators are used to define new flows and functionals for image processing, handling textures and repetitive structures better than classical methods.
[35]
A First-Order Primal-Dual Algorithm for Convex Problems with ...
Dec 21, 2010 · In this paper we study a first-order primal-dual algorithm for non-smooth convex optimization problems with known saddle-point structure.
[36]
[PDF] Conjugate Gradient Acceleration of Non-Linear Smoothing Filters
Dec 14, 2015 · The most efficient signal edge-preserving smoothing filters, e.g., for denoising, are non-linear. Thus, their acceleration is challenging and is ...
[37]
Graph Spectral Image Smoothing - SpringerLink
We then compute the associated Laplacian matrix (the degree matrix minus the adjacency matrix). Anisotropic diffusion across this weighted graph-structure ...
[38]
[PDF] Graph Spectral Image Processing
Relationship between edge-preserving smoothing and image retargeting as signal processing on nonuniform grid. Graph signal processing corresponds to the ...
[39]
[PDF] Accelerated filtering on graphs using Lanczos method
This method, based on Cheby- shev polynomial approximation, only requires multiple application of the Laplacian operator and thus leads to a scalable, efficient ...
[40]
[PDF] Accelerated graph-based spectral polynomial filters
Sep 17, 2015 · LOBPCG is a very efficient eigensolver and gives a fast solution to the spectral image segmentation problem, see [15]. To avoid over ...
[41]
[PDF] Parallel proximal methods for total variation minimization - arXiv
Jan 4, 2016 · It is often interpreted as a sparsity-promoting `1-penalty on the image gradient. TV regularization has proven to be successful in a wide range ...
[42]
Image smoothing in the spatial domain using multigrid conjugate ...
In this paper, we present an alternative implementation for an existing image smoothing algorithm using a spatial iterative method, multigrid conjugate gradient ...
[43]
Locally Optimal Block Preconditioned Conjugate Gradient Method
We describe new algorithms of the locally optimal block preconditioned conjugate gradient (LOBPCG) method for symmetric eigenvalue problems.
[44]
Feature-Oriented Image Enhancement Using Shock Filters
Shock filters for image enhancement are developed. The filters use new nonlinear time dependent partial differential equations and their discretizations.
[45]
Joint bilateral upsampling | ACM Transactions on Graphics
A joint bilateral upsampling procedure to produce a better high resolution solution. We show results for each of the applications above and compare them to ...
[46]
https://epubs.siam.org/doi/10.1137/0727053
[47]
[PDF] Fast End-to-End Trainable Guided Filter - CVF Open Access
The guided filter is a deep learning building block for joint upsampling, generating high-resolution output from low-resolution input and a guidance map, and ...