Mapping-Based Image Diffusion

Abstract

In this work, we introduce a novel tensor-based functional for targeted image enhancement and denoising. Via explicit regularization, our formulation incorporates application-dependent and contextual information using first principles. Few works in literature treat variational models that describe both application-dependent information and contextual knowledge of the denoising problem. We prove the existence of a minimizer and present results on tensor symmetry constraints, convexity, and geometric interpretation of the proposed functional. We show that our framework excels in applications where nonlinear functions are present such as in gamma correction and targeted value range filtering. We also study general denoising performance where we show comparable results to dedicated PDE-based state-of-the-art methods.

Appendices

Appendix 1: Proof of Theorem 1

To prove Theorem 1, compute the variational derivative of the functional (28). The first variation is given by

$$\begin{aligned} \delta R = \left. \frac{\partial }{\partial \varepsilon } R(u+\varepsilon v) \right| _{\varepsilon = 0}. \end{aligned}$$

(82)

Now, we let $u \mapsto u+\varepsilon v$ in R(u) and we get

$$\begin{aligned} R(u+\varepsilon v)&= \nabla m(u+\varepsilon v)^{\top }W(\nabla m(u+\varepsilon v)) \nabla m(u+\varepsilon v) \nonumber \\&= \underbrace{m'(u+\varepsilon v)^2\nabla (u+\varepsilon v)^{\top }}_{=A} \end{aligned}$$

(83a)

$$\begin{aligned}&\qquad \cdot \underbrace{W(\nabla m(u+\varepsilon v)) \nabla (u+\varepsilon v)}_{=B} , \end{aligned}$$

(83b)

then, from the product rule of differentiation, we need to consider the following terms

$$\begin{aligned} \frac{\partial }{\partial \varepsilon } R(u+\varepsilon v)&= \left( \frac{\partial }{\partial \varepsilon } A(u+\varepsilon v) \right) B \nonumber \\&\qquad + A \left( \frac{\partial }{\partial \varepsilon } B(u+\varepsilon v) \right) . \end{aligned}$$

(84)

To simplify the notation define

$$\begin{aligned} z = \begin{pmatrix} z_1 = m'(u+\varepsilon v)(u_1 + \varepsilon v_1) \\ \vdots \\ z_d = m'(u+\varepsilon v)(u_d + \varepsilon v_d) \end{pmatrix}, \end{aligned}$$

(85)

and note the relation

$$\begin{aligned} z|_{\varepsilon = 0} = \nabla m(u) = s. \end{aligned}$$

(86)

We first consider the differentiation of the B-component w.r.t $\varepsilon$. In order to differentiate the B-component use the chain rule of differentiation:

$$\begin{aligned} \frac{\partial W(z)}{\partial \varepsilon } = \frac{\partial W(z)}{\partial z_1}\frac{\partial z_1}{\partial \varepsilon } + \cdots + \frac{\partial W(z)}{\partial z_d}\frac{\partial z_d}{\partial \varepsilon }, \end{aligned}$$

then

$$\begin{aligned} \frac{\partial }{\partial \varepsilon } B(z)= & {} \frac{\partial }{\partial \varepsilon } \left( W(z)\nabla (u+\varepsilon v) \right) \nonumber \\= & {} \left( \frac{\partial }{\partial \varepsilon } W(z) \right) \nabla (u+\varepsilon v) + W(z)\nabla v, \end{aligned}$$

(87)

with

$$\begin{aligned} \frac{\partial }{\partial \varepsilon } W(z)&= \Big [W_{z_1}(z) \Big (m''(u+\varepsilon v)v(u_1+\varepsilon v_1) \nonumber \\&\qquad \qquad + m'(u + \varepsilon v) v_1 \Big ) \nonumber \\&\qquad \vdots \nonumber \\&\qquad + W_{z_d}(z) \Big (m''(u+\varepsilon v)v(u_d+\varepsilon v_d) \nonumber \\&\qquad \qquad + m'(u + \varepsilon v) v_d \Big ) \Big ] , \end{aligned}$$

(88)

evaluating the limit $\varepsilon \rightarrow 0$ in (87) with (88) we get

$$\begin{aligned} \frac{\partial }{\partial \varepsilon } B(z)\bigg |_{\varepsilon = 0}&= \Big [W_{s_1}(s) \Big (m''(u)v u_1 + m'(u) v_1 \Big ) \nonumber \\&\qquad \vdots \nonumber \\&\quad + W_{s_d}(s) \Big (m''(u)v u_d + m'(u) v_d \Big ) \Big ] \nabla u \nonumber \\&\qquad + W(s)\nabla v . \end{aligned}$$

(89)

Next, we focus on the second product in (84). Then, with (89) and after few rewrites, we obtain the following

$$\begin{aligned}&\left. A \left( \frac{\partial }{\partial \varepsilon } B(u+\varepsilon v) \right) \right| _{\varepsilon = 0}\nonumber \\&\quad = m'(u)^2\nabla u^{\top } \Big [W_{s_1}(s) \Big (m''(u)v u_1 + m'(u) v_1 \Big ) \nonumber \\&\quad \qquad \vdots \nonumber \\&\quad \quad + W_{s_d}(s) \Big (m''(u)v u_d + m'(u) v_d \Big ) \Big ]\nabla u \nonumber \\&\quad \quad + m'(u)^2\nabla u^{\top } W(s)\nabla v \end{aligned}$$

(90a)

$$\begin{aligned}&\quad = m'(u)^2\Big [\nabla u^{\top } W_{s_1}(s) \nabla u \Big (m''(u)v u_1 + m'(u) v_1 \Big ) \nonumber \\&\qquad \quad \vdots \nonumber \\&\quad \quad + \nabla u^{\top } W_{s_d}(s) \nabla u \Big (m''(u)v u_d + m'(u) v_d \Big ) \Big ] \nonumber \\&\quad \quad + m'(u)^2\nabla u^{\top } W(s)\nabla v \end{aligned}$$

(90b)

$$\begin{aligned}&\quad = m'(u)^2\Big \langle Q , \Big (m''(u)v \nabla u + m'(u) \nabla v \Big ) \Big \rangle \nonumber \\&\quad \quad + m'(u)^2 \Big \langle \nabla u, W(s)\nabla v \Big \rangle \end{aligned}$$

(90c)

$$\begin{aligned}&\quad = \underbrace{ \Big \langle m'(u)^2 m''(u)Q, \nabla u\Big \rangle }_{=G} v \nonumber \\&\quad \quad + \Big \langle \nabla v, \Big (\underbrace{ m'(u)^3 Q + m'(u)^2W(s)^{\top }}_{=H} \Big ) \nabla u \Big \rangle , \end{aligned}$$

(90d)

where we set

$$\begin{aligned} Q = \begin{pmatrix} \nabla u^{\top } W_{s_1}(s) \nabla u \\ \vdots \\ \nabla u^{\top } W_{s_d}(s) \nabla u \end{pmatrix} = _{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} \nabla u, \end{aligned}$$

(91)

and $_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})}$ is defined in (26). The definitions G and H will be used in the application of Green’s formula (17).

Now we focus on the differentiation of the A-component w.r.t. $\varepsilon$ in (83a), i.e., we set $u \mapsto u+\varepsilon v$ and get

$$\begin{aligned}&\frac{\partial }{\partial \varepsilon } A(u+\varepsilon v) \nonumber \\&\quad = 2m'(u+\varepsilon v)m''(u+\varepsilon v) v \nabla (u+\varepsilon v)^{\top } \nonumber \\&\qquad + m'(u+\varepsilon v)^2 \nabla v^{\top } . \end{aligned}$$

(92)

With (92) we evaluate the limit $\varepsilon \rightarrow 0$ of (83a) which results in

$$\begin{aligned}&\left. \left( \frac{\partial }{\partial \varepsilon } A(u+\varepsilon v) \right) B \right| _{\varepsilon = 0} \nonumber \\&\quad = \Big ( 2m'(u)m''(u) v \nabla u^{\top } + m'(u)^2 \nabla v^{\top } \Big ) W(\nabla m(u)) \nabla u \nonumber \\&\quad = \left( \underbrace{2m'(u)m''(u) \nabla u^{\top } W(\nabla m(u)) \nabla u}_{E} \right) v \nonumber \\&\qquad + \nabla v^{\top } \left( \underbrace{m'(u)^2 W(\nabla m(u))}_{F} \right) \nabla u , \end{aligned}$$

(93)

where the definitions E and F will be used in the application of Green’s theorem. Summing (90d) and (93) and making use of G, H and E, F yields

$$\begin{aligned} \left. \frac{\partial }{\partial \varepsilon } R(u+\varepsilon v) \right| _{\varepsilon = 0}&= \int _\varOmega (G+E)v \; \hbox {d}{\varvec{x}} \nonumber \\&\quad + \int _\varOmega \nabla v^{\top } (H+F)\nabla u \; \hbox {d}{\varvec{x}}. \end{aligned}$$

(94)

The second integral of (94) is now on the form $\nabla v^\top A(\nabla u)$, i.e., we can integrate it w.r.t. $\nabla v$ by using Green’s formula (cmp. (17)) and get

$$\begin{aligned}&\int _\varOmega \nabla v^{\top } (H+F)\nabla u \; {\hbox {d}{\varvec{x}}} \nonumber \\&\quad = \int _{\partial \varOmega } v ({\varvec{n}}\cdot (H+F)\nabla u) \;\hbox {d}S\nonumber \\&\qquad - \int _{\varOmega } v {\text {div}} \left( (H+F)\nabla u \right) \;{\hbox {d}{\varvec{x}}} , \end{aligned}$$

where the natural Nuemann boundary condition is given by ${\varvec{n}}\cdot (H+F) = 0$. Substituting G, H, E, F defined in (90d) and (93) into the above expression, and after rearranging the terms and dropping the parentheses for improved clarity, we get

$$\begin{aligned} \left\{ \begin{array}{ll} m'm'' \nabla u^{\top } \Big [ 2W + _{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m' \Big ] \nabla u \\ - {\text {div}} \left( (m')^2 \Big ( W + W^{\top } + m' _{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} \Big )\nabla u \right) = 0 &{}\quad \hbox {in}\,\varOmega \\ n \cdot (F + H)\nabla u = 0 &{} \quad \hbox {on} \,\partial \varOmega \end{array} \right. \end{aligned}$$

(95)

The above E–L equation can be simplified by considering the following expansion

$$\begin{aligned}&{\text {div}} \left( (m')^2[2W+_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m']\nabla u \right) \nonumber \\&\qquad = 2m'm''\nabla u^{\top } [2W+_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m']\nabla u \nonumber \\&\quad \quad \quad + (m')^2{\text {div}} \left( (2W+_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m')\nabla u \right) . \end{aligned}$$

(96)

Substituting the first term of the right-hand side of (96) into the PDE of (95) we get:

$$\begin{aligned}&{\text {div}} \left( (m')^2[2W+_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m']\nabla u \right) \nonumber \\&\quad - (m')^2{\text {div}} \left( [2W+_{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} m']\nabla u \right) \nonumber \\&\quad - 2{\text {div}} \left( (m')^2 \Big [ W + W^{\top } + m' _{\nabla u}{W_{{\varvec{s}}}({{\varvec{s}}})} \Big ]\nabla u \right) = 0 . \end{aligned}$$

(97)

The final result is obtained after subtracting the last term from the first term in (97), which concludes the proof. $\square$

Appendix 2: Proof of Corollary 4

To derive the gradient energy tensor diffusion scheme we set W and D as specified in (59). With this relation between W and D we introduce E and set $E = 1/|\nabla m(u)|$, then $W = DE$. Before proceeding with the proof, we specify the components of GET. The components of GET (54) are

$$\begin{aligned} a&= (\partial _x u_x)^2 + (\partial _x u_y)^2 - u_x (\partial _{xx}u_x + \partial _{xy}u_y) \end{aligned}$$

(98a)

$$\begin{aligned} b&= (\partial _{x}u_x)(\partial _x u_y) + (\partial _{y}u_x)(\partial _y u_y) \nonumber \\&\quad - \frac{1}{2}( u_x (\partial _{yx}u_x + \partial _{yy}u_y ) + u_y(\partial _{xx}u_x + \partial _{xy}u_y))\end{aligned}$$

(98b)

$$\begin{aligned} c&= (\partial _y u_y)^2 + (\partial _y u_x)^2 - u_y (\partial _{yx}u_x + \partial _{yy}u_y) . \end{aligned}$$

(98c)

Now we need to compute $_{\nabla u}{W_{\nabla u}(\nabla u)}$ in (27) with ${\varvec{s}} = \nabla u$, that is, we have

$$\begin{aligned} _{\nabla u}{W_{\nabla u}(\nabla u)} = \begin{pmatrix} \nabla u^{\top } E_{u_x}D \\ \nabla u^{\top } E_{u_y}D \end{pmatrix} + E \begin{pmatrix} \nabla u^{\top } D_{u_x} \\ \nabla u^{\top } D_{u_y} \end{pmatrix} . \end{aligned}$$

(99)

The derivatives of E reads

$$\begin{aligned} E_{u_x}&= \partial _{u_x} \frac{1}{|\nabla u|} = - \frac{1}{|\nabla u|^3} u_x, \end{aligned}$$

(100a)

$$\begin{aligned} E_{u_y}&= \partial _{u_y} \frac{1}{|\nabla u|} = - \frac{1}{|\nabla u|^3} u_y. \end{aligned}$$

(100b)

The tensor $_{\nabla u}{W_{\nabla u}(\nabla u)}$ in the E–L scheme, can now be expressed as (62). $\square$

Appendix 3: Eigendecomposition

This part decomposes D in its eigendecomposition and computes $D_{u_x}$ and $D_{u_y}$. We have

$$\begin{aligned} D(\nabla u) = U \varLambda U^{\top } = \begin{pmatrix} v^2_1 &{} v_1v_2 \nonumber \\ v_1v_2 &{} v^2_2 \end{pmatrix}\lambda _1 + \begin{pmatrix} w^2_1 &{} w_1w_2 \\ w_1w_2 &{} w^2_2 \end{pmatrix}\lambda _2, \end{aligned}$$

where $\lambda _{1,2} = \exp (-|\mu _{1,2}|/k^2)$ and $\mu _1 = \frac{1}{2}( \mathrm {tr} \left( GET \right) + \alpha )$ and $\mu _2 = \frac{1}{2}(\mathrm {tr} \left( GET \right) - \alpha )$ are the eigenvalues of GET with $\alpha = \sqrt{(a-c)^2 + 4b^2}$. We obtain the eigenvectors of GET by solving $GET \tilde{v} = \mu _1 \tilde{v}$, i.e., we have the following equation system

$$\begin{aligned} \left\{ \begin{array}{l} (a - c - \alpha )\tilde{v}_1 + 2b\tilde{v}_2 = 0 \\ 2b\tilde{v}_1 + (c-a-\alpha )\tilde{v}_2 = 0. \end{array} \right. \end{aligned}$$

(101)

The orthonormal eigenvectors of GET are (67) and (68). Now, we focus on $\partial _{u_x} D$ in (66). After expanding the derivatives of the eigenvectors we obtain

$$\begin{aligned} \partial _{u_x} \tilde{v}_1&= -(\partial _{yx}u_x + \partial _{yy}u_y) , \end{aligned}$$

(102a)

$$\begin{aligned} \partial _{u_x} \tilde{v}_2&= \partial _{xx}u_x + \partial _{xy}u_y + \partial _{u_x} \alpha , \end{aligned}$$

(102b)

$$\begin{aligned} \partial _{u_x} \tilde{w}_1&= \partial _{u_x} \tilde{v}_1, \end{aligned}$$

(102c)

$$\begin{aligned} \partial _{u_x} \tilde{w}_2&= \partial _{xx}u_x + \partial _{xy}u_y - \partial _{u_x} \alpha , \end{aligned}$$

(102d)

and

$$\begin{aligned} \partial _{u_x} |\tilde{v}|&= |\tilde{v}|^{-1}(\tilde{v}_1(\partial _{u_x}\tilde{v}_1)+\tilde{v}_2(\partial _{u_x}\tilde{v}_2)), \end{aligned}$$

(103a)

$$\begin{aligned} \partial _{u_x} |\tilde{w}|&= |\tilde{w}|^{-1}(\tilde{w}_1(\partial _{u_x}\tilde{w}_1)+\tilde{w}_2(\partial _{u_x}\tilde{w}_2)). \end{aligned}$$

(103b)

The derivatives of the tensor’s eigenvalues are:

$$\begin{aligned} \partial _{u_x} \lambda _{1,2}&= \partial _{u_x} \exp (-|\mu _{1,2}|/k^2) \nonumber \\&= - \frac{\hbox {sgn}(\mu _{1,2})}{k^2}\Big (\partial _{u_x}\mathrm {tr} \left( GET \right) \pm \partial _{u_x}\alpha \Big )\lambda _{1,2}, \end{aligned}$$

(104)

where

$$\begin{aligned} \partial _{u_x}\alpha = \alpha ^{-1} \Big ( \mathrm {tr} \left( GET \right) \partial _{u_x}\mathrm {tr} \left( GET \right) - 2\partial _{u_x} \det (GET) \Big ) \end{aligned}$$

(105)

and

$$\begin{aligned} \partial _{u_x}\mathrm {tr} \left( GET \right)&= -(\partial _{xx}u_x + \partial _{xy}u_y), \end{aligned}$$

(106a)

$$\begin{aligned} \partial _{u_x} \det (GET)&= \partial _{u_x} (ac - b^2) \nonumber \\&= -(\partial _{xx}u_x + \partial _{xy}u_y)c \nonumber \\&\quad + b(\partial _{yx}u_x + \partial _{yy}u_y). \end{aligned}$$

(106b)

In the case $b=0$ the difference to the above derivation is that $\det (GET) = ac$, thus $\partial _{u_x} \alpha$ and $\partial _{u_x} {\mathrm {det}}(GET)$ should be modified accordingly. The component $\nabla u^{\top } D_{u_y}$ follows the same line of calculations and is therefore omitted here.