Convergence Rate of Runge-Kutta-Type Regularization for Nonlinear Ill-Posed Problems under Logarithmic Source Condition

Let X and Y be infinite-dimensional real Hilbert spaces with inner products $\langle ·,·\rangle$ and norms $\parallel ·\parallel$. Let us consider a nonlinear ill-posed operator equation where $F:D\left(F\right)\subset X\to Y$ is a nonlinear operator between the Hilbert spaces X and Y. We assume that (1) has a solution $w^{+}$ for exact data (which need not be unique). We have approximate data $g^{\epsilon }$ with

Besides the classical Tikhonov–Phillips regularization, a plethora of interesting variational and iterative approaches for ill-posed problems can be found, e.g., in Morozov [1], Tikhonov and Arsenin [2], Bakushinsky and Kokurin [3], and Kaltenbacher et al. [4]. We focus here on iterative methods, as they are also very popular and effective to use in applications. The simplest iterative regularization is the Landweber method—see, e.g., Hanke et al. [5], where the analysis for convergence rates is done under Hölder-type source condition. A more effective method often used in applications is the Levenberg–Marquardt method

This was investigated in [6,7,8] under the Hölder-type source condition (HSC) and a posteriori discrepancy principle (DP). Jin [7] proved optimal convergence rates for an a priori chosen geometric step size sequence ${\alpha }_k$, whereas Hochbruck and Hönig [6] showed convergence with the optimal rate for quite general step size sequences including the geometric sequence. Later, Hanke [8] avoided any constraints on the rate of decay of the regularization parameter to show the optimal convergence rate.

Tautenhahn [9] proved that asymptotic regularization, i.e., the approximation of problem (1) by a solution of the Showalter differential equation (SDE) where the regularization parameter T is chosen according to the DP under the HSC, is a stable method to solve nonlinear ill-posed problems.

Solving SDE by the family of Runge-Kutta (RK) methods delivers a family of RK-type iterative regularization methods where $𝟙$ denotes the $(s\times 1)$ vector of identity operators, while $\delta$ is the $(s\times s)$ diagonal matrix of bounded linear operators with identity operator on the entire diagonal and zero operator outside of the main diagonal with respect to the appropriate spaces. The parameter ${\tau }_k=1/{\alpha }_k$ in (5) is the step-length, also called the relaxation parameter. The $(s\times s)$ matrix A and the $(s\times 1)$ vector b are the given parameters that correspond to the specific RK method, building the so-called Butcher tableau (succession of stages). Different choices of the RK parameters generate various iterative methods.

Böckmann and Pornsawad [10] showed convergence for the whole RK-type family (including the well-known Landweber and the Levenberg–Marquardt methods). That paper also emphasized advantages of using some procedures from the mentioned family, e.g., regarding implicit A-stable Butcher tableaux. For instance, the Landweber method needs a lot of iteration steps, while those implicit methods need only a few iteration steps, thus minimizing the rounding errors.

Later, Pornsawad and Böckmann [11] filled in the missing results on optimal convergence rates, but only for particular first-stage methods under HSC and using DP.

Our current study considers further the unifying RK-framework described above, as well as a modified version presented below, showing optimality of the RK-regularization schemes under logarithmic source conditions.

An additional term ${\alpha }_k(w_k^{\epsilon }-\xi )$, as in the iteratively regularized Gauss–Newton method (see, e.g., [4]), was added to a modified Landweber method. Thus, Scherzer [12] proved a convergence rate result under HSC without particular assumptions on the nonlinearity of operator F. Moreover, in Pornsawad and Böckmann [13], an the additional term was included to the whole family of iterative RK-type methods (which contains the modified Landweber iteration), where $\zeta \in X$ and $\Pi =\delta +{\tau }_{k}A{F}^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\epsilon }\right)$. Using a priori and a posteriori stopping rules, the convergence rate results of the RK-type family are obtained under HSC if the Fréchet derivative is properly scaled.

Due to the minimal assumptions for the convergence analysis of the modified iterative RK-type methods, an additional term was added to SDE

Pornsawad et al. [14] investigated this continuous version of the modified iterative RK-type methods for nonlinear inverse ill-posed problems. The convergence analysis yields the optimal rate of convergence under a modified DP and an exponential source condition.

Recently, a second-order asymptotic regularization for the linear problem $\widehat{A}x=y$ was investigated in [15] under HSC using DP.

Define with $p>0$ and the usual logarithmic sourcewise representation where $∥v∥$ is sufficiently small and $w_0\in D\left(F\right)$ is an initial guess that may incorporate a priori knowledge on the solution.

In numerous applications—e.g., heat conduction, scattering theory, which are severely ill-posed problems—the Hölder source condition is far too strong. Therefore, Hohage [16] proved convergence and logarithmic convergence rates for the iteratively regularized Gauss–Newton method in a Hilbert space setting, provided a logarithmic source condition (9) is satisfied and DP is used as the stopping rule. Deuflhard et al. [17] showed some convergence rate result for the Landweber iteration using DP and (9) under a Newton–Mysovskii condition on the nonlinear operator. Sufficient conditions for the convergence rate, which is logarithmic in the data noise level, are given.

In Hohage [18], a systematic study of convergence rates for regularization methods under (9) including the case of operator approximations for a priori and a posteriori stopping rules is provided. A logarithmic source condition is considered by Pereverzyev et al. [19] for a derivative-free method, by Mahale and Nair [20] for a simplified generalized Gauss–Newton method, and by Böckmann et al. [21] for the Levenberg–Marquardt method using DP as stopping rule. Pornsawad et al. [22] solved the inverse potential problem, which is exponentially ill-posed, employing the modified Landweber method and proved convergence rate under the logarithmic source condition via DP for this method.

To the best of our knowledge, for the first time, convergence rates are established both for the whole family of RK-methods and for the modified version, when applied to severely ill-posed problems (i.e., under the logarithmic source condition).

The structure of this article is as follows. Section 2 provides assumptions and technical estimations. We derive the convergence rate of the RK-type method (5) in Section 3 and of the modified RK-type method (6) in Section 4 under the logarithmic source conditions (8) and (9). In Section 5, the performed numerical experiments confirm the theoretical results.

Let $e_k:=w^{+}-w_k^{\epsilon }$ be the error of the kth iteration $w_k^{\epsilon }$, $S_k:=F^{\prime }\left(w_k^{\epsilon }\right)$ and $S:=F^{\prime }\left(w^{+}\right)$.

The proof is given in [22] using the mean value theorem.

We note that the explicit Euler method provides $\parallel I-\tau f(-\tau K{K}^{*})\parallel \le |1-\tau |$ and $\parallel I+\tau f(-\tau K{K}^{*})\parallel \le |1+\tau |$. Thus, the conditions (19) and (20) hold if $\tau$ is bounded. For the implicit Euler method, we have and for some positive number ${\lambda }_0$. We observe from Figure 1 that the conditions (19) and (20) hold for $\tau >0$.

Finally, we need a technical result for the next two sections.

To investigate the convergence rate of the RK-type regularization method (5) under the logarithmic source condition, the nonlinear operator F has to satisfy the local property in an open ball $B_{\rho }\left(w_0\right)$ of radius $\rho$ around $w_0$ with $w,\tilde{w}$∈$B_{\rho }\left(w_0\right)$$\subset \mathcal{D}\left(F\right)$. In addition, the regularization parameter $k_{*}$ is chosen according to the generalized discrepancy principle, i.e., the iteration is stopped after $k_{*}$ steps with where $\gamma >\frac{2-\eta }{1-\eta }$ is a positive number. Note that the triangle inequality yields

In the sequel, we establish an error estimate that will be useful in deriving the logarithmic convergence rate.