In this paper, we propose a self adaptive spectral conjugate gradient-based projection method for systems of nonlinear monotone equations. Based on its modest memory requirement and its efficiency, the method is suitable for solving large-scale equations. We show that the method satisfies the descent condition \(F_{k}^{T}d_{k}\leq -c\|F_{k}\|^{2}, c>0\), and also prove its global convergence. The method is compared to other existing methods on a set of benchmark test problems and results show that the method is very efficient and therefore promising.
Introduction
In this paper, we focus on solving large-scale nonlinear system of equations
$$\begin{array}{*{20}l} F(x) = 0, \end{array} $$
(1)
where \(F:\mathbb {R}^{n}\rightarrow \mathbb {R}^{n}\) is continuous and monotone. A function F is monotone if it satisfies the monotonicity condition
Nonlinear monotone equations arise in many practical applications, for example, chemical equilibrium systems [1], economic equilibrium problems [2], and some monotone variational inequality problems [3]. A number of computational methods have been proposed to solve nonlinear equations. Among them, Newton’s method, quasi-Newton method, Gauss-Newton method, and their variants are very popular due to their local superlinear convergence property (see, for example, [4–9]). However, they are not suitable for large-scale nonlinear monotone equations as they need to solve a linear system of equations using the second derivative information (Jacobian matrix or an approximation of it).
Due to their modest memory requirements, conjugate gradient-based projection methods are suitable for solving large-scale nonlinear monotone equations (1). Conjugate gradient-based projection methods generate a sequence {xk} by exploring the monotonicity of the function F. Let zk=xk+αkdk, where αk>0 is the step length that is determined by some line search and
This projection concept on the hyperplane Hk was first presented by Solodov and Svaiter [10].
Following Solodov and Svaiter [10], a lot of work has been done, and continues to be done, to come up with a number of conjugate gradient-based projection methods for nonlinear monotone equations. For example, Hu and Wei [11] proposed a conjugate gradient-based projection method for nonlinear monotone equations (1) where the search direction dk is given as
Recently, three term conjugate gradient-based projection methods have also been presented. One such method is that by Feng et al. [12] who presented their direction as
where \(|\beta _{k}|\leq t\frac {\|F_{k}\|}{\|d_{k-1}\|},\,\, \forall k\geq 1\), and t>0 is a constant. The global convergence of this method was also established using the line search (4). For other conjugate gradient-based projection methods, the reader is referred to [13–27].
In this paper, following the work of Abubakar and Kumam [21], Hu and Wei [11] and that of Liu and Li [22], we propose a self adaptive spectral conjugate gradient-based projection method for solving systems of nonlinear monotone Eq. (1). This method is presented in the next section and the rest of the paper is organized as follows. In “Convergence analysis” section, we show that the proposed method satisfies the descent property \(F_{k}^{T}d_{k}\leq -c\|F_{k}\|^{2}, c>0\), and also establish its global convergence. In “Numerical experiments” section, we present the numerical results and lastly, conclusion is presented in “Conclusion” section.
Algorithm
In this section, we give the details of the proposed method. We start by briefly reviewing the work of Abubakar and Kumam [21] and that of Liu and Li [22].
Most recently, Abubakar and Kumam [21] proposed the direction
and sk−1=zk−1−xk−1=αk−1dk−1. This method was shown to perform well numerically and its global convergence was established using line search (4). In 2015, Liu and Li [22] proposed a spectral DY-type projection method for nonlinear monotone system of Eq. (1) with the search direction dk as
where\(\beta _{k}^{DY}=\frac {\|F_{k}\|^{2}}{d_{k-1}^{T}u_{k-1}}\), uk−1=yk−1+tdk−1, \(t=1+\max \left \{0,-\frac {d_{k-1}^{T}y_{k-1}}{d_{k-1}^{T}d_{k-1}}\right \}\), yk−1=Fk−Fk−1+rsk−1 with sk−1=xk−xk−1, r>0 being a constant and \(\lambda _{k}=\frac {s_{k-1}^{T}s_{k-1}}{s_{k-1}^{T}y_{k-1}}\). The global convergence of this method was established using the line search
with η>0 being a constant and the parameters \(\lambda _{k}^{*}=\frac {s_{k-1}^{T}y_{k-1}}{s_{k-1}^{T}s_{k-1}}\) and \(\mu _{k}>\frac {1}{\lambda _{k}^{*}}\) where sk−1=xk−xk−1 and yk−1=Fk−Fk−1+rsk−1, r∈(0,1). With dk defined by (6), (7), and (8), we now present our algorithm.
Throughout this paper, we assume that the following assumption holds.
Assumption 1
(i) The function F(·) is monotone on \(\mathbb {R}^{n}\), i.e. \((F(x)-F(y))^{T}(x-y) \geq 0, \forall x,y\in \mathbb {R}^{n}\). (ii) The solution set of (1) is nonempty. (iii) The function F(·) is Lipschitz continuous on \(\mathbb {R}^{n}\), i.e. there exists a positive constant L such that
which contradicts Lemma 2. So, the steplength αk is well defined and can be determined within a finite number of trials. Now, we prove inequality (17). If αk≠κ, then \(\alpha '_{k}=\frac {\alpha _{k}}{\rho }\) does not satisfy (5), that is
The following lemma shows that if the sequence {xk} is generated by Algorithm 1, and x∗ is a solution of (1), i.e. F(x∗)=0, then the sequence {∥xk−x∗∥} is decreasing and convergent. Thus, the sequence {xk} is bounded.
Lemma 5
Suppose Assumption 1 holds and the sequence {xk} is generated by Algorithm 1. For any x∗ such that F(x∗)=0, we have that
In this section, results of our proposed method SASCGM are presented together with those of improved three-term derivative-free method (ITDM) [21], the modified Liu-Storey conjugate gradient projection (MLS) method [11], and the spectral DY-type projection method (SDYP) [22]. All algorithms are coded in MATLAB R2016a. In our experiments, we set ε=10−4, i.e., the algorithms are stopped whenever the inequality ∥Fk∥≤10−4 is satisfied, or the total number of iterations exceeds 1000. The method SASCGM is implemented with the parameters σ=10−4, ρ=0.5, r=10−3, \(\mu _{k}=\frac {1}{\lambda _{k}^{*}}+0.1\) and κ=1, while parameters for algorithms ITDM, MLS, and SDYP are set as in respective papers.
The methods are compared using number of iterations, number of function evaluations and CPU time taken for each method to reach the optimal value or termination. We test the algorithms on ten (10) test problems with their dimensions varied from 5000 to 20000, and with four (4) different starting points \(x_{0}=\left (\frac {1}{n},\frac {1}{n},\ldots,\frac {1}{n}\right)^{T}\), x1=(−1,−1,…,−1)T, x2=(0.5,0.5,…,0.5)T and x3=(−0.5,−0.5,…,−0.5)T. The test functions are listed as follows:
Problem 1. Sun and Liu [19] The mapping F is given by
The numerical results are reported in Tables 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10, where “SP” represents the starting point (initial point), “DIM” denotes the dimension of the problem, “NI” refers to the number of iterations, “NFE” stands for the number of function evaluations, and “CPU” is the CPU time in seconds. In Table 3, “*” indicates that the algorithm did not converge within the maximum number of iterations. From the tables, we observe that the proposed method performs better than the other methods in Problems 2, 3, 4, 6, 9, and 10. The proposed method performs slightly lower in Problems 1, 5, 7, and 8. However, overall, the proposed method shows that it is very competitive with the other methods and can be a good addition to the existing methods in the literature.
The performance of the three methods is further presented graphically in Figs. 1, 2, and 3 based on the number of iterations (NI), number of function evaluations (NFE), and the CPU time, respectively, using the performance profile of Dolan and Mor\(\acute {e}\) [28]. That is, we plot the probability ρS(τ) of the test problems for which each of the three methods was within a factor τ. Figures 1, 2, and 3 clearly show the efficiency of the proposed SASCGM method as compared to the other three methods.
In this paper, we proposed a self adaptive spectral conjugate gradient-based projection (SASCGM) method for solving systems of large-scale nonlinear monotone equations. The proposed method is free from derivative evaluations and also satisfies the descent condition \(F_{k}^{T}d_{k}\leq -c\|F_{k}\|^{2}, c>0\), independent of any line search. The global convergence of the proposed method was also established. The proposed algorithm was tested on some benchmark problems with different initial points and different dimensions and the numerical results show that the method is competitive.
Availability of data and materials
All data generated or analyzed during this study are included in this manuscript.
Zhao, Y. B., Li, D.: Monotonicity of fixed point and normal mapping associated with variational inequality and its applications. SIAM J. Optim. 11, 962–973 (2001).
Dehghan, M., Hajarian, M.: New iterative method for solving non-linear equations with fourth-order convergence. Int. J. Comput. Math. 87, 834–839 (2010).
Li, D., Fukushima, M.: A global and superlinear convergent Gauss-Newton-based BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37, 152–172 (1999).
Dehghan, M., Hajarian, M.: On some cubic convergence iterative formulae without derivatives for solving nonlinear equations. Int. J. Numer. Methods Biomed. Eng. 27, 722–731 (2011).
Zhou, W. J., Liu, D. H.: A globally convergent BFGS method for nonlinear monotone equations without any merit functions. Math. Comput. 77, 2231–2240 (2008).
Solodov, M. V., Svaiter, B. F.: A globally convergent inexact newton method for systems of monotone equations, Reformulation: Nonsmooth, Piecewise Smooth, Semismoothing methods. Springer US (1998). https://doi.org/10.1007/978-1-4757-6388-1_18.
Ou, Y., Li, J.: A new derivative-free SCG-type projection method for nonlinear monotone equations with convex constraints. J. Appl. Math. Comput. 56, 195–216 (2018).
The authors appreciate the work of the referees, for their valuable comments and suggestions that led to the improvement of this paper. The authors would also like to thank Prof J. Liu who provided the MATLAB code for SDYP.
Funding
None.
Author information
Authors and Affiliations
Department of Mathematics, University of Botswana, Gaborone, Private Bag UB00704, Botswana
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.