M. J. D.
Powell
Boston, MA
Springer US
Numerical and Computational Mathematics
3-28
The use of Band Matrices for Second Derivative Approximations in Trust Region Algorithms
chapter
chapters
2019-04-16T09:43
https://link.springer.com/10.1007%2F978-1-4613-3335-7_1
en
https://scigraph.springernature.com/explorer/license/
1998-01-01
1998
In many trust region algorithms for optimization calculations, each iteration seeks a vectord ∈ Rn that solves the linear system of equations (B +λI)d = -g, whereB is a symmetric estimate of a second derivative matrix, I is the unit matrix,g is a known gradient vector, and λ is a parameter that controls the length ofd. Several values of λ may be tried on each iteration, and, when there is no helpful sparsity inB, it is usual for each solution to require O(n3) operations. However, if an orthogonal matrix Ω, is available such thatM =ΩTBΩ is an nxn matrix of bandwidth 2s+1, thenΩTd can be calculated in onlyO(ns2) operations for each new λ, by writing the system in the form (M+λI)(ΩTd) = — ΩTg. We find, unfortunately, that the construction ofM and Ω fromB is usually more expensive than the solution of the original system, but in variable metric and quasi-Newton algorithms for unconstrained optimization, each iteration changesB by a matrix whose rank is at most two, and then updating techniques can be applied to Ω. Thus it is possible to reduce the average work per iteration fromO(n3) to O(n7/3) operations. Here the elements of each orthogonal matrix are calculated explicitly, but instead one can express the orthogonal matrix updates as products of Givens rotations, which allows the average work per iteration to be only O(n11/5) operations. Details of procedures that achieve these savings are described, and the O(n7/3) complexity is confirmed by numerical results.
true
Mathematical Sciences
Yuan
Ya-xiang
Springer Nature - SN SciGraph project
Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Silver Street, CB3 9EW, Cambridge, England
University of Cambridge
978-1-4613-3337-1
Advances in Nonlinear Programming
978-1-4613-3335-7
pub.1033777821
dimensions_id
doi
10.1007/978-1-4613-3335-7_1
readcube_id
a0e7c6594a1ed4d4b32a22506a804da845a846b4df37f8ef1fbb4bf6554ef115