Menu
Home Explore People Places Arts History Plants & Animals Science Life & Culture Technology
On this page
Truncated Newton method

The truncated Newton method, originated in a paper by Ron Dembo and Trond Steihaug, also known as Hessian-free optimization, are a family of optimization algorithms designed for optimizing non-linear functions with large numbers of independent variables. A truncated Newton method consists of repeated application of an iterative optimization algorithm to approximately solve Newton's equations, to determine an update to the function's parameters. The inner solver is truncated, i.e., run for only a limited number of iterations. It follows that, for truncated Newton methods to work, the inner solver needs to produce a good approximation in a finite number of iterations; conjugate gradient has been suggested and evaluated as a candidate inner loop. Another prerequisite is good preconditioning for the inner algorithm.

We don't have any images related to Truncated Newton method yet.
We don't have any YouTube videos related to Truncated Newton method yet.
We don't have any PDF documents related to Truncated Newton method yet.
We don't have any Books related to Truncated Newton method yet.
We don't have any archived web articles related to Truncated Newton method yet.

Further reading

  • Grippo, L.; Lampariello, F.; Lucidi, S. (1989). "A Truncated Newton Method with Nonmonotone Line Search for Unconstrained Optimization". J. Optimization Theory and Applications. 60 (3): 401–419. CiteSeerX 10.1.1.455.7495. doi:10.1007/BF00940345. S2CID 18990650.
  • Nash, Stephen G.; Nocedal, Jorge (1991). "A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization". SIAM J. Optim. 1 (3): 358–372. CiteSeerX 10.1.1.474.3400. doi:10.1137/0801023.

References

  1. Dembo, Ron S.; Steihaug, Trond (1983). "Truncated-Newton algorithms for large-scale unconstrained optimization". Mathematical Programming. 26 (2). Springer: 190–212. doi:10.1007/BF02592055. S2CID 40537623.. Convergence results for this algorithm can be found in Dembo, Ron S.; Eisenstat, Stanley C.; Steihaug, Trond (1982). "Inexact newton methods". SIAM Journal on Numerical Analysis. 19 (2): 400–408. Bibcode:1982SJNA...19..400D. doi:10.1137/0719025. JSTOR 2156954.. /wiki/Doi_(identifier) ↩

  2. Martens, James (2010). Deep learning via Hessian-free optimization (PDF). Proc. International Conference on Machine Learning. http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf ↩

  3. Nash, Stephen G. (2000). "A survey of truncated-Newton methods". Journal of Computational and Applied Mathematics. 124 (1–2): 45–59. Bibcode:2000JCoAM.124...45N. doi:10.1016/S0377-0427(00)00426-X. https://doi.org/10.1016%2FS0377-0427%2800%2900426-X ↩

  4. Martens, James (2010). Deep learning via Hessian-free optimization (PDF). Proc. International Conference on Machine Learning. http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf ↩

  5. Nash, Stephen G. (1985). "Preconditioning of truncated-Newton methods" (PDF). SIAM J. Sci. Stat. Comput. 6 (3): 599–616. doi:10.1137/0906042. http://www.ii.uib.no/~lennart/drgrad/Nash1985.pdf ↩