From Newton to Present Days: Minimization of Nonlinear Functions of Several Variables Using Two Criterions, Minimal Function and Minimal Gradient Norm on Anti-gradient Directions

The sequential unconstrained method is suggested and explicated for minimization of nonlinear functions of several variables. The method develops and expands the steepest descent method and uses analytical characteristics of functions without breaking the search. The method uses two optimal points on anti-gradient directions: the Cauchy point with minimal value of function and mini-norm point with minimal value of its norm of gradient. Numerical analysis is made for two functions of two variables to illustrate how the optimal points are collocated on different directions.

The mathematical analysis is made for quadratic functions in canonic form with positive coefficients and six properties are proved, which are used in the algorithm. The program is written in C computer language. The method is checked on twelve test functions with calculations of first partial derivatives by analytical formulas and finite differences that are shown in table 1. A brief historical review of gradients and Newton's methods is represented in the discussion section of the book.