A survey of non-linear optimization techniques.

Optimization means the provision of a set of numerical parameter values which will give the best fit of an equation, or series of equations, to a set of data. For simple systems this can be done by differentiating the equations with respect to each parameter in turn, setting the set of partial differential equations to zero, and solving this set of simultaneous equations (as for exwnple in linear regression). In more complicated cases, however, it may be impossible to differentiate the equations, or very difficultly soluble non-linear equations may result. Many numerical optimization techniques to overcome these difficulties have been developed in the least ten years, and this review explains the logical basis of most of them, without going into the detail of computational procedures.The methods fall naturally into two classes - direct search methods, in which only values of the function to be minimized (or maximized) are used - and gradient methods, which also use derivatives of the function. The author considers all the accepted methods in each class, although warning that gradient methods should not be used unless the analytical differentiation of the function to be minimized is possible.If the solution is constrained, that is, certain values of the parameters are regarded as impossible or certain relations between the parameter values must be obeyed, the problem is more difficult. The second part of the review considers methods which have been proposed for the solution of constrained optimization problems.