Next: Overview of Weighted Nonlinear Least-Squares Fitting, Up: Nonlinear Least-Squares Fitting [Index]

The problem of multidimensional nonlinear least-squares fitting requires
the minimization of the squared residuals of *n* functions,
*f_i*, in *p* parameters, *x_i*,

\Phi(x) = (1/2) || f(x) ||^2 = (1/2) \sum_{i=1}^{n} f_i(x_1, ..., x_p)^2

All algorithms proceed from an initial guess using the linearization,

\psi(\delta) = || f(x+\delta) || ~=~ || f(x) + J \delta ||

where *x* is the initial point, *\delta* is the proposed step
and *J* is the
Jacobian matrix *J_{ij} = d f_i / d x_j*.
Additional strategies are used to enlarge the region of convergence.
These include requiring a decrease in the norm *||f||* on each
step or using a trust region to avoid steps which fall outside the linear
regime.

Note that the model parameters are denoted by *x* in this chapter
since the non-linear least-squares algorithms are described
geometrically (i.e. finding the minimum of a surface). The
independent variable of any data to be fitted is denoted by *t*.