The problem of multidimensional nonlinear least-squares fitting requires
the minimization of the squared residuals of *n* functions,
*f_i*, in *p* parameters, *x_i*,

\Phi(x) = (1/2) || F(x) ||^2 = (1/2) \sum_{i=1}^{n} f_i(x_1, ..., x_p)^2

All algorithms proceed from an initial guess using the linearization,

\psi(p) = || F(x+p) || ~=~ || F(x) + J p ||

where *x* is the initial point, *p* is the proposed step
and *J* is the
Jacobian matrix *J_{ij} = d f_i / d x_j*.
Additional strategies are used to enlarge the region of convergence.
These include requiring a decrease in the norm *||F||* on each
step or using a trust region to avoid steps which fall outside the linear
regime.

To perform a weighted least-squares fit of a nonlinear model
*Y(x,t)* to data (*t_i*, *y_i*) with independent Gaussian
errors *\sigma_i*, use function components of the following form,

f_i = (Y(x, t_i) - y_i) / \sigma_i

Note that the model parameters are denoted by *x* in this chapter
since the non-linear least-squares algorithms are described
geometrically (i.e. finding the minimum of a surface). The
independent variable of any data to be fitted is denoted by *t*.

With the definition above the Jacobian is
*J_{ij} =(1 / \sigma_i) d Y_i / d x_j*, where *Y_i = Y(x,t_i)*.