![]() |
![]() |
![]() |
![]() | Model fitting by least squares | ![]() |
![]() |
Let us minimize the sum of the squares of the components
of the residual vector given by:
A contour plot is based on an altitude function of space.
The altitude is the dot product
.
By finding the lowest altitude,
we are driving the residual vector
as close as we can to zero.
If the residual vector
reaches zero, then we have solved
the simultaneous equations
.
In a two-dimensional world, the vector
has two components,
.
A contour is a curve of constant
in
-space.
These contours have a statistical interpretation as contours
of uncertainty in
, with measurement errors in
.
Let us see how a random search-direction
can be used to reduce the residual
.
Let
be an abstract vector
with the same number of components as the solution
,
and let
contain arbitrary or random numbers.
We add an unknown quantity
of vector
to the vector
,
and thereby create
:
Next, we adjust to minimize the dot product:
A ``computation template'' for the method of random directions is:
A nice thing about the method of random directions is that you do not need to know the adjoint operator![]()
iterate {![]()
![]()
![]()
![]()
![]()
}
In practice, random directions are rarely used.
It is more common to use the gradient direction than a random direction.
Notice that a vector of the size of
is:
![]() |
(59) |
![]() |
(60) |
Starting from a model
(which may be zero),
the following is a template of pseudocode for minimizing the residual
by the steepest-descent method:
![]()
iterate {![]()
![]()
![]()
![]()
![]()
}
Good science and engineering is finding something unexpected.
Look for the unexpected both in data space and in model space.
In data space, you look at the residual .
In model space, you look at the residual projected there
.
What does it mean?
It is simply
,
the changes you need to make to your model.
It means more in later chapters,
where the operator
is a column vector of operators
that are fighting with one another to grab the data.
![]() |
![]() |
![]() |
![]() | Model fitting by least squares | ![]() |
![]() |