In principle, regularization solves the null-space problem,
but that is only for those people lucky enough to have applications
so small they can afford to iterate to completion.
Think of this trivial 2-D null-space situation:
A parabolic penalty on one spatial axis
with no penalty on the other axis.
Imagine a house facing northeast with a parabolic rain gutter mounted perfectly
horizontally on one edge of the house roof.
The null space is anywhere on the center line along the bottom of the gutter.
Anywhere you begin, steepest descent brings you immediately to the gutter bottom
in a location that depends on where you began.
Now tilt the gutter a little bit so the water drips off one end of the rain drain.
Steepest descent now overshoots a little so,
as we saw in Chapter ,
a tortuous path of right-angle turning ensues.
(Recall Figure .)
The conjugate direction method quickly solves this trivial 2-D problem,
but in a 150,000 dimensional lake bottom problem,
conjugate directions taken only a few dozen iterations do not do as well.
When the data-modeling operator contains a null space,
only the regularization can pull us away from it,
and a small number of iterations may be unable to do the job.
So,
we need a good starting location.

Textbook theory may tell us final solutions are independent of the starting location,
but we learn otherwise from nonlinear problems,
and we learn otherwise from linear but large problems.