Iterative methods (like conjugate-directions) can sometimes be accelerated by a change of variables. The simplest change of variable is called a ``trial solution.'' Formally, we write the solution as:
We hope this change of variables has saved effort. For each iteration, there is a little more work: Instead of the iterative application of and , we have iterative application of and .
Our hope is that the number of iterations decreases, because we are clever or because we have been lucky in our choice of . Hopefully, the extra work of the preconditioner operator is not large compared to . If we should be so lucky that , then we get the solution immediately. Obviously we would try any guess with . Where I have known such matrices, I have often found that convergence is accelerated, but not by much. Sometimes, it is worth using for a while in the beginning; but later, it is cheaper and faster to use only . A practitioner might regard the guess of as prior information, like the guess of the initial model .
For a square matrix , the use of a preconditioner should not change the ultimate solution. Taking to be a tall rectangular matrix reduces the number of adjustable parameters, changes the solution, gets it quicker, but lowers resolution.