Effect of limited precision on the BFGS quasi-Newton algorithm

Abstract

Some claim that updating approximate Hessian information via the BFGS formula with a Cholesky factorisation offers greater numerical stability than the more straightforward approach of performing the update directly. Others claim that no such advantage exists and that any such improvement is probably due to early implementations of the DFP formula in conjunction with low accuracy line searches. We find no discernible advantage in choosing factorised implementations (over non-factorised implementations) of BFGS methods when approximate Hessian information is available to full machine precision. However, the type of implementation may have significant effects when approximate Hessian information is only available to limited precision. Furthermore, a conjugate directions factorisation outperforms the other methods explored (including Cholesky factorisation).