Numerical investigations of linear least squares methods for derivative estimation

Abstract

The results of a numerical investigation into the errors for least squares estimates of function gradients are presented. The underlying algorithm is obtained by constructing a least squares problem using a truncated Taylor expansion. An error bound associated with this method contains in its numerator terms related to the Taylor series remainder, while its denominator contains the smallest singular value of the least squares matrix. Perhaps for this reason the error bounds are often found to be pessimistic by several orders of magnitude. The circumstance under which these poor estimates arise is elucidated and an empirical correction of the theoretical error bounds is conjectured and investigated numerically. This is followed by an indication of how the conjecture is supported by a rigorous argument. References

• J. A. Belward, I. W. Turner, and M. Ilic. On derivative estimation and the solution of least squares problems. Journal of Computational and Applied Mathematics, 222:511--523, 2008.
• C. L. Lawson and R. J. Hansen. Solving Least Squares Problems. Prentice--Hall, 1974.
• P. Lancaster and K. Salkauskas. Curve and Surface Fitting, An Introduction. Academic Press, San Diego, 1986.
• M. N. Oqielat, J. A. Belward, I. W. Turner, and B. I. Loch. {A Hybrid Clough-Tocher Radial Basis Function Method for Modelling Leaf Surfaces}. In Oxley, L., and Kulasiri, D., (eds) MODSIM 2007 International Congress on Modelling and Simulation, pages 400--406, 2007. http://mssanz.org.au/MODSIM2007/papers/7_s50
• M. N. Oqielat, I. W. Turner, and J. A. Belward. A hybrid clough-tocher method for surface fitting with application to leaf data. Applied Mathematical Modeling, 2008. doi:10.1016/j.apm.2008.07.023
• I. W. Turner, J. A. Belward, and M. N. Oqielat. Error bounds for least squares gradient estimates. In preparation, 2008.