Maximum a posteriori density estimation and the sparse grid combination technique

Abstract

We study a novel method for maximum a posteriori (MAP) estimation of the probability density function of an arbitrary, independent and identically distributed \(d\)-dimensional data set. We give an interpretation of the MAP algorithm in terms of regularised maximum likelihood. We also present numerical experiments using a sparse grid combination technique and the `opticom' method. The numerical results demonstrate the viability of parallelisation for the combination technique. References

• H. J. Bungartz, M. Griebel, D. Roschke and C. Zenger. Pointwise convergence of the combination technique for the Laplace equation. East-West J. Numer. Math, 2:21--45 (1994). http://zbmath.org/?q=an:00653220
• J. Garcke. Regression with the optimised combination technique. In Proceedings of the 23rd international conference on Machine learning, ICML '06, pages 321--328 (2006). doi:10.1145/1143844.1143885
• J. Garcke. Sparse grid tutorial. Technical report (2011). http://page.math.tu-berlin.de/ garcke/paper/sparseGridTutorial.pdf
• M. Griebel and M. Hegland. A finite element method for density estimation with Gaussian process priors. SIAM J. Numer. Anal., 47:4759--4792 (2010). doi:10.1137/080736478
• M. Griebel, M. Schneider and C. Zenger. A combination technique for the solution of sparse grid problems. In Iterative methods in linear algebra (Brussels, 1991), pages 263--281. North-Holland, Amsterdam (1992).
• M. Hegland. Adaptive sparse grids. ANZIAM J., 44:C335--C353 (2003). http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/685
• M. Hegland. Approximate maximum a posteriori with Gaussian process priors. Constr. Approx., 26:205--224 (2007). doi:10.1007/s00365-006-0661-4
• M. Hegland, J. Garcke, and V. Challis. The combination technique and some generalisations. Linear Algebra Appl., 420:249--275 (2007). doi:10.1016/j.laa.2006.07.014
• C. T. Kelley. Solving nonlinear equations with Newton's method. Fundamentals of Algorithms. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2003).
• H. Kobayashi, B.L. Mark, and W. Turin. Probability, Random Processes, and Statistical Analysis: Applications to Communications, Signal Processing, Queueing Theory and Mathematical Finance. Cambridge University Press (2012).
• C. Pflaum and A. Zhou. Error analysis of the combination technique. Numerische Mathematik, 84:327--350 (1999). doi:10.1007/s002110050474
• D. W. Scott. Multivariate Density Estimation: Theory, Practice, and Visualization. John Wiley and Sons (2004).
• C. Zenger. Sparse grids. Parallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GAMM-Seminar, 31 (1990).