ANZIAM J. 46(E) pp.C1035--C1054, 2005.

Optimisation of a spline based Eulerian--Lagrangian transport solver

S. J. Leak

M. G. Trefry

F. P. Ruan
(Received 29 October 2004, revised 20 September 2005)

Abstract

Consider solute transport in saturated porous media. Eulerian-Lagrangian methods provide efficient numerical solutions with minimal numerical dispersion. We discuss the optimisation and parallelisation of a serial, spline based Eulerian-Lagrangian code (ELM2D , Fortran 90) on a 64 bit NEC SX-5 platform to support high-resolution numerical experiments. The aim was to reduce execution times by an order of magnitude whilst maintaining numerical accuracy. Profiling analysis indicated potential inefficiencies in the spline and diffusion subsystems of the code. Vectorisation of these subsystems achieved more than an order of magnitude speed increase. Use of either OpenMP or SX-5 microtasking directives was also effective in further reducing computational expense. Benchmarking optimised solutions against 32 bit serial solutions generated on a different platform indicated good numerical agreement of overall solute distributions and plume spatial moments for strongly heterogeneous problems with O(107) nodes. More than twenty-fold reduction in SX-5 execution times was achieved through vectorisation and parallelisation, but came at the cost of increasing memory demands. A user configurable striping parameter was introduced to the algorithm to determine the in-core storage, yielding a trade-off between execution speed and resource demand.

Download to your computer

Authors

S. J. Leak
NEC Australia, Melbourne, Victoria, Australia mailto:stephen.leak@nec.com.au
M. G. Trefry
CSIRO Land and Water, Floreat, Western Australia, Australia mailto:mike.trefry@csiro.au
F. P. Ruan
Schlumberger Petrel IS, Oslo, Norway mailto:ruan@slb.com

Published October 9, 2005. ISSN 1446-8735

References

  1. Bierbrauer, F., Soh, W. K. and Yuen, W. Y. D., On some developments and evaluation of an Eulerian--Lagrangian method for the transport equation ANZIAM J. 42(E):C238--C262, 2000. http://anziamj.austms.org.au/V42/CTAC99/Bier
  2. Kinoshita, K., Hardware System of the SX Series NEC Res. and Develop. 39(4):362--368, 1998.
  3. de Oliviera, A. and Baptista, A. M., A comparison of integration and interpolation Eulerian-Lagrangian methods Int. J. Numer. Methods Fluids 21, 183--204, 1995.
  4. Magolu monga Made, M. and van der Vorst, H. A., ParIC: A family of parallel Incomplete Cholesky Preconditioners HPCN Europe, 89--98, 2000. [Online] http://citeseer.ist.psu.edu/made00paric.html.
  5. McLaughlin, D. and Ruan, F., Macrodispersivity and large-scale hydrogeologic variability Transp. Porous Media 42(1):133--154, 2001.
  6. Meuer, H., Strohmaier, E., Dongarra, J. and Simon, H., Top500 Supercomputer Sites [Online] http://www.top500.org. 2005.
  7. Press, W. H., Teukolsky, S. A., Vetterling, W. T. and Flannery, B. P., Numerical Recipes in Fortran 90: The art of parallel scientific computing Volume 2 of Fortran Numerical Recipes, Cambridge University Press, Cambridge, UK, 1996. http://www.nr.com/
  8. Ruan, F. and McLaughlin, D., An investigation of Eulerian--Lagrangian methods for solving heterogeneous advection-dominated transport problems Water Resour. Res. 35(8):2359--2373, 1999.
  9. Trefry, M. G., Ruan, F. P. and McLaughlin, D., Numerical simulations of preasymptotic transport in heterogeneous porous media: Departures from the Gaussian limit Water Resour. Res. 39(3), Article 1063, 2003. http://dx.doi.org/10.1029/2001WR001101
Copyright Austral. Mathematical Soc.; prepared October 9, 2005, from Leak.tex with anziamje.sty (v2.3h)