High Performance Computations for Large Scale Simulations of Subsurface Multiphase Fluid and Heat Flow

TOUGH2 is a widely used reservoir simulator for solving subsurface flow related problems such as nuclear waste geologic isolation, environmental remediation of soil and groundwater contamination, and geothermal reservoir engineering. It solves a set of coupled mass and energy balance equations using a finite volume method. This contribution presents the design and analysis of a parallel version of TOUGH2. The parallel implementation first partitions the unstructured computational domain. For each time step, a set of coupled non-linear equations is solved with Newton iteration. In each Newton step, a Jacobian matrix is calculated and an ill-conditioned non-symmetric linear system is solved using a preconditioned iterative solver. Communication is required for convergence tests and data exchange across partitioning borders. Parallel performance results on Cray T3E-900 are presented for two real application problems arising in the Yucca Mountain nuclear waste site study. The execution time is reduced from 7504 seconds on two processors to 126 seconds on 128 processors for a 2D problem involving 52,752 equations. For a larger 3D problem with 293,928 equations the time decreases from 10,055 seconds on 16 processors to 329 seconds on 512 processors.

[1]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[2]  George Karypis,et al.  A Software Package for Partitioning Unstructured Graphs , Partitioning Meshes , and Computing Fill-Reducing Orderings of Sparse Matrices Version 5 . 0 , 1998 .

[3]  E. Anderson,et al.  Performance of the CRAY T3E Multiprocessor , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[4]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[5]  Barry F. Smith,et al.  Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations , 1996 .

[6]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[7]  K. Pruess,et al.  TOUGH User's Guide , 1987 .

[8]  K. Pruess Proceedings of the TOUGH Workshop 1998 , 1990 .

[9]  Yu-Shu Wu,et al.  A Parallel Implementation of the TOUGH2 Software Package for Large Scale Multiphase Fluid and Heat Flow Simulations , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[10]  K. Pruess,et al.  TOUGH2-A General-Purpose Numerical Simulator for Multiphase Fluid and Heat Flow , 1991 .

[11]  Yousef Saad,et al.  ILUT: A dual threshold incomplete LU factorization , 1994, Numer. Linear Algebra Appl..

[12]  Björn Engquist,et al.  Simulation and Visualization on the Grid , 2000 .

[13]  Erik Elmroth On Grid Partitioning for a High-Performance Groundwater Simulation Software , 2000 .