Development Of A Modular Performance-Portable Climate System Model

Executive Summary The Climate System Model (CSM) and Parallel Climate Model (PCM) of the National Center for Atmospheric Research (NCAR) are two advanced climate models that have seen significant use in climate research and studies of climate variability and global climate change. The community CSM-1 model links atmospheric, oceanic, biologic, cryogenic, and chemical components; it has been and continues to be used for a wide range of climate research. Developed with DOE support, PCM-1 couples similar models and, in addition, has been adapted to execute on scalable parallel computers, hence allowing long-duration simulations in support of DOE missions. Recognizing the strengths of these two models, NCAR scientists began merging the CSM and PCM code bases to produce the Community Climate System Model, CCSM, with the goal of achieving significant improvements in model performance. As they tackled this goal, NCAR staff faced two significant challenges. First, CSM was not designed to exploit the microprocessor-based scalable parallel-architecture computers that are currently being deployed at NSF and DOE centers. A consequence of this limitation is that performance has not increased substantially in the past five years. Second, both CSM and PCM model structures needed to be improved with a view to enabling " plug and play " substitution of important modules, such as dynamical solvers and physics packages. This latter improvement would both facilitate ongoing development of the new merged CSM-2 model and make it easier for scientists to experiment with improvements to individual components. A group of DOE and NCAR scientists thus proposed a joint R&D activity aimed at developing a next-generation modular, performance-portable CCSM. This work was expected to produce two primary outcomes: a performance-enhanced CCSM, better able to exploit microprocessor-based parallel computers, and a detailed design for current and future CCSM versions with substantial improvement in terms of modularity and portability. A substantial challenge in both areas was to evolve software engineering practices without unduly disrupting CCSM development or diverging from a common code base. The R&D activity tackled, in particular, the design and development of (1) a scalable, modular atmosphere model and (2) a next-generation coupler. In the atmosphere domain, work focused on improving node performance, developing a more modular atmosphere model structure that permits the substitution of both dynamics and physics components, and developing the high-performance communication libraries required for good performance on scalable parallel computers. Work on the coupler addressed issues of scalability and configurability. Work on …

[1]  P. Woodward,et al.  The Piecewise Parabolic Method (PPM) for Gas Dynamical Simulations , 1984 .

[2]  Scott R. Kohn,et al.  Toward a Common Component Architecture for High-Performance Scientific Computing , 1999, HPDC.

[3]  Ian T. Foster,et al.  Design and Performance of a Scalable Parallel Community Climate Model , 1995, Parallel Comput..

[4]  B. Vanleer,et al.  Towards the ultimate conservative difference scheme. IV. A new approach to numerical convection , 1977 .

[5]  A. Craig,et al.  Factors that affect the amplitude of El Nino in global coupled climate models , 2001 .

[6]  Thomas Sterling,et al.  Enabling Technologies for Petaflops Computing , 1995 .

[7]  W. G. Strand,et al.  Parallel climate model (PCM) control and transient simulations , 2000 .

[8]  W. Large,et al.  Oceanic vertical mixing: a review and a model with a nonlocal boundary layer parameterization , 1994 .

[9]  Walter Zwieflhofer,et al.  Developments in teracomputing : proceedings of the Ninth ECMWF Workshop on the Use of High Performance Computing in Meteorology : Reading, UK, November 13-17, 2000 , 2001 .

[10]  William D. Hibler,et al.  Modeling Pack Ice as a Cavitating Fluid , 1992 .

[11]  Chris H. Q. Ding,et al.  An Optimal Index Reshuffle Algorithm for Multidimensional Arrays and Its Applications for Parallel Architectures , 2001, IEEE Trans. Parallel Distributed Syst..

[12]  Philip W. Jones.  First- and Second-Order Conservative Remapping Schemes for Grids in Spherical Coordinates , 1999 .

[13]  David L. Williamson,et al.  Accuracy of reduced-grid calculations , 2000 .

[14]  P. Gent,et al.  Isopycnal mixing in ocean circulation models , 1990 .

[15]  David H. Bromwich,et al.  Polar Radiation Budgets of the NCAR CCM3 , 1998 .

[16]  M. Winton,et al.  A Reformulated Three-Layer Sea Ice Model , 2000 .

[17]  Chris H. Q. Ding,et al.  Data Organization and I/O in a Parallel Ocean Circulation Model , 1999, SC.

[18]  G. Bonan.  The Land Surface Climatology of the NCAR Land Surface Model Coupled to the NCAR Community Climate Model , 1998 .

[19]  Shian-Jiann Lin,et al.  An explicit flux‐form semi‐lagrangian shallow‐water model on the sphere , 1997 .

[20]  Marika M. Holland,et al.  Simulating the ice‐thickness distribution in a coupled climate model , 2001 .

[21]  Chris H. Q. Ding,et al.  A ghost cell expansion method for reducing communications in solving PDE problems , 2001, SC.

[22]  P. Kidwell,et al.  The mythical man-month: Essays on software engineering , 1996, IEEE Annals of the History of Computing.

[23]  Shian-Jiann Lin,et al.  A finite‐volume integration method for computing pressure gradient force in general vertical coordinates , 1997 .

[24]  R. C. Malone,et al.  A Reformulation and Implementation of the Bryan-Cox-Semtner Ocean Model on the Connection Machine , 1993 .

[25]  James J. Hack,et al.  Response of Climate Simulation to a New Convective Parameterization in the National Center for Atmospheric Research Community Climate Model (CCM3) , 1998 .

[26]  N. McFarlane,et al.  Sensitivity of Climate Simulations to the Parameterization of Cumulus Convection in the Canadian Climate Centre General Circulation Model , 1995, Data, Models and Analysis.

[27]  Chris H. Q. Ding,et al.  Using accurate arithmetics to improve numerical reproducibility and stability in parallel applications , 2000, ICS '00.

[28]  David L. Williamson,et al.  The Accumulation of Rounding Errors and Port Validation for Global Atmospheric Models , 1997, SIAM J. Sci. Comput..

[29]  James J. Hack,et al.  The Hydrologic and Thermodynamic Characteristics of the NCAR CCM3 , 1998 .

[30]  William H. Lipscomb,et al.  An energy-conserving thermodynamic model of sea ice , 1999 .

[31]  James J. Hack,et al.  The Dynamical Simulation of the NCAR Community Climate Model Version 3 (CCM3) , 1998 .

[32]  Albert A. M. Holtslag,et al.  Local Versus Nonlocal Boundary-Layer Diffusion in a Global Climate Model , 1993 .

[33]  Shian‐Jiann Lin,et al.  Multidimensional Flux-Form Semi-Lagrangian Transport Schemes , 1996 .

[34]  Steve McConnell,et al.  Code complete - a practical handbook of software construction, 2nd Edition , 1993 .

[35]  M. Ashworth,et al.  Optimisation For Vector And Risc Processors , 1998 .

[36]  K. Sato,et al.  Documentation of NCAR CCM0B Program Modules , 1983 .

[37]  R. C. Malone,et al.  Parallel ocean general circulation modeling , 1992 .

[38]  Albert Semtner,et al.  Ocean and climate modeling , 2000, CACM.

[39]  James J. Hack,et al.  The Energy Budget of the NCAR Community Climate Model: CCM3* , 1998 .

[40]  J. Dukowicz,et al.  Implicit free‐surface method for the Bryan‐Cox‐Semtner ocean model , 1994 .

[41]  E. Hunke,et al.  An Elastic–Viscous–Plastic Model for Sea Ice Dynamics , 1996 .

[42]  John Michalakes,et al.  The same-source parallel MM5 , 2000, Sci. Program..