Acceleration of Astrophysics Code using TAU

A team of scientists is investigating the nature of solar wind and the reasons for solar coronal heating. The heliophysics project is led by principal investigator Jean Carlos Perez, of the Space Science Center at the University of New Hampshire. Calculations started at the Argonne Leadership Class Facility (ALCF) in 2011 using Intrepid, an earlier generation Blue Gene/P. The project has been awarded an ALCF computer time grant through the Department of Energy (DOE) sponsored INCITE (Innovative and Novel Computational Impact on Theory Experiment) program each year since.

Perez’s project does their simulation with the Inhomogeneous Reduced Magnetohydrodynamics (IRMHD) code developed by Perez to carry out this research. SUPER Co-PI Sameer Shende offered his expertise to help evaluate performance of the code. After instrumenting the code using TAU and studying the profile information it generated, the team identified a communication volley taking up a suspiciously large amount of run time. Shende reviewed the results using TAU’s ParaProf graphical performance data viewer and saw telltale slopes in the MPI function calls that indicated an inefficient communication pattern. After some refinements in the code, it was clear significant improvements had been achieved in the asynchronous model of communications being used. The overall execution time was reduced from 528.18 CPU hours, to just 70.85 CPU hours (13.41 percent of the original time) for a small 2,048-processor execution on Intrepid at the ALCF.

Researchers will use the improved code to computer simulations to test existing theories of MHD (magnetohydrodynamic) turbulence, develop new theoretical models, and investigate the viability of AW turbulence as a mechanism for generating the solar wind. The results from the simulations will answer a number of key unanswered questions that are being intensely debated in the heliophysics community. The simulations will substantially extend previous, low-resolution simulations to an unprecedented 35 billion-point mesh, to capture more realistic radial variations of the solar atmosphere, as well as the small scale turbulent dynamics that are present in the solar-wind acceleration region.

Please see the full article on page 9 in the February 2013 issue of HPCsource for more details.