Towards fast and accurate exa-scale quantum-mechanical calculations for material modeling using adaptive finite-elements and mixed precision computing

Abstract: Ab-initio calculations based on quantum mechanical theories have played a significant role in determining a wide variety of material properties. In particular, Kohn-Sham density functional theory (DFT) calculations have been instrumental in providing many crucial insights into materials behavior (mechanical, chemical, electronic and optical properties), and occupy a sizable fraction of world’s computational resources today. However, the stringent accuracy requirements in DFT needed to compute meaningful material properties, in conjunction with the asymptotic cubic-scaling computational complexity with number of electrons, demand huge computational resources for accurate DFT calculations. Thus, these calculations are routinely limited to material systems with at most few thousands of electrons. Though numerous efforts have been undertaken over the past two decades to tackle this challenge, the conventional approach of solving the Kohn-Sham DFT equations with a plane-wave discretization despite all its limitations, has remained the method of choice for many material science applications.

In this talk, I will present a significant advance in the state-of-the-art for accurate DFT calculations -via- the development of DFT-FE, which is a result of algorithmic advances combined with implementation innovations, that has enabled fast, scalable and accurate large-scale DFT calculations on material systems with tens of thousands of electrons while allowing for arbitrary boundary conditions and complex geometries. This has been facilitated by (i) the development of efficient and accurate spatially adaptive discretization strategies using higher-order finite-element discretization; (ii) developing efficient and scalable algorithms in conjunction with mixed-precision strategies for the solution of Kohn-Sham equations; (iii) implementation innovations, both on many core and hybrid architectures, that significantly reduce the data movement costs and increase arithmetic intensity. These developments have resulted in DFT-FE providing a time-to solution that is 9x faster than the state-of-art codes for similar accuracy. Furthermore, DFT-FE demonstrated a sustained performance of 53 PFLOPS on a dislocation system in magnesium containing 105,080 electrons using 3800 GPU nodes of ‘Summit’, the current fastest supercomputer in the world. This sustained performance recorded by DFT-FE is unprecedented for DFT codes and 17x greater than that of any previously reported DFT code. The reported advance discussed in this talk has wide ranging implications in tackling critical scientific and technological problems by making used of the predictive capability of DFT calculations for large-scale material systems. These include to name a few: (i) studying catalytic properties of nanoparticles, whose sizes are beyond those that are currently accessible, to accelerate research in catalysis; (ii) designing efficient solid-state electrolytes with high ionic conductivity and interfacial stability, which require large-scale and long time-scale molecular dynamics (MD) simulations; (iii) design of light-weight structural alloys; (iv) understanding charge transport in bio-molecular electronic devices.