SC20 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Task-Parallel In Situ Data Compression of Large-Scale Computational Fluid Dynamics Simulations

Workshop:PAW-ATM 2020: The 3rd Annual Parallel Applications Workshop, Alternatives To MPI+X

Authors: Heather Pacella (Stanford University) and Alec Dunton (University of Colorado Boulder)

Abstract: Present day computational fluid dynamics simulations generate extremely large amounts of data; most of this data is discarded because current storage systems are unable to keep pace. Data compression algorithms can be applied to this data to reduce the overall amount of storage while either exactly retaining the original dataset (lossless compression) or retaining an approximate representation of the original dataset (lossy compression). Interpolative decomposition (ID) is a type of lossy compression that factors the original data matrix as the product of two (smaller) matrices; one of these matrices consists of columns of the original data matrix, while the other is a coefficient matrix. The structure of ID algorithms makes them a natural fit for task-based parallelism. Our presented work will specifically focus on using the task-based Legion programming model to implement a single-pass ID algorithm (SPID) in several fluid dynamics applications. Performance studies, scalability and the accuracy of the compressed results will be discussed in detail during our presentation.


Back to PAW-ATM 2020: The 3rd Annual Parallel Applications Workshop, Alternatives To MPI+X Archive Listing

Back to Full Workshop Archive Listing