SC20 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Data Compression with Deep Learning Based Generative Modeling


Workshop:DRBSD-6: The 6th International Workshop on Data Analysis and Reduction for Big Scientific Data

Authors: Jong Youl Choi, Dave Pugmire, and Scott Klasky (Oak Ridge National Laboratory)


Abstract: We have been developing a VAE-based data compression method, called VAe Physics Optimized Reduction (VAPOR), with a dataset from XGC, a fusion simulation code. VAPOR is based on Vector Quantized Variational Auto Encoder (VQ-VAE), focusing on compressing XGC 5D distribution data as well as preserving physics constraints. Key features of VAPOR are three-fold; i) find a reduced representation of physics data, ii) reconstruct the data with a minimum loss, iii) preserve physics information (e.g., mass, energy, moment conservation) .

We will discuss challenges in XGC 5D data reconstruction and present our initial experiences and results on how we construct Deep Neural Network (DNN) for VAPOR to optimize the reconstruction quality of XGC 5D data and integrate XGC’s physics constraints, and share performance considerations to execute with XGC as an in situ process.





Back to DRBSD-6: The 6th International Workshop on Data Analysis and Reduction for Big Scientific Data Archive Listing



Back to Full Workshop Archive Listing