MiniVite + Metall: A Case Study of Accelerating Graph Analytics Using Persistent Memory Allocator
SessionResearch Poster Display
Event Type
Posters
Research Posters
TP
XO
TimeThursday, 19 November 20208:30am - 5pm EDT
LocationPoster Module
DescriptionA parallel graph generation has additional computation and communication overheads that often exceed the execution time for solving the original problem for which the graph was actually generated. Substantial performance improvements and cost reductions have occurred in persistent memory technology. The promise of persistent memory is that, once constructed, data structures can be re-analyzed and updated beyond the lifetime of a single execution.
miniVite is a distributed-memory graph community detection/clustering mini-application, which supports in-memory random geometric graph generation. We observed significant overhead at scale in the parallel graph generation part. To optionally store the generated graph and provide options to reload it in a successive run, we employ a persistent memory allocator, Metall. Our experiments demonstrate improvements of up to 85x and 65x on NERSC Cori and OLCF Summit supercomputers, respectively (using 128-1024 processes), by trading off the computation/communication attributed to graph generation with I/O.
miniVite is a distributed-memory graph community detection/clustering mini-application, which supports in-memory random geometric graph generation. We observed significant overhead at scale in the parallel graph generation part. To optionally store the generated graph and provide options to reload it in a successive run, we employ a persistent memory allocator, Metall. Our experiments demonstrate improvements of up to 85x and 65x on NERSC Cori and OLCF Summit supercomputers, respectively (using 128-1024 processes), by trading off the computation/communication attributed to graph generation with I/O.
