SC20 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

HOOVER: Leveraging OpenSHMEM for High Performance, Flexible, Streaming Graph Applications

Workshop:PAW-ATM 2020: The 3rd Annual Parallel Applications Workshop, Alternatives To MPI+X

Authors: Max Grossman (Georgia Institute of Technology), Howard Pritchard and Steve Poole (Los Alamos National Laboratory), and Vivek Sarkar (Georgia Institute of Technology)

Abstract: As the adoption of streaming graph applications increases in the finance, defense, social and other industries, so does the size, velocity, irregularity and complexity of these streaming graphs. Most existing frameworks for processing streaming, dynamic graphs are too limited in scalability or functionality to efficiently handle these increasingly massive graphs. Many frameworks are either built for shared memory platforms only (limiting the size of the graph that can be stored) or for distributed platforms, but run on slow, high overhead, interpreted and bulk synchronous platforms.

This paper introduces HOOVER, a high performance streaming graph modeling and analysis framework built from scratch to scale on high performance systems and extremely dynamic graphs. HOOVER offers similar APIs to previous streaming graph frameworks, but sits on top of a high performance runtime system designed for modern supercomputers. HOOVER leverages an eventually-consistent consistency model to improve scalability, and offers a number of unique features to users. On micro-benchmarks, HOOVER is shown to be comparable or faster than existing high performance and distributed graph frameworks. Using mini-apps, we also show that HOOVER easily scales to 2,048 PEs on more realistic applications.


Back to PAW-ATM 2020: The 3rd Annual Parallel Applications Workshop, Alternatives To MPI+X Archive Listing

Back to Full Workshop Archive Listing