Towards Optimizing Memory Mapping of Persistent Memory in UMap
TimeThursday, 19 November 20208:30am - 5pm EDT
DescriptionThe exponential growth in data set sizes across multiple domains creates challenges in terms of storing data efficiently as well as performing scalable computations on such data. Memory mapping files on different storage types offer a uniform interface as well as programming productivity for applications that perform in-place or out-of-core computations. Multi-threaded applications, however, incur an I/O contention on mapped files, hampering their scalability. Also, many applications handle sparse data structures, rendering storage efficiency a desirable feature. To address these challenges, we present SparseStore, a tool for transparently partitioning a memory-mapped persistent region into multiple files with dynamic and sparse allocation. We provide SparseStore as a part of UMap, a user-space page management tool for memory mapping. Our experiments demonstrated that using UMap with SparseStore yielded up to 12x speedup compared to system-level mmap, and up to 2x speedup compared to the default UMap that maps a single file.