Utilizing Persistent Memory in Parallel I/O Libraries
DescriptionScientific applications use parallel I/O (PIO) libraries to manage the complexity of I/O to distributed storage efficiently. PIO libraries, however, have not adequately adapted to the emergence of persistent memory (PMEM), which provides comparable performance to DRAM. They interact with PMEM using either POSIX or MPI-IO, which forces unnecessary network communications and data copies to occur when PMEM is available. In addition, PIO libraries introduce complicated APIs and configuration spaces that cause significant burden on programmers to store basic data structures. In this work, we explore the use of memory mapping as an alternative to POSIX and MPI-IO for interacting with PMEM through pMAP: a lightweight I/O library with a simplistic key-value store interface. We demonstrate that our approach can yield up to 2x improvement in performance over other popular PIO libraries by eliminating unnecessary data copies and network communications.
Event Type
Research Posters