Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
I/O in WRF: A Case Study in Modern Parallel I/O Techniques
SessionHigh Performance I/O
DescriptionLarge-scale parallel applications can face significant I/O performance bottlenecks, making efficient I/O crucial. This work presents a comparative study of several parallel I/O implementations in the Weather Research and Forecasting model, including PnetCDF blocking and non-blocking I/O options, netCDF4, HDF5 Log VOL, and ADIOS. For I/O methods creating files in a canonical data layout, PnetCDF's non-blocking option offers up to 2x improvement over its blocking option and up to 4.5x over HDF5 via netCDF4, demonstrating the effectiveness of the write request aggregation technique. The HDF5 Log VOL outperforms ADIOS with a 4x improvement in write performance when creating files in the log layout, although both require non-negligible time to convert the file back to canonical order for post-run analysis. From these results, we extract some observations that can guide I/O strategies for modern parallel codes.
Event Type
Paper
TimeThursday, 16 November 20233:30pm - 4pm MST
Location401-402
Data Analysis, Visualization, and Storage
I/O and File Systems
State of the Practice
TP
