Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
Accelerating Data Analytics Using Object Based Computational Storage in a HPC
DescriptionHPC not only performs complex calculations at high speed but also processes large amount of data. HPC Systems separates compute node and storage node to effectively process them. All computation is performed on compute node, and all data is stored in storage node. In order to perform data analytics, compute node has to read large amount of data from storage node because simulation output data is large. Compute nodes must have enough memory to hold extremely large data sets but also bandwidth from storage can become a bottleneck as well. However, the actual data required for analytics is only a small part of the total data. One solution to solve this problem is computational storage. Since computational storage transfer only results to compute node by processing where data resides, it can reduce data movement and increase performance. SK hynix is researching computational storage technologies with Los Alamos National Laboratory. We propose Object based Computational Storage (OCS) as a new computational storage platform for data analytics in HPC. OCS has not only high scalability but also data-aware characteristics. Data-aware characteristics enable OCS to perform analytics independently without help from compute nodes. We intend to leverage the Apache analytics ecosystem, including Arrow and Substrait to enhance that ecosystem with the advantages computing near storage enables. Systems that use Arrow can transfer query results using a common transfer format, and Substrait provides a standard and open representation of query plans enabling pushdown of query portions to computational storage. SK hynix’s key technology for OCS is Object based Computational Storage Array(OCSA) used as a backend storage. With OCSA, OCS will provide flexible query pushdown and analytics acceleration as well as less software overhead. This talk will introduce the OCS architecture and discuss why we propose OCS as future direction for computational storage in HPC.
Presenter
Event Type
Exhibitor Forum
TimeThursday, 16 November 20232:30pm - 3pm MST
Location503-504
Artificial Intelligence/Machine Learning
Architecture and Networks
Hardware Technologies
TP
XO/EX