Presentation

· Contributors · Organizations · Search Program · My Schedule · Happening Now · Maps

This content is available for: Tutorial Reg Pass. Upgrade Registration

Compression for Scientific Data

DescriptionLarge-scale numerical simulations, observations, experiments and AI computations are generating or consuming very large datasets that are difficult to analyze, store, and transfer. Data compression is an attractive and efficient technique to significantly reduce scientific datasets. This tutorial reviews the motivations, principles, techniques and error analysis methods for lossy compression of scientific datasets. It details the main compression stages (e.g. decorrelation, approximation and coding) and their variations through the presentation of the state of the art lossy compressors: SZ, ZFP, TThresh, MGARD, SPERR. A special attention is spent on lossy compression trustability. The tutorial addresses the following questions: Why lossy compression? How does compression work? How to measure and control compression error? What are the current use cases? The tutorial uses examples of real-world scientific datasets to illustrate the different compression techniques and their performance. From a participant perspective, the tutorial will detail how to use compression software as executables and as modules integrated in parallel I/O libraries (ADIOS, HDF5). This half-day tutorial, given by two of the leading teams in this domain and targeting primarily beginners interested in learning about lossy compression for scientific data, is improved from the highly rated tutorials given at ISC17-22 and SC17-22.

Presenters