Exascale programming models for extreme data processing
TimeTuesday, June 23rd4:06pm - 4:08pm
DescriptionExtreme Data is an incarnation of Big Data concept distinguished by the massive amounts of data that must be queried, communicated and analyzed in (near) real-time by using a very large number of memory/storage elements and Exascale computing systems. Immediate examples are the scientific data produced at a rate of hundreds of gigabits-per-second that must be stored, filtered and analyzed, the millions of images per day that must be mined (analyzed) in parallel, the one billion of social data posts queried in real-time on an in-memory components database. Nowadays, current disks and commercial storage solutions cannot handle the extreme scale data generation of such applications. Following the need of improvement of current concepts and technologies, ASPIDE’s activities focus on data-intensive applications running on systems composed of up to millions of computing elements (Exascale systems). Practical results will include the methodology and software prototypes that will be designed and used to implement Exascale applications.
The ASPIDE project will contribute with the definition of a new programming paradigm, APIs, runtime tools and methodologies for expressing data-intensive tasks on Exascale systems, which can pave the way for the exploitation of massive parallelism over a simplified model of the system architecture, promoting high performance and efficiency, and offering powerful operations and mechanisms for processing extreme data sources at high speed and/or real-time.
This poster presents some of the current solutions developed in the ASPIDE project, including monitoring at large scale, the DCEX programming framework, and the I/O profiling methodology for current and incoming data intensive applications.