A new framework to improve the performance and flexibility of supercomputing
UC3M/DICYT Researchers from the European research project ASPIDE, coordinated by the Universidad Carlos III de Madrid (UC3M), have created a tool and utility pack for high-performance software developers to improve performance and flexibility when creating applications within the supercomputing industry. As a result, they have accelerated massive data processing in urban environments and mobile telephony industries or in the detection of parasites in beehives, among other fields.
Results of this European research project can be applied to the “Extreme Data” field, in other words, when there is a large amount of data that needs to be stored and analysed mostly in real-time, which requires a lot of memory and Exascale computer systems (anexaFLOP is the equivalent to a quintillion floating point operations per second). Reaching Exascale computers is necessary if we want to analyse the large amounts of information generated every day across high performance simulations and the analysis of social media, for example. In fact, every minute more than 500 hours of video are uploaded to YouTube or approximately 150,000 images to Facebook.
Currently, traditional storage systems cannot manage the extreme scale of this data, say the researchers. “The biggest challenge for new massive computing infrastructures is not their computing capacity, but rather processing and moving data,” explains Francisco Javier García Blas, associate professor at the UC3M’s Department of Computer Science and Engineering and ASPIDE coordinator. It is at this point that the results obtained within the framework of this European R&D&I project can be particularly useful, as they contribute to the definition of a new programming model, APIs (Application Programming Interfaces), and of methodologies for expressing data-intensive tasks in Exascale systems. “In addition to this, practically all of the software being developed is freely available to the community,” he adds.
The framework developed in the ASPIDE project can be used for facilitating the design of software that is commonly used in supercomputing and Big Data-related sectors. The framework has two big benefits. First, it improves the performance of applications using task scheduling, data locality, and intensive parallelism techniques. Second, it implements a processing infrastructure, namely AIDE, which provides a flexible development mechanism of data-intensive application.
Researchers have applied the advantages of these utilities and programming mechanisms to multiple use cases used in the project with a direct reach and impact at a societal level. On the one hand, they have accelerated the massive processing of magnetic resonance studies, which are used to gather metrics about the brain’s microstructure and connectivity, in order to improve the diagnosis of mental illnesses. In addition to this, they have applied Deep Learning techniques to the automatic detection of parasites in beehives, with the aim of improving the bees’ quality of life and preventing the decline of this pollinating species. Finally, the technology developed has also been applied to massive data processing in urban environments and mobile telephony industries.