–
May 16, 2019
Supercomputers are producing large amount of data that need to be analyzed. They produce mostly two kinds of data: scientific data and monitoring data. Scientific data are the results of the execution of numerical simulations and need to be analyzed to extract knowledge. Monitoring data are produced by all kinds of sensors and software components, and can be analyzed to detect, among other things, reliability and performance issues. Considering the scale of such systems, the amount of data to process is huge and analyzing these data with short response time is often necessary. Using techniques and algorithms coming from the Big Data community seems appealing in this context.
This talk will present some of our efforts in trying to apply Big Data and Machine Learning techniques in the HPC context. It will cover 3 main topics: i) The use of Apache Spark Streaming as a tool for in-situ data analysis; ii) The analysis of time series to predict CPU overheating issues in Supercomputers; iii) The application of classification algorithms to the placement problem in NUMA platforms.
