Internship: Efficient scheduling of Novel Ensemble based Workflows

• Level: Master Level Research Internship (M2)
• Location: University Grenoble Alpes Campus, Saint Martin D’heres (close to Grenoble)
• Duration: At least 4 months, possibility to pursue as a PhD.
• Contact: Bruno.Raffin@inria.fr  & pierre-francois.dutot@univ-grenoble-alpes.fr
• Incomes: Gratifications de stage (about 500 euros/month)
• Period: 2020-2021

Work:

Ensemble based workflows are becoming common on supercomputers. They consist in executing the same
application several times (up to millions) with different input parameters, the results being
analyzed through statistical tools for different kind of applications (sensibility analysis, data
assimilation, Hyperparameter tuning for Neural Networks, deep reinforcement learning, Bayesian
optimization,…). We developed a very advanced framework, Melissa (https://melissa-sa.github.io/), to support such applications at very large scale. But today the batch scheduler algorithms running on supercomputers are mainly designed to optimize the resource allocation for jbs with a static level of parallelism (typically MPI applications). These workflows are different as consisting in many parallel jobs. The goal of this internship is to  investigate novel scheduling algorithms capable of leveraging the flexibility of these ensemble based workflows to further optimize resource usage (regarding criteria like job completion time and energy consumption).