PhD position

Advisors: Didier Parigot and Patrick Valduriez, Inria

The Zenith team deals with the management of scientific applications that are computation-intensive and manipulate large amounts of data. These applications are often represented by workflows, which describe sequences of tasks (computations) and data dependencies between these tasks. Several scientific workflow environments have been already proposed [3]. However, they have little support for efficiently managing large data sets. The Zenith team develops an original approach that deals with such large data sets in a way that allows efficient placement of both tasks and data on large-scale (distributed and parallel) infrastructures for more efficient execution. To this end, we propose an original solution that combines the advantages of cloud computing and P2P technologies. This work is part of the IBC project (Institut de Biologie Computationelle – http://www.ibc-montpellier.fr), in collaboration with biologists, in particular from CIRAD and IRD, and cloud providers, in particular Microsoft.

The concept of cloud computing combines several technology advances such as Service-Oriented Architectures, resource virtualization, and novel data management systems referred to as NoSQL. These technologies enable flexible and extensible usage of resources, which is referred to as elasticity. In addition, the Cloud allows users to simply outsource data storage and application executions. For the manipulation of big data, NoSQL database systems, such as Google Bigtable, Hadoop Hbase, Amazon Dynamo, Apache Cassandra, 10gen MongoDB, have been recently proposed.

Existing scientific workflow environments [3] have been developed primarily to simplify the design and execution of a set of tasks in a particular infrastructure. For example, in the field of biology, the Galaxy environment allows users to introduce catalogs of functions/tasks and compose these functions with existing functions in order to build a workflow. These environments propose a design approach that we can classify as “process-oriented”, where information about data dependencies (data flow) is purely syntactic. In addition, the targeted execution infrastructures are mostly computation-oriented, like clusters and grids. Finally, the data produced by scientific workflows are often stored in loosely structured files, for further analysis. Thus, data management is fairly basic, with data are either stored on a centralized disk or directly transferred between tasks. This approach is not suitable for data-intensive applications because data management becomes the major bottleneck in terms of data transfers.

As part of a new project that develops a middleware for scientific workflows (SciFloware), the objective of this thesis is to design a declarative data-centric language for expressing scientific workflows and its associated execution model. A declarative language is important to provide for automatic optimization and parallelization [1].  The execution model for this language will be decentralized, in order to yield flexible execution in distributed and parallel environments. This execution model will capitalize on execution models developed in the context of distributed and parallel database systems [2]. To validate this work, a prototype will be implemented using the SON middleware [4] and a distributed file system like HDFS.

References

[1] E. Ogasawara, J. F. Dias, D. de Oliveira, F. Porto, P. Valduriez, M. Mattoso. An Algebraic Approach for Data-centric Scientific Workflows. Proceedings of the VLDB Endowment (PVLDB), 4(12) : 1328-1339, 2011.

[2] M. T. Özsu, P. Valduriez. Principles of Distributed Database Systems. Third Edition, Springer, 2011.

[3] I. J. Taylor, E. Deelman, D. B. Gannon, M. Shields. Workflows for e-Science : Scientific Workflows for Grids. First Edition, Springer, 2007.

[4] A. Ait-Lahcen, D. Parigot. A Lightweight Middleware for developing P2P Applications with Component and Service-Based Principles. 15th IEEE International Conference on Computational Science and Engineering, 2012.

Contact: Didier Parigot (Firstname.Lastname@inria.fr)

Apply online

PhD. thesis

Director: Patrick Valduriez, senior researcher at INRIA

Supervisor: Hinde Bouziane, associate professor at University Montpellier 2

The ZENITH team is interested in the management of scientific applications that are computation intensive and that manipulate large amounts of data. These applications are often represented by workflows, which describe sequences of tasks (computations) and data dependencies between these tasks. Several scientific workflow
environments have been already proposed [3]. However, they have no support for efficiently managing large data sets. Our team aims to develop an original approach that deals with such large data sets and allows efficient placement of both tasks and data on large scale infrastructures for more efficient execution. To this end, we propose an original solution that combines the advantages of cloud computing and P2P technologies. This work will be part of the IBC project (Institut de Biologie Computationelle – http://www.ibc-montpellier.fr) and will be done in collaboration with biologists, especially from CIRAD, and cloud providers, in particular Microsoft.

The recent concept of cloud Computing mainly relies on the following technology advances: Service-Oriented Architectures, resource virtualization (computational and storage), and modern data management systems, referred to as NoSQL. These technologies enable a flexible and extensible (elastic) usage of resources that is inherent to the Cloud. In addition, the Cloud allows users to simply outsource data storage and application executions. For the manipulation of big data, NoSQL database systems, such as Google Bigtable, Hadoop Hbase, Amazon Dynamo, Apache Cassandra, 10gen MongoDB, have been recently proposed.

Existing scientific workflow environments [3] have been developed primarily to simplify the design and the execution of a set of tasks on parallel-distributed infrastructures. For example, in the field of biology, the Galaxy environment allows users to introduce catalogs of functions/tasks and to compose these functions with other existing ones in order to build a workflow. Therefore, these environments propose a design approach that we can classify as “process-oriented”, where the information about data dependencies (data flow) are purely syntactic. In addition, the targeted execution infrastructures are mostly computation-oriented, like clusters and grid. Finally, data analyzed and produced by a scientific workflow are often stored in loosely structured files. Simple and classical mechanisms for their management are used: they are either stored on a centralized disk or directly transferred between tasks. This approach is not suitable for data-centric applications ( because of a bottleneck, costly data transfer, etc.).

The main goal of this thesis is to propose an original approach for partitioning and mapping data and tasks to distributed resources. This approach will be based on a declarative specification (data flow oriented) of a workflow, on recent NoSQL approaches for data distribution and management and finally, on a Service Oriented Architecture for flexibility. Moreover, a dynamic approach is necessary to take into account the elasticity of the cloud. It will rely on a P2P architecture, in particular SON [4], currently under development in ZENITH.

A declarative specification has the advantage to be able to express a composition of algebraic operators(map, reduce, split, filter, etc) on manipulated data, from which it is possible to automatically parallelize tasks and to optimize the placement of data and tasks. A first algebraic specification for distributed scientific workflows has
been proposed by ZENITH [1]. This specification is close to the one used in Distributed and Parallel Database Management Systems [2], which have been widely studied. We aim to rely on such systems to influence
tasks placement (and scheduling) depending on the data placement managed by NoSQL systems.

With regard to P2P architectures, they have the benefit to be autonomous, dynamic and scalable. Thus, the interest of this thesis is to propose placement algorithms that are decentralized and able to take decisions guided by observed changes withing the execution environment, data placement and performance requirements (e.g. a
maximum execution time constraint). In this context, we can also rely on existing works around adaptation, e.g. [7].

To validate this work, a prototype will be implemented using the SON middleware [4] and a distributed file system like HDFS (Hadoop Distributed File System). It will be then introduced in a workflow environment like Galaxy (used by CIRAD scientists and researchers, with whom we have collaborations). Experimentations will be performed on the experimental Grid5000 platform and on a cloud environment, in particular Microsoft Azure.

Sujet de Stage M2R Info, 2012-13.

Florent Masseglia, Inria-Lirmm, florent.masseglia@inria.fr

François Tardieu, Inra, francois.tardieu@supagro.inra.fr

Patrick Valduriez, Inria-Lirmm, patrick.valduriez@inria.fr

Plus une plante est arrosée et éclairée, plus elle pousse… Cette « analyse » n’est pas très informative, surtout pour la recherche en agronomie qui demande des résultats plus fins sur les données qu’elle produit. Malheureusement, de telles évidences sont dominantes dans certaines études, parce qu’elles sont très caractéristiques de la réalité. Et cette domination est un obstacle pour la découverte de connaissances plus fines et plus instructives dans ces données, en particulier dans le domaine du phénotypage.

Le phénotypage étudie les relations entre le génotype (le patrimoine génétique) et le phénotype (le comportement) des plantes, dans plusieurs scénarios environnementaux. En d’autres termes, il s’agit de comparer l’évolution de plusieurs variétés génétiques d’une plante dans un même environnement. Cette comparaison permet de mieux comprendre certaines caractéristiques (capacité de production, résistance aux conditions climatiques, etc.) des plantes en fonction de leurs variétés.

Pour étudier ces réactions, chaque génotype est représenté plusieurs fois (e.g. de 3 à 10 plantes) afin de diminuer les risques d’exceptions statistiques. L’ensemble des plantes qui partagent le même génotype est appelé une « accession » ci après. La plateforme PhénoArch permet l’analyse de 1650 plantes, qui correspondent à un ensemble de 100 – 400 accessions suivant le nombre de traitements expérimentaux. La plateforme recueille des informations sur les plantes et sur leur environnement à intervalles régulier. Les données issues de la plateforme PhénoArch se présentent sous forme de séries temporelles (des mesures prises à intervalles réguliers) et peuvent concerner l’environnement (e.g. l’éclairement, la température de l’air, l’humidité) ou des variables directement mesurées sur les plantes (e.g. la croissance, le nombre de feuilles, la transpiration).

Analyser ces séries temporelles présente à la fois un enjeu scientifique pour le phénotypage et des défis techniques pour la recherche en informatique.

Nettoyage des données

Les données issues de la plateforme concernent des accessions qui sont chacune représentée par plusieurs plantes. Un premier problème lors de l’analyse de ces données consiste à nettoyer les données issues de plantes qui ont un comportement “déviant” (une plante parmi les 3 à 6 représentant cette accession et qui se comporte de manière anormale). Un premier ensemble d’outils permettrait à ce stade de mieux détecter ces données aberrantes. Il peut s’agir de mettre en place une distance entre les séries afin de détecter si l’une d’elles s’éloigne particulièrement du lot. Cette détection serait alors utilisée sous forme « d’alarme » par les experts afin de mieux cibler les données à examiner pour l’analyse à venir.

Analyse des données

Une fois nettoyées, les données des plantes (i.e. des individus) peuvent permettre d’obtenir des données caractérisant une accession, sous forme de généralisation. Autrement dit, à partir des séries temporelles de 3 à 6 plantes d’une accession, on peut obtenir une série unique (une sorte de série agrégée pour cette accession). Avec une série par accession, on peut alors produire un clustering de l’ensemble des séries temporelles associées à ces accessions.

Le travail de ce stage consiste en trois étapes principales :

1. Etat de l’art. L’étudiant devra proposer un état de l’art sur l’analyse de séries temporelles. Cela devra couvrir les questions de discrétisation, régression et clustering.

2. Application d’une technique de l’état de l’art (choisie en concertation avec les encadrants) sur un jeu de données réelles issues de la plateforme PhénoArch en ne considérant qu’une seule variable phénotypique (e.g. la croissance). Cette application devra être réalisée via une implémentation, par l’étudiant, de la technique sélectionnée, dans un des langages C/C++ ou Java.

3. Proposition d’une méthode permettant de prendre en compte plusieurs variables dans le processus de clustering.