Séminaire IBC du WP5
Mardi 5 novembre, 11h
Salle 127, Batiment Galera
Algebraic Dataflows for Big Data Analysis
Marta Mattoso
UFRJ, Rio de Janeiro
Brazil
Analyzing big data requires the support of dataflows with many activities to extract and explore relevant information from the data. Recent approaches such as Pig Latin propose a high-level language to model such dataflows. However, the dataflow execution is typically delegated to a MapReduce implementation such as Hadoop, which does not follow an algebraic approach, thus it cannot take advantage of the optimization opportunities of PigLatin algebra. In this talk, we discuss some issues of hadoop and propose an approach for big data analysis based on algebraic workflows, which yields optimization and parallel execution of activities and supports user steering using provenance queries. We illustrate how a big data processing dataflow can be modeled using the algebra. Through an experimental evaluation using real datasets and the execution of the dataflow with Chiron, an engine that supports our algebra, we show that our approach yields performance gains of up to 20 % using algebraic transformations in the dataflow and up to 40 % of time saved on a user steering scenario.
