–
March 23, 2017
The scientific process to discovery is composed of various tasks forming a workflow where a node is a computational task and an edge is a data exchange. A task can be a simulation, an analysis, a visualization, or a user interaction performed on a supercomputer. Traditionally, tasks exchange data through files. However, the growing mismatch between computation capabilities and I/O bandwidths available on current and future supercomputers pushes toward the adoption of in situ workflows.
In this talk we present Decaf, an in situ middleware enabling the user to describe an application as a graph compatible with leadership supercomputers. Decaf addresses several key challenges for in situ applications. First it provides a library to redistribute data between parallel tasks while preserving the semantic of the data. Second, Decaf allows to process and transform data between tasks. Third, it provides easily extensible mechanisms to control the flow of data between two tasks. Fourth, Decaf remove the I/O management between tasks from the developer by filtering and routing automatically relevant data between tasks.
