CRC 1404/1: Exploiting SDNs for Efficient Data Management in Next-Generation Data Analysis Workflows (SP B04)

Running a given DAW on different computational infrastructures than it was developed for often incurs severe performance penalties. One reason is that DAWs are typically designed for specific infrastructures, which leads to hard-coded decisions regarding file locations, file movement, or means of net-work-based data exchange between tasks. This subproject will investigate the usage of software-defined networks (SDNs) to bring the requirements of the DAW and the capabilities of the underlying physical infrastructure in terms of data access closer together. It thus aims at improving portability and adaptability of DAW execution engines by means of adapting the underlying infrastructure. Technically, it will develop a light-weight declarative specification language for annotating DAWs with their communication and computation demands, which nicely connects to A02 working in the related field of data access pattern. It will furthermore cooperate with A02 on annotations for specifying data access properties and with B01 on the interplay of file placement and scheduling. The subproject will be led by Prof. Reinefeld, an expert in distributed management of large scientific data sets and high-performance computing, and Prof. Scheuermann, expert in network protocols and communication systems.