Migrating scientific workflows to the cloud: Through graph-partitioning, scheduling and peer-to-peer data sharing

Abstract

In recent years cloud computing has raised significant interest in the scientific community. Running scientific experiments in the cloud has its advantages such as elasticity, scalability and software maintenance. However, the communication latencies added by the virtualization, the technology behind the cloud's service provisioning in general, is observed to be the major hindrance for migrating scientific computing applications to the cloud. The problem escalates further when we consider scientific workflows, where significant data is exchanged across different tasks. So to migrate scientific workflows to the cloud, we propose a way to reduce the data communication by partitioning and scheduling the workflow and adapting a peer-to-peer data sharing among the nodes. Different size Montage workflows were considered for the analysis of the approach. From the study, we observed that the partitioning along with the peer-to-peer file sharing reduced the data communication in cloud up to 80%.