Automate Scientific Workflow Execution between Local Cluster and Cloud

Scientific computational experiments often span multiple computational and analytical steps, and during execution, researchers need to store, access, transfer, and query information. Scientific workflow is a powerful tool to stream-line and organize scientific application. Numbers of tools have been...

Full description

Bibliographic Details
Main Authors: Hao Qian, Daniel Andresen
Format: Article
Language:English
Published: Atlantis Press 2016-01-01
Series:International Journal of Networked and Distributed Computing (IJNDC)
Subjects:
Online Access:https://www.atlantis-press.com/article/25846121.pdf
Description
Summary:Scientific computational experiments often span multiple computational and analytical steps, and during execution, researchers need to store, access, transfer, and query information. Scientific workflow is a powerful tool to stream-line and organize scientific application. Numbers of tools have been developed to help build scientific workflows, they provide mechanisms for creating workflow but lack a native scheduling system for determining where code should be executed. This paper presents Emerald, a system that adds sophisticated computation offloading capabili-ties to scientific workflows. Emerald automatically offloads computation intensive steps of scientific workflow to the cloud in order to enhance workflow performance. Emerald minimizes the burden on developers to build work-flows with computation offloading ability by providing easy-to-use API. Evaluation showed that Emerald can ef-fectively reduce up to 55% of execution time for scientific applications.
ISSN:2211-7946