Genetic Algorithm Based Data-Aware Group Scheduling for Big Data Clouds

Abstract

Cloud computing is a promising cost efficient service oriented computing platform in the fields of science, engineering, business and social networking for delivering the resources on demand. Big Data Clouds is a new generation data analytics platform using Cloud computing as a back end technologies, for information mining, knowledge discovery and decision making based on statistical and empirical tools. MapReduce scheduling models for Big Data computing operate in the cluster mode, where the data nodes are pre-configured with the computing facility for processing. These MapReduce models are based on compute push model-pushing the logic to the data node for analysis, which is primarily for minimizing or eliminating data migration overheads between computing resources and data nodes. Such models, however, substantially perform well in the cluster setups, but are infelicitous for the platforms having the decoupled data storage and computing resources. In this paper, we propose a Genetic Algorithm based scheduler for such Big Data Cloud where decoupled computational and data services are offered as services. The approach is based on evolutionary methods focussed on data dependencies, computational resources and effective utilization of bandwidth thus achieving higher throughputs.