A Performance-Satisfied and Influence-Aware MapReduce Allocation Scheme for Cloud Computing Systems

• Main Authors: CHEN,YI-YING, 陳毅穎
• Other Authors: Lin,Jenn-Wei
• Format: Others
• Language: zh-TW
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/65859497199976921883

碩士 === 輔仁大學 === 資訊工程學系碩士班 === 105 === MapReduce can speed up the execution of an application (job) with big data by dividing the job into a number of map and reduce tasks in cloud computing systems. There are many MapReduce jobs concurrently running in the systems. It is required to efficiently allocate computing resources for the tasks of such jobs. If not, the performance requirements (e.g. execution deadlines) of some jobs cannot be satisfied. Several deadline-constrained MapReduce schedulers have been proposed, but they do not consider the following factors: 1) relaxed data locality, 2) influence of existing tasks, and 3) optimizing allocation contention. To take into consideration the above three factors into the MapReduce scheduling, we first adopt the data locality manner to make a pre-allocation plan. In the pre-allocation plan, if some new tasks will severely affect the existing tasks or their performance requirements cannot be satisfied, a post-allocation plan will be made to update some assignment information of the pre-allocation plan. We transform the post-allocation problem into a well-known network graph problem: minimum cost flow (MCF). By following the solution of the mapping MCF problem to allocate tasks of new jobs, we can take the least total allocation cost while satisfying the performance requirements. In addition, the degree of influence of existing tasks can be mitigated without deteriorating the performance of already slow tasks. Finally, we conduct the performance analysis to demonstrate the effectiveness of our proposed MapReduce schedulers in comparisons with some relative schemes.