Implementation of MPI Cloud Computing Platform Build upon System Kernel Environment

• Main Authors: Bao-Ren Guo, 郭寶仁
• Other Authors: Jih-Ching Chiu
• Format: Others
• Language: zh-TW
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/12110951395425164651

碩士 === 國立中山大學 === 電機工程學系研究所 === 104 === With the age of Big Data coming, the three defining characteristics of Big Data--Volume, variety and Velocity, make Cloud Computing facing new challenges. In response to the demand of Big Data analytics,Using distributed computing cluster to process vast amounts of data is a megatrend .In this paper ,as the outset,we discuss the performance of distributed computing clusters provided by the current cloud computing platforms. Found that for the Message-Passing Interface (MPI) cluster, which is used in Scientific Computing, such as astronomical, atmosphere, physical spectrum...etc., cloud computing platforms provide less relevant integration services. Which made MPI Cluster make inefficient use of cloud computing resources and unable to exert its high computing performance. Following, we propose The MPI Cluster Architecture makes efficient use of cloud computing resources. To break the limitation of constructing traditional MPI Clusters computing environment, The MPI Cluster uses Socket Interface-TCP/IP Server & Client to build Communication System as the message-passing channel, and simplifies Operating System computing resources management mechanism by Group Manager. As the separated way described above,The MPI Cluster can resiliently grasp Operating System computing resources ,and work well on the cloud computing platform,that computing resources are virtualization and flexible. In order to make The MPI Cluster flexible dispatch computing resources on the cloud computing platform more easily, we adopt Kernel Distributed Computing Management (KDCM), proposed by Chiu and Huang. By its ability to unify manage and allocate computing resources, provides The MPI Cluster a path to flexible dispatch computing resources. As the goal of The MPI Cluster: Running in Linux kernel driver, and loading on KDCM, We name it MPI Kernel Cluster (MPIKC). As MPIKC and KDCM fit tightly ,they can efficient dispatch computing resources,exert its high computing performance,and provide Operating System is cloud computing environment on cloud computing platform. At the end, We verify the correctness of running MPIKC on KDCM, and use MPIKC to do distributed computing with high load, in the result of Sum of Absolute Difference and K-means clustering, each computing unit used, enhances a 0.5~1 times performance.We proved that MPIKC fits well with cloud computing platform, and exerts its high performance as the advantage of distributed computing cluster. Keywords：Cloud Computing, Distributed Computing Cluster, Big Data, MPI, Kernel Driver