Q-Learning Based Content Placement Method for Dynamic Cloud Content Delivery Networks

• Main Authors: Yujie Liu, Dianjie Lu, Guijuan Zhang, Jie Tian, Weizhi Xu
• Format: Article
• Language: English
• Published: IEEE 2019-01-01
• Series: IEEE Access
• Subjects: Content placement; dynamic CCDNs; congestion information; Q-learning
• Online Access: https://ieeexplore.ieee.org/document/8717626/

How to reduce the content placement cost of cloud content delivery networks (CCDNs) is a hot topic in recent years. Traditional content placement methods mainly reduce the cost of content placement by constructing delivery trees, but they cannot adapt to the dynamic deployment of cloud proxy servers in the CCDNs. In addition, the traditional content placement method only provides delivery paths according to local decision-making without considering global dynamics of the congestion in the CCDNs, which is also one of the main factors causing high cost of content placement. To solve these problems, we propose a content placement model based on Q-learning for the dynamic CCDNs, called Q-content placement model (Q-CPM). This Q-learning approach can lead to better routing decisions due to up-to-date and more reliable congestion values. Then, based on the Q-CPM model, an algorithm is proposed to construct the Q-adaptive delivery tree (Q-ADT). In this algorithm, local and nonlocal congestion information is propagated over network learning packets. Through this algorithm, the paths with low congestion cost will be selected and can adapt to the dynamic cloud delivery environment. The experimental results show that the method can adapt to the dynamic changes of the CCDNs flexibly and reduce the overall congestion cost of content placement effectively.