MARL-Based Distributed Cache Placement for Wireless Networks

We investigate a distributed caching strategy based on multi-agent reinforcement learning (MARL) in a cache-aided network, where all wireless nodes have limited storage capacity and serve for certain coverage. The wireless nodes can collaboratively optimize distributed caching strategy to maximize t...

Full description

Bibliographic Details
Main Authors: Xiaosheng Lin, Yuhao Tang, Xianfu Lei, Junjuan Xia, Qingfeng Zhou, Huijun Wu, Liseng Fan
Format: Article
Language:English
Published: IEEE 2019-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/8712464/
Description
Summary:We investigate a distributed caching strategy based on multi-agent reinforcement learning (MARL) in a cache-aided network, where all wireless nodes have limited storage capacity and serve for certain coverage. The wireless nodes can collaboratively optimize distributed caching strategy to maximize the network performance measured by the average cache hit probability. Specifically, we firstly model the distributed caching strategy problem as a fully cooperative repeated game and then analyze how to improve the average cache hit probability under the MARL framework. We further propose the caching strategy based on the frequency maximum Q-value (FMQ) and the caching strategy based on the distributed Q-learning (DQ) to optimize the distributed caching strategy. The simulation results show that the proposed FMQ-based strategy significantly improves the average cache hit probability, while the proposed DQ-based strategy can converge to the optimal strategy with probability one. Moreover, the proposed FMQ-based and DQ-based strategies are not only superior to Q-learning based strategy but also superior to the probabilistic caching placement (PCP) and most popular content (MPC) strategies.
ISSN:2169-3536