Decision-making models on perceptual uncertainty with distributional reinforcement learning
Decision-making for autonomous vehicles in the presence of obstacle occlusions is difficult because the lack of accurate information affects the judgment. Existing methods may lead to overly conservative strategies and time-consuming computations that cannot be balanced with efficiency. We propose t...
| 發表在: | Green Energy and Intelligent Transportation |
|---|---|
| Main Authors: | , , , , |
| 格式: | Article |
| 語言: | 英语 |
| 出版: |
Elsevier
2023-04-01
|
| 主題: | |
| 在線閱讀: | http://www.sciencedirect.com/science/article/pii/S2773153722000627 |
| _version_ | 1852672789593456640 |
|---|---|
| author | Shuyuan Xu Qiao Liu Yuhui Hu Mengtian Xu Jiachen Hao |
| author_facet | Shuyuan Xu Qiao Liu Yuhui Hu Mengtian Xu Jiachen Hao |
| author_sort | Shuyuan Xu |
| collection | DOAJ |
| container_title | Green Energy and Intelligent Transportation |
| description | Decision-making for autonomous vehicles in the presence of obstacle occlusions is difficult because the lack of accurate information affects the judgment. Existing methods may lead to overly conservative strategies and time-consuming computations that cannot be balanced with efficiency. We propose to use distributional reinforcement learning to hedge the risk of strategies, optimize the worse cases, and improve the efficiency of the algorithm so that the agent learns better actions. A batch of smaller values is used to replace the average value to optimize the worse case, and combined with frame stacking, we call it Efficient-Fully parameterized Quantile Function (E-FQF). This model is used to evaluate signal-free intersection crossing scenarios and makes more efficient moves and reduces the collision rate compared to conventional reinforcement learning algorithms in the presence of perceived occlusion. The model also has robustness in the case of data loss compared to the method with embedded long and short term memory. |
| format | Article |
| id | doaj-art-57ff9c488bee4ffca80ca3f30c089335 |
| institution | Directory of Open Access Journals |
| issn | 2773-1537 |
| language | English |
| publishDate | 2023-04-01 |
| publisher | Elsevier |
| record_format | Article |
| spelling | doaj-art-57ff9c488bee4ffca80ca3f30c0893352025-08-19T21:32:25ZengElsevierGreen Energy and Intelligent Transportation2773-15372023-04-012210006210.1016/j.geits.2022.100062Decision-making models on perceptual uncertainty with distributional reinforcement learningShuyuan Xu0Qiao Liu1Yuhui Hu2Mengtian Xu3Jiachen Hao4School of Mechanical Engineering, Beijing Institute of Technology, 100081 Beijing, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, 100081 Beijing, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, 100081 Beijing, China; Corresponding author.School of Mechanical Engineering, Beijing Institute of Technology, 100081 Beijing, China; Beili Huidong (Changshu) Vehicle Technology Company, 215513 Suzhou, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, 100081 Beijing, ChinaDecision-making for autonomous vehicles in the presence of obstacle occlusions is difficult because the lack of accurate information affects the judgment. Existing methods may lead to overly conservative strategies and time-consuming computations that cannot be balanced with efficiency. We propose to use distributional reinforcement learning to hedge the risk of strategies, optimize the worse cases, and improve the efficiency of the algorithm so that the agent learns better actions. A batch of smaller values is used to replace the average value to optimize the worse case, and combined with frame stacking, we call it Efficient-Fully parameterized Quantile Function (E-FQF). This model is used to evaluate signal-free intersection crossing scenarios and makes more efficient moves and reduces the collision rate compared to conventional reinforcement learning algorithms in the presence of perceived occlusion. The model also has robustness in the case of data loss compared to the method with embedded long and short term memory.http://www.sciencedirect.com/science/article/pii/S2773153722000627Autonomous vehiclesReinforcement learningSensing occlusionPartially observable markov decision processUnsiganlized intersections |
| spellingShingle | Shuyuan Xu Qiao Liu Yuhui Hu Mengtian Xu Jiachen Hao Decision-making models on perceptual uncertainty with distributional reinforcement learning Autonomous vehicles Reinforcement learning Sensing occlusion Partially observable markov decision process Unsiganlized intersections |
| title | Decision-making models on perceptual uncertainty with distributional reinforcement learning |
| title_full | Decision-making models on perceptual uncertainty with distributional reinforcement learning |
| title_fullStr | Decision-making models on perceptual uncertainty with distributional reinforcement learning |
| title_full_unstemmed | Decision-making models on perceptual uncertainty with distributional reinforcement learning |
| title_short | Decision-making models on perceptual uncertainty with distributional reinforcement learning |
| title_sort | decision making models on perceptual uncertainty with distributional reinforcement learning |
| topic | Autonomous vehicles Reinforcement learning Sensing occlusion Partially observable markov decision process Unsiganlized intersections |
| url | http://www.sciencedirect.com/science/article/pii/S2773153722000627 |
| work_keys_str_mv | AT shuyuanxu decisionmakingmodelsonperceptualuncertaintywithdistributionalreinforcementlearning AT qiaoliu decisionmakingmodelsonperceptualuncertaintywithdistributionalreinforcementlearning AT yuhuihu decisionmakingmodelsonperceptualuncertaintywithdistributionalreinforcementlearning AT mengtianxu decisionmakingmodelsonperceptualuncertaintywithdistributionalreinforcementlearning AT jiachenhao decisionmakingmodelsonperceptualuncertaintywithdistributionalreinforcementlearning |