Emergence of chemotactic strategies with multi-agent reinforcement learning
Reinforcement learning (RL) is a flexible and efficient method for programming micro-robots in complex environments. Here we investigate whether RL can provide insights into biological systems when trained to perform chemotaxis. Namely, whether we can learn about how intelligent agents process given...
| 發表在: | Machine Learning: Science and Technology |
|---|---|
| Main Authors: | , , |
| 格式: | Article |
| 語言: | 英语 |
| 出版: |
IOP Publishing
2024-01-01
|
| 主題: | |
| 在線閱讀: | https://doi.org/10.1088/2632-2153/ad5f73 |
| _version_ | 1850345419665571840 |
|---|---|
| author | Samuel Tovey Christoph Lohrmann Christian Holm |
| author_facet | Samuel Tovey Christoph Lohrmann Christian Holm |
| author_sort | Samuel Tovey |
| collection | DOAJ |
| container_title | Machine Learning: Science and Technology |
| description | Reinforcement learning (RL) is a flexible and efficient method for programming micro-robots in complex environments. Here we investigate whether RL can provide insights into biological systems when trained to perform chemotaxis. Namely, whether we can learn about how intelligent agents process given information in order to swim towards a target. We run simulations covering a range of agent shapes, sizes, and swim speeds to determine if the physical constraints on biological swimmers, namely Brownian motion, lead to regions where reinforcement learners’ training fails. We find that the RL agents can perform chemotaxis as soon as it is physically possible and, in some cases, even before the active swimming overpowers the stochastic environment. We study the efficiency of the emergent policy and identify convergence in agent size and swim speeds. Finally, we study the strategy adopted by the RL algorithm to explain how the agents perform their tasks. To this end, we identify three emerging dominant strategies and several rare approaches taken. These strategies, whilst producing almost identical trajectories in simulation, are distinct and give insight into the possible mechanisms behind which biological agents explore their environment and respond to changing conditions. |
| format | Article |
| id | doaj-art-6a84ca100b9e4fefb4b0dbf963c9b5ca |
| institution | Directory of Open Access Journals |
| issn | 2632-2153 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| spelling | doaj-art-6a84ca100b9e4fefb4b0dbf963c9b5ca2025-08-19T23:12:19ZengIOP PublishingMachine Learning: Science and Technology2632-21532024-01-015303505410.1088/2632-2153/ad5f73Emergence of chemotactic strategies with multi-agent reinforcement learningSamuel Tovey0https://orcid.org/0000-0001-9537-8361Christoph Lohrmann1https://orcid.org/0000-0002-9011-2975Christian Holm2https://orcid.org/0000-0003-2739-310XInstitute for Computational Physics, University of Stuttgart , 70569 Stuttgart, GermanyInstitute for Computational Physics, University of Stuttgart , 70569 Stuttgart, GermanyInstitute for Computational Physics, University of Stuttgart , 70569 Stuttgart, GermanyReinforcement learning (RL) is a flexible and efficient method for programming micro-robots in complex environments. Here we investigate whether RL can provide insights into biological systems when trained to perform chemotaxis. Namely, whether we can learn about how intelligent agents process given information in order to swim towards a target. We run simulations covering a range of agent shapes, sizes, and swim speeds to determine if the physical constraints on biological swimmers, namely Brownian motion, lead to regions where reinforcement learners’ training fails. We find that the RL agents can perform chemotaxis as soon as it is physically possible and, in some cases, even before the active swimming overpowers the stochastic environment. We study the efficiency of the emergent policy and identify convergence in agent size and swim speeds. Finally, we study the strategy adopted by the RL algorithm to explain how the agents perform their tasks. To this end, we identify three emerging dominant strategies and several rare approaches taken. These strategies, whilst producing almost identical trajectories in simulation, are distinct and give insight into the possible mechanisms behind which biological agents explore their environment and respond to changing conditions.https://doi.org/10.1088/2632-2153/ad5f73reinforcement learningmicroroboticschemotaxisactive matterbiophysics |
| spellingShingle | Samuel Tovey Christoph Lohrmann Christian Holm Emergence of chemotactic strategies with multi-agent reinforcement learning reinforcement learning microrobotics chemotaxis active matter biophysics |
| title | Emergence of chemotactic strategies with multi-agent reinforcement learning |
| title_full | Emergence of chemotactic strategies with multi-agent reinforcement learning |
| title_fullStr | Emergence of chemotactic strategies with multi-agent reinforcement learning |
| title_full_unstemmed | Emergence of chemotactic strategies with multi-agent reinforcement learning |
| title_short | Emergence of chemotactic strategies with multi-agent reinforcement learning |
| title_sort | emergence of chemotactic strategies with multi agent reinforcement learning |
| topic | reinforcement learning microrobotics chemotaxis active matter biophysics |
| url | https://doi.org/10.1088/2632-2153/ad5f73 |
| work_keys_str_mv | AT samueltovey emergenceofchemotacticstrategieswithmultiagentreinforcementlearning AT christophlohrmann emergenceofchemotacticstrategieswithmultiagentreinforcementlearning AT christianholm emergenceofchemotacticstrategieswithmultiagentreinforcementlearning |