Intelligent Sensing in Dynamic Environments Using Markov Decision Process
In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environment...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2011-01-01
|
Series: | Sensors |
Subjects: | |
Online Access: | http://www.mdpi.com/1424-8220/11/1/1229/ |
id |
doaj-78f769cd04154fd0a4bf9da4f8a63ae2 |
---|---|
record_format |
Article |
spelling |
doaj-78f769cd04154fd0a4bf9da4f8a63ae22020-11-25T00:24:48ZengMDPI AGSensors1424-82202011-01-011111229124210.3390/s110101229Intelligent Sensing in Dynamic Environments Using Markov Decision ProcessAsad M. MadniPrasanna SridharThrishantha NanayakkaraMalka N. HalgamugeIn a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor’s sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning. http://www.mdpi.com/1424-8220/11/1/1229/sensor networkMarkov decision processsensingreward shaping |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Asad M. Madni Prasanna Sridhar Thrishantha Nanayakkara Malka N. Halgamuge |
spellingShingle |
Asad M. Madni Prasanna Sridhar Thrishantha Nanayakkara Malka N. Halgamuge Intelligent Sensing in Dynamic Environments Using Markov Decision Process Sensors sensor network Markov decision process sensing reward shaping |
author_facet |
Asad M. Madni Prasanna Sridhar Thrishantha Nanayakkara Malka N. Halgamuge |
author_sort |
Asad M. Madni |
title |
Intelligent Sensing in Dynamic Environments Using Markov Decision Process |
title_short |
Intelligent Sensing in Dynamic Environments Using Markov Decision Process |
title_full |
Intelligent Sensing in Dynamic Environments Using Markov Decision Process |
title_fullStr |
Intelligent Sensing in Dynamic Environments Using Markov Decision Process |
title_full_unstemmed |
Intelligent Sensing in Dynamic Environments Using Markov Decision Process |
title_sort |
intelligent sensing in dynamic environments using markov decision process |
publisher |
MDPI AG |
series |
Sensors |
issn |
1424-8220 |
publishDate |
2011-01-01 |
description |
In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor’s sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning. |
topic |
sensor network Markov decision process sensing reward shaping |
url |
http://www.mdpi.com/1424-8220/11/1/1229/ |
work_keys_str_mv |
AT asadmmadni intelligentsensingindynamicenvironmentsusingmarkovdecisionprocess AT prasannasridhar intelligentsensingindynamicenvironmentsusingmarkovdecisionprocess AT thrishanthananayakkara intelligentsensingindynamicenvironmentsusingmarkovdecisionprocess AT malkanhalgamuge intelligentsensingindynamicenvironmentsusingmarkovdecisionprocess |
_version_ |
1725351561735962624 |