Adaptive Planning for Markov Decision Processes with Uncertain Transition Models via Incremental Feature Dependency Discovery

Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive fun...

Full description

Bibliographic Details
Main Authors: Geramifard, Alborz (Contributor), Chowdhary, Girish (Contributor), How, Jonathan P. (Contributor), Ure, Nazim Kemal (Contributor)
Other Authors: Massachusetts Institute of Technology. Department of Aeronautics and Astronautics (Contributor), Massachusetts Institute of Technology. Laboratory for Information and Decision Systems (Contributor)
Format: Article
Language:English
Published: Springer-Verlag, 2013-10-25T13:18:47Z.
Subjects:
Online Access:Get fulltext
LEADER 01848 am a22002533u 4500
001 81767
042 |a dc 
100 1 0 |a Geramifard, Alborz  |e author 
100 1 0 |a Massachusetts Institute of Technology. Department of Aeronautics and Astronautics  |e contributor 
100 1 0 |a Massachusetts Institute of Technology. Laboratory for Information and Decision Systems  |e contributor 
100 1 0 |a Ure, Nazim Kemal  |e contributor 
100 1 0 |a Geramifard, Alborz  |e contributor 
100 1 0 |a Chowdhary, Girish  |e contributor 
100 1 0 |a How, Jonathan P.  |e contributor 
700 1 0 |a Chowdhary, Girish  |e author 
700 1 0 |a How, Jonathan P.  |e author 
700 1 0 |a Ure, Nazim Kemal  |e author 
245 0 0 |a Adaptive Planning for Markov Decision Processes with Uncertain Transition Models via Incremental Feature Dependency Discovery 
260 |b Springer-Verlag,   |c 2013-10-25T13:18:47Z. 
856 |z Get fulltext  |u http://hdl.handle.net/1721.1/81767 
520 |a Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework. 
546 |a en_US 
655 7 |a Article 
773 |t Machine Learning and Knowledge Discovery in Databases