Probabilistic inference for solving discrete and continuous state Markov Decision Processes

Marc Toussaint, Amos Storkey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Inference in Markov Decision Processes has recently received interest as a means to infer goals of an observed action, policy recognition,and also as a tool to compute policies.A particularly interesting aspect of the approach is that any existing inference technique in DBNs now becomes available for answering behavioral questions–including those on continuous, factorial, or hierarchical state representations. Here we present an Expectation Maximization algorithm for computing optimal policies. Unlike previous approaches we can show that this actually optimizes the discounted expected future return for arbitrary reward functions and without assuming an ad-hoc finite total time. The algorithm is generic in that any inference technique can be utilized in the E-step. We demonstrate this for exact inference on a discrete maze and Gaussian belief state propagation in continuous stochastic optimal control problems.
Original languageEnglish
Title of host publicationICML '06 Proceedings of the 23rd international conference on Machine learning
Number of pages8
ISBN (Print)1-59593-383-2
Publication statusPublished - 2006


Dive into the research topics of 'Probabilistic inference for solving discrete and continuous state Markov Decision Processes'. Together they form a unique fingerprint.

Cite this