Sample-based abstraction for hybrid relational MDPs

Davide Nitti, Vaishak Belle, T. De Laet, Luc De Raedt

Research output: Contribution to conferencePaperpeer-review


We study planning in relational Markov Decision Processes involving discrete and continuous states and actions. This combination of hybrid relational domains has so far not received a lot of attention. While several symbolic approaches have been proposed for hybrid and relational domains separately, they generally do not provide an integrated approach and they often make restrictive assumptions to make exact inference possible. Removing those restrictions requires approximations such as Monte-Carlo methods. We propose HyBrel: a sample-based planner for hybrid relational domains that combines model-based approaches with state abstraction. HyBrel samples episodes and uses the previous episodes as well as the model to approximate the Q-function. Abstraction is performed for each sampled episode, this removes typical restrictions of symbolic approaches. In our empirical evaluations, HyBrel is shown to have a wide applicability, confirming the advantage of sampled-based abstraction.
Original languageEnglish
Number of pages9
Publication statusPublished - 2015
Event12th European Workshop on Reinforcement Learning: ICML 2015 - Lille, France
Duration: 10 Jul 201511 Jul 2015


Workshop12th European Workshop on Reinforcement Learning
Abbreviated titleEWRL 2015
Internet address


Dive into the research topics of 'Sample-based abstraction for hybrid relational MDPs'. Together they form a unique fingerprint.

Cite this