Learning Finite State Controllers from Simulation

Matteo Leonetti, Luca Iocchi, Subramanian Ramamoorthy

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We propose a methodology to automatically generate agent controllers, represented as state machines, to act in partially observable environments. We define a multi-step process, in which increasingly accurate models - generally too complex to be used for planning - are employed to generate possible traces of execution by simulation. Those traces are then utilized to induce a state machine, that represents all reasonable behaviors, given the approximate models and planners previously used. The state machine will have multiple possible choices in some of its states. Those states are choice points, and we defer the learning of those choices to the deployment of the agent in the real environment.The controller obtained can therefore adapt to the actual environment,limiting the search space in a sensible way.
Original languageEnglish
Title of host publicationProceedings of The 9th European Workshop on Reinforcement Learning (EWRL-9)
Number of pages12
Publication statusPublished - 2011


Dive into the research topics of 'Learning Finite State Controllers from Simulation'. Together they form a unique fingerprint.

Cite this