Learning Finite State Controllers from Simulation

We propose a methodology to automatically generate agent controllers, represented as state machines, to act in partially observable environments. We define a multi-step process, in which increasingly accurate models - generally too complex to be used for planning - are employed to generate possible traces of execution by simulation. Those traces are then utilized to induce a state machine, that represents all reasonable behaviors, given the approximate models and planners previously used. The state machine will have multiple possible choices in some of its states. Those states are choice points, and we defer the learning of those choices to the deployment of the agent in the real environment.The controller obtained can therefore adapt to the actual environment,limiting the search space in a sensible way.