Model-Free Reinforcement Learning of Impedance Control in Stochastic Environments

F. Stulp, J. Buchli, A. Ellmer, M. Mistry, E. A. Theodorou, S. Schaal

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

For humans and robots, variable impedance control is an essential component for ensuring robust and safe physical interaction with the environment. Humans learn to adapt their impedance to specific tasks and environments; a capability which we continually develop and improve until we are well into our twenties. In this article, we reproduce functionally interesting aspects of learning impedance control in humans on a simulated robot platform. As demonstrated in numerous force field tasks, humans combine two strategies to adapt their impedance to perturbations, thereby minimizing position error and energy consumption: 1) if perturbations are unpredictable, subjects increase their impedance through cocontraction; and 2) if perturbations are predictable, subjects learn a feed-forward command to offset the perturbation. We show how a 7-DOF simulated robot demonstrates similar behavior with our model-free reinforcement learning algorithm PI2, by applying deterministic and stochastic force fields to the robot's end-effector. We show the qualitative similarity between the robot and human movements. Our results provide a biologically plausible approach to learning appropriate impedances purely from experience, without requiring a model of either body or environment dynamics. Not requiring models also facilitates autonomous development for robots, as prespecified models cannot be provided for each environment a robot might encounter.
Original languageEnglish
Pages (from-to)330-341
Number of pages12
JournalIEEE Transactions on Autonomous Mental Development
Issue number4
Publication statusPublished - Dec 2012


Dive into the research topics of 'Model-Free Reinforcement Learning of Impedance Control in Stochastic Environments'. Together they form a unique fingerprint.

Cite this