Homeokinetic Reinforcement Learning

Simón C. Smith; J. Michael Herrmann

doi:10.1007/978-3-642-28258-4_9

Homeokinetic Reinforcement Learning

Simón C. Smith, J. Michael Herrmann

School of Informatics

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

In order to find a control policy for an autonomous robot by reinforcement learning, the utility of a behaviour can be revealed locally through a modulation of the motor command by probing actions. For robots with many degrees of freedom, this type of exploration becomes inefficient such that it is an interesting option to use an auxiliary controller for the selection of promising probing actions. We suggest here to optimise the exploratory modulation by a self-organising controller. The approach is illustrated by two control tasks, namely swing-up of a pendulum and walking in a simulated hexapod. The results imply that the homeokinetic approach is beneficial for high complexity problems.

Original language	English
Title of host publication	Partially Supervised Learning: First IAPR TC3 Workshop, PSL 2011, Ulm, Germany, September 15-16, 2011, Revised Selected Papers
Editors	Friedhelm Schwenker, Edmondo Trentin
Place of Publication	Berlin, Heidelberg
Publisher	Springer
Pages	82-91
Number of pages	10
ISBN (Electronic)	978-3-642-28258-4
ISBN (Print)	978-3-642-28257-7
DOIs	https://doi.org/10.1007/978-3-642-28258-4_9
Publication status	Published - 2012

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer Berlin Heidelberg
Volume	7081
ISSN (Print)	0302-9743

Access to Document

10.1007/978-3-642-28258-4_9

http://link.springer.com/chapter/10.1007%2F978-3-642-28258-4_9

Cite this

@inbook{910085bf08284504bcf809a0aacaddaf,

title = "Homeokinetic Reinforcement Learning",

abstract = "In order to find a control policy for an autonomous robot by reinforcement learning, the utility of a behaviour can be revealed locally through a modulation of the motor command by probing actions. For robots with many degrees of freedom, this type of exploration becomes inefficient such that it is an interesting option to use an auxiliary controller for the selection of promising probing actions. We suggest here to optimise the exploratory modulation by a self-organising controller. The approach is illustrated by two control tasks, namely swing-up of a pendulum and walking in a simulated hexapod. The results imply that the homeokinetic approach is beneficial for high complexity problems.",

author = "Smith, {Sim{\'o}n C.} and Herrmann, {J. Michael}",

year = "2012",

doi = "10.1007/978-3-642-28258-4_9",

language = "English",

isbn = "978-3-642-28257-7",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "82--91",

editor = "Friedhelm Schwenker and Edmondo Trentin",

booktitle = "Partially Supervised Learning: First IAPR TC3 Workshop, PSL 2011, Ulm, Germany, September 15-16, 2011, Revised Selected Papers",

address = "United Kingdom",

}

Homeokinetic Reinforcement Learning. / Smith, Simón C.; Herrmann, J. Michael.
Partially Supervised Learning: First IAPR TC3 Workshop, PSL 2011, Ulm, Germany, September 15-16, 2011, Revised Selected Papers. ed. / Friedhelm Schwenker; Edmondo Trentin. Berlin, Heidelberg: Springer, 2012. p. 82-91 (Lecture Notes in Computer Science; Vol. 7081).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

TY - CHAP

T1 - Homeokinetic Reinforcement Learning

AU - Smith, Simón C.

AU - Herrmann, J. Michael

PY - 2012

Y1 - 2012

N2 - In order to find a control policy for an autonomous robot by reinforcement learning, the utility of a behaviour can be revealed locally through a modulation of the motor command by probing actions. For robots with many degrees of freedom, this type of exploration becomes inefficient such that it is an interesting option to use an auxiliary controller for the selection of promising probing actions. We suggest here to optimise the exploratory modulation by a self-organising controller. The approach is illustrated by two control tasks, namely swing-up of a pendulum and walking in a simulated hexapod. The results imply that the homeokinetic approach is beneficial for high complexity problems.

AB - In order to find a control policy for an autonomous robot by reinforcement learning, the utility of a behaviour can be revealed locally through a modulation of the motor command by probing actions. For robots with many degrees of freedom, this type of exploration becomes inefficient such that it is an interesting option to use an auxiliary controller for the selection of promising probing actions. We suggest here to optimise the exploratory modulation by a self-organising controller. The approach is illustrated by two control tasks, namely swing-up of a pendulum and walking in a simulated hexapod. The results imply that the homeokinetic approach is beneficial for high complexity problems.

U2 - 10.1007/978-3-642-28258-4_9

DO - 10.1007/978-3-642-28258-4_9

M3 - Chapter

SN - 978-3-642-28257-7

T3 - Lecture Notes in Computer Science

SP - 82

EP - 91

BT - Partially Supervised Learning: First IAPR TC3 Workshop, PSL 2011, Ulm, Germany, September 15-16, 2011, Revised Selected Papers

A2 - Schwenker, Friedhelm

A2 - Trentin, Edmondo

PB - Springer

CY - Berlin, Heidelberg

ER -

Homeokinetic Reinforcement Learning

Abstract

Publication series

Access to Document

Fingerprint

Cite this