Attainability of boundary points under reinforcement learning

Ed Hopkins, Martin Posch

Research output: Contribution to journalArticlepeer-review

Abstract

This paper investigates the properties of the most common form of reinforcement learning (the “basic model” of Erev and Roth) [Amer. Econ. Rev. 88 (1998) 848–881]. Stochastic approximation theory has been used to analyse the local stability of fixed points under this learning process. However, as we show, when such points are on the boundary of the state space, for example, pure strategy equilibria, standard results from the theory of stochastic approximation do not apply. We offer what we believe to be the correct treatment of boundary points, and provide a new and more general result: this model of learning converges with zero probability to fixed points which are unstable under the Maynard Smith or adjusted version of the evolutionary replicator dynamics. For two player games these are the fixed points that are linearly unstable under the standard replicator dynamics.

Original languageEnglish
Pages (from-to)110-125
Number of pages16
JournalGames and Economic Behavior
Volume53
Issue number1
DOIs
Publication statusPublished - Oct 2005

Keywords

  • Learning in games
  • Reinforcement learning
  • Stochastic approximation
  • Replicator dynamics

Fingerprint Dive into the research topics of 'Attainability of boundary points under reinforcement learning'. Together they form a unique fingerprint.

Cite this