Edinburgh Research Explorer

Large-Scale Study of Curiosity-Driven Learning

Research output: Contribution to conferencePaper

  • Yuri Burda
  • Harrison Edwards
  • Deepak Pathak
  • Amos Storkey
  • Trevor Darrell
  • Alexei A. Efros

Related Edinburgh Organisations

Open Access permissions

Open

Documents

https://openreview.net/forum?id=rJNwDjAqYX
Original languageEnglish
Number of pages15
Publication statusPublished - 2019
EventSeventh International Conference on Learning Representations - New Orleans, United States
Duration: 6 May 20199 May 2019
https://iclr.cc/

Conference

ConferenceSeventh International Conference on Learning Representations
Abbreviated titleICLR 2019
CountryUnited States
CityNew Orleans
Period6/05/199/05/19
Internet address

Abstract

Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrinsic reward function which uses prediction error as reward signal. In this paper: (a) We perform the first large-scale study of purely curiosity-driven learning, i.e. without any extrinsic rewards, across 54 standard benchmark environments, including the Atari game suite. Our results show surprisingly good performance, and a high degree of alignment between the intrinsic curiosity objective and the handdesigned extrinsic rewards of many game environments. (b) We investigate the effect of using different feature spaces for computing prediction error and show that random features are sufficient for many popular RL game benchmarks, but learned features appear to generalize better (e.g. to novel game levels in Super Mario Bros.). (c) We demonstrate limitations of the prediction-based rewards in stochastic setups. Game-play videos and code are at https://pathak22.github.io/large-scale-curiosity/.

Event

Seventh International Conference on Learning Representations

6/05/199/05/19

New Orleans, United States

Event: Conference

Download statistics

No data available

ID: 92035608