Transformation Equivariant Boltzmann Machines

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We develop a novel modeling framework for Boltzmann machines, augmenting each hidden unit with a latent transformation assignment variable which describes the selection of the transformed view of the canonical connection weights associated with the unit. This enables the inferences of the model to transform in response to transformed input data in a stable and predictable way, and avoids learning multiple features differing only with respect to the set of transformations. Extending prior work on translation equivariant (convolutional) models, we develop translation and rotation equivariant restricted Boltzmann machines (RBMs) and deep belief nets (DBNs), and demonstrate their effectiveness in learning frequently occurring statistical structure from artificial and natural images.
Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning - ICANN 2011
Subtitle of host publication21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proceedings, Part I
EditorsTimo Honkela, Wlodzislaw Duch, Mark Girolami, Samuel Kaski
PublisherSpringer-Verlag GmbH
Pages1-9
Number of pages9
ISBN (Electronic)978-3-642-21735-7
ISBN (Print)978-3-642-21734-0
DOIs
Publication statusPublished - 2011

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Berlin Heidelberg
Volume6791
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • steerable filters
  • image modeling
  • Boltzmann machines
  • transformation invariance
  • transformation equivariant representations
  • convolutional structures

Cite this