Edinburgh Research Explorer

Lending direction to neural networks

Research output: Contribution to journalArticle

Original languageEnglish
Pages (from-to)503-512
Number of pages10
JournalNeural Networks
Volume8
Issue number4
DOIs
Publication statusPublished - 1995

Abstract

We present a general formulation for a network of stochastic directional units. This formulation is an extension of the Boltzmann machine in which the units are not binary, but take on values on a cyclic range, between 0 and 2π radians. This measure is appropriate to many domains, representing cyclic or angular values (e.g., wind direction, days of the week, phases of the moon). The state of each unit in a directional-unit Boltzmann machine (DUBM) is described by a complex variable, where the phase component specifies a direction; the weights are also complex variables. We associate a quadratic energy function, and corresponding probability, with each DUBM configuration. The probability distribution for the state of a given unit conditioned on the state of the rest of the network is a circular version of the Gaussian probability distribution, known as the von Mises distribution. In a mean-field approximation to a stochastic DUBM the phase component of a unit's state represents its mean direction, and the magnitude component specifies the degree of certainty associated with this direction. This combination of a value and a certainty provides additional representational power in a unit. We present a proof that the settling dynamics for a mean-field DUBM cause convergence to a free-energy minimum. Finally, we describe a learning algorithm and simulations that demonstrate a mean-field DUBM's ability to learn interesting mappings.

    Research areas

  • Free-energy minimization

ID: 21979108