Increasing the capacity of a hopfield network without sacrificing functionality

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract / Description of output

Hopfield networks are commonly trained by one of two algorithms. The simplest of these is the Hebb rule, which has a low absolute capacity of n/(2ln n), where n is the total number of neurons. This capacity can be increased to n by using the pseudo-inverse rule. However, capacity is not the only consideration. It is important for rules to be local (the weight of a synapse depends ony on information available to the two neurons it connects), incremental (learning a new pattern can be done knowing only the old weight matrix and not the actual patterns stored) and immediate (the learning process is not a limit process). The Hebbian rule is all of these, but the pseudo-inverse is never incremental, and local only if not immediate. The question addressed by this paper is, ‘Can the capacity of the Hebbian rule be increased without losing locality, incrementality or immediacy?’
Here a new algorithm is proposed. This algorithm is local, immediate and incremental. In addition it has an absolute capacity significantly higher than that of the Hebbian method: n/√2ln n.
In this paper the new learning rule is introduced, and a heuristic calculation of the absolute capacity of the learning algorithm is given. Simulations show that this calculation does indeed provide a good measure of the capacity for finite network sizes. Comparisons are made between the Hebb rule and this new learning rule.
Original languageEnglish
Title of host publicationArtificial Neural Networks — ICANN'97
Subtitle of host publication7th International Conference Lausanne, Switzerland, October 8–10, 1997 Proceeedings
EditorsWulfram Gerstner, Alain Germond, Martin Hasler, Jean-Daniel Nicoud
Place of PublicationBerlin, Heidelberg
PublisherSpringer
Pages451-456
Number of pages6
ISBN (Electronic)978-3-540-69620-9
ISBN (Print)978-3-540-63631-1
DOIs
Publication statusPublished - 1997

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Berlin Heidelberg
Volume1327
ISSN (Print)0302-9743

Fingerprint

Dive into the research topics of 'Increasing the capacity of a hopfield network without sacrificing functionality'. Together they form a unique fingerprint.

Cite this