The Effect of Non-Linear Synapses on Hierarchical Associative Memory

A Hopfield-like network of neurons assuming values 0, 1 is used to store hierarchically correlated patterns at a low level of activity [4, 5]. The interconnections [2]Jij=∑1=1LC1∑α1(Xα1i−<Xα1iα1>)(Xα1j−Xα1jα1)(1)where α1=(α1, …, αl) is a multiindex and < >α1 denotes the average over the 1-th level of the hierachical tree, are modified [1]JijΦ=<Jij2>−−−−−−−√Φ(Jij/<Jij2>−−−−−−−√)/N(2)by the nonlinear w-step functionΦw(x)={bkifx∈(ak−1,ak)−bkifx∈(−ak,−ak−1),0<k<[(w+1)/2],⎧⎩⎨⎪⎪a0=0a[(w+1)/2]=∞,b[(w+1)/2]=1,(3)such that Φw depends on w-2 parameters. The learning rule (1) with suitable chosen constants C1 allows for w-ary synapses to retrieve patterns of the hierarchical tree at levels higher than w with higher quality than lower levels. This is already for one step dynamics (perceptron) in parallel to the effect of synapses destruction investigated in [3]. We consider the case where the number of patterns is proportional to the inverse activity and the case of non-vanishing activity in the thermodynanic limit. By application of the mean field theory these results were sharpened.