Tensors and compositionality in neural systems

Andrea E. Martin, Leonidas A. A. Doumas

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Neither neurobiological nor process models of meaning composition specify the operator through which constituent parts are bound together into compositional structures. In this paper, we argue that a neurophysiological computation system cannot achieve the compositionality exhibited in human thought and language if it were to rely on a multiplicative operator to perform binding, as the tensor product (TP)-based systems that have been widely adopted in cognitive science, neuroscience and artificial intelligence do. We show via simulation and two behavioural experiments that TPs violate variable-value independence, but human behaviour does not. Specifically, TPs fail to capture that in the statements fuzzy cactus and fuzzy penguin, both cactus and penguin are predicated by fuzzy(x) and belong to the set of fuzzy things, rendering these arguments similar to each other. Consistent with that thesis, people judged arguments that shared the same role to be similar, even when those arguments themselves (e.g., cacti and penguins) were judged to be dissimilar when in isolation. By contrast, the similarity of the TPs representing fuzzy(cactus) and fuzzy(penguin) was determined by the similarity of the arguments, which in this case approaches zero. Based on these results, we argue that neural systems that use TPs for binding cannot approximate how the human mind and brain represent compositional information during processing. We describe a contrasting binding mechanism that any physiological or artificial neural system could use to maintain independence between a role and its argument, a prerequisite for compositionality and, thus, for instantiating the expressive power of human thought and language in a neural system.
Original languageEnglish
Article number20190306
Number of pages7
JournalPhilosophical Transactions of the Royal Society B: Biological Sciences
Issue number1791
Early online date16 Dec 2019
Publication statusPublished - 3 Feb 2020

Keywords / Materials (for Non-textual outputs)

  • compositionality
  • tensor products
  • predicates
  • language
  • concepts
  • binding


Dive into the research topics of 'Tensors and compositionality in neural systems'. Together they form a unique fingerprint.

Cite this