You shall know an object by the company it keeps: An investigation of semantic representations derived from object co-occurrence in visual scenes

Zahra Sadeghi , James L. McClelland, Paul Hoffman

Research output: Contribution to journalArticlepeer-review

Abstract

An influential position in lexical semantics holds that semantic representations for words can be derived through analysis of patterns of lexical co-occurrence in large language corpora. Firth (1957) famously summarised this principle as "you shall know a word by the company it keeps". We explored whether the same principle could be applied to non-verbal patterns of object co-occurrence in natural scenes. We performed latent semantic analysis (LSA) on a set of photographed scenes in which all of the objects present had been manually labelled. This resulted in a representation of objects in a high-dimensional space in which similarity between two objects indicated the degree to which they appeared in similar scenes. These representations revealed similarities among objects belonging to the same taxonomic category (e.g., items of clothing) as well as cross-category associations (e.g., between fruits and kitchen utensils). We also compared representations generated from this scene dataset with two established methods for elucidating semantic representations: (a) a published database of semantic features generated verbally by participants and (b) LSA applied to a linguistic corpus in the usual fashion. Statistical comparisons of the three methods indicated significant association between the structures revealed by each method, with the scene dataset displaying greater convergence with feature-based representations than did LSA applied to linguistic data. The results indicate that information about the conceptual significance of objects can be extracted from their patterns of co-occurrence in natural environments, opening the possibility for such data to be incorporated into existing models of conceptual representation.
Original languageEnglish
Pages (from-to)52-61
Number of pages10
JournalNeuropsychologia
Volume76
Early online date6 Sep 2014
DOIs
Publication statusPublished - Sep 2015

Keywords

  • semantic representation
  • object knowledge
  • Latent semantic analysis
  • categorisation

Fingerprint Dive into the research topics of 'You shall know an object by the company it keeps: An investigation of semantic representations derived from object co-occurrence in visual scenes'. Together they form a unique fingerprint.

Cite this