Projects per year
We present a procedure for learning to ground symbols from a sequence of stimuli consisting of an arbitrarily complex noun phrase (e.g. “all but one green square above both red circles.”) and its designation in the visual scene. Our distinctive approach combines: a) lazy fewshot learning to relate open-class words like green and above to their visual percepts; and b) symbolic reasoning with closed-class word categories like quantifiers and negation. We use this combination to estimate new training examples for grounding symbols that occur within a noun phrase but aren’t designated by that noun phase (e.g, red in the above example), thereby potentially gaining data efficiency. We evaluate the approach in a visual reference resolution task, in which the learner starts out unaware of concepts that are part of the domain model and how they relate to visual percepts.
|Title of host publication||Proceedings of The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies|
|Editors||Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz|
|Place of Publication||Stroudsburg, PA, USA|
|Publisher||Association for Computational Linguistics (ACL)|
|Number of pages||12|
|Publication status||Published - 1 Jul 2022|
|Event||2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics|
- Seattle, United States
Duration: 10 Jul 2022 → 15 Jul 2022
|Conference||2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics|
|Abbreviated title||NAACL 2022|
|Period||10/07/22 → 15/07/22|
FingerprintDive into the research topics of 'Interactive Symbol Grounding with Complex Referential Expressions'. Together they form a unique fingerprint.
- 1 Active
UKRI Trustworthy Autonomous Systems Node in Governance and Regulation
Ramamoorthy, R., Belle, V., Bundy, A., Jackson, P., Lascarides, A. & Rajan, A.
1/11/20 → 30/04/24