Semantic Composition of Multimodal Actions in Constraint-based Grammars

Katya Alahverdzhieva; Alexandra Lascarides

Semantic Composition of Multimodal Actions in Constraint-based Grammars

Katya Alahverdzhieva
, Alexandra Lascarides

Research output: Contribution to conference › Poster › peer-review

Abstract

The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we
analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form.

Original language	English
Publication status	Published - 2011
Event	Constraints in Discourse 2011 - Hôtel Club IGeSA - Les Roches Rouges, Agay-Roches Rouges, France Duration: 14 Sept 2011 → 16 Sept 2011

Conference

Conference	Constraints in Discourse 2011
Country/Territory	France
City	Agay-Roches Rouges
Period	14/09/11 → 16/09/11

Access to Document

Semantic Composition of Multimodal Actions in Constraint-base GrammarsSubmitted manuscript, 173 KB

Cite this

@conference{1c4aa21b77a74444ac76e56ca101451d,

title = "Semantic Composition of Multimodal Actions in Constraint-based Grammars",

abstract = "The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form. ",

author = "Katya Alahverdzhieva and Alexandra Lascarides",

year = "2011",

language = "English",

note = "Constraints in Discourse 2011 ; Conference date: 14-09-2011 Through 16-09-2011",

}

TY - CONF

T1 - Semantic Composition of Multimodal Actions in Constraint-based Grammars

AU - Alahverdzhieva, Katya

AU - Lascarides, Alexandra

PY - 2011

Y1 - 2011

N2 - The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form.

AB - The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form.

M3 - Poster

T2 - Constraints in Discourse 2011

Y2 - 14 September 2011 through 16 September 2011

ER -

Semantic Composition of Multimodal Actions in Constraint-based Grammars

Abstract

Conference

Access to Document

Fingerprint

Cite this