Video In Sentences Out

Andrei Barbu, Alexander Bridge, Zachary Burchill, Dan Coroian, Sven Dickinson, Sanja Fidler, Aaron Michaux, Sam Mussman, Siddharth Narayanaswamy, Dhaval Salvi, Lara Schmidt, Jiangnan Shangguan, Jeffrey Mark Siskind, Jarrell Waggoner, Song Wang, Jinlian Wei, Yifan Yin, Zhiqi Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases, spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adjuncts and adverbial modifiers. Extracting the information needed to render these linguistic entities requires an approach to event recognition that recovers object tracks, the track-to-role assignments, and changing body posture.
Original languageEnglish
Title of host publicationUncertainty in Artificial Intelligence
Subtitle of host publicationProceedings of the Twenty-­Eighth Conference
PublisherAssociation for Uncertainty in Artificial Intelligence (AUAI)
Number of pages11
ISBN (Print)978-­0-­9749039-­8-­9
Publication statusPublished - 17 Aug 2012
Event Twenty-­Eighth Conference on Uncertainty in Artificial Intelligence - Catalina Island, United States
Duration: 15 Aug 201217 Aug 2012


Conference Twenty-­Eighth Conference on Uncertainty in Artificial Intelligence
Abbreviated titleUAI 2012
Country/TerritoryUnited States
CityCatalina Island
Internet address


Dive into the research topics of 'Video In Sentences Out'. Together they form a unique fingerprint.

Cite this