"Here's looking at you, kid": Detecting people looking at each other in videos

M. Marin-Jimenez, Andrew Zisserman, Vittorio Ferrari

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The objective of this work is to determine if people are interacting in TV video by detecting whether they are looking at each other or not. We determine both the temporal period of the interaction and also spatially localize the relevant people. We make the following three contributions:
(i) head pose estimation in unconstrained scenarios (TV video) using Gaussian Process regression;
(ii) propose and evaluate several methods for assessing whether and when pairs of people are looking at each other in a video shot;
(iii) introduce new ground truth annotation for this task, extending the TV Human Interactions Dataset [22]. The peformance of the methods is evaluated on this dataset, which consists of 300 video clips extracted from TV shows. despite the variety and difficulty of this video material, our best method obtains an average precision of 86: 2%.
Original languageEnglish
Title of host publicationProceedings of the British Machine Vision Conference (BMVC)
Subtitle of host publicationDundee, September 2011
PublisherBMVA Press
Number of pages12
ISBN (Print)1-901725-43-X
Publication statusPublished - 2011

Cite this