Video Tracking Through Occlusions By Fast Audio Source Localisation

Eleonora D'Arca, Ashley Hughes, James Hopgood, Neil M Robertson

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In this paper we present a novel audio-visual speaker detection and localisation algorithm. Audio source position estimates are computed by a novel stochastic region contraction (SRC) audio search algorithm for accurate speaker localisation. This audio search algorithm is aided by available video information (stochastic region contraction with height estimation (SRC-HE)) which estimates head heights over the whole scene and gives a speed improvement of 56% over SRC. We finally combine audio and video data in a Kalman filter (KF) which fuses person-position likelihoods and tracks the speaker. Our system is composed of a single video camera and 16 microphones. We validate the approach on the problem of video occlusion i.e. two people having a conversation have to be detected and localised at a distance (as in surveillance scenarios vs. enclosed meeting rooms). We show video occlusion can be resolved and speakers can be correctly detected/localised in real data. Moreover, SRC-HE based joint audio-video (AV) speaker tracking outperforms the one based on the original SRC by 16% and 4% in terms of multi object tracking precision (MOTP) and multi object tracking accuracy (MOTA). Speaker change detection improves by 11% over SRC.
Original languageEnglish
Title of host publication2013 20th IEEE International Conference on Image Processing (ICIP)
Pages2660 - 2664
Publication statusPublished - 17 Sep 2013
EventICIP 2013 - Melbourne, Australia
Duration: 15 Sep 201318 Sep 2013


ConferenceICIP 2013


Dive into the research topics of 'Video Tracking Through Occlusions By Fast Audio Source Localisation'. Together they form a unique fingerprint.

Cite this