Edinburgh Research Explorer

DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • Zheng Shou
  • Xudong Lin
  • Yannis Kalantidis
  • Laura Sevilla-Lara
  • Marcus Rohrbach
  • Shih-Fu Chang
  • Zhicheng Yan

Related Edinburgh Organisations

Open Access permissions

Open

Documents

https://ieeexplore.ieee.org/document/8954119
Original languageEnglish
Title of host publication2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1268-1277
Number of pages10
ISBN (Electronic)978-1-7281-3293-8
ISBN (Print)978-1-7281-3294-5
DOIs
Publication statusPublished - 9 Jan 2020
Event2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Long Beach, United States
Duration: 16 Jun 201920 Jun 2019
http://cvpr2019.thecvf.com/

Publication series

Name
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

Conference2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2019
CountryUnited States
CityLong Beach
Period16/06/1920/06/19
Internet address

Abstract

Motion has shown to be useful for video understanding, where motion is typically represented by optical flow. However, computing flow from video frames is very time-consuming. Recent works directly leverage the motion vectors and residuals readily available in the compressed video to represent motion at no cost. While this avoids flow computation, it also hurts accuracy since the motion vector is noisy and has substantially reduced resolution, which makes it a less discriminative motion representation. To remedy these issues, we propose a lightweight generator network, which reduces noises in motion vectors and captures fine motion details, achieving a more Discriminative Motion Cue (DMC) representation. Since optical flow is a more accurate motion representation, we train the DMC generator to approximate flow using a reconstruction loss and an adversarial loss, jointly with the downstream action classification task. Extensive evaluations on three action recognition benchmarks (HMDB-51, UCF-101, and a subset of Kinetics) confirm the effectiveness of our method. Our full system, consisting of the generator and the classifier, is coined as DMC-Net which obtains high accuracy close to that of using flow and runs two orders of magnitude faster than using optical flow at inference time.

    Research areas

  • Action Recognition

Event

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition

16/06/1920/06/19

Long Beach, United States

Event: Conference

Download statistics

No data available

ID: 84955353