Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray

Research output: Contribution to journalArticlepeer-review

Abstract

This paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS. The effort culminates in EPIC-KITCHENS-100, a collection of 100 hours, 20M frames, 90K actions in 700 variable-length videos, capturing long-term unscripted activities in 45 environments, using head-mounted cameras. Compared to its previous version (Damen in Scaling egocentric vision: ECCV, 2018), EPIC-KITCHENS-100 has been annotated using a novel pipeline that allows denser (54% more actions per minute) and more complete annotations of fine-grained actions (+128% more action segments). This collection enables new challenges such as action detection and evaluating the “test of time”—i.e. whether models trained on data collected in 2018 can generalise to new footage collected two years later. The dataset is aligned with 6 challenges: action recognition (full and weak supervision), action detection, action anticipation, cross-modal retrieval (from captions), as well as unsupervised domain adaptation for action recognition. For each challenge, we define the task, provide baselines and evaluation metrics.
Original languageEnglish
Pages (from-to)33-55
Number of pages23
JournalInternational Journal of Computer Vision
Volume130
Issue number1
Early online date20 Oct 2021
DOIs
Publication statusPublished - 1 Jan 2022

Keywords

  • Video dataset
  • Egocentric vision
  • First-person vision
  • Action understanding
  • Multi-benchmark large-scale dataset
  • Annotation quality

Fingerprint

Dive into the research topics of 'Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100'. Together they form a unique fingerprint.

Cite this