From Synthetic to One-Shot Regression of Camera-Agnostic Human Performances

Julian Habekost, Kunkun Pang, Takaaki Shiratori, Taku Komura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Capturing accurate 3D human performances in global space from a static monocular video is an ill-posed problem. It requires solving various depth ambiguities and information about the camera's intrinsics and extrinsics. Therefore, most methods either learn on given cameras or require to know the camera's parameters. We instead show that a camera's extrinsics and intrinsics can be regressed jointly with human's position in global space, joint angles and body shape only from long sequences of 2D motion estimates. We exploit a static camera's constant parameters by training a model that can be applied to sequences with arbitrary length with only a single forward pass while allowing full bidirectional information flow. We show that full temporal information flow is especially necessary when improving consistency through an adversarial network. Our training dataset is exclusively synthetic, and no domain adaptation is used. We achieve one of the best Human3.6M joint's error performances for models that do not use the Human3.6M training data.
Original languageEnglish
Title of host publicationPattern Recognition and Artificial Intelligence
Subtitle of host publicationThird International Conference, ICPRAI 2022, Paris, France, June 1–3, 2022, Proceedings, Part I
EditorsMounîm El Yacoubi, Eric Granger, Pong Chi Yuen, Umapada Pal, Nicole Vincent
Place of PublicationCham
PublisherSpringer International Publishing Switzerland
Pages514-525
Number of pages12
ISBN (Electronic)978-3-031-09037-0
ISBN (Print)978-3-031-09036-3
DOIs
Publication statusPublished - 2 Jun 2022
Event3rd International Conference on Pattern Recognition and Artificial Intelligence 2022 - Paris, France
Duration: 1 Jun 20223 Jun 2022
Conference number: 3
https://icprai2022.sciencesconf.org/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Cham
Volume13363
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd International Conference on Pattern Recognition and Artificial Intelligence 2022
Abbreviated titleICPRAI 2022
Country/TerritoryFrance
CityParis
Period1/06/223/06/22
Internet address

Keywords

  • Human performance
  • Monocular video
  • Synthetic data

Fingerprint

Dive into the research topics of 'From Synthetic to One-Shot Regression of Camera-Agnostic Human Performances'. Together they form a unique fingerprint.

Cite this