Skip to main navigation Skip to search Skip to main content

Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars

  • Dongjiang Cao
  • , Ruofeng Liu
  • , Hao Li
  • , Shuai Wang
  • , Wenchao Jiang
  • , Chris Xiaoxuan Lu

Research output: Contribution to journal › Article › peer-review

Abstract

Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc. This work studies the problem of crossmodal human re-identification (ReID), in response to the regular human movements across camera-allowed regions (e.g., streets) and camera-restricted regions (e.g., offices) deployed with heterogeneous sensors. By leveraging the emerging low-cost RGB-D cameras and mmWave radars, we propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Firstly, to address the fundamental inter-modality discrepancy, we propose a novel signature synthesis algorithm based on the observed specular reflection model of a human body. Secondly, an effective cross-modal deep metric learning model is introduced to deal with interference caused by unsynchronized data across radars and cameras. Through extensive experiments in both indoor and outdoor environments, we demonstrate that our proposed system is able to achieve ∼ 92.5% top-1 accuracy and ∼ 97.5% top-5 accuracy out of 56 volunteers. We also show that our proposed system is able to robustly reidentify subjects even when multiple subjects are present in the sensors’ field of view.
Original languageEnglish
Article number102
Number of pages25
JournalProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume6
Issue number3
DOIs
Publication statusPublished - 7 Sept 2022

Keywords / Materials (for Non-textual outputs)

  • Human-centered computing
  • Ubiquitous and mobile computing.
  • Cross Modal
  • Gait Re-Identification
  • Millimeter Wave Sensing
  • Specular Reflection

Fingerprint

Dive into the research topics of 'Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars'. Together they form a unique fingerprint.

Cite this