Pedestrian Liveness Detection Based on mmWave Radar and Camera Fusion

Hao Li, Ruofeng Liu, Shuai Wang, Wenchao Jiang, Chris Xiaoxuan Lu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Autonomous driving requires vehicles to achieve fine detection of objects in the surrounding environment, especially living pedestrians. Nevertheless, in real world road environments there are living pedestrians and roadside portrait billboards. Existing vision-based object detection technologies fail to accurately distinguish living pedestrians from human figures. As an important sensor of autonomous driving system, mmWave radar has extra help to detect living pedestrians. In this paper, we extract the radar cross section (RCS) of the object from the low-cost mmWave radar signal as a distinguishing feature between living pedestrian and portrait billboard. Based on this observation, we propose a feature fusion network of mmWave radar and computer vision based on attention mechanism, and detect living pedestrians from fusion features. We implement the design with commodity mmWave radar IWR6843ISK-ODS and RGB camera Logitech Pro C920. The evaluation results show that our method effectively detects living pedestrians with an mAP of 97.7% and outperforms existing studies.
Original languageEnglish
Title of host publicationProceedings of the 19th IEEE International Conference on Sensing, Communication, and Networking 2022
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages262-270
Number of pages9
ISBN (Electronic)978-1-6654-8643-9
ISBN (Print)978-1-6654-8644-6
DOIs
Publication statusPublished - 25 Oct 2022
EventIEEE International Conference on Sensing, Communication, and Networking 2022 - Virtual Conference
Duration: 20 Sep 202223 Sep 2022
https://secon2022.ieee-secon.org/

Publication series

NameAnnual IEEE International Conference on Sensing, Communication, and Networking (SECON)
PublisherIEEE
ISSN (Print)2155-5486
ISSN (Electronic)2155-5494

Conference

ConferenceIEEE International Conference on Sensing, Communication, and Networking 2022
Abbreviated titleSECON 2022
Period20/09/2223/09/22
Internet address

Fingerprint

Dive into the research topics of 'Pedestrian Liveness Detection Based on mmWave Radar and Camera Fusion'. Together they form a unique fingerprint.

Cite this