MilliEgo: Single-Chip MmWave Radar Aided Egomotion Estimation via Deep Sensor Fusion

Chris Xiaoxuan Lu, Muhamad Risqi U. Saputra, Peijun Zhao, Yasin Almalioglu, Pedro P. B. de Gusmao, Changhao Chen, Ke Sun, Niki Trigoni, Andrew Markham

Research output: Chapter in Book/Report/Conference proceedingChapter


Robust and accurate trajectory estimation of mobile agents such as people and robots is a key requirement for providing spatial awareness for emerging capabilities such as augmented reality or autonomous interaction. Although currently dominated by optical techniques e.g., visual-inertial odometry these suffer from challenges with scene illumination or featureless surfaces. As an alternative, we propose milliEgo, a novel deep-learning approach to robust egomotion estimation which exploits the capabilities of low-cost mm Wave radar. Although mmWave radar has a fundamental advantage over monocular cameras of being metric i.e., providing absolute scale or depth, current single chip solutions have limited and sparse imaging resolution, making existing point-cloud registration techniques brittle. We propose a new architecture that is optimized for solving this challenging pose transformation problem. Secondly, to robustly fuse mmWave pose estimates with additional sensors, e.g. inertial or visual sensors we introduce a mixed attention approach to deep fusion. Through extensive experiments, we demonstrate our proposed system is able to achieve 1.3% 3D error drift and generalizes well to unseen environments. We also show that the neural architecture can be made highly efficient and suitable for real-time embedded applications.
Original languageEnglish
Title of host publicationProceedings of the 18th Conference on Embedded Networked Sensor Systems
Place of PublicationNew York, NY, USA
PublisherACM Association for Computing Machinery
Number of pages14
ISBN (Print)9781450375900
Publication statusPublished - 16 Nov 2020
Event18th ACM Conference on Embedded Networked Sensor Systems - Yokohama, Japan
Duration: 16 Nov 202019 Nov 2020


Conference18th ACM Conference on Embedded Networked Sensor Systems
Abbreviated titleSenSys 2020
Internet address


Dive into the research topics of 'MilliEgo: Single-Chip MmWave Radar Aided Egomotion Estimation via Deep Sensor Fusion'. Together they form a unique fingerprint.

Cite this