CubeLearn: End-to-end Learning for Human Motion Recognition from Raw mmWave Radar Signals

Peijun Zhao, Chris Xiaoxuan Lu*, Bing Wang, Niki Trigoni, Andrew Markham

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

mmWave FMCW radar has attracted huge amount of research interest for human-centered applications in recent years, such as human gesture and activity recognition. Most existing pipelines are built upon conventional Discrete Fourier Transform (DFT) pre-processing and deep neural network classifier hybrid methods, with a majority of previous works focusing on designing the downstream classifier to improve overall accuracy. In this work, we take a step back and look at the pre-processing module. To avoid the drawbacks of conventional DFT pre-processing, we propose a complex-weighted learnable pre-processing module, named CubeLearn, to directly extract features from raw radar signal and build an end-to-end deep neural network for mmWave FMCW radar motion recognition applications. Extensive experiments show that our CubeLearn module consistently improves the classification accuracies of different pipelines, especially benefiting those simpler models, which are more likely to be used on edge devices due to their computational efficiency. We provide ablation studies on initialization methods and structure of the proposed module, as well as an evaluation of the running time on PC and edge devices. This work also serves as a comparison of different approaches towards data cube slicing. Through our task agnostic design, we propose a first step towards a generic end-to-end solution for radar recognition problems.
Original languageEnglish
Pages (from-to)10236-10249
JournalIEEE Internet of Things Journal
Issue number12
Early online date7 Jun 2023
Publication statusPublished - 15 Jun 2023

Keywords / Materials (for Non-textual outputs)

  • Chirp
  • Convolutional neural networks
  • Discrete Fourier transforms
  • Doppler radar
  • end-to-end neural network
  • Millimeter wave communication
  • mmWave radar
  • motion recognition
  • Neural networks
  • Radar applications


Dive into the research topics of 'CubeLearn: End-to-end Learning for Human Motion Recognition from Raw mmWave Radar Signals'. Together they form a unique fingerprint.

Cite this