Human activity recognition using a single-photon direct time-of-flight sensor

German Mora Martin, Stirling Scholes, Robert K. Henderson, Jonathan Leach, Istvan Gyongy

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Single-Photon Avalanche Diode (SPAD) direct Time-of-Flight (dToF) sensors provide depth imaging over long distances, enabling the detection of objects even in the absence of contrast in colour or texture. However, distant objects are represented by just a few pixels and are subject to noise from solar interference, limiting the applicability of existing computer vision techniques for high-level scene interpretation. We present a new SPAD-based vision system for human activity recognition, based on convolutional and recurrent neural networks, which is trained entirely on synthetic data. In tests using real data from a 64×32 pixel SPAD, captured over a distance of 40 m, the scheme successfully overcomes the limited transverse resolution (in which human limbs are approximately one pixel across), achieving an average accuracy of 89% in distinguishing between seven different activities. The approach analyses continuous streams of video-rate depth data at a maximal rate of 66 FPS when executed on a GPU, making it well-suited for real-time applications such as surveillance or situational awareness in autonomous systems.

Original languageEnglish
Pages (from-to)16645-16656
Number of pages11
JournalOptics Express
Volume32
Issue number10
Early online date22 Apr 2024
DOIs
Publication statusPublished - 6 May 2024

Fingerprint

Dive into the research topics of 'Human activity recognition using a single-photon direct time-of-flight sensor'. Together they form a unique fingerprint.

Cite this