Ground-truthing Large Human Behavior Monitoring Datasets

Tehreem Qasim, Robert B Fisher, Naeem Bhatti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We present a groundtruthing approach which is applicable to large video datasets collected for studying people’s behavior, and which are recorded at a low frame per second (fps) rate. Groundtruthing a large dataset manually is a time consuming task and is prone to errors. The proposed approach is semi-automated (using a combination of deepnet and traditional image analysis) to minimize human labeler’s interaction with the video frames. The framework employs mask-rcnn as a people counter followed by human assisted semi-automated tests to correct the wrong labels. Subsequently, a bounding box extraction algorithm is used which is fully automated for frames with a single person and semi-automated for frames with two or more people. We also propose a methodology for anomaly detection i.e., collapse on table or floor. Behavior recognition is performed by using a fine-tuned alexnet convolutional neural network. The people detection and behavior analysis components of the framework are primarily designed to help reduce human labor in ground-truthing so that minimal human involvement is required. They are not meant to be employed as fully automated state-of-the-art systems. The proposed approach is validated on a new dataset presented in this paper, containing human activity in an indoor office environment and recorded at 1 fps as well as an indoor video sequence recorded at 15 fps. Experimental results show a significant reduction in human labor involved in the process of ground-truthing i.e., the number of potential clicks for office dataset was reduced by 99.2% and for the additional test video by 99.7%.
Original languageEnglish
Title of host publication2020 25th International Conference on Pattern Recognition (ICPR)
Number of pages22
ISBN (Electronic)978-1-7281-8808-9
ISBN (Print)978-1-7281-8809-6
Publication statusPublished - 5 May 2021
Event25th International Conference on Pattern Recognition 2020 - Milan, Italy
Duration: 10 Jan 202115 Jan 2021

Publication series

ISSN (Print)1051-4651


Conference25th International Conference on Pattern Recognition 2020
Abbreviated titleICPR 2020
Internet address


Dive into the research topics of 'Ground-truthing Large Human Behavior Monitoring Datasets'. Together they form a unique fingerprint.

Cite this