What's going on? Discovering spatio-temporal dependencies in dynamic scenes

D. Kuettel, M.D. Breitenstein, L. Van Gool, V. Ferrari

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We present two novel methods to automatically learn spatio-temporal dependencies of moving agents in complex dynamic scenes. They allow to discover temporal rules, such as the right of way between different lanes or typical traffic light sequences. To extract them, sequences of activities need to be learned. While the first method extracts rules based on a learned topic model, the second model called DDP-HMM jointly learns co-occurring activities and their time dependencies. To this end we employ Dependent Dirichlet Processes to learn an arbitrary number of infinite Hidden Markov Models. In contrast to previous work, we build on state-of-the-art topic models that allow to automatically infer all parameters such as the optimal number of HMMs necessary to explain the rules governing a scene. The models are trained offline by Gibbs Sampling using unlabeled training data.
Original languageEnglish
Title of host publicationComputer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages8
ISBN (Print)978-1-4244-6984-0
Publication statusPublished - 1 Jun 2010


  • data mining
  • hidden Markov models
  • image motion analysis
  • image sequences
  • knowledge based systems
  • sampling methods
  • traffic engineering computing
  • DDP-HMM model
  • Gibbs sampling
  • activity sequence
  • behaviour mining
  • complex dynamic scene
  • dependent Dirichlet process
  • hidden Markov model
  • learned topic model
  • moving agents
  • rule extraction
  • spatio-temporal dependency
  • temporal rule discovery
  • traffic light sequence
  • unlabeled training data
  • Computer vision
  • Hidden Markov models
  • Image motion analysis
  • Image sampling
  • Laboratories
  • Layout
  • Mathematical model
  • Motion analysis
  • Traffic control
  • Training data


Dive into the research topics of 'What's going on? Discovering spatio-temporal dependencies in dynamic scenes'. Together they form a unique fingerprint.

Cite this