Segmenting video into classes of algorithm-suitability

Oisin Mac Aodha, Gabriel J. Brostow, Marc Pollefeys

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Given a set of algorithms, which one(s) should you apply to, i) compute optical flow, or ii) perform feature matching? Would looking at the sequence in question help you decide? It is unclear if even a person with intimate knowledge of all the different algorithms and access to the sequence itself could predict which one to apply. Our hypothesis is that the most suitable algorithm can be chosen for each video automatically, through supervised training of a classifier. The classifier treats the different algorithms as black-box alternative “classes,” and predicts when each is best because of their respective performances on training examples where ground truth flow was available.
Our experiments show that a simple Random Forest classifier is predictive of algorithm-suitability. The automatic feature selection makes use of both our spatial and temporal video features. We find that algorithm-suitability can be determined per-pixel, capitalizing on the heterogeneity of appearance and motion within a video. We demonstrate our learned region segmentation approach quantitatively using four available flow algorithms, on both known and novel image sequences with ground truth flow. We achieve performance that often even surpasses that of the one best algorithm at our disposal.
Original languageEnglish
Title of host publication2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages8
ISBN (Electronic)978-1-4244-6985-7
ISBN (Print)978-1-4244-6984-0
Publication statusPublished - 5 Aug 2010
Event 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - San Francisco, United States
Duration: 13 Jun 201018 Jun 2010

Publication series

PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)1063-6919


Conference 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2010
Country/TerritoryUnited States
CitySan Francisco
Internet address

Keywords / Materials (for Non-textual outputs)

  • computer vision
  • image classification
  • image matching
  • image motion analysis
  • image segmentation
  • image sequences
  • video segmentation
  • algorithm suitability
  • optical flow
  • image sequence
  • ground truth flow
  • random forest classifier
  • automatic feature selection
  • temporal video features
  • spatial video features
  • learned region segmentation approach
  • black-box algorithm
  • Image motion analysis
  • Optical computing
  • Image segmentation
  • Testing
  • Prediction algorithms
  • Image sequences
  • Stereo vision
  • Gold
  • Measurement standards
  • Educational institutions


Dive into the research topics of 'Segmenting video into classes of algorithm-suitability'. Together they form a unique fingerprint.

Cite this