Database Saliency for Fast Image Retrieval

Y. Gao, M. Shi, D. Tao, C. Xu

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

The bag-of-visual-words (BoW) model is effective for representing images and videos in many computer vision problems, and achieves promising performance in image retrieval. Nevertheless, the level of retrieval efficiency in a large-scale database is not acceptable for practical usage. Considering that the relevant images in the database of a given query are more likely to be distinctive than ambiguous, this paper defines “database saliency” as the distinctiveness score calculated for every image to measure its overall “saliency” in the database. By taking advantage of database saliency, we propose a saliency- inspired fast image retrieval scheme, S-sim, which significantly improves efficiency while retains state-of-the-art accuracy in image retrieval . There are two stages in S-sim: the bottom-up saliency mechanism computes the database saliency value of each image by hierarchically decomposing a posterior probability into local patches and visual words, the concurrent information of visual words is then bottom-up propagated to estimate the distinctiveness, and the top-down saliency mechanism discriminatively expands the query via a very low-dimensional linear SVM trained on the top-ranked images after initial search, ranking images are then sorted on their distances to the decision boundary as well as the database saliency values. We comprehensively evaluate S-sim on common retrieval benchmarks, e.g., Oxford and Paris datasets. Thorough experiments suggest that, because of the offline database saliency computation and online low-dimensional SVM, our approach significantly speeds up online retrieval and outperforms the state-of-the-art BoW-based image retrieval schemes.
Original languageEnglish
Pages (from-to)359-369
Number of pages11
JournalIEEE transactions on multimedia
Issue number3
Publication statusPublished - 8 Jan 2015

Keywords / Materials (for Non-textual outputs)

  • database management systems
  • image representation
  • image retrieval
  • probability
  • support vector machines
  • BoW model
  • Oxford dataset
  • Paris dataset
  • S-sim scheme
  • bag-of-visual-words model
  • bottom-up saliency mechanism
  • database saliency
  • distinctiveness score
  • image ranking
  • low-dimensional linear SVM
  • posterior probability
  • query expansion
  • saliency-inspired fast image retrieval scheme
  • top-down saliency mechanism
  • video representation
  • Image retrieval
  • Probability
  • Support vector machines
  • Vectors
  • Visual databases
  • Visualization
  • Bag-of-visual-words (BoW)
  • bottom-up saliency
  • top-down saliency


Dive into the research topics of 'Database Saliency for Fast Image Retrieval'. Together they form a unique fingerprint.

Cite this