A sparse kernel relevance model for automatic image annotation

Sean Moran, Victor Lavrenko

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

In this paper, we introduce a new form of the continuous relevance model (CRM), dubbed the SKL-CRM, that adaptively selects the best performing kernel per feature type for automatic image annotation. Previous image annotation models apply a standard selection of kernels to model the distribution of image features. Popular examples include a Gaussian kernel for modelling GIST features or a Laplacian kernel for global colour histograms. In this work, we demonstrate that this standard assignment of kernels to feature types is sub-optimal and a substantially higher image annotation accuracy can be attained by adapting the kernel-feature assignment. We formulate an efficient greedy algorithm to find the best kernel-feature alignment and show that it is able to rapidly find a sparse subset of features that maximises annotation F1 score. In a second contribution, we introduce two data-adaptive kernels for image annotation—the generalised Gaussian and multinomial kernels—which we demonstrate can better model the distribution of image features as compared to standard kernels. Evaluation is conducted on three standard image datasets across a selection of different feature representations. The proposed SKL-CRM model is found to attain performance that is competitive to a suite of state-of-the-art image annotation models.
Original languageEnglish
Pages (from-to)209-229
Number of pages21
JournalInternational Journal of Multimedia Information Retrieval
Issue number4
Publication statusPublished - Nov 2014

Keywords / Materials (for Non-textual outputs)

  • Image annotation
  • Object recognition
  • Kernel density estimation


Dive into the research topics of 'A sparse kernel relevance model for automatic image annotation'. Together they form a unique fingerprint.

Cite this