Connected component in feature space to capture high level semantics in CBIR

Abstract

An important problem in Content Based Image Retrieval (CBIR) systems is the gap between the human high-level semantics and the low-level machine features. In this paper, we develop a novel approach based on the intuition that a query along with the responses from the user during a relevance feedback session provides sufficient cues for learning multiple high-level concepts associated with the query image. For example, a single query image showing a yellow rose contains several high-level semantics such as yellow roses, any rose flower, any yellow coloured flower, a flower, a flower in front-view, etc. Unlike the past approaches that modelled positive responses from the user as a single class with a unimodal probability distribution function, we show that it is more appropriate to group them into multiple connected components in the feature space. It is demonstrated that these components capture and differentiate between the various semantics of an image. We also show that these components may be computed automatically by using a Gaussian Mixture Model. Results on several images illustrate the potential of these connected components to capture the multiple semantics of an image. Copyright 2011 ACM.