Content-based image retrieval


"Content based image retrieval" (CBIR) is also known as "Query by image content" or 'QBIC' in short. As the name suggests, here the query is to find set of the best matched digital images from the set of large databases.Here "Query" can be anything - whether some words or attributes defining the relevant images, or a sample image itself depending on the algorithm used. 

In "Content-based" search, the contents of the image is analyzed rather than metadata of the image like keywords, tags, or description of image. "Content of an image" includes characteristics of an image like colors, textures, shapes or any information which are extracted from image. Naturally, CBIR is more desirable than previous search techniques because its very time consuming having humans to manually annotate the metadata with the images in a large database. Moreover, sometimes humans may not capture the keywords to describe an image. For all these reasons CBIR emerges out to be a time efficient and optimal method.

Various methods have been proposed in order to make the image search more specific and advanced. Some of them are listed below.
  • Query by example - An example image is given in the CBIR system as a query on which the system's search is based upon. The example image can be either a preexisting image or a user can draw a rough approximation of the image they are looking for. This query technique eliminates all the difficulties that can arise when trying to describe images with words.
  • Schematic retrieval: Here user gives a query in words like "Images of mahatma Gandhi". Now its very difficult for the CBIR to perform the search as Mahatma Gandhi wont always be facing the camera. Therefore many CBIR searches make use of low-level features including color, textures and shape.  However, in general, this type of search requires human feedback in order to give more specific results.
  • Relevance Feedback: In this system user progressively refines the search results by marking images in the results as "relevant", "not relevant", or "neutral" to a search query, and then repeating the search with addition of new information.
  • Microsoft research team proposed a technique in which user are provided a text box and the user is allowed to describe spatial positions of the query terms. Now the system gives the result which has the objects having the same orientation or the spatial position in the image.
  • Content comparison using image distance measures:  Here "Image distance" is used to compare between 2 images. Image distance is determined with the help of multiple attributes of an image like color, texture, shape and some others. For the searches it is measured between query and the images in the databases. Lower the Image distance for an image, higher the relevant the image is. And hence this can be used to sort the output result in the preferred of the relevance.

Conclusion:

With all these techniques explained above, CBIR thus becomes a convenient way for searching relevant Images in the databases. After all, no manual work is needed to annotate the images with related keywords and hence less chances of human errors are there. CBIR is currently an active field of research and there is a lot of scope of improvement in this system with the advancement in technologies. CBIR has many applications like in analyzing CCTV footage, Face Finding etc.


References:

  • https://en.wikipedia.org/wiki/Content-based_image_retrieval 
  • https://www.microsoft.com/en-us/research/blog/quest-quality-searches/ 



Comments