A Modular Architecture For Content Based Image Retrieval Systems

Presentation

Our software is a new architecture for building CBIR software systems, based on a client-server architecture, designed and implemented at the IMEDIA Project research team, INRIA Rocquencourt, France.

From the user point of view, it aims to be flexible, easily extensible, easy to use, intuitive, and does not enforce special knowledge or training.

This CBIR engine is organized on a client-server architecture. The server part is written in C++ (for speed reason) and the client part is written in Java and it should normally run on any computer architecture that supports Java Runtime Environment (JRE). The client and the server communicate with each other through a protocol, which is a set of commands the server understands and a set of answers it can return to the client.

By default, the server performs “retrieval by visual similarity” in response to a query, which means that it searches all images in the database and returns a list of the most visually similar images to the query image.

Our prototype allows also region based queries and has hybrid text-image retrieval mode. In the region based mode, the user can select a part of an image and the system will search images (or parts of images) that are visually similar to the selected part.

If an image database has been annotated with keywords, the server can use these keywords for fast retrieval (in this case the search in not based on visual similarities and depends on the quality of keyword indexing). Based on the indexed visual features and the keywords index, it can suggest a number of keywords for a non annotated image. The engine also includes some face detection and recognition signatures.

Relevance feedback (RF) methods try to use the information the user supplies to the system in an attempt to “guess” what are her intentions, thus making it easier to find what she wants. Relevance feedback directed queries may help to protect the user from unwanted technicalities. Our engine has a RF mode for category search in image databases, which means it can help the user to find more rapidly certain categories of images in large databases.

Screenshots

Here we present some screenshots that illustrate the visual searching capabilities of our prototype (click on the thumbnails for a larger version of the screenshot):

The left thumbnail shows an image selected by the user in a generalist database and the right thumbnail shows the answer returned by the server in response to a visual search query.