After several months of intense development
alongside Stacey Kuznetsov,
the Rapid Content-Based Image Search (RCBIS) engine was ready
for use in user oriented programs. The most obvious and appealing
project was to develop an image search engine which took not
text, but a sketch as input. Search by Sketch was born - developed
by Stacey Kuznetsov, Teobaldo Fernandez and myself.
Mirroring the RCBIS color model, a limited
array of colors is available to users. A usability study found
that users wanted to draw unnecessary detail, which wasted
time and actually hurt search accuracy. Only one, large brush
size is provided to preclude this behavior. The red 'X' icon
clears the sketch.
Users can draw into two different sketch
panels. The upper panel is used to search for image content.
Images that closely matched this sketch are ranked highly
in the search results. However, images that have commonalities
with the sketch drawn in the lower panel are negatively weighted
in the search results. These two sketch panels provide a mechanism
that acts like a boolean search.
Search results are presented in order of
relevance, with the closest matches at the top. To provide
additional detail for probable matches and include as many
results on screen as possible, thumbnail size is varied. Since
our image search algorithm is supposed to be fast (as per
the 'Rapid' in 'RCBIS'), we can hit the DB many times a second.
This allows us to present live search results as users are
sketching their input.
Note: Some people may look at the search results and think that
many (or most) of the images don't match in the slightest. It is
important to keep in mind that beyond the intended image (often
the top result), no other images may look similar in the dataset.
The test database contained about 3000 images. During the video,
the database was queried once per second while a user was sketching
their query; the database was remote. The polling frequency can
be configured, typically set at less than 200 ms if the DB is local.
