Question about Bag of Words, detectors, and such

edit

I ran out of space, but I also wanted to know about the BoW vocab size. It seemed like 2000 vocabulary items seemed to be where it started to drop in accuracy(from some paper I saw, I will try ot find it) so I'm curious about how bad it's affected as it grows.

If we have a histogram compare we shoudl be able to at least give a few options, and I would assume that even if we have 100,000 items we still would get the "correct" answer, we might just have a bunch of others that also fit within the spectrum, right?

I'm not sure how many vocab items I will have, but eventually it could be 100,000, I might be able to categorize and such, but my total will definitely be a huge number.

BoW is used to form a global image descriptor from local ones, or in other words: local descriptors are encoded to a global one. Since GIST is already a global one you don't need this.

There are different opinions on the vocabulary size, some papers state that with growing size also the accuracy increases, in others it drops at some point.

I don't know what you exactly meant with 'spectrum'. In an image retrieval case, you are comparing the BoW-Descriptor from one image with all others and take the best one. If it's a recognition task you classify the BoW-Descriptor and get the class of the image.

The last sentence appears that you haven't understood the concept totally (or I misinterpret it). You don't build a vocabulary for each individual image, instead you need to build one single vocabulary from a set of training images (i.e. clustering let's say 100k local features from several images) which is used afterwards to encode the local descriptors for each individual test image --> frequency histogram of nearest cluster-center = BoW descriptor.

Huh, a global descriptor, crazy... So it's a giant descriptor that contains all of the information about it's parts(the individual descriptions of each)? Is this what the "Cluster" function would do?

It seems GIST is what I would be looking at then, you seem to recommend it as well.

Glad to hear that there are varying opinions on size.

I meant spectrum with histograms. I thought that when we get the descriptor we find the images with the closest resemblance, then do a histogram compare and see which ones are the closest resemblance(since there could be more than one that it closely resem

I am going to need to recognize items, but then find out what it is. Like I mentioned a bottle, I could have multiple companies, coke, sprite, fanta, but I need to identify that bottle.

Sorry, the last sentence was something I just made up in case there was an issue with a giant vocab.

I understand that we use multiple images to create this vocabulary. I also want to make sure we can constantly update this vocabulary? I guess it shouldn't matter when we create a new cluster with new features since the old ones will still be there.

I also find it interesting there aren't m any matchers, just a few brute force methods I saw and the FLANN. I'm assuming I will still be using a matcher with the GIST-Descriptor?

Thanks again for the help.

EDIT: It's a bit difficult to find GIST-Desciptors. I found a link here to a C++ implementation http://www.cs.cornell.edu

http://stackoverflow.com/questions/10051221/lear-gist-descriptor-c-code-used-with-c

GIST is a global image descriptor, I don't know how high dimensional it is.

No, this is not what the cluster-function would do, it just clusters the training-features and the means are used to encode the test-features.

Updating the vocabulary is typically not that easy! You can't just add new clusters. For k-means the number of clusters have to be given in advance. But you don't need to do this. Only if your training-features are very different to your test-features.

Well, you need some form of matcher to get the nearest cluster centers to encode your feature vectors.

@Guanta, I'd search a query image in a database of unique images. For those unique images, I'd construct a BoW vocabulary with N number of clusters where N 1)either would be as large as the final number of images I will have in database, 2) or be change dynamically and correspond to the number of images in my database. Does it make sense to create a codebook from BoW visual words and use that to match query images with only one unique image in database? Also, if you can, please answer my question here: http://answers.opencv.org/question/44670/bow-number-of-words-in-vocabulary-for-unique/

1) this is not neccesary, see my answer to your other question. 2) every change of clusters would correspond in a retraining -> costs time -> typically not needed if training set is already diverse enough.

Yes you can train another codebook to use it for matching, however typically you wouldn't use k-means but just a kd-tree (see e.g. the good flann library). However, if the number of classes grows you'd need to retrain that as well (the same would happen with other real classifiers).