How to detect a book!
- We have a bookshelf in a room. It contains ~500 books. The books are either tidily piled on the shelves (horizontal) or are standing side by side (vertical) or tilted/resting on another book or pile of books (not horizontal or vertical, an angle is present in this case) . Note that for each of the books, only the book's spine is visible.
- We have created a database containing the books' names and 1 image for each book (the spine I mentioned before).
- We take a Hi-res picture of the bookshelf. Because we used a camera, perspective causes some of the books near the sides of the image to appear with non-rectangular shapes (maybe not that important though) .
- Some of the books are present in quantities more than one. Some of the books have non-English titles but each of them is unique and distinguishable from others altogether.
Okay, now the quest is :
We want to detect one of the books ( e.g. record 232 in database) in the bookshelf. . We must be able to detect each of the books that we have in database .
------Original Question----- I tried to use SIFT and SURF methods to detect an object (a book in a bookshelf, , scaled / maybe rotated but facing the camera) while having the original image (book cover , fixed and straight image) but I was not successful! At first I thought it was such an easy detection, because " they recognize faces, detecting a book which is only scaled and rotated is easy).
Am I right when trying to use SIFT and SURF? they dont work when I use higher resolution images, and I think they might be overkill for such a task. Please tell me if I'm going the right way :) ?
To make it clear, I have a rectangular image of a book, and an image with many different books (no english letters) , and I want to detect the rotated/scaled book in the bigger image.
Something like this? http://www.pyimagesearch.com/2014/09/... . PS: If you want a C++ version, there's one in the book "OpenCV Essentials" at end of the chapter 3.
Well, not exactly. More like this scenario : I have an image of "Harry Potter Book - Volume 3" 's SPINE (we only have image of the spine of the book) , now we need to detect and draw a red rectangle on this specific book (not any others) in This Bookshelf . )
Oh I see. In that case the edge detector will really not help. But maybe this could help. What you have to do in that case is train your own classifier to be able to reconigze where is the bunch of pixels in an image that can be a Harry Potter Book - Volume 3" 's SPINE. To train your own classifier you can use Haar Cascade technique described here and used on coding-robin.de tutorial (first link). You only need to substitute the banana by the Harry Potter Book - Volume 3" 's SPINE. :)
Thanks, I read about Haar Cascade now; It was ... enlightening ;) Yet, I think there is a difference: In Haar Cascade technique, you try to train detecting an object (banana) , by using various images of banana (40 pics) versus a large number of non-target objects (600 random pics without banana). In my scenario, I have an image from an object (book cover/spine) which is always the same as any instance of it (all the books are printed exactly the same, their covers/spine included). So I have only 1 picture to train for each of the books that I want to detect. And If I have a database of, lets say, 100 books, training a Haar cascade for each of the books would relatively require at least 100 x ( ~600) images, and I assume it takes a long long time to train, too.
I have edited the question to be clearer.
I would go for feature homography I am not sure that you have tried it from your question.
@thdrksdfthmn I think I have tried that. as far as I remember, the problem was there were a lot of feature similarities and a million points and lines everywhere, couldn't actually "detect" and corner out a single book. This method does not take into account the solid rectangular being of a book! letters on a book title may be found on many other books. So is their color. But the whole image of a book spine is unique in comparison to others .Am I wrong here?
It is strange, because they try to detect a book between other books... I have not tried now: You should use homography, because that is going to really detect it, otherwise you will have a lots of matched features... Other idea is template match
I agree with @thdrksdfthmn , you should go for Homography idea. Could you try to write the code using it? That way if some error occurs on that, you could post here and we can try to help you.
I wrote the code using This example . Results? not so good.