How to detect labels with numbers in a black rectangle?
I am an absolute newbie to OpenCV and Computer Vision with only a few weeks of trying to understand this fascinating subject. Please excuse me if my question is "stupid" or otherwise, but we all have to start somewhere. What might be trivial to an expert or Guru is a mountain to a beginner until they are enlightened with the knowledge.
With that aside, I am involved in a project to locate labels with a series of numbers in a black rectangle (see attached photo).
Using the images from a camera, I need to detect and retrieve the numbered rectangles to apply OCR on it and shine a laser pointer on the number that is being searched for. I am able to manually get the rectangle, adjust for its perspective, binarize it and use Tesseract to read it. But I am having difficulty in detecting the label and getting the bounding rectangle of the numbers. I have tried using Cascade Classifiers without much luck and applying adaptive thresholding to get a clearer image of the bounding black rectangle. What would be the best approach to getting the ROI? Is it possible to detect the label using Cascade Classifiers? If it is then please enlighten me as to how train the Cascade Classifier properly because I have followed the instructions, found on the Net, to train them. Do I need to get clearer/cleaner images of the labels and what size is best for the images? I found out there is a limit to how big the positive images can be.
Your suggestions, ideas, constructive comments, and help is greatly appreciated. And I hope to pass on what I learn to others also.
IMHO cascade classifiers are not going to do it, because you may have the black rects in any position, so under any affine perspective transformation (but maybe it will work because it is almost squared)... I think you shall start by a threshold (it is a high contrast there: black - white) and then do some filtering based on the area, and length of the regions.
Another aspect I want to mention is that if you do affine transformations, tesseract is not going to be as good as no transformation. Or at least in my case it did not work well: I have tried OCR on license plates detected on different space positions, and even if I am able to read the text (pretty well), tesseract fails. And if there is someone that is saying it's not true, is welcome to enlight me what am i doing wrong (I've used jpgs) :)
I'm new to OpenCV as well, but I might be able to help. When you take the images from the camera, are they going to be focused on the labels? Or will they be similar to the image you have posted, where the image is distant and you are far away from the labels?
The images are real time from the camera and will be similar distance to that in the picture. The system will check all the numbers in view and shine a laser on the one that is being searched for.if it is there. It's to help people quickly find certain parts by number among others on the shelf. Thanks. Hope to return the favour.
That sounds like cool project! I think you will definitely need closer images than that/possibly take them when you are parallel to the surface so there isn't much perspective. I would guess 2 or 3 feet away would be fine. Once you have the code working on something like that, you can start to optimize it for images that aren't so perfect.
Like thdrksdfthmn said, you should start by setting a threshold so it is high contrast B&W. However, I think that before you do the thresholding to high-contrast black & white, you should invert the image colors. This might make it easier on the Cascade classifiers. Let me know if that helps, and if it doesn't I can think a little bit harder :p. Good luck!!