Template Matching is wrong with specific Reference image

asked 2014-11-26 08:25:48 -0600

Finnish
41 ●2 ●3 ●7

updated 2017-12-24 03:11:37 -0600

6772 ●3 ●48 ●79 https://github.com/stu...

Until now I have always gotten reasonable results withc matching score, Im using "Imgproc.TM_CCOEFF_NORMED" method for template matching below are the results.

Reference image:

image description

Outcome 1:

image description

MinMaxLocResult mmr = Core.minMaxLoc(result);
maxMatchScore = mmr.maxVal;

Value of maxMatchScore is always 1.0

Outcome 2:

image description

MinMaxLocResult mmr = Core.minMaxLoc(result);
maxMatchScore = mmr.maxVal;

Value of maxMatchScore is always 1.0

1.0 is the best match value, which means the image is %100 identical, but it isn't and on other reference images I get other values, so the program is working but only with this specific reference image I'm getting 1.0 also no matter what in file(source image) I give, it always finds the match on the first search upper left corner. A little bit annoying, I would be glad for your suggestions and comments for improvement because TM_CCOEFF_NORMED gives always the first searched square/rectangle as best match with score 1.0, this cant be correct, on the other hand I also tried TM_CCORR_NORMED and TM_SQDIFF_NORMED they gave different match scores, this is promising but still TM_CCORR_NORMED gave good matching score which is still unexpected from my side. I would be glad if someone can explain me the difference between the matching methods or give a link to already an existing page where these methods are being discussed, on opencv docs/tutorials there are only the formulas available but no detailed explanation. In the end I would like to know when to use which match method for what kind of image to get the best results.

Here is some more code:

    Mat img = Highgui.imread(inFile);
    Mat templ = Highgui.imread(templateFile);

    //  Create the result matrix
    int result_cols = img.cols() - templ.cols() + 1;
    int result_rows = img.rows() - templ.rows() + 1;
    Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);

    //  Do the Matching 
    Imgproc.matchTemplate(img, templ, result, match_method);
    //  Localizing the best match with minMaxLoc
    MinMaxLocResult mmr = Core.minMaxLoc(result);

If I add normalization after the matching with

     Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());

Then I become matchscore 0.0, which satisfies me and what I would expect, but this way gives only 1 or 0 nothing else in between, I would accept this if it was always giving correct results but sometimes it gives 0.0 for very good matches as well, i.e. when I cut a part from an image and try to find the cut part in the original image, then it gives 0.0, which should be the case... Also having values like 0.8734543, 0.234356, 0.450985948, helps me a lot to understand and analyze the comparisons, I have threshold of 0.9, any value above 0.9 I accept as correct, found.

edit retag flag offensive close merge delete

Comments

what is `result`?

thdrksdfthmn ( 2014-11-28 02:12:01 -0600 )edit

@thdrksdfthmn result is a Mat not very relevant... with the matchscore being "1.0"

Finnish ( 2015-05-04 08:16:28 -0600 )edit

add a comment

answered 2015-11-18 13:15:47 -0600

pklab

4284 ●7 ●30 ●60

updated 2015-11-19 12:10:17 -0600

Asking about different matching methods is a frequent question... I can integrate my old answer with this.

Before all, formulas in the doc explain exactly what happens for each pixel right. Here is an attempt to explicate them. All contributions are welcome.

TM_SQDIFF is Sum of Square Difference (SSD). It's fast but results depend on overall intensity
TM_CCORR (sometimes called Cross Correlation) is SSD under the circumstance that the sum of portion of the image under matching is constant when the template moves over the image or simply the image is almost constant. I look at this like worst method at all because it depends too much on overall intensity... you can have false detection also using a perfect template, if image has big variation in intensity.ref
TM_CCOEFF (often called Cross Correlation) is correlation coefficient. It consider the mean of template and the mean of the image so that the matching is most robust to local variation ref. I think this is the best method. Downside: is slower.

All of above give results that can't be used for comparison because, matching value depends from image,template, size and intensity. If you want just to find where your template best matches over the image, you can use one of above (I think TM_CCOEFF has more sense)

If you want a good matching (to accept or discard the matching), you have to use the _NORMED version of methods because they scale the result with image,template,size and intensity and provide output in a fixed range 0..1 where 1 means perfect match (with SQDIFF 0 means perfect)

Be careful because many examples uses to normalize (again) the result of the matching. If you do this you will see always a perfect matching because of normalization on result !

here some examples. Green rectangles show the best match found. As you can see non _NORMED methods give useless score.

Template has been brighten image description Results show bad false positive for SQDIFF and CCORR both standard and normed. while COEFF_NORMED is ~1

Template with noise and brlur image description Again CCORR_NORMED has bad false positive with a score of 0.9. COEFF_NORMED is coherent with score 0.6

edit flag offensive delete link

Comments

I made some tests and TM_COEFF_NORMED was giving the best results so I stick with it, but now you are saying TM_COEFF is Normalized Cross Correlation, so when I use TM_COEFF_NORMED, am I using Normalized Normalized Cross Correlation? Although the 2 images are very different, why does it give 1.0 the best match? and why always on first top left corner? Also if I do normalizing after matching after matching like

 Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());

Does it become 3 times Normalized? In case of adding the last line of normalization code, I get matchscore 0.0, which satisfies me but with the last line it is always either 1 or 0... nothing else, and sometimes it gives 0 for good matches

Finnish ( 2015-11-19 05:01:55 -0600 )edit

Ok it was my fault because so named Normalized Cross Correlation correspond to TM_CCOEFF_NORMED formula (now it's correct).

With match_method=TM_CCOEFF_NORMED you should have values between 0..1. (~1 only in case of perfect match). You don't have to normalize the matching result otherwise you will have 1 every time! just select the match where score is above of your desired threshold.

I think you are doing some mistake with the code because you have always perfect match in (0,0). Check the your code with the tutorial

pklab ( 2015-11-19 12:15:23 -0600 )edit

@pklab thanks for updating the answer, my code I assume is correct because out of 100 images I always find the images and have a good match score, it cant be coincidence, just only this specific full green reference image, always finds at 0,0 and with score 1.0 when using TM_COEFF_NORMED, please feel free to give it a try yourself. Not only this specific green image but on full red, blue etc. colors the story is the same.

Finnish ( 2015-11-20 05:34:00 -0600 )edit

I see so how can I overcome this obstacle? Changing the match method for every specific image will be very troublesome, for the specific case when I use "TM_CCORR_NORMED" the score is 0.93 which is misleading and when I use "TM_SQDIFF_NORMED" I get 0.79, which is promising because it is a bad match, but in general I would like to use "TM_CCOEFF_NORMED" for all images... Also one more question is the template image and source image converted to grayscale and then the math operations take place? Or there is no grayscale conversion?

Finnish ( 2015-11-20 07:47:14 -0600 )edit

@Finnish I deleted my previous comment because it was wrong (may be I was hungry and I replaced mul with add in my mind, sorry).

Now:You are using a template that is flat. When the template or the image patch under template is flat (constant), TM_CCOEFF_NORMED is undefined because result is 0/0 but OpenCV implementation returns 1.0 to avoid math exception ! Here is the issue

I will open a new tickets on this, in the mean time avoid to use TM_CCOEFF_NORMED with flat template or in general if TM_CCOEFF==0.

My curiosity, why you are using flat template ? Just for test ?

pklab ( 2015-11-20 09:29:08 -0600 )edit

Thanks again! I am searching for a park polygon in the map or lake which are fully green and blue... Could you please add the mathematical operations again, maybe on your answer, how does the result become zero... for my better understanding.

Finnish ( 2015-11-23 07:17:59 -0600 )edit

@Finnish if you want to search for some constant in your image you could use a running ROI over original image and for each ROI

Check for values within a range around your mean with inRange .
countNonZero on result
than get normalized score with countNonZero/ ROI area

For further information about the issue on TM_CCOEFF_NORMED please follow issue #5688

pklab ( 2015-11-23 12:25:35 -0600 )edit

@pklab - I used your answer but even when I blur the image I still get very high scores on incorrect templates (0.96) just as on correct templates. (correct templates: templates that really appear in the image. incorrect templates: templates that don't really appear in the image). I tried blurring with blur() / GaussianBlur() / medianBlur(). Any thoughts?

AlaaM ( 2017-11-21 10:07:42 -0600 )edit

add a comment

Template Matching is wrong with specific Reference image

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Template Matching is wrong with specific Reference image edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Template Matching is wrong with specific Reference image