Ask Your Question
4

Meaning of Keypoints and Descriptors

asked 2014-07-24 03:18:25 -0600

Keypoints gravatar image

updated 2020-11-30 03:29:06 -0600

Hello,

I have do some research about features detection and I manage to use ORB,SURF,SIFT correctly to dectect objects. But now, I have to write some explanation about how it works. I resume the work in few steps.

  1. Detect Keypoints in image and template
  2. Create Descriptors from the Keypoints
  3. Matches the descriptors of image and template
  4. Keep goodKeypoints from the matches
  5. Find object from the good Keypoints

But there is some points I can't explain very well. What is a Keypoint ? What represent the descriptor of a Keypoint ?

For me a Keypoint is a point of interest detected thanks to methods(SURF,ORB,etc...) which is equivalent to point of interest a human being can see in a image, but i'm not sure if it's the correct definition.

Best regards

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
8

answered 2014-07-24 04:06:14 -0600

FLY gravatar image

Well you can get a very good explanation from here

One important thing to understand is that after extracting the keypoints, you only obtain information about their position, and sometimes their coverage area (usually approximated by a circle or ellipse) in the image. While the information about keypoint position might sometimes be useful, it does not say much about the keypoints themselves.

Depending on the algorithm used to extract keypoint (SIFT, Harris corners, MSER), you will know some general characteristics of the extracted keypoints (e.g. they are centered around blobs, edges, prominent corners...) but you will not know how different or similar one keypoint is to the other.

Here's two simple examples where only the position and keypoint area will not help us:

If you have an image A (of a bear on a white background), and another image B, exact copy of A but translated for a few pixels: the extracted keypoints will be the same (on the same part of that bear). Those two images should be recognized as same, or similar.

But, if the only information we have is their position, and that changed because of the translation, you can not compare the images.

If you have an image A (let's say, of a duck this time), and another image B, exactly the same duck as in A except twice the size: the extracted keypoints will be the same (same parts of the duck). Those are also same (similar) images.

But all their sizes (areas) will be different: all the keypoints from the image B will be twice the size of those from image A.

So, here come descriptors: they are the way to compare the keypoints. They summarize, in vector format (of constant length) some characteristics about the keypoints. For example, it could be their intensity in the direction of their most pronounced orientation. It's assigning a numerical description to the area of the image the keypoint refers to.

Some important things for descriptors are:

they should be independent of keypoint position

If the same keypoint is extracted at different positions (e.g. because of translation) the descriptor should be the same.

they should be robust against image transformations

Some examples are changes of contrast (e.g. image of the same place during a sunny and cloudy day) and changes of perspective (image of a building from center-right and center-left, we would still like to recognize it as a same building).

Of course, no descriptor is completely robust against all transformations (nor against any single one if it is strong, e.g. big change in perspective).

Different descriptors are designed to be robust against different transformations which is sometimes opposed to the speed it takes to calculate them.

they should be scale independent

The descriptors should take scale in to account. If the "prominent" part of the one keypoint is a vertical line of 10px (inside a circular area with radius of 8px), and the prominent part of another a vertical line ... (more)

edit flag offensive delete link more

Comments

1

Thank you very much for the answer. I understand what is the meaning of descriptor now. For the Keypoint, I guess it just an important point estimated by a method (ORB,SURF,SIFT,etc..) and that's all.

Keypoints gravatar imageKeypoints ( 2014-07-24 04:48:00 -0600 )edit

Really good answer I must say!

Doombot gravatar imageDoombot ( 2014-10-28 14:37:23 -0600 )edit
1

answered 2014-07-24 05:53:14 -0600

JohannesZ gravatar image

Who don't you search in a book? The book from Szeliski will give you the answers, that's for sure. It can be downloaded from the author's website (free):

http://szeliski.org/Book/

edit flag offensive delete link more

Question Tools

2 followers

Stats

Asked: 2014-07-24 03:18:25 -0600

Seen: 13,178 times

Last updated: Jul 24 '14