Revision history [back]

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything.

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of points within radius r, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at histogram of oriented gradients for inspiration), ...

you'll need a data structure that gives you nearest neighbor points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything.anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of points within radius r, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at histogram of oriented gradients for inspiration), ...

you'll need a data structure that gives you nearest neighbor points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of points within radius r, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at histogram "histogram of oriented gradients gradients" for inspiration), ...

you'll need a data structure that gives you nearest neighbor points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of points within radius r, distances to nearest n non-black points, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at "histogram of oriented gradients" for inspiration), ...

you'll want to come up with some logic that establishes a mesh relationship from a set of points, so you can walk it. you'll need a mechanism to establish "direct neighbors" for a given point. from your picture that can be 5-7 neighbors. I'd do a... 7-nearest query, figure the median distance of those, and keep those that are close to the median (discarding the ones further away, if there were any).

you'll need a data structure that gives you nearest k-nearest neighbor (KNN) points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of points within radius r, distances to nearest n non-black points, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at "histogram of oriented gradients" for inspiration), ...

you'll want to come up with some logic that establishes a mesh relationship from a set of points, so you can walk it. it, and also detect if you're trying to match points that don't have matching neighborhoods (a feature for the previous paragraph). you'll need a mechanism to establish "direct neighbors" for a given point. from your picture that can be 5-7 neighbors. I'd do a... 7-nearest query, figure the median distance of those, and keep those that are close to the median (discarding the ones further away, if there were any).

you'll need a data structure that gives you k-nearest neighbor (KNN) points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of direct neighbors (see next paragraph), number of points within radius r, distances to nearest n non-black points, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at "histogram of oriented gradients" for inspiration), ...

you'll want to come up with some logic that establishes a mesh relationship from a set of points, so you can walk it, and also detect if you're trying to match points that don't have matching neighborhoods (a feature for the previous paragraph). you'll need a mechanism to establish "direct neighbors" for a given point. from your picture that can be 5-7 neighbors. I'd do a... 7-nearest query, figure the median distance of those, and keep those that are close to the median (discarding the ones further away, if there were any).

you'll need a data structure that gives you k-nearest neighbor (KNN) points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have a lot of structure to work with. there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

if that situation doesn't apply... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of direct neighbors (see next paragraph), number of points within radius r, distances to nearest n non-black points, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at "histogram of oriented gradients" for inspiration), ...

you'll want to come up with some logic that establishes a mesh relationship from a set of points, so you can walk it, and also detect if you're trying to match points that don't have matching neighborhoods (a feature for the previous paragraph). you'll need a mechanism to establish "direct neighbors" for a given point. from your picture that can be 5-7 neighbors. I'd do a... 7-nearest query, figure the median distance of those, and keep those that are close to the median (discarding the ones further away, if there were any).

you'll need a data structure that gives you k-nearest neighbor (KNN) points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

there's literature on graph isomorphism. I'm sure this problem is easier than that. you have you'll also want a procedure to warp your whole mesh (move all the points) given a few control points. you'd find initial matches, warp the previously determined mesh to match, and that gets you a lot of structure to work with. closer to the solution. you'd then refine that mesh's point locations given the current frame's points.

there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.

I'm unaware of anything in OpenCV that is a complete solution to this. I hope you'll get answers that come closer to a solution.

you should first find all/most points, calculate a centroid for each. you'll need that as a starting base for pretty much anything. lots of ways to do that. thresholding and contours/connected components, blob detection, convolution...

if you're lucky you can track them individually... if you have sufficiently small movement frame-to-frame for "nearest point in next frame" to be unambiguous. match a point in one frame to the nearest point in the other frame. see if they're a match in the other direction too for stronger evidence.

you can even try optical flow. that may or may not see the coarser structure of your pattern (which is important for movement farther than half a mesh edge). try opencv's DIS optical flow. it's faster and better than the older methods in OpenCV. you can map points from the previous frame through the flow field and match them to the nearest point in the current frame (or the opposite).

if that situation doesn't apply... none of that works... you'll have to come up smarter ways to give a point an identity. that can be done with "feature descriptors", which use local structure to describe any point in an image. you can match those descriptors to establish correspondence. you'd start from good/strong matches and refine the mesh into the less identifiable points.

throw an existing feature descriptor on the problem, such as SIFT or AKAZE. they come with their own feature detector (source of interesting points) but you can also feed them your points.

if none of those do too well, you can engineer your own descriptor. things you can turn into description/identity: number of direct neighbors (see next paragraph), number of points within radius r, distances to nearest n non-black points, radius of nearest n points, count of each color (histogram) for closest n/radius, some measure of gradients or orientations (look at "histogram of oriented gradients" for inspiration), ...

you'll want to come up with some logic that establishes a mesh relationship from a set of points, so you can walk it, and also detect if you're trying to match points that don't have matching neighborhoods (a feature for the previous paragraph). you'll need a mechanism to establish "direct neighbors" for a given point. from your picture that can be 5-7 neighbors. I'd do a... 7-nearest query, figure the median distance of those, and keep those that are close to the median (discarding the ones further away, if there were any).

you'll need a data structure that gives you k-nearest neighbor (KNN) points for any point efficiently, so you can visit the mesh and propagate correspondence throughout it. OpenCV comes with FLANN, which can do that. it's mostly used to compare feature descriptors (vectors of numbers/bits) but you can use it on plain 2d coordinates too.

you'll also want a procedure to warp your whole mesh (move all the points) given a few control points. you'd find initial matches, warp the previously determined mesh to match, and that gets you a lot closer to the solution. you'd then refine that mesh's point locations given the current frame's points.

there may be literature on your specific problem, maybe even proof of concept source code. you'll have to use search engines and dig through journals. the field of computer graphics likely has info on this. mesh processing is their area of competence.