Ask Your Question
4

2D spline algorithm in OpenCV

asked 2018-06-04 05:29:03 -0600

Martian gravatar image

Does OpenCV have a method for generating a 2D 'spline' contour? IOW, choose gray-scale pixel values at selected points on a 2D map and have them connected by a smoothed 2-dimensional contour. Output would presumably be gray-scale values in Mat form.

The inputs to a regular spline are a set of point coordinates, and output is a continuous curve that connects the input points.

In this case, the input would be a set of point coordinates -each with a gray-scale value-. The output would be a smoothed gray-scale contour that covers the entire square Mat.

That was tougher to explain than I expected. Hope it was clear. Perhaps that would actually be a 3D spline in that gray-scale value would be the 3rd dimension. But this is for generating a 2D image.

edit retag flag offensive close merge delete

Comments

2

no opencv does not have any splines (or any curve-fitting, which is probably more close to what you wanted.)

also note, that the control points for splines are NOT point coords to be approximated (apart from the 1st and last). i just saw this

berak gravatar imageberak ( 2018-06-04 06:08:17 -0600 )edit

The easiest curved line is the Bezier curve:

https://en.wikipedia.org/wiki/Bézier_...

It takes 3D points and interpolates between them. I once used Bezier curves to visualize the quaternions as they are operated on, using Z = Z*Z + C as the iterated equation. If you perform intersection of the curve and the quaternion fractal's surface, you can get a normal (derivative) at the intersection point.

sjhalayka gravatar imagesjhalayka ( 2018-06-04 10:33:17 -0600 )edit

Thanks for the helpful replies. I'm surprised that OpenCV has no built in functions for this. I've seen mention of correcting camera spatial distortion, so I thought this would be related.

OpenCV does have adaptive thresholding, but that goes wrong in areas where there are no foreground objects for contrast. It will detect the more pronounced areas of noise as foreground.

Martian gravatar imageMartian ( 2018-06-04 15:39:27 -0600 )edit

I ran across a good illustration of what I'm looking for in the long run. This shows removal of 'curved' background illumination for a telescope image: http://help.imageanalyst.net/workflow...

Unfortunately, that's a specialized commercial program. And I'm looking for something that could adapt to more complex 2D 'light curvatures.' That's why I had envisioned a spline. But perhaps there are other approaches.

Martian gravatar imageMartian ( 2018-06-04 15:40:55 -0600 )edit

You could accelerate the light (bend the light's natural, inertial path) using gravitation. Don't forget that the bending of light's path (the bend angle) is double when using General Relativity, versus the old Newtonian gravitation. Not that it makes much difference, but you might as well try to do it right the first time.

http://www.einstein-online.info/spotl...

sjhalayka gravatar imagesjhalayka ( 2018-06-04 15:45:42 -0600 )edit

Thanks for the suggestion. I'm not actually working with telescope images. The link illustrated something similar as far as light/illumination correction. It sounds like you're referring to spatial distortion, which I don't need to handle in this case. I have a background field with irregular illumination, somewhat like the telescope image but often more complex. If I could find the general 2D curve for the illumination, I could subtract it to even out the image. Easier said than done.

Martian gravatar imageMartian ( 2018-06-04 15:55:06 -0600 )edit

Yes, good luck. :) And spatial distortion (curvature) only explains Newtonian gravity. The addition of time curvature is what makes GR more like reality.

sjhalayka gravatar imagesjhalayka ( 2018-06-04 15:58:38 -0600 )edit

^^ k,k, enough ... ;)

@Martian, so the whole talk about splines was pretty misleading. maybe you can show an example image of your problem,and what you're trying to achieve ?

I have a background field with irregular illumination, somewhat like the telescope image but often more complex. If I could find the general 2D curve for the illumination, I could subtract it to even out the image

that's probably, what the question should have been.

berak gravatar imageberak ( 2018-06-05 00:42:39 -0600 )edit

Hi Berak, The link that I posted is probably a good example: http://help.imageanalyst.net/workflow... But in this case, instead of telescopes, it's microscopes. Same problem though---irregular illumination, but with variations, like haze or filmy looking areas that sometimes gets misinterpreted as objects.

I didn't think I'd find a specific solution to this so I thought I might be able to 'roll my own' with the spline code.

Martian gravatar imageMartian ( 2018-06-05 02:28:24 -0600 )edit

2 answers

Sort by » oldest newest most voted
3

answered 2018-06-07 11:00:28 -0600

sjhalayka gravatar image

updated 2018-06-22 15:19:55 -0600

OK, now that it's decided that Bezier surfaces might be good enough, here is a link to the function that needs to be implemented:

http://paulbourke.net/geometry/bezier/

The function is:

image description

The function leads to 2D curved surfaces in 3D space:

image description

I’ve implemented the function. When the number of control points spirals upward, so does the amount of computation and variable precision needed. Loading a 1024x1024 image would be only for the very patient. I had to use the Boost multi precision library, with 100 digits, because long long unsigned int and long double were not big enough. If you find that this answer is suitable, please mark it as correct (click on the checkmark).

The code is at:

https://github.com/sjhalayka/bezier_t...

The result is a blurry mess. For instance, the input is 512 x 384:

image description

And the output is 64x64:

image description

You see the main problem now, eh? It's still too smooth?

edit flag offensive delete link more

Comments

1

that's entirely cool, but required was a 2d hyperplane in 3d space ! (x,y,intensity)

(even if it wasn't sooo clear in the question)

berak gravatar imageberak ( 2018-06-07 11:08:41 -0600 )edit

OK cool. Glad you liked my shaggy fractal. :)

sjhalayka gravatar imagesjhalayka ( 2018-06-07 11:14:14 -0600 )edit

i sincerely do. ;)

now go, and solve the problem, lol ;)

your task (if you choose to accept it) would be to model a 2d hyperplane/volume in 3d(x,y,intensity) space, that models additional/artificial lightsources, so those can easily be subtracted from an existing 3d profile (aka- the image)

this message will burn itself in 5 minutes.

berak gravatar imageberak ( 2018-06-07 12:02:06 -0600 )edit

Haha, let me see about it.

sjhalayka gravatar imagesjhalayka ( 2018-06-07 12:42:51 -0600 )edit

@Martian -- do you have a sample of what you want done? I'm not sure I'm catching your drift.

sjhalayka gravatar imagesjhalayka ( 2018-06-08 14:53:40 -0600 )edit

Glad that you're interested, sjh. It sounds like you've done a lot of work with splines and graphics. Below are a few images showing problems that I referred to.

This shows a very simple, gradual shift in background tone: http://genesdev.cshlp.org/content/12/... Ordinarily, adaptiveThreshold will reject that, but if there are no objects in foreground, it starts looking for anything it can find, and ends up detecting peaks in dark/light even if the shift is gradual. This would be a prime example, as you can see that a slowly shifting contour could be subtracted to leave the background completely flat.

This image is tougher, but could possibly be handled after pre-processing w nlmd or bilateral filtering: https://i.ytimg.com/vi/6w1w_Hxvwhs/ma...

Martian gravatar imageMartian ( 2018-06-09 21:56:30 -0600 )edit

The background of these images shows the type of noise that is difficult to reject: https://www.researchgate.net/profile/... Again, that can be pre-filtered with NLMD or whatever, which leaves blurred irregular areas. Those are often rejected, except when there are no nearby objects, in which case they are falsely detected as objects by adaptiveThreshold.

Martian gravatar imageMartian ( 2018-06-09 22:02:29 -0600 )edit

@Martian -- how do curved lines play a role in this? Can you make a sample image with the curved line? Like, make it in GINP.

sjhalayka gravatar imagesjhalayka ( 2018-06-10 17:59:59 -0600 )edit
1

Not a curved line, a curved plane. Start by looking at it as a terrain map, with hills and valleys. If the elevation is then translated into a luminance value, that would look like the gradual shifts in the backgrounds of the images above.

This seems way more difficult to explain than I had envisioned. I had always heard this referred to as a 2D spline, and I believe it is used in exactly this context (background removal in microscope images). So I thought it was common terminology.

Martian gravatar imageMartian ( 2018-06-10 22:15:09 -0600 )edit

@Martian -- Are you at all familiar with OpenGL? This is just for reference: http://www.glprogramming.com/red/chap...

So basically you're looking for a height map, where the intensity is related to the height?

I altered my answer to give you a linear solution. Try that before moving on to curved surfaces -- the OpenGL code I linked to shows you how to get curved surfaces.

sjhalayka gravatar imagesjhalayka ( 2018-06-11 10:47:02 -0600 )edit
4

answered 2018-06-12 03:50:30 -0600

berak gravatar image

well, 2 ideas here:

one could generate a heightmap from x and y curves, like in this silly drawing:

image description

and the resulting intensity distribution would be simply the outer product of the curves:

Mat intensity = curve_x * curve_y.t();

(and that's also where the 2d splines are back !)

-------------------------------------------------------

but also, your background noise is basically a gaussian distribution. (or maybe a mix of gaussians), so probably a simple mouse painting tool (where you subtract such blobs on every click) might do the trick already:

image description

#include<opencv2/opencv.hpp>  
#include<iostream>
using namespace cv;
using namespace std;

Mat gauss_blob(int siz, float sig) {
    Mat k = getGaussianKernel(siz*2+1, sig, CV_32F);
    return k*k.t();
}
int border=800; // max kernel size
int t_int = 220;
int t_sig = 200;
int t_siz = 100;
Mat img;

void on_mouse(int e, int x, int y, int s, void*) {
    if (e==1) {
        Rect r(border+x-t_siz, border+y-t_siz, 2*t_siz+1, 2*t_siz+1);
        Mat patch(img,r);
        Mat blob = gauss_blob(t_siz, float(t_sig)/10);
        patch -= t_int * blob;
    }
    if (e==2) ;// r-click, *add* the blob for undo.
}


int main(void) {
    img = imread("stars.png", 0);
    img.convertTo(img, CV_32F, 1.0/255);
    // make sure we stay inside the image
    Rect show(border,border,img.cols,img.rows);
    copyMakeBorder(img,img,border,border,border,border,BORDER_CONSTANT,Scalar(0));

    namedWindow("gauss");
    createTrackbar("size", "gauss", &t_siz, 800);
    createTrackbar("sigma", "gauss", &t_sig, 800);
    createTrackbar("intensity", "gauss", &t_int, 500);
    setMouseCallback("gauss", on_mouse);
    while(1) {
        imshow("gauss", img(show));
        if (waitKey(5) == 27) break;
    }
    return(0);
}
edit flag offensive delete link more

Comments

@berak -- Please check out this new Bezier curve screenshot: https://github.com/sjhalayka/random_i...

sjhalayka gravatar imagesjhalayka ( 2018-07-14 22:41:09 -0600 )edit

Question Tools

3 followers

Stats

Asked: 2018-06-04 05:29:03 -0600

Seen: 9,744 times

Last updated: Jun 22 '18