Ask Your Question
0

How to write this interpolate function using OpenCV?

asked 2017-05-20 09:45:24 -0600

lovaj gravatar image

updated 2017-05-21 04:23:04 -0600

I want to optimize this code, in particular this function:

bool interpolate(const Mat &im, float ofsx, float ofsy, float a11, float a12, float a21, float a22, Mat &res)
{         
   bool ret = false;
   // input size (-1 for the safe bilinear interpolation)
   const int width = im.cols-1;
   const int height = im.rows-1;
   // output size
   const int halfWidth  = res.cols >> 1;
   const int halfHeight = res.rows >> 1;
   int dim = res.rows * res.cols;
   float *out = res.ptr<float>(0);
   for (int j=-halfHeight; j<=halfHeight; ++j)
   {
      const float rx = ofsx + j * a12;
      const float ry = ofsy + j * a22;
      #pragma omp simd
      for(int i=-halfWidth; i<=halfWidth; ++i)
      {
         float wx = rx + i * a11;
         float wy = ry + i * a21;
         const int x = (int) floor(wx);
         const int y = (int) floor(wy);
         if (x >= 0 && y >= 0 && x < width && y < height)
         {
            // compute weights
            wx -= x; wy -= y;
            // bilinear interpolation
            *out++ = 
               (1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x)   + wx * im.at<float>(y,x+1)) +
               (       wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));
         } else {
            *out++ = 0;
            ret =  true; // touching boundary of the input            
         }
      }
   }
   return ret;
}

Can anybody please help to find an equivalent function in OpenCV for the code above? I'm not expert on image processing, so I don't really know what the fucntion above does in details, but I think this is a warp-affine transformation, even though I don't really know how to define an OpenCV equivalent.

It seems that the input images are the blurred image, the original image or small patches of it. The output result is always a small patch.

Here are some samples:

Sample 1:

Args:

ofx=175.497 ofsy=315.06 a11=1.69477 a12=0.0671724 a21=0.0679493 a22=1.56309

Input image:

enter image description here

Output image:

enter image description here

Sample 2:

Args:

ofx=572.121 ofsy=326.659 a11=0.871508 a12=0 a21=0.346405 a22=1.14744

Input image:

enter image description here

Output image:

enter image description here

Sample 3:

ofx=66.571 ofsy=148.991 a11=1.12027 a12=0.126609 a21=0.126609 a22=2.53436

Input image:

enter image description here

Output image:

enter image description here

These are different sections of the original code where the function is called:

  // warp input according to current shape matrix
  interpolate(wrapper.prevBlur, lx, ly, u11*ratio, u12*ratio, u21*ratio, u22*ratio, img);

  Mat smoothed(patchImageSize, patchImageSize, CV_32FC1, (void *)&workspace.front());
  // interpolate with det == 1
  if (!interpolate(img, x, y, a11, a12, a21, a22, smoothed))
  {

  // subsample with corresponding scale
  interpolate(smoothed, (float)(patchImageSize>>1), (float)(patchImageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);


  // ok, do the interpolation
  interpolate(img, x, y, a11, a12, a21, a22, patch);

This is what happens when I try to use warpAffine in the following way:

  // warp input according to current shape matrix
  interpolate(wrapper.prevBlur, lx, ly, u11*ratio, u12*ratio, u21*ratio, u22*ratio, img);

  Mat warp_mat( 2, 3, CV_32FC1 );
  Mat myPatch(img.rows, img.cols, CV_32FC1);
  warp_mat.at<float>(0,2) = lx;
  warp_mat.at<float>(1,2) = ly;
  warp_mat.at<float ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
2

answered 2017-05-20 17:25:34 -0600

Tetragramm gravatar image

Ok, when I run it, the results didn't match what you uploaded, but I did figure out how to make warpAffine function just like it.

Mat affine( 2, 3, CV_64F );
affine.at<double>( 0, 0 ) = a11;
affine.at<double>( 0, 1 ) = a12;
affine.at<double>( 0, 2 ) = ofsx;
affine.at<double>( 1, 0 ) = a21;
affine.at<double>( 1, 1 ) = a22;
affine.at<double>( 1, 2 ) = ofsy;
invertAffineTransform( affine, affine );
affine.at<double>( 0, 2 ) += output.cols / 2.0;
affine.at<double>( 1, 2 ) += output.rows / 2.0;

warpAffine( image, output, affine, Size( output.cols, output.rows ) );

warpAffine, unlike the function there, is capable of extrapolating past the edge of the image, so you may not need the return value. If you do, just use the BORDER_CONSTANT with a value outside your image range (say -999), then use the checkRange function with your image range to get the boolean value.

edit flag offensive delete link more

Comments

@Tetragramm thanks for your answer, you are the one who went closer to an actual solution between all the people that I've talked to! However (as you can see in my updated answer), the result is kinda approximated, i.e. some values are different...For example, you can see how in img some values are different from myImg obtained through your approach. I don't know if there is a solution for this. Btw, I would need the boolean value, could you please explain a little better the last part?

lovaj gravatar imagelovaj ( 2017-05-21 04:25:54 -0600 )edit

You need it exactly exactly? Ee, that's hard. I'll take a look though.

For the boolean. You have a range of value (0-255) that are valid image values. You need true when the entire patch was within the image, and false if any part of it was outside the image. warpAffine has a flag that determines how it handles the border. The default it BORDER_CONSTANT, where it takes the last parameter and uses that as the value for any portion outside the image. If you set the last value to something far outside the range (say -999), then you only see -999 when you need a false. The checkRange function is just an existing way to see if there is any values outside of the range.

Tetragramm gravatar imageTetragramm ( 2017-05-21 08:53:22 -0600 )edit
1

Yeah, I'm not sure which is the approximated one, but the interpolation works differently between them. You're not going to get them exactly the same, without just re-writing the method you have.

Sorry, I tried them all, and still differences. Small differences, but they are there.

Tetragramm gravatar imageTetragramm ( 2017-05-21 09:06:05 -0600 )edit

Thanks @Tetragramm I think you did the best you could, I really appreaciate it and your help was crucial in my project! Chapeau sir! :)

lovaj gravatar imagelovaj ( 2017-05-21 10:32:59 -0600 )edit

@lovaj. I am also trying to implement the bilinear interpolation after affine transformation. Could you please tell me how did you locate the newly created subpixels??.

Priya KUMARI gravatar imagePriya KUMARI ( 2017-06-12 08:48:02 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2017-05-20 09:45:24 -0600

Seen: 1,625 times

Last updated: May 21 '17