Cuda ORB keypoints

asked 2020-01-19 08:04:36 -0600

MicTie
61 ●1 ●5

I have a little experience with OpenCV (Python) and just received a Jetson Nano and want to test OpenCV with CUDA. So I tried an ORB-example but am stuck with converting GPU-Keypoints to CPU-Keypoints, but I am unable to get any result.


def orb_with_cuda():
        MAX_FEATURES = 100
        GOOD_MATCH_PERCENT = 0.15
        # load images into numpy
        npMat1 = get_sample("im1.jpg")
        npMat2 = get_sample("im2.jpg")        
        # upload into Cuda
        cuMat1 = cv2.cuda_GpuMat()
        cuMat2 = cv2.cuda_GpuMat()
        cuMat1.upload(npMat1)
        cuMat2.upload(npMat2)        
        #convert to Gray
        cuMat1g = cv2.cuda.cvtColor(cuMat1, cv2.COLOR_RGB2GRAY)
        cuMat2g = cv2.cuda.cvtColor(cuMat2, cv2.COLOR_RGB2GRAY)

        #ORB
        corb = cv2.cuda_ORB.create(MAX_FEATURES)
        _kps1, _descs1 = corb.detectAndComputeAsync(cuMat1g, None)
        _kps2, _descs2 = corb.detectAndComputeAsync(cuMat2g, None)

        #convert Keypoints to CPU
        kps1 = [cv2.KeyPoint() for i in range(MAX_FEATURES)]
        kps2 = [cv2.KeyPoint() for i in range(MAX_FEATURES)]
        corb.convert(_kps1, kps1)
        corb.convert(_kps2, kps2)      

        #BruteForce Matching
        cbf = cv2.cuda_DescriptorMatcher.createBFMatcher(cv2.NORM_HAMMING)
        cmatches = cbf.match(_descs1, _descs2) 
        # Sort matches by score
        cmatches.sort(key=lambda x: x.distance, reverse=False)
        # Remove not so good matches
        numGoodMatches = int(len(cmatches) * GOOD_MATCH_PERCENT)
        cmatches = cmatches[:numGoodMatches]  
        # Draw top matches
        imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches, None)
        cv2.imwrite("gpu_matches.jpg", imMatches)
        return()

The code above is running w/o any errors, but it seems the CPU-keypoints are all empty (the created images does not show any matching lines). The equivalent Non-CUDA code is running fine.

Any idea what I am doing wrong? I tried several conversion methods, this is the 'best' I can find. Thanks for your help!

(Using OpenCV4.1.2 on Jetson Nano with Python3.6.9)

edit retag flag offensive close merge delete

Comments

Change this:

# upload into Cuda
cuMat1 = cv2.cuda_GpuMat()
cuMat2 = cv2.cuda_GpuMat()
 cuMat1.upload(npMat1)
 cuMat2.upload(npMat2)        
 #convert to Gray
 cuMat1g = cv2.cuda.cvtColor(cuMat1, cv2.COLOR_RGB2GRAY)
 cuMat2g = cv2.cuda.cvtColor(cuMat2, cv2.COLOR_RGB2GRAY)

to:

cuMat1 = cv2.cuda_GpuMat()
cuMat2 = cv2.cuda_GpuMat()

cuMat1g = cv2.cuda.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
cuMat2g = cv2.cuda.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
cuMat1.upload(cuMat1g)
cuMat2.upload(cuMat2g)

supra56 ( 2020-01-19 10:50:25 -0600 )edit

Have you tried inspecting the result of

_kps1.download()

cudawarped ( 2020-01-19 12:17:18 -0600 )edit

@supra56: Thanks for your help! I changed the lines to


    cuMat1 = cv2.cuda_GpuMat()
    cuMat2 = cv2.cuda_GpuMat()
    cuMat1g = cv2.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
    cuMat2g = cv2.cvtColor(npMat2, cv2.COLOR_RGB2GRAY)
    cuMat1.upload(cuMat1g)
    cuMat2.upload(cuMat2g)

The overall result stays the same.

MicTie ( 2020-01-20 01:44:11 -0600 )edit

@cudawarped: Thanks for your help!

_kps1.download() shows a <class 'numpy.ndarray'=""> with


[[2.54100000e+03 4.51000000e+02 2.27500000e+03 2.21700000e+03
  2.31100000e+03 2.02300000e+03 1.73000000e+03 2.32700000e+03
  4.10000000e+02 2.27400000e+03 3.12000000e+02 2.60800000e+03
  2.27000000e+03 2.28900000e+03 2.29300000e+03 2.23300000e+03
  1.05000000e+03 2.23800000e+03 1.97700000e+03 2.27300000e+03
  2.25500000e+03 2.01500000e+03 1.07040002e+03 2.53920020e+03
  2.30400000e+03 9.21600037e+02 2.54040015e+03 2.63400000e+03
  ......

If I use _kps1 and _kps2 in "imMatches = cv2.drawMatches(npMat1, _kps1, npMat2, _kps2, cmatches, None), I receive an Error Message SystemError: <built-in function="" drawmatches=""> returned NULL without setting an error

MicTie ( 2020-01-20 02:01:44 -0600 )edit

Do you mean

imMatches = cv2.drawMatches(npMat1, _kps1.download(), npMat2, _kps2.download(), cmatches, None)

is cmatches a GpuMat() or numpy array?

The functions in the CUDA namespace (cv2.cuda cv2.cuda_) require the input/output arrays to be GpuMat(), if you want to use them with functions which are not in the CUDA namepsace (cv.) then you will need to download() them to the host first. This allows you to control when the costly memory transfers to/from the device/host take place.

cudawarped ( 2020-01-20 03:37:29 -0600 )edit

cmatches is <class 'list'=""> of <class 'cv2.dmatch'="">.

You, are right I did some nonsense in my earlier reply.

If I use imMatches = cv2.drawMatches(npMat1, _kps1.download(), npMat2, _kps2.download(), cmatches, None), I receive a TypeError: Expected cv::KeyPoint for argument '%s'. My initial attempt (1st post) used the type cv:KeyPoint

MicTie ( 2020-01-20 03:52:14 -0600 )edit

Does this help imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[10], None)? Use any number from 1 to 10 using your owned code. If not try both mine and@ cudawraped together or one at a time.

supra56 ( 2020-01-20 06:34:59 -0600 )edit

@supra56: using

cuMat1 = cv2.cuda_GpuMat() 
cuMat2 = cv2.cuda_GpuMat() 
cuMat1g = cv2.cvtColor(npMat1, cv2.COLOR_RGB2GRAY) 
cuMat2g = cv2.cvtColor(npMat2, cv2.COLOR_RGB2GRAY) 
cuMat1.upload(cuMat1g) cuMat2.upload(cuMat2g) 
...... 
imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[10], None)

returns SystemError: <built-in function="" drawmatches=""> returned NULL without setting an error. Same for other numbers .

`imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[:10], None)` produces no error, but still no matches

I am trying various code-permutations, but no luck ....

MicTie ( 2020-01-20 07:11:07 -0600 )edit

Have you tried passing KeyPoints() instead of numpy arrays?

npKps1 = _kps1.download()
kps1 = [cv.KeyPoint(*npKps1[:,i]) for i in range(npKps1.shape[1])]
npKps2 = _kps2.download()
kps2 = [cv.KeyPoint(*npKps2[:,i]) for i in range(npKps2.shape[1])]

cudawarped ( 2020-01-20 09:02:14 -0600 )edit

Thanks, this seems to be working! I have to wait for a couple of hours before I can show the answer

MicTie ( 2020-01-20 23:54:25 -0600 )edit

add a comment

answered 2020-01-20 12:09:10 -0600

cudawarped
869 ●3 ●10 http://jamesbowley.co.uk

Hi, I have had chance to look at this now. I think the real problem is that the python wrapper for corb.convert() does not allow the host KeyPoints to be an output argument. I would suggest adding CV_OUT as shown below

CV_WRAP virtual void convert(InputArray gpu_keypoints,
                     CV_OUT std::vector<KeyPoint>& keypoints) = 0;

to line 417 of cudafeatures2d.hpp. Hopefully then drawing matches should be as simple as

imMatches = cv.drawMatches(npMat1, corb.convert(_kps1), npMat2, corb.convert(_kps2), cmatches, None)

edit flag offensive delete link

Comments

Thanks for your help! Too bad the solution is not working: The line imMatches = cv2.drawMatches(npMat1, corb.convert(_kps1), npMat2, corb.convert(_kps2), cmatches, None) draws an error: TypeError: Required argument 'keypoints' (pos 2) not found

MicTie ( 2020-01-20 19:20:57 -0600 )edit

It's working on github, are you sure you re-compiled, and copied the new python bindings to your site-packages folder?

cudawarped ( 2020-01-21 02:42:39 -0600 )edit

Sorry, did not know I have to recompile ... Do I have to recompile the entire OpenCV, or is there a faster way? (using Linux 18.04)

MicTie ( 2020-01-21 03:14:12 -0600 )edit

If you start the build again (don't re-build) then only the required elements should be re-built. On windows with ninja this takes less than 30 seconds.

cudawarped ( 2020-01-21 03:32:21 -0600 )edit

I re-compiled the .hpp and everything is working! Thank you so much for your help!

MicTie ( 2020-01-21 07:49:58 -0600 )edit

add a comment

Cuda ORB keypoints

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Cuda ORB keypoints edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Cuda ORB keypoints