OpenCV 4.1.2 CUDA 10 Python on Jeton TX2 Jetpack 4.2

asked 2020-01-06 09:15:56 -0600

pechista
1 ●1

Hello,

I´m using the cv2.calcOpticalFlowFarneback() function on my Jetson TX2 with Python, but unfortunately I´m not getting any improvement in performance after building OpenCV 4.1.2 from source with CUDA support. As long as I know, since OpenCV4, all the functions are optimized and integrated by default, but apparently CUDA optimized functions are not implemented with python Wrappers.

So my goal would be to access the class cv::cuda::FarnebackOpticalFlow to calculate the Optical Flow from a pair of images.

I tried to used the python wrapper for farnerback on gpu developed here https://github.com/NeerajGulia/python..., and I ran the tests, it can do it at a rate of 9 FPS with a test video, compared to the 4 FPS using the normal opencv4 cuda build optimized.

So because I´m new with OpenCV I´d like to know:

1) Is it possible to access OpenCV cv::cuda::FarnebackOpticalFlow class from Python on my Jetson TX2 from Python? I´m currently trying to call calc function from Python like this: opticalFlowGPU = cv.cuda_FarnebackOpticalFlow.create(5, 0.5, False, 15, 3, 5, 1.2, 0) flowCUDA = opticalFlowGPU.calc(frame1, frame2, None)

but I get the following error when I call calc function: Exception has occurred: error OpenCV(4.1.2) /home/nvidia/opencv-4.1.2/modules/core/src/matrix_wrap.cpp:359: error: (-213:The function/feature is not implemented) getGpuMat is available only for cuda::GpuMat and cuda::HostMem in function 'getGpuMat

2) I know Nvidia has developed the new OpticalFlow SDK https://developer.nvidia.com/opticalf.... But I think the Jetson TX2 is not compatible because of the GPU architecture because. Is it posible to build the OpticalFlow SDK for the TX2 so I don´t get an error?

Thanks

answered 2020-01-06 10:44:02 -0600

cudawarped
869 ●3 ●10 http://jamesbowley.co.uk

updated 2020-01-06 11:46:49 -0600

Hi,

1) In 4.1.2 the CUDA python bindings should be working, additionally on the basis that

opticalFlowGPU = cv.cuda_FarnebackOpticalFlow.create(5, 0.5, False, 15, 3, 5, 1.2, 0)

didn't throw an error I think you have them set up correctly.

You may be having a problem because if you are passing numpy arrays instead of GpuMat to the cv2.cuda_FarnebackOpticalFlow.calc() function.

From the help

help(cv2.cuda_FarnebackOpticalFlow.calc)

Help on method_descriptor:

calc(...)
    calc(I0, I1, flow[, stream]) -> flow
    .   @brief Calculates a dense optical flow.
    .   
    .       @Param I0 first input image.
    .       @Param I1 second input image of the same size and the same type as I0.
    .       @Param flow computed flow image that has the same size as I0 and type CV_32FC2.
    .       @Param stream Stream for the asynchronous version.

I0, I1 and flow are required so as long as frame1 and frame2 are GpuMat

flowCUDA = opticalFlowGPU.calc(frame1, frame2, None)

should work without any errors. I would therefore try the below just to check

I0 = cv2.cuda_GpuMat(frame1)
I1 = cv2.cuda_GpuMat(frame2)
flowCUDA = opticalFlowGPU.calc(I0,I1,None)
flowCUDA.download()

ensuring that frame1 and frame2 are single channel arrays.

2) Nvidia optical flow is included in 4.1.2, you should be able to create the class in python by calling

nvidia_of = cv2.cuda.NvidiaOpticalFlow_1_0_create(720,1280)

with the height and width of your frame.

edit flag offensive delete link

Comments

Thanks it is working now after uploading the frames to the GPU as GpuMat.

Nvidia optical flow is testing the following python code OFlow SDK from the docs:

nvof = cv.cuda_NvidiaOpticalFlow_1_0.create(frame1.shape[1], frame1.shape[0], 5, False, False, False, 1)

and the one in your reply:

nvidia_of = cv2.cuda.NvidiaOpticalFlow_1_0_create(720,1280)

But I get the same error:

OpenCV(4.1.2) /home/nvidia/opencv_contrib-4.1.2/modules/cudaoptflow/src/nvidiaOpticalFlow.cpp:172: error: (-6:Unknown error code -6) Cannot find NvOF library. in function 'LoadNvidiaModules'

Is it probably because of the Jetson TX2 GPU architecture or a bug in the OpenCV 4.1.2? they changed the nvidiaOpticalFlow.cpp code in 4.2.0.

pechista ( 2020-01-06 19:49:23 -0600 )edit

add a comment

OpenCV 4.1.2 CUDA 10 Python on Jeton TX2 Jetpack 4.2

1 answer

Comments

Links

Question Tools

Stats

Related questions

OpenCV 4.1.2 CUDA 10 Python on Jeton TX2 Jetpack 4.2 edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

OpenCV 4.1.2 CUDA 10 Python on Jeton TX2 Jetpack 4.2