Stereo SGBM Algorithm

asked 2019-05-20 14:25:01 -0600

kobeking
1 ●2 ●3

Hello everyone,

I have been working on a project in which I need to compute a depth map. In order to do so, I have used both Stereo BM and Stereo SGBM algorithms after computing a stereo calibration. Both gave me not good results at all, even when playing with the different parameters.

After having done some post filtering work with the wls_filter.filter() function, the result with the SGBM algorithm gave me something that was okay, but however, the gray levels in the disparity map dosent stop to change over time, even if nothing moves in the image... which is quite anoying!

Below you will find the result with the SGBM algorithm and the BM algorithm

SGBM:

image description

BM :

image description

Has anyone been able to get a good real time disparity map with any algorithm? If yes, any indication would be helpful !

Thank you.

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2019-05-20 17:22:21 -0600

opalmirror

641 ●6 ●15 http://www.loowit.net/...

updated 2019-05-20 17:24:45 -0600

Calibration: Accurately calibrate your cameras (intrinsic and stereo extrinsics).

Minimize noise: Stereo disparity algorithms are affected by noise in the images, so make sure that your illumination gives good contrast without over or underexposing details. Make sure your sensor's sensitivy (ISO) is low to reduce speckle. Try to avoid lossy compression on images from your sensors (JPG, and other compression techniques) - these produce noise that humans do not notice, but algorithms do.

Adjust parameters: There are many filtering parameters by which you can tweak the SGBM or BM stereo disparity algorithms to provide calmer output. See e.g. Answer #182049

Temporal or Spatial Filtering: you can try smoothing individual frames with e.g. median filter with a small radius, or for a static scene, over time, accumulating multiple depth images and averaging them.

Pattern projection: If there is not enough detail of high contrast in a large area, then using a projector to illuminate a random dot pattern on both objects can assist filling in holes. This also needs gain adjusted so as to provide adequate contrast.

Generally, some set of these techniques can help.

edit flag offensive delete link

add a comment

answered 2019-05-22 15:50:44 -0600

kobeking
1 ●2 ●3

Hello, thank you for your answer.

To verify my stereo calibration results, I put left and right frames side by side and draw horizontal lines, to see if they would pass exactly at the same points. Is that enough? Also, even after having taken 40 calibration pictures, I still get some little errors (see below image), and the horizontal lines don't pass exactly at the same point. What can I do to get a better result ? Does little errors like this influence the result in a significant manner for the disparity map ?

image description

Also, as you can see on the picture, the quality of the image isn't very good. You mentionned that Stereo disparity algorithms are affected by noise, does it mean that if I have better image quality, I will get much better disparity maps ?

Thank you for the explanations on the parameters of Stereo BM & SGB. Could you maybe just explain a bit more how the textureThreshold and uniquenessRation works ? How is texture caracterized in textureThreshold and what does uniquenessRation do exactly ?

Finally, have you ever worked with the WLS filter ? I haven't been able to find explanations on how it really works anywhere... Any help would be helpful.

Thank you.

edit flag offensive delete link

Comments

Are these the left and right images which have been undistorted and rectified? Because that is what you need as input to the match algorithm. After undistort+rectify, I would expect better registration of features to the same line on left and right image (undistort + rectify takes into account camera position/pose relationship, focal length differences, and distortion parameters all determined in mono and stereo calibration).

I haven't studied textureThreshold, uniquenessRatio, or WLS in detail, but there is a paper associated with the SGBM algorithm which you might want to track down which might do so. I've only mapped out the stereo BM match step in detail to understand more about how it is optimized, but that was a couple years ago.

opalmirror ( 2019-05-22 18:24:01 -0600 )edit

add a comment

Stereo SGBM Algorithm

2 answers

Comments

Links

Question Tools

Stats

Related questions

Stereo SGBM Algorithm edit

2 answers

Comments

Links

Question Tools

Stats

Related questions

Stereo SGBM Algorithm