numberOfDisparities clarification (StereoBM)

edit

The left camera may see close objects to the left of the right camera view, and the right camera may see close objects to right of the left camera view.

Imagine the space of valid object surfaces at valid disparties as a truncated pyramid. The base of the pyramid is for the distance corresponding to and inversely proportional to minDisparity. The top and bottom of the rectified image views constrain two opposing sides of the pyramid, and the objects visible to both cameras at the left and right of the rectified image views constrain the other two sides of the pyramid. The truncated end of the pyramid is for the distance inversely proportional to minDisparty + numberOfDisparities.

Recall the resulting disparity image is mapped into the view of the left camera rectified image. From the left edge to the right edge of the disparity image, there are regions with different properties. In detail, at any point (xd,y) in the disparity image:

1. When xd < minDisparity, objects are visible to the left camera but out of view of the right camera, or too distant to be valid. OpenCV does not calculate disparity (it would be geometrically useless).

2. When xd = minDisparity, objects exactly at the maximum range (corresponding to minDisparity) are visible to both cameras and have a valid disparity, but closer objects are only visible to the left camera (and farther objects are too far to be valid). OpenCV does not calculate disparity (it might be useful though, for some applications).

3. when xd > minDisparity and xd < minDisparity + numberOfDisparities, you'll get a blend of of 2) and 4). OpenCV does not attempt to calculate disparity (it might be useful though, for some applications).

4. When xd = minDisparity + numberOfDisparities, objects at all valid ranges are visible to both cameras (although some objects may be too near or too far to be valid). OpenCV calculates disparity and this is the left edge of the valid disparity image pixels.

5. when xd > minDisparity + numberOfDisparities and xd < width - numberOfDisparities, objects at all valid ranges are visible to both cameras (although some objects may be too near or too far to be valid). OpenCV calculates disparity and this is the bulk of the disparity image.

6. when xd >= width - numberOfDisparities, objects at all valid ranges are visible to both cameras (although some objects may be too near or too far to be valid). OpenCV calculates disparity, however the getDisparityValidROI interprets this as out-of-view and so any calculation results are clipped and not returned for this region of the disparity image.

I think that 6. is probably in error and should act just like 5. for most applications. I'm considering filing a bug on that.

I think algorithm in stereoBM is

looks for pixel (xr,y) best matches pixel(xl,y) in left image with xl ranging to xr+minDisparity and xr+minDisparity +numOfdisparities.

Hence if [xr+minDisparity ,xr+minDisparity +numOfdisparities] is outside of images don't try to match pixel because you cannot find the best with given constraint.

In stereoBM I think limits are defined and used here