OpenCV algorithms for computing stereo correspondence (StereoSGBM, StereoBM) require rectified images. If you use them on non-rectified images they won't work correctly.
Rectification is a transformation of images such that projections of the same scene point have equal y-coordinate on both images. E.g. if a projection of some 3D point onto a left image has y-coordinate = 150 it'll have y-coordinate = 150 on the second image.
So rectification simplifies finding corresponding points (and subsequently simplifies computation of a depth map).
If you want to build your stereo vision application use the following approach:
- Calibrate cameras. The simplest approach is to use calibration procedure (stereo_calib.cpp/stereo_calib.exe) distributed in OpenCV examples. This will give intrinsic and extrinsic camera parameters.
From rectification procedure you should get reprojection error on the level of 0.5 pixel. Much bigger reprojection error (above 1 pixel) usually means that there're some issues with a calibration.
- Use stereo camera parameters estimated by stereo calibration procedure to rectify ALL images. You may verify correctness of this step by looking on rectified images: projections of the same scene points on both images should have the same y-coordinate.
- Use stereo correspondence function on RECTIFIED images. I can suggest using StereoSGBM - it usually produces better results than simple StereoBM procedure.
Function cvInitUndistortRectifyMap needs to be used only once - after you get stereo camera parameters from stereo calibration procedure. This funciton computes a mapping needed to rectify images.
Then you rectify ALL images using mapping returned by cvInitUndistortRectifyMap.
