problem with stereoRectify() results [magnified output]

edit

I follow the example of Kaehler and Bradsky book (Learning Opencv3 chapter 19), to get more familiar with the procedure and the techniques because I have a lack of experience. First, I use as calibration targets a checkerboard of 7x10 squares size and also an asymmetric dots patern of 4x11 size. These targets are really small (the side of each square of the checkerboard is 0.45 mm). Each asymmetric dot has a size of about 0.25mm. I achieved the best results with asymmetric dots pattern, with the best result used for the stereorectification process below.

I use cv::calibrateCamera at each camera and after I use stereoCalibrate for the next step. I collect at least 100 successful measurements for a valid result. Then the stereo rectification follows. I use Bouguet method for rectification and pass my calibration results to cv::stereoRectify. After I apply cv::initUndistortRectifyMap to each camera frame and finally remap. At the end I store the results as the ones that uploaded here.

I acquired a better error score for my calibrations after shooting new calibration videos with better light control and being careful to present the targets on the whole field of view. With 135 detected stereo pairs, the reprojection error for stereoCalibrate was 3.3062141238092657e+00 . The error for the left camera is 6.0338782277157943e-01 and for the right camera similarly 6.0971222421947813e-01. The flag I used for stereo calibration was only CALIB_FIX_INTRINSIC, but I didn't see any difference with CALIB_FIX_ASPECT_RATIO | CALIB_SAME_FOCAL_LENGTH. The error score of ~ 0.6 per camera and 3.3 for stereo is better than my previous results. So after running the programm I see that I have a way better than the previous result, but I'm not quite sure that I can assess it correctly as adequate because my final aim is the 3d reconstruction. I run stereoRectify() with alpha set to -1 otherwise it produces quite funny results which doesn't resemble at all my initial frames (strips or discs of image parts). I also put flags=0 for stereoRectify.

Now the new calibration results are: Pattern: Asymmetric dots.

KL:

5.7432741355633716e+03 0. 3.5864063009366998e+02 0.
5.3294883722077411e+03 2.8626995097397503e+02 0. 0. 1.

KR:

5.5855914241748869e+03 0. 3.5893764153026808e+02 0.
5.2368003077143685e+03 2.8626284627542736e+02 0. 0. 1.

DL:

    2.7743096670749008e+00 -2.1332159140099390e+02
-2.3919561436865893e-02 -5.6926044040429592e-02
-1.2539578258517790e+00

DR:

2.4194267392305364e+00 -1.5408901350984709e+02
-1.5277247740632098e-02 -5.7579530702796311e-02
-9.3922783898228168e-01

R:

    9.9937260710853593e-01 -2.1629178382018797e-02
2.8045869635414729e-02 2.1973416854983850e-02 9.9968624039129861e-01
-1.2024546728580759e-02 -2.7776988908176835e-02
1.2633266197999683e-02 9.9953431130320158e-01

T:

-1.7948023556461674e+00 7.6656265194349504e-01
-1.3004269187890980e+00

E:

7.2820204955379997e-03 1.3097030873873605e+00 7.5056862812901337e-01
-1.3494652453095344e+00 5.0801381730943410e-02
1.7574949326414129e+00 -8.0551965631772415e-01
-1.7776590988214844e+00 8.2768590249103491e-05

F:

1.3686591816576201e-07 2.6527131904819632e-05 7.3377205091950545e-02
-2.7052554056256310e-05 1.0974787539235105e-06
2.1173656674781893e-01 -7.6869609851764387e-02
-2.1094641314552989e-01 1.

I suppose that microscope calibration and rectification poses bigger challenge because like FOV, depth of ... (more)