Ask Your Question

Revision history [back]

Erroneous projection matrix due to stereoRectify() ?

I have the following situation: - 2x cameras with 2048x1024 px resolution of the same type - Some hardware black box calculating a depth image from the input with a resolution of 960x440

I have a valid calibration for the camera system with given cameraMatrix1, cameraMatrix2, distCoeffs1 and distCoeffs2. Moreover I have a rectification matrix rect, a rotation matrix R and a translation vector T. The box works very vell, the disparities are very good, hence I don't doubt on this data. (It is originally calculated with Matlab and obtained from a stereoParams object.)

To reproject disparity image points into 3D world points I wanted to use the ROS stereo_image_proc/point_cloud2 node, which needs the projection matrices of both cameras. Hence I crated a python script to calculate it:

import cv2
import numpy as np

if __name__ == '__main__':
    print('Rectify with opencv2')

    cameraMatrix1 = np.matrix([[2565.031253, 0, 919.413174],
                               [0, 2544.873175, 697.398311],
                               [0,0,1]])
    cameraMatrix2 = np.matrix([[2531.349645, 0, 869.425085],
                               [0, 2515.573530, 632.055025],
                               [0,0,1]])
    distCoeffs1 = np.matrix([0.009251, -0.015378, -0.004205, -0.020355, -0.624842]).transpose()
    distCoeffs2 = np.matrix([0.024439, -0.317383, -0.001931, -0.022877, 0.316663]).transpose()
    imageSize = (2048, 1024)
    R = np.matrix([[0.999618864,-0.007622217,-0.026533520],
                   [0.006909877,0.999615975,-0.026835735],
                   [0.026727879,0.026642164,0.999287654]])
    T = np.matrix([497.838267224, -33.349483489, 23.913797009]).transpose()
    camera_name = "zynq_101"

    alpha = -1.0;
    R1 = np.zeros([3,3])
    R2 = np.zeros([3,3])
    P1 = np.zeros([3,4])
    P2 = np.zeros([3,4])
    Q = np.zeros  ([4,4])
    newImageSize = (2048, 1024)

    R1, R2, P1, P2, Q, validPixROI1, validPixROI2 = cv2.stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imageSize, R, T, R1, R2, P1, P2, Q, 0, alpha, newImageSize)

This results in the following projection matrices

left: [2515.57353,  0.0,        807.6421051025391, 0.0, 
       0.0,         2515.57353, 648.6131038665771, 0.0, 
       0.0,         0.0,        1.0,               0.0]
right: [2515.57353, 0.0,        681.3331298828125, 1256596.330467894,
         0.0,       2515.57353, 648.6131038665771, 0.0,
         0.0,       0.0,        1.0,               0.0]

So where is the poblem: I cannot explain the right[1,4] c_x value of 1256596.330467894. In my eyes this value makes no sense. Does anyone see my mistake or can explain why this value might be correct anyway?

Using this projection matrices with the referenced point_cliud2 node of ROS shows that there must be something wrong with my values, the resulting PCL is obviously wrong, the X,Y,Z values are way to large.