Revision history [back]

I'm, trying to implement algorithm find in https://pdfs.semanticscholar.org/8fc2/3d2e7c142c0afc8ff255615e3dbc203f7fbb.pdf using openCv, and I can' get around obtaining frontal images from my camera, after calibration and rectification some position are "cropped" and miss image plane. Input image: https://imageshack.com/a/img922/2971/OCIoQu.png

Example of one of the wrong position: https://imageshack.com/a/img923/3940/XKYwpI.png

cv::Mat distCoeffs;
std::vector<cv::Mat> rvecs, tvecs;
distCoeffs = cv::Mat::zeros(8, 1, CV_64F);

double EPS = 0.00001;
cv::TermCriteria Terms(cv::TermCriteria::COUNT + cv::TermCriteria::EPS, 30, EPS);

int CalibrationControlParameters = 0;

cv::Mat cameraMatrix = cv::Mat::eye(3, 3, CV_64F);

double REPROJECTION = cv::calibrateCamera(Model, KeyPoints, cv::Size(PureImages[0].cols, PureImages[0].rows), 
                                        cameraMatrix, distCoeffs, rvecs, tvecs,
                                        CalibrationControlParameters, Terms);

double alfa = 0;

cv::Mat NewCamera = cv::getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, PureImages[0].size(), alfa,cv::Size(),(cv::Rect*)0,false);

for (int image_num = 0; image_num < PureImages.size(); ++image_num) {

    cv::Mat RotationMatrixCV(3, 3, CV_64FC1);
    cv::Rodrigues(rvecs[image_num], RotationMatrixCV);

    Eigen::Vector3d Translations;
    cv::cv2eigen(tvecs[image_num], Translations);

    cv::Mat CurrentCamera = NewCamera.clone();
    Eigen::Matrix3d CamEig;
    cv::cv2eigen(CurrentCamera, CamEig);

    CamEig(0, 2) += Translations(0);
    CamEig(1, 2) += Translations(1);
    cv::eigen2cv(CamEig, CurrentCamera);

    cv::initUndistortRectifyMap(cameraMatrix, distCoeffs, RotationMatrixCV, CurrentCamera, PureImages[0].size(), CV_32FC1, map1, map2);


    cv::remap(PureImages[image_num], Transformed, map1, map2, 1);

    cv::imwrite(_FolderPath + "perspective" + _PathToImages[image_num], Transformed);
}

Translation can by apply by adding to center of image of CurrentCamera, but I' can't came up with formula with will center all image simultenuisly. Reprojection error is around 0.7 pix ( some images are severly blured), but calibration isn't singular.

Using OpenCv 3.3 and Eigen