SolvePnP detection errors [ios]

edit

Hi,

I've got problem with precise detection of markers using OpenCV.

I've recorded video presenting that issue: http://youtu.be/IeSSW4MdyfU

As you see I'm markers that I'm detecting are slightly moved at some camera angles. I've read on the web that this may be camera calibration problems, so I'll tell you guys how I'm calibrating camera, and maybe you'd be able to tell me what am I doing wrong?

At the beginnig I'm collecting data from various images, and storing calibration corners in _imagePoints vector like this

std::vector<cv::Point2f> corners;
_imageSize = cvSize(image->size().width, image->size().height);

bool found = cv::findChessboardCorners(*image, _patternSize, corners);

if (found) {
    cv::Mat *gray_image = new cv::Mat(image->size().height, image->size().width, CV_8UC1);
    cv::cvtColor(*image, *gray_image, CV_RGB2GRAY);

    cv::cornerSubPix(*gray_image, corners, cvSize(11, 11), cvSize(-1, -1), cvTermCriteria(CV_TERMCRIT_EPS+ CV_TERMCRIT_ITER, 30, 0.1));

    cv::drawChessboardCorners(*image, _patternSize, corners, found);
}

_imagePoints->push_back(_corners);

Than, after collecting enough data I'm calculating camera matrix and coefficients with this code:

std::vector< std::vector<cv::Point3f> > *objectPoints = new std::vector< std::vector< cv::Point3f> >();

for (unsigned long i = 0; i < _imagePoints->size(); i++) {
    std::vector<cv::Point2f> currentImagePoints = _imagePoints->at(i);
    std::vector<cv::Point3f> currentObjectPoints;

    for (int j = 0; j < currentImagePoints.size(); j++) {
        cv::Point3f newPoint = cv::Point3f(j % _patternSize.width, j / _patternSize.width, 0);

        currentObjectPoints.push_back(newPoint);
    }

    objectPoints->push_back(currentObjectPoints);
}

std::vector<cv::Mat> rvecs, tvecs;

static CGSize size = CGSizeMake(_imageSize.width, _imageSize.height);
cv::Mat cameraMatrix = [_userDefaultsManager cameraMatrixwithCurrentResolution:size]; // previously detected matrix
cv::Mat coeffs = _userDefaultsManager.distCoeffs; // previously detected coeffs
cv::calibrateCamera(*objectPoints, *_imagePoints, _imageSize, cameraMatrix, coeffs, rvecs, tvecs);

Results are like you've seen in the video.

What am I doing wrong? is that an issue in the code? How much images should I use to perform calibration (right now I'm trying to obtain 20-30 images before end of calibration).

Should I use images that containg wrongly detected chessboard corners, like this:

or should I use only properly detected chessboards like these:

I've been experimenting with circles grid instead of of chessboards, but results were much worse that now.

In case of questions how I'm detecting marker: I'm using solvepnp function:

solvePnP(modelPoints, imagePoints, [_arEngine currentCameraMatrix], _userDefaultsManager.distCoeffs, rvec, tvec);

with modelPoints specified like this:

    markerPoints3D.push_back(cv::Point3d(-kMarkerRealSize / 2.0f, -kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(kMarkerRealSize / 2.0f, -kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(kMarkerRealSize / 2.0f, kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(-kMarkerRealSize / 2.0f, kMarkerRealSize / 2.0f, 0));

and imagePoints are coordinates of marker corners in processing image (I'm using custom algorithm to do that)