I have an issue using cv::projectPoints in opencv 4.5 projecting 3D Lidar Points into a 2D image.
- There is no roll/pitch/yaw so the rvec is 0.
- Points are already in world space and only have to be transformed to camera space with tvec.
- There is no camera lens distortion
I tested and ran the code below for an image resolution of 785x785 which works fine. Projected Points are on the correct position in the image.
After I've changed the resolution to 1600x1200 the code below does not work correctly anymore. Projected 2D Points are approx 30px off (+ ~30px in direction on top).
I don't really understand whats the issue. Has anyone an idea? I furthermore checked by setting the resolution to 1200x1200 which works correctly again. So the issue comes from not having the same width and height.
My guess is that there might be an issue with cmat.
cv::Mat rvec, tvec, cmat;
rvec.create(1, 3, cv::DataType<float>::type);
rvec.at<float>(0) = 0;
rvec.at<float>(1) = 0;
rvec.at<float>(2) = 0;
tvec.create(3, 1, cv::DataType<float>::type);
tvec.at<float>(0) = camera.opencv_origin.x;
tvec.at<float>(1) = -camera.opencv_origin.y; // In coordinate system y and z axis are inverted
tvec.at<float>(2) = -camera.opencv_origin.z; // In coordinate system y and z axis are inverted
cmat.create(3, 3, cv::DataType<float>::type);
cmat.at<float>(0, 0) = camera.image_width / 2;
cmat.at<float>(1, 1) = camera.image_height / 2;
cmat.at<float>(0, 2) = camera.image_width / 2;
cmat.at<float>(1, 2) = camera.image_height / 2;
cmat.at<float>(2, 2) = 1;
std::vector<cv::Point2f> points_image;
cv::projectPoints(points_world, rvec, tvec, cmat, cv::noArray(), points_image);
for (const auto& p : points_image) {
cv::circle(image, p, 2, cv::Scalar(0, 0, 255), -1);
}