Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.
Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.
Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.
As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?)
Thanks,
Alex
--- Repro case:
Hi Alexander,
What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you.
My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here http://www.uco.es/investiga/grupos/ava/node/26), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.
Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this
vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
and then update the call to solvePnP as
cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around.
Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.)
If you'd like me to grab a bunch of data from bad frames and give you that then I can do, but that's the test harness that I just checked with just now. I don't see any changes to the relevant files recently in SVN, so assume 2.4.2 will have the same issues as 2.4.1.
--
PS. Thanks for creating this answers interface - I hope it works as well as other sites as the yahoo group was unusable. I believe I'd already posted this question there but it never actually showed up!
