I have a problem when using solvePnPRansac to track coplanar markers. I recorded a video of the problem here:
<iframe width="853" height="480" src="https://www.youtube.com/embed/ftijvDPDA0U?showinfo=0" frameborder="0" allowfullscreen=""></iframe>http://youtu.be/ftijvDPDA0U
In the video you see the pink cube drawn with OpenCV flipping about every 2-3 frames. You also see on top a stable coordinate system I render with OpenGL but you can ignore it, I don't apply the rotation values calculated by the tracker, that's why the flipping is not passed to the OpenGL scene.
I assume the problem lies in the solvePnPRansac of OpenCV since the inputs (the 2D and 3D points) are very similar on each frame but the calculated projection matrix differs a log as you can see in the video. The tracked points all look fine. In the video I also render the coordinate axes and you can see that the z-axis is flipped/inverted while the rest of the axes (Y-axis: green pointing down and X-axis: red point to the right) stay correct.
My used parameters for solvePnPRANSAC: I used cv::ITERATIVE on the solvePnPRANSAC with the standard reprojection error 8.0 (tried several times with different values but no luck) and I switched of the extrinsic guess to false.
Are there always 2 correct pnp solutions for coplanar markers and that's why it randomly flips between both? Can I avoid this behaviour. I searched for people having similar problems but all I could find about this are two old questions which sound like it could be the same problem: http://answers.opencv.org/question/6373/cv2solvepnp-axis-flip-with-rotation/# and http://answers.opencv.org/question/11915/solvepnp-similar-input-returns-very-different-output/