OpenCV: Essential Matrix SVD decomp

edit

Hi Folks,

I am trying to get camera motion vector based on OpenCV Optical Flow. I use C# wrapper for Unity of OpenCV 2.4.10, but it is just wrapper Here is the test case:

1. Calibrated my camera and have camera matrix K (3x3)
2. Use 2 100%-identical images framePrev and frameThis as optical flow frames (means no motion)
3. Selected features (2d points) from both images via

goodFeaturesToTrack (frameThis, pointsThis, iGFFTMax, 0.05, 20); 
goodFeaturesToTrack (framePrev, pointsPrev, iGFFTMax, 0.05, 20);

so i have features pointsPrev and pointsThis

4. Use

calcOpticalFlowPyrLK (framePrev, frameThis, pointsPrev, pointsThis, status, err);

to verify flow for points, then I make sure analyzing status and err arrays, so my pointsPrev and pointsThis are identical pairs of points in image pixel coordinates

5. Select first 8 pairs from pointsPrev and poitsThis (simply trunc arrays), then get Fundamental Matrix:

F = Calib3d.findFundamentalMat(pointsPrev, pointsThis, Calib3d.FM_8POINT, 2, 0.99); 

. When points in all the pairs are identical (no motion) - it gives me 3x3 matrix with all zeros, I suggest that is correct (or?)

6. Then getting Essential Matrix based on E = K'.(t) * F * K according to HZ 9.12, I have one camera, so K' = K.

gemm (K.t (),F,1,null,0,tmpMat,Core.GEMM_3_T);
gemm (tmpMat,K,1,null,0,E,Core.GEMM_3_T);

when F = |0|, then E = |0| as well

7. Finally I apply SVD decomposition on E:

SVDecomp(E,W,U,VT);

8. Analyzing W, U, VT output matrices, I can observe these values:

W: 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000
U: -0.577, -0.408, -0.707, -0.577, -0.408, 0.707, 0.577, -0.816, 0.000
Vt: 1.000, 0.000, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 1.000

I suggest these values are strange, as according to books/manuals camera translation vector is U.col(2) is Vector3(-0.7071, 0.7071, 0) that is not correct.

Other observations, when I test for different image frames:

• U values are always between -1 and 1, that should not be translation, similar more on sine/cosine values (again, 0.7071 is sine of pie/4 or cosine of pie/4)
• Fundamental matrix outputs are radically different for different algoriths - 8POINTS, 7POINTS, RANSAC, LMEDS, even for pairs of corresponding poits (features)
• using dirrefent number of pairs of points (features) - say 5, 7, 8, 15, 40 - for the same algoriths also radically changes fundamental matrix output

I do really need your help, thank you in advance!

That is the copy of my question on StackOverflow : http://stackoverflow.com/questions/30...

Kind Regards, Eugene

EDIT 1: Additional observations Then I tried to find the Fund matrix for these frame poits:

MatOfPoint2f p1 = new MatOfPoint2f(new Point(100,100),new Point(100,200),new Point(100,300),
                           new Point (200,100),new Point(200,200),new Point(200,300),
                           new Point(300,100),new Point ...

(more)