Get [R|t] of two kinnect cameras

edit

My aim is to determine the correct tranformation between the coordinate systems of two kinnect cameras, based on chessboard patterns. (the base unit would be meter)

I am basicly using stereo_calib.cpp sample. (with the chessboard unit set correctly)

Using 5 pairs, the reprojection error is 3.32.

R: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ -1.2316533437904904e-002, 5.2568656248225909e-001,
       -8.5058917288527658e-001, -5.7551867406562152e-001,
       6.9190195114274877e-001, 4.3594718235883412e-001,
       8.1769588405826155e-001, 4.9489931100219109e-001,
       2.9402059989690299e-001 ]
T: !!opencv-matrix
   rows: 3
   cols: 1
   dt: d
   data: [ 1.2315362401819163e+000, -9.6293867563953039e-001,
       1.5791089847312716e+000 ]

Having a valid point pair, P_k1 = {0.01334677 -0.3134326 2.604} and P_k2 = {-0.9516979 -0.3950531 2.483} the given R and T does not seem to be right.

I am assuming the P_k2 = R*P_k1 + T formula.

Any idea where the error comes from, or how to improve my results?