Revision history [back]

Hi there!

I'm creating a university project with OpenCV Python and ArUco markers, where I would like to get a (relatively) robust pose estimation for the movement of the camera. I plan on using this for indoor drone flight graphing.

My idea is that the first seen marker will be the origin of my coordinate system. I store the origin of each marker (the first seen position transformed into the first markers coordinate system) and estimate a camera pose from that. My problem is, that I cannot seem to solve the transformation between the two markers' points.

I was able to get a camera position corrected for the marker rotation with the rotation matrix got from rvecs. I've been trying out almost all combinations of signs and matrices, after the theory did not work. Now I feel a little bit lost and am looking for some help with the transformations.

• first aruco marker: tvec_1, rvec_1 --> R1 (rotation matrix)

• second aruco marker: tvec_2, rvec_2 --> R2 (rotation matrix)

To get the camera coordinates in the marker's coordinate system: tvec_m = np.matmul(tvec, -R) I get two vectors like this (tvec_m1 and tvec_m2), both interpreted in their respective marker's Cartesian coordinates. (Just as the aruco.drawAxis function shows them.) These work pretty well, I can graph the camera position and get good values.

The mathematical transformation from the second to the first marker:

• p1 and p2 vectors, M1 and M2 are the respective coordinate system rotation matrices, O1 and O2 are the origins of coordinate systems

• O1 + M1 p1 = O2 + M2 p2 and so p1 = transpose[M1] (O2 - O1 + M2 p2)

In code it looks like this: (the matrix multiplication is reversed, because tvec is row vector instead of column)

d_origins = tvec_2 - tvec_1
tvec_m1 = np.matmul(tvec_1, -R1)
tvec_m2 = np.matmul(tvec_2, -R2)
tvec_m21 = np.matmul(d_origins + tvec_m2, R1.T)

Like this, I should be able to get tvec_m21 == tvec_m1 (of course not exactly, but approximately the same vector), but until yet, no luck.

Is this even a possible way for coordinate transformation between markers? I'm afraid, I keep falling for the same mistake and I won't notice it anymore. What am I doing wrong?

Thanks in advance!

I'm creating a university project with OpenCV Python and ArUco markers, where I would like to get a (relatively) robust pose estimation for the movement of the camera. I plan on using this for indoor drone flight graphing.

My idea is that graphing. For this, I have to transform the camera pose to world coordinates defined by the first seen marker.

I know there must be a transformation matrix between the markers, but I can't seem to figure out, what it is. I am trying with the difference of respective rvecs.

The code for the function in Python:

def TransformBetweenMarkers(tvec_m, tvec_n, rvec_m, rvec_n):
    tvec_m = np.transpose(tvec_m) # tvec of 'm' marker
    tvec_n = np.transpose(tvec_n) # tvec of 'n' marker
    # vector from 'm' to 'n' marker will be the origin of my in the camera's coordinate system. I store the origin of each marker (the first seen position transformed into the first markers coordinate system) and estimate a system
    dtvec = tvec_m - tvec_n

    # get the markers' rotation matrices respectively
    R_m = cv2.Rodrigues(rvec_m)[0]
    R_n = cv2.Rodrigues(rvec_n)[0]

    # camera pose from that. My problem is, that I cannot seem to solve the transformation between the two markers' points.
 
I was able to get a camera position corrected for the marker rotation with the rotation matrix got from rvecs. I've been trying out almost all combinations of signs and matrices, after the theory did not work. Now I feel a little bit lost and am looking for some help with the transformations.
 
 
first aruco marker: tvec_1, rvec_1 --> R1 (rotation matrix)
 
second aruco marker: tvec_2, rvec_2 --> R2 (rotation matrix)
 
 
To get the camera coordinates in the in 'm' marker's coordinate system: system
    tvec_mm = np.matmul(-R_m.T, tvec_m)
    # camera pose in 'n' marker's coordinate system
    tvec_nn = np.matmul(-R_n.T, tvec_n)

    # translational difference between markers in 'm' marker's system,
    # basically the origin of 'n'
    dtvec_m = np.matmul(-R_m.T, dtvec)

    # this gets me the same as tvec_mm,
    # but this only works, if 'm' marker is seen
    # tvec_nm = dtvec_m + np.matmul(-R_m.T, tvec_n)

    # something with the rvec difference must give the transformation(???)
    drvec = rvec_m-rvec_n
    # transformed to 'm' marker
    drvec_m = np.transpose(np.matmul(R_m.T, np.transpose(drvec)))
    dR_m = cv2.Rodrigues(drvec_m)[0]

    # I want to transform tvec_nn with a single matrix,
    # so it would be interpreted in 'm' marker's system
    tvec_nm = dtvec_m + np.matmul(dR_m.T, tvec_nn)

    # objective: tvec_mm == tvec_nm

This is the best I could get, but there is still an error value of +-0.03 meters between the tvec_m = np.matmul(tvec, -R) I get two vectors like this (tvec_m1tvec_mm and tvec_m2tvec_nm), both interpreted in their respective marker's Cartesian coordinates. (Just as the aruco.drawAxis function shows them.) These work pretty well, I can graph the camera position and translation values.

Is it possible to get good values.

The mathematical transformation from the second to the first marker:

• p1 and p2 vectors, M1 and M2 are the respective coordinate system rotation matrices, O1 and O2 are the origins of coordinate systems

• O1 + M1 p1 = O2 + M2 p2 and so p1 = transpose[M1] (O2 - O1 + M2 p2)

In code it looks like this: (the matrix multiplication is reversed, because tvec is row vector instead of column)

d_origins = tvec_2 - tvec_1
tvec_m1 = np.matmul(tvec_1, -R1)
tvec_m2 = np.matmul(tvec_2, -R2)
tvec_m21 = np.matmul(d_origins + tvec_m2, R1.T)

Like this, I should be able to get tvec_m21 == tvec_m1 (of course not exactly, but approximately the same vector), but until yet, no luck.

any better with this? Is this even a possible way for coordinate legit transformation between markers? I'm afraid, I keep falling for or just a huge coincidence, that it gives approximately the same mistake and I won't notice it anymore. What am I doing wrong?

Thanks in advance!values? Any ideas?

Thank you!