Ask Your Question
0

What is the intended result of ChArUco diamond detection?

asked 2016-05-17 15:29:09 -0600

tekulvw gravatar image

I've recently gotten into using the ArUco module in OpenCV 3.1 in order to do pose estimation for a project I'm working on. However, in the course of attempting to estimate pose and draw an axis, the detecting diamonds started behaving like below (diamonds are red borders, for the sake of clarity). My first thought is that diamonds should be surrounding the black checkerboard squares that are bordered by four markers and that the 5 diamonds below aren't accurate but I really don't have a clue. The second issue I've run into is that during live tests, pose estimate is very erratic. The drawn axis (from pose estimation) jumps around the entire image seemingly at random and nothing I've done so far has enabled me to get a reliable pose estimate even with a static camera and ChArUco board.

  1. Are the detected diamonds below accurate? If so, why (ELI5 if possible :) )?
  2. Why isn't the axis for the ChArUco board being drawn in the image below?
  3. Am I doing something wrong in the code below that's causing the live pose estimates to be so erratic?

    image description

import cv2
import numpy as np

def draw_axis(img, charuco_corners, charuco_ids, board):
    vecs = np.load("./calib.npz")  # I already calibrated the camera
    mtx, dist, _, _ = [vecs[i] for i in ('mtx', 'dist', 'rvecs', 'tvecs')]
    ret, rvec, tvec = cv2.aruco.estimatePoseCharucoBoard(
        charuco_corners, charuco_ids, board, mtx, dist)
    if ret is not None and ret is True:
        cv2.aruco.drawAxis(img, mtx, dist, rvec, tvec, 0.1)

def get_image(camera):
    ret, img = camera.read()
    return img

def make_grayscale(img):
    ret = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    return ret

def main():
    camera = cv2.VideoCapture(0)
    img = get_image(camera)
    while True:
        cv2.imshow('calibration', img)
        cv2.waitKey(10)
        img = get_image(camera)
        gray = make_grayscale(img)
        corners, ids, rejected = cv2.aruco.detectMarkers(gray, aruco_dict,
                                                         corners, ids)
        cv2.aruco.drawDetectedMarkers(img, corners, ids)
        if ids is not None and corners is not None \
                and len(ids) > 0 and len(ids) == len(corners):
            diamond_corners, diamond_ids = \
                cv2.aruco.detectCharucoDiamond(img, corners, ids,
                                               0.05 / 0.03, cameraMatrix=mtx,
                                               distCoeffs=dist)
            cv2.aruco.drawDetectedDiamonds(img, diamond_corners, diamond_ids)
            '''if diamond_ids is not None and len(diamond_ids) >= 4:
                break'''
            board = cv2.aruco.CharucoBoard_create(9, 6, 0.05, 0.03,
                                                  aruco_dict)
            if diamond_corners is not None and diamond_ids is not None \
                    and len(diamond_corners) == len(diamond_ids):
                count, char_corners, char_ids = \
                    cv2.aruco.interpolateCornersCharuco(diamond_corners,
                                                        diamond_ids, gray,
                                                        board)
                if count >= 3:
                    draw_axis(img, char_corners, char_ids, board)

if __name__ == '__main__':
    main()
 
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2016-05-17 18:59:54 -0600

Tetragramm gravatar image

In short, no. Look here for an explanation of the Charuco diamonds.

I think you aren't supposed to be looking for diamonds, I think you want a board. Look at this tutorial to see how it's supposed to work. It looks like the formatting broke, so check detect_board_charuco.cpp in the samples folder to see that.

I don't actually see what you're doing wrong, but I think doing the diamond detection before the draw_axis call is messing it up. You should be doing the interpolate corners on the first corners, not the diamond_corners. I think.

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2016-05-17 15:29:09 -0600

Seen: 2,182 times

Last updated: May 17 '16