Unable to calibrate fisheye image from Python

edit

Hi Guys,

I am trying to calibrate a fisheye camera using the cv2.fisheye.calibration function, but I am unable to succeed.

I have succeded using the regular cv2.calibrateCamera() function in the past so I was hoping the fisheye version

I keep getting errors about the dimensions of the object and image points being passed into the function. Some magic happens between python and the c++ so I have a hard time controlling how the point lists are passed to the function.

The error is this:

OpenCV Error: Assertion failed (objectPoints.type() == CV_32FC3 || objectPoints.type() == CV_64FC3) in calibrate, file /Users/jesper/opencv3/opencv/modules/calib3d/src/fisheye.cpp, line 695
Traceback (most recent call last):
  File "extract.py", line 71, in <module>
    print cv2.fisheye.calibrate(obj_points, img_points, (w, h), None, None)
cv2.error: /Users/jesper/opencv3/opencv/modules/calib3d/src/fisheye.cpp:695: error: (-215) objectPoints.type() == CV_32FC3 || objectPoints.type() == CV_64FC3 in function calibrate

I feel that it has something to do about how the arrays are being casted before passed to c++. But I also suspect that there is something fishy about the function requiring CV_64FC3 types the object points are of the format [[x,y,z],[x,y,z],[x,y,z],...]

The file I am trying to calibrate against is uploaded here: https://youtu.be/uFvRySgiXpY

I am on version 3.0.0-dev

A working example would be highly appreciated.

My code is:

#!/usr/local/bin/python

import cv2
import sys
import numpy as np
import glob
import sys
import pickle

print "OpenCV version: ", cv2.__version__

video_file = sys.argv[1]

pattern_size = (8, 6)
square_size = float(1.0)
pattern_points = np.zeros( (np.prod(pattern_size), 3), np.float32 )
pattern_points[:,:2] = np.indices(pattern_size).T.reshape(-1, 2)
pattern_points *= square_size

obj_points = []
img_points = []
h, w = 0, 0

cap = cv2.VideoCapture(video_file)
interleave = int(cap.get(cv2.CAP_PROP_FRAME_COUNT) / 65.0)

cv2.waitKey(1000)
while not cap.isOpened():
    cap = cv2.VideoCapture(video_file)
    cv2.waitKey(1000)
    print "Wait for the header"

pos_frame = cap.get(cv2.CAP_PROP_POS_FRAMES)
print "analyzing video"
nextframe = 0
while True:
    flag, frame = cap.read()
    if flag:
        # The frame is ready and already captured
        pos_frame = cap.get(cv2.CAP_PROP_POS_FRAMES)
        if pos_frame >= nextframe:
            cv2.waitKey(10)
            img = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
            print "frame: ", pos_frame
            h, w = img.shape[:2]
            found, corners = cv2.findChessboardCorners(img, pattern_size)
            nextframe += interleave
            if found:
                cv2.drawChessboardCorners(frame, pattern_size, corners, found) 
                sml = cv2.resize(frame, (w/2,h/2))
                cv2.imshow('video', sml)
                term = ( cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_COUNT, 30, 0.1 )
                cv2.cornerSubPix(img, corners, (5, 5), (-1, -1), term)

            if not found:
                print 'chessboard not found'
                continue
            img_points.append(corners.reshape(-1, 2))
            obj_points.append(pattern_points)
    else:
        cap.set(cv2.CAP_PROP_POS_FRAMES, pos_frame-1)
        print "frame is not ready"
        cv2.waitKey(3000)


    if cap.get(cv2.CAP_PROP_POS_FRAMES) == cap.get(cv2.CAP_PROP_FRAME_COUNT):
        break

cv2.destroyAllWindows()


print cv2.fisheye.calibrate(obj_points, img_points, (w, h), None, None)