Ask Your Question
2

What does projection matrix provided by the calibration represent?

asked 2014-01-16 10:44:50 -0600

matteo gravatar image

updated 2014-01-17 09:29:14 -0600

I'm using such a tool from ROS/OpenCV in order to perform the camera calibration. The procedure ends up providing: camera matrix, distortion parameters, rectification matrix and projection matrix. As far as I know the projection matrix contains the intrinsic parameter matrix of the camera multiplied by the extrinsic parameters matrix of the matrix. The extrinsic parameter matrix itself provides the roto-translation of the camera frame with respect to the world frame. If these assumptions are correct...how the projection matrix is computed by Opencv? I,m not defining any world frame!

camera matrix 414.287922 0.000000 382.549277 0.000000 414.306025 230.875006 0.000000 0.000000 1.000000

distortion -0.278237 0.063338 -0.001382 0.000732 0.000000

rectification 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000

projection 297.051453 0.000000 387.628900 0.000000 0.000000 369.280731 227.051305 0.000000 0.000000 0.000000 1.000000 0.000000

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
4

answered 2014-01-16 15:07:13 -0600

jensenb gravatar image

You are correct that the projection matrix P (dimensions 3 x 4) contains the extrinsic and intrinsic camera parameters and is intended to be used to map homogenous world coordinates (4 x 1) to homogeneous camera coordinates (3 x 1). When performing calibration of a single camera, constructing a projection matrix for the camera doesn't make much sense (since the calibration procedure has no way to determine the relation to an arbitrary world frame), and unsurprisingly the underlying OpenCV calibration function does not return one. The reason you see a projection matrix being created with the ROS calibration tool is that ROS is simply returning a CameraInfo message. This message is meant for passing information about arbitrary cameras (not just during calibration), where it makes sense to include information a camera's extrinsics. In your case the Matrix should be P = [K | 0], since your camera should initially have no orientation and no translation to world frame.

edit flag offensive delete link more

Comments

if K stands for intrinsic matrix...no..in my case that's not true. i have just added the results of the calibration in the post.

matteo gravatar imagematteo ( 2014-01-17 09:41:26 -0600 )edit
1

The convention for the projection matrix is P = K [ R | t ], you will find this definition all the standard references (see Hartley and Zisserman p. 156 eg. 6.8). In the case of mono camera R should be the identity and t a null vector, resulting in P = K [ I | 0 ] = [K | 0]. Note that camera matrix K can be scaled with an arbitrary scale factor, so that you get more or less visible image after undistortion. I cannot see a simple scaling between the camera matrix and the projection matrix, something doesn't seem right with the values you posted. Since the calibration is through ROS. I suggest asking on their question site, after all OpenCV does not return a P matrix as part of the monocular calibration.

jensenb gravatar imagejensenb ( 2014-01-17 11:53:28 -0600 )edit

P is quite confusing here, the answer from @windonground is more correct. To reproject points from camera frame to pixels, you should use different matirces depending on the image you want to project on (raw distorted image or rectified image).

Mehdi gravatar imageMehdi ( 2016-09-12 07:03:35 -0600 )edit
1

answered 2015-05-08 01:25:32 -0600

windonground gravatar image

The meanning is shown in the definition of sesor_msgs/CameraInfo.msg in ros API: http://docs.ros.org/api/sensor_msgs/h...
P is needed in this type of msg.

Projection/camera matrix
[fx' 0 cx' Tx]
P = [ 0 fy' cy' Ty]
[ 0 0 1 0]
By convention, this matrix specifies the intrinsic (camera) matrix
of the processed (rectified) image. That is, the left 3x3 portion
is the normal camera intrinsic matrix for the rectified image.
It projects 3D points in the camera coordinate frame to 2D pixel
coordinates using the focal lengths (fx', fy') and principal point
(cx', cy') - these may differ from the values in K.
For monocular cameras, Tx = Ty = 0. Normally, monocular cameras will
also have R = the identity and P[1:3,1:3] = K.
For a stereo pair, the fourth column [Tx Ty 0]' is related to the
position of the optical center of the second camera in the first
camera's frame. We assume Tz = 0 so both cameras are in the same
stereo image plane. The first camera always has Tx = Ty = 0. For
the right (second) camera of a horizontal stereo pair, Ty = 0 and
Tx = -fx' * B, where B is the baseline between the cameras.
Given a 3D point [X Y Z]', the projection (x, y) of the point onto
the rectified image is given by:
[u v w]' = P * [X Y Z 1]'
x = u / w
y = v / w
This holds for both images of a stereo pair.

It says in monocamera, for rectified image, P is [K | 0]; for raw image P is [K' | 0], the parameter of K' may differ from K, which can be thought as computed by projecting 3D points to rectified images. I guess K' is computed from projecting 3D points to raw images!

edit flag offensive delete link more

Comments

at the end you mean [K' | 0] for rectified image and [K | 0] for raw image right? That is what is written in the ros message comments.

Mehdi gravatar imageMehdi ( 2016-09-12 07:26:48 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2014-01-16 10:44:50 -0600

Seen: 13,834 times

Last updated: May 08 '15