In computer vision a camera matrix or projection matrix is a matrix which
describes the mapping of a pinhole camera from 3D points in the world to 2D
points in an image. Let be a representation of a 3D point
in homogeneous coordinates, and let be a representation of the image of this
point in the pinhole camera. Then the following relation holds
where is the camera matrix and the sign implies that the left and right
hand sides are equal up to a non-zero scalar multiplication.
Since the camera matrix is involved in the mapping between elements of two
projective spaces, it too can be regarded as a projective element. This
means that it has only 11 degrees of freedom since any multiplication by a
non-zero scalar results in an equivalent camera matrix.
Derivation The mapping from the coordinates of a 3D
point P to the 2D image coordinates of the point's projection onto the image
plane, according to the pinhole camera model is given by
where are the 3D coordinates of P relative to a camera centered coordinate
system, are the resulting image coordinates, and f is the camera's focal
length for which we assume f > 0. Furthermore, we also assume that x3 > 0.
To derive the camera matrix this expression is rewritten in terms of
homogeneous coordinates. Instead of the 2D vector we consider the projective
element and instead of equality we consider equality up to scaling by a
non-zero number, denoted . First, we write the homogeneous image coordinates
as expressions in the usual 3D coordinates.
Finally, also the 3D coordinates are expressed in a homogeneous
representation and this is how the camera matrix appears:
or where is the camera matrix, which here
is given by and the corresponding camera matrix now
becomes The last step is a consequence of
itself being a projective element. The camera matrix derived here may
appear trivial in the sense that it contains very few non-zero elements.
This depends to a large extent on the particular coordinate systems which have
been chosen for the 3D and 2D points. In practice, however, other forms of camera
matrices are common, as will be shown below.
Camera position The camera matrix derived in the
previous section has a null space which is spanned by the vector
This is also the homogeneous representation of the 3D point which has
coordinates, that is, the "camera center" is at O. This means that the
camera center cannot be mapped to a point in the image plane by the camera.
For any other 3D point with , the result is well-defined and has the form . This
corresponds to a point at infinity in the projective image plane.
Normalized camera matrix and normalized image coordinates
The camera matrix derived above can be simplified even further if we assume
that f = 1: where here denotes a identity matrix.
Note that matrix here is divided into a concatenation of a matrix and a
3-dimensional vector. The camera matrix is sometimes referred to as a canonical
form. So far all points in the 3D world have
been represented in a camera centered coordinate system, that is, a coordinate
system which has its origin at the camera center. In practice however, the
3D points may be represented in terms of coordinates relative to an arbitrary
coordinate system. Assuming that the camera coordinate axes and the axes are
of Euclidean type, there is a unique Euclidean 3D transformation between the
two coordinate systems. In other words, the camera is not necessarily at the
origin looking along the z axis. The two operations of rotation and
translation of 3D coordinates can be represented as the two matrices
and where is a rotation matrix and is a
3-dimensional translation vector. When the first matrix is multiplied onto the
homogeneous representation of a 3D point, the result is the homogeneous
representation of the rotated point, and the second matrix performs instead a
translation. Performing the two operations in sequence gives a combined
rotation and translation matrix Assuming that and are precisely the
rotation and translations which relate the two coordinate system and above,
this implies that where is the homogeneous representation
of the point P in the coordinate system. Assuming also that the camera matrix is
given by , the mapping from the coordinates in the system to homogeneous
image coordinates becomes Consequently, the camera matrix which
relates points in the coordinate system to image coordinates is
a concatenation of a 3D rotation matrix and a 3-dimensional translation vector.
This type of camera matrix is referred to as a normalized camera matrix, it
assumes focal length = 1 and that image coordinates are measured in a coordinate
system where the origin is located at the intersection between axis X3 and the
image plane and has the same units as the 3D coordinate system. The resulting
image coordinates are referred to as normalized image coordinates.
= The camera position = Again, the null space of the normalized
camera matrix, described above, is spanned by the 4-dimensional vector
This is also, again, the coordinates of the camera center, now relative to the
system. This can be seen by applying first the rotation and then the
translation to the 3-dimensional vector and the result is the homogeneous
representation of 3D coordinates. This implies that the camera center lies
in the null space of the camera matrix, provided that it is represented in terms
of 3D coordinates relative to the same coordinate system as the camera matrix
refers to. The normalized camera matrix can now be
written as where is the 3D coordinates of the
camera relative to the system. General camera matrix
Given the mapping produced by a normalized camera matrix, the resulting
normalized image coordinates can be transformed by means of an arbitrary 2D
homography. This includes 2D translations and rotations as well as
scaling but also general 2D perspective transformations. Such a transformation
can be represented as a matrix which maps the homogeneous normalized image
coordinates to the homogeneous transformed image coordinates :
Inserting the above expression for the normalized image coordinates in terms of
the 3D coordinates gives This produces the most general form of
camera matrix See also
3D projection References
Richard Hartley and Andrew Zisserman. Multiple View Geometry in computer
vision. Cambridge University Press. ISBN 0-521-54051-8.