WO2005073669A1

WO2005073669A1 - Semi and fully-automatic camera calibration tools using laser-based measurement devices

Info

Publication number: WO2005073669A1
Application number: PCT/US2005/003047
Authority: WO
Inventors: Yakup Genc
Original assignee: Siemens Corporate Research, Inc.
Priority date: 2004-01-29
Filing date: 2005-01-27
Publication date: 2005-08-11

Abstract

A system and method for externally calibrating a camera in photogrammetry are disclosed. The system includes a camera for capturing an image; a light source coupled to the camera for projecting beams of coherent light against a target to create a plurality of markers on the target; and a computer communicating with the camera and light source and having a processor and memory containing machine readable instructions which, when executed by the processor, enable the computer to capture images from the camera containing the plurality of markers and distance information from the light source to estimate the calibration parameters using the detected locations of the markers in the images and the measured distances.

Description

SEMI AND FULLY-AUTOMATIC CAMERA CALIBRATION TOOLS USING LASER-BASED MEASUREMENT DEVICES This non-provisional application claims the benefit of Provisional Appl. Serial. No. 60/540,151, entitled "Semi- and Full-Automatic Camera Calibration Tools using Laser-Based Measurement Devices, filed on January 29, 2004.

BACKGROUND OF THE INVENTION [0001] The present invention relates generally to computer vision or photogrammetry, and more particularly, to a system and method for calibrating a camera using a laser distance measurement device. [0002] In the field of computer vision or photogrammetry, a camera is a device that projects a three-dimensional world onto a two-dimensional image. Camera calibration is the process that estimates the parameters of this projection. The camera or projection parameters can be categorized by internal parameters and external parameters. The estimation of these parameters is called internal calibration and external calibration or pose estimation, respectively. [0003] The internal parameters of the camera are commonly referred to as the focal length, aspect ratio, and principal point or image center. These internal parameters and various lens aberrations such as radial and tangential distortion, explain how a camera projects the three-dimensional models measured in its coordinate frame. Internal parameters, therefore, remain fixed so long as the physical characteristics of the camera are fixed. When the three-dimensional model is measured in a different coordinate system, a transformation from this coordinate system to the camera's coordinate system is required to completely define the projection process. This transformation is captured by the external parameters of the camera. The external parameters can also be seen as the position and orientation of the camera or pose. When the camera or the three-dimensional models move, the external parameters change even though the intrinsic parameters of the camera remain fixed. [0004] Computer vision and photogrammetry literature is abundant with camera calibration and pose estimation methods. See, e.g., R.Y. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-The-Shelf TV Cameras," IEEE Journal of Robotics and Automation, RA- 3(4): pp323-344, 1987. The methods for precise calibration assume an available set of correspondences between certain three-dimensional objects and their images. These correspondences, e.g., points, lines and circular objects, provide the data to estimate the calibration parameters. [0005] One of the most time consuming and critical tasks in a camera calibration process is the gathering of calibration data. The calibration data consists of a set of correspondences between known three-dimensional objects and their projections in the image taken by the camera to be calibrated. The first task in data gathering is the placement or measurement of a set of three-dimensional landmarks in a scene. In the minimal case, the position of three scene points and their locations in the image need to be known in order to externally calibrate the camera. The second task is to identify the images of the landmarks in the image and to measure their locations. For static cameras, data gathering can be implemented in two ways. In the first, a necessary number of landmarks are placed in a scene. The three-dimensional locations of the landmarks can be measured in a common coordinate system. Then, one or more images of the scene can be obtained. After the landmarks are identified and their locations computed in the images, the camera can be calibrated using the correspondences. In the second, a single landmark is placed in measured locations in the scene. An image for each location of the marker can be obtained. After identifying the landmark in each image, the correspondences can be used to calibrate the camera. [0006] The first method for data gathering only requires a single image whereas the second method requires several images. The preparation time for the first method may be shorter if pre-fabricated objects are used, e.g., a precisely measured three-dimensional calibration grid. The second method, on the other hand, may be desirable in applications where it is not possible to place a single calibration grid in the scene due to scene complexity. [0007] In the prior art, U.S. Patent No. 5,589,942, discloses discloses a system which utilizes two flexibly located cameras for receiving and recording visual information with respect to a sensed object illuminated by a series of light planes. Each pixel of each image is converted to a digital word and the words are grouped into stripes, each stripe comprising contiguous pixels. One pixel of each stripe in one image is selected and an epi-polar line of that point is drawn in the other image. The three dimensional coordinate of each selected point is determined by establishing the point on said epi-polar line which also lies on a stripe in the second image and which is closest to a known light plane. [0008] U.S. Patent No. 4,627,734, discloses a three-dimensional imaging system operating in accordance with the known active triangulation method, and employing a laser beam that is projected onto an area of the surface to be examined. The solution is characterized by synchronized scanning of projected and deflected beams. [0009] U.S. Patent No. 5,440,392, discloses a technical solution for point by point measurement of spatial coordinates. A touch probe, including three point- sized light sources at known local coordinates, touches the point to be measured. Using a camera to acquire an image of the three light sources, the system calculates the three dimensional coordinates of the point touched. [0010] U.S. Patent No. 6,256,099, discloses a method and system for measuring three dimensional spatial coordinates and for external camera calibration necessary for that measurement by projecting a pattern of light onto a surface used as a photogrammetric target. Images of the pattern are captured by two or more cameras and processed by a computer provided with software using conventional algorithms to identify homologous points. These points are then used to calculate the external calibration of the cameras, or they are used in conjunction with previously calculated external camera calibration in order to calculate three dimensional spatial coordinates on the surface of an object to be measured. SUMMARY OF INVENTION [0011] In accordance with an aspect of the present invention, there is provided a system and methodology for calibrating a camera using a laser measurement device. [0012] In accordance with a first aspect of the invention, a method of externally calibrating a camera comprises the steps of: (a) statically locating the camera;

(b) projecting beams of coherent light from a light source disposed proximal to the camera so as to create a plurality of markers on a target; (c) detecting the markers in an image obtained by the camera; (d) measuring the distances from the light source to the markers on the target; and (f) estimating the calibration parameters using the detected locations of the markers in the images and the measured distances. The light source is preferably a distance measuring laser, but may be any coherent light source within the scope of the invention. In a preferred expedient, the laser is affixed to the camera and permitted to move relative thereto in two degrees of freedom. The method encompasses projecting at least three beams of light to create at least three markers on the target. Preferably, seven to ten markers may be projected to achieve optimal results. The markers can be visually distinguishable or time coded. In accordance with a second aspect of the invention, a system for externally calibrating a camera comprises: a camera for capturing an image; a light source (e.g., a laser) coupled to the camera for projecting beams of coherent light against a target to create a plurality of markers on the target; and a computer communicating with the camera and light source and having a processor and memory containing machine readable instructions which, when executed by the processor, enable the computer to capture images from the camera containing the plurality of markers and distance information from the light source to estimate the calibration parameters using the detected locations of the markers in the images and the measured distances. [0013] These and other advantages of the invention will be apparent to those of ordinary skill in the art by reference to the following detailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS [0014] FIG. 1 is a schematic of a system for externally calibrating a camera in accordance with an aspect of the invention; [0015] FIG. 2 is a three-dimensional illustration of the geometry for the calibration problem addressed by the invention; [0016] FIGS. 3 A and 3B are schematics of different laser projections on a target surface forming part of the calibration process; and [0017] FIG. 4 is a flowchart of an illustrative process in accordance with an aspect of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0018] FIG. 1 depicts a system 100 for measuring three-dimensional spatial coordinates and implementing a process for externally calibrating a camera 102 by using a laser distance measuring device (hereinafter "laser") 104. The system 100 includes a conventional computer 106, comprising a processing unit 108, a system memory 1 10, and a system bus 112 that couples the various system components including the system memory to the processing unit 108. The system bus 112 may be any of several types of bus structures including a memory bus or memory controller, peripheral bus, and a local bus using any of a variety of bus architectures. The system memory 110 includes read only memory (ROM) and random access memory (RAM). The system memory 110 further includes a basic input/ output system (BIOS) which contains the basic routine that helps to transfer information between elements in computer 106. The computer 106 further includes data storage 114 which may comprise a hard disk drive, magnetic disk drive, optical disk drive or the like and the appropriate interfaces to the system bus 112. The drives and their associated computer-readable media provide non- volatile storage of computer readable instructions, data structures, program modules and other data for the computer 106. A number of program modules may be stored on the hard disk, magnetic disk, optical disk, ROM or RAM, including an operating system, one or more application programs, other program modules, and program data. A user of the system can enter commands via a keyboard 116 coupled to the system bus 112 through a serial or USB interface 118. The camera 102 is coupled to the system bus 112 via a camera interface 120 of the type well-known in the art (e.g., USB). Images from the camera 102 may be routed to and stored in the RAM of the system memory 110 or one of the other data storage devices in data storage 114 associated with the computer 106. The computer may be provided with a network interface 122 to enable networked communication with a remote computer(s) 124. The remote computer can be a personal computer, server, router, network PC, peer device or other common network node. A display 125 is coupled to the system bus 112 through a video adapter 127 in a conventional manner. [0019] Lasers 104 for measuring distances are commercially available. The laser 104 is preferably mounted above camera 102 on a movable support assembly 126 that enables the laser to be pointed anywhere along the x, y and z axis as depicted by arrows 128, 130, and 132, respectively. The support assembly 126 may comprise at least one motor (i.e., servo motors, not shown) to enable movement as required. The motion of the laser 104 may be controlled by computer 106 or by an external control panel (not shown). Of course it will be appreciated by those skilled in the art that the laser 104 could be mounted below the camera 102 or even held close to the camera 102 in the hands of a user. The laser projects a coherent beam of light to establish a visually distinguishable marker 134 on the surface 136 of an object 138 in a scene. The laser 104 communicates distance information to the projected marker 134 on the surface 136 to the computer 106 via a laser interface 140 coupled to the system bus 1 12. This may be implemented by a USB or like connection of the type well-known in the art. Alternatively, the laser distance may be read from a display on the device and manually entered by a user via the keyboard 116. [0020] Referring now to FIG. 2, there is depicted a three-dimensional illustration of the geometry for camera calibration in accordance with the present invention. The basic underlying mathematical problem for the proposed calibration method can be described given: n unknown three-dimensional (scene) points (P,- for i = 1, ..., «), the images of -P,'s represented by ;,-, the distances (represented by di) from a fixed point T close to the camera to each _P,-; find: the external parameters of the calibration and the coordinates of the scene points -P,-. The projection equation for the camera that takes a scene point P,- onto the image point/;,- is represented by: p_Pi = RPi+ T, where R and T represent the rotation and translation of the camera with respect to a world coordinate system. The Euclidian distance between the reference point X and scene point ,- is given by:

The task now is to find the motion of the camera, i.e., R and T, and the positions of the points. [0021] In order to externally calibrate the camera, without loss of generality, three of the scene points are set a particular coordinates, such as, for example:

This essentially guarantees a minimal representation for the bundle adjustment process necessary to recover the camera motion and scene points. Of course, it will be appreciated by those skilled in the art that if some or all of the scene points are already known, the optimization problem can be further simplified. It should be noted that if the scene points are to be defined in a different coordinate system, an appropriate coordinate system transformation can be applied to ascertain the correct motion of the camera. [0022] In view of the above, the imaging equations and distance measurements enable the unknown motion and scene point coordinates to be recovered. By simple observation, one can note that each imaging equation yields two independent equations, and thus 2n independent equations for n points. Similarly, each distance measurement provides a single equation, and thus n equations for n points. On the other hand, there are 6 unknowns from the camera motion and 3n - 6 unknown scene point coordinates. Furthermore, given that the reference point X is known with respect to the camera (Xo), i.e., X = R^T(Xo - T), a system of equations is obtained with 3n unknowns and 3n + 3 equations. This system can be solved using optimization techniques. [0023] It should be noted that the effect of not fixing a point from which the distance measurements are taken will cause errors. However, these errors can be quantified given the scene geometry and the distance from the camera to the scene. In applications like surveillance, the camera is typically placed at a relatively far distance from the scene. Accordingly, such errors will be relatively small. Another way to reduce the number of unknowns is by establishing areas in the scene where the scene points are taken from planar surfaces or known geometric shapes. This will reduce the number of unknowns and eliminate the requirement for knowing the reference point X. [0024] Referring now to FIGS. 3 A and 3B, the camera calibration process is illustrated with respect to two points 334a, 334b on the surface 336 an object 338 in a scene. In accordance with the process, a camera 302 is first placed statically in a location where external calibration is needed. A laser 304 is either held close to the camera 302 by a user (in a manual system) or mounted on an apparatus 326 coupled to the camera 302 as described above. For the purpose of this description, it is assumed that the laser 304 is coupled to the camera. The laser 304 is capable of projecting one marker that this visually distinguishable by the camera 302. Alternatively, the laser 304 can project a set of markers, e.g., a cross or a circle, in the scene. Time coded markers can also be utilized to provide a unique marker that can be decoded in the images taken by the camera 302. By orienting the laser 304 such that the originating rays of light approximately pass through a fixed point, the visual marker is projected onto distinct locations on the object 338 in the scene. In FIG. 3A, the marker from the first position of the laser 304 is represented by point 334a, and in FIG. 3B, the marker from the second position of the laser 304 is represented by point 334b. At least three such projections are required (the third is not shown), and the most reliable results will be achieved with approximately 7-10 projections. The 3D locations of these projected points/markers are not required. If a global coordinate system needs to be used, then at least three of these points must be known in that coordinate system. A bundle adjustment process can provide the positions of all of these points in the global coordinate system. The accuracy of the calibration depends on the ability to locate the point from where the laser calibration is implemented and the distances taken. In the case of a laser 304 that is affixed to the camera 302, the originating point for each of the light beams is fixed by design and thus known with respect to the camera's local coordinate system. For each location of the projection, e.g., points 334a, 334b, the marker is detected in the image obtained by the camera 304 on the computer 306. The process can be automated as the projected markers are coded and designed to be detected easily. The distance between the laser and the projected marker is also sent to the computer 306. The detected locations of the markers in the images and the measured distances from the laser to the marker points can then be used to estimate the calibration parameters in accordance with the principles described above. [0025] Referring now to FIG. 4, there is depicted a flow chart of a process for calibrating a camera in accordance with an aspect of the present invention. In step 400, a camera is placed relative to an object for external calibration. In step 402, using a laser disposed at a known location relative to the camera, a marker is projected onto a surface of the object. In step 404, the marker is detected in the image taken by the camera and sent to the computer. In step 406, the distance from the laser to the marker is sent to the computer. In step 408, if additional projections are required as described above (i.e., at least 3 and preferably 7-10), then the process repeats steps 402, 404 and 406 until enough projections have been made. At step 410 the computer estimates the calibration parameters in accordance with the principles described above. [0026] The present invention has been shown and described in what are considered to be the most practical and preferred embodiments. It is anticipated, however, that departures may be made therefrom and that obvious modifications will be implemented by those skilled in the art. It will be appreciated that those skilled in the art will be able to devise numerous arrangements and variations which, although not explicitly shown or described herein, embody the principles of the invention and are within their spirit and scope.

Claims

CLAIMSWe claim:

1. A method of externally calibrating a camera, comprising the steps of: (a) statically locating the camera; (b) projecting beams of coherent light from a light source disposed proximal to the camera so as to create a plurality of markers on a target; (c) detecting the markers in an image obtained by the camera; (d) measuring the distances from the light source to the markers on the target; and (f) estimating the calibration parameters using the detected locations of the markers in the images and the measured distances.

2. The method recited in claim 1, wherein the light source is a laser.

3. The method recited in claim 2, wherein the laser is affixed to the camera and permitted to move relative thereto in two degrees of freedom.

4. The method recited in claim 1, wherein at least three beams of light are projected to create at least three markers on the target.

5. The method recited in claim 1, wherein the markers are visually distinguishable.

6. The method recited in claim 1, wherein the markers are time coded.

7. A system for externally calibrating a camera, comprising: a camera for capturing an image; a light source coupled to the camera for projecting beams of coherent light against a target to create a plurality of markers on the target; and a computer communicating with the camera and light source and having a processor and memory containing machine readable instructions which, when executed by the processor, enable the computer to capture images from the camera containing the plurality of markers and distance information from the light source to estimate the calibration parameters using the detected locations of the markers in the images and the measured distances.

8. The system recited in claim 7, wherein the light source is a laser.

9. The system recited in claim 7, wherein the laser is affixed to the camera and adapted to move relative thereto in two degrees of freedom.

10. The system recited in claim 9, further comprising a motor for moving the laser relative to the camera.

11. The system recited in claim 7, wherein the light source projects markers that are visually distinguishable.

12. The system recited in claim 7, wherein the light source projects markers that are time coded.

13. A system for externally calibrating a camera, comprising: a camera for capturing an image; and a laser affixed to the camera for projecting beams of coherent light against a target to create a plurality of markers on the target, the laser being adapted to move relative to the camera in two degrees of freedom.

14. The system recited in claim 12, further comprising a motor for moving the laser relative to the camera.