WO2013005265A1 - Three-dimensional coordinate measuring device and three-dimensional coordinate measuring method - Google Patents

Abstract

A three-dimensional coordinate measuring device (90) is provided with: a first perspective projection angle extraction unit (103A) that acquires a first target perspective projection angle (θ_A1) corresponding to a target point (W) that is projected by a first target image (60A) imaged from a first perspective (M₁); a second perspective projection angle extraction unit (103B) that acquires a second target perspective projection angle (θ_B1) corresponding to the target point (W) that is projected by a second target image (60B) imaged from a second perspective (M₂); and a coordinate calculation unit (105) that calculates three-dimensional coordinates (112) of the target point (W) by using the first target perspective projection angle (θ_A1), the second target perspective projection angle (θ_B1), the direction of the optical axis (θ_A) of a first image capture device (10A), the direction of the optical axis (θ_B) of a second image capture device (10B), and the distance (d) between the perspectives.

Description

3D coordinate measuring apparatus and 3D coordinate measuring method

The present invention relates to a three-dimensional coordinate measuring apparatus and a three-dimensional coordinate measuring method for measuring three-dimensional coordinates of a target point using two images obtained by photographing the target point from different viewpoints.

A stereo method is known as a method of measuring the coordinates of relative points using an image.

In the stereo method, a target point is imaged by two cameras (imaging devices) placed at different positions, or a target point is imaged from two different positions by one camera. In the stereo method, the coordinates of the target point are calculated using these two images. Specifically, the stereo method calculates three-dimensional coordinates of a target point based on subtle differences in images based on parallax, as in human binocular vision (see, for example, Patent Document 1 and Patent Document 2). .

Japanese Patent Laid-Open No. 9-79845 JP-A-7-218251

However, in the technique described in Patent Document 1, two cameras must be arranged in parallel stereo, and accurate camera settings are required.

On the other hand, the technique described in Patent Document 2 calculates the three-dimensional coordinates of the target point even when the two cameras are deviated from the parallel stereo arrangement by correcting the deviation of the shooting direction of the cameras. It is possible.

However, the technique described in Patent Document 2 has a problem that correction cannot be performed when the arrangement of the two cameras deviates greatly from the stereo arrangement.

Therefore, an object of the present invention is to provide a three-dimensional coordinate measuring apparatus that can improve the degree of freedom of setting of two cameras.

In order to achieve the above object, a three-dimensional coordinate measuring apparatus according to an aspect of the present invention is a three-dimensional coordinate measuring apparatus that measures three-dimensional coordinates of a target point, and is a first imaging device from a first viewpoint. An image acquisition unit that acquires a first target image in which the target point is captured and a second target image in which the target point is captured by a second imaging device from a second viewpoint; For each pixel in the image captured by the imaging device, a line segment connecting each point of the three-dimensional coordinate system projected onto the pixel and the first viewpoint, and the optical axis of the first imaging device Information on the first pixel viewpoint projection angle, which is an angle formed, and for each pixel in the image captured by the second imaging device, each point of the three-dimensional coordinate system projected on the pixel and the second A second image that is an angle formed by a line segment connecting the viewpoint and the optical axis of the second imaging device. Information on viewpoint projection angles, first optical axis information indicating the optical axis direction of the first imaging device, second optical axis information indicating the optical axis direction of the second imaging device, and the first The storage unit that holds the inter-viewpoint information indicating the relative position between the second viewpoint and the second viewpoint, and the information about the first pixel viewpoint projection angle is used to project the first target image onto the first target image. Using a first viewpoint projection angle extraction unit that obtains a first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to a target point, and information about the second pixel viewpoint projection angle, A second viewpoint projection angle extraction unit that acquires a second target viewpoint projection angle that is the second pixel viewpoint projection angle corresponding to the target point projected on the second target image; Two target viewpoint projection angles, the optical axis directions of the first and second imaging devices, and the inter-viewpoint information. And a coordinate calculating unit for calculating three-dimensional coordinates of the point.

According to this configuration, the three-dimensional coordinate measuring apparatus according to an aspect of the present invention can easily calculate the three-dimensional coordinates of the target point using the viewpoint projection angle even when the two cameras are not arranged in parallel stereo. As described above, the three-dimensional coordinate measuring apparatus according to one embodiment of the present invention can improve the degree of freedom of setting two cameras.

In addition, the storage unit further includes information indicating a first rotation angle that is an angle at which the first imaging device rotates about the optical axis of the first imaging device, and the second The imaging device holds information indicating a second rotation angle that is an angle of rotation about the optical axis of the second imaging device, and the coordinate calculation unit includes the first and second objects. The three-dimensional coordinates of the target point are calculated using the viewpoint projection angle, the optical axis directions of the first and second imaging devices, the inter-viewpoint information, and the first and second rotation angles. May be.

According to this configuration, the three-dimensional coordinate measuring apparatus according to one aspect of the present invention can easily calculate the three-dimensional coordinates of the target point even when the camera rotates about the optical axis of the camera.

In addition, the coordinate calculation unit uses the first target viewpoint projection angle, the optical axis direction of the first imaging device, and the first rotation angle to use the first viewpoint and the second viewpoint. A first target angle that is an angle formed by a line segment that connects the first viewpoint and a line segment that connects the first viewpoint and the target point, and calculates the second target viewpoint projection angle, A line segment connecting the first viewpoint and the second viewpoint using the optical axis direction of the second imaging device and the second rotation angle, and the second viewpoint and the target point. A second target angle that is an angle formed by a connecting line segment is calculated, and the three-dimensional coordinates of the target point are calculated using the inter-viewpoint information, the first target angle, and the second target angle. May be calculated.

The first optical axis information includes a first reference line that passes through the first viewpoint and is perpendicular to a line segment that connects the first viewpoint and the second viewpoint; Information indicating a first photographing angle that is an angle formed with an optical axis of one imaging device, and the second optical axis information passes through the second viewpoint and the first viewpoint and the A second reference line that is perpendicular to a line connecting the second viewpoint and is parallel to the first reference line and an optical axis of the second imaging device; Information indicating a shooting angle, wherein the coordinate calculation unit includes an angle formed by a line segment connecting the first viewpoint and the second viewpoint and the first reference line, and the first viewpoint projection angle. And calculating the first target angle using the first imaging angle and the first rotation angle, and a line segment connecting the first viewpoint and the second viewpoint, and the second And the angle formed by the reference line, the second viewpoint projection angle, the second imaging angle, may calculate the second target angle using a said second rotation angle.

Further, the image acquisition unit further includes a first reference point, which is any three of three or more reference points whose relative coordinates are known, from the first viewpoint to the first imaging device. The second reference point, which is any one of the first reference image picked up by (1) and the three or more reference points, was picked up by the second image pickup device from the second viewpoint. A second reference image is acquired, and the three-dimensional coordinate measuring apparatus further uses the information of the first pixel viewpoint projection angle to project the first of the three points to be projected onto the first reference image. A third viewpoint projection angle extraction unit that obtains three first reference viewpoint projection angles that are the first pixel viewpoint projection angles corresponding to the reference point, and information on the second pixel viewpoint projection angle. The second pixels corresponding to the three second reference points projected onto the second reference image A fourth viewpoint projection angle extraction unit that obtains three second reference viewpoint projection angles that are point projection angles; and the relative coordinates of the three first reference viewpoint projection angles and the three first reference points. Are used to calculate a first inclination angle between a first imaging plane that is an imaging plane of the first reference image and a first reference plane that includes the three reference points. A second imaging surface, which is an imaging surface of the second reference image, using an angle calculation unit, the three second reference viewpoint projection angles and the relative coordinates of the three second reference points; A second inclination angle calculating unit for calculating a second inclination angle formed with a second reference plane including the three second reference points, the three first reference points, and the three reference points. Using the relative positional relationship with the second reference point, the first inclination angle, and the second inclination angle, the optical axis direction of the first imaging device and the second imaging device The optical axis direction of the a first rotation angle A, and a setting information calculating unit that calculates a second rotation angle.

According to this configuration, the three-dimensional coordinate measuring apparatus according to an aspect of the present invention can easily calculate the optical axis directions of the first and second imaging apparatuses using the viewpoint projection angle.

In addition, the setting information calculation unit includes a relative positional relationship between the three first reference points and the three second reference points, the first inclination angle, and the second inclination angle. May be used to calculate the three-dimensional coordinates of the first viewpoint and the second viewpoint, and to calculate the inter-viewpoint information from the three-dimensional coordinates of the first viewpoint and the second viewpoint. .

According to this configuration, the three-dimensional coordinate measuring apparatus according to an aspect of the present invention can easily calculate the inter-viewpoint distance using the viewpoint projection angle.

The present invention can be realized not only as such a three-dimensional coordinate measuring apparatus, but also as a three-dimensional coordinate measuring method using characteristic means included in the three-dimensional coordinate measuring apparatus as a step. It can also be realized as a program that causes a computer to execute typical steps. Needless to say, such a program can be distributed via a non-transitory computer-readable recording medium such as a CD-ROM and a transmission medium such as the Internet.

The present invention can provide a three-dimensional coordinate measuring apparatus that can improve the degree of freedom of setting of two cameras.

FIG. 1 is a diagram showing a configuration of a system including a three-dimensional coordinate measuring apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram of the three-dimensional coordinate measuring apparatus according to the embodiment of the present invention. FIG. 3 is a diagram for explaining the three-dimensional coordinate measurement process according to the embodiment of the present invention. FIG. 4 is a flowchart of the three-dimensional measurement process according to the embodiment of the present invention. FIG. 5 is a diagram showing a configuration of a system including a three-dimensional coordinate measuring apparatus during the first adjustment operation according to the embodiment of the present invention. FIG. 6 is a block diagram of the camera setting detection unit according to the embodiment of the present invention. FIG. 7 is a flowchart of the first adjustment operation according to the embodiment of the present invention. FIG. 8 is a diagram for explaining the first adjustment operation according to the embodiment of the present invention. FIG. 9A is a diagram showing a reference point projected on the first imaging surface according to the embodiment of the present invention. FIG. 9B is a diagram showing a reference point projected on the second imaging surface according to the embodiment of the present invention. FIG. 10 is a flowchart of the tilt angle calculation process according to the embodiment of the present invention. FIG. 11 is a diagram for explaining the tilt angle calculation processing according to the embodiment of the present invention. FIG. 12 is a diagram for explaining the tilt angle calculation processing according to the embodiment of the present invention. FIG. 13 is a diagram for explaining a first adjustment operation according to the embodiment of the present invention. FIG. 14 is a diagram showing a configuration of a system including a three-dimensional coordinate measuring apparatus at the time of the second adjustment operation according to the embodiment of the present invention. FIG. 15 is a flowchart of the second adjustment operation according to the embodiment of the present invention. FIG. 16 is a diagram for explaining a viewpoint projection angle according to the embodiment of the present invention. FIG. 17A is a diagram showing an example of an image before distortion correction according to the embodiment of the present invention. FIG. 17B is a diagram showing an example of an image after distortion correction according to the embodiment of the present invention. FIG. 18 is a diagram for explaining a second adjustment operation according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Each of the embodiments described below shows a preferred specific example of the present invention. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. The present invention is limited only by the claims. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present invention are not necessarily required to achieve the object of the present invention. It will be described as constituting a preferred form.

The 3D coordinate measuring apparatus according to the embodiment of the present invention extracts the viewpoint projection angle of the target point in each image from two images obtained by capturing the target point from different viewpoints. Then, the three-dimensional coordinate measuring apparatus calculates the coordinates of the target point using the viewpoint projection angle.

Thereby, the three-dimensional coordinate measuring apparatus according to the embodiment of the present invention can calculate the three-dimensional coordinates of the target point even when the two cameras are not arranged in parallel stereo. As described above, the three-dimensional coordinate measuring apparatus according to the embodiment of the present invention can improve the degree of freedom in setting two cameras.

First, the configuration of a three-dimensional coordinate measurement system including a three-dimensional coordinate measurement apparatus according to an embodiment of the present invention will be described.

FIG. 1 is a diagram showing a configuration of a three-dimensional coordinate measurement system according to an embodiment of the present invention.

1 measures the three-dimensional coordinates of the target point W included in the target structure 50. The three-dimensional coordinate measurement system shown in FIG. This three-dimensional coordinate measurement system includes a three-dimensional coordinate measurement device 90, cameras 10A and 10B that are imaging devices, and cables 30A and 30B.

The camera 10A is the target structure 50 is the first imaging device for generating image data 60A by capturing the first viewpoint M _1. Camera 10B is a target structure 50 is the second imaging device for generating image data 60B by capturing from a second viewpoint M _2. Here, the first viewpoint M ₁ and second viewpoint M ₂ corresponds to the position of the principal point of the photographic lens of the camera 10A and 10B. Further, in FIG. 1, O ₁ and O ₂ written by two-dot chain lines indicate the optical axes of the cameras 10A and 10B (the optical axes of the photographing lenses), respectively.

The relative positional relationship between the camera 10A and the camera 10B and the optical axes O ₁ and O ₂ are set in advance. For example, the camera 10A and the camera 10B are connected via a predetermined member. Thereby, the relative positional relationship between the camera 10A and the camera 10B and the orientation (optical axis) of the camera 10A and the camera 10B are fixed.

The image data 60A generated by the camera 10A is sent to the three-dimensional coordinate measuring device 90 via the cable 30A. The image data 60B generated by the camera 10B is sent to the three-dimensional coordinate measuring apparatus 90 via the cable 30B.

Cables 30A and 30B are, for example, USB (Universal Serial Bus) cables. The cables 30A and 30B may be cables other than USB. The cameras 10A and 10B may send the image data 60A and 60B to the three-dimensional coordinate measuring apparatus 90 via wireless communication. Alternatively, the image data 60A and 60B may be sent to the three-dimensional coordinate measuring device 90 using a recording medium that can be attached to and detached from the cameras 10A and 10B and the three-dimensional coordinate measuring device 90.

The three-dimensional coordinate measuring device 90 is, for example, a personal computer. The three-dimensional coordinate measuring apparatus 90 measures the three-dimensional coordinates of the target point W using the image data 60A and 60B.

FIG. 2 is a block diagram showing a characteristic functional configuration of the three-dimensional coordinate measuring apparatus 90 according to the embodiment of the present invention. For example, the function of the processing unit shown in FIG. 2 is realized by a processor included in a personal computer executing a program.

The three-dimensional coordinate measuring apparatus 90 calculates the three-dimensional coordinates 112 of the target point W using two images obtained by capturing the target point W from different viewpoints. Here, the three-dimensional coordinates 112 is, for example, the coordinates of the first point of view M ₁ in the three-dimensional coordinate system with the origin. Note that the three-dimensional coordinate system of the three-dimensional coordinates 112 is not limited to this, and an arbitrary three-dimensional coordinate system may be used. The three-dimensional coordinates 112 may be relative coordinates between a predetermined point and the target point, or may be information indicating a distance between the predetermined point and the target point W. The point defined here in advance, for example, a first viewpoint M ₁ or the second viewpoint M _2. Incidentally, the point this predetermined, may be any point existing between the first view point M ₁ and the second viewpoint M _2.

As shown in FIG. 2, the three-dimensional coordinate measuring apparatus 90 includes an image acquisition unit 101, a storage unit 102, a first viewpoint projection angle extraction unit 103A, a second viewpoint projection angle extraction unit 103B, and a coordinate calculation unit. 105 and a camera setting detection unit 106.

Image acquisition unit 101, the first target image target point W is imaged (image data 60A) by the first camera 10A from the viewpoint _{M 1,} target point W by the second viewpoint _{M 2} from the camera 10B is imaged The obtained second target image (image data 60B) is acquired.

The storage unit 102 holds the first viewpoint projection angle information 111A, the second viewpoint projection angle information 111B, the first shooting angle θ _A , the second shooting angle θ _B, and the inter-viewpoint distance d. To do.

FIG. 3 is a diagram showing the relationship between the cameras 10A and 10B and the target point W. As shown in FIG. The first reference line L shown in FIG. 3, as the first viewpoint _{M 1,} and a line segment _M 1 _{M 2} perpendicular straight line connecting a first point of view _{M 1} and the second viewpoint _{M 2} is there. The reference line R is a straight line that passes through the second viewpoint M ₂ and is perpendicular to the line segment M ₁ M ₂ . The reference line L and the reference line R are parallel lines.

In FIG. 3, two-dimensional coordinates and angles are illustrated, but actually three-dimensional coordinates and angles are used. Also in this case, since the same method as shown below can be used, the description will be given here with reference to FIG.

The first shooting angle θ _A is an angle formed between the first reference line L and the optical axis O ₁ of the camera 10A. The second shooting angle θ _B is an angle formed by the second reference line R and the optical axis O ₂ of the camera 10B. In other words, the first shooting angle θ _A is first optical axis information indicating the optical axis direction of the camera 10A, and the second shooting angle θ _B is second light indicating the optical axis direction of the camera 10B. Axis information.

Distance between viewpoints d show first viewpoint M ₁ and the distance between the second viewpoint M _2, the first viewpoint M ₁ and the arrangement relationship between the second viewpoint M _2. Wherein the first viewpoint _{M 1} and the positional relationship between the second viewpoint _{M 2,} for example, a camera 10A (first viewpoint _{M 1)} and a camera 10B (first viewpoint _{M 2)} either the right of This is information indicating which is arranged on the left side. That is, distance between viewpoints d is the relative position information indicating the first viewpoint M ₁ and second viewpoint M ₂ and the relative coordinate (relative position).

The first viewpoint projection angle information 111A is information indicating the viewpoint projection angle of each pixel (corresponding to the first pixel viewpoint projection angle of the present invention) in an image taken by the camera 10A. Here, the viewpoint projection angle is an angle formed by a line segment connecting each point of the three-dimensional coordinate system projected onto the pixel and the viewpoint and the optical axis of the imaging device (camera). The viewpoint projection angle is a value determined according to the camera setting, and is a value uniquely determined for each setting state of the camera. For example, the first viewpoint projection angle information 111A is a table indicating viewpoint projection angles corresponding to each of a plurality of pixels.

The second viewpoint projection angle information 111B is information indicating the viewpoint projection angle of each pixel (corresponding to the second pixel viewpoint projection angle of the present invention) in the image captured by the camera 10B. For example, the second viewpoint projection angle information 111B is a table indicating viewpoint projection angles corresponding to each of a plurality of pixels.

The first viewpoint projection angle information 111A, the second viewpoint projection angle information 111B, the first shooting angle θ _A , the second shooting angle θ _B, and the inter-viewpoint distance d are set in advance. It may be stored in the storage unit 102 or may be calculated by the camera setting detection unit 106 as described later.

Further, the first viewpoint projection angle information 111A, the second viewpoint projection angle information 111B, the first shooting angle θ _A , the second shooting angle θ _B, and the inter-viewpoint distance d are stored in a plurality of memories. It may be divided and stored in the unit 102.

Using the first viewpoint projection angle information 111A, the first viewpoint projection angle extraction unit 103A uses the viewpoint projection angle θ _A1 corresponding to the target point W projected on the first target image (the first viewpoint of the present invention). (Corresponding to the target viewpoint projection angle).

The second viewpoint projection angle extraction unit 103B uses the second viewpoint projection angle information 111B to generate a viewpoint projection angle θ _B1 corresponding to the target point W projected on the second target image (the second viewpoint of the present invention). (Corresponding to the target viewpoint projection angle).

The coordinate calculation unit 105 includes the target viewpoint projection angles θ _A1 and θ _B1 acquired by the first viewpoint projection angle extraction unit 103A and the second viewpoint projection angle extraction unit 103B, the first imaging angle θ _A, and the second The three-dimensional coordinates 112 of the target point W are calculated using the shooting angle θ _B and the inter-viewpoint distance d.

The three-dimensional coordinates 112 of the target point W calculated by the coordinate calculation unit 105 are displayed on, for example, a display unit (not shown) included in the three-dimensional coordinate measurement device 90. The three-dimensional coordinates 112 may be output to the outside of the three-dimensional coordinate measuring device 90 or may be stored in a storage unit provided in the three-dimensional coordinate measuring device 90.

The camera setting detection unit 106 includes first viewpoint projection angle information 111A, second viewpoint projection angle information 111B, a first shooting angle θ _A , a second shooting angle θ _B, and an inter-viewpoint distance d. The first viewpoint projection angle information 111A, the second viewpoint projection angle information 111B, the first shooting angle θ _A , the second shooting angle θ _B, and the inter-viewpoint distance d are calculated. Store in the storage unit.

Next, the operation of the three-dimensional coordinate measuring apparatus 90 will be described.

FIG. 4 is a flowchart of the three-dimensional coordinate measurement process performed by the three-dimensional coordinate measurement apparatus 90.

First, the image acquisition unit 101 acquires a first target image and a second target image (S101). Here, the first target image, the target point W by the first camera 10A from the view point M ₁ is an image that has been captured. The second target image, the target point W by the second viewpoint M ₂ from the camera 10B is an image that has been captured.

Next, the first viewpoint projection angle extraction unit 103A extracts the coordinates of the target point W from the first target image. Then, the first viewpoint projection angle extraction unit 103A refers to the first viewpoint projection angle information 111A and acquires the viewpoint projection angle θ _A1 corresponding to the extracted coordinates of the target point W. Similarly, the second viewpoint projection angle extraction unit 103B extracts the coordinates of the target point W from the second target image. Then, the second viewpoint projection angle extraction unit 103B refers to the second viewpoint projection angle information 111B to acquire the viewpoint projection angle θ _B1 corresponding to the extracted coordinates of the target point W (S102).

Next, the coordinate calculation unit 105 uses the first shooting angle θ _A , the second shooting angle θ _B, and the viewpoint projection angles θ _A1 and θ _B1 to use the first target angle θ shown in FIG. _A2 and the second target angle θ _B2 are calculated (S103). Here, as illustrated in FIG. 3, the first target angle θ _A2 is an angle formed by the line segment M ₁ M ₂ and the line segment M ₁ W. The second target angle θ _B2 is an angle formed by the line segment M ₁ M ₂ and the line segment M ₂ W.

Specifically, the coordinate calculation unit 105 calculates the first target angle θ _A2 using the following (Equation 1). In addition, the coordinate calculation unit 105 calculates the second target angle θ _B2 using the following (Formula 2).

θ _A2 = θ _L −θ _A −θ _A1 (Formula 1)
θ _B2 = θ _R −θ _B −θ _B1 (Formula 2)

Here, θ _L is an angle formed by the first reference line L and the line segment M ₁ M ₂ . θ _R is an angle formed by the second reference line R and the line segment M ₁ M ₂ . Further, here, the theta _L and theta _R, are both 90 degrees.

Next, the coordinate calculation unit 105 calculates the three-dimensional coordinate 112 of the target point W using the first target angle θ _A2 , the second target angle θ _B2, and the inter-viewpoint distance d (S104). .

Here, from the first target angle θ _A2 and the second target angle θ _B2 , three triangles WM ₁ M _{2 having} the target point W, the first viewpoint M ₁ and the second viewpoint M ₂ as vertices are used. You can see the inner corner. Furthermore, since the length (inter-viewpoint distance d) of one side M ₁ M ₂ of the triangle WM ₁ M ₂ is known, the three-dimensional coordinates 112 of the target point W are calculated.

Further, distance between viewpoints d is information indicating the first viewpoint M ₁ and the relative positional relationship between the second viewpoint M _2. In other words, by using the distance between viewpoints d, in any three-dimensional coordinate system can define a first viewpoint M ₁ and second three-dimensional coordinates of the viewpoint M _2. For example, the coordinate calculation unit 105 uses a three-dimensional coordinate system in which the coordinates of the first viewpoint M ₁ are the origin (0, 0, 0) and the coordinates of the second viewpoint M ₂ are (d, 0, 0). Set. Then, the coordinate calculation unit 105 calculates the coordinates of the target point W in the set three-dimensional coordinate system.

Also, for example, the three-dimensional coordinate calculation unit 105 can calculate the three-dimensional coordinates 112 of the target point W by the following method.

First, the coordinate calculation unit 105 calculates a first vector passing through the first viewpoint M ₁ and the target point W using the first target angle θ _A2 . Specifically, the coordinate calculation unit 105, through the first view point M _1, and to calculate the direction of the vector represented by the first target angle theta _A2 as the first vector. Note that the coordinate calculation unit 105 may use the first shooting angle θ _A and the viewpoint projection angle θ _A1 instead of the first target angle θ _A2 .

Similarly, the coordinate calculation section 105 uses the second target angle theta _B2, calculates a second vector that passes through the second viewpoint M ₂ and the target point W. Further, the coordinate calculation unit 105 may use the second shooting angle θ _B and the viewpoint projection angle θ _B1 instead of the second target angle θ _B2 .

Next, the coordinate calculation unit 105 calculates the coordinates of the closest point between the first vector and the second vector as the three-dimensional coordinates 112 of the target point W.

Through the above processing, the three-dimensional coordinate measuring apparatus 90 can calculate the three-dimensional coordinates 112 of the target point W.

As described above, the three-dimensional coordinate measuring apparatus 90 according to the embodiment of the present invention holds the viewpoint projection angle with respect to each pixel of the image captured by the camera 10A and the camera 10B in advance, and sets the viewpoint projection angle to be held. Using this, the three-dimensional coordinates 112 of the target point W are calculated. Specifically, the three-dimensional coordinate measuring apparatus 90 extracts the viewpoint projection angle of the target point in each image from two images obtained by capturing the target point from different viewpoints. Then, the three-dimensional coordinate measuring apparatus calculates the coordinates of the target point using the viewpoint projection angle.

Thereby, the three-dimensional coordinate measuring apparatus 90 according to the embodiment of the present invention can easily calculate the three-dimensional coordinates of the target point W. Further, the three-dimensional coordinate measuring device 90 calculates the three-dimensional coordinates of the target point W even when the two cameras are not arranged in parallel stereo by using the first photographing angle θ _A and the second photographing angle θ _B. it can. As described above, the three-dimensional coordinate measuring apparatus 90 according to the embodiment of the present invention can improve the degree of freedom in setting two cameras.

Further, in the techniques described in Patent Document 1 and Patent Document 2, essential information is required for the captured image. Specifically, in Patent Document 1, a license plate is required, and in Patent Document 2, “at least two straight lines that are mutually changed in real space” are required. On the other hand, the three-dimensional coordinate measuring apparatus 90 according to the present embodiment does not need such information.

In the above description, the example in which the first reference line L uses a straight line that passes through the first viewpoint M ₁ and is perpendicular to the line segment M ₁ M ₂ has been described. it may be a straight line passing through the first viewpoint M _1. Even in this case, the same processing as described above can be performed. Similarly, the second reference line R may be any straight line passing through the second point of view M _2. In other words, the first shooting angle θ _A and the second shooting angle θ _B may be information indicating the optical axis directions of the cameras 10A and 10B. For example, the first reference line L and the second reference line R may be straight lines that are parallel to each other and are not perpendicular to the line segment M ₁ M ₂ . Further, the angle formed by the line segment M ₁ M ₂ and the optical axis of the camera may be used as the first shooting angle θ _A and the second shooting angle θ _B.

Further, distance between viewpoints d may be information indicating the first viewpoint M ₁ and the relative positional relationship between the second viewpoint M _2. For example, instead of the distance between viewpoints d, may be used first viewpoint M ₁ coordinates and a second viewpoint M ₂ coordinates in the three-dimensional coordinate system.

Next, a first adjustment operation for detecting the first shooting angle θ _A , the second shooting angle θ _B and the inter-viewpoint distance d by the camera setting detection unit 106 will be described. During the first adjustment operation, the first viewpoint projection angle information 111A and the second viewpoint projection angle information 111B are held in the storage unit 102.

FIG. 5 is a diagram showing a configuration of the three-dimensional coordinate measurement system according to the present embodiment during the first adjustment operation.

As shown in FIG. 5, when the first adjusting operation, the camera 10A generates image data 61A (first reference image) by imaging the reference structure 40 from the first viewpoint M _1. Camera 10B, by imaging the reference structure 40 from the second viewpoint _{M 2,} and generates an image data 61B (second reference image).

The reference structure 40 has three reference points A, B, and C whose relative coordinates are known in a three-dimensional coordinate system. In other words, the triangle defined by the points A, B, and C is referred to as the reference structure 40.

FIG. 6 is a block diagram illustrating a configuration of the camera setting detection unit 106. As shown in FIG. 6, the camera setting detection unit 106 includes a third viewpoint projection angle extraction unit 121A, a fourth viewpoint projection angle extraction unit 121B, a first tilt angle calculation unit 122A, and a second tilt. An angle calculation unit 122B and a setting information calculation unit 123 are provided.

Using the first viewpoint projection angle information 111A, the third viewpoint projection angle extraction unit 121A uses three viewpoint projection angles (corresponding to each of the reference points A, B, and C projected on the first reference image). (Corresponding to the first reference viewpoint projection angle of the present invention).

The fourth viewpoint projection angle extraction unit 121B uses the second viewpoint projection angle information 111B to provide three viewpoint projection angles (corresponding to each of the reference points A, B, and C projected on the second reference image). (Corresponding to the second reference viewpoint projection angle of the present invention).

The first inclination angle calculation unit 122A calculates the first inclination angle using the three first reference viewpoint projection angles and the relative coordinates of the three reference points A, B, and C. Here, the first inclination angle is an angle formed by a first imaging surface that is an imaging surface of the first reference image and a reference plane including three reference points A, B, and C.

The second tilt angle calculation unit 122B calculates the second tilt angle using the three second reference viewpoint projection angles and the relative coordinates of the three reference points A, B, and C. Here, the second inclination angle is an angle formed by the second imaging surface, which is the imaging surface of the second reference image, and the reference plane including the three reference points A, B, and C.

The setting information calculation unit 123 calculates the first shooting angle θ _A and the second shooting angle θ _B using the first tilt angle and the second tilt angle. Also, setting information calculating unit 123 calculates a first inclined angle, with a second inclination angle, the first view point M ₁ and the second three-dimensional coordinates of the viewpoint M _2. Then, setting information calculation unit 123, from the first viewpoint M ₁ and second three-dimensional coordinates of the viewpoint M _2, calculates interview distance d.

FIG. 7 is a flowchart of the first adjustment operation by the three-dimensional coordinate measuring apparatus 90.

First, the image acquisition unit 101 acquires the first reference image and the second reference image as shown in FIG. 8 (S111).

In the present embodiment, for the sake of simplification of description, the case where the three reference points included in the first reference image and the second reference image are the same will be described as an example. The three reference points included in the image and the second reference image may be different. In other words, the first reference image, the first reference point is any three points of the three points or more reference points relative coordinates is known, is captured by the camera 10A from the first viewpoint M ₁ Any image can be used. The second reference image, the second reference point is any three points among the three points or more reference points may be an image captured by the second viewpoint M ₁ from the camera 10B . In this case, in each calculation process, by considering the relative positional relationship between the first reference point and the second reference point, the same as the case where the first reference point and the second reference point are the same. Processing can be performed.

Next, as illustrated in FIG. 9A, the third viewpoint projection angle extraction unit 121A, for example, uses the optical center position C _p1 in the first reference image as the origin and the reference point A projected on the first acquired image. , B and C, the coordinates of points A ₀₁ , B ₀₁ and C ₀₁ are extracted from the first reference image. Next, the third viewpoint projection angle extraction unit 121A uses the first viewpoint projection angle information 111A to calculate the first reference viewpoint projection angle corresponding to the coordinates of the extracted points A ₀₁ , B ₀₁ and C _01. Extract. Similarly, as illustrated in FIG. 9B, the fourth viewpoint projection angle extraction unit 121B, for example, uses the optical center position C _p2 in the second reference image as the origin, and the reference point A projected on the second reference image. , B and C, the coordinates of points A ₀₂ , B ₀₂ and C ₀₂ are extracted from the second reference image. Next, the fourth viewpoint projection angle extraction unit 121B uses the second viewpoint projection angle information 111B to calculate the second reference viewpoint projection angle corresponding to the coordinates of the extracted points A ₀₂ , B ₀₂ and C _02. Extract (S112).

Next, the first inclination angle calculation unit 122A is a first angle that is an angle formed between the first imaging surface that is the imaging surface of the first reference image and the reference plane that includes the reference points A, B, and C. The inclination angle is calculated. Similarly, the second inclination angle calculation unit 122B is a second angle that is an angle formed between the second imaging surface that is the imaging surface of the second reference image and the reference plane that includes the reference points A, B, and C. An inclination angle is calculated (S113).

Hereinafter, an example of processing for calculating the first and second inclination angles will be described in detail.

FIG. 10 is a flowchart of the process (S113) for calculating the first tilt angle and the second tilt angle. FIG. 11 is a diagram for explaining the processing shown in FIG. Hereinafter, the process of calculating the first tilt angle will be described with reference to FIGS. 10 and 11.

First, the first inclination angle calculation unit 122A sets one of the three reference points A, B, and C, for example, an X ₁ -Y ₁ -Z ₁ coordinate system with the point A as the origin. (S141). At this time, the first tilt angle calculating unit 122A has the X ₁ axis and Y ₁ axis is set in parallel to the first imaging plane taken from the first viewpoint M _1. In addition, the first inclination angle calculation unit 122A sets the Z ₁ axis to be perpendicular to the first imaging surface imaged from the first viewpoint M ₁ .

Next, the first inclination angle calculating section 122A sets a reference point B and C and equivalent virtual points _{B 1} and _{C 1} coordinates (S142). Here, the point B ₁ corresponds to the point B, and the point C ₁ corresponds to the point C. For example, in the present embodiment, the coordinates of the point B ₁ are set as (B _1x , 0, 0), and the coordinates of the point C ₁ are set as (0, C _1y , 0).

The first inclination angle calculation unit 122A finely and separately separates the points B ₁ and C ₁ into an angle α _1n around the X ₁ axis, an angle β _1n around the Y ₁ axis, and an angle γ _1n around the Z ₁ axis. A plurality of rotated conversion reference points, points B _1n ′ (B _1nx ′, B _1ny ′, B _1nz ′) and point C _1n ′ (C _1nx ′, C _1ny ′, C _1nz ′) are calculated ( S143). Here, n = 1, 2, 3,...

Next, the first inclination angle calculation unit 122A uses the viewpoint projection angle extracted by the third viewpoint projection angle extraction unit 121A to convert the point B _1n ′ and the point C _1n ′ into the X ₁ -Y ₁ plane. A plurality of transformation projection reference points projected onto the points B _1n ″ (B _1nx ″, B _1ny ″, 0) and point C _1n ″ (C _1nx ″, C _1ny ″, 0) are calculated (S144). . Here, n = 1, 2, 3,...

Here, the viewpoint projection angle of the plane Z ₁ -X _{1 at the} point B ₀₁ extracted by the third viewpoint projection angle extraction unit 121A is ε _1x , and the viewpoint projection angle of the Z ₁ -Y ₁ plane at the point B ₀₁ is epsilon _{1y, 1x} viewpoint projection angle of _Z 1 -X ₁ side of the point _{C 01} phi, the viewpoint projection angle of _Z 1 -Y ₁ side of the point _{C 01} and phi _1y. Reference point A in the first imaging plane, B, triangle formed by connecting the triangular _A 01 _B 01 _{C 01} connecting the _{A 01,} B _{01, C} 01 point that is C, point A, point _{B 1n} ", point _{C 1n"} the When AB _1n “C _1n ” is similar, ∠B _1n ′ M ₁ C _p1 and ∠B _1n ″ M ₁ C _p1 are ε _1x as shown in FIG.

Therefore, B _1nx ″ is expressed by (Equation 3).

Similarly, each of B _1ny ″, C _1nx ″, and C _1ny ″ is expressed by (Expression 4) to (Expression 6).

Finally, the first inclination angle calculation unit 122A determines the converted projection reference points B ₁ ″ and C ₁ ″ in which the triangle A ₀₁ B ₀₁ C ₀₁ and the triangle AB _1n “C _1n ” are most similar. Thus, the first inclination angle (α ₁ , β ₁ , γ ₁ ) is calculated (S145).

As a specific similarity calculation method, the line segment A ₀₁ B ₀₁ has the same length as the line segment AB _1n ″, and the point B ₀₁ and the point C ₀₁ are moved so that the point A ₀₁ coincides with the point A In this state, the first inclination angle (α ₁ , β ₁ , γ ₁ ) is set so that the sum of the length of the line segment B ₀₁ B _1n ″ and the length of the line segment C ₀₁ C _1n ″ is minimized. There is a method to calculate.

However, the similarity comparison calculation method is not limited to the above method. For example, the line segment _A 01 _{B 01,} "with the same length as the line segment _A 01 _{B 01} segment _{AB 1n"} segment _{AB 1n} moving the point _{C 01} to match with. In this state, the first inclination angles (α ₁ , β ₁ , γ ₁ ) may be calculated so that the shift amount between the point C ₀₁ and the point C _1n ″ is minimized.

Similarly, the second inclination angle calculation unit 122B calculates a second inclination angle (α ₂ , β ₂ , γ ₂ ) that is an angle formed by the second imaging surface and a plane including the reference point. .

In this way, the first tilt angle and the second tilt angle are calculated.

As described above, the calculation of the tilt angle has been described based on the embodiment. However, the calculation methods of the first and second tilt angles are not limited to this embodiment.

For example, in the present embodiment, the first inclination angle calculation unit 122A has X ₁ -Y ₁ -Z whose origin is one of three reference points A, B, and C whose coordinates are known. _{Although one} coordinate system has been set, the origin may be selected from other than the three reference points whose coordinates are known.

For example, the first inclination angle calculation unit 122A places the point A at a certain position on the viewpoint projection angle line of the point A. Then, the first inclination angle calculation unit 122A sequentially slides the point B on the viewpoint projection angle line of the point B at a specified pitch and within a certain range. At this time, the first inclination angle calculation unit 122A arranges the point C on the circumference with the line segment AB as an axis for each position of the point B. Then, the first inclination angle calculation unit 122A stores a condition in which the point C is closest to the viewpoint projection angle line of the point C among the positions on the circumference. Thus, the first inclination angle calculation unit 122A slides the point B sequentially, and calculates the closest point of the point C at that time. Then, the first inclination angle calculation unit 122A may calculate the inclination angle from the condition of the closest approach point among the closest points of the point C corresponding to all the slide positions of the point B.

Also, for example, the first inclination angle calculation unit 122A places the point A at a certain position on the viewpoint projection angle line of the point A and places the point B on or near the viewpoint projection angle line of the point B. At this time, the point C exists on the circumference with the line segment AB as an axis. Therefore, the first inclination angle calculation unit 122A stores a condition in which the point C is closest to the viewpoint projection angle line of the point C among the positions on the circumference. The first inclination angle calculation unit 122A moves the point A within a certain range and performs the above calculation for each moved position. Then, the first inclination angle calculation unit 122A may calculate the inclination angle from the condition of the closest approach point among the closest points of the point C corresponding to all positions of the point A.

Again, the process by the three-dimensional coordinate measuring apparatus 90 will be described with reference to FIG.

7, after the calculation of the first and second inclination angle (S113) is completed, the setting information calculation unit 123, a first aspect _{M 1} and second viewpoint _{M 2} coordinates (relative coordinates) Calculate (S114).

Specifically, the setting information calculation unit 123 calculates two vectors. Here, the two vectors, for example, the determined points _B 1 "(point _{B 1n")} in step S145, the straight line _{B 1} 'B connecting the' _{B 1} corresponding to 'point _{B 1n} at that time ₁ ", and, _C 1 point determined in step S145" is (point _{C 1n} ") and, linear _{C 1 'C 1} connecting the' _{C 1} corresponding to the _{'C 1n} terms of their time".

Then, the setting information calculation unit 123 calculates the three-dimensional coordinates with the first viewpoint M ₁ by calculating the coordinates of the closest point of the two vectors.

By the same method, setting information calculating unit 123 calculates a second three-dimensional coordinates of the viewpoint M _2.

Incidentally, using the following method may calculate the first three-dimensional coordinates of the viewpoint M _1.

Here, the triangle ABM ₁ connecting the the point _{M 1} points A and B, the length and ∠M ₁ AB and ∠M ₁ BA line segment AB is known. Therefore, the three-dimensional coordinates of the first viewpoint M ₁ can be calculated using the triangle ABM ₁ . Further, since １AM ₁ B is known from the viewpoint projection angle, similarly, the three-dimensional coordinates of the first viewpoint M ₁ can be calculated using the triangle ABM ₁ .

Then, setting information calculation unit 123, by using the three-dimensional coordinates of the first point of view _{M 1,} and a second three-dimensional coordinates of the viewpoint _{M 2,} and calculates the distance between viewpoints d (S115).

Further, the setting information calculation unit 123 calculates the first shooting angle θ _A and the second shooting angle θ _B (S116). Specifically, by the above process, the first viewpoint M _1, and second viewpoint M _2, reference point A, the three-dimensional coordinates of the B and C are calculated. Therefore, the setting information calculation unit 123 calculates, for example, the angles θ _A4 and θ _B4 illustrated in FIG. 13 from the coordinates of the reference point A, the first viewpoint M ₁ , and the second viewpoint M ₂ . Furthermore, the setting information calculation unit 123 calculates the first shooting angle θ _A using the following (formula 7). In addition, the setting information calculation unit 123 calculates the second shooting angle θ _B using the following (Equation 8).

θ _A = θ _L −θ _A3 −θ _A4 (Formula 7)
θ _B = θ _R −θ _B3 −θ _B4 (Formula 8)

Here, θ _L is an angle formed by the first reference line L and the line segment M ₁ M ₂ . θ _R is an angle formed by the second reference line R and the line segment M ₁ M ₂ . Further, here, the theta _L and theta _R, are both 90 degrees. Θ _A3 is the viewpoint projection angle of the reference point A in the first reference image, and θ _B3 is the viewpoint projection angle of the reference point A in the second reference image.

As described above, the setting information calculation unit 123 can calculate the first shooting angle θ _A and the second shooting angle θ _B.

Note that the calculation method of the first shooting angle θ _A and the second shooting angle θ _B by the setting information calculation unit 123 is not limited to the above method. For example, since the optical axis O ₁ is perpendicular to the first imaging surface, the direction of the optical axis O ₁ is calculated from the first tilt angle, and the first photographing angle θ _A is calculated from the direction of the optical axis O _1. May be. The same applies to the second shooting angle θ _B.

Then, the first shooting angle θ _A , the second shooting angle θ _B and the inter-viewpoint distance d calculated by the setting information calculation unit 123 are stored in the storage unit 102.

As described above, in the present embodiment, the first shooting angle θ _A , the second shooting angle θ _B, and the inter-viewpoint distance d can be easily calculated using the reference structure 40. Thus, even when the arrangement of the cameras 10A and 10B is changed at the shooting site, the first shooting angle θ _A , the second shooting angle θ _B and the inter-viewpoint distance d can be easily calculated. For example, readjustment can be easily performed even when the arrangement of the cameras 10A and 10B is adjusted according to the object or the measurement environment, and even when the arrangement of the cameras 10A and 10B is shifted due to vibration or the like.

Next, a second adjustment operation for detecting the first viewpoint projection angle information 111A and the second viewpoint projection angle information 111B by the camera setting detection unit 106 will be described. In the second adjustment operation, the relative positions of the two cameras 10A and 10B need not be fixed, and the second adjustment operation can be performed individually. In addition, since the process of detecting the first viewpoint projection angle information 111A and the process of detecting the second viewpoint projection angle information 111B are the same, only the process of detecting the first viewpoint projection angle information 111A will be described below. explain.

FIG. 14 is a diagram illustrating a configuration of a system including the three-dimensional coordinate measuring apparatus 90 during the second adjustment operation.

The camera 10A images the calibration plate 70 from the viewpoint M. Here, the viewpoint M corresponds to the position of the principal point of the photographing lens of the camera 10A. In FIG. 14, O written by a two-dot chain line indicates the optical axis of the photographing lens (the optical axis of the camera).

The calibration plate 70 is a transparent plate having high rigidity and has a grid pattern on the surface. A weight 72 is attached to the calibration plate 70 with a thread 71.

The camera setting detection unit 106 performs image pixel distortion correction using the image data 62 </ b> A obtained by capturing the calibration plate 70, the thread 71, and the weight 72 from the viewpoint M. Further, the camera setting detection unit 106 calculates a viewpoint projection angle (corresponding to the first and second pixel viewpoint projection angles of the present invention) of an arbitrary image pixel (pixel).

FIG. 15 is a flowchart of the second adjustment operation.

In the second adjustment operation, the camera setting detection unit 106 sets the optical center position C _p that is the center point of the imaging surface (S201), performs distortion correction (S202), and views the viewpoint of an arbitrary image pixel (pixel). The projection angle θ is calculated (S203).

Here, the viewpoint projection angle θ of an arbitrary image pixel is an arbitrary point P ₃ on the three-dimensional space projected on the two-dimensional coordinates of the arbitrary image pixel P on the image and the viewpoint as shown in FIG. This is the angle between the line segment connecting M and the optical axis O of the camera. In FIG. 16, the optical axis O of the camera and the imaging surface are orthogonal. Therefore, a viewpoint M, and the distance x between the optical center position C _p is the intersection of the optical axis O and the imaging surface of the camera, for example, the origin of the optical center position C _p, the two-dimensional coordinates of an arbitrary image pixels P Can be used to calculate the viewpoint projection angle θ of any image pixel P.

First, the optical center position _Cp is set (S201). Here, the calibration plate 70 is assumed to be kept horizontal by a level or the like. Next, the calibration plate 70 is imaged by the camera 10A from above. Then, the image pickup surface is (1) a grid around the attachment position of the thread 71 such that (1) the attachment position of the calibration plate 70 and the thread 71 and the attachment position of the weight 72 and the thread 71 overlap. Adjust so that the eye display is symmetrical. At this time, the mounting position of the thread 71 on the calibration plate 70 is the optical center position C _p.

Next, the camera setting detection unit 106 calculates parameters for performing image distortion correction (S202). As shown in FIG. 17A, before distortion correction, the image when the calibration plate 70 is photographed is an image in which the grid's eyes are distorted due to the characteristics of the photographing lens of the camera 10A. The camera setting detection unit 106 performs normalization processing on the captured image to calculate parameters for adjusting the captured image to an image as illustrated in FIG. 17B. In the subsequent processing including the above-described measurement processing of the three-dimensional coordinates of the target point W and the first adjustment operation, for example, in the image acquisition unit 101, for the image captured by the camera 10A using the parameter. Then, distortion correction is performed.

Finally, the camera setting detection unit 106 calculates the viewpoint projection angle θ of an arbitrary image pixel P (S203).

FIG. 18 is a diagram for explaining processing for calculating the distance between the calibration plate and the viewpoint.

First, the camera setting detection unit 106 acquires an image of a plate before movement, which is an image after distortion correction obtained by photographing the calibration plate 70 as shown in FIG. 17B. Next, the calibration plate 70 is moved closer to the viewpoint M by the distance y while keeping the calibration plate 70 horizontal. In this state, the camera setting detection unit 106 acquires an image of the plate after movement, which is an image after distortion correction obtained by imaging the calibration plate 70. Then, the camera setting detection unit 106 uses the image of the plate before movement and the image of the plate after movement, for example, _two points P ₁ ′ and P ₂ ′ with the optical center position C _p as the origin. Extract dimension coordinates.

Here, a line connecting the viewpoint M and the optical center position C _p, the intersection of the surface of the calibration plate 70 and C _1. Further, the arbitrary point of the calibration plate 70 surface and P _1. In addition, an intersection of a straight line connecting the viewpoint M and P ₁ and the imaging surface is defined as P ₁ ′. Further, corresponding to P ₁ of the calibration plate 70 surface before the movement, the point of the calibration plate 70 surface after the movement and P _2. Also, let P ₂ ′ be the intersection of the straight line connecting the viewpoints M and P ₂ and the imaging surface. Further, let Q be the intersection of a straight line passing through the point P ₂ and the point P ₂ ′ and the surface of the calibration plate 70 before the movement.

At this time, the length d ₁ of the line segment P ₁ Q is expressed by the following (formula 9).

Here, d ₂ represents the length of the line segment C ₁ P ₁ , d ₃ represents the length of the line segment P ₁ ′ P ₂ ′, and d ₄ represents the length of the line segment C _p P ₁ ′.

Further, the viewpoint projection angle θ ₁ of the point Q is expressed by the following (formula 10).

Therefore, the distance z between the calibration plate 70 and the viewpoint M is expressed by the following (formula 11).

Here, d ₅ indicates the length of the line segment C ₁ Q.

Then, using the distance z between the calibration plate 70 and the viewpoint M, for example, the viewpoint projection angle θ ₂ of P ₁ ′ shown in FIG. 18 is expressed by (Equation 12).

Viewpoint M and the length x of a line connecting the optical center position C _p is expressed by (Equation 13).

Therefore, the viewpoint projection angle θ of any pixel point P can be calculated using (Equation 14).

Here, d ₆ indicates the length of the line segment C _p P.

That is, since the two-dimensional coordinates of an arbitrary image pixel P with the optical center position C _p as the origin are known, the viewpoint projection angle θ of the arbitrary image pixel P can be calculated.

In the above description, the calibration plate 70 is marked with a grid, but the calibration plate may be marked with a known interval.

In the above description, an example in which a thread and a weight are used at the time of calibration has been described. Instead of using a thread and a weight, a light beam or laser light from a lamp light source that is irradiated perpendicularly to the calibration plate is used. Also good.

As described above, the camera setting detection unit 106 calculates the viewpoint projection angle for each pixel in the image captured by the camera 10A, and stores the first viewpoint projection angle information 111A indicating the viewpoint projection angle for each pixel. Store in the unit 102. Similarly, the camera setting detection unit 106 calculates the viewpoint projection angle for each pixel in the image captured by the camera 10B, and stores the second viewpoint projection angle information 111B indicating the viewpoint projection angle for each pixel. To remember.

The three-dimensional coordinate measuring apparatus according to the embodiment of the present invention has been described above, but the present invention is not limited to this embodiment.

For example, in the above description, the first viewpoint projection angle information 111A is a table indicating viewpoint projection angles corresponding to each of a plurality of pixels, but is an expression indicating a correspondence relationship between pixels and viewpoint projection angles. Also good. For example, the said formula is said (Formula 14). In this case, the first viewpoint projection angle extraction unit 103A may calculate the viewpoint projection angle of the target point W in the first target image using (Equation 14). In this case, the camera setting detection unit 106 needs to obtain the distance x between the viewpoint M and the optical center position C _p using the above (Equation 13). Similarly, the second viewpoint projection angle information 111B may be an expression indicating a correspondence relationship between a pixel and a viewpoint projection angle.

In the above description, an example in which shooting is performed from two viewpoints has been described. However, shooting may be performed from three or more viewpoints.

Also, the number of reference points whose relative coordinates are known may be four or more. In this case, it is only necessary that at least three of the four or more reference points are photographed in each of the first and second reference images. Further, the combination of at least three reference points captured in each of the first reference image and the second reference image may be different. Furthermore, it is preferable that four or more reference points are arranged in a three-dimensional manner rather than in the same plane. For example, four or more of the apexes of a triangular pyramid or a quadrangular pyramid can be used as the reference point. Thereby, a wide range and stable angle detection can be performed. For example, when only the above-described three reference points are used, an error is likely to occur if there is no angle between the plane including the three reference points and the viewpoint. On the other hand, for example, by photographing different surfaces of the triangular pyramid from two viewpoints, angle detection can be performed stably even in such a case.

In this case, the first and second inclination angles are angles with respect to different reference planes (first and second reference planes). The first and second inclination angles may be adjusted to one of the reference planes. Alternatively, the first and second tilt angles may be calculated as angles unified in the three-dimensional coordinate system.

Further, the first tilt angle and the second tilt angle may be calculated by performing nonlinear approximation calculation using a large number of reference points whose relative coordinates are known.

Further, for example, among the three reference points A, B, and C whose relative coordinates are known and the points A ₀₁ , B ₀₁ , and C ₀₁ that are the reference points A, B, and C on the first imaging surface, When the point A and the point A ₀₁ , the point B and the point B ₀₁ , and the point C and the point C ₀₁ are not associated with each other, the correspondence can be identified by using a special marker or line. .

In addition, a reference point for verification whose relative coordinates are known may be used in order to prevent reversal (turn over) due to rotation of the reference point and calculation of an incorrect solution (tilt angle).

Also, restrictions (for example, rotation conditions) may be added so as not to calculate an incorrect solution.

Further, the above description, connecting during the three-dimensional coordinate measurement, the baseline of the imaging surface of the camera 10A and 10B (horizontal pixel direction of the image), the first viewpoint M ₁ and the second viewpoint M ₂ segments Description is made on the assumption that M _{1 and} M ₂ are parallel. In other words, it is assumed that the camera 10A and the camera 10B are not rotated about the optical axis. When at least one of the camera 10A and the camera 10B is set to rotate around the optical axis, the following processing may be performed.

For example, the storage unit 102 holds in advance a first rotation angle around the optical axis of the camera 10A and a second rotation angle around the optical axis of the camera 10B. And at the time of three-dimensional coordinate measurement, the image acquisition part 101 performs correction | amendment which rotates the 1st target image acquired from camera 10A according to a 1st rotation angle. Similarly, the image acquisition unit 101 performs correction to rotate the second target image acquired from the camera 10B according to the second rotation angle. Then, the three-dimensional coordinate measuring apparatus 90 calculates the three-dimensional coordinates of the target point W by performing the above-described processing on the corrected first target image and second target image.

Note that the target to be corrected according to the first rotation angle and the second rotation angle is not limited to the first and second target images. For example, the coordinate calculation unit 105 may calculate the three-dimensional coordinates of the target point in consideration of the first and second rotation angles.

Further, the first and second rotation angles can be detected by the following method. For example, the camera setting detection unit 106 calculates the first and second reference viewpoint projection angles of the reference points A, B, and C when the line segment M ₁ M ₂ and the baseline are parallel by the method described above. Keep it. Further, the camera setting detection unit 106 projects the first and second reference viewpoints of the reference points A, B, and C in the actual setting state (when the camera 10A and the camera 10B are rotated around the optical axis). Calculate the corner. Then, the camera setting detection unit 106 calculates the first rotation angle based on the calculated angle difference between the two first reference viewpoint projection angles. Similarly, the camera setting detection unit 106 calculates a second rotation angle based on the calculated angle difference between the two second reference viewpoint projection angles.

Also, the angle information such as the viewpoint projection angle used above is not limited to a value indicating the angle itself, but may be a vector represented by coordinates of two points, for example.

Further, part or all of the functions of the three-dimensional coordinate measuring apparatus according to the embodiment of the present invention may be realized by a processor such as a CPU executing a program.

Furthermore, the connection relationship between the constituent elements is exemplified for specifically explaining the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks. May be. Further, the functions of a plurality of functional blocks having similar functions may be processed by the three-dimensional coordinate measuring apparatus in parallel or in time division.

Furthermore, the present invention may be the above program or a non-transitory computer-readable recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

In addition, the configuration of the three-dimensional coordinate measuring apparatus is for illustration in order to specifically describe the present invention, and the three-dimensional coordinate measuring apparatus according to the present invention is not necessarily provided with all of the above configurations. Absent. In other words, the three-dimensional coordinate measuring apparatus according to the present invention only needs to have a minimum configuration capable of realizing the effects of the present invention.

For example, in the above description, the three-dimensional coordinate measuring apparatus 90 may not include the camera setting detection unit 106. Alternatively, the camera setting detection unit 106 may have only a function of performing one of the first adjustment operation and the second adjustment operation.

Further, the order in which the above steps are executed is for illustration in order to specifically describe the present invention, and may be in an order other than the above. Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

Furthermore, various modifications in which the present embodiment is modified within the scope conceived by those skilled in the art are also included in the present invention without departing from the gist of the present invention.

The present invention can be applied to a three-dimensional coordinate measuring apparatus.

10A, 10B Camera 30A, 30B Cable 40 Reference structure 50 Target structure 60A, 60B, 61A, 61B, 62A Image data 70 Calibration plate 71 Thread 72 Weight 90 Three-dimensional coordinate measuring device 101 Image acquisition unit 102 Storage unit 103A First 1 viewpoint projection angle extraction unit 103B second viewpoint projection angle extraction unit 105 coordinate calculation unit 106 camera setting detection unit 111A first viewpoint projection angle information 111B second viewpoint projection angle information 112 three-dimensional coordinates 121A third viewpoint Projection angle extraction unit 121B Fourth viewpoint projection angle extraction unit 122A First tilt angle calculation unit 122B Second tilt angle calculation unit 123 Setting information calculation unit

Claims (8)

1. A three-dimensional coordinate measuring apparatus for measuring the three-dimensional coordinates of a target point,
   A first target image in which the target point is imaged by a first imaging device from a first viewpoint, and a second target image in which the target point is imaged by a second imaging device from a second viewpoint. An image acquisition unit to acquire;
   For each pixel in the image captured by the first imaging device, a line segment connecting each point of the three-dimensional coordinate system projected onto the pixel and the first viewpoint, and the light of the first imaging device Information on a first pixel viewpoint projection angle that is an angle formed with an axis, and each point of the three-dimensional coordinate system projected onto the pixel for each pixel in an image captured by the second imaging device, Information about the second pixel viewpoint projection angle, which is an angle formed by a line segment connecting the second viewpoint and the optical axis of the second imaging device, and the optical axis direction of the first imaging device. 1 optical axis information, second optical axis information indicating the optical axis direction of the second imaging device, and inter-viewpoint information indicating a relative position between the first viewpoint and the second viewpoint are stored. A storage unit to
   Using the information on the first pixel viewpoint projection angle, a first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to the target point projected on the first target image is acquired. A first viewpoint projection angle extraction unit;
   Using the information on the second pixel viewpoint projection angle, a second target viewpoint projection angle that is the second pixel viewpoint projection angle corresponding to the target point projected on the second target image is acquired. A second viewpoint projection angle extraction unit;
   A coordinate calculation unit that calculates three-dimensional coordinates of the target point using the first and second target viewpoint projection angles, the optical axis directions of the first and second imaging devices, and the inter-viewpoint information. A three-dimensional coordinate measuring device.
2. The storage unit further includes:
   Information indicating a first rotation angle, which is an angle at which the first imaging device rotates around the optical axis of the first imaging device;
   The second imaging device holds information indicating a second rotation angle that is an angle rotating around the optical axis of the second imaging device;
   The coordinate calculation unit includes the first and second target viewpoint projection angles, the optical axis directions of the first and second imaging devices, the inter-viewpoint information, and the first and second rotation angles. The three-dimensional coordinate measurement apparatus according to claim 1, wherein the three-dimensional coordinates of the target point are calculated using a.
3. The coordinate calculation unit
   A line segment connecting the first viewpoint and the second viewpoint using the first target viewpoint projection angle, the optical axis direction of the first imaging device, and the first rotation angle; Calculating a first target angle that is an angle formed by a line segment connecting the first viewpoint and the target point;
   A line segment connecting the first viewpoint and the second viewpoint using the second target viewpoint projection angle, the optical axis direction of the second imaging device, and the second rotation angle; Calculating a second target angle that is an angle formed by a line segment connecting the second viewpoint and the target point;
   The three-dimensional coordinate measurement apparatus according to claim 2, wherein the three-dimensional coordinates of the target point are calculated using the inter-viewpoint information, the first target angle, and the second target angle.
4. The first optical axis information includes a first reference line that passes through the first viewpoint and is perpendicular to a line segment that connects the first viewpoint and the second viewpoint; and Information indicating a first imaging angle that is an angle formed with the optical axis of the imaging device;
   The second optical axis information passes through the second viewpoint, is perpendicular to a line segment connecting the first viewpoint and the second viewpoint, and is parallel to the first reference line. Information indicating a second imaging angle that is an angle formed between the second reference line and the optical axis of the second imaging device,
   The coordinate calculation unit
   An angle formed by a line segment connecting the first viewpoint and the second viewpoint and the first reference line, the first viewpoint projection angle, the first photographing angle, and the first viewpoint Calculating the first target angle using a rotation angle;
   An angle formed by a line segment connecting the first viewpoint and the second viewpoint and the second reference line, the second viewpoint projection angle, the second photographing angle, and the second viewpoint. The three-dimensional coordinate measurement apparatus according to claim 3, wherein the second target angle is calculated using a rotation angle.
5. The image acquisition unit further includes:
   A first reference image that is any one of three or more reference points whose relative coordinates are known is a first reference image captured by the first imaging device from the first viewpoint. A second reference point, which is any three of the three or more reference points, obtains a second reference image captured by the second imaging device from the second viewpoint,
   The three-dimensional coordinate measuring device further includes:
   Using the information of the first pixel viewpoint projection angle, the three first projection points that are the first pixel viewpoint projection angles corresponding to the three first reference points projected on the first reference image. A third viewpoint projection angle extraction unit for acquiring the reference viewpoint projection angle of
   Using the information on the second pixel viewpoint projection angle, three second projections that are the second pixel viewpoint projection angles corresponding to the three second reference points projected on the second reference image. A fourth viewpoint projection angle extraction unit that acquires the reference viewpoint projection angle of
   Using the three first reference viewpoint projection angles and the relative coordinates of the three first reference points, the first imaging surface, which is the imaging surface of the first reference image, and the three reference points A first inclination angle calculation unit for calculating a first inclination angle formed with a first reference plane including a point;
   Using the three second reference viewpoint projection angles and the relative coordinates of the three second reference points, the second imaging surface, which is the imaging surface of the second reference image, and the three points A second inclination angle calculation unit for calculating a second inclination angle formed with a second reference plane including two reference points;
   Using the relative positional relationship between the three first reference points and the three second reference points, the first inclination angle, and the second inclination angle, the first imaging is performed. The setting information calculation unit that calculates the optical axis direction of the apparatus, the optical axis direction of the second imaging apparatus, the first rotation angle, and the second rotation angle is provided. The three-dimensional coordinate measuring apparatus according to any one of the above.
6. The setting information calculation unit
   The first viewpoint using the relative positional relationship between the first reference point of the three points and the second reference point of the three points, the first inclination angle, and the second inclination angle. And calculating the three-dimensional coordinates of the second viewpoint,
   The three-dimensional coordinate measuring apparatus according to claim 5, wherein the inter-viewpoint information is calculated from the three-dimensional coordinates of the first viewpoint and the second viewpoint.
7. A three-dimensional coordinate measurement method for measuring a three-dimensional coordinate of a target point,
   A first target image in which the target point is imaged by a first imaging device from a first viewpoint, and a second target image in which the target point is imaged by a second imaging device from a second viewpoint. A step to obtain,
   For each pixel in the image captured by the first imaging device, a line segment connecting each point of the three-dimensional coordinate system projected onto the pixel and the first viewpoint, and the light of the first imaging device Information on a first pixel viewpoint projection angle that is an angle formed with an axis, and each point of the three-dimensional coordinate system projected onto the pixel for each pixel in an image captured by the second imaging device, Information about the second pixel viewpoint projection angle, which is an angle formed by a line segment connecting the second viewpoint and the optical axis of the second imaging device, and the optical axis direction of the first imaging device. 1 optical axis information, second optical axis information indicating the optical axis direction of the second imaging device, and inter-viewpoint information indicating a relative position between the first viewpoint and the second viewpoint are stored. And steps to
   Using the information on the first pixel viewpoint projection angle, a first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to the target point projected on the first target image is acquired. Steps,
   Using the information on the second pixel viewpoint projection angle, a second target viewpoint projection angle that is the second pixel viewpoint projection angle corresponding to the target point projected on the second target image is acquired. Steps,
   Calculating the three-dimensional coordinates of the target point using the first and second target viewpoint projection angles, the optical axis directions of the first and second imaging devices, and the inter-viewpoint information; Includes 3D coordinate measurement method.
8. A program for causing a computer to execute the three-dimensional coordinate measurement method according to claim 7.

Images