JP2002109518A - Three-dimensional shape restoring method and system therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional shape restoring method and a system therefor capable of restoring a correct three-dimensional shape by preventing degradation of estimation accuracy based on mistaken correspondence, etc., between characteristic data. SOLUTION: A plurality of images of an object picked up by a camera are read (S1). Whether or not the camera used in picking up the images is calibrated is determined (S2), and if it is calibrated, the camera calibration parameters are read (S3). Unless it is calibrated, characteristics are extracted (S4), the images are corresponded to each other (S5) and the camera calibration parameters are calculated (S6), successively to calculate the camera calibration parameters. Next, post-calibration image data excluding the lens distortion is prepared from the input image and the camera calibration parameters (S7). The characteristics on the images are extracted by using the post-calibration image data having no distortion (S8), and the images are corresponded to each other (S9). The three-dimensional shape is restored based on the camera calibration parameters and the correspondence between images by giving the length of a part in the image (S10).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional shape restoration system and method for measuring a three-dimensional shape of an object to be measured from a plurality of still images or moving images.

[0002]

2. Description of the Related Art Image measurement is the measurement of dimensions, shapes and displacements of large structures such as buildings and plants, non-contact measurement of industrial parts, and model data for creating virtual images using computer graphics. There are a wide variety of application fields, such as the input of "". In recent years, various systems for restoring an object shape from a plurality of images or moving images have been proposed. For example, in Japanese Patent Application Laid-Open No. 9-212463, a feature pattern having three-dimensional position data obtained by stereo measurement in which features such as contours, irregularities, and patterns of a target object obtained from an input image are approximated by straight lines, arcs, and the like, A three-dimensional object recognition method for recognizing a three-dimensional shape of a target object by matching with feature data of a model is disclosed. In this method, since the data of the model to be measured must be prepared in advance, the measurement target is specified in advance as a known one.

A three-dimensional measurement method disclosed in Japanese Patent Application Laid-Open No. 10-228542 estimates a three-dimensional shape of a measurement target by performing feature point matching processing using contour features of images photographed from a plurality of parts. I do. This method requires the use of at least three cameras having optical axes parallel to each other and the relative positions of the cameras being arranged side by side in a horizontal direction and a vertical direction, and an apparatus for accurately positioning the cameras. Is required. Further, in the three-dimensional measuring device disclosed in JP-A-10-318715 and JP-A-3181872,
A plurality of image data are associated with each other based on a characteristic pattern image whose positional relationship is known in advance. Therefore, a means for generating a characteristic pattern is required.

On the other hand, in this type of image measurement, a lens distortion of a camera as an image pickup means affects measurement accuracy. Japanese Patent Application Laid-Open No. 11-230745 describes correction of lens distortion.
However, the correction coefficient for distortion correction must be obtained in advance by a lens manufacturer or separately measured. On the other hand, by specifying a distance between two points on an image from a plurality of image data obtained by imaging from an arbitrary position using a commercially available camera or video camera that has not been calibrated, Various systems that can restore a three-dimensional shape have been proposed (QT Luong an
d T. Vieville, Canonic Representation for the Geomet
ries of Multiple Projective Views, Technical Repor
t UCB / CSD 93/772, EECS, UCBerkeley, 1993 and others).

[0005]

In the above-described conventional three-dimensional shape restoration system, corresponding feature data among a plurality of image data is associated as corresponding data, and the three-dimensional shape is restored based on this association. It is based on The feature data is associated manually or automatically by the system based on the similarity between the feature data. However, such a conventional system has a problem that the accuracy of extracting three-dimensional shapes is reduced due to the extraction accuracy of feature data and the occurrence of incorrect association. For example, JP-A-2000-
No. 74641 discloses a technique for extracting a three-dimensional shape of an imaged scene from an image having no information on an imaging device at the time of imaging, but mentions nothing about improving the accuracy of associating feature points. Absent.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and prevents a decrease in estimation accuracy based on an erroneous correspondence between feature data and the like, and enables a three-dimensional shape restoration capable of accurate three-dimensional shape restoration. It is intended to provide a method and a system.

[0007]

A first three-dimensional shape restoring method according to the present invention includes a step of reading a plurality of image data obtained by capturing an image of an object to be measured from different angles; Extracting the feature data of the image from each of the image data, calculating a constraint condition between the image data based on the feature data extracted in this step, and associating the feature data so as to satisfy the constraint condition. Performing, and providing the data of the length of one location in the image data, and the three-dimensional space in which the measurement target is arranged from the associated feature data and the constraint between the image data. A perspective projection transformation matrix between each image data and the image plane is calculated, and the measurement pair is calculated from the perspective projection transformation matrix and the given length data. Characterized by comprising the step of restoring the three-dimensional shape of the object.

Further, the three-dimensional shape restoration system according to the present invention is characterized in that one or a plurality of image pickup means for picking up an image of an object to be measured from different angles, and an image storage means for reading and storing a plurality of image data from the image pickup means. Input means for providing data of one length in the image data, a function of extracting characteristic data of an image from each of a plurality of image data stored in the image storage means, and the extracted characteristic A function of calculating a constraint condition between image data based on data, and associating the feature data so as to satisfy the constraint condition; Calculate a perspective projection transformation matrix between the three-dimensional space where the object is arranged and the image plane of each of the image data, and calculate the perspective projection transformation matrix and the given one Characterized by comprising a from the length of the data and processing means having a function to restore the three-dimensional shape of the measurement object.

According to the present invention, a constraint condition between image data is calculated based on feature data extracted from a plurality of image data taken from different angles, and the correspondence of the feature data satisfying the constraint condition is calculated. Make the attachment. The constraint condition is obtained by repeatedly calculating, for example, until a stable result is obtained. As a result, it is possible to prevent erroneous correspondence and association between feature data with low accuracy, and improve the accuracy of association. Then, data of the length of one place in the image data is given, and from the associated feature data and the constraint condition between the image data, the three-dimensional space where the measurement object is arranged and the By calculating the perspective projection transformation matrix between the image plane and the perspective projection transformation matrix and the given one-piece length data, the three-dimensional shape of the measurement object is restored with high accuracy. The original shape can be restored.

According to a preferred embodiment of the present invention, the method further comprises the step of performing distortion correction on the plurality of read image data based on a camera calibration parameter. The feature data of the image is extracted from each. If the camera calibration parameters are unknown, the method may further include calculating the camera calibration parameters from the plurality of read image data. The steps of calculating the camera calibration parameters include, for example, extracting image feature data from the read plurality of image data, and based on feature data near the center of the image data among the feature data extracted in this step. Calculating a constraint condition between image data, and associating the feature data so as to satisfy the constraint condition; calculating a camera internal parameter from the constraint condition; Calculating a perspective projection transformation matrix for each image data from the image data, and calculating a lens distortion correction coefficient from the perspective projection transformation matrix and associated feature data including a portion other than the vicinity of the optical center. I just need.

The feature data extracted from the image data is preferably a feature point indicating an intersection or a corner of an edge where the density of the image data changes rapidly so as not to extract an erroneous feature point. In addition to the data, feature line segment data indicating edges where the density of the image data changes rapidly is used.

According to a further preferred embodiment of the present invention,
The step of associating the feature data includes, after performing initial association between the plurality of pieces of image data with respect to the extracted feature data, calculating a constraint condition between the image data, Re-associating the feature data so as to satisfy, evaluate the accuracy of the correspondence between images from the position of the corresponding data obtained from the constraint condition and the position of the feature data associated by the constraint condition,
This is a step of repeating the calculation of the constraint condition and the association of the feature data based on the constraint condition until a predetermined accuracy is obtained.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a three-dimensional shape restoration system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a three-dimensional shape restoration system according to one embodiment of the present invention. Measurement object 1
Are imaged by the camera 2 from different angles.
Obtain a plurality of still images captured most typically 2 multiple cameras in _1, 2 _2, ... the measured object 1 from a different direction by. However by imaging from different angles of the measuring object 1 by moving 2 ₁ of one camera to a plurality of positions it may be to obtain a plurality of still images, when the measuring object 1 itself moves is 1 A video image of the measurement object 1 may be obtained by _one video camera 21. In any case,
It is only necessary to obtain a plurality of images of the measurement object 1 from different angles, and various forms of photographing can be taken. The plurality of still images or moving images obtained in this manner (hereinafter, these are collectively referred to as “plural images”) are
It is stored in the image memory 3.

The arithmetic processing unit 4 reads a plurality of images from the image memory 3, generates a calibrated image from which distortion of the camera 2 has been removed based on the camera calibration parameters, and extracts feature points from the calibrated image. , And a process of calculating a three-dimensional shape is performed. The work memory 5 is used to temporarily store intermediate data such as feature points, line segments, corresponding points, and corresponding line segments obtained during these processes. The external storage device 6 stores a camera calibration parameter file, a measurement result file, and the like. The input device 7 is used for inputting necessary data such as the length of one portion in an image in the calculation processing of the three-dimensional shape. The display device 8 displays a read image, extracted feature points, associated feature points, and the like for inputting necessary data. The output device 9 is used to print out a measurement result or the like.

Next, a three-dimensional shape restoring method using the above system will be described. Now, as shown in FIG. 2, the position of a point on the measurement target 1 in a three-dimensional space is

[0016]

(Equation 1)

[0017] and then, the imaging position of the measurement object 1 in the camera 2 _first image plane S,

[0018]

(Equation 2)

Then, the relationship between the two coordinates by the pinhole camera model can be expressed by the following equation using homogeneous coordinates.

[0020]

(Equation 3)

Where k is a non-zero scale factor, P
Is a perspective projection changes a 3 × 4 matrix representing the camera 2 _1. Here, the posture of the camera 2 in the three-dimensional space is R, the position is t, and the position on the image plane at which the optical axis C crosses the image plane S is

[0022]

(Equation 4)

Assuming that the angle formed by the two axes u and ν on the image plane S in space is θ, the scale in the two-axis direction is k _u , k _v , and the focal length is f, the perspective transformation matrix P is expressed as follows. I can do it.

[0024]

(Equation 5)

Here, f, _ku , representing the perspective transformation matrix P,
k _v , θ, u ₀ , and v ₀ are referred to as camera internal parameters.

(1) Constraint condition obtained from two images (epipolar constraint) Now, let the i-th row of the perspective transformation matrix P of the _first camera 21 be a ⁱ and k be a non-zero scale factor, FIG.
As shown in, x point on the space by the first camera 2 ₁ on the image plane S _1, and is reflected in the following manner.

[0027]

(Equation 6)

[0028] Similarly, the i-th row of the second camera 2 ₂ perspective transformation matrix P and b ^i, the k 'as scale factor non-0, x point on the space by the second camera 2 ₂ image on the surface S _2, and is reflected in the following manner.

[0029]

(Equation 7)

To summarize these two equations,

[0031]

(Equation 8)

In order to have a non-trivial solution, the determinant of the 6 × 6 matrix on the left side must be 0.
When this matrix is calculated and the pixel positions are summarized, the following equation is obtained.

[0033]

(Equation 9)

In this equation, the matrix F is defined as a fundamental matr
Called ix.

[0035] (2) constraints determined between three images (trilinear constraint) the i perspective transformation matrix P of the first and second camera 2 _1, 2 ₂ in addition third camera 2 ₃ above the row and c ^i, k "
As the non-zero scale factor, the third camera 2 ₃
Suppose that a point x in space is reflected on the image plane S as follows.

[0036]

(Equation 10)

Summarizing the three equations,

[0038]

[Equation 11]

In order for this to have a non-trivial solution, the rank of the 6 × 7 matrix on the left side must be at most 6. Therefore, the determinant of all 7 × 7 small matrices is 0
Becomes Here, there are a total of 36 ways to take a small matrix, the details of which are as follows.

Three rows are taken from the perspective projection transformation matrix P of the two cameras, and 1 row is taken from the perspective projection transformation matrix P of the remaining cameras.
Go (9 ways). Take three rows from the perspective projection transformation matrix P of one camera,
Two rows are taken from the perspective projection transformation matrix P of the remaining cameras (27 patterns).

In this case, the constraint condition is determined between the two images. The constraint condition newly obtained from the three images is the determinant of the small matrix according to trilinear tens.
Call it or. Now, the 7 × 7 small matrix is

[0042]

(Equation 12)

Let us assume that When this is expanded, it becomes as follows.

[0044]

(Equation 13)

Where ε _ijk is (i, j, k) is (1,
+1 when given by even permutation of 2,3), -1 when given by odd permutation, 0 otherwise, subscript x, y
Represents a line b ^x, c ^y of the camera perspective projection transformation matrix P unincorporated when configuring the submatrix. this,

[0046]

[Equation 14]

Then, the trilinear constraint can be expressed as follows.

[0048]

(Equation 15)

If it is attempted to determine the constraint conditions in the same manner, as will be easily understood, new constraint conditions can no longer be obtained from five or more images. In addition, a constraint condition obtained from four images can be expressed by a constraint condition obtained from two and three images.

The accuracy of the restoration of the three-dimensional shape is determined by a fundamental ma- terial that gives a constraint that satisfies the imaging position on the image plane S of the feature included in the measurement object 1 arranged in the three-dimensional space.
It depends on how accurately the trix or trilinear tensor is estimated. Factors that affect accuracy include:
The extraction position accuracy of the feature, the erroneous correspondence between the features, the stability of the solution, the influence of the distortion of the camera lens, and the like are given. Therefore, the following processing is executed in consideration of these.

FIG. 4 is a main flowchart of the three-dimensional shape restoration processing. First, the measurement object 1 is moved to the camera 2
A plurality of images picked up by are read (S1). It is determined whether the camera 2 used for capturing the read image has been calibrated in the past with reference to a camera calibration parameter file stored in the external storage device 6 (S2). Then, the camera calibration parameters are read from the file (S3). If calibration has not been completed, feature extraction (S4), association between images (S5), and calculation of camera calibration parameters (S6), which will be described later, are sequentially executed, and camera calibration parameters are calculated using the input image. The camera calibration parameters once calculated can be reused if they are stored in the external storage device 6 as a file together with comments and the like. Therefore, steps S4 to S6 do not need to be performed every time.

Once the camera calibration parameters have been determined,
Next, corrected image data is created by removing lens distortion from the input image and the camera calibration parameters (S7). Using the corrected image data without distortion, features on the image are extracted (S8), and the correspondence between the images is performed (S9).
Then, the three-dimensional shape of the object is restored from the correspondence between the camera calibration parameters and the images (S10). However, at this time, only the positional relationship in the three-dimensional space is obtained, and in order to calculate the actual length, it is necessary to input the length of one point in the image. By specifying one location in the image displayed on the display device 8 from the input device 7, the three-dimensional shape is restored.

Next, details of main processing in the main flowchart will be described. FIG. 5 is a flowchart showing details of the feature extraction process (S4) in the main flowchart of FIG. First, a point having a plurality of directions in which the image density value changes rapidly from the image data is detected as a corner. This is stored as feature point data (S11). The feature point extraction error depends on the quantization error and the performance of the feature extraction algorithm, but if there is an extraction position shift, it appears as a large error when the three-dimensional shape is restored. Therefore, for example, it is desirable to execute a plurality of feature extraction algorithms in parallel to extract only clear features. Also, only corners or intersections may cause apparent feature points at occlusion boundaries, which is problematic. Therefore, a feature line segment is also extracted in parallel with the feature point extraction. That is, a point sequence in which the image density value changes abruptly is detected as an edge from the image data (S12), a line segment is detected from this, and this is stored as line data (S12).
13).

FIG. 6 is a flowchart showing details of the association between images (S5) in the main flowchart of FIG. Incorrect feature points can be generated by generating abnormal data.
Since the estimation of the fundamental matrix or trilineartensor is adversely affected, it is necessary to reduce this as much as possible. For this reason, first, from the extracted feature point data, as the initial correspondence, the similarity of the image in the vicinity of the feature point and the similarity that the corresponding point is located at a similar position between adjacent feature points, Correlation between feature points is performed (S21). In parallel with this, the line segment data is also initially associated with the line segment based on the similarity of the line segment parameters (S22).
Subsequently, a constraint condition to be satisfied between the images is calculated from the correspondence between the feature points or the line segments (S23). As described above, this constraint condition is an epipolar constraint between two images,
A trilinear constraint can be used between the three images.

Next, among the feature point data extracted from each image, a set of feature point data having a positional relationship satisfying the calculated constraint condition is selected as corresponding point data (S
24). Similarly, a set of characteristic line segment data having a positional relationship that satisfies the constraint condition among the line segment data is selected as corresponding line segment data (S25). It is determined whether the position of the corresponding data obtained from the constraint condition calculated in step S23 and the position of the corresponding feature data on the image match sufficiently well, and whether similarity is found in the feature data parameters. The accuracy of the correspondence between the images is evaluated based on (S2
6). If the evaluation result is obtained with sufficient accuracy so as to exceed a predetermined value, the result is stored as corresponding point data and corresponding line segment data. Otherwise, the constraint condition is calculated again using this result (S23), and this is repeated until the evaluation result exceeds a predetermined value.

Next, calculation of camera calibration parameters will be described. The points actually obtained on the image plane S in FIG.

[0057]

(Equation 16)

Then, this point is affected by lens aberration distortion of the camera 2. Here, the correction is

[0059]

[Equation 17]

Then, the point after distortion correction is

[0061]

(Equation 18)

Can be expressed as here,

[0063]

[Equation 19]

Then, the correction can be expressed as follows.

[0065]

(Equation 20)

K ₁ , p ₁ , p ₂ , s ₁ , and s ₂ are called lens distortion correction coefficients, which together with the camera internal parameters are called camera calibration parameters.

FIG. 7 is a flowchart showing details of the camera calibration parameter calculation processing (S6) in the main flowchart of FIG. Generally, lens distortion is
The distance increases from the optical center (the intersection of the camera's optical axis and the image plane). Therefore, assuming that the optical center and the center of the image plane are not so far apart, the constraint condition between the images is calculated using only the corresponding data near the center of the image (S3).
1). This is the same as that calculated in step S23 in FIG. Next, equations relating to camera internal parameters such as the optical center and the focal length can be derived from the constraints between the images. One such equation is the Kruppa equation. The camera internal parameters are calculated by solving such an equation (S32). Subsequently, from the calculated optical center and the corresponding data, a perspective projection transformation matrix P, which can be calculated by the aforementioned equation 5, is obtained, and a lens distortion correction coefficient is calculated from the perspective projection transformation matrix P and the corresponding data including a portion other than the vicinity of the optical center. It is calculated (S33). Corresponding data without lens distortion is calculated from the lens distortion correction coefficient calculated in step S33 (S34), and the constraint condition between images is calculated using this (S35). An equation to be solved is obtained from the constraint conditions, and the internal parameters of the camera are calculated (S36). Then, it is evaluated whether or not the lens distortion correction coefficient and the camera's internal parameters have been obtained with sufficient accuracy, and if obtained, these are stored as camera calibration parameters. If not found, step S3
Returning to step S3, the processing after the calculation of the lens distortion correction coefficient is repeated (S37).

In the step of generating image data without distortion (S7) in the main flowchart of FIG. 4, as shown in FIG. 8, lens distortion correction of image data is performed using camera calibration parameters (S41), and an image without distortion is obtained. Create data. In the three-dimensional shape calculation process (S10) in the main flowchart of FIG. 4, a process as shown in FIG. 9 is executed. The perspective projection transformation matrix P can be calculated from the constraint conditions between images calculated from the corresponding point data and the corresponding line data and the camera internal parameters while leaving the scale uncertainty. Therefore, first, this perspective projection transformation matrix P
Is calculated (S51), and the length of one point in the image is input from the input device 7 (S52). Then, the three-dimensional shape of the object is restored from the length of the one portion and the perspective projection transformation matrix P (S53). By the above processing, the three-dimensional shape can be restored with high accuracy.

[0069]

As described above, according to the present invention, a constraint condition between image data is calculated based on feature data extracted from a plurality of image data captured from different angles, and the constraint condition is satisfied. The feature data is correlated as described above, data of a length of one place in the image data is given, and the measurement object is arranged based on the correlated feature data and the constraint condition between the image data. A perspective projection transformation matrix between the three-dimensional space and the image plane of each of the image data is calculated, and the three-dimensional shape of the object to be measured is calculated from the perspective projection transformation matrix and the given length data. Therefore, it is possible to prevent erroneous correspondence or association between feature data having low accuracy, improve the accuracy of the association, and accurately restore the three-dimensional shape. Achieve the cormorant effect.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a three-dimensional shape restoration system according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a perspective projection transformation matrix of a camera in the three-dimensional shape restoration method according to the present invention.

FIG. 3 is a diagram for explaining association of feature points in the restoration method.

FIG. 4 is a main flowchart showing one embodiment of the restoration method.

FIG. 5 is a flowchart of a feature extraction process in the main flowchart.

FIG. 6 is a flowchart showing a process for associating images in the main flowchart.

FIG. 7 is a flowchart of a camera calibration parameter calculation process in the main flowchart.

FIG. 8 is a flowchart of a lens distortion correction process in the main flowchart.

FIG. 9 is a flowchart of a three-dimensional shape calculation process in the main flowchart.

[Explanation of symbols]

1 ... measurement object, 2 ₁ , 2 ₂ ... camera, 3 ... image memory,
4 arithmetic processing device, 5 work memory, 6 external storage device, 7 input device, 8 display device, 9 output device.

Continued on the front page (51) Int.Cl. ⁷ Identification FI FI Theme Court II (Reference) // G01C 3/06 G01B 11/24 N (72) Inventor Shinichi Sadahiro 1-1-1 Kita 7-Nishi, Kita-ku, Sapporo, Hokkaido No. 2 System Technology Institute, Inc. (72) Inventor Naohisa Tsujimura 1-2-1, Kita 7-Jo Nishi, Kita-ku, Sapporo-shi, Hokkaido F-term (Reference) 2F065 AA53 BB05 EE00 EE08 FF05 JJ03 JJ05 JJ19 JJ26 QQ00 QQ03 QQ23 QQ24 QQ28 QQ38 QQ41 SS02 SS06 SS13 2F112 AC02 AC06 BA06 CA08 CA12 FA03 FA21 FA38 FA45 5B057 AA20 BA02 CA12 CA16 CB13 CB16 CD12 DA07 DB02 DC05 DC32 5L0969 FA09 FA09 FA01 FA01 FA07

Claims (6)

[Claims] A step of reading a plurality of image data obtained by imaging a measurement object from different angles; a step of extracting feature data of an image from each of the plurality of read image data; Calculating a constraint condition between the image data based on the extracted feature data, and associating the feature data so as to satisfy the constraint condition; and providing data of a length of one portion in the image data Calculating a perspective projection transformation matrix between the three-dimensional space in which the object to be measured is located and the image plane of each image data from the associated feature data and the constraint condition between the image data. Restoring the three-dimensional shape of the measurement object from the perspective projection transformation matrix and the given length data. Three-dimensional shape reconstruction how to. 2. A step of reading a plurality of image data obtained by imaging an object to be measured from different angles; a step of performing distortion correction on the plurality of read image data based on a camera calibration parameter; Extracting the feature data of the image from each of the plurality of image data having undergone the distortion correction in this step; calculating a constraint condition between the image data based on the feature data extracted in this step; satisfying the constraint condition Associating the characteristic data as described above, providing a length data at one position in the image data, and the constraint condition between the associated characteristic data and the image data. Is calculated, and a perspective projection transformation matrix between the three-dimensional space in which is disposed and the image plane of each image data is calculated. Three-dimensional shape restoring method characterized by comprising the step of restoring the three-dimensional shape of the measurement object from the length of the data in one place given the a. 3. The method according to claim 1, further comprising: calculating the camera calibration parameter from the plurality of read image data; and calculating the camera calibration parameter from the read plurality of image data. Extracting, calculating constraint conditions between the image data based on the feature data near the center of the image data among the feature data extracted in this step, and associating the feature data so as to satisfy the constraint conditions. Performing, calculating a camera internal parameter from the constraint condition, obtains a perspective projection transformation matrix for each image data from the optical center and the associated feature data, and calculates a perspective projection transformation matrix and other than the vicinity of the optical center. Calculating a lens distortion correction coefficient from the associated feature data including the portion. Three-dimensional shape restoration method according to claim 2, wherein a has. 4. The feature data extracted from the image data includes: a feature point data indicating an intersection or a corner of an edge where the density of the image data changes rapidly; and an edge where the density of the image data changes rapidly. The three-dimensional shape restoration method according to any one of claims 1 to 3, wherein the characteristic line segment data indicates the following. 5. The step of associating the feature data includes, after associating the extracted feature data with the plurality of image data, calculating a constraint condition between the image data, The correspondence of the feature data that satisfies the constraint condition between the images is performed again, and the accuracy of the correspondence between the images is determined based on the position of the corresponding data obtained from the constraint condition and the position of the feature data associated with the constraint condition. A step of repeating the calculation of the constraint condition and the association of the feature data based on the constraint condition until a predetermined accuracy is obtained.
5. The method for restoring a three-dimensional shape according to claim 4. 6. One or a plurality of image pickup means for picking up an image of an object to be measured from different angles; an image storage means for reading and storing a plurality of image data from the image pickup means; Input means for providing data of the image data, a function of extracting feature data of an image from each of the plurality of image data stored in the image storage means, and a constraint condition between the image data based on the extracted feature data. The function of calculating and associating the feature data so as to satisfy the constraint condition, the three-dimensional space in which the measurement object is arranged from the associated feature data and the constraint condition between the image data and the A perspective projection transformation matrix between each image data and the image plane is calculated, and the three-dimensional object of the measurement object is calculated from the perspective projection transformation matrix and the given length data. A three-dimensional shape restoration system, comprising: an arithmetic processing unit having a function of restoring a shape.