JP2506500B2 - Image motion detector - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image motion detection device for detecting screen blur caused by camera shake, rotation, and the like.

2. Description of the Related Art Conventionally, for example, Japanese Patent Application No.
The one disclosed in Japanese Patent Publication No. 61-269475 is known.
FIG. 6 shows a block diagram of this conventional image motion detecting apparatus. In FIG. 6, reference numerals 22 to 29 denote correlation values between the signal of the previous image frame and the signal of the current image frame by the so-called block matching method for each of the four divided areas of the screen, and the obtained correlation value Among them, the correlation value having the largest correlation is searched, and the candidate vector giving the correlation value is detected. The valid / invalid decision circuit 30 decides the valid / invalid of the candidate vector using the value of the correlation value, and sends the decision signal D to the vector decision circuit 31.
Is output. The vector determination circuit 31 calculates a motion vector indicating the translation amount of the entire image from the candidate vector determined to be valid. With such a configuration, the candidate vector of the portion in which the pattern is not changed is considered to have low reliability and is invalidated, and is excluded from the calculation of the motion vector to improve the accuracy of the motion vector detection.

Further, as a method of obtaining the motion of an object having a motion including rotation and zoom, for example, “a motion vector detection method of an object whose size and direction are changed” (IEICE Spring National Congress (1989) D-109, p7). -91).

However, in the former configuration as shown in FIG. 6, the validity / invalidity of each candidate vector is determined only by the correlation value obtained for each corresponding divided screen of each candidate vector. , The consideration of the candidate vector which is different from the motion of the whole image was insufficient. For example, even if a part of the screen having a variety of patterns moves differently from the whole, the candidate vector for this motion is determined to be valid by the above-described correlation value determination, and the motion of the entire image There was a case where a large error was given to the judgment of.

Further, in the conventional example, for example, in order to detect the zooming state of the camera, it is necessary that the center point of the optical system of the camera and the center point in signal processing be coincident with each other, and when rotation and parallel movement are combined. Could not detect this.

In addition, the latter IEICE Spring National Convention (1989)
In the method as shown in D-109, p7-91, even if there is a region having a motion that is significantly different from the motion of the entire image, this is not excluded and used for calculating the motion of the entire image. , Ordinary images were affected by local motion, and high accuracy could not always be obtained.

The present invention solves such a conventional problem,
To provide an image motion detection device capable of accurately detecting image motion even when a part of the screen moves differently from the whole, when the camera shakes or rotates, or when the camera zooming operation occurs. It is an object.

Means for Solving the Problem The present invention, means for detecting the amount of movement of a plurality of regions of the image, means for calculating a conversion coefficient representing the movement of the image using a plurality of the detected amount of movement, Means for calculating predicted motion amounts in a plurality of regions on the screen using the conversion coefficient, and motion for extracting a region in which an error between the predicted motion amount and the detected motion amounts of the plurality of regions is equal to or less than a predetermined value. It comprises area extraction means and means for determining the movement of the entire image using the extracted movement amount.

Effect According to the present invention, with the above-described configuration, the amount of movement of a plurality of regions of an image is detected, and a plurality of amounts of movement of these detected regions are selected to perform parallel movement as well as rotation, enlargement, and reduction. A conversion coefficient representing the motion of the entire image caused by the combination of is calculated. Therefore, even when the direction and size of the motion vector obtained depending on the position of the screen are different, such as rotation, enlargement, and reduction, the same conversion coefficient is obtained for the same motion, and the same motion is obtained. It is possible to identify the area that belongs to. Also, based on the calculated conversion coefficient, the motion amount at another position of the image is predicted, and the region in which the difference between the predicted motion amount and the detected motion amount is smaller than a predetermined threshold is extracted. Only moving parts can be obtained as the same area, and the movement of the whole image including rotation, enlargement and reduction as well as parallel movement, and the movement of part of the area that is different from the whole movement can be detected with high accuracy. You can

Examples Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 shows a block diagram of an image motion detecting apparatus in an embodiment of the present invention. In FIG. 1, 1 is an image input terminal, 2 is a motion vector detection circuit,
This is means for detecting the amount of motion of each of the divided regions as motion vectors V _{l to} V _n . Reference numeral 3 denotes a motion vector selection circuit, which selects any 2 of the obtained motion vectors V _{l to} V _n.
Is a motion vector selection circuit that selects one motion vector V _i , V _j (i ≠ j), and 4 is the two motion vectors V _i ,
The conversion coefficient calculation circuit calculates a conversion coefficient based on V _j (i ≠ j). Reference numeral 5 denotes a motion amount prediction circuit, which is means for predicting a motion amount at another position on the screen using the conversion coefficient obtained by the conversion coefficient calculation circuit 4. 6 is an error detection circuit for obtaining an error between the motion amount detected by the motion vector detection circuit 2 and the motion amount predicted by the motion amount prediction circuit 5, 7 is a threshold circuit for comparing the error with a threshold value, and 8 is the result thereof. Is a motion area extraction circuit for extracting a motion area from the motion vector, 9 is a motion determination circuit for determining the motion of the entire image, and 10 is a motion amount output terminal for outputting the result.

The operation of the image motion detecting device of the present embodiment having the above configuration will be described below.

The motion vector detection circuit 2 detects the amount of motion in each region when the screen is divided into _n as motion vectors V _{1 to} V _n by the so-called pattern matching method. The motion vector selection circuit 3 selects two motion vectors from V _{l to} V _n , and using this, the conversion coefficient calculation circuit 4 calculates the motion of the image as a conversion coefficient as follows.

If the point (x, y) in the image moves to (X, Y) due to movements such as parallel movement, enlargement / reduction, and rotation, X, Y are the x, y and the coefficients a, b before movement. It can be expressed as X = ax + by + c Y = -bx + ay + d using c and d. The conversion coefficients a to d can be obtained as follows using the motion vectors at the two positions.

If the X and Y components of the motion vector at the point (x, y) are V _x and V _y , respectively, then V _x = X−x V _y = Y−y, so the conversion coefficient and the point (x, y) motion vector V _x, the relationship of V _y can be expressed as (a-1) x + by + c = V x -bx + (a-1) y + c = V y. Coefficients a, b, c, d are 2 points (x ₁ ,
y ₁ ), (x ₂ , y ₂ ) can be expressed as follows using X components Vx ₁ , Vx ₂ , Y components Vy ₁ , Vy ₂ of the motion vector at the respective positions.

_{(A-1) x 1 +} by 1 + c = Vx 1 (a-1) x 2 + by 2 + c = Vx 2 bx 1 + (a-1) y 1 + d = Vy 1 bx 2 + (a-1) y 2 + D = Vy ₂ Therefore, the coefficients a, b, c and d can be obtained by solving this simultaneous equation.

With the method described above, the conversion coefficient calculation circuit 3 calculates the conversion coefficient using two of the _n motion vectors V _{1 to} V _n . In this way, by expressing the motion of the image using the conversion coefficient, even if the direction and size of the motion vector obtained by the position of the screen are different, such as the motion of rotation, enlargement, reduction, etc., the same motion is obtained. The same conversion factor is obtained for.

Next, the motion amount prediction circuit 5 predicts a motion vector at another position of the image based on the calculated conversion coefficient.
The error detection circuit 6 calculates an error between the predicted motion vector and the motion vector detected by the motion vector detection circuit 2. In a region having a motion similar to the motion of the two regions used to calculate the conversion coefficient, the error between the predicted motion vector and the detected motion vector becomes smaller than a predetermined threshold value. Therefore, such an area is set as the threshold circuit 7.
And the motion area extracting circuit 8 extracts the areas having the same motion.

As described above, according to the present embodiment, since the motion of the image is represented by using the conversion coefficient, the direction and magnitude of the motion vector obtained by the position of the screen, such as motions such as rotation, enlargement, and reduction. Even if the values are different, the same conversion coefficient is obtained for the same motion, and it is possible to identify regions belonging to the same motion. Further, since the region where the difference between the predicted motion amount and the detected motion amount is smaller than the predetermined threshold is extracted, it becomes possible to extract the portion having similar motion, and it is possible to detect the motion of the entire image and the screen. It is possible to detect a part different from the whole movement.

 Therefore, the movement of the image can be accurately detected.

FIG. 2 is a block diagram of an image motion detecting apparatus according to the second embodiment of the present invention. The same means as those in the embodiment of FIG. 1 are designated by the same reference numerals and the description thereof will be omitted. The difference from FIG. 1 is that a coefficient evaluation circuit 41, a coefficient selection circuit 42, a candidate area coefficient circuit 81, and a maximum area selection circuit 82 are provided.

The operation of the image motion detecting apparatus of the second embodiment configured as described above will be described below with respect to the difference from the operation of FIG.

The motion vector selection circuit 3 detects the detected motion vector V _l ~
Any of the three motion vectors V _i from among _{V n, V j, V k} (i ≠
j, i ≠ k, j ≠ k). Then, the conversion coefficient calculation circuit 4 calculates a conversion coefficient that gives the motion of the entire image from those motion vectors. That is, the conversion coefficients a to f
Can be obtained using the motion vectors V _i , V _j , and V _k at the three positions as follows.

If the point (x, y) in the image moves to (X, Y) due to movements such as parallel movement, enlargement / reduction, and rotation, X, Y are the x, y and the coefficients a, b before movement. It can be expressed as X = ax + by + c Y = dx + ey + f by using c, d, e, and f. Coefficients a, b, and c are 3 points (x ₁ ,
y ₁ ), (x ₂ , y ₂ ) (x ₃ , y ₃ ) can be expressed as follows using the X components Vx ₁ , Vx ₂ , Vx ₃ of the motion vector at the respective positions.

_{(A-1) x 1 +} by 1 + c = Vx 1 (a-1) x 2 + by 2 + c = Vx 2 (a-1) x 3 + by 3 + c = Vx 3 Thus coefficients a, b, c is the simultaneous equations Can be obtained by solving. The coefficients d, e, f are 3
Use the Y components Vy ₁ , Vy ₂ , and Vy ₃ of the motion vector at each of the points (x ₁ , y ₁ ) and (x ₂ , y ₂ ) (x ₃ , y ₃ ) to express as follows: You can

dx ₁ + (e-1) y ₁ + f = Vy ₁ dx ₂ + (e-1) y ₂ + f = Vy ₂ dx ₃ + (e-1) y ₃ + f = Vy ₃ Therefore, the coefficients d, e and f are It can be obtained by solving this simultaneous equation.

With the method described above, the conversion coefficient calculation circuit 4 can calculate the conversion coefficient using three of the _n motion vectors V _{1 to} V _n .

Here, when the movement of the entire image is a combination of translation, rotation, enlargement, and reduction, and there is no distortion of the shape, there is a special relationship for the conversion coefficients a to d, that is, a = e = S _x · cos θ. ... (1) a = -d = -S x · sinθ ...... (2) relationship should be established. However, S _x is the enlargement ratio in the X-axis direction, S _y is the enlargement ratio in the Y-axis direction, and θ is the rotation angle of the image. In normal camera operation, when the subject is stationary, the movement of the entire image is considered to be a combination of translation, rotation, enlargement, and reduction. Therefore, three motion vectors V _i ,
After _obtaining the transform coefficient from V _j and V _k , the transform coefficient is evaluated by the coefficient evaluation circuit 41, and the one satisfying the above relationship is selected by the coefficient selection circuit 42 and used for the prediction of the motion amount. It is possible to determine whether or not the motion of the image obtained from the three motion vectors V _i , V _j , and V _k is a motion that can occur in normal camera operation such as parallel movement, rotation, and enlargement / reduction. For example, when a part of the screen moves differently than the whole, or when a foreign body enters the screen,
Assuming that a motion that expands in the axial direction and a motion that shrinks in the Y-axis direction are shown, the conversion coefficient obtained from the motion vector of such an area satisfies the relationships of the expressions (1) and (2). Therefore, it can be seen that it is inappropriate to use it for detecting the motion of the entire image. By using only the transform coefficients that satisfy the relationships of the above equations (1) and (2) for the motion amount prediction, the movement of the area different from the movement of the entire image is eliminated, and the movement of the entire image is accurately detected. be able to.

FIG. 3 is for showing a method of judging the motion of the image by evaluating the coefficients obtained from the three motion vectors. It is assumed that the motion vectors of the four areas of the image have been obtained as shown in FIG. This corresponds to, for example, a case where the object in the upper right area of the screen largely moves to the left when the camera rotates leftward around the center of the screen. Four motion vectors When three of these are selected and the conversion coefficient is obtained, FIG. 3 (b), (c) and (d)
Does not satisfy the relationships of the above equations (1) and (2), and it can be determined that only the combination of FIG. Therefore, the motion of the entire image can be accurately detected by excluding the motion vector in the upper right area of the screen that has a motion different from the entire motion.

The number of transform coefficients is obtained by the number of combinations of detected motion vectors, in the case of the present embodiment, the number of combinations of three selected from n motion vectors, but the motion of the entire image is classified by the group of similar transform coefficients. Be reduced to doing. Usually such a classification determines the parameters when the number of dimensions is large,
Although it is difficult in terms of the number of calculations, as in the present embodiment, the number of regions in which the motion vector is predicted from the transform coefficient and the error between the predicted motion vector and the detected motion vector is smaller than a predetermined value is the maximum. It is possible to easily classify the regions belonging to the representative motions by obtaining the region and using only the detected motion vector belonging to this region.

FIG. 4 is a diagram for explaining the operation of the motion area extraction circuit (candidate area coefficient circuit 81 and maximum area selection circuit 82) in this embodiment. FIG. 4 (a) is a diagram showing an outline of the 16 motion vectors detected by dividing the screen into 16 areas, and most of the image is translated from left to right.
Regions 9, 13, 14 and 16 behave differently. In such a case, for example, when an area in which the error at each position predicted from the motion vectors of the areas 1, 2, and 5 is smaller than a predetermined value (a candidate area) is illustrated, for example, as shown in FIG. 4B. Become. At this time, the number of candidate areas is 13
Therefore, it can be seen that the movements of the regions 1, 2, and 5 are representative of the movement of most of the image. On the other hand, the candidate regions when predicted from the motion vectors of the regions 9, 13 and 14 are as shown in FIG. 4 (c), the number of candidate regions is 3, and the regions 9, 13, 14 are
It can be seen that the movement of is an isolated movement from the whole movement. Therefore, the typical motion of the entire image can be more accurately obtained by eliminating the motion vectors of the isolated regions 9, 13, and 14 and using the motion vectors of the other regions.

The overall operation of this embodiment will be described below with reference to FIG.

(Step S1) The motion vector detection circuit 2 first obtains n motion vectors in each area.

(Step S2) The maximum number of candidate areas is set to 0.

(Step S3) The motion vector selection circuit 3 selects three combinations out of n motion vectors.

(Step S4) The number of candidate areas is set to 0.

(Step S5) The conversion coefficient calculation circuit 4 obtains the conversion coefficients a to f.

(Step S6) The coefficient evaluation circuit 41 evaluates the effectiveness of the conversion coefficient by evaluating whether the motion of the image is parallel movement, rotation, enlargement, or reduction.

(Step S7) The coefficient selection circuit 42 selects one area to be evaluated.

(Step S8) The motion amount prediction circuit 5 predicts a motion vector using only valid motion vectors.

(Step S9) The error detection circuit 6 detects an error between the detected motion vector and the predicted motion vector.

(Step S10) The threshold circuit 7 determines whether the error is less than or equal to a predetermined threshold.

(Step S11) If the error is equal to or smaller than the threshold value, the candidate area counting circuit 81 increments the number of candidate areas by one.

(Step S12) It is judged whether all the areas to be evaluated are finished.

(Step S13) If an area to be evaluated remains, the next area is selected and the process returns to (Step S7).

(Step S14) It is determined whether the number of candidate areas is larger than the maximum number of candidate areas.

(Step S15) If the number of candidate areas is larger than the maximum number of candidate areas, the maximum number of candidate areas is updated.

(Step S16) It is determined whether all motion vector combinations have been completed.

(Step S17) If the combination still remains, the combination is updated and the process returns to (Step S4).

(Step S18) Of all the combinations, the maximum area selection circuit 82 selects only the motion vector of the area having the largest candidate area, and the motion amount determination circuit 9 uses the motion vectors to select the motion of the entire image. It is determined and output to the motion amount output terminal 10.

In this way, the conversion coefficient representing the motion of the image is calculated using the three motion vectors, and from the coefficient value of this conversion coefficient,
By determining whether or not the movement of the image is a combination of parallel movement, rotation, reduction, and enlargement, it is possible to exclude the movement of a region different from the movement of the entire image and accurately detect the movement of the entire image. In addition, the method of obtaining the region in which the number of regions in which the error between the predicted motion vector and the detected motion vector becomes smaller than a predetermined value is the maximum, eliminates the influence of the motion in a region that is significantly different from the predicted motion, and It is possible to easily detect the representative motion of the whole with high accuracy.

The present invention is not limited to the above embodiment,
Various modifications are possible based on the spirit of the present invention. For example, the motion of the entire image may be calculated from the motion amount of 4 or more.

EFFECTS OF THE INVENTION As described above, according to the present invention, the following effects can be achieved.

(1) Since the motion of the entire image is calculated as a conversion coefficient from the detected two or more motion amounts, the direction and size of the motion vector obtained by the position of the screen such as rotation, enlargement, and reduction. Even in the case of different motions, the same conversion coefficient is obtained for the same motion, and it is possible to identify the regions belonging to the same motion.

(2) Since the amount of motion at another position in the image is predicted based on the calculated conversion coefficient, and the region in which the difference between the predicted amount of motion and the detected amount of motion is smaller than a predetermined threshold is extracted, A part with similar movement is obtained as the same area, and the movement of the entire image including rotation, enlargement, and reduction in addition to parallel movement can be detected without being affected by the movement of part of the area that is different from the whole movement. You can

[Brief description of drawings]

FIG. 1 is a block diagram of an image motion detecting apparatus according to the first embodiment of the present invention, FIG. 2 is a block diagram of an image motion detecting apparatus according to the second embodiment of the present invention, and FIG. 3 is a block diagram of the present invention. 2 is a vector configuration diagram for explaining the coefficient evaluation in the second embodiment, FIG. 4 is a vector configuration diagram for explaining the motion region extraction operation in the second embodiment of the present invention, and FIG. 5 is the present invention. FIG. 6 is a block diagram of a conventional image motion detecting device, which is a flowchart of the operation in the second embodiment. 1 ... Image input terminal, 2 ... Motion vector detection circuit, 3
...... Motion vector selection circuit, 4 …… Transform coefficient calculation circuit,
5 ... Motion amount prediction circuit, 6 ... Error detection circuit, 7 ... Threshold circuit, 8 ... Motion area extraction circuit, 9 ... Motion amount determination circuit, 10 ... Motion amount output terminal, 41 ... Coefficient evaluation Circuit, 42 ...
... Coefficient selection circuit, 81 ... Candidate area coefficient circuit, 82 ... Maximum area selection circuit.

Claims (3)

(57) [Claims] 1. A motion vector detecting means for detecting a motion amount of a plurality of areas on a screen, and a transform coefficient calculating means for calculating a transform coefficient representing an image motion using a plurality of the detected motion amounts. A motion amount predicting unit that calculates predicted motion amounts in a plurality of regions on the screen using the conversion coefficient, and an error between the predicted motion amount and the detected motion amounts of the plurality of regions is equal to or less than a predetermined value. An image motion detecting apparatus, comprising: a motion region extracting unit that extracts a region that is defined by the following; and a motion amount determining unit that determines the motion of the entire image using the motion amount of the extracted region. 2. The number of areas in which the error between the predicted motion amount and the motion amounts detected in a plurality of regions on the screen is less than or equal to a predetermined threshold value for each of the calculated predicted motion amounts. 2. The image motion detecting device according to claim 1, wherein the motion amount of the area that gives the maximum number of the areas is selected. 3. The conversion coefficient calculation means calculates the conversion coefficient by using the motion amounts of three or more areas on the screen, and the motion amount prediction means determines the effectiveness of the conversion coefficient by the motion of the image. Judgment is made by evaluating any one of movement, rotation, enlargement, reduction, or a combination thereof, and the invalid conversion coefficients are eliminated and the remaining conversion coefficients are used to display a plurality of areas on the screen. 2. The image motion detection device according to claim 1, wherein the predicted motion amount in is calculated.