JPH03220887A - Video frame selection predictive coding device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Summary] An object of the present invention is to enable more accurate and highly efficient predictive coding in a moving picture frame selection predictive coding device used, for example, in a storage video processing system. a first predictive encoding means that predictively encodes the current screen using a previous screen of the original image as a reference screen; a second predictive encoding means that predictively encodes the current screen using a subsequent screen of the original image as a reference screen; and selecting means for selecting one of the outputs of the first and second predictive encoding means according to a predetermined evaluation function.

[Industrial Application Field 1] The present invention relates to a frame selective predictive encoding device for moving images used, for example, in a storage moving image processing system.

In a storage video processing system, etc., it is necessary to encode video information with high efficiency and store it efficiently.Therefore, an encoding device that can encode video information with high efficiency is also required. It is said that

The predictive encoding device according to the present invention is a storage video system in which video i-brochures are stored in advance on a CD-ROM or the like, for example, and encodes these video information with higher efficiency to M products. This is effective in cases where interpolation between the front and rear screens of a moving image is to be generated by prediction.

[Prior Art] Conventionally, interframe prediction, intraframe prediction, motion compensated prediction, and the like are known as representative predictive coding methods for moving images.

FIG. 12 shows an example of the configuration of a motion compensation predictive encoder. This motion compensation predictive encoding method divides the screen into multiple blocks and encodes each block.
At this time, by calculating a motion vector for each block and correcting the position of the reference screen using this motion vector,
This aims to reduce transmission information by reducing prediction errors.

In FIG. 12, 41 is a subtracter that calculates the difference between the input image X and the predicted value X and outputs a prediction error ε; 42 is a quantizer that quantizes the prediction error ε and outputs a quantization number q;
43 receives the quantization number q and calculates the quantization value ε of the prediction error.
44 is an adder that adds the prediction error ε° and the predicted value *X to generate a local composite signal X°;
45 is a frame memory that stores the image of the previous frame screen (reference screen), 46 is the input image X and the frame memory 5
A motion compensator 47 calculates and outputs a motion vector V of the block of the input image by comparing the input image with the previous screen in the frame memory 45. Reference numeral 47 is a variable delay device that moves the position of the previous screen from the frame memory 45 according to the motion vector V.

This motion compensation predictive encoder uses a motion compensator 6 to calculate the amount of motion of a block 7, and the input image block X is predictively encoded using the previous screen after motion compensation.

In this method, the decoding unit uses the already decoded reproduced image as the reference screen for decoding, so the row encoding unit also performs encoding using the locally decoded reproduced image as the reference screen. There is.

[Problems to be Solved by the Invention] Motion compensated prediction allows highly accurate prediction of motions such as parallel movement, for example.

On the other hand, when a new pattern appears in the background due to the movement of a moving object, it is not always possible to make appropriate predictions, and there are problems such as the prediction efficiency is not increased or the quality of the reproduced image is poor.

Specifically, 1. For example, as shown in Fig. 13, a star pattern in the background that was hidden in the shadow of a person's arm on the previous screen (t-1) appears on the current screen t. , the conventional method has a large prediction error for this star pattern.

Furthermore, in the conventional method, a locally decoded reproduced image is used as a reference screen used when making predictions. This reproduced image contains errors when compared with the original original image, and the worse the image quality of the reproduced image, the lower the correlation with the current image being compared.As a result, prediction efficiency does not improve and the amount of transmitted information decreases. It will increase.

The present invention has been made in view of the above points, and an object thereof is to provide a predictive coding device that can further improve prediction efficiency.

[Means to solve the problem] FIG. 1 is a diagram illustrating the principle of the present invention.

The frame selective predictive encoding device for moving images according to the present invention includes a first predictive encoding means 101 that predictively encodes a current screen using a previous screen of an original image as a reference screen, and a current screen using a subsequent screen of the original image as a reference screen. and a selection means 103 that selects one of the outputs of the first and second predictive encoding means 101 and 102 according to a predetermined evaluation function. In order to improve prediction efficiency, the original image is used as the reference screen.

In addition, as another aspect of the frame selective predictive encoding device for moving images according to the present invention, in the above-described configuration, a variable block size is adaptively selected as a processing block unit, and interframe prediction and interframe prediction are used as predictive encoding methods. One of the prediction schemes including motion compensated prediction is configured to be adaptively selected.

Furthermore, the frame selective predictive encoding device for moving images according to the present invention includes, as another form, a first predictive encoding means for predictively encoding a current screen using a previous screen as a reference screen; comprising a second predictive encoding means for predictively encoding a screen, and a selection means for selecting one of the outputs of the first and second predictive encoding means according to a predetermined evaluation function,
A variable block size is adaptively selected as a processing block unit, and one of prediction methods including interframe prediction and motion compensation prediction is adaptively selected as a predictive encoding method.

[Operation] The inefficiency of the prediction of the star pattern described above is caused by the fact that the conventional method uses inter-frame correlation in only one direction along the flow of time. In other words, the conventional method
As shown in Figure (A), to predict the current frame t, use the previous frame (1-1) as a reference screen and use motion compensation prediction or interframe prediction using the correlation with this previous frame (t-11). It is carried out.

On the other hand, if the correlation between frames in the temporally opposite direction is also used for prediction, it is expected that prediction accuracy can be further improved. That is, as shown in FIG. 3(B),
When predicting the current frame t, the previous frame (t-1
), but also the subsequent frame (t+1) is used as a reference screen for prediction; for example, the previous frame (t+1) is used as a reference screen for prediction.
If there is a large movement on the screen between t-1) and the current frame t, the information from the next frame (t+1) is more correlated with the current frame than the information from the previous frame (1-1). Since this is high, by using this as a reference screen, more accurate predictions can be made.

In the example of FIG. 13 mentioned above, the prediction of the current screen t is based on the subsequent screen (t+1) where the fruit stalk has already appeared, rather than the previous screen (t-1) where the fruit stalk is hidden, as the reference screen. It is possible to make more accurate predictions by making predictions.

Furthermore, in the present invention, the original image is used as it is, rather than the locally decoded reproduced image, as a reference screen used for prediction. As a result, as shown in Fig. 2 (A), when the reference image is the locally decoded previous screen (t-1) or the subsequent screen (t+1) for the current image t, the correlation between frames is However, as shown in Figure 2 (B),
When the previous screen (t-1) or the subsequent screen (t+1) of the original image is used as a reference screen for the current image t, the inter-frame correlation becomes high and the prediction efficiency improves.

The operation of the apparatus of the present invention is based on the premise that the original picture is used as the reference picture, and the first predictive encoding means 101 predictively encodes the current picture t using the previous picture (t-B as the reference picture), while the second The predictive encoding means 102 of the rear screen (t+1)
The current screen t is predictively encoded using the current screen t as a reference screen. The selection means 103 selects these first and second items according to a predetermined evaluation function.
Among the output signals of the predictive encoding means 101 and 102, one with good prediction efficiency etc. is selected and output.

In addition, by adaptively selecting the block size as a processing unit and the prediction method, it is possible to perform accurate prediction processing according to the characteristics of each part of the video image, such as still or moving parts. By combining this with the aforementioned prediction processing in which the original images of the previous frame and the subsequent frame are adaptively selected as reference screens, it is possible to further improve transmission efficiency while reducing errors as a whole.

In addition, even when using a locally decoded image without using the original image as a reference screen, the reference screen can be adaptively switched between the front screen and the back screen, and the variable block size and child
By adaptively selecting the 'jjjl method, more efficient predictive coding can be performed than with conventional methods.

[Actual flow example] Hereinafter, the present invention will be described in detail with reference to the drawings.

This embodiment relates to a predictive coding device used in a storage video processing system, and uses a variable block size motion compensation method disclosed in Japanese Patent Application No. 63-66200 filed by the present applicant as a predictive algorithm. has been done. This variable block size motion compensation method is a method in which prediction is performed while adaptively switching the block size of the processing unit and the prediction method.

FIG. 4 shows an outline of the overall configuration of the embodiment device. In the figure, l is a predictive encoding unit that performs variable block size motion compensation predictive coding of the current frame image (image to be reproduced) using the previous frame image of the original image as a reference screen, and 2 is the current frame image using the subsequent frame image of the original image as a reference screen. This is a predictive encoding unit that performs variable block size motion compensation predictive encoding of an image, and the prediction results and prediction errors output from each predictive encoding unit 1 and 2 are manually input to a prediction error evaluation unit 3. Further, 4 and 5 are frame memories, respectively, which are connected in series to obtain image signals of the current frame, previous frame, and subsequent frame of the original image input, and the images of the current frame and the previous frame are sent to the predictive encoding unit l. , the images of the current frame and the next frame are provided to the predictive encoding unit 2.

The predictive encoding units 1 and 2 predictively encode the video signal block by block, adaptively select any of interframe prediction, motion compensation prediction, and intraframe prediction as the prediction method, and further reduce the prediction error. It is configured to adaptively switch the block size according to the error distribution on the screen, such as reducing the block size for large parts and increasing the block size for parts with relatively small prediction errors. It is something that

The predictive encoding units 1 and 2 have the same configuration, and examples of the configuration are shown in FIGS. 5 and 6. FIG. 5 is a schematic block configuration of a circuit for adaptively selecting a prediction method, and FIG. 6 is a schematic block configuration of a circuit for adaptively selecting a block size.

In FIG. 5, 11 is a motion detection section, 12 is a variable delay section, 13 is an average value calculation section, 14 is an interframe prediction section, 15
1 is a motion compensation prediction unit, 16 is an intra-frame prediction unit, and 17 is a prediction method evaluation unit.

Motion detection unit 11, average value calculation unit 13, interframe prediction unit 14, motion compensation prediction unit 15, and intraframe prediction unit 16
The input image X is manually input to each of the input images X, and the reference image Yc (previous frame image or subsequent frame image) is input to the variable delay unit 12 and the interframe prediction unit 14, respectively.

The output from the variable delay section 12 is input to the motion compensation prediction section 15, and the output from the average value calculation section 13 is input to the intraframe prediction section 16.
is input. Furthermore, the interframe prediction error ε(k) output from the interframe prediction unit 14, the motion compensation prediction unit 15
The motion compensated prediction error εfral output from the intraframe prediction unit 16 and the intraframe prediction error ε(nl output from the intraframe prediction unit 16) are each input to the prediction method evaluation unit 17.

The motion detecting section 11 is a circuit that refers to the reference image Y, tracks the motion of the input image X by pattern matching, etc., calculates a motion vector, and supplies it to the variable delay section 12. The variable delay unit 12 is a circuit that performs prediction by shifting the position of the predicted screen according to the calculated motion vector. The average value calculation unit 13 is a circuit that calculates the average value of the input image X for each encoded block.

The inter-frame prediction unit 14 is a circuit that calculates the inter-frame prediction error ε(k) by taking the difference between the input image
The intra-frame prediction unit 16 is a circuit that calculates the motion compensation prediction error ε(m) by taking the difference between the position-corrected reference image Y from the input image X and the input image from the average value calculation unit 13. This circuit calculates the intra-frame prediction error ε(n) by taking the difference from the average value of the encoded block of X.

The prediction method evaluation unit 17 calculates each input prediction error ε(kl
ε(ml ε(nl) is evaluated by a predetermined evaluation function to select and determine the optimal prediction method for the block to be encoded, and the identification information of the selected prediction method and the prediction error obtained by the prediction method are Output line 17 (a)
, 17 (b), and the output information is stored in the memory circuit. In addition, when calculating the error signal by adding the average value of the own block in intra-frame prediction as in this example, the average value of the own block is also added together with the identification information of the selected prediction method and the prediction error. Output.

In FIG. 6, 20 is a block size evaluation unit;
A prediction error ε determined by the circuit of FIG. 5 for a certain encoded block is inputted via an input line 21. The input line 23 sequentially displays the prediction error ε3 calculated for each block (referred to as a small block for convenience) that is obtained by dividing the block size of the block input to the input line 21 (referred to as a large block for convenience) into wI numbers. This is an input signal line, and these prediction errors ε3 are sent to the prediction error average value calculation unit 22. The prediction error average value calculation unit 22 is a circuit that calculates the average value (εS) of the prediction errors ε3 of these small blocks, and the calculated average value (ε3) is input to the block size evaluation unit 20 via the output line 24. be done.

The block size evaluation unit 20 evaluates the prediction error ε of the input large block and the average value (εS) of the prediction errors ε3 of a plurality of small blocks obtained by dividing the block, according to a predetermined evaluation function, and Evaluate and determine which of the two sizes is the optimal block size according to the characteristics of the image to be encoded.

The operation of the predictive encoding units 1 and 2 will be explained below, taking as an example the case where the maximum size of the block to be encoded is 32 x 32 (pel).

First, the block size of the block to be encoded is the 7th
As shown in the figure, the maximum size is 32X32, and this is divided into several stages and sequentially subdivided into 16X and 16.8 X8.
．． 4 types of block sizes of 4×4 are used. Therefore, a 32x32 block has 1
It is sequentially subdivided into smaller blocks such as 6X 16.8X8.4X4.

Determination of the optimal prediction method by the circuit of FIG. 5 is performed for each block of each block size subdivided as described above. In other words, the input image block
Assuming that the input is input to the circuit shown in the figure, the interframe prediction unit 14,
The motion compensation prediction unit 15 and the intraframe prediction unit 16 respectively perform calculations on this input image block to obtain the interframe prediction error ε(kl), the motion compensation prediction error ε(■), and the intraframe prediction error ε(n), respectively. and sends them to the prediction method evaluation section 17.

The prediction method evaluation unit 17 calculates these prediction errors ε(k)ε(w)
Evaluate ε(n) according to a predetermined evaluation function, determine the optimal prediction method, and output the prediction method identification information and prediction error.

Once the prediction method and prediction error are determined by the prediction method evaluation unit 17, the optimal block size for encoding is determined by the circuit shown in FIG. This block size determination is first made between a 32x32 block and a 16x16 block, and if a 16x16 block is selected, then between 16x16 and 8x8, and an 8x8 block is selected. If it is, it will be between 8x8 and 4x4.

Once the prediction method and prediction error are determined for each block size as described above, the blocks shown in FIG.
Input # to lock size evaluation section 20! A prediction error ε of a large block (for example, an 8×8 block) is inputted via 21. On the other hand, the prediction error average value calculation unit 22 has an input line 2
The prediction errors ε3 of the n small blocks (for example, four blocks of 4×4) constituting the large block are sequentially inputted through 3, and their average value (ε3) is determined, and the average value ( ε3) is sent to the block size evaluation section 20. The block size evaluation unit 20 calculates these prediction errors ε,
and (ε3) are evaluated according to a predetermined evaluation function to determine the optimal block size.

According to the above processing, the optimum block size is selected according to each part of the image, and f' is performed. In this case, for static parts with little movement, the maximum block size is 32x32.
is selected, and as the movement increases, smaller block sizes of 16×16.8×8.4×4 are selected in order.

The overall operation of this embodiment device will be explained below with reference to FIGS. 9 and 1O. This figure 9 and 10
The figure shows the predictive processing algorithm of the embodiment device using the PAD method.

As the overall schematic operation, as shown in FIG. 9, 3
The prediction process shown in FIG. 10 for 2×32 blocks is carried out in two cases: when the previous frame of the original picture is used as the reference picture (step S2), and when the frame after the original picture is used as the reference picture (step S2).
Step S3) and compare the results separately. If the prediction error from the previous frame is smaller than the prediction error from the subsequent frame, use the prediction error from the previous frame and compare the results. Step S5), and in the opposite case, the prediction error from the subsequent frame is used (Step S6). This process is performed for all 32×32 blocks.

The prediction process shown in FIG. 10 will be explained below.

In FIG. 10, MC is a motion compensation prediction subroutine.

INTER frame prediction subroutine, INTRA
represents an intra-frame prediction subroutine, and BEST represents a subroutine for selecting the method with the smallest error among the above three prediction methods.

First, for one 32x32 block, motion compensation prediction MC, interframe prediction I, using the previous frame as a reference screen.
NTER and intraframe prediction INTRA are performed, and the method with the smallest error is selected.

Next, this 32x32 block is divided into 4 blocks of 16x16, and for each block, motion compensation prediction MC, interframe prediction INTER, and intraframe prediction INTRA are performed using the previous frame as a reference screen, and the method with the smallest error is performed. is selected (Step 519), and the errors of the method selected for each block are summed up for the four blocks, and the sum is calculated (Step 5
20).

Then, the error GO3 obtained for this 32×32
Sum of errors G for 16×16 blocks from A32.

5A16 is calculated by 1, and the difference is a predetermined threshold Th
If it is smaller than , then a 32×3.2 block scheme and prediction error is chosen.

On the other hand, if it is larger than the threshold Th, 16X16
further subdivide the block into four 8x8 blocks,
The same process as above is performed for each of the four 16×16 blocks [steps S21 to S532]. Further, depending on the error comparison results, the 8×8 block is also subdivided into 4×4 blocks and similar processing is performed.

Once the prediction process using the previous frame as a reference screen is completed in this way, similar prediction process is performed next using the subsequent frame as a reference screen (step S3), as described above.

Various modifications are possible in implementing the invention. FIG. 11 shows a prediction processing algorithm of such a modified example. In the above-mentioned embodiment, prediction processing is performed by using the previous frame of the original picture as a reference screen and prediction processing is performed using the subsequent frame of the original picture as a reference picture in units of 32x32 blocks, but in this modification,
Prediction processing is performed simultaneously within each variable block using the previous and subsequent frames of the original picture as reference screens. This allows for higher prediction accuracy. You can expect it.

In Fig. 11, (before) and (after) in parentheses respectively represent prediction using the previous frame as the reference screen and prediction using the subsequent frame as the reference screen. When performing prediction processing, processing for the previous frame and processing for the subsequent frame are performed simultaneously, and blocks are selected based on these results.

【Effect of the invention〕

According to the present invention, not only the previous screen but also the subsequent screen is used as a reference screen, and an original image that has a high correlation with the current screen is used as the reference screen, so more accurate and efficient prediction is possible. It becomes possible.

[Brief explanation of drawings]

FIG. 1 is a diagram explaining the principle of the present invention, and FIG. 2 is a diagram explaining how to take a reference image for obtaining correlation information. FIG. 3 is a diagram explaining the temporal direction of prediction, FIG. 4 is a block diagram showing the overall configuration of a moving object frame selection predictive encoding device as an embodiment of the present invention, and FIGS. 5 and 6 are 7 and 8 are block diagrams showing an example of the configuration of the predictive encoding unit in the embodiment device. FIGS. 7 and 8 are diagrams for explaining the operation of the embodiment device. FIGS. 9 and 10 are predictive processing algorithms by the embodiment device. 11 is a flowchart showing a prediction processing algorithm as a modified example of the present invention, FIG. 12 is a block diagram showing a conventional motion compensation predictive encoder, and FIG. 13 is a flowchart showing a prediction processing algorithm as a modification of the present invention. It is a figure explaining the case where the pattern appears. In the figure, 1.2...Prediction encoding unit 3...Prediction error evaluation unit 4.5...Frame memory 11...Motion detection unit 12...Variable delay unit 12.22...Average Value calculation unit 14...Inter frame prediction unit 15...Motion compensation prediction unit 16...Intra frame prediction unit 17...Prediction method evaluation unit 20...Block size evaluation unit

Claims (1)

[Claims] 1. A first predictive encoding means (101) that predictively encodes the current screen using the previous screen of the original picture as a reference screen; and predictively encodes the current screen using the subsequent screen of the original picture as the reference screen. a second predictive encoding means (102), and the first and second predictive encoding means (101, 102) according to a predetermined evaluation function.
) a selection means (103) for selecting one of the outputs of the video frames. 2. Claim 1, wherein a variable block size is adaptively selected as a processing block unit, and one of prediction methods including interframe prediction and motion compensation prediction is adaptively selected as a predictive coding method. The frame selection predictive encoding device for moving images as described above. 3. A first predictive encoding unit that predictively encodes the current screen using the previous screen as a reference screen; a second predictive encoding unit that predictively encodes the current screen using the subsequent screen as a reference screen; and a predetermined evaluation function. a selection means for selecting one of the outputs of the first and second predictive encoding means according to the method, a variable block size is adaptively selected as a processing block unit, and interframe prediction and interframe prediction are selected as the predictive encoding method. A moving image frame selection predictive coding device configured to adaptively select one of prediction methods including motion compensation prediction.