JP7637884B2 - Video encoding device, image encoding method, image encoding program, video decoding device, image decoding method, image decoding program, transmission method and recording method for transmitting bit stream generated by image encoding device - Google Patents

Description

The present invention relates to image encoding and decoding technology that divides an image into blocks and performs prediction.

When encoding and decoding an image, the image to be processed is divided into blocks, which are collections of a certain number of pixels, and processed on a block-by-block basis. Encoding efficiency is improved by dividing the image into appropriate blocks and setting intra-screen prediction (intra-prediction) and inter-screen prediction (inter-prediction) appropriately.

In video encoding and decoding, encoding efficiency is improved by inter-prediction, which predicts from pictures that have already been encoded and decoded. Patent Document 1 describes a technique for applying affine transformation during inter-prediction. In video, it is not uncommon for objects to undergo transformations such as enlargement, reduction, and rotation, and applying the technique in Patent Document 1 enables efficient encoding.

Japanese Patent Application Publication No. 9-172644

However, the technology in Patent Document 1 involves image conversion, which results in a large processing load. In view of the above-mentioned problem, the present invention provides a low-load, efficient encoding technology.

In order to solve the above problem, an image decoding device according to one aspect of the present invention includes an encoding information storage unit that updates candidates included in a history candidate list with inter prediction information used in inter prediction, a spatial candidate derivation unit that derives spatial candidates from inter prediction information of a plurality of blocks adjacent to a block to be decoded and adds the spatial candidates to a first candidate list, a history candidate derivation unit that adds one or more candidates included in the history candidate list to the first candidate list to create a second candidate list, a candidate selection unit that selects one selection candidate from the candidates included in the second candidate list, and an inter prediction unit that performs inter prediction using the selection candidate as inter prediction information, and the history candidate derivation unit switches whether or not to add a candidate that overlaps with a candidate included in the first candidate list depending on the inter prediction mode.

The present invention makes it possible to achieve highly efficient image encoding and decoding processing with low load.

1 is a block diagram of an image encoding device according to a first embodiment. FIG. 1 is a block diagram of an image decoding device according to a first embodiment. 13 is a flowchart for explaining an operation of dividing a tree block. FIG. 13 is a diagram showing how an input image is divided into tree blocks. FIG. 1 is a diagram illustrating z-scanning. FIG. 13 is a diagram showing the divided shapes of blocks. 13 is a flowchart for explaining an operation of dividing a block into four. 11 is a flowchart for explaining an operation of dividing a block into two or three. This is a syntax for expressing the shape of block division. FIG. 1 is a diagram illustrating intra prediction. FIG. 13 is a diagram for explaining reference blocks for inter prediction. This is a syntax for expressing a coding block prediction mode. FIG. 11 is a diagram showing the correspondence between syntax elements and modes relating to inter prediction. FIG. 13 is a diagram for explaining affine transformation motion compensation of two control points. FIG. 13 is a diagram for explaining affine transformation motion compensation of three control points. FIG. 2 is a block diagram showing a detailed configuration of an inter prediction unit 201 in FIG. 1 . 17 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 301 in FIG. 16. 17 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 302 in FIG. 16. 17 is a flowchart for explaining a normal prediction motion vector mode derivation process of the normal prediction motion vector mode derivation unit 301 of FIG. 16 . 11 is a flowchart showing a processing procedure of a normal predictor motion vector mode derivation process having a function common to a normal predictor motion vector mode derivation unit 301 and a normal predictor motion vector mode derivation unit 401 according to the embodiment of the present invention. 11 is a flowchart illustrating a procedure of a merge mode derivation process having a function common to a normal merge mode derivation unit 302 and a normal merge mode derivation unit 402 according to the embodiment of the present invention. FIG. 3 is a block diagram showing a detailed configuration of an inter prediction unit 203 in FIG. 2 . 23 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 301 in FIG. 22. 17 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 302 in FIG. 16. 23 is a flowchart for explaining a normal prediction motion vector mode derivation process of the normal prediction motion vector mode derivation unit 301 of FIG. 22. 3 is a block diagram of a sub-block prediction motion vector mode derivation unit 303 in the encoding device of the present application. FIG. 4 is a block diagram of a sub-block predicted motion vector mode derivation unit 403 in the decoding device of the present application. FIG. FIG. 3 is a block diagram of a sub-block merge mode derivation unit 304 in the encoding device of the present application. FIG. 4 is a block diagram of a sub-block merging mode derivation unit 404 in the decoding device of the present application. FIG. 13 is a diagram for explaining derivation of affine inheritance predicted motion vector candidates. FIG. 13 is a diagram for explaining derivation of affine constructed predicted motion vector candidates. FIG. 13 is a diagram illustrating affine inheritance merge candidate derivation. FIG. 13 is a diagram illustrating the derivation of affine construction merge candidates. 13 is a flowchart of deriving an affine inheritance predicted motion vector candidate. 13 is a flowchart of deriving an affine constructed predicted motion vector candidate. 13 is a flowchart of affine inheritance merge candidate derivation. 13 is a flowchart of affine construct merge candidate derivation. 38 is a flowchart illustrating a procedure for initializing and updating a history motion vector predictor candidate list. 13 is a flowchart of a procedure of a same element confirmation process in the procedure of a process of deriving a history predicted motion vector candidate. 13 is a flowchart of an element shifting process procedure in the history motion vector predictor candidate derivation process procedure. 13 is a flowchart illustrating a procedure of a process of deriving a history predicted motion vector candidate. 13 is a flowchart illustrating a history merging candidate derivation process procedure. FIG. 11 is a diagram for explaining a procedure of a history motion vector predictor candidate list update process. 13 is a flowchart for explaining an operation of the sub-block temporal merging candidate derivation unit 381. 13 is a flowchart illustrating a process of deriving adjacent motion information of a block. 11 is a flowchart illustrating a process of deriving a temporal motion vector. 13 is a flowchart illustrating derivation of inter prediction information. 13 is a flowchart illustrating a process of deriving sub-block motion information. FIG. 2 is a diagram for explaining the temporal relationship between pictures. 13 is a flowchart for explaining a process of deriving a temporal motion vector predictor candidate in a normal motion vector predictor mode derivation unit 301. 13 is a flowchart for explaining the derivation process of ColPic in the derivation process of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. 13 is a flowchart for explaining a process of deriving coding information of ColPic in a process of derivation of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. 13 is a flowchart illustrating a process of deriving inter prediction information. 13 is a flowchart showing the procedure of a process for deriving inter prediction information of a coding block when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI). 13 is a flowchart illustrating a procedure of a motion vector scaling calculation process. 13 is a flowchart illustrating a process of deriving temporal merge candidates. FIG. 13 is a diagram for explaining the prediction direction of motion compensation prediction in a case where uni-prediction is used and the L0 reference picture (RefL0Pic) is at a time earlier than the current picture to be coded (CurPic). FIG. 13 is a diagram for explaining the prediction direction of motion compensation prediction in a case where uni-prediction is performed and the reference picture for L0 prediction is at a later time than the current picture to be coded. FIG. 13 is a diagram for explaining the prediction direction of motion compensation prediction in a bi-predictive case in which the reference picture for L0 prediction is at a time earlier than the picture to be coded and the reference picture for L1 prediction is at a time later than the picture to be coded. FIG. 13 is a diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time earlier than the current picture to be coded. FIG. 13 is a diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are at a time later than the current picture to be coded. 13 is a table showing an example of a history motion vector predictor candidate added by initialization of a history motion vector predictor candidate list. 13 is a table showing another example of a history motion vector predictor candidate added by initialization of the history motion vector predictor candidate list. 13 is a table showing another example of a history motion vector predictor candidate added by initialization of the history motion vector predictor candidate list. 13 is a flowchart illustrating a procedure of a process of deriving a historical motion vector predictor candidate according to the second modification of the first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a coding/decoding device according to a first embodiment;

Definitions of the technologies and technical terms used in this embodiment.

<Tree Block>
In the embodiment, the image to be encoded/decoded is divided equally into a predetermined size. This unit is defined as a tree block. As shown in FIG. 4, in the embodiment, the size of the tree block is set to 128x128 pixels, but the size of the tree block is not limited to this and may be set to any size. The tree blocks to be processed (corresponding to the encoding target in the encoding process and the decoding target in the decoding process) are switched in raster scan order, that is, from left to right and from top to bottom. The inside of each tree block can be further divided recursively. The block to be encoded/decoded after the tree block division is defined as an encoding block. In addition, the tree block and the encoding block are collectively defined as a block. Efficient encoding is possible by performing appropriate block division. The size of the tree block can be a fixed value previously determined by the encoding device and the decoding device, or a configuration can be adopted in which the size of the tree block determined by the encoding device is transmitted to the decoding device.

<Prediction mode>
For each coding block to be processed, the system switches between intra prediction (MODE_INTRA), which predicts from the surrounding image signals of the processed image to be processed (in coding processing, the signal for which coding has been completed is used for the decoded image, image signal, etc., and in decoding processing, it is used for the image for which decoding has been completed, preview image signal, etc.), and inter prediction (MODE_INTER), which predicts from the image signal of the processed image. A mode for distinguishing between this intra prediction (MODE_INTRA) and inter prediction (MODE_INTER) is defined as a prediction mode (PredMode). The prediction mode (PredMode) has values of intra prediction (MODE_INTRA) or inter prediction (MODE_INTER), and can be selected for coding.

<Inter prediction>
In inter prediction, which performs prediction from an image signal of a processed image, multiple processed images can be used as reference pictures. In order to manage multiple reference pictures, two types of lists, L0 (reference list 0) and L1 (reference list L1), are defined, and reference pictures are identified using reference indices. L0 prediction (Pred_L0) is available for P slices. L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bi-prediction (Pred_BI) are available for B slices. L0 prediction (Pred_L0) is inter prediction that refers to a reference picture managed in L0, and L1 prediction (Pred_L1) is inter prediction that refers to a reference picture managed in L1. Bi-prediction (Pred_BI) is inter prediction in which both L0 prediction and L1 prediction are performed, and one reference picture managed in each of L0 and L1 is referenced. Information that identifies L0 prediction, L1 prediction, and bi-prediction is defined as a reference mode. In the following processing, it is assumed that the constants and variables with the subscript LX in the output are processed for each of L0 and L1.

<Predictive motion vector mode>
The predicted motion vector mode is a mode in which an index for identifying a predicted motion vector, a differential motion vector, a reference mode, and a reference index are transmitted to determine inter prediction information for a block to be coded. The predicted motion vector is derived from a predicted motion vector candidate derived from a processed block adjacent to the block to be processed, or a block belonging to a processed image that is located at the same position as the block to be processed or in the vicinity (neighborhood), and an index for identifying the predicted motion vector.

<Merge mode>
The merge mode is a mode in which inter-prediction information for the block to be processed is derived from inter-prediction information for a processed block adjacent to the block to be processed, or a block belonging to a processed image that is located at the same position as the block to be processed or in its vicinity (vicinity), without transmitting a differential motion vector or a reference index.
The spatial merge candidates are defined as processed blocks adjacent to the current block and the inter prediction information of the processed blocks. The temporal merge candidates are defined as blocks belonging to the processed image that are located at the same position as the current block or in the vicinity (neighborhood) and the inter prediction information derived from the inter prediction information of the blocks. Each merge candidate is registered in a merge candidate list, and a merge index identifies the merge candidate to be used in predicting the current block.

<Proximity block>
11 is a diagram for explaining reference blocks to be referenced for deriving inter prediction information in the predicted motion vector mode and merge mode. A0, A1, A2, B0, B1, B2, and B3 are processed blocks adjacent to the block to be processed. T0 is a block belonging to an image that has been coded/decoded and is located at the same position as the block to be coded/decoded in the image to be coded/decoded or in the vicinity (neighborhood).

A1 and A2 are blocks located to the left of the coding block to be processed and adjacent to the coding block to be processed. B1 and B3 are blocks located above the coding block to be processed and adjacent to the coding block to be processed. A0, B0, and B2 are blocks located at the bottom left, top right, and top left, respectively, of the coding block to be processed.

Details on how adjacent blocks are handled in predicted motion vector mode and merge mode will be given later.

<Affine transformation motion compensation>
Affine transformation motion compensation is a method of dividing a block into subblocks of a predetermined unit, determining a motion vector for each subblock individually, and performing motion compensation. The motion vector of each subblock is derived based on one or more control points derived from inter-prediction information of a processed block adjacent to the block to be processed, or a block belonging to a processed image that is located at the same position as the block to be processed or in the vicinity (neighborhood). In this embodiment, the size of the subblock is 4x4 pixels, but the size of the subblock is not limited to this, and the motion vector may be derived in units of pixels.

Figure 14 shows an example of affine transformation motion compensation when there are two control points. In this case, the two control points have two parameters, a horizontal component and a vertical component, so the affine transformation when there are two control points is called a four-parameter affine transformation. CP1 and CP2 in Figure 14 are the control points. Figure 15 shows an example of affine transformation motion compensation when there are three control points. In this case, the three control points have two parameters, a horizontal component and a vertical component, so the affine transformation when there are three control points is called a six-parameter affine transformation. CP1, CP2, and CP3 in Figure 15 are the control points.

Affine transform motion compensation can be used in both predictive motion vector mode and merge mode. The mode in which affine transform motion compensation is applied in predictive motion vector mode is defined as sub-block predictive motion vector mode, and the mode in which affine transform motion compensation is applied in merge mode is defined as sub-block merge mode.

<Inter prediction syntax>
The syntax related to inter prediction will be described with reference to FIG. 12 and FIG. 13. In FIG. 12, merge_flag is a flag indicating whether the processing target coding block is to be in merge mode or predicted motion vector mode. merge_affine_flag is a flag indicating whether or not the sub-block merge mode is applied to the processing target coding block in merge mode. inter_affine_flag is a flag indicating whether or not the sub-block predicted motion vector mode is applied to the processing target coding block in predicted motion vector mode. cu_affine_type_flag is a flag for determining the number of control points in the sub-block predicted motion vector mode. FIG. 13 shows the values of each syntax element and the corresponding prediction method. merge_flag=1, merge_affine_flag=0 correspond to the normal merge mode, which is a merge mode that is not a sub-block merge. merge_flag=1, merge_affine_flag=1 correspond to the sub-block merge mode. merge_flag=0, inter_affine_flag=0 correspond to the normal predicted motion vector mode, which is a predicted motion vector merge that is not a sub-block predicted motion vector mode. The case where merge_flag=0 and inter_affine_flag=1 corresponds to a sub-block predicted motion vector mode. When merge_flag=0 and inter_affine_flag=1, cu_affine_type_flag is further transmitted to determine the number of control points.

<POC>
POC (Picture Order Count) is a variable associated with a picture to be coded, and a value that increases by one in the output order of the picture is set. The value of POC can be used to determine whether the pictures are the same, to determine the order of pictures in the output order, and to derive the distance between pictures. For example, when two pictures have the same POC value, they can be determined to be the same picture. When two pictures have different POC values, it can be determined that the picture with the smaller POC value is the picture to be output first, and the difference between the POCs of the two pictures indicates the distance between the pictures in the time axis direction.

First Embodiment
An image encoding device 100 and an image decoding device 200 according to a first embodiment of the present invention will be described.

Fig. 1 is a block diagram of an image coding device 100 according to a first embodiment. The video coding device according to the embodiment includes an image coding device 100, a block division unit 101, an inter prediction unit 102, an intra prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual signal generation unit 106, an orthogonal transform and quantization unit 107, a bit string coding unit 108, an inverse quantization and inverse orthogonal transform unit 109, a decoded image signal superimposition unit 110, and a coding information storage memory 111.

The block division unit 101 recursively divides an input image to generate coding blocks. The block division unit 101 includes a 4-way division unit that divides the block to be divided horizontally and vertically, and a 2-3 division unit that divides the block to be divided either horizontally or vertically. The block division unit 101 supplies the generated image signal of the coding block to be processed to the inter prediction unit 102, intra prediction unit 103, and residual signal generation unit 106. The block division unit 101 also supplies information indicating the determined recursive division structure to the bit string coding unit 108. The detailed operation of the block division unit 101 will be described later.

The inter prediction unit 102 performs inter prediction of the coding block to be processed. It derives multiple inter prediction information candidates from the prediction mode stored in the coding information storage memory and the decoded image signal stored in the decoded image memory 104, selects an appropriate inter prediction mode from the multiple candidates, and supplies the selected inter prediction mode and a predicted image signal corresponding to the selected inter prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the inter prediction unit 102 will be described later.

The intra prediction unit 103 performs intra prediction of the coding block to be processed. A predicted image signal is generated by intra prediction from the decoded image signal stored in the decoded image memory 104, an appropriate intra prediction mode is selected from a plurality of intra prediction modes, and the selected intra prediction mode and a predicted image signal corresponding to the selected intra prediction mode are supplied to the prediction method determination unit 105. FIG. 10 shows an example of intra prediction. FIG. 10(a) shows the correspondence between the prediction direction of intra prediction and the prediction mode number. For example, prediction mode 50 generates an intra prediction image by copying pixels in the vertical direction. Prediction mode 1 is a DC mode, in which all pixel values of the block to be processed are the average value of the reference pixels. Prediction mode 0 is a Planar mode, in which a two-dimensional intra prediction image is created from reference pixels in the vertical and horizontal directions. FIG. 10(b) is an example of generating an intra prediction image in the case of prediction mode 40. For each pixel of the block to be processed, the value of the reference pixel in the direction indicated by the prediction mode is copied. If the reference pixel for the prediction mode is not an integer position, the reference pixel value is determined by interpolation from the reference pixel values of surrounding integer positions.

The decoded image memory 104 stores the decoded image generated by the decoded image signal superimposition unit 110. The decoded image stored in the decoded image memory is supplied to the inter prediction unit 102 and the inter prediction unit 103.

The prediction method determination unit 105 determines the optimal prediction mode (inter prediction or intra prediction) for each prediction by evaluating the coding information and the code amount of the residual signal, the amount of distortion between the predicted image signal and the image signal, etc. In the case of the inter prediction merge mode, the prediction method determination unit 105 supplies the bit string coding unit 108 with coding information such as the merge index and information indicating whether or not the sub-block merge mode is selected (sub-block merge flag), and in the case of the inter prediction predicted motion vector mode, the prediction method determination unit 105 supplies the bit string coding unit 108 with coding information such as the inter prediction mode, predicted motion vector index, reference indexes for L0 and L1, differential motion vector, and information indicating whether or not the sub-block mode is selected (sub-block predicted motion vector flag). The determined coding information is supplied to the coding information storage memory 111.

The residual signal generation unit 106 generates a residual signal by subtracting the predicted image signal from the image signal to be processed, and supplies the residual signal to the orthogonal transformation and quantization unit 107.

The orthogonal transform and quantization unit 107 performs orthogonal transform and quantization on the residual signal according to the quantization parameter to generate an orthogonally transformed and quantized residual signal, and supplies it to the bit string coding unit 108 and the inverse quantization and inverse orthogonal transform unit 109.

The bit string coding unit 108 codes coding information according to the prediction method determined by the prediction method determination unit 104 for each coding block, in addition to information on a sequence, picture, slice, and coding block basis. Specifically, the coding information, such as the prediction mode PredMode for each coding block, the division mode PartMode, a flag for determining whether or not the merge mode is selected in the case of inter prediction (PRED_INTER), a sub-block merge flag, a merge index in the case of the merge mode, an inter prediction mode in the case of no merge mode, a predicted motion vector index, information on the differential motion vector, and a sub-block predicted motion vector flag, is coded according to a prescribed syntax rule described later, to generate a first coded bit string. The bit string coding unit 108 also entropy codes the orthogonally transformed and quantized residual signal according to a prescribed syntax rule to generate a second coded bit string. The first coded bit string and the second coded bit string are multiplexed according to a prescribed syntax rule, and a bit stream is output.

The inverse quantization and inverse orthogonal transform unit 109 inverse quantizes and inverse orthogonal transforms the orthogonally transformed and quantized residual signal supplied from the orthogonal transform and quantization unit 107 to calculate a residual signal, which it supplies to the decoded image signal superimposition unit 110.

The decoded image signal superimposition unit 110 generates a decoded image by superimposing the predicted image signal according to the decision made by the prediction method decision unit 105 and the residual signal that has been inverse quantized and inverse orthogonal transformed by the inverse quantization and inverse orthogonal transformation unit 109, and stores the decoded image in the decoded image memory 104. Note that the decoded image may be subjected to a filtering process to reduce distortions such as block distortions caused by encoding, before being stored in the decoded image memory 104.

The coding information storage memory 111 stores coding information such as the prediction mode (inter prediction or intra prediction) determined by the prediction method determination unit 105. In the case of inter prediction, the coding information stored in the coding information storage memory 111 includes the determined motion vector, reference list, and reference index, in the case of inter prediction merge mode, the coding information of the merge index and information indicating whether or not the sub-block merge mode (sub-block merge flag), in the case of inter prediction prediction motion vector mode, the inter prediction mode, the predicted motion vector index of L0 and L1, the reference index of L0 and L1, the difference motion vector of L0 and L1, information indicating whether or not the sub-block mode (sub-block prediction motion vector flag), and in the case of intra prediction, the determined intra prediction mode. The construction of the history candidate list managed by the coding information storage memory 111 will be described later.

Figure 2 is a block diagram showing the configuration of a video decoding device according to an embodiment of the present invention, which corresponds to the video encoding device of Figure 1. The video decoding device according to the embodiment includes a bitstream decoding unit 201, a block division unit 202, an inter prediction unit 203, an intra prediction unit 204, an encoding information storage memory 205, an inverse quantization and inverse orthogonal transformation unit 206, a decoded image signal superimposition unit 207, and a decoded image memory 208.

The decoding process of the video decoding device in FIG. 2 corresponds to the decoding process provided inside the video encoding device in FIG. 1, so each of the components of the encoding information storage memory 205, inverse quantization and inverse orthogonal transform unit 206, decoded image signal superimposition unit 207, and decoded image memory 208 in FIG. 2 has a function corresponding to each of the components of the inverse quantization and inverse orthogonal transform unit 109, decoded image signal superimposition unit 110, encoding information storage memory 111, and decoded image memory 104 of the video encoding device in FIG. 1.

The bit stream supplied to the bit string decoding unit 201 is separated according to the rules of the specified syntax. The separated first encoded bit string is decoded to obtain information on a sequence, a picture, a slice, and an encoding block unit, and encoding information on an encoding block unit. Specifically, the encoding information on the prediction mode PredMode that determines whether the encoding block unit is inter prediction (PRED_INTER) or intra prediction (PRED_INTRA), the division mode PartMode, a flag that determines whether the encoding block unit is a merge mode in the case of inter prediction (PRED_INTER), a merge index in the case of merge mode, a sub-block merge flag, an inter prediction mode in the case of a predicted motion vector mode, a predicted motion vector index of L0 and L1, a reference index of L0 and L1, a difference motion vector of L0 and L1, a sub-block predicted motion vector flag, etc. is decoded according to the rules of the specified syntax described later, and the encoding information is supplied to the inter prediction unit 203 or the intra prediction unit 204, and the encoding information storage memory 205. The separated second encoded bit sequence is decoded to calculate an orthogonally transformed and quantized residual signal, and the orthogonally transformed and quantized residual signal is supplied to the inverse quantization and inverse orthogonal transform unit 208.

When the prediction mode PredMode of the coding block to be decoded is inter prediction (PRED_INTER) and a predicted motion vector mode, the inter prediction unit 203 derives multiple predicted motion vector candidates using the coding information of the already decoded image signal stored in the coding information storage memory 205 and registers them in a predicted motion vector candidate list described later, selects a predicted motion vector from the multiple predicted motion vector candidates registered in the predicted motion vector candidate list corresponding to the predicted motion vector index decoded and supplied by the first coding bit string decoding unit 202, calculates a motion vector from the difference vector decoded by the bit string decoding unit 201 and the selected predicted motion vector, and stores the motion vector in the coding information storage memory 205 together with other coding information. The coding information of the coding block supplied and stored here includes a prediction mode PredMode, a division mode PartMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether to use L0 prediction and L1 prediction, reference indexes refIdxL0[xP][yP] and refIdxL1[xP][yP] of L0 and L1, motion vectors mvL0[xP][yP] and mvL1[xP][yP] of L0 and L1, etc. Here, xP and yP are indexes indicating the position of the upper left pixel of the coding block in the picture. When the prediction mode PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the flag predFlagL0 indicating whether to use L0 prediction is 1, and the flag predFlagL1 indicating whether to use L1 prediction is 0. When the inter prediction mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether or not to use L0 prediction is 0, and the flag predFlagL1 indicating whether or not to use L1 prediction is 1. When the inter prediction mode is bi-prediction (Pred_BI), the flag predFlagL0 indicating whether or not to use L0 prediction and the flag predFlagL1 indicating whether or not to use L1 prediction are both 1. Furthermore, when the prediction mode PredMode of the coding block to be decoded is inter prediction (PRED_INTER) and a merge mode, merge candidates are derived. Using the coding information of the already decoded coding block stored in the coding information storage memory 205, multiple merge candidates are derived and registered in a merge candidate list described later, and a merge candidate corresponding to a merge index decoded and supplied by the bitstream decoding unit 201 is selected from the multiple merge candidates registered in the merge candidate list, and inter prediction information such as flags predFlagL0[xP][yP], predFlagL1[xP][yP] indicating whether to use L0 prediction and L1 prediction of the selected merge candidate, reference indexes refIdxL0[xP][yP], refIdxL1[xP][yP] for L0 and L1, and motion vectors mvL0[xP][yP], mvL1[xP][yP] for L0 and L1 are supplied to the motion compensation prediction unit 206 and stored in the coding information storage memory 205. Here, xP and yP are indexes indicating the position of the upper left pixel of the coding block in the picture. The detailed configuration and operation of the inter prediction unit will be described later.

The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the coding block to be decoded is intra prediction (PRED_INTRA). The coding information decoded by the bitstream decoding unit 201 includes an intra prediction mode, and generates a predicted image signal by intra prediction from the decoded image signal stored in the decoded image memory 210 according to the intra prediction mode, and supplies the predicted image signal to the decoded image signal superimposition unit 209. The intra prediction unit 204 corresponds to the intra prediction unit 103 of the image coding device 100, and therefore performs the same processing as the intra prediction unit 103.

The inverse quantization and inverse orthogonal transform unit 208 performs inverse orthogonal transform and inverse quantization on the orthogonally transformed and quantized residual signal decoded by the first encoded bit string decoding unit 202 to obtain an inverse orthogonally transformed and inverse quantized residual signal.

The decoded image signal superimposition unit 209 decodes the decoded image signal by superimposing the predicted image signal inter-predicted by the motion compensation prediction unit 206 or the predicted image signal intra-predicted by the intra prediction unit 204 on the residual signal inversely orthogonally transformed and inversely quantized by the inverse quantization and inverse orthogonal transformation unit 208, and stores the decoded image signal in the decoded image memory 210. When storing in the decoded image memory 210, the decoded image may be subjected to a filtering process to reduce block distortion, etc. caused by encoding, and then stored in the decoded image memory 210.

Next, the operation of the block division unit 101 in the image encoding device 100 will be described. FIG. 3 is a flowchart showing the operation of dividing an image into tree blocks and further dividing each tree block. First, the input image is divided into tree blocks of a predetermined size (step S1001). Each tree block is scanned in a predetermined order, i.e., raster scan order (step S1002), and the inside of the tree block to be processed is divided (step S1003).

Figure 7 is a flowchart showing the detailed operation of the division process in step S1003. First, it is determined whether or not to divide the block to be processed into four (step S1101).

If it is determined that the block to be processed should be divided into four, the block to be processed is divided into four (step S1102). Each block into which the block to be processed is divided is scanned in Z scan order, that is, in the order of upper left, upper right, lower left, and lower right (step S1103). Figure 5 is an example of Z scan order, and 601 in Figure 6 is an example of a block to be processed divided into four. Numbers 0 to 3 in 601 in Figure 6 indicate the order of processing. Then, the flowchart in Figure 7 is called recursively for each block divided in step S1101.

If it is determined that the block to be processed should not be divided into four, a 2-3 division is performed (step S1105).

Figure 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, it is determined whether the block to be processed is to be divided into 2-3, that is, whether to divide into 2 or 3 (step S1201).

If it is not determined that the block to be processed should be divided into 2-3, i.e., if it is determined that no division should be made, the division is terminated (S1211) and the process returns to the block in the upper hierarchy.

If it is determined that the block to be processed should be divided into 2-3 parts, it is then determined whether or not to further divide the block to be processed into 2 parts (step S1202).

If it is determined that the target block should be divided into two, it is determined whether or not to divide the target block vertically (step S1203), and based on the result of that determination, the target block is divided vertically (step S1204) or horizontally (step S1205). As a result of step S1204, the target block is divided into two vertically as shown in FIG. 602, and as a result of step S1205, the target block is divided into two horizontally as shown in FIG. 604.

If it is not determined in step S1202 that the block to be processed should be divided into two, i.e., if it is determined that the block to be processed should be divided into three, it is determined whether or not to divide the block to be processed vertically (step S1206), and based on the result, the block to be processed is divided vertically (step S1207) or horizontally (step S1208). As a result of step S1207, the block to be processed is divided vertically into three as shown in FIG. 603, and as a result of step S1208, the block to be processed is divided horizontally into three as shown in FIG. 605.

After executing one of steps S1204 to S1205, each block into which the processing target block is divided is scanned in the order from left to right and top to bottom (step S1209). Numbers 0 to 3 of 602 to 605 in FIG. 6 indicate the order of processing. The flowchart in FIG. 8 is called recursively for each divided block.

The recursive block division described here may limit the need for division depending on the number of divisions or the size of the block to be processed. The information limiting the need for division may be realized in a configuration in which no information is transmitted by making a prior agreement between the encoding device and the decoding device, or the encoding device may determine the information limiting the need for division and record it in the encoded bit string, thereby transmitting it to the decoding device.

Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 divides tree blocks using the same processing procedure as the block division unit 101 in the image encoding device 101. However, the difference is that the block division unit 101 in the image encoding device 101 determines the optimal block division shape by applying optimization techniques such as optimal shape estimation by image recognition and distortion rate optimization, whereas the block division unit 202 in the image decoding device 200 determines the block division shape by decoding block division information recorded in the encoded bit string.

The syntax for block division (syntax rules for coded bitstreams) in the first embodiment is shown in Figure 9. coding_quadtree() represents syntax for dividing a block into four, and multi_type_tree() represents syntax for dividing a block into two or three. qt_split is a flag indicating whether or not to divide a block into four. If the block is to be divided into four, qt_split=1 is used. If not, qt_split=0 is used. If the block is to be divided into four (qt_split=1), each block is divided into four and divided into four recursively (coding_quadtree(0), coding_quadtree(1), coding_quadtree(2), coding_quadtree(3)). If not, subsequent divisions are determined according to multi_type_tree(). mtt_split is a flag indicating whether or not to divide the block further. If further division is required (mtt_split=1), the mtt_split_vertical flag indicates whether to divide vertically or horizontally, and the mtt_split_binary flag determines whether to divide into two or three are referenced. mtt_split_vertical=1 indicates to divide vertically, and mtt_split_vertical=0 indicates to divide horizontally. mtt_split_binary=1 indicates to divide into two, and mtt_split_binary=0 indicates to divide into three. Hierarchical block division is performed by recursively calling multi_type_tree until mtt_split=0 is reached.

<Inter prediction>
The inter prediction method according to the embodiment is implemented in the inter prediction unit 102 of the video encoding device in FIG. 1 and the inter prediction unit 203 of the video decoding device in FIG.

The inter prediction method according to the embodiment will be described with reference to the drawings. The inter prediction method is performed in both the encoding and decoding processes on a coding block basis.

(Explanation of the inter prediction unit 102 on the encoding side)
Fig. 16 is a diagram showing a detailed configuration of the inter prediction unit 102 of the video encoding device of Fig. 1. The normal predicted motion vector derivation unit 301 derives a plurality of normal predicted motion vector candidates to select a predicted motion vector, and calculates a difference vector with respect to the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated difference vector become inter prediction information of the normal predicted motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the normal predicted motion vector derivation unit 301 will be described later.

The normal merge mode derivation unit 302 derives multiple normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the inter prediction mode determination unit 306. The detailed configuration and processing of the normal merge mode derivation unit 302 will be described later.

The sub-block predictive motion vector derivation unit 303 derives multiple sub-block predictive motion vector candidates, selects a sub-block predictive motion vector, and calculates a difference vector with the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated difference vector become inter prediction information for the normal predictive motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 306. The detailed configuration and processing of the sub-block predictive motion vector derivation unit 303 will be described later.

The subblock merging mode derivation unit 304 derives multiple subblock merging candidates, selects a subblock merging candidate, and obtains inter-prediction information for the subblock merging mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 306. The detailed configuration and processing of the subblock merging mode derivation unit 304 will be described later.

The inter prediction mode determination unit 305 determines the inter prediction mode based on the inter prediction information supplied from the normal prediction motion vector derivation unit 301, the normal merge mode derivation unit 302, the sub-block prediction motion vector derivation unit 303, and the sub-block merge mode derivation unit 304. The inter prediction mode determination unit 305 supplies inter prediction information according to the determination result to the motion compensation prediction unit 306.

Based on the determined inter prediction information, the motion compensation prediction unit 306 performs inter prediction on the reference image signal stored in the decoded image memory 104. The detailed configuration and processing will be described later.

<Description of the Inter Prediction Unit 203 on the Decoding Side>
FIG. 22 is a diagram showing a detailed configuration of the inter prediction unit 203 of the video decoding device of FIG.

The normal prediction motion vector derivation unit 401 derives multiple normal prediction motion vector candidates, selects a prediction motion vector, and calculates a difference vector with the detected motion vector. The detected inter prediction mode, reference index, motion vector, and difference vector become inter prediction information for the normal prediction motion vector mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the normal prediction motion vector derivation unit 401 will be described later.

The normal merge mode derivation unit 402 derives multiple normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the normal merge mode derivation unit 402 will be described later.

The sub-block predictive motion vector derivation unit 403 derives multiple sub-block predictive motion vector candidates, selects a sub-block predictive motion vector, and calculates a difference vector with the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated difference vector become inter prediction information for the normal predictive motion vector mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the sub-block predictive motion vector derivation unit 403 will be described later.

The subblock merge mode derivation unit 404 derives multiple subblock merge candidates, selects a subblock merge candidate, and obtains inter-prediction information for the subblock merge mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the subblock merge mode derivation unit 404 will be described later.

The motion compensation prediction unit 406 performs inter prediction on the reference image signal stored in the decoded image memory 108 based on the determined inter prediction information. The detailed configuration and processing are the same as those on the encoding side.

<Normal predicted motion vector mode derivation part (normal AMVP)>
The normal prediction motion vector mode derivation unit 301 in Figure 16 includes a spatial prediction motion vector candidate derivation unit 321, a temporal prediction motion vector candidate derivation unit 322, a history prediction motion vector candidate derivation unit 323, a prediction motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a prediction motion vector candidate selection unit 327, and a motion vector subtraction unit 328.

The normal prediction motion vector mode derivation unit 402 in FIG. 23 includes a spatial prediction motion vector candidate derivation unit 421, a temporal prediction motion vector candidate derivation unit 422, a history prediction motion vector candidate derivation unit 423, a prediction motion vector candidate supplementation unit 425, a prediction motion vector candidate selection unit 426, and a motion vector addition unit 428.

The processing procedures of the normal prediction motion vector mode derivation unit 301 on the encoding side and the normal prediction motion vector mode derivation unit 401 on the decoding side will be described with reference to the flowcharts of FIG. 19 and FIG. 25, respectively. FIG. 19 is a flowchart showing the normal prediction motion vector mode derivation processing procedure by the normal motion vector mode derivation unit 301 on the encoding side, and FIG. 25 is a flowchart showing the normal prediction motion vector mode derivation processing procedure by the normal motion vector mode derivation unit 401 on the decoding side.

<Normal predicted motion vector mode derivation unit (normal AMVP): Explanation on the encoding side>
The normal predictive motion vector mode derivation process on the encoding side will be described with reference to FIG.

First, the normal motion vector detection unit 326 detects a normal motion vector for each inter prediction mode and reference index (step S100 in FIG. 19).

Next, the spatial prediction motion vector candidate derivation unit 321, the temporal prediction motion vector candidate derivation unit 322, the history prediction motion vector candidate derivation unit 323, the prediction motion vector candidate supplementation unit 325, the prediction motion vector candidate selection unit 327, and the motion vector subtraction unit 328 calculate the difference motion vector of the motion vector used in the inter prediction of the normal prediction motion vector mode for each of L0 and L1 (steps S101 to S106 in FIG. 19). Specifically, when the prediction mode PredMode of the encoding target block is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the prediction motion vector candidate list mvpListL0 of L0 is calculated, the prediction motion vector mvpL0 is selected, and the difference motion vector mvdL0 of the motion vector mvL0 of L0 is calculated. When the inter prediction mode of the block to be coded is L1 prediction (Pred_L1), the L1 predicted motion vector candidate list mvpListL1 is calculated, the predicted motion vector mvpL1 is selected, and the differential motion vector mvdL1 of the L1 motion vector mvL1 is calculated. When the inter prediction mode of the block to be coded is bi-predictive (Pred_BI), both L0 prediction and L1 prediction are performed, the L0 predicted motion vector candidate list mvpListL0 is calculated, the L0 predicted motion vector mvpL0 is selected, and the differential motion vector mvdL0 of the L0 motion vector mvL0 is calculated, and the L1 predicted motion vector candidate list mvpListL1 is calculated, the L1 predicted motion vector mvpL1 is calculated, and the differential motion vector mvdL1 of the L1 motion vector mvL1 is calculated.

Although the differential motion vector calculation process is performed for each of L0 and L1, it is a common process for both L0 and L1. Therefore, in the following explanation, L0 and L1 are represented as a common LX. In the process of calculating the differential motion vector of L0, X is 0, and in the process of calculating the differential motion vector of L1, X is 1. Furthermore, when referring to information from the other list instead of LX during the process of calculating the differential motion vector of LX, the other list is represented as LY.

When calculating the differential motion vector mvdLX of LX (YES in step S102 in FIG. 19), a candidate predicted motion vector of LX is calculated to construct a predicted motion vector candidate list mvpListLX of LX (step S103 in FIG. 19). The spatial predicted motion vector candidate derivation unit 321, the temporal predicted motion vector candidate derivation unit 322, the historical predicted motion vector candidate derivation unit 323, and the predicted motion vector candidate supplementation unit 325 in the normal predicted motion vector mode derivation unit 301 derive multiple predicted motion vector candidates to construct the predicted motion vector candidate list mvpListLX. The detailed processing procedure of step S103 in FIG. 19 will be described later using the flowchart in FIG. 20.

Next, the predicted motion vector candidate selection unit 327 selects the predicted motion vector mvpLX of LX from the predicted motion vector candidate list mvpListLX of LX (step S104 in FIG. 19). Each differential motion vector, which is the difference between the motion vector mvLX and each predicted motion vector candidate mvpListLX[i] stored in the predicted motion vector candidate list mvpListLX, is calculated, the amount of code when these differential motion vectors are encoded is calculated for each element of the predicted motion vector candidate list mvpListLX, and the predicted motion vector candidate mvpListLX[i] with the smallest amount of code for each predicted motion vector candidate among the elements registered in the predicted motion vector candidate list mvpListLX is selected as the predicted motion vector mvpLX. If there are multiple motion vector predictor candidates in the motion vector predictor candidate list mvpListLX that result in the smallest amount of generated code, the motion vector predictor candidate mvpListLX[i] represented by the smallest index i in the motion vector predictor candidate list mvpListLX is selected as the optimal motion vector predictor mvpLX.

Then, the motion vector subtraction unit 328 calculates the differential motion vector mvdLX of LX by subtracting the selected predicted motion vector mvpLX of LX from the motion vector mvLX of LX (step S105 in FIG. 19).

(Normal predicted motion vector mode derivation unit (normal AMVP): explanation on the decoding side)
Next, the normal predicted motion vector mode processing procedure on the decoding side will be described with reference to FIG. 25. On the decoding side, the spatial predicted motion vector candidate derivation unit 421, the temporal predicted motion vector candidate derivation unit 422, the history predicted motion vector candidate derivation unit 423, and the predicted motion vector candidate supplement unit 425 calculate the motion vectors used in the inter prediction of the normal predicted motion vector mode for each of L0 and L1 (steps S201 to S206 in FIG. 25). Specifically, when the prediction mode PredMode of the block to be decoded is inter prediction (MODE_INTER) and the inter prediction mode of the block to be decoded is L0 prediction (Pred_L0), the predicted motion vector candidate list mvpListL0 of L0 is calculated, the predicted motion vector mvpL0 is selected, and the motion vector mvL0 of L0 is calculated. When the inter prediction mode of the block to be decoded is L1 prediction (Pred_L1), the predicted motion vector candidate list mvpListL1 of L1 is calculated, the predicted motion vector mvpL1 is selected, and the motion vector mvL1 of L1 is calculated. When the inter prediction mode of the block to be decoded is bi-prediction (Pred_BI), both L0 prediction and L1 prediction are performed, the predicted motion vector candidate list mvpListL0 for L0 is calculated, the predicted motion vector mvpL0 for L0 is selected, and the motion vector mvL0 for L0 is calculated, while the predicted motion vector candidate list mvpListL1 for L1 is calculated, the predicted motion vector mvpL1 for L1 is calculated, and the motion vector mvL1 for L1 is calculated.

As with the encoding side, the decoding side also performs motion vector calculation processing for each of L0 and L1, but the processing is common to both L0 and L1. Therefore, in the following explanation, L0 and L1 are represented as a common LX. In the processing for calculating the motion vector of L0, X is 0, and in the processing for calculating the motion vector of L1, X is 1. Also, when referring to information in the other list instead of LX during the processing for calculating the motion vector of LX, the other list is represented as LY.

When calculating the motion vector mvLX of LX (YES in step S202 in FIG. 25), a candidate predicted motion vector of LX is calculated and a predicted motion vector candidate list mvpListLX of LX is constructed (step S203 in FIG. 25). The spatial predicted motion vector candidate derivation unit 421, the temporal predicted motion vector candidate derivation unit 422, the historical predicted motion vector candidate derivation unit 423, and the predicted motion vector candidate supplementation unit 425 in the normal predicted motion vector mode derivation unit 401 calculate multiple predicted motion vector candidates and construct the predicted motion vector candidate list mvpListLX. The detailed processing procedure of step S203 in FIG. 25 will be described later using the flowchart in FIG. 20.

Then, the motion vector predictor candidate selection unit 426 extracts, from the motion vector predictor candidate list mvpListLX, the motion vector predictor candidate mvpListLX[mvpIdxLX] corresponding to the motion vector predictor index mvpIdxLX decoded and supplied by the bit string decoding unit 201, as the selected motion vector predictor mvpLX (step S204 in FIG. 25).

Then, the motion vector adder 427 calculates the motion vector mvLX of LX by adding the differential motion vector mvdLX of LX decoded and supplied by the bit string decoding unit 201 and the predicted motion vector mvpLX of LX (step S205 in FIG. 25).

<Normal predicted motion vector mode derivation unit (normal AMVP): Motion vector prediction method>
Figure 20 is a flowchart showing the processing steps of a normal prediction motion vector mode derivation process having functions common to the normal prediction motion vector mode derivation unit 301 of the video encoding device and the normal prediction motion vector mode derivation unit 401 of the video decoding device according to an embodiment of the present invention.

The normal prediction motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 401 are provided with a prediction motion vector candidate list mvpListLXN (N is A or B, the same below). The prediction motion vector candidate list mvpListLXN has a list structure, and is provided with a storage area for storing a prediction motion vector index indicating a location within the prediction motion vector candidate list and a prediction motion vector candidate corresponding to the index as an element. The numbers of the prediction motion vector index start from 0, and the prediction motion vector candidates are stored in the storage area of the prediction motion vector candidate list mvpListLXN. In the following processing, the coding block that is the prediction motion vector candidate of the prediction motion vector index i registered in the prediction motion vector candidate list mvpListLXN is represented by mvpListLXN [i], and is distinguished from the prediction motion vector candidate list mvpListLXN by using an array notation. In this embodiment, the prediction motion vector candidate list mvpListLXN can register a maximum of two prediction motion vector candidates (inter prediction information). Furthermore, the variable numMvpCand, which indicates the number of motion vector predictor candidates registered in the motion vector predictor candidate list mvpListLXN, is set to 0.

The spatial prediction motion vector candidate derivation units 321 and 421 derive prediction motion vector candidates from the adjacent coding block on the left, derive a flag availableFlagLXA indicating whether the prediction motion vector candidate of the adjacent coding block on the left is available, a motion vector mvLXA, a reference index refIdxA, and a list ListA, and add mvLXA to the prediction motion vector candidate list mvpListLXA (step S301 in FIG. 20). Note that X is 0 when L0, and 1 when L1 (same below). Next, the motion vector predictor candidate generation units 121 and 221 derive a candidate motion vector predictor from the adjacent coding block above, derive a flag availableFlagLXB indicating whether the motion vector predictor candidate of the coding block adjacent above is available, a motion vector mvLXB, a reference index refIdxB, and a list ListB, and if mvLXA and mvLXB are not equal, add mvLXB to the motion vector predictor candidate list mvpListLXB (step S302 in FIG. 20). The processes of steps S301 and S302 in FIG. 20 are the same except that the positions and number of the adjacent blocks to be referenced are different, and derive a flag availableFlagLXN indicating whether the motion vector predictor candidate of the coding block is available, a motion vector mvLXN, a reference index refIdxN, and ListN (N is A or B, similarly below).

Then, the temporal motion vector predictor candidate derivation units 322 and 422 derive predictor motion vector candidates from the coding blocks of pictures of different times, derive a flag availableFlagLXCol indicating whether the predictor motion vector candidates of the coding blocks of pictures of different times are available, as well as motion vectors mvLXCol, reference indexes refIdxCol, and a list ListCol, and add mvLXCol to the predictor motion vector candidate list mvpListLX (step S303 in FIG. 20). The derivation process procedure of step S303 will be described in detail later.

Here, it is assumed that the processing of the temporal predictor motion vector candidate derivation units 322 and 422 can be omitted in units of sequences (SPS), pictures (PPS), or slices.

Next, the history prediction motion vector candidate derivation units 323 and 423 add the history prediction motion vector candidates registered in the history prediction motion vector candidate list HmvpCandList to the prediction motion vector candidate list mvpListLX (step S304 in FIG. 20). The registration process procedure in step S304 will be described in detail later using the flowchart in FIG. 41.

Then, the motion vector predictor candidate supplementation units 325 and 425 add motion vectors of a predetermined value, such as (0, 0), until the motion vector predictor candidate list mvpListLX is filled (S305 in FIG. 20).

<Normal merge mode derivation unit (normal merge)>
The normal merge mode derivation unit 302 in FIG. 18 includes a spatial merge candidate derivation unit 341 , a temporal merge candidate derivation unit 342 , an average merge candidate derivation unit 344 , a history merge candidate derivation unit 345 , a merge candidate supplementation unit 346 , and a merge candidate selection unit 347 .

The normal merge mode derivation unit 402 in FIG. 24 includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, a merge candidate supplementation unit 446, and a merge candidate selection unit 447.

Figure 21 is a flowchart explaining the steps of a merge mode derivation process having a function common to the normal merge mode derivation unit 302 of the video encoding device and the normal merge mode derivation unit 402 of the video decoding device according to an embodiment of the present invention.

Below, the various steps are explained step by step. Note that in the following explanation, unless otherwise specified, the slice type slice_type is a B slice, but it can also be applied to the case of a P slice. However, when the slice type slice_type is a P slice, only L0 prediction (Pred_L0) is available as an inter prediction mode, and there is no L1 prediction (Pred_L1) or bi-prediction (Pred_BI), so the processing related to L1 can be omitted.

The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 are provided with a merge candidate list mergeCandList. The merge candidate list mergeCandList has a list structure and is provided with a merge index indicating a location within the merge candidate list and a storage area for storing merge candidates corresponding to the index as elements. The merge index number starts from 0, and merge candidates are stored in the storage area of the merge candidate list mergeCandList. In the following processing, the coding block that is the merge candidate of the merge index i registered in the merge candidate list mergeCandList is represented by mergeCandList[i], and is distinguished from the merge candidate list mergeCandList by using an array notation. In this embodiment, the merge candidate list mergeCandList can register up to six merge candidates (inter prediction information). Furthermore, the variable numMergeCand indicating the number of merge candidates registered in the merge candidate list mergeCandList is set to 0.

The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 derive spatial merge candidates A, B, C, D, and E from the coding blocks A, B, C, D, and E adjacent to the block to be coded/decoded from the coding information stored in the coding information storage memory 115 of the video coding device or the coding information storage memory 210 of the video decoding device, and register the derived spatial merge candidates in the merge candidate list mergeCandList (step S401 in FIG. 21). Here, N is defined to indicate either A, B, C, D, E or the temporal merge candidate Col. A flag availableFlagN indicating whether inter prediction information of coding block N can be used as spatial merge candidate N, an L0 reference index refIdxL0N and an L1 reference index refIdxL1N of spatial merge candidate N, an L0 prediction flag predFlagL0N indicating whether L0 prediction is performed and an L1 prediction flag predFlagL1N indicating whether L1 prediction is performed, an L0 motion vector mvL0N, and an L1 motion vector mvL1N are derived. However, in this embodiment, the merge candidate is derived without referring to the coding block included in the same coding block as the coding block including the coding block to be processed, so a spatial merge candidate included in the same coding block as the coding block including the coding block to be processed is not derived.

Then, the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 derive temporal merge candidates from pictures of different times and register the derived temporal merge candidates in the merge candidate list mergeCandList (step S402 in FIG. 21). A flag availableFlagCol indicating whether the temporal merge candidate is available, an L0 prediction flag predFlagL0Col indicating whether L0 prediction of the temporal merge candidate is performed, an L1 prediction flag predFlagL1Col indicating whether L1 prediction is performed, and an L0 motion vector mvL0Col and an L1 motion vector mvL1Col are derived. The detailed processing procedure of step S402 will be described in detail later using the flowcharts in FIG. 44 to FIG. 47.

Here, it is assumed that the processing of the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 can be omitted in units of sequences (SPS), pictures (PPS), or slices.

Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 derive an average merge candidate from the merge candidate list mergeCandList, and register the derived average merge candidate in the merge candidate list mergeCandList (step S403 in FIG. 21). The average merge candidate is derived when the merge candidate list includes two or more candidates.

Next, the history merge candidate derivation unit 345 and the history merge candidate derivation unit 445 add the history prediction motion vector candidates registered in the history prediction motion vector candidate list HmvpCandList to the merge candidate list mergeCandList (step S404 in FIG. 21). The detailed processing procedure of step S404 will be described in detail later using the flowchart in FIG. 41.

Next, in the merge candidate supplementation unit 346 and the merge candidate supplementation unit 446, when the number of merge candidates registered in the merge candidate list mergeCandList, numMergeCand, is smaller than the maximum number of merge candidates, maxNumMergeCand, the number of merge candidates registered in the merge candidate list mergeCandList, numMergeCand, is set to the upper limit of the maximum number of merge candidates, maxNumMergeCand, and additional merge candidates are derived and registered in the merge candidate list mergeCandList (step S405 in FIG. 21). With the maximum number of merge candidates, maxNumMergeCand, a zero merge candidate with a motion vector of (0,0) value with a different reference index and a prediction mode of L0 prediction (Pred_L0) is added to the P slice. A zero merge candidate with a motion vector of (0,0) value with a different reference index and a prediction mode of bi-prediction (Pred_BI) is added to the B slice.

Next, the merge candidate selection unit 347 and the merge candidate supplement unit 447 select a merge candidate from the merge candidates registered in the merge candidate list mergeCandList. The merge candidate selection unit 347 on the encoding side selects a merge candidate by calculating the code amount and distortion amount, and supplies a merge index indicating the selected merge candidate and inter prediction information of the merge candidate to the motion compensation prediction unit 406. On the other hand, the merge candidate supplement unit 447 on the decoding side selects a merge candidate based on the decoded merge index, and supplies the selected merge candidate to the motion compensation prediction unit 406.

Average Merge Candidates
The average merge candidate will be described. The average merge candidate is derived using the first merge candidate and the second merge candidate, which are two merge candidates included in the merge candidate list. The average merge candidate is a merge candidate having an average motion vector of L0 prediction obtained by averaging the motion vector of L0 prediction of the first merge candidate and the motion vector of L0 prediction of the second merge candidate, and an average motion vector of L1 prediction obtained by averaging the motion vector of L1 prediction of the first merge candidate and the motion vector of L1 prediction of the second merge candidate. The reference index of the L0 prediction of the average merge candidate is the reference index of the L0 prediction of the first merge candidate, and the reference index of the L1 prediction of the average merge candidate is the reference index of the L1 prediction of the first merge candidate. In addition, in deriving the average motion vector, the horizontal component and the vertical component of the motion vector are independently averaged. That is, the horizontal component of the average motion vector of the L0 prediction of the average merge candidate is the average of the horizontal components of the motion vector of the L0 prediction of the first merge candidate and the motion vector of the L0 prediction of the second merge candidate, and the vertical component of the average motion vector of the L0 prediction of the average merge candidate is the average of the vertical components of the motion vector of the L0 prediction of the first merge candidate and the motion vector of the L0 prediction of the second merge candidate. The same is true for the average motion vector of the L1 prediction of the average merge candidate.
By deriving the average merge candidate after deriving the history merge candidate, the history merge candidate becomes the target of the average merge candidate, and it becomes possible to derive the average merge candidate using the history merge candidate even if the total number of spatial merge candidates and temporal merge candidates is one or less, thereby improving coding efficiency.

<Sub-block predictive motion vector mode derivation>
Sub-block predicted motion vector mode derivation will now be described.

Figure 26 is a block diagram of the sub-block prediction motion vector mode derivation unit 303 in the encoding device of the present application.

First, the affine inheritance predicted motion vector candidate derivation unit 361 derives an affine inheritance predicted motion vector candidate. The derivation of the affine inheritance predicted motion vector candidate will be described in detail later.

Next, the affine construction predicted motion vector candidate derivation unit 362 derives affine construction predicted motion vector candidates. The derivation of affine construction predicted motion vector candidates will be described in detail later.

Next, the affine-identical predictor motion vector candidate derivation unit 363 derives affine-identical predictor motion vector candidates. The derivation of affine-identical predictor motion vector candidates will be described in detail later.

The sub-block motion vector detection unit 366 detects a sub-block motion vector suitable for the sub-block prediction motion vector mode, and supplies the detected vector to the sub-block prediction motion vector candidate selection unit 367 and the difference calculation unit 368.

The sub-block predictive motion vector candidate selection unit 367 selects a sub-block predictive motion vector candidate from among the sub-block predictive motion vector candidates derived in the affine inheritance predictive motion vector candidate derivation unit 361, the affine construction predictive motion vector candidate derivation unit 362, and the affine identical predictive motion vector candidate derivation unit 363 based on the motion vector supplied from the sub-block motion vector detection unit 366, and supplies information regarding the selected sub-block predictive motion vector candidate to the inter prediction mode determination unit 305 and the difference calculation unit 368.

The difference calculation unit 368 supplies the difference prediction motion vector obtained by subtracting the sub-block prediction motion vector selected by the sub-block prediction motion vector candidate selection unit 367 from the motion vector supplied from the sub-block motion vector detection unit 366 to the inter prediction mode determination unit 305.

Figure 27 is a block diagram of the sub-block prediction motion vector mode derivation unit 403 in the decoding device of the present application.

First, an affine inheritance predicted motion vector candidate derivation unit 461 derives an affine inheritance predicted motion vector candidate. The processing of the affine inheritance predicted motion vector candidate derivation unit 461 is the same as the processing of the affine inheritance predicted motion vector candidate derivation unit 361 in the encoding device of the present application.

Next, an affine construction predicted motion vector candidate derivation unit 462 derives an affine construction predicted motion vector candidate. The processing of the affine construction predicted motion vector candidate derivation unit 462 is the same as the processing of the affine construction predicted motion vector candidate derivation unit 362 in the encoding device of the present application.

Next, an affine-identical predictor motion vector candidate derivation unit 463 derives an affine-identical predictor motion vector candidate. The processing of the affine-identical predictor motion vector candidate derivation unit 463 is the same as the processing of the affine-identical predictor motion vector candidate derivation unit 363 in the encoding device of the present application.

The sub-block predictive motion vector candidate selection unit 467 selects a sub-block predictive motion vector candidate from among the sub-block predictive motion vector candidates derived in the affine inheritance predictive motion vector candidate derivation unit 461, the affine construction predictive motion vector candidate derivation unit 462, and the affine identical predictive motion vector candidate derivation unit 463, based on the predictive motion vector index transmitted from the encoding device and decoded, and supplies information about the selected sub-block predictive motion vector candidate to the motion compensation prediction unit 406 and the addition calculation unit 467.

The addition calculation unit 467 adds the differential motion vector transmitted and decoded from the encoding device to the sub-block predicted motion vector selected by the sub-block predicted motion vector candidate selection unit 466 to generate a motion vector, and supplies the generated motion vector to the motion compensation prediction unit 406.

<Derivation of affine inheritance predicted motion vector candidates>
The following describes the affine inheritance predicted motion vector candidate derivation unit 361. The affine inheritance predicted motion vector candidate derivation unit 461 is similar to the affine inheritance predicted motion vector candidate derivation unit 361.

Affine inheritance predicted motion vector candidates inherit the motion vector information of the affine control points.

Figure 30 is a diagram explaining the derivation of affine inheritance predicted motion vector candidates.

Affine inheritance predicted motion vector candidates are obtained by searching for motion vectors of affine control points of spatially adjacent encoded and decoded blocks.

Specifically, a maximum of one affine mode is searched for from each of the blocks (A0, A1) adjacent to the left side of the block to be encoded/decoded and the blocks (B0, B1, B2) adjacent to the top side of the block to be encoded/decoded, and these are used as the affine inheritance predicted motion vector.

Figure 34 is a flowchart for deriving affine inheritance predicted motion vector candidates.

First, the blocks (A0, A1) adjacent to the left side of the block to be coded/decoded are set as the left group (S3101), and it is determined whether the block including A0 is a block using affine guarantee (affine mode) (S3102). If A0 is in affine mode (S3102: YES), the affine model used by A0 is obtained (S3103), and processing moves to the adjacent block above. If A0 is not in affine mode (S3102: NO), the affine inheritance predicted motion vector candidate derivation target is set to A0->A1, and an attempt is made to obtain the affine mode from the block including A1.

Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be coded/decoded are set as the upper group (S3104), and it is determined whether the block including B0 is in affine mode (S3105). If B0 is in affine mode (S3105: YES), the affine model used by B0 is obtained (S3106), and the process ends. If B0 is not in affine mode (S3105: NO), the target for deriving the affine inheritance predicted motion vector candidate is set to B0->B1, and an attempt is made to obtain the affine mode from the block including B1. Furthermore, if B1 is not in affine mode (S3105: NO), the target for deriving the affine inheritance predicted motion vector candidate is set to B1->B2, and an attempt is made to obtain the affine mode from the block including B2.

In this way, by dividing the blocks into left and top blocks, and searching for affine models for the left blocks from the bottom left to the top left blocks, and for the left blocks from the top right to the top left blocks, it is possible to obtain two affine models that are as different as possible, and to derive an affine prediction motion vector candidate in which one of the affine prediction motion vectors has a small differential motion vector.

<Derivation of affine constructed predicted motion vector candidates>
The following describes the affine construction predicted motion vector candidate derivation unit 362. The affine construction predicted motion vector candidate derivation unit 462 is similar to the affine construction predicted motion vector candidate derivation unit 362.

Affine constructed predicted motion vector candidates construct motion vector information for affine control points from motion information of spatially adjacent blocks.

Figure 31 is a diagram explaining the derivation of affine construction predicted motion vector candidates.

Affine constructed predicted motion vector candidates are obtained by constructing a new affine model by combining the motion vectors of spatially adjacent encoded and decoded blocks.

Specifically, the motion vector of the upper left affine control point CP0 is derived from the block (B2, B3, A2) adjacent to the upper left of the block to be coded/decoded, the motion vector of the upper right affine control point CP1 is derived from the block (B1, B0) adjacent to the upper right of the block to be coded/decoded, and the motion vector of the lower left affine control point CP2 is derived from the block (A1, A0) adjacent to the lower left of the block to be coded/decoded.

Figure 35 is a flowchart for deriving affine construction predicted motion vector candidates.

First, the upper left control point CP0, the upper right control point CP1, and the lower left affine control point CP2 are derived (S3201). The upper left affine control point CP0 is calculated by searching for reference blocks having the same reference image as the block to be coded/decoded, with the priority order of B2, B3, A2 reference blocks. The upper right affine control point CP1 is calculated by searching for reference blocks having the same reference image as the block to be coded/decoded, with the priority order of B1, B0 reference blocks. The lower left affine control point CP2 is calculated by searching for reference blocks having the same reference image as the block to be coded/decoded, with the priority order of A1, A0 reference blocks.

When the three affine control points mode is selected as the affine construction predicted motion vector (S3202: YES), it is determined whether all three affine control points (CP0, CP1, CP2) have been derived (S3203). When all three affine control points (CP0, CP1, CP2) have been derived (S3203: YES), an affine model using the three affine control points (CP0, CP1, CP2) is set as the affine construction predicted motion vector (S3204). When the three affine control points mode is not selected and the two affine control points mode is selected (S3202: NO), it is determined whether all two affine control points (CP0, CP1) have been derived (S3205). If the two affine control points (CP0, CP1) have all been derived (S3205: YES), an affine model using the two affine control points (CP0, CP1) is set as the affine constructed predicted motion vector (S3206).

<Derivation of Affine Same Predictor Motion Vector Candidates>
The affine co-predictor motion vector candidate derivation unit 363 will now be described. The affine co-predictor motion vector candidate derivation unit 463 is similar to the affine co-predictor motion vector candidate derivation unit 363.

Affine identical predictor motion vector candidates are obtained by deriving identical motion vectors at each affine control point.

Specifically, similar to the affine construction predicted motion vector candidate derivation units 362 and 462, the affine control point information is derived and all affine control points are set to the same value, either CP0 or CP2, to obtain the affine control vector. Also, similar to the normal predicted motion vector mode, the affine control vector can be obtained by setting the derived temporal motion vector to all affine control points.

<Sub-block merge mode derivation>
Sub-block merge mode derivation will now be described.

Figure 28 is a block diagram of the subblock merge mode derivation unit 304 in the encoding device of the present application. The subblock merge mode derivation unit 304 has a subblock merge candidate list subblockMergeCandList. This is similar to the merge candidate list mergeCandList in the normal merge mode derivation unit 302, and differs only in that the candidate list is different for each subblock.

First, the sub-block time merging candidate derivation unit 381 derives sub-block time merging candidates. The derivation of sub-block time merging candidates will be described in detail later.

Next, the affine inheritance merge candidate derivation unit 382 derives affine inheritance merge candidates. Details of affine inheritance merge candidate derivation will be described later.

Next, the affine construction merge candidate derivation unit 383 derives affine construction merge candidates. Details of the derivation of affine construction merge candidates will be described later.

Next, the affine-fixed merge candidate derivation unit 384 derives affine-fixed merge candidates. Details of affine-fixed merge candidate derivation will be described later.

The subblock merging candidate selection unit 386 selects a subblock merging candidate from among the subblock merging candidates derived in the subblock temporal merging candidate derivation unit 381, the affine inheritance merging candidate derivation unit 382, the affine construction merging candidate derivation unit 383, and the affine fixed merging candidate derivation unit 384, and supplies information about the selected subblock merging candidate to the inter prediction mode determination unit 305.

Figure 29 is a block diagram of the subblock merge mode derivation unit 404 in the decoding device of the present application. The subblock merge mode derivation unit 404 has a subblock merge candidate list subblockMergeCandList. This is the same as the subblock merge mode derivation unit 304.

First, the sub-block time merge candidate derivation unit 481 derives sub-block time merge candidates. The processing of the sub-block time merge candidate derivation unit 481 is the same as the processing of the sub-block time merge candidate derivation unit 381.

Next, the affine inheritance merge candidate derivation unit 482 derives affine inheritance merge candidates. The processing of the affine inheritance merge candidate derivation unit 482 is the same as the processing of the affine inheritance merge candidate derivation unit 382.

Next, the affine construct merge candidate derivation unit 483 derives affine construct merge candidates. The processing of the affine construct merge candidate derivation unit 483 is the same as the processing of the affine construct merge candidate derivation unit 383.

Next, the affine-fixed merge candidate derivation unit 485 derives affine-fixed merge candidates. The processing of the affine-fixed merge candidate derivation unit 485 is the same as the processing of the affine-fixed merge candidate derivation unit 485.

The subblock merging candidate selection unit 486 selects a subblock merging candidate from among the subblock merging candidates derived in the subblock temporal merging candidate derivation unit 481, the affine inheritance merging candidate derivation unit 482, the affine construction merging candidate derivation unit 483, and the affine fixed merging candidate derivation unit 484 based on an index transmitted from the encoding device and decoded, and supplies information about the selected subblock merging candidate to the motion compensation prediction unit 406.

<Sub-block Temporal Merging Candidate Derivation>
The operation of the sub-block temporal merging candidate derivation unit 381 will be described later.

<Affine inheritance merge candidate derivation>
The following describes the affine inheritance merge candidate derivation unit 382. The affine inheritance merge candidate derivation unit 482 is similar to the affine inheritance merge candidate derivation unit 382.

Affine inheritance merge candidate inherits the affine model of the affine control points from the affine model of the spatially adjacent block.

Figure 32 is a diagram explaining the derivation of affine inheritance merge candidates. Similar to the derivation of affine inheritance prediction motion vectors, affine merge inheritance merge mode candidates are derived by searching for motion vectors of affine control points of spatially adjacent encoded and decoded blocks.

Specifically, a maximum of one affine mode is searched for from each of the blocks (A0, A1) adjacent to the left side of the block to be encoded/decoded and the blocks (B0, B1, B2) adjacent to the top side of the block to be encoded/decoded, and used for the affine merge mode.

Figure 36 is a flowchart for deriving affine inheritance merge candidates.

First, the blocks (A0, A1) adjacent to the left side of the block to be encoded/decoded are set as the left group (S3301), and it is determined whether the block including A0 is in affine mode (S3302). If A0 is in affine mode (S3102: YES), the affine model used by A0 is obtained (S3303), and processing moves to the adjacent block above. If A0 is not in affine mode (S3302: NO), the target for deriving affine inheritance merge candidates is set to A0->A1, and an attempt is made to obtain the affine mode from the block including A1.

Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be encoded/decoded are set as the upper group (S3304), and it is determined whether the block including B0 is in affine mode (S3305). If B0 is in affine mode (S3305: YES), the affine model used by B0 is obtained (S3306), and the process ends. If B0 is not in affine mode (S3305: NO), the target for deriving affine inheritance merge candidates is set to B0->B1, and an attempt is made to obtain the affine mode from the block including B1. Furthermore, if B1 is not in affine mode (S3305: NO), the target for deriving affine inheritance merge candidates is set to B1->B2, and an attempt is made to obtain the affine mode from the block including B2.

<Affine Construction Merge Candidate Derivation>
The affine construct merge candidate derivation unit 383 will now be described. The affine construct merge candidate derivation unit 483 is similar to the affine construct merge candidate derivation unit 383.

Figure 33 is a diagram explaining the derivation of affine construction merge candidates. Affine construction merge candidates construct an affine model of affine control points from motion information of spatially adjacent blocks and temporal coding blocks.

Specifically, the motion vector of the upper-left affine control point CP0 is derived from the block (B2, B3, A2) adjacent to the upper left of the block to be coded/decoded, the motion vector of the upper-right affine control point CP1 is derived from the block (B1, B0) adjacent to the upper right of the block to be coded/decoded, the motion vector of the lower-left affine control point CP2 is derived from the block (A1, A0) adjacent to the lower left of the block to be coded/decoded, and the motion vector of the lower-right affine control point CP3 is derived from the temporal coding block (T0) adjacent to the lower right of the block to be coded/decoded.

Figure 37 is a flowchart for deriving affine construction merge candidates.

First, the upper left affine control point CP0, the upper right affine control point CP1, the lower left affine control point CP2, and the lower right affine control point CP3 are derived (S3401). The upper left control point CP0 is calculated by searching for blocks having motion information in the order of priority: B2, B3, A2 blocks. The upper right control point CP1 is calculated by searching for blocks having motion information in the order of priority: B1, B0 blocks. The lower left control point CP2 is calculated by searching for blocks having motion information in the order of priority: A1, A0 blocks. The lower right control point CP3 is calculated by searching for motion information of time blocks.

Next, it is determined whether an affine model can be constructed using the derived three affine control points CP0, CP1, and CP2 (S3402), and if so (S3402: YES), the three affine control point affine model using CP0, CP1, and CP2 is set as an affine merge candidate (S3403).

Next, it is determined whether an affine model can be constructed using the derived three affine control points CP0, CP1, and CP3 (S3404), and if so (S3404: YES), the three affine control point affine model using CP0, CP1, and CP3 is set as an affine merge candidate (S3405).

Next, it is determined whether an affine model can be constructed using the derived three affine control points CP0, CP2, and CP3 (S3406), and if so (S3406: YES), the three affine control point affine model using CP0, CP2, and CP3 is set as an affine merge candidate (S3407).

Next, it is determined whether an affine model can be constructed using the three affine control points CP1, CP2, and CP3 derived (S3408), and if so (S3408: YES), the three affine control point affine model using CP1, CP2, and CP3 is set as an affine merge candidate (S3409).

Next, it is determined whether an affine model can be constructed using two affine control points from the derived CP0 and CP1 (S3410), and if so (S3410: YES), the two affine control point affine model using CP0 and CP1 is set as an affine merge candidate (S3411).

Next, it is determined whether an affine model can be constructed using two affine control points from the derived CP0 and CP2 (S3412), and if so (S3412: YES), the two affine control point affine model using CP0 and CP2 is set as an affine merge candidate (S3413).

Here, whether or not an affine model is constructed is determined based on the following conditions:

1. The reference images for all affine control points are the same. (Affine transformation is possible)
2. At least one affine control point has a different motion vector (cannot be expressed by translation).
In addition, for affine models other than the three-line affine control point affine model using CP0, CP1, and CP2 and the two-line affine control point affine model using CP0 and CP1, the three-line control affine model is converted to a three-line affine control point affine model using CP0, CP1, and CP2, and the two-line control affine model is converted to a two-line affine control point affine model using CP0 and CP1.

<Affine fixed merge candidate derivation>
The affine-anchored merging candidate derivation unit 385 will now be described. The affine-anchored merging candidate derivation unit 485 is similar to the affine-anchored merging candidate derivation unit 385.

Affine fixed merge candidates fix the motion information of affine control points with fixed motion information.

Specifically, the motion vector of each affine control point is fixed to (0,0).

<Temporal Predictive Motion Vector>
Prior to the description of the temporal prediction motion vector, the temporal relationship of pictures will be described. Figure 49 (a) shows the relationship between a coding block to be coded and a coded picture that is temporally different from the coding target picture. In the coding target picture, a specific coded picture to be referenced for coding is defined as ColPic. ColPic is specified by syntax.

Figure 49(b) also shows coded blocks in ColPic that are in the same position as the coding block to be coded and in its vicinity. These coding blocks T0 and T1 are coding blocks in almost the same position in a picture that is temporally different from the coding block to be coded.

The above explanation of the temporal relationship of pictures is for encoding, but it is the same for decoding. In other words, the above explanation for decoding can be applied in the same way, with the word "encoding" substituted for "decoding."

The operation of the temporal prediction motion vector candidate derivation unit 322 in the normal prediction motion vector mode derivation unit 301 in FIG. 17 will be described with reference to FIG. 50.

First, ColPic is derived (step S4201). The derivation of ColPic is explained with reference to FIG. 51.

If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0 (YES in step S4211, YES in step S4212), RefPicList1[0], i.e., the picture with a reference index of 0 in reference list L1, becomes a picture colPic of a different time (step S4213). Otherwise, i.e., if the slice type slice_type is a B slice and the above-mentioned flag collocated_from_l0_flag is 1 (YES in step S4211, NO in step S4212), or if the slice type slice_type is a P slice (NO in step S4211, YES in step S4214), RefPicList0[0], i.e., the picture with a reference index of 0 in reference list L0, becomes a picture colPic of a different time (step S4215). If slice_type is not a P slice (NO in step S4214), the process ends.

Refer to FIG. 50 again. After deriving ColPic, derive the coding block colCb and obtain the coding information (step S4202). This process will be described with reference to FIG. 52.

First, the coding block located at the lower right (outside) of the same position as the coding block to be processed in the picture colPic of a different time is set as the coding block colCb of a different time (step S4221). This coding block corresponds to coding block T0 in FIG. 49.

Next, the coding information of the coding block colCb of the different time is obtained (step S4222). If the PredMode of the coding block colCb of the different time is unavailable or the prediction mode PredMode of the coding block colCb of the different time is intra prediction (MODE_INTRA) (step S4223: NO, step S4224: YES), the coding block located in the lower right corner of the same position as the coding block to be processed in the picture colPic of the different time is set as the coding block colCb of the different time (step S4225). This coding block corresponds to the coding block T1 in Figure 49.

Refer to FIG. 50 again. Next, inter prediction information is derived for each reference list (S4203, S4204). Here, for the coding block colCb, a motion vector mvLXCol for each reference list and a flag availableFlagLXCol indicating whether the coding information is valid or not are derived. LX indicates a reference list, and when deriving reference list 0, LX is L0, and when deriving reference list 1, LX is L1. The derivation of inter prediction information will be described with reference to FIG. 53.

If the coding block colCb of a different time is not available (S4231S4231: NO), or if the prediction mode PredMode is intra prediction (MODE_INTRA) (S4232: NO), the flag availableFlagLXCol and the flag predFlagLXCol are both set to 0 (step S4233), the motion vector mvLXCol is set to (0, 0) (S4234), and the processing ends.

If the coding block colCb is available (S4231: Yes) and the prediction mode PredMode is not intra prediction (MODE_INTRA) (S4232: YES), mvCol, refIdxCol, and availableFlagCol are calculated using the following procedure.

When the flag PredFlagL0[xPCol][yPCol] indicating whether L0 prediction is used for the coding block colCb is 0 (YES in S4235), the prediction mode of the coding block colCb is Pred_L1, so the motion vector mvCol is set to the same value as the L1 motion vector MvL1[xPCol][yPCol] of the coding block colCb (S4236), the reference index refIdxCol is set to the same value as the L1 reference index RefIdxL1[xPCol][yPCol] (S4237), and the list ListCol is set to L1 (S4238). Here, xPCol and yPCol are indexes indicating the position of the top left pixel of the coding block colCb in a picture colPic of a different time.

On the other hand, if the L0 prediction flag PredFlagL0[xPCol][yPCol] of the coding block colCb is not 0 (NO in S4235), it is determined whether the L1 prediction flag PredFlagL1[xPCol][yPCol] of the coding block colCb is 0. If the L1 prediction flag PredFlagL1[xPCol][yPCol] of the coding block colCb is 0 (YES in S4239), the motion vector mvCol is set to the same value as the L0 motion vector MvL0[xPCol][yPCol] of the coding block colCb (S4240), the reference index refIdxCol is set to the same value as the L0 reference index RefIdxL0[xPCol][yPCol] (S4241), and the list ListCol is set to L0 (S4242).

If the L0 prediction flag PredFlagL0[xPCol][yPCol] of the coding block colCb and the L1 prediction flag PredFlagL1[xPCol][yPCol] of the coding block colCb are both not 0 (NO in S4235, NO in S4239), the inter prediction mode of the coding block colCb is bi-predictive (Pred_BI), so one of the two motion vectors L0 and L1 is selected (S4243).

Figure 54 is a flowchart showing the process steps for deriving inter prediction information for a coding block when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI).

First, it is determined whether the POC of all pictures registered in all reference lists is smaller than the POC of the current picture to be coded (S4251). If the POC of all pictures registered in all reference lists, L0 and L1, of the coding block colCb is smaller than the POC of the current picture to be coded (YES in S4251), and LX has derived a prediction vector candidate for the motion vector of L0, i.e., L0 of the coding block to be coded (YES in S4252), the inter prediction information of L0 of the coding block colCb is selected. If LX has derived a prediction vector candidate for the motion vector of L1, i.e., L1 of the coding block to be coded (NO in S4252), the inter prediction information of L1 of the coding block colCb is selected. On the other hand, if at least one of the POCs of the pictures registered in all reference lists L0 and L1 of the coding block colCb is larger than the POC of the current picture to be coded (NO in S4251), and if the flag collocated_from_l0_flag is 0 (YES in S4253), the inter prediction information of L0 of the coding block colCb is selected, and if the flag collocated_from_l0_flag is 1 (NO in S4253), the inter prediction information of L1 of the coding block colCb is selected.

When selecting inter prediction information for L0 of coding block colCb (YES in S4252, YES in S4253), the motion vector mvCol is set to the same value as MvL0[xPCol][yPCol] (S4254), the reference index refIdxCol is set to the same value as RefIdxL0[xPCol][yPCol] (S4255), and the list ListCol is set to L0 (S4256).

When selecting inter prediction information for L1 of the coding block colCb (NO in S4252, NO in S4253), the motion vector mvCol is set to the same value as MvL1[xPCol][yPCol] (S4257), the reference index refIdxCol is set to the same value as RefIdxL1[xPCol][yPCol] (S4258), and the list ListCol is set to L1 (S4259).

Returning to FIG. 53, once inter prediction information has been obtained from the coding block colCb, the flag availableFlagLXCol and the flag predFlagLXCol are both set to 1 (S4244).

Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (S4245S4245). The procedure for the scaling calculation process of this motion vector mvLXCol will be explained using FIG. 55.

The inter-picture distance td is calculated by subtracting the POC of the reference picture corresponding to the reference index refIdxCol referenced in the list ListCol of the coding block colCb from the POC of the picture colPic of the different time (S4261). Note that if the POC of the reference picture referenced in the list ListCol of the coding block colCb is earlier in the display order than the picture colPic of the different time, the inter-picture distance td is a positive value, and if the POC of the reference picture referenced in the list ListCol of the coding block colCb is later in the display order than the picture colPic of the different time, the inter-picture distance td is a negative value.
td = POC of a picture colPic at a different time - POC of a reference picture referenced in the list ListCol of coding blocks colCb
The POC of the reference picture referred to by the list LX of the current picture to be coded is subtracted from the POC of the current picture to be coded to calculate the inter-picture distance tb (S4262). Note that if the reference picture referred to by the list LX of the current picture to be coded is earlier in display order than the current picture to be coded, the inter-picture distance tb is a positive value, and if the reference picture referred to by the list LX of the current picture to be coded is later in display order, the inter-picture distance tb is a negative value.
tb = POC of the current picture to be coded/decoded - POC of the reference picture corresponding to the reference index of the LX of the temporal merge candidate
Next, the inter-picture distances td and tb are compared (S4263), and if the inter-picture distances td and tb are equal (YES at S4263), a motion vector mvLXCol is calculated by the following equation (S4264), and this scaling calculation process is terminated.
mvLXCol = mvCol
On the other hand, if the inter-picture distances td and tb are not equal (NO at S4263), the variable tx is calculated by the following formula (S4265).
tx = ( 16384 + Abs( td ) >> 1 ) / td
Next, the scaling coefficient distScaleFactor is calculated using the following equation (S4266).
distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 )
Here, Clip3(x, y, z) is a function that limits the value z to a minimum value of x and a maximum value of y. Next, a motion vector mvLXCol is calculated by the following equation (S4267), and this scaling calculation process ends.
mvLXCol = Clip3( -32768, 32767, Sign( distScaleFactor * mvLXCol )
* ((Abs( distScaleFactor * mvLXCol ) + 127 ) >> 8 ) )
Here, Sign(x) is a function that returns the sign of the value x, and Abs(x) is a function that returns the absolute value of the value x.

Refer to FIG. 50 again. Then, the motion vector mvL0Col of L0 is added as a candidate to the predicted motion vector candidate list mvpListLXN in the normal predicted motion vector mode derivation unit 301 described above (S4205). However, this addition is only made when the flag availableFlagL0Col=1 indicating whether the coding block colCb of reference list 0 is valid or not. Furthermore, the motion vector mvL1Col of L1 is added as a candidate to the predicted motion vector candidate list mvpListLXN in the normal predicted motion vector mode derivation unit 301 described above (S4205). However, this addition is only made when the flag availableFlagL1Col=1 indicating whether the coding block colCb of reference list 1 is valid or not. With the above, the processing of the temporal predicted motion vector candidate derivation unit 322 is terminated.

The above description of the normal prediction motion vector mode derivation unit 301 is for encoding, but it is similar for decoding. In other words, the operation of the temporal prediction motion vector candidate derivation unit 422 in the normal prediction motion vector mode derivation unit 401 in FIG. 23 can be similarly described by replacing encoding in the above description with decoding.

<Time Merge>
The operation of the temporal merge candidate derivation unit 342 in the normal merge mode derivation unit 302 in FIG. 18 will be described with reference to FIG.

First, ColPic is derived (step S4301). Next, the coding block colCb is derived and coding information is obtained (step S4302). Furthermore, inter prediction information is derived for each reference list (S4303, S4304). The above processing is the same as S4201 to S4204 in the temporal motion vector predictor candidate derivation unit 322, and therefore a description thereof will be omitted.

Next, the flag availableFlagCol indicating whether the coding block colCb is valid or not is calculated (S4305). If the flag availableFlagL0Col or the flag availableFlagL1Col is 1, then availableFlagCol is 1. In any other case, availableFlagCol is 0.

Then, the L0 motion vector mvL0Col and the L1 motion vector mvL1Col are added as candidates to the merge candidate list mergeCandList in the normal merge mode derivation unit 302 described above (S4306). However, this addition is only made when the flag indicating whether the coding block colCb is valid or not is availableFlagCol=1. This ends the processing of the temporal merge candidate derivation unit 342.

The above description of the temporal merge candidate derivation unit 342 is for encoding, but the same applies for decoding. In other words, the operation of the temporal merge candidate derivation unit 442 in the normal merge mode derivation unit 402 in FIG. 24 can be similarly described by replacing encoding in the above description with decoding.

<Updating the Historical Motion Vector Predictor Candidate List>
Next, a detailed description will be given of a method for updating the historical motion vector predictor candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the decoding information storage memory 205. Fig. 38 is a flowchart illustrating a historical motion vector predictor candidate derivation process procedure.

In this embodiment, the update of the history prediction motion vector candidate list HmvpCandList is performed in the coding information storage memory 111 and the coding information storage memory 205. A history candidate list update unit may be provided in the inter prediction unit 102 and the inter prediction unit 203 to update the history prediction motion vector candidate list HmvpCandList.

The historical prediction motion vector candidate list HmvpCandList is initially set at the beginning of the slice, and on the encoding side, the historical prediction motion vector candidate list HmvpCandList is updated when the normal prediction vector mode or normal merge mode is selected by the prediction method determination unit 106, and on the decoding side, the historical prediction motion vector candidate list HmvpCandList is updated when the inter prediction mode decoded by the bit string decoding unit 201 is the normal prediction vector mode or the normal merge mode. The inter prediction information used when performing inter prediction in the normal prediction vector mode or normal merge mode is set as the inter prediction information candidate hMvpCand to be registered, and if an inter prediction with the same value as the inter prediction information candidate hMvpCand to be registered exists among the inter prediction information registered in the historical prediction motion vector candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side, the element (inter prediction information) is deleted from the historical prediction motion vector candidate list HmvpCandList, and if an inter prediction with the same value as the inter prediction information candidate hMvpCand to be registered does not exist, the first element (inter prediction information) of the historical prediction motion vector candidate list HmvpCandList is deleted, and the inter prediction information candidate hMvpCand to be registered is added to the end of the historical prediction motion vector candidate list HmvpCandList.

In the present invention, the number of elements in the historical predicted motion vector candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side is set to 6.

First, the historical prediction motion vector candidate list HmvpCandList is initialized on a slice-by-slice basis. Historical prediction motion vector candidates are added to all elements of the historical prediction motion vector candidate list HmvpCandList at the beginning of the slice, and the value of the number of historical prediction motion vector candidates NumHmvpCand registered in the historical prediction motion vector candidate list HmvpCandList is set to 6 (step S2101 in FIG. 38).

Here, the initialization of the historical predicted motion vector candidate list HmvpCandList is performed on a slice-by-slice basis (the first coding block of the slice), but it can also be performed on a picture-by-picture, tile-by-tile, or tree block row-by-tree block basis.

Figure 62 is a table showing an example of a history prediction motion vector candidate added by initializing the history prediction motion vector candidate list HmvpCandList. An example is shown in which the slice type is a B slice and the number of reference pictures is 4. Inter prediction information with a history prediction motion vector index ranging from (the number of history prediction motion vector candidates NumHmvpCand-1) to 0 and a motion vector value of (0, 0) according to the slice type is added to the history prediction motion vector candidate list HmvpCandList as a history prediction motion vector candidate, and the history prediction motion vector candidate list is filled with history candidates. At this time, the history prediction motion vector index starts from (the number of history prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1). After that, overlapping of history prediction motion vector candidates is permitted, and refIdxLX is set to a value of 0. By setting all values to the number of historical motion vector predictor candidates NumHmvpCand and setting the value of the number of historical motion vector predictor candidates NumHmvpCand to a fixed value, invalid historical motion vector predictor candidates are eliminated. In this way, by assigning small reference index values, which generally have a high selection rate, to candidates with large historical motion vector predictor index values that are likely to be added to the motion vector predictor candidate list or merge candidate list, it is possible to improve coding efficiency.

In addition, by filling the history prediction motion vector candidate list with history prediction motion vector candidates on a slice-by-slice basis, the number of history prediction motion vector candidates can be treated as a fixed value, which can simplify, for example, the history prediction motion vector candidate derivation process and the history merge candidate derivation process.

Here, the value of the motion vector is set to (0, 0), which is generally selected with a high probability, but any predetermined value may be used. For example, values such as (4, 4), (0, 32), and (-128, 0) may be used to improve the coding efficiency of the differential motion vector, or multiple predetermined values may be set to improve the coding efficiency of the differential motion vector.

In addition, although the historical prediction motion vector index is set to (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), the historical prediction motion vector index may also start from 0.

Figure 63 is a table showing another example of a history prediction motion vector candidate added by initializing the history prediction motion vector candidate list HmvpCandList. An example is shown in which the slice type is a B slice and the number of reference pictures is two. In this example, inter-prediction information with different reference indexes or motion vector values is added as a history prediction motion vector candidate to fill the history prediction motion vector candidate list so that there is no overlap between the history prediction motion vector candidates in each element of the history prediction motion vector candidate list HmvpCandList. At this time, the history prediction motion vector index starts from (the number of history prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (X is 0 or 1) is set to a value incremented by one from 0 to (the number of reference pictures numRefIdx-1). After that, a motion vector with a different value, 0, is added as a history prediction motion vector candidate to refIdxLX. By setting all values to the number of historical prediction motion vector candidates NumHmvpCand and setting the value of the number of historical prediction motion vector candidates NumHmvpCand to a fixed value, invalid historical prediction motion vector candidates are eliminated.

In this way, by filling the history prediction motion vector candidate list with non-overlapping history prediction motion vector candidates on a slice-by-slice basis, it is possible to omit the processing of the merge candidate supplementation unit 346 subsequent to the history merge candidate derivation unit 345 in the normal merge mode derivation unit 302 described below, which is performed on an encoding block basis, thereby reducing the amount of processing.

Here, the motion vector values are small values such as (0,0) or (1,0), but the motion vector values may be large as long as there is no overlap between historical predicted motion vector candidates.

In addition, although the historical prediction motion vector index is set to (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), the historical prediction motion vector index may also start from 0.

Figure 64 is a table showing another example of historical predicted motion vector candidates added by initializing the historical predicted motion vector candidate list HmvpCandList.

An example is shown for the case where the slice type is a B slice. In this example, inter-prediction information with a reference index of 0 and different motion vector values is added as a history prediction motion vector candidate to fill the history prediction motion vector candidate list so that there is no overlap between history prediction motion vector candidates in each element of the history prediction motion vector candidate list HmvpCandList. At this time, the history prediction motion vector index starts from (the number of history prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (X is 0 or 1) is set to 0. All values are set in the number of history prediction motion vector candidates NumHmvpCand, and the value of the number of history prediction motion vector candidates NumHmvpCand is set to a fixed value, thereby eliminating invalid history prediction motion vector candidates.

In this way, by setting the reference index to 0, the process can be further simplified since initialization can be performed without taking into account the number of reference pictures.

Here, the motion vector value is a multiple of 2, but other values are acceptable as long as the reference index is 0 and there is no overlap between historical motion vector predictor candidates.

In addition, although the historical prediction motion vector index is set to (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), the historical prediction motion vector index may also start from 0.

Then, the following update process of the historical predicted motion vector candidate list HmvpCandList is repeated for each coding block in the slice (steps S2102 to S2111 in FIG. 38).

First, initial settings are made on a coding block basis. The flag identicalCandExist, which indicates whether an identical candidate exists, is set to FALSE, and the deletion target index removeIdx is set to 0 (step S2103 in FIG. 38).

It is determined whether the inter prediction information candidate hMvpCand to be registered exists in the history prediction motion vector candidate list HmvpCandList (step S2104 in FIG. 38). If the prediction method determination unit 105 on the encoding side determines that the mode is the normal prediction motion vector mode or the normal merge mode, or if the bit string decoding unit on the decoding side decodes the normal prediction motion vector mode or the normal merge mode, the inter prediction mode is set to hMvpCand. If the prediction method determination unit 105 on the encoding side determines that the mode is the intra prediction mode, the sub-block prediction motion vector mode, or the sub-block merge mode, or if the bit string decoding unit on the decoding side decodes the intra prediction mode, the sub-block prediction motion vector mode, or the sub-block merge mode, the update process of the history prediction motion vector candidate list HmvpCandList is not performed, and the inter prediction information candidate hMvpCand to be registered does not exist. If the inter prediction information candidate hMvpCand to be registered does not exist, steps S2105 to S2110 are skipped (NO in step S2104 in FIG. 38). If there is an inter prediction information candidate hMvpCand to be registered, perform the processing from step S2105 onwards (YES in step S2104 of FIG. 38).

Next, it is determined whether or not there is an element identical to the inter-prediction information candidate hMvpCand to be registered among the elements of the history prediction motion vector candidate list HmvpCandList (step S2105 in FIG. 38). FIG. 39 is a flowchart of this identical element confirmation process procedure. If the value of the number of history prediction motion vector candidates NumHmvpCand is 0 (NO in step S2121 in FIG. 39), the history prediction motion vector candidate list HmvpCandList is empty and there is no identical candidate, so steps S2122 to S2125 in FIG. 39 are skipped and this identical element confirmation process procedure ends. If the value of the number of history prediction motion vector candidates NumHmvpCand is greater than 0 (YES in step S2121 in FIG. 39), the processes of steps S2122 to S2125 are repeated for the history prediction motion vector index hMvpIdx from 0 to NumHmvpCand-1 (steps S2121 to S2125 in FIG. 39). First, the hMvpIdx-th element HmvpCandList[MvpIdx] counting from 0 in the history predicted motion vector candidate list is compared with the inter prediction information candidate hMvpCand to see if it is the same (step S2123 in FIG. 39). If they are the same (YES in step S2123 in FIG. 39), a flag identicalCandExist indicating whether an identical candidate exists is set to TRUE, the deletion target index removeIdx is set to the value of hMVpIndex, and this identical element confirmation process ends. If they are not the same (NO in step S2123 in FIG. 39), hMvpIdx is incremented by 1 (steps S2121 and S2125 in FIG. 39), and the processes from step S2123 onwards are performed.

Here, by filling the history prediction motion vector candidate list with history prediction motion vector candidates, step S2121 in FIG. 39 can be omitted.

Returning to the flowchart of FIG. 38 again, the process of shifting and adding elements of the historical prediction motion vector candidate list HmvpCandList is performed (step S2106 of FIG. 38). FIG. 40 is a flowchart of the process procedure of shifting/adding elements of the historical prediction motion vector candidate list HmvpCandList in step S2106 of FIG. 38. First, it is determined whether to remove elements stored in the historical prediction motion vector candidate list HmvpCandList and then add a new element, or to add a new element without removing elements. Specifically, it is compared whether the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE (true) or NumHmvpCand is 6 (step S2141 of FIG. 40). If either of the conditions of the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE (true) or NumHmvpCand is 6 is satisfied (YES in step S2141 of FIG. 40), the elements stored in the historical prediction motion vector candidate list HmvpCandList are removed and then a new element is added. The initial value of index i is set to the value of removeIdx + 1. The element shift process of step S2143 is repeated from this initial value to NumHmvpCand (steps S2142 to S2144 in FIG. 40). The elements of HMVPCandList[i] are shifted forward by copying them to HMVPCandList[i - 1] (step S2143 in FIG. 40), and i is incremented by 1 (steps S2142 and S2145 in FIG. 40). Next, the inter prediction information candidate hMvpCand is added to the (NumHmvpCand-1)-th HMVPCandList[NumHmvpCand-1] counting from 0, which corresponds to the end of the history prediction motion vector candidate list (step S2145 in FIG. 40), and the element shift/addition process of this history prediction motion vector candidate list HMVPCandList is terminated. On the other hand, if neither of the conditions that the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE and NumHmvpCand is 6 is satisfied (NO in step S2141 in FIG. 40), a new element is added without removing elements stored in the historical motion vector predictor candidate list HmvpCandList. The inter prediction information candidate hMvpCand is added to the (NumHmvpCand-1)th HMVPCandList[NumHmvpCand] counting from 0, which corresponds to the end of the historical motion vector predictor candidate list, and NumHmvpCand is incremented by 1 (step S2145 in FIG. 40), and the element shift/addition process of this historical motion vector predictor candidate list HMVPCandList is terminated.

Here, the historical motion vector predictor candidate list is assumed to apply to both the motion vector predictor mode and the merge mode, but it may also be applied to only one of them.

As described above, when updating the historical motion vector prediction candidate list, identical elements stored in the historical motion vector prediction candidate list are removed before new elements are added, so there are no duplicate elements in the historical motion vector prediction candidate list, and the historical motion vector prediction candidate list is composed of all different elements.

<Historical Prediction Motion Vector Candidate Derivation Process>
Next, a detailed description will be given of a method for deriving a historical prediction motion vector candidate from the historical prediction motion vector candidate list HMVPCandList, which is a processing procedure of step S304 in Fig. 20 and is common to the historical prediction motion vector candidate derivation unit 323 of the normal prediction motion vector mode derivation unit 301 on the encoding side and the historical prediction motion vector candidate derivation unit 423 of the normal prediction motion vector mode derivation unit 401 on the decoding side. Fig. 41 is a flowchart illustrating the historical prediction motion vector candidate derivation processing procedure.

If the number of current motion vector predictor candidates numCurrMvpCand is equal to or greater than the maximum number of elements in the motion vector predictor candidate list (here, 2) or the number of historical motion vector predictor candidates is equal to 0 (NO in step S2201 in FIG. 41), steps S2202 to S2208 in FIG. 41 are omitted, and the historical motion vector predictor candidate derivation process procedure is terminated. If numCurrMvpCand is smaller than 2, which is the maximum number of elements in the motion vector predictor candidate list (YES in step S2201 in FIG. 41), steps S2202 to S2208 in FIG. 41 are performed.

Next, the process of steps S2203 to S2207 in FIG. 41 is repeated for index i from 1 to the smaller of 4 and NumHmvpCand (steps S2202 to S2208 in FIG. 41). If numCurrMvpCand is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list (NO in step S2203 in FIG. 41), the process of steps S2204 to S2208 in FIG. 41 is omitted, and this history motion vector predictor candidate derivation process procedure is terminated. If numCurrMvpCand is smaller than 2, which is the maximum number of elements in the motion vector predictor candidate list (YES in step S2203 in FIG. 41), the process of step S2204 and subsequent steps in FIG. 41 is performed.

Next, the processes from step S2205 to S2206 are performed for variables Y of 0 and 1 (L0 and L1), respectively (steps S2204 to S2207 in FIG. 41). If numCurrMvpCand is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list (NO in step S2205 in FIG. 41), steps S2206 to S2208 in FIG. 41 are omitted, and this history motion vector predictor candidate derivation process procedure ends. If numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list (YES in step S2205 in FIG. 41), the processes from step S2206 onwards in FIG. 41 are performed.

Next, the LY motion vector of the historical predicted motion vector candidate HmvpCandList[NumHmvpCand - i] is added to the numCurrMvpCand-th element mvpListLY[numCurrMvpCand], counting from 0, of the predicted motion vector candidate list for LY prediction, and numCurrMvpCand is incremented by 1 (step S2206 in Figure 41).

The above processing of steps S2205 to S2206 in FIG. 41 is performed on both L0 and L1 (steps S2204 to S2207 in FIG. 41).

Index i is incremented by 1 (steps S2202 and S2208 in FIG. 41), and if index i is equal to or less than the smaller of 4 and NumHmvpCand, processing from step S2203 onwards is performed again (steps S2202 to S2208 in FIG. 41).

In this embodiment, as described above, in the historical prediction motion vector candidate derivation process, the motion vector of an element in the historical prediction motion vector candidate list is added to the predicted motion vector candidate list without comparing the motion vector of the element in the historical prediction motion vector candidate list with the motion vector of an element in the predicted motion vector candidate list.

By adopting such a configuration, if the number of historical prediction motion vector candidates is two or more, it is possible to ensure that the predicted motion vector candidate list reaches the maximum number of elements after the historical prediction motion vector candidate list candidate derivation process is completed. In addition, it is possible to reduce the amount of processing and the circuit size required to check whether the motion vectors are identical.

Normal motion vector prediction mode is a mode in which motion information for the block to be processed is determined by the motion vector prediction candidates and difference vectors included in the motion vector prediction candidate list. Since there is room for determining an appropriate motion vector using the difference vector, even in cases where elements in the motion vector prediction candidate list overlap, it is possible to minimize the decrease in coding efficiency due to a reduction in options.

In addition, in the normal predicted motion vector mode, the predicted motion vector candidate list is processed separately for L0 prediction and L1 prediction. Therefore, even if elements overlap in the predicted motion vector candidate list for L0 prediction, the elements may not overlap in the predicted motion vector candidate list for L1 prediction.

Also, the normal motion vector predictor mode processes motion vector predictor candidates and reference indices separately. Therefore, even if elements in the motion vector predictor candidate list are duplicated, this does not affect the reference index.

In addition, since the motion vector predictor candidate list contains at most two elements, even if elements in the motion vector predictor candidate list overlap, the number of options is reduced by only one.

In addition, since the historical prediction motion vector candidate list does not contain identical elements, a comparison between elements of the historical prediction motion vector candidate list and elements of the predictor motion vector candidate list is practically meaningful only when only one element of the predictor motion vector candidate list is generated by the spatial prediction motion vector candidate derivation unit 421 and the temporal prediction motion vector candidate derivation unit 422, and therefore occurs very rarely. In addition, the normal predictor motion vector mode is generally selected when the motion with adjacent blocks is not similar, and elements of the historical prediction motion vector candidate list are unlikely to overlap with elements already added to the predictor motion vector candidate list.

For the reasons described above, even if elements in the motion vector predictor candidate list overlap, it is possible to reduce the comparison process between the motion vectors of the historical motion vector predictor candidates and the motion vectors of the motion vector predictor candidates while suppressing a decrease in coding efficiency due to a reduction in options.

In addition, by filling the history prediction motion vector candidate list with non-overlapping history prediction motion vector candidates and adding elements of the history prediction motion vector candidate list to the prediction motion vector candidate list without comparing the elements of the history prediction motion vector candidate list with the elements of the prediction motion vector candidate list, it is possible to omit the processing of the prediction motion vector supplementation unit 325 subsequent to the history prediction motion vector candidate derivation unit 323 in the normal prediction motion vector mode derivation unit 301.

<History Merge Candidate Deriving Process>
Next, a detailed description will be given of a method for deriving a history merge candidate from the history merge candidate list HmvpCandList, which is a processing procedure of step S304 in Fig. 20 and is common to the history merge candidate derivation unit 345 of the normal merge mode derivation unit 302 on the encoding side and the history merge candidate derivation unit 423 of the normal merge mode derivation unit 401 on the decoding side. Fig. 42 is a flowchart illustrating the history merge candidate derivation processing procedure.

First, an initialization process is performed (step S2301 in FIG. 42). The value FALSE is set for each of the elements from 0 to (numCurrMergeCand - 1) of isPruned[i], and the variable numOrigMergeCand is set to the number of elements currently registered in the merge candidate list, numCurrMergeCand.

Next, elements in the history prediction motion vector candidate list that are not included in the merge candidate list are added to the merge candidate list. At this time, the elements are checked and added in descending order from the end of the history prediction motion vector candidate list. The initial value of the index hMvpIdx is set to 1, and the addition process from step S2303 to step S2328 in FIG. 42 is repeated from this initial value to NumHmvpCand-1 (steps S2302 to S2329 in FIG. 42). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is not equal to or less than (maximum number of merge candidates, MaxNumMergeCand-1), merge candidates have been added to all elements in the merge candidate list, so this history merge candidate derivation process is terminated (NO in step S2303 in FIG. 42). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is equal to or less than (maximum number of merge candidates, MaxNumMergeCand-1), the process from step S2304 onwards is performed. A value of FALSE is set to sameMotion (step S2304 in FIG. 42). Next, the initial value of index i is set to 0, and the processes of steps S2306 and S2307 in FIG. 42 are performed from this initial value to numOrigMergeCand-1 (S2305 to S2308 in FIG. 42). A comparison is made to see whether the (NumHmvpCand-hMvpIdx)-th element HmvpCandList[NumHmvpCand-hMvpIdx] counting from 0 of the history motion vector prediction candidate list is the same as the i-th element mergeCandList[i] counting from 0 of the merge candidate list (step S2306 in FIG. 42). The same value of merge candidates is determined to be the case when the values of all components (inter prediction mode, reference index, motion vector) of the merge candidates are the same. If they are the same value (YES in step S2306 in FIG. 39), sameMotion and isPruned[i] are both set to TRUE (step S2307 in FIG. 42). If they are not the same value (NO in step S2306 in FIG. 39), the process in step S2307 is skipped. When the repeated process from step S2305 to step S2308 in Fig. 42 is completed, a comparison is made to see whether sameMotion is FALSE (step S2309 in Fig. 42 ). If sameMotion is FALSE (YES in step S2309 in Fig. 42 ), that is, the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] counting from 0 in the history motion vector predictor candidate list does not exist in mergeCandList, so the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] is added and numCurrMergeCand is incremented by 1 (step S2310 in FIG. 42). Index hMvpIdx is incremented by 1 (step S2302 in FIG. 42), and steps S2302 to S2311 in FIG. 42 are repeated.

When all elements in the history motion vector predictor candidate list have been checked, or when merge candidates have been added to all elements in the merge candidate list, this history merge candidate derivation process is completed.
In this embodiment, as described above, in the history merge candidate derivation process, elements of the history prediction motion vector candidate list are compared with elements of the current merge candidate list, and only elements of the history prediction motion vector candidate list that are not present in the current merge candidate list are added to the merge candidate list.
Unlike the normal predicted motion vector mode, the normal merge mode is a mode that directly determines the motion information of the block to be processed without using a difference vector, so that the coding efficiency can be improved by prohibiting the addition of elements of the history merge candidate list that overlap with elements of the current merge candidate list. Here, the elements of the history predicted motion vector candidate list are compared with all candidates of the current merge candidate list, but this is not limited to this as long as the coding efficiency can be improved by comparing at least the elements of the history predicted motion vector candidate list with the elements of the current merge candidate list.
For example, the number of elements of the historical motion vector predictor candidate list to be compared may be limited to one, two, etc. Also, the number of elements of the current merge candidate list to be compared may be limited to one or two.
In the normal merge mode, the merge candidate list includes both the motion vector of L0 prediction and the motion vector of L1 prediction. Therefore, unlike the normal prediction motion vector mode, the prediction motion vector of L0 prediction and the prediction motion vector of L1 prediction cannot be adjusted separately.
In addition, in the normal merge mode, the merge candidate list includes both the reference indexes for L0 prediction and the reference indexes for L1 prediction. Therefore, unlike the normal prediction motion vector mode, the reference indexes for L0 prediction and the reference indexes for L1 prediction cannot be adjusted separately.
In addition, since the merge candidate list contains up to six elements, which is more than the predicted motion vector candidate list, if a duplicate element is added to the merge candidate list, the number of duplicate elements in the merge candidate list increases, and the merge candidate list cannot be used efficiently.
In addition, the element of the history prediction motion vector candidate list that is added to the merge candidate list is the element that is added most recently among the elements of the history prediction motion vector candidate list.Therefore, the element of the history prediction motion vector candidate list that is added to the merge candidate list is the motion information that is spatially closest to the coding block to be processed.Generally, normal merge mode is selected when the motion of adjacent blocks is similar, and the element of the history prediction motion vector candidate list is likely to overlap with the element that has already been added to the merge candidate list.
In order to address the above-mentioned issues, the addition of elements to the historical motion vector predictor candidate list that overlap with elements to the merge candidate list can be prohibited, thereby increasing the number of valid selection elements and improving the coding efficiency.

In addition, by making the maximum number of elements that can be included in a merge candidate list, which is 6, greater than the maximum number of elements that can be included in a predicted motion vector candidate list, which is 2, the probability of selecting the normal merge mode is increased, thereby reducing the comparison process between the motion vectors of the historical predicted motion vector candidates and the motion vectors of the predicted motion vector candidates while suppressing a decrease in encoding efficiency due to overlapping elements in the predicted motion vector candidate list.
In addition, by including a variety of merge candidates such as temporal merge candidates, history merge candidates, average merge candidates, and zero merge candidates in addition to spatial merge candidates, it is possible to increase the probability of selecting the normal merge mode, while suppressing a decrease in encoding efficiency due to overlapping elements in the predictor motion vector candidate list, and reduce the comparison process between the motion vector of the history predictor motion vector candidate and the motion vector of the predictor motion vector candidate.

<Sub-block Temporal Merging Candidate Derivation>
The operation of the sub-block temporal merging candidate derivation unit 381 in the sub-block merging mode derivation unit 304 in FIG. 16 will be described with reference to FIG.

First, it is determined whether the coding block is less than 8x8 pixels (S4002).

If the coding block is less than 8x8 pixels (S4002: Yes), the flag indicating the presence of a sub-block temporal merge candidate is set to availableFlagSbCol=0 (S4003), and the processing of the sub-block temporal merge candidate derivation unit is terminated. Here, if temporal motion vector prediction is prohibited by the syntax, or if sub-block temporal merging is prohibited, the same processing is performed as when the coding block is less than 8x8 pixels (S4002: Yes).

On the other hand, if the coding block is 8x8 pixels or larger (S4002: No), adjacent motion information of the coding block in the coded picture is derived (S4004).

The process of deriving adjacent motion information of a coding block will be described with reference to FIG. 45. The process of deriving adjacent motion information is similar to the process of the spatial prediction motion vector candidate derivation unit 321 described above. However, the order of searching for adjacent blocks is A0, B0, B1, A1, and B2 is not searched. First, coding information is obtained for adjacent block n=A0 (S4052). The coding information indicates a flag availableFlagN indicating whether the adjacent block is available, a reference index refIdxLXN for each reference list, and a motion vector mvLXN.

Next, it is determined whether adjacent block n is valid or invalid (S4054). If the flag indicating whether the adjacent block is available or not is availableFlagN=1, it is valid; otherwise it is invalid.

If adjacent block n is valid (S4054: Yes), the reference index refIdxLXN is set to the reference index refIdxLXn of adjacent block n (S4056). In addition, the motion vector mvLXN is set to the motion vector mvLXn of adjacent block n (S4056), and the process of deriving adjacent motion information of blocks is terminated.

On the other hand, if adjacent block n is invalid (S4106: No), adjacent block n=B0 is set and coding information is obtained (S4104). Similar processing is then repeated, looping in the order of B1 and A1. The process of deriving adjacent motion information loops until the adjacent blocks become valid, and if all adjacent blocks A0, B0, B1, and A1 are invalid, the process of deriving adjacent motion information for blocks ends.

Referring again to FIG. 44. Once the adjacent motion information has been derived (S4004), a temporal motion vector is derived (S4006).

The process of deriving a temporal motion vector will be described with reference to FIG. 46. First, the temporal motion vector tempMv is initialized to (0,0) (S4062).

Next, it is determined whether the adjacent motion information is valid or invalid (S4064). If the flag indicating whether the adjacent block is available or not is available, availableFlagN=1, the information is valid; otherwise, the information is invalid. If the adjacent motion information is invalid (S4064: No), the process of deriving the temporal motion vector is terminated.

On the other hand, if the adjacent motion information is valid (S4064: Yes), it is determined whether the flag predFlagL1N, which indicates whether L1 prediction is used in adjacent block N, is 1 (S4066). If predFlagL1N=0 (S4066: No), proceed to the next process (S4078). If predFlagL1N=1 (S4066: Yes), it is determined whether the POCs of all pictures registered in all reference lists are equal to or lower than the POC of the current picture to be coded (S4068). If this determination is true (S4068: Yes), proceed to the next process (S4070).

If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0 (S4070: Yes and S4072: Yes), it is determined whether ColPic is the same as the reference picture RefPicList1[refIdxL1N] (the picture with the reference index refIdxL1N in the reference list L1) (S4074). If this determination is true (S4074: Yes), the temporal motion vector tempMv=mvL1N is set (S4076). If this determination is false (S4074: No), proceed to the next process (S4078). If the slice type slice_type is not a B slice and the flag collocated_from_l0_flag is not 0 (S4070: No or S4072: No), proceed to the next process (S4078).

Then, it is determined whether the flag predFlagL0N, which indicates whether L0 prediction is used in adjacent block N, is 1 (S4078). If predFlagL0N=1 (S4078: Yes), it is determined whether ColPic is the same as the reference picture RefPicList0[refIdxL0N] (the picture at reference index refIdxL0N in reference list L0) (S4080). If this determination is true (S4080: Yes), the temporal motion vector tempMv=mvL0N is set (S4082). If this determination is false (S4080: No), the process of deriving the temporal motion vector is terminated.

Refer to FIG. 44 again. Next, ColPic is derived (S4016). This process is the same as S4201 in the temporal motion vector predictor candidate derivation unit 322, so a description thereof will be omitted.

Then, a coding block colCb of a different time is set (S4017). This involves setting the coding block located in the lower right corner of the same position as the coding block to be processed in the picture ColPic of the different time as colCb. This coding block corresponds to coding block T1 in Figure 49.

Next, the position obtained by adding the temporal motion vector tempMv to the coding block colCb is set as new colCb (S4018). If the position of the upper left corner of the coding block colCb is (xColCb, yColCb) and the temporal motion vector tempMv is (tempMv[0], tempMv[1]) with 1/16 pixel accuracy, the position of the upper left corner of the new colCb is as follows:
xColCb = Clip3( xCtb, xCtb + CtbSizeY + 3, xcolCb + ( tempMv[0] >> 4 ) )
yColCb = Clip3( yCtb, yCtb + CtbSizeY - 1, ycolCb + ( tempMv[1] >> 4 ) )
Here, the position of the top left corner of the tree block is (xCtb, yCtb), and the size of the tree block is CtbSizeY. As shown in the above formula, the position after adding tempMv is corrected to within the range of the size of the tree block so that it does not deviate significantly from the position before adding tempMv. If this position is outside the screen, it is corrected to within the screen.

Then, it is determined whether the prediction mode PredMode of this coding block colCb is inter prediction (MODE_INTER) (S4020). If the prediction mode of colCb is not inter prediction (S4020: No), a flag indicating the presence of a sub-block temporal merge candidate is set to availableFlagSbCol=0 (S4003), and the processing of the sub-block temporal merge candidate derivation unit is terminated.

On the other hand, if the prediction mode of colCb is inter prediction (S4020: Yes), inter prediction information is derived for each reference list (S4022, S4023). Here, for colCb, a central motion vector ctrMvLX for each reference list and a flag ctrPredFlagLX indicating whether LX prediction is used or not are derived. LX indicates a reference list, and when deriving reference list 0, LX is L0, and when deriving reference list 1, LX is L1. The derivation of inter prediction information will be described with reference to FIG. 47.

If the coding block colCb of a different time is not available (S4112: NO), or if the prediction mode PredMode is intra prediction (MODE_INTRA) (S4114: NO), the flags availableFlagLXCol and predFlagLXCol are both set to 0 (step S4116), the motion vector mvCol is set to (0, 0) (S4118), and the process of deriving inter prediction information is terminated.

If the coding block colCb is available (S4112: Yes) and the prediction mode PredMode is not intra prediction (MODE_INTRA) (S4114: YES), mvCol, refIdxCol, and availableFlagCol are calculated using the following procedure.

When the flag PredFlagLX[xPCol][yPCol] indicating whether LX prediction of the coding block colCb is used is 1 (YES in S4120), the motion vector mvCol is set to the same value as the LX motion vector MvLX[xPCol][yPCol] of the coding block colCb (S4122), the reference index refIdxCol is set to the same value as the LX reference index RefIdxLX[xPCol][yPCol] (S4124), and the list listCol is set to LX (S4126). Here, xPCol and yPCol are indexes indicating the position of the top left pixel of the coding block colCb in a picture colPic of a different time.

On the other hand, if the flag PredFlagLX[xPCol][yPCol] indicating whether LX prediction of the coding block colCb is used is 0 (NO in S4120), the following process is performed. First, it is determined whether the POC of all pictures registered in all reference lists is equal to or lower than the POC of the current picture to be coded (S4128). It is also determined whether the flag PredFlagLY[xPCol][yPCol] indicating whether LY prediction of colCb is used is 1 (S4128). Here, LY prediction is defined as a reference list different from LX prediction. In other words, when LX=L0, LY=L1, and when LX=L1, LY=L0.

If this determination is true (S4128: Yes), the motion vector mvCol is set to the same value as MvLY[xPCol][yPCol], which is the motion vector of LY of the coding block colCb (S4130), the reference index refIdxCol is set to the same value as the reference index RefIdxLY[xPCol][yPCol] of LY (S4132), and the list listCol is set to LX (S4134).

On the other hand, if this determination is false (S4128: No), the flags availableFlagLXCol and predFlagLXCol are both set to 0 (step S4116), the motion vector mvCol is set to (0, 0) (S4118), and the process of deriving inter prediction information is terminated.

When inter prediction information is obtained from the coding block colCb, the flag availableFlagLXCol and the flag predFlagLXCol are both set to 1 (S4136).

Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (S4138). This process is the same as S4245 in the temporal motion vector predictor candidate derivation unit 322, so a description thereof will be omitted.

Refer to FIG. 44 again. Once the inter prediction information is derived for each reference list, the calculated motion vector mvLXCol is set as the central motion vector ctrMvLX, and the calculated flag predFlagLXCol is set as the flag ctrPredFlagLX (S4022, S4023).

Then, it is determined whether the central motion vector is valid or invalid (S4024). If ctrPredFlagL0=0 and ctrPredFlagL1=0, it is invalid; otherwise, it is invalid. If the central motion vector is invalid (S4024: No), the flag indicating the presence of a sub-block temporal merge candidate is set to availableFlagSbCol=0 (S4003), and the processing of the sub-block temporal merge candidate derivation unit is terminated.

On the other hand, if the central motion vector is valid (S4024: Yes), the flag indicating the presence of a sub-block temporal merge candidate is set to availableFlagSbCol=1 (S4025), and sub-block motion information is derived (S4026). This process will be described with reference to FIG. 48.

First, the number of sub-blocks in the width direction, numSbX, and the number of sub-blocks in the height direction, numSbY, are calculated from the width cbWidth and height cBheight of the coding block colCb (S4152). In addition, refIdxLXSbCol = 0 is set (S4152). After this process, the process is repeated in units of predicted sub-blocks colSb. This repetition is performed while changing the height direction index ySbIdx from 0 to numSbY and the width direction index xSbIdx from 0 to numSbX.

If the top left position of the coding block colCb is (xCb, yCb), the top left position (xSb, ySb) of the prediction sub-block colSb is calculated as follows.
xSb = xCb + xSbIdx * sbWidth
ySb = yCb + ySbIdx * sbHeight
Next, the position obtained by adding the temporal motion vector tempMv to the prediction sub-block colSb is set as new colSb (S4154). If the position of the upper left corner of the prediction sub-block colSb is (xColSb, yColSb) and the temporal motion vector tempMv is (tempMv[0], tempMv[1]) with 1/16 pixel accuracy, the position of the upper left corner of the new colSb is as follows:
xColSb = Clip3( xCtb, xCtb + CtbSizeY + 3, xSb + ( tempMv[0] >> 4 ) )
yColSb = Clip3( yCtb, yCtb + CtbSizeY - 1, ySb + ( tempMv[1] >> 4 ) )
Here, the position of the top left corner of the tree block is (xCtb, yCtb), and the size of the tree block is CtbSizeY. As shown in the above formula, the position after adding tempMv is corrected to within the range of the size of the tree block so that it does not deviate significantly from the position before adding tempMv. If this position is outside the screen, it is corrected to within the screen.

Then, inter prediction information is derived for each reference list (S4156, S4158). Here, for the prediction sub-block colSb, a motion vector mvLXSbCol for each reference list on a sub-block basis and a flag availableFlagLXSbCol indicating whether the prediction sub-block is valid or not are derived. LX indicates a reference list, and when deriving reference list 0, LX is L0, and when deriving reference list 1, LX is L1. The derivation of inter prediction information is the same as S4022 and S4023 in FIG. 47, so a description thereof will be omitted.

After deriving the inter prediction information (S4156, S4158), it is determined whether or not the predicted sub-block colSb is valid (S4160). If availableFlagL0SbCol=0 and availableFlagL1SbCol=0, colSb is invalid; otherwise, it is determined to be valid. If colSb is invalid (S4160: No), the motion vector mvLXSbCol is set to the central motion vector ctrMvLX (S4162). Furthermore, the flag predFlagLXSbCol indicating whether or not LX prediction is being used is set to the flag ctrPredFlagLX in the central motion vector (S4162). This completes the derivation of sub-block motion information.

Refer to FIG. 44 again. Then, the L0 motion vector mvL0SbCol and the L1 motion vector mvL1SbCol are added as candidates to the subblock merge candidate list subblockMergeCandList in the subblock merge mode derivation unit 304 described above (S4028). However, this addition is only made when the flag availableSbCol=1, which indicates the presence of a subblock temporal merge candidate, is set. This completes the processing of the temporal merge candidate derivation unit 342.

The above description of the subblock temporal merge candidate derivation unit 381 is for encoding, but the same applies for decoding. In other words, the operation of the subblock temporal merge candidate derivation unit 481 in the subblock merge mode derivation unit 404 in FIG. 22 can be similarly described by replacing encoding with decoding in the above description.

<Motion Compensation Prediction Processing>
The motion compensation prediction unit 306 acquires the position and size of a block currently being subjected to prediction processing in encoding. The motion compensation prediction unit 306 also acquires inter prediction information from the inter prediction mode determination unit 305. A reference index and a motion vector are derived from the acquired inter prediction information, and a reference picture specified by the reference index in the decoded image memory is acquired at a position where the reference picture is moved by the amount of the motion vector from the same position as the image signal of the prediction block, and then a prediction signal is generated.

When the reference mode in inter prediction is prediction from a single reference picture, such as L0 prediction or L1 prediction, the prediction signal obtained from one reference picture is used as a motion compensation prediction signal, and when the reference mode is prediction from two reference pictures, such as BI prediction, the weighted average of the prediction signals obtained from the two reference pictures is used as a motion compensation prediction signal, and the motion compensation prediction signal is supplied to the prediction method determination unit. Here, the weighted average ratio for bi-prediction is set to 1:1, but weighted averaging may be performed using other ratios. For example, the weighting ratio may be set to be larger as the picture interval between the picture to be predicted and the reference picture becomes closer. The weighting ratio may also be calculated using a correspondence table of combinations of picture intervals and weighting ratios.

The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The motion compensation prediction unit 406 obtains inter prediction information from the normal prediction motion vector mode derivation unit 401, the normal merge mode derivation unit 402, the sub-block prediction motion vector mode derivation unit 403, and the sub-block merge mode derivation unit 404 via the switch 408.

The motion compensation prediction unit 406 supplies the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207.

<About prediction direction>
57 to 61 are diagrams for explaining the prediction direction of motion compensation prediction. A process of performing prediction from a single reference picture is defined as uni-prediction, and in the case of uni-prediction, prediction is performed using either one of two reference pictures registered in a reference list, called L0 prediction or L1 prediction.

Figure 57 shows a case where uni-prediction is performed and the L0 reference picture (RefL0Pic) is at a time earlier than the picture to be coded (CurPic). Figure 58 shows a case where uni-prediction is performed and the reference picture for L0 prediction is at a time later than the picture to be coded. Similarly, uni-prediction can be performed by replacing the reference picture for L0 prediction in Figures 57 and 58 with the reference picture for L1 prediction (RefL1Pic).

A process of making predictions from two reference pictures is defined as bi-prediction, and in the case of bi-prediction, both L0 prediction and L1 prediction are used and expressed as BI prediction. Figure 59 shows a case where the reference picture for L0 prediction is before the picture to be coded, and the reference picture for L1 prediction is after the picture to be coded. Figure 60 shows a case where the reference picture for L0 prediction and the reference picture for L1 prediction are before the picture to be coded. Figure 61 shows a case where the reference picture for L0 prediction and the reference picture for L1 prediction are after the picture to be coded. In this way, the relationship between the prediction type and time of L0/L1 can be used without being limited to L0 being the past direction and L1 being the future direction. In addition, in the case of bi-prediction, the same reference picture may be used to make each of L0 prediction and L1 prediction. The decision as to whether to perform motion compensation prediction in a uni-predictive or bi-predictive manner is made based on information (e.g., a flag) indicating, for example, whether to use L0 prediction and whether to use L1 prediction.

<About reference index>
In an embodiment of the present invention, in order to improve the accuracy of motion compensation prediction, it is possible to select an optimal reference picture from among multiple reference pictures in motion compensation prediction. To this end, the reference picture used in the motion compensation prediction is used as a reference index, and the reference index is coded into the coding stream together with a coding vector.

<Motion compensation process based on normal predicted motion vector mode>
16 , when the inter prediction information by the normal prediction motion vector mode derivation unit 301 is selected in the inter prediction mode determination unit 305, the motion compensation unit 306 acquires the inter prediction information from the inter prediction mode determination unit 305, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter prediction unit 203 on the decoding side in FIG. 22, when the switch 408 is connected to the normal prediction motion vector mode derivation unit 401 during the decoding process, the motion compensation unit 406 obtains inter prediction information from the normal prediction motion vector mode derivation unit 401, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation process based on normal merge mode>
As also shown in the inter prediction unit 102 on the encoding side in Fig. 16, when the inter prediction mode determination unit 305 selects inter prediction information by the normal merge mode derivation unit 302, the motion compensation unit 306 acquires this inter prediction information from the inter prediction mode determination unit 305, derives a reference mode, a reference index, and a motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter prediction unit 203 on the decoding side in FIG. 22, when the switch 408 is connected to the normal merge mode derivation unit 402 during the decoding process, the motion compensation unit 406 obtains inter prediction information from the normal merge mode derivation unit 402, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation process based on sub-block predicted motion vector mode>
16 , when inter prediction information is selected by the sub-block prediction motion vector mode derivation unit 303 in the inter prediction mode determination unit 305, the motion compensation unit 306 acquires the inter prediction information from the inter prediction mode determination unit 305, derives a reference mode, a reference index, and a motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter prediction unit 203 on the decoding side in FIG. 22, when the switch 408 is connected to the sub-block prediction motion vector mode derivation unit 403 during the decoding process, the motion compensation unit 406 obtains inter prediction information from the sub-block prediction motion vector mode derivation unit 403, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation process based on sub-block merge mode>
16 , when inter prediction information by the sub-block merge mode derivation unit 304 is selected in the inter prediction mode determination unit 305, the motion compensation unit 306 acquires the inter prediction information from the inter prediction mode determination unit 305, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter prediction unit 203 on the decoding side in FIG. 22, when the switch 408 is connected to the sub-block merging mode derivation unit 404 during the decoding process, the motion compensation unit 406 obtains inter prediction information from the sub-block merging mode derivation unit 404, derives the reference mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation processing based on affine transformation prediction>
In the normal prediction motion vector mode and normal merge mode, motion compensation using an affine model can be used based on the following flags. The following flags are reflected in the following flags based on the inter prediction conditions determined by the inter prediction mode determination unit 305 in the encoding process, and are encoded in the encoded stream. In the decoding process, whether or not to perform motion compensation using an affine model is specified based on the following flags in the encoded stream.

sps_affine_enabled_flag indicates whether motion compensation using the affine model can be used in inter prediction. If sps_affine_enabled_flag is 0, motion compensation using the affine model is suppressed on a sequence-by-sequence basis. In addition, inter_affine_flag and cu_affine_type_flag are not transmitted in the CU syntax of the encoded video sequence. If sps_affine_enabled_flag is 1, motion compensation using the affine model can be used in the encoded video sequence.

sps_affine_type_flag indicates whether motion compensation using the 6-parameter affine model is available for inter prediction.

If sps_affine_type_flag is 0, motion compensation using the 6-parameter affine model is suppressed. In addition, cu_affine_type_flag is not transmitted in the CU syntax of the encoded video sequence. If sps_affine_type_flag is 1, motion compensation using the 6-parameter affine model can be used in the encoded video sequence.

If sps_affine_type_flag is not present, it is assumed to be 0.

When decoding a P or B slice, if inter_affine_flag is 1 for the currently processed CU, motion compensation using the affine model is used to generate a motion compensation prediction signal for the currently processed CU.

If inter_affine_flag is 0, no affine model is used for the currently processed CU.

If inter_affine_flag is not present, it is assumed to be 0.

When decoding a P or B slice, if cu_affine_type_flag is 1 for the currently processed CU, motion compensation using a 6-parameter affine model is used to generate a motion compensation prediction signal for the currently processed CU.

If cu_affine_type_flag is 0, motion compensation using a four-parameter affine model is used to generate a motion compensation prediction signal for the currently processed CU.

In affine model motion compensation, reference indexes and motion vectors are derived on a subblock basis, so a motion compensation prediction signal is generated using the reference indexes and motion vectors being processed on a subblock basis.

<Modification 1>
A first modified example of the present embodiment will be described. This modified example differs from the present embodiment in that a merge differential motion vector mode is added, and only the differences from the present embodiment will be described. If umve_flag in FIG. 12 is 1, the merge differential motion vector mode is selected, and if umve_flag is 0, the normal merge mode is selected.

Next, the operation of the merge differential motion vector mode will be described. This mode allows one merge differential motion vector to be added to each of the L0 prediction motion vector and the L1 prediction motion vector of one of the top two merge candidates (merge candidates with merge indexes 0 and 1 in the merge candidate list).

In the case of the merge differential motion vector mode, the merge differential motion vector is coded by the bit string coding unit 108 and decoded by the bit string decoding unit 201.

As described above, since the merge candidate list is also used in the merge motion vector difference mode, the coding efficiency can be improved by prohibiting the addition of elements in the history predicted motion vector candidate list that overlap with elements in the merge candidate list and increasing the number of valid selection elements.

In addition, by adding the merge differential motion vector mode and reducing the probability of selecting the normal predicted motion vector mode, it is possible to reduce the comparison process between the motion vectors of the history predicted motion vector candidate and the motion vector of the predicted motion vector candidate while suppressing a decrease in encoding efficiency due to overlapping elements in the predicted motion vector candidate list.
<Modification 2>
Modification 2 of the present embodiment will be described. This modification differs from the present embodiment in the operation of the history motion vector candidate derivation units 323 and 423 shown in Fig. 41, and only the differences from the present embodiment will be described. Fig. 65 is a flowchart for explaining the history motion vector candidate derivation process procedure of modification 2. Fig. 65 differs from Fig. 41 in that step S2209 is added. Fig. 65 is the same operation as Fig. 41 except for step S2209.

The following describes the processing performed when numCurrMvpCand is smaller than 2, which is the maximum number of elements in the predicted motion vector candidate list (YES in step S2205 in Figure 65).

Check whether the LY reference index of the history prediction motion vector candidate HmvpCandList[NumHmvpCand-i] is the same as the LY reference index of the coding block to be processed (step S2209 in FIG. 65). If the LY reference index of the history prediction motion vector candidate HmvpCandList[NumHmvpCand-i] is the same as the LY reference index of the coding block to be processed (YES in step S2209 in FIG. 65), proceed to S2206. If the LY reference index of the history prediction motion vector candidate HmvpCandList[NumHmvpCand-i] is not the same as the LY reference index of the coding block to be processed (NO in step S2209 in FIG. 65), proceed to S2207.

As described above, when the LY reference index of the history prediction motion vector candidate is the same as the LY reference index of the coding block to be processed, the motion vector of the history prediction motion vector candidate is added to the prediction motion vector candidate list for LY prediction, so that a highly accurate history prediction motion vector candidate can be added to the prediction motion vector candidate list without comparing the motion vector of the history prediction motion vector candidate with the motion vector of the prediction motion vector candidate.

In this embodiment, in the history prediction motion vector candidate derivation process, elements of the history prediction motion vector candidate list are added to the prediction motion vector candidate list without comparing the elements of the history prediction motion vector candidate list with the elements of the prediction motion vector candidate list, while in the history merge candidate derivation process, elements of the history merge candidate list are compared with elements of the merge list, and only history merge candidate lists that do not exist in the merge list are added to the merge list. By adopting the above configuration, the following effects can be obtained.

1. In the process of deriving candidates from the historical motion vector predictor candidate list, there is no need to perform additional candidate derivation processing, which reduces the amount of processing and the circuit size. In addition, even in cases where elements in the motion vector predictor candidate list are inevitably duplicated, it is possible to minimize the decrease in coding efficiency due to a reduction in options.

2. In the history merge candidate derivation process, by prohibiting the addition of elements to the history merge candidate list that overlap with elements of the merge list, it is possible to construct an appropriate merge candidate list in the normal merge mode, which determines the motion information of the block to be processed without using a difference vector, and thus improve coding efficiency.

3. By filling the history prediction motion vector candidate list with non-overlapping history prediction motion vector candidates, the processing of the merge candidate supplementation unit 346 subsequent to the history merge candidate derivation unit 345 in the normal merge mode derivation unit 302 can be omitted, thereby reducing the amount of processing.

4. By filling the history prediction motion vector candidate list with non-overlapping history prediction motion vector candidates and adding elements of the history prediction motion vector candidate list to the prediction motion vector candidate list without comparing the elements of the history prediction motion vector candidate list with the elements of the prediction motion vector candidate list, it is possible to omit the processing of the prediction motion vector supplementation unit 325 subsequent to the history prediction motion vector candidate derivation unit 323 in the normal prediction motion vector mode derivation unit 301.

All of the above-mentioned embodiments may be combined in multiple ways.

In all of the embodiments described above, the coded bitstream output by the image coding device has a specific data format so that it can be decoded according to the coding method used in the embodiment. Furthermore, an image decoding device corresponding to this image coding device can decode the coded bitstream in this specific data format.

When a wired or wireless network is used to exchange coded bitstreams between an image coding device and an image decoding device, the coded bitstream may be converted into a data format suitable for the transmission mode of the communication channel before being transmitted. In this case, a transmitting device is provided that converts the coded bitstream output by the image coding device into coded data in a data format suitable for the transmission mode of the communication channel and transmits the coded bitstream to the network, and a receiving device is provided that receives the coded data from the network, restores it to a coded bitstream, and supplies it to the image decoding device.

The transmitting device includes a memory that buffers the coded bitstream output by the image coding device, a packet processing unit that packetizes the coded bitstream, and a transmitting unit that transmits the packetized coded data via the network. The receiving device includes a receiving unit that receives the packetized coded data via the network, a memory that buffers the received coded data, and a packet processing unit that packetizes the coded data to generate a coded bitstream and provides it to the image decoding device.

In addition, a display unit that displays the image decoded by the image decoding device may be added to the configuration to form a display device. In this case, the display unit reads out the decoded image signal generated by the decoded image signal superimposition unit 205 and stored in the decoded image memory 206, and displays it on a screen.

Also, an imaging unit may be added to the configuration and the captured image may be input to the image encoding device to form an imaging device. In this case, the imaging unit inputs the captured image signal to the block division unit 101.

Figure 66 shows an example of the hardware configuration of the encoding/decoding device of the present application. The encoding/decoding device includes the configuration of the image encoding device and image decoding device according to the embodiment of the present invention. The encoding/decoding device 9000 has a CPU 9001, a codec IC 9002, an I/O interface 9003, a memory 9004, an optical disk drive 9005, a network interface 9006, and a video interface 9009, and each part is connected by a bus 9010.

The image encoding unit 9007 and the image decoding unit 9008 are typically implemented as a codec IC 9002. The image encoding process of the image encoding device according to the embodiment of the present invention is executed by the image encoding unit 9007, and the image decoding process of the image decoding device according to the embodiment of the present invention is executed by the image encoding unit 9007. The I/O interface 9003 is realized by, for example, a USB interface, and is connected to an external keyboard 9104, mouse 9105, etc. The CPU 9001 controls the encoding/decoding device 9000 so as to execute the operation desired by the user based on the user operation input via the I/O interface 9003. The user operation by the keyboard 9104, mouse 9105, etc. includes the selection of whether to execute the encoding or decoding function, the setting of the encoding quality, the input/output destination of the encoded stream, the input/output destination of the image, etc.

When a user wishes to play back an image recorded on the disc recording medium 9100, the optical disc drive 9005 reads out an encoded bit stream from the inserted disc recording medium 9100 and sends the read encoded stream to the image decoding unit 9008 of the codec IC 9002 via the bus 9010. The image decoding unit 9008 executes image decoding processing in the image decoding device according to the embodiment of the present invention on the input encoded bit stream, and sends the decoded image to the external monitor 9103 via the video interface 9009. The encoding/decoding device 9000 also has a network interface 9006, and can be connected to an external distribution server 9106 or a mobile terminal 9107 via the network 9101. When a user wishes to play back an image recorded on the distribution server 9106 or a mobile terminal 9107 instead of the image recorded on the disc recording medium 9100, the network interface 9006 acquires an encoded stream from the network 9101 instead of reading out the encoded bit stream from the input disc recording medium 9100. Furthermore, when a user wishes to play back the image recorded in memory 9004, an image decoding process is performed on the encoded stream recorded in memory 9004 in the image decoding device according to an embodiment of the present invention.

When the user wishes to encode an image captured by an external camera 9102 and record it in memory 9004, the video interface 9009 inputs the image from the camera 9102 and sends it to the image encoding unit 9007 of the codec IC 9002 via the bus 9010. The image encoding unit 9007 executes image encoding processing in the image encoding device according to the embodiment of the present invention on the image input via the video interface 9009, and creates an encoded bit stream. The encoded bit stream is then sent to the memory 9004 via the bus 9010. When the user wishes to record the encoded stream on the disc recording medium 9100 instead of the memory 9004, the optical disc drive 9005 writes the encoded stream to the inserted disc recording medium 9100.

It is also possible to realize a hardware configuration that has an image encoding device but not an image decoding device, or a hardware configuration that has an image decoding device but not an image encoding device. Such a hardware configuration is realized, for example, by replacing the codec IC 9002 with an image encoding unit 9007 or an image decoding unit 9008, respectively.

The above encoding and decoding processes may of course be realized as transmission, storage, and receiving devices using hardware, but may also be realized by firmware stored in ROM (read-only memory) or flash memory, or by software on a computer, etc. The firmware and software programs may be provided by recording them on a recording medium that can be read by a computer, etc., or may be provided from a server via a wired or wireless network, or may be provided as data broadcasting on terrestrial or satellite digital broadcasting.

The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing process, and that such modifications are also within the scope of the present invention.

100 Image encoding device, 101 Block division unit, 102 Inter prediction unit, 103 Intra prediction unit, 104 Decoded image memory, 105 Prediction method determination unit, 106 Residual signal generation unit, 107 Orthogonal transformation and quantization unit, 108 Bit string encoding unit, 109 Inverse quantization and inverse orthogonal transformation unit, 110 Decoded image signal superposition unit, 111 Encoded information storage memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block division unit, 203 Inter prediction unit 204 Intra prediction unit, 205 Encoded information storage memory 205 Inverse quantization and inverse orthogonal transformation unit, 207 Decoded image signal superposition unit, 208 Decoded image memory.

Claims (8)

a spatial candidate derivation unit that derives spatial candidates from inter prediction information of a plurality of blocks adjacent to a current block to be coded, and registers the spatial candidates in a first candidate list;
a history candidate derivation unit that adds history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection unit that selects a selection candidate from the candidates included in the second candidate list;
an inter prediction unit that performs inter prediction using the selection candidates;
Equipped with
when a prediction mode of the encoding target block is a first prediction mode, the history candidate derivation unit determines whether to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list, and when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
A video encoding device, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the encoding target block, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selection candidates;
having
the history candidate derivation step, when a prediction mode of the encoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
An image encoding method, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. On the computer,
a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the encoding target block, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selected candidates;
Run the command,
the history candidate derivation step, when a prediction mode of the encoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
An image encoding program, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. a spatial candidate derivation unit that derives spatial candidates from inter prediction information of a plurality of blocks adjacent to a decoding target block, and registers the spatial candidates in a first candidate list as candidates;
a history candidate derivation unit that adds history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection unit that selects a selection candidate from the candidates included in the second candidate list;
an inter prediction unit that performs inter prediction using the selection candidates;
Equipped with
the history candidate derivation unit, when a prediction mode of the decoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
A video decoding device, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the block to be decoded, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selection candidates;
having
the history candidate derivation step, when a prediction mode of the decoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
An image decoding method, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. On the computer,
a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the decoding target block, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selected candidates;
Run the command,
the history candidate derivation step, when a prediction mode of the decoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
An image decoding program, characterized in that a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode. A transmission method for transmitting a bitstream generated by an image encoding device , comprising the steps of:
a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the encoding target block, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selection candidates;
a bitstream generating step of encoding the encoding information to generate a bitstream;
a bitstream transmitting step of transmitting the bitstream;
having
the history candidate derivation step, when a prediction mode of the encoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode.
Transmission method. A recording method for recording a bit stream generated by an image encoding device onto a recording medium, comprising the steps of:
a spatial candidate derivation step of deriving spatial candidates from inter prediction information of a plurality of blocks adjacent to the encoding target block, and registering the spatial candidates in a first candidate list as candidates;
a history candidate derivation step of adding history candidates included in the history candidate list as candidates to the first candidate list to generate a second candidate list;
a candidate selection step of selecting a selection candidate from the candidates included in the second candidate list;
an inter prediction step of performing inter prediction using the selection candidates;
a bitstream generating step of encoding the encoding information to generate a bitstream;
a bitstream recording step of recording the bitstream on the recording medium;
having
the history candidate derivation step, when a prediction mode of the encoding target block is a first prediction mode, determines whether or not to add the history candidate to the first candidate list based on whether a motion vector and a reference index included in the history candidate are identical to a motion vector and a reference index included in a candidate included in the first candidate list; and, when the prediction mode is a second prediction mode, adds the history candidate to the first candidate list regardless of whether a motion vector of the history candidate is identical to a motion vector of a candidate included in the first candidate list;
a maximum number of candidates included in the second candidate list when the prediction mode is a first prediction mode is greater than a maximum number of candidates included in the second candidate list when the prediction mode is a second mode.
Recording method.

Images