JPWO2015141696A1 - Image decoding apparatus, image encoding apparatus, and prediction apparatus - Google Patents

Abstract

DBBP has a problem that the processing amount of interpolation processing for generating two interpolation images, division mode derivation processing for deriving a division mode from a depth image, and synthesis processing for synthesizing two complementary images according to segmentation is large. It was. A depth base comprising a segmentation deriving unit for deriving segmentation information from a depth image, an image interpolating unit for generating two motion compensated images, and an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image In the block prediction image generation device, the image interpolation unit generates the two motion compensated images by bilinear prediction, and the depth base block prediction image generation device. A depth-based block predicted image generation apparatus that derives a division mode from pixels at four corners of a depth block. In addition, a depth-based block predicted image generation apparatus characterized in that synthesis is performed by selecting one of the two interpolation images in each pixel of a block. Also, a depth-based block prediction image generation apparatus characterized by not performing bi-prediction.

Description

  The present invention relates to an image decoding device, an image encoding device, and a prediction device.

  The multi-view image encoding technique includes a parallax predictive encoding that reduces the amount of information by predicting a parallax between images when encoding images of a plurality of viewpoints, and a decoding method corresponding to the encoding method. Has been proposed. A vector representing the parallax between viewpoint images is called a displacement vector. The displacement vector is a two-dimensional vector having a horizontal element (x component) and a vertical element (y component), and is calculated for each block which is an area obtained by dividing one image. In order to acquire images from a plurality of viewpoints, it is common to use cameras arranged at the respective viewpoints. In multi-viewpoint encoding, each viewpoint image is encoded as a different layer in each of a plurality of layers. A method for encoding a moving image composed of a plurality of layers is generally referred to as scalable encoding or hierarchical encoding. In scalable coding, high coding efficiency is realized by performing prediction between layers. A reference layer without performing prediction between layers is called a base layer, and other layers are called enhancement layers. Scalable encoding in the case where a layer is composed of viewpoint images is referred to as view scalable encoding. At this time, the base layer is also called a base view, and the enhancement layer is also called a non-base view. Furthermore, in addition to view scalable, scalable coding when a layer is composed of a texture layer (image layer) and a depth layer (distance image layer) is called three-dimensional scalable coding.

  For scalable coding, in addition to view scalable coding, spatial scalable coding (pictures with low resolution as the base layer and pictures with high resolution in the enhancement layer), SNR scalable coding (image quality as the base layer) Low picture, high resolution picture as an enhancement layer). In scalable coding, for example, a base layer picture may be used as a reference picture in coding an enhancement layer picture.

  In Non-Patent Document 1, it is called depth-based block partitioning (DBBP), in which partition information (segmentation) is derived from a depth image, and one prediction image is synthesized from two interpolated images using the segmentation as a mask. Technology is known. In DBBP, the segmentation is derived from the region division based on the depth pixels, thereby forming a rectangle (2N × 2N, 2N × N, 2N × nU, 2N × nD, N × 2N, nL × 2N, nR × 2N). Division with a high degree of freedom which is not limited is possible.

  Non-Patent Document 2 is a technique for unifying viewpoint synthesis prediction (VSP) and a disparity vector (DV) used for depth pixel reference in DBBP. Both VSP and DBBP are adjacent base disparity vectors before depth refinement ( NBDV).

F. Jager, J. Konieczny, and G. Cordara, “CE3: Results on Depth-based Block Partitioning (DBBP)”, JCT3V-G0106, JCT-3V 7th Meeting: San Jose, USA, 11 Jan.-17 Jan. 2013 (released January 3, 2014) MW Park, JY Lee, B. Choi, Y. Cho, C. Kim, “Disparity Vector for DBBP in 3D-HEVC”, JCT3V-H0070, JCT-3V 8th Meeting: Valencia, ES, 29 March-4 April 2014 （ (March 21, 2014 release)

  Since Non-Patent Document 1 uses different partitioning methods for VSP and DBBP, there is a problem that the implementation becomes complicated.

  Non-Patent Document 2 can unify disparity vectors (DV) used for reference to depth pixels of VSP and DBBP, but has a problem that encoding efficiency decreases.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. In the depth-based block predicted image generation apparatus including the image synthesizing unit, the image interpolation unit generates the two motion compensation images by bilinear prediction.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. A depth-based block prediction image generation apparatus including an image synthesizing unit that further includes a depth division mode deriving unit for deriving a division mode from the depth image, wherein the depth division mode deriving unit is divided from pixels at four corners of the depth block A mode is derived.

  One aspect of the present invention is characterized in that the depth division mode deriving unit derives a division mode from a comparison of the upper left and lower right of the depth and a comparison of the upper right and lower left of the depth.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. And a depth division mode deriving unit for deriving a division mode, wherein the depth division mode deriving unit derives a 2N × N or N × 2N division mode.

  In one embodiment of the present invention, in the depth-based block prediction image generation device, the segmentation deriving unit derives segmentation information that takes 0 or 1 for each pixel, and the image synthesizing unit includes two pieces of information for each pixel of the block. The composition is performed by selecting one of the interpolation images.

  One embodiment of the present invention is an image decoding device including the depth base block prediction image generation device and the DBBP flag decoding unit, and the depth base block prediction image generation device performs DBBP prediction when the DBBP flag is 1. It is characterized by performing.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a view synthesis prediction unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images, an image synthesis unit that generates one motion compensation image by synthesizing the two interpolation images, and a division mode deriving unit that derives a division mode. The composite prediction means includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the division mode derivation unit and the partition division unit include a common division mode. A derivation unit is provided.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a merge mode parameter derivation unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images and an image synthesis unit that combines the two interpolation images to generate one motion compensation image. The image decoding apparatus further includes a DBBP flag decoding unit. The merge mode parameter derivation unit converts bi-prediction to uni-prediction when the DBBP flag is 1.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and an inter prediction parameter decoding unit, wherein the depth base block prediction image generation unit includes a segmentation deriving unit for deriving segmentation information from the depth image, An image interpolation unit that generates two motion compensation images and an image synthesis unit that combines the two interpolation images to generate one motion compensation image. The image decoding apparatus further includes a DBBP flag decoding unit. The inter prediction parameter decoding unit does not decode a bi-prediction value as an inter prediction identifier when the DBBP flag is 1.

  One embodiment of the present invention is an image encoding device including the depth base block prediction image generation device and the DBBP flag encoding unit, and the depth base block prediction image generation device is configured to perform DBBP when the DBBP flag is 1. An image encoding apparatus that performs prediction.

  One embodiment of the present invention is an image encoding device including a depth base block prediction image generation unit and a viewpoint synthesis prediction unit, wherein the depth base block prediction image generation unit is a segmentation deriving unit that derives segmentation information from a depth image. An image interpolation unit that generates two motion compensation images, an image synthesis unit that combines the two interpolation images to generate one motion compensation image, and a division mode derivation unit that derives a division mode, The viewpoint synthesis prediction unit includes a partition division unit that performs partition division in the depth, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the division mode derivation unit and the partition division unit include: A common division mode deriving unit is provided.

  One aspect of the present invention is an image encoding device including a depth base block prediction image generation unit and a merge mode parameter derivation unit, wherein the depth base block prediction image generation unit derives segmentation information from the depth image. And an image interpolating unit for generating two motion compensated images and an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image. The image encoding device encodes the DBBP flag. And a merge mode parameter deriving unit that performs conversion from uni-prediction to bi-prediction when the DBBP flag is 1.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a view synthesis prediction unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images, an image synthesis unit that generates one motion compensation image by synthesizing the two interpolation images, and a division mode deriving unit that derives a division mode. The composite prediction unit includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image. The segmentation derivation unit of the depth base block prediction image generation unit, the division mode The derivation unit derives the position of the depth image to be referenced. A disparity vector used, characterized by a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit.

  One embodiment of the present invention is an image encoding device including a depth base block prediction image generation unit and a viewpoint synthesis prediction unit, wherein the depth base block prediction image generation unit is a segmentation deriving unit that derives segmentation information from a depth image. An image interpolation unit that generates two motion compensation images, an image synthesis unit that combines the two interpolation images to generate one motion compensation image, and a division mode derivation unit that derives a division mode, The viewpoint synthesis prediction unit includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the segmentation derivation unit of the depth base block prediction image generation unit, the division Deriving the position of the depth image referenced by the mode deriving unit Wherein the disparity vector, that a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit for use in.

  According to the present invention, there is an effect of simplifying the mounting by sharing the block division processing between DBBP and VSP. In addition, according to the present invention, there is an effect of simplifying the mounting by sharing the parallax used in the DBBP and the VSP.

It is a block diagram which shows the structure of the DBBP prediction part 3095 which concerns on this embodiment. 1 is a schematic diagram illustrating a configuration of an image transmission system according to an embodiment of the present invention. It is a figure which shows the hierarchical structure of the data of the encoding stream which concerns on this embodiment. It is a figure which shows the pattern of a division | segmentation mode, (a)-(h) is a division | segmentation mode, respectively 2Nx2N, 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nR. The partition shapes in the case of × 2N and N × N are shown. It is a conceptual diagram which shows an example of a reference picture list. It is a conceptual diagram which shows the example of a reference picture. It is the schematic which shows the structure of the image decoding apparatus 31 which concerns on this embodiment. It is the schematic which shows the structure of the inter prediction parameter decoding part 303 which concerns on this embodiment. It is the schematic which shows the structure of the merge mode parameter derivation | leading-out part 3036 which concerns on this embodiment. It is the schematic which shows the structure of the AMVP prediction parameter derivation | leading-out part 3032 which concerns on this embodiment. It is a figure which shows an example of a merge candidate list. It is a figure which shows the position of the adjacent block which a space merge candidate refers. It is a figure which shows the reference position of a depth in the division mode which concerns on this embodiment, and division flag derivation. It is a figure which shows the structure of the VSP merge candidate derivation | leading-out part 30374 (VSP prediction part 30374) of this embodiment. It is the schematic which shows the structure of the inter prediction parameter decoding control decoding part 303 which concerns on this embodiment. It is the schematic which shows the structure of the inter estimated image generation part 309 which concerns on this embodiment. It is the schematic which shows the structure of the residual prediction part 3092 which concerns on this embodiment. It is a conceptual diagram (in the case of a motion vector) of the residual prediction which concerns on this embodiment. It is a conceptual diagram (in the case of a disparity vector) of residual prediction according to the present embodiment. It is a syntax table regarding DBBP flag dbbp_flag of this embodiment. It is a block diagram which shows the structure of DBBP prediction part 3095C which concerns on this embodiment. It is a figure explaining the image synthetic | combination part 30953 which concerns on this embodiment. It is the schematic which shows the structure of the inter prediction parameter decoding control decoding part 3031A which concerns on this embodiment. It is a figure explaining derivation | leading-out of inter_pred_flag in the inter prediction parameter decoding control part 3031A which concerns on this embodiment. It is a block diagram which shows the structure of merge mode parameter derivation | leading-out part 3036A which concerns on this embodiment. It is a block diagram which shows the structure of the image coding apparatus 11 which concerns on this embodiment. It is the schematic which shows the structure of the inter prediction parameter encoding part 112 which concerns on this embodiment. It is a block diagram which shows the structure of the modification of DBBP prediction part 3095C which concerns on this embodiment. It is a flowchart for demonstrating operation | movement of the image decoding apparatus 31 and the image coding apparatus 11 of a structure which uses a common displacement vector in the VSP prediction part 30374 and the DBBP prediction part 3095. It is a figure which shows the data flow of the example which uses a common displacement vector in VSP prediction part 30374 and DBBP prediction part 3095. It is a figure which shows the data flow of the example using a common displacement vector in VSP prediction part 30374 and DBBP prediction part 3095C.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.

  The image transmission system 1 is a system that transmits a code obtained by encoding a plurality of layer images and displays an image obtained by decoding the transmitted code. The image transmission system 1 includes an image encoding device 11, a network 21, an image decoding device 31, and an image display device 41.

  A signal T indicating a plurality of layer images (also referred to as texture images) is input to the image encoding device 11. A layer image is an image that is viewed or photographed at a certain resolution and a certain viewpoint. When performing view scalable coding in which a three-dimensional image is coded using a plurality of layer images, each of the plurality of layer images is referred to as a viewpoint image. Here, the viewpoint corresponds to the position or observation point of the photographing apparatus. For example, the plurality of viewpoint images are images taken by the left and right photographing devices toward the subject. The image encoding device 11 encodes each of the signals to generate an encoded stream Te (encoded data). Details of the encoded stream Te will be described later. A viewpoint image is a two-dimensional image (planar image) observed at a certain viewpoint. The viewpoint image is indicated by, for example, a luminance value or a color signal value for each pixel arranged in a two-dimensional plane. Hereinafter, one viewpoint image or a signal indicating the viewpoint image is referred to as a picture. In addition, when performing spatial scalable coding using a plurality of layer images, the plurality of layer images include a base layer image having a low resolution and an enhancement layer image having a high resolution. When SNR scalable encoding is performed using a plurality of layer images, the plurality of layer images are composed of a base layer image with low image quality and an extended layer image with high image quality. Note that view scalable coding, spatial scalable coding, and SNR scalable coding may be arbitrarily combined. In the present embodiment, encoding and decoding of an image including at least a base layer image and an image other than the base layer image (enhancement layer image) is handled as the plurality of layer images. Of the multiple layers, for two layers that have a reference relationship (dependency relationship) in the image or encoding parameter, the image on the reference side is referred to as a first layer image, and the image on the reference side is referred to as a second layer image . For example, when there is an enhancement layer image (other than the base layer) that is encoded with reference to the base layer, the base layer image is treated as a first layer image and the enhancement layer image is treated as a second layer image. Note that examples of the enhancement layer image include an image of a viewpoint other than the base view and a depth image.

  A depth image (also referred to as a depth map, “depth image”, or “distance image”) is a signal value (“depth value”) corresponding to the distance from the viewpoint (such as a photographing device) of the subject or background included in the subject space. ”,“ Depth value ”,“ depth ”, etc.), and is an image signal composed of signal values (pixel values) for each pixel arranged in a two-dimensional plane. The pixels constituting the depth image correspond to the pixels constituting the viewpoint image. Therefore, the depth map is a clue for representing the three-dimensional object space by using the viewpoint image which is a reference image signal obtained by projecting the object space onto the two-dimensional plane.

  The network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional or bidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may be replaced by a storage medium that records an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray Disc).

  The image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates a plurality of decoded layer images Td (decoded viewpoint images Td) respectively decoded.

  The image display device 41 displays all or part of the plurality of decoded layer images Td generated by the image decoding device 31. For example, in view scalable coding, a 3D image (stereoscopic image) and a free viewpoint image are displayed in all cases, and a 2D image is displayed in some cases. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. In addition, in the spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have a high processing capability, a high-quality enhancement layer image is displayed and only a lower processing capability is provided. Displays a base layer image that does not require higher processing capability and display capability as an extension layer.

<Structure of Encoded Stream Te>
Prior to detailed description of the image encoding device 11 and the image decoding device 31 according to the present embodiment, the data structure of the encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described. .

  FIG. 3 is a diagram illustrating a hierarchical structure of data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. (A) to (f) of FIG. 3 respectively show a sequence layer that defines the sequence SEQ, a picture layer that defines the picture PICT, a slice layer that defines the slice S, a slice data layer that defines slice data, and slice data. It is a figure which shows the encoding unit layer which prescribes | regulates the encoding tree layer which prescribes | regulates the encoding tree unit contained, and the coding unit (Coding Unit; CU) contained in a coding tree.

(Sequence layer)
In the sequence layer, a set of data referred to by the image decoding device 31 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 3A, the sequence SEQ includes a video parameter set (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an additional extension. Information SEI (Supplemental Enhancement Information) is included. Here, the value indicated after # indicates the layer ID. FIG. 3 shows an example in which encoded data of # 0 and # 1, that is, layer 0 and layer 1, exists, but the type of layer and the number of layers are not dependent on this.

  The video parameter set VPS is a set of encoding parameters common to a plurality of moving images, a plurality of layers included in the moving image, and encoding parameters related to individual layers in a moving image composed of a plurality of layers. A set is defined.

  In the sequence parameter set SPS, a set of encoding parameters referred to by the image decoding device 31 in order to decode the target sequence is defined. For example, the width and height of the picture are defined.

  In the picture parameter set PPS, a set of encoding parameters referred to by the image decoding device 31 in order to decode each picture in the target sequence is defined. For example, a quantization width reference value (pic_init_qp_minus26) used for picture decoding and a flag (weighted_pred_flag) indicating application of weighted prediction are included. A plurality of PPS may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.

(Picture layer)
In the picture layer, a set of data referred to by the image decoding device 31 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 3B, the picture PICT includes slices S0 to SNS-1 (NS is the total number of slices included in the picture PICT).

  In addition, hereinafter, when it is not necessary to distinguish each of the slices S0 to SNS-1, the subscripts may be omitted. The same applies to data included in an encoded stream Te described below and to which other subscripts are attached.

(Slice layer)
In the slice layer, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 3C, the slice S includes a slice header SH and slice data SDATA.

  The slice header SH includes a coding parameter group that the image decoding device 31 refers to in order to determine a decoding method of the target slice. The slice type designation information (slice_type) that designates the slice type is an example of an encoding parameter included in the slice header SH.

  As slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

  Note that the slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the sequence layer.

(Slice data layer)
In the slice data layer, a set of data referred to by the image decoding device 31 in order to decode the slice data SDATA to be processed is defined. As shown in FIG. 3D, the slice data SDATA includes a coded tree block (CTB). The CTB is a fixed-size block (for example, 64 × 64) constituting a slice, and may be referred to as a maximum coding unit (LCU).

(Encoding tree layer)
As shown in (e) of FIG. 3, the coding tree layer defines a set of data referred to by the image decoding device 31 in order to decode a coding tree block to be processed. The coding tree unit is divided by recursive quadtree division. A tree-structured node obtained by recursive quadtree partitioning is called a coding tree. An intermediate node of the quadtree is a coded tree unit (CTU), and the coded tree block itself is also defined as the highest CTU. The CTU includes a split flag (split_flag). When the split_flag is 1, the CTU is split into four coding tree units CTU. When split_flag is 0, the coding tree unit CTU is divided into four coding units (CU: Coded Unit). The coding unit CU is a terminal node of the coding tree layer and is not further divided in this layer. The encoding unit CU is a basic unit of the encoding process.

  When the size of the coding tree block CTB is 64 × 64 pixels, the size of the coding unit CU is any of 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels. Can take.

(Encoding unit layer)
As shown in (f) of FIG. 3, the encoding unit layer defines a set of data referred to by the image decoding device 31 in order to decode the processing target encoding unit. Specifically, the encoding unit includes a CU header CUH, a prediction unit (prediction unit), a conversion tree, and a CU header CUF. In the CU header CUH, it is defined whether the coding unit is a unit using intra prediction or a unit using inter prediction. In the CU header CUH, the encoding unit indicates a residual prediction index iv_res_pred_weight_idx indicating a weight used for residual prediction (or whether or not to perform residual prediction), and illuminance indicating whether or not illuminance compensation prediction is used. A compensation flag ic_flag is included. The encoding unit is the root of a prediction unit (PU) and a transform tree (TT). The CU header CUF is included between the prediction unit and the conversion tree or after the conversion tree.

  In the prediction unit, the encoding unit is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the coding unit. Further, the prediction unit includes one or a plurality of prediction blocks obtained by the above-described division.

  Prediction processing is performed for each prediction block. Hereinafter, a prediction block that is a unit of prediction is also referred to as a prediction unit. More specifically, since prediction is performed in units of color components, hereinafter, blocks for each color component, such as a luminance prediction block and a color difference prediction block, are referred to as prediction blocks, and blocks of multiple color components (luminance prediction blocks). A block and a color difference prediction block) are collectively called a prediction unit. A block whose index cIdx (colour_component Idx) indicating a color component type is 0 indicates a luminance block (predicted luminance block) (usually displayed as L or Y), and a block whose cIdx is 1 or 2 is Cb, The Cr color difference block (color difference prediction block) is shown.

  Roughly speaking, there are two types of division in the prediction unit: intra prediction and inter prediction. Intra prediction is prediction within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).

  In the case of intra prediction, there are 2N × 2N (the same size as the encoding unit) and N × N division methods.

Also, in the case of inter prediction, the division method is encoded by the encoded data division mode part_mode. The division mode specified by the division mode part_mode includes the following eight types of patterns in total, assuming that the size of the target CU is 2N × 2N pixels. That is, 4 symmetric splittings of 2N × 2N pixels, 2N × N pixels, N × 2N pixels, and N × N pixels, and 2N × nU pixels, 2N × nD pixels, nL × 2N pixels, And four asymmetric motion partitions (AMP) of nR × 2N pixels. N = 2 ^m (m is an arbitrary integer of 1 or more). Hereinafter, a prediction block whose division mode is asymmetric division is also referred to as an AMP block. Since the number of divisions is one of 1, 2, and 4, PUs included in the CU are 1 to 4. These PUs are expressed as PU0, PU1, PU2, and PU3 in order.

  4A to 4H specifically illustrate the positions of the PU partition boundaries in the CU for each partition mode.

  FIG. 4A shows a 2N × 2N division mode in which no CU division is performed. FIGS. 4B and 4E show the shapes of partitions when the division modes are 2N × N and N × 2N, respectively. FIG. 4H shows the shape of the partition when the division mode is N × N.

  4 (c), (d), (f), and (g) are 2N × nU, 2N × nD, nL × 2N, and nR × 2N, which are asymmetric divisions (AMP), respectively. Shows the shape of the partition.

  Also, in FIGS. 4A to 4H, the numbers given to the respective regions indicate the region identification numbers, and the regions are processed in the order of the identification numbers. That is, the identification number represents the scan order of the area.

  In the prediction block in the case of inter prediction, seven types other than N × N (FIG. 4 (h)) are defined among the eight types of division modes.

  A specific value of N is defined by the size of the CU to which the PU belongs, and specific values of nU, nD, nL, and nR are determined according to the value of N. For example, 32 × 32 pixel CUs are 32 × 32 pixels, 32 × 16 pixels, 16 × 32 pixels, 32 × 16 pixels, 32 × 8 pixels, 32 × 24 pixels, 8 × 32 pixels, and 24 × 32. It can be divided into prediction blocks for inter prediction of pixels.

  In the transform tree, the encoding unit is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding unit. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.

  There are two types of division in the transformation tree: one in which an area having the same size as the encoding unit is allocated as a transformation block, and the other in division by recursive quadtree division, similar to the above-described division in the tree block.

  The conversion process is performed for each conversion block. Hereinafter, a transform block that is a unit of transform is also referred to as a transform unit (TU).

(Prediction parameter)
The prediction image of the prediction unit is derived by a prediction parameter associated with the prediction unit. The prediction parameters include a prediction parameter for intra prediction or a prediction parameter for inter prediction. Hereinafter, prediction parameters for inter prediction (inter prediction parameters) will be described. The inter prediction parameter includes prediction use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and vectors mvL0 and mvL1. The prediction use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list and L1 list are used, respectively, and a reference picture list corresponding to a value of 1 is used. In this specification, when “flag indicating whether or not XX” is described, 1 is XX, 0 is not XX, 1 is true and 0 is false in logical negation and logical product. (The same applies hereinafter). However, other values can be used as true values and false values in an actual apparatus or method. When two reference picture lists are used, that is, (predFlagL0, predFlagL1) = (1, 1) corresponds to bi-prediction, and when one reference picture list is used, that is, (predFlagL0, predFlagL1) = ( 1, 0) or (predFlagL0, predFlagL1) = (0, 1) corresponds to single prediction. Note that the prediction use flag information can also be expressed by an inter prediction identifier inter_pred_idc described later. Normally, a prediction use flag is used in a prediction image generation unit and a prediction parameter memory, which will be described later, and an inter prediction identifier inter_pred_idc is used when decoding information about which reference picture list is used from encoded data. .

  Syntax elements for deriving inter prediction parameters included in the encoded data include, for example, a partition mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector flag mvp_LX_flag, and a difference There is a vector mvdLX. Note that LX is a description method used when L0 prediction and L1 prediction are not distinguished. By replacing LX with L0 and L1, parameters for the L0 list and parameters for the L1 list are distinguished (the same applies hereinafter). For example, refIdxL0 is a reference picture index used for L0 prediction, refIdxL1 is a reference picture index used for L1 prediction, and refIdx (refIdxLX) is a notation used when refIdxL0 and refIdxL1 are not distinguished.

(Example of reference picture list)
Next, an example of the reference picture list will be described. The reference picture list is a column composed of reference pictures stored in the reference picture memory 306. FIG. 5 is a conceptual diagram showing an example of the reference picture list RefPicListX. In the reference picture list RefPicListX, five rectangles arranged in a line on the left and right indicate reference pictures, respectively. The codes P1, P2, Q0, P3, and P4 shown in order from the left end to the right are codes indicating respective reference pictures. P such as P1 indicates the viewpoint P, and Q of Q0 indicates a viewpoint Q different from the viewpoint P. The subscripts P and Q indicate the picture order number POC. A downward arrow directly below refIdxLX indicates that the reference picture index refIdxLX is an index that refers to the reference picture Q0 in the reference picture memory 306.

(Reference picture example)
Next, an example of a reference picture used for deriving a vector will be described. FIG. 6 is a conceptual diagram illustrating an example of a reference picture. In FIG. 6, the horizontal axis indicates the display time, and the vertical axis indicates the viewpoint. The rectangles shown in FIG. 6 with 2 rows and 3 columns (6 in total) indicate pictures. Among the six rectangles, the rectangle in the second column from the left in the lower row indicates a picture to be decoded (target picture), and the remaining five rectangles indicate reference pictures. A reference picture Q0 indicated by an upward arrow from the target picture is a picture that has the same display time as the target picture and a different viewpoint (view ID). In the displacement prediction based on the target picture, the reference picture Q0 is used. A reference picture P1 indicated by a left-pointing arrow from the target picture is a past picture at the same viewpoint as the target picture. A reference picture P2 indicated by a right-pointing arrow from the target picture is a future picture at the same viewpoint as the target picture. In motion prediction based on the target picture, the reference picture P1 or P2 is used.

(Inter prediction identifier and prediction usage flag)
The relationship between the inter prediction identifier inter_pred_idc and the prediction usage flags predFlagL0 and predFlagL1 is
inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
Can be converted to each other. Here, >> is a right shift, and << is a left shift. Therefore, as the inter prediction parameter, the prediction use flags predFlagL0 and predFlagL1 may be used, or the inter prediction identifier inter_pred_idc may be used. In addition, hereinafter, the determination using the prediction usage flags predFlagL0 and predFlagL1 may be replaced with the inter prediction identifier inter_pred_idc. Conversely, the determination using the inter prediction identifier inter_pred_idc can be replaced with the prediction use flags predFlagL0 and predFlagL1.

(Merge mode and AMVP prediction)
The prediction parameter decoding (encoding) method includes a merge mode and an AMVP (Adaptive Motion Vector Prediction) mode. The merge flag merge_flag is a flag for identifying these. In both the merge mode and the AMVP mode, the prediction parameter of the target PU is derived using the prediction parameter of the already processed block. The merge mode is a mode that uses the prediction parameter already derived without including the prediction use flag predFlagLX (inter prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the vector mvLX in the encoded data. The AMVP mode is an inter prediction identifier. In this mode, inter_pred_idc, reference picture index refIdxLX, and vector mvLX are included in the encoded data. The vector mvLX is encoded as a prediction vector flag mvp_LX_flag indicating a prediction vector and a difference vector (mvdLX).

  The inter prediction identifier inter_pred_idc is data indicating the type and number of reference pictures, and takes one of Pred_L0, Pred_L1, and Pred_BI. Pred_L0 and Pred_L1 indicate that reference pictures stored in reference picture lists called an L0 list and an L1 list are used, respectively, and that both use one reference picture (single prediction). Prediction using the L0 list and the L1 list are referred to as L0 prediction and L1 prediction, respectively. Pred_BI indicates that two reference pictures are used (bi-prediction), and indicates that two reference pictures stored in the L0 list and the L1 list are used. The prediction vector flag mvp_LX_flag is an index indicating a prediction vector, and the reference picture index refIdxLX is an index indicating a reference picture stored in the reference picture list. The merge index merge_idx is an index that indicates whether one of the prediction parameter candidates (merge candidates) derived from the processed block is used as a prediction parameter of the prediction unit (target block).

(Motion vector and displacement vector)
The vector mvLX includes a motion vector and a displacement vector (disparity vector). A motion vector is a positional shift between the position of a block in a picture at a certain display time of a layer and the position of the corresponding block in a picture of the same layer at a different display time (for example, an adjacent discrete time). It is a vector which shows. The displacement vector is a vector indicating a positional shift between the position of a block in a picture at a certain display time of a certain layer and the position of a corresponding block in a picture of a different layer at the same display time. The pictures in different layers may be pictures from different viewpoints or pictures with different resolutions. In particular, a displacement vector corresponding to pictures of different viewpoints is called a disparity vector. In the following description, when a motion vector and a displacement vector are not distinguished, they are simply referred to as a vector mvLX. A prediction vector and a difference vector related to the vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively. Whether the vector mvLX and the difference vector mvdLX are motion vectors or displacement vectors is determined using a reference picture index refIdxLX associated with the vectors.

(Configuration of image decoding device)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 7 is a schematic diagram illustrating a configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit 302, a reference picture memory (reference image storage unit, frame memory) 306, a prediction parameter memory (prediction parameter storage unit, frame memory) 307, and a prediction image generation unit 308. , An inverse quantization / inverse DCT unit 311, an addition unit 312, and a depth DV derivation unit 351 (not shown).

  The prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.

  The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside, and separates and decodes individual codes (syntax elements). The separated codes include prediction information for generating a prediction image and residual information for generating a difference image.

The entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302. Some of the separated codes are, for example, prediction mode PredMode, split mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector flag mvp_LX_flag, difference vector mvdLX, residual prediction index iv_res_pred_weight_idx and illuminance compensation flag ic_flag. Control of which code to decode is performed based on an instruction from the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311. This quantization coefficient is a coefficient obtained by performing quantization and performing DCT (Discrete Cosine Transform) on the residual signal in the encoding process. The entropy decoding unit 301 outputs the depth DV conversion table DepthToDisparityB to the depth DV deriving unit 351. The depth DV conversion table DepthToDisparityB is a table for converting the pixel value of the depth image into a parallax indicating the displacement between the viewpoint images, and an element DepthToDisparityB [d] of the depth DV conversion table DepthToDisparityB has an inclination cp_scale and an offset cp_off, Using the slope precision cp_precision,
log2Div = BitDepth _Y -1 + cp_precision
offset = (cp_off << BitDepthY) + ((1 << log2Div) >> 1)
scale = cp_scale
DepthToDisparityB [d] = (scale * d + offset) >> log2Div
It can be calculated by the following formula. The parameters cp_scale, cp_off, and cp_precision are decoded from the parameter set in the encoded data for each viewpoint to be referred to. BitDepthY indicates the bit depth of the pixel value corresponding to the luminance signal, and takes, for example, 8 as the value.

  The prediction parameter decoding unit 302 receives a part of the code from the entropy decoding unit 301 as an input. The prediction parameter decoding unit 302 decodes the prediction parameter corresponding to the prediction mode indicated by the prediction mode PredMode that is a part of the code. The prediction parameter decoding unit 302 outputs the prediction mode PredMode and the decoded prediction parameter to the prediction parameter memory 307 and the prediction image generation unit 308.

  Based on the code input from the entropy decoding unit 301, the inter prediction parameter decoding unit 303 refers to the prediction parameter stored in the prediction parameter memory 307 and decodes the inter prediction parameter. The inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.

  Based on the code input from the entropy decoding unit 301, the intra prediction parameter decoding unit 304 refers to the prediction parameter stored in the prediction parameter memory 307 and decodes the intra prediction parameter. The intra prediction parameter is a parameter used in a process of predicting a picture block within one picture, for example, an intra prediction mode IntraPredMode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.

  The reference picture memory 306 stores the decoded picture block recSamples generated by the addition unit 312 at the position of the decoded picture block.

  The prediction parameter memory 307 stores the prediction parameter at a predetermined position for each picture and block to be decoded. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode PredMode separated by the entropy decoding unit 301. . The stored inter prediction parameters include, for example, a prediction use flag predFlagLX, a reference picture index refIdxLX, and a vector mvLX.

  The prediction image generation unit 308 receives the prediction mode PredMode and the prediction parameter from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates prediction picture blocks predSamples (prediction images) using the input prediction parameter and the read reference picture in the prediction mode indicated by the prediction mode PredMode.

  Here, when the prediction mode PredMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameters input from the inter prediction parameter decoding unit 303 and the read reference pictures to perform prediction picture block predSamples by inter prediction. Is generated. The prediction picture block predSamples corresponds to the prediction unit PU. The PU corresponds to a part of a picture composed of a plurality of pixels as a unit for performing the prediction process as described above, that is, a target block on which the prediction process is performed at a time.

  For the reference picture list RefPicListLX in which the prediction usage flag predFlagLX is 1, the inter predicted image generation unit 309 is located at the position indicated by the vector mvLX with reference to the prediction unit from the reference picture RefPicListLX [refIdxLX] indicated by the reference picture index refIdxLX. A reference picture block is read from the reference picture memory 306. The inter prediction image generation unit 309 performs motion compensation on the read reference picture block to generate prediction picture blocks predSamplesLX. The inter prediction image generation unit 309 further generates prediction picture blocks predSamples by weighted prediction from the prediction picture blocks predSamplesL0 and predSamplesL1 derived from the reference pictures in each reference picture list, and outputs the prediction picture blocks predSamples to the addition unit 312.

  When the prediction mode PredMode indicates the intra prediction mode, the intra predicted image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference picture. Specifically, the intra predicted image generation unit 310 reads, from the reference picture memory 306, a reference picture block that is a decoding target picture and is in a predetermined range from a prediction unit among blocks that have already been processed. The predetermined range is, for example, the range of adjacent blocks on the left, upper left, upper, and upper right, and differs depending on the intra prediction mode.

  The intra predicted image generation unit 310 performs prediction in the prediction mode indicated by the intra prediction mode IntraPredMode for the read reference picture block, generates predicted picture block predSamples, and outputs the prediction picture block predSamples to the adding unit 312.

  The inverse quantization / inverse DCT unit 311 performs inverse quantization on the quantization coefficient input from the entropy decoding unit 301 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 311 performs inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 311 outputs the calculated decoded residual signal to the adder 312.

  The addition unit 312 performs pixel value processing on the prediction picture block predSamples input from the inter prediction image generation unit 309 and the intra prediction image generation unit 310 and the signal value resSamples of the decoded residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. To generate a decoded picture block recSamples. The adder 312 outputs the generated decoded picture block recSamples to the reference picture memory 306. The decoded picture block is integrated for each picture. A loop filter such as a deblocking filter and an adaptive offset filter is applied to the decoded picture. The decoded picture is output to the outside as a decoded layer image Td.

(Configuration of inter prediction parameter decoding unit)
Next, the configuration of the inter prediction parameter decoding unit 303 will be described. FIG. 8 is a schematic diagram illustrating a configuration of the inter prediction parameter decoding unit 303 according to the present embodiment. The inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter deriving unit 3032, an adding unit 3035, a merge mode parameter deriving unit 3036, and a displacement deriving unit 30363.

  The inter prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode a code related to the inter prediction (the syntax element) includes, for example, a division mode part_mode, a merge included in the encoded data. A flag merge_flag, a merge index merge_idx, an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector flag mvp_LX_flag, a difference vector mvdLX, a residual prediction index iv_res_pred_weight_idx, an illumination compensation flag ic_flag, and a DBBP flag dbbp_flag are extracted. When 3031 expresses that a certain syntax element is extracted, it means that the entropy decoding unit 301 is instructed to decode a certain syntax element, and the corresponding syntax element is read out from the encoded data.

  When the merge flag merge_flag is 1, that is, when the prediction unit is in the merge mode, the inter prediction parameter decoding control unit 3031 extracts the merge index merge_idx from the encoded data. The inter prediction parameter decoding control unit 3031 outputs the extracted residual prediction index iv_res_pred_weight_idx, the illumination compensation flag ic_flag, and the merge index merge_idx to the merge mode parameter deriving unit 3036.

  When the merge flag merge_flag is 0, that is, when the prediction block is in the AMVP prediction mode, the inter prediction parameter decoding control unit 3031 uses the entropy decoding unit 301 to calculate the inter prediction identifier inter_pred_idc, the reference picture index refIdxLX, and the prediction vector flag. mvp_LX_flag and difference vector mvdLX are extracted. The inter prediction parameter decoding control unit 3031 outputs the prediction use flag predFlagLX derived from the extracted inter prediction identifier inter_pred_idc and the reference picture index refIdxLX to the AMVP prediction parameter derivation unit 3032 and the prediction image generation unit 308, and also the prediction parameter memory 307 To remember. The inter prediction parameter decoding control unit 3031 outputs the extracted prediction vector flag mvp_LX_flag to the AMVP prediction parameter derivation unit 3032 and outputs the extracted difference vector mvdLX to the addition unit 3035.

  The inter prediction parameter decoding control unit 3031 decodes the DBBP flag dbbp_flag from the encoded data when the division mode PartMode is a specific value. In other cases, when dbbp_flag is not included in the encoded data, 0 is estimated as dbbp_flag. FIG. 20 is a syntax table related to the DBBP flag dbbp_flag of this embodiment. The inter prediction parameter decoding control unit 3031 decodes cu_skip_flag, pred_mode, part_mode, and dbbp_flag shown in SE1001 to SE1004 in the figure. Here, cu_skip_flag is a flag indicating whether or not the target CU is skipped. In the case of skipping, PartMode is limited to 2N × 2N, and decoding of the split mode part_mode is omitted. The division mode part_mode decoded from the encoded data is set to the division mode PredMode. In this example, the inter prediction parameter decoding control unit 3031 decodes dbbp_flag when the partition mode PartMode (= part_mode) is 2N × N, but may decode dbbp_flag when the partition mode is another value. . Further, the dbbp flag may be derived by a method different from the above.

  In addition, the inter prediction parameter decoding control unit 3031 displays the displacement vector (NBDV) derived when the inter prediction parameter is derived and the VSP mode flag VspModeFlag that is a flag indicating whether to perform viewpoint synthesis prediction, as an inter prediction image generation unit. To 309.

  FIG. 9 is a schematic diagram illustrating a configuration of the merge mode parameter deriving unit 3036 according to the present embodiment. The merge mode parameter derivation unit 3036 includes a merge candidate derivation unit 30361, a merge candidate selection unit 30362, and a bi-prediction restriction unit 30363. The merge candidate derivation unit 30361 includes a merge candidate storage unit 303611, an extended merge candidate derivation unit 30370, and a basic merge candidate derivation unit 30380.

  The merge candidate storage unit 303611 stores the merge candidates input from the extended merge candidate derivation unit 30370 and the basic merge candidate derivation unit 30380 in the merge candidate list mergeCandList. The merge candidate includes a prediction usage flag predFlagLX, a vector mvLX, a reference picture index refIdxLX, a VSP mode flag VspModeFlag, a displacement vector MvDisp, and a layer ID RefViewIdx. In the merge candidate storage unit 303611, an index is assigned to the merge candidates stored in the merge candidate list mergeCandList according to a predetermined rule.

  FIG. 11 shows an example of the merge candidate list mergeCandList derived by the merge candidate deriving unit 30361. FIG. 11A shows merge candidates derived by the merge candidate storage unit 303611 in the base layer (layer IDnal_unit_layer = 0). If two merge candidates have the same prediction parameter, excluding the process of reducing the order (pruning process), the merge index order is spatial merge candidate (A1), spatial merge candidate (B1), spatial merge candidate (B0), and space The merge candidate (A0) and the spatial merge candidate (B2) are in this order. The parentheses are nicknames of merge candidates, and in the case of spatial merge candidates, they correspond to the positions of reference blocks used for derivation. Further, there are a merge merge candidate and a zero merge candidate thereafter, which are omitted in FIG. These merge candidates, that is, the spatial merge candidate, temporal merge candidate, join merge candidate, and zero merge candidate are derived by the basic merge candidate deriving unit 30380. FIG. 11B shows merge candidates derived by the merge candidate storage unit 303611 in the enhancement layer (layer IDnal_unit_layer! = 0) that is a layer other than the base layer. Texture merge candidate (T), interview merge candidate (IvMC), spatial merge candidate (A1), spatial merge candidate (B1), spatial merge candidate (B0), displacement merge candidate (IvDC), VSP merge in order of merge index Candidate (VSP), spatial merge candidate (A0), spatial merge candidate (B2), motion shift merge candidate (IvMCShift), displacement shift merge candidate (IvDCShift), and temporal merge candidate (Col). The parentheses are nicknames of merge candidates. Further, there are a merge merge candidate and a zero merge candidate thereafter, which are omitted in FIG. Texture merge candidate (T), Interview merge candidate (IvMC), Displacement merge candidate (IvDC), VSP merge candidate (VSP), Motion shift merge candidate (IvMCShift), Displacement shift merge candidate (IvDCShift) are derived as extended merge candidates. Derived in part 30370.

  FIG. 12 is a diagram illustrating the positions of adjacent blocks to which spatial merge candidates refer. A0, A1, B0, B1, and B2 each correspond to the position shown in FIG. 12, and the coordinates are as follows. When the upper left coordinates of the prediction unit are xPb and yPb, and the width and height of the prediction unit are nPbW and nPbH, the positions of adjacent blocks are as follows.

A0: (xPb-1, yPb + nPbH)
A1: (xPb-1, yPb + nPbH-1)
B0: (xPb + nPbW, yPb-1)
B1: (xPb + nPbW-1, 1, yPb-1)
B2: (xPb-1, yPb-1)
The extended merge candidate derivation unit 30370 includes an inter-layer merge candidate derivation unit 30371 (interview merge candidate derivation unit 30371), a displacement merge candidate derivation unit 30373, and a VSP merge candidate derivation unit 30374 (VSP prediction unit 30374). . The extended merge candidate is a merge candidate different from a basic merge candidate described later, and includes at least a texture merge candidate (T), an interview merge candidate (IvMC), a displacement merge candidate (IvDC), a VSP merge candidate (VSP), Either a motion shift merge candidate (IvMCShift) or a displacement shift merge candidate (IvDCShift) is included.

(Texture merge candidate)
The inter-layer merge candidate derivation unit 30371 derives a texture merge candidate (T), an inter-view merge candidate (IvMC), and a motion shift merge candidate (IvMCShift). For these merge candidates, a block corresponding to a prediction unit is selected from reference pictures of different layers (for example, a base layer and a base view) having the same POC as the target picture, and a prediction parameter that is a motion vector included in the block is selected as a prediction parameter. It is derived by reading from the memory 307.

The texture merge candidate (T) is derived by the inter-layer merge candidate deriving unit 30371 when the target picture is depth. The texture merge candidate (T) is derived by specifying a reference block from a depth picture having the same view ID as the target picture and reading a motion vector of the reference block.
The coordinates (xRef, yRef) of the reference block are derived from the following equations when the upper left coordinates of the prediction unit are xPb and yPb, and the width and height of the prediction unit are nPbW and nPbH.

xRefFull = xPb + ((nPbW-1) >> 1)
yRefFull = yPb + ((nPbH-1) >> 1)
xRef = Clip3 (0, PicWidthInSamplesL-1, (xRefFull >> 3) << 3)
yRef = Clip3 (0, PicHeightInSamplesL-1, (yRefFull >> 3) << 3)
Note that PicWidthInSamples _L and PicHeightInSamples _L represent the width and height of the image, respectively, and the function Clip3 (x, y, z) restricts (clips) z to not less than x and not more than y, and returns the restricted result. It is a function.

  If the motion vector of the reference block is textMvLX, the motion vector mvLXT of the texture merge candidate is derived by the following equation.

mvLXT [0] = (textMvLX [xRef] [yRef] [0] + 2) >> 2
mvLXT [1] = (textMvLX [xRef] [yRef] [1] + 2) >> 2
For texture merge candidates, prediction parameters may be assigned in units of sub-blocks obtained by further dividing the prediction unit.

(Interview merge candidate)
The inter-view merge candidate moves from the reference block of the reference picture ivRefPic having the same POC as the target picture specified by the later-described displacement vector deriving unit 352 in the inter-layer merge candidate deriving unit 30371 and having a different view ID (refViewIdx). It is derived by reading prediction parameters such as vectors. This process is called a temporal inter-view motion candidate derivation process. As the temporal inter-view motion candidate derivation process, the inter-layer merge candidate derivation unit 30371 is first derived from the upper left coordinates of the block (xPb, yPb), the block width and height from the nPbW, nPbH, and the displacement vector derivation unit 352. Reference coordinates (xRef, yRef) are derived from the following equations when the displacement vector to be used is (mvDisp [0], mvDisp [1]).

xRefFull = xPb + (nPbW >> 1) + ((mvDisp [0] + 2) >> 2)
yRefFull = yPb + (nPbH >> 1) + ((mvDisp [1] + 2) >> 2)
xRef = Clip3 (0, PicWidthInSamplesL-1, (xRefFull >> 3) << 3)
yRef = Clip3 (0, PicHeightInSamplesL-1, (yRefFull >> 3) << 3)
Next, the inter-layer merge candidate derivation unit 30371 performs temporal inter-view motion candidate derivation processing in a temporal inter-view motion candidate derivation unit 303711 (not shown).

The temporal inter-view motion candidate derivation unit 3037111 derives the reference block position (xRef, yRef) from the block coordinates (xPb, yPb), the block widths nPbW, nPbH, and the block displacement vector mvDisp by the above processing, and further A vector of a temporal inter-view motion candidate is derived by referring to the vector of the prediction unit on the reference picture ivRefPic located at the reference block position (xRef, yRef). First, reference block position (xRef, yRef)
(XIvRefPb, yIvRefPb) is the upper left coordinate of the prediction unit (luminance prediction block) on the reference picture ivRefPic including the coordinates indicated by the reference picture list, prediction list flag, vector, and reference picture index included in the prediction unit on the reference picture ivRefPic Are set as refPicListLYIvRef, predFlagLYIvRef [x] [y], mvLYIvRef [x] [y], and refIdxLYIvRef [x] [y], respectively.

  When the prediction usage flag predFlagLYIvRef [xIvRefPb] [yIvRefPb] is 1, the temporal inter-view motion candidate derivation unit 3037111 performs prediction on the reference picture ivRefPic for the index i from 0 to the reference picture list element number −1 (num_ref_idx_lX_active_minus1). It is determined whether or not PicOrderCnt (RefPicListLYIvRef [refIdxLYIvRef [xIvRefPb] [yIvRefPb]]) that is the unit POC is equal to PicOrderCnt (RefPicListLX [i]) that is the POC of the reference picture of the target prediction unit (that is, mvLYIRef). When [xIvRefPb] [yIvRefPb] is a displacement vector), the predictable flag availableFlagLXInterView, the vector mvLXInterView, and the reference picture index refIdxLX are derived by the following equations.

availableFlagLXInterView = 1
mvLXInterView = mvLYIvRef [xIvRefPb] [yIvRefPb]
refIdxLX = i
That is, the temporal inter-view motion candidate derivation unit 303711 predicts the prediction unit on the reference picture ivRefPic when the reference picture referred to by the target prediction unit and the reference picture referenced by the prediction unit on the reference picture ivRefPic are the same. A vector mvLXInterView and a reference picture index refIdxLX are derived using parameters. In the inter-view merge candidate, the prediction parameter may be assigned in units of sub-blocks obtained by further dividing the prediction unit. For example, when the width and height of the prediction unit are nPbW and nPbH, and the minimum size of the subblock is SubPbSize, the width nSbW and the height nSbH of the subblock are derived by the following equations.

nSbW = nPbW / SubPbSize <= 1? nPbW: SubPbSizen
nSbH = nPbH / SubPbSize <= 1? nPbH: SubPbSize
Subsequently, the temporal inter-view motion candidate derivation unit 3037111 described above applies the vector spMvLX [xBlk] [yBlk], the reference picture index spRefIdxLX [xBlk] [yBlk], and the prediction usage flag spPredFlagLX [xBlk] [yBlk] ] Is derived.

  Where (xBlk, yBlk) is the relative coordinates in the prediction unit of the sub-block (coordinates based on the upper left coordinate of the prediction unit), from 0 (nPbW / nSbW-1), 0 to (nPbH / nSbH, respectively) -Takes an integer value of 1). If the coordinates of the prediction unit are (xPb, yPb) and the relative coordinates (xBlk, yBlk) in the prediction unit of the sub-block, the coordinates in the picture of the sub-block are (xPb + xBlk * nSbW, yPb + yBlk * nSbH) Expressed.

  The intra-picture coordinates (xPb + xBlk * nSbW, yPb + yBlk * nSbH) of the sub-block, the width nSbW and the height nSbH of the sub-block, (xPb, yPb), nPbW, As nPbH, temporal inter-view motion candidate derivation processing is performed in units of sub-blocks.

  In the above processing, the temporal interview motion candidate derivation unit 3037111 supports subblocks from the intermerge candidate vector mvLXInterView, the reference picture index refIdxLXInterView, and the prediction usage flag availableFlagLXInterView for the subblock for which the predictable flag availableFlagLXInterView is 0. A vector spMvLX, a reference picture index spRefIdxLX, and a prediction usage flag spPredFlagLX are derived by the following equations.

spMvLX [xBlk] [yBlk] = mvLXInterView
spRefIdxLX [xBlk] [yBlk] = refIdxLXInterView
spPredFlagLX [xBlk] [yBlk] = availableFlagLXInterView
Note that xBlk and yBlk are sub-block addresses and take values from 0 to (nPbW / nSbW-1) and from 0 to (nPbH / nSbH-1), respectively. Note that the vector mvLXInterView, the reference picture index refIdxLXInterView, and the prediction usage flag availableFlagLXInterView are (xPb + (nPbW / nSbW / 2) * nSbW, yPb + (nPbH / nSbH / 2) * nSbH) with reference block coordinates As a temporal inter-view motion candidate derivation process.

(Motion shift merge candidate)
The motion shift merge candidate also reads a prediction parameter such as a motion vector from a reference block of a picture having the same POC as the target picture identified by the displacement vector deriving unit 352 and having a different view ID in the inter-layer merge candidate deriving unit 30371. It is derived by this. The coordinates (xRef, yRef) of the reference block, the upper left coordinates of the prediction unit are xPb, yPb, the width and height of the prediction unit are nPbW, nPbH, and the displacement vector derived from the displacement vector deriving unit 352 is mvDisp [0], When mvDisp [1], it is derived from the following equation.

xRefFull = xPb + (nPbW >> 1) + ((mvDisp [0] + nPbW * 2 + 4 + 2) >> 2
)
yRefFull = yPb + (nPbH >> 1) + ((mvDisp [1] + nPbH * 2 + 4 + 2) >> 2
)
xRef = Clip3 (0, PicWidthInSamplesL-1, (xRefFull >> 3) << 3)
yRef = Clip3 (0, PicHeightInSamplesL-1, (yRefFull >> 3) << 3)
(Displacement merge candidate)
The displacement merge candidate derivation unit 30373 derives a displacement merge candidate (IvDC) and a shift displacement merge candidate (IvDcShift) from the displacement vector input from the displacement vector derivation unit 352. The displacement merge candidate derivation unit 30373 is the horizontal component mvDisp [0] of the displacement vector (mvDisp [0], mvDisp [1]) to which the horizontal component is input as the displacement merge candidate (IvDC), and the vertical component is 0. A vector is generated by the following equation.

mvL0IvDC [0] = DepthFlag? (mvDisp [0] + 2) >> 2: mvDisp [0]
mvL0IvDC [1] = 0
Here, DepthFlag is a variable that becomes 1 in the case of depth.

  The displacement merge candidate derivation unit 30373 stores the generated vector and the reference picture index refIdxLX of the previous layer image pointed to by the displacement vector (for example, the index of the base layer image having the same POC as the decoding target picture) as a merge candidate. Output to the unit 303611.

  The displacement merge candidate derivation unit 30373 derives, as a shift displacement merge candidate (IvDC), a merge candidate having a vector obtained by shifting the displacement merge candidate in the horizontal direction by the following expression.

mvLXIvDCShift [0] = mvL0IvDC [0] + 4
mvLXIvDCShift [1] = mvL0IvDC [1]
(VSP merge candidate)
The VSP merge candidate derivation unit 30374 (hereinafter, VSP prediction unit 30374) derives a VSP (View Synthesis Prediction) merge candidate. The VSP prediction unit 30374 divides the prediction unit into a plurality of sub-blocks (sub-prediction units), and sets the vector mvLX, the reference picture index refIdxLX, and the view ID RefViewIdx for each divided sub-block. The VSP prediction unit 30374 outputs the derived VSP merge candidate to the merge candidate storage unit 303611.

  FIG. 14 is a block diagram showing the relationship between the VSP prediction unit 30374 and other means. The VSP prediction unit 30374 operates using the split flag horSplitFlag derived by the split flag deriving unit 353 and the displacement vector disparitySamples derived by the depth DV deriving unit 351.

  The partition division unit (not shown) of the VSP prediction unit 30374 selects either a horizontally long rectangle (here 8 × 4) or a vertically long rectangle (here 4 × 8) according to the partition flag horSplitFlag derived by the partition flag deriving unit 353. The sub-block size is determined by selection. Specifically, the sub-block width nSubBlkW and height nSubBlkH are set using the following equations.

nSubBlkW = horSplitFlag? 8: 4
nSubBlkH = horSplitFlag? 4: 8
The depth vector derivation unit (not shown) of the VSP prediction unit 30374 uses the motion vector disparitySamples [] derived from the depth DV derivation unit 351 for each subblock of the derived subblock size, as a horizontal component motion vector mvLX [ A vector mvLX [] is derived using 0], 0 as a vertical component motion vector mvLX [1], and a prediction parameter of a VSP merge candidate is derived.

  Also, the VSP prediction unit 30374 controls whether or not to add the VSP merge candidate to the merge candidate list mergeCandList according to the residual prediction index iv_res_pred_weight_idx and the illumination compensation flag ic_flag input from the inter prediction parameter decoding control unit 3031. Also good. Specifically, the VSP prediction unit 30374 may add the VSP merge candidate to the elements of the merge candidate list mergeCandList only when the residual prediction index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0.

  The basic merge candidate derivation unit 30380 includes a spatial merge candidate derivation unit 30382, a temporal merge candidate derivation unit 30382, a combined merge candidate derivation unit 30383, and a zero merge candidate derivation unit 30384. The basic merge candidate is a merge candidate used in the base layer, that is, a merge candidate used in HEVC (for example, HEVC main profile) instead of scalable, and includes at least one of a spatial merge candidate and a temporal merge candidate.

  The spatial merge candidate derivation unit 30381 reads the prediction parameters (prediction usage flag predFlagLX, vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule, and uses the read prediction parameters as spatial merge candidates. To derive. Prediction parameters to be read are predictions related to each of adjacent blocks that are blocks within a predetermined range from the prediction unit (for example, all or a part of blocks that touch the lower left end, the upper left end, and the upper right end of the prediction unit, respectively). It is a parameter. The derived spatial merge candidate is stored in the merge candidate storage unit 303611.

  The spatial merge candidate derivation unit 30381 sets a merge candidate VSP mode flag mergeCandIsVspFlag that is derived by inheriting the VSP mode flag VspModeFlag of the adjacent block. That is, when the VSP mode flag VspModeFlag of the adjacent block is 1, the VSP mode flag mergeCandIsVspFlag of the corresponding spatial merge candidate is 1, and otherwise, the VSP mode flag mergeCandIsVspFlag is 0.

  Hereinafter, the VSP mode flag VspModeFlag is set to 0 in the merge candidates derived by the time merge candidate derivation unit 30382, the merge merge candidate derivation unit 30383, and the zero merge candidate derivation unit 30384.

  The temporal merge candidate derivation unit 30382 reads the prediction parameter of the block in the reference image including the lower right coordinate of the prediction unit from the prediction parameter memory 307 and sets it as a merge candidate. The reference image can be specified by using, for example, the collocated picture col_ref_idx specified by the slice header and the reference picture index refIdxLX specified by RefPicListX [col_ref_idx] specified by the reference picture list RefPicListX. It is stored in the merge candidate storage unit 303611.

  The merge merge candidate derivation unit 30383 derives a merge merge candidate by combining two different derived merge candidate vectors and reference picture indexes that have already been derived and stored in the merge candidate storage unit 303611 as L0 and L1 vectors, respectively. To do. The derived merge candidates are stored in the merge candidate storage unit 303611.

  The zero merge candidate derivation unit 30384 derives merge candidates whose reference picture index refIdxLX is i and whose X component and Y component of the vector mvLX are both 0 until the number of derived merge candidates reaches the maximum value. The value of i indicating the reference picture index refIdxLX is assigned in order from 0. The derived merge candidates are stored in the merge candidate storage unit 303611.

  The merge candidate selection unit 30362 selects, from the merge candidates stored in the merge candidate storage unit 303611, a merge candidate to which an index corresponding to the merge index merge_idx input from the inter prediction parameter decoding control unit 3031 is assigned. As an inter prediction parameter. That is, when the merge candidate list is mergeCandList, the prediction parameter indicated by mergeCandList [merge_idx] is selected and output to the bi-prediction restriction unit 30363.

  Merge candidate selection section 30362 sets subblock motion compensation flag subPbMotionFlag to 1 when an inter-view merge candidate is selected as a merge candidate. The merge candidate selection unit 30362 may set the sub-block motion compensation flag subPbMotionFlag to 1 even when the merge candidate VSP mode flag vspModeFlag is 1. In other cases, the sub-block motion compensation flag subPbMotionFlag is set to 0.

The bi-prediction restriction unit 30363 sets the reference picture index refIdxL1 of L1 and the prediction use flag predFlagL1 of L1 in the case of the following bi-prediction restriction condition 1 to:
refIdxL1 = -1, predFlagL1 = 0
To convert bi-prediction (predFlagL0 = 1 and predFlagL1 = 1) to single prediction.

Bi-prediction restriction condition 1: The selected prediction parameter is bi-prediction (predFlagL0 = 1 and predFlagL1 = 1) and the size of the prediction unit is smaller than a predetermined size (the sum of the width nOrigPbW and height nOrigPbH of the prediction unit is equal to 12) )
The bi-prediction restriction unit 30363 stores the selected merge candidate in the prediction parameter memory 307 and outputs it to the prediction image generation unit 308.

predSamplesLX´ [x] [y] = predSamplesLX [x] [y]
FIG. 10 is a schematic diagram illustrating a configuration of the AMVP prediction parameter derivation unit 3032 according to the present embodiment. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033, a prediction vector selection unit 3034, and an inter prediction identifier derivation unit 3035. The vector candidate derivation unit 3033 reads a vector stored in the prediction parameter memory 307 based on the reference picture index refIdx, and generates a vector candidate list mvpListLX. The reference block is a block (for example, a block at the lower left end, an upper right end, or a temporally adjacent block of the prediction unit) at a predetermined position based on the position of the prediction unit.

  The prediction vector selection unit 3034 selects the vector mvpListLX [mvp_lX_flag] indicated by the prediction vector flag mvp_LX_flag input from the inter prediction parameter decoding control unit 3031 as the prediction vector mvpLX from the vector candidates mvpListLX derived by the vector candidate derivation unit 3033. . The prediction vector selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3035.

  The addition unit 3035 adds the prediction vector mvpLX input from the prediction vector selection unit 3034 and the difference vector mvdLX input from the inter prediction parameter decoding control unit to calculate a vector mvLX. The adding unit 3035 outputs the calculated vector mvLX to the predicted image generation unit 308.

  FIG. 15 is a block diagram illustrating a configuration of the inter prediction parameter decoding control unit 3031 according to the embodiment of this invention. As illustrated in FIG. 15, the inter prediction parameter decoding control unit 3031 includes a split mode decoding unit 30311, an inter prediction identifier decoding unit 30312, a DBBP flag decoding unit 30313, and a merge flag decoding unit, a merge index decoding unit, and inter prediction that are not illustrated. An identifier decoding unit, a reference picture index decoding unit, a vector candidate index decoding unit, a vector difference decoding unit, a residual prediction index decoding unit, and an illuminance compensation flag decoding unit are configured. The partition mode decoding unit, the merge flag decoding unit, the merge index decoding unit, the reference picture index decoding unit, the vector candidate index decoding unit, and the vector difference decoding unit are respectively divided mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc The reference picture index refIdxLX, the prediction vector flag mvp_LX_flag, and the difference vector mvdLX are decoded.

  The inter prediction identifier decoding unit 30312 decodes an inter prediction identifier inter_pred_flag indicating that the prediction unit indicates L0 prediction (PRED_L0), L0 prediction (PRED_L1), and bi-prediction (PRED_BI).

  The residual prediction index decoding unit uses the entropy decoding unit 301 to decode the residual prediction index iv_res_pred_weight_idx from the encoded data when the division mode PartMode (part_mode) of the encoding unit CU is 2Nx2N. In other cases, the residual prediction index decoding unit sets (infers) 0 to iv_res_pred_weight_idx. The residual prediction index decoding unit outputs the decoded residual prediction index iv_res_pred_weight_idx to the merge mode parameter derivation unit 3036 and the inter prediction image generation unit 309. The residual prediction index is a parameter for changing the operation of residual prediction. In this embodiment, it is an index indicating the weight of residual prediction, and takes values of 0, 1, and 2. When iv_res_pred_weight_idx is 0, residual prediction is not performed. Note that the vector used for residual prediction may be changed instead of changing the weight of residual prediction according to the index. Instead of the residual prediction index, a flag (residual prediction flag) indicating whether to perform residual prediction may be used.

The illuminance compensation flag decoding unit uses the entropy decoding unit 301 to decode the illuminance compensation flag ic_flag from the encoded data when the division mode PartMode is 2Nx2N. In other cases, the illuminance compensation flag decoding unit sets (infers) 0 to ic_flag. The illuminance compensation flag decoding unit outputs the decoded illuminance compensation flag ic_flag to the merge mode parameter derivation unit 3036 and the inter predicted image generation unit 309.
Hereinafter, the displacement vector deriving unit 352, the division flag deriving unit 353, and the depth DV deriving unit 351, which are means used for deriving the prediction parameters, will be described in order.

(Displacement vector deriving unit 352)
The displacement vector deriving unit 352 stores the displacement vector (hereinafter referred to as MvDisp [x] [y] or mvDisp [x] [y]) of the encoding unit (target CU) to which the target PU belongs in the space of the encoding unit. From adjacent blocks temporally or temporally. Specifically, a block Col that is temporally adjacent to the target CU, a second block AltCol that is temporally adjacent, a block A1 that is spatially adjacent to the left, and a block B1 that is adjacent to the top are used as reference blocks. A block prediction flag predFlagLX, a reference picture index refIdxLX, and a vector mvLX are extracted in order. When the extracted vector mvLX is a displacement vector, the displacement vector of the adjacent block is output. If there is no displacement vector in the prediction parameter of the adjacent block, the prediction parameter of the next adjacent block is read and the displacement vector is derived in the same manner. If displacement vectors cannot be derived in all adjacent blocks, a zero vector is output as a displacement vector. The displacement vector deriving unit 352 also outputs a reference picture index and a view ID (RefViewIdx [x] [y], where (xP, yP) are coordinates) of the block from which the displacement vector is derived.

  The displacement vector obtained by the above is called NBDV (Neighbour Base Disparity Vector). The displacement vector deriving unit 352 further outputs the obtained displacement vector NBDV to the depth DV deriving unit 351. The depth DV deriving unit 351 derives depth-derived displacement vectors (displacement array disparitySamples). The depth DV deriving unit 351 updates (refines) the displacement vector by using the displacement vector disparitySamples obtained from the depth as the horizontal component mvLX [0] of the motion vector. The updated displacement vector is called DoNBDV (Depth Orientated Neighbor Base Disparity Vector). The displacement vector deriving unit 352 outputs the displacement vector (DoNBDV) to the inter-layer merge candidate deriving unit 30371, the displacement merge candidate deriving unit, and the viewpoint synthesis prediction merge candidate deriving unit. Further, the obtained displacement vector (NBDV) is output to the inter predicted image generation unit 309.

(Displacement vector of VSP prediction unit 30374 and DBBP prediction unit 3095)
Hereinafter, an image decoding apparatus configured to use a common displacement vector (disparity vector) in the VSP prediction unit 30374 and the DBBP prediction unit 3095 will be described. As illustrated in FIG. 14, the VSP prediction unit 30374 performs coordinates (xTL, yTL) obtained by shifting the target block by the displacement vector MvDisp in the partition division of the division flag derivation unit 353 and the displacement vector array disparitySamples derivation of the depth DV derivation unit 351. And refer to the point on the depth block of coordinates (xTL, yTL). Similarly, the DBBP prediction unit 3095 also derives the coordinates (xTL, yTL) obtained by shifting the target block by the displacement vector MvDisp in the segmentation unit 30952 and the DBBP division mode derivation unit 30954, and points on the depth block of the coordinates (xTL, yTL) Refer to

  The VSP prediction unit 30374 derives the coordinates (xTL, yTL) of the depth block using the displacement vector NBDV that has not been updated (refined) by the depth reference as the displacement vector mvDisp, and the depth DV derivation unit 351 A block displacement vector is derived. In this case, the depth reference is not necessary for the derivation of the displacement vector mvDisp, and the depth reference is performed when the displacement vector of the sub-block is derived. Therefore, the depth reference only needs to be performed once. On the other hand, the DBBP prediction unit 3095 derives the coordinates (xTL, yTL) of the depth block using the displacement vector DoNBDV updated by the depth reference as the displacement vector mvDisp, and a segmentation unit 30952 and a DBBP division mode derivation unit 30954. In reference to depth. In this case, the depth reference by the displacement vector NBDV is performed for the derivation of the displacement vector mvDisp, and the depth reference (depth transfer) by another displacement vector DoNBDV is performed for the segmentation unit 30952 and the DBBP division mode derivation unit 30954. In this case, since depth transfer is required twice, the transfer and processing amount of depth images are large. In addition, if the VSP prediction unit 30374 and the DBBP prediction unit 3095 perform different depth transfers, the process cannot be shared, so that one process is complicated, and the other process is easy, resulting in a design imbalance. Since the overall worst case complexity is determined by the maximum value of the two processes, even if one of the processes is easy, the overall complexity is not reduced. Therefore, in design, it is preferable to allow the same degree of complexity as long as the worst case complexity can be shared. Therefore, it is preferable to use the same displacement vector by standardizing the criteria for how much the updated displacement vector is used in the viewpoint synthesis prediction and DBBP.

  In this embodiment, the processing amount of the worst case of depth reference is reduced while making the processing of the VSP prediction unit 30374 and the DBBP prediction unit 3095 common. The worst case processing amount is to perform depth reference many times in a small block. Therefore, in this embodiment, when the size of the target block is larger than the predetermined size, the displacement vector (DoNBDV) updated with reference to the depth is used, and in other cases, the update is performed with reference to the depth. A displacement vector (NBDV) that is not used is used.

  More specifically, when the size of the target block is larger than a predetermined size, the VSP prediction unit 30374 uses the displacement vector DoNBDV updated by depth reference as the displacement vector mvDisp, and coordinates of the depth block (xTL, yTL) is derived, and the depth DV deriving unit 351 derives the displacement vector of the sub-block. On the other hand, when the size of the target block is equal to or smaller than the predetermined size, the VSP prediction unit 30374 uses the displacement vector NBDV that has not been updated by the depth reference as the displacement vector mvDisp, and calculates the coordinates (xTL, yTL) of the depth block. Then, the depth DV deriving unit 351 derives the displacement vector of the sub-block. Similarly, when the size of the target block is larger than the predetermined size, the DBBP prediction unit 3095 uses the displacement vector DoNBDV updated by the depth reference as the displacement vector mvDisp, and uses the depth block coordinates (xTL, yTL). The segmentation unit 30952 and the DBBP split mode deriving unit 30954 refer to the depth. On the other hand, when the size of the target block is equal to or smaller than the predetermined size, the DBBP prediction unit 3095 uses the displacement vector NBDV that has not been updated by the depth reference as the displacement vector mvDisp and uses the coordinates (xTL, yTL) of the depth block. The segmentation unit 30952 and the DBBP division mode deriving unit 30954 refer to the depth.

  FIG. 29 is a flowchart for explaining operations of the image decoding device 31 and the image encoding device 11 configured to use a common displacement vector as the VSP prediction unit 30374 and the DBBP prediction unit 3095.

  (S3001) It is determined whether or not the prediction unit that is the target block is larger than a predetermined size. If YES, the processing proceeds to S3002. If NO, the processing proceeds to S3003.

  (S3002) When the prediction unit is larger than the predetermined size, the disparity vector DoNBDV obtained by refining the depth image is used as the disparity vector.

  (S3003) When the prediction unit is larger than the predetermined size, a disparity vector NBDV obtained without refining with a depth image is used as a disparity vector.

  More specifically, the image decoding device 31 and the image encoding device 11 perform the following processing. In the partitioning of the partition flag deriving unit 353 and the depth DV deriving of the depth DV deriving unit 351 included in the VSP prediction unit 30374, the sum of the predicted block width nPbW and height nPbH is a predetermined value ( Here, the displacement vector MvRefinedDisp [xPb] [yPb] updated using the depth image if 16) is exceeded, and the displacement vector MvDisp [xPb not updated using the depth image otherwise. ] [yPb], partition by disparity array DisparitySamples for deriving sub-block displacement vector mvLX by depth DV unit 351, partition by partition partition unit by horSplitFlag using partition flag derived by partition flag deriving unit 353 Splitting is performed.

mvLXVSP = nPbW + nPbH> 16? MvRefinedDisp [xPb] [yPb]: MvDisp [xPb] [yPb] Formula A1
Similarly, in the DBBP prediction unit 3095, the sum of the prediction block width (here, nTbS) and the prediction block height (here, nTbS) is a predetermined value (here, 16). The displacement vector MvRefinedDisp [xTb] [yTb] refined using the depth image is exceeded if not, and the displacement vector MvDisp [xTb] [yTb] not refined using the depth image otherwise. , SegMask derivation in the segmentation block 30952 and division mode PartMode derivation in the DBBP division mode derivation unit 30954 are performed.

nTbS + nTbS> 16? MvRefinedDisp [xTb] [yTb]: MvDisp [xTb] [yTb] Formula A2
The predetermined size may be larger than that described above. For example, instead of the expressions A1 and A2, it is preferable to set the predetermined size to 24 as in the following expressions A1 ′ and A2 ′.

mvLXVSP = nPbW + nPbH> 24? MvRefinedDisp [xPb] [yPb]: MvDisp [xPb] [yPb] Formula A1 ′
nTbS + nTbS> 24? MvRefinedDisp [xTb] [yTb]: MvDisp [xTb] [yTb] Formula A2 ′
FIG. 30 is a diagram illustrating a data flow of an example in which a common displacement vector is used in the VSP prediction unit 30374 and the DBBP prediction unit 3095. As described above, the depth DV deriving unit 351 uses the depth image # 1 which is a depth block determined by the displacement vector MvDisp [] [] before being updated by the upper left coordinates of the block and the depth reference from the reference picture memory 306, The displacement vector MvRefinedDisp [] [] updated by the depth reference is derived. The switch 354 selects MvRefinedDisp [] [] when the block size is larger than a predetermined size, and sets MvDisp [] [] to the displacement vector mvDisp in other cases. In view synthesis prediction, the depth DV deriving unit 351 derives disparity arrays DispariytSamples derived in subblock units using the displacement vector mvDisp, and the VSP predicting unit 30374 uses the values indicated by the disparity array DispariytSamples as horizontal vectors. A predicted image is generated by compensation. At this time, the depth DV deriving unit 351 refers to the depth image # 2 that is a depth block determined by the displacement vector mvDisp. In view synthesis prediction, an 8 × 4 or 4 × 8 sub-block size is selected using the depth image # 2 that is referred to by the division flag deriving unit 353 using the displacement vector mvDisp. Here, when the displacement vector mvDisp is equal to MvDisp [] [], the depth block to be referenced for deriving MvRefinedDisp [] is equal to the depth block for deriving the disparity array DispariytSamples and the split flag horSplitFlag. Image # 2 is equal to depth image # 1. Therefore, a series of processes of derivation of the displacement vector MvRefinedDisp, the disparity array DispariytSamples, and the division flag horSplitFlag can be performed by transferring the depth image once, and two depth transfers are not necessary to obtain the depth image. On the other hand, when the displacement vector mvDisp is MvRefinedDisp [] [], the depth image # 2 is a different block from the depth image # 1, and two depth transfers are necessary. Similarly, also in DBBP prediction, the segmentation unit 30952 and the DBBP partition mode deriving unit 30954 derive the segmentation information segMask and the partition mode PartMode using the displacement vector mvDisp. At this time, the segmentation unit 30952 and the DBBP division mode deriving unit 30954 refer to the depth image # 2 that is a depth block determined by the displacement vector mvDisp. Again, if the displacement vector mvDisp is equal to MvDisp [] [], the depth image # 2 is equal to the depth image # 1 and two depth transfers are not required. On the other hand, when the displacement vector mvDisp is MvRefinedDisp [] [], the depth image # 2 is different from the depth image # 1, and two depth transfers are necessary.

  In addition, in FIG. 30, although the example using the division | segmentation flag derivation | leading-out part 353, the segmentation part 30952, DBBP division | segmentation mode derivation | leading-out part 30954 etc. was demonstrated, each means is not limited to this, It is a modification as described in this specification. Also good. For example, an example in which the DBBP split mode deriving unit 30954C is used is preferable because the split processing is shared between the viewpoint synthesis prediction and DBBP. FIG. 31 shows an example in which a DBBP prediction unit 3095C (DBBP division mode deriving unit 30954C) is used instead of the DBBP prediction unit 3095 (DBBP division mode deriving unit 30954). Also in this case, by switching between MvDisp and MvRefinedDisp according to the size of the target block as described above, the disparity array DisparitySamples derivation process of the depth DV derivation unit 351 of the viewpoint synthesis prediction process and the division flag horSplitFlag derivation process of the division flag derivation unit The segMask derivation process of the segmentation unit 30952 is derived using a common displacement vector. The image decoding apparatus 31 and the image encoding apparatus 11 configured as described above are the above-described depth base block prediction image generation means in an image decoding apparatus including a depth base block prediction image generation unit (DBBP prediction unit 3095) and a viewpoint synthesis prediction unit. Includes a segmentation deriving unit 30952 for deriving segmentation information from a depth image, a DBBP image interpolating unit 30951 for generating two motion compensated images, and image synthesis for synthesizing the two interpolated images to generate one motion compensated image. A partition division unit for deriving a partition flag horSplitFlag from the depth image and obtaining a sub-block size; and a depth division unit 30953, and a DBBP partition mode deriving unit 30954 for deriving a partition mode PartMode. Disparity array from image A depth motion DV deriving unit 351 for deriving ySamples and obtaining a motion vector mvLX is provided, and a position of a depth image referred to by the segmentation deriving unit 30952 and the DBBP division mode deriving unit 30954 of the depth base block prediction image generating unit is derived. And the disparity vector used for deriving the position of the depth image by the partition dividing unit and the depth motion DV deriving unit 351 of the viewpoint synthesis prediction unit are used as a common disparity vector. In the modification of the above configuration, the image decoding device 31 and the image encoding device 11 are the above-described depth base block in an image decoding device including a depth base block predicted image generation unit (DBBP prediction unit 3095C) and a viewpoint synthesis prediction unit. The predicted image generation means includes a segmentation deriving unit 30951 for deriving segmentation information from the depth image, a DBBP image interpolating unit 30951 for generating two motion compensated images, and combining the two interpolated images into one motion compensated image. A partitioning unit that generates an image synthesis unit 30953 and a DBBP partitioning mode deriving unit 30954C for deriving a partitioning mode PartMode, wherein the viewpoint synthesis prediction unit performs partitioning by deriving a partitioning flag horSplitFlag from a depth image and obtaining a subblock size Part and depth image A depth motion DV deriving unit 351 for deriving a disparity array DisparitySamples to obtain a motion vector mvLX, and a disparity vector used for deriving a position of a depth image referred to by the segmentation deriving unit 30952 of the depth-based block prediction image generating unit And the disparity vector used for deriving the position of the depth image by the partition dividing unit and the depth motion DV deriving unit 351 of the viewpoint synthesis prediction unit are set as a common disparity vector. Also, the DBBP partition mode deriving unit 30954C and the partition partitioning unit respectively perform partition mode PartMode derivation and subblock size derivation using the output of the common partition flag deriving unit 353. Note that a division flag deriving unit 353A or the like may be used instead of the division flag deriving unit 353.

  Preferably, the common disparity vector is a disparity vector refined by depth when the block size is larger than a predetermined size, and refined by depth when the block size is equal to or smaller than the predetermined size. This is the disparity vector before being processed.

  More preferably, the common disparity vector is a disparity vector refined by depth when the sum of the width and height of the prediction block is larger than 16, and is refined by depth otherwise. This is the disparity vector before being processed.

  The common disparity vector is a disparity vector refined by depth when the sum of the width and height of the prediction block is greater than 24. In other cases, the disparity before being refined by depth It may be a vector.

  According to the image decoding device 31 and the image encoding device 11 configured as described above, since a common displacement vector is used as the VSP prediction unit 30374 and the DBBP prediction unit 3095, an effect of simplifying the processing amount and implementation of depth transfer. Play.

  According to the image decoding device 31 and the image encoding device 11 configured as described above, a small block having a predetermined size or less does not use a disparity vector that needs to be refined by a depth image. The memory bandwidth for transferring the image and the processing amount referring to the depth image are reduced.

(Division flag deriving unit 353)
The division flag deriving unit 353 refers to the depth image corresponding to the target block and derives the division flag horSplitFlag. The following description will be made assuming that the coordinates of the target block set as the input of the division flag deriving unit 353 are (xP, yP), the width and height are nPSW, nPSH, and the displacement vector is mvDisp. The division flag deriving unit 353 refers to the depth image when the width and height of the target block are equal, but when the width and height of the target block are not equal, the division flag deriving unit 353 refers to the division flag without referring to the depth image. Deriving horSplitFlag may be derived. Details of the division flag deriving unit 353 will be described below.

  The division flag deriving unit 353 reads, from the reference picture memory 306, the depth image refDepPels that has the same POC as the decoding target picture and has the same view ID as the view ID (RefViewIdx) of the reference picture indicated by the displacement vector mvDisp.

  Next, the division flag deriving unit 353 derives coordinates (xTL, yTL) obtained by shifting the upper left coordinates (xP, yP) of the target block by the displacement vector MvDisp by the following formula.

xTL = xP + ((mvDisp [0] + 2) >> 2)
yTL = yP + ((mvDisp [1] + 2) >> 2)
Here, mvDisp [0] and mvDisp [1] are the X component and the Y component of the displacement vector MvDisp, respectively. The derived coordinates (xTL, yTL) indicate the coordinates of the block corresponding to the target block on the depth image refDepPels.

  The division flag deriving unit 353 sets the flag minSubBlkSizeFlag to 1 using the following expression when the width nPSW or the height nPSH of the target block is other than a multiple of 8.

minSubBlkSizeFlag = (nPSW% 8! = 0) | | (nPSH% 8! = 0)
When the flag minSubBlkSizeFlag is 1, the split flag deriving unit 353 determines that the horSplitFlag is 1 when the height of the target block is not a multiple of 8 (when nPSH% 8 is true), and In this case, 0 is set.

horSplitFlag = (nPSH% 8! = 0)
In other words, when the target block height is not a multiple of 8 (when nPSH% 8 is true), horSplitFlag is 1, and when the target block width is other than a multiple of 8 (when nPSW% 8 is true) Is set to 0 in horSplitFlag.

The division flag deriving unit 353 derives the sub block size from the depth value. The sub-block size is derived from the comparison of the four points (TL, TR, BL, BR) at the corners of the prediction block. When the flag minSubBlkSizeFlag is 0, the pixel value of the depth image of the upper left (TL) coordinates of the target block is refDepPelsP0, the pixel value of the upper right end (TR) is refDepPelsP1, the pixel value of the lower left end (BL) is refDepPelsP2, and the lower right end When the pixel value of (BR) is refDepPelsP3,
horSplitFlag = (refDepPelsP0> refDepPelsP3) == (refDepPelsP1> refDepPelsP2)
It is determined whether the conditional expression (horSplitFlag) is satisfied.
In order to derive horSplitFlag, the following expression with a changed sign may be used.

horSplitFlag = (refDepPelsP0 <refDepPelsP3) == (refDepPelsP1 <refDepPelsP2)
The split flag derivation unit 353 outputs horSplitFlag to the split mode derivation unit 30954C and the VSP prediction unit 30374.

  The division flag deriving unit 353 may derive as follows. When the width nPSW and the height nPSH of the target block are different, it is derived by the following formula according to the width and height of the target block.

If nPSW> nPSH, horSplitFlag = 1
Otherwise, if nPSH> nPSW, horSplitFlag = 0
Otherwise, when the width and height of the target block are equal, the depth is referred to and derived according to the following formula.

horSplitFlag = (refDepPelsP0> refDepPelsP3) == (refDepPelsP1> refDepPelsP2)
The target block of the division flag deriving unit 353 is a prediction unit in the case of viewpoint synthesis prediction, and a block having the same width and height in the case of DBBP. In the case of DBBP, since the width and the height are equal, in the above derivation method, the division flag horSplitFlag is derived with reference to the four corners of the depth image.

(Division flag deriving unit 353A)
Hereinafter, a split flag deriving unit 353A that is a modification of the split flag deriving unit 353 will be described. The division flag deriving unit 353A refers to the depth image corresponding to the target block and derives the division flag horSplitFlag. In the following description, it is assumed that the coordinates of the target block set as the input of the division flag deriving unit 353A are (xP, yP), the width and height are nPSW, nPSH, and the displacement vector is mvDisp.

  The division flag deriving unit 353A reads out the depth image refDepPels having the same POC as the decoding target picture from the reference picture memory 306 and having the same view ID as the reference ID (RefViewIdx) of the reference picture indicated by the displacement vector mvDisp.

  Next, the division flag deriving unit 353A derives coordinates (xTL, yTL) obtained by shifting the upper left coordinates (xP, yP) of the target block by the displacement vector MvDisp by the following formula.

xTL = xP + ((mvDisp [0] + 2) >> 2)
yTL = yP + ((mvDisp [1] + 2) >> 2)
Here, mvDisp [0] and mvDisp [1] are the X component and the Y component of the displacement vector MvDisp, respectively. The derived coordinates (xTL, yTL) indicate the coordinates of the block corresponding to the target block on the depth image refDepPels.

  The division flag deriving unit 353A sets the flag minSubBlkSizeFlag to 1 using the following expression when the width nPSW or the height nPSH of the target block is other than a multiple of 8.

minSubBlkSizeFlag = (nPSW% 8! = 0) | | (nPSH% 8! = 0)
When the flag minSubBlkSizeFlag is 1, when the flag minSubBlkSizeFlag is 1, the split flag deriving unit 353A sets 1 to horSplitFlag when the height of the target block is not a multiple of 8 (when nPSH% 8 is true), otherwise In this case, 0 is set.

horSplitFlag = (nPSH% 8! = 0)
In other words, when the target block height is not a multiple of 8 (when nPSH% 8 is true), horSplitFlag is 1, and when the target block width is other than a multiple of 8 (when nPSW% 8 is true) Is set to 0 in horSplitFlag.

  The division flag deriving unit 353A derives the sub-block size from the depth value when both the width of the target block and the height of the target block are multiples of 8. Specifically, the sub-block size is derived from a comparison of three points (TL, TR, BL) at the corners of the prediction block. When the flag minSubBlkSizeFlag is 0, the pixel value of the depth image at the upper left (TL) coordinates of the target block is refDepPelsP0, the upper right (TR) pixel value is refDepPelsP1, and the lower left (BL) pixel value is refDepPelsP2. When the horizontal absolute value difference abs (ab) is larger than the vertical absolute value difference abs (ac) (abs (ab)> abs (ac)), the vertical sub-block (4 × 8) is HorSplitFlag = 0 is derived, and in other cases, that is, when the absolute value difference abs (ab) in the horizontal direction is less than or equal to the absolute value difference abs (a-c) in the vertical direction (abs (ab) <= abs (ac)), horSplitFlag = 1 which is a horizontally long sub-block (8 × 4) is derived. Specifically, the split flag deriving unit 353A derives horSplitFlag by the following equation.

a = refDepPelsP0
b = refDepPelsP1
c = refDepPelsP2
horSplitFlag = abs (a-b)> abs (a-c)? 0: 1
In order to derive horSplitFlag, the following expression in which the handling of the equal sign in the determination is changed may be used.

horSplitFlag = abs (a-b)> = abs (a-c)? 0: 1
The split flag derivation unit 353A outputs the horSplitFlag to the split mode derivation unit 30954C and the VSP prediction unit 30374.

(Depth DV deriving unit 351)
The depth DV deriving unit 351 derives disparity arrays disparitySamples (horizontal vectors), which are horizontal components of depth-derived displacement vectors, in designated block units (sub-blocks). The input of the depth DV derivation unit 351 includes the depth DV conversion table DepthToDisparityB, the block width nBlkW and height nBlkH, the split flag splitFlag, the depth image refDepPels, and the coordinates (xTL, yTL) of the corresponding block on the depth image refDepPels. , View IDrefViewIdx, output is disparity array disparitySamples (horizontal vector).

  Note that the coordinates (xTL, yTL) of the corresponding block on the depth image refDepPels are derived by the following equations when the displacement vector is mvDisp.

xTL = xP + ((mvDisp [0] + 2) >> 2)
yTL = yP + ((mvDisp [1] + 2) >> 2)
The depth DV deriving unit 351 sets a pixel used for deriving the depth representative value maxDep for each target block. Specifically, as shown in FIG. 13, when the relative coordinates from the upper left prediction block (xTL, yTL) of the target block are (xSubB, ySubB), the X coordinate xP0 of the left end of the sub-block and the right end The X coordinate xP1, the upper end Y coordinate yP0, and the lower end Y coordinate yP1 are obtained from the following equations.

xP0 = Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB)
yP0 = Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB)
xP1 = Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB + nBlkW-1)
yP1 = Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB + nBlkH-1)
Here, pic_width_in_luma_samples and pic_height_in_luma_samples represent the width and height of the image, respectively.

  Next, the depth DV deriving unit 351 derives the depth representative value maxDep of the target block. Specifically, the pixel values refDepPels [xP0] [yP0], refDepPels [xP0], refDepPels [xP1] [yP0], refDepPels [xP1] [yP1] of the depth image at the corner of the sub-block and in the vicinity thereof ], The representative depth value maxDep is derived from the following equation.

maxDep = 0
maxDep = Max (maxDep, refDepPels [xP0] [yP0])
maxDep = Max (maxDep, refDepPels [xP0] [yP1])
maxDep = Max (maxDep, refDepPels [xP1] [yP0])
maxDep = Max (maxDep, refDepPels [xP1] [yP1])
Here, the function Max (x, y) is a function that returns x if the first argument x is greater than or equal to the second argument y, and returns y otherwise.

  The depth DV deriving unit 351 uses the representative depth value maxDep, the depth DV conversion table DepthToDisparityB, and the view ID refViewIdx of the layer indicated by the displacement vector (NBDV) to target the disparity array disparitySamples that is the horizontal component of the displacement vector derived from the depth. For each pixel (x, y) in the block (where x is a value from 0 to nBlkW-1, and y is a value from 0 to nBlkH-1), it is derived by the following equation.

disparitySamples [x] [y] = DepthToDisparityB [refViewIdx] [maxDep] ... (Formula A)
The depth DV deriving unit 351 outputs the derived parallax array disparitySamples [] to the displacement vector deriving unit 352 as the displacement vector DoNBDV (a horizontal component thereof). The depth DV deriving unit 351 also outputs the displacement vector (the horizontal component thereof) to the VSP prediction unit 30374.

(Inter prediction image generation unit 309)
FIG. 16 is a schematic diagram illustrating a configuration of the inter predicted image generation unit 309 according to the present embodiment. The inter prediction image generation unit 309 includes a motion displacement compensation unit 3091, a residual prediction unit 3092, an illuminance compensation unit 3093, a DBBP prediction unit 3095 (depth base block prediction image generation device 3095), and a weighted prediction unit 3096. .

  The inter prediction image generation unit 309 performs processing in units of subblocks when the subblock motion compensation flag subPbMotionFlag input from the inter prediction parameter decoding unit 303 is 1, and performs prediction when the subblock motion compensation flag subPbMotionFlag is 0. The following processing is performed for each unit. The sub-block motion compensation flag subPbMotionFlag is set to 1 when the inter-view merge candidate is selected as the merge mode or when the VSP merge candidate is selected. The inter prediction image generation unit 309 derives prediction images predSamples using the motion displacement compensation unit 3091 based on the prediction parameters. In addition, when the residual prediction index iv_res_pred_weight_idx is not 0, the inter predicted image generation unit 309 sets the residual prediction execution flag resPredFlag to 1 indicating that residual prediction is to be performed, and the motion displacement compensation unit 3091 The result is output to the difference prediction unit 3092. On the other hand, when the residual prediction index iv_res_pred_weight_idx is 0, the residual prediction execution flag resPredFlag is set to 0 and output to the motion displacement compensation unit 3091 and the residual prediction unit 3092.

  The motion displacement compensation unit 3091, the residual prediction unit 3092, the illuminance prediction unit 3093, and the DBBP prediction unit 3095 each perform motion compensation of the motion compensation image predSamplesL0 or L1 of L0 in the case of simple prediction (predFlagL0 = 1 or predFlagL1 = 1). The image predSamplesL1 is derived, and in the case of bi-prediction (predFlagL0 = 1 and predFlagL1 = 1), the L0 motion compensated image predSamplesL0 and the L1 motion compensated image predSamplesL1 are derived and output to the weighted prediction unit 3096. The weighted prediction unit 3096 derives a predicted image predSamples from one motion compensated image predSamplesL0 or predSamplesL1 in the case of uni-prediction, and calculates a predicted image predSamples from two two motion compensated images predSamplesL0 and predSamplesL1 in the case of bi-prediction. To derive.

(Motion displacement compensation)
The motion displacement compensation unit 3091 generates a motion prediction image predSampleLX based on the prediction use flag predFlagLX, the reference picture index refIdxLX, and the vector mvLX (motion vector or displacement vector). The motion displacement compensation unit 3091 reads out a block at a position shifted by the vector mvLX from the reference picture memory 306, starting from the position of the prediction unit of the reference picture specified by the reference picture index refIdxLX, and interpolates the predicted image. Generate. Here, when the vector mvLX is not an integer vector, a prediction image is generated by applying a filter called a motion compensation filter (or displacement compensation filter) for generating a pixel at a decimal position. In general, when the vector mvLX is a motion vector, the above processing is called motion compensation, and when the vector mvLX is a displacement vector, it is called displacement compensation. Here, it is collectively referred to as motion displacement compensation. Hereinafter, the prediction image of L0 prediction is referred to as predSamplesL0, and the prediction image of L1 prediction is referred to as predSamplesL1. If the two are not distinguished, they are called predSamplesLX. Hereinafter, an example in which residual prediction and illuminance compensation are further performed on the prediction image predSamplesLX obtained by the motion displacement compensation unit 3091 will be described. These output images are also referred to as prediction images predSamplesLX. In the following residual prediction and illuminance compensation, when an input image and an output image are distinguished, the input image is expressed as predSamplesLX and the output image is expressed as predSamplesLX ′.

  When the residual prediction execution flag resPredFlag is 0, the motion displacement compensation unit 3091 generates a motion compensated image predSamplesLX using a motion compensation filter having 8 taps for the luminance component and 4 taps for the chrominance component. When the residual prediction execution flag resPredFlag is 1, a motion compensation image predSamplesLX is generated by a 2-tap motion compensation filter for both the luminance component and the chrominance component.

  When the sub block motion compensation flag subPbMotionFlag is 1, the motion displacement compensation unit 3091 performs motion compensation in units of sub blocks. Specifically, a sub-block vector of coordinates (xCb, yCb), a reference picture index, and a reference list use flag are derived from the following equations.

MvL0 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbMvL0 [xCb + x] [yCb + y]: mvL0
MvL1 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbMvL1 [xCb + x] [yCb + y]: mvL1
RefIdxL0 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbRefIdxL0 [xCb + x] [yCb + y]: refIdxL0
RefIdxL1 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbRefIdxL1 [xCb + x] [yCb + y]: refIdxL1
PredFlagL0 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbPredFlagL0 [xCb + x] [yCb + y]: predFlagL0
PredFlagL1 [xCb + x] [yCb + y] = subPbMotionFlag? SubPbPredFlagL1 [xCb + x] [yCb + y]: predFlagL1
Here, SubPbMvLX, SubPbRefIdxLX, and SubPbPredFlagLX (X is 0, 1) correspond to subPbMvLX, subPbRefIdxLX, and subPbPredFlagLX described in the inter-layer merge candidate derivation unit 30371.

(Residual prediction)
The residual prediction unit 3092 performs residual prediction when the residual prediction execution flag resPredFlag is 1. When the residual prediction execution flag resPredFlag is 0, the residual prediction unit 3092 outputs the input predicted image predSamplesLX as it is. The refResSamples residual prediction is performed by estimating the residual of the motion compensated image predSamplesLX generated by motion prediction or displacement prediction and adding it to the predicted image predSamplesLX of the target layer. Specifically, when the prediction unit is motion prediction, it is assumed that a residual similar to the reference layer also occurs in the target layer, and the residual of the reference layer already derived is estimated as the residual of the target layer. Use as a value. When the prediction unit is displacement prediction, a residual between a reference layer picture and a target layer picture at a time (POC) different from that of the target picture is used as an estimated value of the residual.

  Similar to the motion displacement compensation unit 3091, the residual prediction unit 3092 also performs residual prediction in units of subblocks when the subblock motion compensation flag subPbMotionFlag is 1.

  FIG. 17 is a block diagram showing a configuration of the residual prediction unit 3092. The residual prediction unit 3092 includes a reference image interpolation unit 30922 and a residual synthesis unit 30923.

  When the residual prediction execution flag resPredFlag is 1, the reference image interpolation unit 30922 receives the vector mvLX and the residual prediction displacement vector mvDisp input from the inter prediction parameter decoding unit 303, and the reference picture stored in the reference picture memory 306. Are used to generate two residual prediction motion compensated images (corresponding block rpSamplesLX, reference block rpRefSamplesLX).

  The residual prediction unit 3092 derives an inter-view prediction flag ivRefFlag, which is a flag indicating whether the target block is motion prediction or displacement prediction, by (DiffPicOrderCnt (currPic, RefPicListX [refIdxLX]) == 0). Here, DiffPicOrderCnt (X, Y) indicates the difference between the POC of picture X and picture Y (the same applies hereinafter). Therefore, when the POC of the target picture currPic and the POC of the reference picture RefPicListX [refIdxLX] indicated by the reference picture index refIdxLX and the reference picture list RefPicListX are 0, the target block is assumed to be subject to displacement prediction and ivRefFlag is set to 1. Otherwise, ivRefFlag is set to 0, assuming that motion prediction is applied to the target block.

  FIG. 18 is a diagram for describing the corresponding block rpSamplesLX and the reference block rpRefSamplesLX when the vector mvLX is a motion vector (when the interview prediction flag ivRefFlag is 0). As shown in FIG. 18, the corresponding block corresponding to the prediction unit on the target layer is a displacement vector mvDisp that is a vector indicating the positional relationship between the reference layer and the target layer, starting from the position of the prediction unit of the image on the reference layer. It is located in a block that is displaced by a certain amount.

  FIG. 19 is a diagram for explaining the corresponding block rpSamplesLX and the reference block rpRefSamplesLX when the vector mvLX is a displacement vector (when the interview prediction flag ivRefFlag is 1). As shown in FIG. 19, the corresponding block rpSamplesLX is a block on the reference picture rpPic that has a different time from the target picture and the same view ID as the target picture. The residual prediction unit 3092 derives mvT that is a vector of a prediction unit on the previous picture mvPicT indicated by the target block vector mvLX (= displacement vector mvDisp). The corresponding block rpSamplesLX is located in the block that is shifted by the vector mvT starting from the position of the prediction unit (target block).

(Derivation of reference picture for residual prediction)
The residual prediction unit 3092 includes reference pictures rpPic and rpPicRef, which are reference pictures to be referred to in derivation of residual prediction motion compensated images (rpSamplesLX and rpRefSamplesLX), and the position of the reference block (the reference block based on the coordinates of the target block). Relative coordinates) vectors mvRp and mvRpRef are derived.

  The residual prediction unit 3092 sets a picture having the same display time (POC) or the same view ID as the target picture to which the target block belongs as rpPic.

  Specifically, when the target block is motion prediction (when the interview prediction flag ivRefFlag is 0), the residual prediction unit 3092 has the same POC of the reference picture rpPic and PicOrderCntVal that is the POC of the target picture, and the reference The view ID of the picture rpPic and the reference view ID RefViewIdx [xP] [yP] of the prediction unit (this is different from the view ID of the target picture). The reference picture rpPic is derived from the above condition. Further, the residual prediction unit 3092 sets the displacement vector MvDisp to the rpPic vector mvRp.

  The residual prediction unit 3092 sets the reference picture used for generating the predicted image of the target block to rpPic when the target block is displacement prediction (when the interview prediction flag ivRefFlag is 1). That is, when the reference index of the target block is RpRefIdxLY and the reference picture list is RefPicListY, the reference picture rpPic is derived from RefPicListY [RpRefIdxLY]. Furthermore, a residual prediction vector deriving unit 30924 (not shown) included in the residual prediction unit 3092 is included. The residual prediction vector deriving unit 30924 is the same POC as the target picture to which the target block vector mvLX (which is equal to the displacement vector MvDisp) points to the rpPic vector mvRp. MvT which is a vector of the prediction unit is derived, and the motion vector mvT is set to mvRp.

  Next, the residual prediction unit 3092 sets, as rpPicRef, a reference picture having a different display time (POC) and a different view ID from the current picture.

  Specifically, when the target block is motion prediction (when the interview prediction flag ivRefFlag is 0), the residual prediction unit 3092 has the same POC of the reference picture rpPicRef and the POC of the reference picture RefPicListY [RpRefIdxLY] of the target block. The reference picture rpPicRef is derived from the condition that the view ID of the reference picture rpPicRef is equal to the view ID RefViewIdx [xP] [yP] of the reference picture of the displacement vector MvDisp. Further, the residual prediction unit 3092 sets the vector mvLX obtained by scaling the motion vector of the prediction block to the vector mvRpRef of rpPicRef and the sum (mvRp + mvLX) of the vector mvRp.

  When the target prediction unit is displacement prediction (when the inter prediction prediction flag ivRefFlag is 1), the residual prediction unit 3092 has the POC of the reference picture rpPicRef equal to the POC of the reference picture rpPic and the view ID of the reference picture rpPicRef. A reference picture rpPicRef is derived from the condition that the view IDs RefViewIdx [xP] [yP] of the prediction units are equal. Further, the residual prediction unit 3092 sets the motion vector mvLX of the prediction block and the sum (mvRp + mvLX) of the prediction block to the vector mvRpRef of the rpPicRef.

  That is, in the residual prediction unit 3092, mvRp and mvRpRef are derived as follows.

When the interview prediction flag ivRefFlag is 0
mvRp = MvDisp formula (B-1)
mvRpRef = mvRp + mvLX (= mvLX + MvDisp) Formula (B-2)
When interview prediction flag ivRefFlag is 1
mvRp = mvT formula (B-3)
mvRpRef = mvRp + mvLX (= mvLX + mvT) Formula (B-4)
(Residue prediction vector deriving unit 30924)
The residual prediction vector deriving unit 30924 derives a vector mvT of a prediction unit on a picture different from the current picture. The residual prediction vector deriving unit 30924 receives the reference picture, the target block coordinates (xP, yP), the target block size nPSW, nPSH, and the vector mvLX, and receives motion compensation parameters (vector, reference picture) of the prediction unit on the reference picture. The vector mvT and view ID are derived from the index and view ID). The residual prediction vector deriving unit 30924 derives the reference coordinates (xRef, yRef) as the center coordinates of the block at the position shifted by the vector mvLX from the target block on the reference picture instructed as an input by the following expression: To do.

xRef = Clip3 (0, PicWidthInSamplesL-1, xP + (nPSW >> 1) + ((mvDisp [0] + 2) >> 2))
yRef = Clip3 (0, PicHeightInSamplesL-1, yP + (nPSH >> 1) + ((mvDisp [1] + 2) >> 2))
The residual prediction vector deriving unit 30924 derives a refPU vector mvLX and a reference picture index refPicLX that are prediction units including reference block coordinates (xRef, yRef). If the target prediction unit is displacement prediction (DiffPicOrderCnt (currPic, refPic) is 0) and the reference prediction unit refPU is motion prediction (DiffPicOrderCnt (refPic, refPicListRefX [refIdxLX]) is non-zero), the refPU vector is mvT And the referable flag availFlagT is set to 1. With the above processing, a block vector having the same POC as the target picture and a picture with a different view ID as a reference picture can be derived as mvT.

  The residual prediction vector deriving unit 30924 derives a vector of prediction units on a picture different from the current picture. The residual prediction vector deriving unit 30924 derives the following reference block coordinates (xRef, yRef) using the target block coordinates (xP, yP), the target block sizes nPbW, nPbH, and the displacement vector mvDisp as inputs.

xRef = Clip3 (0, PicWidthInSamplesL-1, xP + (nPSW >> 1) + ((mvDisp [0] + 2) >> 2))
yRef = Clip3 (0, PicHeightInSamplesL-1, yP + (nPSH >> 1) + ((mvDisp [1] + 2) >> 2))
The residual prediction vector deriving unit 30924 derives a refPU vector mvLX and a reference picture index refPicLX that are prediction units including reference block coordinates (xRef, yRef). When the target prediction unit is motion prediction (DiffPicOrderCnt (currPic, refPic) is other than 0) and the reference prediction unit refPU is displacement prediction (DiffPicOrderCnt (refPic, refPicListRefX [refIdxLX]) is 0), the reference flag availFlagT is set to 1. To do. As a result, a block vector having the same POC as the target picture and a picture with a different view ID as a reference picture can be derived as mvT.

(Reference image interpolation unit 30922)
The reference image interpolation unit 30922 generates an interpolation image of the reference block rpSamplesLX by setting the vector mvC to the vector mvLX. A pixel at a position where the coordinates (x, y) of the pixel of the interpolation image is shifted by the vector mvLX of the prediction unit is derived by linear interpolation (bilinear interpolation). Considering that the displacement vector LX has a 1/4 pel decimal precision, the reference image interpolating unit 30922 uses the X of the pixel R0 with integer precision corresponding to the case where the pixel coordinates of the prediction unit are (xP, yP). The coordinate xInt, the Y coordinate yInt, and the fractional part xFrac of the X component of the displacement vector mvDisp and the fractional part yFrac of the Y component are expressed as follows (formula C-1)
xInt = xPb + (mvLX [0] >> 2)
yInt = yPb + (mvLX [1] >> 2)
xFrac = mvLX [0] & 3
yFrac = mvLX [1] & 3
It is derived by the following formula. Here, X & 3 is a mathematical expression for extracting only the lower 2 bits of X.

Next, the reference image interpolation unit 30922 generates an interpolation pixel predPartLX [x] [y] in consideration of the fact that the vector mvLX has a 1/4 pel decimal precision. First, the coordinates of integer pixels A (xA, yB), B (xB, yB), C (xC, yC), and D (xD, yD) are expressed by the following (formula C-2)
xA = Clip3 (0, picWidthInSamples-1, xInt)
xB = Clip3 (0, picWidthInSamples-1, xInt + 1)
xC = Clip3 (0, picWidthInSamples-1, xInt)
xD = Clip3 (0, picWidthInSamples-1, xInt + 1)
yA = Clip3 (0, picHeightInSamples-1, yInt)
yB = Clip3 (0, picHeightInSamples-1, yInt)
yC = Clip3 (0, picHeightInSamples-1, yInt + 1)
yD = Clip3 (0, picHeightInSamples-1, yInt + 1)
It is derived by the following formula. Here, the integer pixel A is a pixel corresponding to the pixel R0, and the integer pixels B, C, and D are integer precision pixels adjacent to the right, bottom, and bottom right of the integer pixel A, respectively. The reference image interpolation unit 30922 includes reference pixels refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], and refPicLX [corresponding to the integer pixels A, B, C, and D, respectively. xD] [yD] is read from the reference picture memory 306.

The reference image interpolation unit 30922 then subtracts the X component of the reference pixel refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], refPicLX [xD] [yD] and the vector mvLX. An interpolated pixel predPartLX [x] [y], which is a pixel shifted by a decimal part of the vector mvLX from the pixel R0, is derived by linear interpolation (bilinear interpolation) using the part xFrac and the fractional part yFrac of the Y component. Specifically, the following formula (C-3)
predPartLX [x] [y] = (refPicLX [xA] [yA] * (8-xFrac) * (8-yFrac)
+ refPicLX [xB] [yB] * (8-yFrac) * xFrac
+ refPicLX [xC] [yC] * (8-xFrac) * yFrac
+ refPicLX [xD] [yD] * xFrac * yFrac) >> 6
It is derived by the following formula.

  In the above description, the four-point pixels around the target pixel are used for deriving by one-step bilinear interpolation. However, horizontal linear interpolation and vertical linear interpolation are separated, and the remaining pixels are separated by two-step linear interpolation. A difference prediction interpolation image may be generated.

  The reference image interpolation unit 30922 performs the above interpolation pixel derivation process on each pixel in the prediction unit, and sets a set of interpolation pixels as an interpolation block predPartLX. The reference image interpolation unit 30922 outputs the derived interpolation block predPartLX to the residual synthesis unit 30923 as the corresponding block rpSamplesLX.

  The reference image interpolation unit 30922 derives the reference block rpRefSamplesLX by performing the same processing except that the corresponding block rpSamplesLX is derived and the displacement vector mvLX is replaced with the vector mvR. The reference image interpolation unit 30922 outputs the reference block rpRefSamplesLX to the residual synthesis unit 30923.

(Residual synthesis unit 30923)
When the residual prediction execution flag resPredFlag is 1, the residual synthesis unit 30923 derives a residual from the difference between the two residual prediction motion compensated images (rpSamplesLX, rpRefSamplesLX), and uses this residual in the motion compensated image. A predicted image is derived by adding the predicted images. Specifically, the residual synthesis unit 30923 derives a corrected predicted image predSamplesLX ′ from the predicted image predSamplesLX, the corresponding block rpSamplesLX, the reference block rpRefSamplesLX, and the residual prediction index iv_res_pred_weight_idx. The corrected predicted image predSamplesLX´
predSamplesLX´ [x] [y] = predSamplesLX [x] [y] + ((rpSamplesLX [x] [y]-rpRefSamplesLX [x] [y]) >> (iv_res_pred_weight_idx-1))
It is calculated using the following formula. x is 0 to the width -1 of the prediction block, and y is 0 to the height of the prediction block -1. When the residual prediction execution flag resPredFlag is 0, the residual synthesis unit 30923 outputs the predicted image predSamplesLX as it is as in the following equation.

predSamplesLX´ [x] [y] = predSamplesLX [x] [y]
(Illuminance compensation)
When the illumination compensation flag ic_flag is 1, the illumination compensation unit 3093 performs illumination compensation on the input predicted image predSamplesLX. When the illumination compensation flag ic_flag is 0, the input predicted image predSamplesLX is output as it is.

(Weighted prediction)
In the case of single prediction (predFlagL0 = 1 / predFlagL1 = 0 or predFlagL0 = 0 / predFlagL1 = 1), the weighted prediction unit 3096 derives the predicted image predSamples from the L0 motion compensated image predSampleL0 or the L1 motion compensated image predSampleL1. . Specifically, the prediction from L0 and the prediction from L1 are respectively derived using the following equations.

predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, predSamplesL0 [x] [y] * w0 + o0)
predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, predSamplesL1 [x] [y] * w1 + o1)
)
Here, w0, w1, o0, and o1 are weights and offsets encoded in the parameter set, respectively. bitDepth is a value indicating the bit depth. In the case of bi-prediction (predFlagL0 = 1 / predFlagL1 = 1), the weighted prediction unit 3096 generates a prediction image from the weighted prediction of the L0 motion compensated image predSampleL0 and the L1 motion compensated image predSampleL1.

(predSamplesL0 [x] [y] * w0 + predSamplesL1 [x] [y] * w1 +
((o0 + o1 + 1) << log2Wd)) >> (log2Wd + 1))
Here, w0, w1, o0, o1, and log2Wd are a weight, an offset, and a shift value encoded by the parameter set, respectively. bitDepth is a value indicating the bit depth.

(DBBP prediction)
When the DBBP mode flag dbbp_flag is 1, the DBBP prediction unit 3095 generates predicted images predSamples by depth-based block partitioning (DBBP). The depth-based block division divides the target block into two regions (region 1 and region 2) based on the segmentation of the depth image corresponding to the target block. An interpolation image of region 1 (hereinafter referred to as predSamplesA), an interpolation image of region 2 (hereinafter referred to as predSamplesB) and a segmentation indicating region division are derived, and further, two interpolation images are synthesized according to the segmentation, thereby obtaining one interpolation image (Predicted image) is generated.

  In DBBP, in order to perform suitable division based on the depth image, the division mode (PartMode) decoded from the encoded data is different from the division (segmentation) actually applied. Prediction parameters such as motion vectors decoded from the encoded data are used to derive prediction parameters of subsequent prediction units and prediction units of different pictures, so use a division mode that is as close to actual division (segmentation) as possible. It is appropriate to store the prediction parameters applied to the DBBP. Therefore, the DBBP prediction unit 3095 derives a division mode PartMode from the depth image in a DBBP division mode derivation unit 30954 described later. The DBBP prediction unit 3095 replaces the partition mode obtained by decoding with the derived partition mode PartMode. Further, the prediction parameter decoded by the inter prediction parameter decoding control unit 3031 is stored in the prediction parameter memory 307 according to the replaced partition mode PartMode. Note that segmentation indicates division in units of pixels, and the division mode indicates division in units of rectangles that are preliminarily dished.

  FIG. 1 is a block diagram illustrating a configuration of the DBBP prediction unit 3095 according to the embodiment of this invention. The DBBP prediction unit 3095 includes a DBBP image interpolation unit 30951, a segmentation unit 30952, an image synthesis unit 30953, and a DBBP split mode derivation unit 30954. Note that the segmentation unit 30952 and the DBBP division mode derivation unit 30954 may be performed by the prediction parameter decoding unit 302 instead of the prediction image generation unit 101.

(DBBP image interpolation unit 30951)
The DBBP image interpolation unit 30951 generates two interpolated images (predSamplesA, predSampleB) for each reference picture list L0 or L1 by bilinear interpolation based on the two vectors input to the DBBP prediction unit 3095. Note that the operation of the DBBP image interpolation unit 30951 related to bilinear interpolation is the same as the operation of the ARP reference image interpolation unit 30922. First, the integer position xInt, yInt and the phases xFrac, yFrac are calculated from the motion vector mvLX by (Equation C-1). Next, the positions of the four pixels refPicLx [xA] [yA], refPicLx [xB] [yB], refPicLx [xC] [yC], refPicLx [xD] [yD] are obtained by (Expression C-2). Is derived. Finally, the pixels are interpolated from four points according to (Formula C-3) according to the phases xFrac and yFrac. Note that the ARP reference image interpolation unit 30922 and the DBBP image interpolation unit 30951 perform common bilinear interpolation. May be used.

  Note that the two interpolated images are distinguished by partIdx taking 0 or 1. That is, the DBBP image interpolation unit 30951 derives an interpolated image predSample with partIdx = 0 or an interpolated image predSample with partIdx = 1.

  The segmentation unit 30952 derives the segmentation information segMask from the depth block corresponding to the target block input from the reference picture memory 306. Note that the depth block corresponding to the target block has the same POC as the decoding target picture, and is on the image above the depth picture having the same view ID as the view ID (RefViewIdx) of the reference picture indicated by the displacement vector MvDisp. This is a block whose coordinates are (xP + mvDisp [0], xP + mvDisp [1]). Here, (xP, yP) represents the coordinates of the target block, and MvDisp represents the displacement vector of the target block as MvDisp.

  Specifically, the segmentation unit 30952 derives the representative value thresVal of the pixel value of the depth block, and segMask is set to 1 when each pixel value of the depth block is larger than the representative value thresVal, and 0 when it is less than or equal to the representative value thresVal. [] [] Is derived.

  Specifically, the representative value thresVal is the depth pixel refSamples [x] [for each pixel (x = 0 .. nCbSL-1, y = 0 .. nCbSL-1 x, y) constituting the depth block. The sum SumVals of y] is derived as shown in the following equation, and is further derived by the segmentation unit 30952 by right shifting by a value corresponding to the logarithm of 2 of the depth block size.

sumRefVals + = refSamples [x] [y]
threshVal = (sumRefVals >> (2 * log2 (nTbS)))
segMask [] [] is the depth pixel refSamples [x] [y] for each pixel (x = 0 .. nCbSL-1, y = 0 .. nCbSL-1 x, y) making up the depth block. Depending on whether or not the representative value thresVal is exceeded, it is derived by the segmentation unit 30952 according to the following equation.

segMask [x] [y] = (refSamples [x] [y]> threshVal)
segMask [x] [y] is a block having the same size as the target block and each pixel value having 0 or 1.

  For each reference picture list L0 or L1, the image composition unit 30953 uses the segMask derived by the segmentation unit 30952 and the two interpolated images (predSamplesA and predSampleB) derived by the DBBP image interpolation unit 30951 to interpolate the image predSamplesLX. Is derived. The derived interpolation image predSamplesL0 and interpolation image predSamplesL1 are output to the weighted prediction unit 3096.

  FIG. 22 is a diagram illustrating the image composition unit 30953. Based on the segmentation information segMask, the image composition unit 30953 selects one of the two interpolated images for each pixel, and further performs a filtering process to derive a predicted image predSamplesLX (here, PredSamplesDbbp).

  Specifically, the interpolated image predSamples is set in predSamplesDbbp according to the values of partIdx and segmentation information segMask [x] [y] that distinguish the two interpolated images.

  The image composition unit 30953 may select a pixel from two interpolation images based on the upper left pixel segMask [0] [0] of segMask. In this case, the image composition unit 30953 selects pixels from the interpolated image corresponding to partIdx different from segMask [0] [0] as follows. The image composition unit 30953 derives a flag curSegmentFlag indicating whether or not the values of partIdx and segMask [0] [0] are equal to each other by the following expression.

curSegmentFlag = (partIdx! = segMask [0] [0]).
When curSegmentFlag is 1, an interpolated image predSamples corresponding to predIdx = curSegmentFlag is assigned to the interpolated image predSamplesLX for pixels whose segMask [x] [y] is equal to curSegmentFlag. For example, if partIdx is 0 and the segmentation information segMask [0] [0] of the upper left image of the block is 0, the segmentation information segMask [x] [y] of each pixel is 1 ((partIdx! = SegMask [0] [0 ]) = (0! = 0) = 1), the interpolated image predSamples of partIdx = 0 is set to PredSamplesDbbp. When partIdx is 0 and the segmentation information segMask [0] [0] of the upper left image of the block is 1, the segmentation information segMask [x] [y] of each pixel is 0 ((partIdx! = segMask [0] [0]) If (0! = 1) = 0), the interpolated image predSamples of partIdx = 0 is set to PredSamplesDbbp.

  In summary, the image synthesis unit 30953 assigns the interpolated image predSamples to the predicted image PredSamplesDbbp according to the segmentation information segMask [x] [y] of each pixel according to the following equation.

for (y = 0; y <nCbS _L ; y ++) {
for (x = 0; x <nCbS _L ; x ++) {
if (segMask [x] [y] = = (partIdx! = segMask [0] [0]))
PredSamplesDbbp _L [x] [y] = predSamples _L [x] [y]
if ((x% 2 = = 0) && (y% 2 = = 0)) {
PredSamplesDbbp _Cb [x / 2] [y / 2] = predSamples _Cb [x / 2] [y / 2]
PredSamplesDbbp _Cr [x / 2] [y / 2] = predSamples _Cr [x / 2] [y / 2]
}
}
}
The image composition unit 30953 may further filter each pixel according to the segmentation information segMask [x] [y]. Specifically, the image composition unit 30953 includes segmentation information cFlag (= segMask [x] [y]) of the target pixel and segmentation information lFlag (= segMask [x-1] [y]) of the pixel adjacent to the left. , Refer to the segmentation information rFlag (segMask [x + 1] [y]) of the pixel adjacent to the right, and if they are different (cFlag! = LFlag || cFlag! = RFlag), the left pixel p [x− 1] [y], the target pixel p [x] [y], and the right pixel p [x + 1] [y] are filtered using a weight of 1: 2: 1. Subsequently, the image composition unit 30953 includes the segmentation information cFlag (= segMask [x] [y]) of the target pixel and the segmentation information tFlag (= segMask [x] [y-1]) of the adjacent pixel on the upper side. Refer to the segmentation information bFlag (segMask [x] [y + 1]) of the adjacent pixel below, and if they are different (cFlag! = TFlag || cFlag! = BFlag), the upper pixel p [x] [ A filter using a weight of 1: 2: 1 is applied to y−1], target pixel p [x] [y], and lower pixel p [x] [y + 1]. The pseudo code for this process is as follows.

for (y = 0; y <nCbS _X ; y ++)
for (x = 0; x <nCbS _X ; x ++) {
tFlag = segMask [n * x] [Max (0, n * (y-1))]
lFlag = segMask [Max (0, (n * (x-1))] [n * y]
bFlag = segMask [n * x] [Min (n * (y + 1), nCbS _L -1)]
rFlag = segMask [Min (n * (x + 1), nCbS _L -1)] [n * y]
cFlag = segMask [n * x] [n * y]
filt = p [x] [y]
if ((lFlag | | cFlag | | rFlag) && (! lFlag | |! cFlag | |! rFlag))
filt = (p [Max (0, x-1)] [y] + (filt << 1) + p [Min (x + 1, nCbS _X -1)] [y]) >> 2
if ((tFlag | | cFlag | | bFlag) && (! tFlag | |! cFlag | |! bFlag))
filt = (p [x] [Max (0, y-1)] + (filt << 1) + p [x] [Min (y + 1, nCbS _X -1)]) >> 2
predSamples [x] [y] = filt
}
The DBBP division mode deriving unit 30954 derives the division mode partMode from the depth block refSamples corresponding to the target block. Specifically, based on the segmentation information segMask derived from the depth block refSamples, for each x and y (x = 0..nCbS-1, y = 0..nCbS-1) of the target block, 2N × 2N (SIZE_Nx2N), 2N × N (SIZE_2NxN), 2N × nU (SIZE_2NxnU), 2N × nD (SIZE_2NxnD), nL × The following part of the pseudo code is the sum partSum [0], partSum [1], partSum [2], partSum [3], partSum [4], partSum [5] corresponding to 2N (SIZE_nLx2N) and nR × 2N (SIZE_nRx2N) Derived based on.

for (y = 0; y <nCbS; y ++)
for (x = 0; x <nCbS; x ++) {
segFlag = segMask [x] [y]
partSum [0] [(x <(nCbS >> 1))? segFlag:! segFlag] ++
partSum [1] [(y <(nCbS >> 1))? segFlag:! segFlag] ++
if (nCbS> 8) {
partSum [2] [(y <(nCbS >> 2))? segFlag:! segFlag] ++
partSum [3] [(y <(nCbS >> 2 + nCbS >> 1))? segFlag:! segFlag] ++
partSum [4] [(x <(nCbS >> 2))? segFlag:! segFlag] ++
partSum [5] [(x <(nCbS >> 2 + nCbS >> 1))? segFlag:! segFlag] ++
}
}
Further, the DBBP partitioning mode deriving unit 30954 derives p that maximizes partSum [p] for p = 0..5, and the partitioning mode corresponding to p is SIZE_Nx2N, SIZE_2NxN, Derived as corresponding to SIZE_2NxnU, SIZE_2NxnD, SIZE_nLx2N, SIZE_nRx2N. For example, when p = 0 and partSum [p] is maximum (partIdc = 0), SIZE_Nx2N is derived as the division mode.

partIdc = 0
maxPartSum = 0
for (p = 0; p <6; p ++)
for (i = 0; i <2; i ++) {
if (partSum [p] [i]> maxPartSum) {
maxPartSum = partSum [p] [i]
partIdc = p
}
}
According to the DBBP prediction unit 3095 having the above configuration, the DBBP image interpolation unit 30951 generates an interpolation image by bilinear prediction from two interpolation images used for DBBP prediction synthesis. Compared with the case where the interpolated image of the DBBP prediction unit is generated with 8 taps or 4 taps used, the processing amount and the transfer amount are greatly reduced. According to the inventor's calculation, when motion compensation is performed by the motion displacement compensation unit 3091 that uses 8 taps for luminance and 4 taps for color difference, compared to motion compensation in bi-prediction of 8 × 8 blocks, DBBP motion compensation has a very high complexity, with a maximum of 170% for multiplication, a maximum of 170% for addition, and a maximum of 119% for memory bandwidth. In contrast, by using bilinear prediction for the DBBP prediction unit, the maximum complexity of DBBP can be reduced to a maximum of 44% by multiplication, a maximum of 26% by addition, and a memory bandwidth of 70%. Further, according to the experiment by the inventor, the decrease in encoding efficiency due to the use of this bilinear interpolation is 0.00% on an average of 8 sequences, and the processing amount can be decreased without decreasing the encoding efficiency. .

  According to the DBBP prediction unit 3095 having the above configuration, the segmentation unit 30952 derives segmentation information segMask that takes 0 or 1 for each pixel, and the image synthesis unit 30953 determines each of the target blocks based on the segmentation information segMask. The pixel is synthesized by selecting one of the two motion compensation images. Accordingly, an effect of reducing the processing of the image synthesizing unit 30953 is achieved as compared with a case where there are pixels that are synthesized by weighting, for example, two motion compensated images with a weight of 1/2. Also, not only segmentation information segMask [x] [y] corresponding to each pixel (x, y), but also segmentation information segMask [x] [y-1], segMask [x] [y + 1] , SegMask [x-1] [y] and segMask [x + 1] [y] are also referred to, and the effect of significantly reducing the processing of the image composition unit 30953 is achieved compared to the case of composition. Note that the DBBP split mode deriving unit 30954 included in the DBBP prediction unit 3095 can use another process. For example, a DBBP split mode deriving unit 30954A and a DBBP split mode deriving unit 30954B, which will be described later, may be used.

(DBBP prediction unit 3095A)
Hereinafter, a DBBP prediction unit 3095A, which is another configuration of the DBBP prediction unit 3095, will be described. The DBBP prediction unit 3095A includes a DBBP image interpolation unit 30951, a segmentation unit 30952, an image synthesis unit 30953, and a DBBP division mode derivation unit 30954A. The DBBP prediction unit 3095A basically has the same configuration as the DBBP prediction unit 3095, but includes a DBBP split mode deriving unit 30954A instead of the DBBP split mode deriving unit 30954. Since the DBBP image interpolation unit 30951 and the segmentation unit 30952 have already been described, description thereof will be omitted.

  The DBBP division mode deriving unit 30954A derives one of SIZE_Nx2N and SIZE_2NxN as the division mode partMode from the depth block refSamples corresponding to the target block. Specifically, based on the segmentation information segMask derived from the depth block refSamples, for each x and y (x = 0..nCbS-1, y = 0..nCbS-1) of the target block, Then, the sums partSum [0] and partSum [1] corresponding to the respective division modes, SIZE_Nx2N, and SIZE_2NxN are derived based on the following pseudo code.

for (y = 0; y <nCbS; y ++)
for (x = 0; x <nCbS; x ++) {
segFlag = segMask [x] [y]
partSum [0] [(x <(nCbS >> 1))? segFlag:! segFlag] ++
partSum [1] [(y <(nCbS >> 1))? segFlag:! segFlag] ++
}
Further, the DBBP partition mode deriving unit 30954 derives p that maximizes partSum [p] for p = 0.0.1, and sets the partition mode corresponding to p to SIZE_Nx2N and SIZE_2NxN. Derived as corresponding.

partIdc = 0
maxPartSum = 0
for (p = 0; p <2; p ++)
for (i = 0; i <2; i ++) {
if (partSum [p] [i]> maxPartSum) {
maxPartSum = partSum [p] [i]
partIdc = p
}
}
Unlike the DBBP prediction unit 3095, the DBBP prediction unit 3095A does not derive an asymmetric partition (AMP partition, SIZE_2NxnU, SIZE_2NxnD, SIZE_nLx2N, SIZE_nRx2N) as a partition mode, and only two partition modes N × 2N and 2N × N of symmetric partition Is targeted.

  According to the DBBP predicting unit 3095A having the above configuration, the DBBP split mode deriving unit 30954A uses only N × 2N or 2N × N as the split mode. There is an effect to reduce. That is, since only the total values partSum [0] and partSum [1] for the two partition modes are derived as the total values, the total values partSum [0], partSum [1], and partSum [2] for the six partition modes are derived. ], PartSum [3], partSum [4], and partSum [5] can be reduced to a third of the processing amount.

(DBBP prediction unit 3095B)
Hereinafter, a DBBP prediction unit 3095B, which is another configuration of the DBBP prediction unit 3095, will be described. The DBBP prediction unit 3095B basically has the same configuration as the DBBP prediction unit 3095, but includes a DBBP split mode deriving unit 30954B instead of the DBBP split mode deriving unit 30954.

  The DBBP split mode deriving unit 30954B determines the split mode PartMode with reference to only the four corner pixels of the target block shown in FIG. 13 of refSamples (hereinafter referred to as refDepPels) corresponding to the target block.

  Specifically, the coordinates xP0, xP0, yP0, yP1 corresponding to the upper left coordinates (xP0, yP0), upper right coordinates (xP1, yP0), lower left coordinates (xP0, xP1), and lower right coordinates (xP1, yP1) are as follows: Derived by the formula.

xP0 = Clip3 (0, pic_width_in_luma_samples-1, x _TL )
yP0 = Clip3 (0, pic_height_in_luma_samples-1, y _TL )
xP1 = Clip3 (0, pic_width_in_luma_samples-1, _xTL + nPSW-1)
yP1 = Clip3 (0, pic_height_in_luma_samples-1, y _TL + nPSH-1)
Furthermore, the upper left pixel TLrefDepPels [xP0] [yP0] and the lower right pixel BR are compared (refDepPels [xP0] [yP0] <refDepPels [xP1] [yP1]) and the lower left pixel BLrefDepPels [xP0] [yP0] From the comparison (refDepPels [xP1] [yP0] <refDepPels [xP0] [yP1]), a split flag horSplitFlag is derived by the following equation.

horSplitFlag = (refDepPels [xP0] [yP0] <refDepPels [xP1] [yP1])
= = (refDepPels [xP1] [yP0] <refDepPels [xP0] [yP1]))
The DBBP split mode deriving unit 30954B assigns 2N × N or N × 2N according to the split flag horSplitFlag. Specifically, the split mode PartMode is derived by assigning 2N × N when horSplitFlag is 1 and N × 2N when horSplitFlag is 0.

  According to the DBBP prediction unit 3095B having the above configuration, only the limited pixels of the depth block (here, the four pixels in the block and the pixels at the four corners of the block) are referred to. Compared to the above, the amount of processing is greatly reduced. Further, since it is not necessary to calculate the depth representative value or to derive the total value partSum [] for each division mode, the processing amount can be further reduced.

  According to the DBBP prediction unit 3095B having the above configuration, the division mode is derived by a simple process of comparing the upper left pixel and the lower right pixel of the depth and comparing the upper right pixel and the lower left pixel of the depth. As compared with the case where the process is performed, the processing amount is greatly reduced.

  Further, according to the DBBP prediction unit 3095B having the above configuration, as in the DBBP prediction unit 3095A, only N × 2N or 2N × N is used as the division mode. It has the effect of reducing

(DBBP prediction unit 3095C)
Hereinafter, a DBBP prediction unit 3095C, which is another configuration of the DBBP prediction unit 3095, will be described. The DBBP prediction unit 3095C has basically the same configuration as the DBBP prediction unit 3095B, but includes a DBBP split mode deriving unit 30954C instead of the DBBP split mode deriving unit 30954B.

  FIG. 21 is a block diagram illustrating a configuration of the DBBP prediction unit 3095C according to the embodiment of this invention. The DBBP prediction unit 3095C includes a DBBP image interpolation unit 30951, a segmentation unit 30952, an image synthesis unit 30953, and a DBBP division mode derivation unit 30954C.

  The DBBP split mode deriving unit 30954C uses the split flag deriving unit 353 to derive a split flag horSplitFlag having a value of 0 or 1. The DBBP split mode deriving unit 30954C derives a split mode based on horSplitFlag. For example, the DBBP split mode deriving unit 30954C derives a split mode by assigning 2N × N when horSplitFlag is 1 and N × 2N when horSplitFlag is 0.

  An image decoding apparatus including a DBBP prediction unit 3095C and a VSP prediction unit 30374 includes, as the DBBP prediction unit 3095C, a segmentation derivation unit 30952 that derives segmentation information from a depth image, and a DBBP image interpolation unit 30951 that generates two motion compensation images. The image synthesizing unit 30953 for synthesizing the two interpolation images to generate one motion compensation image, and the DBBP division mode deriving unit 30954C for deriving the division mode are provided. The VSP prediction unit 30374 includes a partition division unit that performs partition division according to depth, and a depth motion DV derivation unit 351 that derives a motion vector from the depth image. Further, the partition mode deriving unit 30954C and the partition division unit of the VSP prediction unit 30374 include a common partition mode deriving unit 353.

  According to the DBBP prediction unit 3095C having the above configuration, only the limited pixels of the depth block (here, the pixels at the four corners) are referred to, so that the processing amount is significantly reduced as compared with the case of referring to all the pixels. The effect to do.

  According to the image decoding apparatus included in the DBBP prediction unit 3095C having the above-described configuration, the DBBP prediction unit 3095C and the VSP prediction unit 30374 use the common partition flag derivation unit 353, and thus different methods are used in the DBBP prediction unit and the VSP prediction unit. As compared with the case where the division method is derived, the effect of simplifying the mounting is obtained.

  According to the DBBP prediction unit 3095C having the above-described configuration, since the division mode PartMode is derived by a simple process of comparing the upper left pixel and the lower right pixel of the depth block and comparing the upper right pixel and the lower left pixel of the depth block, There is an effect that the amount of processing is significantly reduced as compared with the case of performing comparison with respect to pixels.

  Further, according to the DBBP prediction unit 3095C having the above configuration, as in the case of the DBBP prediction unit 3095A, only N × 2N or 2N × N is used as the division mode. There is an effect of reducing the amount.

  Note that the DBBP prediction unit 3095C and the VSP prediction unit 30374 may use the division flag derivation unit 353A instead of the division flag derivation unit 353 as a common division flag derivation unit. Also in this case, according to the image decoding apparatus included in the DBBP prediction unit 3095C, the DBBP prediction unit 3095C and the VSP prediction unit 30374 use the same division flag derivation unit 353A, and therefore different methods are used in the DBBP prediction unit and the VSP prediction unit. As compared with the case where the division method is derived, the effect of simplifying the mounting is obtained.

(DBBP bi-prediction restriction)
The image decoding apparatus according to the modification of the present embodiment is configured not to apply bi-prediction in the case of DBBP. The image decoding apparatus according to the modified example includes an inter prediction parameter decoding control unit 3031A instead of the inter prediction parameter decoding control unit 3031 and a merge mode parameter deriving unit 3036A instead of the merge mode parameter deriving unit 3036. Since operations other than the inter prediction parameter decoding control unit 3031A and the merge mode parameter deriving unit 3036A are as described above, description thereof will be omitted.

  FIG. 23 is a schematic diagram showing the configuration of the inter prediction parameter decoding control decoding unit 3031A according to the present embodiment. The inter prediction parameter decoding control decoding unit 3031A has the same configuration as the inter prediction parameter decoding control decoding unit 3031, but includes an inter prediction identifier decoding unit 30312A instead of the inter prediction identifier decoding unit 30312.

  FIG. 24 is a diagram illustrating derivation of inter_pred_flag in the inter prediction parameter decoding control unit 3031A. inter_pred_flag is decoded when the slice type is B (bi-prediction is available). FIG. 24A is a diagram showing values that inter_pred_flag can take, and FIG. 24B shows a bit string (binarization) after CABAC decoding of inter_pred_flag. As shown in the figure, when the DBBP flag is 1, the inter prediction parameter decoding control unit 3031A (inter prediction identifier decoding unit 30312A) of the present embodiment does not decode inter_pred_flag = 2 (PRED_BI). That is, the inter-prediction parameter decoding control unit 3031A (inter-prediction identifier decoding unit 30312A) of the present embodiment has a case where the prediction unit is not a predetermined size ((nPbW + nPbH)! = 12) and the DBBP flag is 0. (Dbbp_flag == 0), 0 (PRED_L0), 1 (PRED_L1), 2 (PRED_BI) are decoded as inter_pred_flag, and the prediction unit has a predetermined size ((nPbW + nPbH)! = 12), or When the DBBP flag is 1 (dbbp_flag == 1), 0 (PRED_L0) and 1 (PRED_L1) are decoded as inter_pred_flag. The condition that the prediction unit is a predetermined size (the sum of the width and height of the prediction unit is 12) corresponds to the above-described bi-prediction restriction condition 1, and the condition that the DBBP flag is 1 is the following bi-prediction restriction condition. It corresponds to 2.

  Bi-prediction restriction condition 2: DBBP flag dbbp_flag is 1.

  Also, as shown in FIG. 24 (b), inter_pred_flag is calculated when the prediction unit is not a predetermined size ((nPbW + nPbH)! = 12) and the DBBP flag is 0 (dbbp_flag == 0). The bit string of inter_pred_flag is 00, 01, 1 and 0 (PRED_L0), 1 (PRED_L1), and 2 (PRED_BI) correspond to each. In the above case, the inter prediction parameter decoding control unit 3031A (inter prediction identifier decoding unit 30312A) decodes 00, 01, and 1 bit strings, and inter_pred_flag is set to 0 (PRED_L0), 1 (PRED_L1), and 2 (PRED_BI). Decrypt by assigning. Otherwise, when the prediction unit has a predetermined size ((nPbW + nPbH)! = 12) or the DBBP flag is 1 (dbbp_flag == 1), the bit string of inter_pred_flag is 0, 1 and 0 (PRED_L0 ), 1 (PRED_L1) corresponds. In the above case, the inter prediction parameter decoding control unit 3031A decodes 0 and 1 bit strings, and assigns 0 (PRED_L0) and 1 (PRED_L1) to inter_pred_flag.

  FIG. 25 is a block diagram illustrating a configuration of the merge mode parameter deriving unit 3036A. The merge mode parameter derivation unit 3036 includes a merge candidate derivation unit 30361, a merge candidate selection unit 30362, and a bi-prediction restriction unit 30363A. The partial operation of the merge mode parameter deriving unit 3036A and the means other than the bi-prediction limiting unit 30363A are as described above, and thus the description thereof is omitted.

  The merge mode parameter deriving unit 3036A adds the DBBP flag dbbp_flag to the bi-prediction restriction unit 30363A in addition to the prediction parameter derived by the merge candidate selection unit 30362, the width nOrigPbW and the height nOrigPbH of the prediction unit. Output.

  The bi-prediction restriction unit 30363A sets refIdxL1 = -1 and predFlagL1 = 0 to the reference picture index refIdxL1 of L1 and the prediction use flag predFlagL1 of L1 in the case of the above-described bi-prediction restriction condition 1 or bi-prediction restriction condition 2. Thus, bi-prediction (predFlagL0 = 1 and predFlagL1 = 1) is converted to single prediction.

  According to the bi-prediction restriction unit 30363A having the above-described configuration, the interpolation image derived by the DBBP prediction unit 3095 is limited to the case of uni-prediction (in the case of a reference picture of L0 or L1). Compared with the case of generating an interpolated image using prediction, the processing amount and the transfer amount are greatly reduced.

  Further, when both the inter prediction parameter decoding control unit 3031A and the merge mode parameter deriving unit 3036A having the above-described configuration are provided, the case where bi-prediction is applied is completely prohibited in DBBP prediction performed when dbbp_flag is 1. can do.

(Configuration of image encoding device)
Next, the configuration of the image encoding device 11 according to the present embodiment will be described. FIG. 26 is a block diagram illustrating a configuration of the image encoding device 11 according to the present embodiment. The image encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a prediction parameter memory (prediction parameter storage). Section, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, and prediction parameter coding unit 111. The prediction parameter encoding unit 111 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113.

  The predicted image generation unit 101 generates predicted picture blocks predSamples for each block which is an area obtained by dividing the picture for each viewpoint of the layer image T input from the outside. Here, the predicted image generation unit 101 reads the reference picture block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter encoding unit 111. The prediction parameter input from the prediction parameter encoding unit 111 is, for example, a motion vector or a displacement vector. The predicted image generation unit 101 reads the reference picture block of the block at the position indicated by the motion vector or the displacement vector predicted from the encoded prediction unit. The predicted image generation unit 101 generates predicted picture blocks predSamples using one prediction method among a plurality of prediction methods for the read reference picture block. The predicted image generation unit 101 outputs the generated predicted picture block predSamples to the subtraction unit 102 and the addition unit 106. Note that since the predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 already described, details of generation of the predicted picture block predSamples are omitted.

  In order to select a prediction method, the predicted image generation unit 101, for example, calculates an error value based on a difference between a signal value for each pixel of a block included in the layer image and a signal value for each corresponding pixel of the predicted picture block predSamples. Select the prediction method to minimize. Note that the method of selecting the prediction method is not limited to this.

  When the encoding target picture is a base view picture, the plurality of prediction methods are intra prediction, motion prediction, and merge mode. Motion prediction is prediction between display times among the above-mentioned inter predictions. The merge mode is a prediction that uses the same reference picture block and prediction parameter as a block that has already been encoded and is within a predetermined range from the prediction unit. When the encoding target picture is a picture other than the base view, the plurality of prediction methods are intra prediction, motion prediction, merge mode (including viewpoint synthesis prediction), and displacement prediction. The displacement prediction (disparity prediction) is prediction between different layer images (different viewpoint images) in the above-described inter prediction. For displacement prediction (disparity prediction), there are predictions with and without additional prediction (residual prediction and illuminance compensation).

  When the intra prediction is selected, the predicted image generation unit 101 outputs a prediction mode PredMode indicating the intra prediction mode used when generating the predicted picture block predSamples to the prediction parameter encoding unit 111.

  When motion prediction is selected, the predicted image generation unit 101 stores the motion vector mvLX used when generating the predicted picture block predSamples in the prediction parameter memory 108 and outputs the motion vector mvLX to the inter prediction parameter encoding unit 112. The motion vector mvLX indicates a vector from the position of the encoded prediction unit to the position of the reference picture block when the predicted picture block predSamples is generated. The information indicating the motion vector mvLX may include information indicating a reference picture (for example, a reference picture index refIdxLX, a picture order number POC), and may represent a prediction parameter. Further, the predicted image generation unit 101 outputs a prediction mode PredMode indicating the inter prediction mode to the prediction parameter encoding unit 111.

  When the prediction image generation unit 101 selects displacement prediction, the prediction image generation unit 101 stores the displacement vector used when generating the prediction picture block predSamples in the prediction parameter memory 108 and outputs the prediction vector to the inter prediction parameter encoding unit 112. The displacement vector dvLX indicates a vector from the position of the encoded prediction unit to the position of the reference picture block when the predicted picture block predSamples is generated. The information indicating the displacement vector dvLX may include information indicating a reference picture (for example, reference picture index refIdxLX, view IDview_id) and may represent a prediction parameter. Further, the predicted image generation unit 101 outputs a prediction mode PredMode indicating the inter prediction mode to the prediction parameter encoding unit 111.

  When the merge mode is selected, the predicted image generation unit 101 outputs a merge index merge_idx indicating the selected reference picture block to the inter prediction parameter encoding unit 112. Further, the predicted image generation unit 101 outputs a prediction mode PredMode indicating the merge mode to the prediction parameter encoding unit 111.

  In the merge mode described above, when the VSP mode flag VspModeFlag indicates that the viewpoint synthesis prediction is performed, the prediction image generation unit 101 performs the viewpoint in the VSP prediction unit 30374 included in the prediction image generation unit 101 as described above. Perform synthetic prediction. Further, in the motion prediction, displacement prediction, and merge mode, the prediction image generation unit 101 includes the prediction image generation unit 101 as described above when the residual prediction execution flag resPredFlag indicates that the residual prediction is performed. The residual prediction unit 3092 performs residual prediction.

  The subtraction unit 102 subtracts the signal value of the prediction picture block predSamples input from the prediction image generation unit 101 for each pixel from the signal value of the corresponding block of the layer image T input from the outside, and generates a residual signal. Generate. The subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103 and the encoding parameter determination unit 110.

  The DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102 and calculates a DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain a quantization coefficient. The DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy encoding unit 104 and the inverse quantization / inverse DCT unit 105.

  The entropy coding unit 104 receives the quantization coefficient from the DCT / quantization unit 103 and the coding parameter from the coding parameter determination unit 110. The input encoding parameters include codes such as a reference picture index refIdxLX, a prediction vector flag mvp_LX_flag, a difference vector mvdLX, a prediction mode PredMode, a merge index merge_idx, a residual prediction index iv_res_pred_weight_idx, and an illumination compensation flag ic_flag.

  The entropy encoding unit 104 generates an encoded stream Te by entropy encoding the input quantization coefficient and encoding parameter, and outputs the generated encoded stream Te to the outside.

  The inverse quantization / inverse DCT unit 105 inversely quantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 105 performs inverse DCT on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 105 outputs the calculated decoded residual signal to the addition unit 106 and the encoding parameter determination unit 110.

  The addition unit 106 adds the signal value of the prediction picture block predSamples input from the prediction image generation unit 101 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and refers to them. Generate a picture block. The adding unit 106 stores the generated reference picture block in the reference picture memory 109.

  The prediction parameter memory 108 stores the prediction parameter generated by the prediction parameter encoding unit 111 at a predetermined position for each picture and block to be encoded.

  The reference picture memory 109 stores the reference picture block generated by the addition unit 106 at a predetermined position for each picture and block to be encoded.

  The encoding parameter determination unit 110 selects one set from among a plurality of sets of encoding parameters. The encoding parameter is a parameter to be encoded that is generated in association with the above-described prediction parameter or the prediction parameter. The predicted image generation unit 101 generates predicted picture blocks predSamples using each of these sets of encoding parameters.

  The encoding parameter determination unit 110 calculates a cost value indicating the amount of information and the encoding error for each of the plurality of sets. The cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient λ. The code amount is the information amount of the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameter. The square error is the sum between pixels regarding the square value of the residual value of the residual signal calculated by the subtracting unit 102. The coefficient λ is a real number larger than a preset zero. The encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the set of unselected encoding parameters.

  The prediction parameter encoding unit 111 derives a prediction parameter used when generating a prediction picture based on the parameter input from the prediction image generation unit 101, and encodes the derived prediction parameter to generate a set of encoding parameters. To do. The prediction parameter encoding unit 111 outputs the generated set of encoding parameters to the entropy encoding unit 104.

  The prediction parameter encoding unit 111 stores, in the prediction parameter memory 108, a prediction parameter corresponding to the one selected by the encoding parameter determination unit 110 from the generated set of encoding parameters.

  The prediction parameter encoding unit 111 operates the inter prediction parameter encoding unit 112 when the prediction mode PredMode input from the prediction image generation unit 101 indicates the inter prediction mode. The prediction parameter encoding unit 111 operates the intra prediction parameter encoding unit 113 when the prediction mode PredMode indicates the intra prediction mode.

  The inter prediction parameter encoding unit 112 derives an inter prediction parameter based on the prediction parameter input from the encoding parameter determination unit 110. The inter prediction parameter encoding unit 112 includes the same configuration as the configuration in which the inter prediction parameter decoding unit 303 derives the inter prediction parameter as a configuration for deriving the inter prediction parameter. The configuration of the inter prediction parameter encoding unit 112 will be described later.

  The intra prediction parameter encoding unit 113 determines the intra prediction mode IntraPredMode indicated by the prediction mode PredMode input from the encoding parameter determination unit 110 as a set of inter prediction parameters.

(Configuration of inter prediction parameter encoding unit)
Next, the configuration of the inter prediction parameter encoding unit 112 will be described. The inter prediction parameter encoding unit 112 is means corresponding to the inter prediction parameter decoding unit 303. FIG. 27 is a schematic diagram illustrating the configuration of the inter prediction parameter encoding unit 112 according to the present embodiment. The inter prediction parameter encoding unit 112 includes a merge mode parameter deriving unit 1121, an AMVP prediction parameter deriving unit 1122, a subtracting unit 1123, and an inter prediction parameter encoding control unit 1126.

  The merge mode parameter deriving unit 1121 has the same configuration as the merge mode parameter deriving unit 3036 (see FIG. 9).

  The AMVP prediction parameter derivation unit 1122 has the same configuration as the AMVP prediction parameter derivation unit 3032 (see FIG. 10).

  The subtraction unit 1123 subtracts the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the vector mvLX input from the coding parameter determination unit 110 to generate a difference vector mvdLX. The difference vector mvdLX is output to the inter prediction parameter encoding control unit 1126.

  The inter prediction parameter coding control unit 1126 instructs the entropy coding unit 104 to decode a code related to inter prediction (the syntax element) includes, for example, a code (syntax element) included in the coded data. , Merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector flag mvp_LX_flag, and difference vector mvdLX are encoded.

  The inter prediction parameter encoding control unit 1126 includes a residual prediction index encoding unit 10311, an illumination compensation flag encoding unit 10312, a merge index encoding unit, a vector candidate index encoding unit, a split mode encoding unit, and a merge flag encoding. , An inter prediction identifier encoding unit, a reference picture index encoding unit, and a vector difference encoding unit. The division mode encoding unit, the merge flag encoding unit, the merge index encoding unit, the inter prediction identifier encoding unit, the reference picture index encoding unit, the vector candidate index encoding unit, and the vector difference encoding unit are respectively divided mode part_mode , Merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector flag mvp_LX_flag, and difference vector mvdLX are encoded.

  The residual prediction index encoding unit 10311 encodes the residual prediction index iv_res_pred_weight_idx to indicate whether or not residual prediction is performed.

  The illuminance compensation flag encoding unit 10312 encodes the illuminance compensation flag ic_flag to indicate whether or not illuminance compensation is performed.

  When the prediction mode PredMode input from the prediction image generation unit 101 indicates the merge mode, the inter prediction parameter encoding control unit 1126 uses the merge index merge_idx input from the encoding parameter determination unit 110 as the entropy encoding unit 104. To be encoded.

  Also, the inter prediction parameter encoding control unit 1126 performs the following process when the prediction mode PredMode input from the prediction image generation unit 101 indicates the inter prediction mode.

  The inter prediction parameter encoding control unit 1126 integrates the reference picture index refIdxLX and the prediction vector flag mvp_LX_flag input from the encoding parameter determination unit 110 and the difference vector mvdLX input from the subtraction unit 1123. The inter prediction parameter encoding control unit 1126 outputs the integrated code to the entropy encoding unit 104 to be encoded. The inter prediction parameter encoding control unit 1126 includes a DBBP flag encoding unit dbbp_flag (not shown).

  The predicted image generation unit 101 is means corresponding to the predicted image generation unit 308 described above, and the process for generating a predicted image from the prediction parameters is the same.

  In the present embodiment, the predicted image generation unit 101 also includes the above-described residual synthesis unit 30923 as with the predicted image generation unit 308. That is, residual prediction is not performed when the size of the target block (predicted block) is equal to or smaller than a predetermined size. Also, the predicted image generation unit 101 of the present embodiment performs residual prediction only when the division mode part_mode of the coding unit CU is 2N × 2N. That is, the residual prediction index iv_res_pred_weight_idx is set to 0. Also, the residual prediction index encoding unit 10311 of the present embodiment encodes the residual prediction index iv_res_pred_weight_idx only when the division mode part_mode of the encoding unit CU is 2N × 2N.

  The image encoding device including the residual prediction unit 3092 is the image encoding device including the residual prediction index encoding unit that encodes the residual prediction index, and the division mode of the encoding unit including the target block is 2N ×. In the case of 2N, the residual prediction index is encoded. In other cases, the residual prediction index is not encoded, and when the residual prediction index is other than 0, residual prediction is performed.

(DBBP prediction)
Further, the predicted image generation unit 101 included in the image encoding device 11 of the present embodiment includes a DBBP prediction unit 3095. Details of the operation of the DBBP prediction unit 3095 have already been described, and are omitted here. The DBBP prediction unit 3095 performs depth-based block prediction when 1 is encoded as the DBBP flag encoding unit dbbp_flag described above.

  According to the image coding apparatus including the DBBP prediction unit 3095 having the above configuration, the DBBP image interpolation unit 30951 generates an interpolation image by bilinear prediction of two interpolation images, so that the processing amount and the transfer amount are greatly reduced. The effect to do.

  According to the image coding apparatus including the DBBP prediction unit 3095 having the above configuration, the segmentation unit 30952 derives segmentation information segMask that takes 0 or 1 for each pixel, and the image synthesis unit 30953 outputs the segmentation information segMask to the segmentation information segMask. Based on this, synthesis is performed by selecting one of the two motion compensation images in each pixel of the target block. Accordingly, an effect of reducing the processing of the image synthesizing unit 30953 is achieved as compared with a case where there are pixels that are synthesized by weighting, for example, two motion compensated images with a weight of 1/2.

  In addition, according to the image coding apparatus including the DBBP prediction unit 3095 having the above configuration, not only segmentation information segMask [x] [y] corresponding to each pixel (x, y), but also up / down / left / right segmentation information segMask Image compositing compared to compositing with reference to [x] [y-1], segMask [x] [y + 1], segMask [x-1] [y], segMask [x + 1] [y] The effect of greatly reducing the processing of the unit 30953 is achieved.

(Modification of DBBP)
The predicted image generation unit 101 included in the image encoding device 11 may include any of the DBBP prediction unit 3095A, the DBBP prediction unit 3095B, and the DBBP prediction unit 3095C instead of the DBBP prediction unit 3095.

  According to the image coding apparatus including the DBBP prediction units 3095A to 3095C having the above-described configuration, only the limited pixels of the depth block (here, the four corner pixels) are referred to, and thus all the pixels are referred to. Compared with the case where it does, there exists an effect which reduces a processing amount significantly.

  According to the image coding apparatus including the DBBP prediction units 3095A to 3095C having the above configuration, the division is performed by a simple process of comparing the upper left pixel and the lower right pixel of the depth and comparing the upper right pixel and the lower left pixel of the depth. Since the mode is derived, the processing amount is greatly reduced as compared with the case where all the pixels are compared.

  In addition, according to the image encoding device including the DBBP prediction unit 3095B and the DBBP prediction unit 3095C having the above configuration, as in the DBBP prediction unit 3095A, only N × 2N or 2N × N is used as the division mode. There is an effect of reducing the processing amount as compared with the case where AMP division is targeted.

  According to the image coding apparatus included in the DBBP prediction unit 3095C having the above configuration, the DBBP prediction unit 3095C and the VSP prediction unit 30374 use the common split mode deriving unit 3096. Therefore, different methods are used in the DBBP prediction unit and the VSP prediction unit. As compared with the case where the division method is derived by using the method, the effect of simplifying the mounting is obtained.

(DBBP bi-prediction restriction)
The image encoding device 11 according to the modification of the present embodiment is configured not to apply bi-prediction in the case of DBBP. Specifically, an inter prediction parameter encoding unit 103A (not shown) instead of the inter prediction parameter encoding unit 103, a merge mode parameter encoding unit 1036A and a merge mode parameter deriving unit 3036A (not shown) instead of the merge mode parameter deriving unit 3036. Is provided. The inter prediction parameter encoding unit 103A is means corresponding to the above-described inter prediction parameter decoding control unit 3031A.

  When the DBBP flag is 1, the inter prediction parameter encoding unit 103A does not encode inter_pred_flag = 2 (PRED_BI). That is, the inter prediction parameter encoding unit 103A of the present embodiment, when the prediction unit is not a predetermined size ((nPbW + nPbH)! = 12) and the DBBP flag is 0 (dbbp_flag == 0), When 0 (PRED_L0), 1 (PRED_L1), and 2 (PRED_BI) are decoded as inter_pred_flag and the prediction unit has a predetermined size ((nPbW + nPbH)! = 12) or the DBBP flag is 1 (dbbp_flag == 1), 0 (PRED_L0) and 1 (PRED_L1) are encoded as inter_pred_flag.

  Also, inter_pred_flag indicates that when the prediction unit is not a predetermined size ((nPbW + nPbH)! = 12) and the DBBP flag is 0 (dbbp_flag == 0), the inter prediction parameter encoding unit 103A sets inter_pred_flag 00, 01, and 1 are encoded as a bit string. Otherwise, when the prediction unit has a predetermined size ((nPbW + nPbH)! = 12) or the DBBP flag is 1 (dbbp_flag == 1), the inter prediction parameter encoding unit 103A sets the bit string of inter_pred_flag to 0, Encode as 1.

  In addition, since the image coding apparatus according to the present embodiment includes the merge mode parameter deriving unit 3036A, in the case of the above-described bi-prediction restriction condition 1 or bi-prediction restriction condition 2, the L1 reference picture indexes refIdxL1 and L1 are used for prediction. By setting refIdxL1 = −1 and predFlagL1 = 0 to the flag predFlagL1, bi-prediction is converted into single prediction.

  That is, the image coding apparatus according to the present embodiment includes a DBBP prediction unit 3095 and a merge mode parameter derivation unit 3036A. The DBBP prediction unit 3095 includes a segmentation derivation unit 30952 that derives segmentation information from a depth image, and two motion compensations. A DBBP image interpolation unit 30951 that generates an image and an image synthesis unit 30953 that combines the two interpolation images to generate one motion compensation image, and the image encoding device encodes a DBBP flag (not shown). A DBBP flag encoding unit is further provided, and the merge mode parameter deriving unit 3036A is an image encoding device that performs conversion from bi-prediction to single prediction when the DBBP flag is 1.

  According to the image coding unit including the merge mode parameter deriving unit 3036A having the above-described configuration, the interpolated image derived by the DBBP prediction unit 3095 is uni-prediction (in the case of a reference picture of L0 or L1. Since 1 or predFlagL1 is limited to 1), the worst case of the processing amount and the transfer amount is greatly reduced as compared with the case where an interpolated image can be generated using DBBP prediction in bi-prediction. Play.

  Similarly, according to the image encoding unit including the inter prediction parameter encoding unit 103A having the above configuration, when the DBBP flag dbbp_flag is 1, a single prediction (PRED_L0 or PRED_L1) value is encoded as the inter prediction identifier inter_pred_idc. Since the bi-prediction PRED_BI value is not encoded, it is prohibited to perform bi-prediction in the case of DBBP prediction. Therefore, the worst case of the processing amount and the transfer amount is greatly reduced as compared with the case where an interpolation image can be generated using DBBP prediction in bi-prediction.

  Further, when both the inter prediction parameter encoding unit 103A and the merge mode parameter deriving unit 3036A having the above configuration are provided, the case where bi-prediction is applied is completely prohibited in the DBBP prediction performed when dbbp_flag is 1. can do.

  Note that a part of the image encoding device 11 and the image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the predicted image generation unit 101, the DCT / quantization unit 103, and entropy encoding. Unit 104, inverse quantization / inverse DCT unit 105, encoding parameter determination unit 110, prediction parameter encoding unit 111, entropy decoding unit 301, prediction parameter decoding unit 302, predicted image generation unit 308, inverse quantization / inverse DCT unit 311 may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. Here, the “computer system” is a computer system built in either the image encoding device 11 or the image decoding device 31 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In such a case, a volatile memory inside a computer system serving as a server or a client may be included and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  Moreover, you may implement | achieve part or all of the image coding apparatus 11 in the embodiment mentioned above and the image decoding apparatus 31 as integrated circuits, such as LSI (Large Scale Integration). Each functional block of the image encoding device 11 and the image decoding device 31 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to

(Additional notes)
The present invention can also be expressed as follows.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. In the depth-based block predicted image generation apparatus including the image synthesizing unit, the image interpolation unit generates the two motion compensation images by bilinear prediction.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. A depth-based block prediction image generation apparatus including an image synthesizing unit that further includes a depth division mode deriving unit for deriving a division mode from the depth image, wherein the depth division mode deriving unit is divided from pixels at four corners of the depth block A mode is derived.

  One aspect of the present invention is characterized in that the depth division mode deriving unit derives a division mode from a comparison of the upper left and lower right of the depth and a comparison of the upper right and lower left of the depth.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. The depth-based block prediction image generation apparatus including the image synthesizing unit further includes a depth division mode deriving unit for deriving a division mode from the depth image, and the depth division mode deriving unit has a 2N × N or N × 2N division mode. It is derived.

  According to one aspect of the present invention, a segmentation deriving unit that derives segmentation information from a depth image, an image interpolating unit that generates two motion compensation images, and a single motion compensation image are generated by combining the two interpolation images. A depth-based block prediction image generation apparatus including an image synthesis unit that further includes a depth division mode deriving unit that derives a division mode from a depth image, and the segmentation deriving unit derives segmentation information that takes 0 or 1 for each pixel. The image synthesizing unit synthesizes by selecting one of the two interpolated images at each pixel of the block.

  One embodiment of the present invention is an image decoding device including the depth base block prediction image generation device and the DBBP flag decoding unit, and the depth base block prediction image generation device performs DBBP prediction when the DBBP flag is 1. It is characterized by performing.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a view synthesis prediction unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images, an image synthesis unit that generates one motion compensation image by synthesizing the two interpolation images, and a division mode deriving unit that derives a division mode. The composite prediction means includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the division mode derivation unit and the partition division unit include a common division mode. A derivation unit is provided.

  In one aspect of the present invention, the division mode deriving unit is configured such that the depth division mode deriving unit compares the upper left pixel and the lower right pixel of the depth block corresponding to the target block, and compares the upper right pixel and the lower left pixel of the depth block. The division mode is derived from

  In one form of the present invention, the division mode deriving unit derives an absolute value difference between the horizontal directions from two depth pixels having the same vertical component coordinates from the depth block corresponding to the target block, and An absolute value difference between vertical directions is derived from two depth pixels having the same coordinates, and when the absolute value difference in the horizontal direction is larger than the absolute value difference in the vertical direction, the absolute value difference is divided vertically. In this case, a partition mode is derived by horizontally dividing, and the partition dividing unit divides into 4 × 8 sub-blocks when the absolute value difference in the horizontal direction is larger than the absolute value difference in the vertical direction. In other cases, it is divided into 8 × 4 sub-blocks horizontally long.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a merge mode parameter derivation unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images and an image synthesis unit that combines the two interpolation images to generate one motion compensation image. The image decoding apparatus further includes a DBBP flag decoding unit. The merge mode parameter deriving unit converts from single prediction to bi-prediction when the DBBP flag is 1.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and an inter prediction parameter decoding unit, wherein the depth base block prediction image generation unit includes a segmentation deriving unit for deriving segmentation information from the depth image, An image interpolation unit that generates two motion compensation images and an image synthesis unit that combines the two interpolation images to generate one motion compensation image. The image decoding apparatus further includes a DBBP flag decoding unit. The inter prediction parameter decoding unit does not decode a bi-prediction value as an inter prediction identifier when the DBBP flag is 1.

  One embodiment of the present invention is an image encoding device including the depth base block prediction image generation device and the DBBP flag encoding unit, and the depth base block prediction image generation device is configured to perform DBBP when the DBBP flag is 1. An image encoding apparatus that performs prediction.

  One embodiment of the present invention is an image encoding device including a depth base block prediction image generation unit and a viewpoint synthesis prediction unit, wherein the depth base block prediction image generation unit is a segmentation deriving unit that derives segmentation information from a depth image. An image interpolation unit that generates two motion compensation images, an image synthesis unit that combines the two interpolation images to generate one motion compensation image, and a division mode derivation unit that derives a division mode, The viewpoint synthesis prediction unit includes a partition division unit that performs partition division in the depth, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the division mode derivation unit and the partition division unit include: A common division mode deriving unit is provided.

  One aspect of the present invention is an image encoding device including a depth base block prediction image generation unit and a merge mode parameter derivation unit, wherein the depth base block prediction image generation unit derives segmentation information from the depth image. And an image interpolating unit for generating two motion compensated images, and an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and the image encoding device includes a DBBP flag encoding unit And the merge mode parameter derivation unit converts from single prediction to bi-prediction when the DBBP flag is 1.

  One aspect of the present invention is an image decoding apparatus including a depth base block prediction image generation unit and a view synthesis prediction unit, wherein the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from a depth image, and An image interpolation unit that generates two motion compensation images, an image synthesis unit that generates one motion compensation image by synthesizing the two interpolation images, and a division mode deriving unit that derives a division mode. The composite prediction unit includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image. The segmentation derivation unit of the depth base block prediction image generation unit, the division mode The derivation unit derives the position of the depth image to be referenced. A disparity vector used, characterized by a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit.

  According to one aspect of the present invention, the common disparity vector is a disparity vector refined by depth when the block size is larger than a predetermined size, and is reduced when the block size is equal to or smaller than the predetermined size. It is a disparity vector before being refined by.

  In one aspect of the present invention, the common disparity vector is a disparity vector refined by depth when the sum of the width and height of the prediction block is greater than 16, and by depth otherwise. It is a disparity vector before being refined.

  In one aspect of the present invention, the common disparity vector is a disparity vector refined by depth if the sum of the width and height of the prediction block is greater than 24, and by depth otherwise. It is a disparity vector before being refined.

  One embodiment of the present invention is an image encoding device including a depth base block prediction image generation unit and a viewpoint synthesis prediction unit, wherein the depth base block prediction image generation unit is a segmentation deriving unit that derives segmentation information from a depth image. An image interpolation unit that generates two motion compensation images, an image synthesis unit that combines the two interpolation images to generate one motion compensation image, and a division mode derivation unit that derives a division mode, The viewpoint synthesis prediction unit includes a partition division unit that performs partition division from the depth image, and a depth motion vector derivation unit that derives a motion vector from the depth image, and the segmentation derivation unit of the depth base block prediction image generation unit, the division Deriving the position of the depth image referenced by the mode deriving unit Wherein the disparity vector, that a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit for use in.

  The present invention can be suitably applied to an image decoding apparatus that decodes encoded data obtained by encoding image data and an image encoding apparatus that generates encoded data obtained by encoding image data. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.

DESCRIPTION OF SYMBOLS 1 ... Image transmission system 11 ... Image encoding apparatus 101 ... Predictive image generation part 102 ... Subtraction part 103 ... DCT / quantization part 10311 ... Residual prediction index encoding part 10312 ... Illuminance compensation flag encoding part 104 ... Entropy encoding Unit 105 ... inverse quantization / inverse DCT unit 106 ... addition unit 108 ... prediction parameter memory (frame memory)
109 ... Reference picture memory (frame memory)
DESCRIPTION OF SYMBOLS 110 ... Coding parameter determination part 111 ... Prediction parameter coding part 112 ... Inter prediction parameter coding part 1121 ... Merge mode parameter derivation part 1122 ... AMVP prediction parameter derivation part 1123 ... Subtraction part 1126 ... Inter prediction parameter coding control part 113 ... Intra prediction parameter encoding unit 21 ... Network 31 ... Image decoding device 301 ... Entropy decoding unit 302 ... Prediction parameter decoding unit 303 ... Inter prediction parameter decoding units 3031 and 3031A ... Inter prediction parameter decoding control unit 30311 ... Division mode decoding unit 30312 , 30312A ... Inter prediction identifier decoding unit 30313 ... DBBP flag decoding unit 3032 ... AMVP prediction parameter derivation unit 3035 ... Addition unit 3036 ... Merge mode parameter derivation unit 30361 ... Merge candidate Output unit 303611 ... Merge candidate storage unit 30362 ... Merge candidate selection unit 30363, 30363A ... Bi-prediction restriction unit 30370 ... Extended merge candidate derivation unit 30371 ... Inter-layer merge candidate derivation unit 30373 ... Displacement merge candidate derivation unit 30374 ... VSP merge candidate derivation (VSP prediction unit, view synthesis prediction means, partition division unit, depth vector derivation unit)
30380 ... Basic merge candidate derivation unit 30281 ... Spatial merge candidate derivation unit 30382 ... Temporal merge candidate derivation unit 30383 ... Combined merge candidate derivation unit 30384 ... Zero merge candidate derivation unit 304 ... Intra prediction parameter decoding unit 306 ... Reference picture memory (frame memory)
307 ... Prediction parameter memory (frame memory)
308 ... Prediction image generation unit 309 ... Inter prediction image generation unit 3091 ... Motion displacement compensation unit 3092 ... Residual prediction unit 30922 ... Reference image interpolation unit 30923 ... Residual synthesis unit 30924 ... Residual prediction vector derivation unit 3093 ... Illuminance compensation Units 3095, 3095A, 3095B, 3095C ... DBBP prediction unit (depth base block prediction image generation device)
30951 ... DBBP image interpolation unit (image interpolation unit, image interpolation means)
30952 ... Segmentation unit 30953 ... Image composition units 30954, 30954A, 30954B, 30954C ... DBBP division mode deriving unit (depth division mode deriving means)
3096 ... Weighted prediction unit 310 ... Intra prediction image generation unit 311 ... Inverse quantization / inverse DCT unit 312 ... Addition unit 351 ... Depth DV derivation unit 352 ... Displacement vector derivation unit 353 ... Split mode derivation unit 354 ... Switch 41 ... Image display apparatus

Claims (19)

  In an image decoding apparatus including a depth base block predicted image generation unit and a merge mode parameter derivation unit, the depth base block prediction image generation unit generates a segmentation derivation unit that derives segmentation information from a depth image, and generates two motion compensation images. An image interpolating unit that combines the two interpolated images to generate one motion compensated image, the image decoding device further includes a DBBP flag decoding unit, and the merge mode parameter deriving unit includes: An image decoding apparatus that converts from bi-prediction to uni-prediction when the DBBP flag is 1.   In an image decoding apparatus including a depth base block prediction image generation unit and an inter prediction parameter decoding unit, the depth base block prediction image generation unit generates a segmentation deriving unit for deriving segmentation information from the depth image, and generates two motion compensation images. And an image synthesizing unit that synthesizes the two interpolated images to generate one motion compensated image. The image decoding apparatus further includes a DBBP flag decoding unit, and the inter prediction parameter decoding unit includes: An image decoding apparatus that does not decode a bi-prediction value as an inter prediction identifier when the DBBP flag is 1.   In the image coding apparatus including a depth base block predicted image generation unit and a merge mode parameter derivation unit, the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from the depth image, and two motion compensation images. An image interpolating unit for generating and an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and the image encoding device further includes a DBBP flag encoding unit for encoding the DBBP flag. And the merge mode parameter deriving unit converts from single prediction to bi-prediction when the DBBP flag is 1.   A segmentation deriving unit for deriving segmentation information from a depth image, an image interpolating unit for generating two motion compensated images, an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and a division mode. A depth-based block predicted image generation apparatus comprising a depth division mode deriving unit for deriving, wherein the depth division mode deriving unit derives a 2N × N or N × 2N division mode.   A depth base comprising a segmentation deriving unit for deriving segmentation information from a depth image, an image interpolating unit for generating two motion compensated images, and an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image In the block prediction image generation device, the image interpolation unit generates the two motion compensated images by bilinear prediction, and the depth base block prediction image generation device.   A segmentation deriving unit for deriving segmentation information from a depth image, an image interpolating unit for generating two motion compensated images, an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and a division mode. A depth-based block prediction image generation apparatus comprising a depth division mode deriving unit for deriving, wherein the depth division mode deriving unit derives a division mode from pixels at four corners of a depth block.   5. The depth division mode deriving unit derives a division mode from a comparison between an upper left pixel and a lower right pixel of a depth block corresponding to a target block and a comparison between an upper right pixel and a lower left pixel of the depth block. Or the depth-based block prediction image generation device according to 6.   The depth-based block prediction image generation device according to any one of claims 4 to 7, wherein the segmentation derivation unit derives segmentation information that takes 0 or 1 for each pixel, and the image synthesis unit includes: A depth-based block predictive image generation apparatus, characterized in that synthesis is performed by selecting one of the two interpolated images in each pixel of a block.   The depth-based block prediction image generation device according to any one of claims 4 to 8, and an image decoding device including the DBBP flag decoding unit, wherein the depth-base block prediction image generation device has a DBBP flag of 1. In this case, an image decoding apparatus that performs DBBP prediction. In an image decoding apparatus including a depth-based block prediction image generation unit and a viewpoint synthesis prediction unit,
The depth-based block predicted image generation means includes a segmentation deriving unit for deriving segmentation information from the depth image, an image interpolation unit for generating two motion compensation images, and a single motion compensation image by combining the two interpolation images. An image synthesizing unit for generating a partition mode, and a partition mode deriving unit for deriving a division mode, wherein the viewpoint synthesis predicting means includes a partition division unit that performs partition division from a depth image, and a depth motion DV that derives a motion vector from the depth image. An image decoding apparatus comprising: a derivation unit, wherein the partition mode derivation unit and the partition division unit include a common partition mode derivation unit.   The depth division mode deriving unit derives a division mode from a comparison between an upper left pixel and a lower right pixel of a depth block corresponding to a target block and a comparison between an upper right pixel and a lower left pixel of the depth block. The image decoding device described.   The division mode deriving unit derives an absolute value difference between the horizontal directions from two depth pixels having the same vertical component coordinates from the depth block corresponding to the target block, and from the two depth pixels having the same horizontal component coordinates. By deriving the absolute value difference between the vertical directions, if the absolute value difference in the horizontal direction is larger than the absolute value difference in the vertical direction, it is divided vertically, otherwise it is divided horizontally A partition mode is derived, and the partition division unit divides the block into 4 × 8 sub-blocks when the horizontal absolute value difference is larger than the vertical absolute value difference, and 8 otherwise. The image decoding device according to claim 9, wherein the image decoding device is divided into × 4 sub-blocks horizontally long.   The depth-based block prediction image generation device according to any one of claims 3 to 7 and an image encoding device including a DBBP flag encoding unit, wherein the depth-based block prediction image generation device includes a DBBP flag. An image encoding apparatus that performs DBBP prediction when is 1.   In the image coding apparatus including the depth base block prediction image generation unit and the viewpoint synthesis prediction unit, the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from the depth image, and two motion compensation images. An image interpolating unit for generating, an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and a split mode deriving unit for deriving a split mode. A partition division unit that performs partition division and a depth motion vector derivation unit that derives a motion vector from the depth image are provided. The division mode derivation unit and the partition division unit include a common division mode derivation unit. An image encoding apparatus characterized by that.   In an image decoding apparatus including a depth base block prediction image generation unit and a viewpoint synthesis prediction unit, the depth base block prediction image generation unit generates a segmentation deriving unit for deriving segmentation information from the depth image, and generates two motion compensation images. An image interpolating unit for synthesizing the image, an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and a division mode deriving unit for deriving a division mode. A partition division unit that performs partition division and a depth motion vector derivation unit that derives a motion vector from the depth image, and the depth image that is referred to by the segmentation derivation unit and the division mode derivation unit of the depth-based block prediction image generation unit. The disparity vector used to derive the position and Image decoding apparatus characterized by a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit.   The common disparity vector is a disparity vector refined by depth when the block size is larger than a predetermined size, and a disparity vector before being refined by depth when the block size is equal to or smaller than the predetermined size. The image decoding apparatus according to claim 15, wherein:   The common disparity vector is a disparity vector refined by depth when the sum of the width and height of the prediction block is larger than 16, and is a disparity vector before being refined by depth otherwise. The image decoding apparatus according to claim 16, wherein the image decoding apparatus is provided.   The common disparity vector is a disparity vector refined by depth when the sum of the width and height of the prediction block is greater than 24, and is a disparity vector before being refined by depth otherwise. The image decoding device according to claim 17, wherein the image decoding device is provided.   In the image coding apparatus including the depth base block prediction image generation unit and the viewpoint synthesis prediction unit, the depth base block prediction image generation unit includes a segmentation derivation unit that derives segmentation information from the depth image, and two motion compensation images. An image interpolating unit for generating, an image synthesizing unit for synthesizing the two interpolated images to generate one motion compensated image, and a division mode deriving unit for deriving a division mode. A depth division image that is referred to by the segmentation derivation unit and the division mode derivation unit of the depth-based block prediction image generation unit, including a partition division unit that performs partition division from a depth image and a depth motion vector derivation unit that derives a motion vector from the depth image. Disparity vector used to derive the position of The image encoding apparatus characterized by a common disparity vector and a disparity vector used to derive the position of the depth image in the partitioning portion of the view synthesized predicting means and the depth motion vector derivation unit.

Images