WO2018174618A1

WO2018174618A1 - Prediction method and device using reference block

Info

Publication number: WO2018174618A1
Application number: PCT/KR2018/003393
Authority: WO
Inventors: 이진호; 임성창; 강정원; 고현석; 이하현; 전동산; 조승현; 김휘용; 김재곤; 박도현; 최해철
Original assignee: 한국전자통신연구원; 한국항공대학교 산학협력단; 한밭대학교 산학협력단
Priority date: 2017-03-22
Filing date: 2018-03-22
Publication date: 2018-09-27
Also published as: KR20220136316A

Abstract

The present invention relates to a decoding method, a decoding device, an encoding method and an encoding device for a video, wherein, in encoding and decoding a video, a prediction mode, among multiple prediction modes, may be selected using prediction mode information for a block to be predicted, and prediction may be executed for the block to be predicted on the basis of the selected prediction mode. The prediction mode information may include information associated with a reference block, wherein the reference block may include at least one among a spatially neighboring block spatially adjacent to the block to be predicted, and a temporally neighboring block temporally adjacent to the block to be predicted. If multiple prediction modes are selected, a prediction mode, among the multiple selected prediction modes, to be used for the prediction of the block to be predicted may be determined by the signaling of prediction mode determining information.

Description

Prediction method and apparatus using reference blocks

The following embodiments relate to a video decoding method, a decoding apparatus, an encoding method, and an encoding apparatus, and a method and an apparatus for performing prediction using a reference block in encoding and decoding of a video.

The present invention claims the benefits of the Korean Patent Application No. 10-2017-0036257 filed March 22, 2017 and the Korean Patent Application No. 10-2018-0033279 filed March 22, 2018, the contents of all Is included herein.

Through the continuous development of the information and telecommunications industry, broadcasting services having high definition (HD) resolution have spread worldwide. This proliferation has resulted in many users becoming accustomed to high resolution, high quality images and / or video.

In order to satisfy users' demand for high image quality, many organizations are spurring the development of next generation video devices. Users are interested in Ultra High Definition (UHD) TVs, which have four times the resolution of FHD TVs, as well as High Definition TV (HDTV) and Full HD (FHD) TVs. This has increased, and as the interest increases, an image encoding / decoding technique for an image having higher resolution and image quality is required.

An image encoding / decoding apparatus and method include an inter prediction technique, an intra prediction technique, an entropy encoding technique, etc. in order to perform encoding / decoding of high resolution and high quality images. Can be used. The inter prediction technique may be a technique of predicting a value of a pixel included in a current picture by using a previous picture temporally and / or a later picture temporally. An intra prediction technique may be a technique of predicting a value of a pixel included in a current picture by using information of a pixel in a current picture. The entropy encoding technique may be a technique of allocating a short code to a symbol having a high appearance frequency and a long code to a symbol having a low appearance frequency.

Various prediction methods have been developed to improve the efficiency and accuracy of intra prediction and / or inter prediction. The efficiency of the prediction may vary greatly depending on which prediction method among the various applicable prediction methods is used for encoding and / or decoding of the block.

An embodiment may provide an encoding apparatus, an encoding method, a decoding apparatus, and a decoding method that perform prediction by using information about a reference block.

An embodiment may provide an encoding apparatus, an encoding method, a decoding apparatus, and a decoding method that perform prediction by using information about a spatial reference block and / or a temporal reference block.

The processor may include a processor configured to select a prediction mode from a plurality of prediction modes using prediction mode information on a target block, and to perform prediction on the target block based on the selected prediction mode. The information includes information related to a reference block, and the reference block is provided with an encoding apparatus including one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.

The processor may further include a processor configured to select a prediction mode from a plurality of prediction modes using prediction mode information on the target block, and to perform prediction on the target block based on the selected prediction mode. Mode information includes information related to a reference block, and the reference block is provided with a decoding apparatus including one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.

In another aspect, the method includes: selecting a prediction mode from among a plurality of prediction modes by using prediction mode information on a target block; And performing prediction on the target block based on the selected prediction mode, wherein the prediction mode information includes information related to a reference block, wherein the reference block is a spatial neighboring block spatially adjacent to the target block and the reference block. A decoding method including one or more of temporal neighboring blocks temporally adjacent to a target block is provided.

The selected prediction mode may be plural.

A prediction mode to be used for prediction of the target block among the plurality of selected prediction modes may be determined using prediction mode determination information.

The decoding method may further include receiving a bitstream including the prediction mode determination information.

The prediction mode determination information may include one or more partial prediction mode determination information.

The selected prediction mode may be one or more.

At least some of the one or more partial prediction mode determination information may be omitted in the signaling of the prediction mode determination information according to the selected one or more prediction modes.

The prediction mode determination information may include a matching mode identifier.

When there are a plurality of selected prediction modes, the matching mode identifier may indicate a prediction mode used for decoding the target block among the plurality of selected prediction modes.

When there is one selected prediction mode, the prediction mode determination information may not include the matching mode identifier.

The prediction mode determination information may include a matching prediction identifier.

The matching prediction identifier may indicate whether the prediction mode is selected from the plurality of prediction modes.

The information related to the reference block includes a motion vector of the reference block, a prediction direction of the reference block, a reference picture index of the reference block, an encoding mode of the reference block, a reference picture list of the reference block, and a picture order of the reference block. One or more of a picture order count (POC), a quantization parameter (QP) of the reference block, and a coded block flag (CBF) of the reference block.

The plurality of prediction modes may include a template matching mode and a lateral matching mode.

The reference block may be plural.

The motion vectors of the plurality of reference blocks may be derived, the correlation of the derived motion vectors may be determined, and the prediction mode may be selected based on the correlation.

According to the comparison between the correlation and the threshold, one of a template matching mode and a bilateral matching mode may be selected as the prediction mode of the target block.

The prediction mode may be selected from among the plurality of prediction modes using a prediction direction.

The prediction direction may be a prediction direction of the target block or a prediction direction of the reference block.

The prediction mode may be selected according to whether the inter prediction of the target block is unidirectional prediction or bidirectional prediction.

If the inter prediction is bidirectional prediction, a bilateral matching mode may be selected from the plurality of prediction modes.

If the inter prediction is unidirectional prediction, a template matching mode may be selected from the plurality of prediction modes.

The prediction mode may be selected from among the plurality of prediction modes using the coding parameter of the reference block.

It is determined whether a prediction mode of the reference block is an intra prediction mode or an inter prediction mode according to a coding parameter of the reference block, and a plurality of prediction modes according to whether the prediction mode of the reference block is an intra prediction mode or an inter prediction mode. Among these, the selected prediction mode may be determined.

The prediction mode may be selected from the plurality of prediction modes based on the difference between the specified POCs.

Provided are an encoding device, an encoding method, a decoding device, and a decoding method for performing prediction using information about a reference block.

Provided are an encoding device, an encoding method, a decoding device, and a decoding method for performing prediction using information on a spatial reference block and / or a temporal reference block.

1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.

2 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

4 is a diagram illustrating a form of a prediction unit PU that a coding unit CU may include.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

6 is a diagram for explaining an embodiment of an intra prediction process.

7 is a diagram for describing a position of a reference sample used in an intra prediction process.

8 is a diagram for explaining an embodiment of an inter prediction process.

9 illustrates spatial candidates according to an example.

10 illustrates an addition order of spatial information of motion candidates to a merge list according to an example.

11 illustrates a process of transform and quantization according to an example.

12 is a structural diagram of an encoding apparatus according to an embodiment.

13 is a structural diagram of a decoding apparatus according to an embodiment.

14 is a flowchart of a prediction method, according to an exemplary embodiment.

15 is a flowchart of a method of selecting a prediction mode, according to an exemplary embodiment.

16 illustrates reference blocks according to an example.

17 illustrates signaling of prediction mode determination information when two prediction modes are selected according to an example.

18 illustrates signaling of prediction mode determination information when one prediction mode is selected according to an example.

19 illustrates signaling of prediction mode determination information when one prediction mode is selected according to an example.

20 illustrates prediction of a target block using a bilateral matching mode according to an example.

21 illustrates prediction for a target block using a template matching mode according to an example.

22 is a flowchart of a method of predicting a target block and a method of generating a bitstream, according to an embodiment.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

DETAILED DESCRIPTION For the following detailed description of exemplary embodiments, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention in connection with one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled.

Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects. Shape and size of the elements in the drawings may be exaggerated for clarity.

In the present invention, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is said to be "connected" or "connected" to another component, the above two components may be directly connected to or connected to each other, It is to be understood that other components may exist in the middle of the two components. When a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that no other component exists between the two components.

The components shown in the embodiments of the present invention are independently shown to represent different characteristic functions, and do not mean that each component is composed of separate hardware or one software unit. In other words, each component is listed as each component for convenience of description, and at least two of each component is combined into one component, or one component is divided into a plurality of components to provide a function. Combined and separate embodiments of each of these components, which can be carried out, are also included within the scope of the present invention without departing from the spirit of the invention.

In addition, the description "including" a specific configuration in the exemplary embodiments does not exclude a configuration other than the specific configuration described above, the additional configuration is the implementation of the exemplary embodiments or the technical spirit of the exemplary embodiments. It can be included in the range.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present invention, the terms "comprise" or "having" and the like are intended to indicate that there exists a feature, number, step, operation, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof. In other words, the description "include" a specific configuration in the present invention does not exclude a configuration other than the configuration, it means that additional configuration may be included in the scope of the technical spirit of the present invention or the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the embodiments. In describing the embodiments, when it is determined that the detailed description of the related well-known configuration or function may obscure the subject matter of the present specification, the detailed description thereof will be omitted. In addition, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

In the following description, an image may mean one picture constituting a video and may represent a video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video" and may mean "encoding and / or decoding of one of images constituting the video." It may be.

Hereinafter, the terms "video" and "motion picture" may be used interchangeably and may be used interchangeably.

Hereinafter, the target image may be an encoding target image that is a target of encoding and / or a decoding target image that is a target of decoding. The target image may be an input image input to the encoding apparatus or may be an input image input to the decoding apparatus.

Hereinafter, the terms "image", "picture", "frame" and "screen" may be used in the same sense and may be used interchangeably.

Hereinafter, the target block may be an encoding target block that is a target of encoding and / or a decoding target block that is a target of decoding. In addition, the target block may be a current block that is a target of current encoding and / or decoding. For example, the terms "target block" and "current block" may be used interchangeably and may be used interchangeably.

In the following, the terms “block” and “unit” may be used interchangeably and may be used interchangeably. Or “block” may indicate a particular unit.

In the following, the terms “region” and “segment” may be used interchangeably.

In the following, the specific signal may be a signal representing a specific block. For example, the original signal may be a signal representing a target block. The prediction signal may be a signal representing a prediction block. The residual signal may be a signal representing a residual block.

In embodiments, each of the specified information, data, flags and elements, attributes, etc. may have a value. The value "0" of information, data, flags and elements, attributes, etc. may represent a logical false or first predefined value. In other words, the value "0", false, logical false and the first predefined value can be used interchangeably. The value "1" of information, data, flags and elements, attributes, etc. may represent logical true or second predefined values. In other words, the value "1", true, logical true and the second predefined value can be used interchangeably.

When a variable such as i or j is used to indicate a row, column, or index, the value of i may be an integer of 0 or more and may be an integer of 1 or more. In other words, in embodiments, rows, columns, indexes, etc. may be counted from zero and counted from one.

In the following, terms used in the embodiments are described.

Encoder: Refers to a device that performs encoding.

Decoder: Refers to an apparatus for performing decoding.

Unit: A unit may represent a unit of encoding and decoding of an image. The terms "unit" and "block" may be used interchangeably and may be used interchangeably.

The unit may be an M × N array of samples. M and N may each be a positive integer. A unit can often mean an array of two-dimensional samples.

In encoding and decoding of an image, a unit may be an area generated by division of one image. One image may be divided into a plurality of units. Alternatively, the unit may mean the divided portion when an image is divided into subdivided portions, and encoding or decoding is performed on the divided portion.

In encoding and decoding of an image, predefined processing for a unit may be performed according to the type of the unit.

Depending on the function, the type of unit may be a macro unit, a coding unit (CU), a prediction unit (PU), a residual unit, a transform unit (TU), or the like. Can be classified as. Alternatively, depending on the function, the unit may be a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit. It may mean a (Prediction Unit), a prediction block, a residual unit, a residual block, a transform unit, and a transform block.

A unit may refer to information including a luma component block and a corresponding chroma component block, and a syntax element for each block, to refer to the block separately.

The size and shape of the unit may vary. In addition, the unit may have various sizes and various shapes. In particular, the shape of the unit may include not only square but also geometric figures that can be expressed in two dimensions such as rectangles, trapezoids, triangles, and pentagons.

In addition, the unit information may include at least one of a type of a unit indicating a coding unit, a prediction unit, a residual unit, a transform unit, and the like, a size of a unit, a depth of a unit, an encoding and decoding order of the unit, and the like.

One unit can be further divided into sub-units having a smaller size than the unit.

Unit Depth: The unit depth may refer to the degree of division of the unit. In addition, the unit depth may indicate the level at which the unit exists when the unit is represented in a tree structure.

The unit splitting information may comprise a unit depth with respect to the depth of the unit. Unit depth may indicate the number and / or degree of unit division.

In the tree structure, the root node has the shallowest depth and the leaf node has the deepest depth.

One unit may be divided into a plurality of sub-units hierarchically with depth information based on a tree structure. In other words, the unit and the lower unit generated by the division of the unit may correspond to the node and the child node of the node, respectively. Each divided subunit may have a unit depth. Since the unit depth indicates the number and / or degree of division of the unit, the division information of the lower unit may include information about the size of the lower unit.

In the tree structure, the highest node may correspond to the first unit that is not split. The highest node may be called the root node. In addition, the highest node may have a minimum depth value. At this time, the highest node may have a depth of level 0.

A node with a depth of level 1 may represent a unit created as the first unit is divided once. A node with a depth of level 2 may represent a unit created as the first unit is split twice.

A node with a depth of level n may represent a unit generated as the first unit is divided n times.

The leaf node may be the lowest node or may be a node that cannot be further divided. The depth of the leaf node may be at the maximum level. For example, the predefined value of the maximum level may be three.

Sample: A sample may be a base unit constituting a block. The sample may be represented as values from 0 to 2 ^Bd- 1 depending on the bit depth Bd.

The sample may be a pixel or pixel value.

In the following, the terms "pixel", "pixel" and "sample" may be used in the same sense and may be used interchangeably.

Coding Tree Unit (CTU): A CTU may consist of one luminance component (Y) coding tree block and two color difference component (Cb, Cr) coding tree blocks associated with the luminance component coding tree block. have. In addition, the CTU may mean that the block and the syntax element for each block of the blocks.

Each coding tree unit may be split using one or more partitioning schemes such as a quad tree and a binary tree to form sub-units such as a coding unit, a prediction unit, and a transform unit.

-CTU can be used as a term for referring to a pixel block which is a processing unit in a decoding and encoding process of an image, as in the division of an input image.

Coding Tree Block (CTB): A coding tree block may be used as a term for referring to any one of a Y coding tree block, a Cb coding tree block, and a Cr coding tree block.

Neighbor block: A block adjacent to the target block. The block adjacent to the target block may mean a block in which a boundary of the target block abuts or a block located within a predetermined distance from the target block. The neighboring block may mean a block adjacent to a vertex of the target block. The block adjacent to the vertex of the target block may be a block vertically adjacent to a neighboring block horizontally adjacent to the target block or a block horizontally adjacent to a neighboring block vertically adjacent to the target block. The neighboring block may mean a restored neighboring block.

Prediction unit: This may mean a base unit for prediction such as inter prediction, intra prediction, inter compensation, intra compensation, and motion compensation.

One prediction unit may be divided into a plurality of partitions or lower prediction units having a smaller size. The plurality of partitions may also be the base unit in performing prediction or compensation. The partition generated by the partitioning of the prediction unit may also be the prediction unit.

Prediction unit partition: This may mean a form in which a prediction unit is divided.

Reconstructed neighboring unit: The reconstructed neighboring unit may be a unit that is already decoded and reconstructed around the target unit.

The reconstructed neighbor unit may be a spatial neighbor unit or a temporal neighbor unit to the target unit.

The reconstructed spatial peripheral unit may be a unit within the current picture and already reconstructed through encoding and / or decoding.

The reconstructed temporal peripheral unit may be a unit in the reference image and a unit already reconstructed through encoding and / or decoding. The position in the reference image of the reconstructed temporal peripheral unit may be the same as the position in the current picture of the target unit or may correspond to the position in the current picture of the target unit.

Parameter set: The parameter set may correspond to header information among structures in the bitstream. For example, the parameter set may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate-distortion optimization: The encoding apparatus uses a combination of the size of the coding unit, the prediction mode, the size of the prediction unit, the motion information, and the size of the transform unit to provide high coding efficiency. Distortion optimization can be used.

The rate-distortion optimization method can calculate the rate-distortion cost of each combination in order to select the optimal combination among the above combinations. Rate-distortion cost can be calculated using Equation 1 below. In general, a combination in which the rate-distortion cost is minimized may be selected as an optimal combination in the rate-distortion optimization scheme.

[Equation 1]

D may represent distortion. D may be the mean square error of the squares of difference values between the original transform coefficients and the reconstructed transform coefficients in the transform unit.

-R can represent the rate R may indicate a bit rate using the associated context information.

λ may represent the Lagrangian multiplier. R may include not only coding parameter information such as prediction mode, motion information, coded block flag, etc., but also bits generated by encoding of transform coefficients.

The encoding apparatus may perform processes such as inter prediction and / or intra prediction, transformation, quantization, entropy encoding, inverse quantization, inverse transformation, etc. to calculate accurate D and R. These processes can greatly increase the complexity in the encoding apparatus.

Bitstream: A bitstream may mean a string of bits including encoded image information.

Parameter set: The parameter set may correspond to header information among structures in the bitstream.

The parameter set may comprise at least one of a video parameter set, a sequence parameter set, a picture parameter set and an adaptation parameter set. In addition, the parameter set may include information of a slice header and information of a tile header.

Parsing: Parsing may mean entropy decoding a bitstream to determine a value of a syntax element. Alternatively, parsing may refer to entropy decoding itself.

A symbol: may mean at least one of syntax elements, coding parameters, transform coefficients, etc. of the encoding target unit and / or decoding target unit. In addition, the symbol may mean an object of entropy encoding or a result of entropy decoding.

Reference picture: The reference picture may mean an image referenced by a unit for inter prediction or motion compensation. Alternatively, the reference picture may be an image including a reference unit referenced by the target unit for inter prediction or motion compensation.

Hereinafter, the terms "reference picture" and "reference picture" may be used in the same sense and may be used interchangeably.

Reference picture list: The reference picture list may be a list including one or more reference pictures used for inter prediction or motion compensation.

The types of reference picture lists are List Combined (LC), List 0 (List 0; L0), List 1 (List 1; L1), List 2 (List 2; L2), and List 3 (List 3; L3). ) And the like.

One or more reference picture lists may be used for inter prediction.

Inter prediction indicator: The inter prediction indicator may indicate the direction of inter prediction of the target unit. The inter prediction may be one of unidirectional prediction and bidirectional prediction. Alternatively, the inter prediction indicator may indicate the number of reference pictures used when generating the prediction unit of the target unit. Alternatively, the inter prediction indicator may mean the number of prediction blocks used for inter prediction or motion compensation for the target unit.

Reference picture index: The reference picture index may be an index indicating a specific reference picture in the reference picture list.

Motion Vector (MV): The motion vector may be a two-dimensional vector used in inter prediction or motion compensation. The motion vector may mean an offset between an encoding target image / decoding target image and a reference image.

For example, MV can be expressed in the form (mv _x , mv _y ). mv _x may represent a horizontal component and mv _y may represent a vertical component.

Search range: The search range may be a two-dimensional area in which a search for MV is performed during inter prediction. For example, the size of the search region may be M × N. M and N may each be a positive integer.

Motion vector candidate: A motion vector candidate may mean a motion vector of a block that is a prediction candidate or a block that is a prediction candidate when predicting a motion vector.

The motion vector candidate may be included in the motion vector candidate list.

Motion vector candidate list: A motion vector candidate list may mean a list constructed using one or more motion vector candidates.

Motion vector candidate index: The motion vector candidate index may mean an indicator indicating a motion vector candidate in the motion vector candidate list. Alternatively, the motion vector candidate index may be an index of a motion vector predictor.

Motion information: The motion information includes not only a motion vector, a reference picture index and an inter prediction indicator, but also reference picture list information, a reference picture, a motion vector candidate, a motion vector candidate index, a merge candidate and a merge index, and the like. It may mean information including at least one.

Merge candidate list: The merge candidate list may mean a list constructed using merge candidates.

Merge candidate: means a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-prediction merge candidate, a zero merge candidate, and the like. The merge candidate may include motion information such as an inter prediction indicator, a reference picture index for each list, and a motion vector.

Merge index: The merge index may be an indicator indicating a merge candidate in the merge candidate list.

The merge index may indicate a reconstructed unit that induced a merge candidate among the reconstructed units spatially adjacent to the target unit and the reconstructed units temporally adjacent to the target unit.

The merge index may indicate at least one of motion information of the merge candidate.

Transform unit: A transform unit may be a basic unit in residual signal encoding and / or residual signal decoding such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, transform coefficient decoding, and the like. One transform unit can be divided into a plurality of transform units of smaller size.

Scaling: Scaling may refer to a process of multiplying a transform coefficient level by a factor.

As a result of scaling for the transform coefficient level, a transform coefficient can be generated. Scaling may be referred to as dequantization.

Quantization Parameter (QP): The quantization parameter may mean a value used when generating a transform coefficient level for a transform coefficient in quantization. Or, it may mean a value used when generating transform coefficients by scaling transform coefficient levels in inverse quantization. Alternatively, the quantization parameter may be a value mapped to a quantization step size.

Delta quantization parameter: The delta quantization parameter refers to a differential value of the predicted quantization parameter and the quantization parameter of the encoding / decoding target unit.

Scan: A scan may refer to a method of ordering coefficients within a unit, block, or matrix. For example, aligning a two-dimensional array in the form of a one-dimensional array may be referred to as a scan. Alternatively, arranging the one-dimensional array in the form of a two-dimensional array may also be referred to as a scan or inverse scan.

Transform coefficient: The transform coefficient may be a coefficient value generated by performing a transform in the encoding apparatus. Alternatively, the transform coefficient may be a coefficient value generated by performing at least one of entropy decoding and inverse quantization in the decoding apparatus.

A quantized level or a quantized transform coefficient level obtained by applying quantization to a transform coefficient or a residual signal may also be included in the meaning of the transform coefficient.

Quantized level: A quantized level may mean a value generated by performing quantization on a transform coefficient or a residual signal in an encoding apparatus. Alternatively, the quantized level may mean a value that is an object of inverse quantization in performing inverse quantization in the decoding apparatus.

The quantized transform coefficient level resulting from the transform and quantization may also be included in the meaning of the quantized level.

Non-zero transform coefficient: A non-zero transform coefficient may mean a transform coefficient having a nonzero value or a transform coefficient level having a nonzero value. Alternatively, the non-zero transform coefficient may mean a transform coefficient whose magnitude is not zero or a transform coefficient level whose magnitude is not zero.

Quantization Matrix: A quantization matrix may refer to a matrix used in a quantization or inverse quantization process to improve subjective or objective image quality of an image. The quantization matrix may also be called a scaling list.

Quantization matrix coefficient: The quantization matrix coefficient may mean each element in the quantization matrix. Quantization matrix coefficients may also be referred to as matrix coefficients.

Default matrix: The default matrix may be a quantization matrix predefined in the encoding apparatus and the decoding apparatus.

Non-default Matrix: The non-default matrix may be a quantization matrix that is not predefined in the encoding apparatus and the decoding apparatus. The non-default matrix may be signaled from the encoding device to the decoding device.

The encoding apparatus 100 may be an encoder, a video encoding apparatus, or an image encoding apparatus. The video may include one or more images. The encoding apparatus 100 may sequentially encode one or more images of the video.

Referring to FIG. 1, the encoding apparatus 100 may include an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, and entropy encoding. The unit 150 may include an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on the target image using an intra mode and / or an inter mode.

In addition, the encoding apparatus 100 may generate a bitstream including encoding information through encoding of the target image, and may output the generated bitstream. The generated bitstream can be stored in a computer readable recording medium and can be streamed via wired / wireless transmission media.

As an prediction mode, when an intra mode is used, the switch 115 may be switched to intra. When the inter mode is used as the prediction mode, the switch 115 may be switched to the inter.

The encoding apparatus 100 may generate a prediction block for the target block. In addition, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the target block and the prediction block.

When the prediction mode is the intra mode, the intra prediction unit 120 may use a pixel of an already coded / decoded block around the target block as a reference sample. The intra prediction unit 120 may perform spatial prediction on the target block by using the reference sample, and generate prediction samples on the target block through spatial prediction.

The inter predictor 110 may include a motion predictor and a motion compensator.

When the prediction mode is the inter mode, the motion predictor may search an area that best matches the target block from the reference image in the motion prediction process, and derive a motion vector for the target block and the searched area using the searched area. can do.

The reference picture may be stored in the reference picture buffer 190 and may be stored in the reference picture buffer 190 when encoding and / or decoding of the reference picture is processed.

The motion compensator may generate a prediction block for the target block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional vector used for inter prediction. In addition, the motion vector may indicate an offset between the target image and the reference image.

If the motion predictor and the motion compensator have non-integer values, the motion predictor and the motion compensator may generate a prediction block by applying an interpolation filter to a portion of the reference image. In order to perform inter prediction or motion compensation, methods of motion prediction and motion compensation of a PU included in a CU based on a CU include a skip mode, a merge mode, and an advanced motion vector prediction. Whether it is a prediction (AMVP) mode or a current picture reference mode may be determined, and inter prediction or motion compensation may be performed according to each mode.

The subtractor 125 may generate a residual block that is a difference between the target block and the prediction block. The residual block may be referred to as the residual signal.

The residual signal may mean a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing the difference between the original signal and the prediction signal, or by transforming and quantizing. The residual block may be a residual signal on a block basis.

The transform unit 130 may generate transform coefficients by performing transform on the residual block, and output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block.

When the transform skip mode is applied, the transform unit 130 may omit the transform of the residual block.

By applying quantization to the transform coefficients, a quantized transform coefficient level or quantized level can be generated. In the following embodiments, the quantized transform coefficient level and the quantized level may also be referred to as transform coefficients.

The quantization unit 140 may generate a quantized transform coefficient level or a quantized level by quantizing the transform coefficients according to the quantization parameter. The quantization unit 140 may output the generated quantized transform coefficient level or the generated quantized level. In this case, the quantization unit 140 may quantize the transform coefficients using the quantization matrix.

The entropy encoder 150 may generate a bitstream by performing entropy encoding according to a probability distribution based on the values calculated by the quantizer 140 and / or the coding parameter values calculated in the encoding process. . The entropy encoder 150 may output the generated bitstream.

The entropy encoder 150 may perform entropy encoding on information about pixels of an image and information for decoding an image. For example, the information for decoding the image may include a syntax element.

The coding parameter may be information required for encoding and / or decoding. The coding parameter may include information encoded by the encoding apparatus 100 and transferred from the encoding apparatus 100 to the decoding apparatus, and may include information that may be inferred in the encoding or decoding process. For example, there is a syntax element as information transmitted to the decoding apparatus.

For example, coding parameters may include prediction modes, motion vectors, reference picture indexes, coding block patterns, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block sizes, block partitions. ) May include information such as information or statistics. The prediction mode may indicate an intra prediction mode or an inter prediction mode.

The residual signal may represent a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the prediction signal.

When entropy coding is applied, a small number of bits may be allocated to a symbol having a high occurrence probability, and a large number of bits may be allocated to a symbol having a low occurrence probability. As the symbol is represented through this assignment, the size of the bitstring for the symbols to be encoded may be reduced. Therefore, compression performance of image encoding may be improved through entropy encoding.

In addition, the entropy encoder 150 may use exponential golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding for entropy encoding. A coding method such as Arithmetic Coding (CABAC) may be used. For example, the entropy encoder 150 may perform entropy coding using a variable length coding (VLC) table. For example, the entropy encoder 150 may derive a binarization method for the target symbol. Also, the entropy encoder 150 may derive a probability model of the target symbol / bin. The entropy encoder 150 may perform arithmetic coding using the derived binarization method, probability model, and context model.

The entropy encoder 150 may change a coefficient of a form of a 2D block into a form of a 1D vector through a transform coefficient scanning method to encode a transform coefficient level.

The coding parameter may include information derived from an encoding process or a decoding process, as well as information (or a flag, an index, etc.) encoded by the encoding apparatus and signaled from the encoding apparatus to the decoding apparatus, as the syntax element. have. In addition, the coding parameter may include information required for encoding or decoding an image. For example, the size of the unit / block, the depth of the unit / block, the partition information of the unit / block, the partition structure of the unit / block, the information indicating whether the unit / block is divided into quad tree form, the unit / block is binary Information indicating whether or not to be split into a tree, binary tree split direction (horizontal or vertical), binary tree split type (symmetric split or asymmetric split), prediction method (intra prediction or inter prediction), intra prediction Mode / direction, reference sample filtering method, prediction block filtering method, prediction block boundary filtering method, filter tab of filtering, filter coefficient of filtering, inter prediction mode, motion information, motion vector, reference picture index, inter prediction direction, inter prediction Indicator, reference picture list, reference picture, motion vector predictor, motion vector prediction candidate, motion vector candidate list, merge mode Information indicating whether the interpolation filter is used, the merge candidate, the merge candidate list, whether to use the skip mode, the type of interpolation filter, the filter tab of the interpolation filter, the filter coefficient of the interpolation filter, the motion vector size, the motion vector representation accuracy, Transform type, transform size, information indicating whether to use a primary transform, information indicating whether to use an additional (secondary) transform, primary transform index, secondary transform index, information indicating the presence or absence of a residual signal, code Coded block patterns, coded block flags, quantization parameters, quantization matrices, information about intra-loop filters, information about whether intra-loop filters are applied, and information on intra-loop filters Coefficients, the filter tab of the intra-loop, the shape / form of the intra loop filter, information indicating whether or not the deblocking filter is applied, the deblocking filter coefficients , Deblocking filter tab, deblocking filter strength, deblocking filter shape / shape, information indicating whether to apply adaptive sample offset, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, adaptive Whether to apply in-loop filter, adaptive in-loop filter coefficients, adaptive in-loop filter tab, adaptive in-loop filter shape / shape, binarization / debinarization method, context model, context Model Determination Method, Context Model Update Method, Whether to Perform Regular Mode, Bypass Mode, Context Bean, Bypass Bin, Transform Coefficient, Transform Coefficient Level, Transform Coefficient Level Scanning Method, Display / Output Order of Image Information about slice identification information, slice type, slice partition information, tile identification information, tile type, tile partition information, picture type, bit depth, luminance signal, and At least one value or a combined form of the information on the chrominance signal may be included in the coding parameter.

Here, signaling a flag or an index means that the encoding apparatus 100 converts an entropy coded flag or an entropy coded index generated by performing entropy encoding on a flag or an index into a bitstream. In the decoding apparatus 200, the decoding apparatus 200 may mean obtaining an flag or an index by performing entropy decoding on an entropy coded flag or an entropy coded index extracted from the bitstream. .

Since encoding through inter prediction is performed by the encoding apparatus 100, the encoded target image may be used as a reference image with respect to other image (s) to be processed later. Accordingly, the encoding apparatus 100 may reconstruct or decode the encoded target image and store the reconstructed or decoded image in the reference picture buffer 190 as a reference image. Inverse quantization and inverse transform on the encoded target image may be processed for decoding.

The quantized level may be inversely quantized in the inverse quantization unit 160 and inversely transformed in the inverse transformer 170. The inverse quantized and / or inverse transformed coefficients may be summed with the prediction block via the adder 175. A reconstructed block may be generated by adding the inverse quantized and / or inverse transformed coefficients with the prediction block. Here, the inverse quantized and / or inversely transformed coefficient may refer to a coefficient in which at least one or more of dequantization and inverse-transformation are performed, and may mean a reconstructed residual block.

The reconstructed block may pass through the filter unit 180. The filter unit 180 may reconstruct or reconstruct at least one or more of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF). It can be applied to a picture. The filter unit 180 may be referred to as an in-loop filter.

The deblocking filter may remove block distortion generated at boundaries between blocks. To determine whether to apply the deblocking filter, it may be determined whether to apply the deblocking filter to the target block based on the pixel (s) included in some columns or rows included in the block. When the deblocking filter is applied to the target block, the filter applied may vary depending on the strength of the required deblocking filtering. In other words, a filter determined according to the strength of deblocking filtering among different filters may be applied to the target block.

The SAO may add an appropriate offset to the pixel value of the pixel to compensate for coding errors. The SAO may perform correction using an offset on a difference between an original image and a deblocked image in units of pixels for the deblocked image. After dividing the pixels included in the image into a predetermined number of areas, a method of determining an area to be offset from the divided areas and applying an offset to the determined area may be used. The edge information of each pixel of the image may be used. In consideration, a method of applying an offset may be used.

The ALF may perform filtering based on a value obtained by comparing the reconstructed image with the original image. After dividing a pixel included in an image into predetermined groups, a filter to be applied to the divided group may be determined, and filtering may be performed for each group. Information related to whether to apply the adaptive loop filter may be signaled for each CU. The shape and filter coefficients of the ALF to be applied to each block may vary from block to block.

The reconstructed block or the reconstructed image that has passed through the filter unit 180 may be stored in the reference picture buffer 190. The reconstructed block that has passed through the filter unit 180 may be part of a reference picture. In other words, the reference picture may be a reconstructed picture composed of reconstructed blocks that have passed through the filter unit 180. The stored reference picture can then be used for inter prediction.

The decoding apparatus 200 may be a decoder, a video decoding apparatus, or an image decoding apparatus.

Referring to FIG. 2, the decoding apparatus 200 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, and an adder 255. The filter unit 260 may include a reference picture buffer 270.

The decoding apparatus 200 may receive a bitstream output from the encoding apparatus 100. The decoding apparatus 200 may receive a bitstream stored in a computer readable recording medium, and may receive a bitstream streamed through a wired / wireless transmission medium.

The decoding apparatus 200 may perform intra mode and / or inter mode decoding on the bitstream. In addition, the decoding apparatus 200 may generate a reconstructed image or a decoded image through decoding, and output the generated reconstructed image or the decoded image.

For example, switching to the intra mode or the inter mode according to the prediction mode used for decoding may be made by a switch. When the prediction mode used for decoding is an intra mode, the switch may be switched to intra. When the prediction mode used for decoding is an inter mode, the switch may be switched to inter.

The decoding apparatus 200 may obtain a reconstructed residual block by decoding the input bitstream, and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate the reconstructed block to be decoded by adding the reconstructed residual block and the prediction block.

The entropy decoder 210 may generate symbols by performing entropy decoding on the bitstream based on a probability distribution on the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be similar to the entropy encoding method described above. For example, the entropy decoding method may be an inverse process of the above-described entropy encoding method.

The quantized coefficient may be inverse quantized by the inverse quantization unit 220. The inverse quantization unit 220 may generate inverse quantized coefficients by performing inverse quantization on the quantized coefficients. In addition, the inverse quantized coefficient may be inversely transformed by the inverse transformer 230. The inverse transform unit 230 may generate the reconstructed residual block by performing an inverse transform on the inverse quantized coefficients. As a result of inverse quantization and inverse transformation on the quantized coefficients, a reconstructed residual block may be generated. In this case, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients in generating the reconstructed residual block.

When the intra mode is used, the intra predictor 240 may generate the predictive block by performing spatial prediction using pixel values of the already decoded block around the target block.

The inter predictor 250 may include a motion compensator. Alternatively, the inter predictor 250 may be referred to as a motion compensator.

When the inter mode is used, the motion compensator may generate a prediction block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270.

When the motion vector has a non-integer value, the motion compensator may apply an interpolation filter to a portion of the reference image, and generate a prediction block using the reference image to which the interpolation filter is applied. The motion compensation unit may determine which of the skip mode, the merge mode, the AMVP mode, and the current picture reference mode is a motion compensation method used for the PU included in the CU based on the CU to perform motion compensation. According to the present invention, motion compensation may be performed.

The reconstructed residual block and prediction block may be added via adder 255. The adder 255 may generate the reconstructed block by adding the reconstructed residual block and the predictive block.

The reconstructed block may pass through the filter unit 260. The filter unit 260 may apply at least one of the deblocking filter, SAO, and ALF to the reconstructed block or the reconstructed picture.

The reconstructed block that has passed through the filter unit 260 may be stored in the reference picture buffer 270. The reconstructed block that has passed through the filter unit 260 may be part of the reference picture. In other words, the reference image may be an image composed of reconstructed blocks that have passed through the filter unit 260. The stored reference picture may then be used for inter prediction.

3 may schematically illustrate an example in which one unit is divided into a plurality of sub-units.

In order to efficiently divide an image, a coding unit (CU) may be used in encoding and decoding. A unit may be a term that collectively refers to 1) a block including image samples and 2) a syntax element. For example, "division of a unit" may mean "division of a block corresponding to a unit".

The CU may be used as a base unit of image encoding / decoding. In addition, the CU may be used as a unit to which one selected mode of intra mode and inter mode is applied in image encoding / decoding. In other words, in image encoding / decoding, it may be determined which mode of intra mode and inter mode is applied to each CU.

In addition, a CU may be a base unit in encoding / decoding of prediction, transform, quantization, inverse transform, inverse quantization, and transform coefficients.

Referring to FIG. 3, the image 300 may be sequentially divided in units of a largest coding unit (LCU), and the division structure of the image 300 may be determined according to the LCU. Here, the LCU may be used as the same meaning as a coding tree unit (CTU).

The division of the unit may mean division of a block corresponding to the unit. The block division information may include depth information regarding a depth of a unit. The depth information may indicate the number and / or degree of division of the unit. One unit may be divided into sub-units hierarchically with depth information based on a tree structure. Each divided subunit may have depth information. Depth information may be information indicating the size of a CU. Depth information may be stored for each CU. Each CU may have depth information.

The partition structure may mean a distribution of a CU in the LCU 310 for efficiently encoding an image. This distribution may be determined according to whether to divide one CU into a plurality of CUs. The number of partitioned CUs may be two or more positive integers, including 2, 3, 4, 8, 16, and the like. The horizontal size and the vertical size of the CU generated by the split may be smaller than the horizontal size and the vertical size of the CU before the split, depending on the number of CUs created by the split.

The partitioned CU may be recursively divided into a plurality of CUs in the same manner. By recursive partitioning, the size of at least one of the horizontal size and vertical size of the divided CU can be reduced compared to at least one of the horizontal size and vertical size of the CU before splitting.

Partitioning of a CU can be done recursively up to a predefined depth or a predefined size. For example, the depth of the LCU may be zero, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, the LCU may be a CU having a maximum coding unit size as described above, and the SCU may be a CU having a minimum coding unit size.

The division may begin from the LCU 310, and the depth of the CU may increase by one each time the division reduces the horizontal and / or vertical sizes of the CU.

For example, for each depth, a CU that is not divided may have a size of 2N × 2N. In addition, in the case of a partitioned CU, a CU of 2N × 2N size may be divided into four CUs having an N × N size. The size of N can be reduced by half for every 1 increase in depth.

Referring to FIG. 3, an LCU having a depth of 0 may be 64x64 pixels or 64x64 block. 0 may be the minimum depth. An SCU of depth 3 may be 8x8 pixels or 8x8 block. 3 may be the maximum depth. In this case, a CU of a 64x64 block, which is an LCU, may be represented by depth 0. A CU of a 32x32 block may be represented by depth 1. A CU of a 16x16 block may be represented by depth 2. A CU of an 8x8 block, which is an SCU, may be represented by depth 3.

Information on whether a CU is split may be expressed through split information of the CU. The split information may be 1 bit of information. All CUs except the SCU may include partition information. For example, the value of partition information of a CU that is not divided may be 0, and the value of partition information of a CU that is divided may be 1.

For example, when one CU is divided into four CUs, the horizontal size and vertical size of each CU of the four CUs generated by the split are each half the horizontal size and half the vertical size of the CU before splitting. Can be. When a 32x32 size CU is divided into four CUs, sizes of the divided four CUs may be 16x16. When one CU is divided into four CUs, it may be said that the CU is divided into quad-tree shapes.

For example, when one CU is divided into two CUs, the horizontal size or vertical size of each CU of the two CUs generated by the split is half the horizontal size or half the vertical size of the CU before splitting, respectively. Can be. When a 32x32 size CU is vertically divided into two CUs, sizes of the divided two CUs may be 16x32. When one CU is divided into two CUs, it can be said that the CU is divided into a binary-tree.

In addition to quad-tree splitting and binary-tree splitting, the LCU 310 may be applied.

A CU that is no longer split among CUs split from the LCU may be split into one or more prediction units (PUs).

The PU may be a base unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. PU may be divided into various types according to each mode. For example, the target block described above with reference to FIG. 1 and the target block described above with reference to FIG. 2 may be a PU.

In skip mode, there may be no partition in the CU. In the skip mode, 2N × 2N mode 410 having the same size of PU and CU without splitting may be supported.

In the inter mode, eight divided forms in a CU may be supported. For example, in the inter mode, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435, nLx2N mode 440, and nRx2N Mode 445 may be supported.

In the intra mode, 2Nx2N mode 410 and NxN mode 425, 2NxN mode and Nx2N may be supported.

In the 2Nx2N mode 410, a PU having a size of 2Nx2N may be encoded. A PU having a size of 2N × 2N may mean a PU having a size equal to the size of a CU. For example, a PU having a size of 2N × 2N may have a size of 64 × 64, 32 × 32, 16 × 16, or 8 × 8.

In the NxN mode 425, a PU having a size of NxN may be encoded.

For example, in intra prediction, when the size of the PU is 8x8, four divided PUs may be encoded. The size of the partitioned PU may be 4 × 4.

When the PU is encoded by the intra mode, the PU may be encoded using one intra prediction mode among the plurality of intra prediction modes. For example, High Efficiency Video Coding (HEVC) technology can provide 35 intra prediction modes, and the PU can be coded in one of the 35 intra prediction modes.

Which of the 2Nx2N mode 410 and NxN mode 425 is to be coded may be determined by the rate-distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2N × 2N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a 2N × 2N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding a 2N × 2N size PU among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

In addition, the encoding apparatus 100 may sequentially perform encoding operations on each PU of the PUs divided by N × N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a N × N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding of a PU of an N × N size among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

A transform unit (TU) may be a base unit used for a process of transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding in a CU. The TU may have a square shape or a rectangular shape.

Of the CUs partitioned from the LCU, a CU that is no longer split into CUs may be split into one or more TUs. In this case, the partition structure of the TU may be a quad-tree structure. For example, as shown in FIG. 5, one CU 510 may be divided one or more times according to the quad-tree structure. Through division, one CU 510 may be configured with TUs of various sizes.

In the encoding apparatus 100, a 64x64 coding tree unit (CTU) may be divided into a plurality of smaller CUs by a recursive quad-tree structure. One CU may be divided into four CUs having the same sizes. CUs may be recursively split, and each CU may have a quad tree structure.

The CU may have a depth. If a CU is split, the CUs created by splitting may have a depth increased by one from the depth of the split CU.

For example, the depth of the CU may have a value of 0 to 3. The size of the CU may be from 64x64 to 8x8 depending on the depth of the CU.

Through recursive partitioning for a CU, an optimal partitioning method can be selected that produces the smallest rate-distortion ratio.

6 is a diagram for explaining an embodiment of an intra prediction process.

Arrows outward from the center of the graph of FIG. 6 may indicate prediction directions of intra prediction modes. In addition, the number displayed near the arrow may represent an example of a mode value allocated to the intra prediction mode or the prediction direction of the intra prediction mode.

Intra encoding and / or decoding may be performed using reference samples of units around the target block. The surrounding block may be a surrounding rebuilt block. For example, intra coding and / or decoding may be performed using a value or a coding parameter of a reference sample included in a neighboring reconstructed block.

The encoding apparatus 100 and / or the decoding apparatus 200 may generate the prediction block by performing intra prediction on the target block based on the information of the sample in the target image. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may generate a prediction block for the target block by performing intra prediction based on information of a sample in the target image. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may perform directional prediction and / or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may mean a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, a PU, and a TU.

The unit of a prediction block may be the size of at least one of a CU, a PU, and a TU. The prediction block may have a square shape, having a size of 2N × 2N or a size of N × N. The size of NxN may include 4x4, 8x8, 16x16, 32x32 and 64x64.

Alternatively, the prediction block may be a rectangular block having MxN sizes such as 2x8, 4x8, 2x16, 4x16, and 8x16.

Intra prediction may be performed according to an intra prediction mode for a target block. The number of intra prediction modes that the target block may have may be a predetermined fixed value or may be a value determined differently according to the properties of the prediction block. For example, the attributes of the prediction block may include the size of the prediction block and the type of the prediction block.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction block. Or, for example, the number of intra prediction modes may be 3, 5, 9, 17, 34, 35, 36, or the like.

The intra prediction mode may be a non-directional mode or a directional mode. For example, the intra prediction mode may include two non-directional modes and 33 directional modes as shown in FIG. 6.

The non-directional mode may include a DC mode and a planar mode. For example, the mode value of the DC mode may be 1. The mode value of the planner mode may be zero.

The directional modes may be prediction modes with a specific direction or a specific angular. Among the plurality of intra prediction modes, a mode other than the DC mode and the planner mode may be a directional mode.

The intra prediction mode may be expressed by at least one of a mode number, a mode value, a mode number, and a mode angle. The number of intra prediction modes may be M pieces. M may be one or more. In other words, the intra prediction mode may be M pieces including the number of non-directional modes and the number of directional modes.

The number of intra prediction modes may be fixed to M regardless of the size of the block. Alternatively, the number of intra prediction modes may differ depending on the size of the block and / or the type of color component. For example, the number of prediction modes may vary depending on whether the color component is a luma signal or a chroma signal. For example, as the size of a block increases, the number of intra prediction modes may increase. Alternatively, the number of intra prediction modes of the luma component block may be greater than the number of intra prediction modes of the chrominance component block.

For example, in the vertical mode having a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample.

Even in the directional mode other than the above-described mode, the encoding apparatus 100 and the decoding apparatus 200 may perform intra prediction on the target unit using the reference sample according to the angle corresponding to the directional mode.

The intra prediction mode located on the right side of the vertical mode may be referred to as a vertical right mode. The intra prediction mode located at the bottom of the horizontal mode may be referred to as a horizontal-below mode. For example, in FIG. 6, intra prediction modes in which the mode value is one of 27, 28, 29, 30, 31, 32, 33, and 34 may be vertical right modes 613. Intra prediction modes with a mode value of one of 2, 3, 4, 5, 6, 7, 8, and 9 may be horizontal bottom modes 616.

The number of intra prediction modes described above and the mode value of each intra prediction modes may be exemplary only. The number of intra prediction modes described above and the mode value of each intra prediction modes may be defined differently according to an embodiment, implementation, and / or need.

A step of checking whether samples included in the reconstructed neighboring block to perform intra prediction on the target block can be used as a reference sample of the target block may be performed. If there are samples of the neighboring block that are not available as reference samples of the current block, the values generated by copying and / or interpolation using at least one sample value of the samples included in the restored neighboring block are It can be replaced with sample values of samples that are not available as reference samples. If the value generated by copying and / or interpolation is replaced with the sample value of the sample, the sample can be used as a reference sample of the target block.

In intra prediction, a filter may be applied to at least one of a reference sample or a prediction sample based on at least one of an intra prediction mode and a size of a target block.

When the intra prediction mode is the planner mode, in generating the prediction block of the target block, the upper reference sample of the target sample, the left reference sample of the target sample, and the right top reference sample of the target block, according to the position in the prediction block of the target sample, And a sample value of the prediction target sample using the weighted sum of the lower left reference samples of the target block.

When the intra prediction mode is the DC mode, in generating the prediction block of the target block, an average value of the top reference samples and the left reference samples of the target block may be used.

When the intra prediction mode is the directional mode, the prediction block may be generated using an upper reference sample, a left reference sample, a right upper reference sample, and / or a lower left reference sample.

Real unit interpolation may be performed to generate the above-described prediction sample.

The intra prediction mode of the target block may be predicted from the intra prediction of the neighboring blocks of the target block, and information used for the prediction may be entropy encoded / decoded.

For example, if the intra prediction modes of the target block and the neighboring block are the same, it may be signaled that the intra prediction modes of the target block and the neighboring block are the same using a predefined flag.

For example, an indicator indicating the same intra prediction mode as the intra prediction mode of the target block among the intra prediction modes of the plurality of neighboring blocks may be signaled.

If the intra prediction modes of the target block and the neighboring block are different from each other, the information of the intra prediction mode of the target block may be entropy encoded / decoded based on the intra prediction mode of the neighboring block.

7 shows the location of a reference sample used for intra prediction of a target block. Referring to FIG. 7, the reconstructed reference samples used for intra prediction of a target block include lower-left reference samples 731, left reference samples 733, and upper-above- left corner reference sample 735, upper reference samples 737, upper right-above reference samples 739, and the like.

For example, the left reference samples 733 may refer to a reconstructed reference pixel adjacent to the left side of the target block. The top reference samples 737 may refer to a reconstructed reference pixel adjacent to the top of the target block. The upper left corner reference sample 735 may refer to a reconstructed reference pixel located at the upper left corner of the target block. Also, the lower left reference samples 731 may refer to a reference sample located at the bottom of the left sample line among samples positioned on the same line as the left sample line composed of the left reference samples 733. The upper right reference samples 739 may refer to reference samples positioned to the right of the upper pixel line among samples positioned on the same line as the upper sample line formed of the upper reference samples 737.

When the size of the target block is N × N, the lower left reference samples 731, the left reference samples 733, the upper reference samples 737, and the upper right reference samples 739 may each be N pieces.

The prediction block may be generated through intra prediction on the target block. Generation of the predictive block may include determining a value of pixels of the predictive block. The size of the target block and the prediction block may be the same.

The reference sample used for intra prediction of the target block may vary according to the intra prediction mode of the target block. The direction of the intra prediction mode may indicate a dependency relationship between the reference samples and the pixels of the prediction block. For example, the value of the specified reference sample can be used as the value of the specified one or more pixels of the prediction block. In this case, the specified one or more specified pixels of the specified reference sample and prediction block may be samples and pixels designated by a straight line in the direction of the intra prediction mode. In other words, the value of the specified reference sample may be copied to the value of the pixel located in the reverse direction of the intra prediction mode. Alternatively, the pixel value of the prediction block may be a value of a reference sample located in the direction of the intra prediction mode based on the position of the pixel.

For example, when the intra prediction mode of the target block is a vertical mode having a mode value of 26, the upper reference samples 737 may be used for intra prediction. When the intra prediction mode is the vertical mode, the value of a pixel of the prediction block may be a value of a reference sample located vertically above the position of the pixel. Thus, the top reference samples 737 adjacent to the top of the target block can be used for intra prediction. Also, the values of the pixels of one row of the prediction block may be the same as the values of the top reference samples 737.

For example, when the mode value of the intra prediction mode of the target block is 18, at least some of the left reference samples 733, the upper left corner reference sample 735 and the at least some intra prediction of the top reference samples 737 are included. Can be used. When the mode value of the intra prediction mode is 18, the value of the pixel of the prediction block may be the value of the reference sample positioned diagonally on the upper left side with respect to the pixel.

The reference sample used to determine the pixel value of one pixel of the prediction block may be one, or may be two or more.

As described above, the pixel value of the pixel of the prediction block may be determined according to the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode. If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is an integer position, the value of one reference sample indicated by the integer position may be used to determine the pixel value of the pixel of the prediction block.

If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is not an integer position, an interpolated reference sample may be generated based on the two reference samples closest to the position of the reference sample. have. The value of the interpolated reference sample can be used to determine the pixel value of the pixel of the prediction block. In other words, when the position of the reference sample indicated by the position of the pixel of the prediction block and the direction of the intra prediction mode indicates between the two reference samples, an interpolated value is generated based on the values of the two samples. Can be.

The prediction block generated by the prediction may not be the same as the original target block. In other words, there may be a prediction error that is a difference between the target block and the prediction block, and the prediction error may exist between the pixels of the target block and the pixels of the prediction block.

Hereinafter, the terms "difference", "error" and "residual" may be used in the same sense and may be used interchangeably.

For example, in the case of directional intra prediction, the greater the distance between the pixel and the reference sample of the prediction block, the larger prediction error may occur. Discontinuity may occur between the prediction block and the neighboring block generated by such a prediction error.

Filtering on the prediction block may be used to reduce the prediction error. The filtering may be to adaptively apply a filter to a region that is considered to have a large prediction error in the prediction block. For example, an area considered to have a large prediction error may be a boundary of a prediction block. In addition, according to the intra prediction mode, an area considered to have a large prediction error among the prediction blocks may be different, and characteristics of the filter may be different.

8 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle illustrated in FIG. 8 may represent an image (or a picture). In addition, arrows in FIG. 8 may indicate prediction directions. That is, the image may be encoded and / or decoded according to the prediction direction.

Each picture may be classified into an I picture (Intra Picture), a P picture (Uni-prediction Picture), and a B picture (Bi-prediction Picture). Each picture may be encoded according to an encoding type of each picture.

If the target image to be encoded is an I picture, the target image may be encoded using data in the image itself without inter prediction referring to another image. For example, an I picture can only be encoded with intra prediction.

When the target image is a P picture, the target image may be encoded through inter prediction using only a reference picture existing in one direction. Here, the unidirectional may be forward or reverse.

When the target picture is a B picture, the target picture may be encoded through inter prediction using reference pictures existing in both directions or inter prediction using reference pictures existing in one of the forward and reverse directions. Here, the bidirectional can be forward and reverse.

P pictures and B pictures that are encoded and / or decoded using the reference picture may be regarded as an image using inter prediction.

In the following, inter prediction in inter mode according to an embodiment is described in detail.

Inter prediction may be performed using motion information.

In the inter mode, the encoding apparatus 100 may perform inter prediction and / or motion compensation on a target block. The decoding apparatus 200 may perform inter prediction and / or motion compensation corresponding to inter prediction and / or motion compensation in the encoding apparatus 100 with respect to the target block.

The motion information on the target block may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200. The motion information may be derived using motion information of the restored neighboring block, motion information of a collocated block (col block), and / or motion information of a block adjacent to the call block. The call block may be a block in a collocated picture (col picture). The position in the call picture of the call block may correspond to the position in the target image of the target block. The call picture may be one picture of at least one reference picture included in the reference picture list.

For example, the encoding apparatus 100 or the decoding apparatus 200 uses the motion information of the spatial candidate and / or the temporal candidate as the motion information of the target block to perform prediction and / or motion compensation. Can be done. The target block may mean a PU and / or a PU partition.

The spatial candidate may be a reconstructed block spatially adjacent to the target block.

The temporal candidate may be a reconstructed block corresponding to a target block in a collocated picture (col picture).

In inter prediction, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency and decoding efficiency by using motion information of spatial candidates and / or temporal candidates. The motion information of the spatial candidate may be referred to as spatial motion information. The motion information of the temporal candidate may be referred to as temporal motion information.

Hereinafter, the motion information of the spatial candidate may be motion information of the PU including the spatial candidate. The motion information of the temporal candidate may be motion information of the PU including the temporal candidate. The motion information of the candidate block may be motion information of the PU including the candidate block.

Inter prediction may be performed using a reference picture.

The reference picture may be at least one of a previous picture of the target picture or a subsequent picture of the target picture. The reference picture may mean an image used for prediction of the target block.

In inter prediction, an area within a reference picture can be specified by using a reference picture index (or refIdx) indicating a reference picture, a motion vector to be described later, and the like. Here, the specified region in the reference picture may represent a reference block.

Inter prediction may select a reference picture, and may select a reference block corresponding to the target block within the reference picture. In addition, inter prediction may generate a prediction block for a target block using the selected reference block.

The motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200.

The spatial candidate may be 1) present in the target picture, 2) already reconstructed through encoding and / or decoding, and 3) adjacent to the target block or located at the corner of the target block. The block located at the corner of the target block may be a block vertically adjacent to a neighboring block horizontally adjacent to the target block or a block horizontally adjacent to a neighboring block vertically adjacent to the target block. "Block located at the corner of the target block" may have the same meaning as "block adjacent to the corner of the target block". The "block located at the corner of the target block" may be included in the "block adjacent to the target block".

For example, a spatial candidate may be a reconstructed block located to the left of the target block, a reconstructed block located to the top of the target block, a reconstructed block located at the lower left corner of the target block, or a top right corner of the target block. It may be a reconstructed block or a reconstructed block located at the upper left corner of the target block.

Each of the encoding apparatus 100 and the decoding apparatus 200 may identify a block that exists at a position spatially corresponding to the target block in the col picture. The position of the target block in the target picture and the position of the identified block in the call picture may correspond to each other.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a coll block existing at a predetermined relative position with respect to the identified block as a temporal candidate. The predefined relative position may be a position inside and / or outside of the identified block.

For example, the call block may include a first call block and a second call block. When the coordinates of the identified block are (xP, yP) and the size of the identified block is (nPSW, nPSH), the first call block may be a block located at coordinates (xP + nPSW, yP + nPSH). The second call block may be a block located at coordinates (xP + (nPSW >> 1), yP + (nPSH >> 1)). The second call block can optionally be used if the first call block is unavailable.

The motion vector of the target block may be determined based on the motion vector of the call block. Each of the encoding apparatus 100 and the decoding apparatus 200 may scale a motion vector of a call block. The scaled motion vector of the call block can be used as the motion vector of the target block. In addition, the motion vector of the motion information of the temporal candidate stored in the list may be a scaled motion vector.

The ratio of the motion vector of the target block and the motion vector of the call block may be equal to the ratio of the first distance and the second distance. The first distance may be a distance between the reference picture and the target picture of the target block. The second distance may be a distance between the reference picture and the call picture of the call block.

The derivation method of the motion information may vary according to the inter prediction mode of the target block. For example, as the inter prediction mode applied for inter prediction, there may be an enhanced motion vector predictor (AMVP) mode, a merge mode and a skip mode, a current picture reference mode, and the like. have. The merge mode may be referred to as a motion merge mode. In the following, each of the modes is described in detail.

1) AMVP Mode

When the AMVP mode is used, the encoding apparatus 100 may search for a similar block around the target block. The encoding apparatus 100 may obtain the prediction block by performing prediction on the target block using the retrieved motion information of the similar block. The encoding apparatus 100 may encode a residual block that is a difference between the target block and the prediction block.

1-1) Preparation of predictive motion vector candidate list

When the AMVP mode is used as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a predicted motion vector candidate list using the motion vector of the spatial candidate, the motion vector of the temporal candidate, and the zero vector. have. The predictive motion vector candidate list may include one or more predictive motion vector candidates. At least one of a motion vector of a spatial candidate motion vector, a motion vector of a temporal candidate, and a zero vector may be determined and used as a predictive motion vector candidate.

In the following, the terms “predictive motion vector (candidate)” and “motion vector (candidate)” may be used in the same sense and may be used interchangeably.

The spatial motion candidate may include the reconstructed spatial neighboring block. In other words, the motion vector of the reconstructed neighboring block may be referred to as a spatial prediction motion vector candidate.

The temporal motion candidate may include a call block and a block adjacent to the call block. In other words, the motion vector of the call block or the motion vector of the block adjacent to the call block may be referred to as a temporal prediction motion vector candidate.

The zero vector may be a (0, 0) motion vector.

The predictive motion vector candidate may be a motion vector predictor for prediction of the motion vector. Also, in the encoding apparatus 100, the predicted motion vector candidate may be a motion vector initial search position.

1-2) Searching for Motion Vectors Using Predictive Motion Vector Candidate List

The encoding apparatus 100 may determine a motion vector to be used for encoding a target block within a search range using the predictive motion vector candidate list. Also, the encoding apparatus 100 may determine a prediction motion vector candidate to be used as a prediction motion vector of the target block among the prediction motion vector candidates of the prediction motion vector candidate list.

The motion vector to be used for encoding the target block may be a motion vector that can be encoded at a minimum cost.

In addition, the encoding apparatus 100 may determine whether to use the AMVP mode in encoding the target block.

1-3) Transmission of Inter Prediction Information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The decoding apparatus 200 may perform inter prediction on the target block by using inter prediction information of the bitstream.

The inter prediction information includes 1) mode information indicating whether the AMVP mode is used, 2) a predicted motion vector index, 3) a motion vector difference (MVD), 4) a reference direction, and 5) a reference picture index. can do.

In addition, the inter prediction information may include a residual signal.

When the mode information indicates that the AMVP mode is used, the decoding apparatus 200 may obtain a predicted motion vector index, a motion vector difference, a reference direction, and a reference picture index from the bitstream through entropy decoding.

The prediction motion vector index may indicate a prediction motion vector candidate used for prediction of a target block among prediction motion vector candidates included in the prediction motion vector candidate list.

1-4) Inter Prediction in AMVP Mode Using Inter Prediction Information

The decoding apparatus 200 may derive the predictive motion vector candidate using the predictive motion vector candidate list, and determine the motion information of the target block based on the derived predicted motion vector candidate.

The decoding apparatus 200 may determine the motion vector candidate for the target block from among the prediction motion vector candidates included in the prediction motion vector candidate list using the prediction motion vector index. The decoding apparatus 200 may select a prediction motion vector candidate indicated by the prediction motion vector index from among prediction motion vector candidates included in the prediction motion vector candidate list as the prediction motion vector of the target block.

The motion vector to be actually used for inter prediction of the target block may not match the prediction motion vector. The motion vector to be actually used for inter prediction of the target block and MVD may be used to indicate the difference between the predicted motion vector. The encoding apparatus 100 may derive a predictive motion vector similar to the motion vector actually used for inter prediction of the target block in order to use the MVD of the smallest possible size.

The MVD may be a difference between the motion vector and the predicted motion vector of the target block. The encoding apparatus 100 may calculate an MVD and entropy encode the MVD.

The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive the motion vector of the target block by adding the decoded MVD and the predictive motion vector. In other words, the motion vector of the target block derived from the decoding apparatus 200 may be the sum of the entropy decoded MVD and the motion vector candidate.

The reference direction may point to the reference picture list used for prediction of the target block. For example, the reference direction may point to one of the reference picture list L0 and the reference picture list L1.

The reference direction only points to the reference picture list used for prediction of the target block, but may not indicate that the directions of the reference pictures are limited in the forward direction or the backward direction. In other words, each of the reference picture list L0 and the reference picture list L1 may include pictures in the forward and / or reverse direction.

That the reference direction is uni-direction may mean that one reference picture list is used. The bi-direction of the reference direction may mean that two reference picture lists are used. That is to say, the reference direction may indicate that only the reference picture list L0 is used, that only the reference picture list L1 is used and one of the two reference picture lists.

The reference picture index may indicate a reference picture used for prediction of a target block among reference pictures of the reference picture list. The reference picture index may be entropy encoded by the encoding apparatus 100. The entropy coded reference picture index may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

When two reference picture lists are used for prediction of the target block. One reference picture index and one motion vector may be used for each reference picture list. In addition, when two reference picture lists are used for prediction of the target block, two prediction blocks may be specified for the target block. For example, the (final) prediction block of the target block may be generated through an average or weighted-sum of two prediction blocks for the target block.

The motion vector of the target block may be derived by the prediction motion vector index, the MVD, the reference direction, and the reference picture index.

The decoding apparatus 200 may generate a prediction block for the target block based on the derived motion vector and the reference picture index. For example, the prediction block may be a reference block indicated by the derived motion vector in the reference picture indicated by the reference picture index.

By encoding the predicted motion vector index and the MVD without encoding the motion vector itself of the target block, the amount of bits transmitted from the encoding device 100 to the decoding device 200 may be reduced, and encoding efficiency may be improved.

The motion information of the neighboring blocks reconstructed with respect to the target block may be used. In a particular inter prediction mode, the encoding apparatus 100 may not separately encode motion information about the target block. The motion information of the target block is not encoded, and other information that can derive the motion information of the target block through the motion information of the reconstructed neighboring block may be encoded instead. As other information is encoded instead, the amount of bits transmitted to the decoding apparatus 200 may be reduced, and encoding efficiency may be improved.

For example, the inter prediction mode in which the motion information of the target block is not directly encoded may include a skip mode and / or a merge mode. In this case, the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and / or an index indicating which motion information of which unit among the reconstructed neighboring units is used as the motion information of the target unit.

2) merge mode

Merge is a method of deriving the motion information of the target block. Merge may mean merging of motions for a plurality of blocks. Merge may mean applying motion information of one block to other blocks. In other words, the merge mode may mean a mode in which the motion information of the target block is derived from the motion information of the neighboring block.

When the merge mode is used, the encoding apparatus 100 may predict the motion information of the target block by using the motion information of the spatial candidate and / or the motion information of the temporal candidate. The spatial candidate may include a reconstructed spatial neighboring block spatially adjacent to the target block. Spatially adjacent blocks may include left adjacent blocks and top adjacent blocks. Temporal candidates may include call blocks. The terms “spatial candidate” and “spatial merge candidate” may be used interchangeably and may be used interchangeably. The terms "temporal candidate" and "temporary merge candidate" may be used interchangeably and may be used interchangeably.

The encoding apparatus 100 may obtain a prediction block through prediction. The encoding apparatus 100 may encode a residual block that is a difference between a target block and a prediction block.

2-1) Creating a merge candidate list

When the merge mode is used, each of the encoding apparatus 100 and the decoding apparatus 200 may generate the merge candidate list using the motion information of the spatial candidate and / or the motion information of the temporal candidate. The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction may be unidirectional or bidirectional.

The merge candidate list may include merge candidates. The merge candidate may be motion information. In other words, the merge candidate list may be a list in which motion information is stored.

The merge candidates may be motion information such as temporal candidates and / or spatial candidates. In addition, the merge candidate list may include a new merge candidate generated by a combination of merge candidates already present in the merge candidate list. In other words, the merge candidate list may include new motion information generated by a combination of motion information already present in the merge candidate list.

In addition, the merge candidate list may include motion information of the zero vector. The zero vector may be referred to as a zero merge candidate.

In other words, the motion information in the merge candidate list may include: 1) motion information of a spatial candidate, 2) motion information of a temporal candidate, 3) motion information generated by a combination of motion information already present in the merge candidate list, and 4) zero vector. It may be at least one of.

The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction may be referred to as an inter prediction indicator. The reference direction may be unidirectional or bidirectional. The unidirectional reference direction may indicate L0 prediction or L1 prediction.

The merge candidate list may be generated before prediction by the merge mode is performed.

The number of merge candidates in the merge candidate list may be predefined. The encoding apparatus 100 and the decoding apparatus 200 may add the merge candidates to the merge candidate list according to a predefined method and a predefined rank so that the merge candidate list has a predetermined number of merge candidates. The merge candidate list of the encoding apparatus 100 and the merge candidate list of the decoding apparatus 200 may be identical through the predefined scheme and the predefined ranking.

Merge may be applied in a CU unit or a PU unit. When merging is performed in a CU unit or a PU unit, the encoding apparatus 100 may transmit a bitstream including predefined information to the decoding apparatus 200. For example, the predefined information may include 1) information indicating whether or not to perform merge for each block partition, and 2) any block among blocks that are spatial candidates and / or temporal candidates for the target block. It may include information about whether it is.

2-2) Searching for Motion Vectors Using Merge Candidate Lists

The encoding apparatus 100 may determine a merge candidate to be used for encoding the target block. For example, the encoding apparatus 100 may perform predictions on the target block by using merge candidates of the merge candidate list and generate residual blocks for the merge candidates. The encoding apparatus 100 may use, for encoding the target block, a merge candidate that requires a minimum cost in prediction and encoding of the residual block.

In addition, the encoding apparatus 100 may determine whether to use the merge mode in encoding the target block.

2-3) Inter prediction information transmission

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The encoding apparatus 100 may generate entropy encoded inter prediction information by performing entropy encoding on the inter prediction information, and may transmit a bitstream including the entropy encoded inter prediction information to the decoding apparatus 200. Entropy encoded inter prediction information may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through the bitstream.

The decoding apparatus 200 may perform inter prediction on the target block by using inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether to use the merge mode and 2) the merge index.

In addition, the inter prediction information may include a residual signal.

The decoding apparatus 200 may obtain the merge index from the bitstream only when the mode information indicates that the merge mode is used.

The mode information may be a merge flag. The unit of the mode information may be a block. The information on the block may include mode information, and the mode information may indicate whether a merge mode is applied to the block.

The merge index may indicate a merge candidate used for prediction of the target block among merge candidates included in the merge candidate list. Alternatively, the merge index may indicate which one of neighboring blocks spatially or temporally adjacent to the target block is performed.

2-4) Inter prediction in merge mode using inter prediction information

The decoding apparatus 200 may perform prediction on the target block by using the merge candidate indicated by the merge index among the merge candidates included in the merge candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the merge index, the reference picture index, and the reference direction.

3) Skip Mode

The skip mode may be a mode in which the motion information of the spatial candidate or the motion information of the temporal candidate is applied to the target block as it is. In addition, the skip mode may be a mode that does not use the residual signal. In other words, when the skip mode is used, the reconstructed block may be a prediction block.

The difference between the merge mode and the skip mode may be whether to transmit or use the residual signal. In other words, the skip mode may be similar to the merge mode except that no residual signal is transmitted or used.

When the skip mode is used, the encoding apparatus 100 transmits, to the decoding apparatus 200, information indicating which of the blocks that are spatial candidates or temporal candidates is used as the motion information of the target block. Can transmit The encoding apparatus 100 may generate entropy encoded information by performing entropy encoding on the information, and may signal the entropy encoded information to the decoding apparatus 200 through a bitstream.

In addition, when the skip mode is used, the encoding apparatus 100 may not transmit other syntax element information such as MVD to the decoding apparatus 200. For example, when used with a skip mode, the encoding apparatus 100 may not signal syntax elements regarding at least one of an MVC, a coded block flag, and a transform coefficient level to the decoding apparatus 200.

3-1) Creation of merge candidate list

Skip mode can also use the merge candidate list. In other words, the merge candidate list can be used in both merge mode and skip mode. In this regard, the merge candidate list may be named "skip candidate list" or "merge / skip candidate list."

Alternatively, the skip mode may use a separate candidate list different from the merge mode. In this case, in the following description, the merge candidate list and the merge candidate may be replaced with the skip candidate list and the skip candidate, respectively.

The merge candidate list may be generated before the prediction by the skip mode is performed.

3-2) Motion Vector Search Using Merge Candidate List

The encoding apparatus 100 may determine a merge candidate to be used for encoding the target block. For example, the encoding apparatus 100 may perform predictions on the target block by using merge candidates of the merge candidate list. The encoding apparatus 100 may use a merge candidate that requires a minimum cost in prediction for encoding a target block.

In addition, the encoding apparatus 100 may determine whether to use the skip mode in encoding the target block.

3-3) Inter prediction information transmission

The inter prediction information may include 1) mode information indicating whether to use a skip mode and 2) a skip index.

The skip index may be the same as the merge index described above.

When the skip mode is used, the target block may be encoded without a residual signal. The inter prediction information may not include the residual signal. Or, the bitstream may not include the residual signal.

The decoding apparatus 200 may obtain the skip index from the bitstream only when the mode information indicates that the skip mode is used. As described above, the merge index and the skip index may be the same. The decoding apparatus 200 may obtain the skip index from the bitstream only when the mode information indicates that the merge mode or the skip mode is used.

The skip index may indicate a merge candidate used for prediction of the target block among merge candidates included in the merge candidate list.

3-4) Inter Prediction in Skip Mode Using Inter Prediction Information

The decoding apparatus 200 may perform prediction on the target block by using the merge candidate indicated by the skip index among the merge candidates included in the merge candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the skip index, the reference picture index, and the reference direction.

4) Current picture reference mode

The current picture reference mode may mean a prediction mode that uses a pre-restored region in the current picture to which the target block belongs.

A vector can be defined to specify the pre-restored region. Whether the target block is encoded in the current picture reference mode may be determined using the reference picture index of the target block.

A flag or index indicating whether the target block is a block encoded in the current picture reference mode may be signaled from the encoding device 100 to the decoding device 200. Alternatively, whether the target block is a block encoded in the current picture reference mode may be inferred through the reference picture index of the target block.

When the target block is encoded in the current picture reference mode, the current picture may be added at a fixed position or any position within the reference picture list for the target block.

For example, the fixed position may be the position at which the reference picture index is zero or the last position.

When the current picture is added at an arbitrary position in the reference picture list, a separate reference picture index indicating this arbitrary position may be signaled from the encoding apparatus 100 to the decoding apparatus 200.

In the above-described AMVP mode, merge mode, and skip mode, the motion information to be used for prediction of the target block among the motion information in the list may be specified through an index to the list.

In order to improve encoding efficiency, the encoding apparatus 100 may signal only an index of an element causing a minimum cost in inter prediction of a target block among elements of a list. The encoding apparatus 100 may encode the index and may signal the encoded index.

Therefore, the aforementioned lists (that is, the prediction motion vector candidate list and the merge candidate list) may be derived in the same manner based on the same data in the encoding apparatus 100 and the decoding apparatus 200. Here, the same data may include the reconstructed picture and the reconstructed block. Also, to specify elements by index, the order of the elements in the list may have to be constant.

9 illustrates spatial candidates according to an example.

In FIG. 9, the positions of the spatial candidates are shown.

The large block in the middle may represent the target block. Five small blocks may represent spatial candidates.

The coordinates of the target block may be (xP, yP), and the size of the target block may be (nPSW, nPSH).

The spatial candidate A ₀ may be a block adjacent to the lower left corner of the target block. A ₀ may be a block occupying a pixel of coordinates (xP − 1, yP + nPSH + 1).

The spatial candidate A ₁ may be a block adjacent to the left side of the target block. A ₁ may be the lowest block among blocks adjacent to the left side of the target block. Alternatively, A ₁ may be a block adjacent to the top of A ₀ . A ₁ may be a block occupying a pixel of coordinates (xP-1, yP + nPSH).

The spatial candidate B ₀ may be a block adjacent to the upper right corner of the target block. B ₀ may be a block occupying a pixel of coordinates (xP + nPSW + 1, yP-1).

The spatial candidate B ₁ may be a block adjacent to the top of the target block. B ₁ may be the rightmost block among blocks adjacent to the top of the target block. Alternatively, B ₁ may be a block adjacent to the left side of B ₀ . B ₁ may be a block occupying a pixel of coordinates (xP + nPSW, yP-1).

The spatial candidate B ₂ may be a block adjacent to the upper left corner of the target block. B ₂ may be a block occupying a pixel of coordinates (xP-1, yP-1).

Judgment of the availability of spatial and temporal candidates

In order to include the motion information of the spatial candidate or the motion information of the temporal candidate in the list, it is determined whether the motion information of the spatial candidate or the motion information of the temporal candidate is available.

In the following, the candidate block may include a spatial candidate and a temporal candidate.

For example, the above determination may be made by sequentially applying steps 1) to 4) below.

Step 1) If the PU including the candidate block is outside the boundary of the picture, the availability of the candidate block may be set to false. "Availability is set to false" may mean the same as "set to unavailable".

Step 2) If the PU containing the candidate block is outside the boundary of the slice, the availability of the candidate block may be set to false. If the target block and the candidate block are located in different slices, the availability of the candidate block may be set to false.

Step 3) If the PU containing the candidate block is outside the boundary of the tile, the availability of the candidate block may be set to false. If the target block and the candidate block are located in different tiles, the availability of the candidate block may be set to false.

Step 4) If the prediction mode of the PU including the candidate block is an intra prediction mode, the availability of the candidate block may be set to false. If the PU including the candidate block does not use inter prediction, the availability of the candidate block may be set to false.

As shown in FIG. 10, in adding motion information of spatial candidates to a merge list, an order of A ₁ , B ₁ , B ₀ , A _0, and B ₂ may be used. That is, motion information of available spatial candidates may be added to the merge list in the order of A ₁ , B ₁ , B ₀ , A _0, and B ₂ .

Derivation method of merge list in merge mode and skip mode

As described above, the maximum number of merge candidates in the merge list may be set. The maximum number set is indicated by N. The set number may be transmitted from the encoding apparatus 100 to the decoding apparatus 200. The slice header of the slice may include N. In other words, the maximum number of merge candidates of the merge list for the target block of the slice may be set by the slice header. For example, by default, the value of N may be five.

The motion information (ie, merge candidate) may be added to the merge list in the order of steps 1) to 4) below.

Step 1) Available spatial candidates among the spatial candidates may be added to the merge list. The motion information of the available spatial candidates may be added to the merge list in the order shown in FIG. 10. In this case, when the motion information of the available spatial candidates overlaps with other motion information already existing in the merge list, the motion information may not be added to the merge list. Checking whether it overlaps with other motion information present in the list may be abbreviated as "redundancy check".

The added motion information may be up to N pieces.

Step 2) If the number of motion information in the merge list is smaller than N and a temporal candidate is available, the motion information of the temporal candidate may be added to the merge list. At this time, if the motion information of the available temporal candidate overlaps with other motion information already existing in the merge list, the motion information may not be added to the merge list.

Step 3) If the number of motion information in the merge list is less than N and the type of the target slice is "B", the combined motion information generated by the combined bi-prediction is added to the merge list. Can be.

The target slice may be a slice including the target block.

The combined motion information may be a combination of L0 motion information and L1 motion information. The L0 motion information may be motion information referring only to the reference picture list L0. The L1 motion information may be motion information referring only to the reference picture list L1.

Within the merge list, the L0 motion information may be one or more. Also, within the merge list, there may be one or more L1 motion information.

The combined motion information may be one or more. Which L0 motion information and which L1 motion information among one or more L0 motion information and one or more L1 motion information are used in generating the combined motion information may be defined. One or more combined motion information may be generated in a predefined order by combined bidirectional prediction using a pair of different motion information in the merge list. One of the pairs of different motion information may be L0 motion information and the other may be L1 motion information.

For example, the combined motion information added first may be a combination of L0 motion information having a merge index of 0 and L1 motion information having a merge index of 1. If the motion information having the merge index of 0 is not the L0 motion information or the motion information having the merge index of 1 is not the L1 motion information, the combined motion information may not be generated and added. Next, the additional motion information may be a combination of L0 motion information having a merge index of 1 and L1 motion information having a merge index of 0. The following specific combinations may follow other combinations in the field of encoding / decoding of video.

In this case, when the combined motion information is overlapped with other motion information already existing in the merge list, the combined motion information may not be added to the merge list.

Step 4) If the number of motion information in the merge list is smaller than N, zero vector motion information may be added to the merge list.

The zero vector motion information may be motion information in which the motion vector is a zero vector.

The zero vector motion information may be one or more. The reference picture indices of the one or more zero vector motion information may be different from each other. For example, the value of the reference picture index of the first zero vector motion information may be zero. The value of the reference picture index of the second zero vector motion information may be one.

The number of zero vector motion information may be equal to the number of reference pictures in the reference picture list.

The reference direction of the zero vector motion information may be bidirectional. Both motion vectors may be zero vectors. The number of zero vector motion information may be smaller than the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1. Alternatively, when the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1 are different from each other, a unidirectional reference direction may be used for the reference picture index that can be applied to only one reference picture list.

The encoding apparatus 100 and / or the decoding apparatus 200 may sequentially add zero vector motion information to the merge list while changing the reference picture index.

When the zero vector motion information overlaps with other motion information already existing in the merge list, the zero vector motion information may not be added to the merge list.

The order of steps 1) to 4) described above is merely exemplary, and the order between the steps may be interchanged. In addition, some of the steps may be omitted depending on predefined conditions.

Derivation Method of Predictive Motion Vector Candidate List in AMVP Mode

The maximum number of predicted motion vector candidates in the predicted motion vector candidate list may be predefined. The predefined maximum number is denoted by N. For example, the predefined maximum number may be two.

The motion information (ie, the predicted motion vector candidate) may be added to the predicted motion vector candidate list in the order of steps 1) to 3) below.

Step 1) Available spatial candidates of the spatial candidates may be added to the predicted motion vector candidate list. Spatial candidates may include a first spatial candidate and a second spatial candidate.

The first spatial candidate may be one of A ₀ , A ₁ , scaled A _0, and scaled A ₁ . The second spatial candidate may be one of B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B _1, and scaled B ₂ .

The motion information of the available spatial candidates may be added to the predicted motion vector candidate list in the order of the first spatial candidate and the second spatial candidate. At this time, if the motion information of the available spatial candidates overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information may not be added to the predicted motion vector candidate list. In other words, when the value of N is 2, if the motion information of the second spatial candidate is the same as the motion information of the first spatial candidate, the motion information of the second spatial candidate may not be added to the predicted motion vector candidate list.

The added motion information may be up to N pieces.

Step 2) If the number of motion information in the predicted motion vector candidate list is smaller than N and a temporal candidate is available, motion information of the temporal candidate may be added to the predicted motion vector candidate list. At this time, if the motion information of the available temporal candidate overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information may not be added to the predicted motion vector candidate list.

Step 3) If the number of motion information in the predicted motion vector candidate list is smaller than N, zero vector motion information may be added to the predicted motion vector candidate list.

The zero vector motion information may be one or more. The reference picture indices of the one or more zero vector motion information may be different from each other.

The encoding apparatus 100 and / or the decoding apparatus 200 may sequentially add zero vector motion information to the predicted motion vector candidate list while changing the reference picture index.

When zero vector motion information overlaps with other motion information already existing in the predicted motion vector candidate list, the zero vector motion information may not be added to the predicted motion vector candidate list.

The description of the zero vector motion information described above with respect to the merge list may also be applied to the zero vector motion information. Duplicate explanations are omitted.

The order of steps 1) to 3) described above is merely illustrative, and the order between the steps may be interchanged. In addition, some of the steps may be omitted depending on predefined conditions.

11 illustrates a process of transform and quantization according to an example.

As illustrated in FIG. 11, a quantized level may be generated by performing a transform and / or quantization process on the residual signal.

The residual signal may be generated by the difference between the original block and the prediction block. Here, the prediction block may be a block generated by intra prediction or inter prediction.

The transform may include at least one of a primary transform and a secondary transform. The transform coefficients may be generated by performing the primary transform on the residual signal, and the secondary transform coefficients may be generated by performing the secondary transform on the transform coefficients.

The primary transform may be performed using at least one of a plurality of pre-defined transform methods. For example, the plurality of pre-defined transformation methods are based on a discrete cosine transform (DCT), a discrete sine transform (DST), and a karhunen-Lo ㅸ ve transform (KLT). Conversion and the like.

A secondary transform may be performed on the transform coefficients generated by performing the primary transform.

The transform method (s) applied to the primary transform and / or the secondary transform may be determined according to at least one of the coding parameters for the target block and / or the neighboring block. Alternatively, transformation information indicating a transformation method may be signaled from the encoding apparatus 100 to the decoding apparatus 200.

Quantized levels can be generated by performing quantization on the residual signal or the result generated by performing the primary and / or secondary transform.

The quantized level may be scanned according to at least one of an upper right diagonal scan, a vertical scan, and a horizontal scan, according to at least one of an intra prediction mode, a block size, and a block type.

For example, the coefficients can be changed to a one-dimensional vector form by scanning the coefficients of the block using up-right diagonal scanning. Alternatively, a vertical scan that scans two-dimensional block shape coefficients in a column direction or a horizontal scan that scans two-dimensional block shape coefficients in a row direction may be used instead of the upper right diagonal scan depending on the size of the transform block and / or the intra prediction mode. have.

The scanned quantization level may be entropy coded and the bitstream may include entropy coded quantization level.

The decoding apparatus 200 may generate a quantized level through entropy decoding on the bitstream. The quantized levels can be aligned in the form of two-dimensional blocks through inverse scanning. In this case, at least one of the upper right diagonal scan, the vertical scan, and the horizontal scan may be performed as a method of reverse scanning.

Inverse quantization may be performed at the quantized level. For the result generated by performing inverse quantization, the second inverse transform may be performed depending on whether or not the second inverse transform is performed. Also, with respect to a result generated by performing the second inverse transform, the first inverse transform may be performed depending on whether the first inverse transform is performed. The reconstructed residual signal may be generated by performing the first inverse transform on the result generated by the second inverse transform.

12 is a structural diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1200 may correspond to the encoding apparatus 100 described above.

The encoding apparatus 1200 may include a processor 1210, a memory 1230, a user interface (UI) input device 1250, a UI output device 1260, and a storage that communicate with each other through a bus 1290. 1240. In addition, the encoding apparatus 1200 may further include a communication unit 1220 connected to the network 1299.

The processor 1210 may be a semiconductor device that executes processing instructions stored in the central processing unit (CPU), the memory 1230, or the storage 1240. The processor 1210 may be at least one hardware processor.

The processor 1210 may generate and process a signal, data, or information input to the encoding apparatus 1200, output from the encoding apparatus 1200, or used in the encoding apparatus 1200. Inspection, comparison, and judgment related to data or information can be performed. In other words, in an embodiment, generation and processing of data or information, and inspection, comparison, and determination related to the data or information may be performed by the processor 1210.

The processor 1210 may include an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, an entropy encoder 150, and inverse quantization. The unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190 may be included.

The inter predictor 110, the intra predictor 120, the switch 115, the subtractor 125, the transformer 130, the quantizer 140, the entropy encoder 150, the inverse quantizer 160, At least some of the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may be program modules, and may communicate with an external device or system. The program modules may be included in the encoding apparatus 1200 in the form of an operating system, an application program module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the encoding apparatus 1200.

Program modules perform routines or subroutines, programs, objects, components, and data to perform functions or operations, or to implement abstract data types, according to one embodiment. Data structures and the like, but is not limited thereto.

The program modules may be composed of instructions or codes performed by at least one processor of the encoding apparatus 1200.

The processor 1210 may include an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, an entropy encoder 150, and inverse quantization. Instructions or codes of the unit 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may be executed.

The storage unit can represent memory 1230 and / or storage 1240. The memory 1230 and the storage 1240 may be various types of volatile or nonvolatile storage media. For example, the memory 1230 may include at least one of a ROM 1231 and a RAM 1232.

The storage unit may store data or information used for the operation of the encoding apparatus 1200. In an embodiment, data or information included in the encoding apparatus 1200 may be stored in the storage.

For example, the storage unit may store a picture, a block, a list, motion information, inter prediction information, a bitstream, and the like.

The encoding apparatus 1200 may be implemented in a computer system including a recording medium that may be read by a computer.

The recording medium may store at least one module required for the encoding apparatus 1200 to operate. The memory 1230 may store at least one module, and the at least one module may be configured to be executed by the processor 1210.

Functions related to communication of data or information of the encoding apparatus 1200 may be performed through the communication unit 1220.

For example, the communication unit 1220 may transmit the bitstream to the decoding device 1300 to be described later.

13 is a structural diagram of a decoding apparatus according to an embodiment.

The decoding device 1300 may correspond to the decoding device 200 described above.

The decoding apparatus 1300 communicates with each other via a bus 1390 (1310), a memory 1330, a user interface (UI) input device 1350, a UI output device 1360 and storage ( 1340). In addition, the decryption apparatus 1300 may further include a communication unit 1320 connected to the network 1399.

The processor 1310 may be a semiconductor device that executes processing instructions stored in the central processing unit (CPU), the memory 1330, or the storage 1340. The processor 1310 may be at least one hardware processor.

The processor 1310 may generate and process a signal, data, or information input to the decoding apparatus 1300, output from the decoding apparatus 1300, or used in the decoding apparatus 1300. Inspection, comparison, and judgment related to data or information can be performed. In other words, in an embodiment, generation and processing of data or information, and inspection, comparison, and determination related to the data or information may be performed by the processor 1310.

The processor 1310 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, an adder 255, a filter 260, and the like. The reference picture buffer 270 may be included.

Entropy decoder 210, inverse quantizer 220, inverse transformer 230, intra predictor 240, inter predictor 250, adder 255, filter 260, and reference picture buffer 270. At least some) may be program modules, and may communicate with an external device or system. The program modules may be included in the decryption apparatus 1300 in the form of an operating system, an application program module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the decryption apparatus 1300.

The program modules may be composed of instructions or codes performed by at least one processor of the decoding apparatus 1300.

The processor 1310 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, an adder 255, a filter 260, and the like. Instructions or codes of the reference picture buffer 270 may be executed.

The storage unit may represent the memory 1330 and / or the storage 1340. The memory 1330 and the storage 1340 may be various types of volatile or nonvolatile storage media. For example, the memory 1330 may include at least one of a ROM 1331 and a RAM 1332.

The storage unit may store data or information used for the operation of the decoding apparatus 1300. In an embodiment, data or information included in the decryption apparatus 1300 may be stored in the storage.

The decryption apparatus 1300 may be implemented in a computer system including a recording medium that may be read by a computer.

The recording medium may store at least one module required for the decoding apparatus 1300 to operate. The memory 1330 may store at least one module, and the at least one module may be configured to be executed by the processor 1310.

Functions related to communication of data or information of the decoding apparatus 1300 may be performed through the communication unit 1320.

For example, the communication unit 1320 may receive a bitstream from the encoding apparatus 1200.

14 is a flowchart of a prediction method, according to an exemplary embodiment.

The prediction method may be performed by the encoding apparatus 1200 and / or the decoding apparatus 1300.

For example, the encoding apparatus 1200 may perform the prediction method of the embodiment to compare the efficiencies of the plurality of prediction methods for the target block, and the prediction method of the embodiment to generate a reconstructed block for the target block. Can be performed.

For example, the decoding apparatus 1300 may perform the prediction method of the embodiment to generate a reconstructed block for the target block.

Hereinafter, the processing unit may correspond to the processing unit 1210 of the encoding apparatus 1200 and / or the processing unit 1310 of the decoding apparatus 1300.

In operation 1410, the processor may select a prediction mode to be used for prediction of the target block. The selected prediction mode may be one or more.

In one embodiment, when there are a plurality of selected prediction modes, prediction mode determination information for determining a prediction mode to be used for prediction of a target block among the plurality of selected prediction modes may be used. In encoding of the target block, when there are a plurality of selected prediction modes, the processor may generate prediction mode determination information indicating a prediction mode to be used for prediction of the target block among the plurality of selected prediction modes.

In one embodiment, the selected prediction mode may be an inter prediction mode. The prediction mode may be a specified inter prediction method among a plurality of inter prediction methods.

In one embodiment, the selected prediction mode may be an intra prediction mode. The prediction mode may be a specified intra prediction method of the plurality of intra prediction methods.

An example of step 1410 is described in detail below with reference to FIG. 15.

In operation 1420, the processor may perform prediction of the target block based on the selected prediction mode.

In one embodiment, the prediction may be intra prediction or inter prediction.

In decoding the target block, when there are a plurality of selected prediction modes, the processor may determine the prediction mode to be used for prediction of the target block among the plurality of selected prediction modes using the prediction mode determination information.

An example of step 1420 is described in detail below with reference to FIGS. 20 and 21.

The prediction mode selection method of the embodiment may correspond to step 1410 described above. Step 1410 may include

steps

1510 and 1520.

In operation 1410, the processor may select one or more prediction modes from among a plurality of prediction modes in performing inter prediction on the target block.

The plurality of prediction modes may be referred to as a plurality of available prediction modes available in the encoding apparatus 1200 and / or the decoding apparatus 1300. The selected one or more prediction modes may be named one or more prediction mode candidates. The selected one or more prediction modes may be one or more prediction mode candidates that may be used for encoding and / or decoding of the target block.

In one embodiment, the plurality of prediction modes include AMVP mode, merge mode, skip mode, Advanced Temporal Motion Vector Prediction (ATMVP) mode, Spatial-Temporal Motion Vector Predictor (STMVP). ), Pattern Matched Motion Vector Derivation (PMMVD) template matching mode, bilateral matching mode, affine mode, and bi-directional optical flow flow (BIO) mode, decoder-side motion vector refinement (DMVR) mode, and local illumination compensation (LIC) mode.

In one embodiment, the affine mode may include an affine inter mode and an affine merge mode.

In one embodiment, the merge or AMVP candidate list may be constructed based on the motion information of the first prediction mode. For example, a candidate having a motion vector similar or identical to the motion vector of the first prediction mode may be excluded from the merge or AMVP candidate list, or the order of these candidates may be changed. The first prediction mode may be at least one of an AMVP mode, a merge mode, a skip mode, a template matching mode, a bilateral matching mode, and an affine merge mode.

In one embodiment, the plurality of prediction modes may include the other modes described above.

The processor may select one or more prediction modes from among the plurality of prediction modes by using at least one of a method of selecting a prediction mode in step 1510 and a method of selecting a prediction mode in step 1520.

In operation 1510, the processor may select a prediction mode based on statistical values of the plurality of prediction modes.

For example, the prediction mode having the lowest statistics value or the highest statistics value may be selected by comparing the statistics values of the plurality of prediction modes.

The statistical value may mean at least one of an average value, a maximum value, a minimum value, a mode value, a median value, a weighted average value, a distortion value, and a rate-distortion value.

Alternatively, the aforementioned statistical value may be a statistical value for the rate-distortion cost or the rate-distortion cost. The processor may calculate rate-distortion costs of the plurality of prediction modes by performing entropy encoding on the target block using the respective prediction modes for each prediction mode of the plurality of prediction modes. The processor may select one or more prediction modes of the plurality of prediction modes according to the rate-distortion costs. The processor may select the prediction mode having the lowest rate-distortion cost by comparing the rate-distortion costs of the plurality of prediction modes.

In operation 1520, the processor may induce prediction mode information on the target block, and select one or more prediction modes among the plurality of prediction modes using the derived prediction mode information.

In one embodiment, the processor may use the information related to the reference block in deriving the prediction mode information. Alternatively, the prediction mode information may include information related to the reference block.

In one embodiment, the reference block may include one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.

For example, the processing unit derives prediction mode information using a spatial neighboring block and deriving prediction mode information using a temporal neighboring block in deriving prediction mode information about a target block using information related to a reference block. The above can be used.

In an embodiment, the information related to the reference block may include a motion vector of the reference block, a prediction direction of the reference block, a reference picture index of the reference block, an encoding mode of the reference block, a reference picture list of the reference block, and a picture order count of the reference block ( One or more of a Picture Order Count (POC), a Quantization Parameter (QP) of the reference block, and a Coded Block Flag (CBF) of the reference block.

Also, in one embodiment, the information related to the reference block may further include information used for encoding and / or decoding the aforementioned reference block.

In one embodiment, the processor may select a prediction mode from a template matching mode and a bilateral matching mode. The plurality of prediction modes may include a template matching mode and a bilateral matching mode.

The processor may induce motion vectors of the reference blocks in selecting one or more prediction modes from among a plurality of prediction modes, such as a template matching mode and a bilateral matching mode, and determine the correlation of the derived motion vectors. And select one or more prediction modes of the plurality of prediction modes based on the determined correlation. Derivation of motion vectors, determination of correlation and selection of prediction mode are described below with reference to FIG. 16.

In one embodiment, the processor may select one or more prediction modes from among the plurality of prediction modes using the prediction direction. The prediction direction may be the prediction direction of the target block or the prediction direction of the reference block.

In one embodiment, the processor may select one or more prediction modes according to whether inter prediction of the target block is unidirectional prediction or bidirectional prediction.

For example, if the inter prediction of the target block is bidirectional prediction, the processor may select a lateral matching mode from among a plurality of prediction modes. The processor may select a template matching mode from among a plurality of prediction modes if the inter prediction of the target block is unidirectional prediction.

For example, if the inter prediction of the reference block is bidirectional prediction, the processor may select a bilateral matching mode among the plurality of prediction modes. The processor may select a template matching mode among a plurality of prediction modes if the inter prediction of the reference block is unidirectional prediction.

In one embodiment, the processor may select one or more prediction modes from among a plurality of prediction modes using a coding parameter. The coding parameter may be a coding parameter of the target block or a coding parameter of the reference block.

For example, the processor may determine whether the prediction mode of the reference block is the intra prediction mode or the inter prediction mode according to the coding parameter of the reference block. The processor may select one or more prediction modes from among a plurality of prediction modes according to whether the prediction mode of the reference block is an intra prediction mode or an inter prediction mode. The reference block may be a temporal neighboring block.

In one embodiment, the processor may select one or more prediction modes from among a plurality of prediction modes using the POC. The POC may be a POC of a target picture including a target block or a POC of a reference picture including a reference block.

For example, the processor may select one or more prediction modes from among the plurality of prediction modes based on the difference between the specified POCs.

For example, the processor may select a template matching mode from among a plurality of prediction modes when the difference between the specified POCs is smaller than the threshold. Here, the specified POCs may be a POC of a target picture and a POC of a reference picture.

16 illustrates reference blocks according to an example.

As described above, in selecting one or more prediction modes for a target block from among a plurality of prediction modes, motion vectors of reference blocks may be used. The motion vector can be derived as follows.

The reference blocks may include spatially adjacent blocks of the target block.

In Fig. 16, "a" represents block a. Block a may be the leftmost block among blocks adjacent to the top of the target block. "b" represents block b. Block b may be a second block from the left of the blocks adjacent to the top of the target block. "c" represents block c. Block c may be a second block from the right of the blocks adjacent to the top of the target block. "d" represents block d. The block d may be the rightmost block among blocks adjacent to the top of the target block.

In Fig. 16, "e" represents block e. Block a may be a topmost block among blocks adjacent to the left side of the target block. "f" represents block f. The block f may be the second block from the top among blocks adjacent to the left side of the target block. "g" represents block g. The block g may be the second block from the bottom of the blocks adjacent to the left side of the target block. "h" represents block h. The block h may be the lowest block among blocks adjacent to the left side of the target block.

The processor may use the motion vectors of the n or more merge candidates in deriving the motion vector of the reference block. In other words, the reference block may be n or more merge candidates. n can be a positive integer.

For example, when the processor derives two motion vectors, the processor may use the first merge candidate and the second merge candidate as reference blocks.

For example, when the processor derives three motion vectors, the processor may use the first merge candidate, the second merge candidate, and the third merge candidate as reference blocks.

In one embodiment, the merge candidate may be a block at a location specified relative to the target block.

In one embodiment, the merge candidate may be a block generated based on blocks at locations specified with respect to the target block. Alternatively, the merge candidate may be a virtual block or derived block that is determined based on blocks at locations specified for the target block. Alternatively, the above virtual block or derived block may be regarded as a reference block. In this case, the reference block may be a block generated based on blocks at positions specified with respect to the target block.

Blocks at specified locations may include spatial neighboring blocks and temporal neighboring blocks.

For example, when two merge candidates are used, the plurality of merge candidates may include a top position block and a left position block. The upper position block may be a block adjacent to the top of the target block or a block generated based on blocks adjacent to the top of the target block. The left position block may be a block adjacent to the left side of the target block or a block generated based on blocks adjacent to the left side of the target block.

In one embodiment, the processor may derive a motion vector of one of blocks a, b, c and d, and use the derived motion vector as the motion vector of the upper position block.

For example, the processor may use the motion vector of block a as the motion vector of the upper position block.

For example, the processor may use the motion vector of the block d as the motion vector of the upper position block.

In one embodiment, the processor may derive the motion vector of one of the blocks e, block f, block g and block h, and use the derived motion vector as the motion vector of the left position block.

For example, the processor may use the motion vector of the block e as the motion vector of the left position block.

For example, the processor may use the motion vector of the block h as the motion vector of the left position block.

In one embodiment, the processor may derive the motion vectors of one or more blocks of block a, block b, block c, and block d, and use the combination of derived motion vectors as the motion vector of the upper position block.

For example, the combination of derived motion vectors may be an average of the derived motion vectors or a weighted average of the derived motion vectors. Alternatively, the combination of derived motion vectors may be the result of a calculation or motion vector determination scheme using the derived motion vectors.

For example, the processor may use the average or the weighted average of the motion vector of block a and the motion vector of block b as the motion vector of the upper position block.

For example, the processor may use the average or the weighted average of the motion vectors of the leftmost m or the rightmost m blocks among blocks adjacent to the top of the target block as the motion vector of the upper position block. m may be an integer of 2 or more.

For example, the processor may use the average or weighted average of the motion vectors of blocks a, b, c and d as the motion vector of the upper position block.

For example, the processor may use the average or the weighted average of the motion vectors of the blocks adjacent to the top of the target block as the motion vector of the top position block.

In one embodiment, the processor may derive the motion vectors of one or more blocks of block e, block f, block g and block h, and use the combination of the derived motion vectors as the motion vector of the left position block.

For example, the processor may use the average or the weighted average of the motion vector of block e and the motion vector of block f as the motion vector of the left position block.

For example, the processor may use the average or the weighted average of the motion vectors of the uppermost m or the lowermost m blocks among the blocks adjacent to the left side of the target block as the motion vector of the left position block. m may be an integer of 2 or more.

For example, the processor may use the average or the weighted average of the motion vectors of blocks e, block f, block g and block h as the motion vector of the left position block.

For example, the processor may use the average or the weighted average of the motion vectors of the blocks adjacent to the left side of the target block as the motion vector of the left position block.

In one embodiment, the processor may perform scaling of the motion vector in deriving a motion vector used for the motion vector of the left position block.

For example, if the POC of the reference picture of the neighboring block does not match ((POC of the target picture of the target block)-1) in the reference picture list L0, the processing unit performs scaling of the motion vector of the neighboring block. Can be. In this case, the processor may perform scaling of the motion vector of the adjacent block in accordance with ((POC of the target picture) −1).

For example, if the POC of the reference picture of the neighboring block does not match ((POC of the target picture of the target block) + 1) in the reference picture list L1, the processing unit performs scaling of the motion vector of the neighboring block. Can be. In this case, the processor may perform scaling of the motion vector of the adjacent block in accordance with ((POC) + 1 of the target picture).

In the above embodiments, it has been described that one or more motion vectors of one or more adjacent blocks are used to derive the motion vector of the merge candidate. In addition, one or more motion vectors of one or more adjacent blocks may be considered to be used to select one or more prediction modes of the plurality of prediction modes. In other words, the merge candidate may be a virtual block or derived block for defining a motion vector used to select one or more prediction modes. In addition, the motion vector of the merge candidate may be regarded as a motion vector calculated, determined or derived based on one or more motion vectors of one or more adjacent blocks.

Referring back to FIG. 15, as described above, in operation 1520, the processor may determine the correlation of the derived motion vectors of the reference blocks.

In one embodiment, there may be three reference blocks. The processor may calculate correlation using the motion vectors of the three reference blocks. In calculating the correlation, Equation 2 below may be used.

[Formula 2]

In the "MV _nm " of the formula, n may represent the index or number of the reference block. For example, MV ₀ may represent the first reference block. m may mean the x coordinate or the y coordinate of the motion vector. For example, MV _nx may be the x coordinate of a motion vector of a reference block having an index of n. MV _ny may be the y coordinate of the motion vector of the reference block whose index is n.

"Abs {}" in the expression may represent an absolute value.

For example, the three reference blocks may be the first merge candidate, the second merge candidate, and the third merge candidate described above.

In one embodiment, the processor may calculate correlation using the motion vectors of the two reference blocks. In calculating the correlation, Equation 3 below can be used.

[Equation 3]

In one embodiment, the processor may calculate correlation using motion vectors present in one reference block. In this case, the reference block may be a block decoded by bidirectional prediction, and correlation may be calculated using two motion vectors present. In calculating the correlation, Equation 4 below can be used.

[Equation 4]

In the expression "MVN _m ", N may represent the index or number of the reference picture list of the reference block. For example, MV0 may represent the reference picture list 0 and m may mean the x coordinate or the y coordinate of the motion vector. For example, MVNx may be the x coordinate of the motion vector of the N reference picture list of the reference block.

"Abs {}" in the expression may represent an absolute value.

For example, the reference block may be the first merge candidate, the second merge candidate, or the third merge candidate described above.

In selecting the prediction mode of the target block, the processor may select one of the template matching mode and the bilateral matching mode as the prediction mode of the target block based on the calculated correlation.

In one embodiment, the processor may select one of the template matching mode and the bilateral matching mode as the prediction mode of the target block according to the comparison between the calculated correlation and the threshold.

In one embodiment, when the calculated correlation is greater than the threshold, the processor may select the template matching mode as the prediction mode of the target block.

In one embodiment, when the calculated correlation is smaller than the threshold, the processor may select the template matching mode as the prediction mode of the target block.

In one embodiment, if the calculated correlation is greater than the threshold, the processor may add the template matching mode to one or more prediction modes selected for the target block.

In one embodiment, when the calculated correlation is smaller than the threshold, the processor may add the template matching mode to one or more prediction modes selected for the target block.

In one embodiment, if the calculated correlation is greater than the threshold, the processor may add the bilateral matching mode to one or more prediction modes selected for the target block.

In one embodiment, when the calculated correlation is smaller than the threshold, the processor may add the bilateral matching mode to one or more prediction modes selected for the target block.

In one embodiment, regardless of the calculated correlation, the processor may select both template matching mode and bilateral matching mode as one or more prediction modes for the target block.

In an embodiment, the threshold may be determined by a user of the encoding apparatus 1200 and / or the decoding apparatus 130. Alternatively, the threshold may be determined by information related to the target block and / or information related to the reference block.

In one or more prediction modes selected by prediction mode information among the plurality of prediction modes, one of the selected one or more prediction modes may be used for encoding and / or decoding of the target block.

Prediction mode determination information may be used to indicate a prediction mode used for encoding and / or decoding a target block among the selected one or more prediction modes. In other words, the prediction mode determination information may indicate a prediction mode for encoding and / or decoding the target block. The processor may perform encoding and / or decoding of the target block using the prediction mode indicated by the prediction mode determination information.

In order to determine a prediction mode for decoding a target block, prediction mode determination information for determining a prediction mode may be signaled from the encoding apparatus 1200 to the decoding apparatus 1300.

When one or more prediction modes are selected among the plurality of prediction modes according to the prediction mode information or the like, prediction mode determination information may be at least partially omitted according to the selected one or more prediction modes.

In other words, the prediction mode determination information may include one or more partial prediction mode determination information, and the one or more partial prediction mode determination information in signaling of the prediction mode determination information according to one or more prediction modes selected from a plurality of prediction modes. At least some of these may be omitted.

In other words, if it is determined by the prediction mode information that the specified prediction mode among the plurality of prediction modes is excluded from the selection, the signaling of the prediction mode determination information may be performed so that the information related to the excluded prediction mode is omitted. . Alternatively, if it can be determined by the prediction mode information that only the specified prediction mode is selected among the plurality of prediction modes, the signaling of the prediction mode determination information may be performed so that the information on the specified prediction mode is included.

17 to 19 below illustrate examples of signaling for prediction mode determination information according to an example.

In FIG. 17, "1" may represent a first predefined value (eg, 1, logical 1, or true).

"0" may represent a second predefined value (eg, 0, logical 0, or false).

"X" may indicate that it is not signaled.

The prediction mode determination information may include at least one of a merge identifier, a matching prediction identifier, a condition, and a matching mode identifier. For example, the plurality of partial prediction mode determination information described above may be a merge identifier, a matching prediction identifier, a state, and a matching mode identifier. Alternatively, the state may be coding parameters or prediction mode information of the target block that are not included in the prediction mode determination information.

The merge identifier may indicate whether the merge mode is used. The merge mode may be a premise of selecting one or more prediction modes from among a plurality of prediction modes of the embodiments. In other words, only one or more prediction modes may be selected from among a plurality of prediction modes when the merge mode is performed.

If the merge identifier has the first predefined value, the merge for the target block may be performed. When the merge for the target block is performed as the merge identifier has the first predefined value, the prediction mode determination information may include at least one of a matching prediction identifier, a state, and a matching mode identifier.

If the merge identifier has the second predefined value, the merge for the target block may not be performed. When the merge identifier is not performed as the merge identifier has the second predefined value, the matching prediction identifier, the state and the matching mode identifier may be excluded from the prediction mode determination information. That is, when merge is not performed on the target block, the prediction mode determination information may not include the matching prediction identifier, the state and the matching mode identifier.

According to an embodiment, the matching prediction identifier may indicate whether one or more prediction modes are selected from among a plurality of prediction modes.

When the matching prediction identifier has the first predefined value, one or more prediction modes among the plurality of prediction modes may be selected using prediction mode information or the like as described above. As the matching prediction identifier has a first predefined value, when one or more prediction modes of the plurality of prediction modes are selected, the prediction mode determination information may include at least one of a state and a matching mode identifier.

If the matching prediction identifier has a second predefined value, one or more prediction modes may not be selected, and the processor may use the specified prediction mode for decoding the target block. For example, the specified prediction mode may be merge mode. As the matching prediction identifier has a second predefined value, when one or more prediction modes are not selected among the plurality of prediction modes, the state and the matching mode identifier may be excluded from the prediction mode determination information. That is, when one or more prediction modes are not selected among the plurality of prediction modes, the prediction mode determination information may not include a state and a matching mode identifier.

The processor may select one or more prediction modes among the plurality of prediction modes using the state. In other words, the number of selected one or more prediction modes may be determined by the state. Depending on the state, the selected one or more prediction modes may be plural or only one.

When there are a plurality of selected one or more prediction modes, a prediction mode to be used for encoding a target block among the selected plurality of prediction modes may be determined.

When a plurality of prediction modes are selected, a matching mode identifier may be required to determine prediction modes used for encoding and / or decoding a target block among the selected plurality of prediction modes.

The matching mode identifier may indicate a prediction mode used for encoding and / or decoding of the target block among the selected prediction modes. The value of the matching mode identifier may be a number or index of a prediction mode used for encoding and / or decoding a target block among the selected plurality of prediction modes. For example, when the value of the matching mode identifier is L, the prediction mode used for encoding and / or decoding of the target block is the Lth prediction mode or the L + 1th prediction mode among the plurality of selected prediction modes. Can be.

When a plurality of prediction modes are selected, the prediction mode determination information may include a matching mode identifier indicating prediction modes used for encoding and / or decoding of a target block among the selected plurality of prediction modes.

When one prediction mode is selected or when the selected prediction mode is one, the matching mode identifier may be excluded from the prediction mode determination information. That is, when only one prediction mode is selected so that the prediction mode used for encoding and / or decoding of the target block can be determined without the matching mode identifier, the prediction mode determination information may not include the matching mode identifier.

In one embodiment, the state may represent a slice of the target picture of the target block. For example, when the slice of the target picture is a B-slice, the selected one or more prediction modes may include a template matching mode and a bilateral matching mode.

For example, when the selected one or more prediction modes include a template matching mode and a bilateral matching mode, if the value of the matching mode identifier is a first predefined value, the template matching mode may be used for encoding and / or decoding the target block. It may be a matching mode used. Also, in this case, if the value of the matching mode identifier is a second predefined value, the bilateral matching mode may be a matching mode used for encoding and / or decoding of the target block.

In FIG. 18, the functions of the merge identifier and the matching prediction identifier may be the same and / or similar to those described above with reference to FIG. 17. Duplicate explanations are omitted.

When the matching prediction identifier has a first predefined value, a prediction mode may be selected among the plurality of prediction modes using prediction mode information and the like.

For example, the prediction mode selected using the prediction mode information may be one template matching mode. If there is only one prediction mode selected, the matching mode identifier may not be required. In other words, when there is one selected prediction mode, the matching mode identifier may be excluded from the prediction mode determination information. Or, if there is one selected prediction mode, the prediction mode determination information may not include a matching mode identifier.

In FIG. 19, the functions of the merge identifier and the matching prediction identifier may be the same and / or similar to those described above with reference to FIG. 17. Duplicate explanations are omitted.

For example, the prediction mode selected using the prediction mode information or the like may be one lateral matching mode. If there is only one prediction mode selected, the matching mode identifier may not be required. In other words, when there is one selected prediction mode, the matching mode identifier may be excluded from the prediction mode determination information. Or, if there is one selected prediction mode, the prediction mode determination information may not include a matching mode identifier.

The processor may use the above-described merge identifier, matching prediction identifier and / or state values to identify what information the prediction mode determination information includes and does not contain.

In addition, the processor may determine the prediction mode used for encoding and / or decoding the target block using the values of the above-described merge identifier, matching prediction identifier, state and / or matching mode identifier.

As described above, the bilateral matching mode and / or template matching mode may be used for encoding and / or decoding of the target block. In FIG. 20 and FIG. 21, the prediction using the bilateral matching mode and the prediction using the template matching mode are described.

In operation 1420, the processor may perform prediction on the target block using bilateral matching.

In one embodiment, the processor may determine whether to perform a bilateral matching according to the type of slice. The slice may be a slice of the target block.

For example, the processor may perform lateral matching only when the slice is a B slice. Alternatively, the processor may perform a lateral match only when the slice is a P slice or a B slice.

In one embodiment, in performing the lateral matching, the processor may derive the motion vector MV0 using the initial motion vector that can be derived from the target block, and derive the motion vector MV1 according to the motion vector MV0. have.

The direction of the motion vector MV1 may be opposite to the direction of the induced motion vector MV0, and the motion trajectory of the motion vector MV1 may be the same as the motion trajectory of the derived motion vector MV0. In other words, the motion vector MV1 may be a motion vector present on the same locus as the locus of the induced motion vector MV0 in a direction opposite to the direction of the induced motion vector MV0.

The initial motion vector may be a motion vector that is initially derived.

"Ref0" may be reference picture 0 of MV0. "Ref1" may be reference picture 1 of MV1.

TD0 may be a trajectory distance between the target picture and reference picture 0. TD1 may be a trajectory distance between the target picture and reference picture 1.

In one embodiment, the initial motion vector may be one or more.

In one embodiment, as shown in FIG. 20, the processor may derive an initial motion vector according to one or more directions of at least one motion vector among MV0 and MV1.

For example, the processor may derive an initial motion vector from the reference picture list L0 with respect to the target block.

For example, the processor may derive an initial motion vector from the reference picture list L1 with respect to the target block.

In performing the lateral matching, the processor may construct an initial motion vector candidate list in deriving an initial motion vector for the target block, and may include at least one of one or more initial motion vector candidates of the configured initial motion vector candidate list. The initial motion vector candidate can be used as the initial motion vector.

For example, the processor may use the AMVP mode to construct an initial motion vector candidate list for the target block. The predicted motion vector candidates of the predicted motion vector candidate list in the AMVP mode may be one or more initial motion vector candidates of the initial motion vector candidate list. The processor may add the predicted motion vector candidates of the predicted motion vector candidate list in the AMVP mode to the initial motion vector candidate list.

For example, the processor may use the merge mode to construct an initial motion vector candidate list for the target block. The merge candidates of the merge candidate list in the merge mode may be one or more initial motion vector candidates of the initial motion vector candidate list. The processor may add merge candidates of the merge mode to the initial motion vector candidate list.

For example, the processor may configure the matching prediction unidirectional motion vector of the target block as an initial motion vector candidate list. The processor may add the matching prediction unidirectional motion vector of the target block to the initial motion vector candidate list.

For example, the processor may configure the motion vector of the neighboring block of the target block as the initial motion vector candidate list. The processor may add the motion vector of the neighboring block of the target block to the initial motion vector candidate list.

For example, the processor may configure the above-described combinations of motion vectors as an initial motion vector candidate list. Combinations of motion vectors can be N or more. N may be a positive integer. The processor may add the aforementioned combinations of motion vectors to the initial motion vector candidate list.

For example, the processor may use a motion vector for at least one of a direction of the reference picture list L0 and a direction of the reference picture list L1 in constructing the initial motion vector list. The processor may add a motion vector for at least one of the direction of the reference picture list L0 and the direction of the reference picture list L1 to the initial motion vector list.

In one embodiment, in performing the lateral matching, the processor may derive a bidirectional motion vector that best matches the initial motion vector indication block and the opposite block using the initial motion vector list. Here, the initial motion vector indication block may be a block indicated by the initial motion vector. The opposite block may be a block existing on the same trajectory as the trajectory of the initial motion vector indicating block in a direction opposite to the direction of the initial motion vector indicating block. In other words, the direction of the initial motion vector block and the direction of the opposite block may be opposite to each other, and the trajectory of the initial motion vector block and the trajectory of the opposite block may be the same.

For example, as shown in FIG. 20, when the motion vector present in the initial motion vector list is MV0, the processor 1) in the reference picture in the opposite direction of the direction of MV0, 2) on the same locus as the locus of MV0 3) can derive MV1, which is a motion vector pointing to the block that best matches the block indicated by MV0.

In one embodiment, the processor may perform the improvement of the initial motion vector.

For example, the processor may search for blocks around the block indicated by the derived MV0 and search blocks around the block indicated by the derived MV1. The processor may improve the initial motion vector to indicate blocks that most closely match each other among blocks around the block indicated by MV0 and blocks near the block indicated by MV1.

In one embodiment, in performing the lateral matching, the processor may induce motion information in units of sub-blocks. The motion information may include a motion vector.

In deriving motion information in units of sub-blocks, the processor may use the same method as described above of deriving an initial motion vector in units of blocks.

For example, a sub block that is a unit of derivation of motion information may be a 4 × 4 block. Alternatively, the sub block may be an 8 × 8 block. In other words, in deriving the motion information, the reference block may be divided into subblocks of a predefined size, such as 4x4 size or 8x8 size.

In one embodiment, the sub block may be a block of a combination of one or more predefined sizes. The sub block may be blocks of one or more sizes of 4x4 size and 8x8 size. For example, the sub block may be 4x4 blocks and 8x8 blocks.

In one embodiment, in performing the lateral matching, the processor may define the degree of matching of the blocks. In other words, the processor may use one of the defined defined methods in determining the degree of matching of the blocks.

For example, if the sum of absolute differences (SAD) of the two blocks is the lowest, the processor may determine that the two blocks are the best match. In other words, the processor may determine that the lower the SAD between the two blocks, the better the two blocks match.

For example, if the sum of Absolute Transformed Differences (SATD) is the lowest between the two blocks, the processor may determine that the two blocks match best. In other words, the processor may determine that the lower the SATD between the two blocks, the better the two blocks match.

In operation 1420, the processor may perform prediction on the target block using template matching.

In one embodiment, the processor may determine whether to perform template matching according to the type of slice. The slice may be a slice of the target block. For example, the processor may perform template matching only when the slice is a B slice. Alternatively, the processor may perform template matching only when the slice is a P slice or a B slice.

In one embodiment, in performing template matching, the processor may use a neighboring block of the target block as a template. The size and position of the template can be determined by a predefined manner.

For example, the processor may use a neighboring block adjacent to the upper end of the target block as a template.

For example, the processor may use a neighboring block adjacent to the left side of the target block as a template.

For example, the processor may use a neighboring block adjacent to the top or left side of the target block as a template.

For example, the size of the template may be (width, 4) or (4, height). width may be the horizontal size of the target block. The height may be the vertical size of the target block.

For example, the size of the template may be (width, 8) or (8, height).

For example, the size of the template may be determined based on the size of the target block. For example, the size of the template may be 1/2 or 1/4 of the size of the target block.

In one embodiment, in performing template matching, the processor may search for a block in a reference picture using a template. In FIG. 21, "Ref0" may be a reference picture.

The processor may search for a block having a shape and size equal to the shape and size of the template in the reference picture. The processor may derive the motion vector of the target block by using the motion vector of the searched block. The processor may determine the motion vector for the searched block as the motion vector of the target block.

At this time, the direction of the reference picture from which the block is found and the direction of the reference picture indicated by the motion vector of the target block may be opposite to each other.

For example, the processor may search for a block that best matches the template of the target block in the reference picture in the reference picture list L0.

For example, the processor may search for a block that best matches the template of the target block in the reference picture in the reference picture list L1.

For example, the processor may search for a block by using at least one list of the presented reference picture lists.

In one embodiment, the processor may perform the improvement of the motion vector.

For example, the processor may search for blocks in the vicinity of the block indicated by the current template and the derived motion vector, and the motion information so that the motion vector points to a block that best matches the template among the blocks found in the search. Can be improved. In other words, the improved motion information may be a motion vector for a block that best matches the template among blocks around the block indicated by the derived motion vector.

In one embodiment, in performing template matching, the processor may induce motion information in units of sub-blocks. The motion information may include a motion vector.

In deriving motion information in units of sub-blocks, the processor may use the same method as inducing a motion vector in units of blocks.

For example, a sub block that is a unit of derivation of motion information may be a 4 × 4 block. Alternatively, the sub block may be an 8 × 8 block. In other words, in deriving the motion information, the block may be divided into sub-blocks of a predefined size, such as 4x4 size or 8x8 size.

In one embodiment, in performing template matching, the processor may define a degree of matching of the blocks. In other words, the processor may use one of the defined defined methods in determining the degree of matching of the blocks. One of the blocks may be a template.

The prediction method and the bitstream generation method of the target block according to the embodiment may be performed by the encoding apparatus 1200. The embodiment may be part of an encoding method or a video encoding method of a target block.

In operation 2210, the processor 1210 may select a prediction mode. Step 2210 may correspond to step 1410 described above with reference to FIG. 14.

In operation 2220, the processor 1210 may perform prediction using the selected prediction mode. Step 2220 may correspond to step 1420 described above with reference to FIG. 14.

In operation 2230, the processor 1210 may generate prediction mode determination information. The prediction mode determination information may be generated at least partially in

step

2210 or 2220.

In operation 2240, the processor 1210 may generate a bitstream.

The bitstream may include information about the encoded target block. For example, the information about the encoded target block may include transform and quantized coefficients of the target block, and may include coding parameters of the target block. The bitstream may include prediction mode determination information.

The processor 1210 may perform entropy encoding on prediction mode determination information, and may generate a bitstream including entropy-coded prediction mode determination information.

The processor 1210 may store the generated bitstream in the storage 1240. Alternatively, the communication unit 1220 may transmit the bitstream to the decoding device 1300.

23 is a flowchart of a method of predicting a target block using a bitstream, according to an embodiment.

The prediction method of the target block using the bitstream of the embodiment may be performed by the decoding apparatus 1300. An embodiment may be part of a decoding method or a video decoding method of a target block.

In operation 2310, the communicator 1320 may acquire a bitstream. The communication unit 1320 may receive a bitstream from the encoding apparatus 1200.

The processor 1310 may store the obtained bitstream in the storage 1240.

In operation 2320, the processor 1310 may obtain prediction mode determination information from the bitstream.

The processor 1310 may obtain prediction mode determination information by performing entropy decoding on the entropy-coded prediction mode determination information of the bitstream.

In operation 2330, the processor 1310 may select a prediction mode based on the prediction mode determination mode. Step 2330 may correspond to step 1410 described above with reference to FIG. 14.

In operation 2340, the processor 1310 may perform prediction using the selected prediction mode. Step 2340 may correspond to step 1420 described above with reference to FIG. 14.

In the above-described embodiments, the methods are described based on a flowchart as a series of steps or units, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or simultaneously from other steps as described above. Can be. Also, one of ordinary skill in the art appreciates that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps in the flowcharts may be deleted without affecting the scope of the present invention. I can understand.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims

A processor that selects a prediction mode from among a plurality of prediction modes using prediction mode information on the target block, and performs prediction on the target block based on the selected prediction mode.

Including,

The prediction mode information includes information related to a reference block,

And the reference block includes one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.
A processor that selects a prediction mode from among a plurality of prediction modes using prediction mode information on the target block, and performs prediction on the target block based on the selected prediction mode.

Including,

The prediction mode information includes information related to a reference block,

And the reference block includes one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.
Selecting a prediction mode from among a plurality of prediction modes by using prediction mode information on the target block; And

Performing prediction on the target block based on the selected prediction mode

Including,

The prediction mode information includes information related to a reference block,

Wherein the reference block includes one or more of a spatial neighboring block spatially adjacent to the target block and a temporal neighboring block temporally adjacent to the target block.
The method of claim 3,

The selected prediction mode is plural,

And a prediction mode to be used for prediction of the target block among the plurality of selected prediction modes is determined using prediction mode determination information.
The method of claim 4, wherein

Receiving a bitstream including the prediction mode determination information

Decryption method further comprising.
The method of claim 4, wherein

The prediction mode determination information includes one or more partial prediction mode determination information,

The selected prediction mode is one or more,

And at least some of the one or more partial prediction mode determination information are omitted in the signaling of the prediction mode determination information according to the selected one or more prediction modes.
The method of claim 4, wherein

The prediction mode determination information includes a matching mode identifier,

And if the selected prediction mode is plural, the matching mode identifier indicates a prediction mode used for decoding the target block among the selected plurality of prediction modes.
The method of claim 7, wherein

And when the selected prediction mode is one, the prediction mode determination information does not include the matching mode identifier.
The method of claim 4, wherein

The prediction mode determination information includes a matching prediction identifier,

The matching prediction identifier indicates whether the prediction mode is selected among the plurality of prediction modes.
The method of claim 3,

The information related to the reference block includes a motion vector of the reference block, a prediction direction of the reference block, a reference picture index of the reference block, an encoding mode of the reference block, a reference picture list of the reference block, and a picture order of the reference block. And at least one of a Picture Order Count (POC), a Quantization Parameter (QP) of the reference block, and a Coded Block Flag (CBF) of the reference block.
The method of claim 3,

The plurality of prediction modes include a template matching mode and a lateral matching mode.
The method of claim 3,

The reference block is a plurality,

Motion vectors of the plurality of reference blocks are derived, correlation of the derived motion vectors is determined, and the prediction mode is selected based on the correlation.
The method of claim 12,

And one of a template matching mode and a bilateral matching mode is selected as the prediction mode of the target block according to the comparison between the correlation and the threshold.
The method of claim 3,

The prediction mode is selected from among the plurality of prediction modes using a prediction direction,

The prediction direction is a prediction direction of the target block or a prediction direction of the reference block.
The method of claim 3,

And the prediction mode is selected according to whether the inter prediction of the target block is unidirectional prediction or bidirectional prediction.
The method of claim 15,

And if the inter prediction is bidirectional prediction, a bilateral matching mode is selected from the plurality of prediction modes.
The method of claim 15,

And if the inter prediction is one-way prediction, a template matching mode is selected from the plurality of prediction modes.
The method of claim 3,

And the prediction mode is selected from among the plurality of prediction modes using the coding parameter of the reference block.
The method of claim 18,

It is determined whether a prediction mode of the reference block is an intra prediction mode or an inter prediction mode according to a coding parameter of the reference block, and a plurality of prediction modes according to whether the prediction mode of the reference block is an intra prediction mode or an inter prediction mode. Decoding method in which the selected prediction mode is determined.
The method of claim 3,

And the prediction mode is selected from among the plurality of prediction modes based on the difference between the specified POCs.