CN117336483A - Encoding method, decoding method, encoding device, and decoding device - Google Patents

Abstract

The present invention provides an encoding method, a decoding method, an encoding device, and a decoding device, including: initializing a history candidate motion information list corresponding to a current coding tree unit, wherein the history candidate motion information list comprises N storage spaces, the initialized history candidate motion information list comprises at least M empty storage spaces, M is less than or equal to N, the current coding tree unit is contained in a coding tree unit set (Slice) formed by a plurality of coding tree units, and the current coding tree unit is not the first in the coding tree unit set according to a preset processing sequence; adding motion information at L positions in an adjacent block of a space domain of the current coding tree unit to the history candidate motion information list according to a preset sequence, wherein M is less than or equal to L and less than or equal to N; inter-prediction is performed on the current coding tree unit or the current coding unit based on the historical candidate motion information list.

Description

Encoding method, decoding method, encoding device, and decoding device

This application is a divisional application of a patent application with application number 201810990347.3 and a filing date of August 28, 2018, titled "Inter-frame prediction method, device and encoding/decoding method and device for its application".

Technical field

The embodiments of the present application relate to the field of video coding, and more specifically, to coding methods, decoding methods, coding devices, and decoding devices.

Background technique

Video encoding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video dissemination on the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray discs, and video content capture and editing systems and security applications for camcorders.

With the development of the block-based hybrid video coding method in the H.261 standard in 1990, new video coding technologies and tools have been developed and formed the basis for the subsequent evolution of video coding standards. Video coding standards include MPEG-1 video, MPEG-2 video, ITU-TH.262/MPEG-2, ITU-T H.263, ITU-T H.264/MPEG-4 Part 10 Advanced Video Coding (Advanced Video Coding (AVC), ITU-T H.265/High Efficiency Video Coding (HEVC)... and extensions to such standards, such as scalability and/or 3D (three-dimensional) extensions. As video creation and sharing become more widespread, video traffic becomes the largest burden on communication networks and data storage. Therefore one of the goals of most video coding standards is to reduce the bitrate compared to previous standards without reducing the subjective quality of the picture. Even though the latest High Efficiency video coding (HEVC) can compress video approximately twice as much as AVC without reducing the subjective quality of the picture, there is still an urgent need for new technologies to further compress video compared to HEVC, and a new generation of video coding Technology VVC (Versatile Video Coding) technology is in the process of being formulated. Its goal is to further increase the compression rate by about 50% compared to HEVC without reducing the subjective quality of the picture.

In the HEVC/H.265 video coding standard, or the VVC/H.266 video coding standard under development, a frame of image will be divided into non-overlapping Coding Tree Units (CTU). The size of the CTU can be set to 64×64 or 128×128 size. Taking a 64×64 size CTU as an example, it contains 64 columns of pixels, each column contains 64 pixels, and each pixel contains a luminance component or/and a chrominance component. A CTU is divided into one or more coding units (Coding Unit, CU). A CU contains basic coding information, including prediction mode, transform coefficients and other information. The decoder can perform corresponding prediction, inverse quantization, inverse transformation, reconstruction, filtering and other decoding processes on the CU based on these encoding information to generate a reconstructed image corresponding to the CU. A CU corresponds to a predicted image and a residual image, and the predicted image and the residual image are added to obtain a reconstructed image. The prediction image is generated by intra prediction or inter prediction, and the residual image is generated by inverse quantization and inverse transformation of the transform coefficients.

Inter-frame prediction is a prediction technology based on motion compensation. In inter-frame predictive coding, since there is a certain temporal correlation between the same objects in adjacent frames of the image, each frame of the image sequence can be divided into many non-overlapping blocks, and the motion of all pixels in the block is considered to be the same. . The main processing process is to determine the motion information of the current block, obtain the reference image block from the reference frame of the current block according to the motion information, and generate the predicted image of the current block, where the current block (current block) refers to the encoding/decoding process being performed. Image block, where the current block may be a luma block or a chroma block in a coding unit. Motion information includes inter-frame prediction direction, reference frame index (reference index, ref_idx), motion vector (motion vector, MV), etc., where the inter-frame prediction direction indicates whether the current block uses forward prediction, backward prediction, or bidirectional prediction. A prediction direction, the motion vector indicates the displacement vector of the reference image block used to predict the current block in the reference frame relative to the current block, so a motion vector corresponds to a reference image block in a reference frame. Inter-frame prediction of an image block can only use one motion vector to generate a predicted image using pixels in a reference frame, which is called unidirectional prediction; it can also use two motion vectors to use pixels in two reference frames to combine Generating predicted images is called bidirectional prediction. That is, an image block may typically contain one or two motion vectors. For some multi-hypothesis inter prediction techniques, one image block may contain more than two motion vectors.

An MV is a two-dimensional vector, including a horizontal displacement component and a vertical displacement component; an MV corresponds to two frames, each frame has a picture order count (POC), which is used to represent the display order of the image. number, so an MV also corresponds to a POC difference. The POC difference has a linear relationship with the time interval. The scaling of motion vectors usually uses a scaling method based on POC differences to convert the motion vectors between a pair of images into the motion vectors between another pair of images.

During encoding, video coding standards such as H.265/HEVC and H.266/VVC divide a frame of image into non-overlapping Coding Tree Units (CTU). A CTU is divided into one or more coding Unit(CodingUnit,CU). A CU contains coding information, including prediction mode, transform coefficients and other information. Decoding end: According to these coding information, the CU is subjected to corresponding prediction, inverse quantization, inverse transformation and other decoding processes to generate the reconstructed image corresponding to the CU.

In the code stream, motion information occupies a large amount of data. In order to reduce the amount of data required, prediction is usually used to transmit motion information. The prediction methods are divided into inter-frame prediction and intra-frame prediction. Intra-frame prediction uses the reference block in the same frame image as the prediction block, while inter-frame prediction uses Reference blocks in different frames are used as prediction blocks.

There are three commonly used inter prediction modes:

1) Advanced Motion Vector Prediction (AMVP) mode: identifies the inter-frame prediction direction (forward, backward or bidirectional), reference frame index (reference index), and motion vector prediction used by the current block in the code stream Value index (motion vector predictor index, MVP index), motion vector residual value (motion vector difference, MVD); the reference frame queue used is determined by the inter-frame prediction direction, and the reference frame pointed to by the current block MV is determined by the reference frame index. The motion vector predictor index indicates an MVP in the MVP list as the predictor of the current block MV, and an MVP is added to an MVD to obtain an MV.

2) Merge/skip mode: The code stream identifies the fusion index (merge index), selects a merge candidate from the fusion candidate list (mergecandidate list) according to the merge index, and the motion information of the current block (including prediction direction, reference frame, motion vector) is determined by this merge candidate. The main difference between merge mode and skip mode is that merge mode implies that the current block has residual information, that is, the motion vector obtained from the motion candidate list is used as the motion vector predictor of the current block, and the motion vector of the current block is The predicted value of the motion vector is obtained by adding the residual value of the motion vector. The residual value of the motion vector is obtained by decoding the code stream; and the skip mode implies that the current block has no residual information (or the residual is 0), that is, from The motion vector obtained in the motion vector list is directly used as the motion vector of the current block for inter-frame prediction; the two modes derive motion information in the same way.

3) Affine transformation mode: The motion vector of each sub-block in the current block is obtained from the motion vectors of two or three control points through affine transformation.

In AMVP mode and merge/skip mode, each needs to first establish a candidate list. For AMVP, a candidate motion vector list (AMVP candidate list) needs to be established, and a better motion vector will be selected as the current The motion vector prediction value of the block, and the index value of the motion vector will be written into the code stream; for the Merge/skip mode, it is recommended that a motion candidate list (Merge candidate list) needs to be established, and the current candidate Candidate motion information in the motion information list includes unidirectional or bidirectional reference information, reference frame index, and motion vector information corresponding to the reference direction. Figure 6 shows the specific positions of the candidate blocks in the spatial domain and the candidate blocks in the time domain that need to be referenced to establish the candidate motion vector list and the current candidate motion information list in AMVP and merge/skip modes. The picture on the left of Figure 6 shows that it has been determined The blocks at the lower right and center of the current block are the most suitable to provide good temporal motion vector predictors (TMVP). The right diagram of Figure 6 shows that two spatial MVP candidates A and B are derived from five spatial neighboring blocks. AMVP allows up to two candidate motion vectors, that is, the maximum value of the AMVP candidate list is 2, while the merge/skip mode allows more candidate motion information, and the maximum candidate motion information allowed in HEVC is 5. That is, the maximum value of the current candidate motion information list is 5.

In the development process of the latest video coding technology, versatile video coding (versatile video coding), it is proposed to use historical motion information to expand the above-mentioned AMVP and optional motion vectors or candidate motion information in Merge/skip mode. The JVET-K0104 proposal proposes a method of adding historical candidate motion information (history candidates) to the fused motion information candidate list and candidate motion vector prediction list, adding merge/skip fused motion information candidates and Inter MVP mode motion vector prediction. The number of candidates improves prediction efficiency. The historical candidate motion information list is composed of historical candidate motion information, where the historical candidate motion information is the motion information of previously encoded blocks. In the JVET-K0104 proposal, the use of the history candidate motion information list (history candidate list) and the construction method of the history candidate motion information list are introduced.

The construction method of the fused motion information candidate list that adds historical candidate motion information (how to use the historical candidate motion information list) is as follows:

Step 1: Add the spatial candidates and temporal candidates adjacent to the current block's spatial domain to the fused motion information candidate list of the current block, the method is the same as that in HEVC. As shown in Figure 6, the spatial candidates include A0, A1, B0, B1, and B2, and the time domain candidates include T0 and T1. In VTM, temporal candidates also include candidates provided by adaptive temporal motion vector prediction (ATMVP) technology.

Step 2: Add the historical candidate motion information in the historical candidate motion information list to the fused motion information candidate list, and check the preset number of historical candidate motion information in order from the tail to the head of the historical candidate motion information list, as shown in the figure 7 shown. Starting from the historical candidate motion information at the end of the historical candidate motion information list, check whether it is the same as the fused motion information candidate in the fused motion information candidate list obtained in step 1. If it is different, add it to the fused motion information candidate list. If it is the same, check The next historical candidate motion information in the historical candidate motion information list.

Step 3: Add other types of fused motion information candidates, such as bi-predictive candidates and zero motion vector candidates.

In the JVET-K0104 proposal, the historical candidate motion information list is constructed using the motion information of the encoded blocks in the current frame, and the historical candidate motion information list is accessed in a first-in, first-out manner. The overall historical candidate motion information list in the encoding/decoding end is constructed and used as follows:

Step 1: Initialize the historical candidate motion information list at the beginning of slice (SLICE) decoding and clear it.

Step 2: Decode the current CU. If the current CU or the current block is in the merge or inter prediction mode, generate a fusion motion information candidate list or a candidate motion vector prediction list, and add the historical candidate motion information in the historical candidate motion information list. to the fused motion information candidate list or candidate motion vector prediction list.

Step 3: After decoding the current CU or current block, use the motion information of the current block as new historical candidate motion information, add it to the historical candidate motion information list, and update the historical candidate motion information list, as shown in Figure 8. First, starting from the head of the historical candidate motion information list, the motion information of the current block is compared with the historical candidate motion information in the historical candidate motion information list. If a certain historical candidate motion information (for example, MV2 in Figure 3) is the same as the motion information of the current block, then this historical candidate motion information MV2 is removed. Then, the size of the historical candidate motion information list is checked. If the size of the list exceeds a preset size, the historical candidate motion information located at the head of the list is removed. Finally, the motion information of the current block is added to the end of the historical candidate motion information list.

However, in the process of constructing the above-mentioned historical candidate motion information list, the existing technology uses the historical candidate motion information list to be initialized when each slice starts encoding and decoding, which is not conducive to parallel encoding and decoding at the row level and CTU level. Moreover, this method updates the historical candidate motion information list table in each coding block. When the historical candidate motion information list is long, its construction and update take a long time.

Contents of the invention

In view of this, the present invention provides an inter-frame prediction method and device, a coding and decoding method using the method, and a coding and decoding device using the device. A first aspect of the present invention provides an inter-frame prediction method, which includes: initializing a historical candidate motion information list corresponding to the current coding tree unit, wherein the historical candidate motion information list includes N storage spaces, so The N storage spaces are used to store historical candidate motion information. The initialized historical candidate motion information list includes at least M vacant storage spaces, where M≤N, M and N are integers, and the current coding tree unit includes In a coding tree unit set (Slice) composed of multiple coding tree units, the current coding tree unit is not the first one in the coding tree unit set according to a predetermined processing order; all the coding tree units are processed in a predetermined order. The motion information at L positions in the spatial adjacent blocks of the current coding tree unit is added to the historical candidate motion information list, where M≤L≤N, and the L positions in the spatial adjacent blocks are based on the preset The rules are obtained; constructing the current candidate motion information list of the current coding tree unit or the current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; and according to the current coding tree unit A combination of the current candidate motion information list and the historical candidate motion information list, or a combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list, performs frame processing on the current coding tree unit or the current coding unit. prediction.

In this method, during the coding process of the current coding tree unit, the historical candidate motion information list is initialized, that is, an independent historical candidate motion information list corresponding to the current coding tree unit is constructed, thereby cutting off the coding of the coding tree unit. The dependency relationship caused by constructing the historical candidate motion information list during the process allows the coding tree unit to independently encode according to its own historical candidate motion information list. While having considerable coding efficiency, it is more conducive to the design of row-level and CTU-level encoding and decoding parallelism, through parallel processing, can greatly reduce encoding and decoding time without basically losing the encoding quality.

In an optional implementation combined with the first aspect of the present invention, the initializing the historical candidate motion information list corresponding to the current coding tree unit includes clearing the historical candidate motion information list such that M=N. This method allows the current coding tree unit to construct a new historical candidate motion information list to increase the accuracy of inter-frame prediction.

In another optional implementation combined with the first aspect of the present invention or based on the first aspect, M predetermined positions in adjacent blocks in the airspace are used as the M historical candidates. The source position of the motion information, specifically, the M positions in the adjacent block in the spatial domain are, and the first candidate motion information is obtained from the preset position in the adjacent block in the spatial domain to obtain the first candidate motion information. The position is the starting point, and the remaining M-1 candidate motion information is obtained at preset step intervals. In the process of constructing the historical candidate motion information list, in order to be consistent with the existing historical candidate motion information list construction method and have a relatively simplified algorithm, the motion vectors at the M positions are usually calculated from a starting position to a preset The motion vectors at M positions are obtained sequentially at intervals, where the preset interval can also be called a step size, and the step size can be fixed, for example, using 4 or 8 pixels as a unit; in addition, the The step size can also be changed, for example, different step sizes are set according to the size of the current coding tree unit.

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, the adding order of the motion information of the M position may be in a preset order, for example, in a clockwise manner. In order, taking the spatial adjacent block in the lower left corner of the current coding tree unit as the starting point, taking the spatial adjacent block in the upper right corner of the current coding tree unit as the end point, convert the spatial adjacent blocks at L positions in the spatial adjacent block The motion information is added to the historical candidate motion information list. This acquisition method aims to well match the processing order of adjacent blocks in the spatial domain and simplify the reading and writing logic of historical motion information. Therefore, a variety of different methods can be used. For example, in a counterclockwise manner, or by simultaneously reading in opposite directions starting from adjacent blocks of space at both endpoints.

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, before performing inter prediction on the current coding tree unit or the current coding unit, the method may also Combining the current candidate motion information list of the current coding tree unit and the historical candidate motion information list or combining the current candidate motion information list of the current coding unit and the historical candidate motion information list, which specifically may be Yes: Add the historical candidate motion information to the current candidate motion information list of the current coding tree unit or the current candidate motion information list of the previous coding unit, and then based on the current candidate motion information list of the current coding tree unit or the previous The inter-frame prediction is performed in the current candidate motion information list of the coding unit. This processing method can simplify the indexing operation of the motion information of the current motion coding tree unit or coding unit, and add the motion information in the historical candidate motion information list to the current candidate motion information list of the current motion coding tree unit or coding unit. Finally, the original candidate motion information and historical candidate motion information adopt a unified index sequence and index number, and there is no need to establish an additional current candidate motion information list index, which can effectively simplify the indexing process.

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, if the current coding tree unit needs to be further divided into coding units for coding, the method In addition to performing inter prediction on the current coding unit according to the acquired motion information, the method may further include updating the historical candidate motion information list based on the current coding unit motion information. This method can enable the historical candidate motion information list corresponding to the current coding tree unit to be continuously updated to improve the accuracy of inter-frame prediction.

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, the above-mentioned updating of the historical candidate motion information list can be divided into two situations, that is, if If the M positions are not filled, add the current coding unit motion information as historical motion information to the vacant storage space closest to the N-M position among the M positions in the historical candidate motion information list; or; If the M positions have been filled, the historical motion information that was first added to the historical candidate motion information list will be removed according to the first-in, first-out principle, and the remaining historical motion information will exceed the position of the removed historical motion information. After the shift, the current coding unit motion information is added to the end of the historical candidate motion information list as historical motion information, wherein the end of the historical candidate motion information list containing the latest added historical motion information is the historical candidate The end of the sports information list. This method provides flexibility in the application of the historical candidate motion information list, that is, the historical candidate motion information list can also be used for inter-frame prediction of the current block when it is not completely filled, and when the historical candidate motion information list is full, Under this condition, the motion information/motion vector of the current coding block can still be used to update the historical candidate motion information list.

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, when the historical candidate motion information list is filled, the motion information/motion vector of the current coding block remains The historical candidate motion information list may no longer be updated, that is, inter prediction is performed on another coding unit based on the same method as the current coding unit, wherein the another coding unit is located in the preset processing order. After the current coding unit and belonging to the coding tree unit with the current coding unit, the historical motion information list used in the inter-frame prediction of the other coding unit includes the history used in the inter-frame prediction of the current coding unit. Historical sports information in the sports information list. Specifically, if the M positions are not filled, the current coding unit motion information is added as historical motion information to the vacancy closest to the N-M position among the M positions in the historical candidate motion information list. within the storage space; if the M positions are filled, perform inter-frame prediction processing on the next coding unit based on the current candidate motion information list. This processing method allows parallel processing of coding blocks within the current coding tree unit,

In another optional implementation combined with the first aspect of the present invention or based on any optional implementation of the first aspect, if the historical candidate motion information list is not filled after traversing the spatial adjacent image blocks, In this case, or if the current coding tree unit is located at the top of a frame of image, or if the current coding tree unit is located at the left of a frame of image, you can refer to any of the following methods to handle unfilled historical candidate motion information lists. part. Method 1: No more motion information from other sources is filled, and the motion information of the current coding unit obtained when encoding and decoding the current coding unit in the current coding tree unit is added to the historical candidate motion information as the historical candidate motion information. List. Method 2: Fill in the motion information of the coding block at a preset non-nearby position in the current coding tree unit spatial domain. The preset non-adjacent location can be a fixed interval from the adjacent location, or it can be a preset template. Method 3: Fill in the coding block time domain motion information from the preset position corresponding to the current coding tree unit in the reference frame and the corresponding position of the current coding tree unit adjacent to the coding block. The preset positions in the corresponding positions of the current coding tree unit may be extracted at fixed intervals, or may be extracted in a specific rule or order. The preset position in the corresponding position of the current coding tree unit adjacent to the coding block may be extracted in a specific order according to a specific rule. Method 4: Fill in the coding block time domain motion information from the preset position in the reference frame, the corresponding position of the current coding tree unit and the coding block corresponding to the preset non-adjacent position of the previous coding tree unit. The preset positions in the corresponding positions of the current coding tree unit may be extracted at fixed intervals, or may be extracted in a specific rule or order. The preset positions in the coding blocks corresponding to the preset non-adjacent positions of the current coding tree unit may be extracted in a specific order according to a specific rule. Method 5: Fill in the historical candidate motion information from the historical candidate motion information list of coding tree units adjacent to the current coding tree unit. Any of the above filling methods for the unfilled historical candidate information motion list can enrich the historical candidate motion information in the historical candidate motion information list, and can be used when there are insufficient adjacent blocks in the spatial domain or insufficient MVs of adjacent blocks in the spatial domain. In this case, the current candidate motion information list is supplemented so that the encoding and decoding gains brought by the historical candidate motion information list can be fully utilized.

A second aspect of the present invention provides a method for encoding using the inter-frame prediction of the first aspect of the present invention: it includes: performing inter-frame prediction on the current coding tree unit or coding unit based on the inter-frame prediction method of the first aspect of the present invention. Predict to obtain an inter-frame prediction image; subtract the current coding tree unit or the original image of the current coding unit from the obtained inter-frame prediction image to obtain a residual image; index the residual image and the motion information Encode to form a code stream. Wherein, it should be noted that as a decoding method, during the inter-frame prediction process, it is necessary to obtain the current coding tree unit from a combination of the historical candidate motion information list and the current candidate motion information list or Motion information of the current coding unit, according to the motion information of the current coding tree unit or the current coding unit, perform inter-frame prediction on the current coding tree unit or the current coding unit to obtain an inter-frame prediction image, wherein the motion information is obtained Specifically, the code stream can be parsed, the current coding tree unit or the motion information index corresponding to the current coding unit, and the current candidate motion information list is obtained from the combination of the historical candidate motion information list and the current candidate motion information list. Coding tree unit or motion information of the current coding unit.

A third aspect of the present invention provides a method for encoding using the inter-frame prediction of the first aspect of the present invention: it includes: performing inter-frame prediction on the current coding tree unit or coding unit based on the inter-frame prediction method of the first aspect of the present invention. Predict to obtain an inter-frame prediction image, subtract the current coding tree unit or the original image of the current coding unit from the obtained inter-frame prediction image to obtain a residual image; encode the residual image to form a code stream. Wherein, in the process of obtaining the inter-frame prediction image based on the inter-frame prediction method of the first aspect, the method further includes: obtaining the current coding tree from a combination of the historical candidate motion information list and the current candidate motion information list. The motion information of the unit or the current coding unit, and the motion information index of the motion information; according to the motion information of the current coding tree unit or the current coding unit, perform inter-frame prediction on the current coding tree unit or the current coding unit to obtain inter-predicted images; and encoding the motion index.

Compared with the existing technology, the above encoding/decoding method adopts the update of the historical candidate motion information list at the coding tree level, allowing parallel encoding and decoding at the row level and CTU level, which can effectively reduce the encoding time.

In addition, the present invention also provides inter-array prediction devices, encoding devices and encoding equipment corresponding to the first, second and third aspects of the present invention, and a decoding device and decoding equipment corresponding to the third aspect of the present invention.

In addition, the present invention also provides an inter-array prediction device, an encoding device and an encoding device corresponding to the first, second and third aspects of the invention, which include a digital processor and a memory, and executable programs are stored in the memory. Instruction set, the digital processor reads the instruction set stored in the memory to implement the method provided by the first, second or third aspect of the present invention.

Description of drawings

Figure 1 is a block diagram of an example of a video encoding system for implementing an embodiment of the present invention;

Figure 2 is a block diagram of an example structure of a video encoder used to implement an embodiment of the present invention;

Figure 3 is a block diagram of an example structure of a video decoder used to implement an embodiment of the present invention;

FIG. 4 shows a diagram including the encoder 20 of FIG. 2 and the decoder 30 of FIG. 3

Figure 5 is a block diagram showing another example of an encoding device or a decoding device;

Figure 6 is a schematic diagram of the positions of adjacent blocks in the spatial domain and adjacent blocks in the time domain of the current block;

Figure 7 is a schematic diagram of historical candidate motion information added to the current candidate motion information list;

Figure 8 is a schematic diagram of the construction of a historical candidate motion information list;

Figure 9 shows the motion information of adjacent image blocks in the left and upper spatial areas of the front coding tree unit;

Figure 10 shows the acquisition process of two spatial candidate motion information A and B.

Figure 11 is a flowchart of an example operation of a video encoder implementing the inter-frame prediction method of the present invention according to an embodiment;

Figure 12 is a flow chart of a method for decoding by a video decoder based on the inter-array prediction method of Figure 11 according to another embodiment;

Figure 13 is a flow chart of a method for encoding by a video decoder based on the inter-array prediction method of Figure 11 according to another embodiment;

FIG. 14 is a schematic diagram of an inter-frame prediction device for implementing the method described in FIG. 11 according to another embodiment;

FIG. 15 is a schematic diagram of an inter-frame prediction device for implementing the method described in FIG. 12 provided according to another embodiment;

Figure 16 is a schematic diagram of an inter-frame prediction device provided according to another embodiment to implement the method described in Figure 13;

FIG. 17 is a schematic diagram of a device for implementing any of the methods in FIGS. 11 to 13 provided according to another embodiment.

In the following, if there is no specific note regarding the same reference signs, the same reference signs refer to the same or at least functionally equivalent features.

Detailed ways

Hereinafter, specific embodiments of the present invention and application examples using the specific embodiments of the present invention will be described with reference to the accompanying drawings.

It should be understood that the embodiments of the present invention are not limited to the examples listed herein, but may be used in other aspects and may include structural or logical changes not shown in the drawings.

For example, it should be understood that content contained in inter prediction methods herein may equally apply to corresponding devices or systems for performing said methods, and vice versa. For example, when one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, to perform the one or more described method steps (for example, one unit performs one or more steps , or a plurality of units, each of which performs one or more of a plurality of steps), even if such unit or units are not explicitly depicted or illustrated in the drawings. On the other hand, for example, if a specific apparatus is described based on one or more units such as functional units, the corresponding method may include a step to perform the functionality of the one or more units (e.g., a step to perform the functionality of the one or more units) functionality, or a plurality of steps, each of which performs the functionality of one or more of a plurality of units), even if such one or more steps are not explicitly depicted or illustrated in the drawings. Further, it should be understood that features of various exemplary embodiments and/or aspects described herein may be combined with each other unless expressly stated otherwise.

Video coding generally refers to the processing of sequences of pictures that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" may be used as synonyms. Video coding as used in this application (or this disclosure) means video encoding or video decoding. Video encoding is performed on the source side and typically involves processing (eg, by compressing) the original video picture to reduce the amount of data required to represent the video picture (and thereby store and/or transmit it more efficiently). Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video picture. The "encoding" of video pictures (or pictures in general, as will be explained below) referred to in the embodiments shall be understood to refer to the "encoding" or "decoding" of a video sequence. The combination of encoding part and decoding part is also called codec (encoding and decoding).

In the case of lossless video encoding, the original video picture can be reconstructed, that is, the reconstructed video picture has the same quality as the original video picture (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed by e.g. quantization to reduce the amount of data required to represent the video picture, while the decoder side cannot fully reconstruct the video picture, i.e. the quality of the reconstructed video picture is compared to the original video picture of lower or poorer quality.

Several video coding standards from H.261 to H.265 belong to "lossy hybrid video coding" (that is, combining spatial and temporal prediction in the sample domain with 2D transform coding in the transform domain for applying quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, which are usually encoded at the block level. In other words, the encoder side usually processes i.e. encodes the video at the block (video block) level, e.g. by spatial (intra-picture) prediction and temporal (inter-picture) prediction to generate prediction blocks, starting from the current block (currently processed or to be processed). The processed block) subtracts the prediction block to obtain the residual block, transforms the residual block in the transform domain and quantizes the residual block to reduce the amount of data to be transmitted (compressed), while the decoder side will perform the inverse processing relative to the encoder Partially applied to encoded or compressed blocks to reconstruct the current block for representation. Additionally, the encoder-decoder processing loop is replicated such that the encoder and decoder generate the same predictions (eg, intra- and inter-prediction) and/or reconstructions for processing, ie encoding, subsequent blocks.

As used herein, the term "block" may be a portion of a picture or frame. For ease of description, refer to VVC or the Joint Collaboration Team on Video by the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Motion Picture Experts Group (MPEG). High-Efficiency Video Coding (HEVC) developed by JCT-VC (JCT-VC) describes the embodiment of the present invention. Persons of ordinary skill in the art understand that embodiments of the present invention are not limited to HEVC or VVC. Can refer to CU, PU and TU. In HEVC, a CTU is split into multiple CUs by using a quadtree structure represented as a coding tree. The decision is made at the CU level whether to code a picture region using inter-picture (temporal) or intra-picture (spatial) prediction. Each CU can be further split into one, two or four PUs depending on the PU split type. The same prediction process is applied within a PU and the relevant information is transferred to the decoder on a PU basis. After obtaining the residual block by applying a prediction process based on the PU split type, the CU may be split into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU. In the latest development of video compression technology, quad-tree and binary tree (QTBT) are used to segment frames to segment coding blocks. In QTBT block structure, CU can be in square or rectangular shape. In VVC, a coding tree unit (CTU) is first divided by a quadtree structure. Quad leaf nodes are further divided by a binary tree structure. The binary leaf node is called a coding unit (CU), and the segmentation is used for prediction and transformation processing without any other segmentation. This means that CU, PU and TU have the same block size in the QTBT encoding block structure. At the same time, it is also proposed to use multiple partitioning, such as ternary tree partitioning, together with the QTBT block structure.

Embodiments of the encoder 20, the decoder 30 and the encoding system 10 are described below based on Figures 1 to 3 (before embodiments of the invention are described in more detail based on Figure 6).

1 is a conceptual or schematic block diagram of an exemplary encoding system 10, such as a video encoding system 10, that may utilize the techniques of the present disclosure. The encoder 20 (eg, video encoder 20 ) and decoder 30 (eg, video decoder 30 ) representations of the video encoding system 10 may be used to perform fusion candidate list construction in accordance with the various examples described in this application, and a device instance of the technology for encoding and decoding based on the fused selection list. As shown in Figure 1, encoding system 10 includes a source device 12 for providing encoded data 13, eg, encoded pictures 13, to a destination device 14, eg, decoding the encoded data 13.

The source device 12 includes an encoder 20 and, optionally, a picture source 16 , such as a pre-processing unit 18 such as a picture pre-processing unit 18 , and a communication interface or unit 22 .

Picture source 16 may include or be any class of picture capture device, used for example to capture real world pictures, and/or any class of pictures or comments (for screen content encoding, some text on the screen is also considered to be a picture to be encoded or part of an image) generating device, for example, a computer graphics processor for generating computer animated pictures, or for acquiring and/or providing real world pictures, computer animated pictures (for example, screen content, virtual reality (VR) images), and/or any combination thereof (e.g., augmented reality (AR) images).

A (digital) picture is or can be viewed as a two-dimensional array or matrix of sample points with brightness values. The sampling points in the array can also be called pixels (short for picture element) or pels. The number of sample points in the horizontal and vertical directions (or axes) of an array or image defines the size and/or resolution of the image. In order to represent color, three color components are usually used, that is, the picture can be represented as or contain three sample arrays. In RBG format or color space, the picture includes corresponding red, green and blue sample arrays. However, in video coding, each pixel is usually represented in a luminance/chrominance format or color space, for example, YCbCr, including a luminance component indicated by Y (sometimes also indicated by L) and two chrominances indicated by Cb and Cr Portion. The luminance (abbreviated luma) component Y represents brightness or gray level intensity (for example, in a grayscale picture they are the same), while the two chrominance (abbreviated chroma) components Cb and Cr represent chrominance or color information components . Correspondingly, a picture in YCbCr format includes a brightness sample array of brightness sample values (Y), and two chroma sample arrays of chroma values (Cb and Cr). Pictures in RGB format can be converted or transformed into YCbCr format and vice versa, this process is also called color transformation or conversion. If the picture is black and white, the picture may only include an array of luminance samples.

Picture source 16 (eg, video source 16) may be, for example, a camera for capturing pictures, a memory such as a picture memory, including or storing previously captured or generated pictures, and/or any class (internal) of acquiring or receiving pictures. or external) interface. The camera may be, for example, local or an integrated camera integrated in the source device, and the memory may be local or, for example, an integrated memory integrated in the source device. The interface may be, for example, an external interface that receives pictures from an external video source, such as an external picture capture device, such as a camera, an external memory, or an external picture generation device, such as an external computer graphics processor, computer or server. The interface may be any class of interface according to any proprietary or standardized interface protocol, such as wired or wireless interfaces, optical interfaces. The interface for obtaining the picture data 17 may be the same interface as the communication interface 22 or a part of the communication interface 22 .

Different from the preprocessing unit 18 and the processed pictures or picture data performed by the preprocessing unit 18 , the pictures or picture data 17 (eg, video data 16 ) may also be referred to as original pictures or original picture data 17 .

The preprocessing unit 18 is configured to receive (original) picture data 17 and perform preprocessing on the picture data 17 to obtain preprocessed pictures 19 or preprocessed picture data 19 . For example, the preprocessing performed by preprocessing unit 18 may include trimming, color format conversion (eg, from RGB to YCbCr), color grading, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

An encoder 20 (eg video encoder 20) is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (details will be described further below, eg based on Figure 2 or Figures 4, 5). In one example, the encoder 20 may select the most suitable prediction method for the current block (current image block to be encoded) based on rate-distortion cost evaluation, for example, using intra prediction or inter prediction. When the encoder 20 chooses to use the inter prediction mode, the encoder may perform the following method to perform inter prediction on the current block. That is, the encoder 20 first initializes the historical candidates corresponding to the current coding tree unit. A motion information list, wherein the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, and the initialized historical candidate motion information list includes at least M vacant storage space, the M≤N, M and N are integers, the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, and the current coding tree unit is not the coding tree unit. The first one in the tree unit set according to the predetermined processing order; then, according to the predetermined order, the motion information at L positions in the spatial domain adjacent blocks of the current coding tree unit is added to the historical candidate motion information list , where M≤L≤N, the L positions in the spatial adjacent blocks are obtained according to preset rules; construct the current candidate motion information list of the current coding tree unit or the current coding unit, wherein the coding unit It is obtained by dividing the coding tree unit; finally, according to the combination of the current candidate motion information list of the current coding tree unit or the current coding unit and the historical candidate motion information list, the current coding tree unit or the current coding unit is Perform inter-frame prediction. Based on the above method, the encoder 20 can obtain more diverse motion information during the encoding process to predict the current block. In the application process of the historical candidate motion information list, there is no need to wait for the previous coding tree unit to be encoded. Only then starts processing the last coding tree unit, which can improve the parallel processing capability of coding tree units in inter-frame prediction and improve coding efficiency.

The communication interface 22 of the source device 12 may be used to receive and transmit the encoded picture data 21 to other devices, such as the destination device 14 or any other device, for storage or direct reconstruction, or for storing the encoded picture data 21 in a corresponding manner. The encoded picture data 21 is processed before encoding the data 13 and/or transmitting the encoded data 13 to other devices, such as the destination device 14 or any other device for decoding or storage.

Destination device 14 includes a decoder 30 (eg, video decoder 30) and, optionally, a communication interface or unit 28, a post-processing unit 32, and a display device 34.

The communication interface 28 of the destination device 14 is used, for example, to receive the encoded picture data 21 or the encoded data 13 directly from the source device 12 or any other source, such as a storage device, such as a storage device for encoded picture data. equipment.

Communication interface 22 and communication interface 28 may be used to transmit or receive encoded picture data 21 or encoded data 13 directly via a direct communication link between source device 12 and destination device 14 or over any type of network. The link may be, for example, a direct wired or wireless connection, any type of network, such as a wired or wireless network or any combination thereof, or any type of private network or public network, or any combination thereof.

The communication interface 22 may, for example, be used to encapsulate the encoded picture data 21 into a suitable format, such as a packet, for transmission over a communication link or communication network.

The communication interface 28 forming a corresponding part of the communication interface 22 may, for example, be used to depackage the encoded data 13 to obtain the encoded picture data 21 .

Both communication interface 22 and communication interface 28 may be configured as a one-way communication interface, as indicated by the arrow from source device 12 to destination device 14 for encoded picture data 13 in FIG. 1 , or as a bi-directional communication interface, and May be used, for example, to send and receive messages to establish connections, confirm and exchange any other information related to the communication link and/or data transmission, such as encoded picture data transmission.

Decoder 30 is configured to receive encoded picture data 21 and provide decoded picture data 31 or decoded pictures 31 (details will be described further below, eg based on Figure 3 or Figure 5). In one example, the decoder 30 may be used to decode the data encoded by the encoder. Specifically, the decoder 30 may parse the code stream to obtain the fusion candidate index; and obtain the corresponding fusion candidate list from the fusion candidate list according to the fusion candidate index. fusion candidate and use the fusion candidate as the motion information of the current block; perform inter-frame prediction on the current block according to the motion information of the current block to obtain the predicted image of the current block; obtain the prediction image of the current block Residual image: Add the prediction image of the current block and the residual image of the current block to obtain the reconstructed image of the current block.

Post-processor 32 of destination device 14 is used to post-process decoded picture data 31 (also referred to as reconstructed picture data), e.g., decoded picture 131, to obtain post-processed picture data 33, e.g., post-processed Picture 33. Post-processing performed by post-processing unit 32 may include, for example, color format conversion (eg, from YCbCr to RGB), color correction, trimming or resampling, or any other processing, for example, to prepare decoded picture data 31 for processing by Display device 34 displays.

The display device 34 of the destination device 14 is used to receive the post-processed picture data 33 for displaying the picture to a user or viewer, for example. Display device 34 may be or may include any type of display for presenting reconstructed pictures, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital Digital light processor (DLP) or other display of any kind.

Although FIG. 1 shows source device 12 and destination device 14 as separate devices, device embodiments may also include both source device 12 and destination device 14 or the functionality of both, ie, source device 12 or the corresponding functionality and the destination device 14 or corresponding functionality. In such embodiments, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and/or software, or using separate hardware and/or software, or any combination thereof. .

Based on the description, it will be obvious to a person skilled in the art that the existence and (accurate) division of the functionality of the different units or the functionality of the source device 12 and/or the destination device 14 shown in Figure 1 may vary depending on the actual device and application.

Both encoder 20 (e.g., video encoder 20) and decoder 30 (e.g., video decoder 30) may be implemented as any of a variety of suitable circuits, e.g., one or more microprocessors, digital signal processors (digital signal processor, DSP), application-specific integrated circuit (application-specific integrated circuit, ASIC), field-programmable gate array (field-programmable gate array, FPGA), discrete logic, hardware, or any combination thereof. If the techniques are implemented partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium, and may execute the instructions in hardware using one or more processors to perform the techniques of the present disclosure. . Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be regarded as one or more processors. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as a combined encoder/decoder in the corresponding device. part of the codec.

Source device 12 may be referred to as a video encoding device or video encoding device. Destination device 14 may be referred to as a video decoding device or video decoding device. Source device 12 and destination device 14 may be examples of video encoding devices or video encoding apparatuses.

Source device 12 and destination device 14 may include any of a variety of devices, including any class of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a video camera, a desktop Computers, set-top boxes, televisions, display devices, digital media players, video game consoles, video streaming devices (such as content serving servers or content distribution servers), broadcast receiver devices, broadcast transmitter devices, etc., and can be used without Or use any kind of operating system.

In some cases, source device 12 and destination device 14 may be equipped for wireless communications. Accordingly, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video encoding system 10 shown in FIG. 1 is only an example, and the techniques of this application may be applicable to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any data communication between encoding and decoding devices. . In other instances, data can be retrieved from local storage, streamed over a network, etc. The video encoding device may encode the data and store the data to memory, and/or the video decoding device may retrieve the data from memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but merely encode data to and/or retrieve data from memory and decode data.

It should be understood that for each of the examples described above with reference to video encoder 20, video decoder 30 may be used to perform the reverse process. Regarding signaling syntax elements, video decoder 30 may be configured to receive and parse such syntax elements and decode related video data accordingly. In some examples of the present invention, video encoder 20 may combine one or more syntax elements that define the specific position of the fusion candidate in the fusion candidate list and the syntax element of the inter-frame coding type of the spatially non-adjacent block of the current block. Entropy encoding into an encoded video bitstream. In such instances, video decoder 30 may parse such syntax elements and decode the associated video data accordingly.

Encoders & Encoding Methods

2 illustrates a schematic/conceptual block diagram of an example of video encoder 20 for implementing the techniques of this disclosure. In the example of FIG. 2, video encoder 20 includes residual calculation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filtering processor unit 220, decoded picture buffer (DPB) 230, prediction processing unit 260, and entropy encoding unit 270. Prediction processing unit 260 may include inter prediction unit 244, intra prediction unit 254, and mode selection unit 262. Inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 shown in FIG. 2 may also be called a hybrid video encoder or a video encoder according to a hybrid video codec.

For example, residual calculation unit 204, transform processing unit 206, quantization unit 208, prediction processing unit 260, and entropy encoding unit 270 form the forward signal path of encoder 20, while, for example, inverse quantization unit 210, inverse transform processing unit 212, re- The structural unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (decoded picture buffer, DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder, where the backward signal path of the encoder corresponds to in the signal path of the decoder (see decoder 30 in Figure 3).

The encoder 20 receives, for example via an input 202, a picture 201 or a block 203 of a picture 201, for example a picture in a sequence of pictures forming a video or video sequence. The picture block 203 can also be called the current picture block or the picture block to be encoded, and the picture 201 can be called the current picture or the picture to be encoded (especially when the current picture is distinguished from other pictures in video encoding, other pictures such as the same video sequence That is, also including previously encoded and/or decoded pictures in the video sequence of the current picture).

segmentation

Embodiments of the encoder 20 may include a segmentation unit (not shown in Figure 2) for segmenting the picture 201 into a plurality of blocks, such as block 203, typically into a plurality of non-overlapping blocks. Split units can be used to use the same block size for all pictures in a video sequence and the corresponding raster that defines the block size, or to change the block size between pictures or subsets or groups of pictures and split each picture into corresponding block. The "Block partitioning structure for next generation video coding" proposed by J.An et al. was introduced in VVC (International Telecommunication Union, COM16-C966, September 2015, hereinafter referred to as The quad-tree-binary-tree (QTBT) partitioning technique proposed in "VCEG Recommendation COM16-C966"). Simulations have shown that the proposed QTBT structure is more efficient than the quad-tree structure used in HEVC Efficient. Furthermore, in QTBT, a CU can have a square or rectangular shape. As shown in Figure 3, the coding tree unit (CTU) is first divided through a quad-tree structure. The quad-leaf nodes can be further divided through a binary tree structure Partitioning. There are two types of partitioning in binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. In each case, a node is divided by bisecting it horizontally or vertically along the middle. Binary leaf nodes are called coding units (coding unit, CU), and prediction and transform processing respectively without any further partitioning. This means that CU, PU and TU have the same block size in the QTBT coding block structure. CU is sometimes composed of different color components It is composed of coding block (CB). For example, in the case of P and B strips in the 4:2:0 chroma format, a CU contains one luma CB and two chroma CBs, and the CU is sometimes composed of A single component CB is constituted, for example, in the case of an I-strip, a CU contains only one luma CB or only two chroma CBs.

In addition, a block partitioning structure named multi-type-tree (MTT) is proposed in US Patent Application Publication No. 20170208336 to replace the CU structure based on QT, BT and/or QTBT. The MTT partition structure is still a recursive tree structure. In MTT, multiple different partitioning structures (eg three or more) are used. For example, according to the MTT technique, three or more different partitioning structures may be used for each corresponding non-leaf node of the tree structure at each depth of the tree structure. The depth of a node in a tree structure may refer to the length of the path (eg, the number of divisions) from the node to the root of the tree structure. The partitioning structure may generally refer to how many different blocks a block can be divided into. The partitioning structure may be a quadtree partitioning structure that can divide a block into four blocks, a binary tree partitioning structure that can divide a block into two blocks, or a ternary tree partitioning structure that can divide a block into three blocks. In addition, a ternary tree The partition structure may not divide the blocks through the center. A partition structure can have multiple different partition types. The partitioning type may additionally define how the block is divided, including symmetrical or asymmetrical partitioning, even or uneven partitioning, and/or horizontal or vertical partitioning.

In MTT, at each depth of the tree structure, the encoder 100 can be used to further partition the subtree using a specific partition type of one of three additional partition structures. For example, the encoder 100 may be used to determine specific partition types from QT, BT, triple-tree (TT), and other partition structures. In one example, the QT partition structure may include a square quadtree or a rectangular quadtree partition type. Encoder 100 may partition a square block using square quadtree partitioning by bisecting the block horizontally and vertically along the center into four equally sized square blocks. Likewise, encoder 100 may use rectangular quadtree partitioning to partition a rectangular (eg, non-square) block by bisecting the rectangular block horizontally and vertically along the center into four equally sized rectangular blocks.

The BT partition structure may include at least one of a horizontal symmetric binary tree, a vertical symmetric binary tree, a horizontal asymmetric binary tree or a vertical asymmetric binary tree partition type. For the horizontally symmetric binary tree partitioning type, the encoder 100 may be used to bisect the block horizontally along the center of the block into two symmetric blocks of the same size. For the vertically symmetric binary tree partitioning type, the encoder 100 may be used to bisect the block vertically along the center of the block into two symmetric blocks of the same size. For the horizontal asymmetric binary tree partitioning type, the encoder 100 may be used to horizontally divide the block into two blocks of different sizes. For example, one block can be 1/4 the size of the parent block, while another block can be 3/4 the size of the parent block, similar to the PART_2N×nU or PART_2N×nD partitioning types. For the vertical asymmetric binary tree partitioning type, the encoder 100 may be used to vertically divide the block into two blocks of different sizes. For example, one block can be 1/4 of the parent block size, while another block can be 3/4 of the parent block size, similar to the PART_nL×2N or PART_nR×2N partitioning types. In other examples, an asymmetric binary tree partitioning type may divide a parent block into parts of different sizes. For example, one child block can be 3/8 of the parent block, while another child block can be 5/8 of the parent block. Of course, such division types may be vertical or horizontal.

TT partition structures differ from types of QT or BT structures in that TT partition structures do not divide blocks along the center. The central areas of the blocks are kept together in the same sub-block. Unlike QT, which produces four blocks, or binary tree, which produces two blocks, partitioning according to the TT partition structure produces three blocks. Example division types according to the TT division structure include symmetric division types (two types, horizontal and vertical) and asymmetric division types (two types, horizontal and vertical). Furthermore, the symmetric partition type according to the TT partition structure can be unequal/uneven or equal/uniform. The asymmetric partition type according to the TT partition structure is uneven/uneven. In one example, the TT partition structure may include at least one of the following partition types: horizontal equal/uniform symmetric ternary tree, vertical equal/uniform symmetric ternary tree, horizontal uneven/uneven symmetric ternary tree, vertical uneven/ Uneven symmetric ternary tree, horizontal uneven/uneven asymmetric ternary tree, or vertical uneven/uneven asymmetric ternary tree partition type.

In general, the unequal/uneven symmetric ternary tree partitioning type is a partitioning type that is symmetrical about the center line of the block but where at least one of the resulting three blocks is not the same size as the other two. A preferred example is where the side pieces are 1/4 the size of the block and the center piece is 1/2 the size of the block. The equal/uniformly symmetric ternary tree partitioning type is a partitioning type that is symmetrical around the center line of the blocks and the resulting blocks are all the same size. Such divisions are possible if the block height or width - depending on the vertical or horizontal division - is an integral multiple of 3. The unequal/uneven asymmetric ternary tree partitioning type is a partitioning type that is not symmetrical about the center line of the blocks and where at least one of the resulting blocks is not of the same size as the other two.

In one example, prediction processing unit 260 of video encoder 20 may be used to perform any combination of the above-described segmentation techniques.

Like picture 201 , block 203 is also or can be viewed as a two-dimensional array or matrix of sample points with brightness values (sampling values), although its size is smaller than that of picture 201 . In other words, block 203 may include, for example, one sample array (e.g., a luma array in the case of a black and white picture 201) or three sample arrays (e.g., a luma array and two chrominance arrays in the case of a color picture) or based on Arrays of any other number and/or category of color formats to apply. The number of sample points in the horizontal and vertical directions (or axes) of the block 203 defines the size of the block 203 .

The encoder 20 as shown in Figure 2 is used to encode the picture 201 block by block, eg, performing encoding and prediction for each block 203.

Residual calculation

The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (additional details of the prediction block 265 are provided below), for example, by subtracting the prediction from the sample value of the picture block 203 on a sample-by-sample (pixel-by-pixel) basis. sample values of block 265 to obtain the residual block 205 in the sample domain.

transform

The transform processing unit 206 is configured to apply a transform such as a discrete cosine transform (DCT) or a discrete sine transform (DST) on the sample values of the residual block 205 to obtain the transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.

Transform processing unit 206 may be operable to apply an integer approximation of DCT/DST, such as the transform specified for HEVC/H.265. In contrast to the orthogonal DCT transform, this integer approximation is usually scaled by some factor. In order to maintain the norm of the forward and inverse transformed residual blocks, additional scaling factors are applied as part of the transformation process. The scaling factor is usually chosen based on certain constraints, e.g. the scaling factor is a power of 2 used for the shift operation, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor may be specified on the decoder 30 side for the inverse transform (and for the corresponding inverse transform on the encoder 20 side for example by the inverse transform processing unit 212), and accordingly, the inverse transform may be specified at the encoder 30 On the side 20, a corresponding scaling factor is specified for the forward transformation through the transformation processing unit 206.

Quantify

Quantization unit 208 is used to quantize transform coefficients 207, such as by applying scalar quantization or vector quantization, to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all transform coefficients 207. For example, n-bit transform coefficients may be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, with scalar quantization, different scales can be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. The appropriate quantization step size can be indicated by a quantization parameter (QP). For example, the quantization parameter may be an index into a predefined set of suitable quantization steps. For example, a smaller quantization parameter can correspond to fine quantization (smaller quantization step size), and a larger quantization parameter can correspond to coarse quantization (larger quantization step size), and vice versa. Quantization may include dividing by the quantization step size and corresponding quantization or inverse quantization, such as performed by inverse quantization 210, or may include multiplying by the quantization step size. Embodiments according to some standards such as HEVC may use quantization parameters to determine the quantization step size. In general, the quantization step size can be calculated based on the quantization parameters using a fixed-point approximation of the equation involving division. Additional scaling factors can be introduced for quantization and inverse quantization to recover the norm of the residual block that may be modified due to the scaling used in the fixed-point approximation of the equations for quantization step size and quantization parameters. In one example implementation, inverse transformation and inverse quantization scaling may be combined. Alternatively, one can use a custom quantization table and signal it from the encoder to the decoder in e.g. the bitstream. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 on the quantized coefficients to obtain the inverse quantized coefficients 211 , for example, applying the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208 inverse quantization scheme. The inverse quantized coefficients 211 may also be called inverse quantized residual coefficients 211, corresponding to the transform coefficients 207, although the loss due to quantization is generally not the same as the transform coefficients.

The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain in the sample domain Inverse transform block 213. The inverse transform block 213 may also be referred to as the inverse transform inverse quantized block 213 or the inverse transform residual block 213.

The reconstruction unit 214 (eg, summer 214) is used to add the inverse transform block 213 (ie, the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain, eg, The sample values of reconstructed residual block 213 are added to the sample values of prediction block 265 .

Optionally, a buffer unit 216 (or simply "buffer" 216), such as line buffer 216, is used to buffer or store reconstructed block 215 and corresponding sample values, for example, intra prediction. In other embodiments, the encoder may be used to perform any class of estimation and/or prediction using the unfiltered reconstructed blocks and/or corresponding sample values stored in buffer unit 216, such as intra predict.

For example, embodiments of encoder 20 may be configured such that buffer unit 216 is used not only to store reconstructed block 215 for intra prediction 254, but also for loop filter unit 220 (not shown in FIG. 2 out), and/or, for example, causing buffer unit 216 and decoded picture buffer unit 230 to form one buffer. Other embodiments may be used for using filtered block 221 and/or blocks or samples from decoded picture buffer 230 (neither shown in FIG. 2 ) as the input or basis for intra prediction 254 .

The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain the filtered block 221, thereby smoothly performing pixel conversion or improving video quality. Loop filter unit 220 is intended to represent one or more loop filters, such as deblocking filters, sample-adaptive offset (SAO) filters, or other filters, such as bilateral filters, automatic Adaptive loop filter (ALF), or sharpening or smoothing filter, or collaborative filter. Although loop filter unit 220 is shown in FIG. 2 as an in-loop filter, in other configurations, loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed encoding block after the loop filter unit 220 performs a filtering operation on the reconstructed encoding block.

Embodiments of encoder 20 (correspondingly, loop filter unit 220) may be configured to output loop filter parameters (e.g., sample adaptive offset information), e.g., directly or by entropy encoding unit 270 or any other The entropy encoding unit entropy encodes the output, for example, so that the decoder 30 can receive and apply the same loop filter parameters for decoding.

Decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use by video encoder 20 in encoding video data. DPB 230 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM) (resistive RAM, RRAM)) or other types of memory devices. DPB 230 and buffer 216 may be provided by the same memory device or separate memory devices. In one example, decoded picture buffer (DPB) 230 is used to store filtered block 221 . The decoded picture buffer 230 may further be used to store other previous filtered blocks of the same current picture or different pictures such as a previous reconstructed picture, such as the previously reconstructed and filtered block 221, and may provide a complete previous filtered block. Reconstruction is a decoded picture (and corresponding reference blocks and samples) and/or a partially reconstructed current picture (and corresponding reference blocks and samples), for example for inter prediction. In one example, if reconstructed block 215 is reconstructed without in-loop filtering, decoded picture buffer (DPB) 230 is used to store reconstructed block 215 .

Prediction processing unit 260, also referred to as block prediction processing unit 260, for receiving or retrieving block 203 (current block 203 of current picture 201) and reconstructed picture data, such as a reference to the same (current) picture from buffer 216 samples and/or reference picture data 231 of one or more previously decoded pictures from the decoded picture buffer 230, and for processing such data for prediction, i.e. providing an inter-predicted block 245 or an intra-predicted block 245. Prediction block 265 of prediction block 255.

Mode selection unit 262 may be used to select a prediction mode (eg, intra or inter prediction mode) and/or corresponding prediction block 245 or 255 used as prediction block 265 to calculate residual block 205 and reconstruct the reconstructed block 215.

Embodiments of mode selection unit 262 may be used to select a prediction mode (e.g., from those supported by prediction processing unit 260) that provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provide minimal signaling overhead (minimum signaling overhead means better compression in transmission or storage), or consider or balance both at the same time. The mode selection unit 262 may be configured to determine a prediction mode based on rate distortion optimization (RDO), that is, select a prediction mode that provides minimum rate distortion optimization, or select a prediction mode whose associated rate distortion at least meets the prediction mode selection criteria. .

Prediction processing performed (e.g., by prediction processing unit 260) and mode selection performed (e.g., by mode selection unit 262) by an example of encoder 20 will be explained in detail below.

As mentioned above, the encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) set of prediction modes. The set of prediction modes may include, for example, intra prediction modes and/or inter prediction modes.

The intra prediction mode set may include 35 different intra prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in H.265, or may include 67 Different intra prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in the developing H.266.

The set of (possible) inter-prediction modes depends on the available reference pictures (i.e. at least part of the decoded pictures stored in DBP 230, for example) and other inter-prediction parameters, e.g. on whether the entire reference picture or only the reference picture is used A portion of the picture, such as a search window area surrounding the area of the current block, is searched for the best matching reference block, and/or e.g. depending on whether pixel interpolation such as half-pixel and/or quarter-pixel interpolation is applied.

In addition to the above prediction modes, skip mode and/or direct mode can also be applied.

Prediction processing unit 260 may further be used to partition block 203 into smaller block partitions or sub-blocks, for example, by iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning, or Triple-tree (TT) partitioning, or any combination thereof, and for performing prediction, for example, for each of the block partitions or sub-blocks, wherein the mode selection includes selecting the tree structure of the partitioned block 203 and selecting the tree structure to be applied to the block. Prediction mode for each of the partitions or sub-blocks.

The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is operable to receive or obtain the picture block 203 (the current picture block 203 of the current picture 201) and the decoded picture 231, or at least one or more previously reconstructed blocks, eg, one or more other/different previously reconstructed blocks. The reconstructed blocks of picture 231 are decoded for motion estimation. For example, the video sequence may comprise the current picture and the previously decoded picture 31, or in other words, the current picture and the previously decoded picture 31 may be part of a sequence of pictures forming the video sequence, or form the sequence of pictures. The construction of the fusion candidate list in this application can be implemented through the motion estimation module.

For example, encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different pictures in a plurality of other pictures and provide a reference picture (or reference picture index) to the motion estimation unit (not shown in FIG. 2 ) ...) and/or provide the offset (spatial offset) between the position of the reference block (X, Y coordinates) and the position of the current block as an inter prediction parameter. This offset is also called a motion vector (MV).

The motion compensation unit is configured to obtain, for example, receive inter prediction parameters and perform inter prediction based on or using the inter prediction parameters to obtain the inter prediction block 245 . Motion compensation performed by a motion compensation unit (not shown in Figure 2) may involve fetching or generating prediction blocks based on motion/block vectors determined by motion estimation (possibly performing interpolation to sub-pixel accuracy). Interpolation filtering can generate additional pixel samples from known pixel samples, potentially increasing the number of candidate prediction blocks available for encoding a picture block. Upon receiving the motion vector for the PU of the current picture block, motion compensation unit 246 may locate the prediction block to which the motion vector points in a reference picture list. Motion compensation unit 246 may also generate syntax elements associated with the blocks and the video slice for use by video decoder 30 when decoding the picture blocks of the video slice.

The intra prediction unit 254 is configured to obtain, eg, receive a picture block 203 of the same picture (the current picture block) and one or more previously reconstructed blocks, eg, reconstructed neighboring blocks, for intra estimation. For example, the encoder 20 may be used to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

Embodiments of encoder 20 may be used to select an intra prediction mode based on optimization criteria, such as minimum residual (e.g., an intra prediction mode that provides prediction block 255 that is most similar to current picture block 203) or minimum code rate distortion ( For example……).

Intra-prediction unit 254 is further operable to determine intra-prediction block 255 based on intra-prediction parameters as selected for the intra-prediction mode. In any case, after selecting an intra prediction mode for a block, intra prediction unit 254 is also operable to provide intra prediction parameters to entropy encoding unit 270 , ie, providing an indication of the selected intra prediction mode for the block. Information. In one example, intra prediction unit 254 may be used to perform any combination of intra prediction techniques described below.

The entropy coding unit 270 is used to apply an entropy coding algorithm or scheme (for example, a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (context adaptive binary arithmetic coding, CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or other entropy coding methods, or technique) is applied (or not applied) to any or all of the quantized residual coefficients 209, the inter prediction parameters, the intra prediction parameters and/or the loop filter parameters to obtain a code that can be encoded via the output 272, for example. The encoded picture data 21 is output in the form of a bit stream 21. The encoded bitstream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may also be used to entropy encode other syntax elements of the current video slice being encoded.

Other architectural variations of video encoder 20 may be used to encode video streams. For example, the non-transform based encoder 20 may directly quantize the residual signal without the transform processing unit 206 for certain blocks or frames. In another embodiment, encoder 20 may have quantization unit 208 and inverse quantization unit 210 combined into a single unit.

FIG. 3 illustrates an exemplary video decoder 30 for implementing the techniques of the present application, namely, fusion candidate list construction of a block to be decoded (current block) and decoding of a compressed image based on the constructed fusion candidate list. Video decoder 30 is configured to receive encoded picture data (eg, encoded bitstream) 21 , such as encoded by encoder 20 , to obtain decoded picture 231 . During the decoding process, video decoder 30 receives video data from video encoder 20, such as an encoded video bitstream representing picture blocks of an encoded video slice and associated syntax elements.

In the example of Figure 3, decoder 30 includes entropy decoding unit 304, inverse quantization unit 310, inverse transform processing unit 312, reconstruction unit 314 (eg, summer 314), buffer 316, loop filter 320, Decoded picture buffer 330 and prediction processing unit 360. Prediction processing unit 360 may include inter prediction unit 344, intra prediction unit 354, and mode selection unit 362. In some examples, video decoder 30 may perform a decoding pass that is generally reciprocal to the encoding pass described with reference to video encoder 20 of FIG. 2 .

The entropy decoding unit 304 is configured to perform entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and/or decoded encoding parameters (not shown in FIG. 3 ), such as inter prediction, intra prediction parameters. , loop filter parameters, and/or other syntax elements (decoded). Entropy decoding unit 304 is further configured to forward inter prediction parameters, intra prediction parameters, and/or other syntax elements to prediction processing unit 360 . Video decoder 30 may receive syntax elements at the video slice level and/or the video block level.

The inverse quantization unit 310 may be functionally the same as the inverse quantization unit 110 , the inverse transform processing unit 312 may be functionally the same as the inverse transform processing unit 212 , the reconstruction unit 314 may be functionally the same as the reconstruction unit 214 , and the buffer 316 may be functionally the same. Like buffer 216, loop filter 320 may be functionally identical to loop filter 220, and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230.

Prediction processing unit 360 may include inter prediction unit 344 , which may be functionally similar to inter prediction unit 244 , and intra prediction unit 354 , which may be functionally similar to intra prediction unit 254 . Prediction processing unit 360 is generally operable to perform block predictions and/or obtain prediction blocks 365 from encoded data 21 , and to receive or obtain prediction-related parameters and/or information about the prediction blocks 365 from, for example, entropy decoding unit 304 (explicitly or implicitly). Information about the selected prediction mode.

When a video slice is encoded as an intra-coded (I) slice, intra-prediction unit 354 of prediction processing unit 360 is used to represent an intra-prediction mode based on the signal and a previously decoded block from the current frame or picture. data to generate prediction blocks 365 for the picture blocks of the current video slice. When a video frame is encoded as an inter-coded (i.e., B or P) slice, inter prediction unit 344 (eg, motion compensation unit) of prediction processing unit 360 operates based on the motion vectors and the motion vectors received from entropy decoding unit 304 Other syntax elements generate prediction blocks 365 for the video blocks of the current video slice. For inter prediction, the prediction block can be generated from a reference picture within a reference picture list. Video decoder 30 may construct the reference frame lists: List 0 and List 1 based on the reference pictures stored in DPB 330 using default construction techniques.

Prediction processing unit 360 is configured to determine prediction information for the video block of the current video slice by parsing the motion vectors and other syntax elements, and use the prediction information to generate prediction blocks for the current video block being decoded. For example, prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (eg, intra or inter prediction), an inter prediction slice type (eg, B slice, B slice, P slice or GPB slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, each inter-coded video block of the slice The inter prediction state and other information of inter-encoded video blocks are used to decode the video blocks of the current video slice.

Inverse quantization unit 310 may be used to inversely quantize (ie, inversely quantize) the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304 . The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in the video slice to determine the degree of quantization that should be applied and likewise determine the degree of inverse quantization that should be applied.

The inverse transform processing unit 312 is configured to apply an inverse transform (eg, inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to generate a residual block in the pixel domain.

Reconstruction unit 314 (eg, summer 314) is used to add inverse transform block 313 (ie, reconstructed residual block 313) to prediction block 365 to obtain reconstructed block 315 in the sample domain, e.g., by The sample values of reconstructed residual block 313 are added to the sample values of prediction block 365 .

The loop filter unit 320 (during or after the encoding loop) is used to filter the reconstructed block 315 to obtain the filtered block 321, thereby smoothing the pixel conversion or improving the video quality. Loop filter unit 320 is intended to represent one or more loop filters, such as deblocking filters, sample-adaptive offset (SAO) filters, or other filters, such as bilateral filters, adaptive Adaptive loop filter (ALF), or sharpening or smoothing filter, or collaborative filter. Although loop filter unit 320 is shown in FIG. 3 as an in-loop filter, in other configurations, loop filter unit 320 may be implemented as a post-loop filter.

The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330 that stores reference pictures for subsequent motion compensation.

The decoder 30 is configured to output the decoded picture 31 , for example via an output 332 , for presentation to or viewing by a user.

Other variations of video decoder 30 may be used to decode compressed bitstreams. For example, decoder 30 may generate the output video stream without loop filter unit 320. For example, the non-transform based decoder 30 may directly inverse-quantize the residual signal without the inverse-transform processing unit 312 for certain blocks or frames. In another embodiment, video decoder 30 may have inverse quantization unit 310 and inverse transform processing unit 312 combined into a single unit.

FIG. 4 is an illustration of an example of a video encoding system 40 including the encoder 20 of FIG. 2 and/or the decoder 30 of FIG. 3 according to an exemplary embodiment. The system 40 can implement the technology of the present application, and is used to construct a fusion candidate list of the current block based on the fusion candidate construction method proposed by the present invention, and perform image encoding or decoding based on the fusion candidate list. In the illustrated embodiment, video encoding system 40 may include imaging device 41, video encoder 20, video decoder 30 (and/or a video encoder implemented by logic circuitry 47 of processing unit 46), antenna 42, One or more processors 43, one or more memories 44 and/or display devices 45.

As shown, imaging device 41, antenna 42, processing unit 46, logic circuit 47, video encoder 20, video decoder 30, processor 43, memory 44 and/or display device 45 are capable of communicating with each other. As discussed, although video encoding system 40 is shown with video encoder 20 and video decoder 30 , in different examples, video encoding system 40 may include only video encoder 20 or only video decoder 30 .

In some examples, video encoding system 40 may include antenna 42 as shown. For example, antenna 42 may be used to transmit or receive an encoded bitstream of video data. Additionally, in some examples, video encoding system 40 may include display device 45. Display device 45 may be used to present video data. In some examples, logic circuitry 47 may be implemented by processing unit 46 as shown. The processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. Video encoding system 40 may also include an optional processor 43, which may similarly include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. In some examples, the logic circuit 47 can be implemented by hardware, such as video encoding dedicated hardware, etc., and the processor 43 can be implemented by general software, operating system, etc. In addition, the memory 44 may be any type of memory, such as volatile memory (eg, static random access memory (Static Random Access Memory, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.) or non-volatile memory. memory (e.g., flash memory, etc.), etc. In a non-limiting example, memory 44 may be implemented by cache memory. In some examples, logic circuitry 47 may access memory 44 (eg, for implementing an image buffer). In other examples, logic circuitry 47 and/or processing unit 46 may include memory (eg, cache, etc.) for implementing image buffers, etc.

In some examples, video encoder 20 implemented by logic circuitry may include an image buffer (eg, implemented by processing unit 46 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). A graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 20 implemented with logic circuitry 47 to implement the various modules discussed with respect to FIG. 2 and/or any other encoder system or subsystem described herein. Logic circuits can be used to perform the various operations discussed herein.

Video decoder 30 may be implemented in a similar manner with logic circuitry 47 to implement the various modules discussed with respect to decoder 30 of FIG. 3 and/or any other decoder system or subsystem described herein. In some examples, logic circuitry implemented video decoder 30 may include an image buffer (implemented by processing unit 2820 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). A graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video decoder 30 implemented with logic circuitry 47 to implement the various modules discussed with reference to FIG. 3 and/or any other decoder system or subsystem described herein.

In some examples, antenna 42 of video encoding system 40 may be used to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to encoded video frames, indicators, index values, mode selection data, etc., as discussed herein, such as data related to encoded partitions (e.g., transform coefficients or quantized transform coefficients , optional indicators (as discussed, and/or data defining encoding splits). Video encoding system 40 may also include video decoder 30 coupled to antenna 42 and for decoding the encoded bitstream. Display device 45 is used to present video frames.

FIG. 5 is a simplified block diagram of an apparatus 500 operable as either or both source device 12 and destination device 14 in FIG. 1 , according to an exemplary embodiment. The device 500 can implement the technology of the present application, and is used to construct a fusion candidate list and perform image encoding or decoding based on the constructed fusion candidate list. Apparatus 500 may take the form of a computing system including multiple computing devices, or a single computing device such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, or the like.

The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices that is currently available or may be developed in the future that is capable of manipulating or processing information. As shown, although the disclosed embodiments may be practiced using a single processor, such as processor 502, speed and efficiency advantages may be realized using more than one processor.

In one implementation, the memory 504 in the device 500 may be a read only memory (Read Only Memory, ROM) device or a random access memory (random access memory, RAM) device. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by processor 502 using bus 512 . Memory 504 may further include an operating system 508 and application programs 510 including at least one program that permits processor 502 to perform the methods described herein. For example, applications 510 may include Applications 1 through N, which further include a video encoding application that performs fusion candidate list construction as described herein. Apparatus 500 may also include additional memory in the form of slave memory 514, which may be, for example, a memory card for use with a mobile computing device. Because a video communication session may contain a large amount of information, this information may be stored in whole or in part in slave memory 514 and loaded into memory 504 for processing as needed.

Device 500 may also include one or more output devices, such as display 518. In one example, display 518 may be a touch-sensitive display that combines a display and a touch-sensitive element operable to sense touch input. Display 518 may be coupled to processor 502 via bus 512 . Other output devices may be provided in addition to display 518 or as an alternative to display 518 to permit a user to program or otherwise use device 500 . When the output device is or includes a display, the display may be implemented in different ways, including through a liquid crystal display (LCD), a cathode-ray tube (CRT) display, a plasma display, or a light emitting diode (light emitting diode). diode, LED) display, such as organic LED (organic LED, OLED) display.

The apparatus 500 may also include or be connected to an image sensing device 520, such as a camera or any other image sensing device 520 existing or to be developed in the future that can sense an image, such as is an image of a user running device 500. Image sensing device 520 may be placed directly facing the user running device 500 . In one example, the position and optical axis of image sensing device 520 may be configured so that its field of view includes an area immediately adjacent to display 518 and from which display 518 is visible.

The device 500 may also include or be connected to a sound sensing device 522 , such as a microphone or any other sound sensing device that is existing or will be developed in the future and can sense sounds near the device 500 . The sound sensing device 522 may be placed directly facing the user operating the device 500 and may be used to receive sounds, such as speech or other vocalizations, emitted by the user while operating the device 500 .

Although the processor 502 and memory 504 of the device 500 are shown in Figure 5 as being integrated in a single unit, other configurations may be used. Operation of processor 502 may be distributed among multiple directly coupled machines (each machine having one or more processors), or distributed over a local area or other network. The memory 504 may be distributed among multiple machines, such as network-based memory or memory among multiple machines running the apparatus 500 . Although only a single bus is shown here, bus 512 of device 500 may be formed from multiple buses. Further, slave memory 514 may be directly coupled to other components of device 500 or accessible through a network, and may comprise a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Accordingly, apparatus 500 may be implemented in a variety of configurations.

FIG. 11 is a flowchart of an example operation of a method for constructing a fusion candidate list according to an embodiment of the present invention implemented by the video encoder 20 and the video decoder 30 shown in FIG. 1 . One or more functional units of video encoder 20 or video decoder 30 include prediction processing units 260/360, which may be used to perform the method of Figure 11. In the example of Figure 11, an update method for improving the historical candidate motion information list is proposed. This method is applied to inter-frame prediction and allows the reconstruction of the historical candidate motion information list at the CTU (coding tree) level without adding additional While the storage area has considerable coding efficiency, it is more conducive to designing row-level and CTU-level encoding and decoding parallelism. This method can greatly reduce encoding and decoding time while ensuring that the encoding quality is basically not lost during the inter-frame encoding process. The inter prediction method of the candidate list includes:

S1101 initializes the historical candidate motion information list corresponding to the current coding tree unit;

Wherein, the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, the initialized historical candidate motion information list includes at least M vacant storage spaces, and the M≤N, M and N are integers, the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, and the current coding tree unit is not in the coding tree unit set. First in a predetermined processing sequence;

S1103 Add the motion information at L positions in the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, where M≤L≤N, the spatial adjacent blocks The inner L positions are obtained according to the preset rules;

S1105 constructs a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; and

S1107 Code the current encoding according to the combination of the current candidate motion information list of the current coding tree unit and the historical candidate motion information list or the combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list. The tree unit or the current coding unit performs inter prediction.

In this method, during the coding process of the current coding tree unit, the historical candidate motion information list is initialized, that is, an independent historical candidate motion information list corresponding to the current coding tree unit is constructed, thereby cutting off the coding of the coding tree unit. The dependency relationship caused by constructing the historical candidate motion information list during the process allows the coding tree unit to independently encode according to its own historical candidate motion information list. While having considerable coding efficiency, it is more conducive to the design of row-level and CTU-level encoding and decoding parallelism, through parallel processing, can greatly reduce encoding and decoding time without basically losing the encoding quality.

Optionally, the initializing the historical candidate motion information list corresponding to the current coding tree unit includes clearing the historical candidate motion information list so that M=N. This method allows the current coding tree unit to construct brand new historical candidate motion information. list to increase the accuracy of inter prediction.

Optionally, the M positions in the spatial adjacent block are to obtain the first candidate motion information from the preset positions in the spatial adjacent block, with the position where the first candidate motion information is obtained as the starting point, The remaining M-1 candidate motion information is obtained at preset step intervals. In the process of constructing the historical candidate motion information list, in order to be consistent with the existing historical candidate motion information list construction method and have a relatively simplified algorithm, the motion vectors at the M positions are usually calculated from a starting position to a preset The motion vectors at M positions are obtained sequentially at intervals, where the preset interval can also be called a step size, and the step size can be fixed, for example, using 4 or 8 pixels as a unit; in addition, the The step size can also be changed, for example, different step sizes are set according to the size of the current coding tree unit. The order in which the motion information/motion vector of the M position is added may be a preset order, for example, in clockwise order, starting from the spatial adjacent block in the lower left corner of the current coding tree unit, starting from the current coding The spatial adjacent block in the upper right corner of the tree unit is the end point, and the motion information at L positions within the spatial adjacent block is added to the historical candidate motion information list.

Optionally, the current candidate motion information list of the current coding tree unit and the historical candidate motion information list are combined, or the current candidate motion information list of the current coding unit and the historical candidate motion information list are combined. Before the current coding tree unit or the current coding unit performs inter-frame prediction, the method may also combine the current candidate motion information list of the current coding tree unit and the historical candidate motion information list or combine the current candidate motion information list of the current coding unit. The current candidate motion information list and the historical candidate motion information list are combined, which may specifically include: adding the historical candidate motion information to the current candidate motion information list of the current coding tree unit or the current candidate motion information of the previous coding unit. In the list, the inter-frame prediction is then performed based on the current candidate motion information list of the current coding tree unit or the current candidate motion information list of the previous coding unit.

If the current coding tree unit needs to be further divided into coding units for coding, the method may also include: obtaining based on a combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list. motion information of the current coding unit, and perform inter-frame prediction on the current coding unit according to the acquired motion information; and update the historical candidate motion information list based on the motion information of the current coding unit. This method can enable the historical candidate motion information list corresponding to the current coding tree unit to be continuously updated to improve the accuracy of inter-frame prediction. Optionally, the above-mentioned updating of the historical candidate motion information list can be divided into two situations, that is, if the M positions are not filled, the current coding unit motion information is added to all the historical motion information as historical motion information. In the vacant storage space closest to the N-M position among the M positions in the historical candidate motion information list; or if the M positions are filled, the earliest candidate will be added to the historical candidate according to the first-in, first-out principle. Remove the historical motion information in the motion information list, shift the remaining historical motion information beyond the removed historical motion information, and then add the current coding unit motion information as historical motion information to the historical candidate motion information list. Tail, wherein the end of the historical candidate motion information list containing the latest added historical motion information is the tail of the historical candidate motion information list. This method provides flexibility in the application of the historical candidate motion information list, that is, the historical candidate motion information list can also be used for inter-frame prediction of the current block when it is not completely filled, and when the historical candidate motion information list is full, Under this condition, the motion information/motion vector of the current coding block can still be used to update the historical candidate motion information list. Of course, when the historical candidate motion information list is filled, the motion information/motion vector of the current coding block may still no longer be updated on the historical candidate motion information list. That is, if the M positions are not filled, Then add the current coding unit motion information as historical motion information to the vacant storage space closest to the N-M position among the M positions in the historical candidate motion information list; if the M positions have been filled, Then perform inter prediction processing on the next coding unit based on the current candidate motion information list. This processing method may allow parallel processing of coding blocks within the current coding tree unit. Specifically, it may be to perform inter prediction on another coding unit based on the same method as the current coding unit, wherein the other coding unit The coding unit is located after the current coding unit in a preset processing order and belongs to the coding tree unit with the current coding unit. The historical motion information list used in inter prediction of another coding unit includes the current coding unit. The historical motion information in the historical motion information list used for the unit's inter prediction.

The inventive solution allows the reconstruction of the historical candidate motion information list at the CTU (coding tree) level. While having considerable coding efficiency, it is more conducive to designing row-level and CTU-level encoding and decoding parallelism. This method can ensure that inter-frame When encoding quality is basically not lost during the encoding process, encoding and decoding time is greatly reduced.

The following are examples of specific implementations of the inter-frame prediction method of the present invention:

Example 1:

When starting to encode the current coding tree unit, initialize the historical candidate motion information list;

The initialization process of the historical candidate motion information list can refer to the existing technology. For example, the process can be carried out using the same method as the JVET-K0104 proposal (that is, at the beginning of the slice (SLICE), the historical candidate motion information list is cleared), or Other initialization methods of the historical candidate motion information list may be used, and the present invention is not limited thereto; but in this embodiment, the initialization is to clear the historical candidate motion information list. The coding tree unit is an image block to be processed based on which the prediction mode can be determined during the encoding process. It may or may not be further divided. Its definition is consistent with the definition of the coding tree unit in HEVC and VVC. In H.264 In the middle is a macroblock. Coding tree units are used below. If the coding tree unit is further divided, multiple coding units will be formed. Therefore, in this case, the coding tree unit can also be understood as a coding unit combination.

Then, the motion information of the spatial adjacent blocks of the current coding tree unit is added to the historical candidate motion information list.

The motion information of the adjacent image blocks in the spatial domain includes the motion information of the adjacent image blocks in the left spatial domain (A, An) and the motion information of the adjacent image blocks in the upper spatial domain (B, Bn, C), as shown in Figure 10. Bn and An in Figure 10 are motion information extracted according to predetermined rules from all upper and left adjacent encoded/decoded image blocks. Among them, the predetermined rule may be to extract at fixed intervals M and N (M and N are positive integers greater than 0), the interval M is suitable for extracting motion information in the adjacent image blocks on the left, and the interval N is suitable for the upper adjacent image blocks. Other extraction predetermined rule methods can also be used to extract motion information within image blocks. The present invention does not involve specific extraction predetermined rule methods. The above-mentioned adjacent image blocks preferably refer to image blocks located in the same SLICE as the current coding tree unit.

If the historical candidate motion information list is not filled up after traversing the spatial adjacent image blocks, or if the current coding tree unit is located at the top of a frame of image, or if the current coding tree unit is located at the top of a frame of image On the other hand, you can refer to any of the following methods to deal with the unfilled parts of the historical candidate motion information list.

method one:

Without filling in any other source of motion information, the motion information of the current coding unit acquired when encoding and decoding the current coding unit in the current coding tree unit is added to the historical candidate motion information list as the historical candidate motion information.

Method Two:

Fill the motion information of the coding block at a preset non-nearby position in the spatial domain of the current coding tree unit. The preset non-adjacent location can be a fixed interval from the adjacent location, or it can be a preset template.

Method three:

The filling comes from the coding block time domain motion information of the preset position corresponding to the current coding tree unit and the coding block adjacent to the current coding tree unit in the reference frame. The preset positions in the corresponding positions of the current coding tree unit may be extracted at fixed intervals, or may be extracted in a specific rule or order. The preset position in the corresponding position of the current coding tree unit adjacent to the coding block may be extracted in a specific order according to a specific rule.

Method four:

The filling comes from the coding block time domain motion information of the preset position in the reference frame, the corresponding position of the current coding tree unit and the coding block corresponding to the preset non-adjacent position of the previous coding tree unit. The preset positions in the corresponding positions of the current coding tree unit may be extracted at fixed intervals, or may be extracted in a specific rule or order. The preset positions in the coding blocks corresponding to the preset non-adjacent positions of the current coding tree unit may be extracted in a specific order according to a specific rule.

Method five:

Populate historical candidate motion information from the historical candidate motion information list of coding tree units adjacent to the current coding tree unit.

Add A, An, B, Bn and C to the historical candidate motion information list in a predetermined order until the traversal of all adjacent blocks ends, or until the number of historical candidate motion information in the historical candidate motion information list exceeds the preset maximum value, in which the predetermined order in which A, An, B, Bn and C are added to the historical candidate motion information list can be the order of C, An, Bn, or the order of C, A0, B0, A1, B1 up to An and Bn , other predetermined sequences can also be adopted, and the present invention does not involve specific predetermined sequences. In this embodiment, the historical candidate motion information list is initialized when each coding tree unit starts coding, so that each coding tree unit does not need to wait for the completion of coding of the last coding unit of the previous coding tree unit. Only when processing can be started, it can be processed in parallel with the previous coding tree unit, thus greatly saving processing time.

Example 2:

Different from the embodiment that initializes the historical candidate motion information list when starting to encode the current coding tree unit, this embodiment initializes the spatial domain of a preset number of current coding tree units when starting to encode the current coding tree unit. The motion information of adjacent image blocks is added to the historical candidate motion information list, where the preset number is a positive integer greater than 0. Among them, a possible implementation method of adding the motion information of a preset number of spatially adjacent image blocks of the current coding tree unit to the historical candidate motion information list may be already in the existing historical candidate motion information list. On the basis of the historical candidate motion information, a predetermined number of motion information is additionally supplemented as new historical candidate motion information, regardless of whether the historical candidate motion information list is filled or not; another possible implementation may be to add the current candidate motion information to the historical candidate motion information. A predetermined number of historical candidate motion information existing in the historical candidate motion information list is cleared according to predetermined rules, and then a preset number of motion information in adjacent image blocks in the spatial domain of the current coding tree unit is used as new The historical candidate sports information is added to the historical candidate sports information list.

Embodiment three:

This embodiment is to initialize or reconstruct the historical candidate motion information list in Embodiment 1 and 2 and apply it to the inter-frame prediction of the current coding unit, which specifically includes:

The method in Embodiment 1 or Embodiment 2 is used to update or reconstruct the historical candidate motion information list. For specific reconstruction methods, please refer to Embodiment 1 or Embodiment 2;

(1) Perform inter-frame prediction on at least one coding unit in the current coding tree unit or in the current coding tree unit;

Performing inter-frame prediction on at least one coding unit in the current coding tree unit or the current coding tree unit takes the luminance block or chrominance block included in the coding unit or coding tree unit as the basic processing unit. A coding tree unit or coding unit includes one luma coding block and two chroma coding blocks. At least one color component (chrominance or luminance) coding block in the previous coding tree unit or the current coding tree unit in which encoding or decoding processing is being performed is called a current block.

The above-mentioned inter-frame prediction for the current coding tree unit or at least one coding unit in the current coding tree unit may include:

(2) Analyze the inter prediction mode of the current block. If the current block is in merge/skip mode, generate a fusion motion information candidate list; if the current CU or current block is in AMVP mode, generate a motion vector prediction list.

Optionally, the historical candidate motion information in the above historical candidate motion information list is added to the fused motion information candidate list or the candidate motion vector prediction list. It should be noted that the historical candidate motion information in the historical candidate motion information list may not be added to the fused motion information candidate list or the candidate motion vector prediction list, but the historical candidate motion information list may be kept independent, and the current block may be When making predictions, the historical candidate motion information list is directly indexed. If the historical candidate motion information in the historical candidate motion information list is added to the fused motion information candidate list, the method in the JVET-K0104 proposal can be followed, or other methods can be used to add the historical candidate motion information in the historical candidate motion information list according to Adding to the fused motion information candidate list is not limited by the present invention. It should be noted that if historical candidate motion information is added to the fusion motion information candidate list, the position of the historical candidate motion information in the historical candidate motion information list may be before other types of fusion candidates, such as bidirectional prediction fusion candidates (bi -predictive merge candidate) and zero motion vector fusion candidate (zero motion vector mergecandidate).

If the current block is in merge/skip mode, the detailed process of generating a fusion motion information candidate list is as follows.

The fused motion information candidate list is constructed based on the following candidates: a. Up to four spatial fused motion information candidate lists obtained from five spatially adjacent blocks; b. One temporal fused motion information candidate obtained from two temporally co-located blocks; c. .An additional fused motion information candidate containing the combined bidirectional prediction candidate and the zero motion vector candidate. The first candidate in the fused motion information candidate list is the spatial neighbor. According to the right part of Figure 6, by checking A1, B1, B0, A0 and B2 in sequence, up to four candidates can be inserted in the merge list in the stated order. Some additional redundancy checks are performed before considering all motion data of neighboring blocks as candidates for fused motion information. These redundancy checks can be divided into two categories and serve two different purposes: a. To avoid candidates with redundant motion data in the list; b. To prevent the merging of two candidates that can be expressed in other ways to produce redundant syntax. partition.

When N is the number of spatially fused motion information candidate lists, a complete redundancy check will be Comparison of sub-movement data. In the case of five latent space fusion motion information candidates, ten motion data comparisons would be required to ensure that all candidates in the merged list have different motion data. During the development of HEVC, checks for redundant motion data have been reduced to a subset, resulting in significantly reduced comparison logic while maintaining coding efficiency. In the final design, no more than two comparisons were performed for each candidate, resulting in a total of five comparisons. Given the order {A1, B1, B0, A0, B2}, B0 only checks B1, A0 only checks A1, and B2 only checks A1 and B1. In the embodiment of partition redundancy checking, the bottom PU of the 2NxN partition is merged with the top PU by selecting candidate B1. This will result in a CU with two PUs with the same motion data, which can be equally signaled as a 2N×2N CU. Overall, this check applies to all second PUs of rectangular and asymmetric partitions 2N×N, 2N×nU, 2N×nD, N×2N, nR×2N and nL×2N. It should be noted that for the spatial fusion motion information candidate list, only the redundancy check is performed and the motion data is copied from the candidate blocks as is. Therefore, motion vector scaling is not required here.

The motion vector of the temporal fusion motion information candidate list is obtained in the same way as that of TMVP. Since the fused motion information candidate list includes all motion data and TMVP is only a motion vector, the acquisition of the entire motion data only depends on the type of slice. For bidirectionally predicted slices, TMVP is obtained for each reference picture list. Depending on the availability of TMVP for each list, set the prediction type to Bidirectional Prediction or to the list where TMVP is available. All relevant reference picture indices are set equal to zero. Therefore, for unidirectional predicted slices, only the TMVP of list 0 is obtained together with the reference picture index equal to zero.

When at least one TMVP is available and the temporal fusion motion information candidate list is added to the list, no redundancy check is performed. This allows merge list construction to be independent of co-located images, thereby improving error tolerance. Consider the case where the temporally fused motion information candidate list would be redundant and therefore not included in the fused motion information candidate list list. In the case of missing co-located pictures, the decoder cannot obtain the temporal candidate motion information and therefore does not check whether it is redundant. The indexes of all subsequent candidates will be affected by this.

For parsing robustness reasons, the length of the fused motion information candidate list is fixed. After the spatial and temporal fusion motion information candidate list has been added, it may occur that the list does not yet have a fixed length. To compensate for the coding efficiency loss that occurs with non-length adaptive list index signaling, additional candidates are generated. Depending on the type of slice, up to two candidates can be used to completely populate the list: a. Combined bidirectional prediction candidates; b. Zero motion vector candidates.

In a bi-predictive slice, additional candidates may be generated based on existing candidates by combining the reference picture list 0 motion data of one candidate with the list 1 motion data of another candidate. This is done by copying Δx ₀ , Δy ₀ , Δt ₀ from one candidate, such as the first candidate, and copying Δx ₁ , Δy ₁ , Δt ₁ from another candidate, such as the second candidate. Different combinations are predefined and given in Table 1.1.

Table 1.1

After adding combined bi-prediction candidates or when the list is still incomplete for uni-prediction slices, zero motion vector candidates are calculated to make the list complete. All zero motion vector candidates have one zero displacement motion vector for unidirectional prediction slices and two zero displacement motion vectors for bidirectional prediction slices. The reference index is set equal to zero and is incremented by one for each additional candidate until the maximum number of reference indices is reached. If this is the case and there are other candidates missing, these candidates are created using a reference index equal to zero. For all additional candidates, redundancy checks are not performed because results show that omitting these checks does not cause a loss in coding efficiency.

For each PU encoded in inter-picture prediction mode, a so-called merge_flag indicates that the block merge is used to derive motion data. merge_idx further identifies candidates in the merge list that provide all motion data required by the MCP. In addition to this PU-level signaling, the number of candidates in the merge list is also signaled in the slice header. Since the default value is five, it is expressed as the difference from five (five_minus_max_num_merge_cand). This way, five is signaled with the short codeword of 0, while using only one candidate is signaled with the longer codeword of 4. As for the impact on the fused motion information candidate list construction process, the entire process remains unchanged, but the process is terminated after the list contains the maximum number of fused motion information candidate lists. In the initial design, the maximum value of the merge index encoding is given by the number of available spatial and temporal candidate motion information in the list. The index can be efficiently encoded as a flag when, for example, only two candidates are available. However, in order to parse the merge index, the entire list of fused motion information candidate lists must be constructed to know the actual number of candidates. Assuming adjacent blocks are unavailable due to sending errors, it will no longer be possible to parse the merge index.

The key application of the block merging concept in HEVC is in combination with skip mode. In previous video coding standards, skip mode was used to indicate blocks in which motion data was inferred rather than explicitly signaled, and the prediction residual was zero, i.e., no transform coefficients were sent. In HEVC, skip_flag is signaled at the beginning of each CU in the inter-picture prediction slice, which means the following: a. The CU contains only one PU (2N×2N partition type); b. Use merge mode to Obtain motion data (merge_flag equals 1); c. There is no residual data in the code stream.

A parallel merge estimation hierarchy of indication regions is introduced in HEVC, where the fused motion information candidate list can be obtained independently by checking whether a candidate block is located in the merge estimation region (MER). Candidate blocks in the same MER are not included in the fused motion information candidate list. Therefore, its motion data does not need to be available at list building time. When this level is, for example, 32, then all prediction units in the 32×32 region can construct the fused motion information candidate list list in parallel, because all fused motion information candidate lists in the same 32×32 MER are not inserted into the list. As shown in Figure 5, there is a CTU partition with seven CUs and ten PUs. All potential fused motion information candidate lists for the first PU0 are available because they are outside the first 32×32 MER. For the second MER, when the merged estimates within the MER should be independent, the fused motion information candidate list list for PU2-6 cannot contain motion data from these PUs. Therefore, for example when viewing PU5, no fused motion information candidate list is available and therefore is not inserted into the fused motion information candidate list list. In this case, PU5's merge list consists only of temporal candidate motion information (if available) and zero MV candidates. To enable the encoder to trade off parallelism and coding efficiency, the parallel merge estimation level is adaptive and signaled in the picture parameter set as log2_parallel_merge_level_minus2.

If the current block is in Inter MVP mode, also called AMVP, the detailed process of generating a candidate motion vector prediction list is as follows.

The initial design of the AMVP mode contains five MVPs from three different categories of predictors: three motion vectors from spatial neighbors, the median of the three spatial predictors, and scaled motion vectors from co-located temporal neighboring blocks. Additionally, the predictor list is modified by reordering to place the most likely motion predictor first and by removing redundant candidates to ensure minimal signaling overhead. Subsequently, the original AMVP underwent several simplifications, such as removing the median predictor, reducing the number of candidates in the list from five to two, fixing the order of candidates in the list, and reducing the number of redundant checks. The final design of AMVP candidate list construction contains the following two MVP candidates: a. at most two spatial candidate MVPs obtained from five spatially adjacent blocks; b. when two spatial candidate MVPs are not available or they are the same, from A temporal candidate MVP obtained from two temporally co-located blocks; c. Zero motion vector when spatial candidate motion information, temporal candidate motion information, or both are not available. As already mentioned, two spatial MVP candidates A and B are obtained from five spatial neighboring blocks, as shown in the right part of Figure 6. For AMVP and inter prediction block merging, the positions of spatial candidate blocks are the same. Figure 10 depicts the process flow of obtaining two spatial candidate motion information A and B. For candidate A, motion data from the two blocks A0 and A1 in the lower left corner are considered in a two-pass approach. In the first pass, it is checked whether any candidate block contains a reference index equal to the reference index of the current block. The first motion vector found will be candidate A. When all reference indexes from A0 and A1 point to reference pictures that are different from the reference index of the current block, the associated motion vectors cannot be used as is. Therefore, in the second pass, the motion vectors need to be scaled based on the temporal distance between the candidate reference picture and the current reference picture. Equation (1.1) shows how the candidate motion vector mvcand is scaled according to the scaling factor. ScaleFactor is calculated based on the temporal distance between the current picture and the reference picture of candidate block td and the temporal distance between the current picture and the reference picture of current block tb. The temporal distance is expressed as the difference between picture order count (POC) values that define the order in which pictures are displayed. The scaling operation is basically the same scheme used for time pass-through mode in H.264/AVC. This decomposition allows ScaleFactor to be pre-computed at the slice level, since it only depends on the reference picture list structure signaled in the slice header. It should be noted that MV scaling is only performed when both the current reference picture and the candidate reference picture are short-term reference pictures. The parameter td is defined as the POC difference between the co-located picture of the co-located candidate block and the reference picture.

mv＝sign(mvcand·ScaleFactor)·((|mvcand·ScaleFactor|+27)>>8) (1.1)

ScaleFactor＝clip(-212,212 -1,(tb·tx+25)>>6) (1.2)

For candidate B, candidates B0 through B2 are checked sequentially in the same way as A0 and A1 were checked in the first pass. However, the second pass is only performed when blocks A0 and A1 do not contain any motion information, ie are not available or are encoded using intra-picture prediction. Next, if candidate A is found, candidate A is set equal to the unscaled candidate B, and candidate B is set equal to the second unscaled or scaled variant of candidate B. The second pass searches the unscaled and scaled MVs from candidates B0 to B2. Overall, this design allows A0 and A1 to be processed independently of B0, B1 and B2. The getter of B should only know the availability of both A0 and A1 in order to search for scaled or otherwise unscaled MVs from B0 to B2. This dependency is acceptable considering that it significantly reduces the complex motion vector scaling operation of candidate B. Reducing the amount of motion vector scaling represents a significant reduction in complexity in deriving motion vector predictors.

In HEVC, the blocks at the lower right and center of the current block have been determined to be most suitable to provide good temporal motion vector predictors (TMVP). These candidates are shown in the left part of Figure 3, where C0 represents the lower right neighbor and C1 represents the center block. Here again the motion data of C0 is considered first, and if not available, the motion data from the co-located candidate block at the center is used to derive the temporal MVP candidate C. Motion data for C0 is also considered unavailable when the associated PU belongs to a CTU outside the current CTU row. This minimizes memory bandwidth requirements for storing co-located motion data. In contrast to spatial MVP candidates where the motion vectors may refer to the same reference picture, motion vector scaling is mandatory for TMVP. Therefore, the same scaling operation as spatial MVP is used.

While the temporal pass-through mode in H.264/AVC always refers to the second reference picture list, i.e. the first reference picture in List 1, and is only allowed in bidirectional predictive slices, HEVC provides an indication for each picture which The possibility that a reference picture is treated as a homotopic picture. This is accomplished by signaling the co-located reference picture list and reference picture index in the slice header and requiring that these syntax elements in all slices in the picture should specify the same reference picture.

Because temporal MVP candidates introduce additional dependencies, its use may need to be disabled for error robustness reasons. In H.264/AVC, it is possible to disable temporal direct mode (direct_spatial_mv_pred_flag) for bidirectionally predicted slices in the slice header. The HEVC syntax extends this signaling by allowing TMVP to be disabled at the sequence level or at the picture level (sps/slice_temporal_mvp_enabled_flag). Although the flag is signaled in the header, its value should be the same for all slices in a picture, which is a requirement for code stream consistency. Since the signaling of picture-level flags depends on the SPS flag, signaling the picture-level flag in PPS will introduce a parsing dependency between SPS and PPS. Another advantage of this kind of preamble signaling is that if you only want to change the value of this flag in the PPS without changing other parameters, you do not need to send a second PPS.

Generally, motion data signaling in HEVC is similar to that in H.264/AVC. The inter-picture prediction syntax element inter_pred_idc signals whether reference list 0, 1, or both is used. For each MCP obtained from a reference picture list, the corresponding reference picture (Δt) is signaled by the index ref_idx_l0/1 of the reference picture list, and the MV (Δx, Δy) is signaled by the index mvp_l0/1_flag of the MVP and its MVD . The newly introduced flag mvd_l1_zero_flag in the slice header indicates whether the MVD of the second reference picture list is equal to zero and therefore not signaled in the code stream. When the motion vector is completely reconstructed, the final clipping operation ensures that the value of each component of the final motion vector will always be in the range of -215 to 215-1, endpoint values inclusive.

If the historical candidate motion information in the historical candidate motion information list is added to the current candidate motion information list, you can follow the method in the JVET-K0104 proposal, or you can use other methods to add the historical candidate motion information in the historical candidate motion information list as follows The present invention does not limit the list of fused motion information candidates.

(3) Obtain the motion information of the current block;

Through the above method, the current candidate motion information list of the current block (including the current block's own motion information candidate list and the historical candidate motion information list) can be obtained from the current block's motion information according to existing methods.

More specifically, at the decoding end: if the current block is in merge/skip mode, the motion information of the current block is determined based on the fusion index carried in the code stream. If the current block is in Inter MVP mode, the motion information of the current block is determined based on the inter-frame prediction direction, reference frame index, motion vector predictor index, and motion vector residual value transmitted in the code stream.

(4) Obtain the inter-frame prediction image of the current block based on the motion information;

More specifically, the decoding end: performs motion compensation according to the motion information to obtain the predicted image; further, the above step (4) may also include obtaining the residual image of the current block, and combining the inter-frame predicted image with the residual image. The difference images are added to obtain the reconstructed image of the current block; if the current block has no residual, the predicted image is the reconstructed image of the current block.

Optionally, the above-described third embodiment may also include updating the historical candidate motion information list using the motion information of the current block. This step may be before step (4) and after step (2), or after step (4);

Specifically, the method in the JVET-K0104 proposal can be followed, or other methods can be used to update the historical candidate sports information list. In the JVET-K0104 proposal, starting from the head of the historical candidate motion information list, the motion information of the current block is compared with the historical candidate motion information in the historical candidate motion information list; if there is a certain historical candidate motion information and the current block motion information Similarly, the historical candidate motion information is removed from the historical candidate motion information list. Then, check the size of the historical candidate motion information list. If the list size exceeds the preset size, remove the historical candidate motion information at the head of the list, and remove the remaining historical candidate motion information in the current candidate motion information list. After shifting to the head of the list, the motion information of the current block is added to the tail of the historical candidate motion information list. It should be noted that in the present invention, the above-mentioned step of determining whether the motion information of the current block is the same as a certain historical candidate motion information in the historical candidate motion information list does not need to be performed. That is, the historical candidate motion information list may contain Two identical motion information can also be the same after some kind of processing. For example, the result of two motion vectors right-shifted by 2 bits is the same.

It should be noted that if the above-described third embodiment does not include using the motion information of the current block to update the historical candidate motion information list, it means that the coding block in the current coding tree unit can be updated using the same historical candidate motion information list. Inter prediction, thereby allowing parallel operations in the processing of coding blocks within coding tree units.

In general, the above-mentioned Embodiments 1 to 3 can be more conducive to row-level and CTU-level encoding and decoding parallelism without adding additional storage areas and achieving considerable encoding efficiency, and can effectively reduce encoding and decoding time. FIG. 12 is a flowchart of an example operation of applying the inter-frame prediction method in FIG. 11 to perform image decoding in an embodiment of the present invention implemented by the video decoder 30 shown in FIG. 1 . One or more functional units of video decoder 30 include prediction processing unit 360, which may be used to perform the method of Figure 12. In the example of Figure 12, pictures are decoded based on the inter-frame prediction method in the method of Figure 11. The decoding method 1200 specifically includes:

S1201 initializes the historical candidate motion information list corresponding to the current coding tree unit;

Wherein, the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, the initialized historical candidate motion information list includes at least M vacant storage spaces, and the M≤N, M and N are integers, the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, and the current coding tree unit is not in the coding tree unit set. First in a predetermined processing sequence;

S1203 Add the motion information at L positions in the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, where M≤L≤N, the spatial adjacent blocks The inner L positions are obtained according to the preset rules;

When the inter prediction type of the current coding unit or the current coding tree unit is AMVP mode or merge/skip mode, generating a current candidate motion information list of the current coding unit or the current coding tree unit; wherein AMVP In the merge/skip mode, the current candidate motion information list includes a motion vector, and in the merge/skip mode, the current candidate motion information list includes a bidirectional reference or a unidirectional reference indication, a reference frame index, and a reference direction. The corresponding motion vector;

S1205 constructs the current candidate motion information list of the current coding tree unit or the current coding unit;

S1207 Obtains the motion information of the current coding tree unit or the current coding unit from the combination of the historical candidate motion information list and the current candidate motion information list. According to the motion information of the current coding tree unit or the current coding unit, Perform inter-frame prediction on the current coding tree unit or current coding unit to obtain an inter-frame prediction image;

This step includes parsing the code stream, the motion information index corresponding to the current coding tree unit or the current coding unit, and obtaining the current candidate motion information list from the combination of the historical candidate motion information list and the current candidate motion information list. Coding the motion information of the current coding tree unit or the current coding unit, and performing inter-frame prediction on the current pair of the current coding tree unit or the current coding unit according to the motion information to obtain an inter-frame prediction image;

S1209 Add the obtained inter-frame prediction image to the current coding tree unit or the residual image of the current coding unit to obtain a reconstructed image of the current coding tree unit or the current coding unit.

Compared with the existing technology, the above decoding method adopts the update of the historical candidate motion information list at the CTU level, allowing parallel encoding and decoding at the row level and CTU level, which can effectively reduce the decoding time.

FIG. 13 is a flowchart of an example operation of image encoding using the fusion candidate list construction method constructed in FIG. 11 in an embodiment of the present invention implemented by the video encoder 20 shown in FIG. 1 . One or more functional units of video encoder 20 include prediction processing unit 260, which may be used to perform the method of Figure 13. In the example of Figure 13, the picture is encoded based on the inter-frame prediction method in the method of Figure 11, and the encoding method 1300 specifically includes:

S1301 initializes the historical candidate motion information list corresponding to the current coding tree unit;

Wherein, the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, the initialized historical candidate motion information list includes at least M vacant storage spaces, and the M≤N, M and N are integers, the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, and the current coding tree unit is not in the coding tree unit set. First in a predetermined processing sequence;

S1303 Add the motion information at L positions in the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, where M≤L≤N, the spatial adjacent blocks The inner L positions are obtained according to the preset rules;

When the inter prediction type of the current coding unit or the current coding tree unit is AMVP mode or merge/skip mode, generating a current candidate motion information list of the current coding unit or the current coding tree unit; wherein AMVP In the merge/skip mode, the current candidate motion information list includes a motion vector, and in the merge/skip mode, the current candidate motion information list includes a bidirectional reference or a unidirectional reference indication, a reference frame index, and a frame corresponding to the reference direction. motion vector;

S1305 constructs the current candidate motion information list of the current coding tree unit or the current coding unit;

S1307 obtains the motion information of the current coding tree unit or the current coding unit, and the motion information index of the motion information from the combination of the historical candidate motion information list and the current candidate motion information list;

S1309 Perform inter-frame prediction on the current coding tree unit or current coding unit according to the motion information of the current coding tree unit or current coding unit to obtain an inter-frame prediction image;

S1311 Subtract the original image of the current coding tree unit or the current coding unit from the obtained inter-frame prediction image to obtain a residual image;

S1213 Encode the residual image and the motion information index to form a code stream.

Compared with the existing technology, the above decoding method adopts the update of the historical candidate motion information list at the CTU level, allowing parallel encoding and decoding at the row level and CTU level, which can effectively reduce the encoding time.

Figure 14 is an inter-frame prediction device 1400 provided by the present invention. The inter-frame prediction device has the function of implementing the inter-frame prediction method in Figure 11. It includes: an initialization module 1401 for initializing the current coding tree unit. Corresponding historical candidate motion information list, wherein the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, and the initialized historical candidate motion information list includes at least M vacant storage space, the M≤N, M and N are integers, the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, and the current coding tree unit is not the first one in the coding tree unit set according to a predetermined processing order; the historical candidate motion information list construction module 1403 is used to convert L in the spatial adjacent blocks of the current coding tree unit in a predetermined order. The motion information at positions is added to the historical candidate motion information list, where M≤L≤N positions in the spatial adjacent block are obtained according to preset rules; the current candidate motion information list construction module 1405, Be used to construct a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; and a prediction module 1407, which is used to, According to the combination of the current candidate motion information list of the current coding tree unit and the historical candidate motion information list or the combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list, the current candidate motion information list is The coding tree unit or the current coding unit performs inter prediction.

Optionally, the initializing the historical candidate motion information list corresponding to the current coding tree unit includes clearing the historical candidate motion information list such that M=N. Optionally, the M positions in the spatial adjacent block are to obtain the first candidate motion information from the preset positions in the spatial adjacent block, with the position where the first candidate motion information is obtained as the starting point, The remaining M-1 candidate motion information is obtained at intervals of a preset step size, where the preset step size is a fixed value, or the preset step size changes according to a preset rule.

Optionally, the prediction module 1405 is configured to: obtain the motion information of the current coding unit according to a combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list, and calculate the motion information based on the obtained motion information. The information performs inter-frame prediction on the current coding unit; the device 1400 further includes: a historical motion information list update module 1407, which is configured to update the historical candidate motion information list based on the current coding unit motion information.

Optionally, the historical motion information list update module updates the historical candidate motion information list according to the following rules: if the M positions are not filled, the current coding unit motion information is added to the history as historical motion information. In the vacant storage space closest to the N-M position among the M positions in the candidate motion information list; or if the M positions are filled, the earliest historical candidate motion information will be added according to the first-in, first-out principle. Remove the historical motion information in the list, shift the remaining historical motion information to the position of the removed historical motion information, and then add the current coding unit motion information to the end of the historical candidate motion information list as historical motion information, Wherein, the end of the historical motion information candidate list containing the latest added historical motion information is the tail of the historical motion candidate information list.

Optionally, the current candidate motion information list building module 1403 is also configured to add the historical candidate motion information in the historical candidate motion information list to the current candidate motion information list of the current coding unit; correspondingly, the prediction module 1405 The motion information of the current coding unit will be obtained according to the current candidate motion information list of the current coding unit, and the current coding unit will be inter-predicted based on the obtained motion information.

Optionally, the prediction module 1405 is configured to obtain the motion information of the current coding unit according to a combination of the current candidate motion information list of the current coding unit and the historical candidate motion information list, and based on the obtained The motion information performs inter-frame prediction on the current coding unit; the prediction module 1405 is also configured to perform inter-frame prediction on another coding unit based on the same method as the current coding unit, wherein the other coding unit Located after the current coding unit according to the preset processing order and belonging to the coding tree unit with the current coding unit, the historical motion information list used in inter prediction of the other coding unit includes the current coding unit's Historical motion information in the historical motion information list used for inter-frame prediction.

Optionally, the historical candidate motion information list building module 1403 is configured to, in clockwise order, start from the spatial adjacent block in the lower left corner of the current coding tree unit, and start from the upper right corner of the current coding tree unit. The spatial adjacent block is the end point, and the motion information at L positions within the spatial adjacent block is added to the historical candidate motion information list.

Figure 15 is a coding device 1500 provided by the present invention. The coding device has the function of implementing the coding method in Figure 12. It includes: an inter-frame prediction device 1501 (same as the inter-frame prediction device 1400), which obtains the current encoding The inter-frame prediction image of the tree unit or the inter-frame prediction image of the current coding unit; the obtaining the inter-frame prediction image of the current coding tree unit or the inter-frame prediction image of the current coding unit includes: obtaining the inter-frame prediction image from the historical candidate motion information list and the Obtain the motion information of the current coding tree unit or current coding unit and the motion information index of the motion information from a combination of the current candidate motion information lists; according to the motion information of the current coding tree unit or current coding unit, The current coding tree unit or the current coding unit performs inter-frame prediction to obtain an inter-frame prediction image; the residual calculation module 1503 is used to compare the original image of the current coding tree unit or the current coding unit with the obtained inter-frame prediction image. Predicted images are subtracted to obtain a residual image; and an encoding module 1505 is used to encode the residual image and the motion information index to form a code stream.

Figure 16 is a decoding device 1600 provided by the present invention. The decoding device has the function of implementing the decoding method in Figure 13. It includes: an inter-frame prediction device 1601 (same as the inter-frame prediction device 1400), used for, Obtain the inter-frame prediction image of the current coding tree unit or the inter-frame prediction image of the current coding unit; and, a reconstruction module (1603), used to compare the obtained inter-frame prediction image with the current coding tree unit or the current coding tree unit. The residual images of the coding units are added to obtain the current coding tree unit or the reconstructed image of the current coding unit.

Figure 17 is a general schematic diagram of a device for implementing the methods in Figures 11 to 13 provided by the present invention. The device 1700 can be an inter-frame prediction device, a coding device, and a decoding device. The device includes There is an inter-frame prediction device, which includes a digital processor 1701 and a memory 1702. An executable instruction set is stored in the memory. The digital processor reads the instruction set stored in the memory for use. Implement the methods described in Figures 11 to 13.

In one or more instances, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted on a computer-readable medium as one or more instructions or code, and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media that correspond to tangible media, such as data storage media or communication media, including any medium that facilitates transfer of a computer program from one place to another according to a communications protocol. . In this manner, computer-readable media generally may correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or may be used to store instructions or data structures in the form of any other medium that contains the required program code and is accessible by a computer. Also, any connection is properly termed a computer-readable medium. For example, if coaxial cables, fiber optic cables, twisted pairs, digital subscriber lines (DSL), or wireless technologies such as infrared, radio, and microwave are used to transmit instructions from a website, server, or other remote source, the same Axial cables, fiber optic cables, twisted pairs, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are actually directed to non-transitory tangible storage media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks are usually magnetically Reproduce data, whereas discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (Application Specific Integrated Circuits). ASIC), field programmable logic arrays (FPGA) or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules for encoding and decoding, or incorporated into a synthetic codec. Furthermore, the techniques described may be entirely implemented in one or more circuits or logic elements.

The techniques of the present disclosure may be implemented in a variety of devices or devices including wireless handsets, integrated circuits (ICs), or collections of ICs (eg, chipsets). This disclosure describes various components, modules or units in order to emphasize functional aspects of devices for performing the disclosed techniques, but does not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit in conjunction with suitable software and/or firmware, or provided by a collection of interoperating hardware units including One or more processors.

Claims (19)

1. An encoding method, characterized in that it includes: Initialize the historical candidate motion information list corresponding to the current coding tree unit, wherein the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, and the initialized history The candidate motion information list includes at least M vacant storage spaces, where M≤N, M and N are integers, and the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, so The current coding tree unit is not the first one in the coding tree unit set according to a predetermined processing order; Add motion information at L positions within the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, where M≤L≤N within the spatial adjacent blocks The position is obtained according to preset rules; Constructing a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; and Obtained from a combination of the historical candidate motion information list and a current candidate motion information list of the current coding tree unit or from a combination of the historical candidate motion information list and a current candidate motion information list of the current coding unit The motion information of the current coding tree unit or the current coding unit, and the motion information index of the motion information; According to the motion information of the current coding tree unit or the current coding unit, perform inter prediction on the current coding tree unit or the current coding unit to obtain an inter prediction block; Subtract the current coding tree unit or the current coding unit from the obtained inter prediction block to obtain a residual block; and encode the residual block and the motion information index to form a code stream. 2. The method of claim 1, wherein the initializing the historical candidate motion information list corresponding to the current coding tree unit includes clearing the historical candidate motion information list such that M=N. 3. The method according to claim 1, characterized in that: the M positions in the adjacent blocks in the spatial domain are: the first candidate motion information is obtained from the preset positions in the adjacent blocks in the spatial domain to obtain the The position of the first candidate motion information is used as the starting point, and the remaining M-1 candidate motion information is obtained at preset step intervals. 4. The method of claim 3, wherein the preset step size is a fixed value, or the preset step size changes according to a preset rule. 5. The method according to any one of claims 1-4, characterized in that: the method further includes: The historical candidate motion information list is updated based on the motion information of the current coding unit. 6. The method of claim 5, wherein updating the historical candidate motion information list based on the motion information of the current coding unit includes: If the M positions are not filled, the motion information of the current coding unit is used as historical motion information and added to the vacant storage space closest to the N-M position among the M positions in the historical candidate motion information list. within; or; If the M positions have been filled, the historical motion information that was first added to the historical candidate motion information list will be removed according to the first-in, first-out principle, and the remaining historical motion information will be moved to the removed historical motion information position. After the shift, the motion information of the current coding unit is added to the end of the historical candidate motion information list as historical motion information, wherein the end of the historical candidate motion information list containing the latest added historical motion information is the The end of the list of historical candidate motion information. 7. The method according to any one of claims 1 to 6, characterized in that: adding the motion information at L positions in the spatial adjacent blocks of the current coding tree unit to the said The list of historical candidate sports information includes: In clockwise order, with the spatial adjacent block in the lower left corner of the current coding tree unit as the starting point and the spatial adjacent block in the upper right corner of the current coding tree unit as the end point, L within the spatial adjacent block is The motion information at positions is added to the historical candidate motion information list. 8. A decoding method, characterized by comprising: Initialize the historical candidate motion information list corresponding to the current coding tree unit, wherein the historical candidate motion information list includes N storage spaces, the N storage spaces are used to store historical candidate motion information, and the initialized history The candidate motion information list includes at least M vacant storage spaces, where M≤N, M and N are integers, and the current coding tree unit is included in a coding tree unit set (Slice) composed of multiple coding tree units, so The current coding tree unit is not the first one in the coding tree unit set according to a predetermined processing order; Add motion information at L positions within the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, where M≤L≤N within the spatial adjacent blocks The position is obtained according to preset rules; Constructing a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; and Obtained from a combination of the historical candidate motion information list and a current candidate motion information list of the current coding tree unit or from a combination of the historical candidate motion information list and a current candidate motion information list of the current coding unit The motion information of the current coding tree unit or the current coding unit, and the motion information index of the motion information; According to the motion information of the current coding tree unit or the current coding unit, perform inter-frame prediction on the current coding tree unit or the current coding unit to obtain an inter-frame prediction block; The obtained inter prediction block is added to the current coding tree unit or the residual block of the current coding unit to obtain the current coding tree unit or the reconstruction block of the current coding unit. 9. An encoding device, characterized by comprising: An initialization module, configured to initialize a historical candidate motion information list corresponding to the current coding tree unit, wherein the historical candidate motion information list includes N storage spaces, and the N storage spaces are used to store historical candidate motion information, The initialized historical candidate motion information list includes at least M vacant storage spaces, where M≤N, M and N are integers, and the current coding tree unit is included in a coding tree unit set composed of multiple coding tree units. (Slice), the current coding tree unit is not the first one in the coding tree unit set according to the predetermined processing order; A historical candidate motion information list building module configured to add motion information at L positions within the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, wherein: The M≤L≤N positions in the adjacent blocks in the above-mentioned airspace are obtained according to the preset rules; A current candidate motion information list construction module, configured to construct a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; as well as Obtaining module, configured to obtain from a combination of the historical candidate motion information list and the current candidate motion information list of the current coding tree unit or from the historical candidate motion information list and the current candidate motion information of the current coding unit. In the combination of lists, obtain the motion information of the current coding tree unit or the current coding unit, and the motion information index of the motion information; A prediction module, configured to perform inter-frame prediction on the current coding tree unit or the current coding unit according to the motion information of the current coding tree unit or the current coding unit, and obtain an inter-frame prediction block; A residual calculation module, configured to subtract the current coding tree unit or the current coding unit from the obtained inter prediction block to obtain a residual block; An encoding module, configured to encode the residual block and the motion information index to form a code stream. 10. The device of claim 9, wherein the initialization module is configured to initialize a historical candidate motion information list corresponding to the current coding tree unit, and is specifically configured to clear the historical candidate motion information list, so that M=N. 11. The device of claim 9, wherein the M positions in the spatial adjacent block are: the first candidate motion information is obtained from a preset position in the spatial adjacent block to obtain the The position of the first candidate motion information is used as the starting point, and the remaining M-1 candidate motion information is obtained at preset step intervals. 12. The device of claim 11, wherein the preset step size is a fixed value, or the preset step size changes according to a preset rule. 13. The device according to any one of claims 9-12, characterized in that: the device further includes: A historical motion information list update module, configured to update the historical candidate motion information list based on the motion information of the current coding unit. 14. The device of claim 13, wherein the historical motion information list update module updates the historical candidate motion information list according to the following rules: If the M positions are not filled, the motion information of the current coding unit is added as historical motion information to the vacant storage space closest to the N-M position among the M positions in the historical candidate motion information list. ;or; If the M positions have been filled, the historical motion information that was first added to the historical candidate motion information list will be removed according to the first-in, first-out principle, and the remaining historical motion information will be moved to the removed historical motion information position. After the shift, the motion information of the current coding unit is added to the end of the historical candidate motion information list as historical motion information, wherein the end of the historical candidate motion information list containing the latest added historical motion information is the historical motion information. The end of the list of candidate motion information. 15. The device according to any one of claims 9-14, characterized in that: the historical candidate motion information list building module is used to, In clockwise order, with the spatial adjacent block in the lower left corner of the current coding tree unit as the starting point and the spatial adjacent block in the upper right corner of the current coding tree unit as the end point, L within the spatial adjacent block is The motion information at positions is added to the historical candidate motion information list. 16. A decoding device, characterized in that it includes: An initialization module configured to initialize a historical candidate motion information list corresponding to the current coding tree unit, wherein the historical candidate motion information list includes N storage spaces, and the N storage spaces are used to store historical candidate motion information. , the initialized historical candidate motion information list includes at least M vacant storage spaces, where M≤N, M and N are integers, and the current coding tree unit is included in a coding tree unit composed of multiple coding tree units. In the set (Slice), the current coding tree unit is not the first one in the set of coding tree units according to the predetermined processing order; A historical candidate motion information list building module configured to add motion information at L positions within the spatial adjacent blocks of the current coding tree unit to the historical candidate motion information list in a predetermined order, wherein: The M≤L≤N positions in the adjacent blocks in the above-mentioned airspace are obtained according to the preset rules; A current candidate motion information list construction module, configured to construct a current candidate motion information list of the current coding tree unit or a current candidate motion information list of the current coding unit, wherein the coding unit is divided by the coding tree unit; as well as Obtaining module, configured to obtain from a combination of the historical candidate motion information list and the current candidate motion information list of the current coding tree unit or from the historical candidate motion information list and the current candidate motion of the current coding unit. In the combination of information lists, obtain the motion information of the current coding tree unit or the current coding unit, and the motion information index of the motion information; A prediction module, configured to perform inter-frame prediction on the current coding tree unit or the current coding unit according to the motion information of the current coding tree unit or the current coding unit, and obtain an inter-frame prediction block; A reconstruction module configured to add the obtained inter prediction block to the current coding tree unit or the residual block of the current coding unit to obtain the reconstruction of the current coding tree unit or the current coding unit. piece. 17. An encoding device, characterized in that: it includes a digital processor and a memory, an executable instruction set is stored in the memory, and the digital processor reads the instruction set stored in the memory for use Implement the encoding method as described in any one of claims 1-7. 18. A decoding device, characterized in that: it includes a digital processor and a memory, an executable instruction set is stored in the memory, and the digital processor reads the instruction set stored in the memory for use Implement the decoding method as claimed in claim 8. 19. A server, characterized in that the server includes a communication interface and a memory, the communication interface is used to receive and/or send a video code stream, the memory is used to store the video code stream, and the video code stream The stream is obtained based on the encoding method described in any one of claims 1-7.