BR122023025846A2 - DECODING APPARATUS FOR IMAGE DECODING, CODING APPARATUS FOR IMAGE ENCODING, COMPUTER READABLE STORAGE MEDIUM AND APPARATUS FOR TRANSMITTING DATA TO AN IMAGE - Google Patents

Abstract

According to embodiments herein, the subjective/objective quality of an image can be improved through image coding based on subimages and/or virtual boundaries, and consumption of hardware resources required for coding can be reduced.

Description

FUNDAMENTALS OF DISCLOSURE Field of Disclosure

[001] The present document relates to an image coding apparatus and method for controlling loop filtering.

Related Technique

[002] Recently, the demand for high-resolution and high-quality image/video such as 4K or 8K or ultra high definition (UHD) image/video has increased in various fields. Since image/video data has high resolution and high quality, the amount of information or bits to be transmitted increases relative to existing image/video data, and therefore transmission of image data using a medium such as a line existing wired or wireless broadband or an existing storage medium or storing image/video data using the existing storage medium increases the transmission cost and storage cost.

[003] Furthermore, interest and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms has increased recently and image/video transmission is having different characteristics from reality images, like game images, increased.

[004] Consequently, a highly efficient image/video compression technology is required to effectively compress, transmit, store and reproduce information from a high-resolution, high-quality image/video with various characteristics, as described above.

[005] Specifically, there is a discussion of a method for effectively controlling the loop filtering performed across virtual boundaries and a method for effectively signaling information in the subimages. summary

[006] According to an embodiment of the present document, a method and apparatus for increasing image coding efficiency are provided.

[007] According to an embodiment of the present document, efficient filtering application methods and apparatus are provided.

[008] According to an embodiment of the present document, a method and apparatus for effectively applying deblocking, adaptive sample shifting (SAO) and adaptive loop filtering (ALF) are provided.

[009] According to one embodiment of the present document, loop filtering can be performed based on virtual thresholds.

[010] According to one embodiment of the present document, whether a sequence parameter set (SPS) includes additional information related to virtual boundaries (e.g., information about the positions and number of virtual boundaries) can be determined based on the resampling for reference images is enabled.

[011] According to an embodiment of the present document, image coding can be performed based on subimages.

[012] According to one embodiment of the present document, the sub-images used in image coding can be independently encoded.

[013] According to one embodiment of the present document, an image may include only one subimage. Furthermore, the subimage can be encoded independently.

[014] According to an embodiment of the present document, an image can be generated based on a sub-image fusion process. Furthermore, the sub-images may be independently encoded sub-images.

[015] According to an embodiment of the present document, each of the sub-images used in image coding can be treated as an image.

[016] According to an embodiment of the present document, a coding apparatus for performing video/image coding is provided.

[017] In accordance with one embodiment of the present document, a computer-readable digital storage medium is provided on which encoded video/image information generated in accordance with the video/image coding method disclosed in at least one of the embodiments of this document, is stored.

[018] According to one embodiment of the present document, a computer-readable digital storage medium is provided on which encoded information or encoded video/image information, causing the video/image decoding method disclosed in at least at least one of the embodiments of this document by the decoding apparatus is stored.

[019] According to an embodiment of the present document, the overall image/video compression efficiency can be improved.

[020] According to one embodiment of the present document, subjective/objective visual quality can be improved through efficient filtering.

[021] According to an embodiment of the present document, the loop filtering process based on virtual thresholds can be effectively carried out and the filtering performance can be improved.

[022] According to one embodiment of the present document, information for loop filtering based on virtual boundaries can be signaled effectively.

[023] According to one embodiment of the present document, information related to the subimage can be signaled effectively. Therefore, the subjective/objective image quality can be improved and there can be a decrease in the consumption of hardware resources required for encoding.

BRIEF DESCRIPTION OF THE DRAWINGS

[024] Figure 1 schematically shows an example of a video/image coding system to which embodiments of the present disclosure can be applied.

[025] Figure 2 is a view that schematically illustrates the configuration of a video/image coding apparatus to which embodiments of the present disclosure can be applied.

[026] Figure 3 is a view that schematically illustrates the configuration of a video/image decoding apparatus to which embodiments of the present disclosure can be applied.

[027] Figure 4 exemplifies a hierarchical architecture for an encoded video/image.

[028] Figure 5 illustrates an example of a form of adaptive loop filtering (ALF) filter.

[029] Figure 6 is a flowchart illustrating a coding method based on filtering in a coding apparatus.

[030] Figure 7 is a flowchart illustrating a decoding method based on filtering in a decoding apparatus.

[031] Figure 8 and Figure 9 schematically show an example of a video/image coding method and related components in accordance with the embodiment(s) of the present document.

[032] Figure 10 and Figure 11 schematically show an example of an image/video decoding method and related components in accordance with an embodiment(s) of the present document.

[033] Figure 12 shows an example of a content streaming system to which the modality(s) disclosed in this document can be applied.

DESCRIPTION OF EXAMPLE MODALITIES

[034] The present disclosure can be modified in various ways, and its specific embodiments will be described and illustrated in the drawings. However, the modalities are not intended to limit disclosure. The terms used in the following description are used to merely describe specific embodiments, but are not intended to limit the disclosure. A singular number expression includes a plural number expression as long as it is clearly read differently. Terms such as “include” and “have” are intended to indicate that there are features, numbers, steps, operations, elements, components or combinations thereof used in the following description and therefore it should be understood that the possibility of existence or addition of one or more different features, numbers, steps, operations, elements, components or combinations thereof is not excluded.

[035] Meanwhile, each configuration of the drawings described in this document is an independent illustration to explain functions as characteristics that are different from each other, and does not mean that each configuration is implemented by mutually different hardware or software. For example, two or more configurations can be combined to form a configuration, and a configuration can also be divided into multiple configurations. Without departing from the essence of this document, embodiments in which configurations are combined and/or separated are included within the scope of the claims.

[036] In the following, examples of the present embodiment will be described in detail with reference to the attached drawings. In addition, similar reference numbers are used to indicate similar elements throughout the drawings, and the same descriptions on similar elements will be omitted.

[037] This document refers to video/image coding. For example, the methods/embodiments disclosed in this document may be related to the versatile video coding (VVC) standard (ITU-T Rec. H.266), the next generation video/image coding standard after VVC, or other standards related to video coding (e.g. High Efficiency Video Coding (HEVC) standard (ITU-T Rec. H.265), Essential Video Coding (EVC) standard, AVS2 standard, and the like).

[038] This document suggests several video/image coding modalities, and the above modalities can also be carried out in combination with each other, unless otherwise specified.

[039] In this document, a video may refer to a series of images over time. An image generally refers to the unit that represents an image in a given period of time, and a slice/block refers to the unit that constitutes a part of the image in terms of encoding. A slice/block may include one or more coding tree units (CTUs). An image can consist of one or more slices/blocks. An image can consist of one or more groups of blocks. A block group can include one or more blocks.

[040] A pixel or a pel can mean a smaller unit that constitutes an image (or image). Additionally, 'sample' can be used as a term corresponding to a pixel. A swatch can generally represent a pixel or a value of a pixel, and can represent just a pixel/pixel value of a luma component or just a pixel/pixel value of a chroma component.

[041] A unit may represent a basic image processing unit. The unit may include at least one specific region of the image and information related to the region. A unit may include one luma block and two chroma blocks (e.g. cb, cr). The unit may be used interchangeably with terms such as block or area in some cases. In a general case, an M x N block may include samples (or samples matrices) or a set (or matrix) of M column and N row transform coefficients. Alternatively, the sample may mean a pixel value in the spatial domain, and when such a pixel value is transformed in the frequency domain, it may mean a transform coefficient in the frequency domain.

[042] In this document, the terms “/” and “,” should be interpreted to indicate “and/or”. For example, the expression “A/B” can mean “A and/or B”. Furthermore, “A, B” can mean “A and/or B”. Furthermore, “A/B/C” can mean “at least one of A, B and/or C”. Furthermore, “A/B/C” can mean “at least one of A, B and/or C”.

[043] Furthermore, in the document, the term “or” must be interpreted to indicate “and/or”. For example, the expression “A or B” may comprise 1) A only, 2) B only, and/or 3) A and B. In other words, the term “or” in this document should be interpreted to indicate “additionally or alternatively ”.

[044] In the present specification, “at least one of A and B” may mean “only A”, “only B” or “both A and B”. Furthermore, in this specification, the expression “at least one of A or B” or “at least one of A and/or B” can be interpreted in the same way as “at least one of A and B”.

[045] Furthermore, in the present specification, “at least one of A, B and C” may mean “A only”, “B only”, “C only” or “any combination of A, B and C”. Furthermore, “at least one of A, B or C” or “at least one of A, B and/or C” can mean “at least one of A, B and C”.

[046] Furthermore, the parentheses used in this specification may mean “for example”. Specifically, in case “prediction (intra prediction)” is expressed, it can be indicated that “intra prediction” is proposed as an example of “prediction”. In other words, the term “prediction” in the present specification is not limited to “intra prediction”, and it may be indicated that “intra prediction” is proposed as an example of “prediction”. Furthermore, even in the case where “prediction (i.e., intra prediction)” is expressed, it can be indicated that “intra prediction” is proposed as an example of “prediction”.

[047] In the present specification, the technical characteristics explained individually in a drawing can be implemented individually, or can be implemented simultaneously.

[048] Figure 1 illustrates an example of a video/image coding system to which the disclosure of this document can be applied.

[049] Referring to Figure 1, a video/image coding system may include a source device and a receiving device. The source device may transmit information or encoded video/image data to the receiving device via a digital storage medium or network in the form of a file or stream.

[050] The source device may include a video source, an encoding apparatus, and a transmitter. The receiving device may include a receiver, a decoding apparatus, and a renderer. The encoding apparatus may be called a video/image encoding apparatus, and the decoding apparatus may be called a video/image decoding apparatus. The transmitter may be included in the coding apparatus. The receiver may be included in the decoding apparatus. The renderer can include a view, and the view can be configured as a separate device or an external component.

[051] The video source may acquire video/image through a video/image capture, synthesis or generation process. The video source may include a video/image capture device and/or a video/image generating device. The video/image capture device may include, for example, one or more cameras, video/image files including previously captured video/images, and the like. The video/image generating device may include, for example, computers, tablets and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated using a computer or the like. In this case, the video/image capture process can be replaced by a related data generation process.

[052] The encoding apparatus can encode input video/image. The coding apparatus can perform a series of processes such as prediction, transform, and quantization for compression and coding efficiency. The encoded data (encoded video/image information) can be output in the form of a bit stream.

[053] The transmitter may transmit the encoded image/image information or data output in the form of a bit stream to the receiver of the receiving device via a digital storage medium or a network in the form of a file or stream. Digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like. The transmitter may include an element for generating a media file via a predetermined file format and may include an element for transmission over a transmission/communication network. The receiver can receive/extract the bit stream and transmit the received bit stream to the decoding apparatus.

[054] The decoding apparatus can decode the video/image by performing a series of processes, such as dequantization, inverse transform and prediction corresponding to the operation of the encoding apparatus.

[055] The renderer can render the decoded video/image. The rendered video/image can be displayed across the screen.

[056] Figure 2 is a diagram that schematically illustrates the configuration of a video/image coding apparatus to which the disclosure of the present document can be applied. Next, what is referred to as the video coding apparatus may include an image coding apparatus.

[057] Referring to Figure 2, the coding apparatus 200 may include and be configured with an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter predictor 221 and an intra predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a reconstructed block generator or reconstructor. The image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which have been described above, may be configured by one or more hardware components (e.g., computer chipsets). encoder or processors) according to an embodiment. Additionally, memory 270 may include a decoded picture buffer (DPB) and may also be configured by a digital storage medium. The hardware component may further include memory 270 as an internal/external component.

[058] The image partitioner 210 may divide an input image (or image, frame) input to the encoding apparatus 200 into one or more processing units. As an example, the processing unit may be called the coding unit (CU). In this case, the coding unit can be divided recursively according to a Quad-tree binary tree ternary tree (QTBTTT) structure of a coding tree unit (CTU) or the largest coding unit (LCU). For example, a coding unit may be divided into a plurality of coding units of a deeper depth based on a quadruple tree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, the quadruple tree structure is applied first and the binary tree structure and/or the ternary tree structure can be applied later. Alternatively, the binary tree structure can also be applied first. A coding process in accordance with the present disclosure may be carried out based on a final coding unit that is not further divided. In this case, based on the coding efficiency according to image characteristics or the like, the maximum coding unit can be directly used as the final coding unit, or as necessary, the coding unit can be recursively divided into deeper coding, such that a coding unit with an optimal size can be used as the final coding unit. Here, the coding process may include a process such as prediction, transformation and reconstruction to be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, each prediction unit and transform unit can be divided or partitioned from the aforementioned final coding unit. The prediction unit may be a sample prediction unit and the transform unit may be a unit for inducing a transform coefficient and/or a unit for inducing a residual signal of the transform coefficient.

[059] The unit can be used interchangeably with the term as a block or an area in some cases. Generally, an M x N block can represent samples composed of M columns and N rows or a group of transform coefficients. The sample may generally represent a pixel or a pixel value, and may also represent only the pixel/pixel value of a luma component, and may also represent only the pixel/pixel value of a chroma component. Sample can be used as the term corresponding to a pixel or pel configuring an image (or image).

[060] Subtractor 231 may generate a residual signal (residual block, residual samples, or residual sample matrix) by subtracting a prediction signal (predicted block, prediction samples, or prediction sample matrix) output of the predictor 220 from a signal of input image (original block, original samples or original sample matrix), and the generated residual signal is transmitted to the transformer 232. The predictor 220 can perform the prediction for a processing target block (hereinafter, referred to as a “block current”) and generate a predicted block including prediction samples for the current block. The predictor 220 can determine whether the intra prediction or inter prediction is applied to a current block or a CU unit. As described later in the description of each prediction mode, the predictor can generate various types of prediction-related information, such as prediction mode information, and transfer the generated information to the entropy encoder 240. The prediction information can be encoded into the entropy encoder 240 and output in the form of a bit stream.

[061] The intra predictor 222 may predict a current block with reference to samples within a current image. The referenced samples can be located neighboring the current block or can also be located far from the current block according to the prediction mode. The prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode or a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the fine degree of the prediction direction. However, this is illustrative and directional prediction modes that are more or less than the above number can be used according to the configuration. The intra predictor 222 may also determine the prediction mode applied to the current block using the prediction mode applied to the neighboring block.

[062] The inter predictor 221 may induce a predicted block from the current block based on a reference block (reference sample array) specified by a motion vector in a reference image. At this time, in order to decrease the amount of motion information transmitted in the between prediction mode, the motion information can be predicted in units of a block, a sub-block or a sample based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference image index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction or the like). In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current image and a temporal neighboring block existing in the reference image. The reference image including the reference block and the reference image including the temporal neighboring block may also be the same as each other and may also be different from each other. The temporal neighbor block may be called such a name as a collocated reference block, a collocated CU (colCU), or the like, and the reference image including the temporal neighbor block may also be called a collocated image (colPic). For example, inter predictor 221 may configure a candidate list of motion information based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and/or reference image index of the current block. Inter prediction can be performed based on various prediction modes, and for example, in the case of a skip mode and a merge mode, the inter predictor 221 can use the movement information of the neighboring block as the movement information of the current block. In case of skip mode, the residual signal cannot be transmitted unlike in blending mode. A motion vector prediction (MVP) mode can indicate the motion vector of the current block by using the motion vector of the neighboring block as a motion vector predictor and signaling a motion vector difference.

[063] The predictor 220 may generate a prediction signal based on various prediction methods described below. For example, the predictor can not only apply intra prediction or inter prediction to predict a block, but also simultaneously apply intra prediction and inter prediction. This can be called combined inter and intra prediction (CIIP). Furthermore, the predictor can perform an intra-block copy (IBC) to predict a block. Intrablock copy can be used for content/moving image coding of a game or the like, for example screen content coding (SCC). IBC basically performs prediction on the current image, but it can be performed in a similar way to inter prediction where a reference block is derived on the current image. That is, the IBC may use at least one of the interprediction techniques described in this document.

[064] The prediction signal generated through the inter predictor 221 and/or the intra predictor 222 can be used to generate a reconstructed signal or to generate a residual signal. Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). Here, the GBT means transform obtained from a graph when the relationship information between pixels is represented by the graph. CNT refers to the transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform process can be applied to blocks of square pixels with the same size, or it can be applied to blocks with a variable size instead of a square.

[065] The quantizer 233 can quantify the transform coefficients and transmit them to the entropy encoder 240, and the entropy encoder 240 can encode the quantized signal (information about the quantized transform coefficients) and output a bit stream. The information about the quantized transform coefficients can be termed as residual information. The quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector form based on a coefficient sweep order and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 can realize various encoding methods, such as, for example, exponential Golomb, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and the like. The entropy encoder 240 may encode information necessary for video/image reconstruction together or separately from the quantized transform coefficients (e.g., values of syntax elements and the like). The encoded information (e.g., encoded video/image information) may be transmitted or stored in the unit of a network abstraction layer (NAL) in the form of a bit stream. The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), an image parameter set (PPS), a sequence parameter set (SPS), or a set of video parameters (VPS). Additionally, video/image information may also include general restriction information. In the present document, information and/or syntax elements being signaled/transmitted to be described later may be encoded through the encoding process described above and be included in the bit stream. The bit stream can be transmitted over a network or can be stored on a digital storage medium. Here, the network may include a transmission network and/or a communication network and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like. A transmitter (not illustrated) transmitting a signal output from the entropy encoder 240 and/or a storage unit (not illustrated) storing the signal may be configured as an internal/external element of the coding apparatus 200 and, alternatively, the transmitter may be included in the entropy encoder 240.

[066] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual samples) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal to the output of the prediction signal. of predictor 220 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed samples, or reconstructed sample matrix). If there is no residue for the processing target block, as in the case where a skip mode is applied, the predicted block can be used as the reconstructed block. The generated reconstructed signal can be used for intra prediction of a next processing target block in the current image and can be used for inter prediction of a next image through filtering as described below.

[067] Meanwhile, chroma-scaled luma mapping (LMCS) can be applied during an image encoding and/or reconstruction process.

[068] Filter 260 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 270, specifically, in a DPB of memory 270. The various filtering methods may include, for example, deblocking filtering, an adaptive sample offset (SAO), an adaptive loop filter, a two-sided filter and the like. The filter 260 may generate various types of filtering-related information and transfer the generated information to the entropy encoder 290 as described later in the description of each filtering method. Information related to filtering may be encoded by entropy encoder 290 and output in the form of a bit stream.

[069] The modified reconstructed image transmitted to memory 270 can be used as a reference image in the inter predictor 221. When the inter prediction is applied through the coding apparatus, the prediction mismatch between the coding apparatus 200 and the coding apparatus decoding can be avoided and coding efficiency can be improved.

[070] The DPB of memory 270 may store the modified reconstructed image for use as a reference image in interpredictor 221. Memory 270 may store motion information of a block from which motion information in the current image is derived (or encoded). ) and/or movement information of blocks in the image, having already been reconstructed. The stored motion information may be transferred to the inter predictor 221 to be used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples from reconstructed blocks in the current image and may transfer the reconstructed samples to intra predictor 222.

[071] Figure 3 is a diagram to schematically explain the configuration of a video/image decoding apparatus to which the disclosure of the present document can be applied.

[072] Referring to Figure 3, the decoding apparatus 300 may include and configured with an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. The entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350, which have been described above, may be configured by one or more hardware components (e.g., chipsets or decoder processors) according to one embodiment. Additionally, memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include memory 360 as an internal/external component.

[073] When the bit stream including the video/image information is input, the decoding apparatus 300 may reconstruct the image in response to a process in which the video/image information is processed in the encoding apparatus illustrated in Figure 2 For example, the decoding apparatus 300 may derive the units/blocks based on block division-related information acquired from the bit stream. The decoding apparatus 300 may perform decoding using the processing unit applied to the encoding apparatus. Therefore, the processing unit for decoding may be, for example, a coding unit, and the coding unit may be divided according to the quadruple tree structure, the binary tree structure and/or the ternary tree structure of the tree encoding unit or the maximum encoding unit. One or more transform units may be derived from the coding unit. Furthermore, the reconstructed image signal decoded and output through the decoding apparatus 300 can be reproduced via a playback apparatus.

[074] The decoding apparatus 300 may receive an output signal from the encoding apparatus of Figure 2 in the form of a bit stream and the received signal may be decoded via the entropy decoder 310. For example, the entropy decoder 310 can analyze the bit stream to derive information (e.g., video/image information) necessary for image reconstruction (or image reconstruction). The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), an image parameter set (PPS), a sequence parameter set (SPS), or a set of video parameters (VPS). Additionally, video/image information may also include general restriction information. The decoding apparatus may further decode the image based on the parameter set information and/or the general constraint information. Signaled/received information and/or syntax elements described later in this document may be decoded and obtained from the bit stream. For example, entropy decoder 310 decodes information in the bitstream based on a coding method, such as exponential Golomb coding, CAVLC, or CABAC, and output syntax elements required for image reconstruction and quantized values of transform coefficients. for residue. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bit stream, determine a context model using a decoding target syntax element information, decoding information of a decoding target block or information from a symbol/bin decoded in a previous step, and perform an arithmetic decoding on the bin, predicting a probability of occurrence of a bin according to the determined context model, and generating a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model using the information from the decoded symbol/bin to a context model of a next symbol/bin after determining the context model. Information related to the prediction among the information decoded by the entropy decoder 310 may be provided to the predictor 330 and information about the residue on which the entropy decoding was performed in the entropy decoder 310, that is, the quantized transform coefficients and information of Related parameters may be introduced into the dequantizer 321. Additionally, information about filtering between information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not illustrated) for receiving an output signal from the encoding apparatus may further be configured as an internal/external element of the decoding apparatus 300, or the receiver may be a constituent element of the entropy decoder 310. Meanwhile, the decoding apparatus according to the present document may be termed as a video/image/ image decoding apparatus, and the decoding apparatus can be classified into an information decoder (video/image/image information decoder) and a sample decoder (video/image/image sample decoder). The information decoder may include the entropy decoder 310 and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the predictor 330, the adder 340, the filter 350, and the memory 360.

[075] The dequantizer 321 can dequantize the quantized transform coefficients to output the transform coefficients. The dequantizer 321 can rearrange the quantized transform coefficients into a two-dimensional block shape. In this case, the rearrangement can be performed based on a coefficient sweep order performed by the coding apparatus. The dequantizer 321 may perform dequantization for the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) and acquire the transform coefficients.

[076] The inverse transformer 322 inversely transforms the transform coefficients to acquire the residual signal (residual block, residual sample matrix).

[077] The predictor 330 can perform the prediction of the current block and generate a predicted block including the prediction samples from the current block. The predictor may determine whether intra prediction is applied or inter prediction is applied to the current block based on information about the prediction output of the entropy decoder 310 and determine a specific intra/inter prediction mode.

[078] The predictor may generate a prediction signal based on various prediction methods described below. For example, the predictor can not only apply intra prediction or inter prediction to predict a block, but also simultaneously apply intra prediction and inter prediction. This can be called combined inter and intra prediction (CIIP). Furthermore, the predictor can perform an intra-block copy (IBC) to predict a block. Intrablock copy can be used for content/moving image coding of a game or the like, for example screen content coding (SCC). IBC basically performs prediction on the current image, but it can be performed in a similar way to inter prediction in which a reference block is derived on the current image. That is, the IBC may use at least one of the interprediction techniques described in this document.

[079] The intra predictor 332 can predict the current block by referring to the samples in the current image. Said samples may be located in the vicinity of the current block, or may be located away from the current block according to the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra predictor 332 may determine the prediction mode to be applied to the current block using the prediction mode applied to the neighboring block.

[080] The inter predictor 331 may derive a predicted block for the current block based on a reference block (reference sample matrix) specified by a motion vector in a reference image. In this case, in order to reduce the amount of motion information being transmitted in inter-prediction mode, motion information can be predicted in unit blocks, sub-blocks or samples based on the correlation of motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference image index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction, and the like). In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current image and a temporal neighboring block existing in the reference image. For example, the inter predictor 331 may construct a candidate list of motion information based on neighboring blocks and derive a current block motion vector and/or a reference image index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information about the prediction may include information that indicates an inter-prediction mode for the current block.

[081] The adder 340 may generate a reconstructed signal (reconstructed image, reconstructed block or reconstructed sample matrix) by adding the obtained residual signal to the prediction signal output (predicted block or predicted sample matrix) of the predictor 330. If there is none residue to the processing target block, as in the case where a skip mode is applied, the predicted block can be used as the reconstructed block.

[082] Adder 340 can be called a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra prediction of a next block to be processed in the current image and, as described later, can also be output through filtering or can also be used for inter prediction of a next image.

[083] Meanwhile, a chroma scaled luma mapping (LMCS) can also be applied in the image decoding process.

[084] Filter 350 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 360, specifically, in a DPB of memory 360. The various filtering methods may include, for example, deblocking filtering, an adaptive sample shift, an adaptive loop filter, a two-sided filter and the like.

[085] The reconstructed (modified) image stored in the DPB of memory 360 can be used as a reference image in interpredictor 331. Memory 360 can store the motion information of the block from which the motion information in the current image is derived (or decoded) and/or the movement information of the blocks in the image has already been reconstructed. The stored motion information can be transferred to the inter predictor 331 to be used as the motion information of the spatial neighbor block or the motion information of the temporal neighbor block. The memory 360 may store reconstructed samples from reconstructed blocks in the current image and transfer the reconstructed samples to the intra predictor 332.

[086] In the present specification, the modalities described in the predictor 330, the dequantizer 321, the inverse transformer 322 and the filter 350 of the decoding apparatus 300 can also be applied in the same way or correspondingly to the predictor 220, the dequantizer 234, the inverse transformer 235 and the filter 260 of the coding apparatus 200.

[087] Meanwhile, as described above, when performing video coding, prediction is performed to improve compression efficiency. Through this, a predicted block including prediction samples for a current block as a block to be encoded (i.e., an encoding target block) can be generated. Here, the predicted block includes prediction samples in a spatial domain (or pixel domain). The predicted block is derived in the same way in an encoding apparatus and a decoding apparatus, and the encoding apparatus may signal information (residual information) about residues between the original block and the predicted block, instead of an original sample value of an original block, to the decoding apparatus, thus increasing the efficiency of image coding. The decoding apparatus may derive a residual block including residual samples based on the residual information, add the residual block and the predicted block to generate reconstructed blocks including reconstructed samples, and generate a reconstructed image including the reconstructed blocks.

[088] Residual information can be generated through a transform and quantization process. For example, the coding apparatus may derive a residual block between the original block and the predicted block, perform a transform process on residual samples (residual sample matrix) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients and signal-related residual information for the decoding apparatus (via a bit stream). Here, the residual information may include value information of the quantified transform coefficients, location information, a transform technique, a transform kernel, a quantification parameter, and the like. The decoding apparatus may perform the dequantization/inverse transform process based on the residual information and derive residual samples (or residual blocks). The decoding apparatus can generate a reconstructed image based on the predicted block and the residual block. Furthermore, for reference for interprediction of a subsequent image, the coding apparatus may also dequantize/back transform the quantized transform coefficients to derive a residual block and generate a reconstructed image based thereon.

[089] In this document, at least one of quantization/dequantization and/or transform/inverse transform may be omitted. When quantization/dequantization is omitted, the quantified transform coefficient can be termed as a transform coefficient. When the transform/inverse transform is omitted, the transform coefficient can be called the coefficient or residual coefficient or it can be called the transform coefficient for uniformity of expression.

[090] In this document, the quantified transform coefficient and the transform coefficient may be referred to as a transform coefficient and a scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient(s) and the information about the transform coefficient(s) may be signaled using the residual coding syntax. Transform coefficients can be derived based on the residual information (or information in the transform coefficient(s), and scaled transform coefficients can be derived by inversely transforming (scaling) the transform coefficients. Residual samples can be derived based on the inverse transform (transform) of the scaled transform coefficients. This may also be applied/expressed elsewhere in this document.

[091] The predictor of the encoding apparatus/decoding apparatus may derive prediction samples by performing inter-prediction in block units. The inter prediction may be a prediction derived in a manner that depends on data elements (e.g., sample values or motion information, etc.) of image(s) other than the current image. When inter prediction is applied to the current block, based on the reference block (reference sample matrices) specified by the motion vector in the reference image pointed by the reference image index, the predicted block (prediction sample matrix) for the current block can be derived. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks or samples based on the correlation between the motion information between neighboring blocks and the current block. The motion information may include the motion vector and the reference image index. The movement information may also include inter prediction type information (L0 prediction, L1 prediction, Bi prediction, etc.). When inter prediction is applied, the neighboring blocks may include a spatial neighboring block existing in the current image and a temporal neighboring block existing in the reference image. The reference image including the reference block and the reference image including the temporal neighboring block may be the same or different. The temporal neighboring block can be called a collocated reference block, collocated CU (colCU), etc., and a reference image including the temporally neighboring block can be called a collocated image (colPic). For example, a candidate list of motion information can be constructed based on blocks neighboring the current block and a flag or index information indicating which candidate is selected (used) to derive the motion vector and/or image index. reference of the current block can be signaled. Inter prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of a selected neighboring block. In skip mode, unlike blending mode, a residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, a motion vector from a selected neighboring block can be used as a motion vector predictor and a motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[092] The motion information may include L0 motion information and/or L1 motion information according to a type of inter prediction (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction can be termed as an L0 or MVL0 motion vector, and a motion vector in the L1 direction can be termed as an L1 or MVL1 motion vector. The prediction based on motion vector L0 can be called L0 prediction, the prediction based on motion vector L1 can be called L1 prediction, and the prediction based on motion vector L0 and motion vector L1 can be called bi- prediction. Here, the motion vector L0 may indicate a motion vector associated with the list of reference images L0 (L0), and the motion vector L1 may indicate a motion vector associated with the list of reference images L1 (L1). The L0 reference image list may include images that are prior to the current image in the output order as reference images, and the L1 reference image list may include images that are subsequent to the current image in the output order. Previous images can be called forward (reference) images and subsequent images can be called backward (reference) images. The L0 reference image list may further include images that are subsequent to the current image in the output order as reference images. In this case, previous images can be indexed first and subsequent images can be indexed next into the L0 reference image list. The L1 reference image list may also include images prior to the current image in the output order as reference images. In this case, subsequent images can be indexed first in reference image list 1 and previous images can be indexed next. Here, the output order may correspond to a picture order count (POC) order.

[093] Figure 4 exemplarily shows a hierarchical structure for an encoded image/video.

[094] With reference to Figure 4, the encoded image/video is divided into VCL (video coding layer) which deals with an image/video decoding process and itself, a subsystem that transmits and stores the encoded information and a network abstraction layer (NAL) that exists between the VCL and subsystems and is responsible for network adaptation functions.

[095] The VCL can generate VCL data including compressed image data (slice data) or generate parameter sets including a set of image parameters (Image Parameter Set: PPS), a set of sequence parameters (Set of sequence parameters: SPS), a video parameter set (Video Parameter Set: VPS) etc. or a supplemental enhancement information (SEI) message additionally required for the process of decoding an image.

[096] In NAL, a NAL unit can be generated by adding header information (NAL unit header) to a raw byte sequence payload (RBSP) generated in the VCL. In this case, RBSP refers to slice data, parameter sets, SEI messages, etc. generated in the VCL. The NAL unit header may include NAL unit type information specified in accordance with the RBSP data included in the corresponding NAL unit.

[097] As shown in the figure, the NAL unit can be divided into a VCL NAL unit and a non-VCL NAL unit according to the RBSP generated in the VCL. The VCL NAL unit may mean a NAL unit including information (sliced data) about an image, and the non-VCL NAL unit may mean a NAL unit containing information (parameter set or SEI message) necessary to decode an image.

[098] The above-described VCL NAL unit and the non-VCL NAL unit can be transmitted over a network by attaching header information in accordance with a subsystem data standard. For example, the NAL unit can be transformed into a data format of a predetermined standard, such as H.266/VVC file format, real-time transport protocol (RTP), transport stream (TS), etc. and transmitted across multiple networks.

[099] As described above, in the NAL unit, the NAL unit type can be specified according to the RBSP data structure included in the corresponding NAL unit, and information about this NAL unit type can be stored and signaled in the header of the NAL unit.

[0100] For example, the NAL unit can be roughly classified into the VCL NAL unit type and the non-VCL NAL unit type, depending on whether the NAL unit includes image information (slice data). The VCL NAL unit type can be classified according to the property and a type of image included in the VCL NAL unit, and the non-VCL NAL unit type can be classified according to the type of a parameter set.

[0101] The following is an example of the NAL unit type specified according to the parameter set type included in the non-VCL NAL unit type. - APS NAL Unit (Adaptation Parameter Set): Type for NAL unit including APS - DPS NAL Unit (Decoding Parameter Set): Type for NAL unit including DPS - VPS NAL Unit (Video Parameter Set) : Type for NAL unit including VPS - NAL Unit of SPS (Sequence Parameter Set): Type for NAL unit including SPS - NAL Unit of PPS (Picture Parameter Set): Type for NAL unit including PPS - NAL Unit PH (image header): Type for NAL unit including PH

[0102] The NAL unit types described above have syntax information for the NAL unit type, and the syntax information can be stored and signaled in the NAL unit header. For example, syntax information can be nal_unit_type and NAL unit types can be specified by a nal_unit_type value.

[0103] Meanwhile, as described above, an image may include a plurality of slices, and a slice may include a slice header and slice data. In this case, an image header may further be added to a plurality of slices (a slice header and a slice data set) in an image. The image header (image header syntax) may include information/parameters commonly applicable to the image. In this document, a slice can be mixed or replaced by a group of blocks. Additionally, in this document, a slice header can be mixed or replaced with a type group header.

[0104] The slice header (slice header syntax or slice header information) may include information/parameters commonly applicable to the slice. APS (APS syntax) or PPS (PPS syntax) may include information/parameters commonly applicable to one or more slices or images. The SPS (SPS syntax) may include information/parameters commonly applicable to one or more sequences. VPS (VPS syntax) may include information/parameters commonly applicable to multiple layers. DPS (DPS syntax) may include information/parameters commonly applicable to the entire video. The DPS may include information/parameters related to the concatenation of a coded video sequence (CVS). In this document, high-level syntax (HLS) may include at least one APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, image header syntax, and slice header syntax.

[0105] Herein, image/video information encoded in the encoding apparatus and signaled in the form of a bit stream to the decoding apparatus may include, as well as information relating to image-in-image partitioning, intra/inter-image information. prediction, residual information, loop filtering information, etc. the information included in the slice header, the information included in the image header, the information included in the APS, the information included in the PPS, the information included in the SPS, the information included in the VPS, and/or the information included in the DPS. Additionally, the image/video information may also include NAL unit header information.

[0106] Meanwhile, in order to compensate for a difference between an original image and a reconstructed image due to an error occurring in a compression coding process such as quantization, a loop filtering process can be performed on reconstructed samples or reconstructed images as described above. As described above, loop filtering can be performed by the encoding apparatus filter and the decoding apparatus filter, and a deblocking filter, SAO and/or adaptive loop filter (ALF) can be applied. For example, the ALF process may be performed after the unlocking filtering process and/or the SAO process is completed. However, even in this case, the unlock filtering process and/or the SAO process can be omitted.

[0107] Next, image reconstruction and filtering will be described in detail. In image/video coding, a reconstructed block can be generated based on intra prediction/inter prediction in each block unit and a reconstructed image including the reconstructed blocks can be generated. When the current image/slice is an I image/slice, the blocks included in the current image/slice can be reconstructed based on intra prediction alone. Meanwhile, when the current image/slice is a P or B image/slice, the blocks included in the current image/slice can be reconstructed based on intra prediction or inter prediction. In this case, intra prediction can be applied to some blocks in the current image/slice and inter prediction can be applied to the remaining blocks.

[0108] The intra prediction may represent a prediction to generate prediction samples for the current block based on reference samples in the image (hereinafter, current image) to which the current block belongs. If intra prediction is applied to the current block, neighboring reference samples can be derived to be used for intra prediction of the current block. The neighboring reference samples of the current block may include a sample adjacent to a left boundary of the current block with a size of nWxnH, total 2xnH samples neighboring the bottom left, a sample adjacent to the upper boundary of the current block, total of 2xnW samples neighboring the top right corner and a sample neighboring the top left corner of the current block. Alternatively, the neighboring reference samples of the current block may include multi-column top neighbor sample and multi-row left neighbor sample. Alternatively, the neighboring reference samples of the current block may include total nH samples adjacent to the right boundary of the current block with a size of nWxnH, total nH samples adjacent to the right boundary of the current block, total nW samples adjacent to the lower boundary of the current block. current block and a neighboring sample to the bottom right corner of the current block.

[0109] However, some of the neighboring reference samples of the current block may not yet have been decoded or may not be available. In this case, the decoder can configure neighboring reference samples to be used for prediction by replacing available samples with unavailable samples. Alternatively, neighboring reference samples to be used for prediction can be configured by interpolating the available samples.

[0110] When neighboring reference samples are derived, there are two cases, that is, a case (i) in which a prediction sample can be derived based on an average or interpolation of neighboring reference samples of a current block, and a case (ii) in which the prediction sample can be derived based on a reference sample present in a specific (prediction) direction for the prediction sample among the neighboring reference samples of the current block. Case (i) can be called non-directional mode or non-angular mode, and case (ii) can be called directional mode or angular mode. Furthermore, the prediction sample can also be generated through a first neighboring sample and a second neighboring sample located in a direction opposite to the prediction direction of the current block intra prediction mode based on the prediction sample of the current block between the neighboring reference samples. The above case can be termed as linear interpolation intra prediction (LIP). Additionally, chroma prediction samples can be generated based on luma samples using a linear model. This case can be called LM mode. Furthermore, a temporary prediction sample of the current block can be derived based on filtered neighboring reference samples. At least one reference sample, which is derived according to the intra prediction mode between the existing neighboring reference samples, that is, unfiltered neighboring reference samples, and the temporary prediction sample can be weighted summed to derive the sample prediction of the current block. The above case can be called position-dependent intra prediction (PDPC). Furthermore, a reference sample line with the highest prediction accuracy among the multiple neighboring reference sample lines of the current block can be selected to derive the prediction sample using the reference sample located in the prediction direction on the corresponding line and the reference sample The line used in this document can be indicated (flagged) to a decoding device, thus performing intra prediction coding. The above case can be called multiple reference line (MRL) intra prediction or MRL-based intra prediction. Furthermore, intra prediction can be performed based on the same intra prediction mode by dividing the current block into vertical or horizontal sub-partitions, and neighboring reference samples can be derived and used in units of the sub-partitions. That is, in this case, the intra prediction mode for the current block is equally applied to the sub-partitions, and the intra prediction performance can be improved in some cases by deriving and using the neighboring reference samples in units of the sub-partitions. Such a prediction method can be called intra sub-partition (ISP) or ISP-based intra-prediction. The above-mentioned intra prediction methods can be called a type of intra prediction distinct from the intra prediction mode in sections 1.2. The intra prediction type may be called in various terms, such as an intra prediction technique or an additional intra prediction mode or the like. For example, the intra prediction type (or additional intra prediction mode or the like) may include at least one of the aforementioned LIP, PDPC, MRL and ISP. A general intra prediction method, except for the specific intra prediction type such as LIP, PDPC, MRL or ISP, can be called a normal intra prediction type. The normal intra prediction type can be generally applied when the specific intra prediction type is not applied, and the prediction can be performed based on the intra prediction mode mentioned above. Meanwhile, optionally, post-processing filtering can be performed on the derived prediction sample.

[0111] Specifically, the intra prediction process may include an operation of determining an intra prediction mode/type, an operation of deriving a neighboring reference sample, and an operation of deriving a prediction sample based on the mode/ type of intra prediction. Additionally, optionally, a post-processing filtering operation can be performed on the derived prediction sample.

[0112] A modified reconstructed image can be generated through the loop filtering process, and the modified reconstructed image can be output as a decoded image in the decoding apparatus and can also be stored in a decoded image buffer or memory of the decoding apparatus. encoding/decoding and used as a reference image in the inter prediction process when the image is encoded/decoded later. The loop filtering process may include a deblocking filtering process, an adaptive sample shift (SAO) process, and/or an adaptive loop filter (ALF) process as described above. In this case, one or some of the unlocking filtering process, SAO process, ALF process and bilateral filtering process may be applied sequentially, or all of the processes may be applied sequentially. For example, the SAO process can be performed after the deblocking filtering process is applied to the reconstructed image. Alternatively, for example, the ALF process can be performed after the deblocking filtering process is applied to the reconstructed image. This can also be carried out on the coding device.

[0113] Deblocking filtering is a filtering technique that removes distortion that occurs at the boundaries between blocks in the reconstructed image. The deblocking filtering process may, for example, derive a target boundary on the reconstructed image, determine a threshold strength (bS) for the target boundary, and perform deblocking filtering on the target boundary based on the bS. The bS can be determined based on a prediction mode, a motion vector difference, whether the reference images are identical, whether there is a significant non-zero coefficient, etc., of two blocks adjacent to the target boundary.

[0114] SAO is a method in which a difference in displacement between the reconstructed image and the original image is compensated based on the sample. For example, SAO can be applied based on a type such as a band shift, an edge shift, or the like. According to SAO, samples can be classified into different categories according to each type of SAO, and a compensation value can be added to each sample based on the category. Filter information for SAO may include information about whether SAO is applied, SAO type information, SAO offset value information, or the like. SAO can be applied to the reconstructed image after applying deblocking filtering.

[0115] ALF is a technique for filtering the reconstructed image on a sample basis, based on filter coefficients according to a filter shape. The encoding apparatus may determine whether ALF is applied, a form of ALF and/or an ALF filtering coefficient or the like by comparing the reconstructed image and the original image and may signal the result of the determination to the decoding apparatus. That is, the filtering information for the ALF may include information about whether the ALF is applied, ALF filter shape information, ALF filtering coefficient information, or the like. ALF can be applied to the reconstructed image after applying deblocking filtering.

[0116] Figure 5 illustrates an example of an ALF filter form.

[0117] Figure 5, (a) illustrates a 7x7 diamond filter shape and (b) illustrates a 5 x 5 diamond filter shape. In Figure 5, Cn in the filter shape represents a filter coefficient. When n is identical in Cn, it means that the same filter coefficient can be allocated. In the present document, a location and/or unit in which the filter coefficient is allocated according to the shape of the ALF filter may be called a filter tap. In this case, a filter coefficient can be assigned to each filter tap and a shape in which the filter tap is arranged can correspond to a filter shape. A filter tap located in the center of the filter shape can be called a center filter tap. The same filter coefficient can be assigned to two filter taps with the same value n and present in corresponding locations relative to the central filter tap. For example, in the case of the 7 x 7 diamond filter shape, 25 filter taps are included and the filter coefficients C0 to C11 are allocated in a central symmetrical shape. Therefore, filter coefficients can be allocated to the 25 filter taps using only 13 filter coefficients. Furthermore, for example, in the case of the 5 x 5 diamond filter shape, 13 filter taps are included and the filter coefficients C0 to C5 are allocated to the central symmetrical shape. Therefore, filter coefficients can be allocated to the 13 filter taps using only 7 filters. For example, in order to reduce a data amount of information about the filter coefficients to be flagged, 12 filter coefficients out of the 13 filter coefficients for the 7 x 7 diamond filter shape can be (explicitly) flagged and a coefficient filter can be (implicitly) derived. Furthermore, for example, 6 filter coefficients of the 7 filter coefficients for the 5 x 5 diamond filter shape can be (explicitly) signed and a filter coefficient can be (implicitly) derived.

[0118] Figure 6 is a flowchart illustrating a coding method based on filtering in a coding apparatus. The method of Figure 6 may include steps S600 to S630.

[0119] In step S600, the coding apparatus can generate a reconstructed image. Step S600 can be performed based on the above-mentioned reconstructed image (or reconstructed samples) generation process.

[0120] In step S610, the coding apparatus may determine whether loop filtering is applied (through a virtual threshold) based on information related to loop filtering. Here, the loop filtering may include at least one of the above-mentioned deblocking filtering, SAO and ALF.

[0121] In step S620, the encoding apparatus may generate a modified reconstructed image (modified reconstructed samples) based on the determination in step S610. Here, the modified reconstructed image (modified reconstructed samples) can be a filtered reconstructed image (filtered reconstructed samples).

[0122] In step S630, the encoding apparatus may encode image/video information including information related to loop filtering, based on the loop filtering process.

[0123] Figure 7 is a flowchart illustrating a decoding method based on filtering in a decoding apparatus. The method of Figure 7 may include steps S700 to S730.

[0124] In step S700, the decoding apparatus may obtain image/video information including information related to loop filtering of a bit stream. Here, the bit stream may be based on encoded image/video information transmitted from the encoding apparatus.

[0125] In step S710, the decoding apparatus may generate a reconstructed image. Step S710 can be performed based on the above-mentioned reconstructed image (or reconstructed samples).

[0126] In step S720, the decoding apparatus may determine whether loop filtering is applied (through a virtual threshold) based on information related to loop filtering. Here, the loop filtering may include at least one of the above-mentioned deblocking filtering, SAO and ALF.

[0127] In step S730, the decoding apparatus may generate a modified reconstructed image (modified reconstructed samples) based on the determination in step S720. Here, the modified reconstructed image (modified reconstructed samples) can be a filtered reconstructed image (filtered reconstructed samples).

[0128] As described above, the loop filtering process can be applied to the reconstructed image. In this case, a virtual threshold can be defined to further improve the subjective/objective visual quality of the reconstructed image, and the loop filtering process can be applied across the virtual threshold. The virtual boundary may include, for example, a discontinuous border such as a 360-degree image, a VR image, a boundary, a Picture In Picture (PIP), or the like. For example, the virtual boundary may be present at a predetermined location and a presence and/or location thereof may be signaled. For example, the virtual boundary may be located on an upper fourth sample line of a CTU line (specifically, for example, above the upper fourth sample of the CTU line). As another example, information about the presence and/or location of the virtual boundary can be signaled via HLS. HLS may include SPS, PPS, image header, slice header, or the like, as described above.

[0129] Next, a high-level syntax signaling and semantics will be described in accordance with embodiments of the present disclosure.

[0130] An embodiment of the present document may include a method of controlling circuit filters. The present method for controlling loop filters can be applied to a reconstructed image. Loop filters (loop filters) can be used for decoding encoded bit streams. Loop filters can include the above-mentioned deblocking, SAO and ALF. The SPS may include flags related to each of unlock, SAO and ALF. Flags can indicate whether each of the tools is available for encoding a coded layer video sequence (CLVS) or coded video sequence (CVS) relative to the SPS.

[0131] In one example, when loop filters are enabled for encoding images in CVS, the application of the loop filters can be controlled so as not to exceed specific limits. For example, loop filters can be controlled not to cross subimage boundaries, loop filters can be controlled not to cross block boundaries, loop filters can be controlled not to cross slice boundaries, and/or The loop filters can be controlled not to exceed virtual limits.

[0132] Information related to loop filtering may include information, syntax, syntax and/or semantic elements described herein (or embodiments therein). Information related to loop filtering may include information related to whether (all or part of) a loop filtering process is enabled at specific boundaries (e.g., a virtual boundary, a subimage boundary, a slice boundary, and/or or a block limit). Image information included in a bitstream may include high-level syntax (HLS), and HLS may include information related to loop filtering. Modified (or filtered) reconstructed samples (reconstructed images) can be generated based on determining whether the loop filtering process is applied across the specific thresholds. In one example, when the loop filtering process is disabled for all blocks/thresholds, the modified reconstructed samples may be identical to the reconstructed samples. In another example, the modified reconstructed samples may include modified reconstructed samples derived based on in-loop filtering. However, in this case, some of the reconstructed samples (e.g., samples reconstructed across the virtual boundaries) may not be loop filtered based on the aforementioned determination. For example, samples reconstructed at a specific boundary (at least one of a virtual boundary, subimage boundary, slice boundary, and/or block boundary for which loop filtering is enabled) can be loop filtered, but reconstructed samples other boundaries (for example, a virtual boundary, subimage boundary, slice boundary, and/or block boundary for which loop filtering is disabled) cannot be loop filtered.

[0133] In an example, whether the loop filtering process is performed across the virtual boundary, the loop filtering related information may include an SPS virtual boundaries presence flag, an image header virtual boundaries flag , information on the number of virtual limits, information on positions of virtual limits or the like.

[0134] In the embodiments included herein, information about the position of the virtual boundaries may include information about an x-coordinate of a vertical virtual boundary and/or information about a y-coordinate of a horizontal virtual boundary. Specifically, information about the position of the virtual boundaries may include information on the x-coordinate of the vertical virtual boundary and/or information on the y-axis of the horizontal virtual boundary in units of luma samples. Additionally, information about the position of the virtual boundaries may include information about the number of pieces of information (syntax elements) in the x-coordinate of the vertical virtual boundary that are present in the SPS. Additionally, information about the position of the virtual boundaries may include information about the number of pieces of information (syntax elements) in the y-coordinate of the horizontal virtual boundary that are present in the SPS. Alternatively, information about the position of the virtual boundaries may include information about the number of pieces of information (syntax elements) in the x-coordinate of the vertical virtual boundary that are present in an image header. Additionally, information about the position of the virtual boundaries may include information about the number of pieces of information (syntax elements) in the y-coordinate of the horizontal virtual boundary that is present in the image header.

[0135] The following tables show an exemplary syntax and semantics of an SPS according to the present embodiment. Table 1

[0136] The following tables show an example of syntax and semantics of an image parameter set (PPS) according to the present embodiment. Table 3 Table 4

[0137] The following tables show an exemplary syntax and semantics of an image header in accordance with the present embodiment. Table 5

[0138] The following tables show an exemplary syntax and semantics of a slice header in accordance with the present embodiment. Table 7

[0139] Next, information related to subimages, information related to virtual limits usable in mesh filtering and their signaling will be described.

[0140] When there is no sub-image with boundaries treated as image boundaries, even if an image includes multiple sub-images, the advantage of using the sub-image cannot be utilized. In one embodiment of this document, the image/video information for image coding may include information for treating a subpicture as a picture, which may be referred to as an image manipulation flag (i.e., subpic_treated_as_pic_flag[i]).

[0141] In order to signal a subpicture layout, a flag (i.e., subpic_present_flag) related to the presence of subpictures is flagged. This can be called a subimage present flag. When a value of subpic_present_flag is 1, information (i.e. sps_num_subpics_minus1) related to the number of subpictures to partition the image is flagged. In an example, the number of subpics to partition the image might be equal to sps_num_subpics_minus1 + 1 (1 is added to sps_num_subpics_minus1). The available values of sps_num_subpics_minus1 include 0, which means that only one subpicture can be present in the image. If the image only includes one sub-image, signaling information related to the sub-image can be considered a redundant process since the sub-image itself is an image.

[0142] In the existing embodiment, if the image includes only one subimage and subimage flag is present, a value of the image manipulation flag (i.e., subpic_treated_as_pic_flag[i]) and/or a value of a flag (i.e., loop_filter_across_subpic_enabled_flag) related to whether loop filtering is performed on subpictures can be 0 or 1. Here, there is an issue in which a case in which the value of subpic_treated_as_pic_flag[i] is 0 is inconsistent with a case in which the boundaries of the subimage are the boundaries of the image. This requires an additional redundant process in which a decoder can confirm that the image boundaries are the subimage boundaries.

[0143] When an image is generated based on a merging process of two or more subimages, all subimages used in the merging process must be independently encoded subimages (a subimage in which the value of the image manipulation flag (subpic_reated_as_pic_flag[ i]) is 1). This is because when a sub-image (referred to as a 'first sub-image' in this paragraph) other than the independently encoded sub-image is merged, the blocks in the first sub-image are encoded by reference to a reference block existing outside the first sub-image, which may cause a problem after the merger.

[0144] Furthermore, when the image is divided into subimages, subimage identification signaling may or may not be present. When sub-picture ID signaling is present, the sub-picture ID signaling may be present (included) in an SPS, a PPS, and/or a picture header (PH). A case in which subimage ID signaling is not present in the SPS may include a case in which a bit stream is generated as a result of a subimage merging process. Consequently, it is preferred that all subpictures be independently encoded when subpicture ID signaling is not included in the SPS.

[0145] In an image coding process in which virtual boundaries are used, information about the positions of the virtual boundaries may be signaled in an SPS or an image header. Signaling information about virtual boundary positions in the SPS means that there is no change in positions across the entire CLVS. However, when reference image resampling (RPR) is enabled for CLVS, images in CLVS may have different sizes. Here, RPR (also called adaptive resolution shifting (ARC)) is performed for a normal encoding operation of images with different resolutions (spatial resolutions). For example, the RPR may include increasing sampling rate and decreasing sampling rate. High coding efficiency for adapting a bitrate and spatial resolution can be achieved through RPR. It is necessary to ensure that all virtual boundary positions are present in an image, considering the RPR.

[0146] In the existing ALF process, the exponential Golomb code of order k with k=3 is used to signal absolute values of luma and chroma ALF coefficients. However, exponential Golomb coding of order k is problematic because it causes significant computational overhead and complexity.

[0147] The modalities described below can propose solutions to the aforementioned problem. The modalities can be applied independently. Alternatively, at least two modalities can be applied in combination.

[0148] In one embodiment of this document, when subpicture signaling is present and a picture has only one subpicture, the only subpicture is an independently encoded subpicture. For example, when the image has only one subimage, the only subimage is an independently encoded subimage, and a value of an image manipulation flag (that is, subpic_treated_as_pic_flag[i]) for the single subimage is 1. Consequently, a redundant process related to the subimage can be omitted.

[0149] In one embodiment of this document, when subpicture flag is present, the number of subpictures may be greater than 1. In one example, if subpicture flag is present (i.e., a value of subpics_present_flag is 1), information about the number of subpics (that is, sps_num_subpics_minus1) can be greater than 0, and the number of subpics can be sps_num_subpics_minus1 + 1 (1 is added to sps_num_subpics_minus1). In another example, the information about the number of subpics might be sps_num_subpics_minus2, and the number of subpics might be sps_num_subpics_minus2 + 2 (2 is added to sps_num_subpics_minus2). In another example, a subpicture present flag (subpics_present_flag) may be replaced by information about the number of subpictures (sps_num_subpics_minus1), and therefore the subpicture flag may be present when sps_num_subpics_minus1 is greater than 0.

[0150] In one embodiment of this document, when an image is divided into sub-images, at least one sub-image among the sub-images may be an independently encoded sub-image. Here, a value of an image manipulation flag (i.e., subpic_treated_as_pic_flag[i]) for the independently encoded subimage can be 1.

[0151] In one embodiment of this document, subimages of an image based on a process of merging two or more subimages may be independently encoded subimages.

[0152] In one embodiment of this document, when the sub-picture identification (ID) flag is present in a position other than an SPS (other syntax, other high-level syntax information), all sub-pictures may be independently encoded sub-pictures and a value of an image manipulation flag (i.e., subpic_treated_as_pic_flag) for all subimages may be 1. In an example, the subimage ID flag may be present in a PPS, in which case all subimages may be subimages independently coded. In another example, the sub-image ID flag may be present in an image header, in which case all sub-images are independently encoded sub-images.

[0153] In one embodiment of this document, when virtual boundary flagging is present in an SPS for CLVS and reference image resampling is enabled, all horizontal virtual boundary positions may be within a minimum image height of images relative to the SPS and all vertical positions, the virtual boundary positions may be within a minimum image width of the images relative to the SPS.

[0154] In one embodiment of this document, when reference image resampling (RPR) is enabled, virtual boundary flagging may be included in an image header. That is, when RPR is enabled, virtual boundary signaling may not be included in the SPS.

[0155] In one embodiment of this document, fixed-length coding (FLC) with the number of bits (or bit length) can be used to signal ALF data. In an example, information about the ALF data may include information about a bit length of an absolute value of an ALF luma coefficient (i.e., alf_luma_coeff_abs_len_minus1) and/or information about a bit length of an absolute value of a ALF chroma coefficient (i.e. alf_chroma_coeff_abs_len_minus1). For example, information about the bit length of the absolute value of the ALF luma coefficient and/or the information about the bit length of the absolute value of the ALF chroma coefficient may be encoded with ue(v).

[0156] The following table shows an exemplary syntax of the SPS according to the present embodiment. Table 9

[0157] The following table shows an exemplary semantics of syntax elements included in the syntax. Table 10

[0158] The following table shows an exemplary syntax of the SPS according to the present embodiment. Table 11

[0159] The following table shows an exemplary semantics of syntax elements included in the syntax. Table 12

[0160] The following table shows an exemplary syntax of ALF data according to the present embodiment. Table 13

[0161] The following table shows an exemplary semantics of syntax elements included in the syntax. Table 14

[0162] According to the embodiments of the present document described in conjunction with the tables above, through image coding based on subimages and/or virtual thresholds, the subjective/objective image quality may be improved and there may be a decrease in the consumption of hardware resources required for encoding.

[0163] Figure 8 and Figure 9 schematically show an example of a video/image coding method and related components in accordance with the embodiment(s) of the present document.

[0164] The method disclosed in Figure 8 can be carried out by the coding apparatus disclosed in Figure 2 or Figure 9. Specifically, for example, S800 and S810 of Figure 8 can be carried out by a residual processor 230 of the coding apparatus of Figure 9 , S820 and/or S830 of Figure 8 may be realized by a filter 260 of the coding apparatus of Figure 9 and S840 of Figure 8 may be realized by an entropy encoder 240 of the coding apparatus of Figure 9. Furthermore, although not shown in Figure 8, prediction samples or prediction-related information may be derived by a predictor 220 of the coding apparatus of Figure 8, and a bit stream may be generated from residual information or prediction-related information by the entropy encoder. 240 of the coding device. The method disclosed in Figure 8 may include the above-mentioned embodiments herein.

[0165] Referring to Figure 8, the coding apparatus can derive residual samples (S800). The encoding apparatus may derive residual samples for a current block, and residual samples for the current block may be derived based on original samples and prediction samples of the current block. Specifically, the coding apparatus may derive prediction samples from the current blocks based on a prediction mode. In this case, various prediction methods disclosed herein, such as inter prediction or intra prediction, can be applied. Residual samples can be derived based on the prediction samples and the original samples. For inter prediction, the coding apparatus may derive at least one reference image and may perform the inter prediction based on the at least one reference image. Prediction samples can be generated based on inter prediction. The encoding apparatus may generate information related to the reference image based on at least one reference image.

[0166] The coding apparatus can derive transform coefficients. The coding apparatus may derive the transform coefficients based on a transform process for the residual samples. For example, the transform process may include at least one of a DCT, a DST, a GBT and a CNT.

[0167] The coding apparatus can derive quantized transform coefficients. The coding apparatus may derive the quantified transform coefficients based on a quantification process for the transform coefficients. Quantized transform coefficients can have a one-dimensional vector form, based on a coefficient sweep order.

[0168] The coding apparatus may generate residual information (S810). The coding apparatus can generate residual information based on the transform coefficients. The coding apparatus may generate residual information indicating the quantized transform coefficients. Residual information can be generated through various coding methods, such as exponential Golomb, CAVLC, CABAC or similar.

[0169] The coding apparatus can generate reconstructed samples. The coding apparatus can generate the reconstructed samples based on the residual information. The reconstructed samples can be generated by adding the prediction sample and residual samples based on the residual information. Specifically, the coding apparatus can perform prediction (intra or inter prediction) on the current block and can generate reconstructed samples, based on the original samples and the prediction samples generated from the prediction.

[0170] The reconstructed samples may include reconstructed luma samples and reconstructed chroma samples. Specifically, residual samples may include residual luma samples and residual chroma samples. Residual luma samples can be generated based on original luma samples and prediction luma samples. Residual chroma samples can be generated based on the original chroma samples and the prediction chroma samples. The coding apparatus may derive transform coefficients for the residual luminance samples (luminance transform coefficients) and/or transform coefficients for the residual chroma samples (chroma transform coefficients). Quantized transform coefficients may include quantized luma transform coefficients and/or quantized chroma transform coefficients.

[0171] The coding apparatus can determine whether the loop filtering process is carried out through virtual boundaries (S820). The virtual limits can be the same as the aforementioned virtual limits. Furthermore, the loop filtering process may include at least one of an unlocking process, an SAO process, and an ALF process.

[0172] The coding apparatus may generate information related to loop filtering (S830). The coding apparatus may generate loop filtering-related information based on the determination of step S820. Here, information related to loop filtering may refer to information used to carry out the loop filtering process. For example, information related to loop filtering may include information about virtual boundaries described herein (an SPS virtual boundaries presence flag, an image header virtual boundaries presence flag, information about the number of virtual boundaries , information about virtual boundary positions, etc.).

[0173] The encoding apparatus can encode video/image information (S840). Image information may include residual information, prediction-related information, reference image-related information, sub-image related information, loop filtering-related information, and/or virtual boundary-related information (and/or additional virtual boundary-related information ). The encoded video/image information can be output in the form of a bit stream. The bit stream may be transmitted to a decoding apparatus via a network or storage medium.

[0174] The image/video information may include a variety of information in accordance with an embodiment of the present document. For example, the image/video may include information disclosed in at least one of tables 1 to 14 above.

[0175] In one embodiment, the image information may include a set of sequence parameters (SPS) and image header information pertaining to the SPS. For example, whether additional information related to virtual boundaries is included in the SPS or image header information can be determined based on whether reference image resampling is enabled. Here, resampling of the reference image can be performed according to the above-mentioned RPR. Additional information related to virtual boundaries may also be simply referred to as virtual boundary-related information. The term 'additional' is used to be distinguished from information relating to virtual boundaries, such as an SPS virtual boundaries present flag and/or a PH virtual boundaries present flag.

[0176] In one embodiment, additional information related to the virtual boundaries may include the number of virtual boundaries and positions of the virtual boundaries.

[0177] In one embodiment, additional information related to the virtual boundaries may include information about the number of vertical virtual boundaries, information about the positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about the positions of the horizontal boundaries virtual limits.

[0178] In one embodiment, the image information may include a flag enabled for reference image resampling. For example, whether reference image resampling is enabled can be determined based on the reference image resampling enabled flag.

[0179] In one embodiment, the SPS may include an SPS virtual limits present flag related to whether the SPS includes the additional information related to the virtual limits. A value of the SPS Virtual Limits Present flag can be determined as 0, based on the fact that resampling of the reference image is enabled.

[0180] In one embodiment, additional information related to virtual boundaries may not be included in the SPS, based on the fact that resampling of the reference image is enabled. In this case, for example, the image header information may include additional information related to virtual boundaries.

[0181] In one embodiment, the current image may include a subimage as just a subimage. The subimage can be encoded independently. Reconstructed samples can be generated based on the subimage, and information related to the subimage can be generated based on the subimage. Additionally, image information may include information related to the subimage.

[0182] In one embodiment, an image manipulation flag for the subimage (a subpicture_treated_as_picture flag) may not be present in the image information. Therefore, a subpicture_treated_as_picture flag value can be set through inference (estimation or prediction) at the end of decoding. In one example, the value of the subpicture_treated_as_picture flag might be set to 1.

[0183] In one embodiment, the current image may include subimages. In one example, subimages may be derived based on a process of merging two or more independently encoded subimages. Reconstructed samples can be generated based on the subimages. Sub-image related information can be generated based on sub-images. Additionally, image information may include information related to the subimage.

[0184] Figure 10 and Figure 11 schematically show an example of a video/image decoding method and related components in accordance with the embodiment(s) of the present document.

[0185] The method disclosed in Figure 10 can be realized by the decoding apparatus disclosed in Figure 3 or Figure 11. Specifically, for example, S1000 of Figure 10 can be realized by an entropy decoder 310 of the decoding apparatus, S1010 can be performed by a residual processor 320 and/or adder 340 of the decoding apparatus and S1020 may be performed by a filter 350 of the decoding apparatus. The method disclosed in Figure 10 may include the above-mentioned embodiments herein.

[0186] Referring to Figure 10, the decoding apparatus can receive/obtain video/image information (S1000). Video/image information may include residual information, prediction-related information, reference image-related information, sub-image-related information, loop filtering-related information, and/or virtual boundary-related information (and/or additional virtual boundaries- related information). The decoding apparatus can receive/obtain the image/video information via a bit stream.

[0187] The image/video information may include a variety of information in accordance with an embodiment of the present document. For example, the image/video may include information disclosed in at least one of tables 1 to 14 above.

[0188] The decoding apparatus can derive quantized transform coefficients. The decoding apparatus may derive the quantized transform coefficients based on the residual information. Quantized transform coefficients can have a one-dimensional vector form, based on a coefficient sweep order. Quantized transform coefficients may include quantized luma transform coefficients and/or quantized chroma transform coefficients.

[0189] The decoding apparatus can derive the transform coefficients. The decoding apparatus may derive the transform coefficients based on a dequantization process for the quantized transform coefficients. The decoding apparatus may derive luma transform coefficients through dequantization based on the quantized luma transform coefficients. The decoding apparatus may derive chroma transform coefficients through dequantization based on the quantized chroma transform coefficients.

[0190] The decoding apparatus may generate/derive residual samples. The decoding apparatus may derive the residual samples based on the inverse transform process for the transform coefficients. The decoding apparatus may derive residual luma samples through the inverse transform process based on the luma transform coefficients. The decoding apparatus may derive residual chroma samples via inverse transform based on the chroma transform coefficients.

[0191] The decoding apparatus may derive at least one reference image, based on information related to the reference image. The decoding apparatus may perform a prediction process based on at least one reference image. Specifically, the decoding apparatus may derive prediction samples from the current blocks based on a prediction mode. In this case, various prediction methods disclosed herein, such as inter prediction or intra prediction, can be applied. The decoding apparatus may generate prediction samples for the current block in the current image based on the prediction process. For example, the decoding apparatus may perform an inter-prediction process based on at least one reference image, and may generate prediction samples based on the inter-prediction process.

[0192] The decoding apparatus can generate/derive reconstructed samples (S1010). For example, the decoding apparatus may generate/derive reconstructed luma samples and/or reconstructed chroma samples. The decoding apparatus may generate the reconstructed luma samples and/or the reconstructed chroma samples based on the residual information. The decoding apparatus can generate the reconstructed samples based on the residual information. The reconstructed samples may include the reconstructed luma samples and/or the reconstructed chroma samples. A luma component of the reconstructed samples may correspond to the reconstructed luma samples, and a chroma component of the reconstructed samples may correspond to the reconstructed chroma samples. The decoding apparatus may generate prediction luma samples and/or prediction chroma samples via a prediction process. The decoding apparatus can generate the reconstructed luma samples based on the prediction luma samples and the residual luma samples. The decoding apparatus can generate the constructed chroma samples based on the prediction chroma samples and the residual chroma samples.

[0193] The decoding apparatus can generate modified (filtered) reconstructed samples (S1020). The decoding apparatus may generate the modified reconstructed samples based on a loop filtering process for the reconstructed samples. The decoding apparatus may generate the modified reconstructed samples based on information related to loop filtering. The decoding apparatus may use an unlocking process, an SAO process and/or an ALF process to generate the modified reconstructed samples.

[0194] In one embodiment, the image information may include an SPS and image header information regarding the SPS. For example, whether additional information related to virtual boundaries is included in the SPS or image header information can be determined based on whether reference image resampling is enabled. Here, resampling of the reference image can be performed according to the above-mentioned RPR. Additional information related to virtual boundaries may also be simply referred to as virtual boundary-related information. The term 'additional' is used to be distinguished from information relating to virtual boundaries, such as an SPS virtual boundaries present flag and/or a PH virtual boundaries present flag.

[0195] In one embodiment, additional information related to the virtual boundaries may include the number of virtual boundaries and positions of the virtual boundaries.

[0196] In one embodiment, additional information related to the virtual boundaries may include information about the number of vertical virtual boundaries, information about the positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about the positions of the virtual boundaries horizontal.

[0197] In one embodiment, the image information may include a flag enabled for reference image resampling. For example, whether reference image resampling is enabled can be determined based on the reference image resampling enabled flag.

[0198] In one embodiment, the SPS may include an SPS virtual limits present flag related to whether the SPS includes the additional information related to the virtual limits. A value of the SPS Virtual Limits Present flag can be determined as 0, based on the fact that resampling of the reference image is enabled.

[0199] In one embodiment, additional information related to virtual boundaries may not be included in the SPS, based on the fact that resampling of the reference image is enabled. In this case, for example, the image header information may include additional information related to virtual boundaries.

[0200] In one embodiment, the current image may include a subimage as just a subimage. The subimage can be encoded independently. Reconstructed samples can be generated based on the subimage, and information related to the subimage can be generated based on the subimage. Additionally, image information may include information related to the subimage.

[0201] In one embodiment, an image manipulation flag for the subimage (a subpicture_treated_as_picture flag) may not be present in the image information. Therefore, a subpicture_treated_as_picture flag value can be set through inference (estimation or prediction) at the end of decoding. In one example, the value of the subpicture_treated_as_picture flag might be set to 1.

[0202] In one embodiment, the current image may include subimages. In one example, subimages may be derived based on a process of merging two or more independently encoded subimages. Reconstructed samples can be generated based on the subimages. Sub-image related information can be generated based on sub-images. Additionally, image information may include information related to the subimage.

[0203] In the presence of the residual sample for the current block, the decoding apparatus may receive residual information for a current block. The residual information may include a transformation coefficient for residual samples. The decoding apparatus may derive residual samples (or a residual sample matrix) for the current block based on the residual information. Specifically, the decoding apparatus may derive quantized transform coefficients based on the residual information. Quantized transform coefficients can have a one-dimensional vector form, based on a coefficient sweep order. The decoding apparatus may derive the transform coefficients based on a dequantization process for the quantized transform coefficients. The decoding apparatus may derive residual samples based on the transform coefficients.

[0204] The decoding apparatus may generate reconstructed samples, based on prediction (intra) samples and residual samples, and may derive a reconstructed block or reconstructed image, based on the reconstructed samples. Specifically, the decoding apparatus may include reconstructed samples, based on a sum between the prediction (intra) samples and the residual samples. Then, as described above, the decoding apparatus may optionally apply the loop filtering process such as deblocking filtering and/or the SAO process to the reconstructed image to improve the quality of the subjective/objective image.

[0205] For example, the decoding apparatus may obtain image information including all or parts of the above-described information (or syntax elements) by decoding the bit stream or encoded information. Furthermore, the bit stream or encoded information may be stored on a computer-readable storage medium and may cause the decoding method described above to be performed.

[0206] Although the methods have been described based on a flowchart in which the steps or blocks are listed in sequence in the embodiments described above, the steps of the present document are not limited to a certain order and a given step can be performed in a different step or in a different order or simultaneously in relation to that described above. Furthermore, it will be understood by those skilled in the art that the steps of the flowcharts are not exclusive, and another step may be included therein or one or more steps of the flowchart may be excluded without exerting influence on the scope of the present disclosure.

[0207] The above-mentioned method according to the present disclosure may be in the form of software and the encoding apparatus and/or decoding apparatus according to the present disclosure may be included in a device for performing image processing, e.g. , a TV, a computer, a smartphone, a set-top box, a display device or the like.

[0208] When the embodiments of the present disclosure are implemented by software, the aforementioned method can be implemented by a module (process or function) that performs the aforementioned function. The module can be stored in memory and executed by a processor. Memory can be installed inside or outside the processor and can be connected to the processor through several well-known means. The processor may include an Application Specific Integrated Circuit (ASIC), other chipsets, a logic circuit and/or a data processing device. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium and/or other storage device. In other words, embodiments according to the present disclosure can be implemented and executed on a processor, a microprocessor, a controller, or a chip. For example, the functional units illustrated in the respective figures can be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip. In this case, implementation information (e.g. instruction information) or algorithms may be stored on a digital storage medium.

[0209] Furthermore, the decoding apparatus and the encoding apparatus to which the embodiment(s) of the present document are applied may be included in a multimedia transmission transceiver, a mobile communication terminal, a video device home cinema device, a digital cinema video device, a surveillance camera, a video chat device and a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service provider, an Over The Top (OTT) video device, an internet streaming service provider, a 3D video device, a Virtual Reality (VR) device, an Augmented Reality (AR) device, an image phone video device, a vehicle terminal (e.g., a vehicle (including a vehicle), an airplane terminal, or a ship terminal), and a medical video device ; and can be used to process an image or data signal. For example, the OTT video device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, and a digital video recorder (DVR).

[0210] Furthermore, the processing method to which the embodiment(s) of the present document is applied may be produced in the form of a program executed by a computer and may be stored on a computer-readable recording medium. Multimedia data with a data structure according to the embodiment(s) of this document may also be stored on the computer-readable recording medium. Computer-readable recording medium includes all types of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device. Computer-readable recording media also includes media embedded in the form of a carrier wave (e.g., Internet transmission). Furthermore, a bit stream generated by the encoding method may be stored on the computer-readable recording medium or transmitted over a wired or wireless communication network.

[0211] Furthermore, the embodiment(s) of the present document may be embodied as a computer program product based on a program code, and the program code may be executed in a computer in accordance with the embodiment(s) of this document. The program code can be stored on a computer-readable medium.

[0212] Figure 12 represents an example of a content streaming system to which the embodiment of the present document can be applied.

[0213] Referring to Figure 12, the content streaming system to which embodiments of the present document are applied may generally include an encoding server, a streaming server, a web server, a media store, a user device and a multimedia input device.

[0214] The encoding server functions to compress the input content of multimedia input devices, such as the smartphone, camera, camcorder, and the like, into digital data to generate a bit stream and transmit it to the streaming server. As another example, in a case where the multimedia input device such as smart phone, camera, camcorder or the like directly outputs a bit stream, the encoding server may be omitted.

[0215] The bitstream may be generated by a coding method or a bitstream generation method to which embodiments of the present document are applied. And the streaming server can temporarily store the bitstream in a process of transmitting or receiving the bitstream.

[0216] The streaming server transmits multimedia data to the user's equipment based on a user's request through the web server, which functions as an instrument that informs the user which service exists. When the user requests a service that the user wants, the web server transfers the request to the streaming server, and the streaming server transmits multimedia data to the user. In this regard, the content streaming system may include a separate control server, and in this case, the control server functions to control commands/responses between respective equipment in the content streaming system.

[0217] The streaming server may receive content from the media store and/or the encoding server. For example, if the content is received from the encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to provide the streaming service smoothly.

[0218] For example, user equipment may include a cell phone, a smartphone, a portable computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a watch-type terminal (smart watch), a glass-type terminal (smart glass), a head-mounted display (HMD)), a digital TV, a desktop computer, digital signage or similar.

[0219] Each of the servers of the content streaming system can be operated as a distributed server, and in this case, the data received by each server can be processed in a distributed manner.

[0220] The claims in the present description can be combined in various ways. For example, the technical features in the method claims of the present disclosure may be combined to be implemented or carried out in an apparatus, and the technical features in the apparatus claims may be combined to be implemented or carried out in a method. Furthermore, the technical features in the method claim(s) and apparatus claim(s) may be combined to be implemented or performed in an apparatus. Furthermore, the technical features in the method claim(s) and apparatus claim(s) may be combined to be implemented or carried out in a method.

Claims (4)

1. Decoding apparatus for image decoding, the decoding apparatus CHARACTERIZED by the fact that it comprises: a memory; and at least one processor connected to the memory, the at least one processor configured to: obtain image information including residual information through a bit sequence; generate reconstructed samples of a current image, based on residual information; and generating modified reconstructed samples, based on a loop filtering process for the reconstructed samples, wherein the image information includes a sequence parameter set (SPS) and image header information that refers to the SPS, wherein the SPS includes a reference image resampling enabled flag, wherein the reference image resampling enabled flag is related to whether reference image resampling is enabled, wherein whether information related to additional virtual boundaries is included in the SPS or image header information is determined based on the reference image resampling enabled flag; and wherein information relating to additional virtual boundaries includes information about the number of vertical virtual boundaries, information about positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about positions of the horizontal virtual boundaries. 2. Coding apparatus for image coding, the coding apparatus CHARACTERIZED by the fact that it comprises: a memory; and at least one processor connected to the memory, the at least one processor configured to: derive residual samples for a current block; derive residual information, based on the residual samples for the current block; determining whether a loop filtering process is performed across the virtual boundaries; generate information related to loop filtering for reconstructed samples of the current image, based on the determination; and encoding image information including residual information and loop filtering related information, wherein image information includes a set of sequence parameters (SPS) and image header information that refers to the SPS, wherein the SPS includes a reference image resampling enabled flag, wherein the reference image resampling enabled flag is related to whether reference image resampling is enabled, wherein information related to additional virtual boundaries is included in the SPS, or the information image header values are determined based on the reference image resampling enabled flag; and wherein information relating to additional virtual boundaries includes information about the number of vertical virtual boundaries, information about positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about positions of the horizontal virtual boundaries. 3. Computer-readable storage medium CHARACTERIZED by the fact that it stores a bit sequence generated by the encoding apparatus for image encoding as defined in claim 2. 4. Apparatus for transmitting data to an image, the apparatus CHARACTERIZED by the fact that it comprises: at least one processor configured to obtain a bit sequence for the image, wherein the bit sequence is generated based on deriving residual samples for a current block, derive residual information, based on residual samples for the current block, determine whether a loop filtering process is performed across virtual boundaries, generate loop filtering-related information for reconstructed samples of the current image, based on the determining and encoding image information including residual information and information related to loop filtering; and a transmitter configured to transmit the data comprising the bit sequence, wherein the image information includes a sequence parameter set (SPS) and image header information that refers to the SPS, wherein the SPS includes an enable flag for reference image resampling, wherein the reference image resampling enabled flag is related to whether reference image resampling is enabled, wherein whether information related to additional virtual boundaries is included in the SPS or the reference image header information image are determined based on the reference image resampling enabled flag; and wherein information relating to additional virtual boundaries includes information about the number of vertical virtual boundaries, information about positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about positions of the horizontal virtual boundaries.