CN119895864A

CN119895864A - Decoding method, encoding method, decoder, and encoder

Info

Publication number: CN119895864A
Application number: CN202280100092.3A
Authority: CN
Inventors: 杨铀; 刘琼; 蒋小广; 黎振行
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-10-14
Filing date: 2022-10-14
Publication date: 2025-04-25
Also published as: WO2024077611A1

Abstract

The embodiment of the present application provides a decoding method, an encoding method, a decoder and an encoder, wherein the decoding method comprises: decoding a code stream to determine a current sub-image block in a current image; determining whether the current sub-image block is a sub-image block that needs to be illuminated; and when the current sub-image block is a sub-image block that needs to be illuminated, performing an illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation. The decoding method provided by the present application can improve decoding performance.

Description

Decoding method, encoding method, decoder, and encoder

Technical Field

Embodiments of the present application relate to the field of encoding and decoding, and more particularly, to a decoding method, an encoding method, a decoder, and an encoder.

Background

For a moving picture expert group immersive video (Moving Picture Experts Group Immersive Video, MIV) encoding process, a limited number of views among the reference views may be first selected as base views, and the selected base views can be enabled to express the visual range of a scene as much as possible. Wherein an image at a base view (which may also be referred to as a base view) may be transmitted as a complete image; then, redundant pixels between the base view and the image under the non-base view (also referred to as an additional view) can be removed, namely only effective information which is not repeatedly expressed is reserved, and then the effective information is extracted into sub-tiles and the image under the base view to be reorganized, so that a larger rectangular image (also referred to as a mosaic image or a mosaic image) is formed, and the mosaic image can be used for generating video data, so that an encoder encodes the video data to obtain a code stream. For example, a pixel clipping (Prune Pixel) module may be utilized to detect repeated pixels between the base view and the additional view and clip (prune) repeated pixels in the additional view. However, since the detection of the repeated pixel points between the base view and the additional view is based on the detection of the depth difference value and the brightness difference value, the retained pixels may include pixels (such as highlight pixels) which are greatly affected by illumination, and the quality degradation problem, such as the loss of texture information of the sub-tiles, exists after the information of the highlight pixels is subjected to video encoding and decoding, so that the decoding performance is reduced.

Disclosure of Invention

The embodiment of the application provides a decoding method, an encoding method, a decoder and an encoder, which can improve decoding performance.

In a first aspect, an embodiment of the present application provides a decoding method, including:

Decoding the code stream to determine a current sub-block in the current image;

Determining whether to perform illumination transformation on the current sub-image block;

And when the illumination transformation is determined to be carried out on the current sub-block, carrying out the illumination transformation on the current sub-block to obtain an illumination transformed sub-block.

In a second aspect, an embodiment of the present application provides an encoding method, including:

Determining a current sub-tile in the current image;

when the illumination transformation is determined to be carried out on the current sub-block, carrying out the illumination transformation on the current sub-block to obtain an illumination transformed sub-block;

and coding the sub-image block after illumination transformation to obtain a code stream.

In a third aspect, an embodiment of the present application provides a decoder, including:

the decoding unit is used for decoding the code stream and determining a current sub-block in the current image;

A determining unit, configured to determine whether to perform illumination transformation on the current sub-image block;

and the transformation unit is used for carrying out illumination transformation on the current sub-block when determining to carry out illumination transformation on the current sub-block, so as to obtain an illumination transformed sub-block.

In a fourth aspect, an embodiment of the present application provides an encoder, including:

A first determining unit configured to determine a current sub-tile in a current image;

A second determining unit, configured to determine whether to perform illumination transformation on the current sub-image block;

the transformation unit is used for carrying out illumination transformation on the current sub-image block when determining to carry out illumination transformation on the current sub-image block, so as to obtain an illumination transformed sub-image block;

and the coding unit is used for coding the sub-image block after illumination transformation to obtain a code stream.

In a fifth aspect, an embodiment of the present application provides a decoder, including:

a processor adapted to implement computer instructions, and

A computer readable storage medium storing computer instructions adapted to be loaded by a processor and to perform the decoding method of the first aspect or implementations thereof referred to above.

In one implementation, the processor is one or more and the memory is one or more.

In one implementation, the computer-readable storage medium may be integral to the processor or separate from the processor.

In a sixth aspect, an embodiment of the present application provides an encoder, including:

a processor adapted to implement computer instructions, and

A computer readable storage medium storing computer instructions adapted to be loaded by a processor and to perform the encoding method of the second aspect or implementations thereof referred to above.

In a seventh aspect, embodiments of the present application provide a computer-readable storage medium storing computer instructions that, when read and executed by a processor of a computer device, cause the computer device to perform the decoding method as referred to in the first aspect or the encoding method as referred to in the second aspect.

In an eighth aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the decoding method as referred to above in the first aspect or the encoding method as referred to above in the second aspect.

In a ninth aspect, an embodiment of the present application provides a code stream, where the code stream is a code stream as referred to in the method of the first aspect referred to above or a code stream generated by the method of the second aspect referred to above.

Based on the above technical scheme, the illumination transformation is introduced into the current sub-block requiring illumination transformation at the decoder side, so that the encoder is beneficial to strengthening the quality of the current sub-block through the illumination transformation of the current sub-block, the illumination transformation of the current sub-block can restore the illumination effect of the current sub-block, the sub-block decoded by the decoder can be ensured to be a sub-block with enhanced quality after the encoder, and further, the problem that the quality of the information of pixels is degraded after the video encoding and decoding can be relieved, for example, the problem that the texture information of the sub-block is lost can be relieved, and the decoding performance is improved.

Drawings

Fig. 1 is an example of a codec system of an MIV provided by an embodiment of the present application.

Fig. 2 is an example of a preprocessing procedure provided by an embodiment of the present application.

Fig. 3 is an example of a depth map provided by an embodiment of the present application.

Fig. 4 is an example of a texture map provided by an embodiment of the present application.

Fig. 5 is an example of an occupancy map provided by an embodiment of the present application.

Fig. 6 is an example of a generation process of a splice map provided by an embodiment of the present application.

Fig. 7 is a schematic flow chart of a decoding method provided by an embodiment of the present application.

Fig. 8 is a schematic flow chart of an encoding method provided by an embodiment of the present application.

Fig. 9 is an example of a mapping relationship between a pixel in a base view and a pixel block in which a pixel in a current sub-tile is located, provided by an embodiment of the present application.

Fig. 10 is an example of a judgment flow of similar pixel pairs provided by the embodiment of the present application.

Fig. 11 is an example of a determination procedure for combining illumination conversion into similar pixel pairs according to an embodiment of the present application.

Fig. 12 is a schematic block diagram of a decoder provided by an embodiment of the present application.

Fig. 13 is a schematic block diagram of an encoder provided by an embodiment of the present application.

Fig. 14 is an example of an electronic device provided by an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 1 is an example of an MIV codec system 10 provided by an embodiment of the present application.

As shown in fig. 1, the codec system 10 of the MIV may be roughly divided into a data acquisition module 11, a preprocessing module 12, a video encoding frame 13, a video decoding frame 14, and a post-processing module 15 according to task lines.

The data acquisition module 11 is used for acquiring data. The data collected by the data collection module 11 may include not only texture information, but also depth information corresponding to the texture information one by one, where the texture information may be a three-channel color image, and the depth information may be a depth map. Alternatively, the pixel values of the depth map may be used to reflect the distance measure of the corresponding point to the acquisition device, i.e. the pixel values of the depth map may be used to reflect the corresponding point geometry information.

The preprocessing module 12 is used for organizing and expressing the data acquired by the data acquisition module 11 to obtain a video to be encoded. The input source format of the preprocessing module 12 is multi-viewpoint texture deepening video, and the video format can be perspective projection plane video or panoramic video. Specifically, after the data acquisition module 11 acquires the corresponding video sequences through the cameras at different positions, the preprocessing module 12 may perform illumination equalization, color correction, and the like on the images in the video sequences, and may perform camera calibration and image calibration. If the video format acquired by the data acquisition module 11 is panoramic video, the preprocessing module 12 may further splice images in the panoramic video by using an image stitching technology, and map the spliced splicing image into a two-dimensional planar video.

As shown in fig. 2, the reference views are views used by the data acquisition module 11 to acquire data, and the reference views may be associated by view parameters, texture data, and geometric data, and the preprocessing module 12 may select a limited number of views among the reference views as base views, and may enable the selected base views to express the visual range of the scene as much as possible.

The image under the base view (may also be referred to as a base view) may be transmitted as a complete image, and further, redundant pixels between the remaining non-base view (may also be referred to as an additional view) and the base view may be removed, that is, only valid information that is not repeatedly expressed is retained, and then the valid information is extracted as a sub-tile and the image under the base view is reorganized, so as to form a larger rectangular image (may also be referred to as a mosaic image or a mosaic image).

Further, the stitched image may be used to generate video data, which may include texture video data and geometry video data. The video data may be used as an input image of the video encoding frame 13, and the output of the video encoding frame 13 is a code stream obtained by encoding the input image. The output code streams of the video coding framework 13 may include a geometric code stream, which is a code stream generated by encoding a geometric depth map (e.g., the depth map shown in fig. 3) to represent geometric information, and an attribute code stream, which is a code stream generated by encoding a texture map (e.g., the texture map shown in fig. 4) to represent attribute information. Alternatively, the output code stream of the video encoding frame 13 may also include an occupied code stream, which is a code stream generated by encoding an occupied map (such as the occupied map shown in fig. 5) and used to indicate the effective areas in the depth map and the texture map, and these three types of images are encoded and decoded by using the video encoder. The output stream of the video encoding framework 13 may be used as the input stream of the video decoding framework 14.

Of course, for stitched images, there may also be auxiliary data related to sub-picture stitching.

As shown in fig. 2, the auxiliary data related to the splicing information of the sub-tiles may also be written into the code stream, so that the transmission pixel rate can be reduced while the scene information is kept as much as possible, and further, sufficient information is ensured for the post-processing module 15 to render the reconstructed image to obtain the final view.

As shown in fig. 6, the reference view may include view 0, view 1, and view 2, the reference view image acquired at view 0 may be used to extract sub-tile 2 and sub-tile 4, the reference view image acquired at view 1 may be used to extract sub-tile 8, the reference view image acquired at view 2 may be used to extract sub-tile 3 and sub-tile 7, further, a mosaic image 0 may be obtained based on sub-tile 2, sub-tile 4, and sub-tile 8, a mosaic image 1 may be obtained based on sub-tile 3 and sub-tile 7, and then the mosaic image 0 and the mosaic image 1 may be stitched to the image under the base view to obtain a final mosaic image, and the final mosaic image may be used as an input image of the video encoding frame 13 to encode the input image to obtain a code stream.

The video encoding frame 13 is used for encoding and compressing video, and the video decoding frame 14 is used for decoding and reconstructing video.

The specific implementation of the video encoding framework 13 and the video decoding framework 14 is not limited in the present application. For example, a codec framework may be employed that applies three-dimensional video coding techniques such as MV-HEVC and 3D-HEVC to multi-view video, where the codec efficiency of the codec framework is higher than HEVC. Of course, the codec framework of the conventional planar video hybrid coding techniques such as HEVC, VVC, etc. may be used.

The post-processing module 15 is configured to synthetically render the image decoded by the video decoding framework 14 to obtain a final view. For example, a target view rendered at the user viewpoint position orientation may be synthesized from the decoded reconstructed image and pose information of the user at the current time.

Fig. 7 is a schematic flow chart of a decoding method 200 provided by an embodiment of the present application. It should be appreciated that the decoding method 200 may be performed by a decoder. For example, to the decoding framework 14 shown in fig. 1. For convenience of description, a decoder is described as an example.

As shown in fig. 7, the decoding method 200 may include some or all of the following:

S210, decoding the code stream to determine a current sub-block in the current image;

s220, determining whether illumination transformation is carried out on the current sub-image block;

And S230, when the illumination transformation is determined to be carried out on the current sub-block, carrying out the illumination transformation on the current sub-block to obtain an illumination transformed sub-block.

Illustratively, the current image may be an additional view.

The current sub-tile may be, for example, an image block processed by the encoder from the additional view. For example, the encoder may form a larger rectangular image (which may also be referred to as a stitched image or a stitched image) by removing redundant pixels between the base view and the additional view, i.e., retaining only non-repeatedly expressed active pixels (which may also be referred to as active pixel clusters (clusters)), forming regular-shaped image blocks (i.e., sub-tiles) by dividing and packing (pack) the active pixel clusters, and then reorganizing the base view with one or more sub-tiles derived based on the additional view (e.g., stitching the base view with sub-tiles derived based on the additional view), which may be used to generate video data such that the encoder encodes the video data to obtain a bitstream.

Accordingly, the decoder may restore or decode additional views based on the current sub-tile.

Specifically, the decoder may determine a cluster of valid pixels (cluster) that the encoder retains after clipping redundant pixels in the additional view based on the current sub-tile, and then restore the additional view based on the resulting cluster of valid pixels. For example, the decoder may determine a position of the current sub-tile in the additional view based on a clipping mask of the additional view, and restore the redundant pixels clipped in the additional view with pixels in the base view based on the decoded positions of the valid pixel clusters, thereby restoring the additional view based on pixels of the current sub-tile and clipped points in the additional view.

The current sub-tile may be a rectangular image block, for example.

In this embodiment, by introducing illumination transformation to a current sub-block that needs to perform illumination transformation at the decoder side, the encoder is facilitated to strengthen the quality of the current sub-block through the illumination transformation of the current sub-block, for the decoder, the illumination transformation of the current sub-block not only can restore the illumination effect of the current sub-block, but also can ensure that the sub-block decoded by the decoder is a sub-block with enhanced quality after the encoder, and further, the problem that the quality of the information of the pixels of the sub-block is degraded after the information is encoded and decoded by the video can be alleviated, for example, the problem that the texture information of the sub-block is lost can be alleviated, and the decoding performance is improved.

In some embodiments, the S220 may include:

decoding the code stream, and determining a sub-image block identifier of the current sub-image block, wherein the sub-image block identifier of the current sub-image block indicates whether illumination transformation is carried out on the current sub-image block or not;

based on the sub-image block identification of the current sub-image block, determining whether to perform illumination transformation on the current sub-image block.

For example, if the sub-block identification of the current sub-block indicates that the current sub-block is subjected to illumination transformation, the decoder determines to perform illumination transformation on the current sub-block, and if the sub-block identification of the current sub-block indicates that the current sub-block is not subjected to illumination transformation, the decoder determines to not perform illumination transformation on the current sub-block.

Of course, in other alternative embodiments, it may be determined in other ways whether to perform the illumination transform for the current sub-tile.

In one possible implementation, the decoder may determine whether to perform illumination transform on the current sub-block based on whether the code stream carries an identification of the current sub-block. For example, if the code stream carries the identifier of the current sub-block, the decoder determines to perform illumination transformation on the current sub-block, and if the code stream does not carry the identifier of the current sub-block, the decoder determines not to perform illumination transformation on the current sub-block.

In another possible implementation, the decoder may also perform a pixel-level illumination transform or a view-level illumination transform. For example, the decoder may also determine whether to perform the illumination transform on the current pixel based on an identification corresponding to the current pixel in the current sub-tile, the identification corresponding to the current pixel indicating whether to perform the illumination transform on the current pixel. For another example, the decoder may further determine whether to perform illumination transformation on the sub-image block in the additional view based on an identifier corresponding to an additional view to which the current sub-image block belongs, where the identifier corresponding to the additional view indicates whether to perform illumination transformation with the sub-image block in the additional view.

In some embodiments, the decoder may also decode the code stream, determine an enable flag for a sub-block flag of the current sub-block, and indicate whether the code stream carries the sub-block flag of the current sub-block.

Illustratively, the enable identity may be carried within an Atlas sequence parameter set MIV extension syntax (Atlas sequence PARAMETER SET MIV extension syntax) in the code stream.

Illustratively, the Atlas sequence parameter set MIV extension syntax may be as shown in table 1:

TABLE 1

As shown in Table 1, asme _ic_enabled_flag is an enable flag.

Wherein asme _ic_enabled_flag is 1 means that the syntax element pdu_ic_flag [ tileID ] [ p ] exists in the syntax structure pdu_ miv _extension (). asme _ic_enabled_flag is 0 means that the syntax element pdu_ic_flag [ tileID ] [ p ] does not exist in the syntax structure pdu_ miv _extension (). When asme _ic_enabled_flag does not exist, its value defaults to 0.(asme_ic_enabled_flag equal to 1 specifies that the pdu_ic_flag[tileID][p]syntax elements are present in the pdu_miv_extension()syntax structure.asme_ic_enabled_flag equal to 0 specifies that the pdu_ic_flag[tileID][p]syntax elements are not present in the pdu_miv_extension()syntax structure.When not present,the value of asme_ic_enabled_flag is inferred to be equal to 0).

Illustratively, the sub-block identification of the current sub-block may be carried within a sub-block data unit MIV expansion syntax (PATCH DATA unit MIV extension syntax) in the bitstream.

Illustratively, the sub-picture block data unit MIV extension syntax may be as shown in table 2:

TABLE 2

As shown in Table 2, pdu_ic_flag [ tileID ] [ p ] is a sub-picture block identification.

Wherein a pdu_ic_flag [ tileID ] [ p ] of 1 means that the sub-picture block performs illumination transform. A pdu_ic_flag [ tileID ] [ p ] of 0 means that the sub-block is not illumination transformed. When pdu_ic_flag [ tileID ] [ p ] is not present, its value defaults to 0.(pdu_ic_flag[tileID][p]equal to 1 specifies that the patch is an IC patch.When pdu_ic_flag[tileID][p]is equal to 0,then the patch is not an IC patch.When not present,the value of pdu_ic_flag[tileID][p]is inferred to be equal to 0.)

In some embodiments, the S230 may include:

Determining a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-tile;

and carrying out illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters, and obtaining the sub-image block after illumination transformation.

Illustratively, when determining to perform illumination transformation on the current sub-block, the decoder determines the target transformation mode and the target transformation parameters, and then performs illumination transformation on the current sub-block by using the target transformation mode based on the target transformation parameters to obtain the sub-block after illumination transformation.

In some embodiments, the target transformation is a default transformation.

Illustratively, the default transformation may be a logarithmic transformation, a gamma transformation, or other transformation, or even the default transformation may be a transformation based on machine learning, which is not particularly limited in the present application.

Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transformation mode, which is not particularly limited by the present application.

In some embodiments, the target transformation parameters are transformation parameters that the target transformation style defaults to.

Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transform parameter, which is not particularly limited by the present application.

In some embodiments, the contrast of the attribute information of the pixels in the current sub-block is adjusted based on the target transformation mode and the target transformation parameter to obtain the sub-block after illumination transformation, wherein the contrast between the attribute information of the pixels in the sub-block after illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-block.

In other words, the target transform parameters used by the current sub-tile can increase the contrast between the attribute information of the pixels in the current sub-tile.

For example, the contrast of the attribute information of the pixels in the current sub-tile may be a contrast between texture information of the pixels in the current sub-tile. In other words, the target transform parameters used by the current sub-tile can increase the contrast between the texture information of the pixels in the current sub-tile.

For example, the contrast of the attribute information of the pixels in the current sub-tile may be a contrast between the color information of the pixels in the current sub-tile. In other words, the target transform parameters used by the current sub-tile can increase the contrast between the color information of the pixels in the current sub-tile.

It is noted that, in the case where the target transform manner and the target transform manner are fixed, the difference between the contrast between the attribute information of the pixels in the sub-tile after the illumination transform and the contrast between the attribute information of the pixels in the current sub-tile is also fixed, and therefore, the decoder may determine the target transform manner and the target transform parameter by decoding the code stream, and then determine the target transform parameter based on the difference between the contrast between the attribute information of the pixels in the sub-tile after the illumination transform and the contrast between the attribute information of the pixels in the current sub-tile, and the contrast of the sub-tile after the illumination transform, which is not particularly limited by the present application.

In some embodiments, the method 200 may further comprise:

And determining a decoded image of the current image based on the illumination transformed sub-image block.

Illustratively, the decoder determines, based on the illumination transformed sub-picture block, a decoded image of an additional image to which the current sub-picture block belongs.

The decoding method according to the embodiment of the present application is described in detail above from the viewpoint of a decoder, and the encoding method according to the embodiment of the present application will be described below from the viewpoint of an encoder.

Fig. 8 is a schematic flow chart of an encoding method 310 provided by an embodiment of the present application. It should be appreciated that the encoding method 310 may be performed by an encoder. For example, to the coding framework 13 shown in fig. 1. For convenience of description, an encoder is described as an example.

As shown in fig. 8, the decoding method 310 may include some or all of the following:

S311, determining a current sub-block in the current image;

s312, determining whether illumination transformation is carried out on the current sub-image block;

S313, when the illumination transformation is determined to be carried out on the current sub-block, carrying out the illumination transformation on the current sub-block to obtain a sub-block after the illumination transformation;

s314, coding the sub-image block after illumination transformation to obtain a code stream.

Illustratively, the current image may be an additional view.

The current sub-tile may be a rectangular image block, for example.

In this embodiment, by introducing illumination transformation to a current sub-block that needs to perform illumination transformation at the encoder side, the encoder is facilitated to strengthen the quality of the current sub-block through the illumination transformation of the current sub-block, for the decoder, the illumination transformation of the current sub-block not only can restore the illumination effect of the current sub-block, but also can ensure that the sub-block decoded by the decoder is a sub-block with enhanced quality after the encoder, and further, the problem that the quality of the information of the pixels of the sub-block is degraded after the information is encoded and decoded by the video can be alleviated, for example, the problem that the texture information of the sub-block is lost can be alleviated, and the decoding performance is improved.

In some embodiments, the S314 may include:

The sub-block identification of the current sub-block indicates whether illumination transformation is carried out on the current sub-block or not;

And coding the sub-image block identification of the current sub-image block and the sub-image block after illumination transformation to obtain the code stream.

In this embodiment, the sub-block identifier of the current sub-block is written into the code stream, which is favorable for the decoder to decode the code stream, and determines whether to perform illumination transformation along with the current sub-block, so that it can be ensured that the decoder and the encoder keep consistent understanding of whether to perform illumination transformation on the current sub-block, and the encoding and decoding efficiency is improved.

Of course, in other alternative embodiments, the decoder may be indicated by other means as well whether to perform the illumination transform on the current sub-tile.

In one possible implementation, the encoder encodes the identity of the sub-tile requiring illumination conversion so that the decoder determines whether to do illumination conversion based on the decoded sub-tile identity. For example, the decoder may determine whether to perform illumination transform on the current sub-block based on whether the code stream carries an identification of the current sub-block. For example, if the code stream carries the identifier of the current sub-block, the decoder determines to perform illumination transformation on the current sub-block, and if the code stream does not carry the identifier of the current sub-block, the decoder determines not to perform illumination transformation on the current sub-block.

In another possible implementation, the encoder may also encode a pixel-level illumination transform identification or a view-level illumination transform identification. For example, the decoder may determine whether to perform the illumination transform on the current pixel based on an identification corresponding to the current pixel in the current sub-tile, the identification corresponding to the current pixel indicating whether to perform the illumination transform on the current pixel. For another example, the decoder may further determine whether to perform illumination transformation on the sub-image block in the additional view based on an identifier corresponding to an additional view to which the current sub-image block belongs, where the identifier corresponding to the additional view indicates whether to perform illumination transformation with the sub-image block in the additional view.

In some embodiments, the S312 may include:

Acquiring a basic view;

Detecting illumination of a pixel pair formed by the pixel in the base view and the pixel in the current sub-block, and determining whether the pixel pair is a pixel pair with inconsistent illumination;

And determining whether to perform illumination transformation on the current sub-image block based on the number of the pixel pairs with inconsistent illumination.

For example, the encoder may first map any pixel of a base view to an additional view to which the current sub-block belongs based on parameters of a camera for acquiring the base view, parameters of a camera for acquiring an additional view to which the current sub-block belongs, depth information of the base view, and depth information of the additional view to which the current sub-block belongs, to obtain one or more pixel pairs formed by pixels in the base view and pixels in the current sub-block.

Illustratively, the pair of pixels is from pixels of the current sub-tile that are at a projection or mapping position when the pixels from the base view are projected or mapped to the current sub-tile.

As shown in fig. 9, assuming that the view v0 is a base view and the view v1 is an additional view to which the current sub-tile belongs, any pixel of the view v0 may be mapped into the view v1 to obtain one or more pixel pairs formed by a pixel in the view v0 and a pixel in the current sub-tile, for example, a pixel pair formed by a pixel 1 in the view v0 and a pixel 2 in the current sub-tile is exemplarily shown in the figure, based on a parameter of a camera for acquiring the view v0, a parameter of a camera for acquiring the view v1, depth information of the view v0, and depth information of the view v 1.

In some embodiments, a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-tile is determined, and based on the distribution difference value, whether the pixel pair is a non-uniform illumination pixel pair is determined.

For example, the distribution difference value may be used to characterize a difference between an illumination effect of color information of pixels in the base view and an illumination effect of color information of pixels in the current sub-tile, e.g., if the distribution difference value is smaller, the closer the illumination effect of color information of pixels in the base view and the illumination effect of color information of pixels in the current sub-tile are, and thus the pixel pair may be described as a pixel pair that is not non-uniform in illumination. Otherwise, if the distribution difference value is larger, the difference between the illumination effect of the color information of the pixels in the base view and the illumination effect of the color information of the pixels in the current sub-block is larger, so that the pixel pair can be illustrated as a pixel pair with inconsistent illumination.

In some embodiments, the maximum difference value between (1-R ₁)/(1-R ₂)、(1-G ₁)/(1-G ₂) and (1-B ₁)/(1-B ₂) is determined as the distribution difference value, wherein R ₁、G ₁、B ₁ represents the B component, G component, and B component values of the pixels in the current sub-tile, respectively, and R ₂、G ₂、B ₂ represents the B component, G component, and B component values of the pixels in the base view, respectively.

Illustratively, the encoder may first determine the difference between (1-R ₁)/(1-R ₂) and (1-G ₁)/(1-G ₂), the difference between (1-R ₁)/(1-R ₂) and (1-B ₁)/(1-B ₂), and the difference between (1-B ₁)/(1-B ₂) and (1-G ₁)/(1-G ₂), and determine the largest of the differences between (1-R ₁)/(1-R ₂) and (1-G ₁)/(1-G ₂), the difference between (1-R ₁)/(1-R ₂) and (1-B ₁)/(1-B ₂), and the difference between (1-B ₁)/(1-B ₂) and (1-G ₁)/(1-G ₂) as the distribution difference value.

The illumination effect of the pixels in the current sub-block is closer to the illumination effect of high light, and the values of the B component, the G component and the B component of the pixels in the current sub-block are closer to 1, so that the embodiment of the application is closer to each other between (1-R ₁)/(1-R ₂)、(1-G ₁)/(1-G ₂) and (1-B ₁)/(1-B ₂), which means that the illumination effect between the color information of the pixels in the current sub-block is closer to the highlight effect, the detection of the highlight sub-block can be realized, and the encoder can perform illumination transformation on the highlight sub-block, thereby alleviating the problem of quality degradation of the information of the pixels after video encoding and decoding, such as the problem of texture information loss of the sub-block, and improving the decoding performance.

Of course, in other alternative embodiments, the encoder may also determine the distribution difference value from the difference between two of (1-R ₁)/(1-R ₂)、(1-G ₁)/(1-G ₂) and (1-B ₁)/(1-B ₂). The present application is not particularly limited thereto.

For example, the encoder may determine the distribution difference value by comparing (1-R ₁)/(1-R ₂) with (1-G ₁)/(1-G ₂). For another example, the encoder may determine the distribution difference value by comparing (1-R ₁)/(1-R ₂) with (1-B ₁)/(1-B ₂). For another example, the encoder may determine the distribution difference value by comparing (1-B ₁)/(1-B ₂) with (1-G ₁)/(1-G ₂).

In some embodiments, the pixel pair is determined to be a non-uniform illumination pixel pair if the distribution difference value is less than or equal to a first threshold, and the pixel pair is determined not to be a non-uniform illumination pixel pair if the distribution difference value is greater than the first threshold.

The first threshold value may be a preset threshold value, for example a standard predefined threshold value.

Of course, in other alternative embodiments, the encoder may also determine whether the pixel pair is a non-uniform illumination pixel pair based on the range to which the distribution difference value belongs. For example, if the range to which the distribution difference value belongs is a first range, the pixel pair is determined to be a pixel pair with inconsistent illumination, and if the range to which the distribution difference value belongs is a second range, the pixel pair is determined to be a pixel pair with inconsistent illumination. Optionally, the maximum value of the first range is less than or equal to the minimum value of the second range.

In some embodiments, a determination is made as to whether to perform an illumination transform on the current sub-block based on the number of illumination-inconsistent pixel pairs and the duty cycle of the illumination-inconsistent pixel pairs.

For example, the encoder may determine the ratio of the number of non-uniform illumination pixel pairs to the number of pixels in the current sub-tile as the duty cycle of the non-uniform illumination pixel pairs.

In some embodiments, if the duty ratio of the non-uniform illumination pixel pair is greater than or equal to a second threshold, the current sub-tile is determined to be a sub-tile requiring illumination conversion, and if the duty ratio of the non-uniform illumination pixel pair is less than the second threshold, the current sub-tile is determined to be a sub-tile requiring no illumination conversion.

The second threshold value may be a predefined threshold value, for example a standard predefined threshold value.

In some embodiments, determining whether the pixel pair is a similar pixel pair, and if the pixel pair is not a similar pixel pair, performing illumination detection on the pixel pair, determining whether the pixel pair is a non-uniform illumination pixel pair.

For example, the encoder may trigger illumination detection of the pixel pair if it is determined that the pixel pair is not a similar pixel pair, thereby determining whether the pixel pair is an inconsistent illumination pixel pair.

In some embodiments, the S312 may include:

determining a depth difference value between the pixel pairs;

Determining a luminance difference value between a pixel in the base view and a pixel block in which a pixel in the current sub-tile is located;

Determining that the pixel pair is a similar pixel pair when the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold;

And determining that the pixel pair is not a similar pixel pair when the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold.

For example, the encoder may determine a difference value between the luminance of the pixel in the base view and the luminance of each pixel in the pixel block, and determine a maximum value of the determined difference values as the maximum difference value. For example, assuming that the pixel block is a3×3 pixel block centered on a pixel in the current sub-block, the encoder may determine a difference value (i.e., 9 difference values) between the luminance of the pixel in the base view and the luminance of each pixel in the 3×3 pixel block and determine a maximum value of the determined 9 difference values as the maximum difference value.

Notably, the encoder may determine the depth difference value in a pixel-to-pixel (pixel-to-pixel) manner and the luminance difference value in a pixel-to-pixel block (pixel-to-block) manner. The application is not particularly limited in this regard and, for example, in other alternative embodiments, the encoder may determine the luminance difference value in a pixel-to-pixel (pixel-to-pixel) manner.

Of course, in other alternative embodiments, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to the third threshold value, to determine whether the pixel pair is a non-uniform illumination pixel pair. That is, the encoder may trigger illumination detection on the pixel pair if it is determined that the pixel pair is not a similarly bright pixel pair, thereby determining whether the pixel pair is an inconsistent illumination pixel pair.

Fig. 10 is an example of a judgment flow 320 of similar pixel pairs provided in an embodiment of the present application.

As shown in fig. 10, the determining process 320 may include:

S321, the encoder determines whether the depth difference value is smaller than t1.

Illustratively, the encoder may determine a depth difference value for a pixel pair formed for a pixel in the base view and a pixel in the additional image, and determine whether the depth difference value is less than t1. For example, if the depth difference value is less than t1, the encoder determines that the pixel pair is a depth-similar pixel pair, otherwise the encoder determines that the pixel pair is not a depth-similar pixel pair. Wherein the pixels in the base view may be pixels 1 as shown in fig. 9 and the pixels in the additional image may be pixels 2 as shown in fig. 9.

S322, the encoder determines whether the brightness difference value is less than t2.

For example, the encoder may determine a difference value between the luminance of the pixel in the base view and the luminance of each pixel in the pixel block in which the pixel in the additional image is located, and determine a maximum value of the determined difference values as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on a pixel in the additional image, the encoder may determine a difference value (i.e., 9 difference values) between the luminance of the pixel in the base view and the luminance of each pixel in the 3×3 pixel block and determine the maximum value of the determined 9 difference values as the luminance difference value, and determine whether the luminance difference value is less than t2. For example, if the luminance difference value is less than t2, the encoder determines that the pixel pair is a luminance-similar pixel pair, otherwise, the encoder determines that the pixel pair is not a luminance-similar pixel pair.

S323, executing a second stage.

Illustratively, S321 and S322 may be understood as a first stage of performing similarity detection on the pixel pair, after the first stage is performed, a second stage may be performed on the suspected pixel pair output by the first stage, and after the second stage is performed, a final similar pixel pair may be determined. Optionally, the encoder may determine, when performing the second stage of detection on the suspected similar pixel pair, whether the suspected similar pixel pair is a similar pixel based on a color difference value between the suspected similar pixel pair. It is noted that the step of performing the second stage may be an optional step.

In this embodiment, after the encoder detects the pixels of the pixels in the additional view and the base view by using the pixel clipping (Prune Pixel) module, the pixels of the pixels may be clipped (prune) in a subsequent process, that is, only the valid pixels that are not repeatedly expressed (may also be referred to as valid pixel clusters (clusters)) are retained, and then the valid pixel clusters are divided and packed (pack) to form the regular-shaped image block (i.e., the sub-block), and then the base view and one or more sub-blocks obtained based on the additional view are reorganized (for example, the base view and the sub-block obtained based on the additional view are spliced) to form a larger rectangular image (may also be referred to as a spliced image or a spliced image), and the spliced image may be used to generate video data, so that the encoder encodes the video data to obtain a code stream.

FIG. 11 is an example of a decision flow 330 for combining illumination transformations into similar pixel pairs provided by an embodiment of the present application.

As shown in fig. 11, the determining process 330 may include:

s331, the encoder determines whether the depth difference value is smaller than t1.

S332, the encoder determines whether the luminance difference value is less than t2.

S333, executing the second stage.

Illustratively, S321 and S322 may be understood as a first stage of performing similarity detection on the pixel pair, after the first stage is performed, a second stage may be performed on the suspected pixel pair output by the first stage, and after the second stage is performed, a final similar pixel pair may be determined. Optionally, when the encoder performs the second stage of detection on the suspected similar pixel pair, the encoder may determine whether the suspected similar pixel pair is a similar pixel based on a color difference value between the suspected similar pixel pair. It is noted that the step of performing the second stage may be an optional step.

S334, the encoder determines whether the distribution difference value is smaller than t3.

For example, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to t1, and determine whether the pixel pair is a pixel pair with inconsistent illumination. That is, the encoder may trigger illumination detection on the pixel pair if it is determined that the pixel pair is not a similarly bright pixel pair, thereby determining whether the pixel pair is an inconsistent illumination pixel pair. For example, if the distribution difference value of the pixel pair is smaller than t3, determining that the pixel pair is an illumination inconsistent pixel pair, otherwise, determining that the pixel pair is not an illumination inconsistent pixel pair.

S335, the encoder determines whether to perform illumination transformation on the current sub-tile based on the detection result of the illumination inconsistent pixel pairs.

S336, the current sub-block of the encoder performs the care transform.

It should be understood that S335 and S336 are described above with reference to the method 310, and are not repeated here.

In this embodiment, after detecting the pixels of the pixels in the additional view and the pixels in the base view by using the pixel clipping (Prune Pixel) module, the encoder may clip (prune) the pixels in the additional view, that is, only hold the valid pixels (which may also be referred to as valid pixel clusters (clusters)) that are not repeatedly expressed, divide and pack the valid pixel clusters (pack) to form a regular-shaped image block (i.e., a sub-tile), then perform the illumination transform on the sub-tile that needs to be subjected to the illumination transform in the obtained one or more sub-tiles, and obtain an illumination transformed sub-tile, and then reorganize the base view and the sub-tile that needs to be subjected to the illumination transform obtained based on the additional view, and perform the illumination transform on the sub-tile that needs to be subjected to the illumination transform (for example, splice the base view and the sub-tile obtained based on the additional view) to form a larger rectangular image (which may also be referred to as a splice image or a splice image), and the splice image may be used to generate video data, so as to encode the video data stream and obtain the encoded video data.

In some embodiments, determining a target transformation manner used by the current sub-tile, determining target transformation parameters used by the current sub-tile, and performing illumination transformation on the current sub-tile based on the target transformation manner and the target transformation parameters to obtain an illumination transformed sub-tile.

In an exemplary embodiment, when determining to perform illumination transformation on the current sub-tile, the encoder determines the target transformation mode and the target transformation parameters, and then performs illumination transformation on the current sub-tile by using the target transformation mode based on the target transformation parameters to obtain the sub-tile after illumination transformation.

In some embodiments, the target transformation is a default transformation.

Of course, in other alternative embodiments, the encoder may also write the target transform into the code stream, which is not particularly limited by the present application.

Of course, in other alternative embodiments, the encoder may also write the target transform parameters to the code stream, which is not particularly limited by the present application.

In some embodiments, the contrast of the attribute information of the pixels in the current sub-block is adjusted based on the target transformation mode and the target transformation parameter to obtain the sub-block after illumination transformation, wherein the contrast between the attribute information of the pixels in the sub-block after illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-block.

In other words, the target transform parameters used by the current sub-tile can reduce the contrast between the attribute information of the pixels in the current sub-tile.

For example, the contrast of the attribute information of the pixels in the current sub-tile may be a contrast between texture information of the pixels in the current sub-tile. In other words, the target transform parameters used by the current sub-tile can reduce the contrast between the texture information of the pixels in the current sub-tile.

For example, the contrast of the attribute information of the pixels in the current sub-tile may be a contrast between the color information of the pixels in the current sub-tile. In other words, the target transform parameters used by the current sub-tile can reduce the contrast between the color information of the pixels in the current sub-tile.

Notably, since the encoder emphasizes the contrast between the attribute information of the pixels in the current sub-tile, the decoder needs to weaken the contrast between the attribute information of the pixels in the current sub-tile when recovering the attribute information of the pixels in the current sub-tile. Of course, in other alternative embodiments, the encoder performs illumination transform on the current sub-tile, and the decoder performs illumination transform on the current sub-tile as well as illumination inverse transform on the current sub-tile by the encoder. The present application is not particularly limited thereto.

It is noted that, in the case where the target transform manner and the target transform manner are fixed, the difference between the contrast between the attribute information of the pixels in the sub-tile after the illumination transform and the contrast between the attribute information of the pixels in the current sub-tile is also fixed, and therefore, the encoder may determine the target transform manner and the target transform parameter by decoding the code stream, and then determine the target transform parameter based on the difference between the contrast between the attribute information of the pixels in the sub-tile after the illumination transform and the contrast between the attribute information of the pixels in the current sub-tile, and the contrast of the sub-tile after the illumination transform, which is not particularly limited by the present application.

In some embodiments, the sub-blocks after illumination transformation are spliced with the base view to obtain a spliced image, and the pixels in the spliced image and the pixels in the sub-blocks after illumination transformation are encoded to obtain the code stream.

The encoder may illustratively remove redundant pixels between the base view and the additional view, that is, only valid pixels that are not repeatedly expressed (which may also be referred to as a valid pixel cluster (cluster)), divide and pack (pack) the valid pixel cluster to form an image block (that is, a sub-tile) with a regular shape, then perform illumination transformation on a sub-tile that needs to be subjected to illumination transformation in the obtained one or more sub-tiles, and obtain an illumination transformed sub-tile, and then reorganize the base view and the sub-tile that needs not to be subjected to illumination transformation and the sub-tile that needs to be subjected to illumination transformation after performing illumination transformation on the sub-tile that needs to be subjected to illumination transformation (for example, splice the base view and the sub-tile obtained based on the additional view) to form a larger rectangular image (which may also be referred to as a splice image or a splice image), and the splice image may be used to generate video data, so that the encoder encodes the video data and obtains a code stream.

The preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings, but the present application is not limited to the specific details of the embodiments, and various simple modifications can be made to the technical solution of the present application within the scope of the technical concept of the present application, and all the simple modifications belong to the protection scope of the present application. For example, the individual features described in the above-mentioned embodiments can be combined in any suitable manner, without contradiction, and the application will not be described in any way in any possible combination in order to avoid unnecessary repetition. As another example, any combination of the various embodiments of the present application may be made without departing from the spirit of the present application, which should also be regarded as the disclosure of the present application. It should also be understood that, in the various method embodiments of the present application, the sequence numbers of the processes referred to above do not mean the sequence of execution, and the execution sequence of the processes should be determined by the functions and internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The method embodiments of the present application are described above in detail, and the apparatus embodiments of the present application are described below in detail with reference to fig. 12 to 14.

Fig. 12 is a schematic block diagram of a decoder 400 of an embodiment of the present application.

As shown in fig. 12, the decoder 400 may include:

a decoding unit 410, configured to decode the code stream, and determine a current sub-tile in the current image;

A determining unit 420, configured to determine whether to perform illumination transformation on the current sub-picture block;

and the transforming unit 430 is configured to, when determining to perform illumination transformation on the current sub-tile, and obtain an illumination transformed sub-tile.

In some embodiments, the determining unit 420 is specifically configured to:

In some embodiments, the transforming unit 430 is specifically configured to:

Determining a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-tile;

In some embodiments, the target transformation is a default transformation.

In some embodiments the transformation unit 430 is specifically configured to:

And adjusting the contrast of the attribute information of the pixels in the current sub-block based on the target transformation mode and the target transformation parameters to obtain the sub-block after illumination transformation, wherein the contrast of the attribute information of the pixels in the sub-block after illumination transformation is larger than the contrast of the attribute information of the pixels in the current sub-block.

In some embodiments, the decoding unit 410 is further configured to:

Fig. 13 is a schematic block diagram of an encoder 500 of an embodiment of the present application.

As shown in fig. 13, the encoder 500 may include:

A first determining unit 510 for determining a current sub-tile in the current image;

A second determining unit 520, configured to determine whether to perform illumination transformation on the current sub-picture block;

A transforming unit 530, configured to, when determining to perform illumination transformation on the current sub-tile, perform illumination transformation on the current sub-tile to obtain an illumination transformed sub-tile;

And the encoding unit 540 is configured to encode the sub-picture block after the illumination transformation to obtain a code stream.

In some embodiments, the encoding unit 540 is specifically configured to:

In some embodiments, the second determining unit 520 is specifically configured to:

Acquiring a basic view;

determining a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-tile;

Based on the distribution difference value, it is determined whether the pixel pair is a non-uniform illumination pixel pair.

And determining the maximum difference value between (1-R ₁)/(1-R ₂)、(1-G ₁)/(1-G ₂) and (1-B ₁)/(1-B ₂) as the distribution difference value, wherein R ₁、G ₁、B ₁ respectively represents the values of the B component, the G component and the B component of the pixel in the current sub-block, and R ₂、G ₂、B ₂ respectively represents the values of the B component, the G component and the B component of the pixel in the basic view.

If the distribution difference value is smaller than or equal to a first threshold value, determining that the pixel pair is a pixel pair with inconsistent illumination;

And if the distribution difference value is larger than the first threshold value, determining that the pixel pair is not inconsistent in illumination.

Determining a duty cycle of the illumination-inconsistent pixel pairs based on the number of illumination-inconsistent pixel pairs;

And determining whether to perform illumination transformation on the current sub-image block based on the duty ratio of the pixel pairs with inconsistent illumination.

If the duty ratio of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold value, determining the current sub-block as a sub-block needing illumination conversion;

And if the duty ratio of the pixel pairs with inconsistent illumination is smaller than the second threshold value, determining the current sub-block as a sub-block which does not need illumination conversion.

determining whether the pixel pair is a similar pixel pair;

And when the pixel pairs are not similar pixel pairs, carrying out illumination detection on the pixel pairs, and determining whether the pixel pairs are inconsistent illumination pixel pairs.

determining a depth difference value between the pixel pairs;

In some embodiments, the transforming unit 530 is specifically configured to:

Determining a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-tile;

In some embodiments, the target transformation is a default transformation.

In some embodiments, the transforming unit 530 is specifically configured to:

And adjusting the contrast of the attribute information of the pixels in the current sub-block based on the target transformation mode and the target transformation parameters to obtain the sub-block after illumination transformation, wherein the contrast of the attribute information of the pixels in the sub-block after illumination transformation is smaller than the contrast of the attribute information of the pixels in the current sub-block.

In some embodiments, the encoding unit 540 is specifically configured to:

splicing the sub-blocks after illumination transformation with the basic view to obtain a spliced image;

And encoding the pixels in the spliced image and the pixels in the illumination transformed sub-blocks to obtain the code stream.

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. In particular, the decoder 400 shown in fig. 12 may correspond to a respective subject in the method 200 of performing an embodiment of the present application, and the foregoing and other operations and/or functions of the respective units in the decoder 400 are respectively for implementing a respective flow in the method 210. The encoder 500 shown in fig. 13 may correspond to a respective subject in performing the methods 310-330 of the embodiments of the present application, i.e., the foregoing and other operations and/or functions of the respective units in the encoder 500 are respectively for implementing the respective flows in the methods 310-330.

It should also be understood that each unit in the decoder 400 or the encoder 500 according to the embodiments of the present application may be separately or all combined into one or several other units to form a structure, or some unit(s) thereof may be further split into a plurality of units with smaller functions to form a structure, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The units referred to above are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the decoder 400 or the encoder 500 may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of a plurality of units. According to another embodiment of the present application, the decoder 400 or the encoder 500 according to the embodiment of the present application may be constructed by running a computer program (including program code) capable of executing steps involved in the respective methods on a general-purpose computing device of a general-purpose computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and implementing the encoding method or the decoding method of the embodiment of the present application. The computer program may be recorded on a computer readable storage medium, and loaded into an electronic device and executed therein to implement a corresponding method of an embodiment of the present application.

In other words, the units referred to above may be implemented in hardware, or may be implemented by instructions in software, or may be implemented in a combination of hardware and software. Specifically, each step of the method embodiment in the embodiment of the present application may be implemented by an integrated logic circuit of hardware in a processor and/or an instruction in software form, and the steps of the method disclosed in connection with the embodiment of the present application may be directly implemented as a hardware decoding processor or implemented by a combination of hardware and software in the decoding processor. Alternatively, the software may reside in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in a memory, and the processor reads information in the memory and, in combination with its hardware, performs the steps in the method embodiments referred to above.

Fig. 14 is a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application.

As shown in fig. 14, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. Wherein the processor 610 and the computer-readable storage medium 620 may be connected by a bus or other means. The computer readable storage medium 620 is used to store a computer program 621, the computer program 621 including computer instructions, and the processor 610 is used to execute the computer instructions stored by the computer readable storage medium 620. Processor 610 is a computing core and a control core of electronic device 600 that are adapted to implement one or more computer instructions, in particular to load and execute one or more computer instructions to implement a corresponding method flow or a corresponding function.

The processor 610 may also be referred to as a central processing unit (Central Processing Unit, CPU), for example. The Processor 610 may include, but is not limited to, a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field programmable gate array (Field Programmable GATE ARRAY, FPGA) or other programmable logic device, transistor logic device, discrete hardware components, and the like.

The computer readable storage medium 620 may be, for example, a high speed RAM memory or a Non-volatile memory (Non-VolatileMemory), such as at least one disk memory, or alternatively, at least one computer readable storage medium located remotely from the processor 610. In particular, computer-readable storage media 620 includes, but is not limited to, volatile memory and/or nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate Synchronous dynamic random access memory (Double DATA RATE SDRAM, DDR SDRAM), enhanced Synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCH LINK DRAM, SLDRAM), and Direct memory bus RAM (DR RAM).

The electronic device 600 may be an encoder or an encoding framework according to an embodiment of the present application, the computer readable storage medium 620 stores first computer instructions, the processor 610 loads and executes the first computer instructions stored in the computer readable storage medium 620 to implement corresponding steps in the encoding method provided by the embodiment of the present application, in other words, the first computer instructions in the computer readable storage medium 620 are loaded by the processor 610 and execute corresponding steps, which are not repeated herein.

The electronic device 600 may be, for example, a decoder or a decoding framework according to an embodiment of the present application, the computer readable storage medium 620 stores therein second computer instructions, the processor 610 loads and executes the second computer instructions stored in the computer readable storage medium 620 to implement corresponding steps in the decoding method provided by the embodiment of the present application, in other words, the second computer instructions in the computer readable storage medium 620 are loaded by the processor 610 and execute corresponding steps, which are not repeated herein.

According to another aspect of the present application, there is also provided a codec system including the encoder and decoder referred to above.

According to another aspect of the present application, there is also provided a computer-readable storage medium (Memory) which is a Memory device in the electronic device 600 for storing programs and data. Such as computer-readable storage medium 620. It is understood that the computer readable storage medium 620 herein may include a built-in storage medium in the electronic device 600, and may include an extended storage medium supported by the electronic device 600. The computer-readable storage medium provides storage space that stores an operating system of the electronic device 600. Also stored in this memory space are one or more computer instructions, which may be one or more computer programs 621 (including program code), adapted to be loaded and executed by the processor 610.

According to another aspect of the application, the application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. Such as computer program 621. At this time, the data processing apparatus 600 may be a computer, and the processor 610 reads the computer instructions from the computer-readable storage medium 620, and the processor 610 executes the computer instructions so that the computer performs the encoding method or the decoding method provided in the various alternatives referred to above.

In other words, when implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, runs the processes of, or implements the functions of, embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, from one website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

Those of ordinary skill in the art will appreciate that the elements and process steps of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

Finally, it should be noted that the above is only a specific embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about the changes or substitutions within the technical scope of the present application, and the changes or substitutions are all covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A decoding method, characterized by comprising:

Decode the code stream and determine the current sub-block in the current image;

Determining whether to perform illumination transformation on the current sub-block;

When it is determined to perform illumination transformation on the current sub-image block, illumination transformation is performed on the current sub-image block to obtain a sub-image block after illumination transformation.

The method according to claim 1, characterized in that the determining whether to perform a lighting transformation on the current sub-block comprises:

Decoding the bitstream to determine a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.

The method according to claim 1 or 2, characterized in that the step of performing illumination transformation on the current sub-image block to obtain the sub-image block after illumination transformation comprises:

Determining a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

The method according to claim 3 is characterized in that the target transformation mode is a default transformation mode.

The method according to claim 3 is characterized in that the target transformation parameters are transformation parameters used by default in the target transformation mode.

The method according to claim 3, characterized in that the step of performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters and obtaining the sub-image block after illumination transformation comprises:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-image block.

The method according to any one of claims 1 to 6, characterized in that the method further comprises:

Based on the sub-image block after the illumination transformation, a decoded image of the current image is determined.

A coding method, characterized by comprising:

Determine a current sub-block in a current image;

When it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation;

The sub-image block after the illumination transformation is encoded to obtain a bit stream.

The method according to claim 8, characterized in that encoding the sub-image block after the illumination transformation to obtain a bit stream comprises:

Acquire a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.

The method according to claim 8, characterized in that the determining whether to perform a lighting transformation on the current sub-block comprises:

Get the base view;

Performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-image block to determine whether the pixel pair is a pixel pair with inconsistent illumination;

Based on the number of the pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

The method according to claim 10, characterized in that the step of performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination comprises:

Determine a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block;

Based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

The method according to claim 11, characterized in that the determining of the distribution difference value between the color information of the pixel in the base view and the color information of the pixel in the current sub-block comprises:

The maximum difference value among (1-R ₁ )/(1-R ₂ ), (1-G ₁ )/(1-G ₂ ) and (1-B ₁ )/(1-B ₂ ) is determined as the distribution difference value; wherein R ₁ , G ₁ , B ₁ respectively represent the values of the B component, G component, and B component of the pixel in the current sub-block; and R ₂ , G ₂ , B ₂ respectively represent the values of the B component, G component, and B component of the pixel in the base view.

The method according to claim 11, characterized in that the step of determining whether the pixel pair is a pixel pair with inconsistent illumination based on the distribution difference value comprises:

If the distribution difference value is less than or equal to a first threshold, determining that the pixel pair is a pixel pair with inconsistent illumination;

If the distribution difference value is greater than the first threshold, it is determined that the pixel pair is not a pixel pair with inconsistent illumination.

The method according to claim 10, characterized in that the determining whether to perform illumination transformation on the current sub-block based on the number of pixel pairs with inconsistent illumination comprises:

Determining a proportion of the pixel pairs with inconsistent illumination based on the number of the pixel pairs with inconsistent illumination;

Based on the proportion of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-image block.

The method according to claim 14, characterized in that the determining whether to perform illumination transformation on the current sub-block based on the proportion of the pixel pairs with inconsistent illumination comprises:

If the proportion of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold, determining that the current sub-image block is a sub-image block that needs illumination transformation;

If the proportion of the pixel pairs with inconsistent illumination is less than the second threshold, it is determined that the current sub-image block is a sub-image block that does not require illumination transformation.

Determining whether the pixel pair is a similar pixel pair;

When the pixel pair is not a similar pixel pair, light detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent light.

The method according to claim 16, characterized in that the determining whether the pixel pair is a similar pixel pair comprises:

determining a depth difference value between the pixel pairs;

Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located;

When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair;

When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.

The method according to any one of claims 8 to 17, characterized in that performing illumination transformation on the current sub-image block to obtain the sub-image block after illumination transformation comprises:

Determining a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

The method according to claim 18 is characterized in that the target transformation mode is a default transformation mode.

The method according to claim 18 is characterized in that the target transformation parameters are transformation parameters used by default in the target transformation method.

The method according to claim 18, characterized in that the step of performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters and obtaining the sub-image block after illumination transformation comprises:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-image block.

The method according to any one of claims 8 to 21, characterized in that encoding the sub-image block after the illumination transformation to obtain a bit stream comprises:

Splicing the sub-image block after the illumination transformation with the base view to obtain a spliced image;

The pixels in the spliced image and the pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.

A decoder, comprising:

A decoding unit, used for decoding the code stream and determining a current sub-block in a current image;

A determination unit, used to determine whether to perform illumination transformation on the current sub-block;

A transformation unit is used to perform a lighting transformation on the current sub-image block when it is determined that the current sub-image block is to be subjected to a lighting transformation, so as to obtain a sub-image block after the lighting transformation.

An encoder, characterized in that it comprises:

A first determining unit, configured to determine a current sub-block in a current image;

A second determining unit, used to determine whether to perform illumination transformation on the current sub-block;

A transformation unit, configured to perform a lighting transformation on the current sub-image block to obtain a sub-image block after the lighting transformation when it is determined to perform a lighting transformation on the current sub-image block;

The encoding unit is used to encode the sub-image block after the illumination transformation to obtain a code stream.

An electronic device, comprising:

a processor adapted to execute a computer program;

A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21.

A computer-readable storage medium, characterized in that it is used to store a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21.

A computer program product, comprising a computer program/instruction, characterized in that when the computer program/instruction is executed by a processor, the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21 is implemented.

A code stream, characterized in that the code stream is a code stream in the method according to any one of claims 1 to 7 or a code stream generated by the method according to any one of claims 8 to 21.