CN114339408A - Video decoding method - Google Patents

Abstract

The invention relates to the technical field of multimedia playing, and provides a video decoding method, which comprises the steps of storing data to obtain a video packet queue after a decapsulated video code stream is obtained, and then screening the data of the video packet queue according to a preset screening rule, so that illegal data can be filtered before decoding, the generation of decoding abnormity is reduced, and the decoding efficiency is improved; and then, determining a corresponding target decoding mode according to the attribute information of the target video packet data, creating a hard decoding plug-in and performing video decoding when the target decoding mode is hard decoding, realizing the check of the coding format of the video frame by identifying the attribute information of the video frame in the video code stream in advance, and further allocating the corresponding decoding mode to the video frame, thereby ensuring that the data format transmitted by the player to the soft decoder or the hard decoder is supported, and ensuring that the video can be normally played.

Description

A video decoding method

technical field

The invention relates to the technical field of multimedia playback, in particular to a video decoding method.

Background technique

The complete playback process of a video file is divided into three steps: video decapsulation, video decoding, and video picture display.

Decoders are divided into software decoders, hardware decoders and wireless decoders, among which:

Software decoder: The decoder mentioned in the computer is a soft decoder, that is, the audio and video data are decoded by software methods. Corresponding to it are DVD and vcd machines, which belong to hardware decoders. Usually, if a computer wants to play a certain format of video, it needs a decoder that supports the video encoding, and the video decoder comes into being. RM/RMVB Real Media Decoder MOV Quick Time Decoder 3GP/MP4 Decoder DVD/VOB Decoder Divx Decoder xvid Decoder WMV Decoder.

Hardware decoder: The existence of the decoder is because the audio and video data storage must first be compressed, otherwise the amount of data is too large, and compression requires a certain encoding, in order to use the smallest capacity to store the highest quality audio and video data. Therefore, when the data needs to be played, it must be decoded by the decoder first. The digital encoding formats that can be decoded are AC-3, HDCD, DTS, etc. These are multi-channel audio and video encoding formats. If you want to achieve a high fidelity level, there are two-channel PCM digital encoding. Therefore, when choosing a hardware decoder, you should pay attention to whether the software supports these formats.

In the existing decoding scheme, when the player plays H264 video files, some H264 video files are often played with a blurry screen, a black screen, and the SEEK cannot be played normally during playback. This is because the player only checks the encapsulation format of the H264 video file while ignoring the encoding format during playback. Then, the data format transmitted by the player to the soft decoder is wrong, and the data transmitted by the player is not supported by the hard decoder, etc., resulting in abnormal video decoding of the soft decoder or hard decoder, thus making the H264 video playback display abnormal phenomenon. .

SUMMARY OF THE INVENTION

The present invention provides a video decoding method, which solves the technical problem that the existing video decoding scheme lacks the checking of the encoding format of audio and video data, resulting in abnormal video playback display.

In order to solve the above technical problems, the present invention provides a video decoding method, comprising the steps:

S1, obtain the video code stream for decapsulation to obtain the code stream information, and store it in the video packet queue;

S2, according to the preset screening rule, screen out the target video packet data from the video packet queue;

S3, obtain attribute information of each described target video packet data, and determine target decoding mode according to described attribute information;

S4. When the target decoding mode corresponding to the video code stream is hard decoding, create a hard decoding plug-in and perform video decoding.

In this basic scheme, after obtaining the decapsulated video stream, data storage is performed to obtain a video packet queue, and then data filtering is performed on the video packet queue according to the preset filtering rules, which can filter out illegal data before decoding and reduce the occurrence of decoding abnormality. , to improve the decoding efficiency; then, determine the corresponding target decoding method according to the attribute information of the target video packet data, and when the target decoding method is hard decoding, create a hard decoding plug-in and perform video decoding. The attribute information of the frame can check its encoding format, and then assign the corresponding decoding method to it, so as to ensure that the data format transmitted by the player to the soft decoder or hard decoder is supported, so that the video can be played normally.

In a further embodiment, the step S1 is specifically as follows: decapsulate the video code stream obtained in the encapsulation analysis module of the player, obtain the corresponding code stream information, and save the code stream information to the video packet queue;

Each video packet in the video packet queue stores pts, dts and video compression data corresponding to the video stream.

In a further embodiment, the step S2 includes:

S21, sequentially read each video packet in the video packet queue;

S22, determine whether the number of the corresponding NALU headers of the video packet is greater than the first preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded;

S23, determine whether the data type of the corresponding NALU header of the video packet is greater than the second preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded;

S24. Determine whether the data type of the NALU header corresponding to the video packet is the target value, and if so, mark it as the target video packet data, and if otherwise, determine that the video packet data is illegal and discard it.

According to the normal specification attributes of the video stream, this solution sets the first preset threshold and the second preset threshold corresponding to the number of NALU headers and the data type of the NALU header, and performs preliminary screening of video packets, which can quickly filter out illegal data to improve the readability of video packet data, thereby improving the smoothness of video playback.

In a further embodiment, the step S3 includes:

S31. Read the profile attribute and the level attribute in the sequence parameter set of the target video packet data, determine whether the profile attribute and the level attribute are not greater than a third preset threshold, if so, go to the next step, if not, determine the target decoding method for soft decoding;

S32. Read the video definition from the sequence parameter set, and determine whether the video definition is greater than a fourth preset threshold, if so, determine that the target decoding mode is hard decoding, if not, determine that the target decoding mode is soft decoding ;

The third preset threshold includes the maximum hard-decoded profile attribute and the maximum level attribute; the video definition includes video height and video width, and the fourth preset threshold includes the hard-decoded maximum video height, video Maximum width.

This scheme is limited by the decoding ability of hard decoding, sets the third preset threshold and the fourth preset threshold corresponding to the video packet data profile attribute and level attribute, video definition, and preferentially assigns the video files that meet the hard decoding requirements as Hard decoding, thereby reducing CPU usage and improving decoding efficiency.

In a further embodiment, the step S4 includes:

S41. Create a hard decoding plug-in of H264 decoding type:

S42. Start the hard decoding plug-in of the H264 decoding type through the encoder/decoder component;

S43. Send the target video packet data to the VPU decoder for hardware decoding output.

In a further embodiment, the step S41 includes:

A1. Add H264 decoding type to the enumeration type list;

A2. Copy the hard decoding library header file and hard decoding library of H264 decoding type, and set the compilation output directory;

A3. Configure the CMA start address and related parameters of the H264 decoding type;

A4. Compilation steps A1 to A3 complete the creation of a hard decoding plug-in of the H264 decoding type.

In a further embodiment, the step S42 includes:

B1. Configure the MIME type of the encoder/decoder component as H264 decoding type;

B2. Start the MIME type plug-in in the encoder/decoder component, and then start the corresponding H264 decoding type hard decoding plug-in.

In a further embodiment, the step S43 includes:

C1, drive FFmpeg to read described video stream, separate its corresponding extradata information, audio stream, video stream;

C2. Set the address and cache size of extradata according to the playback requirements;

C3. Parse the extradata through FFmpeg to obtain the sequence parameter set and the image parameter set and combine them into a memory buffer;

C4. Start the VPU decoder to read the memory buffer, perform decoding configuration, and perform hardware decoding and output according to the target video packet data.

By setting the player, this solution creates a H264 decoding type hard decoding plug-in for the video stream, and then starts the VPU decoder for hardware decoding output, which can fully utilize the hardware resources in the system, reduce the CPU load, and effectively improve the video decoding efficiency.

In further embodiments, the present invention also includes:

S5. When the target decoding mode corresponding to the video code stream is soft decoding, adopt the FFmpeg soft decoding mode to perform video decoding on the target video packet data according to the H264 video coding standard format.

In a further embodiment, after step S23, it also includes: judging whether the video packet is a key frame, if so, setting the key frame flag bit key_flag in its data structure to 1, otherwise, setting its key frame flag bit key_flag is 0.

In this solution, in the process of video data screening, key frames are identified, which is beneficial to quickly identify whether the current frame is a key frame during video decoding, and then adopt an appropriate decoding method to improve decoding efficiency.

Description of drawings

FIG. 1 is a working flowchart of a video decoding method provided by an embodiment of the present invention.

Detailed ways

The embodiments of the present invention will be explained in detail below in conjunction with the accompanying drawings. The examples are given only for the purpose of illustration and should not be construed as a limitation of the present invention. The accompanying drawings are only used for reference and description, and do not constitute a limitation on the protection scope of the patent of the present invention. limitation, since many changes may be made in the present invention without departing from the spirit and scope of the invention.

A video decoding method provided by an embodiment of the present invention is shown in FIG. 1 . In this embodiment, an H264 video file is used as an example for description, including steps S1 to S5:

S1. Obtain the video code stream and decapsulate to obtain the code stream information, and store it in the video packet queue, specifically: decapsulate the video code stream obtained in the encapsulation analysis module of the player, and obtain the corresponding code stream information, And save the stream information to the video packet queue;

Each video packet in the video packet queue (ie, the Avpacket queue) stores the pts, dts, and video compression data of the corresponding video stream.

S2. According to preset screening rules, screen out target video packet data from the video packet queue, including steps S21-S24:

S21, sequentially read the video compression data of each video packet in the video packet queue;

S22, determine whether the number of NALU headers corresponding to the video packet is greater than the first preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded;

In this embodiment, the first preset threshold is 0, the full name of NALU (Network Abstract Layer Unit) is the network abstraction layer unit, which is the basic unit of the H264 code stream. A frame of H264 code stream (annex-b packing method) is composed of a series of NALUs, and NALUs are composed of NALU header and NALU body.

The code stream of a frame of H264 (for example, the code stream in the annex-b packaging mode) is composed of a series of NALUs, and the NALU is composed of a NALU header and a NALU body. The NALU header is preceded by a 00 00 00 01 or 00 00 01 delimiter, which can be used to calculate the number of NALU headers.

S23, determine whether the data type of the corresponding NALU header of the video packet is greater than the second preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded;

In this embodiment, the second preset threshold is set to 31.

According to the definition of the H264 standard document, the data type of the NALU header is within the range of 0 to 31. Therefore, if the NALU header exceeds 31, the data is illegal.

In this embodiment, after step S23, it further includes: judging whether the video packet is a key frame, if so, setting the key frame flag key_flag in the data structure to 1, otherwise, setting the key frame flag key_flag to 0.

Among them, the algorithm for judging whether it is a key frame is to judge whether the following operation result is 5:

The binary value corresponding to the NALU data type is logically ANDed with the binary value corresponding to 31.

In this embodiment, in the process of video data screening, the key frame is identified, which is beneficial to quickly identify whether the current frame is a key frame during video decoding, and then adopt an appropriate decoding method to improve decoding efficiency.

S24. Determine whether the data type of the NALU header corresponding to the video packet is the target value, and if so, mark it as the target video packet data, if otherwise, determine that the video packet data is illegal and discard it.

In this embodiment, the target video packet data is a data packet including an SPS unit, and SPS is an abbreviation of SequenceParamater Set, which means sequence parameter set; the target value is set to 7.

In this embodiment, according to the normal specification attribute of the video stream, the first preset threshold and the second preset threshold corresponding to the number of NALU headers and the data type of the NALU header are set, and the video packets are preliminarily screened, which can be quickly filtered out. Illegal data, improve the readability of video packet data, thereby improving the smoothness of video playback.

S3. Obtain attribute information of each target video packet data, and determine the target decoding mode according to the attribute information, including steps S31-S32:

S31, read the profile attribute and the level attribute in the sequence parameter set (ie SPS, Sequence Paramater Set) of the target video packet data, and determine whether the profile attribute and the level attribute are not greater than the third preset threshold, and if so, go to the next step, If otherwise, determine that its target decoding mode is soft decoding;

The third preset threshold includes the hard-decoded maximum value of the profile attribute and the maximum value of the level attribute.

S32, read the video definition from the sequence parameter set, and determine whether the video definition is greater than the fourth preset threshold, if so, determine that its target decoding mode is hard decoding, if otherwise, determine that its target decoding mode is soft decoding;

The video definition includes video height and video width, and the fourth preset threshold includes hard-decoded maximum video height and maximum video width.

This embodiment is limited by the decoding capability of hard decoding, and sets a third preset threshold and a fourth preset threshold corresponding to the video packet data profile attribute and level attribute, video definition, and preferentially allocates video files that meet the hard decoding requirements. It is hard decoding, thereby reducing CPU usage and improving decoding efficiency.

S4. When the target decoding mode corresponding to the video code stream is hard decoding, create a hard decoding plug-in and perform video decoding, including steps S41 to S43:

S41. Create a hard decoding plug-in of the H264 decoding type, including steps A1 to A4:

A1. Add H264 decoding type to the enumeration type list;

A2. Copy the hard decoding library header file and hard decoding library of H264 decoding type, and set the compilation output directory;

A3. Configure the starting address and related parameters of the CMA (Contiguous Memory Allocator, continuous memory block) of the H264 decoding type;

In this embodiment, the relevant parameters include parameters such as the name of the hard decoder, the MIME type, the parsed maximum and minimum resolution, and the baud rate.

A4. Compilation steps A1 to A3 complete the creation of a hard decoding plug-in of the H264 decoding type.

S42, start the hard decoding plug-in of the H264 decoding type through the encoder/decoder (ie MediaCodec) component, including steps B1-B2:

B1. Configure the MIME type of the encoder/decoder component as H264 decoding type;

B2. Start the MIME type plug-in in the encoder/decoder component, and then start the corresponding H264 decoding type hard decoding plug-in.

S43, sending the target video packet data to the VPU decoder for hardware decoding output, including steps C1-C4:

C1. Drive FFmpeg to read the video stream, and separate its corresponding extradata information, audio stream, and video stream;

C2. Set the address and cache size of extradata according to the playback requirements;

C3. Parse the extradata through FFmpeg to obtain the sequence parameter set and the image parameter set and combine them into a memory buffer;

C4. Start the VPU decoder to read the memory buffer, perform decoding configuration, and perform hardware decoding output according to the target video packet data.

Specifically, after starting the VPU decoder, the address of the memory buffer (memory buffer) is sent to the VPU decoder, and after reading the information of the memory buffer, the VPU decoder completes the configuration work before decoding. Then, the memory buffer is sent to the VPU decoder in the form of CMA memory through the interface of MediaCodec for hardware decoding output.

In this embodiment, by setting the player, creating a hard decoding plug-in of H264 decoding type for the video stream, and then starting the VPU decoder for hardware decoding output, the hardware resources in the system can be fully utilized, the CPU load can be reduced, and the effective Improved video decoding efficiency.

S5. When the target decoding mode corresponding to the video code stream is soft decoding, the FFmpeg soft decoding mode is adopted to perform video decoding on the target video packet data according to the H264 video coding standard format.

In this embodiment of the present invention, after the decapsulated video stream is acquired, data is stored to obtain a video packet queue, and then data filtering is performed on the video packet queue according to preset screening rules, so that illegal data can be filtered out before decoding, thereby reducing abnormal decoding. Then, according to the attribute information of the target video packet data, the corresponding target decoding mode is determined, and when the target decoding mode is hard decoding, a hard decoding plug-in is created and video decoding is performed. The attribute information of the video frame can check its encoding format, and then assign the corresponding decoding method to it, so as to ensure that the data format transmitted by the player to the soft decoder or hard decoder is supported, so that the video can be played normally.

The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (10)

1. a video decoding method, is characterized in that, comprises the steps: S1, obtain the video code stream for decapsulation to obtain the code stream information, and store it in the video packet queue; S2, according to the preset screening rule, screen out the target video packet data from the video packet queue; S3, obtain attribute information of each described target video packet data, and determine target decoding mode according to described attribute information; S4. When the target decoding mode corresponding to the video code stream is hard decoding, create a hard decoding plug-in and perform video decoding. 2. a kind of video decoding method as claimed in claim 1 is characterized in that, described step S1 is specifically: carry out decapsulation to the video code stream obtained in the encapsulation analysis module of the player, obtain the corresponding code stream information, and save the code stream information in the video packet queue; Each video packet in the video packet queue stores pts, dts and video compression data corresponding to the video stream. 3. A kind of video decoding method as claimed in claim 1, is characterized in that, described step S2 comprises: S21, sequentially read each video packet in the video packet queue; S22, determine whether the number of the corresponding NALU headers of the video packet is greater than the first preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded; S23, determine whether the data type of the corresponding NALU header of the video packet is greater than the second preset threshold, if so, enter the next step, if otherwise, determine that the video packet data is illegal and discarded; S24. Determine whether the data type of the NALU header corresponding to the video packet is the target value, and if so, mark it as the target video packet data, and if otherwise, determine that the video packet data is illegal and discard it. 4. a kind of video decoding method as claimed in claim 1, is characterized in that, described step S3 comprises: S31. Read the profile attribute and the level attribute in the sequence parameter set of the target video packet data, determine whether the profile attribute and the level attribute are not greater than a third preset threshold, if so, go to the next step, if otherwise, determine the target decoding method is soft decoding; S32. Read the video definition from the sequence parameter set, and determine whether the video definition is greater than a fourth preset threshold, if so, determine that the target decoding mode is hard decoding, if not, determine that the target decoding mode is soft decoding ; The third preset threshold includes the maximum hard-decoded profile attribute and the maximum level attribute; the video definition includes video height and video width, and the fourth preset threshold includes the hard-decoded maximum video height, video Maximum width. 5. A kind of video decoding method as claimed in claim 1, is characterized in that, described step S4 comprises: S41. Create a hard decoding plug-in of H264 decoding type: S42. Start the hard decoding plug-in of the H264 decoding type through the encoder/decoder component; S43. Send the target video packet data to the VPU decoder for hardware decoding output. 6. A video decoding method according to claim 5, wherein the step S41 comprises: A1. Add H264 decoding type to the enumeration type list; A2. Copy the hard decoding library header file and hard decoding library of H264 decoding type, and set the compilation output directory; A3. Configure the CMA start address and related parameters of the H264 decoding type; A4. Compilation steps A1 to A3 complete the creation of a hard decoding plug-in of the H264 decoding type. 7. A video decoding method according to claim 5, wherein the step S42 comprises: B1. Configure the MIME type of the encoder/decoder component to be the H264 decoding type; B2. Start the MIME type plug-in in the encoder/decoder component, and then start the corresponding H264 decoding type hard decoding plug-in. 8. A video decoding method according to claim 5, wherein the step S43 comprises: C1, drive FFmpeg to read described video stream, separate its corresponding extradata information, audio stream, video stream; C2. Set the address and cache size of extradata according to the playback requirements; C3. Parse the extradata through FFmpeg to obtain the sequence parameter set and the image parameter set and combine them into a memory buffer; C4. Start the VPU decoder to read the memory buffer, perform decoding configuration, and perform hardware decoding and output according to the target video packet data. 9. A kind of video decoding method as claimed in claim 1, is characterized in that, also comprises: S5. When the target decoding mode corresponding to the video code stream is soft decoding, adopt the FFmpeg soft decoding mode to perform video decoding on the target video packet data according to the H264 video coding standard format. 10. a kind of video decoding method as claimed in claim 3, is characterized in that, also comprises after step S23: judge whether described video bag is key frame, if so, the key frame flag bit key_flag in its data structure is set is 1, otherwise its key frame flag bit key_flag is set to 0.

Images