Method and device that summary of the invention is realized based on the infra-frame prediction of correlation between future position
The present invention is directed to above-mentioned the deficiencies in the prior art, a kind of video encoding/decoding method and decoding device are provided, it is finished video decoding process through resolving with entropy decoding, parameter decoding and three steps of pixel decoding, processing in each step has similar character and data local characteristics, and provide corresponding decoding device in conjunction with three steps, thereby help reasonably Resources allocation, improve the efficient of video decode.
The present invention is achieved through the following technical solutions:
The video encoding/decoding method that the present invention relates to comprises the steps:
Step 1, the video bit stream that input has been compressed according to the video compression standard that this bit stream is followed, is resolved and the entropy decoding it, obtains syntax element information at different levels;
Described syntax element information, be meant corresponding with the code word in the bit stream, can be by variable length decoding and the information that obtains based on entropy decoding methods such as context decodings.
Step 2, carry out the video parameter decoding according to resulting syntax element information in the step 1, obtain new parameter information, from circulation for the second time, then use resulting syntax element information in the step 1 and the preceding parameter information that once circulates and obtained during execution in step two, carry out the video parameter decoding, obtain new parameter information;
Described parameter information is meant directly and describes image with parametric form, but the code word information corresponding in not direct and the bit stream, they are obtained by syntax element information and existing parameter information computing.
Step 3 is carried out the pixel decoding according to resulting parameter information in resulting syntax element information, the step 2 in the step 1, video image information after obtaining reducing and output; From circulation for the second time, the video image information that is obtained when then using resulting parameter information and a preceding execution in step three in the syntax element information that obtains in the step 1, the step 2 carries out the pixel decoding, new video image information after obtaining reducing and output;
Step 4, the above step 1 that circulates is finished all incoming bit streams to step 3 up to decoding.
The video decoder that the present invention relates to comprises: resolve and entropy decoder module, parameter decoder module, main decoder module, first buffer, second buffer and the 3rd buffer.
Described parsing and entropy decoder module are resolved and the entropy decoding the bit stream of input, obtain syntax element information at different levels, and deposit first buffer in.
Described parsing and entropy decoder module, comprise embedded type CPU (central processing unit) and entropy decoding accelerator, described embedded type CPU is responsible for controlling the operation of entropy decoding accelerator, parameter decoder module and main decoder module, after the initialization of each sheet is finished, send enabling signal to entropy decoding accelerator, parameter decoder module and main decoder module, after the decoding of a sheet was finished, embedded type CPU was accepted the end signal of entropy decoding accelerator, parameter decoder module and main decoder module; Described entropy decoding accelerator is responsible for other syntactic element parsing/entropy decoding work of the following level of macro block rank and macro block.
Described embedded type CPU also is responsible for the parsing/entropy decoding work of the above rank syntactic element of macro block, embedded type CPU uses index Columbus coding/decoding method and the fixed length coding/decoding method based on software, the video bit stream of input is decoded into sequence head information, the header of picture header information and sheet, resulting sequence head information, picture header information, the header of sheet outputs to first buffer on the sheet, the work period of embedded type CPU is unit with the sheet, after finishing decoding to the above rank syntactic element of the macro block of a sheet at every turn, start the entropy decoding accelerator, also finish processing up to the entropy decoding accelerator, begin the work of next sheet then this sheet.
Described entropy decoding accelerator, be responsible for other syntactic element parsing/entropy decoding work of the following level of macro block rank and macro block, the entropy decoding accelerator is after the enabling signal that obtains from embedded type CPU, from first buffer, read the header of decoding sequence of corresponding this film respectively, the picture header information syntactic element also stores in first buffer, first buffer has had the header syntactic element of interior decoded of sheet, the entropy decoding accelerator is according to all syntactic elements of having decoded in first buffer, use hardware based index Columbus and context-adaptive variable length decoding method that incoming bit stream is resolved/the entropy decoding, and store into decoded result in first buffer in order successively, and be maintained as the form of syntactic element fifo queue, finish the decoding of a sheet when the entropy decoding accelerator after, issue end signal of embedded type CPU, and enter wait state, till embedded type CPU sends enabling signal next time.
Described parameter decoder module obtains to be calculated the parameter information that makes new advances, and deposited in second buffer by the syntax element information of resolving and the entropy decoder module deposits in from first buffer.
Described parameter decoder module, be responsible for decoded syntactic element is converted to parameter information, as long as have data to use in the syntactic element, the parameter decoder module is promptly started working, the work period of parameter decoder module is unit with the sheet, when each sheet begins, the parameter decoder module reads the above rank syntactic element of the macro block corresponding with this film from first buffer, and store second buffer into, the parameter decoder module is according to the macro block higher level syntactic element that is stored in second buffer, macro block rank and the following rank syntactic element of macro block are carried out the parameter decoding, its result comprises: motion vector, intra prediction mode, boundary filtering strength and quantization parameter, the parameter decoded results is deposited into second buffer, distance of swimming amplitude information and reference key need not parameter decoding and can directly be used as parameter in the syntactic element, and this class syntactic element directly is transferred in second buffer.
Described main decoder module, from first buffer, obtain by the syntax element information of resolving and the entropy decoder module deposits in, the parameter information that acquisition is deposited in by the parameter decoder module from second buffer calculates the video image information that makes new advances, and deposits the 3rd buffer and output in.
Described main decoder module comprises following submodule: re-quantization submodule, inverse transformation submodule, infra-frame prediction submodule, reference pixel obtain submodule, fractional samples interpolation submodule, compensation reconstruct submodule and loop filtering submodule.
Described re-quantization submodule reads macro block (mb) type, quantization parameter and distance of swimming amplitude information etc. from second buffer, foundation is inverse scan, Hadamard transform and the inverse quantization method of prescribed by standard H.264, with the 4x4 piece is unit, obtain transform coefficient matrix, and this matrix is sent to the inverse transformation submodule.
Thereby described inverse transformation submodule is responsible for that the transform coefficient matrix that obtains is carried out the inverse transformation of 4x4 integer and is obtained the residual sample matrix, and this matrix is sent to compensation reconstruct submodule.
Described infra-frame prediction submodule reads macro block (mb) type and intra prediction mode etc. from second buffer, according to the reference sample without loop filtering from compensation reconstructed module feedback, carry out infra-frame prediction and will predict the outcome being sent to the compensation reconstructed module.
Described reference pixel obtains submodule according to macro block (mb) type, reference key and the motion vector etc. that read from second buffer, from the 3rd buffer, read the video image information that carries out the required reference of corresponding fractional samples interpolation, and result and the required parameter information of fractional samples interpolation are sent to fractional samples interpolation submodule.The video image information of the required reference of described fractional samples interpolation is meant before this decoded video image information in the main decoder module running.
Described fractional samples interpolation submodule carries out the filtering interpolation according to the parameter information and the reference pixel that are obtained, and the result is sent to compensation reconstruct submodule.
Described compensation reconstruct submodule is the work period with macro block (MB), when each macro block begins, at first reads macro block (mb) type from second buffer.If current macro is I_PCM (type pulse code modulation in a frame) type, then from second buffer, read PCM (pulse code modulation) value successively, directly output to the loop filtering submodule, and feed back to the infra-frame prediction submodule; If current macro is I (in the frame) type of mb except I_PCM (type pulse code modulation in the frame), then obtain the residual sample matrix from the inverse transformation submodule, obtain the infra-frame prediction result from the infra-frame prediction submodule, output to the loop filtering submodule after then both being compensated, and feed back to the infra-frame prediction submodule; If current macro is P (single directional prediction) type or B (bi-directional predicted) type of mb, then obtain the difference sample matrix from the inverse transformation submodule, obtain interpolation result from fractional samples interpolation submodule, output to the loop filtering submodule after then both being compensated, and feed back to the infra-frame prediction submodule.
Described loop filtering submodule reads the information of decoded picture in anter from the 3rd buffer, from compensating the image pattern after the reconstruct submodule reads the current macro compensation, according to parameters such as macro block (mb) type that from second buffer, reads and boundary filtering strengths, carry out block-eliminating effect filtering, and the filtering result write the 3rd buffer as video image information, and output image information.
Compared with prior art, the present invention has following beneficial effect: the inventive method is according to the relation between the various piece in the video decoding process, comprise sequencing, operand, data dependency and data local characteristics, video decoding process is finished through resolving with entropy decoding, parameter decoding and three steps of pixel decoding.The present invention also provides a kind of video decoder, comprise: parsing and entropy decoder, parameter decoder, main decoder module, be used for respectively finishing and resolve and entropy decoding, parameter decoding and three steps of pixel decoding, and first buffer, second buffer and the 3rd buffer are used for storing intermediate data.Contents processing in each step has similar computing granularity and data local characteristics, thereby helps reasonably Resources allocation, simplifies the internal interface of video decode.
Embodiment
Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment is being to implement under the prerequisite with the technical solution of the present invention, provided detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to following embodiment.
Present embodiment is based on standard basis specification (BASELINE PROFILE) H.264.
Present embodiment is achieved by the following technical solution.
As shown in Figure 2, the decoding device that present embodiment provided comprises: resolve and entropy decoder module, parameter decoder module, main decoder module, first buffer, second buffer and the 3rd buffer.
Described parsing and entropy decoder module are resolved and the entropy decoding the bit stream of input, obtain syntax element information at different levels, and deposit first buffer in.
Described parameter decoder module obtains to be calculated the parameter information that makes new advances, and deposited in second buffer by the syntax element information of resolving and the entropy decoder module deposits in from first buffer.
Described main decoder module, from first buffer, obtain by the syntax element information of resolving and the entropy decoder module deposits in, the parameter information that acquisition is deposited in by the parameter decoder module from second buffer calculates the video image information that makes new advances, and deposits the 3rd buffer and output in.
Described first buffer is responsible for storage by resolving the syntax element information that deposits in the entropy decoder module.
Described second buffer is responsible for the parameter information that storage is deposited in by the parameter decoder module.
Described the 3rd buffer is responsible for the video image information that storage main decoder module calculates.
As shown in Figure 3, be the The general frame of present embodiment device, wherein solid line bar arrow is a data transfer direction, the dashed bars arrow is the control transmission direction.
Described parsing and entropy decoder module, comprise embedded type CPU (central processing unit) and entropy decoding accelerator, described embedded type CPU is responsible for controlling the operation of entropy decoding accelerator, parameter decoder module and main decoder module, after the initialization of each sheet is finished, send enabling signal to entropy decoding accelerator, parameter decoder module and main decoder module, after the decoding of a sheet was finished, embedded type CPU was accepted the end signal of entropy decoding accelerator, parameter decoder module and main decoder module; Described entropy decoding accelerator is responsible for other syntactic element parsing/entropy decoding work of the following level of macro block rank and macro block.
Described first buffer comprises being used to resolve the part of decoding with entropy in first memory and the synchronous DRAM (SDRAM).
Described second buffer comprises the part that is used for the parameter decoding on the sheet in the second memory and synchronous DRAM.
Described the 3rd buffer comprises the part that is used to store output image data and reference image data in the synchronous DRAM.
Described first memory is responsible for the header syntactic element of storage through the sheet of embedded type CPU decoding; Described first memory, also be responsible for resolving at the entropy decoding accelerator/the entropy decode procedure in interim the storage header of decoding sequence, the picture header information syntactic element that from synchronous DRAM, read.
Described second memory is responsible for the above rank syntactic element of macro block that interim storage is read in parameter decoding module parameter decode procedure from synchronous DRAM; Described second memory also is responsible for the parameter information that stored parameter is decoded.
Described synchronous DRAM is responsible for storage and is outputed to synchronous DRAM through the sequence head information and the picture header information of embedded type CPU decoding; Described synchronous DRAM also is responsible for macro block rank and other syntactic element of level below the macro block that storage obtains through the decoding of entropy decoding accelerator; Described synchronous DRAM also is responsible for the video image information that storage is obtained by the decoding of main decoder module.
The image of present embodiment outputs to display device by display interface.
Described embedded type CPU also is responsible for the parsing/entropy decoding work of the above rank syntactic element of macro block, embedded type CPU uses index Columbus coding/decoding method and the fixed length coding/decoding method based on software, the video bit stream of input is decoded into sequence head information, the header of picture header information and sheet, resulting sequence head information and picture header information output to synchronous DRAM, resulting header outputs to first memory on the sheet, the work period of embedded type CPU is unit with the sheet, after finishing decoding to the above rank syntactic element of the macro block of a sheet at every turn, start the entropy decoding accelerator, parameter decoder module and main decoder module, up to the entropy decoding accelerator, parameter decoder module and main decoder module are also finished the processing to this sheet, begin the work of next sheet then.
Described entropy decoding accelerator, be responsible for other syntactic element parsing/entropy decoding work of the following level of macro block rank and macro block, the entropy decoding accelerator is after the enabling signal that obtains from embedded type CPU, from synchronous DRAM, read the header of decoding sequence of corresponding this film respectively, the picture header information syntactic element also stores in the first memory, first memory has had the header syntactic element of interior decoded of sheet, the entropy decoding accelerator is according to all syntactic elements of having decoded in the first memory, use hardware based index Columbus and context-adaptive variable length decoding method that incoming bit stream is resolved/the entropy decoding, and store in the synchronous DRAM successively decoded result in order, and be maintained as the form of syntactic element fifo queue, finish the decoding of a sheet when the entropy decoding accelerator after, issue end signal of embedded type CPU, and enter wait state, till embedded type CPU sends enabling signal next time.
Described embedded type CPU and entropy decoding accelerator have been finished the step 1 of coding/decoding method in the present embodiment, as shown in Figure 1, that is: the video bit stream that compressed of input, according to the video compression standard that this bit stream is followed, it is resolved and the entropy decoding, obtain syntax element information at different levels;
Described syntax element information, be meant corresponding with the code word in the bit stream, can be by variable length decoding and the information that obtains based on entropy decoding methods such as context decodings.In the present embodiment, syntactic element comprises macro block (mb) type (MB TYPE), distance of swimming amplitude information (RUN-LEVEL), motion vector residual error (MVD) and reference key macro block rank and the following rank syntactic elements of macro block such as (REF INDEX), also comprises the header above rank syntactic elements of macro block such as (SLICE HEADER) of sequence head information (SEQUENCE HEADER), picture header information (PICTURE HEADER) and sheet.
Described parameter decoder module, be responsible for decoded syntactic element is converted to parameter information, as long as have data to use in the syntactic element fifo queue, the parameter decoder module is promptly started working, the work period of parameter decoder module is unit with the sheet, when each sheet begins, the parameter decoder module reads the above rank syntactic element of the macro block corresponding with this film from synchronous DRAM, and store second memory into, the parameter decoder module is according to the macro block higher level syntactic element that is stored in the second memory, macro block rank and the following rank syntactic element of macro block that obtains successively carried out the parameter decoding, its parameter information comprises: motion vector (MV), intra prediction mode (INTRAMODE), boundary filtering strength (BS) and quantization parameter (QP), the parameter information of parameter decoding is deposited into second memory.Distance of swimming amplitude information and reference key need not parameter decoding and can directly be used as parameter in the syntactic element, and directly are transferred in the second memory.
Described parameter decoder module has been finished the step 2 of the coding/decoding method that present embodiment provided, and as shown in Figure 1, that is: carries out the video parameter decoding according to resulting syntax element information in the step 1, obtains new parameter information.From circulation for the second time, then use resulting syntax element information in the step 1 and the preceding parameter information that once circulates and obtained during execution in step two, carry out video parameter and decode, obtain new parameter information.
Described parameter information is meant directly and describes image with parametric form, but the code word information corresponding in not direct and the bit stream, they are obtained by syntax element information and existing parameter information computing.In the present embodiment, parameter information comprises motion vector, boundary filtering strength, quantization parameter and intra prediction mode.
As shown in Figure 4, described main decoder module comprises following submodule: re-quantization submodule, inverse transformation submodule, infra-frame prediction submodule, reference pixel obtain submodule, fractional samples interpolation submodule, compensation reconstruct submodule and loop filtering submodule.Among the figure, solid line bar arrow is the parameter information transmission direction, and hollow lines arrow is the data information transfer direction.
Described re-quantization submodule reads macro block (mb) type, quantization parameter and distance of swimming amplitude information from second memory, foundation is inverse scan, Hadamard transform and the inverse quantization method of prescribed by standard H.264, with the 4x4 piece is unit, obtain transform coefficient matrix, and this matrix is sent to the inverse transformation submodule.
Thereby described inverse transformation submodule is responsible for that the transform coefficient matrix that obtains is carried out the inverse transformation of 4x4 integer and is obtained the residual sample matrix, and this matrix is sent to compensation reconstruct submodule.
Described infra-frame prediction submodule reads macro block (mb) type and intra prediction mode etc. from second memory, according to the reference sample without loop filtering from compensation reconstructed module feedback, carry out infra-frame prediction and will predict the outcome being sent to the compensation reconstructed module.
Described reference pixel obtains submodule according to macro block (mb) type, reference key and the motion vector etc. that read from second memory, from synchronous DRAM, read the video image information that carries out the required reference of corresponding fractional samples interpolation, and result and the required parameter information of fractional samples interpolation are sent to fractional samples interpolation submodule.The video image information of the required reference of described fractional samples interpolation is meant decoded video image information in the main decoder module running before anter.
Described fractional samples interpolation submodule carries out the filtering interpolation according to the parameter information and the reference pixel that are obtained, and the result is sent to compensation reconstruct submodule.
Described compensation reconstruct submodule is the work period with macro block (MB), when each macro block begins, at first reads macro block (mb) type from second memory.If current macro is I_PCM (type pulse code modulation in a frame) type, then from second memory, read PCM (pulse code modulation) value successively, directly output to the loop filtering submodule, and feed back to the infra-frame prediction submodule; If current macro is I (in the frame) type of mb except I_PCM (type pulse code modulation in the frame), then obtain the residual sample matrix from the inverse transformation submodule, obtain the infra-frame prediction result from the infra-frame prediction submodule, output to the loop filtering submodule after then both being compensated, and feed back to the infra-frame prediction submodule; If current macro is P (single directional prediction) type or B (bi-directional predicted) type of mb, then obtain the difference sample matrix from the inverse transformation submodule, obtain interpolation result from fractional samples interpolation submodule, output to the loop filtering submodule after then both being compensated, and feed back to the infra-frame prediction submodule.
Described loop filtering submodule is reading decoded video image information in anter from synchronous DRAM, from compensating the image pattern after the reconstruct submodule reads the current macro compensation, according to macro block (mb) type that from second memory, reads and boundary filtering strength parameter, carry out block-eliminating effect filtering, and the filtering result write synchronous DRAM as video image information, and output image information.
Described main decoder module, finished the step 3 of method that present embodiment provides, as shown in Figure 1, that is: according to resulting parameter information in resulting syntax element information, the step 2 in the step 1, carry out the pixel decoding, video image information after obtaining reducing and output; From circulation for the second time, the video image information that then uses resulting parameter information in the syntax element information that obtains in the step 1 and the step 2 to decode and obtained during a preceding execution in step three, carry out the pixel decoding, new video image information after obtaining reducing and output.
The step 4 of method that present embodiment provides is that circulation step one is finished all incoming bit streams to step 3 up to decoding.
The overall work process of present embodiment device is as follows: the video bit stream of input at first carries out the parsing/entropy decoding work of the above rank syntactic element of macro block in embedded type CPU, resulting sequence head information and picture header information output to synchronous DRAM, resulting header outputs to first memory on the sheet, the work period of embedded type CPU is unit with the sheet, after finishing decoding, start entropy decoding accelerator, parameter decoder module and main decoder module to the above rank syntactic element of the macro block of a sheet at every turn.The entropy decoding accelerator is after the enabling signal that obtains from embedded type CPU, carry out other syntactic element parsing/entropy decoding work of the following level of macro block rank and macro block, and store in the synchronous DRAM successively decoded result in order, and be maintained as the form of syntactic element fifo queue, finish the decoding of a sheet when the entropy decoding accelerator after, issue end signal of embedded type CPU; Described parameter decoder module, obtain from the enabling signal of CPU and detect in the syntactic element fifo queue data available is arranged after, just decode by parameter, syntactic element in the formation is converted to parameter information, and deposit in the second memory, finish the decoding of a sheet when the parameter decoder module after, issue end signal of embedded type CPU; Described main decoder module, obtain from the enabling signal of CPU and detect in the second memory data available is arranged after, just decode by pixel, calculate image information when anter, finish the decoding of a sheet when the main decoder module after, issue end signal of embedded type CPU, the image information of main decoder module decoding gained deposits synchronous dynamic random access memory in and shows for display interface output.
Present embodiment is gone up at FPGA (field programmable gate array) and is realized, concrete chip model is XILINXVIRTEX-4; Embedded type CPU has adopted PowerPC 405 processors of IBM (International Business Machines Corporation); Synchronous dynamic random access memory (SDRAM) has adopted two generation Double Data Rate (DDR-2) chips.The operating frequency of embedded type CPU is 200 megahertzes (MHZ), and the operating frequency of FPGA and SDRAM is 50 megahertzes.
The contents processing of present embodiment in each step has similar computing granularity and data local characteristics, thereby help reasonably Resources allocation, simplify the internal interface of video decode, decode by the present embodiment device and to meet H.264HIGH SD size (720x576 pixel) video of PROFILE (high-end standard), frame per second is per second 30 frames.