CN101472174A - Method and device for recuperating original image data in video decoder - Google Patents

Abstract

The invention relates to a method and a device for reducing the transposition calculation in a video decoder through the reconstruction. The method comprises the following steps: adding residual block matrixes generated by the conversion of inverted discrete cosines and decoded, and the reconstructed predication block matrixes, so as to generate consolidated value of block matrixes; conducting the transposition on the consolidated value of block matrixes so as to generate reconstructed image data of the current macro blocks; and outputting the reconstructed image data according to the line sequence of the original image. The method has the advantages that the operation of the block matrix transposition is conducted after the pixels reconstruct units, so as to avoid the resource of a motion compensation unit, which serves as a system bottleneck, and greatly reduces the system delay.

Description

The method and the device that are used for recuperating original image data in the Video Decoder

Technical field

The present invention relates to video coding and decoding technology, relate in particular to the method and apparatus that in Video Decoder, is used for recuperating original image data.

Background technology

For example H.264, in AVS or the Real standard in a lot of important video encoding standards,, decoder is recuperating original image data from the bit stream of compression.Whole process is as follows, at first, and entropy decoder decoding conversion coefficient and motion vector; Secondly, conversion coefficient is rebuild the residual block matrix by the inverse transformation of frequency-domain model; Simultaneously, decoder produces the prediction block matrix according to motion vector parameter and corresponding reference frame; At last, prediction block matrix and residual image addition obtain the reconstructed image of decoding end.

Particularly, as shown in Figure 1, when prediction block matrix and the addition of residual image matrix, further comprising the steps of: the residual block matrix of IDCT (reverse discrete cosine transform) unit output and prediction block matrix in input pixel reconstruction unit earlier process transposition cell translation become matrix according to the original image line output, deposit corresponding cache region in; , export line by line residual block matrix and the addition of prediction block matrix in the pixel reconstruction unit.Wherein, according to existing standard, the direction of the relative original image of residual block of IDCT unit output is by row output, and therefore the reconstructed image of final output need carry out transposition by row output before final output.In like manner, for prediction block matrix, also need before final output, become the direction of relative original image direction by transposition by line output by row output.

In the hardware system of supporting many video encoding and decoding standards, the IDCT unit need be finished just outputing to reconfiguration unit behind the transposition of result of calculation behind the reverse discrete cosine transform of finishing two dimension.Therefore in the process of recuperating original image data, need a large amount of matrix operations, especially transposition computing, and the transposition unit must be made of register.The complexity height that prior art scheme hardware is realized and used, computing is slow, and the transposition computing has increased system delay greatly all finishing as the motion compensation units of coding and decoding video streamline bottleneck in a large number, has badly influenced code efficiency.

Summary of the invention

In view of there being the problems referred to above in the prior art, the present invention improves the Video Decoder that is used for recuperating original image data, the residual block matrix is merged with the prediction block matrix need not under the situation of transposition, behind the pixel reconstruction unit, be combined matrix again and carry out transposition, export in proper order, reduce transposition unit number thereby reach according to relative original image is capable, and finish after the matrix transpose operation of macrooperation amount is placed on the pixel reconstruction unit, reduced the load of motion compensation units.The merging that it will be understood by those skilled in the art that prediction block matrix mentioned herein and residual block matrix is not limited in does simple addition to two matrixes, and this merging process also can be weighting summation, the perhaps operation in the reconstruct.

According to an aspect of the present invention, a kind of method that is used for recuperating original image data in Video Decoder is provided, wherein, may further comprise the steps: with the decoded residual block matrix of reverse discrete cosine transform generation and the prediction block matrix addition of reconstruction, to generate the merging value of block matrix; Merging value to described block matrix is carried out transposition, to generate the reconstructed image data of current macro; Export the reconstructed image data of described current macro by the row order of original image.

According to another aspect of the present invention, a kind of video decoder that is used for recuperating original image data also is provided, comprise: merge device, be used for the decoded residual block matrix of reverse discrete cosine transform generation and the prediction block matrix addition of reconstruction, to generate the merging value of block matrix; The transposition device is used for the merging value of described block matrix is carried out transposition, to generate the reconstructed image data of current macro; Output device is used for the capable reconstructed image data of exporting described current macro in proper order by original image.

In the technical scheme of the present invention, Video Decoder carries out the operation of block matrix transposition behind the pixel reconstruction unit, has avoided taking the resource as the motion compensation units of system bottleneck, has significantly reduced time-delay.In addition, by to adjustment, hardware is done under the prerequisite of bigger change avoiding, reduced the transposition unit, reduced the volume of chip and reduced cost transposition and addition order of steps.

Description of drawings

By reading the detailed description of doing with reference to the following drawings that non-limiting example is done, it is more obvious that other features, objects and advantages of the present invention will become:

Fig. 1 is the Organization Chart of the recovery initial data coding of prior art;

Fig. 2 is the schematic diagram of video decode pipelined architecture;

Fig. 3 is the Organization Chart according to recovery initial data coding of the present invention;

Fig. 4 is the method flow diagram that is used for recuperating original image data in Video Decoder according to a specific embodiment of the present invention;

Fig. 5 is the method flow diagram that is used for obtaining at inter-frame forecast mode the prediction block matrix in Video Decoder according to a specific embodiment of the present invention;

Fig. 6 is the method flow diagram that is used for obtaining at inter-frame forecast mode the prediction block matrix in Video Decoder according to a specific embodiment of the present invention;

Fig. 7 is the schematic diagram of embodiment pixel position corresponding among Fig. 6;

Fig. 8 is the method flow diagram that is used for obtaining at intra prediction mode the prediction block matrix in Video Decoder according to a specific embodiment of the present invention;

Fig. 9 a is the schematic diagram of embodiment pixel position corresponding among Fig. 8;

Fig. 9 b is the predictive mode direction schematic diagram of the 4*4 luminance component piece of H.264 video encoding and decoding standard defined;

Figure 10 is the block diagram that is used for recuperating original image data in Video Decoder according to a specific embodiment of the present invention.

Embodiment

The present invention introduces the device/module of the recuperating original image data that second aspect present invention provided in the pipelined architecture (as shown in Figure 2) of video decode, with the method that realizes that first aspect present invention was provided.

Below in conjunction with Fig. 2 the pipelined architecture of whole video decoding and the background technology that hereinafter will mention are further described.In video decode, be elementary cell with the macro block, adopt streamline to realize whole decoding.Whole pipelined architecture is divided into 4 grades, and it is synchronous between level and the level with the macro block to be that unit adopts Handshake Protocol to realize.In the first order, basic layer bit stream decoded, extract the raw information of macro block and residual block, and the needed motion vector information of inter prediction.In the second level, obtain the residual block matrix by reverse discrete cosine transform, for the situation of inter prediction, obtain predicting the information of block matrix by the motion vector that is obtained in the first order.In the third level, residual block matrix and prediction block matrix are merged, generate according to the capable reconstructed image of exporting in proper order of original image, for the situation of infra-frame prediction, predict that accordingly block matrix also obtains in this one-level.In the fourth stage, select the suitable reconstructed image that the third level is generated to carry out filtering, optimize picture quality.Emphasis of the present invention is in the second level and the third level, comprises and rebuilds the prediction block matrix, and itself and the residual block matrix that generates are merged the process of sending into filtering stage in the ring according to relative original image by line output.

Predicted picture in the coding and decoding video in cataloged procedure, is predicted by the data of previous coding, generates each coded macroblocks.The prediction of current macro is that the image pixel from encoded (be arranged in same frame or be arranged in the previous coding frame) produces.Current macro deducts this predicted value, with the end value compression, generates residual block and is transferred to encoder.And the information that is transferred to encoder comprises that also decoder repeats the forecasting process information necessary, motion vector for example, predictive mode etc.Decoder produces an identical predicted macroblock, and extracts with the form of matrix, its residual block matrix with decoding is merged again.Suppose that the original image matrix of input is X, the prediction block matrix is P (X), and corresponding residual block matrix is R (X), then residual block matrix R (X)=X-P (X) coding transmission.In decoding end, video decoder decoded residual block matrix R (X) produces identical prediction block matrix P (X), rebuilds reconstructed blocks matrix X, then X=P (X)+R (X).In actual operation, because the influence of quantizing process and some other factorses, but the matrix before and after the decoding is similar not necessarily identical.For convenience of description, residual block matrix and former residual block matrix after present embodiment changes with R (X) expression IDCT are referred to as the residual block matrix.In like manner, the prediction block matrix before and after the decoding is referred to as the prediction block matrix, with P (X) expression.For satisfying many video encoding and decoding standards, original image matrix after the reconstruct need be according to the row order output of original image matrix, in other words, in decoding end, the merging value of prediction block matrix and residual block matrix is final also must be by the row order output of original image matrix.

That is:

X＝(X′) ^T＝(P′(X)+R′(X)) ^T

Wherein matrix X ', P ' (X) and R ' be respectively original image matrix X, prediction block matrix P (X) and residual block matrix R (X) matrix (X) according to the arrangement of original image column direction, for example, the row vector of matrix X ' equals the column vector of matrix X.From formula, we are not difficult to find out that the original image matrix after the reconstruct can be represented with the transposition of the merging value of predicting block matrix and residual block matrix.Prediction block matrix here and residual block matrix all are to arrange according to the column direction of original image matrix.

In order to understand the present invention better, be further described below in conjunction with specific embodiments of the invention.

Fig. 3 is according to the Organization Chart that is used for recuperating original image data in coding and decoding video of the present invention.Fig. 4 is the method flow diagram of an embodiment.Following present invention is described with reference to Fig. 3, Fig. 4 and in conjunction with first embodiment, all is that 4 x, 4 block matrix are example with residual block and predicted macroblock wherein.Suppose that the original image matrix of input is $X=\left[\begin{array}{cccc}A0& A1& A2& A3\\ B0& B1& B2& B3\\ C0& C1& C2& C3\\ D0& D1& D2& \mathrm{<figure-callout\; id="3"\; label="D"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">D</figure-callout>}3\end{array}\right],$ The prediction block matrix is $P\left(X\right)=\left[\begin{array}{cccc}a\&prime;0& a\&prime;1& a\&prime;2& a\&prime;3\\ b\&prime;0& b\&prime;1& b\&prime;2& b\&prime;3\\ c\&prime;0& c\&prime;1& c\&prime;2& c\&prime;3\\ d\&prime;0& d\&prime;1& d\&prime;2& d\&prime;3\end{array}\right],$ Corresponding residual block matrix is $R\left(X\right)=\left[\begin{array}{cccc}a0& a1& a2& a3\\ b0& b1& b2& b3\\ c0& c1& c2& c3\\ d0& d1& d2& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3\end{array}\right].$

Describe for simplifying, the direction that numeral changes in the present embodiment is the line direction of original image, and the direction that letter changes is the column direction of original image.Then, residual block matrix R (X) is encoded, transmits, and for example in DCT Video Codec mixed model, the residual error relevant information is transmitted with the form of VLCs, comprises the macro block header, (run is level) to conversion coefficient of, each macro block etc.Simultaneously, rebuild the relevant information of predicting block matrix P (X) and also be transferred to decoding end.

At first, in step S40, video decoder obtains the residual block matrix of IDCT generation and the prediction block matrix of reconstruction.

Video decoder decodes the (run of macro block header, conversion coefficient and each piece from compression bit stream, relevant information such as level), encode and reorder by reverse run-level, sample block matrix after obtaining quantizing, the sample block matrix of reverse discrete cosine transform unit as shown in Figure 3 after to change of scale carries out idct transform, to generate decoded residual block matrix $R\&prime;\left(X\right)=\left[\begin{array}{cccc}a0& b0& c0& d0\\ a1& b1& c1& d1\\ a2& b2& c2& d2\\ a3& b3& c3& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3\end{array}\right].$

The operation principle of IDCT can adopt following formula to be described:

R(X)＝A ^TFA＝((FA) ^TA) ^T＝(R′(X)) ^T

Then R ' (X)=(FA) ^TA

Wherein, R (X) is the residual block matrix of 4x4, and F is the sample block matrix, and A is the transformation matrix of 4x4, and T is the transpose of a matrix factor.

From formula, as can be seen,, establish for each 4x4 sample block matrix F $F=\left[\begin{array}{cccc}{a}_{F}0& a_{F}1& a_{F}2& a_{F}3\\ b_{F}0& b_{F}1& b_{F}2& b_{F}3\\ c_{F}0& c_{F}1& c_{F}2& c_{F}3\\ d_{F}0& d_{F}1& d_{F}2& d_{F}3\end{array}\right],$ Earlier according to [a _F0, a _F1, a _F2, a _F3], [b _F0, b _F1, b _F2, b _F3] ... order is done conversion line by line, and then according to $\left[\begin{array}{c}{a}_{F}0\\ b_{F}0\\ c_{F}0\\ d_{F}0\end{array}\right],$ $\left[\begin{array}{c}{a}_{F}1\\ b_{F}1\\ c_{F}1\\ {\mathrm{<figure-callout\; id="1"\; label="d\; F"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}}_{F}1\end{array}\right],$ ... order do conversion by row.Shu Chu residual block matrix at last $R\&prime;\left(X\right)=\left[\begin{array}{cccc}a0& b0& c0& d0\\ a1& b1& c1& d1\\ a2& b2& c2& d2\\ a3& b3& c3& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3\end{array}\right],$ Wherein [a0, a1, a2, a3] is the capable orientation of raw image data, and in other words, the residual block matrix shown in the matrix is to arrange according to the column direction of original image.The residual block matrix that is obtained is deposited in the residual error data buffering area.

Decoder is rebuild the prediction block matrix according to the information that parses in the elementary bit stream $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right],$ Described reconstruction can comprise interframe conversion or frame inner conversion.According to the difference of Forecasting Methodology, the prediction block matrix after the reconstruction can be stored in inter prediction buffering area or infra-frame prediction buffering area.Can addition for the component that guarantees prediction block matrix and residual block matrix relevant position, the prediction block matrix must be to arrange according to the column direction of original image.

Then, in step S41, video decoder merges residual block matrix and prediction block matrix, promptly obtain P ' (X)+R ' (X).Video decoder reads earlier the residual block matrix from buffering area $\left[\begin{array}{cccc}a0& b0& c0& d0\\ a1& b1& c1& d1\\ a2& b2& c2& d2\\ a3& b3& c3& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3\end{array}\right]$ With the prediction block matrix after the reconstruction $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right],$ By the pixel reconstruction unit data on residual block matrix and the prediction block matrix correspondence position are merged again.For example the data on the relevant position are carried out addition, generate the merging value of block matrix, promptly $\left[\begin{array}{cccc}a0+a\&prime;0& b0+b\&prime;0& c0+c\&prime;0& d0+d\&prime;0\\ \mathrm{<figure-callout\; id="1"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}1+a\&prime;1& \mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}1+b\&prime;1& \mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}1+c\&prime;1& \mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}1+d\&prime;1\\ a2+a\&prime;2& b2+b\&prime;2& c2+c\&prime;2& d2+d\&prime;2\\ \mathrm{<figure-callout\; id="3"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}3+a\&prime;3& \mathrm{<figure-callout\; id="3"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}3+b\&prime;3& \mathrm{<figure-callout\; id="3"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}3+c\&prime;3& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3+d\&prime;3\end{array}\right].$ The merging value of described block matrix is admitted to the transposition unit.

Afterwards, in step S42, the transposition unit carries out transposition with the merging value of institute's write-in block matrix, promptly finishes (P ' (X)+R ' (X)) ^T, and deposit the restructuring matrix of the macro block that generates behind the transposition in the reconstructed image data buffer zone.Described transposition is that the position of the row and column of the merging value of block matrix is changed, and with first row of the merging value of block matrix, first row as restructuring matrix, the secondary series of the merging value of block matrix is as second row of restructuring matrix, and the like.The reconstructed blocks matrix that is generated in this example is $\left[\begin{array}{cccc}a0+a\&prime;0& a1+a\&prime;1& a2+a\&prime;2& a3+a\&prime;3\\ b0+b\&prime;0& \mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}1+b\&prime;1& b2+b\&prime;2& \mathrm{<figure-callout\; id="3"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}3+b\&prime;3\\ c0+c\&prime;0& \mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}1+c\&prime;1& c2+c\&prime;2& \mathrm{<figure-callout\; id="3"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}3+c\&prime;3\\ d0+d\&prime;0& \mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}1+d\&prime;1& d2+d\&prime;2& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3+d\&prime;3\end{array}\right],$ Promptly $\left[\begin{array}{cccc}A0& A1& A2& A3\\ B0& B1& B2& B3\\ C0& C1& C2& C3\\ D0& D1& D2& \mathrm{<figure-callout\; id="3"\; label="D"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">D</figure-callout>}3\end{array}\right].$

At last, in step S43, video decoder reads the reconstructed blocks matrix from the reconstructed image data buffer zone, and it outputs to the next stage of streamline by being about to.

In addition, present embodiment is not limited to 4 x, 4 block matrix, and according to above description, those skilled in the art should be able to expand to present embodiment the matrix of any dimension without what creative work still.

In supporting the hardware system of many video encoding and decoding standards, the prediction block matrix obtained various ways, for example interframe conversion and frame inner conversion.The prediction block matrix of interframe encode is to predict acquisition by the image of previous coding frame.The prediction block matrix of intraframe coding is to be predicted through the image of coding, decoding and reconstruction by present frame.The implementation of obtaining the prediction block matrix among the step S40 is also therefore different, is discussed below respectively:

Situation 1: the prediction block matrix is for to obtain by inter prediction

Variation between the frame of video may come from object of which movement, the automobile that for example goes at express speed, and variable object of which movement moves etc. as arm.For this class situation, the most general method is exactly in the past or seeks the prediction block matrix of current macro in certain following two field picture.

Its concrete steps are, coding side in reference frame (after coding and the transmission, the picture frame before or after the present frame) same 4 x, 4 sizes of search the match block matrix.Be about to current 4 x, 4 block matrix and compare, find out the grand fast matrix of optimum Match wherein with some or all 4 x, 4 block matrix in reference frame search zone (generally being to be the zone at center with the current macro position).Matching criterior commonly used is the residual block energy after calculating current block matrix and comparison block matrix subtract each other, and makes the minimum comparison block matrix of residual block energy be the prediction block matrix.Utilize motion vector to describe relative position between predicted macroblock and current macro, and it is transferred to decoding end.

In decoding end, promptly among the step S40, from elementary bit stream, decode motion vector, find the predicted macroblock position according to motion vector, fetch data according to the row sequential read of original image matrix then, and the data that read are write prediction block matrix buffering area line by line, institute obtains the prediction block matrix and is $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right].$

Situation 2: the prediction block matrix is for to obtain by inter prediction and interpolation arithmetic

In a lot of video encoding and decoding standards, need utilize location of pixels location prediction block matrix after the interpolation to improve the precision of motion compensation.For example, H.264 standard code, movement compensation process need be finished sub-pixel interpolation.For this kind situation, on the basis of situation 1, situation 2 also needs to increase the step of interpolation calculation.How to specify comprising below in conjunction with Fig. 5 and to obtain the prediction block matrix of exporting in proper order by the original image row under the situation of interpolation calculation.

At first, shown in step S50, determine the value of the reference block matrix of the pairing whole pixel value of prediction block matrix, establish this reference block matrix and be by motion vector $\left[\begin{array}{cccc}{a}_r\&prime;0& a_r\&prime;1& a_r\&prime;2& a_r\&prime;3\\ b_r\&prime;0& b_r\&prime;1& b_r\&prime;2& b_r\&prime;3\\ c_r\&prime;0& c_r\&prime;1& c_r\&prime;2& c_r\&prime;3\\ d_r\&prime;0& d_r\&prime;1& d_r\&prime;2& d_r\&prime;3\end{array}\right],$ [a wherein _r' 0, a _r' 1, a _r' 2, a _r' 3] be the line direction of original image.

Then, shown in step S51, according to line direction, i.e. [a _r' 0, a _r' 1, a _r' 2, a _r' 3, b _r' 0, b _r' 1, b _r' 2, b _r' 3 ..., d _r' 2, d _r' 3] order is done the half-pix row interpolation one by one to pixel in the current reference block matrix.Shown in step S52, finish according to line direction each pixel in the described current reference block matrix is done the half-pix row interpolation, obtain the block matrix after the interpolation $\left[\begin{array}{cccc}{a}_r\&prime;0& a_r\&prime;1& a_r\&prime;2& a_r\&prime;3\\ {\mathrm{<figure-callout\; id="1"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}}_{1/2}\&prime;0& {\mathrm{<figure-callout\; id="1"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}}_{1/2}\&prime;1& {\mathrm{<figure-callout\; id="1"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}}_{1/2}\&prime;2& {\mathrm{<figure-callout\; id="1"\; label="a"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">a</figure-callout>}}_{1/2}\&prime;3\\ b_r\&prime;0& b_r\&prime;1& b_r\&prime;2& b_r\&prime;3\\ {\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}}_{1/2}\&prime;0& {\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}}_{1/2}\&prime;1& {\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}}_{1/2}\&prime;2& {\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">b</figure-callout>}}_{1/2}\&prime;3\\ c_r\&prime;0& c_r\&prime;1& c_r\&prime;2& c_r\&prime;3\\ {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}}_{1/2}\&prime;0& {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}}_{1/2}\&prime;1& {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}}_{1/2}\&prime;2& {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">c</figure-callout>}}_{1/2}\&prime;3\\ d_r\&prime;0& d_r\&prime;1& d_r\&prime;2& d_r\&prime;3\\ {\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}}_{1/2}\&prime;0& {\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}}_{1/2}\&prime;1& {\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}}_{1/2}\&prime;2& {\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}}_{1/2}\&prime;3\end{array}\right],$ Wherein, 1/2 is half-pix point subscript.

Afterwards, shown in step S53, according to reference block matrix column direction, i.e. [a _r' 0, b _r' 0, c _r' 0, d _r' 0] order is done half-pix row interpolation to each pixel in the block matrix after the interpolation one by one, determines to wait to ask position a little to obtain the value of waiting to ask a little by motion vector afterwards.Shown in step S54, the value a little of waiting to ask that will obtain according to the row order of reference block matrix deposits the prediction block matrix in proper order in, obtains the prediction block matrix at last $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right].$ The corresponding reference block matrix column of the line direction of prediction matrix direction, the i.e. column direction of original image.

Below with still with 4 of 4 x, 1/4 interpolation determines that earlier the half-pix value on the vertical direction is an example, specifies implementation method in conjunction with Fig. 5, Fig. 6 and Fig. 7.With Fig. 6 is example, suppose that Z waits to ask 1/4 pixel, describe for convenient, four points setting whole pixel E, F, H, G and be 2 of one 2 x among 4 of current 4 x (for example, a ' 0, a ' 1, b ' 0, and b ' 1), and in the square area of waiting to ask 1/4 pixel Z to be in E, F, H, G formation, E, F, H, G are provided with in the direction of the clock, wherein E is the top left corner apex among 2 of 2 x, is initial point with E, and abscissa is along E-〉the F direction is provided with, the horizontal axis motion vector is mvx, and the vertical axis motion vector is mvy.E, E0, E1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress against the current, F, F0, F1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress downstream, in like manner, H, H0, H1 waits to ask near the downside of the 1/4 pixel Z square adjacent whole pixel that makes progress against the current, G, G0, G1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress downstream, in like manner, N1, N0, E and H, P0, the left side was contrary near P1 was respectively this pixel Z, along adjacent whole pixel on the vertical direction, O1, O0, F and G, Q0, the right side was contrary near Q1 was respectively this pixel Z, along adjacent whole pixel on the vertical direction.I, L, M, J are respectively the mid points of pixel EF, FG, GH, HE, i.e. half-pix point, and K is the mid point of J and L line, also is the half-pix point.

At first, in step S70, interpolation device reads in 6 adjacent whole pixels of whole pixel E, H one row, is followed successively by N1, N0, E, H, P0 and P1 from top to bottom, calculates the value of half-pix point J according to formula given in the standard.For example, H.264 standard code, for luminance component, ask the method for half-pix point as described below: utilize institute to ask about half-pix point (or up and down) nearest respectively 3 adjacent whole pixels to make weighted average, weight coefficient is respectively 1, and-5,20,20 ,-5 ,-1.As calculate the value of half-pix point J, then J=(N1-5*N0+20*E+20*H-5*P0+P1+16)/32 with above formula.In like manner, read in 6 adjacent whole pixels of whole pixel F, G one row, obtain the value of half-pix point L with same procedure.Shown in step S71, one by one each the whole pixel repeating step S70 in the reference block matrix is asked for the value of the half-pix point on the vertical direction according to the row order of prediction matrix.

Then, in step S72, interpolation device reads the value of whole pixel E1, E0, E, F, F0 and F1, and interpolation filter is according to the value of following formula calculating half-pix point I, I=(E1-5*E0+20E+20F-5*F0+F1+16)/32.In like manner, interpolation device reads the value of whole pixel H1, H0, H, G, G0 and G1, calculates the value of half-pix point M.Half-pix is put J1, J0, J, L, L0 and L1 send into interpolation device, wherein J1, J0, J, L, L0 and L1 obtain in step S70, in like manner, calculate the value of half-pix point K.

In step S73, determine to wait to ask the residing position of 1/4 pixel Z according to the motion vector value of the 4*4 piece of importing.It is the little square of four identical sizes on summit that 2 of current 2 x are divided into EIJK, IFLK, KLGM and JKMH, and each foursquare length of side all is a half-pix.Determine to wait to ask the residing little square of 1/4 pixel Z by horizontal axis motion vector mvx and vertical axis motion vector mvy.For example, can utilize the motion vector lowest bit to represent to wait to ask the accurate position of pixel on reference axis.

In step S74, ask the residing little foursquare summit of 1/4 pixel Z pixel value based on waiting, determine to wait to ask the value of 1/4 pixel Z.At first, determine to wait to ask the exact position of pixel Z by the motion vector value of 4*4 piece of input.If this pixel Z is positioned at little foursquare summit, then export known summit pixel value as waiting to ask pixel Z value.If this pixel Z is positioned on the little foursquare limit, two summits, summit of then exporting the limit, place ask weighted average to obtain waiting to ask pixel Z value as reference point to these two reference points; If this pixel Z is positioned at little foursquare central authorities, two summits of then exporting little square diagonal angle, place ask weighted average to obtain waiting to ask pixel Z value as reference point to these two reference points.The value of the 1/4 pixel Z that obtains is deposited in the relevant position of prediction block matrix.

Shown in step S75, press reference block matrix column order to each pixel value repeating step S72, S73 and S74 in the described reference block matrix, the value of waiting to ask pixel of all pixel correspondences in finishing to current reference block matrix, according to the row order storage one by one of reference block matrix, finished the prediction block matrix of arranging according to the original image column direction $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right]$ Obtain.

Situation 3: the prediction block matrix is for to obtain by infra-frame prediction

H.264 introducing intra prediction mode to utilize spatial coherence in the video encoding and decoding standard to improve compression efficiency.Below, be described further in situation 3, obtaining the predicted macroblock technical scheme of arranging in conjunction with specific embodiments by the original image column direction.The predictive mode of being mentioned in the present embodiment has corresponding predictive mode on-45 degree directions.

Briefly, intra prediction mode be exactly in spatial domain, utilize same number of frames previous change and rebuild after the reference block matrix information predict, obtain the prediction block matrix.This prediction is based on piece, and for luminance component (1uma), the size of piece can be selected between 16 * 16 and 4 * 4, and 16 * 16 have 4 kinds of predictive modes, and 4 * 4 have 9 kinds of predictive modes; For chromatic component (chroma), prediction is carried out whole 8 * 8, and 4 kinds of predictive modes are arranged.Here be that example specifies with 4 * 4 each predictive mode of Block Brightness component.

Fig. 8 (a) has shown the 4x4 luminance block that need predict, and E-M is the reference pixel value that can be used for predicting of current macro top and left.Value a ' 0 in the prediction block matrix, a ' 1, and a ' 2 ..., d ' 2, and d ' 3 can calculate according to reference pixel value E-M according to certain predictive mode.Selectable predictive mode is as shown in the table:

Pattern 0	Infer from reference pixel value E, F, G, the H of top
		Pattern
1	Infer from reference pixel value N, O, P, the Q on the left side
		Pattern 2	Infer with E, N, H, Q
Mode
	3	The lower-left and upper right between infer at 45 degree
Pattern
	4	45 degree are inferred in the bottom right
Pattern
	5	Infer at nearly 26.6 degree of vertical left (wide/high=1/2)
Pattern 6			Nearly 26.6 degree are inferred under level
	Mode
7		Infer at nearly 26.6 degree of vertical-right
	Pattern
8		Infer at level nearly 26.6 degree that make progress

As can be seen from the table, except the DC prediction, other the every kind all corresponding different prediction direction of predictive mode is shown in Fig. 8 b.By prediction direction is contrasted, we find that the prediction direction of some predictive modes is with-45 degree direction symmetries.For example, pattern 0 and pattern 1, pattern 5 and pattern 6, mode 7 and pattern 8.Generally we need to select a kind of each pixel to current macro to obtain predicted value earlier, change sequence of matrices again and obtain being listed as tactic prediction block matrix according to original image.Consider prediction direction, we are not difficult to find, if select to be with original predictive mode-predictive mode that miter angle is corresponding, only need to change the order of input pixel, just can directly obtain the prediction block matrix by the arrangement of original image column direction.As, be the situation of pattern 5 to original predictive mode, directly choose pattern 6 and predict, will change into by row input pixel by row input pixel, just can obtain the prediction block matrix of arranging by the original image column direction.Right for other predictive modes about-miter angle correspondence, same being suitable for.Below we just prove.

If the value that adopts certain point in the matrix that pattern 5 predictions obtain for certain particular matrix is for (x, y), the value that adopts identical point in the matrix that pattern 6 predictions obtain for same matrix is (x ', y ').According to standard H.264, under certain condition, the predictor formula of pattern 5 correspondences is P ₅(x, y)=(p[x-(y〉〉 1)-1 ,-1]+p[x-(y〉〉 1) ,-1]+1) 1,

Predictor formula for pattern 6 correspondences is

P ₆(x′，y′)＝(p[-1，y′-(x′>>1)-1]+p[-1，y′-(x′>>1)]+1)>>1

When x=y ' and y=x ' time,

P ₆(x′，y′)＝(p[-1，x-(y>>1)-1]+p[-1，x-(y>>1)]+1)>>1

P then ₆(x ', y ')=P ₅(y, x), the matrix that 6 predictions of employing pattern obtain is the transpose of a matrix that 5 predictions of employing pattern obtain, and x=y ' and y=x ' they are the two matrixes conditions of transposition each other.

It should be appreciated by those skilled in the art, the situation that is not limited in predictive mode 5 and predictive mode 6 is more than described, all are satisfied predictive mode about-miter angle correspondence to all setting up, the pattern 0 and the pattern 1 that comprise 4 x, 4 luminance components, pattern 5 and pattern 6, mode 7 and pattern 8, pattern 4 (about-the miter angle prediction, need not symmetrical predictive mode); The horizontal or vertical predictive mode of 16 * 16 luminance components; The horizontal or vertical predictive mode of 8 x, 8 chromatic components.

For the predictive mode of this class about-miter angle correspondence, the process of obtaining the prediction block matrix as shown in Figure 9.In step S90, video decoder is earlier by to the elementary bit stream decoding, the predictive mode type when obtaining coding, find then with its about-predictive mode that miter angle is corresponding, determine the predictive mode of this corresponding predictive mode for decoding.Selected predictive mode is a pattern 5 when supposing coding, determines that then the predictive mode of decoding is a pattern 6.In step S91, the order of the input reference pixel value of the predictive mode type defined when at first determining decoding, find one by one again and the value of each reference pixel about the-pixel that miter angle is corresponding, value with the corresponding pixel points that found replaces described reference pixel value at last, according to the reference pixel buffering area of the order input prediction unit of the predictive mode defined of decoding.In conjunction with Fig. 8 a as seen, if the order of when decoding reference pixel value input of setting of predictive mode is [M, E, F, G, H, N, O, P, Q], when then using with it about the corresponding predictive mode of-miter angle, the pixel of E, F, G, H point correspondence is respectively N, O, P, Q, therefore, the order of reference pixel value input should be [M, N, O, P, Q, E, F, G, H].In step S92, prediction unit is predicted based on the predictive mode of step S90 and the determined decoding of S91 and the reference pixel value of input, is obtained the prediction block matrix.Predictive mode when encoding as direct use, the matrix that directly obtains is $\left[\begin{array}{cccc}a\&prime;0& a\&prime;1& a\&prime;2& a\&prime;3\\ b\&prime;0& b\&prime;1& b\&prime;2& b\&prime;3\\ c\&prime;0& c\&prime;1& c\&prime;2& c\&prime;3\\ d\&prime;0& d\&prime;1& d\&prime;2& d\&prime;3\end{array}\right],$ The matrix that then adopts the technical scheme of present embodiment to obtain is $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right],$ The i.e. prediction block matrix of arranging according to the original image column direction.

For the piece of 4 x 4, prior art need be realized with two transposition unit, and this transposition unit must be realized with the higher register of cost.In technical scheme of the present invention, only need a transposition unit, saved system cost greatly.Simultaneously, also reduced delay as the motion compensation units level of system bottleneck.

Followingly second aspect of the present invention is described in detail with reference to Figure 10.Figure 10 is the block diagram at the video decoder that is used for recuperating original image data 1 shown in Fig. 4 of the embodiment according to the present invention.

This video decoder 1 comprises: reconstructing device 100, merging device 101, transposition device 102 and output device 103.Below describing with residual block and predicted macroblock all is that 4 x, 4 block matrix are example.

Suppose that the original image matrix of coding side input is $X=\left[\begin{array}{cccc}A0& A1& A2& A3\\ B0& B1& B2& B3\\ C0& C1& C2& C3\\ D0& D1& D2& \mathrm{<figure-callout\; id="3"\; label="D"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">D</figure-callout>}3\end{array}\right],$ The prediction block matrix is $P\left(X\right)=\left[\begin{array}{cccc}a\&prime;0& a\&prime;1& a\&prime;2& a\&prime;3\\ b\&prime;0& b\&prime;1& b\&prime;2& b\&prime;3\\ c\&prime;0& c\&prime;1& c\&prime;2& c\&prime;3\\ d\&prime;0& d\&prime;1& d\&prime;2& d\&prime;3\end{array}\right],$ Corresponding residual block matrix is $R\left(X\right)=\left[\begin{array}{cccc}a0& a1& a2& a3\\ b0& b1& b2& b3\\ c0& c1& c2& c3\\ d0& d1& d2& \mathrm{<figure-callout\; id="3"\; label="d"\; filenames="A200710173884E00332.png"\; state="\{\{state\}\}">d</figure-callout>}3\end{array}\right].$ Describe for simplifying, the direction that numeral changes in the present embodiment is the line direction of original image, and the direction that letter changes is the column direction of original image.

The signal of the residual block matrix that is generated by IDCT and the prediction block matrix of reconstruction arrives described merging transposition 101, and described merging device 101 merges processing from residual block matrix and prediction block matrix, to generate the merging value of block matrix.Described residual block matrix is obtained by reverse discrete cosine transform, and wherein the operation principle of reverse discrete cosine transform is as follows:

R(X)＝A ^TFA＝((FA) ^TA) ^T＝(R′(X)) ^T

Then R ' (X)=(FA) ^TA

Wherein, R (X) is the residual block matrix of 4 x 4, and F is the sample block matrix, and A is the transformation matrix of 4x4, and T is the transpose of a matrix factor.Can derive from formula, the residual block matrix that input merges device 101 is to arrange according to the column direction of original image.

For the component that guarantees prediction block matrix and residual block matrix relevant position can addition, input merges the prediction block matrix after the reconstruction of device 101 $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right]$ Also must be to arrange according to the column direction of original image, wherein said reconstruction can comprise interframe conversion or frame inner conversion.

Merge 101 pairs of residual block matrixes of device $\left[\begin{array}{cccc}a0& b0& c0& d0\\ a1& b1& c1& d1\\ a2& b2& c2& d2\\ a3& b3& c3& d3\end{array}\right]$ With the prediction block matrix after the reconstruction $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right]$ Data on the correspondence position merge.For example, the data on the relevant position are carried out addition, generate the merging value of block matrix, promptly $\left[\begin{array}{cccc}a0+a\&prime;0& b0+b\&prime;0& c0+c\&prime;0& d0+d\&prime;0\\ a1+a\&prime;1& b1+b\&prime;1& c1+c\&prime;1& d1+d\&prime;1\\ a2+a\&prime;2& b2+b\&prime;2& c2+c\&prime;2& d2+d\&prime;2\\ a3+a\&prime;3& b3+b\&prime;3& c3+c\&prime;3& d3+d\&prime;3\end{array}\right].$ This merging device 101 passes to described transposition device 102 with the merging value of the block matrix of its generation.

The merging value of the block matrix that 102 pairs of these merging devices 101 of described transposition device obtain is carried out transposition, to generate the reconstructed image data of current macro.Described transposition is that the position of the row and column of the merging value of block matrix is changed, and with first row of the merging value of block matrix, first row as restructuring matrix, the secondary series of the merging value of block matrix is as second row of restructuring matrix, and the like.The reconstructed blocks matrix that is generated in this example is $\left[\begin{array}{cccc}a0+a\&prime;0& a1+a\&prime;1& a2+a\&prime;2& a3+a\&prime;3\\ b0+b\&prime;0& b1+b\&prime;1& b2+b\&prime;2& b3+b\&prime;3\\ c0+c\&prime;0& c1+c\&prime;1& c2+c\&prime;2& c3+c\&prime;3\\ d0+d\&prime;0& d1+d\&prime;1& d2+d\&prime;2& d3+d\&prime;3\end{array}\right],$ Promptly $\left[\begin{array}{cccc}A0& A1& A2& A3\\ B0& B1& B2& B3\\ C0& C1& C2& C3\\ D0& D1& D2& D3\end{array}\right].$ Transposition device 102 passes to described output device 103 with the reconstructed blocks matrix of its generation.

Described output device 103 is according to the capable restructuring matrix of exporting described current macro in proper order of original image.The raw image data that the next stage of streamline obtains rebuilding.So far, decoding device 1 has been finished the recovery to raw image data.

In supporting the hardware system of many video encoding and decoding standards, the prediction block matrix obtained various ways, for example interframe conversion and frame inner conversion.The prediction block matrix of interframe encode is to predict acquisition by the image of previous coding frame.The prediction block matrix of intraframe coding is to be predicted through the image of coding, decoding and reconstruction by present frame.Therefore, the execution mode of the prediction block matrix that generation merging device 101 is obtained is also different, is discussed below respectively:

Situation 1: the prediction block matrix is for to obtain by inter prediction

In decoding end, the elementary bit stream that comprises the relevant information of prediction block matrix arrives reconstructing device 100,100 pairs of elementary bit stream of described reconstructing device are decoded, row sequential read according to original image matrix is fetched data, and the data that read are write prediction block matrix buffering area line by line, institute obtains the prediction block matrix and is $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right].$ The prediction block matrix that described reconstructing device 100 will generate is delivered to and merges device 101.

Situation 2: the prediction block matrix is for to obtain by inter prediction and interpolation arithmetic

In decoding end, the elementary bit stream that comprises the relevant information of prediction block matrix arrives reconstructing device 100, described reconstructing device 100 finds the matching area of described current macro by motion vector in the reference image frame of preserving, and according to the pixel value in the described matching area by described relatively original image earlier line by line the back carry out linear interpolation by the order of row.Its detailed process is as follows:

Reconstructing device 100 is determined the value of the reference block matrix of the pairing whole pixel value of prediction block matrix by motion vector, establishes this reference block matrix and is $\left[\begin{array}{cccc}{a}_r\&prime;0& a_r\&prime;1& a_r\&prime;2& a_r\&prime;3\\ b_r\&prime;0& b_r\&prime;1& b_r\&prime;2& b_r\&prime;3\\ c_r\&prime;0& c_r\&prime;1& c_r\&prime;2& c_r\&prime;3\\ d_r\&prime;0& d_r\&prime;1& d_r\&prime;2& d_r\&prime;3\end{array}\right],$ [a wherein _r' 0, a _r' 1, a _r' 2, a _r' 3] be the line direction of original image.Then, reconstructing device 100 is according to line direction, i.e. [a _r' 0, a _r' 1, a _r' 2, a _r' 3, b _r' 0, b _r' 1, b _r' 2, b _r' 3 ..., d _r' 2, d _r' 3] order is done the half-pix row interpolation one by one to each pixel in the current reference block matrix, obtains the block matrix after the interpolation $\left[\begin{array}{cccc}{a}_r\&prime;0& a_r\&prime;1& a_r\&prime;2& a_r\&prime;3\\ a_{1/2}\&prime;0& a_{1/2}\&prime;1& a_{1/2}\&prime;2& a_{1/2}\&prime;3\\ b_r\&prime;0& b_r\&prime;1& b_r\&prime;2& b_r\&prime;3\\ b_{1/2}\&prime;0& b_{1/2}\&prime;1& b_{1/2}\&prime;2& b_{1/2}\&prime;3\\ c_r\&prime;0& c_r\&prime;1& c_r\&prime;2& c_r\&prime;3\\ c_{1/2}\&prime;0& c_{1/2}\&prime;1& c_{1/2}\&prime;2& c_{1/2}\&prime;3\\ d_r\&prime;0& d_r\&prime;1& d_r\&prime;2& d_r\&prime;3\\ d_{1/2}\&prime;0& d_{1/2}\&prime;1& d_{1/2}\&prime;2& d_{1/2}\&prime;3\end{array}\right],$ Wherein, 1/2 is half-pix point subscript.Afterwards, reconstructing device 100 is according to reference block matrix column direction, i.e. [a _r' 0, b _r' 0, c _r' 0, d _r' 0] order is done half-pix row interpolation to each pixel in the block matrix after the interpolation one by one, determine to wait to ask position a little to obtain the value of waiting to ask a little by motion vector afterwards, the value a little of waiting to ask that to obtain according to the row order of reference block matrix deposits the prediction block matrix in proper order at last, obtains the prediction block matrix $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right].$ The corresponding reference block matrix column of the line direction of prediction matrix direction, the i.e. column direction of original image.The prediction block matrix that described reconstructing device 100 will generate is delivered to and merges device 101.

Below with still with 4 of 4 x, 1/4 interpolation determines that earlier the half-pix value on the vertical direction is an example, specifies implementation method in conjunction with Fig. 6.With Fig. 6 is example, suppose that Z waits to ask 1/4 pixel, describe for convenient, four points setting whole pixel E, F, H, G and be 2 of one 2 x among 4 of current 4 x (for example, a ' 0, a ' 1, b ' 0, and b ' 1), and in the square area of waiting to ask 1/4 pixel Z to be in E, F, H, G formation, E, F, H, G are provided with in the direction of the clock, wherein E is the top left corner apex among 2 of 2 x, is initial point with E, and abscissa is along E-〉the F direction is provided with, the horizontal axis motion vector is mvx, and the vertical axis motion vector is mvy.E, E0, E1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress against the current, F, F0, F1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress downstream, in like manner, H, H0, H1 waits to ask near the downside of the 1/4 pixel Z square adjacent whole pixel that makes progress against the current, G, G0, G1 waits to ask near the upside of the 1/4 pixel Z square adjacent whole pixel that makes progress downstream, in like manner, N1, N0, E and H, P0, the left side was contrary near P1 was respectively this pixel Z, along adjacent whole pixel on the vertical direction, O1, O0, F and G, Q0, the right side was contrary near Q1 was respectively this pixel Z, along adjacent whole pixel on the vertical direction.I, L, M, J are respectively the mid points of pixel EF, FG, GH, HE, i.e. half-pix point, and K is the mid point of J and L line, also is the half-pix point.

Reconstructing device 100 reads in 6 adjacent whole pixels of whole pixel E, H one row, is followed successively by N1, N0, E, H, P0 and P1 from top to bottom, calculates the value of half-pix point J according to formula given in the standard.For example, H.264 standard code, for luminance component, ask the method for half-pix point as described below: utilize institute to ask about half-pix point (or up and down) nearest respectively 3 adjacent whole pixels to make weighted average, weight coefficient is respectively 1, and-5,20,20 ,-5 ,-1.As calculate the value of half-pix point J, then J=(N1-5*N0+20*E+20*H-5*P0+P1+16)/32 with above formula.In like manner, read in 6 adjacent whole pixels of whole pixel F, G one row, obtain the value of half-pix point L with same procedure.One by one each the whole pixel repeating step S70 in the reference block matrix is asked for the value of the half-pix point on the vertical direction according to the row order of prediction matrix.

Then, reconstructing device 100 reads the value of whole pixel E1, E0, E, F, F0 and F1, and interpolation filter is according to the value of following formula calculating half-pix point I, I=(E1-5*E0+20E+20F-5*F0+F1+16)/32.In like manner, interpolation device reads the value of whole pixel H1, H0, H, G, G0 and G1, calculates the value of half-pix point M.Half-pix is put J1, J0, J, L, L0 and L1 send into interpolation device, wherein J1, J0, J, L, L0 and L1 obtain in step S70, in like manner, calculate the value of half-pix point K.

Afterwards, reconstructing device 100 determines to wait to ask the residing position of 1/4 pixel Z according to the motion vector value of the 4*4 piece of input.It is the little square of four identical sizes on summit that 2 of current 2 x are divided into EIJK, IFLK, KLGM and JKMH, and each foursquare length of side all is a half-pix.Determine to wait to ask the residing little square of 1/4 pixel Z by horizontal axis motion vector mvx and vertical axis motion vector mvy.For example, can utilize the motion vector lowest bit to represent to wait to ask the accurate position of pixel on reference axis.

Then, reconstructing device 100 is asked the residing little foursquare summit of 1/4 pixel Z pixel value based on waiting, determines to wait to ask the value of 1/4 pixel Z.At first, determine to wait to ask the exact position of pixel Z by the motion vector value of 4*4 piece of input.If this pixel Z is positioned at little foursquare summit, then export known summit pixel value as waiting to ask pixel Z value.If this pixel Z is positioned on the little foursquare limit, two summits, summit of then exporting the limit, place ask weighted average to obtain waiting to ask pixel Z value as reference point to these two reference points; If this pixel Z is positioned at little foursquare central authorities, two summits of then exporting little square diagonal angle, place ask weighted average to obtain waiting to ask pixel Z value as reference point to these two reference points.The value of the 1/4 pixel Z that obtains is deposited in the relevant position of prediction block matrix.

At last, reconstructing device 100 repeats to ask relevant half-pix point by reference block matrix column order to each pixel value in the described reference block matrix, obtain the step of waiting to ask pixel Z value based on described half-pix point, the value of waiting to ask pixel of all pixel correspondences in finishing to current reference block matrix, according to the row order storage one by one of reference block matrix, finished the prediction block matrix of arranging according to the original image column direction $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right]$ Obtain.The prediction block matrix that described reconstructing device 100 will generate is delivered to and merges device 101.

Situation 3: the prediction block matrix is for to obtain by infra-frame prediction

H.264 introducing intra prediction mode to utilize spatial coherence in the video encoding and decoding standard to improve compression efficiency.Below, be described further in situation 3, obtaining the predicted macroblock technical scheme of arranging in conjunction with specific embodiments by the original image column direction.The predictive mode of being mentioned in the present embodiment has corresponding predictive mode on-45 degree directions, be that example is specifically described with 4 * 4 each predictive mode of Block Brightness component.

Fig. 8 (a) has shown the 4x4 luminance block that need predict, and E-M is the reference pixel value that can be used for predicting of current macro top and left.Value a ' 0 in the prediction block matrix, a ' 1, and a ' 2 ..., d ' 2, and d ' 3 can calculate according to reference pixel value E-M according to certain predictive mode.Wherein, pattern 0 and pattern 1, pattern 5 and pattern 6, the prediction direction of mode 7 and pattern 8 is about-miter angle correspondence.We are not difficult to find, if select to be with original predictive mode-predictive mode that miter angle is corresponding, only need to change the order of input pixel, just can directly obtain the prediction block matrix by the arrangement of original image column direction.Wherein, described and original predictive mode is-predictive mode that miter angle is corresponding, comprises the pattern 0 and the pattern 1 of 4 x, 4 luminance components, pattern 5 and pattern 6, and mode 7 and pattern 8, pattern 4 (about-the miter angle prediction, need not symmetrical predictive mode); The horizontal or vertical predictive mode of 16 * 16 luminance components; The horizontal or vertical predictive mode of 8 x, 8 chromatic components.

For the predictive mode of this class about-miter angle correspondence, reconstructing device 100 is earlier by decoding to elementary bit stream, obtain the predictive mode type in when coding, find then with its about-predictive mode that miter angle is corresponding, determine the predictive mode of this corresponding predictive mode for decoding.Selected predictive mode is a pattern 5 when supposing coding, determines that then the predictive mode of decoding is a pattern 6.The order of the input reference pixel value of the predictive mode type defined when reconstructing device 101 is at first determined decoding, find one by one again and the value of each reference pixel about the-pixel that miter angle is corresponding, value with the corresponding pixel points that found replaces described reference pixel value at last, according to the reference pixel buffering area of the order input prediction unit of the predictive mode defined of decoding.In conjunction with Fig. 8 a as seen, if the order of when decoding reference pixel value input of setting of predictive mode is [M, E, F, G, H, N, O, P, Q], when then using with it about the corresponding predictive mode of-miter angle, the pixel of E, F, G, H point correspondence is respectively N, O, P, Q, therefore, the order of reference pixel value input should be [M, N, O, P, Q, E, F, G, H].Reconstructing device 101 is predicted based on the predictive mode of determined decoding and the reference pixel value of input, is obtained the prediction block matrix.Predictive mode when encoding as direct use, the matrix that directly obtains is $\left[\begin{array}{cccc}a\&prime;0& a\&prime;1& a\&prime;2& a\&prime;3\\ b\&prime;0& b\&prime;1& b\&prime;2& b\&prime;3\\ c\&prime;0& c\&prime;1& c\&prime;2& c\&prime;3\\ d\&prime;0& d\&prime;1& d\&prime;2& d\&prime;3\end{array}\right],$ The matrix that then adopts the technical scheme of present embodiment to obtain is $\left[\begin{array}{cccc}a\&prime;0& b\&prime;0& c\&prime;0& d\&prime;0\\ a\&prime;1& b\&prime;1& c\&prime;1& d\&prime;1\\ a\&prime;2& b\&prime;2& c\&prime;2& d\&prime;2\\ a\&prime;3& b\&prime;3& c\&prime;3& d\&prime;3\end{array}\right],$ The i.e. prediction block matrix of arranging according to the original image column direction.The prediction block matrix that described reconstructing device 100 will generate is delivered to and merges device 101.

For the piece of 4x4, prior art need be realized with two transposition unit, and this transposition unit must be realized with the higher register of cost.In technical scheme of the present invention, only need a transposition unit, saved system cost greatly.Simultaneously, also reduced delay as the motion compensation units level of system bottleneck.

More than specific embodiments of the invention are described.It will be appreciated that the present invention is not limited to above-mentioned specific implementations, those skilled in the art can make various distortion or modification within the scope of the appended claims.

Claims (12)

1. method that is used for recuperating original image data in Video Decoder wherein, may further comprise the steps:

B. the decoded residual block matrix that reverse discrete cosine transform is generated and the prediction block matrix addition of reconstruction is to generate the merging value of block matrix;

C. the merging value of described block matrix is carried out transposition, to generate the reconstructed image data of current macro;

D. export the reconstructed image data of described current macro by the row order of original image.

2. method according to claim 1 is characterized in that, also comprises before the described step b:

A. rebuild described prediction block matrix in proper order by the row of described original image.

3. method according to claim 2 is characterized in that, described step a also comprises:

A '. in the reference image frame of preserving, find the matching area of described current macro by motion vector, and rebuild described prediction block matrix according to described matching area.

4. method according to claim 3 is characterized in that, the level or the vertical component of described motion vector comprise fractional value, and described step a ' also comprises:

-pixel value in the described matching area is carried out linear interpolation by the first back line by line of described relatively original image by the order of row.

5. method according to claim 2 is characterized in that, described step a also comprises:

A ". based on preassigned pattern, predict according to the pixel of macro block formerly adjacent with described current macro, rebuild described prediction block matrix, the direction of the prediction of the direction of the prediction of wherein said preassigned pattern and the predictive mode of coding is with respect to-miter angle symmetry.

6. method according to claim 5 is characterized in that, one of the predictive mode 0,1,4,5,6,7,8 of the 4*4 macro block that described line output predictive mode is a H.264 standard or horizontal or vertical predictive mode of 16*16 macro block.

7. video decoder that is used for recuperating original image data comprises:

Merge device, be used for the decoded residual block matrix of reverse discrete cosine transform generation and the prediction block matrix addition of reconstruction, to generate the merging value of block matrix;

The transposition device is used for the merging value of described block matrix is carried out transposition, to generate the reconstructed image data of current macro;

Output device is used for the capable reconstructed image data of exporting described current macro in proper order by original image.

8. video decoder according to claim 7 is characterized in that, also comprises:

Reconstructing device is used for rebuilding described prediction block matrix in proper order by the row of described original image.

9. video decoder according to claim 8, it is characterized in that, described reconstructing device also is used for, and finds the matching area of described current macro by motion vector in the reference image frame of preserving, and rebuilds described prediction block matrix according to described matching area.

10. video decoder according to claim 9, it is characterized in that, when the level of described motion vector or vertical component composition comprise fractional value, described reconstructing device also is used for, and the pixel value in the described matching area is carried out linear interpolation by the first back line by line of described relatively original image by the order of row.

11. video decoder according to claim 8, it is characterized in that, described reconstructing device also is used for, predict according to the formerly pixel of macro block adjacent based on preassigned pattern with described current macro, rebuild described prediction block matrix, the direction of the prediction of the direction of the prediction of wherein said preassigned pattern and the predictive mode of coding is with respect to-miter angle symmetry.

12. video decoder according to claim 11 is characterized in that, one of the predictive mode 0,1,4,5,6,7,8 of the 4*4 macro block that described line output predictive mode is a H.264 standard or horizontal or vertical predictive mode of 16*16 macro block.

Images