JPH1098695A - Image information converter and its device and product sum arithmetic unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To select the number of taps of an estimating equation to perform an optimal converting processing in accordance with characteristics of am image in converting SD (a video signal of an NTSC method) image into an HD (the video signal of a high vision) image. SOLUTION: Ratio DRratio between dynamic range DRall of all SD data segmented by an area segmenting circuit 2 and a part of the dynamic range DRpartial is calculated in a tap selecting parameter calculating circuit 3. A class code class is generated from a space class and a moving class mv-class in a class code generating circuit 8 and coefficient data is read out of ROM tables 9, 10 by regarding the class as an address. HD data is generated by a linear estimating equation consisting of twenty-five pixels in an estimating operation circuit 13 and the HD data is generated by the linear estimating equation consisting of fifteen pixels in an estimating operation circuit 14. The DRratio is compared with a threshold value in a switch circuit 15 and the HD data is selected and outputted in accordance with the comparison result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information converter suitable for use in, for example, a television receiver or a video tape recorder, and more particularly, to a method for converting image information of a normal resolution supplied from outside into a high-resolution image. The present invention relates to an image information conversion device and method for converting image data into an image and outputting the converted image information, and a product-sum operation device.

[0002]

2. Description of the Related Art Today, with the rise of audio-visual orientation, it has been desired to develop a television receiver capable of obtaining a higher-resolution image. In response to this demand, a so-called Hi-Vision has been developed. . In this high-definition television system, the number of scanning lines specified in the so-called NTSC system is 525, which is twice or more, that is, 1125 lines. And has become a wide-angle screen. For this reason, it is possible to obtain a high-resolution and realistic screen.

[0003] Here, although it is a Hi-Vision having such excellent characteristics, an image cannot be displayed even if an NTSC video signal is supplied as it is. This is because the standards differ between the NTSC system and the high-vision system as described above. Therefore, when an image corresponding to an NTSC video signal is to be displayed in Hi-Vision, the rate of the video signal has been conventionally converted using, for example, an image information conversion device as shown in FIG.

In FIG. 15, the conventional image information conversion apparatus described above uses NTS supplied through an input terminal 100.
It is composed of a horizontal interpolation filter 101 for performing a horizontal interpolation process on a video signal (SD data) of the C system and a vertical interpolation filter 102 for performing a vertical interpolation process on a video signal on which a horizontal interpolation process has been performed. ing.

More specifically, the horizontal interpolation filter 101
It has a configuration as shown in FIG. The example of FIG.
For example, the horizontal interpolation filter 101 is constituted by a cascade connection type FIR filter. In FIG. 16, 1
Reference numeral 10 denotes an input terminal to which SD data is supplied, and 11
1 ₀ - 111 _m is a multiplier for multiplying the filter coefficient alpha ₀ to? _M, respectively SD data. 112 _{0 to} 112
_m-1 is an adder, and 113 _{1 to} 113
_m is a delay element of time T (T: 1 sampling period). At the output terminal 114, output data obtained by horizontal interpolation is obtained. This output data is used as the vertical interpolation filter 102
Supplied to

The vertical interpolation filter 102 has the same configuration as the horizontal interpolation filter 101, and performs vertical pixel interpolation on a video signal on which horizontal interpolation processing has been performed. Thus, vertical pixel interpolation is performed on the NTSC video signal. The Hi-Vision video signal (HD data) thus converted is supplied to the Hi-Vision receiver. Thus, an image corresponding to the NTSC video signal can be displayed on the high-vision receiver.

However, the above-mentioned conventional image information conversion apparatus merely performs horizontal and vertical interpolation on the basis of an NTSC video signal. It was no different from the video signal. In particular, when a normal moving image is to be converted, it is common to perform vertical interpolation by in-field processing. In this case, since the inter-field correlation of the image is not used, in the image still unit, There is a disadvantage that the resolution is deteriorated rather than the video signal of the NTSC system due to the conversion loss.

On the other hand, the applicant filed Japanese Patent Application No. Hei 6-205.
In the image signal conversion device of No. 934, storage means for performing class division according to a three-dimensional (spatio-temporal) distribution of an image signal level as an input signal and storing a prediction coefficient value previously acquired by learning for each class is provided. And output an optimum estimated value by a calculation based on a prediction formula.

This technique is used to create HD (High Definition) pixels when SD (High Definition) pixels are created.
(Standerd Definition) Class division is performed using pixel data, and a prediction coefficient value is obtained for each class by learning, so that an intra-frame correlation is used in an image stationary part and an intra-field correlation is used in a moving part. Thus, an HD pixel value closer to the true value is obtained.

For example, when the purpose is to create HD pixels y _{1 to} y ₄ as shown in FIGS.
, The SD pixels m _{1 to} m ₅ and the SD pixels n _{1 to} n ₅
By calculating the average value of the differences between the frames of the pixels at the same spatial position, and performing threshold processing on the average value, the classification is performed mainly on the expression of the degree of motion.

At the same time, as shown in FIG. 5, SD pixels k _{1 to} k ₅ are connected to an ADRC (Adaptive Dynamic Range Code).
ng) By processing, class classification is performed with a small number of bits mainly for the purpose of representing waveforms in space. For each of the classes determined by the above-described two types of class classification, a linear linear expression is obtained using SD pixels x _{1 to} x ₂₅ as shown in FIG. 7 to obtain prediction coefficient values by learning. In this method, class conversion representing mainly the degree of movement and class classification representing mainly waveforms in space are individually performed in a form suitable for each, so that a high conversion performance can be obtained with a relatively small number of classes. There is a feature that is.

[0012] estimation calculation of HD pixel y is performed in the formula (1) as follows using the prediction coefficients w _n obtained by the above procedure.

Y = w ₁ x ₁ + w ₂ x ₂ +... + W _n x _n (1) In this example, n = 9.

As described above, the prediction coefficient value for estimating the HD data corresponding to the SD data is previously obtained for each class by learning, and stored in the ROM table.
By outputting the input SD data and the prediction coefficient value read from the ROM table, the input SD data is output.
Unlike the data simply interpolated, the actual HD
There is a feature that data closer to data can be output.

FIG. 17 shows this image information conversion apparatus. SD data is supplied from an input terminal 121, and the SD data is supplied to region cutout circuits 122, 124 and 128. The area extraction circuit 122 extracts SD data that is the basis of the class generated by the ADRC circuit 123. The area cutout circuit 124 extracts SD data that is the basis of the motion class generated by the motion class determination circuit 125. The class code generation circuit 126 generates a class code from the class from the ADRC circuit 123 and the motion class from the motion class determination circuit 125.

The input SD data is delayed for a predetermined time, and the area extracting circuit 128
The SD data to be operated at 29 is extracted. RO
In the M table 127, a prediction coefficient value corresponding to the supplied class code is read. In the estimation operation circuit 129, the SD data from the area extraction circuit 128 and the ROM
An estimation operation is performed using the class code from the table 127 to generate HD data. The generated HD data is output from the output terminal 130.

Incidentally, learning in the image information conversion apparatus has been performed by a configuration as shown in FIG. For example, HD data obtained as an imaging output of an HD video camera is input from an input terminal 131, and the HD data is supplied to a vertical thinning filter 132 and a normal equation adding circuit 141. In the vertical thinning filter 132, the frequency of the vertical scanning line in the field is 1 /
The thinning process is performed to reduce the HD data in the horizontal direction in the field to 1/2 in the horizontal thinning filter 133 to obtain SD data. This SD data is stored in the area extracting circuit 1
34, 136 and a delay circuit (DL) 139.

In the area extracting circuit 134, the SD data which is the basis of the class generated by the ADRC circuit 135 is extracted. The area cutout circuit 136 extracts the SD data that is the basis of the motion class generated by the motion class determination circuit 137. The class code generation circuit 138 generates a class code from the class from the ADRC circuit 135 and the motion class from the motion class determination circuit 137. The SD data supplied via the delay circuit 139 is extracted by the area extraction circuit 140 by the normal equation addition circuit 141 to extract the SD data. In the normal equation adding circuit 141, for example, normal equations for one frame are added by SD data and HD data for each class. Prediction coefficient determination circuit 14
In 2, a prediction coefficient value is determined for each class from the normal equation, and the determined prediction coefficient value is stored in the memory 1 for each class.
43.

That is, the input HD data is subjected to thinning processing by a vertical thinning filter 132 so that the vertical frequency in the field is reduced to １ ／, and furthermore, the horizontal thinning filter 133 is used to reduce the horizontal frequency of the HD data. Is decimated so that becomes 1/2, and SD
By obtaining data and learning the relationship between the SD data and the input HD data, a prediction coefficient value has been obtained.

By the way, S which is the object of the actual signal conversion is
D data is not always obtained by thinning out HD data as in learning. For example, an image sensor (HD data) of an HD video camera may be converted. Typically,
The output signal of the television camera decreases as the spatial frequency component increases, due to the effects of the resolution characteristics of the lens, the aperture characteristics of the electron beam during imaging, and the like. In addition to compensating for this, the contour corrector is designed to enhance the outline of the video signal and transmit it in anticipation of the lowering of the spatial frequency characteristic in the display device in order to produce a sharp image. Also called.

The contour corrector generally includes many nonlinear processing such as coring and nonlinear processing on a contour signal. Coring is a process in which most of the small contour signals are regarded as noise components, and the process of suppressing them is a non-linear process for contour signals. The gain of the small-amplitude correction signal is large and the gain of the large-amplitude correction signal is large. Is generally reduced. In addition, the gain of the correction signal is changed according to the brightness of the image.
The correction signal may be controlled in consideration of the SC encoding.

[0022]

By the way, in the image signal conversion device proposed above, as described above,
Learning has been performed between an image signal of HD data on which contour correction has been performed and an image signal of SD data created by down-converting the image signal using an ideal filter. Accordingly, the obtained SD data is SD data obtained by down-converting the image signal of the HD data on which the contour correction has been performed in the HD video camera.
In particular, the characteristics of the data near the contour were different from the SD data captured by the SD video camera.

Therefore, in the image signal conversion device,
The ideal conversion was realized for the up-conversion of SD data obtained by down-converting the HD data, but the up-conversion of SD data captured by an SD video camera was not always ideal conversion, especially near the contour. There was a drawback that should not be. Especially when there is a sharp edge adjacent to a flat part where there is little change in luminance, the presence of the edge may cause considerable remarkable ringing on the flat part, which causes a large visual deterioration. was there.

It should be noted that not only enhancers but also HD
The difference in signal characteristics between the SD data obtained by down-converting the data and the SD data that is actually going to change causes image degradation after conversion. It is conceivable to reduce the number of taps in the estimation calculation in order to prevent the image quality deterioration described above. However, this method is not preferable because it lowers the accuracy of signal conversion. If the number of taps is increased in order to increase the accuracy of the signal conversion, the above-described image quality deterioration is likely to occur.

Accordingly, it is an object of the present invention to provide an image information conversion apparatus and method, and a product-sum operation apparatus, which can increase the precision of signal conversion and prevent image quality deterioration.

Further, another object of the present invention is to increase the accuracy of signal conversion while reducing hardware,
Moreover, it is an object of the present invention to provide an image information conversion device and method and a product-sum operation device capable of preventing image quality deterioration.

[0027]

According to a first aspect of the present invention, there is provided an image information converting apparatus for converting a first digital image signal into a second digital image signal having a larger number of pixels. A pixel extracting means for extracting pixel data at a predetermined position from one digital image signal, and a level distribution pattern of the pixel data extracted by the pixel extracting means are detected, and based on the pattern, the pixel data to be estimated belongs. Class determination means for determining a class and outputting class information, and coefficient data of an estimation formula as information for converting the first digital image signal into a second digital image signal are stored for each class. A coefficient data storage means for outputting coefficient data according to the class information from the class determination means; Image signal generation for generating a predicted value of a second digital image signal using an estimated expression formed by a linear combination of pixel data of the selected first tap or a second tap longer than the first tap and coefficient data Means for switching the processing by the first and second taps.

According to a tenth aspect of the present invention, there is provided an image information conversion method for converting a first digital image signal into a second digital image signal having a larger number of pixels. A step of cutting out pixel data at a predetermined position from the signal, detecting a pattern of a level distribution of the pixel data cut out by the pixel cutout means, determining a class to which the pixel data to be estimated belongs based on the pattern, and Outputting information, and coefficient data of an estimation formula, which is information for converting the first digital image signal into a second digital image signal, is stored for each class. Outputting the coefficient data according to the following equation: and the pixel data of the second digital image signal to be estimated Determining whether there is an edge in the vicinity of the flat portion, and pixel data of a first tap cut out from the first digital image signal or a second tap longer than the first tap And generating a predicted value of the second digital image signal using an estimation expression formed by a linear combination with coefficient data,
Switching the processing by the first and second taps.

Further, the invention according to claim 11 is a product-sum operation device for performing a product-sum operation of pixel data of a plurality of taps of an input digital image signal and a plurality of coefficient data stored in a multiplier memory. In a multiply-accumulation device in which M and N longer than M are set as a plurality of taps,
An N tap register for outputting tap pixel data;
An M tap register for outputting tap pixel data;
Shrinking means for shrinking taps to a multiplicand of L taps shorter than N taps, selecting means for selecting whether to perform product-sum operation using L taps or M taps, and L tap or M tap selected by the selecting means And a means for executing a sum-of-products operation using the selected L tap or M tap and the coefficient data.

As described above, the present invention detects a pattern of the level distribution of SD pixels located near an HD pixel to be created from an input SD signal and, based on the detected pattern, detects an image of the area. The class to which the information belongs is determined, and the class detection information is output. In the coefficient data storage means, coefficient data of a linear estimation expression that is information for converting image information supplied from the outside into image information having a higher resolution than this image information is stored for each class,
This coefficient data is output according to the class detection information. Then, the image information conversion unit converts the image information into image information having a higher resolution than the image information supplied from the coefficient data storage unit. At this time, in addition to the processing by the tap having the normal size, there is provided a device that performs the processing with a small tap. When processing an image of a signal source having a characteristic different from that of the learning target, processing of an edge portion adjacent to a flat portion, which is likely to deteriorate, is performed by processing with a small tap,
The conversion performance for an image of a signal source having a characteristic different from that of a learning target is greatly improved.

[0031]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image signal conversion device according to the present invention will be described below in detail with reference to FIG. FIG. 1 shows a schematic configuration of this embodiment, that is, the signal processing of the image signal conversion device. For example, a so-called NTSC video is digitized and supplied as SD data as an image signal supplied from the outside from an input terminal denoted by reference numeral 1.

In this embodiment, the positional relationship between SD pixels and HD pixels to be created is as shown in FIGS. FIG. 2 shows SD pixels of the current field, SD pixels of the previous field, HD pixels of the current field, and HD pixels of the previous field in the horizontal and vertical directions. FIG. 3 shows SD pixels and HD pixels in the time direction and the vertical direction. As described above, the HD pixel to be created has SD when viewed in the same field.
HD pixels y ₁ , y ₂ and SD existing close to the pixel
There are two types of HD pixels y ₃ and y ₄ existing at positions far from the pixels. Thereafter, the HD existing at a position close to the SD pixel
The mode for estimating pixels is referred to as mode 1, and the mode for estimating HD pixels located far from the SD pixels is referred to as mode 2.

The area extracting circuit 2 extracts pixels necessary for calculating tap selection parameters from the SD data supplied from the input terminal 1. In this example, cut out HD pixel y ₁ ~y 17 amino located ₄ fields and the same field SD pixels t ₁ ~t ₁₇ should create, for example, as shown in FIG. The SD data extracted by the area cutout circuit 2 is output to a tap selection parameter calculation circuit 3
Supplied to

The tap selection parameter calculation circuit 3 is a circuit for calculating parameters necessary for switching between long tap processing and short tap processing, which will be described later.
This parameter indicates whether or not the condition that the HD data to be estimated is located in a flat portion where there is almost no change in luminance and an edge exists near the flat portion is satisfied. Specifically, the dynamic range DR _all which is a difference between the maximum value and the minimum value of the SD data (t _{1 to} t ₁₇ ) in the same field supplied from the region cutout circuit 2
And the dynamic range DR _partial of a part of the SD data (t _{1 to} t ₉ ) of the SD data supplied from the region cutout circuit 2 and calculates a ratio DR _ratio (= DR
_all / DR _partial ) is supplied to a switching circuit 15 for switching between long tap / short tap processing.

Here, the SD data t _{1 to} t ₁₇ are pixels of the same field as the HD data to be estimated, among the long tap SD data x _{1 to} x ₂₅ described later,
Some of the SD data t _{1 to} t ₉ are the HDs to be estimated from short tap SD data x _{1 to} x ₁₅ described later.
These are pixels in the same field as the data.

In general, conversion from SD data to HD data using long taps tends to have better conversion performance. However, if there is a steep edge adjacent to a flat part where there is little change in brightness, the conversion processing using a similarly long tap will cause a considerably noticeable linking to the flat part due to the presence of the edge. May appear after a change. In order to prevent this image quality deterioration, the length of the tap for performing the conversion process is switched according to the dynamic range ratio DR _ratio . Alternatively, it may be determined whether or not the pixel data to be estimated satisfies the condition that the pixel data to be estimated is located in the flat portion and the edge exists near the flat portion, using a parameter other than the dynamic range.

On the other hand, the area extracting circuit 4 classifies the SD image signal supplied from the input terminal 1 into a class mainly for representing a waveform in a space (hereinafter, referred to as a space class).
Cut out the pixels required for. In this embodiment, for example, FIG.
Five SD pixels k ₁ to k ₅ located in the vicinity of HD pixels y ₁ ~y ₄ to create, as shown in cut. The SD data extracted by the area extracting circuit 4 is ADRC
It is supplied to the circuit 5.

The ADRC circuit 5 converts the data in each area from, for example, 8-bit SD data to 2-bit S data for the purpose of patterning the level distribution of the SD data in the above-mentioned area.
An operation for compressing the data into D data is performed. As a result, the formed pattern compression data is transferred to the class code generation circuit 8.
To supply.

ADRC is an adaptive quantization method originally developed for high-efficiency coding for VTRs. However, since a local pattern of a signal level can be efficiently represented by a short word length, the present invention provides an ADRC. In the embodiment, the code is used to generate a code for classifying a signal pattern. The ADRC circuit calculates the maximum value MAX and the minimum value MIN in the area by the following equation (2), where the dynamic range DR in the area, the bit allocation is n, the data level of the pixel in the area is L, and the requantization code is Q. Are re-quantized by equally dividing the space between them by a designated bit length.

DR = MAX−MIN + 1 Q = [(L−MIN + 0.5) · 2 ⁿ / DR] (2) where [] means truncation processing.

In this embodiment, it is assumed that the SD data of 5 pixels each separated by the area extracting circuit 4 is compressed to 2 bits. The compressed SD data is defined as q _{1 to} q ₅ respectively.

On the other hand, the SD image signal supplied from the input terminal 1 is also supplied to an area extracting circuit 6. The area cutout circuit 6 mainly functions to cut out pixels necessary for class classification (motion class) for representing the degree of movement. In this embodiment, for example, from the supplied SD image signal, ten SD pixels m _{1 to} m ₅ and n _{1 to} n existing at the positions shown in FIG. 6 with respect to the HD pixels y _{1 to} y ₄ to be created.
Extract ₅ .

The data extracted by the area extracting circuit 6 is supplied to a motion class determining circuit 7. The motion class determination circuit 7 calculates a difference between frames of the supplied SD data, and calculates a motion parameter as an index of motion by performing a threshold process on an average value of the absolute values.
Specifically, the motion class determination circuit 7 calculates the following equation (3)
The average value of the absolute value of the difference between the supplied SD data
Calculate param.

[0044]

(Equation 1) However, in this embodiment, n = 5.

The average value param of the absolute value of the difference between the SD data calculated by the above-described method is determined by, for example, a threshold value set in advance so that the histogram of the absolute value of the difference between the SD data is divided into n equal parts. Of the absolute values of the differences
Calculate the motion class mv-class using param. For example, here, four motion classes are provided and SD
If the average value of the absolute values of the data differences is param ≦ 2, the motion class mv-class is determined to be 0, and if the average value param ≦ 4, the motion class mv-class is determined to be 1, and the average value param ≦
In the case of 8, the motion class mv-class is determined to be 2, and the average value pa
If ram> 8, the motion class mv-class is determined to be 3.
The motion class mv-class determined in this way is supplied to the class code generation circuit 8.

The class code generation circuit 8 includes pattern compression data (space class) supplied from the ADRC circuit 5,
And the motion class supplied from the motion class determination circuit 7
By performing the operation of the following equation (4) based on mv-class, the class to which the block belongs is detected, and the class code class indicating the class is supplied to the ROM tables 9 and 10. The class code class indicates an address read from the ROM tables 9 and 10.

[0047]

(Equation 2) In this embodiment, n = 5 and p = 2.

The ROM tables 9 and 10 store coefficient data for calculating HD data corresponding to SD data by using a linear estimation formula by learning the relationship between the pattern of SD data and HD data. It is stored every time. This is information for converting the SD data into HD data that conforms to the so-called Hi-Vision standard, which is image information having a higher resolution than this image information, using a linear estimation formula. In this embodiment, coefficient data is prepared independently in mode 1 and mode 2. Note that R
A method of creating the coefficient data stored in the OM tables 9 and 10 will be described later. From the ROM tables 9 and 10, _wi (class) which is coefficient data of the class is read from the address indicated by the class code class. The coefficient data is supplied to the estimation calculation circuits 13 and 14.

On the other hand, the input SD data is also supplied to the area extracting circuits 11 and 12. Area extraction circuit 1
Reference numeral ₁ denotes ₂₅ pieces of SD data x _{1 to} x 25 used for the estimation calculation, which are located at positions as shown in FIG. 7 from the input SD data.
Cut out. The output signal of the area cutout circuit 11 is supplied to the estimation calculation circuit 13. The estimation calculation circuit 13 calculates HD data corresponding to the input SD data based on the SD data supplied from the area cutout circuit 11 and the coefficient data supplied from the ROM table 9.

More specifically, the estimation operation circuit 13 outputs the x data, which is the SD data supplied from the region extraction circuit 11,
By ₁ ~x ₂₅ and ROM table 9 w ₁ to w ₂₅ is a coefficient data supplied from, by performing the calculation shown in each formula (5), corresponding to the input SD data H
The D data hd 'is calculated. SD data x ₁ ~x ₂₅ is,
There is output data of the second tap (long tap) set for the SD data. At this time, coefficient data w _i (cl is used for mode 1 using the coefficient for block 1 and for mode 2 using the coefficient for block 2.
An operation is performed based on ass). The created HD data is supplied to the switching circuit 15.

Hd ′ = w ₁ x ₁ + w ₂ x ₂ +... + W ₂₅ x ₂₅ (5)

On the other hand, the area extracting circuit 12
From the data in the position as shown in FIG. 8, cut out 15 SD data x ₁ ~x ₁₅ to be used for estimation operation. That, SD data x ₁ ~x ₁₅ is the output data of the first tap which is set with respect to SD data (short taps). Output signal of region cutout circuit 12 (tap output)
Is supplied to the estimation operation circuit 14. Estimation operation circuit 14
Is the SD data supplied from the area cutout circuit 12,
Based on the coefficient data supplied from the ROM table 10, HD data corresponding to the input SD data is calculated.

More specifically, the estimation calculation circuit 14 calculates the x data which is the SD data supplied from the area cutout circuit 12.
By ₁ ~x ₁₅ and the ROM table 10 w ₁ to w ₁₅ is a coefficient data supplied from, by performing the calculation shown in each formula (6), calculates HD data hd' corresponding to the input SD data I do. At this time, with respect to the mode 1 by using the coefficient for the block 1, with respect to mode 2 using the coefficients of the block 2, the coefficient data w _i
An operation is performed based on (class). H created
The D data is supplied to the switching circuit 15.

Hd ′ = w ₁ x ₁ + w ₂ x ₂ +... + W ₁₅ x ₁₅ (6)

In the switching circuit 15, the DR _ratio (= DR _all / D) supplied from the tap selection parameter calculation circuit 3
If the DR _ratio is smaller than a preset threshold (for example, 4) based on R _partial ), the estimation calculation circuit 1
The switching is performed so as to select the output of No. 3; otherwise, the switching is performed so as to select the output of the estimation operation circuit 14.

The output of the switching circuit 15 is output via an output terminal 16. The HD data output via the output terminal 16 is supplied to, for example, an HD television receiver, an HD video tape recorder, or the like.

As described above, the most significant feature of the present invention is that
The greatest feature lies in that two types of processing, a long tap (25 taps in this embodiment) and a short tap (15 taps in this embodiment), are performed, and adaptively switched and output.

FIG. 9 conceptually shows a one-dimensional level change of pixel data. When trying to estimate the HD data at the position x, it will be briefly described whether to take out the HD data estimated / calculated by a short tap or to take out the HD data estimated / calculated by a long tap. In the case of FIG. 9A showing a smooth level change, HD data estimated by a long tap is extracted from the switching circuit 15. When trying to estimate HD data located near a steep edge as shown in FIG. 9B, the HD data estimated by a short tap is taken out from the switching circuit 15. When trying to estimate HD data located on a steep edge as shown in FIG. 9C, the HD data estimated by a long tap is taken out from the switching circuit 15. Thus, in the switching circuit 15, the DR _ratio from the tap selection parameter calculation circuit 3 is output.
Thus, it is determined whether or not the HD data to be estimated is located in the flat portion and whether an edge exists near the flat portion.

In general, in the case of the image information conversion apparatus of the present invention, when the characteristics of the image to be processed and the characteristics of the image to be learned match, the conversion performance is better when the processing is performed with a long tap. There is a tendency. When converting an image signal captured by an SD video camera into an HD signal, the characteristics of the image to be processed and the characteristics of the image to be learned do not always match, but in most cases, it is better to perform the conversion process with a long tap. , The conversion result is good.

However, a good change result is not always obtained in the vicinity of the contour or the like. In particular, when there is a steep edge adjacent to a flat portion where there is almost no change in luminance, the presence of an edge may cause the change. In some cases, quite remarkable ringing appears on the flat portion, which may cause a large visual deterioration, and may cause a large overall image quality deterioration.

In the image information conversion device of the present invention,
When there is an edge adjacent to a flat part where there is not much change in luminance, the above-described problem is caused by performing image information conversion processing with a short tap, and otherwise performing image information conversion processing with a long tap. The point is greatly improved, and a good conversion result can be obtained even when the characteristics of the processing target image and the characteristics of the learning target image are slightly different.

Next, a method of creating coefficient data stored in the ROM tables 9 and 10 will be described with reference to FIG. In order to obtain coefficient data by learning,
First, one of the HD images corresponding to the already known HD images
An SD image having / 4 pixels is formed. First, input terminal 2
The HD data is supplied via the H. 1 and the pixels in the vertical direction of the HD data are thinned out by the vertical thinning filter 22 so that the vertical frequency in the field becomes １ ／. The pixels in the horizontal direction of the HD data are thinned out.

The created SD data is supplied to the region cutout circuits 24, 26 and 28, and further to the switching circuit 31 for switching between long tap / short tap processing. The HD data supplied from the input terminal 1 is supplied to normal equation addition circuits 34 and 38.

In the area extracting circuit 24, pixels necessary for calculating tap selection parameters are extracted from the SD data supplied from the horizontal thinning filter 23. This is exactly the same as the region cutout circuit 2 already described. SD extracted by the region extraction circuit 24
The data is supplied to the tap selection parameter calculation circuit 25.

The tap selection parameter calculation circuit 25 is a circuit for calculating parameters required for switching between long tap processing and short tap processing, and has the same function as the tap selection parameter calculation circuit 3 already described. I do. DR calculated by tap selection parameter calculation circuit 3
_{The ratio} is supplied to the switching circuit 31.

On the other hand, in the area extracting circuit 26, necessary pixels are extracted from the supplied SD data in order to perform the space class classification. Specifically, the region cutout circuit 26 has the same function as the region cutout circuit 4 described above. The extracted SD data is supplied to the ADRC circuit 27.
Supplied to

The ADRC circuit 27 detects a one-dimensional or two-dimensional level distribution pattern of the SD data supplied for each area, and as described above, all data in each area or a part of the data. An operation of compressing data from, for example, 8-bit SD data to 2-bit SD data is performed to form pattern compressed data, and this pattern compressed data is converted to a class code generation circuit 3.
Supply 0. The ADRC circuit 27 has the AD
This is the same as the RC circuit 5.

On the other hand, the SD image signal supplied to the area extracting circuit 28 is subjected to data extracting for motion class classification. Specifically, the region cutout circuit 2
Numeral 8 has the same function as that of the area extracting circuit 6 described above. The SD data clipped by the region clipping circuit 28 is supplied to a motion class determination circuit 29. The motion class determination circuit 29 specifically has the same function as the motion class determination circuit 7 described above. The motion class mv determined by the motion class determination circuit 29
-class is supplied to the class code generation circuit 30.

The class code generation circuit 30 is the same as the class code generation circuit 8 described above,
By performing the operation of the above equation (4) based on the pattern compression data (space class) supplied from the C circuit 27 and the motion class mv-class supplied from the motion class determination circuit 29, the class to which the block belongs Is detected, and a class code class indicating the class is output. The class code generating circuit 30
class is output to the normal equation adding circuits 33 and 37.

On the other hand, the SD from the horizontal thinning filter 23
The signal is supplied to the switching circuit 31, and in the switching circuit 31,
The region cutout circuit 32 or 36 is selected based on the dynamic range ratio DR _ratio supplied from the tap selection parameter calculation circuit 25. Specifically, when the ratio DR _{ratio of the} dynamic range is smaller than a preset threshold value (for example, 4), the region cutout circuit 3
2 is larger than the threshold value, the area cutout circuit 36 is selected, and the selected area cutout circuit 3 is selected.
2 or 36 is supplied with the SD signal.

The area extracting circuit 32 extracts SD data used for normal equation addition. Area cutout circuit 3
2 is specifically the same as the area extracting circuit 11 described above, and functions to extract SD data necessary for normal equation addition. The output of the area extracting circuit 32 is supplied to a normal equation adding circuit 33.

On the other hand, the area extracting circuit 36 also extracts SD data used for normal equation addition. The area cutout circuit 36 is, specifically, the area cutout circuit 1 described above.
2 and serves to cut out SD data necessary for normal equation addition. The output of the area extracting circuit 36 is supplied to a normal equation adding circuit 37.

Here, the normal equation addition circuits 33 and 3
For explanation of No. 6, learning of conversion formulas from a plurality of SD data to HD data and signal conversion using the prediction formulas will be described. In the following, for the sake of explanation, a case will be described where pixels are more generalized and prediction is performed with n pixels. SD
X _1, x ₂ pixel level, respectively, as · · · x _n,
Requantization data results of p-bit ADRC, each q _1, q _2, and · · · q _n. At this time, the class code class of this area is defined by Expression (4).

As described above, when the SD pixel levels are x ₁ , x ₂ ,..., X _n and the HD pixel levels are y, coefficient data w ₁ , w ₂ ,. _n
Is set as an n-tap linear estimation equation. This is shown in equation (1) above. Learning ago, w _i is undetermined coefficients.

Learning is performed on a plurality of signal data for each class. When the number of data is m, the following equation (7) is set according to the equation (1).

Y _k = w ₁ × _{k 1} + w ₂ × _{k 2} +... + W _n × _kn (7) (k = 1, 2,..., M)

If m> n, w ₁ , w ₂ ,..., W
_{Since n} is not uniquely determined, the element of the error vector e is defined by Expression (8), and coefficient data that minimizes Expression (9) is obtained. This is a so-called least squares method.

E _k = y _k- [w ₁ x _k1 + w ₂ x _k2 +... + W _n x _kn ] (8) (k = 1, 2,..., M)

[0079]

(Equation 3)

Here, a differential coefficient by w _{i in} equation (9) is obtained. What is necessary is just to find each w _i so that the following equation (10) is set to 0.

[0081]

(Equation 4)

Hereinafter, when X _ji Y _i is defined as in equations (11) and (12), equation (10) can be rewritten into equation (13) using a matrix.

[0083]

(Equation 5)

[0084]

(Equation 6)

[0085]

(Equation 7)

This equation is generally called a normal equation. The normal equation addition circuits 33 and 36 generate the class code class supplied from the class code generation circuit 30.
, SD data x ₁ from the region cutout circuit 32,
x ₂ ,... x ₂₅ and S from the area cutout circuit 36
This normal equation addition is performed by using the HD data y corresponding to the SD data supplied from the input terminal 21 for the D data x ₁ , x ₂ ,..., X ₁₅ .

After the input of all the training data is completed, the normal equation adding circuit 33
4 outputs the normal equation data. Prediction coefficient determination circuit 3
In step 4, a normal equation is solved for w _i using a general matrix solution method such as a sweeping out method, and coefficient data (prediction coefficient value) is calculated. The prediction coefficient determination circuit 34 writes the calculated coefficient data to the memory 35.

Similarly, after the input of all the training data is completed, the normal equation adding circuit 36 outputs the normal equation data to the prediction coefficient determining circuit 38. Prediction coefficient determining circuit 38, using a general matrix solution such as a method sweeping the normal equation is solved have w _i Nitsu to calculate the coefficient data. The prediction coefficient determination circuit 38 writes the data into the calculated prediction coefficient memory 39.

As a result of training as described above,
A long tap (25 in this embodiment) is stored in the memory 35.
In this case, coefficient data for estimating the target HD data y for each class in the case of using (tap), which can be estimated statistically closest to the true value, is stored. As described above, the table stored in the memory 35 is the ROM table 9 used in the image signal conversion device of the present invention.

Similarly, in the memory 39, a coefficient for estimating the target HD data y for each class in the case of using short taps (15 taps in this embodiment), which can be estimated statistically closest to the true value Data is stored. As described above, the table stored in the memory 39 is the ROM table 10 used in the image signal conversion device of the present invention. By the above processing, SD
Learning of coefficient data for creating HD data from data ends.

FIG. 11 shows a circuit diagram around the estimation operation circuits 13 and 14 in the above-described embodiment. The address control circuit 41 includes a class code generation circuit 8
, The coefficient memory 43 corresponds to the ROM table 10, the product-sum machine 44 corresponds to the estimation operation circuit 13, the multiplier memory 46 corresponds to the ROM table 9, and the product-sum machine 47.
Corresponds to the estimation operation circuit 14, and the MUX (multiplexer) 48 corresponds to the switching circuit 15. Further, the long tap multiplicand register 42 holds the output of the area cutout circuit 11, and the short tap multiplicand register 45 holds the output of the area cutout circuit 12.

That is, FIG. 11 shows a configuration which can be used by switching between long taps and short taps. Pixel data of N taps (for example, 25 taps) is supplied from the long tap multiplicand register 42 to the accumulator 44. The coefficient data corresponding to the pixel data is stored in the address control circuit 41.
Are read from the multiplier memory 43 by the class code "class", and the read 25 coefficient data are supplied from the multiplier memory 43 to the multiply-accumulator 44. The product-sum unit 44 performs a product-sum operation on the corresponding pixel data and coefficient data as shown in Expression (5), and the product-sum output, that is, HD data, is supplied to a MUX (multiplexer) 48.

Pixel data of M taps (for example, 15 taps) is supplied from the short tap multiplicand register 45 to the multiply-accumulator 47. The coefficient data corresponding to the pixel data is read from the multiplier memory 46 by the class code class from the address control circuit 41, and the read 1
The five coefficient data are supplied from the multiplier memory 46 to the accumulator 47. The product-sum unit 47 performs a product-sum operation on the corresponding pixel data and coefficient data as shown in Expression (6), and the product-sum output, that is, HD data, is supplied to the MUX 48. In the MUX 48, the switching signal supplied from outside with the HD data from the long tap and the HD data from the short tap, the above-mentioned DR _ratio (= DR _all / DR
_Partial ), the selected HD data is output from the output terminal 49 after being switched.

Here, as shown by the equations (5) and (6), the difference between the sum-of-products 44 and 47 is only the number of taps, and as shown in FIG. Waki 4
By using only 4, the product-sum operation of short taps can also be performed. The multiplier memory 51 corresponds to the ROM tables 9 and 10, and the MUX 52 corresponds to the switch 15.

The 25 tap pixel data from the long tap multiplicand register 42 is supplied to the MUX 52, and the 15 tap pixel data from the short tap
UX52. In the MUX 52, DR _ratio (= DR _all / DR _partial ) which is a switching signal from outside is used.
Thereby, the pixel data from the long tap multiplicand register 42 and the pixel data from the short tap multiplicand register 45 are switched. The pixel data selected from the MUX 52 is supplied to the accumulator 44.

MUX from address control circuit 41
A class code class corresponding to the pixel data selected by 52 is supplied to the multiplier memory 51. In the multiplier memory 51, coefficient data is read based on the DR _ratio , which is an external switching signal, and the supplied class code class, and the read coefficient data is supplied to the multiply-accumulator 44. The sum-of-products unit 44 performs the sum-of-products operation as described above, and as a result of the sum-of-products operation, HD data is obtained. That H
The D data is output from the output terminal 53.

As described above, when the circuit in FIG. 12 is compared with the circuit in FIG. 11, only one product-sum device can be used, and the hardware can be considerably reduced. Therefore, as shown in FIG. 13, in order to further reduce the hardware, the used sum of products is changed from the long tap sum of products 44 to the short tap sum of products 47. The long / short tap determination circuit 63 corresponds to the class code generation circuit 8, and the multiplier memory 64
The MUX 67 corresponds to the M tables 9 and 10, and the MUX 67 corresponds to the switch 15.

In the address control circuit 41, the class code class of the corresponding pixel data is supplied to the control memory 61 and the multiplier memory 64. The control memory 61 controls the tap degeneration operation circuit 62 based on the supplied class code class. In the tap reduction operation circuit 62, the N tap pixel data from the long tap multiplicand register 42 is reduced to L tap pixel data and supplied to the MUX 67. At this time, a relationship of N taps ≧ L taps is established. For example, in the above-described embodiment, N = 25 and L = 15. Pixel data of M taps (for example, 15 taps) is supplied from the short tap multiplicand register 45 to the MUX 67.

The long / short tap determination circuit 63 determines whether to use the long tap pixel data or the short tap pixel data.
And to the multiplier memory 64. The determination result output from the long / short tap determination circuit 63 is similar to the DR _ratio output from the tap selection parameter calculation circuit 3 described above. The MUX 67 selects the pixel data from the tap degeneration operation circuit 62 or the pixel data from the short tap multiplicand register 45 using the determination result from the long / short tap determination circuit 63 as a switching signal. The selected pixel data is supplied from the MUX 67 to the accumulator 47.

In the multiplier memory 64, coefficient data is selected from the class code class from the address control 41 and the determination result from the long / short tap determination circuit 63. The selected coefficient data is supplied from the multiplier memory 64 to the accumulator 47. The sum-of-products unit 47 performs a sum-of-products operation based on the above equation (6), and outputs the sum-of-products, that is, HD data, from the output terminal 66.

Here, the reduction from N taps to L taps performed by the tap reduction operation circuit 62 will be briefly described. First, N-tap pixel data is subjected to absolute value conversion. An average value and a maximum value are obtained from the N-tap pixel data subjected to the absolute value conversion, and 0 is set as a temporary representative value A.

The pixel data of N taps for each provisional representative value A is divided into groups. An average value of the pixel data is obtained for each of the divided groups, and the obtained average value is set as a temporary representative value B. At this time, it is assumed that 0 does not always fluctuate. It is determined whether or not the provisional representative value B is equal to L. If it is determined that the provisional representative value B is not equal to the provisional representative value B, the N-tap pixel data that has been converted into the absolute value is again divided into groups based on the provisional representative value B. . An error between the tentative representative value B and the pixel data in the group is calculated, and ± 0.0001 is added to the tentative representative value B of the group having the maximum error to divide the data into two. The two values and the temporary representative value B that did not result in the maximum error are reset as the temporary representative value A. This process is repeated until the temporary representative value B becomes equal to L.

Further, when it is determined whether or not the provisional representative value B is equal to L, when it is determined that the provisional representative value B is equal to L, the provisional representative value B becomes pixel data reduced to L taps.

Next, using the circuit diagram shown in FIG.
The short tap determination circuit 63 will be described in detail. The long tap determination area register 71 determines pixel data included in a long tap area from input data. When the pixel data included in the long tap area is selected, the pixel data is supplied from the long tap determination area register 71 to the dynamic range calculation circuit 72. The dynamic range calculation circuit 72 calculates a dynamic range from the supplied pixel data, and supplies the calculated dynamic range to the comparator 79 via the register 73.

The short tap determination area register 74 determines the pixel data included in the short tap area from the input data. When the pixel data included in the short tap area is selected, the pixel data is supplied from the short tap determination area register 74 to the dynamic range calculation circuit 75. In the dynamic range calculation circuit 75, a dynamic range is obtained from the supplied pixel data, and the dynamic range is supplied to the multiplier 77. In the multiplier 77, the dynamic range is multiplied by a threshold value TH set in advance via the register 76. The result of the multiplication is supplied to the comparator 79 via the register 78.

In the comparator 79, the dynamics for a long tap
For short taps multiplied by cleanse and threshold TH
As shown in equation (14), the dynamic range of
Are compared. The comparison result is output from the output terminal 80.
You. The long tap DR shown in the equation (14) is as described above.
DR_allUnlike the SD data x shown in FIG.₁~ X _{twenty five}
And the short tap DR is the DR
_partialUnlike the SD data x shown in FIG.₁~ X_Fifteen
Is similar to

Long tap DR ≧ TH × Short tap DR (14)

That is, the threshold value TH is obtained by multiplying the dynamic range for the short tap by the value of the preset setting register, and is compared with the dynamic range for the long tap.

In the above description of the embodiment, the ADRC is provided as information compression means for patterning a spatial waveform with a small number of bits. However, this is only an example, and a class having a small signal waveform pattern is used. Any information compression means can be provided as long as the information compression means can be represented by, for example, DPCM (Differential Pulse Code Modula).
) or VQ (Vector Quantization).

In the above description of the embodiment, in order to calculate a motion parameter, a difference between frames of SD pixel data is calculated, and an average value of absolute values thereof is subjected to threshold processing. It is not necessary to use this method.For example, normalized data may be calculated by dividing a time difference by a spatial difference, and thresholding may be performed on the normalized data to obtain a motion parameter. It is possible.

[0111]

According to the present invention, pixel data is predicted by estimating long taps, so that the accuracy of prediction can be improved. Of course, when performing the long tap estimation calculation, the image quality degradation can be prevented by performing the short tap estimation calculation on the edge portion adjacent to the flat portion where the image quality is likely to deteriorate and there is almost no change in luminance. As a result, the conversion performance for an image can be greatly improved.

Further, according to the present invention, in a product-sum operation circuit having a plurality of inputs having different numbers of taps, the number of taps necessary for the data having the largest number of taps is not required for the input of the multiplier as in the prior art. Therefore, the hardware scales of the multiplier memory and the accumulator are largely eliminated, and the entire hardware scale can be reduced. Moreover, due to the effect of the tap degeneration arithmetic circuit,
The performance of the long tap is not significantly degraded, and a filter operation optimal for image quality can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an image information conversion device according to the present invention.

FIG. 2 is a schematic diagram for explaining a positional relationship between SD data and HD data.

FIG. 3 is a schematic diagram for explaining a positional relationship between SD data and HD data.

FIG. 4 is a schematic diagram for explaining data used in a tap selection parameter calculation circuit.

FIG. 5 is a schematic diagram for explaining data used for space class classification.

FIG. 6 is a schematic diagram for explaining data used for motion class classification.

FIG. 7 is a schematic diagram for explaining pixels used for estimation calculation for long tap processing.

FIG. 8 is a schematic diagram illustrating pixels used for an estimation calculation for short tap processing.

FIG. 9 is a schematic diagram used for describing a switching circuit according to the present invention.

FIG. 10 is a block diagram of an embodiment when creating a correction table according to the present invention.

FIG. 11 is an example of a long / short tap product-sum operation circuit.

FIG. 12 is an example of a long / short tap product-sum operation circuit.

FIG. 13 shows an embodiment of the long / short tap product-sum operation circuit of the present invention.

FIG. 14 is a circuit diagram of one embodiment of a long / short tap determination circuit according to the present invention.

FIG. 15 is a circuit diagram of a conventional image information conversion device.

FIG. 16 is a circuit diagram of a main part of a conventional image information conversion device.

FIG. 17 is a block diagram of a conventional image conversion device.

FIG. 18 is a block diagram when a correction table according to a conventional image conversion apparatus is created.

[Explanation of symbols]

2, 4, 6, 11, 12,... An area cutout circuit,
..Tap selection parameter calculation circuit, 5 ... ADRC
Circuit, 7: motion class determination circuit, 8: class code generation circuit, 9, 10: ROM table, 13,
14 ... Estimation calculation circuit, 15 ... Switching circuit

Claims (11)

[Claims] 1. An image information conversion apparatus for converting a first digital image signal into a second digital image signal having a larger number of pixels, comprising: a pixel data at a predetermined position from the first digital image signal; A pixel cutout means for cutting out, a pattern of a level distribution of the pixel data cut out by the pixel cutout means, a class to which the pixel data to be estimated belongs is determined based on the pattern, and class information is output. Class determination means, and coefficient data of an estimation formula, which is information for converting the first digital image signal into the second digital image signal, are stored for each class. Coefficient data storage means for outputting said coefficient data in accordance with said class information; and said first digital image signal The prediction value of the second digital image signal is calculated using an estimation expression formed by a linear combination of the pixel data of the first tap extracted from the second tap or the second tap longer than the first tap and the coefficient data. An image information conversion device, comprising: an image signal generation unit that generates a signal; and switching between processing by the first and second taps. 2. The image information conversion apparatus according to claim 1, wherein pixel data of the second digital image signal to be estimated is located in a flat portion, and whether an edge exists near the flat portion. An image information conversion apparatus, further comprising: a determination unit that determines whether the processing is performed by the first and second taps, based on a determination result of the determination unit. 3. The image information conversion device according to claim 1, wherein a first data storing unit stores coefficient data corresponding to the first tap.
And a second memory for storing coefficient data corresponding to the second tap.
A first area extracting means for extracting the pixel data of the first tap from the first digital image signal; and a second area extracting means for extracting the pixel data of the second tap from the first digital image signal. A first calculating unit that generates a first predicted value by the estimation formula using the pixel data from the first region extracting unit and the coefficient data from the first memory. And second calculation means for generating a second predicted value by the estimation formula using the pixel data from the second area cutout means and the coefficient data from the second memory. An image information conversion device characterized by the above-mentioned. 4. The image information conversion apparatus according to claim 1, wherein the pixel data of the second digital image signal to be estimated is located in a flat portion and an edge exists in the vicinity of the flat portion. Is satisfied, the process of the first tap is selected, and if the condition is not satisfied, the process of the second tap is selected. 5. The image information conversion device according to claim 4, wherein a ratio between a dynamic range of a plurality of pixel data of the first tap and a dynamic range of a plurality of pixel data of the second tap is determined. An image information conversion apparatus, wherein one of a first tap process and a second tap process is selected. 6. The image information conversion apparatus according to claim 1, wherein the coefficient data stored in the coefficient data storage means is obtained by learning in advance, and the method of learning the coefficient data is a second digital data conversion method. The image signal is converted into a first digital image signal having a smaller number of pixels than the second digital image signal, and pixel data at a predetermined position is cut out from the converted first digital image signal. Detecting the level distribution pattern of the extracted pixel data, determining the class to which the pixel data to be estimated belongs based on the pattern, outputting class information, and determining the position and the position of the pixel data to be estimated. One of the first and second taps is selected according to pixel data around the position of the pixel data to be estimated. The coefficient data that minimizes the sum of the squares of the error between the pixel data from the second digital image signal and the prediction value obtained from the estimation formula including the linear combination is transmitted to the first tap. An image information conversion device, wherein each of the second taps is obtained for each of the class information. 7. The image information conversion device according to claim 1, wherein the image signal generating unit uses a pixel data of the first tap and the coefficient data from the coefficient data storage unit to perform a product sum. First sum-of-products means for performing an operation, second sum-of-products means for performing a sum-of-products operation using the pixel data of the second tap, and the coefficient data from the coefficient data storage means, Switching signal generating means for generating a switching signal for switching between the first tap and the second tap; a first signal from the first sum-of-products means responsive to the switching signal;
And a selecting means for selecting the second sum-of-products output from the second sum-of-products means. 8. The image information conversion apparatus according to claim 1, wherein said image signal generating means generates a switching signal for generating a switching signal for switching between said first tap and said second tap. Means for selecting pixel data of the first tap and pixel data of the second tap in response to the switching signal; storing the pixel data from the selecting means; and storing the coefficient data. And a product-sum means for performing a product-sum operation using the coefficient data from the means. 9. The image information conversion device according to claim 1, wherein the image signal generating unit includes: a shrinking unit that shrinks the second tap to a third tap shorter than the second tap; Switching signal generating means for generating a switching signal for switching between the second tap and the third tap; pixel data of the second tap and pixel data of the third tap in response to the switching signal And a product sum means for performing a product sum operation using the pixel data from the selection means and the coefficient data from the coefficient data storage means. Information conversion device. 10. An image information conversion method for converting a first digital image signal into a second digital image signal having a larger number of pixels, comprising the steps of: And a step of detecting a level distribution pattern of the pixel data extracted by the pixel extraction means, determining a class to which the pixel data to be estimated belongs based on the pattern, and outputting class information. And coefficient data of an estimation formula, which is information for converting the first digital image signal into the second digital image signal, is stored for each class, and the class information from the class determination means is stored. Outputting the coefficient data according to the following equation: Determining whether the data is located in the flat portion and whether there is an edge near the flat portion; and a first tap or the first tap cut out from the first digital image signal Generating a predicted value of the second digital image signal using an estimation expression formed by a linear combination of longer pixel data of the second tap and the coefficient data; Switching the processing by tapping the image information. 11. A product-sum operation device for performing a product-sum operation of pixel data of a plurality of taps of an input digital image signal and a plurality of coefficient data stored in a multiplier memory, wherein M is used as the plurality of taps. In the multiply-accumulation device in which N longer than M is set, an N-tap register that outputs the N-tap pixel data, an M-tap register that outputs the M-tap pixel data, Shrinking means for shrinking to a multiplicand of L taps shorter than the tap; selecting means for selecting whether the product-sum operation is performed by the L taps or M taps; A memory for outputting coefficient data corresponding to the M tap, and using the selected L tap or the M tap and the coefficient data, Means for performing a sum-of-products operation.