JP2001245130A - Image processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit, equipped in satisfactory balance for both high speed computation at compositing images and continuity of the composited images. SOLUTION: This image processing unit composites multiple images with different exposures into a single image with a wide dynamic range. Namely, it is provided with an extraction part 13 for proper exposures which extracts the proper exposure thresholds, based on the image signal level for each of the images, provided with an extraction part 14 of the duplicate areas which extracts the duplicate areas in each of the proper exposure threshold extracted, provided with the correction part 15 of the gradients which corrects the gray scale in the proper exposure thresholds, and provided the image compositing part 16, which has a selection part 22 of the standard image which selects the a single image as the standard image such that the proper exposure thresholds have the largest area, by the correction part 23 of the exposures which compensates for the exposures, except for the standard image, through a process of obtaining the exposure difference between the duplicate areas and the standard image, and by the weighted addition part 24, which composites through addition weighted on the function, based on the change of the image signal level in the standard image, when they are duplicate areas on the proper exposure thresholds after compensation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus for generating one wide dynamic range image from a plurality of images captured under different exposure conditions.

[0002]

2. Description of the Related Art Conventionally, an image processing apparatus has been proposed which combines a plurality of images captured under different exposure conditions to generate one wide dynamic range image.

[0003] As an example of such a device, Japanese Patent Application No.
No. 338551 discloses that each image is divided into a proper exposure area and an improper exposure area, gradation correction is performed for each proper exposure area, and a proper exposure area for each gradation-corrected image is synthesized. An image processing device that generates one wide dynamic range image is described, and further, as an example of a device to which the image processing device is applied, a super latitude digital camera capable of imaging a subject with a wider dynamic range is described. Have been.

[0004] As another example, Patent Publication No. 27
No. 55366, for example, when two images of a long exposure image and a short exposure image are combined to generate one wide dynamic range image, each image is divided into an appropriate exposure area and an inappropriate exposure area. However, at this time, a digital camera that provides an intermediate area that belongs to both the proper exposure area for long-time exposure and the appropriate area for short-time exposure, and performs weighted addition within this intermediate area to synthesize an image. Has been described.

[0005]

However, in the technique disclosed in Japanese Patent Publication No. 2755366, since the synthesizing process is performed using a different function for each image, it takes time to calculate each function. There are issues.

Further, two straight lines in the intermediate area mid are obtained from data on the end points of the proper exposure area long for long exposure and data on the end points of the proper exposure area short for short exposure. A weight coefficient is obtained.

When such a means is used, when the shape of the intermediate area on the screen becomes complicated, it becomes difficult to obtain the data of the end points along the change in the density direction. (It is necessary to find the center of gravity of mid and scan in the direction perpendicular to this.)

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus having a good balance between the high-speed operation for synthesizing images and the continuity of the synthesized images. The purpose is.

[0009]

In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention processes an image group consisting of a plurality of images of the same subject captured under different exposure conditions. An image processing apparatus for generating one wide dynamic range image, comprising: an extracting unit for extracting an appropriate exposure area based on an image signal level for each image in the image group; Overlapping area extracting means for extracting an area overlapping between the exposure areas; gradation correcting means for performing gradation correction on the proper exposure area; and overlapping the proper exposure area gradation corrected by the gradation correcting means. Synthesizing means for generating one wide dynamic range image by synthesizing based on the information of the area.

An image processing apparatus according to a second aspect of the present invention
In the image processing apparatus according to the first aspect of the present invention, the synthesizing unit includes a selecting unit that selects one image from the image group as a reference image, and an exposure difference between the appropriate exposure area overlapping the image group and the reference image. Exposure correction means for obtaining exposure correction other than the reference image, and a change in the image signal level of the reference image if the above-described exposure-corrected appropriate exposure area does not overlap with the above-mentioned overlapping area. And weighting and adding means for performing composition by performing weighting and adding with a function based on.

Further, in the image processing apparatus according to a third aspect of the present invention, in the image processing apparatus according to the second aspect of the present invention, the selecting means may select one image having the largest appropriate exposure area from the image group as a reference image. Is to be selected as

[0012]

Embodiments of the present invention will be described below with reference to the drawings. 1 to 4 show one embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of an electronic camera.

In the present embodiment, the image processing apparatus of the present invention is applied to an electronic camera. For simplicity, two images of an image obtained by short-time exposure and an image obtained by long-time exposure are combined to form one wide area. Although the case where a dynamic range image is obtained will be described, it is needless to say that the present invention can be applied to a case where a larger number of images are combined.

The electronic camera comprises a single-chip type color CCD or the like having an electronic shutter function. The CCD 4 is for photoelectrically converting a subject image and outputting it as an image signal, and for forming the subject image on the CCD 4. Lens system 1
A stop 2 for controlling a passing range of a light beam passing through the lens system 1, a low-pass filter 3 serving as an optical filter for removing unnecessary high-frequency components from the light beam passing through the stop 2, and an output from the CCD 4. After that, an A / D converter 5 for converting an analog image signal, which has been subjected to noise component removal by a correlated double sampling circuit (not shown) or the like and then amplified to a digital signal, into a digital signal, and the A / D converter A first image buffer 6a and a second image buffer 6b for storing image data of one screen digitized by the converter 5 and storing an image obtained by long-time exposure and an image obtained by short-time exposure, respectively. The image data is read out from the first image buffer 6a, which is also used to store the data for photometry and focus detection, and The exposure control unit 7 for controlling the electronic shutter of the opening diameter and the CCD4 of the diaphragm 2 so that the proper exposure when imaging seek, the first image buffer 6a
And a focusing point detecting unit 8 for detecting an in-focus position and controlling an AF motor 9 to be described later based on the detection result, and an AF of the lens system 1 controlled by the in-focus detecting unit 8. An AF motor 9 for driving an imaging lens to form a subject image on the CCD 4, and interpolating the single-plate image data read out from the first and second image buffers 6a and 6b by 3 An interpolating unit 10 for converting the image data of the plate, a working buffer 11 for storing the interpolated image data, and image data of the three plates read from the working buffer 11 for the luminance signal Y and the color difference signals Cb, Cr. And a Y / C separation unit 12 for separating the
First, the luminance signal Y is read out, and for each pixel constituting the entire screen, whether or not the pixel is appropriately exposed is determined based on the signal level, and the divided image information is extracted and output based on the result. Appropriate exposure extraction unit 13 as a means
And an overlapping area extracting means for extracting an overlapping area, which is an overlapping portion between a proper exposure area for long-time exposure and a proper exposure area for short-time exposure, based on the divided image information extracted by the proper exposure extracting unit 13. A conversion characteristic is calculated by calculating a histogram of an edge, which is a feature amount, of the image data of the appropriate exposure area output from the overlapping area extraction unit 14 and the appropriate exposure extraction unit 13, and the floor characteristic of each image is calculated based on the conversion characteristic. Tone correction unit 15 as tone correction means for performing tone conversion
And the gradation output from the gradation correction unit 15 with reference to the information on the proper exposure area output from the proper exposure extraction unit 13 and the information on the overlapping area output from the overlapping area extraction unit 14. An image synthesizing unit 16 as synthesizing means for synthesizing an image related to long-time exposure and an image related to short-time exposure after conversion to generate one wide dynamic range image, and a wide dynamic range synthesized by the image synthesizing unit 16 An output unit 17 that outputs an image to, for example, a recording medium or a display device, and the above-described interpolation unit 10, an appropriate exposure extraction unit 13, an overlapping area extraction unit 14, a gradation correction unit 15, an image synthesis unit 16, and an output unit 17 are included. A control unit 1 for controlling the entire electronic camera
And 8.

FIG. 2 is a block diagram showing a detailed configuration of the image synthesizing unit.

The image synthesizing section 16 is provided with the gradation correcting section 1.
When the proper exposure area related to the long exposure and the proper exposure area related to the short exposure after the gradation correction by 5 are combined, the appropriate exposure area related to the long exposure and the short exposure An intermediate area (overlap area) included in both of the appropriate exposure areas is synthesized so as to maintain practical image continuity while emphasizing the processing speed, thereby generating an image with a wide dynamic range. For example, a first working buffer 20 for storing image data of a proper exposure area related to long-time exposure, and a second working buffer 2 for storing image data of a proper exposure area related to, for example, short-time exposure
1 and one of these images having the larger area of the appropriate exposure area (the image having the largest area of the appropriate exposure area when three or more images are combined) is selected as the reference image. A reference image selecting unit 22 as a selecting unit, and an exposure correcting unit 23 as an exposure correcting unit that calculates an exposure difference between the reference image and the reference image from the intermediate region and corrects exposure of an image other than the reference image.
With respect to the proper exposure area subjected to the exposure correction, the non-overlapping portions are left as they are, and the overlapping portions are weighted and added by a function based on a change in the image signal level of the reference image (see Equation 1 described later). And a weighting and adding unit 24 serving as a weighting and adding means for performing the following.

FIG. 3 is a diagram showing how a long-time exposure image and a short-time exposure image are synthesized in an intermediate area, and FIG. 4 is a flowchart showing image synthesis processing. The operation of the image synthesizing unit 16 will be described with reference to FIG. 4 while referring to FIGS.

The image data whose gradation has been corrected by the gradation correcting section 15 is temporarily stored in a first work buffer 20 and a second work buffer 21, respectively.

More specifically, first, image data relating to the luminance signal Y of a long-time exposure image (image (1) in the figure) is stored in the first work buffer 20 (step S1), Image data related to the luminance signal Y of the exposure image (image (2) in the figure) is stored in the second work buffer 21 (step S4), and image synthesis processing as described later is performed on them. At this time, image data of an intermediate region (overlapping region) related to long-time exposure and image data of an intermediate region (overlapping region) related to short-time exposure are also read (Step S3).

Thereafter, the image data relating to the color difference signal Cb of the long-time exposure is stored in the first working buffer 20, and the image data relating to the color difference signal Cb of the short-time exposure is stored in the second working buffer 21. The image data is stored and similarly subjected to image synthesis processing.
r is stored in the first working buffer 20, and the image data related to the short-time exposure color difference signal Cr is stored in the second working buffer 21, so that image combining processing is performed. .

At this time, the processing relating to the luminance signal Y needs to be performed first, but the processing order of the color difference signal Cb and the color difference signal Cr may be reversed.

Then, the information of the proper exposure area input from the proper exposure extracting unit 13 is read (step S).
2) The reference image selection unit 22 sets, for example, the larger one of the appropriate exposure area related to long-time exposure and the appropriate exposure area related to short-time exposure as the reference image (step S5).

On the other hand, the image data stored in the first working buffer 20 and the second working buffer 21 is the information of the overlapping area output from the overlapping area extracting unit 14,
The exposure correction unit 2 controlled by the control unit 18 based on the selected image information output from the reference image selection unit 22
3 is used to calculate the exposure ratio (step S6).
If the long exposure image (image (1)) stored in the work buffer 20 is not selected as the reference image (step S7), exposure correction is performed on the long exposure image (step S8). On the other hand, if the short-time exposure image (image (2)) stored in the second working buffer 21 is not selected as the reference image (step S9), exposure correction is performed on the short-time exposure image. (Step S10). That is, exposure correction is performed on images other than the reference image.

In the weighting and adding section 24, the control section 18
Control, the selection result by the reference image selection unit 22 (for example, description will be made assuming that image data related to long exposure has been selected as the reference image) and the data output from the appropriate exposure extraction unit 13 Based on this, the luminance Ymid and the chrominance signals Cbmid and Crmid in the intermediate area are weighted and added as shown in the following equation 1 to perform arithmetic processing (step S13).

[0025]

(Equation 1)

Here, long_Y, long_Cb,
long_Cr indicates a luminance signal Y and two color difference signals Cb and Cr in an intermediate area (overlapping area) related to long-time exposure, respectively, and short_Y, short_Cb, and short
t_Cr indicates a luminance signal Y and two color difference signals Cb and Cr in an intermediate area (overlapping area) related to short-time exposure, respectively. In addition, max_Ylong and min_Ylong indicate the maximum luminance value and the minimum luminance value of the intermediate area (overlap area) related to the long-time exposure, respectively.

FIG. 3 shows the appearance of signals when synthesizing images using Equation 1 as described above. In the non-overlapping area, the appropriate exposure area after exposure correction (image (1 ) And the image (2)) are used as they are, and for the overlap region, a composite signal that has been subjected to the processing shown in the above equation 1 is used.

The signal synthesized in the overlapping area is not always connected at both ends to a long-time exposure image or a short-time exposure image, but information on a long-time image serving as a reference image (as a reference image) When a short-time image is selected, it becomes information of a short-time image, and when it is selected from a group of three or more images, it becomes information of the selected image). To have done,
The load on the processing circuit can be reduced, and the processing time can be reduced.

The weighted and added image data of the overlapping area, the image data of the non-overlapping proper exposure area related to the long-time exposure (step S11), and the image data of the non-overlapping proper exposure area related to the short-time exposure Data (Step S
12) and (Step S14), and outputs the combined image (Step S15), and ends.

In the above description, as shown in Expression 1, the weighted addition processing is performed using the information of the selected reference image (the above-described long-time exposure image). Equation 2 may be used.

[0031]

Ymid = long_Y / 2 + short_Y / 2 Cbmid = long_Cb / 2 + short_Cb / 2 Crmid = long_Cr / 2 + short_Cr / 2

When this equation 2 is used, it is possible to obtain a practical composite image having almost no uncomfortable feeling even in the intermediate region by performing only a very simple operation.

According to such an embodiment, the selected 1
Since the weighted addition is performed using a function based on only the information of one reference image (a single function), the load of the calculation when combining the images is reduced as compared with the case where the weighted addition is performed using a plurality of different functions. Processing can be sped up. Also,
Since it is not necessary to scan along the change in the density direction of the image, the processing is also easy at this point. Further, by using a constant that does not depend on the independent variable, it is possible to further speed up the processing.

Thus, an image processing apparatus having a good balance between the high-speed operation and the continuity of the synthesized image is obtained.

It should be noted that the present invention is not limited to the above-described embodiment, and it is needless to say that various modifications and applications can be made without departing from the gist of the invention.

[0036]

As described above, according to the image processing apparatus of the first aspect of the present invention, since the gradation-corrected proper exposure area is synthesized based on the information of the overlapping area,
It is possible to have a good balance between the high-speed operation at the time of combining images and the continuity of the combined image.

Further, according to the image processing apparatus of the present invention, the same effects as those of the first aspect of the present invention can be obtained, and the reference image is selected and the overlapping appropriate exposure area is selected. To calculate the exposure difference between the reference image and the exposure correction other than the reference image, and perform weighted addition with a function based on the change in the image signal level of the reference image to combine them. Good balance can be achieved.

Further, according to the image processing apparatus of the present invention, the same effect as that of the invention described in the second aspect is obtained, and one image in which the area of the appropriate exposure area is maximum is selected as the reference image. As a result, efficient processing can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an electronic camera according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of an image synthesizing unit according to the embodiment.

FIG. 3 is a diagram showing a state in which a long-time exposure image and a short-time exposure image are combined in an intermediate region in the embodiment.

FIG. 4 is a flowchart showing image synthesis processing in the embodiment.

[Explanation of symbols]

 Reference Signs List 4 CCD 6a First image buffer 6b Second image buffer 10 Interpolation unit 11 Working buffer 12 Y / C separation unit 13 Appropriate exposure extraction unit (extraction means) 14 Overlap area extraction unit ( Overlapping area extracting means) 15: gradation correcting section (gradation correcting means) 16: image synthesizing section (synthesizing means) 17 ... output section 18 ... control section 20 ... first work buffer 21 ... second work buffer 22 ... Reference image selecting unit (selecting unit) 23: exposure correcting unit (exposure correcting unit) 24: weighting adding unit (weighting adding unit)

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/335 G06F 15/66 470J 15/68 310J // H04N 101: 00 H04N 1/40 101E F term ( Reference) 5B057 AA20 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CE10 CE11 DA07 DA08 DB02 DB06 DB09 DC22 DC32 5C022 AA13 AB01 5C024 BX01 CX46 DX01 GY01 HX00 HX14 HX28 HX30 HX50 5C076 AA11 AA12PPA PQ18 TT09

Claims (3)

[Claims] 1. An image processing apparatus for processing an image group consisting of a plurality of images of the same subject imaged under different exposure conditions to generate one wide dynamic range image, comprising: Extracting means for extracting a proper exposure area based on an image signal level for each image; overlapping area extracting means for extracting an area overlapping between the proper exposure areas for each of the extracted images; and gradation for the proper exposure area Tone correcting means for performing correction, and synthesizing means for generating one wide dynamic range image by synthesizing an appropriate exposure area whose tone has been corrected by the tone correcting means based on the information of the overlapping area. An image processing apparatus comprising: 2. The image processing apparatus according to claim 1, wherein the synthesizing unit selects one image from the image group as a reference image, and obtains an exposure difference from the reference image from an appropriate exposure area overlapping the image group to obtain an image other than the reference image. Exposure correction means for performing the exposure correction of the above, the appropriate exposure area that has been subjected to the exposure correction, if it is not the overlapping area, as it is, if it is the overlapping area, the function based on the change of the image signal level of the reference image The image processing apparatus according to claim 1, further comprising: weighting and adding means for performing composition by weighting and adding. 3. The image processing apparatus according to claim 2, wherein the selecting means selects one image having the largest area of the appropriate exposure area from the image group as a reference image.