JP2010226314A - Movie processing apparatus, movie processing method, and movie processing program - Google Patents

Abstract

The present invention alleviates a blurring path that causes a motion that gives a discontinuous impression during reproduction, which may occur when an exposure time is extremely short compared to a frame period.
A moving image processing apparatus according to the present invention includes a motion detecting unit that detects a motion between frames from moving image data, a filter coefficient generating unit that generates a filter coefficient based on a detection result of the motion detecting unit, and the filter A filter that averages pixel values of the moving image data in a path of movement between the frames by filtering the moving image data by a coefficient.
[Selection] Figure 2

Description

  The present invention relates to moving image processing, and relates to, for example, a moving image processing apparatus, a moving image processing method, and a moving image processing program that alleviate a motion that gives a discontinuous impression.

  Conventionally, when capturing a moving image, a digital camera has set the exposure time (charge accumulation time) to almost the full frame period in order to ensure smooth reproduction image quality.

  In addition, in moving image processing, a method has been proposed in which image data between successive frames is generated using a motion vector and the frame rate of the moving image is converted (see, for example, Patent Document 1).

JP 2007-166198 A

  By the way, in order to set the exposure time to almost the full frame period, it is necessary to perform exposure control by aperture adjustment called a galvano system using a special aperture device in order to limit the amount of light. For this reason, instead of exposure control by adjusting the aperture, the exposure time is adjusted to control the amount of light, that is, if exposure control is performed by adjusting the exposure time, the components related to the aperture can be reduced, and the digital camera The overall configuration can also be simplified. Further, if the exposure time for capturing a moving image can be shortened to such an extent that a still image is captured, it is possible to record each frame of a moving image as a still image with little blur. However, when performing exposure control by adjusting the exposure time, it may be assumed that the exposure time becomes extremely short compared to the frame period, and the motion of the moving image may be discontinuous and give an unnatural impression.

  In addition, in the method of generating image data between successive frames using a motion vector and converting the frame rate of a moving image, it is necessary to generate image data that interpolates between frames, so that the amount of data is inevitably generated. In addition, there is a problem that the process of interpolating using image data becomes complicated.

  The present invention has been made in consideration of the above points, and a moving image processing apparatus and moving image that can alleviate the motion of giving a discontinuous impression that may occur when the exposure time is extremely short compared to the frame period. It is an object to provide a processing method and a moving image processing program.

  In order to solve the above problems, the invention of claim 1 is applied to a moving image processing apparatus, and a motion detection unit that detects a motion between frames from moving image data, and a filter coefficient based on a detection result of the motion detection unit. A filter coefficient generation unit that generates the image data, and a filter that averages pixel values of the moving image data along a path of movement between the frames by filtering the moving image data using the filter coefficient.

  According to a second aspect of the present invention, in the configuration of the first aspect, the moving image data output from the filter is recorded, and a recording unit that records one frame of the moving image data as a still image.

  According to a third aspect of the present invention, in the configuration according to the first or second aspect, the motion detecting unit detects a motion between the frames by detecting a global motion vector of the moving image data.

  According to a fourth aspect of the present invention, in the configuration of the first aspect, the motion detection unit includes at least a motion from the previous frame in the previous frame, a motion from the previous frame in the current frame, and a current frame from the current frame. The motion is approximated by a curve to detect the motion between the frames.

  According to a fifth aspect of the present invention, in the configuration of the first aspect, the motion detecting unit detects a motion between the frames by a gyro sensor.

  According to a sixth aspect of the present invention, in the configuration of the fifth aspect, the motion detection unit repeatedly detects a motion of the moving image data at a time interval shorter than a frame period of the moving image data, and approximates the detection result to a curve. Detect motion between frames.

  According to a seventh aspect of the present invention, in any one of the first to sixth aspects, the difference calculation unit that detects difference information from the previous frame of the moving image data, and the filter coefficient according to the difference information And a coefficient correction unit for correcting.

  The invention according to claim 8 is the structure according to any one of claims 1 to 7, further comprising a high frequency component removing unit that removes a high frequency component from the moving image data, wherein the motion detecting unit is the high frequency component removing unit. The motion between frames is detected from the moving image data from which the high-frequency components have been removed.

  The invention according to claim 9 is applied to a moving image processing method to detect a motion between frames from moving image data, and a filter coefficient generation step to generate a filter coefficient based on the detection result of the motion detection step. And a filtering step of averaging the pixel values of the moving image data in the path of movement between the frames by filtering the moving image data with the filter coefficient.

  The invention of claim 10 is applied to a program for a moving image processing method to detect a motion between frames from moving image data, and a filter coefficient for generating a filter coefficient based on a detection result of the motion detection step. A generating step; and a filtering step of averaging pixel values of the moving image data along a movement path between the frames by filtering the moving image data using the filter coefficient.

  According to the present invention, it is possible to alleviate a blurring path that causes a motion that gives a discontinuous impression during reproduction, which may occur when the exposure time is extremely short compared to the frame period.

1 is a block diagram illustrating a digital camera according to a first embodiment of the present invention. It is a functional block diagram of CPU in the digital camera of FIG. It is a figure explaining the image processing in the digital camera of FIG. It is a functional block diagram of CPU in the digital camera which concerns on the 2nd Embodiment of this invention. It is a figure explaining the image processing in the digital camera of FIG. It is a figure explaining the filter coefficient in the digital camera of FIG. It is a block diagram which shows the digital camera which concerns on the 3rd Embodiment of this invention. It is a figure explaining the image processing of the digital camera of FIG. It is a functional block diagram of CPU in the digital camera which concerns on the 4th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

  (1) First embodiment

<First Embodiment>
FIG. 1 is a block diagram showing an electrical configuration of a digital camera which is an embodiment of the imaging apparatus according to the first embodiment of the present invention. This digital camera is provided with a recording mode for shooting as a basic operation mode and a playback mode for playing back a shot image.

  The digital camera 1 includes a camera unit 2, a storage medium 3, a bus 4, a CPU (Central Processing Unit) 5, a buffer memory 6, an image display unit 7, a memory 8, a key input unit 9, a subject detection unit 17, an SRAM 18, and a matching unit. 19. An image deformation synthesis adding unit 20 and the like are provided, and moving image data output from the camera unit 2 is recorded and reproduced.

  Here, the camera unit 2 includes a lens driving block 10, a lens 11, an aperture 12, an image sensor 13, a driver 14, a TG (Timing Generator) 15, a signal processing unit 16, and the like, and outputs moving image data D1. That is, the camera unit 2 sets the focus and magnification of the lens 11 with the lens driving block 10 under the control of the CPU 5 via the bus 4, and condenses incident light on the imaging surface of the imaging device 13 with the lens 10. The imaging device 13 is a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and photoelectrically converts an optical image formed on the imaging surface. The signal processing unit 16 performs correlated double sampling processing (CDS: Correlated Double Sampling) on the output signal of the image sensor 13, and then performs automatic gain adjustment (AGC) and analog-to-digital conversion processing (AD: Analog to Digital converter). The moving image data D1 is output to the bus 4. The TG 15 outputs a timing signal to each unit under the control of the CPU 5 via the bus 4, and the driver 14 generates a drive signal for the image sensor 13 from the timing signal output from the TG 15.

  The CPU 5 is arithmetic processing means for executing a program stored in the memory 8, and executes control of the digital camera 1, processing of moving image data, etc. in response to an operation of the key input unit 9. That is, when the reproduction mode is selected by the key input unit 9, the CPU 5 reproduces moving image data and still image data from the recording medium 3, decompresses the data, and displays them on the image display unit 7. Here, the recording medium 3 is, for example, a memory card, and the image display unit 7 is, for example, a liquid crystal display device. When the recording mode is selected by the key input unit 9, the moving image data D 1 output from the camera unit 2 is converted into YUV data, and then compressed into, for example, MPEG and recorded in the storage medium 3. When a predetermined operator of the key input unit 9 is operated while shooting a moving image, one frame of the moving image data D1 is compressed by JPEG and recorded in the storage medium 3 as a still image file. The recording medium 3 constitutes a recording unit. The buffer memory 6 is a work area for the CPU 5 and temporarily stores moving image data D1 output from the camera unit 2, data compression, decompressed moving image data, and the like.

  In this digital camera 1, the diaphragm 12 of the camera unit 2 is configured to be simple enough to be used in a still image camera. For example, the amount of incident light on the image sensor 13 is limited by switching the aperture step by step by a user's manual operation. .

  Corresponding to the configuration of the diaphragm 12, the CPU 5 controls exposure by adjusting the exposure time. That is, the CPU 5 detects the brightness level from the image data D1 output from the camera unit 2 to detect an evaluation value for exposure control, and outputs it from the TG 15 so that the evaluation value for exposure control becomes a predetermined value. The drive of the image sensor 13 by the driver 14 is controlled by adjusting the timing signal. Note that this evaluation value generation method for exposure control can be applied to an evaluation value generation method based on various photometry methods such as center-weighted photometry and spot photometry.

  Accordingly, when the subject is bright, the image data D1 output from the camera unit 2 has an extremely short exposure time compared to the frame period, and the motion is jerky and discontinuous. Therefore, in the digital camera 1, a panning mode is provided as one of the operation modes in the recording mode so as to be selectable by the key input unit 9. In the digital camera 1, in the panning mode, the moving image data D1 is image-processed by the CPU 5 to relieve the jerky movement. When a still image is recorded, the moving image data D1 before image processing is recorded on the storage medium 3, and a sharp still image with little motion blur is recorded.

  Furthermore, the subject detection unit 17, SRAM 18, matching unit 19, and image deformation synthesis addition unit 20 are connected to the CPU 5.

  The subject detection unit 17 functions as a detection unit of the present invention that separately detects a main subject and a background other than the main subject from the subject image. The main subject is the amount of movement based on the object recognized when the photographer presses the shutter key halfway so that the photographer is positioned at the center of the frame during panning, and the motion vector detected for each block. An object recognized as having a small is recognized as a feature. Note that the CPU 5 may perform the process of detecting the feature portion from the captured subject image by causing the detection program to be executed, or may be performed by a filtering process such as a Roberts filter. The subject detection unit 17 constitutes a motion detection unit of the present invention.

  When shooting in the recording mode, the matching unit 19 uses the SRAM 18 as a working memory, performs matching between subject images captured by the image sensor 13, and changes with the subject image before and after the change before and after the change. A motion vector indicating the relative change amount and change direction of the subsequent subject image is acquired and output to the CPU 5. The matching is performed for each block obtained by dividing the subject image into a plurality of blocks such as 3 × 3 and 4 × 4.

  The image deformation synthesis adding unit 20 is an image generation processing unit according to the present invention that generates a panning image by applying a deformation to the subject image captured by the image sensor 13 in accordance with an instruction from the CPU 5 and applying deformation. The image deformation synthesis adding unit 20 includes LPFs 5A to 5D that are low-pass filters. The image deformation synthesis adding unit 20 constitutes a filter of the present invention. The LPFs 5A to 5D constitute a high-frequency component removal unit, and the CPU 5 and the image deformation synthesis addition unit 20 constitute a filter coefficient generation unit.

  Next, in the digital camera having the above configuration, for example, an operation when the panning mode which is a lower mode of the recording mode is set will be described.

  FIG. 2 is a functional block diagram of the CPU 5 in the digital camera according to the first embodiment of this invention. Here, when the panning mode is set by the photographer when the recording mode is set, the CPU 5 controls the image deformation / synthesis adding unit 20 and uses the low-pass filter included in the image deformation / synthesis adding unit 20. A description will be given centering on processing performed on two or more images including moving image data D1A and D1B of the previous frame image and the current frame image.

  The CPU 5 controls the image deformation synthesis adder 20 to band-limit the moving image data D1A and D1B of the previous frame image and the current frame image by the low-pass filters 5A and 5B indicated by LPF to remove high-frequency components, A previous frame LPF image D2A and a current frame LPF image D2B composed of frequency components are generated. In the moving image data D2A and D2B from which the high-frequency component has been removed by the low-pass filter, detailed information such as the edge of the subject included in the moving image data D1A and D1B is reduced. Therefore, by processing with the low-pass filter, the CPU 5 can smooth the motion of the moving image between the moving image data D2A and D2B. In addition, a minute spatial change in the details of an image is likely to be erroneously detected as a motion. Accordingly, it is possible to reduce false detection of motion caused by minute spatial changes. In order to facilitate understanding, the processing of the previous frame image and the current frame image has been described, but moving image data is processed continuously. Therefore, the CPU 5 continues processing while constantly updating the current frame LPF image D2B created in the immediately preceding frame as the previous frame LPF image D2A.

  The CPU 5 detects the movement between frames for each frame of the moving image data D1. Further, the moving image data D1 is subjected to filtering processing by setting the filter characteristics in accordance with the movement, and the pixel value is averaged based on the movement path between frames (hereinafter referred to as blurring path) as will be described in detail later. Turn into. As a result, the CPU 5 creates a blurred image blurred along the blurring path, and smoothes the motion of the moving image by relaxing the blurring path that causes a motion that gives a discontinuous impression during reproduction.

  In the digital camera 1, the motion between the frames is detected using a global motion vector indicating the amount of deviation from the previous frame to the current frame, and the global motion vector itself is set as a blur path. A simple averaging process is applied to the averaging process.

  The CPU 5 calculates an inter-frame GMV (GMV: global motion vector) from the previous frame LPF image D2A and the current frame LPF image D2B (5C). For the calculation method of inter-frame GMV, various calculation methods such as a block matching method that calculates a motion vector by performing a matching process in units of blocks, a representative point matching method that calculates a motion vector by matching representative points, and the like are applied. Can do.

  The CPU 5 sets a filter coefficient in accordance with the blur path (5D), filters the image data D1B of the current frame with the filter coefficient, and records it on the recording medium 3 (5E, D3).

  Here, since the blur path is a path of movement from the previous frame to the current frame, the incident light of the pixels existing in the blur path is sequentially applied to each pixel of the image sensor 13 from the previous frame to the current frame. It is incident. Therefore, when the exposure time is a frame period, the image data D1 of the current frame is exposed to incident light of pixels existing in these blur paths, and as a result, smooth movement is ensured.

  On the other hand, when the exposure time is extremely short compared with the frame period, only the incident light of the neighboring pixels among the incident light of the pixels existing in the blur path is selectively exposed, and as a result, the motion It feels crisp. Therefore, if the pixel values of the blur path are averaged, the jerky movement can be alleviated.

  Specifically, as shown in FIG. 3, it is assumed that the blurring path is “10 pixels rightward in the horizontal direction”. In this case, incident light for “10 pixels horizontally rightward” sequentially enters the pixels of the image sensor 13 corresponding to the specific pixel A of the current frame from the previous frame to the current frame. Therefore, when the exposure time is a frame period, even the incident light corresponding to “10 pixels rightward in the horizontal direction” is exposed.

  On the other hand, when the exposure time is extremely shorter than the frame period, only the incident light of the neighboring pixels among these “horizontal rightward 10 pixels” is exposed. Therefore, in this case, by averaging the pixel values of the “horizontal rightward 10 pixels” from the specific pixel A in the current frame, the discontinuous motion that is jerky is relieved, and a moving image with smooth motion is obtained. be able to.

  Thus, the CPU 5 performs the filtering process on the moving image data of the current frame by a convolution operation using a two-dimensional filter centered on the processing target pixel. The coefficient of this filter is set to a coefficient that simply averages the pixel values of the blur path. Therefore, in the example of FIG. 3, the tap coefficients for 10 pixels in the horizontal direction from the processing target pixel A are set to the tap coefficients represented by the coefficient matrix Coef = (1111111111) / 10, and the remaining tap coefficients are set to the value 0. Set to.

  In addition, when recording a still image in the panning mode, the CPU 5 does not apply a filter so that the current frame image D1B before the filtering process is recorded on the storage medium 3 with the current frame still image DM.

  According to this embodiment, the exposure time is compared with the frame period by detecting the motion between frames, setting the filter coefficient according to this motion, and averaging the pixel values of the current frame in the motion path Even in an extremely short time, a blurring path that causes a motion that gives a discontinuous impression during reproduction can be relaxed by image processing, and a moving image with smooth motion can be captured.

  In addition, according to the present embodiment, by further controlling exposure by adjusting the exposure time, it is possible to simplify the configuration of the aperture and obtain a moving image with smooth motion, so that the components related to the aperture can be reduced. The overall configuration of the digital camera can be simplified.

  Further, according to the present embodiment, by recording one frame of a moving image before image processing as a still image, it is possible to record a sharp still image with little motion blur, and to record a sharp still image. And recording of a moving image with smooth motion can be realized at the same time.

  Further, according to the present embodiment, it is possible to capture a moving image with smooth motion by detecting motion between frames by processing moving image data, without requiring a special configuration for motion detection such as a gyro. can do.

  The digital camera according to the first embodiment has been described above, but the present invention is not limited to this. In the above-described embodiment, for example, the case where the blur path is linear has been described, but the blur path may be nonlinear. This will be described in the next embodiment.

  (2) Second embodiment

  FIG. 4 is a functional block diagram of the CPU 5 in the digital camera according to the second embodiment of the present invention. Also in this case, when the panning mode is set by the photographer when the recording mode is set, the CPU 5 controls the image deformation / synthesis adding unit 20 and uses the low-pass filter included in the image deformation / synthesis adding unit 20. A description will be given centering on processing performed on two or more images including moving image data D1A and D1B of the previous frame image and the current frame image.

  The digital camera of this embodiment is the same as that of the first embodiment except that the configuration shown in FIG. 4 is different from the digital camera of the first embodiment. Similar components are denoted by the same reference numerals, and description thereof is omitted.

  In the present embodiment, the blur path is not limited to a linear shape, and may be a curved line. For this reason, the blur path is calculated by approximating the inter-frame GMV detected in successive frames. In the present embodiment, a spline function is applied to this curve approximation. The CPU 5 calculates the blurring path by performing spline interpolation on the inter-frame GMV detected in successive frames. Note that various curves such as a Bezier curve and a quadratic function curve can be applied to the approximate curve.

  Therefore, as shown by the low-pass filters 5A, 5B, 5C, and 5D, the CPU 5 detects the previous frame image (frame (i-2) image), the previous frame image (frame (i-1) image), and the current frame image ( Frame (i-2) consisting of low-frequency components by band-limiting the moving image data D1A, D1B, D1C, D1D of the frame (i) image) and the next frame image (frame (i + 1) image) An LPF image D2A, a frame (i-1) LPF image D2B, a frame (i) LPF image D2C, and a frame (i + 1) LPF image D2D are generated.

  The CPU 5 calculates an interframe GMV0, an interframe GMV1, and an interframe GMV2 among these four LPF images D2A to D2D (5E, 5F, and 5G).

  Further, as shown in FIG. 5, the CPU 5 calculates the relative position between the previous frame and the current frame using the inter-frame GMV1 obtained from the previous frame image and the current frame image. In addition, the gradient of the blur path in the previous frame and the current frame is expressed by an average value of GMV between frames of the previous frame and the previous frame ((GMV0 + GMV1) / 2), and an average value of GMV between frames of the current frame and the next frame ((GMV1 + GMV2)). / 2). The CPU 5 calculates a blur path by spline interpolation using these relative positions and inclinations (5H).

  Further, as shown in FIG. 6, the pixel value of the blur path described with reference to FIG. 5 is averaged by a simple average to set a filter coefficient (5I), and the moving image data of the current frame is filtered by this filter coefficient Then, a blurred image is created (5J), and this blurred image is recorded (D3).

  In addition, when recording a still image in the panning mode, the CPU 5 does not apply the filter so that the current frame image D1C before the filtering process is recorded on the storage medium 3 by the current frame still image DM.

  In the present embodiment, the motion between frames detected in consecutive frames is not limited to a linear motion, and even if the motion between frames is non-linear, continuous blur images can be smoothed by approximating the curve. Therefore, it is possible to create a moving image with a smooth movement, so that a moving image with a smoother movement can be provided.

  The digital camera according to the first and second embodiments has been described above, but the present invention is not limited to this. In the above-described embodiment, the blur path is detected using the motion vector, but may be detected using a measuring device such as a gyro sensor, for example. This will be described in the next embodiment.

  (3) Third embodiment

  FIG. 7 is a block diagram showing a digital camera according to the third embodiment of the present invention. The digital camera according to the present embodiment has a gyro sensor 22, and uses this gyro sensor 22 to detect the movement of the digital camera and calculate a blur path. Further, the digital camera according to the present embodiment repeatedly performs the motion detection process using the gyro sensor at a time interval shorter than one frame period to approximate the blurring path with a curve. As described above, the digital camera according to the present embodiment is the same as the digital camera according to the first or second embodiment except that the configuration regarding the blur path detection and the motion detection process is different. Similar components are denoted by the same reference numerals, and description thereof is omitted.

  As shown in FIG. 7, the gyro sensor is connected to the CPU 5 via the bus 4, and detects the movement of the moving image from the movement of the camera to calculate the blur path. In addition, the process at the time of camera-shake correction | amendment is applicable to the process which detects the motion of a moving image using this gyro sensor.

  FIG. 8 is a diagram for explaining the blur path calculation process. In this digital camera, for example, the frame period is divided into eight equal parts, the output signal of the gyro sensor is processed, and the movement of the camera is detected respectively.

  In the present embodiment, by detecting a motion between frames using a gyro sensor, it is possible to further simplify the processing of moving image data and obtain the same effect as that of the above-described embodiment.

  In addition, by repeatedly detecting motion at time intervals shorter than the frame period and approximating the blur path with a curve, the fine motion within the frame period is reflected in the filter coefficient setting, and a blur image close to the actual camera motion is generated. can do.

  (4) Fourth embodiment

  FIG. 9 is a functional block diagram of the CPU 5 in the digital camera according to the fourth embodiment of the present invention. The digital camera of the present embodiment is configured in the same way as the first embodiment except that the configuration shown in FIG. 9 is different.

  Here, for example, when the movement of the subject is tracked by panning, the inter-frame difference value is small in the pixel where the subject is photographed, and the filtering process using the blur path according to the above-described embodiment is performed on this pixel. The subject will be blurred. Therefore, in this case, the image quality can be improved by reducing the filtering strength.

  On the other hand, the inter-frame difference value is large in the pixel that has captured the background, and the blurring path that causes the motion that gives the discontinuous impression at the time of reproduction by executing the filtering process by the blurring path according to the above-described embodiment Can be relaxed.

  Therefore, in the present embodiment, difference information with respect to the previous frame is calculated (5F), a weighting coefficient is generated based on this difference information, and the filter coefficient is corrected (5G, 5H). More specifically, the smaller the difference value is, the closer the filter coefficient assigned to the blur path pixels other than the target pixel is to the value 0, thereby reducing the filtering strength. On the contrary, as the difference value is larger, the filter coefficient assigned to each pixel is brought closer to the value 1 and closer to the process by simple averaging, and the filtering strength is increased. The filtering strength may be adjusted by adjusting the number of pixels used for the filtering process. Alternatively, after the blur image is created, the filtering strength may be indirectly adjusted by weighted addition of the blur image and the original current frame image using a weighting coefficient based on the difference information. The CPU 5 constitutes a difference calculation unit and a coefficient correction unit of the present invention.

  In this embodiment, by adjusting the filtering strength based on the difference information from the previous frame, the blur of the main subject is reduced when the positional relationship between the main subject and the digital camera does not change as in the case of panning. It is possible to record a moving image with smooth motion.

  (5) Other embodiments

  In the above-described embodiment, the case where the motion between frames is calculated and the filter coefficient is set for each frame has been described. However, the present invention is not limited to this, and one screen is divided and generated for each of a plurality of frames. The filter coefficient may be set by calculating the motion between frames for each block and further for each pixel.

  For example, when a filter coefficient is set for each block, a motion vector representing each block is detected for each block by detecting a correlation with the previous frame, and a blur path is calculated for each block based on the motion vector. Should be set. When setting a filter coefficient for each pixel, a motion vector may be detected for each pixel, and a blur path may be calculated for each pixel using this motion vector to set the filter coefficient.

  In the above-described embodiment, the case where the filtering process is performed by the two-dimensional filter has been described. However, the present invention is not limited to this, and a moving image has a lot of movement in the horizontal direction. Then, filtering processing may be performed. If the filtering process is performed only in the horizontal direction as described above, continuous blurred images can be smoothly connected to obtain a moving image with smoother motion. In addition, it is possible to simplify the structure and relax a blurring path that causes a motion that gives a discontinuous impression during reproduction sufficiently in practice.

  In the above-described embodiment, the case where a blurred image is generated by simple averaging or by weighted averaging according to difference information has been described. However, the present invention is not limited to this, and for example, the filter is determined by the exposure time with respect to the frame period. The filtering characteristics may be adjusted according to various shooting conditions, such as when adjusting the characteristics.

  That is, a motion that gives a discontinuous impression when a moving image is reproduced becomes more intense as the exposure time becomes shorter than the frame period, and hardly occurs when the exposure time is approximately the frame period. Therefore, if the filtering process is simply performed by a simple average or the like, an excessive filtering process may occur. Therefore, if the filter characteristics are adjusted according to the exposure time with respect to the frame period, excessive filtering processing can be prevented and the image quality can be further improved.

  In the above-described embodiment, the case where the current frame is filtered by the motion path from the previous frame to the current frame has been described. However, the present invention is not limited to this, and the motion path from the current frame to the next frame is used. The current frame may be subjected to filtering processing, and further, the movement path on the current frame side 1/2 of the movement path from the previous frame to the current frame, and the current frame side 1 of the movement path from the current frame to the next frame The current frame may be filtered according to the / 2 motion path.

  In the above-described embodiment, the case where the exposure control is performed by adjusting the exposure time has been described. However, the present invention is not limited to this and can be widely applied to the case where the exposure control is performed by adjusting the aperture. For ease of explanation, the case of panning shot shooting has been described as an example. However, the present invention is not limited to panning shot shooting but can be applied to processing of moving images.

  In the above-described embodiment, the case where the present invention is applied to a digital camera and moving image data is input and processed from the camera unit has been described. However, the present invention is not limited to this, for example, from a network or various recording media. The present invention can be widely applied to a case where moving image data is input and displayed.

  Further, in the above-described embodiment, the case where the present invention is applied to a digital camera and the moving image data D1 input from the imaging system is processed has been described. However, the present invention is not limited to this. The present invention can be widely applied to a case where moving image data D1 provided by a medium is recorded on a recording medium such as a hard disk. Further, it can be operated by a moving image processing program that causes a computer included in an imaging apparatus including a CPU and a memory to function as each unit described above. The moving image processing program can be distributed via a communication line, or can be distributed by writing on a recording medium such as a CD-ROM.

  The present invention is not limited to the above-described embodiments, and the above-described embodiments can be variously combined and various modifications can be made to the above-described embodiments without departing from the spirit of the present invention. It can be set as the form which added.

    The present invention can be applied to, for example, a digital camera.

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Camera part, 3 ... Storage medium, 5 ... CPU, 6 ... Buffer memory, 13 ... Image sensor, 16 ... Signal processing part

Claims (10)

A motion detector for detecting motion between frames from the video data;
A filter coefficient generation unit that generates a filter coefficient based on a detection result of the motion detection unit;
A moving image processing apparatus comprising: a filter that averages pixel values of the moving image data in a path of movement between the frames by filtering the moving image data with the filter coefficient.   The moving image processing apparatus according to claim 1, further comprising a recording unit that records moving image data output from the filter and records one frame of the moving image data as a still image.   The moving image processing apparatus according to claim 1, wherein the motion detection unit detects a motion between the frames by detecting a global motion vector of the moving image data.   The motion detection unit detects the motion between the frames by approximating at least the motion from the previous frame in the previous frame, the motion from the previous frame in the current frame, and the motion from the current frame in the next frame. The moving image processing apparatus according to claim 3, wherein:   The moving image processing apparatus according to claim 1, wherein the motion detection unit detects a motion between the frames by a gyro sensor.   6. The motion detection unit repeatedly detects motion of the moving image data at a time interval shorter than a frame period of the moving image data, and detects a motion between the frames by approximating a detection result to a curve. The moving image processing apparatus described in 1. A difference calculating unit for detecting difference information from the previous frame of the moving image data;
The moving image processing apparatus according to claim 1, further comprising: a coefficient correction unit that corrects the filter coefficient according to the difference information. It has a high frequency component removal unit that removes high frequency components from video data,
The moving image processing apparatus according to claim 1, wherein the motion detection unit detects a motion between frames from the moving image data from which the high frequency component is removed by the high frequency component removal unit. A motion detection step for detecting motion between frames from video data;
A filter coefficient generation step for generating a filter coefficient based on the detection result of the motion detection step;
And a filtering step of averaging pixel values of the moving image data along a path of motion between the frames by filtering the moving image data with the filter coefficient. A motion detection step for detecting motion between frames from video data;
A filter coefficient generation step for generating a filter coefficient based on the detection result of the motion detection step;
A moving image characterized by causing the arithmetic processing means to perform a filtering step of averaging pixel values of the moving image data in a path of movement between the frames by filtering the moving image data with the filter coefficient. Processing program.

Images