+

WO2018155087A1 - Image processing device, image forming device and image processing method - Google Patents

Image processing device, image forming device and image processing method Download PDF

Info

Publication number
WO2018155087A1
WO2018155087A1 PCT/JP2018/002735 JP2018002735W WO2018155087A1 WO 2018155087 A1 WO2018155087 A1 WO 2018155087A1 JP 2018002735 W JP2018002735 W JP 2018002735W WO 2018155087 A1 WO2018155087 A1 WO 2018155087A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
image
still image
unit
frame
Prior art date
Application number
PCT/JP2018/002735
Other languages
French (fr)
Japanese (ja)
Inventor
田中 邦彦
Original Assignee
京セラドキュメントソリューションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京セラドキュメントソリューションズ株式会社 filed Critical 京セラドキュメントソリューションズ株式会社
Priority to JP2019501159A priority Critical patent/JP6870728B2/en
Publication of WO2018155087A1 publication Critical patent/WO2018155087A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/91Television signal processing therefor

Definitions

  • the present invention relates to an image processing technique for extracting still image data from moving image data, and more particularly to a technique that can be used for creating an album.
  • Patent Document 1 discloses a technique for automatically creating an album by detecting an image of a predetermined person from a moving image and selecting a representative part from a plurality of frames in which the person's image is reflected. is suggesting.
  • the moving image data is simply regarded as a set of a plurality of frame image data. For this reason, sufficient studies have not been made in terms of extracting still image data by taking advantage of temporal continuity, which is a characteristic of the moving image data.
  • the present disclosure has been made in view of such a situation, and an object thereof is to provide a technique for extracting still image data by taking advantage of temporal continuity which is a characteristic of the moving image data.
  • the image processing apparatus includes a still image data generation unit and a continuous still image data extraction unit.
  • the still image data generation unit generates a plurality of still image data from moving image data.
  • the continuous still image data extraction unit is still image data that includes images of the same subject from the plurality of generated still image data and that is temporally continuous in a predetermined cycle, and the same Continuous still image data in which the movement of the subject image satisfies a preset condition is extracted.
  • An image forming apparatus includes the image processing apparatus and an image forming unit that forms an image on a print medium.
  • An image processing method generates a plurality of still image data from moving image data, includes an image of the same subject from the plurality of generated still image data, and has a predetermined cycle. And extracting continuous still image data that is temporally continuous still image data and that satisfies the preset condition of the motion of the same subject image.
  • FIG. 2 is a block diagram illustrating a functional configuration of an image forming apparatus 100 according to an embodiment of the present disclosure. It is a flowchart which shows the content of the still image acquisition process which concerns on one Embodiment. It is a flowchart which shows the content of the person registration process which concerns on one Embodiment. It is a data flow diagram which shows the content of the frame image data generation process which concerns on one Embodiment. It is explanatory drawing which shows the content of several frame image data containing the image of the same person. It is explanatory drawing which shows the content of two frame image data F1, F2 containing the image of the same person. It is explanatory drawing which shows the content of the frame image data which the to-be-photographed object is moving in the perspective direction.
  • the image forming apparatus 100 includes a control unit 110, an image forming unit 120, an operation display unit 130, a storage unit 140, and a communication interface unit 150.
  • the communication interface unit 150 is also called a communication I / F unit.
  • the control unit 110, the image forming unit 120, the operation display unit 130, the storage unit 140, and the communication interface unit 150 are examples of an image processing device, a print processing device, an operation display device, a storage device, and a communication interface device, respectively.
  • the image forming apparatus 100 is connected to the smartphone 200 through short-range wireless communication via the communication interface unit 150. Thereby, the image forming apparatus 100 can receive the moving image data generated by imaging with the smartphone 200.
  • CLASS 2 of BLUETOOTH registered trademark
  • CLASS 2 of BLUETOOTH registered trademark
  • CLASS 2 of BLUETOOTH is a communication with an output of 2.5 mW, and is a short-range wireless communication that enables communication between the image forming apparatus 100 and the smartphone 200 within about 10 m.
  • the control unit 110 includes storage means such as RAM and ROM, and a processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit).
  • a processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit).
  • the processor is an example of a control unit.
  • the control unit 110 also has controller functions related to various I / O, USB (Universal Serial Bus), bus, and other hardware interfaces.
  • the control unit 110 controls the entire image forming apparatus 100.
  • the control unit 110 further includes a frame extraction unit 111, a person verification unit 112, a speed detection unit 113, a high-speed moving section extraction unit 114, a still image output unit 115, and a cycle setting unit 116.
  • the person verification unit 112 includes a person registration unit 112a.
  • the processor executes a program stored in the ROM or the like.
  • the control unit 110 functions as a frame extraction unit 111, a person verification unit 112, a speed detection unit 113, a high-speed moving section extraction unit 114, a still image output unit 115, and a period setting unit 116.
  • the frame extraction unit 111, the person verification unit 112, the speed detection unit 113, the high-speed movement section extraction unit 114, the still image output unit 115, and the period setting unit 116 are respectively a frame extraction device, a person verification device, a speed detection device, and a high-speed movement section It is an example of an extracting device, a still image output device, and a period setting device.
  • the image forming unit 120 forms an image on a sheet or other sheet-like print medium.
  • the operation display unit 130 functions as a touch panel including an operation unit and a display unit.
  • the operation display unit 130 displays various menu screens as input screens on the display unit, and receives user operation inputs through the operation unit.
  • the storage unit 140 is a storage device including a hard disk drive or flash memory which is a non-temporary recording medium.
  • the storage unit 140 stores a control program and data corresponding to processing executed by the control unit 110, respectively.
  • the storage unit 140 includes a frame memory 141 for temporarily storing frame image data, a still image storage area 142, and a person registration data storage area 143.
  • the storage area is a part of the data storage unit in the storage unit 140.
  • the frame memory 141, the still image storage area 142, and the person registration data storage area 143 are part of a data storage unit in one storage device.
  • the still image storage area 142 and the person registration data storage area 143 may be separate storage devices.
  • the control unit 110 can execute a still image acquisition process according to an embodiment. 2, S10, S20, S30,... Are identification codes of a plurality of steps in the still image acquisition process.
  • FIG. 2 is a flowchart showing the content of a still image acquisition process according to an embodiment, as shown in FIG.
  • the person verification unit 112 executes a person registration process that involves the use of the operation display unit 130 by the user.
  • the person registration unit 112a can register information about a person to be detected in a still image to be extracted as one of the conditions for extracting still image data from moving image data. .
  • S11, S12, S13,... are identification codes of a plurality of steps in the still image acquisition process.
  • FIG. 3 is a flowchart showing the contents of person registration processing according to an embodiment, as shown in FIG.
  • the person registration unit 112a executes a moving image data take-in process.
  • the person registration unit 112a selects the moving image data MD according to the operation of the operation display unit 130 by the user.
  • the person registration unit 112a can capture the moving image data MD into the image forming apparatus 100 through, for example, a wireless communication device (not shown) or a portable storage medium (not shown).
  • step S12 the person verification unit 112 of the control unit 110 executes a person detection process.
  • the person verification unit 112 generates frame image data from the moving image data MD.
  • the person collation unit 112 extracts a person detection area, which is an image area having the characteristics of a person, from a still image represented by the frame image data file.
  • the person collation unit 112 can extract a person detection area using machine learning such as SVM (Support Vector Machine) based on, for example, HOG (Histograms of Oriented Gradients) features.
  • SVM Small Vector Machine
  • HOG Hemograms of Oriented Gradients
  • step S13 the person verification unit 112 executes a person classification process.
  • the person verification unit 112 classifies the person in the person detection area by determining which of the family images registered in advance is the person image in the person detection area, for example. .
  • family information includes information on father A, mother B, son C, and daughter D.
  • the person registration unit 112a registers the family information in the storage unit 140 in advance in response to an operation on the operation display unit 130 by the user.
  • the person verification unit 112 selects frame image data including a face image having a size larger than a preset image size. Furthermore, the person verification unit 112 classifies the selected frame image data into a plurality of groups according to the classification result of the person classification process, and causes the operation display unit 130 to display the group image data.
  • the person verification unit 112 determines whether each of the plurality of groups corresponds to the father A, the mother B, the son C, the daughter D, or another person according to the input operation to the operation display unit 130 by the user. Can be corrected.
  • the user can perform an operation to correct a misrecognition that a still image of father A is included in the group of son C, for example. Thereby, the person collation part 112 can improve the precision of machine learning.
  • the person verification unit 112 generates a database using the father A, mother B, son C, and daughter D as records.
  • HOG feature amounts of face images of father A, mother B, son C, and daughter D are registered.
  • step S14 the person collation unit 112 executes a clothing selection process.
  • the person collation unit 112 extracts HOG feature values for the clothing worn by each of the father A, mother B, son C, and daughter D from the frame image data.
  • the person collation part 112 can specify a person using the HOG feature-value of the clothes image in addition to the HOG feature-value of the face image of father A, mother B, son C, and daughter D. This is because each person often wears the same clothes, and different persons tend to wear different clothes.
  • step S15 the person registration unit 112a of the person verification unit 112 executes a database registration process.
  • the person registration unit 112 a stores a database for the father A, mother B, son C, and daughter D in the person registration data storage area 143 of the storage unit 140.
  • the database includes HOG feature values of face images, HOG feature values of clothes images, machine learning data of face images, machine learning data of clothes images, and height and other attribute data that can be input by the user.
  • the person registration unit 112a can also register the face image and clothing image data of each person using still image data captured by a digital camera in accordance with a user operation.
  • the person registration unit 112a can generate the HOG feature value of the face image and the HOG feature value of the clothes image by using such image data, and register them in the database.
  • the HOG feature amount is generated based on YUV image data with a small calculation load in image recognition.
  • step S ⁇ b> 20 the user sets a still image extraction mode via the operation display unit 130.
  • the still image extraction mode includes, for example, a continuous photo mode in which lively continuous photos are extracted, in addition to various modes such as a mode for extracting a close-up photo of a person's face and a mode for extracting a group photo. Yes.
  • the control unit 110 causes the operation display unit 130 to display a screen (not shown) that accepts selection of each mode.
  • the cycle setting unit 116 causes the operation display unit 130 to display an operation display screen (not shown) that receives a cycle setting input.
  • the cycle setting input is an input operation for setting a cycle, which is a time interval between frame images. The period is used for setting an extraction period of a frame image as a still image. However, if no user input is made, 0.2 seconds is used as the initial setting.
  • step S30 the person verification unit 112 selects a person through the operation of the operation display unit 130 by the user.
  • the person verification unit 112 causes the operation display unit 130 to display a screen for accepting selection of father A, mother B, son C, and daughter D. In this example, it is assumed that son C is selected.
  • the frame extraction unit 111 executes a frame image generation process.
  • the frame extraction unit 111 is an example of an apparatus that functions as a still image data generation unit.
  • the frame extraction unit 111 generates frame image data having a predetermined cycle from moving image data MD having a frame rate of 30 fps, for example.
  • the predetermined period is set in advance by the user, for example. For example, it is conceivable that the predetermined cycle of the initial setting is 0.2 seconds.
  • FIG. 4 is a data flow diagram showing the contents of frame image data generation processing according to an embodiment.
  • a data flow diagram is shown on the upper side, and GOP (Group of Pictures) is shown on the lower side.
  • the data flow diagram shows a flow from extraction of frame image data from moving image data MD to conversion and storage of the extracted data.
  • the frame image data is configured as YUV image data.
  • the frame image data generation process is a process for extracting a plurality of frame image data from the moving image data MD, and is executed by the frame extraction unit 111.
  • Frame image data generation processing includes, for example, MPEG-4 (ISO / IEC 14496) and H.264. H.264 is included.
  • the frame extraction unit 111 generates frame image data from an I frame (Intra-coded Frame), a P frame (Predicted Frame), and a B frame (Bi-directional Predicted Frame).
  • An I frame is a frame that is encoded without using inter-frame prediction.
  • the I frame is also called an intra frame or a key frame.
  • the I frame constitutes a GOP together with a P frame (Predicted Frame) and a B frame (Bi-directional Predicted Frame).
  • Frame image data can be generated by subjecting the P frame to inter-frame processing with the I frame.
  • Frame image data can be generated by subjecting the B frame to inter-frame processing with the I frame, P frame, and other B frames.
  • the moving image data is generated from a plurality of frame image data arranged in time series.
  • a plurality of frame image data is often approximated between frames before and after time series.
  • Inter-frame prediction is a process that uses such characteristics of moving image data.
  • Inter-frame prediction is a process of predicting the current frame image from the previous frame image in time series.
  • the movement for each pixel block is estimated, and further, the difference of the pixel block between the frames after the movement is DCT transformed and quantized. Thereby, the compression rate per GOP increases.
  • P frames are reconstructed from I frames by using motion vectors.
  • the motion vector is a movement vector of each pixel block.
  • the frame extraction unit 111 generates frame image data as YUV image data including luminance data and color difference data by performing inverse discrete cosine transform (also called inverse DCT transform) on the I frame.
  • the inverse DCT transform is executed for each 8 ⁇ 8 pixel or 16 ⁇ 16 pixel block, for example.
  • the frame extraction unit 111 stores the reproduced frame image data in the frame memory 141.
  • the frame extraction unit 111 generates difference data by performing inverse discrete cosine transform on the P frame and the B frame.
  • the frame extraction unit 111 generates frame image data by performing inter-frame processing using the difference data and the motion vector.
  • the motion vector is data generated when the moving image data MD is encoded. This processing is performed using MPEG-4 or H.264. H.264 is a normal decoding process.
  • the frame extraction unit 111 executes frame image data generation processing based on the P frame and the B frame on the RAM (not shown) of the control unit 110.
  • the frame extraction unit 111 stores frame image data having a preset period in the frame memory 141. Specifically, when the frame rate of the moving image data MD is 30 fps, the cycle is set to 0.2 seconds. In this case, the frame extraction unit 111 stores the frame image data in the frame memory 141 every six frames. On the other hand, the frame extraction unit 111 discards other frame image data. Thereby, the frame extraction unit 111 can reduce excessive consumption of the frame memory 141.
  • the frame extraction unit 111 stores the frame image data in the frame memory 141
  • the frame extraction unit 111 stores the motion vector used in generating the frame image data together with each frame image data.
  • step S50 the person verification unit 112 executes a person detection process.
  • the person verification unit 112 determines whether or not the person selected in step S30 is included for each of the plurality of frame image data stored in the frame memory 141.
  • the person selected in step S30 is the son C.
  • the person verification unit 112 tries to detect the son C from the frame image data as YUV image data including luminance data and color difference data.
  • the person collation unit 112 can detect and specify a person using, for example, the well-known OpenCV (Open Source Computer Vision Library). First, the person verification unit 112 detects a person from the frame image data. Furthermore, the person verification unit 112 determines whether the face of the person detected using the HOG feature amount of the son C face image is the face of the son C.
  • OpenCV Open Source Computer Vision Library
  • the person verification unit 112 uses the HOG feature value of the clothes image of the son C and determines whether or not the face of the person is the face of the son C. Determine whether.
  • the HOG feature amount of the clothes image can be used as an auxiliary. This is because, generally, when a continuous image with a dynamic feeling is taken, it is difficult to capture a face image stably.
  • step S60 the speed detector 113 of the controller 110 detects the moving speed of the person who is the subject.
  • the person is son C.
  • the speed detection unit 113 specifies a temporal section in which the same subject image is included in a plurality of temporally continuous frame image data.
  • the speed detection unit 113 can estimate the speed of the subject image using the motion vector of the pixel block that forms the image area including the same subject image, that is, the same person image. This is because the pixel block constituting the image area including the image of the son C has a motion vector corresponding to the translational movement speed of the son C.
  • FIG. 5 images of nine frame image data F1 to F9 in which the son C is photographed as the same person are shown.
  • the frame image data F1 to F9 represent a series of continuous images with a feeling of dynamism in which the son C is running against the background of three stationary trees.
  • FIG. 6 is an explanatory diagram showing the contents of two pieces of frame image data F1, F2 representing the same person.
  • the image of the son C is translated to the right side in FIG.
  • the translational movement amount of the image of the son C is a movement amount between the frames of the two pieces of frame image data F1 and F2.
  • the translational movement speed is estimated from a motion vector assuming 30 fps.
  • step S70 the high speed movement section extraction unit 114 of the control unit 110 determines whether or not the moving speed of the image of the son C is equal to or higher than a preset threshold value. If the moving speed of the son C image is equal to or higher than the threshold, the high-speed moving section extraction unit 114 proceeds to step S80. If the moving speed of the son C image is lower than the threshold, the high-speed moving section extraction unit 114 performs the process. Proceed to S90.
  • step S80 the high-speed movement section extraction unit 114 executes a frame image data grouping process.
  • the frame image data grouping process is frame image data that is temporally continuous in a predetermined cycle before and after the two pieces of frame image data F1 and F2, and includes the image of the son C.
  • FIG. 5 shows an example in which the photographing of the son C is started with the frame image data F1, and the photographing of the son C is continued until the frame image data F9.
  • the high-speed movement section extraction unit 114 groups the nine pieces of frame image data F1 to F9 and stores them in the still image storage area 142. As a result, the nine pieces of frame image data F1 to F9 can be managed and handled as one continuous image data file. In the still image storage area 142, nine frame image data F1 to F9 are DCT converted and stored as JPEG still image data.
  • the high-speed moving section extraction unit 114 is an example of a continuous still image data extraction unit.
  • Nine pieces of frame image data F1 to F9 are examples of continuous still image data.
  • the state in which the moving speed of the image of the subject (son C) is equal to or higher than the threshold may be instantaneous for one continuous image data file. That is, it is only necessary that the moving speed of the subject image is equal to or greater than the threshold value once in one continuous image data file. It is not necessary for the moving speed of the image to exceed the threshold in one continuous image data file. However, it is also conceivable that the high speed moving section extracting unit 114 determines that the moving speed exceeds a threshold in all sections of one continuous image data file as a necessary condition.
  • the control unit 110 repeatedly executes the processing from step S40 to step S80 for a plurality of frame image data until the final frame image data becomes a processing target (step S90).
  • step S100 the still image output unit 115 executes frame image data output processing.
  • the still image output unit 115 displays one continuous image data file on the operation display unit 130 as nine thumbnail images.
  • one continuous image data file includes nine grouped frame image data F1 to F9.
  • the control unit 110 selects an arbitrary frame image as a processing target from the nine pieces of frame image data F1 to F9 according to an operation of touching a thumbnail image by the user, and processes all the frame images at once. It can also be selected as a target.
  • the still image output unit 115 controls the image forming unit 120 based on the selected frame image data in accordance with a user instruction to execute processing for forming an image on the print medium. Can be made.
  • the printed matter output by the image forming unit 120 can be used for an album or the like as a continuous photograph with a lively feeling.
  • the still image output unit 115 can also transmit a continuous image data file to the smartphone 200 or a personal computer (not shown) via the communication interface unit 150 in response to an instruction operation by the user.
  • the temporal continuity that is the characteristic of the moving image data is utilized to obtain a continuous image (continuous photograph) with a dynamic feeling at a predetermined cycle.
  • a plurality of still image data (frame image data) can be extracted.
  • frame image data in which the translational movement speed of an image of a person as a subject exceeds a threshold is extracted as a continuous image with a sense of dynamism.
  • a continuous image with a feeling of dynamism is not limited to an image having a high translational movement speed.
  • an image showing a scene in which a subject approaches in the perspective direction is also included in a continuous image having a dynamic feeling.
  • the speed detection unit 113 makes a determination based on whether or not the change rate of the image size of the subject is equal to or greater than a preset threshold value.
  • the change rate of the image size of the subject is a reduction rate or an enlargement rate per unit time.
  • the motion vector associated with the pixel block constituting the image of the person as the subject has many components in the divergence direction when the person as the detection target approaches.
  • the motion vector has many components in the convergence direction when the person to be detected moves away.
  • Translational movement, perspective movement, and combinations thereof can be detected by analyzing the translational component, the divergent component, and the convergence component of the motion vector.
  • the speed detector 113 determines whether or not the image is a continuous image based on at least one of a translation component, a divergence component, and a convergence component of a motion vector associated with the moving speed of an image of a person as a subject. Can do.
  • frame image data is extracted as a continuous image with a sense of movement based on the moving speed of an image of a person as a subject.
  • a continuous image with a feeling of dynamism is not limited to an image having a high moving speed.
  • the image of the person as the subject is an image when the person's limb is moving or rotating without moving.
  • the control unit 110 may extract a continuous image including an image of a person who is moving or rotating a limb as a continuous image having a sense of movement.
  • a motion vector associated with a pixel block constituting an image of a person as a subject may have a random direction and size.
  • the speed detection unit 113 has a continuous image corresponding to the motion vector with a lively feeling. Can be determined.
  • a motion vector is used as data used for determining whether or not a continuous image is lively.
  • the motion vector is not necessarily used, and other data may be adopted.
  • the speed detection unit 113 may be configured to detect the movement speed of the person with reference to, for example, the position or size of the person area existing in each frame image. For example, the speed detection unit 113 calculates the difference in the position of the person region between temporally adjacent frame images as the movement distance, and if the distance is larger than a predetermined threshold, the person in the moving image moves at high speed. Can be determined.
  • the person area is an area including an image of a person.
  • the speed detection unit 113 compares not only the position of the person area but also the size of the person area. Even when the difference in size is larger than a predetermined threshold, the image of the person in the moving image moves at high speed. You may judge.
  • the method using the motion vector in the above embodiment has an advantage that the calculation load is small and the processing can be speeded up.
  • the method according to the present modification has an advantage that the degree of freedom in design of processing for determining whether or not a continuous image is lively is high.
  • one of the difference in the position of the person area and the difference in the size of the person area may be mainly used, and the other may be used in a complementary manner.
  • the speed detection unit 113 only needs to determine whether or not the movement of the image of the same subject satisfies a preset condition.
  • the present invention is applied to the image forming apparatus.
  • the present invention can also be applied to an apparatus that functions as an image processing apparatus such as a smartphone or a personal computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)

Abstract

The purpose of the present invention is to extract still image data by using temporal continuity that is a characteristic of moving image data. An image processing device (110) is provided with: a still image data generation unit (111) which generates, from moving image data, a plurality of pieces of still image data; and a continuous still image data extraction unit (114) which extracts, from the generated plurality of pieces of still image data, continuous still image data in which a movement of an identical subject satisfies a preset condition, as still image data which shows the identical subject and is temporally continuous in a prescribed period.

Description

画像処理装置、画像形成装置及び画像処理方法Image processing apparatus, image forming apparatus, and image processing method
 本発明は、動画像データから静止画像データを抽出する画像処理技術に関し、特にアルバムの作成等に利用可能な技術に関する。 The present invention relates to an image processing technique for extracting still image data from moving image data, and more particularly to a technique that can be used for creating an album.
 近年、ビデオカメラやスマートフォンの性能向上や画質向上に伴い、動画像データから抽出可能な静止画像データを生成することが行われている。 In recent years, with improvement in performance and image quality of video cameras and smartphones, still image data that can be extracted from moving image data has been generated.
 また、動画像データから所望の静止画を抽出し、抽出された前記静止画のアルバムなどを作成可能とする技術も提案されている。たとえば特許文献1は、動画中から所定の人物の画像を検出し、その人物の画像が写りこんでいる複数のフレームから代表的な一部を選択することでアルバムを自動的に作成する技術を提案している。 Also, a technique has been proposed in which a desired still image is extracted from moving image data and an album of the extracted still image can be created. For example, Patent Document 1 discloses a technique for automatically creating an album by detecting an image of a predetermined person from a moving image and selecting a representative part from a plurality of frames in which the person's image is reflected. is suggesting.
特開2009-88687号公報JP 2009-88687 A
 ところで、従来は、前記動画像データが単に複数のフレーム画像データの集合として捉えられている。そのため、前記動画像データの特性である時間的な連続性を活かして静止画像データを抽出するという観点での十分な検討が行われていなかった。 Incidentally, conventionally, the moving image data is simply regarded as a set of a plurality of frame image data. For this reason, sufficient studies have not been made in terms of extracting still image data by taking advantage of temporal continuity, which is a characteristic of the moving image data.
 本開示は、このような状況に鑑みてなされたものであり、前記動画像データの特性である時間的な連続性を活かして静止画像データを抽出する技術を提供することを目的とする。 The present disclosure has been made in view of such a situation, and an object thereof is to provide a technique for extracting still image data by taking advantage of temporal continuity which is a characteristic of the moving image data.
 本開示の一の局面に係る画像処理装置は、静止画像データ生成部と、連続静止画像データ抽出部と、を備える。前記静止画像データ生成部は、動画像データから複数の静止画像データを生成する。前記連続静止画像データ抽出部は、生成された前記複数の静止画像データから、同一の被写体の画像を含むとともに所定の周期で時間的に連続している静止画像データであり、かつ、前記同一の被写体の画像の動きが予め設定されている条件を満たしている連続静止画像データを抽出する。 The image processing apparatus according to one aspect of the present disclosure includes a still image data generation unit and a continuous still image data extraction unit. The still image data generation unit generates a plurality of still image data from moving image data. The continuous still image data extraction unit is still image data that includes images of the same subject from the plurality of generated still image data and that is temporally continuous in a predetermined cycle, and the same Continuous still image data in which the movement of the subject image satisfies a preset condition is extracted.
 本開示の他の局面に係る画像形成装置は、前記画像処理装置と、印刷媒体に画像を形成する画像形成部と、を備える。 An image forming apparatus according to another aspect of the present disclosure includes the image processing apparatus and an image forming unit that forms an image on a print medium.
 本開示の他の局面に係る画像処理方法は、動画像データから複数の静止画像データを生成することと、前記生成された複数の静止画像データから、同一の被写体の画像を含むとともに所定の周期で時間的に連続している静止画像データであり、かつ、前記同一の被写体の画像の動きが予め設定されている条件を満たしている連続静止画像データを抽出することと、を含む。 An image processing method according to another aspect of the present disclosure generates a plurality of still image data from moving image data, includes an image of the same subject from the plurality of generated still image data, and has a predetermined cycle. And extracting continuous still image data that is temporally continuous still image data and that satisfies the preset condition of the motion of the same subject image.
 本開示によれば、動画像データの特性である時間的な連続性を活かした静止画像データの抽出を実現する技術を提供することが可能である。 According to the present disclosure, it is possible to provide a technology that realizes the extraction of still image data that takes advantage of temporal continuity that is a characteristic of moving image data.
本開示の一実施形態に係る画像形成装置100の機能構成を示すブロックダイアグラムである。2 is a block diagram illustrating a functional configuration of an image forming apparatus 100 according to an embodiment of the present disclosure. 一実施形態に係る静止画像取得処理の内容を示すフローチャートである。It is a flowchart which shows the content of the still image acquisition process which concerns on one Embodiment. 一実施形態に係る人物登録処理の内容を示すフローチャートである。It is a flowchart which shows the content of the person registration process which concerns on one Embodiment. 一実施形態に係るフレーム画像データ生成処理の内容を示すデータフローダイアグラムである。It is a data flow diagram which shows the content of the frame image data generation process which concerns on one Embodiment. 同一人物の画像を含む複数のフレーム画像データの内容を示す説明図である。It is explanatory drawing which shows the content of several frame image data containing the image of the same person. 同一人物の画像を含む2枚のフレーム画像データF1,F2の内容を示す説明図である。It is explanatory drawing which shows the content of two frame image data F1, F2 containing the image of the same person. 被写体が遠近方向において移動しているフレーム画像データの内容を示す説明図である。It is explanatory drawing which shows the content of the frame image data which the to-be-photographed object is moving in the perspective direction.
 以下、本開示の実施形態を、図面を参照して説明する。なお、前記実施形態は、本開示を実施するための形態である。 Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In addition, the said embodiment is a form for implementing this indication.
 図1に示されるように、本開示の一実施形態に係る画像形成装置100の機能構成を示すブロックダイアグラムである。画像形成装置100は、制御部110と、画像形成部120と、操作表示部130と、記憶部140と、通信インターフェース部150とを備えている。通信インターフェース部150は、通信I/F部とも呼ばれる。 1 is a block diagram illustrating a functional configuration of an image forming apparatus 100 according to an embodiment of the present disclosure, as illustrated in FIG. The image forming apparatus 100 includes a control unit 110, an image forming unit 120, an operation display unit 130, a storage unit 140, and a communication interface unit 150. The communication interface unit 150 is also called a communication I / F unit.
 制御部110、画像形成部120、操作表示部130、記憶部140および通信インターフェース部150は、それぞれ画像処理装置、印刷処理装置、操作表示装置、記憶装置および通信インターフェース装置の一例である。 The control unit 110, the image forming unit 120, the operation display unit 130, the storage unit 140, and the communication interface unit 150 are examples of an image processing device, a print processing device, an operation display device, a storage device, and a communication interface device, respectively.
 画像形成装置100は、通信インターフェース部150を介して近距離無線通信でスマートフォン200と接続される。これにより、画像形成装置100は、スマートフォン200で撮像して生成された動画像データを受信することができる。 The image forming apparatus 100 is connected to the smartphone 200 through short-range wireless communication via the communication interface unit 150. Thereby, the image forming apparatus 100 can receive the moving image data generated by imaging with the smartphone 200.
 近距離無線通信は、本実施形態では、BLUETOOTH(登録商標)のCLASS2を使用している。BLUETOOTH(登録商標)のCLASS2は、出力2.5mWの通信であり、画像形成装置100とスマートフォン200との距離が10m以内程度での通信が可能な近距離無線通信である。 Near field communication uses CLASS 2 of BLUETOOTH (registered trademark) in this embodiment. CLASS 2 of BLUETOOTH (registered trademark) is a communication with an output of 2.5 mW, and is a short-range wireless communication that enables communication between the image forming apparatus 100 and the smartphone 200 within about 10 m.
 制御部110は、RAMやROM等の記憶手段、及びMPU(Micro Processing Unit)又はCPU(Central Processing Unit)等のプロセッサーを備えている。前記プロセッサーは、制御手段の一例である。 The control unit 110 includes storage means such as RAM and ROM, and a processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit). The processor is an example of a control unit.
 また、制御部110は、各種I/O、USB(ユニバーサル・シリアル・バス)、バス、その他ハードウェア等のインターフェースに関連するコントローラの機能を備えている。制御部110は、画像形成装置100の全体を制御する。 The control unit 110 also has controller functions related to various I / O, USB (Universal Serial Bus), bus, and other hardware interfaces. The control unit 110 controls the entire image forming apparatus 100.
 制御部110は、さらに、フレーム抽出部111と、人物照合部112と、速度検出部113と、高速移動区間抽出部114と、静止画像出力部115と、周期設定部116とを備えている。人物照合部112は、人物登録部112aを備えている。 The control unit 110 further includes a frame extraction unit 111, a person verification unit 112, a speed detection unit 113, a high-speed moving section extraction unit 114, a still image output unit 115, and a cycle setting unit 116. The person verification unit 112 includes a person registration unit 112a.
 なお、前記プロセッサーは、前記ROM等に記憶されたプログラムを実行する。これにより、制御部110は、フレーム抽出部111、人物照合部112、速度検出部113、高速移動区間抽出部114、静止画像出力部115および周期設定部116として機能する。 Note that the processor executes a program stored in the ROM or the like. Thereby, the control unit 110 functions as a frame extraction unit 111, a person verification unit 112, a speed detection unit 113, a high-speed moving section extraction unit 114, a still image output unit 115, and a period setting unit 116.
 フレーム抽出部111、人物照合部112、速度検出部113、高速移動区間抽出部114、静止画像出力部115および周期設定部116は、それぞれフレーム抽出装置、人物照合装置、速度検出装置、高速移動区間抽出装置、静止画像出力装置および周期設定装置の一例である。 The frame extraction unit 111, the person verification unit 112, the speed detection unit 113, the high-speed movement section extraction unit 114, the still image output unit 115, and the period setting unit 116 are respectively a frame extraction device, a person verification device, a speed detection device, and a high-speed movement section It is an example of an extracting device, a still image output device, and a period setting device.
 画像形成部120は、紙その他のシート状の印刷媒体上に画像を形成する。操作表示部130は、操作部及び表示部を含むタッチパネルとして機能する。操作表示部130は、様々なメニュー画面を入力画面として前記表示部に表示し、さらに前記操作部を通じてユーザーの操作入力を受け付ける。 The image forming unit 120 forms an image on a sheet or other sheet-like print medium. The operation display unit 130 functions as a touch panel including an operation unit and a display unit. The operation display unit 130 displays various menu screens as input screens on the display unit, and receives user operation inputs through the operation unit.
 記憶部140は、非一時的な記録媒体であるハードディスクドライブやフラッシュメモリー等からなる記憶装置である。 The storage unit 140 is a storage device including a hard disk drive or flash memory which is a non-temporary recording medium.
 記憶部140は、それぞれ制御部110が実行する処理に対応する制御プログラム及びデータを記憶する。記憶部140は、フレーム画像データを一時的に格納するためのフレームメモリ141と、静止画像格納領域142と、人物登録データ格納領域143とを有している。なお、格納領域とは、記憶部140におけるデータ記憶部の一部である。 The storage unit 140 stores a control program and data corresponding to processing executed by the control unit 110, respectively. The storage unit 140 includes a frame memory 141 for temporarily storing frame image data, a still image storage area 142, and a person registration data storage area 143. The storage area is a part of the data storage unit in the storage unit 140.
 フレームメモリ141、静止画像格納領域142および人物登録データ格納領域143が、1つの記憶装置におけるデータ記憶部の一部であることが考えられる。一方、静止画像格納領域142および人物登録データ格納領域143が、それぞれ個別の記憶装置であることも考えられる。 It is conceivable that the frame memory 141, the still image storage area 142, and the person registration data storage area 143 are part of a data storage unit in one storage device. On the other hand, the still image storage area 142 and the person registration data storage area 143 may be separate storage devices.
 制御部110は、一実施形態に係る静止画像取得処理を実行可能である。図2におけるS10,S20,S30,・・・は、前記静止画像取得処理における複数のステップの識別符号である。 The control unit 110 can execute a still image acquisition process according to an embodiment. 2, S10, S20, S30,... Are identification codes of a plurality of steps in the still image acquisition process.
 図2に示されるように、一実施形態に係る静止画像取得処理の内容を示すフローチャートである。前記静止画像取得処理のステップS10では、人物照合部112が、ユーザーによる操作表示部130の使用を伴う人物登録処理を実行する。前記人物登録処理では、人物登録部112aが、動画像データから静止画像データを抽出する際の条件の一つとして、抽出対象となる静止画像で検出されるべき人物の情報を登録することができる。 FIG. 2 is a flowchart showing the content of a still image acquisition process according to an embodiment, as shown in FIG. In step S10 of the still image acquisition process, the person verification unit 112 executes a person registration process that involves the use of the operation display unit 130 by the user. In the person registration process, the person registration unit 112a can register information about a person to be detected in a still image to be extracted as one of the conditions for extracting still image data from moving image data. .
 図3におけるS11,S12,S13,・・・は、前記静止画像取得処理における複数のステップの識別符号である。 3, S11, S12, S13,... Are identification codes of a plurality of steps in the still image acquisition process.
 図3に示されるように、一実施形態に係る人物登録処理の内容を示すフローチャートである。前記人物登録処理のステップS11では、人物登録部112aが、動画像データ取込処理を実行する。動画像データ取込処理では、人物登録部112aは、ユーザーによる操作表示部130の操作に従って、動画像データMDを選択する。さらに、人物登録部112aは、動画像データMDを、たとえば不図示のワイヤレス通信装置又は不図示の可搬記憶媒体を通じて、画像形成装置100に取り込むことが可能である。 FIG. 3 is a flowchart showing the contents of person registration processing according to an embodiment, as shown in FIG. In step S11 of the person registration process, the person registration unit 112a executes a moving image data take-in process. In the moving image data capturing process, the person registration unit 112a selects the moving image data MD according to the operation of the operation display unit 130 by the user. Furthermore, the person registration unit 112a can capture the moving image data MD into the image forming apparatus 100 through, for example, a wireless communication device (not shown) or a portable storage medium (not shown).
 ステップS12では、制御部110の人物照合部112が、人物検出処理を実行する。人物検出処理では、人物照合部112は、動画像データMDからフレーム画像データを生成する。さらに、人物照合部112は、フレーム画像データファイルによって表される静止画像中から人物の特徴を有する画像領域である人物検出領域を抽出する。 In step S12, the person verification unit 112 of the control unit 110 executes a person detection process. In the person detection process, the person verification unit 112 generates frame image data from the moving image data MD. Furthermore, the person collation unit 112 extracts a person detection area, which is an image area having the characteristics of a person, from a still image represented by the frame image data file.
 人物照合部112は、たとえばHOG(Histograms of Oriented Gradients)特徴量に基づいて、SVM(Support Vector Machine)などの機械学習を利用して人物検出領域を抽出することができる。 The person collation unit 112 can extract a person detection area using machine learning such as SVM (Support Vector Machine) based on, for example, HOG (Histograms of Oriented Gradients) features.
 ステップS13では、人物照合部112は、人物分類処理を実行する。人物分類処理では、人物照合部112は、たとえば人物検出領域中の人物の画像が予め登録されている家族の画像のいずれに該当するかを判定することにより、人物検出領域中の人物を分類する。 In step S13, the person verification unit 112 executes a person classification process. In the person classification process, the person verification unit 112 classifies the person in the person detection area by determining which of the family images registered in advance is the person image in the person detection area, for example. .
 例えば、家族の情報は、父親A、母親B、息子C及び娘Dの情報を含む。人物登録部112aは、ユーザーによる操作表示部130に対する操作に応じて、前記家族の情報を予め記憶部140に登録する。 For example, family information includes information on father A, mother B, son C, and daughter D. The person registration unit 112a registers the family information in the storage unit 140 in advance in response to an operation on the operation display unit 130 by the user.
 人物照合部112は、予め設定されている画像サイズよりも大きなサイズの顔の画像を含むフレーム画像データを選択する。さらに、人物照合部112は、選択したフレーム画像データを、人物分類処理の分類結果に従って、複数のグループに分類して操作表示部130に表示させる。 The person verification unit 112 selects frame image data including a face image having a size larger than a preset image size. Furthermore, the person verification unit 112 classifies the selected frame image data into a plurality of groups according to the classification result of the person classification process, and causes the operation display unit 130 to display the group image data.
 さらに、人物照合部112は、ユーザーによる操作表示部130への入力操作に応じて、複数のグループのそれぞれが父親A、母親B、息子C及び娘Dあるいは他人の何れに該当するかの分類結果を修正可能である。 Furthermore, the person verification unit 112 determines whether each of the plurality of groups corresponds to the father A, the mother B, the son C, the daughter D, or another person according to the input operation to the operation display unit 130 by the user. Can be corrected.
 ユーザーは、たとえば息子Cのグループに父親Aの静止画像が含まれているといった誤認識を修正する操作を行うことができる。これにより、人物照合部112は、機械学習の精度を向上させることができる。 The user can perform an operation to correct a misrecognition that a still image of father A is included in the group of son C, for example. Thereby, the person collation part 112 can improve the precision of machine learning.
 人物照合部112は、父親A、母親B、息子C及び娘Dをレコードとしてデータベースを生成する。データベースには、父親A、母親B、息子C及び娘Dの顔画像のHOG特徴量が登録される。 The person verification unit 112 generates a database using the father A, mother B, son C, and daughter D as records. In the database, HOG feature amounts of face images of father A, mother B, son C, and daughter D are registered.
 ステップS14では、人物照合部112は、服装選択処理を実行する。服装選択処理では、人物照合部112は、父親A、母親B、息子C及び娘Dのそれぞれが着ている服装についてのHOG特徴量をフレーム画像データから抽出する。これにより、人物照合部112は、父親A、母親B、息子C及び娘Dの顔画像のHOG特徴量に加えて、その服装画像のHOG特徴量を使用して人物を特定することができる。各人物は、同一の服装を着ることが多く、相違する人物は、相違する服装を着る傾向があるからである。 In step S14, the person collation unit 112 executes a clothing selection process. In the clothing selection process, the person collation unit 112 extracts HOG feature values for the clothing worn by each of the father A, mother B, son C, and daughter D from the frame image data. Thereby, the person collation part 112 can specify a person using the HOG feature-value of the clothes image in addition to the HOG feature-value of the face image of father A, mother B, son C, and daughter D. This is because each person often wears the same clothes, and different persons tend to wear different clothes.
 ステップS15では、人物照合部112の人物登録部112aは、データベース登録処理を実行する。データベース登録処理では、人物登録部112aは、父親A、母親B、息子C及び娘Dについてのデータベースを記憶部140の人物登録データ格納領域143に格納する。 In step S15, the person registration unit 112a of the person verification unit 112 executes a database registration process. In the database registration process, the person registration unit 112 a stores a database for the father A, mother B, son C, and daughter D in the person registration data storage area 143 of the storage unit 140.
 データベースにおいて、父親A、母親B、息子C及び娘Dがレコードである。データベースには、顔画像それぞれのHOG特徴量、服装画像のHOG特徴量、顔画像の機械学習データ、服装画像の機械学習データに加え、身長その他のユーザー入力可能な属性データが含まれる。 In the database, father A, mother B, son C, and daughter D are records. The database includes HOG feature values of face images, HOG feature values of clothes images, machine learning data of face images, machine learning data of clothes images, and height and other attribute data that can be input by the user.
 さらに、人物登録部112aは、ユーザーによる操作に従って、デジタルカメラで撮像した静止画像データを使用して各人物の顔画像や服装画像のデータを登録することもできる。人物登録部112aは、このような画像データを使用して、顔画像のHOG特徴量及び服装画像のHOG特徴量を生成して、データベースに登録することができる。なお、本実施形態では、HOG特徴量は、画像認識における計算負荷の小さいYUV画像データに基づいて生成されているものとする。 Furthermore, the person registration unit 112a can also register the face image and clothing image data of each person using still image data captured by a digital camera in accordance with a user operation. The person registration unit 112a can generate the HOG feature value of the face image and the HOG feature value of the clothes image by using such image data, and register them in the database. In the present embodiment, it is assumed that the HOG feature amount is generated based on YUV image data with a small calculation load in image recognition.
 図2に示されるように、ステップS20では、ユーザーは、操作表示部130を介して静止画像抽出モードを設定する。静止画像抽出モードには、たとえば人物の顔のアップ写真を抽出するモードおよび集合写真を抽出するモードといった各種のモードに加えて、躍動感のある連続写真が抽出される連続写真モードが含まれている。静止画像抽出モードの設定では、制御部110は、操作表示部130に各モードの選択を受け付ける画面(図示せず)を表示させる。 As shown in FIG. 2, in step S <b> 20, the user sets a still image extraction mode via the operation display unit 130. The still image extraction mode includes, for example, a continuous photo mode in which lively continuous photos are extracted, in addition to various modes such as a mode for extracting a close-up photo of a person's face and a mode for extracting a group photo. Yes. In setting the still image extraction mode, the control unit 110 causes the operation display unit 130 to display a screen (not shown) that accepts selection of each mode.
 例えば、連続写真モードが選択された場合、周期設定部116は、周期設定入力を受け付ける操作表示画面(図示せず)を操作表示部130に表示させる。前記周期設定入力は、フレーム画像の時間的な間隔である周期を設定する入力操作である。周期は、静止画像としてのフレーム画像の抽出周期の設定に使用される。ただし、ユーザー入力がなされなかった場合には、初期設定として0.2秒が使用されるものとする。 For example, when the continuous photo mode is selected, the cycle setting unit 116 causes the operation display unit 130 to display an operation display screen (not shown) that receives a cycle setting input. The cycle setting input is an input operation for setting a cycle, which is a time interval between frame images. The period is used for setting an extraction period of a frame image as a still image. However, if no user input is made, 0.2 seconds is used as the initial setting.
 ステップS30では、人物照合部112が、ユーザーによる操作表示部130の操作を通じて、人物を選択する。人物の選択では、人物照合部112は、操作表示部130に父親A、母親B、息子C及び娘Dの選択を受け付ける画面を表示させる。この例では、息子Cが選択されたものとする。 In step S30, the person verification unit 112 selects a person through the operation of the operation display unit 130 by the user. In the selection of a person, the person verification unit 112 causes the operation display unit 130 to display a screen for accepting selection of father A, mother B, son C, and daughter D. In this example, it is assumed that son C is selected.
 ステップS40では、フレーム抽出部111は、フレーム画像生成処理を実行する。フレーム抽出部111は、静止画像データ生成部とし機能する装置の一例である。フレーム画像生成処理では、フレーム抽出部111は、たとえば30fpsのフレームレートの動画像データMDから所定の周期のフレーム画像データを生成する。所定の周期は、たとえばユーザーによって予め設定される。例えば、初期設定の所定の周期が、0.2秒であることが考えられる。 In step S40, the frame extraction unit 111 executes a frame image generation process. The frame extraction unit 111 is an example of an apparatus that functions as a still image data generation unit. In the frame image generation process, the frame extraction unit 111 generates frame image data having a predetermined cycle from moving image data MD having a frame rate of 30 fps, for example. The predetermined period is set in advance by the user, for example. For example, it is conceivable that the predetermined cycle of the initial setting is 0.2 seconds.
 図4は、一実施形態に係るフレーム画像データ生成処理の内容を示すデータフローダイアグラムである。図4には、上側にデータフローダイアグラムが示され、下側にGOP(Group of Pictures)が示されている。データフローダイアグラムは、動画像データMDからフレーム画像データが抽出され、抽出されたデータが変換および保存されるまでの流れを示している。フレーム画像データは、YUV画像データとして構成されている。フレーム画像データ生成処理は、動画像データMDから複数のフレーム画像データを抽出する処理であり、フレーム抽出部111によって実行される。 FIG. 4 is a data flow diagram showing the contents of frame image data generation processing according to an embodiment. In FIG. 4, a data flow diagram is shown on the upper side, and GOP (Group of Pictures) is shown on the lower side. The data flow diagram shows a flow from extraction of frame image data from moving image data MD to conversion and storage of the extracted data. The frame image data is configured as YUV image data. The frame image data generation process is a process for extracting a plurality of frame image data from the moving image data MD, and is executed by the frame extraction unit 111.
 フレーム画像データ生成処理には、たとえばMPEG-4(ISO/IEC 14496)やH.264に規定される処理が含まれる。フレーム画像データ生成処理では、フレーム抽出部111は、Iフレーム(Intra-coded Frame)、Pフレーム(Predicted Frame)及びBフレーム(Bi-directional Predicted Frame)からフレーム画像データを生成する。 Frame image data generation processing includes, for example, MPEG-4 (ISO / IEC 14496) and H.264. H.264 is included. In the frame image data generation process, the frame extraction unit 111 generates frame image data from an I frame (Intra-coded Frame), a P frame (Predicted Frame), and a B frame (Bi-directional Predicted Frame).
 Iフレームは、フレーム間予測を用いずに符号化されるフレームである。Iフレームは、イントラフレームやキーフレームとも呼ばれる。Iフレームは、Pフレーム(Predicted Frame)およびBフレーム(Bi-directional Predicted Frame)とともにGOPを構成する。 An I frame is a frame that is encoded without using inter-frame prediction. The I frame is also called an intra frame or a key frame. The I frame constitutes a GOP together with a P frame (Predicted Frame) and a B frame (Bi-directional Predicted Frame).
 Pフレームが、Iフレームとのフレーム間処理を施されることによって、フレーム画像データを生成することが可能である。Bフレームが、Iフレーム、Pフレーム及び他のBフレームとのフレーム間処理を施されることによって、フレーム画像データを生成することが可能である。 It is possible to generate frame image data by subjecting the P frame to inter-frame processing with the I frame. Frame image data can be generated by subjecting the B frame to inter-frame processing with the I frame, P frame, and other B frames.
 動画像データは、時系列順に配列されている複数のフレーム画像データから生成される。複数のフレーム画像データは、時系列の前後のフレーム間で近似していることが多い。フレーム間予測は、このような動画像データの性質を利用した処理である。フレーム間予測は、時系列的に前のフレーム画像から現在のフレーム画像を予測する処理である。 The moving image data is generated from a plurality of frame image data arranged in time series. A plurality of frame image data is often approximated between frames before and after time series. Inter-frame prediction is a process that uses such characteristics of moving image data. Inter-frame prediction is a process of predicting the current frame image from the previous frame image in time series.
 具体的には、フレーム間予測において、画素ブロック毎の移動が推定され、さらに、移動後のフレーム間での画素ブロックの差分が、DCT変換されるとともに量子化される。これにより、GOP単位での圧縮率が高まる。Pフレームは、動きベクトルを使用することによりIフレームから再現される。動きベクトルは、各画素ブロックの移動ベクトルである。 Specifically, in the inter-frame prediction, the movement for each pixel block is estimated, and further, the difference of the pixel block between the frames after the movement is DCT transformed and quantized. Thereby, the compression rate per GOP increases. P frames are reconstructed from I frames by using motion vectors. The motion vector is a movement vector of each pixel block.
 フレーム抽出部111は、Iフレームに対して逆離散コサイン変換(逆DCT変換とも呼ばれる。)を行うことによって、輝度データと色差データとを含むYUV画像データとしてのフレーム画像データを生成する。逆DCT変換は、たとえば8×8画素あるいは16×16の画素ブロック毎に実行される。フレーム抽出部111は、再現されたフレーム画像データをフレームメモリ141に格納する。 The frame extraction unit 111 generates frame image data as YUV image data including luminance data and color difference data by performing inverse discrete cosine transform (also called inverse DCT transform) on the I frame. The inverse DCT transform is executed for each 8 × 8 pixel or 16 × 16 pixel block, for example. The frame extraction unit 111 stores the reproduced frame image data in the frame memory 141.
 フレーム抽出部111は、Pフレーム及びBフレームに対して逆離散コサイン変換を行うことによって差分データを生成する。フレーム抽出部111は、差分データと動きベクトルとを使用してフレーム間処理を実行することによりフレーム画像データを生成する。動きベクトルは、動画像データMDのエンコード時に生成されるデータである。本処理は、MPEG-4やH.264に規定される通常の復号化処理である。 The frame extraction unit 111 generates difference data by performing inverse discrete cosine transform on the P frame and the B frame. The frame extraction unit 111 generates frame image data by performing inter-frame processing using the difference data and the motion vector. The motion vector is data generated when the moving image data MD is encoded. This processing is performed using MPEG-4 or H.264. H.264 is a normal decoding process.
 フレーム抽出部111は、制御部110のRAM(図示せず)上でPフレームおよびBフレームに基づくフレーム画像データ生成処理を実行する。フレーム抽出部111は、予め設定されている周期のフレーム画像データをフレームメモリ141に格納する。具体的には、動画像データMDのフレームレートが30fpsである場合、周期が0.2秒に設定されている。この場合、フレーム抽出部111は、6フレーム毎にフレーム画像データをフレームメモリ141に格納する。一方、フレーム抽出部111は、他のフレーム画像データを廃棄する。これにより、フレーム抽出部111は、フレームメモリ141の過剰な消費を低減させることができる。 The frame extraction unit 111 executes frame image data generation processing based on the P frame and the B frame on the RAM (not shown) of the control unit 110. The frame extraction unit 111 stores frame image data having a preset period in the frame memory 141. Specifically, when the frame rate of the moving image data MD is 30 fps, the cycle is set to 0.2 seconds. In this case, the frame extraction unit 111 stores the frame image data in the frame memory 141 every six frames. On the other hand, the frame extraction unit 111 discards other frame image data. Thereby, the frame extraction unit 111 can reduce excessive consumption of the frame memory 141.
 フレーム抽出部111は、フレーム画像データをフレームメモリ141に格納する際に、一緒にフレーム画像データの生成で使用された動きベクトルを各フレーム画像データに紐づけて格納する。 When the frame extraction unit 111 stores the frame image data in the frame memory 141, the frame extraction unit 111 stores the motion vector used in generating the frame image data together with each frame image data.
 ステップS50では、人物照合部112が、人物検出処理を実行する。人物検出処理では、人物照合部112は、フレームメモリ141に格納されている複数のフレーム画像データのそれぞれについてステップS30で選択された人物が含まれているか否かを判断する。 In step S50, the person verification unit 112 executes a person detection process. In the person detection process, the person verification unit 112 determines whether or not the person selected in step S30 is included for each of the plurality of frame image data stored in the frame memory 141.
 例えば、ステップS30で選択された人物が息子Cであることが考えられる。この場合、人物照合部112は、輝度データと色差データとを含むYUV画像データとしてのフレーム画像データから息子Cの検出を試みる。 For example, it is conceivable that the person selected in step S30 is the son C. In this case, the person verification unit 112 tries to detect the son C from the frame image data as YUV image data including luminance data and color difference data.
 人物照合部112は、たとえば周知のOpenCV(Open Source Computer Vision Library)を使用して人物の検出や特定を実行することができる。まず、人物照合部112は、フレーム画像データの中から人物を検出する。さらに、人物照合部112は、息子Cの顔画像のHOG特徴量を使用して検出された人物の顔が息子Cの顔であるか否かを判断する。 The person collation unit 112 can detect and specify a person using, for example, the well-known OpenCV (Open Source Computer Vision Library). First, the person verification unit 112 detects a person from the frame image data. Furthermore, the person verification unit 112 determines whether the face of the person detected using the HOG feature amount of the son C face image is the face of the son C.
 人物照合部112は、息子Cの顔であるか否かの判断の確度が低い場合に、息子Cの服装画像のHOG特徴量を使用して、人物の顔が息子Cの顔であるか否かを判断する。特に、息子Cが横方向から撮影されており、かつ、顔画像のサイズが小さい場合等に、服装画像のHOG特徴量を補助的に使用することができる。一般に、躍動感のある連続画像が撮影される場合、アップで顔画像を安定的に捉えることが難しいからである。 When the accuracy of the determination as to whether or not the face is the son C's face is low, the person verification unit 112 uses the HOG feature value of the clothes image of the son C and determines whether or not the face of the person is the face of the son C. Determine whether. In particular, when the son C is photographed from the lateral direction and the size of the face image is small, the HOG feature amount of the clothes image can be used as an auxiliary. This is because, generally, when a continuous image with a dynamic feeling is taken, it is difficult to capture a face image stably.
 ステップS60では、制御部110の速度検出部113は、被写体である人物の移動速度を検出する。この例では、人物は息子Cである。速度検出部113は、時間的に連続している複数のフレーム画像データにおいて同一の被写体の画像が含まれる時間的な区間を特定する。 In step S60, the speed detector 113 of the controller 110 detects the moving speed of the person who is the subject. In this example, the person is son C. The speed detection unit 113 specifies a temporal section in which the same subject image is included in a plurality of temporally continuous frame image data.
 速度検出部113は、同一の被写体の画像、すなわち同一人物の画像を含む画像領域を構成する画素ブロックの動きベクトルを使用して被写体の画像の速度を推定することができる。息子Cの画像を含む画像領域を構成する画素ブロックは、息子Cの併進移動速度に対応する動きベクトルを有しているからである。 The speed detection unit 113 can estimate the speed of the subject image using the motion vector of the pixel block that forms the image area including the same subject image, that is, the same person image. This is because the pixel block constituting the image area including the image of the son C has a motion vector corresponding to the translational movement speed of the son C.
 図5に示される例では、同一人物として息子Cが撮影されている9枚のフレーム画像データF1~F9の画像が示されている。フレーム画像データF1~F9は、静止している3本の樹木を背景として、息子Cが疾走している躍動感のある一連の連続画像を表している。 In the example shown in FIG. 5, images of nine frame image data F1 to F9 in which the son C is photographed as the same person are shown. The frame image data F1 to F9 represent a series of continuous images with a feeling of dynamism in which the son C is running against the background of three stationary trees.
 図6は、同一人物が表されている2枚のフレーム画像データF1,F2の内容を示す説明図である。図6における2枚のフレーム画像データF1,F2において、息子Cの画像は、図6に向かって右側に併進移動している。息子Cの画像の併進移動量は、2枚のフレーム画像データF1,F2のフレーム間の移動量である。併進移動量を周期で除することによって、併進移動速度を算出することができる。ただし、本実施形態では、併進移動速度は、30fpsを想定する動きベクトルから推定される。 FIG. 6 is an explanatory diagram showing the contents of two pieces of frame image data F1, F2 representing the same person. In the two pieces of frame image data F1 and F2 in FIG. 6, the image of the son C is translated to the right side in FIG. The translational movement amount of the image of the son C is a movement amount between the frames of the two pieces of frame image data F1 and F2. By dividing the translational movement amount by the period, the translational movement speed can be calculated. However, in the present embodiment, the translational movement speed is estimated from a motion vector assuming 30 fps.
 ステップS70では、制御部110の高速移動区間抽出部114は、息子Cの画像の移動速度が予め設定されている閾値以上であるか否かを判断する。高速移動区間抽出部114は、息子Cの画像の移動速度が閾値以上である場合には、処理をステップS80に進め、息子Cの画像の移動速度が閾値未満である場合には、処理をステップS90に進める。 In step S70, the high speed movement section extraction unit 114 of the control unit 110 determines whether or not the moving speed of the image of the son C is equal to or higher than a preset threshold value. If the moving speed of the son C image is equal to or higher than the threshold, the high-speed moving section extraction unit 114 proceeds to step S80. If the moving speed of the son C image is lower than the threshold, the high-speed moving section extraction unit 114 performs the process. Proceed to S90.
 ステップS80では、高速移動区間抽出部114は、フレーム画像データグループ化処理を実行する。フレーム画像データグループ化処理は、2枚のフレーム画像データF1,F2の前後に所定の周期で時間的に連続し、息子Cの画像を含むフレーム画像データである。 In step S80, the high-speed movement section extraction unit 114 executes a frame image data grouping process. The frame image data grouping process is frame image data that is temporally continuous in a predetermined cycle before and after the two pieces of frame image data F1 and F2, and includes the image of the son C.
 図5は、フレーム画像データF1で息子Cの撮影が開始され、フレーム画像データF9まで息子Cの撮影が継続している例を示す。 FIG. 5 shows an example in which the photographing of the son C is started with the frame image data F1, and the photographing of the son C is continued until the frame image data F9.
 高速移動区間抽出部114は、9枚のフレーム画像データF1~F9をグループ化し、静止画像格納領域142に格納する。これにより、9枚のフレーム画像データF1~F9は、一つの連続画像データファイルとしての管理および取り扱いが可能である。静止画像格納領域142には、9枚のフレーム画像データF1~F9が、それぞれDCT変換されてJPEG静止画像データとして格納される。 The high-speed movement section extraction unit 114 groups the nine pieces of frame image data F1 to F9 and stores them in the still image storage area 142. As a result, the nine pieces of frame image data F1 to F9 can be managed and handled as one continuous image data file. In the still image storage area 142, nine frame image data F1 to F9 are DCT converted and stored as JPEG still image data.
 なお、高速移動区間抽出部114は、連続静止画像データ抽出部の一例である。9枚のフレーム画像データF1~F9は、連続静止画像データの一例である。 Note that the high-speed moving section extraction unit 114 is an example of a continuous still image data extraction unit. Nine pieces of frame image data F1 to F9 are examples of continuous still image data.
 また、本実施形態では、被写体(息子C)の画像の移動速度が閾値以上である状態が、一つの連続画像データファイルにおいて一瞬でもよい。即ち、被写体の画像の移動速度が閾値以上である状態が、一つの連続画像データファイルにおいて1回生じていればよい。画像の移動速度が一つの連続画像データファイルの全体において閾値を超えている必要はない。ただし、高速移動区間抽出部114が、一つの連続画像データファイルの全区間において移動速度が閾値を超えていることを必要条件として判断することも考えられる。 In the present embodiment, the state in which the moving speed of the image of the subject (son C) is equal to or higher than the threshold may be instantaneous for one continuous image data file. That is, it is only necessary that the moving speed of the subject image is equal to or greater than the threshold value once in one continuous image data file. It is not necessary for the moving speed of the image to exceed the threshold in one continuous image data file. However, it is also conceivable that the high speed moving section extracting unit 114 determines that the moving speed exceeds a threshold in all sections of one continuous image data file as a necessary condition.
 制御部110は、ステップS40乃至ステップS80の処理を、複数のフレーム画像データについて、最終フレーム画像データが処理対象となるまで繰り返して実行する(ステップS90)。 The control unit 110 repeatedly executes the processing from step S40 to step S80 for a plurality of frame image data until the final frame image data becomes a processing target (step S90).
 ステップS100では、静止画像出力部115は、フレーム画像データ出力処理を実行する。フレーム画像データ出力処理では、静止画像出力部115は、一つの連続画像データファイルを操作表示部130に9枚のサムネイル画像として表示する。ここでは、一つの連続画像データファイルは、グループ化された9枚のフレーム画像データF1~F9を含む。 In step S100, the still image output unit 115 executes frame image data output processing. In the frame image data output process, the still image output unit 115 displays one continuous image data file on the operation display unit 130 as nine thumbnail images. Here, one continuous image data file includes nine grouped frame image data F1 to F9.
 なお、制御部110は、ユーザーによるサムネイル画像をタッチする操作に従って、9枚のフレーム画像データF1~F9から任意のフレーム画像を処理対象として選択すること、及び、一括して全てのフレーム画像を処理対象として選択することもできる。 The control unit 110 selects an arbitrary frame image as a processing target from the nine pieces of frame image data F1 to F9 according to an operation of touching a thumbnail image by the user, and processes all the frame images at once. It can also be selected as a target.
 静止画像出力部115は、ユーザーの指示に応じて、選択されたフレーム画像データに基づいて画像形成部120を制御することにより、印刷媒体上に画像を形成する処理を、画像形成部120に実行させることができる。 The still image output unit 115 controls the image forming unit 120 based on the selected frame image data in accordance with a user instruction to execute processing for forming an image on the print medium. Can be made.
 画像形成部120により出力された印刷物は、躍動感のある連続写真としてアルバムなどに利用可能である。一方、静止画像出力部115は、ユーザーによる指示操作に応じて、通信インターフェース部150を介してスマートフォン200又は図示しないパーソナルコンピュータに連続画像データファイルを送信することもできる。 The printed matter output by the image forming unit 120 can be used for an album or the like as a continuous photograph with a lively feeling. On the other hand, the still image output unit 115 can also transmit a continuous image data file to the smartphone 200 or a personal computer (not shown) via the communication interface unit 150 in response to an instruction operation by the user.
 このように、本実施形態に係る画像形成装置100によれば、動画像データの特性である時間的な連続性を活かして、躍動感のある連続画像(連続写真)として、所定の周期で時間的に連続する複数の静止画像データ(フレーム画像データ)を抽出することができる。 As described above, according to the image forming apparatus 100 according to the present embodiment, the temporal continuity that is the characteristic of the moving image data is utilized to obtain a continuous image (continuous photograph) with a dynamic feeling at a predetermined cycle. A plurality of still image data (frame image data) can be extracted.
 本開示は、上記実施形態だけでなく、以下のような変形例でも実施することができる。 The present disclosure can be implemented not only in the above-described embodiment but also in the following modifications.
 [変形例1]
 上記実施形態では、躍動感のある連続画像として、被写体としての人物の画像の並進移動速度が閾値を超えたフレーム画像データが抽出されている。しかしながら、躍動感のある連続画像は、並進移動速度が速い画像に限定されない。
[Modification 1]
In the above-described embodiment, frame image data in which the translational movement speed of an image of a person as a subject exceeds a threshold is extracted as a continuous image with a sense of dynamism. However, a continuous image with a feeling of dynamism is not limited to an image having a high translational movement speed.
 具体的には、たとえば図7に示されるように、被写体が遠近方向において近づいてくるようなシーンを示す画像も躍動感のある連続画像に含まれる。 Specifically, as shown in FIG. 7, for example, an image showing a scene in which a subject approaches in the perspective direction is also included in a continuous image having a dynamic feeling.
 図7に示される例では、検出対象としての人物が検出された領域のサイズが、検出領域Fr1のサイズから検出領域Fr2のサイズに変化している。ステップS70において、速度検出部113が、被写体の画像サイズの変化率が予め設定されている閾値以上であるか否かに基づいて判断することが考えられる。被写体の画像サイズの変化率は、単位時間あたりの縮小率または拡大率である。 In the example shown in FIG. 7, the size of the area where the person as the detection target is detected is changed from the size of the detection area Fr1 to the size of the detection area Fr2. In step S70, it is conceivable that the speed detection unit 113 makes a determination based on whether or not the change rate of the image size of the subject is equal to or greater than a preset threshold value. The change rate of the image size of the subject is a reduction rate or an enlargement rate per unit time.
 Pフレーム又はBフレームにおいて、被写体としての人物の画像を構成する画素ブロックに紐づけられている動きベクトルは、検出対象としての人物が近づいてくる場合には発散方向の成分を多く有する。一方、動きベクトルは、検出対象としての人物が遠ざかる場合には収束方向の成分を多く有する。 In the P frame or B frame, the motion vector associated with the pixel block constituting the image of the person as the subject has many components in the divergence direction when the person as the detection target approaches. On the other hand, the motion vector has many components in the convergence direction when the person to be detected moves away.
 並進移動、遠近移動及びその組合せは、動きベクトルの並進成分と、発散成分と、収束成分とを解析することによって検出することができる。速度検出部113は、被写体としての人物の画像の移動速度に伴う動きベクトルの並進成分、発散成分及び収束成分の少なくとも1つに基づいて躍動感のある連続画像であるか否かを判断することができる。 Translational movement, perspective movement, and combinations thereof can be detected by analyzing the translational component, the divergent component, and the convergence component of the motion vector. The speed detector 113 determines whether or not the image is a continuous image based on at least one of a translation component, a divergence component, and a convergence component of a motion vector associated with the moving speed of an image of a person as a subject. Can do.
 [変形例2]
 上記実施形態では、躍動感のある連続画像として、被写体としての人物の画像の移動速度に基づいてフレーム画像データが抽出されている。しかしながら、躍動感のある連続画像は、移動速度が速い画像に限定されない。
[Modification 2]
In the above-described embodiment, frame image data is extracted as a continuous image with a sense of movement based on the moving speed of an image of a person as a subject. However, a continuous image with a feeling of dynamism is not limited to an image having a high moving speed.
 具体的には、たとえば被写体としての人物が体操をしているような場合が考えられる。この場合、被写体としての人物の画像は、移動を伴わずに、人物の手足が動いている場合又は回転している場合の画像である。このような場合に、制御部110が、手足が動いている場合又は回転している人物の画像を含む連続画像を、躍動感のある連続画像として抽出してもよい。 Specifically, for example, a case where a person as a subject is doing gymnastics can be considered. In this case, the image of the person as the subject is an image when the person's limb is moving or rotating without moving. In such a case, the control unit 110 may extract a continuous image including an image of a person who is moving or rotating a limb as a continuous image having a sense of movement.
 このように、検出対象としての人物の手足が動いている場合又は回転している場合が考えられる。この場合、被写体としての人物の画像を構成する画素ブロックに紐づけられている動きベクトルが、ランダムな方向と大きさとを有することがある。この場合、速度検出部113は、被写体としての人物の画像を構成する画素ブロックの動きベクトルがランダムな方向と大きさとを有する場合に、その動きベクトルに対応する連続画像が躍動感のある連続画像であると判断することができる。 Thus, the case where the limb of the person as the detection target is moving or rotating may be considered. In this case, a motion vector associated with a pixel block constituting an image of a person as a subject may have a random direction and size. In this case, when the motion vector of the pixel block that forms the image of the person as the subject has a random direction and size, the speed detection unit 113 has a continuous image corresponding to the motion vector with a lively feeling. Can be determined.
 [変形例3]
 上記実施形態では、躍動感のある連続画像であるか否かの判断に使用するデータとして、動きベクトルが使用されている。しかしながら、必ずしも動きベクトルが使用される必要はなく他のデータが採用されてもよい。
[Modification 3]
In the above-described embodiment, a motion vector is used as data used for determining whether or not a continuous image is lively. However, the motion vector is not necessarily used, and other data may be adopted.
 具体的には、速度検出部113は、たとえば各フレーム画像に存在する人物領域の位置又は大きさを参照し、人物の移動速度を検知するように構成されていてもよい。たとえば、速度検出部113は、時間的に隣接するフレーム画像間での人物領域の位置の差を移動距離として計算し、その距離が所定の閾値より大きければ、動画中の人物が高速で移動していると判断することができる。ここで、人物領域は、人物の画像を含む領域である。 Specifically, the speed detection unit 113 may be configured to detect the movement speed of the person with reference to, for example, the position or size of the person area existing in each frame image. For example, the speed detection unit 113 calculates the difference in the position of the person region between temporally adjacent frame images as the movement distance, and if the distance is larger than a predetermined threshold, the person in the moving image moves at high speed. Can be determined. Here, the person area is an area including an image of a person.
 速度検出部113は、さらに、人物領域の位置だけでなく人物領域の大きさを比較し、大きさの差が所定の閾値より大きい場合にも動画中の人物の画像が高速で移動していると判断してもよい。 Further, the speed detection unit 113 compares not only the position of the person area but also the size of the person area. Even when the difference in size is larger than a predetermined threshold, the image of the person in the moving image moves at high speed. You may judge.
 上記実施形態における動きベクトルを使用する方法は、計算負荷が小さく処理を高速化することができるという利点を有している。一方、本変形例における方法は、躍動感のある連続画像であるか否かの判断のための処理の設計自由度が高いという利点を有している。 The method using the motion vector in the above embodiment has an advantage that the calculation load is small and the processing can be speeded up. On the other hand, the method according to the present modification has an advantage that the degree of freedom in design of processing for determining whether or not a continuous image is lively is high.
 さらに、たとえば人物領域の位置の差及び人物領域の大きさの差の一方を主として利用し、他方を補完的に利用することによって、両者を組み合わせてもよい。このように、速度検出部113は、同一の被写体の画像の動きが予め設定されている条件を満たしているか否かを判断するものであればよい。 Furthermore, for example, one of the difference in the position of the person area and the difference in the size of the person area may be mainly used, and the other may be used in a complementary manner. As described above, the speed detection unit 113 only needs to determine whether or not the movement of the image of the same subject satisfies a preset condition.
 [変形例4]
 上記実施形態では、画像形成装置に本発明が適用されている。しかしながら、本発明を、たとえばスマートフォンやパーソナルコンピュータといった画像処理装置として機能する装置に適用することもできる。
[Modification 4]
In the above embodiment, the present invention is applied to the image forming apparatus. However, the present invention can also be applied to an apparatus that functions as an image processing apparatus such as a smartphone or a personal computer.

Claims (7)

  1.  動画像データから複数の静止画像データを生成する静止画像データ生成部と、
     生成された前記複数の静止画像データから、同一の被写体の画像を含むとともに所定の周期で時間的に連続している静止画像データであり、かつ、前記同一の被写体の画像の動きが予め設定されている条件を満たしている連続静止画像データを抽出する連続静止画像データ抽出部と、
    を備える画像処理装置。
    A still image data generating unit that generates a plurality of still image data from moving image data;
    From the generated still image data, the still image data includes the same subject image and is temporally continuous in a predetermined cycle, and the movement of the same subject image is preset. A continuous still image data extraction unit that extracts continuous still image data that satisfies the following conditions:
    An image processing apparatus comprising:
  2.  請求項1記載の画像処理装置であって、
     前記連続静止画像データ抽出部は、前記被写体の画像が予め設定されている閾値を超える速度で並進移動していると判断したときに、前記同一の被写体の画像の動きが予め設定されている条件を満たしていると判断する画像処理装置。
    The image processing apparatus according to claim 1,
    When the continuous still image data extraction unit determines that the image of the subject is moving in translation at a speed exceeding a preset threshold, the condition that the movement of the same subject image is preset An image processing apparatus that determines that the above is satisfied.
  3.  請求項1に記載の画像処理装置であって、
     前記連続静止画像データ抽出部は、前記被写体の画像サイズの変化率が予め設定されている閾値以上であると判断されたときに、前記同一の被写体の画像の動きが予め設定されている条件を満たしていると判断する画像処理装置。
    The image processing apparatus according to claim 1,
    The continuous still image data extraction unit sets a condition in which the movement of the same subject image is preset when it is determined that the rate of change in the image size of the subject is greater than or equal to a preset threshold value. An image processing apparatus that determines that the condition is satisfied.
  4.  請求項1に記載の画像処理装置であって、
     前記静止画像データ生成部は、前記動画像データに含まれている動きベクトルを使用して前記動きベクトルが紐づけられている複数の画素ブロックの画像を再現して前記複数の静止画像データを生成し、
     前記連続静止画像データ抽出部は、前記被写体の画像を構成する少なくとも1つの前記画素ブロックに紐づけられている前記動きベクトルを使用して前記被写体の動きが予め設定されている条件を満たしているか否かを判断する画像処理装置。
    The image processing apparatus according to claim 1,
    The still image data generation unit generates the plurality of still image data by reproducing an image of a plurality of pixel blocks to which the motion vector is linked using a motion vector included in the moving image data. And
    Whether the continuous still image data extraction unit satisfies a condition in which the motion of the subject is set in advance using the motion vector associated with at least one pixel block constituting the image of the subject An image processing apparatus that determines whether or not.
  5.  請求項1に記載の画像処理装置であって、さらに、
     画面を表示し、ユーザーからの入力を受け付ける操作表示部と、
     前記所定の周期の設定である周期設定入力を受け付ける画面を前記操作表示部に表示させる周期設定部と、
    を備える画像処理装置。
    The image processing apparatus according to claim 1, further comprising:
    An operation display that displays a screen and accepts input from the user;
    A cycle setting unit that causes the operation display unit to display a screen that accepts a cycle setting input that is a setting of the predetermined cycle;
    An image processing apparatus comprising:
  6.  請求項1に記載の画像処理装置と、
     印刷媒体に画像を形成する画像形成部と、
    を備える画像形成装置。
    An image processing apparatus according to claim 1;
    An image forming unit that forms an image on a print medium;
    An image forming apparatus comprising:
  7.  動画像データから複数の静止画像データを生成することと、
     前記生成された複数の静止画像データから、同一の被写体の画像を含むとともに所定の周期で時間的に連続している静止画像データであり、かつ、前記同一の被写体の画像の動きが予め設定されている条件を満たしている連続静止画像データを抽出することと、
    を含む画像処理方法。
    Generating a plurality of still image data from moving image data;
    From the plurality of generated still image data, still image data including an image of the same subject and continuous in time with a predetermined cycle, and movement of the image of the same subject is preset. Extracting continuous still image data that satisfies the following conditions:
    An image processing method including:
PCT/JP2018/002735 2017-02-23 2018-01-29 Image processing device, image forming device and image processing method WO2018155087A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2019501159A JP6870728B2 (en) 2017-02-23 2018-01-29 Image processing device, image forming device and image processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-032729 2017-02-23
JP2017032729 2017-02-23

Publications (1)

Publication Number Publication Date
WO2018155087A1 true WO2018155087A1 (en) 2018-08-30

Family

ID=63252626

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/002735 WO2018155087A1 (en) 2017-02-23 2018-01-29 Image processing device, image forming device and image processing method

Country Status (2)

Country Link
JP (1) JP6870728B2 (en)
WO (1) WO2018155087A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007082240A (en) * 2006-09-26 2007-03-29 Casio Comput Co Ltd Image processing apparatus and image processing method
JP2008167220A (en) * 2006-12-28 2008-07-17 Canon Inc Image processing apparatus and method, and program and recording medium
JP2010192957A (en) * 2009-02-16 2010-09-02 Nikon Corp Electronic camera
JP2012253784A (en) * 2008-04-02 2012-12-20 Panasonic Corp Imaging device
JP2016192593A (en) * 2015-03-30 2016-11-10 キヤノン株式会社 Zoom controller, control method of zoom controller

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007082240A (en) * 2006-09-26 2007-03-29 Casio Comput Co Ltd Image processing apparatus and image processing method
JP2008167220A (en) * 2006-12-28 2008-07-17 Canon Inc Image processing apparatus and method, and program and recording medium
JP2012253784A (en) * 2008-04-02 2012-12-20 Panasonic Corp Imaging device
JP2010192957A (en) * 2009-02-16 2010-09-02 Nikon Corp Electronic camera
JP2016192593A (en) * 2015-03-30 2016-11-10 キヤノン株式会社 Zoom controller, control method of zoom controller

Also Published As

Publication number Publication date
JP6870728B2 (en) 2021-05-12
JPWO2018155087A1 (en) 2019-11-07

Similar Documents

Publication Publication Date Title
JP4775306B2 (en) Image processing apparatus, imaging apparatus, image display control method, and computer program
US8675065B2 (en) Video monitoring system
JP6049448B2 (en) Subject area tracking device, control method thereof, and program
US10582211B2 (en) Neural network to optimize video stabilization parameters
TW201701187A (en) Method and system of gesture detection and recognition
CN111567030B (en) Method for automatically switching video capture and playback frame rates
KR20140045897A (en) Device and method for media stream recognition based on visual image matching
CN110381247B (en) Image pickup apparatus and control method of image pickup apparatus
TWI521473B (en) Device, method for image analysis and computer-readable medium
US10657407B2 (en) Image processing apparatus, image processing method, and recording medium
JP5156982B2 (en) Image processing system, image processing method, and program
JP6870728B2 (en) Image processing device, image forming device and image processing method
JP7211373B2 (en) MOVING IMAGE ANALYSIS DEVICE, MOVING IMAGE ANALYSIS SYSTEM, MOVING IMAGE ANALYSIS METHOD, AND PROGRAM
JP6737247B2 (en) Image processing apparatus, image processing method, and image processing program
US20160172004A1 (en) Video capturing apparatus
JP6768222B2 (en) Image processing device, image processing method and image processing program
JP6624106B2 (en) Image reading device, image forming system, image reading method, and image reading program
JP6203007B2 (en) Image processing apparatus, image processing method, and program
JP6756311B2 (en) Image processing device, image processing method and image processing program
JP6768223B2 (en) Image processing device, image processing method and image processing program
JP6841181B2 (en) Image processing device, image processing method and image processing program
JP2007049449A (en) Image recorder
JP6778398B2 (en) Image forming apparatus, image forming method and image forming program
US20200252637A1 (en) Moving image processor, moving image processing system, and moving image processing method
JP2017098897A (en) Information processing system, information processing device and method of controlling the same, and computer program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18757345

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019501159

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18757345

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载