JP2007193590A - Image processing method, image processing device, reception terminal, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable proper selection of an image to be used as an indicator image (a representative image) indicating contents of an image group, from images in the image group. <P>SOLUTION: In a control part 11 of the reception terminal 1 for receiving print orders, at least one image group is extracted based on predetermined conditions, from a plurality of images recorded in a medium M recording images having date and time information as added information, and a representative image indicating contents of images included in the image group is selected per extracted image group. When selecting the representative image, the control part 11 calculates light distribution characteristics in regard to at least one image in the image group, it carries out face extracting processing of extracting a face area of a person, and it selects the representative image on the basis of a calculation result of the light distribution characteristics and/or face area information of the image from which the face area is extracted. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing apparatus, a reception terminal, and an image processing program.

  In recent years, digital cameras have been widely used in place of analog cameras. In addition to the increase in the number of shots, the memory (media) for recording and storing captured images is also increasing in capacity and price, and a huge number of images can be recorded in one memory. Yes. Due to the increase in the number of recorded images, the necessity of image management arises, and for example, a technique of grouping for each series of captured images such as travel is desired and studied. Furthermore, since there are a plurality of groups, it has become necessary to devise a way to understand at a glance the contents of each group.

A technique for selecting a representative image from an image group has been proposed so that the contents of the image group can be understood at a glance. As such a technique, Patent Document 1 discloses a technique in which the first frame (the frame with the youngest date) or the last frame in the image group is used as a representative image (representative frame) in the image group. Further, in Patent Document 2, an image having a feature amount closest to an average value of feature amounts (hue, saturation, brightness, etc.) of all images in the image group or an image having the earliest shooting date is represented as a representative in the image group. A technique for making an image is disclosed.
JP 2004-297176 A JP 2004-310436 A

However, the above-described conventional technology has the following problems.
In the technique disclosed in Patent Document 1, a representative image is used as a representative image regardless of what the first or last frame is captured. Therefore, a poorly captured image may be a representative image and may not be appropriate as a representative image. there were. In the technique described in Patent Document 2, an image having a feature value closest to the average value of the feature values of all the images in the image group is used as a representative image. For example, most of the images in the image group are captured. In the case of an image taken in a dark place as an environment, there is a problem that an image having a feature amount closest to the average value is not necessarily an easy-to-view image as a representative image.

  An object of the present invention is to make it possible to select an appropriate image as a sign image (representative image) suggesting the contents of an image group from images in the image group.

  In order to solve the above-mentioned problem, the invention according to claim 1 is characterized in that at least one image based on a predetermined condition is selected from a plurality of images recorded in a recording unit in which an image having date information as additional information is recorded. An image processing method comprising: an extraction step of extracting a group; and a selection step of selecting a sign image suggesting the content of an image included in the image group from images belonging to the extracted image group. In the selection step, light distribution is calculated for at least one image in the image group, and the marker image is selected based on a calculation result of the light distribution.

  The invention according to claim 2 is an extraction step of extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording unit in which an image having date / time information is recorded as additional information. A selection step of selecting a sign image suggesting the content of the image included in the image group from the images belonging to the extracted image group, wherein the selection step includes: A face extraction process for extracting a human face area from at least one image in the image group is performed, and the marker image is selected based on face area information of the image from which the face area is extracted.

The invention according to claim 3 is an extraction step of extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording unit in which an image having date / time information is recorded as additional information. A selection step of selecting a sign image suggesting the content of the image included in the image group from the images belonging to the extracted image group, and an image processing method comprising:
In the selection step, light distribution is calculated for at least one image in the image group, and a face extraction process for extracting a human face area is performed, and the light distribution calculation result and the face area are extracted. The marker image is selected based on face area information of the captured image.

  According to a fourth aspect of the present invention, in the image processing method according to the first or third aspect, in the selection step, the light intensity and the strobe degree are calculated as the light distribution.

  A fifth aspect of the present invention is the image processing method according to the second or third aspect, wherein the face area information includes at least one of a face size and a face position.

  According to a sixth aspect of the present invention, in the image processing method according to any one of the first to fifth aspects, an operation step of accepting an operation of selecting image data to be printed out of the image data in the image group; And a transmission step of transmitting the selected image data as image data for printing to the image forming means.

  The invention according to claim 7 is an extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date / time information is recorded as additional information. A selection means for selecting a sign image suggesting the content of the image included in the image group from the images belonging to the extracted image group, wherein the selection means includes: Light distribution is calculated for at least one image in the image group, and the marker image is selected based on the calculation result of the light distribution.

  According to an eighth aspect of the present invention, there is provided an extracting means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date / time information is recorded as additional information. A selection means for selecting a sign image suggesting the content of the image included in the image group from the images belonging to the extracted image group, wherein the selection means includes: A face extraction process for extracting a human face area from at least one image in the image group is performed, and the marker image is selected based on face area information of the image from which the face area is extracted.

  The invention according to claim 9 is an extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information. A selection means for selecting a sign image suggesting the content of the image included in the image group from the images belonging to the extracted image group, wherein the selection means includes: The light distribution is calculated for at least one image in the image group, and a face extraction process for extracting a human face area is performed, and the calculation result of the light distribution and the face area of the image from which the face area is extracted The marker image is selected based on information.

  According to a tenth aspect of the present invention, in the image processing apparatus according to the seventh or ninth aspect, the selection means calculates a light intensity and a strobe degree as the light distribution.

  According to an eleventh aspect of the present invention, in the image processing apparatus according to the eighth or ninth aspect, the face area information includes at least one of a face size and a face position.

  According to a twelfth aspect of the present invention, in the reception terminal for receiving a print order from a customer, the image processing apparatus according to any one of the seventh to eleventh aspects and a print out of the image data in the image group. An operation unit that receives an operation of selecting image data to be ordered and a transmission unit that transmits the selected image data as image data for printing to an image forming unit.

According to a thirteenth aspect of the present invention, there is provided a computer that controls image processing.
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
Light distribution is calculated for at least one image in the extracted image group, and based on the calculation result of the light distribution, an image included in the image group is selected from images belonging to the image group. It is an image processing program for functioning as a selection means for selecting a sign image suggesting the contents.

The invention according to claim 14 is a computer for controlling image processing.
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
A face extraction process is performed to extract a human face area for at least one image in the extracted image group, and based on the face area information of the image from which the face area is extracted, To an image processing program for functioning as selection means for selecting a sign image suggesting the content of an image included in the image group.

The invention according to claim 15 is a computer for controlling image processing.
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
The light distribution is calculated for at least one image in the extracted image group, and a face extraction process for extracting a human face area is performed, and the calculation result of the light distribution and the image from which the face area is extracted This is an image processing program for functioning as selection means for selecting a sign image suggesting the content of an image included in the image group from images belonging to the image group based on the face area information.

  According to a sixteenth aspect of the present invention, in the image processing program according to the thirteenth or fifteenth aspect, the selection unit calculates a light intensity and a strobe degree as the light distribution.

  According to a seventeenth aspect of the present invention, in the image processing program according to the fourteenth or fifteenth aspect, the face area information includes at least one of a face size and a face position.

  According to an eighteenth aspect of the present invention, in the image processing program according to any one of the thirteenth to seventeenth aspects, the computer performs an operation of selecting image data to be printed out of the image data in the image group. It is characterized by functioning as an operating means for receiving and a transmitting means for transmitting the selected image data as image data for printing to the image forming means.

  According to the present invention, it is possible to select an appropriate image as a sign image by selecting the sign image based on at least one of the light distribution of the images in the image group and the face area information. Therefore, it is easy to assume the contents of the image group.

  In particular, when a sign image (specifically, an image with high light intensity) is selected based on the light distribution in the image group, the visibility of the image content is enhanced when the sign image is displayed on the display unit.

  Further, when a sign image is selected based on face area information, it becomes easy to grasp the contents of the image group.

  Furthermore, when a sign image is selected based on the light distribution and face area information in the image group, the visibility of the image content is enhanced when the sign image is displayed on the display unit, and the content of the image group is grasped. It becomes easy. In addition, the extraction accuracy of the face extraction process can be improved.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the configuration in the present embodiment will be described.

  FIG. 1 shows a configuration of an image processing system 100 of the present embodiment. As shown in FIG. 1, the image processing system 100 includes a reception terminal 1 for receiving a print order from a customer (user), an order management apparatus 2 for managing information related to the print order of each customer, and printing an image. An image printer unit 4 serving as an image forming unit, and an image media writing unit 5 that writes image information to a medium. The order management device 2 includes a receiving terminal 1, an image printer unit 4, and an image media writing unit 5. Connected. Note that the numbers of the reception terminal 1, the image printer unit 4, and the image media writing unit 5 connected to the order management apparatus 2 are not particularly limited.

  In FIG. 2, the principal part structure of the reception terminal 1 is shown. The reception terminal 1 is installed in a store for receiving print orders, and as shown in FIG. 2, a control unit 11, a display unit 12, an operation input unit 13, a media reading unit 14, a storage unit 15, and an order confirmation slip issuing unit. 16 and a communication unit 17, which are connected to each other via a bus 18.

  The control unit 11 is configured by a CPU (Central Processing Unit) or the like, and controls the operation of each unit constituting the reception terminal 1 according to various processing programs stored in the storage unit 15. For example, the control unit 11 extracts at least one image group from a plurality of images recorded on the medium M based on a predetermined condition (date and time information added to the image, details will be described later). And a sign image (representative image) that suggests the content of the image included in the image group based on at least one of light distribution and human face area information from the extracted image group images The representative image selection process is executed (see FIG. 5). The control unit 11 has functions as an extraction unit and a selection unit of the present invention, and the reception terminal 1 has a configuration including the image processing apparatus of the present invention.

  The display unit 12 is configured by a display such as an LCD (Liquid Crystal Display), and performs a required display process according to a display control signal input from the control unit 11.

  The operation input unit 13 includes a numeric keypad, cursor keys, various function keys, and the like, and outputs an operation signal generated by pressing the key to the control unit 11. Further, the operation input unit 13 has a touch panel provided so as to cover the display display of the display unit 12, and detects a coordinate instructed to be touched by a coordinate reading principle such as an electromagnetic induction type, a magnetostriction type, a pressure sensitive type or the like. The detected coordinates are output to the control unit 11 as a position signal.

  The media reading unit 14 is a medium (recording device) such as a memory stick (registered trademark), smart media (registered trademark), compact flash (registered trademark), multimedia card (registered trademark), SD memory card (registered trademark), or PC card. Means) M is configured to be mountable, and reads data recorded on the mounted medium M. In the present embodiment, these media brought by the user are used as the recording means of the present invention. However, a hard disk built in the reception terminal 1 or an external hard disk, an image server connected to the reception terminal 1, and the like are used. Recording means may be used.

  The storage unit 15 stores various processing programs executed by the control unit 11 and data used when these processing programs are executed.

  The order confirmation form issuing unit 16 prints out an order confirmation form or a receipt for allowing the user to confirm the contents of the print order in accordance with a control signal from the control unit 11.

  The communication unit 17 includes a modem, a terminal adapter, a LAN adapter, and the like, and performs communication with an external device (order management device 2) via a communication network such as a telephone line, an ISDN line, or a dedicated line. Take control.

  FIG. 3 shows a main part configuration of the order management apparatus 2. As shown in FIG. 3, the order management apparatus 2 includes a control unit 21, a display unit 22, an operation input unit 23, a storage unit 24, an I / F (interface) unit 25, a communication unit 26, and a DB reference unit 27. These units are connected to each other via a bus 28.

  The control unit 21 is configured by a CPU or the like, and controls the operation of each unit constituting the order management device 2 according to various processing programs stored in the storage unit 24.

  The display unit 22 is configured by a display such as an LCD, and performs a required display process according to a display control signal input from the control unit 21.

  The operation input unit 23 includes a numeric keypad, cursor keys, various function keys, and the like, and outputs an operation signal generated by pressing the key to the control unit 11. The operation input unit 23 may be provided with a touch panel so as to cover the display display of the display unit 22.

  The storage unit 24 stores various processing programs executed by the control unit 21 and data used when these processing programs are executed.

  The I / F unit 25 is a communication module capable of data communication with an external device, and outputs data to the image printer unit 4 and the image media writing unit 5 in accordance with a control signal from the control unit 21.

  The communication unit 26 includes a modem, a terminal adapter, a LAN adapter, and the like, and performs communication control for communicating with an external device (receiving terminal 1) via a communication network such as a telephone line, an ISDN line, or a dedicated line. I do.

  The DB reference unit 27 reads information stored in a DB (DataBase) unit 3 and writes information in accordance with an instruction from the control unit 21.

  Returning to FIG. The DB unit 3 stores service history information indicating a history of past print order services performed for the customer.

  The image printer unit 4 performs print output by a predetermined printing method (electrophotographic method, ink jet method, thermal sublimation method, etc.) according to an instruction from the order management device 2.

  The image media writing unit 5 is configured to be able to load a medium such as a CD-R, and writes image information to the loaded medium according to an instruction from the order management apparatus 2.

Next, the operation in this embodiment will be described.
First, with reference to the flowchart of FIG. 4, the process (a) performed in the reception terminal 1 at the time of reception of a print order and the process (b) performed in the order management apparatus 2 are demonstrated.

  When the medium M on which the image is recorded is inserted into the media reading unit 14 of the receiving terminal 1, the medium M is read, and all the images recorded on the medium M are set as grouping target images and images from the grouping target image are displayed. A grouping process for extracting groups is performed (step S1). Next, for each extracted image group, a representative image that selects a sign image (representative image) that suggests the contents of the image included in the image group based on at least one of light distribution and human face area information A selection process is performed (step S2).

  The grouping / representative image selection process shown in steps S1 and S2 will be described in detail with reference to the flowchart of FIG.

  First, the number of grouping target images read from the medium M is counted (step S10), and it is determined whether the number of images is greater than a predetermined number (step S11). If it is determined in step S11 that the number of grouping target images is equal to or less than the predetermined number (step S11; NO), the grouping process is not performed, and the number of image groups is set to 1 (N = 1), which will be described later. Migrate to

  If it is determined in step S11 that the number of grouping target images is greater than the predetermined number (step S11; YES), a grouping process for extracting image groups from the grouping target images is performed (step S12). Hereinafter, the number of image groups obtained by the grouping process in step S12 is N.

  As a grouping processing method, it is preferable to group images that are continuously photographed with the same intention for the user, such as trips and various events, into the same image group, and a method known in the art can be used. Specifically, it is preferable to perform grouping using date and time information added to the image. For example, as described in JP-A-2005-174060, GPS (Global Positioning System) information or the like is used. A method of grouping according to the shooting location may be used in combination.

  The date / time information is information indicating the date / time of image generation or update, and is information indicating the date / time when a predetermined image is generated by shooting if the image is generated by shooting by a shooting device such as a digital camera. So-called shooting date / time information is preferable. If the image is generated using an editor, such as a CG (Computer Graphics) image, the generation date / time or update date / time of the image file may be used as the date / time information.

  As the date and time information, existing tag information defined in various general-purpose image formats represented by JPEG (Joint Photographic Coding Experts Group), TIFF (Tagged Image File Format), Exif (Exchangeable Image File Format), etc. is used. However, the present invention is not limited to this, and information set in a freely usable area such as a maker note may be used.

  As a specific method of grouping using date and time information, for example, as described in Japanese Patent Application Laid-Open No. 2001-228528, a method of forming the same image group when the difference in date and time information is smaller than a threshold value can be given. It is done. More preferably, in the transition of the appearance frequency of the captured image with respect to the date / time information, a valley is detected using a technique of primary differentiation or histogram division, and this is used as an image group separator, or the difference between the date / time information is a threshold value. In the grouping method, the threshold value is preferably converted according to the date / time information (the older the shooting date and time, the larger the threshold value).

  After completion of the grouping process in step S12 or when it is determined in step S11 that the number of grouping target images is equal to or smaller than a predetermined number (step S11; NO), the counter value I for counting the image groups is 1. And at least one image in the I-th image group is calculated with respect to light distribution (forward light intensity, strobe light degree) (step S13).

  The light distribution can be calculated by extracting shooting conditions using tag information such as Exif (for example, an Exif flash tag in which the presence or absence of flash emission at the time of shooting is described) and calculating the light distribution. It is also possible to use a method known in the art, such as calculation by performing image analysis processing. Specifically, the captured image data is divided into regions each having a predetermined combination of brightness and hue, and an occupation ratio indicating the proportion of the entire captured image data is calculated for each of the divided regions. It is preferable to use a method of calculating the light distribution at the time of photographing by performing an operation of multiplying the rate by a coefficient set in advance according to the photographing conditions. A specific example of this light distribution calculation method will be described later in detail with reference to FIGS.

  Next, it is determined whether or not there is an image having a light intensity greater than or equal to a threshold value among the images of the I-th image group (step S14). If it is determined in step S14 that there is no image with a forward light intensity equal to or greater than the threshold (step S14; NO), it is determined whether or not there is an image with a strobe degree equal to or greater than the threshold in the images of the I-th image group. (Step S15). If it is determined in step S15 that there is no image with a strobe degree equal to or greater than the threshold value (step S15; NO), the image with the latest shooting date and time is selected as the representative image from the images of the I-th image group ( Step S21).

  In step S14, when it is determined that there is an image having a light intensity equal to or greater than the threshold (step S14; YES), a face extraction process for extracting a person's face area is performed on the image (step S16). If it is determined in step S15 that there is an image with a strobe degree equal to or greater than the threshold value (step S15; YES), face extraction processing for extracting a human face area is performed on the image (step S16).

  As a method of face extraction processing, for example, a method using a two-dimensional histogram of hue and saturation described in JP-A-6-67320, JP-A-8-122944, JP-A-8-184925, A method using the pattern matching process described in Kaihei 9-138471, a pattern search method described in Japanese Patent Laid-Open No. 9-138471, and the like can be used. In addition, various methods such as a method for determining a human face candidate region using a neural network described in Japanese Patent Laid-Open Nos. 5-282457 and 6-214970 can be used. A specific example of face extraction processing using a neural network will be described later in detail with reference to FIGS.

  When the face extraction process ends, it is determined whether or not a face area has been extracted by the face extraction process (whether there is a face image) (step S17). In step S17, when the face area is not extracted (step S17; NO), the representative image of the I-th image group is selected based on the light distribution calculated in step S13 (step S20). In step S20, an image having the best light distribution, that is, an image having the highest luminous intensity is selected as a representative image from the images of the I-th image group. If there is no image having a light intensity equal to or greater than the threshold value, an image having the highest strobe light is selected as the representative image.

  If a face area is extracted in step S17 (step S17; YES), face area information is acquired from the image from which the face area is extracted (step S18). Even if a face is shown, if the face is not the main subject, it is not appropriate as a representative image. Therefore, in order to select an appropriate representative image, in step S18, the size of the face relative to the image area is used as face area information. One of the face positions, more preferably both, is acquired. In order to select a more preferable representative image, in addition to the face size and the face position, the brightness, saturation, hue, and number of faces of the face may be acquired as face area information.

  When the face area information is acquired, the representative image of the I-th image group is selected based on the face area information (step S19). When selecting based on the size of the face, the selection condition of the representative image is that the occupation ratio of the size of the face (the largest face when there are a plurality of faces) is 2% or more and 50% or less. It is preferable that When selecting based on the position of the face, it is preferable that 80% or more of the face area (the largest face when there are a plurality of faces) is located at the center 50% of the image area. The representative image selection condition is preferably changed according to whether the light distribution of the image is normal light or strobe light.

  When a representative image from which a face area is extracted is selected, correction processing is performed on the representative image so that the extracted face has an appropriate color value, and the representative image after correction processing is processed in step S3 described later. May be displayed on the display unit 12. For example, the average brightness of the face area is acquired from the extracted face area information of the face, and gradation conversion processing is performed so that the average brightness value becomes a predetermined value (for example, L * = 65). May be corrected to an image that is easy to see.

  When the representative image of the I-th image group is selected, it is determined whether or not the counter value I is equal to or less than N−1 (step S22). If it is determined in step S22 that the counter value I is equal to or less than N−1 (step S22; YES), the counter value I is incremented (step S23), and the process of steps S13 to S22 is performed for the new counter value I. Is repeated.

  If it is determined in step S22 that the counter value I is greater than N−1 (step S22; NO), the grouping / representative image selection process ends, and the process proceeds to step S3 in FIG.

  Returning to FIG. When the grouping / representative image selection process is completed, the grouping result and the representative image are displayed on the display unit 12 (step S3). Various methods can be used to display the grouping result on the display unit 12. For example, when the number of grouped image groups is larger than a predetermined number, first, only the representative image is arranged and displayed on the display screen, and the representative image is selected by clicking or the like, thereby corresponding to the representative image. Images belonging to an image group may be displayed.

  Alternatively, the image group may be displayed two-dimensionally such that the top of the image group is arranged in one direction on the display screen and the image belonging to the image group is arranged on the other side, or based on date and time information added to the image. Then, a calendar as shown in FIG. 7 may be displayed. In the calendar display, it is preferable to display on the calendar an index that allows the group date and time range (the latest and oldest range of date and time information in the image group) to be visually recognized. Specifically, the background color, the colored frame, the mark, and the like are displayed on the calendar for easy visual recognition. Further, when a plurality of image groups are obtained, it is preferable to assign a different index for each image group, such as changing a background color or a colored frame for each image group.

  Furthermore, when the group date and time range extends over a plurality of months (for example, July 31 to August 6), as shown in FIG. 7, a plurality of months are displayed continuously without being divided into months. It is preferable to set the date arrangement on the calendar. In this case, in order to distinguish each month, it is preferable to display a boundary line on the calendar or change the background color for each month. Moreover, you may make it mark on the date on which the customer visited the store where the reception terminal 1 was installed on a calendar.

  In the display of the grouping result, it is preferable to display a sign that allows the contents of the images belonging to the image group to be estimated. In addition to the representative image, the number of images in the image group (number of images), the shooting location, and the like can be used as such a sign. It is preferable to set a plurality of signs for one image group. For example, a plurality of representative images can be selected and displayed in order in a slide show format. Alternatively, for example, it is also preferable to set a combination of different types of signs such as a representative image and the number of images. Furthermore, this indicator is preferably set for each date in the image group. For example, for each date, the number of images taken on that day is displayed, or representative images are selected for each date in the group date and time range, and thumbnail images of representative images for each date are displayed on the calendar as shown in FIG. You may make it do. When a representative image is displayed on a calendar for each date, the corresponding representative image may be enlarged and displayed by designating a date on the calendar.

  When the grouping result and the representative image are displayed on the display unit 12, an ordering operation by the user is performed based on the display content (step S4). When the calendar as shown in FIG. 7 is displayed on the display unit 12, in step S4, for example, buttons for ordering all the images included in the image group displayed on the calendar are displayed on the screen (touch panel). You can place an order for each image group by selecting the images in the image group that include the date by specifying a button on the calendar or by specifying a button on the calendar, or by specifying a day on the calendar. It is possible to use the method to do. Also, you can select an image group by specifying on the calendar, display thumbnail images of that image group, confirm all the images in the image group, place a batch order, and if necessary, order images It is also possible to select.

  When the ordering operation is completed, a confirmation message for confirming the execution of printing is displayed on the display unit 12 (step S5). When execution of printing is confirmed by the operation of the operation input unit 13 by the user (step S6; YES), an order confirmation slip is issued from the order confirmation slip issuing unit 16 (step S7), and provisional order information (order) Information) is transmitted to the order management apparatus 2 (step S8).

  The order management apparatus 2 that has received the provisional order information outputs a print output instruction signal to the image printer unit 4 based on the received information (step T1), and the image printer 4 performs print output. The print order acceptance process ends with the print output.

<Modification>
Next, a case where an image print order having solid identification basic information as additional information is placed will be described. The solid identification basic information is data such as a media unique number, a volume label, and a file name. In the service history information stored in the DB unit 3, data such as the file name of a printed image and the service usage status (usage frequency) are stored for each individual identification information (ID number).

An operation according to a modification of the present embodiment will be described.
With reference to the flowchart of FIG. 6, the process (a) executed in the receiving terminal 1 when the print order is received and the process (b) executed in the order management apparatus 2 will be described.

  When the medium M on which the image data is recorded is inserted into the media reading unit 14 of the reception terminal 1, the solid identification basic information is acquired by searching the recorded content of the medium M (step S 30). Individual identification basic information is transmitted to the order management apparatus 2.

  In the order management device 2, the DB unit 3 searches for the solid identification information that matches the received solid identification basic information, thereby reading the service history information corresponding to the solid identification basic information (step T10). Information indicating the presence or absence (whether there is corresponding service history information) is transmitted to the receiving terminal 1 (step T11). If there is service history information, in step T11, service history information is also transmitted in addition to the presence / absence information of matching information.

  In the reception terminal 1, it is determined from the received information from the order management device 2 whether there is matching information (step S31). If it is determined in step S31 that there is matching information (step S31; YES), an unprinted image is selected from the images recorded on the medium M based on the service history information received from the order management apparatus 2. (Step S32). If it is determined in step S31 that there is no match information (step S31; NO), all images recorded on the medium M are selected (step S33).

  When an image is selected in step S32 or S33, a grouping process for extracting an image group from the grouping target image is performed using the selected image as a grouping target image (step S34), and a representative image is selected for each image group. A representative image selection process to be selected is performed (step S35). Next, the grouping result in step S34 and the representative image selected in step S35 are displayed on the display unit 12 (step S36). Since the processes of steps S34 to S36 are the same as steps S1 to S3 of FIG. 4, description thereof is omitted here.

  Next, an ordering operation by the user is performed based on the content displayed on the display unit 12 in step S36 (step S37). When the ordering operation is completed, a confirmation message for confirming the execution of printing is displayed on the display unit 12 (step S38). When the execution of the printing is confirmed by the operation of the operation input unit 13 by the user (step S39; YES), the order confirmation slip is issued from the order confirmation slip issuing portion 16 (step S40), and the provisional order information (order) Information) is transmitted to the order management apparatus 2 (step S41).

  The order management apparatus 2 that has received the provisional order information outputs a print output instruction signal to the image printer unit 4 based on the received information (step T12), and the image printer 4 performs print output.

  Next, the order management apparatus 2 determines whether or not the customer of the current print order is new based on the presence / absence of matching information in step T10 (step T13). In step T10, when there is no solid identification information that matches the solid identification basic information received from the reception terminal 1 in the DB unit 3, it is determined as a new customer in step T13 (step T13; YES). New solid identification information is issued for the print order, and the solid identification information is registered in the DB unit 3 (step T14).

  In step T13, if it is determined that the customer of the current print order is not new (step T13; NO), or if solid identification information is newly registered in step T14, the DB unit is based on the current print order. 3 is updated (step T15), and the print order acceptance process is terminated.

  Next, the light distribution calculation process (step S13 in FIG. 5) will be described in detail with reference to the flowchart in FIG.

  First, for each photographic image data, the photographic image data is divided into predetermined image areas, and an occupancy ratio calculation process is performed to calculate an occupancy ratio indicating the ratio of each divided area to the entire photographic image data (step S45). The occupation rate calculation process in step S45 will be described in detail later with reference to FIGS.

  Next, based on the occupation ratio calculated in step S45 and a coefficient set in advance according to the shooting conditions, an index (index 1 to 5) for specifying the light distribution (light source state) is calculated, Based on the calculated index, the light distribution at the time of photographing (forward light intensity, backlight intensity, strobe intensity) is calculated (step S46). Here, the index for specifying the light distribution is a numerical value that quantitatively represents the light distribution at the time of photographing. The index calculation process and the light distribution calculation process in step S46 will be described in detail later.

<Occupancy rate calculation process>
Next, the occupation rate calculation process shown in step S45 of FIG. 8 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S50). FIG. 10 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 10, the values of the digital image data as the input image data are defined as InR, InG, InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S51). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the light distribution calculation is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following equation (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the equation (1) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance Y = InR × 0.30 + InG × 0.59 + InB × 0.11
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formula (1).

When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S52), and the main occupancy ratio calculation is performed. Processing ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting obtained by discriminant analysis, that is, the lightness state of the face area at the time of strobe shooting. Show. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupation ratio Rij of each divided area shown in Table 1 is multiplied.

  FIG. 11 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 11, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 13 shows a curve (coefficient curve) in which the first coefficient in the skin color area (H1) and the first coefficient in the other areas (green hue area (H3)) continuously change over the entire brightness. It is shown. According to Table 2 and FIG. 13, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(2-1)〜式(2-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) (2-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) (2-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) (2-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) (2-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (2-1) to (2-4).
H1 area sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) (2-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) (2-2)
Sum of H3 area = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) (2-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) (2-4)

The index 1 is defined as Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 1 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 4.424 (3)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy of backlight photographing obtained by discriminant analysis, that is, the brightness state of the face region at the time of backlight photographing. Show. The coefficient of each divided area shown in Table 3 is a weighting coefficient that is multiplied by the first occupation ratio Rij of each divided area shown in Table 1.

  FIG. 12 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupancy calculated from the area (r4) distributed in the intermediate lightness of the flesh color hue area in FIG. 12, and the low lightness (shadow) area ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 14 shows the second coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 14, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(4-1)〜式(4-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) (4-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) (4-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 (4-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 (4-4) When the second coefficient in the lightness area vi and the hue area Hj is Dij, the sum of the Hk areas for calculating the index 2 is defined as in Expression (4).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) (4-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (4-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (4-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (4-4)

The index 2 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (5)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, they are effective in calculating the light distribution when the captured image data is a color image.

  Next, the occupation rate calculation process (step S45 in FIG. 8) for calculating the index 3 will be described in detail with reference to the flowchart in FIG.

  First, the RGB values of the captured image data are converted into the HSV color system (step S60). Next, the photographed image data is divided into regions each composed of a combination of the distance from the outer edge of the photographed image screen and brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S61). Hereinafter, the area division of the captured image data will be described in detail.

  FIGS. 16A to 16D show four regions n1 to n4 divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 16A is an outer frame, a region n2 shown in FIG. 16B is a region inside the outer frame, and a region n3 shown in FIG. A region n4 shown in FIG. 16D is an inner region and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S62), and the main occupancy ratio calculation is performed. The process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１７は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is obtained by discriminant analysis.

FIG. 17 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(6-1)〜式(6-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 (6-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 (6-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 (6-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) (6-4) If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (6).

Accordingly, the sum of the n1 to n4 regions is expressed as the following formulas (6-1) to (6-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (6-1)
Sum of n2 area = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (6-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (6-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) (6-4)

The index 3 is defined as in Expression (7) using the sum of the N1 to H4 regions shown in Expressions (6-1) to (6-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (7)
Since the index 3 is calculated based on the compositional feature (distance from the outer edge of the screen of the captured image data) according to the distribution position of the brightness of the captured image data, the light distribution of not only the color image but also the monochrome image is calculated. It is also effective to do.

The index 4 is defined as Expression (8) using the indices 1 and 3, and the index 5 is defined as Expression (9) using the indices 1 to 3.
Index 4 = 0.565 x Index 1 + 0.565 x Index 3 + 0.457 (8)
Indicator 5 = (-0.121) x Indicator 1 + 0.91 x Indicator 2 + 0.113 x Indicator 3-0.072 (9)
Here, the weighting coefficient by which each index is multiplied in Expression (8) and Expression (9) is set in advance according to the shooting conditions.

このように指標４、５の値により、配光性を定量的に算出することができる。 <Calculation method of light distribution>
Next, the light distribution calculation method (step S46 in FIG. 8) will be described.
In FIG. 18, 60 images were taken under each condition of forward light, backlight, and strobe, and indexes 4 and 5 were calculated for a total of 180 digital image data, and the values of indexes 4 and 5 under each condition were plotted. Is. According to FIG. 18, when the value of index 4 is greater than 0.5, there are many strobe scenes, and when the value of index 4 is 0.5 or less and the value of index 5 is greater than −0.5, the backlight scene is I understand that there are many. Table 6 shows the calculation contents of the light distribution by the values of the indexes 4 and 5.

Thus, the light distribution can be calculated quantitatively based on the values of the indexes 4 and 5.

<Face extraction processing conditions>
Next, a method for determining processing conditions for the face extraction process (step S16 in FIG. 5) will be described in detail.

  The determination of the processing conditions for the face extraction process executed according to the light distribution at the time of shooting is performed in order to improve the extraction accuracy of the face extraction process. As a processing method of the face extraction processing, a face candidate area is extracted from captured image data, and it is determined whether or not the extracted face candidate area is a person's face. A human face is included in the face candidate area. For example, there is a method for determining whether or not it is. When such a method is used, the extraction accuracy of the face extraction process includes the extraction accuracy of the face candidate region and whether the face candidate region is a human face (or whether a face is included in the face candidate region). Both of the determination accuracy contribute to determining whether or not the face extraction processing conditions are determined. When the processing conditions for the face extraction process are determined, the processing conditions for the process of determining the face of the person from the face candidate area Either one or both are preferably determined according to the light distribution calculation result.

  Hereinafter, the process of extracting a face candidate area from captured image data and the process of determining whether or not the extracted face candidate area is a person's face are referred to as a “face extraction process”. Of these, the part of the process for determining whether or not the face candidate area is a person's face is referred to as “face determination process”.

First, a method for determining the extraction condition of the face candidate area will be described.
As a method for extracting a face candidate area, a method can be used in which an initial point corresponding to a skin color is searched from captured image data, and a skin color area is extracted as a face candidate area using a simple area expansion method. The simple area expansion method is an image processing method for extracting a specific image area by expanding adjacent areas with pixels adjacent to each other whose data difference between pixels is equal to or less than a threshold value belonging to the same image area. . In other words, the simple area expansion method starts from an initial pixel that matches a specified condition, and a data difference between adjacent pixels (either 4-connected or 8-connected) is equal to or less than a threshold value. In this case, it is assumed that the adjacent pixel and the initial pixel belong to the same image area, and the same determination is performed for a pixel adjacent to the pixel that belongs to the same pixel. In this manner, starting from the initial pixel, a specific image region is extracted by gradually expanding the same image region. In the present embodiment, it is preferable to further use a process for extracting a face component region by extracting an edge component in an image and ending simple region expansion when the image edge is reached.

  When a skin color area is extracted as the face candidate area, even if the subject is the same person, the color of the skin color area varies significantly depending on the light distribution at the time of shooting. For example, based on the skin color of an image taken in front light, the skin tone becomes whiter (especially the closer it is) in strobe shooting, and if it is shooting in backlight, Skin tone becomes darker. In consideration of such a situation, it is preferable to switch and / or adjust (stepwise or continuously) the search condition for the initial pixel in accordance with the light distribution calculation result.

  For example, for a predetermined basic search condition for searching for an initial pixel, if the light distribution is “forward light”, the basic search condition is used. If the light distribution is “strobe”, the basic search condition is used. If the search condition with the brightness corrected (brighter) is used and the light distribution is “backlight”, the search condition with the brightness reduced (darker) from the basic search condition may be used. As described above, when extracting a face candidate region using the simple region expansion method, the search condition for the initial pixel may be switched according to the light distribution at the time of photographing. Moreover, since the edge intensity | strength which complete | finishes simple area expansion differs with light distribution, it is preferable to adjust the edge extraction conditions for ending simple area expansion according to light distribution.

  When the calculation result of the light distribution is obtained as an index (indicator 4, 5) indicating the light distribution stepwise or continuously, such as forward light intensity, strobe light intensity, backlight intensity, etc., the value of the obtained index Accordingly, the search condition for the initial pixel may be determined by adjusting the degree of increasing the brightness (in the case of a strobe) and the degree of decreasing in the brightness (in the case of backlight). Therefore, the simple region expansion method using the initial pixel search condition can be determined as the extraction condition for the face candidate region.

  In the above description, the example in which the search condition for the initial pixel is corrected using the fact that the brightness difference is generated in the skin color area according to the light distribution is shown. However, not only the brightness but also saturation is used. Thus, the search condition correction requirement may be increased. For example, with flash and flash photography, the saturation is lower than with direct light. Further, not only the initial pixel search condition but also the area expansion condition may be corrected. Furthermore, it is preferable to correct the initial pixel search condition and the region expansion condition for various processing conditions for extracting the face candidate region, reflecting the difference in the characteristics of the skin color region obtained for each light distribution.

  Next, a method for determining a face determination processing condition for determining a face from a face candidate area will be described. In the following, a method for determining a face determination processing condition when a feature amount is extracted from a face candidate region and face determination processing is performed using a neural network based on the extracted feature amount will be described.

  First, prior to a specific description of a method for determining face determination processing conditions, a neural network used in face determination processing will be briefly described. As shown in FIG. 19, this neural network is a hierarchical neural network having an input layer, an intermediate layer, and an output layer (hereinafter simply referred to as a neural network). The neural network preferably uses a digital neural network chip, but can also be realized by using a general-purpose DSP (Digital Signal Processor) and a dedicated emulation program, or using a normal CPU and an emulation program. It doesn't matter.

ここで、ｆ（ｘ）は一般に式（１１）に示すような非線形のシグモイド関数が用いられるが、必ずしもこれに限定されるわけではない。

Each layer of the neural network is composed of structural units called neurons. Except for the input layer, an arbitrary neuron receives a large number of inputs called synapses as shown in FIG. 20, and multiplies each input value x _i by a predetermined weight w _i called connection strength to sum the sum. It functions as a functional element that obtains and evaluates a result obtained by evaluating it with a predetermined output function f as an output y. The output function f is expressed as in Expression (10).

Here, f (x) is generally a non-linear sigmoid function as shown in Equation (11), but is not necessarily limited thereto.

Here, the input weight w _i of each neuron is important information for determining the relationship between the input and output of the neural network, in other words, the operation of the neural network, and is determined by performing a learning operation on the neural network. In FIG. 19, only one intermediate layer is shown, but this may have a configuration of one or more arbitrary hierarchies. Hierarchical networks are known to be able to achieve any function that can be synthesized if there is only one intermediate layer, but increasing the number of intermediate layers is advantageous in terms of learning efficiency and the number of neurons. It has been known.

  A characteristic of the neural network is a generalization ability as a system. This is not necessarily achieved by learning by using teacher data (a pair of an input pattern and a desired output pattern for the input pattern. Usually, a plurality of pairs are prepared. This is the ability to give an almost correct output even for an unlearned input pattern that is not given.

  Various learning methods using teacher data for neural networks are known. For example, by performing a sufficient number of learning operations using the widely used back propagation learning method (Back Propagation learning method). Thus, an output approximate to the teacher data can be obtained.

  Next, a method for determining the face determination processing condition will be specifically described. If the same face determination processing conditions are used even though the light distribution is different, a difference in face determination accuracy occurs. This is because the feature amount contributing to the face determination process is different for each light distribution and the extraction condition for optimally extracting the feature amount is different. Therefore, it is preferable to determine the face determination processing condition according to the light distribution.

  Specifically, it is preferable to determine the learning result of the neural network used for the design of the neural network and the face determination process according to the light distribution. Here, the design of the neural network indicates, for example, designing the type and number of input parameters of the neural network, the number of intermediate layers, and the like. The input parameter includes a feature amount extracted from the face candidate region. The learning result of the neural network represents a weight (connection strength) assigned to each neuron constituting the neural network, a slope value of the sigmoid function, and the like.

  Regardless of the nature of the feature quantity used as the input parameter of the neural network and the light distribution, if the feature quantity is extracted from the face candidate region under the same extraction condition, the extraction accuracy differs. Further, depending on the nature of the feature quantity, a difference occurs in the separability between a person's face and a part other than the face according to the light distribution. For this reason, it is preferable to change the feature amount extracted from the face candidate region and the feature amount extraction condition in accordance with the light distribution.

  For example, using features that make up the face such as eyes and mouth as feature values, and extraction conditions for feature values, differences in skin and eyes or mouth color (skin color and almost black, skin color and redness) and contours of parts In the case of using the extraction condition using the edge of the backlight, there is a tendency for the backlight to be backlit and the backlight to be less accurate in extracting feature quantities. That is, in the backlight, the entire face of the person is photographed darker than the normal light, so that the difference between the color values of the skin and the part is small, and the edge of the contour is difficult to extract compared to the normal light.

  In this way, the feature quantity extraction conditions may be switched or the type and number of input parameters of the neural network may be changed according to the light distribution. Further, the feature amount extraction condition may be adjusted stepwise or continuously in accordance with the calculated index values (forward light intensity, strobe light intensity, backlight intensity, etc.). It is also preferable to change the number of intermediate layers and the number of neurons in the neural network according to the number of input parameters.

  Further, when determining the face determination processing conditions, it is preferable to switch the neural network previously learned in each of the forward light, strobe, and backlight images according to the light distribution. In addition, if the light distribution calculation result is a system in which the light distribution is obtained as a stepwise or continuous index such as forward light intensity, strobe light intensity, and reverse light intensity, a neural network can be connected to an intensity strobe light. If you prepare it step by step according to the degree of light distribution, such as for light, weak strobe, strong backlight, weak backlight, etc., it will improve the judgment accuracy of face judgment processing Can do. In addition, it is preferable to operate the system by appropriately updating it using a version upgrade method of the self-learning neural network.

<Face extraction processing>
Next, the face extraction process shown in step S16 of FIG. 5 will be described in detail with reference to the flowchart of FIG.

  First, in accordance with the processing conditions of face extraction processing (face candidate region extraction conditions), face candidate regions are extracted from photographed image data with a light intensity equal to or greater than a threshold value or strobe strength equal to or greater than a threshold value (step S70). It is determined whether or not there is a face candidate area (step S71).

  If it is determined in step S71 that there is no face candidate area in the captured image data (step S71; NO), the face extraction process ends. If it is determined in step S71 that there is a face candidate area in the captured image data (step S71; YES), face determination is performed to determine whether the face candidate area extracted in step S70 is a human face. Processing is performed (step S72), and the face extraction processing ends. The face determination process in step S72 will be described in detail later with reference to FIG.

  In the above description, the simple area expansion method is used as an example of the face candidate area extraction process. However, the face candidate area extraction method is not limited to this. For example, it is possible to use a method of extracting an edge from photographed image data and extracting a specific shape pattern such as an ellipse using a technique such as Hough transform or template matching.

  Furthermore, as disclosed in JP-A-4-346332, an original image (photographed image data) is divided into a large number of pixels, and a histogram of hue values and saturation values is created from the RGB values of each pixel, A method is used in which a histogram is divided from the shape, an image is divided into areas composed of pixels corresponding to the divided parts, and an area corresponding to a human face is estimated from the divided areas. be able to.

  Further, as disclosed in JP-A-6-309433, JP-A-6-67320, JP-A-5-158164, and JP-A-5-165120, a hue value, a saturation value, and a luminance value are disclosed. A method of determining a skin color area based on a histogram such as the above and using the skin color area as a face candidate area can be used. Further, a line process as disclosed in JP-A-8-221567, JP-A-2000-20694, JP-A-2000-32272, JP-A-2000-201158, and JP-A-2000-20769. The face candidate region can be extracted using the method.

  In addition, an edge and a low frequency image are generated from a color digital image, and predetermined parameters (for example, saturation value, hue value, and lightness value) are initially matched with predetermined initial conditions (values in each predetermined range). Extract a pixel, perform simple area expansion from the extracted initial pixel for the low-frequency image, and forcibly end simple area expansion when the image edge is reached to extract a face candidate area May be.

  Next, the face determination process shown in step S72 of FIG. 21 will be described with reference to the flowchart of FIG.

  First, a feature value is extracted from the face candidate region according to the feature value determined as the processing condition of the face extraction process and the extraction condition (step S80). Next, the neural network determined according to the light distribution (for example, when the light distribution is backlit) using the feature amount extracted in step S80 as the input signal (input parameter) and the parameter representing the facial appearance as the output signal. The neural network learned for backlighting is used to determine whether the face candidate area is a human face (step S81), and the face determination process ends.

  As described above, according to the reception terminal 1 of the present embodiment, an image suitable as a representative image is obtained by selecting a representative image based on at least one of the light distribution of the images in the image group and the face area information. Selectable. Therefore, it is easy to assume the contents of the image group.

  Note that the description in the present embodiment can be changed as appropriate without departing from the spirit of the present invention.

  For example, in the present embodiment, the case where the reception terminal 1 side performs the overall processing related to the print order is shown. However, when the reception terminal 1 has low processing capability, the processing target image is transmitted to the external device, and the external terminal It is also possible to distribute the processing load by performing processing with a high processing load (for example, light distribution calculation processing, face extraction processing) on the device side. On the other hand, depending on the processing capability of the reception terminal 1, the function of the order management device 2 may be added to the reception terminal 1 side.

1 is a diagram showing a configuration of an image processing system according to an embodiment of the present invention. The block diagram which shows the principal part structure of the reception terminal which concerns on this embodiment. The block diagram which shows the principal part structure of the order management apparatus which concerns on this embodiment. 6 is a flowchart illustrating processing executed by a reception terminal and an order management apparatus when a print order is received. The flowchart which shows the grouping and representative image selection process performed by a reception terminal. 9 is a flowchart illustrating processing executed by a reception terminal and an order management apparatus when a print order is received according to a modification of the embodiment. The figure which shows an example of the calendar displayed on the display part of a reception terminal. The flowchart which shows the light distribution calculation process performed with a reception terminal. The flowchart which shows the occupation rate calculation process which calculates the 1st occupation rate for every area | region of brightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. The figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. The flowchart which shows the occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient for multiplying the 2nd occupation rate for calculating the parameter | index 3 according to area | region (n1-n4). The plot figure of the parameter | index 4 and the parameter | index 5 calculated according to light distribution (forward light, strobe, back light). The figure which shows the structure of a hierarchical neural network typically. The figure which shows the structure of the neuron which comprises a neural network. The flowchart which shows the face extraction process performed with a reception terminal. The flowchart which shows a face determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Reception terminal 11 Control part 12 Display part 13 Operation input part 14 Media reading part 15 Storage part 16 Order confirmation slip issuing part 17 Communication part 2 Order management apparatus 21 Control part 22 Display part 23 Operation input part 24 Storage part 25 I / F Unit 26 communication unit 27 DB reference unit 3 DB unit 4 image printer unit 5 image media writing unit 100 image processing system

Claims (18)

An extraction step of extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording unit in which an image having date and time information is recorded as additional information;
A selection step of selecting a sign image suggesting the content of an image included in the image group from the images belonging to the extracted image group, and an image processing method comprising:
The image processing method characterized in that, in the selection step, light distribution is calculated for at least one image in the image group, and the marker image is selected based on a calculation result of the light distribution. An extraction step of extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording unit in which an image having date and time information is recorded as additional information;
A selection step of selecting a sign image suggesting the content of an image included in the image group from the images belonging to the extracted image group, and an image processing method comprising:
In the selection step, face extraction processing is performed to extract a human face area for at least one image in the image group, and the sign image is selected based on face area information of the image from which the face area has been extracted. An image processing method characterized by the above. An extraction step of extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording unit in which an image having date and time information is recorded as additional information;
A selection step of selecting a sign image suggesting the content of an image included in the image group from the images belonging to the extracted image group, and an image processing method comprising:
In the selection step, light distribution is calculated for at least one image in the image group, and a face extraction process for extracting a human face area is performed, and the light distribution calculation result and the face area are extracted. An image processing method comprising: selecting the sign image based on face area information of a captured image.   The image processing method according to claim 1, wherein, in the selection step, a light intensity and a strobe degree are calculated as the light distribution.   4. The image processing method according to claim 2, wherein the face area information includes at least one of a face size and a face position. An operation step of receiving an operation of selecting image data to be printed out of the image data in the image group;
A transmission step of transmitting the selected image data as image data for printing to an image forming unit;
The image processing method according to claim 1, further comprising: Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which images having date and time information are recorded as additional information;
A selection means for selecting a sign image suggesting the content of an image included in the image group from the images belonging to the extracted image group,
The image processing apparatus, wherein the selection unit calculates a light distribution for at least one image in the image group and selects the marker image based on a calculation result of the light distribution. Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which images having date and time information are recorded as additional information;
A selection means for selecting a sign image suggesting the content of an image included in the image group from images belonging to the extracted image group,
The selection means performs a face extraction process for extracting a human face area for at least one image in the image group, and selects the sign image based on face area information of the image from which the face area is extracted. An image processing apparatus. Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which images having date and time information are recorded as additional information;
A selection means for selecting a sign image suggesting the content of an image included in the image group from the images belonging to the extracted image group,
The selection means calculates a light distribution for at least one image in the image group and performs a face extraction process for extracting a human face area, and the calculation result of the light distribution and the face area are extracted. An image processing apparatus, wherein the sign image is selected based on face area information of the captured image.   The image processing apparatus according to claim 7, wherein the selection unit calculates a forward light degree and a strobe degree as the light distribution.   The image processing apparatus according to claim 8, wherein the face area information includes at least one of a face size and a face position. In the reception terminal for receiving print orders from customers,
The image processing apparatus according to any one of claims 7 to 11,
Operation means for accepting an operation of selecting image data to be printed out of the image data in the image group;
Transmitting means for transmitting the selected image data as image data for printing to image forming means;
A reception terminal comprising: A computer that controls image processing
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
Light distribution is calculated for at least one image in the extracted image group, and based on the calculation result of the light distribution, an image included in the image group is selected from images belonging to the image group. A selection means for selecting a sign image suggesting the content,
Image processing program to function as A computer that controls image processing
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
A face extraction process is performed to extract a human face area for at least one image in the extracted image group, and based on the face area information of the image from which the face area is extracted, Selecting means for selecting a sign image suggesting the content of the image included in the image group;
Image processing program to function as A computer that controls image processing
Extraction means for extracting at least one image group based on a predetermined condition from a plurality of images recorded in a recording means in which an image having date and time information is recorded as additional information;
The light distribution is calculated for at least one image in the extracted image group, and a face extraction process for extracting a human face area is performed, and the calculation result of the light distribution and the image from which the face area is extracted Selecting means for selecting a sign image suggesting the content of an image included in the image group from images belonging to the image group based on the face area information of
Image processing program to function as   The image processing program according to claim 13 or 15, wherein the selection unit calculates a light intensity and a strobe degree as the light distribution.   16. The image processing program according to claim 14, wherein the face area information includes at least one of a face size and a face position. The computer,
Operation means for accepting an operation of selecting image data to be printed out of the image data in the image group;
Transmitting means for transmitting the selected image data as image data for printing to an image forming means;
The image processing program according to any one of claims 13 to 17, wherein the image processing program is made to function as:

Images