JP2004298285A - Judgment device and judgment method of walking condition and pedestrian attribute - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a judgment device and a judgment method of a walking condition and a pedestrian attribute with which judgment of the walking condition and the pedestrian attribute can be performed with high accuracy and wide utility. <P>SOLUTION: A photographing apparatus 11 takes a picture of a pedestrian and obtains a two-dimensional picture every constant period of time. A height sensor 13 detects the height of the pedestrian and a speed sensor 14 detects a travel speed of the pedestrian. A controlling member 161 detects a vertex of the pedestrian in the two-dimensional picture and acquires a movement track of the vertex on a time axis from a temporal transition of the position of the vertex in the picture. An FFT 18 performs a frequency analysis of the orbit and acquires a frequency spectrum of the movement of the vertex. The controlling member 161 judges a basic frequency of the walking motion from the frequency spectrum and compares the basic frequency with a predetermined model of the walking condition to judge the walking condition of the pedestrian. The pedestrian attribute is judged by the speed and the height of the pedestrian detected by the height sensor 13 and the speed sensor 14 in relation to the walking condition. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for determining a pedestrian's walking state and attributes using image processing.
[0002]
[Prior art]
As a method for determining the walking state, a method using a pressure sensor has been proposed (for example, see Non-Patent Document 1). This method measures the foot pressure of a pedestrian using a pressure sensor formed on a large area, and analyzes the distribution of foot pressure and the spatial parameters of walking (stride, overlapping stride, stride) and temporal parameters (one step, This is a method of displaying the pedestrian's walking state by displaying the overlapping walking time, the free leg time, and both foot contact time.
On the other hand, as a method of determining a walking state, there is a method of obtaining information such as a walking speed and a stride over a wide range at low cost by using an image (for example, see Patent Document 1).
[0003]
In addition, as a method for estimating pedestrian attributes, a method of estimating gender from the shape characteristics of the whole body (for example, see Non-Patent Document 2) and a method of estimating age from characteristics of a pedestrian's face (for example, , Non-Patent Document 3). The method in Non-Patent Document 2 focuses on the fact that gender identification information such as differences in clothes and hairstyles appears in the shape of a human silhouette, performs morphological processing on a silhouette image of a person, and identifies males and females from the differences in the pattern spectrum. Method. The method in Non-Patent Document 3 uses a CCD camera to photograph a person's face, performs weblets transform processing on each feature point of the face to determine the directionality and frequency of each point, and obtains them in advance. This is a method of recognizing a face as compared with a standard face image.
[0004]
[Non-patent document 1]
Development of walking pattern measurement device using large area pressure sensor (Transactions of the Institute of Electronics, Information and Communication Engineers D-II Vol.J84-D-II No.2 pp.380-389)
[Patent Document 1]
JP-A-11-339139
[Non-patent document 2]
A gender discrimination method using a very small amount of pattern spectrum by morphological processing (Transactions of the Institute of Electronics, Information and Communication Engineers D-II Vol. J80-D-II No. 5 pp. 1037-1045)
[Non-Patent Document 3]
Face Recognition under Free Walking (Interaction '99 (Information Processing Society of Japan) IA-01 March 1999)
[0005]
[Problems to be solved by the invention]
In a method of determining a walking state using a pressure sensor, a thin film sensor is used. Such a sensor is very delicate, and in a situation where the weight of a human is hung or a person is strongly kicked with a foot, anxiety arises in terms of durability. Further, since a sensor having a large area corresponding to a space where humans can walk is required, the price is high.
[0006]
On the other hand, when an image is used, it is extremely difficult to determine the position of the foot and the state of the foot in a situation where the foot of the pedestrian is hidden by some obstacle. Therefore, it is difficult to determine the absolute value of the speed because the position in the real space cannot be specified. For this reason, the method of Patent Literature 1 that obtains information on the speed and the stride from the image is extremely limited in the situations where it can be used.
[0007]
Regarding the pedestrian attribute estimation method, in any of the above methods (sex estimation method from whole body shape feature, age estimation method from face feature), whole body or face image is clearly displayed in a stable state. It is necessary to be photographed, and the use scene is limited, so that it lacks practicality.
[0008]
The present invention has been made in view of the above situation, and has as its object to enable highly versatile determination of a walking state and determination of a pedestrian attribute while maintaining high accuracy. Another object of the present invention is to make it possible to determine a walking state in a scene where a foot of a pedestrian is not visible due to some obstacle (such as a store shelf).
[0009]
[Means for Solving the Problems]
In order to achieve the above object, a walking state determination device according to a first aspect of the present invention includes:
Storage means for storing a series of two-dimensional images taken of a three-dimensional space;
A specific part detection means for detecting a specific part of the detection target person in the two-dimensional image stored in the storage means,
Trajectory calculation means for calculating a movement trajectory on the time axis in the two-dimensional image of the specific part detected by the specific part detection means,
Frequency analyzing means for obtaining a frequency spectrum of the locus calculated by the locus calculating means,
Walking state determining means for determining a walking state based on the frequency spectrum obtained by the frequency analyzing means,
It is characterized by having.
[0010]
The walking state determination device further includes:
Storage means for storing statistical case data of the fundamental vibration frequency of the walking motion;
Means for classifying the fundamental vibration frequency of the frequency spectrum into one of the categories based on the case data, and determining a walking state of the detection target person by the category assigned to the category;
May be provided.
[0011]
The pedestrian attribute determination device according to the second aspect of the present invention includes:
Speed detection means for detecting the speed of the detection target person,
Storage means for storing statistical case data of walking speed;
Means for classifying the speed of the detection target person based on the case data,
Means for determining the attribute of the detection target person based on the classification of the fundamental vibration frequency and the classification of the speed,
It is characterized by having.
[0012]
The pedestrian attribute determination device according to the third aspect of the present invention includes:
In the walking state determination device, height detection means for detecting the height of the detection target person,
Storage means for storing statistical height case data;
Classification means for classifying the height of the detection target person based on the case data,
Attribute determination means for determining the attribute of the detection target person based on the classification of the fundamental vibration frequency and the classification of the height,
It is characterized by having.
[0013]
The walking state and the pedestrian attribute determining method according to the fourth aspect of the present invention include:
Input a two-dimensional image of a three-dimensional space,
Detect the target person on the input two-dimensional image,
Calculate the moving trajectory of the detected subject in three-dimensional space-time,
Based on the calculated trajectory, the fundamental vibration frequency of the subject's motion is determined,
Detect the speed of the detected subject,
Detects the height of the detected subject,
Based on the detected fundamental vibration frequency, speed, and height, determine the walking state and pedestrian attribute of the subject,
It is characterized by the following.
[0014]
A program according to a fifth aspect of the present invention includes:
Computer
Storage means for storing a continuous two-dimensional image on a time axis taken of a three-dimensional space,
A specific part detection means for detecting a specific part of the detection target person in the two-dimensional image stored in the storage means,
Trajectory calculation means for calculating a movement trajectory in three-dimensional space-time of the specific part detected by the specific part detection means,
Frequency analysis means for extracting the frequency spectrum of the movement trajectory calculated by the trajectory calculation means,
Based on the frequency spectrum extracted by the frequency analysis unit, a walking state and a pedestrian attribute determining unit that determines a walking state and an attribute of the subject,
It is characterized by functioning as
[0015]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a walking state / pedestrian attribute determination device and a determination method according to an embodiment of the present invention will be described with reference to the drawings.
[0016]
As shown in FIG. 1, a walking state / pedestrian attribute determining device 1 according to the present embodiment includes an imaging device (camera) 11, an image buffer 12, a height sensor 13, a speed sensor 14, an input process, It comprises a unit 15, a computer 16, a large-capacity hard disk (HDD) 17, and a frequency analysis processor (FFT) 18.
[0017]
The imaging device 11 is configured by a digital still camera or the like, and is installed so as to photograph a pedestrian passing through the real space RS at an arbitrary position in the three-dimensional real space RS, as schematically shown in FIG. . The height T of the installation position of the imaging device 11 and the inclination α of the optical axis are obtained in advance. The imaging device 11 acquires a still image in the three-dimensional real space at regular intervals of Δt, and outputs image data in an arbitrary format.
[0018]
The image buffer 12 temporarily stores the image data sent from the imaging device 11 and sequentially sends out the image data in accordance with the transfer speed of the input processing unit 15.
[0019]
The height sensor 13 detects the height (tall or low) of a person passing through the three-dimensional real space RS. As shown in FIG. 3, the height sensors 13 are arranged at different height positions on one side of the passage (the three-dimensional real space RS), and each emit a light beam horizontally toward the other side of the passage. 131 and 132, and photodetectors 133 and 134 arranged on the other side of the traffic area.
[0020]
When the passerby passes the installation position of the height sensor 13 and the light receivers 133 and 134 cannot receive any of the light beams from the light emitters 131 and 132, the passerby is a tall person; If the 134 can receive the light beam from the light emitter 132 and the light receiver 133 cannot receive the light from the light emitter 131, the pedestrian is a short person.
[0021]
The speed sensor 14 detects the speed of the pedestrian when the pedestrian crosses the light beam emitted by one of the light emitters 131 and 132 of the height sensor 13 (blocks the beam). That is, when the light receiver 133 cannot detect the light beam, it issues a trigger signal to the speed sensor 14. The speed sensor 14 detects the speed of the pedestrian at the position where the light beam is being emitted in response to the trigger signal.
[0022]
The height sensor 13 and the speed sensor 14 supply measurement results to the input processing unit 15.
[0023]
The input processing unit 15 is connected to the HDD 17 via a communication network. As schematically shown in FIG. 4, the input processing unit 15 associates the image data of a series of still images IM continuously supplied from the imaging device 11 via the image buffer 12 with the shooting time (or order) t. And provide it to the HDD 17.
[0024]
In addition, the input processing unit 15 divides the height of the person who has crossed the light beam into two stages, high and low, based on the output signals of the two light receivers 133 and 134 of the height sensor 13. Further, based on the output of the speed sensor 14, the speed of the person crossing the light beam is divided into four stages of 1 to 4 (note that such a classification corresponds to the classification in the classification table described later with reference to FIG. 9). Do). Next, for example, “t = t ₃ , Tall = high, speed = 3 ”and transmitted to the HDD 17.
[0025]
The computer 16 includes a control unit 161, a memory 162, an input unit 163, and a display unit 164.
[0026]
The memory 162 includes a ROM 162a and a RAM 162b as shown in FIG.
[0027]
The ROM 162a includes a flash memory, a hard disk device, and the like, and stores an operation program that defines the operation of the control unit 161. In addition, the ROM 162a stores control data for controlling the operation of the control unit 161 and statistical data such as a basic frequency, a speed, and a height of the pedestrian when the pedestrian walks, as schematically shown in FIG. The classification data classified based on the typical value is stored.
[0028]
The RAM 162b functions as a work area of the control unit 161.
[0029]
The input unit 163 illustrated in FIG. 1 includes a keyboard, a mouse, and the like, and inputs various commands and data to the control unit 161.
[0030]
The display unit 164 includes a CRT, a liquid crystal display device, and the like, and displays various images such as a two-dimensional image acquired by the imaging device 11. In addition, the trajectory on the image of the top of the human head walking in the three-dimensional real space RS and the result of frequency analysis of the trajectory by the FFT 18 are displayed.
[0031]
The control unit 161 is for controlling the operation of the entire computer 16 and is constituted by a microprocessor or the like, and operates using the RAM 162b as a work area according to an operation program stored in the ROM 162a. The control unit 161 obtains a trajectory on the image of the pedestrian who walks in the three-dimensional real space RS. It determines and outputs the attribute of the pedestrian, such as noh.
[0032]
Specifically, the control unit 161 1) reads a series of two-dimensional images IM stored in the HDD 17 and 2) detects a feature point (head finger) of a detection target (human) in each two-dimensional image IM. 3) determine the position of the feature point to be detected in each two-dimensional image IM; 4) join the positions of the feature points in each two-dimensional image IM to obtain a series of feature points on a series of two-dimensional images IM. 5) The trajectory is sent to the frequency analysis processing unit FFT18 to obtain a frequency spectrum by frequency analysis. 6) A dominant frequency component is extracted from the frequency spectrum to determine a walking state. The pedestrian attribute is determined based on the walking state and the speed or height of the pedestrian, and the determination result is displayed on the display unit 164. Details of the processing executed by the control unit 161 will be described later.
[0033]
The HDD 17 has two input / output ports IOP, receives the two-dimensional image IM and the height / speed data provided from the input processing unit 15 at the first port IOP1, and sequentially stores the data. The stored image data is output from the second port IOP2 in accordance with the request of (1). Further, the trajectory data and the image data thereof processed by the computer 16 are received and stored via the second port IOP2.
[0034]
The frequency analysis (Fast Fourier Transform) processing device FFT18 receives the movement locus of the pedestrian's crown on the time axis on the two-dimensional image IM acquired by the control unit 161 and performs frequency analysis processing to perform walking analysis. Find the frequency spectrum.
[0035]
Next, the operation of the walking state / pedestrian attribute determination device 1 having the above configuration will be described as an example of a case where the walking state of a pedestrian walking in the three-dimensional real space RS is determined.
[0036]
In this example, the detection target is a human, and the top of the head is a feature. Here, a technique described in Japanese Patent Application Laid-Open No. 2002-197463 is used as a process for detecting the top of a human head on the two-dimensional image IM. In this process, the input image is classified into a non-background region and a background region by a difference process, and the coordinates of the highest point on the image in the non-background region are extracted as the top of a human head as a detection target portion. is there. An image processing program for extracting the detection target part is also stored in the memory 162.
[0037]
1. Preparation operation
As a preparatory step before entering a series of operations for determining a pedestrian state, case data is collected and statistically analyzed to create a classification table (model) for associating the pedestrian state with the frequency. 162 is stored. For example, 100 pedestrian models including each age group such as infants, adults, and the elderly walk slowly, walk fast, and have them run. From the locus of the head obtained at that time, the fundamental frequency of each walking motion is calculated. (The fundamental frequency of this walking motion is also called a walking rhythm). Next, an average value of the walking rhythm of the pedestrian model is obtained. If the rhythm is equal to or more than the first predetermined value, the vehicle runs, if the rhythm is equal to or less than the second predetermined value, walks slowly, and the first predetermined value and the second predetermined value are compared. The boundary (first and second predetermined values) is determined, for example, if the distance is between the two, the person walks fast. Further, a classification table for classifying walking states is created based on the boundary division. This classification table takes a form as exemplified in FIG. 9 (a), and the walking rhythm of the pedestrian is applied to this classification table. It has a specification to judge. This classification table is stored in the ROM 162a.
[0038]
It also collects case data on pedestrian walking speeds, determines the boundaries of high and low speeds from the average of walking speeds for each age of the pedestrian model, and determines the attributes of pedestrians in relation to rhythm and speed. A classification table as illustrated in FIG. 10B for determining (age) is created and stored in the ROM 162a. By applying the walking state and the walking speed of the pedestrian to this classification table, for example, it is possible to classify a pedestrian whose walking speed is “3” and “walks fast” as an adult. Note that the information indicating the speed generated from the output of the speed sensor 14 by the input processing unit 15 is information indicating any of the speeds used in this classification table.
[0039]
It also collects case data on the height of pedestrians, determines the boundaries of height and height from the average height of the pedestrian model at each age, and determines the pedestrian's attributes (age ) Is determined, and a classification table as shown in FIG. 10C is created and stored in the ROM 162a. By applying the walking state and the height of the pedestrian to this classification table, it is possible to classify, for example, a pedestrian who is traveling tall and is an adult. The information indicating the speed generated by the input processing unit 15 from the output of the height sensor 13 is information indicating one of the heights used in this classification table.
[0040]
In addition, as a preparation for a difference process for detecting a crown, an image in a state where no pedestrian exists in the three-dimensional real space RS, that is, a so-called background image is acquired and stored in the HDD 17.
[0041]
2. 2D image acquisition operation
The imaging device 11 installed at a predetermined position in the three-dimensional real space RS acquires and outputs a two-dimensional image IM obtained by photographing the three-dimensional space RS at regular time intervals Δt. The two-dimensional image IM is temporarily stored in the image buffer 12 and then transferred to the HDD 17 by the input processing unit 15 in association with the imaging time. The HDD 17 stores the sequentially transferred two-dimensional images IM, as schematically shown in FIG.
[0042]
3. Speed / height acquisition operation
In parallel with the acquisition operation of the two-dimensional image IM, the height sensor 13 and the speed sensor 14 collect information indicating the height and speed of the pedestrian. This information is transferred to the HDD 17 by the input processing unit 15 in association with the time information, and is stored in the HDD 17.
[0043]
4. Walking state / pedestrian attribute judgment operation
With respect to a series of two-dimensional images IM acquired in parallel with the above-described image acquisition operation or in the past in the above-described image acquisition operation, the computer 16 performs the following series of operations to walk through the three-dimensional real space RS. The walking state and the attribute (eg. Age) of the walking pedestrian are obtained. This process includes a feature point detection operation and a state / attribute determination operation described below.
[0044]
4.1 Specific site detection operation
This processing is an operation for obtaining the position of the top of the pedestrian, which is the feature point of the detection target, on each two-dimensional image IM. This feature point detection operation will be described with reference to FIG.
[0045]
The control unit 161 of the computer 16 reads the two-dimensional images IM stored in the HDD 17 in the order of photographing times in response to an input instruction or the like from the input unit 163, and performs a flowchart of FIG. The processing shown in is performed. First, the first two-dimensional image IM (imaging time t ₀ ) To detect the top of the pedestrian in the two-dimensional image IM (step S101).
[0046]
Subsequently, it is determined whether or not an image of a pedestrian exists in the image obtained in the process of step S101 (step S102). When the presence of the pedestrian is not confirmed (step S102: No), the image returns to the main routine without performing any further processing.
[0047]
On the other hand, when a pedestrian is detected (Step S102: Yes), the position of the pedestrian's characteristic point (top), that is, the coordinates (x, y) of the highest point in the human image is detected (Step S103). ).
[0048]
Subsequently, an operation (identification operation) of confirming which pedestrian is the same person as which pedestrian is performed between two temporally adjacent still images (step S104). Where t = t ₀ Is the time at the start of photographing, since there is no information on the time before that and no identification work is required, the process proceeds to step S105 without any processing.
[0049]
In step S105, it is determined whether a new pedestrian has been detected in the two-dimensional image IM. T = t without previous two-dimensional image IM ₀ In the two-dimensional image IM, all pedestrians are new pedestrians. Thus, in step S105, the determination is Yes, and the process proceeds to step S106.
[0050]
In step S106, the control unit 161 assigns information (hereinafter, the object number OBi: i is an integer) for identifying the person to the pedestrian newly detected in the image. For example, the object number OB1 is assigned to the first pedestrian detected in the image. If there are a plurality of pedestrians in the two-dimensional image IM, OB2, OB3,... Are sequentially assigned (step S106).
[0051]
Next, t = t ₀ At the timing, the height and the speed are measured by the height sensor 13 and the speed sensor 14, and it is determined whether or not the height and the speed are stored in the HDD 17. If the height and the speed are stored, the height detected by the corresponding object ( A height (back height) and a speed (walking speed) are assigned (step S107). If there are two or more objects and it is not possible to determine the object corresponding to the person who has traversed the light beam of the height sensor 13, an error processing (correspondence between the object and the height / speed information) is given to the operator. ) May be required.
[0052]
As described above, t = t ₀ Is completed for the two-dimensional image IM.
[0053]
FIG. 7A shows that t = t ₀ 2 schematically illustrates an example of a result of processing on a two-dimensional IM image. In this example, two pedestrians are detected in the two-dimensional image IM, object numbers OB1 and OB2 are assigned, and the coordinates of the top of OB1 are (x ₀₁ , Y ₀₁ ) And the coordinates of the top of OB2 are (x ₀₂ , Y ₀₂ ).
[0054]
Next, the control unit 161 controls the second two-dimensional image IM (t = t ₁ ), And the process of FIG. 6 is performed on this image. First, the top of the pedestrian existing in the two-dimensional image IM is detected (step S101).
[0055]
Next, it is determined whether or not a pedestrian is detected (step S102). If a pedestrian is detected (step S102: Yes), the detected position coordinates (x, y) of the top of each pedestrian are determined. The pedestrian is detected (step S103), and the identified pedestrian is identified (step S104). That is, it is determined whether or not the detected pedestrian is a pedestrian that also existed in the previous two-dimensional image IM. For this processing, the control unit 161 determines that t = t ₀ Comparing the position of the pedestrian present on the two-dimensional image IM with the position of the pedestrian detected this time, and determines that the pedestrian located closest to the two images is the same pedestrian. And assigns the same object number OB.
[0056]
Subsequently, it is determined whether or not there is a pedestrian that does not exist in the previous image among the detected pedestrians (step S105). If the presence of a new pedestrian is confirmed (step S105: Yes), the process proceeds to step S106. On the other hand, if no new pedestrian is detected (step S105: No), the process jumps to step S107.
[0057]
In step S106, the control unit 161 assigns a new object number to a person newly detected in the image. For example, when two pedestrians exist in the previous image, the object number OB3 is assigned to the newly detected third pedestrian. If there are a plurality of newly detected pedestrians, OB4, OB5,... Are sequentially assigned (step S106).
[0058]
Next, t = t ₁ At the timing, the height and speed of the pedestrian are measured, and it is determined whether or not the pedestrian is stored in the HDD 17. If the pedestrian is stored, the height and the height detected for the corresponding object are determined. A speed (walking speed) is assigned (step S107).
[0059]
As described above, t = t ₁ Is completed for the two-dimensional image IM.
[0060]
FIG. 7B shows that t = t ₁ 2 schematically illustrates an example of a result of processing on the two-dimensional image IM. In this example, the two pedestrians shown in FIG. ₀ From t = t ₁ And the coordinates of the top of OB1 are (x ₁₁ , Y ₁₁ ), The coordinates of the top of OB2 are (x ₁₂ , Y ₁₂ ).
[0061]
Similarly, t = t _n-1 Up to this point, the control unit 161 calls the image, detects the pedestrian, obtains the coordinates of the top of the head, and repeats the operation of identifying the OB.
[0062]
FIG. 7C shows that t = t _k 2 schematically illustrates an example of a result of processing on the two-dimensional image IM. In this example, the coordinates of the top of each of OB1 and OB2 are (x _k1 , Y _k1 ) And (x _k2 , Y _k2 ), And a new person OB3 has appeared, and the coordinates of the top are (x _k3 , Y _k3 ).
[0063]
4.2 State / attribute determination operation
Next, the state / attribute determination operation will be described with reference to the flowchart in FIG.
[0064]
First, the control unit 161 determines that t = t stored in the HDD 17 ₀ The coordinates of the top of OB1 are called (step S201). Next, an axis orthogonal to the x-axis and the y-axis of the two-dimensional image IM is defined as a time axis (t-axis) to form three-dimensional coordinates (txy), and t = t in the three-dimensional coordinates. ₀ The position of the top of OB1 is plotted and displayed on the display unit 164 at the position. Then, t = t ₁ At the top of OB1 at t = t ₁ Plot at position. Hereinafter, the control unit 161 determines that t = t _n-1 The same operation is repeated until each time (t = t) aligned in the time axis direction as schematically shown in FIG. ₀ ~ T _n-1 ) Are obtained (step S202).
[0065]
Next, the trajectory on the three-dimensional space-time (txy) obtained in this way is transmitted to the frequency analysis processor FFT18. The frequency analysis processing device FFT18 performs a frequency analysis process on the received trajectory of the OB1 on the three-dimensional spatiotemporal image using the fast Fourier transform (FFT) (step S203). By performing arithmetic processing on the time-series data (x, y) t, a frequency spectrum of the motion locus of the crown can be obtained.
[0066]
The frequency analysis processing device FFT18 transmits the processing result to the control unit 161. The control unit 161 extracts, from the obtained spectrum distribution, a frequency having the largest energy (having a large amplitude) as a fundamental frequency component (step S204). It should be noted that there is a certain limit in human walking, so the upper limit may be about 5 Hz. Further, when a pedestrian walks meandering, for example, the lower limit frequency may be set to about 0.5 Hz so that the frequency component due to the meandering can be removed. Therefore, for example, the frequency with the largest amplitude in the range of 0.5 Hz to 5 Hz is extracted.
[0067]
Next, the extracted fundamental frequency, that is, the walking rhythm is adapted to the classification table shown in FIG. 10A to determine whether the walking state is slow walking, fast walking, or running (step S205).
For example, if the fundamental frequency is about 0.5 to 1.5 Hz, "walk slowly", if the fundamental frequency is about 1.5 to 2.5 Hz, "walk fast", the fundamental frequency is about 2.5 to 5 Hz. If so, it is determined that the vehicle is in the “running” state.
[0068]
Next, the speed assigned to the object to be processed is read, and adapted to the classification state shown in FIG. 10B together with the previously determined walking state (slow walking, fast walking, running), and the object (pedestrian) ) Is determined (step S206). For example, if the person is “walking slowly” and the “speed is 2”, it is determined that the person is an adult.
[0069]
Next, the height assigned to the object to be processed is read out, and the walking state (slow walking, fast walking, running) determined earlier is adapted to the classification table shown in FIG. ) Is determined (step S207). For example, a person who is "slowly walking" and "short" is determined to be an infant.
[0070]
Thus, the processing of the state / attribute determination operation for OB1 ends (step S208).
[0071]
The above processing is repeated for each object in order, and the walking state and the attribute of each object (passer) are obtained (step S209).
[0072]
In the above embodiment, the pedestrian attribute is determined from the information on either the speed or the height of the pedestrian and the information on the walking frequency. The attribute may be determined. In this case, the classification table takes a three-dimensional form as schematically shown in FIG. 11, and allows more detailed attribute determination.
[0073]
In the above embodiment, the speed at which a pedestrian passes through a certain position was measured with a speed gun. However, the measuring method is not limited to this. For example, the speed of a person passing through the image may be obtained by taking an image from the lateral direction of the passage. Alternatively, the light emitters 131 and 132 may be arranged at fixed intervals in the horizontal direction instead of in the height direction, and the time interval in which the light emitted from the light emitters 131 and 132 is blocked may be measured, and the speed may be determined from this. .
[0074]
Further, in the above-described embodiment, the height is determined using the light sources installed above and below, but the determination method is not limited to this. For example, an infrared sensor that is continuous in the height direction may be arranged on one side of the passage (the three-dimensional real space RS), and infrared light emitted from a pedestrian passing therethrough may be detected to measure the height.
[0075]
In the above-described embodiment, the classification table is created by dividing the speed into four levels, the height into two levels, and the pedestrian's age into three levels. However, the classification is not limited to this, and is arbitrary. For example, it is possible to arrange the light source of the height sensor 13 in three stages and divide the pedestrian's age into four stages of infants, boys, adults, and the elderly to perform more detailed classification.
[0076]
In the above-described embodiment, the association between the frequency, the height, and the speed, and the walking state and the attribute are expressed as a discrete classification table and used. However, it is also possible to treat these as statistical data in a two-dimensional or three-dimensional continuous space and stochastically estimate a walking state and attributes. There are various methods of such stochastic treatment, such as Bayesian estimation, but the details are not limited here.
[0077]
In the above-described embodiment, a method of acquiring a continuous still image online and analyzing it is used. However, it is also possible to acquire only the image first and then change the time and place before conducting the analysis.
[0078]
In the above embodiment, the frequency analysis is performed using the frequency analysis processing device FFT18. However, the frequency analysis may be performed in the computer 16 without providing a dedicated device externally.
[0079]
Further, the method of detecting an object, the method of detecting a feature point, and the method of tracking described in the above embodiment are not limited to the above, and may be replaced by various other methods.
[0080]
The device according to the embodiment of the present invention determines the attribute of a pedestrian (adult, infant, or elderly). Therefore, it is effective for obtaining information in marketing, such as analyzing the audience of visitors to amusement parks, concerts, movies, and the like. In addition, by using an infrared camera or the like, a pedestrian can be photographed and information can be acquired in a situation where it is difficult to determine with the naked eye such as in the dark, so that it can be effective in the field of crime prevention and monitoring.
[0081]
【The invention's effect】
As described above, according to the present invention, it is possible to determine the walking status of a pedestrian in the observation target area.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a walking state / pedestrian attribute determination device according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining a walking state / pedestrian attribute determination process, and is a diagram for explaining a process of acquiring an image of a pedestrian;
FIG. 3 is a diagram for explaining a process of measuring the height and speed of a pedestrian;
FIG. 4 is a diagram schematically illustrating a state where images acquired by a camera are stored in an HDD in order of photographing time.
FIG. 5 is a diagram showing a configuration of a memory and an example of data stored in a ROM.
FIG. 6 is a flowchart illustrating a flow of a process of detecting a top portion (feature point) of a pedestrian in an image.
FIG. 7 is a diagram schematically illustrating a result image of the pedestrian detection process.
FIG. 8 is a diagram illustrating an example of a trajectory in the (txy) space of a crown detected by image processing.
FIG. 9 is a flowchart showing the flow of processing for determining a walking state / attribute.
FIG. 10 is a diagram showing an example of a classification table based on walking rhythm, speed, and height.
FIG. 11 is a diagram showing an example of a classification table based on walking rhythm, speed, and height.
[Explanation of symbols]
11 Imaging device
12 Image buffer
13 Height sensor
131, 132 light emitter
133, 134 Receiver
14 Speed sensor
15 Input processing unit
16 Computer
161 control unit
162 memory
162a ROM
162b RAM
163 input section
164 display
17 HDD
18 FFT

Claims (6)

Storage means for storing a series of two-dimensional images taken of a three-dimensional space;
A specific part detection means for detecting a specific part of the detection target person in the two-dimensional image stored in the storage means,
Trajectory calculation means for calculating a movement trajectory on the time axis in the two-dimensional image of the specific part detected by the specific part detection means,
Frequency analyzing means for obtaining a frequency spectrum of the locus calculated by the locus calculating means,
Walking state determining means for determining a walking state based on the frequency spectrum obtained by the frequency analyzing means,
A walking state determination device comprising: Storage means for storing statistical case data of the fundamental vibration frequency of the walking motion;
Means for classifying the fundamental vibration frequency of the frequency spectrum into one of the categories based on the case data, and determining a walking state of the detection target person by the category assigned to the category;
The walking state determination device according to claim 1, further comprising: Speed detection means for detecting the speed of the detection target person,
Storage means for storing statistical case data of walking speed;
Means for classifying the speed of the detection target person based on the case data,
Means for determining the attribute of the detection target person based on the classification of the fundamental vibration frequency and the classification of the speed,
A pedestrian attribute determination device, comprising: In the walking state determination device, height detection means for detecting the height of the detection target person,
Storage means for storing statistical height case data;
Classification means for classifying the height of the detection target person based on the case data,
Attribute determination means for determining the attribute of the detection target person based on the classification of the fundamental vibration frequency and the classification of the height,
A pedestrian attribute determination device, comprising: Input a two-dimensional image of a three-dimensional space,
Detect the target person on the input two-dimensional image,
Calculate the moving trajectory of the detected subject in three-dimensional space-time,
Based on the calculated trajectory, the fundamental vibration frequency of the subject's motion is determined,
Detect the speed of the detected subject,
Detects the height of the detected subject,
Based on the detected fundamental vibration frequency, speed, and height, determine the walking state and pedestrian attribute of the subject,
A method for determining a walking state and a pedestrian attribute. Computer
Storage means for storing a continuous two-dimensional image on a time axis taken of a three-dimensional space,
A specific part detection means for detecting a specific part of the detection target person in the two-dimensional image stored in the storage means,
Trajectory calculation means for calculating a movement trajectory in three-dimensional space-time of the specific part detected by the specific part detection means,
Frequency analysis means for extracting the frequency spectrum of the movement trajectory calculated by the trajectory calculation means,
Based on the frequency spectrum extracted by the frequency analysis unit, a walking state and a pedestrian attribute determining unit that determines a walking state and an attribute of the subject,
A program characterized by functioning as a program.

Images