CN103793056A - Mid-air gesture roaming control method based on distance vector - Google Patents

Abstract

The invention discloses a distance vector-based air gesture roaming control method, which includes the following steps: Step 1: Acquiring, analyzing and processing video image sequences; Step 2: Detecting five-finger open gestures and fist gestures to initialize the control area; Step 3 : Obtain the skin color information in the region of interest; Step 4: Obtain the motion information of the human hand in the region of interest; Step 5: Calculate the position of the human hand in each frame image from the skin color information and motion information obtained in Step 3 and Step 4 Coordinate information; Step 6: Determine the direction and speed of the gesture in the interface; Step 7: The gesture in the interface responds according to the direction and speed determined in Step 6, realizing gesture roaming. It has the advantages of realizing small-scale and full-interface accessible operations, fast gesture roaming with a long distance between the current gesture and the initial position, and precise gesture roaming with a short distance between the current gesture and the initial position.

Description

Air gesture roaming control method based on distance vector

technical field

The invention relates to a human-computer interaction technology, in particular to an air gesture roaming control method based on a distance vector.

Background technique

In daily life, gesture is a commonly used way of expressing will, with strong ideographic function, and it is also the main interaction mode of existing human-computer interaction systems, such as mouse, keyboard, remote control, touch screen, etc. part of the human-computer interaction system. And some emerging human-computer interaction systems capture the user's behavior in the sensor capture range through ordinary cameras or depth cameras, and use image processing, machine learning, pattern recognition and other technologies to identify and track user gestures and other actions, analyze and capture The gesture-based non-contact human-computer interaction can be realized through the interaction with the interface through the user's behavioral intention in the received image sequence.

Gesture roaming is to map the gesture movement to the interface, control the hand movement in the interface with the real hand movement, and realize the selection and browsing of interface information. It is an important function of the gesture-based human-computer interaction system. The existing common mapping method is the direct mapping of gesture coordinates, that is, the coordinates of the gesture in the image sequence captured by the sensor, or the "comfortable motion area" in the image sequence captured by the sensor through some prior knowledge The coordinates of the gesture directly map to the gesture coordinates in the interface. For example, the image size of each frame in the image sequence captured by the sensor is length*width=640 pixels*480 pixels, the position of the gesture is (200,100) pixels, and the size of the interface is length*width=1280 pixels*720 pixels, then Through direct mapping of coordinates, the gesture coordinates in the interface are (1280/640*200=400,720/480*100=150) pixels.

This direct mapping method of gesture coordinates only uses coordinate information, and when the user wants to select some items that are far away from the coordinates of the current gesture during gesture roaming, the gesture needs to move a long distance, which increases the fatigue of the user , when selecting some items with similar coordinates, it is often difficult to select or misselect due to the current technical level constraints and cannot achieve sufficient accuracy. Therefore, the ease of use of the gesture-based human-computer interaction system is reduced and the lack of humanity change.

In addition to coordinate mapping, methods of velocity mapping are mentioned in some inventions. Velocity mapping is a relative mapping method, which calculates the movement speed and direction of the gesture in the image sequence captured by the sensor, regardless of its specific position coordinates. According to a specific proportional relationship, the gesture in the operation interface moves in the corresponding direction A certain distance is related to the length of the distance and the speed.

This gesture speed mapping method only uses the relative coordinate information of the gesture motion, that is, the difference between the absolute coordinates of the gesture in the image captured by the sensor before and after the two frames. In this way, in the actual operation process, especially a new user who is new to this system cannot intuitively grasp the speed and position of the gesture, and the destination that he wants to roam in the interface will exceed the range that the current user gesture can reach. For example, if the user uses his right hand to operate, his right hand has stretched to the right as far as it can reach, and because of the speed mapping adopted, the gestures in the interface may be on the far left of the interface, and the user must retract his hand at this time. Come back and do it again. This reduces the usability of the human-computer interaction system and increases the time for users to get familiar with, learn and adapt.

Therefore, the advantages of these two mapping methods should be combined to develop a new control method for air gesture roaming.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a distance vector-based air gesture roaming control method. This control method solves the problem that in the direct mapping of gesture coordinates, when selecting an item that is far from the current gesture coordinates, the user needs to move the gesture to a far distance and is not accurate enough when selecting an item with similar coordinates; In the speed map of , the position of the item you want to select has exceeded the position that the user's gesture can reach in the current reality.

The object of the present invention is achieved through the following technical solutions: a distance vector-based aerial gesture roaming control method, comprising the following steps:

Step 1, acquiring and analyzing and processing video image sequences;

Step 2. Detect five-finger open gestures and fist gestures, frame the detected human hand area as the area of interest, and record the initial position where the user starts to control to initialize the control area;

Step 3, performing a skin color segmentation algorithm operation on the image in the region of interest to obtain skin color information in the region of interest;

Step 4, performing a differential operation on the images of two adjacent frames in the region of interest to obtain the motion information of the human hand in the region of interest;

Step 5, calculating the position coordinate information of the hands in each frame of images by the skin color information and motion information of the hands obtained in the steps 3 and 4;

Step 6, determine the gesture movement direction and movement speed in the interface;

Step 7. The gestures in the interface make corresponding responses according to the direction and speed determined in step 6, so that the gestures roam.

Described step two comprises the following steps:

Step A, using the fixed gesture detection classifier trained by the Adaboost algorithm to detect five-finger open gestures and fist gestures; the classifiers for five-finger open gestures and fist gestures are respectively trained by a positive sample set and a negative sample set, and the sample set contains Gesture sample pictures of different backgrounds, different lighting conditions, and different people are collected, and the negative sample set also includes images under different backgrounds and different lighting conditions, but gestures are not included;

Step B. Use Haar-like features and integral images to extract and calculate the features of the sample image. The weak classifiers obtained in each round of training have different weights. The weak classifiers with high recognition rates have greater weights, and the recognition rate The weight of a low weak classifier is low. After multiple rounds of training, several weak classifiers obtained are combined to obtain a strong classifier with a high recognition success rate, and multiple strong classifiers obtained through training are combined to form a cascade structure. A classifier with a high detection success rate;

Step C. Use the trained classifier to detect the five-finger open and fist gestures in the image. After successfully finding the human hand area, record the rectangular position information of the human hand area. The upper left corner is (x ₀ , y ₀ ), the width is w, and the height is h; set this rectangular area as the region of interest, and at the same time get the center point (x _c , y _c ) of the human hand, where x _c =x ₀ +0.5*w, y _c =y ₀ +0.5*h, record the center position of the human hand as the initial position where the user starts to control, so as to determine the initial position and initialize the ring control area.

Described step three comprises the following steps:

Step Ⅰ. According to the analysis of skin color samples, human skin color has good clustering in YCrCb color space. After removing the influence of brightness Y, the Cr and Cb channels of skin color are concentrated in a small elliptical area. YCrCb color space and RGB color The space conversion relationship is as follows:

Y=0.257R+0.504G+0.098B+16,

Cb=-0.148R-0.219G+0.439B+128,

Cr=0.439R-0.368G-0.071B+128,

According to the analysis of the hand skin color sample set, the thresholds of the Cr and Cb channels of the hand skin color are:

Thres(Cb,Cr)={Cb,Cr│95<Cb<139,122<Cr<167},

Among them, Thres (Cb, Cr) represents the threshold;

Step Ⅱ, first convert the RGB image obtained in the video sequence into an image in the YCrCb color space, and then use the threshold Thres (Cb, Cr) to perform skin color segmentation on the image to obtain a binary image of skin color, namely:

Wherein, Thres(Cb,Cr) represents the threshold value.

In described step 4, the operation method of carrying out difference operation to the image of two adjacent frames in the region of interest is: let I _t be the current frame image, I _t-1 be the previous frame image, calculate and obtain two frame images The differential result I _diff =I _t -I _t-1 , and binarize the differential result, namely:

And image morphology processing is performed on the difference result.

零阶矩即为图像像素值的总和：In the step five, combine the skin color information and motion information of the hands obtained in steps three and four, that is, take the union of the two, and obtain a binary value that removes background noise interference and describes the hand information in the region of interest. Image I, calculate the center of mass of the target in image I from the zero-order moment and the second-order moment

The zeroth moment is simply the sum of the image pixel values:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

There are two first-order moments, namely:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; xI</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

Therefore:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$

Get the position information of the human hand in the current frame.

In the sixth step, the method for determining the direction and speed of the gesture movement in the interface is: map the position result obtained by the tracking of the human hand, and obtain the coordinate information (x, y) of the gesture in the current frame obtained by the step five and the obtained According to the distance between the initial center point (x _c , y _c ) obtained in the above step 2, determine the movement rate of the gesture in the interface according to the proportional relationship between the distance and the speed; at the same time, determine according to the vector direction of the initial position and the current position of the gesture The direction of movement of the gesture in the interface.

The working principle of the present invention: according to the initial position of the user's control, the present invention limits the user's operation to the circle shown in Figure 1. In Figure 1, the point indicated by 1 represents the user in the image sequence captured by the sensor. The initial position coordinates of the gesture at the beginning of the operation, which will not change when the user completes an operation. The point indicated by 4 in FIG. 1 represents the position where the current user gesture is located. When the user's gesture moves within the circle indicated by 2 in Figure 1, it is regarded as a movement caused by unstable factors such as shaking of the user's own gesture, and the system does not respond. When the user's gesture is outside the circle indicated by 3 in Figure 1, the system does not respond. In Figure 1, when the user gesture moves within the circle formed by the circle indicated by 3 and the circle indicated by 2, it is considered as a valid operation. When the user's gesture is in this circle, the gesture in the interface starts to move, and the direction of movement is the position of the gesture in the circle at this time, that is, the coordinates of the point indicated by 4 in Figure 1 and the initial position of the gesture, that is The vector direction formed by the coordinates of the point indicated by 1 in Figure 1 is directed from the point indicated by 1 to the point indicated by 4. The speed of movement is related to the position of the current gesture in the circle, that is, the point indicated by 4 in Figure 1, and the distance from the initial position, that is, the point indicated by 1 in Figure 1.

这样，用户手势只要保持在圆环中的位置，界面中的手势就会按照一定的方向和速率持续移动，用户改变手势在圆环中的位置就可以改变界面中手势的运动方向和运动速率。The corresponding relationship between the distance between the current gesture position and the initial position and the movement rate of the gesture in the interface is shown in Figure 2. In Figure 2, the abscissa represents the distance between the position of the current gesture in the circle and the initial position, and R1 is The radius of the circle indicated by 2 in Fig. 1, R2 is the radius of the circle indicated by 3 in Fig. 1 . The ordinate is the scale. A rate V0 is specified, which represents the number of pixels moved by gestures in the interface per unit time. The ordinate indicates the ratio of the motion rate of the gesture in the interface to V0, the maximum is Pmax, Pmax>1, and the minimum is Pmin, Pmin≥0. When the distance is less than R1, that is, when the gesture is within the circle indicated by 2 in Figure 1, the ratio is 0, that is, the movement rate is 0, and the effect shown is that the system does not respond. When the distance is greater than R2, that is, when the gesture is outside the circle indicated by 3 in Figure 1, the ratio is also 0, that is, the moving rate is 0, and the effect shown is that the system does not respond. When the distance is greater than R1 and less than R2, that is, the gesture is in the circle formed by the circle indicated by 3 and the circle indicated by 2 in Figure 1, assuming that the distance at this time is RX, the ratio is:

In this way, as long as the user's gesture remains in the ring, the gesture in the interface will continue to move in a certain direction and speed, and the user can change the direction and speed of the gesture in the interface by changing the position of the gesture in the ring.

Through this gesture roaming control method, the distance is used to adjust the speed, and the vector is used to control the direction. When the position of the desired item is far from the current gesture position, the user can move the gesture to a position farther from the initial position in the ring When selecting an item that is closer to the current gesture, the user can move the gesture to a position in the ring that is closer to the initial position to achieve slow and high-precision movement. This solves the problems that arise during the direct mapping of coordinates. In this method, the gestures in the interface are constantly moving, and the user operation changes the direction and speed of the gestures in the interface. Therefore, roaming and control of any distance in the interface can be realized within a small range. Avoids problems that can occur in velocity mapping.

Compared with the prior art, the present invention has the following advantages and effects:

1. Solved the problem that when selecting an item far from the current gesture position in the process of coordinate direct mapping, the user needs to move the gesture correspondingly far away, and the problem of not being accurate enough when selecting an item with similar coordinates. By changing the distance between the current gesture and the initial position, the user can realize rapid gesture roaming with a long distance between the current gesture and the initial position and precise gesture roaming with a short distance between the current gesture and the initial position.

2. Solved the problem that the item that the user wants to select during the speed mapping process is beyond the reachable range of the user's current gesture. Let the gestures in the interface move automatically, and the user controls the direction and speed of its movement to achieve small-scale and fully accessible operations on the interface

3. Provide a speed control mode proportional to the distance between the current gesture position and the initial position in the air gesture roaming control method based on the distance vector.

Description of drawings

Fig. 1 is the schematic diagram of the air gesture roaming control method based on the distance vector according to the present invention; among the figure, 1 is the initial position point, 2 is the circular range that the system does not respond to, 3 refers to the circular range and 2 refers to The ring range formed by the circular range of the generation is the system response range, and 4 is the current gesture position.

Fig. 2 is in the air gesture roaming control method based on the distance vector according to the present invention, the distance between the current gesture position and the initial position and the corresponding relationship diagram of the motion rate of the gesture in the interface; the abscissa is the current gesture position and the initial position The distance between , and the ordinate is the proportional value between the motion rate of the gesture in the interface and the system-specified rate.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Example

As shown in Figure 1, 1 is the initial position point, 2 is the circular range where the system does not respond, the circular range formed by the circular range indicated by 3 and the circular range indicated by 2 is the system response range, and 4 is the system response range. Current gesture position, a kind of air gesture roaming control method based on distance vector, comprises the following steps:

Step 1. Use a common camera or a depth camera as a front-end sensor to process and analyze the image sequence captured by the sensor;

Step 2. The fixed gesture detection classifier trained by the Adaboost algorithm can detect two gestures of five-finger open and fist; the classifiers of the two gestures are trained by different sample sets, and the positive sample sets are included in Gesture sample pictures of different backgrounds, different lighting conditions, and different people, and the negative sample also contains images under different backgrounds and different lighting conditions, but does not contain gestures; use Haar-like features and integral images to sample image features Do extraction calculations. The weak classifiers obtained in each round of training have different weights, the weak classifiers with high recognition rate have greater weights, and the weak classifiers with low recognition rate have lower weights; after multiple rounds of training, the obtained weak classifiers A strong classifier with a high recognition success rate is obtained by combining the trained classifiers; multiple strong classifiers trained to form a cascade structure classifier have a high detection success rate; Two gestures of five fingers open and fist are detected. After successfully finding the human hand area, record the rectangular position information of the human hand area. The upper left corner is (x ₀ , y ₀ ), the width is w, and the height is h; set The rectangular area is the area of interest, and at the same time, the center point (x _c , y _c ) of the human hand can be obtained, where x _c =x ₀ +0.5*w, y _c =y ₀ +0.5*h; record the position of the human hand The center position point is used as the initial position where the user starts to control, so that the position of the initial position 1 in Figure 1 can be determined, and the entire ring control area in Figure 1 can be initialized;

Step 3: Perform skin color segmentation algorithm operation on the image in the region of interest. According to the analysis of skin color samples, human skin color has good clustering in the YCrCb color space. After removing the influence of brightness Y, the Cr and Cb channels of skin color are concentrated in a small elliptical area. The conversion relationship between YCrCb color space and RGB color space is as follows:

Y=0.257R+0.504G+0.098B+16,

Cb=-0.148R-0.219G+0.439B+128,

Cr=0.439R-0.368G-0.071B+128,

According to the analysis of the human skin color sample set, the threshold values of the Cr and Cb channels of the human skin color are:

Thres(Cb,Cr)={Cb,Cr│95<Cb<139,122<Cr<167},

First convert the RGB image obtained in the video sequence into an image in the YCrCb color space, and then use the threshold Thres (Cb, Cr) to perform skin color segmentation on the image to obtain a binary image of skin color, namely:

Among them, Thres(Cb,Cr) represents the threshold;

Step 4: Perform a differential operation on the images of two adjacent frames in the region of interest. Let I _t be the current frame image, I _t-1 be the previous frame image, calculate the difference result I _diff =I _t -I _t-1 of the two frames of images, and perform binarization processing on the difference result, namely:

In order to obtain clearer and more accurate motion contour information and fill the internal cavity of the contour, image morphology processing can be performed on the difference result, mainly dilation and erosion, to further remove the interference of image noise;

Step 5: Combine the hand skin color information and motion information obtained in Step 3 and Step 4, that is, take the union of the two, and obtain a binary image I that removes background noise interference and describes hand information in the region of interest. Calculate the centroid of the object in image I from the zero-order moment and the second-order moment

The zeroth moment is simply the sum of the image pixel values:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

There are two first-order moments, which are

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; xI</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

Therefore:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$

Finally, the position information (x, y) of the current frame is obtained;

Step 6: According to the control method shown in Figure 1, map the position results obtained by the human hand tracking. From the coordinate information (x, y) of the current frame gesture obtained in step 5 and the distance between the initial center point (x _c , y _c ) obtained in step 2, determine the interface according to the distance and speed ratio relationship shown in Figure 2 The movement rate of the medium gesture. At the same time, determine the movement direction of the gesture in the interface according to the initial position and the vector direction of the current gesture position;

Step 7: The gestures in the interface make corresponding responses according to the moving rate and moving direction mapped in step 6 to realize gesture roaming.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (6)

1. a method for controlling air gesture roaming based on distance vector, is characterized in that, comprises the following steps: Step 1, acquiring and analyzing and processing video image sequences; Step 2. Detect five-finger open gestures and fist gestures, frame the detected human hand area as the area of interest, and record the initial position where the user starts to control to initialize the control area; Step 3, performing a skin color segmentation algorithm operation on the image in the region of interest to obtain skin color information in the region of interest; Step 4, performing a differential operation on the images of two adjacent frames in the region of interest to obtain the motion information of the human hand in the region of interest; Step 5, calculating the position coordinate information of the hands in each frame of images by the skin color information and motion information of the hands obtained in the steps 3 and 4; Step 6, determine the gesture movement direction and movement speed in the interface; Step 7. The gestures in the interface make corresponding responses according to the direction and speed determined in step 6, so that the gestures roam. 2. the air gesture roaming control method based on distance vector according to claim 1, is characterized in that, described step 2 comprises the following steps: Step A, using the fixed gesture detection classifier trained by the Adaboost algorithm to detect five-finger open gestures and fist gestures; the classifiers for five-finger open gestures and fist gestures are respectively trained by a positive sample set and a negative sample set, and the sample set contains Gesture sample pictures of different backgrounds, different lighting conditions, and different people are collected, and the negative sample set also includes images under different backgrounds and different lighting conditions, but gestures are not included; Step B. Use Haar-like features and integral images to extract and calculate the features of the sample image. The weak classifiers obtained in each round of training have different weights. The weak classifiers with high recognition rates have greater weights, and the recognition rate The weight of a low weak classifier is low. After multiple rounds of training, several weak classifiers obtained are combined to obtain a strong classifier with a high recognition success rate, and multiple strong classifiers obtained through training are combined to form a cascade structure. A classifier with a high detection success rate; Step C. Use the trained classifier to detect the five-finger open and fist gestures in the image. After successfully finding the human hand area, record the rectangular position information of the human hand area. The upper left corner is (x ₀ , y ₀ ), the width is w, and the height is h; set this rectangular area as the region of interest, and at the same time get the center point (x _c , y _c ) of the human hand, where x _c =x ₀ +0.5*w, y _c =y ₀ +0.5*h, record the center position of the human hand as the initial position where the user starts to control, so as to determine the initial position and initialize the ring control area. 3. the air gesture roaming control method based on distance vector according to claim 1, is characterized in that, described step 3 comprises the following steps: Step Ⅰ. According to the analysis of skin color samples, human skin color has good clustering in YCrCb color space. After removing the influence of brightness Y, the Cr and Cb channels of skin color are concentrated in a small elliptical area. YCrCb color space and RGB color The space conversion relationship is as follows: Y=0.257R+0.504G+0.098B+16, Cb=-0.148R-0.219G+0.439B+128, Cr=0.439R-0.368G-0.071B+128, According to the analysis of the hand skin color sample set, the thresholds of the Cr and Cb channels of the hand skin color are: Thres(Cb,Cr)={Cb,Cr│95<Cb<139,122<Cr<167}, Among them, Thres (Cb, Cr) represents the threshold; Step Ⅱ, first convert the RGB image obtained in the video sequence into an image in the YCrCb color space, and then use the threshold Thres (Cb, Cr) to perform skin color segmentation on the image to obtain a binary image of skin color, namely:

Wherein, Thres(Cb,Cr) represents the threshold value. 4. The air gesture roaming control method based on distance vector according to claim 1, characterized in that, in the step 4, the operation method of performing differential operation on the images of two adjacent frames in the region of interest is: Let I _t be the current frame image, I _t-1 be the previous frame image, calculate the difference result I _diff =I _t -I _t-1 of the two frames of images, and perform binarization on the difference result, namely:

And image morphology processing is performed on the difference result. 5. the air gesture roaming control method based on distance vector according to claim 1, is characterized in that, in described step 5, combine the skin color information and motion information of the staff obtained in step 3 and step 4, namely take The two are combined to obtain a binary image I that removes background noise interference and describes human hand information in the region of interest, and calculates the center of mass of the target in image I from the zero-order moment and the second-order moment The zeroth moment is simply the sum of the image pixel values: ${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ There are two first-order moments, namely: ${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; xI</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ ${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ Therefore: $\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$ $\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>$ Get the position information of the human hand in the current frame. 6. The air gesture roaming control method based on distance vector according to claim 1, characterized in that, in said step 6, the method for determining the motion direction and motion speed of the gesture in the interface is: performing the position result obtained by tracking the hand Mapping, and the distance between the coordinate information (x, y) of the current frame gesture obtained in step five and the initial center point (x _c , y _c ) obtained in step two, according to the proportional relationship between distance and speed , to determine the movement rate of the gesture in the interface; at the same time, determine the movement direction of the gesture in the interface according to the initial position and the vector direction of the current gesture position.

Images