CN108549878B - Depth information-based hand detection method and system - Google Patents
Depth information-based hand detection method and system Download PDFInfo
- Publication number
- CN108549878B CN108549878B CN201810391268.0A CN201810391268A CN108549878B CN 108549878 B CN108549878 B CN 108549878B CN 201810391268 A CN201810391268 A CN 201810391268A CN 108549878 B CN108549878 B CN 108549878B
- Authority
- CN
- China
- Prior art keywords
- hand
- depth image
- depth
- image
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a hand detection method and system based on depth information. The method comprises the following steps: acquiring a current depth image of a human body; marking initial position coordinates of the hand in the current depth image according to the current depth image; normalizing the current depth image; obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing; carrying out non-maximum suppression operation according to each selected area to obtain an optimal selected area; and carrying out image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image. The hand detection method based on the depth information eliminates the limitation of the traditional method for detecting the hand by the depth map on the condition that the hand needs to be positioned at the forefront of the camera, can accurately find the position of the hand, and can further improve the position precision of the palm of the hand through non-maximum suppression operation and image post-processing.
Description
Technical Field
The invention relates to the technical field of hand detection, in particular to a depth information-based hand detection method and a depth information-based hand detection system.
Background
With the continuous development of new interaction technologies and applications, especially the rapid development of AR (augmented reality) and VR (virtual reality), gesture-related technologies are considered as the next generation of new most representative interaction technologies. Non-contact remote human-computer interaction can be realized. In the field of intelligent home furnishing, intelligent household appliances or robots can carry out control and command through gestures; realistic actual experience can be achieved in VR and AR neighborhoods, and user experience is greatly enhanced in the process of games and teaching. All gesture interaction techniques involve hand detection algorithms. General common hand detection algorithms are mainly based on 2D color images for detection or traditional image processing methods for finding out the area where the closest point is located on a depth map, wherein the area is the position where the hand is located by default, and the methods either completely depend on pixel information and have poor anti-interference capability or have certain limitation on the area where the hand is located. When the range of motion of the human hand is large, the detection is often not accurate enough.
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art and provides a hand detection method based on depth information and a hand detection system based on depth information.
In order to achieve the above object, a first aspect of the present invention provides a hand detection method based on depth information, including:
step S120, obtaining a current depth image of a human body;
step S130, marking initial position coordinates of the hand in the current depth image according to the current depth image;
step S140, carrying out normalization processing on the current depth image;
s150, obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing;
step S160, carrying out non-maximum value inhibition operation according to each selected area to obtain an optimal selected area;
and S170, performing image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image.
Optionally, the method further comprises:
s110: and updating the preset multilayer convolutional neural network model.
Optionally, the step S110 includes:
acquiring each depth image of a hand of a human body, wherein each depth image comprises depth images of various postures of the hand;
marking initial position coordinates of the hand in the corresponding depth images according to the depth images;
normalizing each depth image;
and inputting each depth image after normalization processing into a convolutional neural network model, training until iteration times or loss convergence is met, and finishing updating the preset multilayer convolutional neural network model.
Optionally, the step of normalizing each depth image includes:
and normalizing each depth image in a preset depth range of [ -1,1] or [0,1 ].
Optionally, the image post-processing the optimal selected region includes:
at least one of smoothing filtering, dilation-erosion, and contour detection.
In a second aspect of the present invention, there is provided a hand detection system based on depth information, comprising:
the acquisition module is used for acquiring a current depth image of a human body;
the marking module is used for marking the initial position coordinates of the hand in the current depth image according to the current depth image;
the normalization module is used for performing normalization processing on the current depth image;
the selection module is used for obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing;
the evaluation module is used for carrying out non-maximum value inhibition operation according to each selected area to obtain an optimal selected area;
and the image processing module is used for carrying out image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image.
Optionally, the method further comprises:
and the updating module is used for updating the preset multilayer convolutional neural network model.
Optionally, the acquiring module is further configured to acquire each depth image of a hand of a human body, where each depth image includes depth images of various gestures of the hand;
the marking module is further used for marking the initial position coordinates of the hand in the corresponding depth images according to the depth images;
the normalization module is also used for performing normalization processing on each depth image;
and the updating module is used for inputting each depth image after the normalization processing into the convolutional neural network model, training until the iteration times or loss convergence is met, and finishing updating the preset multilayer convolutional neural network model.
Optionally, the normalization module is configured to perform normalization processing on each depth image within a preset depth range [ -1,1] or [0,1 ].
Optionally, the acquisition module comprises a depth camera or a depth camera.
The hand detection method based on the depth information eliminates the limitation of the traditional method for detecting the hand by the depth map on the condition that the hand needs to be positioned at the forefront of the camera, can accurately find the position of the hand (including both hands), and can further improve the position precision of the palm center of the hand through non-maximum value inhibition operation and image post-processing.
The hand detection system based on the depth information eliminates the limitation of the traditional method for detecting the hand by the depth map on the condition that the hand needs to be positioned at the forefront of the camera, can accurately find the position of the hand (including both hands), and can further improve the position precision of the palm center of the hand through non-maximum value inhibition operation and image post-processing.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flowchart of a hand detection method based on depth information according to a first embodiment of the present invention;
fig. 2 is a schematic structural diagram of a hand detection system based on depth information according to a second embodiment of the present invention.
Description of the reference numerals
100: a hand detection system based on depth information;
110: an acquisition module;
120: a marking module;
130: a normalization module;
140: selecting a module;
150: an evaluation module;
160: image processing module
170: and updating the module.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
Referring to fig. 1, a first aspect of the present invention relates to a method S100 for detecting a hand based on depth information, comprising:
and step S120, acquiring a current depth image of the human body.
Specifically, in this step, a current depth image of the human body may be acquired using a depth image device such as a depth camera or a depth camera.
And S130, marking the initial position coordinates of the hand in the current depth image according to the current depth image.
It should be noted that no specific limitation is made on how to mark the initial position coordinates of the hand in the current depth image, and for example, the initial position coordinates may be determined by depth information of each pixel point in the depth image. Of course, the calibration of the position coordinates of the hand part can also be realized according to other modes.
And step S140, carrying out normalization processing on the current depth image.
And S150, obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing.
It should be noted that the preset multilayer convolutional neural network model is substantially equivalent to a classifier model, and the current depth image after normalization processing can be substituted into the preset multilayer convolutional neural network model, that is, into the classifier model, and for the current depth image, each selected region of the hand detection can be determined according to the output result of the classifier model.
And step S160, performing non-maximum suppression operation according to each selected area to obtain an optimal selected area.
And S170, performing image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image.
In this step, the final position coordinates of the hand obtained are more accurate than the initial position coordinates of the hand.
The depth information-based hand detection method S100 in this embodiment eliminates the limitation of the conventional depth map-related hand detection method on the condition that the hand must be located at the forefront of the camera, and can accurately find the hand position (including both hands), and further improve the position accuracy of the palm of the hand by non-maximum suppression operation and image post-processing.
Optionally, as shown in fig. 1, the method further includes:
s110: and updating the preset multilayer convolutional neural network model.
Specifically, step S110 includes:
acquiring each depth image of a hand of a human body, wherein each depth image comprises depth images of various postures of the hand.
And marking the initial position coordinates of the hand in the corresponding depth images according to the depth images.
And carrying out normalization processing on each depth image.
And inputting each depth image after normalization processing into a convolutional neural network model, training until iteration times or loss convergence is met, and finishing updating the preset multilayer convolutional neural network model.
In the hand detection method S100 based on depth information in this embodiment, the preset multilayer convolutional neural network model may be updated, that is, the preset multilayer convolutional neural network model may be trained, so that the position of the hand (including both hands) may be further accurately found, and the position accuracy of the palm center of the hand may be further improved.
Optionally, the step of normalizing each depth image includes:
and normalizing each depth image in a preset depth range of [ -1,1] or [0,1 ]. Of course, normalization processing can be performed in other preset depth ranges according to actual needs.
In the hand detection method S100 based on depth information in this embodiment, the positions of the hands (including both hands) can be further accurately found, and the position accuracy of the palm center of the hand can be further improved.
Optionally, the image post-processing the optimal selected region includes:
at least one of smoothing filtering, dilation-erosion, and contour detection.
In a second aspect of the present invention, as shown in fig. 2, there is provided a hand detection system 100 based on depth information, comprising:
an obtaining module 110, configured to obtain a current depth image of a human body;
the marking module 120 is configured to mark an initial position coordinate of the hand in the current depth image according to the current depth image;
a normalization module 130, configured to perform normalization processing on the current depth image;
the selecting module 140 is configured to obtain each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after the normalization processing;
the evaluation module 150 is used for performing non-maximum suppression operation according to each selected area to obtain an optimal selected area;
and the image processing module 160 is configured to perform image post-processing on the optimal selected area to obtain a final position coordinate of the hand in the current depth image.
The hand detection system 100 based on depth information in this embodiment eliminates the limitation of the conventional method for detecting the hand by using a depth map on the condition that the hand must be located at the forefront of the camera, and can accurately find the position of the hand (including both hands), and further improve the position accuracy of the palm center of the hand by non-maximum suppression operation and image post-processing.
Optionally, the method further comprises:
and the updating module 170 is configured to update the preset multilayer convolutional neural network model.
Optionally, the obtaining module 110 is further configured to obtain each depth image of a hand of a human body, where each depth image includes depth images of various gestures of the hand;
the marking module 120 is further configured to mark initial position coordinates of the hand in the corresponding depth images according to the depth images;
the normalization module 130 is further configured to perform normalization processing on each depth image;
and the updating module 170 is configured to input each depth image after the normalization processing into the convolutional neural network model, train the depth image until the iteration number is satisfied or the loss is converged, and complete updating of the preset multilayer convolutional neural network model.
The hand detection system 100 based on depth information in this embodiment can update the preset multilayer convolutional neural network model, that is, can train the preset multilayer convolutional neural network model, so that the position of the hand (including both hands) can be further accurately found, and the position accuracy of the palm of the hand can be further improved.
Optionally, the normalization module 130 is configured to normalize each depth image within a preset depth range [ -1,1] or [0,1 ].
Optionally, the acquisition module 110 comprises a depth camera or a depth camera.
It will be understood that the above embodiments are merely exemplary embodiments taken to illustrate the principles of the present invention, which is not limited thereto. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit and substance of the invention, and these modifications and improvements are also considered to be within the scope of the invention.
Claims (10)
1. A hand detection method based on depth information is characterized by comprising the following steps:
step S120, obtaining a current depth image of a human body;
step S130, marking initial position coordinates of the hand in the current depth image according to the current depth image;
step S140, carrying out normalization processing on the current depth image;
s150, obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing;
step S160, carrying out non-maximum value inhibition operation according to each selected area to obtain an optimal selected area;
and S170, performing image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image.
2. The hand detection method of claim 1, further comprising:
s110: and updating the preset multilayer convolutional neural network model.
3. The hand detection method according to claim 2, wherein the step S110 includes:
acquiring each depth image of a hand of a human body, wherein each depth image comprises depth images of various postures of the hand;
marking initial position coordinates of the hand in the corresponding depth images according to the depth images;
normalizing each depth image;
and inputting each depth image after normalization processing into a convolutional neural network model, training until iteration times or loss convergence is met, and finishing updating the preset multilayer convolutional neural network model.
4. The hand detection method according to claim 3, wherein the step of normalizing each depth image includes:
and normalizing each depth image in a preset depth range of [ -1,1] or [0,1 ].
5. A hand detection method as claimed in any of claims 1 to 4, wherein said image post-processing of said optimal selected area comprises:
at least one of smoothing filtering, dilation-erosion, and contour detection.
6. A depth information based hand detection system, comprising:
the acquisition module is used for acquiring a current depth image of a human body;
the marking module is used for marking the initial position coordinates of the hand in the current depth image according to the current depth image;
the normalization module is used for performing normalization processing on the current depth image;
the selection module is used for obtaining each selected area including hand detection based on a preset multilayer convolutional neural network model according to the current depth image after normalization processing;
the evaluation module is used for carrying out non-maximum value inhibition operation according to each selected area to obtain an optimal selected area;
and the image processing module is used for carrying out image post-processing on the optimal selected area to obtain the final position coordinate of the hand in the current depth image.
7. The hand detection system of claim 6, further comprising:
and the updating module is used for updating the preset multilayer convolutional neural network model.
8. The hand detection system of claim 7,
the acquisition module is further configured to acquire each depth image of a hand of a human body, where each depth image includes depth images of various gestures of the hand;
the marking module is further used for marking the initial position coordinates of the hand in the corresponding depth images according to the depth images;
the normalization module is also used for performing normalization processing on each depth image;
and the updating module is used for inputting each depth image after the normalization processing into the convolutional neural network model, training until the iteration times or loss convergence is met, and finishing updating the preset multilayer convolutional neural network model.
9. The hand detection system of claim 8, wherein the normalization module is configured to normalize each of the depth images within a predetermined depth range [ -1,1] or [0,1 ].
10. A hand detection system as claimed in any of claims 6 to 9 wherein the acquisition module comprises a depth camera or a depth camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810391268.0A CN108549878B (en) | 2018-04-27 | 2018-04-27 | Depth information-based hand detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810391268.0A CN108549878B (en) | 2018-04-27 | 2018-04-27 | Depth information-based hand detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108549878A CN108549878A (en) | 2018-09-18 |
| CN108549878B true CN108549878B (en) | 2020-03-24 |
Family
ID=63512809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810391268.0A Active CN108549878B (en) | 2018-04-27 | 2018-04-27 | Depth information-based hand detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108549878B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109243248A (en) * | 2018-09-29 | 2019-01-18 | 南京华捷艾米软件科技有限公司 | A kind of virtual piano and its implementation based on 3D depth camera mould group |
| CN109696958A (en) * | 2018-11-28 | 2019-04-30 | 南京华捷艾米软件科技有限公司 | A kind of gestural control method and system based on depth transducer gesture identification |
| CN111462548A (en) * | 2019-01-21 | 2020-07-28 | 北京字节跳动网络技术有限公司 | A paragraph reading method, apparatus, device and readable medium |
| CN111459443A (en) * | 2019-01-21 | 2020-07-28 | 北京字节跳动网络技术有限公司 | Character point-reading method, device, equipment and readable medium |
| WO2021098666A1 (en) | 2019-11-20 | 2021-05-27 | Oppo广东移动通信有限公司 | Hand gesture detection method and device, and computer storage medium |
| CN111462234B (en) * | 2020-03-27 | 2023-07-18 | 北京华捷艾米科技有限公司 | A method and device for determining a location |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999152A (en) * | 2011-09-09 | 2013-03-27 | 康佳集团股份有限公司 | Method and system for gesture recognition |
| CN106648103A (en) * | 2016-12-28 | 2017-05-10 | 歌尔科技有限公司 | Gesture tracking method for VR headset device and VR headset device |
| CN106778435A (en) * | 2016-11-30 | 2017-05-31 | 上海正雅齿科科技有限公司 | Feature extracting method based on Image neighborhood structure tensor equation |
| CN107292904A (en) * | 2016-03-31 | 2017-10-24 | 北京市商汤科技开发有限公司 | A kind of palm tracking and system based on depth image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10320437B2 (en) * | 2014-10-24 | 2019-06-11 | Usens, Inc. | System and method for immersive and interactive multimedia generation |
-
2018
- 2018-04-27 CN CN201810391268.0A patent/CN108549878B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999152A (en) * | 2011-09-09 | 2013-03-27 | 康佳集团股份有限公司 | Method and system for gesture recognition |
| CN107292904A (en) * | 2016-03-31 | 2017-10-24 | 北京市商汤科技开发有限公司 | A kind of palm tracking and system based on depth image |
| CN106778435A (en) * | 2016-11-30 | 2017-05-31 | 上海正雅齿科科技有限公司 | Feature extracting method based on Image neighborhood structure tensor equation |
| CN106648103A (en) * | 2016-12-28 | 2017-05-10 | 歌尔科技有限公司 | Gesture tracking method for VR headset device and VR headset device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108549878A (en) | 2018-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108549878B (en) | Depth information-based hand detection method and system | |
| CN106598227B (en) | Gesture identification method based on Leap Motion and Kinect | |
| US10043308B2 (en) | Image processing method and apparatus for three-dimensional reconstruction | |
| JP4278979B2 (en) | Single camera system for gesture-based input and target indication | |
| CN104598915B (en) | A kind of gesture identification method and device | |
| WO2023000119A1 (en) | Gesture recognition method and apparatus, system, and vehicle | |
| CN105528082A (en) | Three-dimensional space and hand gesture recognition tracing interactive method, device and system | |
| CN105500370B (en) | A kind of robot off-line teaching programing system and method based on body-sensing technology | |
| CN102982557A (en) | Method for processing space hand signal gesture command based on depth camera | |
| CN107357428A (en) | Man-machine interaction method and device based on gesture identification, system | |
| JP2014501011A (en) | Method, circuit and system for human machine interface with hand gestures | |
| CN102999152A (en) | Method and system for gesture recognition | |
| CN113192138B (en) | A robot autonomous repositioning method, device, robot and storage medium | |
| CN109839827B (en) | Gesture recognition intelligent household control system based on full-space position information | |
| EP3127586B1 (en) | Interactive system, remote controller and operating method thereof | |
| CN112488059B (en) | Spatial gesture control method based on deep learning model cascade | |
| CN111597987B (en) | Method, apparatus, device and storage medium for generating information | |
| CN113971835A (en) | A control method, device, storage medium and terminal device for household electrical appliances | |
| CN110456904B (en) | Augmented reality glasses eye movement interaction method and system without calibration | |
| Xompero et al. | Multi-view shape estimation of transparent containers | |
| CN111002289A (en) | Robot online teaching method and device, terminal device and storage medium | |
| JP2017033556A (en) | Image processing method and electronic apparatus | |
| CN113918013A (en) | Gesture directional interaction system and method based on AR glasses | |
| CN107391289A (en) | A kind of three-dimensional pen-based interaction Interface Usability appraisal procedure | |
| CN109934155B (en) | Depth vision-based collaborative robot gesture recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Hand detection method and system based on depth information Effective date of registration: 20220311 Granted publication date: 20200324 Pledgee: Industrial Bank Co.,Ltd. Beijing West Railway Station sub branch Pledgor: BeiJing Hjimi Technology Co.,Ltd. Registration number: Y2022110000054 |
|
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20231101 Granted publication date: 20200324 |