CN112861818A

CN112861818A - Living body detection method and device and full-automatic safe deposit box

Info

Publication number: CN112861818A
Application number: CN202110350681.4A
Authority: CN
Inventors: 崔伟; 王义锦
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2021-03-31
Filing date: 2021-03-31
Publication date: 2021-05-28

Abstract

The invention provides a method and a device for detecting a living body and a full-automatic safe deposit box, which can be used in the technical field of biological identification, the financial field or other fields. The method comprises the following steps: acquiring a common photo and an infrared photo of a living body to be detected, and fusing the common photo and the infrared photo to obtain a fused photo; obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fused photo according to the fused photo; and performing living body detection on the fusion photo according to the frequency domain distribution, the facial light stream histogram and the facial spectrum information to obtain a living body detection result of the living body to be detected. According to the invention, by fusing the common photo and the infrared photo and utilizing the fused photo for living body detection, the detection accuracy is ensured, no matching action is required, the face information is not required to be uploaded to a server for detection, network transmission links such as service calling and the like are not required, and the detection speed is higher.

Description

Living body detection method and device and full-automatic safe deposit box

Technical Field

The invention relates to the technical field of biological identification, in particular to a method and a device for detecting a living body and a full-automatic safe deposit box.

Background

Nowadays, biometric recognition technology represented by face recognition is rapidly developed, and usage scenarios are increasingly abundant. In order to prevent attacks on the face recognition technology by means of false photos, face changing, masks and the like, before face recognition, the human body is often judged to be a real person or not through living body detection. The two most used biopsy methods at present are: a back-end live body test and a back-end live body test + an action live body test.

(I) posterior end Living body detection

The principle of the method is as follows: the human face is photographed through the camera, the human face information is uploaded to the living body detection interface of the server side, and whether the human face is a real person or not is judged through a certain algorithm in the interface. However, the greatest disadvantage of this method is that the detection capability depends heavily on the quality of the algorithm model, and is also affected by environmental conditions, shooting angle, and quality of the picture collected by the camera, and the detection success rate is not very high because a real person under conditions such as too strong light, too dark light, too poor picture quality, and shooting deflection angle can be mistakenly considered as a prosthetic attack. However, the method is simple in technical implementation and is suitable for scenes with staff on-site guidance (so that the scenes are not dare to be counterfeited by adopting fake photos, masks and the like). If the living body detection is unsuccessful, the staff can guide the client to repeatedly detect by adjusting the photographing angle and the like. The method is adopted when the self-service terminal transacts the auditing service.

(II) rear end biopsy + motion biopsy

The principle of the method is as follows: on the basis of back end living body detection, action living body detection is added, whether a real person is present or not is judged mainly under the coordination of a series of actions such as blinking, shaking, mouth opening and the like, and the method is mostly used for the mobile phone end living body detection. However, the technical implementation difficulty of the motion living body detection is high, the relevant motion must be completed, and the whole detection time may be long.

However, for some complex scenes, such as full-automatic safe deposit box service, the method requires 24-hour service, is unattended, is simple to operate, has high success rate and the like, and does not have an ideal solution at present.

Disclosure of Invention

Aiming at the problems in the prior art, the embodiments of the present invention mainly aim to provide a method, an apparatus and a full-automatic safe deposit box for detecting a living body, which do not need to perform any matching actions and do not need to send face information to a service end to complete accurate living body detection.

In order to achieve the above object, an embodiment of the present invention provides a method for detecting a living body, including:

acquiring a common photo and an infrared photo of a living body to be detected, and fusing the common photo and the infrared photo to obtain a fused photo;

obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fused photo according to the fused photo;

and performing living body detection on the fusion photo according to the frequency domain distribution, the facial light stream histogram and the facial spectrum information to obtain a living body detection result of the living body to be detected.

Optionally, in an embodiment of the present invention, the fusing the common photograph and the infrared photograph to obtain a fused photograph includes:

preprocessing the common photo and the infrared photo, and obtaining edge maps corresponding to the preprocessed common photo and the preprocessed infrared photo respectively by utilizing an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing;

carrying out feature point detection and feature point matching on the edge image of the common photo and the edge image of the infrared photo to obtain a transformation relation between the common photo and the infrared photo, and registering the common photo and the infrared photo by utilizing the transformation relation;

IHS transformation is carried out on the registered common picture to obtain an I component, an H component and an S component, and the I component is utilized to carry out image decomposition on the registered infrared picture to obtain a fused I component;

and performing IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fused picture.

Optionally, in an embodiment of the present invention, the obtaining, according to the fused photo, frequency domain distribution, a facial optical flow histogram, and facial spectrum information of the fused photo includes:

obtaining the frequency domain distribution of the fusion photo according to the distribution of the fusion photo on the frequency domain;

extracting the face optical flow histogram from the fused photo by using an optical flow technology;

and carrying out spectral analysis on the fused photo to obtain the facial spectral information of the fused photo.

Optionally, in an embodiment of the present invention, the performing living body detection on the fusion photo according to the frequency domain distribution, the facial optical flow histogram, and the facial spectral information to obtain a living body detection result of the living body to be detected includes:

according to the frequency domain distribution, carrying out heart rate distribution detection on the fusion photo to obtain a photo attack result;

carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph;

according to the facial spectrum information, performing material detection on the living body to be detected in the fusion photo to obtain a material detection result;

and obtaining a living body detection result of the living body to be detected by using the photo attack result, the screen attack result and the material detection result.

An embodiment of the present invention further provides a living body detection apparatus, including:

the photo fusion module is used for acquiring a common photo and an infrared photo of a living body to be detected, and fusing the common photo and the infrared photo to obtain a fusion photo;

the information extraction module is used for obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fusion photo according to the fusion photo;

and the living body detection module is used for carrying out living body detection on the fusion photo according to the frequency domain distribution, the facial light stream histogram and the facial spectrum information to obtain a living body detection result of the living body to be detected.

Optionally, in an embodiment of the present invention, the photo fusion module includes:

the preprocessing unit is used for preprocessing the common photo and the infrared photo and obtaining edge maps respectively corresponding to the preprocessed common photo and the preprocessed infrared photo by utilizing an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing;

the photo registration unit is used for detecting and matching the characteristic points of the edge image of the common photo and the edge image of the infrared photo to obtain a transformation relation between the common photo and the infrared photo, and registering the common photo and the infrared photo by utilizing the transformation relation;

the IHS transformation unit is used for carrying out IHS transformation on the registered common picture to obtain an I component, an H component and an S component, and carrying out image decomposition on the registered infrared picture by utilizing the I component to obtain a fused I component;

and the photo fusion unit is used for carrying out IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fusion photo.

Optionally, in an embodiment of the present invention, the information extracting module includes:

the frequency distribution unit is used for obtaining the frequency domain distribution of the fusion photo according to the distribution of the fusion photo on the frequency domain;

an optical flow histogram unit for extracting the face optical flow histogram from the fused photograph by an optical flow technique;

and the spectrum information unit is used for carrying out spectrum analysis on the fused photo to obtain the facial spectrum information of the fused photo.

Optionally, in an embodiment of the present invention, the living body detecting module includes:

the photo attack unit is used for carrying out heart rate distribution detection on the fusion photo according to the frequency domain distribution to obtain a photo attack result;

the screen attack unit is used for carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy and obtaining a screen attack result according to the subspace graph;

the material detection unit is used for detecting the material of the living body to be detected in the fusion photo according to the facial spectrum information to obtain a material detection result;

and the detection result unit is used for obtaining the living body detection result of the living body to be detected by utilizing the photo attack result, the screen attack result and the material detection result.

The embodiment of the invention also provides a full-automatic safe deposit box, which comprises a camera, an inner box, a safe deposit box operation module and a living body detection module;

the safe deposit box operation module controls the camera to collect common photos and infrared photos of the user according to the received access request of the user;

the living body detection module acquires a common photo and an infrared photo which are acquired by the camera, and fuses the common photo and the infrared photo to obtain a fused photo; obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fused photo according to the fused photo; according to the frequency domain distribution, the facial optical flow histogram and the facial spectrum information, performing living body detection on the fusion photo to obtain a living body detection result, and sending the living body detection result to the safe deposit box operation module;

and the safe deposit box operation module controls the inner box to be opened according to the living body detection result.

Optionally, in an embodiment of the present invention, the living body detection module is further configured to preprocess the ordinary photograph and the infrared photograph, and obtain edge maps corresponding to the preprocessed ordinary photograph and the preprocessed infrared photograph by using an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing; carrying out feature point detection and feature point matching on the edge image of the common photo and the edge image of the infrared photo to obtain a transformation relation between the common photo and the infrared photo, and registering the common photo and the infrared photo by utilizing the transformation relation; IHS transformation is carried out on the registered common picture to obtain an I component, an H component and an S component, and the I component is utilized to carry out image decomposition on the registered infrared picture to obtain a fused I component; and performing IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fused picture.

Optionally, in an embodiment of the present invention, the living body detecting module is further configured to obtain frequency domain distribution of the fusion photo according to distribution of the fusion photo in a frequency domain; extracting the face optical flow histogram from the fused photo by using an optical flow technology; and carrying out spectral analysis on the fused photo to obtain the facial spectral information of the fused photo.

Optionally, in an embodiment of the present invention, the in-vivo detection module is further configured to perform heart rate distribution detection on the fusion photo according to the frequency domain distribution to obtain a photo attack result; carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph; according to the facial spectrum information, performing material detection on the living body to be detected in the fusion photo to obtain a material detection result; and obtaining a living body detection result of the living body to be detected by using the photo attack result, the screen attack result and the material detection result.

Optionally, in an embodiment of the present invention, the fully automatic safe deposit box further includes a touch screen, which is used for receiving an access request from a user.

Optionally, in an embodiment of the present invention, the fully automatic safe deposit box further includes a manipulator, a conveyor belt, and a box outlet.

Optionally, in an embodiment of the present invention, the safe deposit box operating module is further configured to control the manipulator to obtain an article to be taken from a corresponding inner box according to the living body detection result and a pick-up request in the access request, and place the article to be taken on the conveyor belt; and controlling the conveyor belt to convey the articles to be taken to the box outlet and controlling the box outlet to be opened.

Optionally, in an embodiment of the present invention, the safe deposit box operating module is further configured to control the box outlet to be opened and control the conveyor belt to convey the object to be stored to the manipulator according to the living body detection result and a storage request in the access request; and controlling the manipulator to place the articles to be stored into the corresponding inner box.

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method when executing the program.

The present invention also provides a computer-readable storage medium storing a computer program for executing the above method.

According to the invention, by fusing the common photo and the infrared photo and utilizing the fused photo for living body detection, the detection accuracy is ensured, no matching action is required, the face information is not required to be uploaded to a server for detection, network transmission links such as service calling and the like are not required, and the detection speed is higher.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.

FIG. 1 is a flow chart of a method of detecting a living organism according to an embodiment of the present invention;

FIG. 2 is a flow chart of photo fusion in an embodiment of the present invention;

FIG. 3 is a flow chart of information extraction for fusing photos in an embodiment of the present invention;

FIG. 4 is a flowchart of in vivo testing in an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a fully-automatic safe deposit box according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating operation of the fully automatic safe deposit box according to an embodiment of the present invention;

FIG. 7 is a flow chart illustrating authorization of an SDK in a fully automatic safe box according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a living body detecting apparatus according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a photo fusion module according to an embodiment of the present invention;

FIG. 10 is a block diagram of an information extraction module according to an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of a biopsy module according to an embodiment of the invention;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a living body detection method, a living body detection device and a full-automatic safe deposit box, which can be used in the financial field or other fields.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, a flowchart of a living body detection method according to an embodiment of the present invention is shown, and an implementation subject of the living body detection method provided by the embodiment of the present invention includes, but is not limited to, a computer. The method shown in the figure comprises the following steps:

and step S1, acquiring a common photo and an infrared photo of the living body to be detected, and fusing the common photo and the infrared photo to obtain a fused photo.

Wherein, the living body to be detected generally refers to a person, and a camera is used for collecting a common photo and an infrared photo of the person. Preferably, a binocular camera can be adopted for photo collection.

Further, the photo fusion process comprises: and strictly registering the common photo and the infrared photo by using any anny edge detection and ORB feature detection. Specifically, preprocessing such as denoising and smoothing is carried out on the common photo and the infrared photo, canny edge images of the common photo and the infrared photo are obtained based on a canny edge detection algorithm, SURF feature point detection and feature point preliminary matching are respectively carried out on the canny edge images, an accurate transformation relation between the common photo and the infrared photo is obtained, and registration of the common photo and the infrared photo is completed through the obtained transformation relation; carrying out IHS transformation on the common picture to obtain three components of I, H and S; based on a two-dimensional harr wavelet algorithm, taking an I component of a common photo and an infrared photo as input, and decomposing an image into high-frequency information in horizontal, vertical and diagonal directions and a low-frequency component under corresponding resolution to obtain a fused I component; and performing IHS inverse transformation on the fused I component and the H and S components of the common picture to obtain a fused image.

And step S2, obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fused photo according to the fused photo.

The frequency domain distribution of the fusion photo can be obtained according to the different distribution of the fusion photo on the frequency domain. Using optical flow techniques, a facial optical flow histogram may be extracted from the fused photograph. Specifically, each pixel point in the fused picture is given a velocity vector so as to form a motion field of the image, and a limit point of a feature vector is calculated in a motion matrix so as to obtain a face optical flow histogram. Furthermore, by analyzing the spectral information of the face reflection in the fused photograph, the face spectral information is obtained.

And step S3, performing living body detection on the fusion photo according to the frequency domain distribution, the facial optical flow histogram and the facial spectrum information to obtain a living body detection result of the living body to be detected.

And judging whether the photo attack is the photo attack or not according to the frequency domain distribution of the fused photo. Because the fusion photograph will contain a heart rate distribution, it is judged that there is no heart rate if it is a photograph, depending on the distribution difference in frequency.

Furthermore, the extracted face optical flow histogram is decomposed through a dynamic mode to obtain a subspace graph with the maximum motion energy, and then texture is analyzed to judge whether the face optical flow histogram is a screen video attack. Furthermore, the difference between the real human face skin and other attacking materials can be distinguished by analyzing the spectrum information of the fused photo. Specifically, a spectrum curve chart is analyzed by using the difference of the reflectivity of the skin and other materials, if the human face is detected, the reflectivity shows a W-shaped trend, and other materials have no characteristics.

Further, after the live body detection judgment is carried out on the fusion photo by utilizing the frequency domain distribution, the facial optical flow histogram and the facial spectrum information, if the photo attack result, the screen attack result and the material detection result are all passed, the live body detection result of the to-be-detected live body is passed. And if any result of the photo attack result, the screen attack result and the material detection result is judged to be failed, namely photo attack, screen video attack or other material attacks are identified, the living body detection result of the living body to be detected is failed.

As an embodiment of the present invention, as shown in fig. 2, fusing the normal photograph and the infrared photograph to obtain a fused photograph includes:

step S11, preprocessing the common photo and the infrared photo, and obtaining edge images respectively corresponding to the preprocessed common photo and the preprocessed infrared photo by using an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing;

step S12, feature point detection and feature point matching are carried out on the edge graph of the ordinary photo and the edge graph of the infrared photo to obtain a transformation relation between the ordinary photo and the infrared photo, and the ordinary photo and the infrared photo are registered by utilizing the transformation relation;

step S13, IHS transformation is carried out on the registered common picture to obtain an I component, an H component and an S component, and the I component is utilized to carry out image decomposition on the registered infrared picture to obtain a fused I component;

and step S14, performing IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fused picture.

Wherein, the common photo and the infrared photo are strictly registered by utilizing anny edge detection and ORB characteristic detection. Specifically, preprocessing such as denoising and smoothing is carried out on the common photo and the infrared photo, canny edge images of the common photo and the infrared photo are obtained based on a canny edge detection algorithm, SURF feature point detection and feature point preliminary matching are respectively carried out on the canny edge images, an accurate transformation relation between the common photo and the infrared photo is obtained, and registration of the common photo and the infrared photo is completed through the obtained transformation relation; carrying out IHS transformation on the common picture to obtain three components of I, H and S; based on a two-dimensional harr wavelet algorithm, taking an I component of a common photo and an infrared photo as input, and decomposing an image into high-frequency information in horizontal, vertical and diagonal directions and a low-frequency component under corresponding resolution to obtain a fused I component; and performing IHS inverse transformation on the fused I component and the H and S components of the common picture to obtain a fused image.

As an embodiment of the present invention, as shown in fig. 3, obtaining the frequency domain distribution, the face optical flow histogram, and the face spectral information of the fused photograph from the fused photograph includes:

and step S21, obtaining the frequency domain distribution of the fusion photo according to the distribution of the fusion photo on the frequency domain.

The frequency domain distribution of the fusion photo can be obtained according to the different distribution of the fusion photo on the frequency domain.

In step S22, the face optical flow histogram is extracted from the fused photograph by an optical flow technique.

Wherein, by using the optical flow technology, a face optical flow histogram can be extracted from the fusion photo. Specifically, each pixel point in the fused photo is given a velocity vector so as to form a motion field of the image, and limit points of a feature vector are calculated in a motion matrix so as to obtain a facial optical flow histogram

And step S23, performing spectral analysis on the fused photo to obtain the facial spectral information of the fused photo.

The facial spectral information is obtained by analyzing the spectral information of the facial reflection in the fused photograph.

As an embodiment of the present invention, as shown in fig. 4, performing a living body detection on the fused photograph according to the frequency domain distribution, the facial optical flow histogram and the facial spectrum information, and obtaining a living body detection result of the living body to be detected includes:

and step S31, according to the frequency domain distribution, carrying out heart rate distribution detection on the fusion photo to obtain a photo attack result.

Whether the photo attack is detected can be distinguished according to different distribution of the fusion photo on the frequency domain. Since the fusion shots will contain heart rate distributions, they are judged by the difference in frequency distribution. If a photograph, it is without heart rate.

And step S32, carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph.

The method comprises the steps of extracting a face optical flow histogram by using an optical flow technology, extracting feature vectors in a fusion photo from the fusion photo, and obtaining a subspace graph with maximum motion energy through dynamic mode decomposition. Specifically, the dynamic mode decomposition is a dimension reduction method, and since the obtained feature vector has a high dimension, the calculation of the feature value is very difficult, and the dynamic mode can solve the problem, and further obtain the feature value. And analyzing the texture of the subspace graph with the maximum motion energy to judge whether the attack is a screen video attack.

Specifically, how the modes of the feature vectors change in different spaces is known according to the calculated feature values, and the texture features of the video attack have the characteristics of small correlation, small energy value, no obvious change, large inverse difference distance and the like, so that whether the video attack is a screen video attack or not is distinguished.

And step S33, performing material detection on the living body to be detected in the fusion photo according to the facial spectrum information to obtain a material detection result.

The difference between the real human face skin and other attack materials can be distinguished by analyzing the spectral information reflected by the human face.

Specifically, the spectral information is extracted from the fused photograph in the following extraction process: and performing data modeling on the fused photo, adjusting model parameters to obtain a Gaussian model, and calculating based on a DLFA algorithm to obtain a spectral curve graph of the wavelength and the reflectivity. Furthermore, the spectrum curve chart is analyzed by utilizing the difference of the reflectivity of the skin and other materials, if the human face is detected, the reflectivity shows a W-shaped trend, and other materials have no characteristics.

And step S34, obtaining the living body detection result of the living body to be detected by using the photo attack result, the screen attack result and the material detection result.

If the photo attack result, the screen attack result and the material detection result are all passed through the judgment of the steps S31-S33, the living body detection result of the living body to be detected is passed. And if any result of the photo attack result, the screen attack result and the material detection result is judged to be failed, namely photo attack, screen video attack or other material attacks are identified, the living body detection result of the living body to be detected is failed.

Fig. 5 is a schematic structural diagram of a full-automatic safe deposit box according to an embodiment of the present invention, where the full-automatic safe deposit box includes: a camera 2, an inner box 7, a safe deposit box operation module (not shown in the figure) and a living body detection module (not shown in the figure);

and the safe deposit box operation module controls the camera 2 to collect the common photos and the infrared photos of the user according to the received access request of the user.

The safe deposit box operation module is used for handling the safe deposit box service for the user. The camera 2 can adopt a binocular camera for collecting face information, including common pictures and infrared pictures. Further, the safe box operation module can receive an access request of a user through the touch screen. The access request comprises a pickup request and a deposit request.

A living body detection module acquires a common photo and an infrared photo collected by the camera, and fuses the common photo and the infrared photo to obtain a fused photo; obtaining the frequency domain distribution, the face optical flow histogram and the face spectrum information of the fused photo according to the fused photo; and performing living body detection on the fusion photo according to the frequency domain distribution, the facial light stream histogram and the facial spectrum information to obtain a living body detection result, and sending the living body detection result to the safe deposit box operation module.

Wherein, the photo fusion process comprises the following steps: and strictly registering the common photo and the infrared photo by using any anny edge detection and ORB feature detection. Specifically, preprocessing such as denoising and smoothing is carried out on the common photo and the infrared photo, canny edge images of the common photo and the infrared photo are obtained based on a canny edge detection algorithm, SURF feature point detection and feature point preliminary matching are respectively carried out on the canny edge images, an accurate transformation relation between the common photo and the infrared photo is obtained, and registration of the common photo and the infrared photo is completed through the obtained transformation relation; carrying out IHS transformation on the common picture to obtain three components of I, H and S; based on a two-dimensional harr wavelet algorithm, taking an I component of a common photo and an infrared photo as input, and decomposing an image into high-frequency information in horizontal, vertical and diagonal directions and a low-frequency component under corresponding resolution to obtain a fused I component; and performing IHS inverse transformation on the fused I component and the H and S components of the common picture to obtain a fused image.

Furthermore, according to different distribution of the fusion photos on the frequency domain, the frequency domain distribution of the fusion photos can be obtained. Using optical flow techniques, a facial optical flow histogram may be extracted from the fused photograph. Specifically, each pixel point in the fused picture is given a velocity vector so as to form a motion field of the image, and a limit point of a feature vector is calculated in a motion matrix so as to obtain a face optical flow histogram. Furthermore, by analyzing the spectral information of the face reflection in the fused photograph, the face spectral information is obtained.

And if the living body detection result is that the living body detection result passes, the safe deposit box operation module controls the corresponding inner box to be opened so as to store or take the articles. And if the living body detection result is that the living body detection result does not pass, the safe deposit box operation module does not open the inner box.

As an embodiment of the present invention, the living body detection module is further configured to preprocess the ordinary photograph and the infrared photograph, and obtain edge maps corresponding to the preprocessed ordinary photograph and the preprocessed infrared photograph by using an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing; carrying out feature point detection and feature point matching on the edge image of the common photo and the edge image of the infrared photo to obtain a transformation relation between the common photo and the infrared photo, and registering the common photo and the infrared photo by utilizing the transformation relation; IHS transformation is carried out on the registered common picture to obtain an I component, an H component and an S component, and the I component is utilized to carry out image decomposition on the registered infrared picture to obtain a fused I component; and performing IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fused picture.

As an embodiment of the present invention, the living body detecting module is further configured to obtain frequency domain distribution of the fusion photo according to distribution of the fusion photo in a frequency domain; extracting the face optical flow histogram from the fused photo by using an optical flow technology; and carrying out spectral analysis on the fused photo to obtain the facial spectral information of the fused photo.

As an embodiment of the present invention, the living body detection module is further configured to perform heart rate distribution detection on the fusion photo according to the frequency domain distribution to obtain a photo attack result; carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph; according to the facial spectrum information, performing material detection on the living body to be detected in the fusion photo to obtain a material detection result; and obtaining a living body detection result of the living body to be detected by using the photo attack result, the screen attack result and the material detection result.

As an embodiment of the present invention, the fully automatic safe deposit box further comprises a touch screen 3 for receiving an access request of a user.

The touch screen 3 is used for receiving an access request of a user, and specifically, the touch screen 3 is responsible for interacting with a customer, such as displaying and selecting a box number. The full-automatic safe deposit box also comprises an end machine input device 1 which mainly comprises a display and an operating system and is responsible for living body detection and face recognition.

The fully automatic safe deposit box further comprises a mechanical arm 6, a conveyor belt 5 and a box outlet 4.

In this embodiment, the safe deposit box operating module is further configured to control the manipulator to obtain an article to be taken from a corresponding inner box according to the living body detection result and a pick-up request in the access request, and place the article to be taken on the conveyor belt; and controlling the conveyor belt to convey the articles to be taken to the box outlet and controlling the box outlet to be opened.

In this embodiment, the safe deposit box operating module is further configured to control the box outlet to be opened and control the conveyor belt to convey the object to be stored to the manipulator according to the living body detection result and a storage request in the access request; and controlling the manipulator to place the articles to be stored into the corresponding inner box.

Wherein, go out the incasement mouth 4 and be responsible for taking the inner box, conveyer belt 5 is responsible for transmitting the inner box to the case mouth of going out, and manipulator 6 is responsible for snatching the inner box, and inner box 7 is responsible for the access of article.

Specifically, after a customer transacts the safe deposit box service on the safe deposit box operation module, if the customer accesses articles, the customer needs to firstly enter a watching room (a completely independent privacy space without monitoring and working personnel), living body detection and face recognition are carried out in the watching room through the terminal input device 1, the number of the box which is displayed on the touch screen 3 is selected after the verification is passed, the manipulator 6 grabs the inner box 7 to be taken out, and then the inner box is conveyed to the box outlet 4 through the conveyor belt 5, so that the customer can operate the articles stored in the inner box 7.

In one embodiment of the present invention, the liveness detection is accomplished by an SDK kit installed on the input device 1 of the terminal, as shown in FIG. 6. The internal processing of the SDK toolkit includes the following parts: the system comprises a photographing driving module, a binocular camera, a visible light and infrared fusion module, an image preprocessing module, a living body detection and analysis module and an output detection result.

And the photographing driving module drives the binocular camera to photograph the human face through a js function at the front end, and comprises a common picture and an infrared picture.

And the picture preprocessing module is used for preprocessing the two pictures such as gray correction, noise filtration, normalization, image enhancement and the like and preparing for the next processing of the pictures.

The visible light and infrared fusion module adopts an IHS and wavelet transformation fusion algorithm to synthesize a fusion photo with high spatial resolution, clear color texture, rich image information and the like.

The specific fusion process is as follows:

(1) based on canny edge detection and ORB feature detection, the normal photo and the infrared photo are strictly registered. Firstly, preprocessing such as denoising and smoothing is carried out on a common photo and an infrared photo, canny edge images of the common photo and the infrared photo are obtained based on a canny edge detection algorithm, SURF feature point detection and feature point preliminary matching are respectively carried out on the canny edge images, an accurate transformation relation between the common photo and the infrared photo is obtained, and registration of the common photo and the infrared photo is completed through the obtained transformation relation.

(2) And performing IHS transformation on the common picture to obtain three components of I, H and S.

(3) Based on a two-dimensional harr wavelet algorithm, the I component of a common photo and an infrared photo are used as input, and the fused I component is obtained by decomposing an image into high-frequency information in the horizontal direction, the vertical direction and the diagonal direction and a low-frequency component under the corresponding resolution.

(4) And performing IHS inverse transformation on the fused I component and the H and S components of the common picture to obtain a fused image.

And the living body detection analysis module can distinguish whether the photo attack is existed according to different distribution of the fused photos on the frequency domain. Since the fusion shots will contain heart rate distributions, they are judged by the difference in frequency distribution. If it is a photograph, it is without heart rate; the extraction process of the extracted facial optical flow histogram comprises the steps of extracting by using an optical flow technology, specifically, giving a velocity vector to each pixel point in an image so as to form a motion field of the image, and calculating a limit point of a characteristic vector in a motion matrix so as to obtain the optical flow histogram.

Further, extracting the feature vectors in the fusion photo, and obtaining a subspace graph with maximum motion energy through dynamic mode decomposition. The dynamic mode decomposition is a dimension reduction method, because the obtained feature vector has high dimension, the calculation of the feature value is very difficult, and the dynamic mode can solve the problem and further obtain the feature value. Whether the video attack is a screen video attack can be judged by analyzing the texture, specifically, how the modes of the feature vector change in different spaces can be known according to the feature values obtained by calculation, and the texture features of the video attack have the characteristics of small correlation, small energy value, no obvious change, large inverse difference distance and the like, so that whether the video attack is the video attack or not can be distinguished.

Further, spectral information is extracted from the fused photograph by analyzing spectral information reflected by the human face. The extraction process is as follows: and performing data modeling on the fused photo, adjusting model parameters to obtain a Gaussian model, and calculating based on a DLFA algorithm to obtain a spectral curve graph of the wavelength and the reflectivity. The difference of the real human face skin and other attacking materials can be distinguished by utilizing the facial spectrum information, the spectrum curve graph is analyzed, if the human face exists, the reflectivity presents W-shaped trend, and other materials have no characteristics.

And outputting the detection result, outputting the final result of the living body detection to a display of the terminal equipment input device 1, converting the json data format, and displaying whether the json data passes or not on a screen, wherein the data format is json.

The invention carries out detection at the client end provided with the camera, and a packaged SDK toolkit (SDK for short) is arranged on the client end machine, and the internal main function of the SDK is to carry out living body detection on two photos collected by a binocular camera so as to judge whether the two photos are live bodies of real people. In order to ensure the uniqueness and the safety of the SDK use, the client device and the SDK are ensured to be bound one by one in a remote authorization mode. And generating a pre-authorization file by using an authorization tool, requesting an authorization service deployed by a server, generating an authorization file, and putting the authorization file into a specified directory to finish the authorization of the SDK. Authorization services need only be requested once at the time of initial use.

The detection process is completed at the client side, and network transmission processes such as service calling and the like are not needed, so that the detection speed is higher. The SDK can be used only after authorization, so that the safety and reliability in the in-vivo detection process are ensured, and the requirement of the full-automatic safe deposit box service on the in-vivo detection is realized.

In an embodiment of the invention, as shown in fig. 7, it is a flow chart of SDK authorization in the in-vivo detection, and the SDK can be normally used only after authorization is successful.

And installing an authorization tool on the terminal input device 1, wherein the authorization tool is an executable exe tool, and the operation acquires relevant physical information (including mac, ip, cpu and the like) of the terminal input device 1 and generates a pre-authorization file.

And the authorization service is deployed at the service end and is used for acquiring the physical information of the terminal equipment by analyzing the pre-authorization file and generating an authorization file containing the unique authorization ID according to the physical information.

Requesting an authorization service through http on the terminal input device 1, uploading a pre-authorization file, returning an authorization file, and placing the authorization file under a specified directory to complete the authorization of the SDK.

Authorization services need only be requested once at the time of initial use.

Compared with the back-end live body detection and action live body detection technology, the full-automatic safe deposit box does not need to do any matching action, does not need to send face information to a server side for detection, and detects the face information at a client side according to an SDK packet. The detection speed is higher because network transmission links such as service calling and the like are not needed. Meanwhile, the client equipment and the SDK are bound one by one in a remote authorization mode, the SDK is authorized by generating an authorization file, and the safety and reliability in the living body detection process are ensured, so that a full-automatic safe box service scene with 24-hour service, unattended operation and high detection success rate requirement is met.

Fig. 8 is a schematic structural diagram of a living body detecting apparatus according to an embodiment of the present invention, wherein the apparatus includes:

the photo fusion module 10 is configured to acquire a common photo and an infrared photo of a living body to be detected, and fuse the common photo and the infrared photo to obtain a fusion photo.

And the information extraction module 20 is configured to obtain frequency domain distribution, a facial optical flow histogram, and facial spectrum information of the fused photo according to the fused photo.

And the living body detection module 30 is configured to perform living body detection on the fusion photo according to the frequency domain distribution, the facial optical flow histogram and the facial spectrum information to obtain a living body detection result of the living body to be detected.

As an embodiment of the present invention, as shown in fig. 9, the photo fusion module 10 includes:

the preprocessing unit 11 is configured to preprocess the ordinary photograph and the infrared photograph, and obtain edge maps corresponding to the preprocessed ordinary photograph and the preprocessed infrared photograph by using an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing;

the photo registration unit 12 is configured to perform feature point detection and feature point matching on the edge image of the ordinary photo and the edge image of the infrared photo to obtain a transformation relationship between the ordinary photo and the infrared photo, and register the ordinary photo and the infrared photo by using the transformation relationship;

an IHS transformation unit 13, configured to perform IHS transformation on the registered common picture to obtain an I component, an H component, and an S component, and perform image decomposition on the registered infrared picture by using the I component to obtain a fused I component;

and a picture fusion unit 14, configured to perform IHS inverse transformation according to the H component, the S component, and the fused I component to obtain the fused picture.

As an embodiment of the present invention, as shown in fig. 10, the information extraction module 20 includes:

the frequency distribution unit 21 is configured to obtain frequency domain distribution of the fused photo according to distribution of the fused photo in a frequency domain;

an optical flow histogram unit 22 for extracting the face optical flow histogram from the fused photograph by an optical flow technique;

and the spectrum information unit 23 is configured to perform spectrum analysis on the fused photo to obtain facial spectrum information of the fused photo.

As one embodiment of the present invention, as shown in fig. 11, the living body detecting module 30 includes:

the photo attack unit 31 is configured to perform heart rate distribution detection on the fusion photo according to the frequency domain distribution to obtain a photo attack result;

the screen attack unit 32 is used for performing dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph;

a material detection unit 33, configured to perform material detection on the living body to be detected in the fusion photograph according to the facial spectrum information, so as to obtain a material detection result;

and the detection result unit 34 is configured to obtain a living body detection result of the living body to be detected by using the photo attack result, the screen attack result, and the material detection result.

Based on the same application concept as the living body detection method, the invention also provides the living body detection device. Because the principle of solving the problems of the living body detection device is similar to that of a living body detection method, the implementation of the living body detection device can refer to the implementation of the living body detection method, and repeated parts are not repeated.

As shown in fig. 12, the electronic device 600 may further include: communication module 110, input unit 120, audio processing unit 130, display 160, power supply 170. It is noted that the electronic device 600 does not necessarily include all of the components shown in fig. 12; furthermore, the electronic device 600 may also comprise components not shown in fig. 12, which may be referred to in the prior art.

As shown in fig. 12, the central processor 100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, the central processor 100 receiving input and controlling the operation of the various components of the electronic device 600.

The memory 140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 100 may execute the program stored in the memory 140 to realize information storage or processing, etc.

The input unit 120 provides input to the cpu 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display an object to be displayed, such as an image or a character. The display may be, for example, an LCD display, but is not limited thereto.

The memory 140 may be a solid state memory such as Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 140 may also be some other type of device. Memory 140 includes buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application/function storage section 142, and the application/function storage section 142 is used to store application programs and function programs or a flow for executing the operation of the electronic device 600 by the central processing unit 100.

The memory 140 may also include a data store 143, the data store 143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by the electronic device. The driver storage portion 144 of the memory 140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging application, address book application, etc.).

The communication module 110 is a transmitter/receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter/receiver) 110 is coupled to the central processor 100 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.

Based on different communication technologies, a plurality of communication modules 110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132 to implement general telecommunications functions. Audio processor 130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, an audio processor 130 is also coupled to the central processor 100, so that recording on the local can be enabled through a microphone 132, and so that sound stored on the local can be played through a speaker 131.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A method of in vivo detection, the method comprising:

2. The method of claim 1, wherein the fusing the normal photo and the infrared photo to obtain a fused photo comprises:

3. The method of claim 1, wherein obtaining the frequency domain distribution, the face optical flow histogram and the face spectral information of the fused photo from the fused photo comprises:

4. The method according to claim 1 or 3, wherein the performing the living body detection on the fused photo according to the frequency domain distribution, the facial optical flow histogram and the facial spectrum information to obtain the living body detection result of the living body to be detected comprises:

5. A living body detection apparatus, the apparatus comprising:

6. The apparatus of claim 5, wherein the photo fusion module comprises:

7. The apparatus of claim 5, wherein the information extraction module comprises:

8. The apparatus of claim 5 or 7, wherein the liveness detection module comprises:

9. A full-automatic safe deposit box is characterized by comprising a camera, an inner box, a safe deposit box operation module and a living body detection module;

10. The full-automatic safe deposit box according to claim 9, wherein the living body detection module is further configured to preprocess the normal photo and the infrared photo, and obtain edge maps corresponding to the preprocessed normal photo and the preprocessed infrared photo respectively by using an edge detection algorithm; the preprocessing comprises denoising processing and smoothing processing; carrying out feature point detection and feature point matching on the edge image of the common photo and the edge image of the infrared photo to obtain a transformation relation between the common photo and the infrared photo, and registering the common photo and the infrared photo by utilizing the transformation relation; IHS transformation is carried out on the registered common picture to obtain an I component, an H component and an S component, and the I component is utilized to carry out image decomposition on the registered infrared picture to obtain a fused I component; and performing IHS inverse transformation according to the H component, the S component and the fused I component to obtain the fused picture.

11. The fully-automatic safe deposit box according to claim 9, wherein the living body detecting module is further configured to obtain a frequency domain distribution of the fused photo according to the distribution of the fused photo in the frequency domain; extracting the face optical flow histogram from the fused photo by using an optical flow technology; and carrying out spectral analysis on the fused photo to obtain the facial spectral information of the fused photo.

12. The full-automatic safe deposit box according to claim 9 or 11, wherein the living body detection module is further configured to perform heart rate distribution detection on the fused photo according to the frequency domain distribution to obtain a photo attack result; carrying out dynamic mode decomposition on the face optical flow histogram to obtain a subspace graph with maximum motion energy, and obtaining a screen attack result according to the subspace graph; according to the facial spectrum information, performing material detection on the living body to be detected in the fusion photo to obtain a material detection result; and obtaining a living body detection result of the living body to be detected by using the photo attack result, the screen attack result and the material detection result.

13. The fully automatic safe deposit box according to claim 9, further comprising a touch screen for receiving a user access request.

14. The fully automated safe deposit box of claim 9, further comprising a robot, a conveyor belt, and a box exit.

15. The fully automatic safe deposit box according to claim 14, wherein the safe deposit box operation module is further configured to control the robot to take the article to be taken from the corresponding inner box and place the article to be taken on the conveyor belt according to the result of the living body detection and the article taking request in the access request; and controlling the conveyor belt to convey the articles to be taken to the box outlet and controlling the box outlet to be opened.

16. The fully automatic safe deposit box according to claim 14, wherein the safe deposit box operation module is further configured to control the box outlet to open and control the conveyor belt to convey the object to be stored to the manipulator according to the living body detection result and a storage request in the access request; and controlling the manipulator to place the articles to be stored into the corresponding inner box.

17. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 4 when executing the computer program.

18. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 4.