KR102202342B1 - All-in-one light-emitting display and sensor to detect biological properties - Google Patents

Abstract

A biosensor device integrated with a display portion, and includes a surface contacted by a body portion such as a finger. Light sources such as LED arrays emit light through the surface, which is reflected and partially absorbed by body parts. The array of photo detectors detects the light reflected by the body part and generates a signal corresponding to the image of the light reflection, which corresponds to the light absorption pattern at the body part. The light absorption pattern may be associated with fingerprints, blood vessel patterns, blood movement within blood vessels, combinations thereof, or other biometric features. The processor receives the signal from the photo detector and analyzes the signal to determine the characteristics of the body part. This characteristic can be used to authenticate the user of the biosensor device by comparing the detected characteristic to the stored characteristic.

Description

All-in-one light-emitting display and sensor to detect biological properties

Cross-reference to related applications

This application is a continuation in part of U.S. Patent Application No. 15/600,480 filed May 19, 2017, and U.S. Provisional Patent Application Nos. 62/348,096, filed June 9, 2016 by Jerome Chandra Bhat, and Jerome Based on and claims priority to U.S. Provisional Patent Application No. 62/340,218, filed May 23, 2016 by Chandra Bhat and Richard Ian Olsen, which is assigned to and incorporated herein by reference.

Field of invention

The present invention relates to detection of biological properties such as blood flow, blood components, fingerprints, blood vessel patterns, faces, and the like, and more particularly, to detection of such properties using light and photo detectors.

Applying light of a specific wavelength (e.g., red or IR) to a human body part (e.g., a finger) and measuring the light transmitted through the body part is blood flow (e.g., pulse), components in the blood (e.g., hemoglobin) It is known that it can be used to detect fat, fat and other properties. In general, light absorption is related to certain properties.

However, such systems are generally limited to medical devices that perform only one function. In addition, since the measurement of light absorption is through a body part, the device must be specifically made to surround the specific body part being examined, such as a finger.

There is a need for a more flexible and compact biological sensor that can be used for various functions and can be used for medical purposes as well as non-medical purposes such as user authentication. In the case of user authentication, biological sensors must be able to be easily integrated into existing consumer products.

In U.S. Patent No. 9,570,002 column 31 discloses the addition of an IR LED emitter with a photo detector to a matrix of a full color display screen. Thereafter, the next user can touch the screen, and the fingerprint image of the user can be detected by detecting reflection from the ridge of the fingertip touching the screen. This detection is made only on the surface of the fingertip, not by measuring light absorption deep in the finger to detect blood vessels or blood flow. Thus, if the device is used for authentication, it is possible to trick the device with a simulated/fake fingerprint. Furthermore, the display screen is a dense array of red, green and blue LEDs, and it is difficult to add an IR LED and a photo detector to an existing LED array without loss of resolution. In addition, it will be very expensive to develop a new display screen with integrated display pixels and detection pixels.

An integrated compact light emitting display and sensor device that can be used to detect the biological properties of a person using the device is described. This trait can be used for medical/diagnostic purposes as well as human authentication. For example, a device can be installed on a smartphone, computer, or even a firearm to authenticate users.

In one embodiment, high-resolution pixels formed by multi-wavelength light sources, such as LEDs or filtered white light, provide light that penetrates a body part, such as a finger, when applied to the skin. An array of photo detectors is integrated into the light emitting display to detect the amount of light reflected off a body part, where the amount of reflected light is affected by the absorption of light by the body part. The array of photo detectors can produce high-resolution images of light absorption. Pixels (eg, emission wavelength) can be controlled to target specific biological properties, and the resolution of the device can be high to detect detailed properties such as fingerprints and blood vessel patterns. Video images may be captured. For the purpose of detecting the location of blood vessels, etc., optics may be used to detect absorption only to a certain depth of the skin. The processor in the device can be programmed to analyze the signal from the photo detector and generate results such as authentication of the user.

Since it would be very difficult to falsify both the fingerprint and blood vessel pattern, this device has much higher reliability in authentication compared to a simple fingerprint detector.

In another embodiment, LED light (e.g., IR light) is connected to the edge of a transparent light guide, which light is guided within the light guide by TIR, since the light guide and air have very different refractive indices. . Assuming that a user's finger is pressed against a light guide surface such as glass for fingerprint detection, the refractive indexes of the finger and the glass are quite matched. If the fingerprint protrusion (ridge) contacts the glass surface, it will create an air gap in the trough. Thus, the light is extracted with the finger from the light guide only in the pattern of the protrusion. Thus, the light guide can be transparent. A high-resolution photo detector array is positioned behind the light guide. Light reflected from a body part (eg, a finger) in contact with the upper surface of the light guide is transmitted back through the light guide and is detected by the photo detector array. Optionally, the device may act as a display through the display pixels behind the light guide. The resulting detected image can be used to authenticate the user by fingerprint, blood vessel pattern or other combination of biological characteristics. In one embodiment, since the light guide is also a display glass for the display screen laterally spaced from the detector portion, the detector and the display are integrated. The display pixel can be conventional, such as an OLED, and only the detector light source uses the display glass as a light guide. Synergy and integration are achieved by using the display glass as a dual use of a light guide and a protective layer for a display (or a touch screen for a display), and TIR uniformly diffuses light for fingerprint/vessel detection. The detector's light source can be optically coupled near any edge of the display glass.

The detector array may be disposed over the entire display pixel or may be laterally spaced from the display portion.

Light enters the finger at a distance through the fingerprint protrusion, is scattered within the finger, and is absorbed by blood vessels within a certain depth. This will create a second reflective pattern that can be used with the fingerprint pattern to detect the positional relationship between the fingerprint and the blood vessel pattern. The fingerprint pattern has a strong contrast, and the blood vessel pattern is effectively superimposed on the fingerprint pattern.

In one embodiment, the light source and light sensor portions are spaced laterally from the display pixels, eg along one side of the display portion, so as not to use any space in the display portion. The display portion produces an image by emitting light through a light-passing panel, for example a thin glass panel. The glass panel may also include a capacitive touch sensor. The light source for the bio-detector portion may be laterally spaced from the display portion, such as next to the light sensor portion, or, as described above, this light source is optically coupled to the edge of the light-passing panel and the light-passing panel is a waveguide for the light source. Plays the role of. Thus, detection is made through an area spaced apart from the display area. The light source can emit IR light, and the display portion emits red, green and blue light from the pixel array. In this way, the light source and the detector portion do not affect the display portion, and the display portion and the detector portion use the same light passing plate. Thus, the display and sensor are integrated but laterally spaced apart.

This device can also be used as a sensor or pulse sensor for specific blood components that absorb target wavelengths.

Different LEDs emitting different wavelengths can be activated individually, and the reflected light can be detected by a broad spectrum light detector to determine different types of biological properties of the body part.

Like blood vessels, optics can be used to focus light to detect biological features only at specific depths in a body part.

Additional sensors, such as gas sensors, electrodes for pulse detection, and the like, may be integrated into the optical sensor.

The device is compact and can be inexpensive, so it can be used in handheld devices for authentication or for analyzing the biological properties of users.

Various different designs and uses of these devices are described.

1 is a plan view of a high-resolution light-emitting display capable of emitting light of a selected wavelength in a selected area, the device also comprising an array of photo detectors within each of the multi-wavelength pixel areas, from a body part of a person in contact with the light-emitting surface Detects light absorption.
FIG. 2 is a cross-sectional view of the device of FIG. 1 showing a human finger placed on a light emitting surface to detect fingerprints, blood flow, hemoglobin in blood, or other properties for authentication or medical diagnosis.
Figure 3 shows the correlation of light wavelength versus absorption for hemoglobin and oxygen-hemoglobin in blood, which can be used by the processor of the device of Figures 1 and 2 to determine the concentration of these components in human blood. have.
4 is a cross-sectional view of another embodiment of a device comprising a light source and a light sensor for detecting biological properties, the light being injected into the edge portion of a transparent or translucent light guide and emitted from the front side of the light guide, wherein the photodiode array is It is located at the rear of the light guide to detect light absorption images by body parts.
5 shows the device of FIG. 4 attached to a transparent layer, such as a protective outer layer of a smart phone, laptop computer or other consumer device.
6 shows the device of FIG. 4 with reinforced electrodes for detecting an electric signal from a human body part, such as detection of an EKG signal, along with detection of reflected light.
7 shows the device of FIG. 5 further reinforced with a sensitive gas detection element for detecting gases emitted from a person.
FIG. 8 shows the device of FIG. 6 further enhanced with a non-contact infrared temperature sensor for measuring the temperature of a body part (eg, a finger) in contact with the device.
9 shows another embodiment of the present invention, in which a focusing lens is attached to the optical sensor array to form a high resolution, high signal-to-noise ratio, and compact device.
10 is a front view of a single integrated circuit chip with a high resolution array of photo detectors. A lens array is placed over the detector array to reduce the amount of direct light entering the photo detector to focus the incident light and increase the signal-to-noise ratio. The light source may have many suitable designs, such as a transparent light guide that emit light only in the direction of the body part in contact with the light guide. The reflected light is transmitted back to the photo detector through the light guide.
Figure 11 is similar to Figure 10, but incorporates a light source (eg, LED or laser) on the same surface as the photo detector.
12 shows functions that can be performed with various devices.
13 is a flowchart of an example of the basic steps performed by a device to detect a biological characteristic of a user.
14 is a plan view of another embodiment of an integrated sensor and display, the sensor portion being laterally spaced from the display portion but using the same cover glass as the display portion.
Fig. 15 is a cross-sectional view of the device of Fig. 14, showing that the sensor portion and the IR emitter are laterally spaced from the display portion and do not affect the display portion. For ease of understanding, the sensor and IR emitter are shown side-by-side in FIG. 15 as opposed to the more accurate cross section of FIG. 14.
16 is a cross-sectional view of an integrated display/sensor showing details of the sensor portion along the edge of the display portion.
17 is a cross-sectional view of an integrated display/sensor, showing the conductive traces being electrically connected to the bottom of the detector element.
Fig. 18 shows the device of Fig. 15 that emits light reflected and absorbed by a body part, and the reflected light is detected to generate a detected image of the reflected/absorbed light to authenticate the user.
19 shows another embodiment in which the display area is partially transparent and the sensor receives light from a body part through the display area.
FIG. 20 is a cross-sectional view of the embodiment of FIG. 19, in which transparent areas are dispersed in the display portion, so that light reflected from the body portion is detected by the detector array under the display portion. The display pixels (eg, red, green and blue sub-pixels) may provide a light source for the detector array or the light source may be a separate light source such as an IR emitter.
21 is a cross-sectional view of another embodiment, in which red, green, and blue OLED sub-pixels are formed on an upper substrate, and at least one photodiode is formed together with a thin film transistor on the bottom substrate to detect light reflected from a body part, and a thin film transistor Controls the display pixels.
22 is a cross-sectional view of another embodiment, in which an IR emitter, such as one or more IR LEDs, is connected to the edge of the display glass of the display serving as a light guide, and the display glass is reflected from a body part in contact with the glass. It emits the light that will be. This light may be detected only in an area laterally spaced from the display area, or may be detected by a photo detector dispersed within the display area.
23 is a cross-sectional view of another embodiment, in which a liquid crystal pixel is formed with a photodiode on a transparent substrate (eg, glass), and light from the backlight passes through the liquid crystal pixel and is reflected from a body part and is detected by the photodiode. .
The same or similar elements in the various drawings are indicated by the same number.

Various types of bio-sensor devices are described that emit light of a specific wavelength and use image capture to detect the absorption of light by a part of the body of a person in contact with (or close to) the light emitting window. Other uses are also described.

1 is a plan view of a bio-sensor device 10 having a size suitable for a body part reflection image to be captured. In one embodiment, the device 10 is only large enough to detect finger contact with its surface.

The device 10 includes a pixel array 11 formed by a micro-LED 12 or another light source such as a vertical-cavity surface emitting laser (VCSEL). These lasers are considered a subset of light emitting diodes. The LED 12 may be of different types to emit various intended peak wavelengths, or the LED 12 may be of the same type (eg, UV LED) with different phosphors to emit various intended wavelengths. May be. In the illustrated example, the LEDs 12 in a single pixel 11 include red (R), blue (B), green (G) and IR-emitting LEDs. Different wavelengths penetrate the skin to different degrees. Short wavelengths (blue, green) penetrate less into the skin than long wavelengths (red, infrared). In another embodiment, the device 10 is only for a specific function, such as detecting blood flow in a finger, and the LED 12 emits a single narrowband wavelength for a specific function, such as red or IR. Each pixel 11 also includes a broadband photodiode 14 or other type of photo detector. Since the LEDs 12 emitting different wavelengths can be activated individually, the output of the photodiode 14 can be related to the wavelength emitted by the activated LEDs 12. The resolution of the device 10 may be the same as the high-resolution display.

LEDs of any one peak wavelength in the pixel array are selectively illuminated sequentially and one or more detectors in the pixel array are read simultaneously to eliminate cross-talk, thereby eliminating the image obtained by digital processing. The resolution can be further improved. For example, this sequential illumination makes it possible to better distinguish the locations of absorbing features (eg, blood vessels) within a body part from locations that only scatter light and do not absorb light. Furthermore, only the detector(s) adjacent to the illuminated LED can be illuminated, so that the body part area being sampled can be highly localized.

The LED can be an OLED or an inorganic LED.

Alternatively, various color pixels can be formed by a liquid crystal display (LCD). Typical LCDs use white light backlights with a wide range of wavelengths. Color filters such as red, green, blue and IR filters are positioned behind the controllable liquid crystal layer forming subpixels. The liquid crystal layer effectively has an optical shutter for each sub-pixel position. By controlling the optical shutter, a different color of each pixel is controlled. The photodiode 14 may be formed in a transparent lamination layer above the LCD layer or below the LCD layer.

In either embodiment, the photodiode 14 and the light emitting pixel are on the same plane as the body part of the person being analyzed, so that the photodiode 14 detects the absorption of light by the human skin. Detect the image. This contrasts with known devices that detect the amount of light passing through a human body part. Since the localized absorption of the detected body part is basically determined by detecting the difference between the reflected light received by different photodiodes 14, the image of the reflected light corresponds to the image of the absorbed light. This difference can be used to map the absorption pattern and compare it to the stored absorption pattern. Thus, the absolute magnitude of the reflected light may change based on the body part being sampled, any ambient light and current to the LED, etc., but the difference in light reflection intensity will still correspond to the absorption pattern within the body part.

An example of the pixel array 11 for a combination fingerprint detector and a blood vessel position detector is 2 cm×2 cm, and may be an array including 800 pixels 11. The resolution can be as small as 0.25mm. This array size can be integrated into a smart phone or other portable device to authenticate the user using two different tests.

The LED controller 18 controls the activation of the LED 12 using row and column addressing, for example, and the photodiode 14 is controlled by the photo detector controller 20. The processor 22 provides full control of the controllers 18 and 20 and may include a variety of photodiodes 14 for further processing depending on the desired function. In one embodiment, the processor 22 controls the device to detect the fingerprint as well as the location of blood vessels in the finger, the pulse rate, and components in the blood. Multiple images can be obtained, such as a video, to analyze blood flow. The processor 22 may compare this data with data stored in the memory to authenticate the user or use the data for medical analysis.

In one embodiment, the LED controller 18 rasterizes or otherwise spatially sequence the illumination of the LED, while the photo detector controller 20 captures a sequence of images corresponding to a number of distinct illumination states. . The processor 22 can process the sequence of data to compensate for the captured image for the scattered light, thus increasing the resolution of the combined image formed by absorbing features in the body part that contacts the device.

The entire device 10, including the processor 22 and the controller 18/20, can be formed as a single modular device requiring only power and data reads.

Any of the pixels 11 or different sections of the pixel array may be individually activated to reduce cross-talk between the photodiode 14 and the LED 12. Ideally, the body part is in direct contact with the device 10 to minimize collision and scattering of reflected light to obtain the highest resolution image. To further improve the resolution, focusing optics and directional optics may be used.

FIG. 2 is a schematic cross-sectional view of the device 10 of FIG. 1, showing a person's finger 23 in direct contact with the protective glass cover 24 of the device. The blood vessel 25 of the finger 23 is also shown. Since the finger 23 is similar to the refractive index of the glass cover 24, but has a much higher refractive index (e.g., 1.5) than air, the fingerprint ridge 26 (in direct contact with the glass) and fingerprint Valley 27 (separated from the glass) will cause different light reflections 28 in that area. The valley region will reflect a relatively high proportion of light from the glass surface due to the TIR at the glass/air interface, while the ridge region will absorb a relatively high proportion of light. Due to the high resolution of the photodiode 14, some photodiodes 14 will be located below the valley 27 and others will be located below the ridge 26, whereby the image of the fingerprint is displayed on the processor 22 (Fig. It is detectable by 1) and may be associated with a fingerprint stored for user authentication.

The glass cover 24 may include inorganic glass, a crystalline material such as sapphire, glass ceramic, or polymer.

There are known ways to trick a fingerprint detector. To further protect against spoofing, the processor 22 also processes the photodiode 14 data to determine a pattern of low-wavelength (e.g., red, IR) absorption that matches the blood vessels of the finger. Gwang enters the finger at a certain distance through the fingerprint ridge. The shape and location of blood vessels related to fingerprints is unique to each individual and is very difficult to forge. This data is associated with the data stored for authentication. For example, the user performs initial reference detection, where a fingerprint pattern and a blood vessel pattern are detected and stored. Upon subsequent authentication, the user is authenticated if the new detection matches the reference pattern. Other blood data may be obtained for medical analysis. FIG. 2 shows the light reflectance 28 at the skin depth of the blood vessel 25, where the absorption of red/IR is higher at the location of the blood vessel than in the finger region that does not contain the blood vessel within the target depth. A map of blood vessels detected by the processor 22 may be created (with or separately from the fingerprint pattern) and compared with the stored reference blood vessel map.

The photodiode 14 signals can be subtracted from each other by the processor 22, so that the difference can be correlated with the absorption of the wavelength of interest. The difference (not absolute) in the signal output by the high-resolution photodiode 14 enables common-mode rejection of the light directly received by the photodiode 14 from the LED, resulting in a signal-to-noise ratio. Greatly increases

In general, by placing the array of photodiodes 14 over an extended area, the device 10 can be in contact with or in proximity to the glass cover 24 and act as an image capture device for items of appropriate contrast. To be. Thus, when body parts such as fingers, palms, wrists or faces are in contact with or in close proximity to the glass cover 24, the device 10 will be able to detect shallow subcutaneous biometric data, such as fingerprints, blood vessel patterns, and It can be used to capture color tone information, and beneficial biometric and biometric data, such as the presence of skin-resident pathogens.

Further, the pixel array can be simply used to detect the presence and movement of one or more fingers or other body parts in contact with or in proximity to the glass cover 24, whereby this array may be used to detect various gestures and Or it could be used as a touch screen or simple "button" without a dedicated touch detection array, mechanical button, or a separate gesture subsystem.

Further, by sequencing the LEDs 12 of various colors, spectroscopy of a body part is performed to obtain optically collected biometric data. When the wavelength penetrates the skin to some extent, subcutaneous biometric data may be obtained. For example, both red and green can be used to detect blood flow and blood vessels and to check the heart rate, and a combination of red and infrared at the appropriate wavelength can be used to check the supply of blood oxygen.

Each wavelength penetrates the skin to a different degree. Short wavelengths (blue, green) penetrate less into the skin than long wavelengths (red, infrared). Thus, by scanning across pixel wavelengths, images sampling various skin depths can be obtained. Thereafter, data from various images may be subtracted from each other to calculate an additional resolution. For example, a blue image capturing epidermal data may be subtracted from a red image capturing both epidermal data and subcutaneous data to represent only subcutaneous data.

More information can be collected by expanding the wavelength response range. By mixing or adding pixels that are reactive at higher or lower frequencies, additional functions such as pathogen detection, UV exposure, moisture and body chemistry reactions may be identified. In some cases, UV may be used to detect fluorescent components in blood or skin.

Furthermore, by capturing a series of images (i.e., video), the movement of the blood pulse along the blood vessel can be obtained, which is the rate at which the heart rate, blood pressure can be inferred, the electrocardiogram output, stroke volume, and The same biometric data is further provided either directly or by inference. The sequence of images also monitors blood flow, the body part actually being examined contains a living body part, and triggers biometric events, such as using a spoof material sample or a detached or deceased body part. It can be used to deduce that it does not include attempts to deceive. Likewise, the use of a combination of light detectors and LEDs that produce various colors can provide identification of substances applied to human fingers, for example, having a copy of the fingerprint but at the same time exhibiting physiological effectiveness. With this "video" feature, you can prevent fingerprint spoofing with conformal materials.

When capturing and processing an extended sequence of images, it may be necessary to compensate for the motion of the subject during video capture. In this case, for example, movement of the epidermal image captured with blue, green or other suitable light source can be used to track the movement of the subject during video capture, and compensate for the movement of the subcutaneous image captured with red or infrared light, for example. Can be used to

Given that the light penetrating the finger undergoes changes in backscatter and reflection based on the structure of the finger, fingerprints will be best detected from light that penetrates the finger least, such as blue light. However, fingerprints can be read in a wide range of visible and near-infrared wavelengths.

If you use long wavelengths, you can take subcutaneous images. For example, by using an illumination wavelength and power that can penetrate the skin by 2~3mm or more, light can penetrate the subcutaneous area where capillaries and veins can be found. Considering that the blood vessels coincide with the fingerprint ridges and valleys, the images collected from the blood vessels are likely to be affected by the presence of the fingerprint ridges and valleys. Thus, the presence of fingerprint ridges and valleys can enhance the specificity of layers used for recognition or can be removed from the image of blood vessels. In one example where it is desirable to remove the fingerprint, a technique of merely subtracting the image of the fingerprint captured by the short wavelength light as described above may be used. Alternatively, this may be done through image processing, such as by filtering or other techniques with similar effects. By setting the detector array so that the detector's focus is essentially solely or largely subcutaneously, such as, for example, including optics in the system (see figure below), the influence of the fingerprint can be minimized.

In addition, by detecting both the fingerprint image and the blood vessel image, since the fingerprint image can provide a reference for the blood vessel image, the orientation of the finger on the device can be a random direction. Accordingly, images relative to each other can be accurately compared with the stored relative images regardless of the position of the finger on the device.

As an artifact of this biometric authentication method, important physiological information can be simultaneously extracted. By studying the image of blood vessels that are captured under different illuminations, it is possible to perform a spectroscopic analysis of the tissue just below the detection area where the chemistry of blood, interstitial fluid or tissue can take place. For example, by studying images with two wavelengths (FIG. 3), such as 680 nm and 850 nm, where hemoglobin and oxygen-hemoglobin have a change in extinction coefficient with each other, SpO2, which is blood oxygen saturation, can be obtained. By studying the captured image under the appropriate illumination wavelength through expansion, blood sugar, red blood cell count, white blood cell count, blood CO2, blood sugar and other solutes in blood and interstitial fluid can be detected.

In all these configurations, the influence of ambient light may need to be considered. Specifically, when studying a thin body part such as a finger, ambient light may propagate through the finger to the detector array and interfere with the received signal or image. Therefore, it is necessary to consider the influence of ambient light. The ambient light may include a basic steady-state light source such as sunlight, or a modulated light source such as an incandescent lamp or modulated LED lighting. A portion of the detected optical signal due to ambient light can be quantified by first sampling the detector array with the array light source turned off. This signal is subtracted from the signal captured by the detector array when the array light source is in the "on" state, so that a signal related only to illumination by the array light source can be inferred. Furthermore, since all photodiodes 14 can detect the same amount of ambient light, the ambient light is canceled by subtracting the common-mode signal.

This interrelated double sampling method can be further improved by both the photo detector configuration and the option for multiple photo detectors of each pixel. This "ambient light rejection" can be facilitated by modulating the array light source and array detector sampling times at a much higher frequency than that of the ambient light source. The frequency of any ambient light modulation can be further detected by the detector array itself.

The above description describes an example of using a micro-LED display, but functionally the same modality is achieved through an extended detector array integrated into any other display of suitable resolution, such as an OLED display or an active matrix LCD display. can do. Specifically, when the display comprises an integrated semiconductor device such as amorphous silicon, polycrystalline silicon or organic semiconductor, the photodiode array can be formed on the display using substantially similar semiconductor processing, thus minimizing additional processing costs and Higher.

Functionally equivalent aspects may be achieved, for example, with individually expanded detector arrays in an optical module, chip on glass (a chip attached directly to the cover glass) or chip on display (a chip attached directly to the display glass). Such a module, chip on glass or chip on display may be directly integrated into the display or may include a standalone array; It may contain its own light source or may be arranged to use an external light source such as from the display or any other suitable existing pixelated, uniform, side, point or other light source.

The extended detector array may include an integrated sensor array such as a CMOS image sensor, and optics as shown in FIG. 4 to be described later may be integrated. The module cover glass itself may optionally be fully or partially coated with an optically strengthening layer, an optically filtering layer or an optically blocking layer, an antireflective coating or the like.

Such a module can act as an integration point of multiple human interfaces and a physiological improvement to the end application. By way of non-limiting example, features such as optical functions on/off or other "buttons" on consumer devices, features such as gesture recognition for advanced functions such as pinch, zoom, scroll, joystick, trackball, and signature can be implemented through algorithms or software. Can be added to existing work.

In order to provide additional mechanical button action, a pressure sensor or a pressure detection array may be augmented in an integral sensor module or a standalone sensor module. The stacked capacitive sensor array layers can also be integrated, from which the occurrence of a “touch” event and the force associated with the touch event can be determined. Determining the force of the touch event can also be used to feedback to the consumer that the size of the touch event has not been optimized for optimal biometric authentication or biometric data. For example, pressing the module too hard can limit blood flow to the capillaries and affect the signals received therefrom.

An integrated sensor module or a standalone sensor module may be bonded directly to the display or cover glass through an optically clear adhesive such as epoxy, silicone, acrylic or low-temperature molten glass (eg frit glass or compound semiconductor glass). In this case, the bonding may include an adhesive that extends over the entire interface of the module cover glass and the display glass. Alternatively, the adhesive can be dispersed only on a portion of the interface (eg, corners), and most of the module cover glass and display glass can simply be in contact or close.

4 shows another embodiment in which light sources 30, such as LEDs emitting the same or different wavelengths, are located only around the outer edge of the light guide 32, which serves as a protective cover. The light is connected to the edge portion of the light guide 32 by embedding the LED in the light guide 32 or by otherwise connecting the light to the edge. The light is transmitted to all areas of the light guide by TIR and exits only through the upper surface of the light guide 32.

When the body part directly contacts the light guide 32, since the skin has a refractive index similar to that of the light guide, light will be naturally extracted from the light guide at the contact point of the body part. When the body part to be detected is spaced from the light guide 32, the light guide 32 may have a light extraction feature such as a molded micro-reflector (prism) that directs the light upward and mixes the light. It is known to use such distributed micro-reflectors in light guides for backlights. Thus, light from an activated LED can be emitted substantially uniformly upwards into a body part in contact with or proximate the light guide surface. Little or no light is emitted downward by the light guide 32. Most of the light guide 32 is transparent, so that light reflected from the body portion passes through the light guide 32. This design using a light guide is relatively inexpensive compared to integrating an IR LED and a light detector with a display pixel LED into a display screen. In addition, the resolution of the photo detector array is independent of any display pixel array, and thus can be made with a higher resolution.

The optics 34 may include a focusing micro lens that focuses light reflected only from a specific distance toward the body part to the photodiode array 38. The optics 34 may limit the incident light from the body part to only a narrow angle perpendicular to the light guide to reduce cross-talk and better map the features of the body part to be analyzed. The photodiode array 38 may be a CMOS image sensor, a CCD image sensor, or any other image sensor.

The light blocking wall 40 may be used to block direct light from colliding with the photodiode array 38. This module may include an opaque thermally conductive vessel 42 to attenuate heat from the LED. The control electronics of FIG. 1 may be attached to or separated from the module. This module may have contact pads for soldering to the printed circuit board.

5 shows that the module of FIG. 4 is attached with a protective transparent cover 44, such as a transparent surface layer of a smart phone or other device.

As will be described in detail below, the IR emitter of FIG. 2 can be removed from the display pixel array and edge-connected to the display glass. The IR light is uniformly diffused within the display glass by TIR until it encounters skin tissue (eg, a finger) in direct contact with the display glass. At the point of contact, the light will be reflected/absorbed by the skin tissue and detected by the photodiodes 14 distributed within the display pixel array. Thus, the body part can touch any part of the display glass in any direction for detection.

As shown in FIG. 6, the integrated sensor module or standalone sensor module may include one or more electrodes 48, 50 extending past or around the light guide 32 or located outside the module but electrically connected to the module. It may further include, whereby the user can be electrically monitored. For example, these electrodes 49 and 50 may be used to detect a user's electrocardiogram (ECG or EKG) or a living body impedance. The ECG signal can be used separately to determine a heart rate, an impending medical condition, or an actual medical condition, determined by the shape of the ECG signal. Alternatively, the ECG can be used in conjunction with optically induced photovoltaic pulse waves (PPG) to determine blood pressure and other medically important indicators. Bioimpedance can be used to determine moisture, fat content, or other vital signs. Muscle activity can also be monitored electrically. All of these monitors can be combined into a single module.

The ECG signal can also be used as a biometric signature.

Thus, biometric authentication can be accomplished through a single aspect such as fingerprint, vascular imaging or ECG; Or through a combination of aspects; It can be performed through a form of multifactor authentication. Since errors may occur in the form of false positives and false negatives in all biometric authentication events, the accuracy of biometric authentication can be improved by using multifactor biometric authentication. For example, the multifactor biometric authentication scheme may be configured to confirm authentication events including positive authentication of two factors and two factors of third factor and rejected authentication, thus reducing security, but not While reducing the likelihood that the user will be locked-out due to negatives, it is possible to increase security against false positives and spoofing by requesting that at least two factors be authenticated. Alternatively, considering that different aspects have different authentication times, the multifactor authentication scheme provides a first fast low security authentication based on a single fast authentication factor, and collects additional slower authentication factors. It may be configured to provide one or more subsequent levels of increased security authentication over an extended period of time required to process.

As shown in FIG. 7, the module may be further reinforced with an integrated gas detection element 54. Instead, the gas detection element 54 may be a separate adjacent device. The use of highly specific, highly directional gas sensors, such as suitable electrochemical detection elements, can be used to detect current or impending medical conditions, just as dogs can smell human diseases. These data may be used separately, or may be used in conjunction with the optical and electrical data described above to provide a more complete context in which the data can be interpreted, thereby further (statistically) a medical diagnosis or a general health/health diagnosis. You can do it exactly.

Using a complementary gas detection element 54 may also be used in biometric authentication applications. The fingerprint spoof sample may include regenerating fingerprints on organic materials such as wood, adhesives, putty, acetate sheets, and the like. These materials emit predominantly volatile organic compounds, especially immediately after formation or curing. Furthermore, all humans emit volatile organic compounds (VOCs) through skin and sweat. Thus, a contact fingerprint or proximity fingerprint or other biometric authentication event, in which the gaseous environment at the point of contact is sampled substantially simultaneously with the biometric authentication event, has the opportunity to "smell" the presence of spoofing material or the skin directly. Furthermore, if sufficient resolving power exists for the gas detection element, a certain proportion of VOCs (that person's "odor") emitted by any one individual can be sufficiently distinguished and can be used as other biological identification elements or modalities. . This detection method is shown in FIG. 7 with a gas detection element 54 that detects the presence of an actual human finger 23 and a photodiode array 38 that detects light absorption by a fingerprint and/or blood vessel position. .

The integrated sensor module or the standalone sensor module is arranged as a small line scanner (single or narrow pixel line), for which a finger can be physically scanned or swiped. This may provide the advantage of being able to scan an enlarged portion of a finger or other body part while reducing the sensor footprint. In the case of a standalone sensor module, the footprint is reduced, so that space-limited platforms such as cell phones, watches and other wearables can be easily designed and module costs can be reduced. Through the swiping motion of a finger, a small form factor sensor can be used to investigate an extended range of body tissue. Being able to detect an extended range of body tissue provides the advantage that specific biometric and biometric markers can be sampled in a specific tissue area in which the biometric marker is the strongest or the most clearly defined. For example, in the case of obtaining a traditional fingerprint, the most useful information can be obtained by scanning the tissue on the phalange at the end because it is an area rich in epidermal characteristics (ridges and valleys). On the other hand, capturing unique subcutaneous information, such as finger vein identification data, can be most successful on the middle phalanx with large veins, so during biometric authentication events, it is easy to detect and the finger touches the sensor module to It is less impacted by the applied pressure. Thus, the optical biometric authentication line scanner can be configured to optimally capture fingerprint data from an end phalanx and finger vein identification data from an intermediate phalanx when a finger is scanned thereon. This module can also detect the passage of a finger on the module and change the behavior of the module during a finger scan, such as changing the illumination source or focal length when the sensor passes under one of the interphalangeal joints.

The line array sensor can also be formed on or behind a flat cover glass or curved cover glass, the curve of which closely matches the curve of the finger or wrist. Thereafter, this sensor may be integrated into a wearable device such as a ring or a watch, and in the case of networked, it may be used as a wearable device providing biometric authentication of a user. Such a device can perform a one-time biometric authentication event. After that, it is possible to check whether the wearable device has not been removed and the user is still alive by polling the sensor array continuously or periodically. Thereafter, the wearable device can confirm that the user maintains the biometric authentication without performing any subsequent biometric authentication event. Thereafter, the device can be wirelessly networked to facilitate fast user authentication, for example, a telephone, a credit card payment system, an ATM, a car, a door, a data vault, and the like. This aspect is particularly useful when the biometric authentication event includes a long-term event, such as when the biometric authentication event involves analysis of an ECG signal in which a number of heartbeats must be captured.

As shown in FIG. 8, the signal from the integrated sensor module or the standalone sensor module may be further augmented with data from an integrated or separate adjacent contact or non-contact temperature sensor 58. A low blue (red or IR) filter 60 may cover the sensor 58. The temperature data can be used as a standalone biometric or can be used to provide additional context in which other biometrics are interpreted.

9 shows an embodiment in which the control circuit and the photodiode are integrated into the sensor chip 62, and the focusing optics 64 are formed directly on the photodiode array for more precise focusing. This focusing may be within a certain distance within the body part in contact with the light guide 32. An LED light source 30 connected to the corner is also shown. The resulting module is so thin that it can be easily incorporated into a wide variety of applications.

10 is a plan view of the sensor chip 62 and optics of FIG. 9 according to an embodiment. A plenoptic lens may be superimposed on a group of the arranged photodiodes 14, and a micro lens 66 may cover each photodiode 14. The pitch between the photodiodes 14 is less than 0.25 mm, which can form a compact array that is small. In this way, the photodiode array 14 (detector pixel) under each plenoptic lens 68 produces one macro pixel with a resolution of 0.25 mm. Each micro-lens 66 on each photodiode 14 can enhance light capture and provide directionality. Each plenoptic lens 68 in this example may be disposed on about 16 photodiodes 14. The plenoptic lens 68 can be focused to various depths and oriented to collect more information about the reflected light from the body part. Various lenses may be formed by molding a transparent sheet and laminating the sheet on the photodiode array, or may be formed by directly molding on the photodiode array. A hemispherical lens 68 is shown for simplicity, but this lens can be of any shape suitable for focusing. A chip can be attached to the back side of any light guide glass. In another embodiment, the optics are spaced apart from the photodiode array by a gap. A reference photodiode 69 may be positioned outside the plenoptic lens 68 to detect ambient light. For miniaturization and improvement in signal-to-noise ratio, system logic 70 for processing the detected signal is formed on the same chip as the photodiode array. This logic may include analog-to-digital converters and digital processing circuitry.

11 shows the module of FIG. 10 to which the LED light source 72 is reinforced. As described above, the light source 72 may inject light into an upper light guide to diffuse light onto the photodiode array. The light reflected from the body part is directed to a photodiode array through a light guide. In another embodiment, a light guide is not required because the module may be small (eg, less than 2×2 cm), and light from the LED light source 72 is scattered within the body part and reflected back to the photodiode array.

Since LED performance degrades over time, photodiode 14 can be used to compare the LED light output against baseline and provide feedback to the LED activation circuit to achieve baseline performance.

Although the above-described embodiments include spectral analysis performed by an array of broadband detectors with a narrowband emitter, its function is to perform phosphorescence conversion of a wavelength specific detector, such as a detector having an optical filter thereon or in their optical path. It can also be implemented with broadband emitters such as phosphor-converted) white LEDs.

12 shows various possible applications of the modules described herein.

13 is a flowchart illustrating basic steps performed by various modules according to an embodiment of the present invention.

In step 80, an LED display panel, a light guide panel or other light source is disposed for the photo detector array to form a bio-sensor.

In step 82, the bio-sensor's output window is touched by the body part for analysis. The device may include a touch sensor or a light shadow sensor to detect when the body part is above the output window and start the image detection process.

In step 84, the LED is activated to apply a desired wavelength to the body part, and detect localized light absorption by the body part for fingerprint detection, blood vessel detection, pulse, and the like. The LEDs can be activated simultaneously or can be activated sequentially. When the LEDs are activated sequentially, the light detector can be selectively read out to better correlate the absorption of light with a location within the body part proximate to the activated LED. The sequential illumination can be done in any pattern.

In step 86, the signal from the photo detector array is scanned in columns and rows, etc. to detect the relative magnitude of light detected across the array, thereby effectively obtaining a detailed image of light absorption by the body part.

If the LEDs are activated sequentially, the method returns to step 84 after each activation to provide additional spatial absorption data.

In step 88, the raw data is processed by the processor of the module to obtain results such as user authentication, medical analysis, and the like.

Hereinafter, various other embodiments including an embodiment in which the photo detector and the light source are located outside the display area and do not affect the display portion itself are disclosed. Thus, the display portion may generally be conventional. The photo detector is covered with the same glass that covers the display screen. The light emitted for the biosensor function may be from a corner portion of the display or a light source spaced apart from the display portion.

Further, an embodiment in which transparent portions are dispersed in the display portion, and the light reflected by the body portion passes through and detected by the photo detector array below, will be described.

14-18 show an integrated display and an optical sensor, where the sensor is laterally spaced from the display portion but uses the same display glass or cover glass. The display area 90 includes an array of red, green and blue pixels, which are addressed to display any image. This device may be a smartphone display. The display area 90 is covered by a thin display glass 92 (or other type of light passing panel), which may include a touch screen sensor layer.

The active area of the display area 90 is surrounded by black ink 94 for aesthetic purposes. Ink is generally black, but can be any color.

A sensor area 96 is adjacent to the display area 90, which uses the same glass 92. When the black ink 94 is opaque to IR, the sensor area 96 is not covered with ink or is covered with IR ink 98 that passes the IR. The IR ink 98 may appear black.

As shown in FIG. 16, one or more photo detector elements 100, such as photodiode dies, are mounted under the IR ink 98 so as to detect light passing through the IR ink. An IR emitter 102, such as an IR LED, is activated to irradiate light onto the body part that contacts the glass 92 over the sensor area 96. There may be other wavelength LEDs, such as red, in the sensor area 96 depending on the biological characteristics to be detected. Assuming that the body part is a finger, IR light or red light is partially reflected from the surface of the finger, and the protrusion of the finger, especially the fingerprint, enters the part that contacts the glass 92. As described above, some of the light is absorbed by the blood and blood vessels or by specific components in the blood, and the pattern of absorption is detected.

In other embodiments, the light emitted by the sensor area 96 may be white light, and the detector element 100 may be from a body part in front of the sensor area 96 or from any object including a printed code. The image of reflection of white light is detected.

By separating the detector element 100 and the IR emitter 102 laterally from the display area 90, the display area 90 is not affected by the sensor area 96. Accordingly, the display area 90 may have a pixel array of very high resolution.

Optics can be integrated into the optical path of the detector. For example, the optics can be molded or mounted on the detector die. Alternatively, the optics may be molded or mounted on the display glass 92 or IR ink 98. Optics may also optionally be etched into display glass 92. Optical filters, such as organic dyes, dielectric filters or metallic dielectric filters, may also be placed directly on the photo detector dice, display glass 92, or any other suitable element in the optical path. The detector may comprise an array of pixels or an array of focal planes.

16 shows the detector element 100 in more detail. The reflected light 104 passes through the IR ink 98 and is detected by the detector element 100, and the detector element 100 is a single element for generating a high-resolution image of reflection/absorption of light by body parts such as fingers. It may be a detector element or an array of detector elements.

In another embodiment, the LED in the display area 90 supplies light, which light is reflected from the body part because there is some scattering of light within the body part reaching the detector element 100.

The advantage of the device of FIGS. 14 to 18 is that the sensor area 96 does not affect the configuration or operation of the display area 90, which may be a smartphone display.

The sensor area 96 may be used to detect a user's fingerprint and/or blood vessel pattern for authentication.

As shown in Figure 16, the electrical interconnection to the detector element 100 is the conductive traces 106, 108 deposited on the IR ink 98 and suitable interconnects 110, 112 (e.g., conductive Adhesive, solder, etc.). Alternatively, conductive traces 106/108 may be deposited directly on glass 92, and patterned IR ink 98 may be deposited thereon. The conductive traces 106/108 may include, for example, a conductive oxide, a conductive organic material, a semiconductor, a metal, or a mixture thereof. The interconnects 110/112 may comprise, for example, solder, metal, conductive paste, conductive film or anisotropically conductive element. The detector element 100 may comprise a stand-alone light receiver or a two-dimensional or linear array thereof, and one or more light receiver elements, such as a photodiode with an analog electronic circuit and an optional analog-to-digital converter, digital control logic, digital -It may include elements incorporating analog circuitry and processing circuitry. The detector element 100 may comprise a CMOS image sensor or similar device.

In FIG. 16, the back cover 113 is also shown.

As shown in Fig. 17, the electrical connection to the detector element 100 is, for example, a flexible PCB (flex circuit), a PCB, a ribbon connector, a wiring assembly, or any other connected to the back side of the detector element 100. It can be made independently of the glass 92 by means of a suitable element. Metal traces 114 are shown in this example. Electrical connection to the back side of the detector element 100 may also be made through a spring connector or other similar interconnection scheme. In this manner, the detector element 100 may include a technology such as that used in a back-side illuminated (BSI) CMOS image sensor.

One or more light emitting devices such as LEDs or VCSELs may also be mounted on the glass 92. The light emitting device and detector element 100 can be mounted on the glass 92 in a similar manner. Optics may be incorporated in the optical path of the light emitting device. For example, the optics may be molded or mounted on a light emitter. Alternatively, the optics may be molded or mounted on the display glass 92 or IR ink 98. Optics can also optionally be etched into display glass 92. Optical filters such as organic dyes, dielectric filters or metallic dielectric filters may also be placed directly on the photo detector dice, display glass, or any other suitable element in the optical path. In this case, light is emitted from the light emitting device through IR ink 98 and is incident on a partial reflector (such as a user's face, hand or finger) on or in contact with the display glass 92. As shown in FIG. 18, the light 117 emitted from the IR emitter 102 is reflected from the body part 116 (beam 118) and is detected by the detector element 100. Alternatively, the display itself can be used as a light emitter, and the detector element 100 can simply detect the reflected light from the display.

The above example includes the detector element 100 positioned behind the IR ink 98, but the detector element 100 may be positioned on the glass 92 when there is no ink.

In the above example where the emitted light should be part of the detection scheme, it can help to differentiate the reflected signal from the background light through double correlation sampling at the point of turning the emitter on and off.

19 and 20 illustrate a partially transparent display 119, such as a partially-transparent OLED display, wherein one or more photo detector elements 100 on the back side of the display 119, such as on the back glass plate 120 Is mounted. The display 119 includes an array of intentionally transparent regions 121 ('transparent pixels'), shown in Fig. 20, the purpose of which is to provide an optical path through the display 119 to make it partially transparent. . In this example, the detector element 100 may be an array of photodiodes or other photo detectors mounted behind the partially-transparent display 119, which means that the photosensitive element in the detector array is a transparent area of the partially transparent display 119 In a manner that lies directly below 121, it is selectively aligned in the pixel array of the partially transparent display 119. Light reflected from the body part 16 may be emitted from the display pixel 124 or the dedicated light emitter 102. In FIG. 19, light rays 125 are shown as being emitted by a dedicated light emitter 102, and in FIG. 20 rays 126 emitted by red pixels 124 are shown. Reflected light 127 is shown to be detected by detector element 100.

Optics may be incorporated into the optical path of the detector element 100. For example, the optics may be molded to the detector die or mounted to the detector die. Alternatively, the optics may be molded or mounted on the back glass plate 120, or may be etched or patterned thereon. Optical filters such as organic dyes, dielectric filters or metallic dielectric filters may be placed directly on the photo detector dynamo, display glass or other suitable element of the optical path. The detector element 100 may include a pixel array or a focal plane array.

Any display device may be used as the light source, and light reflected from the body part may be detected by the detector device 100. Red, green and blue LEDs 124 (indicated by RGB) are shown in each display pixel area adjacent to the transparent area 121. The display pixel array itself, typically comprising red, green and blue array pixels, can optionally be enhanced to include infrared emitting pixels, ultraviolet emitting pixels or any other suitable wavelength emitting pixels. In this way, OLED materials that emit in infrared, ultraviolet or other suitable wavelength ranges can be used. In the case of micro-LED displays, for example infrared LEDs and ultraviolet LEDs formed of each of Al _x In _y Ga _z P and Al _x Ga _y N can be integrated as part of a red, green and blue LED pixel array, and these same Other suitable wavelengths may be produced by materials or other compound semiconductor alloys.

Alternatively, one or more individual optical light emitters 102 may be mounted behind a partially transparent display 119. Light from these light emitters 102 can be transmitted through a partially transparent display 119, reflected from an external body part 116, and detected by the detector element 100. Optics may be incorporated into the optical path of the light emitter 102. For example, optics may be molded or mounted on a light emitter die. Alternatively, the optics may be molded or mounted on the back glass plate 120, or may be etched or patterned thereon. Optical filters such as organic dyes, dielectric filters or metallic dielectric filters may be placed directly on the photo detector die, display glass, or other suitable element in the optical path.

Alternatively, similar to the design of FIG. 14, separate light emitters may be mounted along the side of the display. These separate light emitters may include infrared emitters, ultraviolet emitters, or emitters of any suitable wavelength, such as, for example, LEDs or LEDs converted to phosphors. Alternatively, a separate light emitter in the final product, such as a camera flash, an indicator LED, a light source integrated in another optical communication link, or any other light source, can be used as the illumination source, whereby a signal reflected from a nearby body can be detected. . Further, as described above, the light source may be edge-connected to the display glass, and the display glass internally diffuses light by TIR until it leaves the contact point of the body part and the display glass.

21 shows a single pixel in a display 130, such as an AMOLED display, which shows a thin film transistor (TFT) array 134 formed on low-temperature polycrystalline silicon (LPTS), amorphous silicon (a-Si), or crystalline silicon. It further includes a semiconductor backplane 132 including. In this embodiment, one or more photosensitive detection elements 100 are formed on the semiconductor backplane 132 prior to deposition of the surface organic electroluminescent layer. The detector element 100 may include a photodiode such as a p-i-n photodiode, for example. The OLED material is grown or deposited over the TFT array 134 to form the red, green and blue sub-pixels 135. Infrared sub-pixels can also be formed as part of an array. Optionally, an optic 136 may be formed or disposed on the detector device 100. Such optics may include, for example, lenses formed by a photoresist reflow process or a printing process. One or more detector elements 100 may be used as a photo detector or detector array integrated directly into the display. Ray 138 is also shown.

An optical filter such as an organic dye, a dielectric filter or a metal-dielectric filter can be selectively integrated over the detector element 100 to create a pixel with sensitivity to a specific wavelength. For example, red, green, blue, cyan, magenta, yellow, IR and UV absorbing dyes may be selectively disposed on different detector elements, providing a multi-pixel detector capable of color detection or spectroscopy. The OLED material itself can optionally be disposed on the photodiode, and the red, green and blue OLED materials all have different absorption spectra and thus act as separate optical filters.

Figure 22 shows another embodiment of an integrated display and detector, and Figure 23 shows a single pixel area of the display/detector of Figure 22.

The full color display 140 includes a backplane 142, such as an LPTS, amorphous silicon (a-Si), or a TFT-LCD comprising an array of thin film transistors (TFTs) formed on crystalline silicon. The transistor may be patterned and doped amorphous silicon deposited on the transparent glass substrate 146 (FIG. 23 ), and the patterned conductor may be transparent, such as ITO. On the glass substrate 146, an array of thin film transistors 147 is formed with the LCD layer 148, and the transistor 144 is turned on by a conventional column and row decoder to control each LCD pixel, one of which is It is shown in Figure 23.

Color filters 150 such as blue, red, and green filters are formed above or below the LCD layer 148 to form blue, red, and green sub-pixels. Infrared sub-pixels can also be formed as part of a pixel. A black mask 152 is printed on the upper display glass 154 to more optically separate the sub-pixels.

The display portion of the backplane 142 may be conventional.

The white backlight 156 is located under the backplane 142, and the LCD sub-pixels act like a controllable shutter to emit a selected amount of blue, red, and green light for each pixel to create a wide color gamut.

In general, photolithography is used to form the backplane 142, and very high resolution can be obtained.

To augment this display with a detector, one or more photo detector elements 158, such as amorphous silicon photodiodes, are also deposited and patterned on the backplane 142 by photolithography. The photodiode may be a variation of the silicon transistor 147 formed in a TFT array and formed using an additional masking step. The TFT array and the photodiode can be formed on the same plane on the glass substrate 146. A suitable conductor on substrate 146 connects the photodiode to the detector circuit. The photodiodes may be outside the display area or may be distributed throughout the display area. The photodetector element 158 may comprise a photodiode, such as a p-i-n photodiode, for example.

22, the IR emitter 159 or other peak wavelength LED is optically connected to the edge of the display glass 154, where the display glass 154 diffuses the IR light 164, so that the IR light It is used as a waveguide that emits only on the photo detector element 158 so as to move 164 away from the backplane 142. This light is extracted from the display glass 154 only in the area where the body part 116 contacts the display glass 154 as described above. If the body part does not need to contact the display glass, the surface of the display glass 154 may be roughened over the photo detector element 158 to extract the guided light. The display glass can be clear on the display portion of the pixel.

The IR filter 162 may also be printed on the lower surface of the display glass 154 and may be the same side as the RGB color filter 150.

23 shows a backlight 156 providing light 168 for the detector. However, the detector light may be coupled to the edge of the display glass 154 as shown in FIG. 22.

Optionally, optics may be formed or disposed on the photo detector element 158. Such optics may include, for example, lenses formed by a photoresist reflow process or a printing process. These one or more photo detector elements can be used as a photo detector or detector array integrated directly into the display. In this example, a light-shielding layer may be incorporated between the growth substrate and the photo detector element to block the backlight from directly illuminating the photo detector element.

Optionally, an optical filter such as an organic dye, a dielectric filter or a metal-dielectric filter may be incorporated into the detector pixel to create a pixel with sensitivity to a specific wavelength. For example, red, green, blue, cyan, magenta, yellow, IR and UV absorbing dyes may be selectively disposed over different detector pixels, providing a multi-pixel detector capable of color detection or spectroscopy. Optionally, a standard LCD color filter material itself may be disposed on the photodiode.

In some embodiments, the detector pixels may be arranged directly under the LCD element so that the LCD matrix may be used to subtract the detector pixels, which facilitates capture of the image by the array.

In FIG. 22, the display portion operates normally to display any image or command, and the infrared LED 159 of FIG. 22 is activated to cause the display glass 154 to emit IR light in the area of the photo detector element 158. do. The emitted light is reflected and absorbed by the body part 116 on or in contact with the display glass 154 in proximity to the photo detector element 158, and the signal from the photo detector element 158 is transmitted by the detector circuit. It is processed to identify features of body parts 116 such as fingerprints, blood vessels, facial recognition, combinations thereof, and the like. The photo detector element 158 can include any number of pixels depending on the desired resolution. Therefore, all light emission for detection is generated by a side light source such as the IR LED 159, so that the display pixel area is not occupied by the IR light emitter.

Since the refractive indexes of the glass 154 and the body tissue are similar, light is transmitted into the tissue at the point where the glass contacts the body tissue. The light is then attenuated and scattered back from the tissue to the detector so that it can be detected from the detector. The biometric signature or biometric data can be confirmed from the detected light.

Wavelengths other than IR can be used to detect other biological properties, especially facial recognition.

In one embodiment, the display portion may be high resolution for displaying conventional images, and the detector portion may include low resolution visible light pixels to identify the location of the detection portion under the display glass. For example, a luminescent pixel (eg, RGB or white) in the detector can provide an outline of the detector area or identify simple instructions for the user. In this embodiment, the pixels in the display portion operate independently of the light emitting pixels in the detection area. The luminescent pixels in the detector area can also serve as illumination pixels for the body part to be detected.

In all embodiments, the detection circuit and control circuit of Fig. 1 can be incorporated into the device.

While specific embodiments of the present invention have been shown and described, those skilled in the art will understand that changes and modifications may be made without departing from the scope of the present invention, and the appended claims therefore fall within the true spirit and scope of the present invention. It includes all changes and modifications.

Claims (33)

As a biosensor device,
A display portion comprising a two-dimensional pixel array configured to display a full color image and comprising light emitting diodes emitting a first set of peak wavelengths, having a light-passing panel,
A two-dimensional array of light sources distributed within the pixel array and configured to emit light through the light passing panel to be partially reflected and absorbed by blood in a person's finger placed on the display portion, the light source comprising the pixel and Emits light with a peak wavelength that is coplanar and is not at least within the peak wavelength of the first set-and,
A two-dimensional array of photo detectors distributed within the pixel array and arranged to detect light reflected by the surface of the finger and the blood within the finger, the photo detector receiving a first signal corresponding to the reflected light And the photodetector is coplanar with the pixel-with,
A controller connected to the display portion and the array of light sources-the controller controls the pixels in the display portion during a first period to emit light of one or more peak wavelengths to detect the fingerprint of the finger and a first image of the fingerprint And the array of light sources does not emit light during the first period, and the controller controls the array of light sources during a second period to detect the blood in the finger to obtain a second image of the blood. Configured to capture-wow,
Connected to the photodetector and analyzing the first signal to detect the fingerprint using only the pixels that are activated within the display portion during the first period, and at least the array of light sources during the second period A detection circuit configured to detect the blood in the finger using light emitted by
Device comprising a.
The method of claim 1,
The pixels generate red, green and blue light,
The light source generates infrared light
Device.
The method of claim 1,
Each pixel of the display portion includes a sub-pixel emitting light of a different color,
The light source includes at least one of the sub-pixels
Device.
The method of claim 3,
The sub-pixels include blue sub-pixels, green sub-pixels, and red sub-pixels,
The light source includes at least some of the red sub-pixels
Device.
The method of claim 1,
The light source is laterally spaced from the pixel of the display portion
Device.
The method of claim 1,
The light source includes one or more light emitting diodes
Device.
The method of claim 1,
The light source includes light emitting diodes emitting light of different peak wavelengths.
Device.
The method of claim 1,
The light source emits light of a wavelength absorbed by blood vessels.
Device.
delete delete delete The method of claim 1,
Further comprising a lens over the photo detector to limit the angular range of light reflected from the finger
Device.
The method of claim 1,
Further comprising a lens on the photodetector to focus only light reflected from a predetermined distance into the finger onto the photodetector.
Device.
The method of claim 1,
Non-optical sensor integrated in the device
The device further comprising.
The method of claim 14,
The non-optical sensor includes an electrode contacted with the finger
Device.
delete The method of claim 1,
The device is configured to detect blood vessels in the finger
Device.
delete The method of claim 1,
The controller comprises a processor integrated in the device,
The processor controls activation of the light source and detection of the reflected light received by the photo detector.
Device.
The method of claim 1,
The controller includes a processor configured to process data from the photo detector, compare the data to stored data, and indicate authentication of a user of the device.
Device.
The method of claim 1,
The controller includes a processor,
The light source includes a plurality of light emitting diodes,
The processor is configured to control the light emitting diodes to be sequentially illuminated,
The signal from the photo detector is read and processed by the processor.
Device.
The method of claim 1,
The controller includes a processor configured to detect the light reflection image of the finger to determine an absorption area of the light of the finger.
Device.
The method of claim 1,
The controller comprises a processor configured to process data from the photo detector,
The processor detects the movement of the finger to detect a gesture
Device.
As a biosensor device,
A display portion comprising a two-dimensional pixel array configured to display a full color image and comprising light emitting diodes emitting a first set of peak wavelengths, having a light-passing panel,
A light source optically connected adjacent to the edge of the light-passing panel-The light-passing panel guides light from the light source, and serves as a light guide for emitting light from the light source through the light-emitting surface of the light-passing panel. And the light source is different from the pixel array in the display portion used to generate an image-and,
A two-dimensional array of photo detectors arranged to detect light reflected by a finger of a person in contact with the light-passing panel-the light from the light source is in an area where the finger contacts the light-passing panel on the display portion. Extracted from the light passing panel, and the photo detector generates a first signal corresponding to the reflected light-and,
A controller connected to the display portion and the light source-the controller controls the pixels in the display portion during a first period to emit light of one or more peak wavelengths to detect the fingerprint of the finger and capture a first image of the fingerprint Wherein the light source does not emit light during the first period, and the controller is configured to control the light source during a second period to detect blood in the finger and capture a second image of the blood-and,
It is connected to one or more of the photo detectors and analyzes the first signal to detect the fingerprint using only the pixels activated within the display portion during the first period, and at least the light source during the second period A detection circuit configured to detect the blood in the finger using light emitted by
Device comprising a.
delete delete The method of claim 24,
The detection circuit detects a fingerprint at a position where a crest of the fingerprint contacts the light-passing panel, and extracts light from the light-passing panel.
Device.
The method of claim 27,
The detection circuit also detects the blood vessel pattern of the finger
Device.
The method of claim 28,
The detection circuit detects a combined pattern of a fingerprint of the finger and a blood vessel
Device. The method of claim 1,
The controller controls one or more pixels of the display portion to emit light during the second period, while controlling the array of light sources during the second period to detect the blood in the finger and Configured to capture the second image
Device.
The method of claim 1,
The controller controls at least one of a red light emitting pixel and a green light emitting pixel of the display portion to emit light during the second period, and controls the array of light sources during the second period to detect the blood in the finger And configured to capture the second image of the blood in the finger.
Device.
The method of claim 1,
The detection circuit is configured to generate a difference between the first image and the second image to obtain a resultant image at a desired depth into the finger.
Device.
The method of claim 1,
The detection circuit subtracts the first signal generated by some of the photo detectors from the first signal generated by another of the photo detectors, and the difference is related to absorption of the wavelength of interest.
Device.

Images