+

WO2008017077A2 - Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée - Google Patents

Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée Download PDF

Info

Publication number
WO2008017077A2
WO2008017077A2 PCT/US2007/075221 US2007075221W WO2008017077A2 WO 2008017077 A2 WO2008017077 A2 WO 2008017077A2 US 2007075221 W US2007075221 W US 2007075221W WO 2008017077 A2 WO2008017077 A2 WO 2008017077A2
Authority
WO
WIPO (PCT)
Prior art keywords
optical waveguide
light
touch sensing
imaging
sensing display
Prior art date
Application number
PCT/US2007/075221
Other languages
English (en)
Other versions
WO2008017077A4 (fr
WO2008017077A3 (fr
Inventor
Jefferson Y. Han
Original Assignee
Perceptive Pixel, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Perceptive Pixel, Inc. filed Critical Perceptive Pixel, Inc.
Priority to EP07813780A priority Critical patent/EP2047308A4/fr
Priority to JP2009523076A priority patent/JP2009545828A/ja
Publication of WO2008017077A2 publication Critical patent/WO2008017077A2/fr
Publication of WO2008017077A3 publication Critical patent/WO2008017077A3/fr
Publication of WO2008017077A4 publication Critical patent/WO2008017077A4/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
    • G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
    • G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
    • G02B6/122—Basic optical elements, e.g. light-guiding paths
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
    • G02B6/24—Coupling light guides
    • G02B6/26—Optical coupling means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0412—Digitisers structurally integrated in a display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/04166—Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04104—Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04109—FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location

Definitions

  • the present invention relates to techniques for enabling high- resolution multi-touch sensing displays based on frustrated total internal reflection.
  • the Fingerworks iGesturePad is another example of a device that employs multiple discrete sensors in a matrix configuration with active transistors at each node.
  • U.S. Patent 6,323,846 to Westerman et al. discloses additional examples of using an array of proximity sensors in a multi-touch surface system.
  • Multi-touch sensing may be achieved by carefully employing a purely passive matrix of force-sensitive-resistors (FSRs), as discussed in Hillis, W. D., "A High Resolution Imaging Touch Sensor," International Journal of Robotics Research, pages 1, 2, 33-44 (1982), incorporated herein by reference.
  • FSRs force-sensitive-resistors
  • Patent 4,134,063 to Nicol et al. discloses the use of capacitive electrodes for this purpose.
  • SmartSkin An Infrastructure for Freehand Manipulation on Interactive Surfaces
  • Such systems while less complex than systems that employ multiple active sensors, still entail numerous electrical connections and thus disadvantageous ⁇ limit their application to uses that require relatively low resolution (e.g., under 100 x 100).
  • Such systems are visually opaque and thus require the use of top- projection if to be integrated with a graphic display.
  • such systems have had problems with robustness given the feeble nature of the electrical signals they utilize.
  • FTIR frustrated total internal reflection
  • U.S. Patent 4,346,376 to Mallos discloses a CRT-based touch sensor, which replaced the bulky prism with a thin platen waveguide and operates by detecting the light scattered away by an object in optical contact. More recent fingerprint sensors use this approach, as disclosed in Fujieda, L, Haga, H., "Fingerprint Input based on Scattered-Light Detection,” Applied Optics-IP, 36, 35, 9152-9156 (1997), incorporated herein by reference.
  • a multi-touch sensing display comprises an optical waveguide adapted (i.e., designed) to receive light and adapted to cause some of the received light to undergo total internal reflection within the optical waveguide, the optical waveguide adapted to allow total internal reflection to be frustrated upon occurrence of a physical phenomena and adapted to allow some of the received light to escape when total internal reflection is frustrated, and an imaging camera adapted to detect light escaping the optical waveguide.
  • the senor includes infrared light emitting diodes disposed directly against the edge of the optical waveguide.
  • the optical waveguide is a sheet of acrylic with edges treated to admit light.
  • the physical phenomena entails contacting the optical waveguide with an object so that total internal reflection is frustrated upon such contact.
  • the object is a finger of a user of the multi-touch sensing display.
  • the optical waveguide is adapted to allow total internal reflection to be frustrated simultaneously at multiple positions upon occurrence of multiple physical phenomenon, and some of those positions are disposed apart from one another.
  • the physical phenomenon include at least contacting a first location of a surface of the optical waveguide with a first object and contacting a second location of the surface of the optical waveguide with a second object, the first and second locations being disposed apart from one another, and the optical waveguide is adapted to cause total internal reflection to be frustrated at corresponding first and second positions.
  • the first and second objects are first and second fingers of a user of the multi-touch sensing display.
  • an imaging sensor is adapted to discriminately detect light output from positions along a two-dimensional surface of the optical waveguide.
  • the imaging camera is adapted to discriminately detect light simultaneously escaping from multiple positions along a two-dimensional surface of the optical waveguide.
  • the senor includes a light source adapted to emit light of a predetermined wavelength that is received by the optical waveguide, and the imaging camera is adapted to detect light only substantially at the predetermined wavelength.
  • the senor includes a light source adapted to emit light of first and second predetermined wavelengths that are received by the optical waveguide, and the imaging camera is adapted to detect light only substantially at the first predetermined wavelength, and the multi-touch sensing display further includes a second imaging camera adapted to detect light only substantially at the second predetermined wavelength escaping the optical waveguide.
  • the senor includes a light source adapted to emit pulses of light at a rate substantially synchronized to the shutter of the imaging camera.
  • the optical waveguide has a non- planar surface, and the physical phenomena comprises contacting the non-planar surface of the optical waveguide so that total internal reflection is frustrated upon such contact.
  • the senor includes a video projector adapted to project a two-dimensional video image onto the optical waveguide.
  • the multi-touch sensing display includes a diffuser onto which the video image is projected.
  • the optical waveguide includes non-contact and contact surfaces
  • the diffuser is disposed on the non-contact surface and the physical phenomena comprises contacting the contact surface of the optical waveguide so that total internal reflection is frustrated upon such contact.
  • the video projector is disposed on a non-contact side of the optical waveguide corresponding to a side on which the non- contact surface is disposed, and the video projector is adapted to project the video image toward the optical waveguide for viewing by a user disposed on a contact side of the optical waveguide.
  • the senor includes a video projector adapted to project a two-dimensional video image and a diffuser disposed adjacent to a surface of the optical waveguide, and the video image is projected onto the diffuser.
  • the diffuser is an electro- switchable diffuser screen synchronized to a rate of projection of the video projector, and is made non-diffusive according to the shutter periods of the imaging sensor.
  • the diffuser is a directional diffuser (e.g. holographic, Lumisty, etc.) that diffuses for certain angles of incidence, and is non-diffuse for others.
  • a directional diffuser e.g. holographic, Lumisty, etc.
  • additional cameras are used to view through the diffuser when it is made non-diffusive to observe the touching object.
  • the imaging camera includes multiple imaging cameras adapted to detect light escaping from respectively different portions of the optical waveguide.
  • the senor includes multiple video projectors and each video projector is adapted to project a respective two-dimensional video image onto a respective one of the different portions of the optical waveguide.
  • the senor includes an LCD display panel disposed between the imaging camera and the optical waveguide, and the LCD display panel is transparent to light escaping the optical waveguide so as to not prevent the escaping light from being detected by the imaging camera.
  • a multi-touch sensing display comprises an optical waveguide adapted to receive light and adapted to cause some of the received light to undergo total internal reflection within the optical waveguide, a compliant surface overlay disposed sufficiently close to the optical waveguide to enable depression of the compliant surface overlay by an external force to cause the compliant surface overlay to contact the optical waveguide, the optical waveguide and the compliant surface overlay adapted to cause total internal reflection within the optical waveguide to be frustrated upon contact of the optical waveguide by the compliant surface overlay, the optical waveguide adapted to allow some of the light undergoing total internal reflection to escape when total internal reflection is frustrated, and an imaging camera adapted to detect light escaping the optical waveguide.
  • a small gap is disposed between a substantial portion of the compliant surface overlay and the optical waveguide so that frustration of total internal reflection is minimized during non-depression of the compliant surface overlay.
  • the compliant surface overlay is adapted to be depressed by a finger of a user of the multi-touch sensing display.
  • the compliant surface overlay is adapted to be depressed by passive styluses, gloved hands, and arbitrary objects.
  • the compliant surface overlay is adapted to contact the optical waveguide at multiple locations simultaneously in response to simultaneous depression of the compliant surface overlay at multiple corresponding locations, while maintaining a gap in locations that are not depressed, and the optical waveguide is adapted to cause total internal reflection to be frustrated simultaneously within the optical waveguide at those locations.
  • At least two of the depressed locations are disposed apart from one another.
  • the imaging camera is adapted to discriminately detect light output from positions along a two-dimensional surface of the optical waveguide.
  • the imaging camera is adapted to discriminately detect light simultaneously escaping from multiple positions along a two-dimensional surface of the optical waveguide.
  • the senor further includes a light source adapted to emit light of a predetermined wavelength that is received by the optical waveguide, and the imaging camera is adapted to detect light only substantially at the predetermined wavelength.
  • the light source emits light of first and second predetermined wavelengths
  • the imaging camera is adapted to detect light only substantially at the first predetermined wavelength
  • the multi-touch sensing display further includes a second imaging camera adapted to detect light only substantially at the second predetermined wavelength escaping the optical waveguide.
  • the light source emits pulses of light at a rate substantially synchronized to the shutter of the imaging sensor.
  • the compliant surface overlay includes a non- planar contact surface.
  • the senor includes a video projector adapted to project a two-dimensional video image onto the optical waveguide.
  • the multi-touch sensing display includes a diffuser onto which the video image is projected.
  • the diffuser is an electro-switchable diffuser screen synchronized to a rate of projection of the video projector, and is made non-diffusive according to the shutter periods of the imaging sensor.
  • the diffuser is a directional diffuser (e.g. holographic, Lumisty, etc.) that diffuses for certain angles of incidence, and is non- diffuse for others.
  • additional cameras are used to view through the diffuser when it is made non-diffusive to observe the touching object.
  • the compliant surface overlay is adapted to diffuse the video image projected by the video projector.
  • the compliant surface overlay includes a component that rejects ambient light of the same wavelengths as the light source, a component that reduces friction at the interaction surface, a component that reduces glare, and a component that provides a comfortable cushion for the user to depress.
  • the senor includes a video projector adapted to project a two-dimensional video image through the optical waveguide onto the compliant surface overlay, and the compliant surface overlay has a thin layer of rubber adapted to diffuse the video image projected by the video projector.
  • a method of multi-touch sensing comprises the steps of receiving light within an optical waveguide, internally reflecting the received light within the optical waveguide, frustrating the internally reflected light within the optical waveguide to cause some of the reflected light to escape the optical waveguide, and imaging the escaped light.
  • the light is emitted by a light source having multiple infrared light emitting diodes.
  • the method includes providing a sheet of acrylic with edges treated to admit light as the optical waveguide.
  • the optical waveguide is contacted with an object to frustrate the internally reflected light.
  • one or more fingers of a user contact the optical waveguide to frustrate the internally reflected light.
  • the internally reflected light is frustrated simultaneously at multiple positions within the optical waveguide, and some of those positions are disposed apart from one another.
  • first and second locations of the optical waveguide disposed apart from one another are simultaneously contacted to cause reflected light to escape simultaneously from corresponding positions of the optical waveguide.
  • the light escaping the optical waveguide is imaged discriminately along a two-dimensional surface.
  • light of a predetermined wavelength is received within the optical waveguide, and light only substantially at the predetermined wavelength escaping the optical waveguide is imaged.
  • light of two different wavelengths is received within the optical waveguide, and light only substantially at those wavelengths escaping the optical waveguide is imaged.
  • two different cameras image light at the two different wavelengths.
  • pulses of light are received within the optical waveguide, and an imaging camera having a shutter synchronized to the rate of the pulses detects the escaped light.
  • an optical waveguide having a non-planar surface is provided.
  • the method includes projecting a two- dimensional video image onto the optical waveguide.
  • a diffuser is provided to diffuse the projected video image.
  • a two-dimensional video image is projected on and diffused by a diffuser disposed on a non-contact surface of the optical waveguide, and a contact surface of the optical waveguide is contacted to cause the internally reflected light to be frustrated.
  • the video image is projected from a non- contact side of the optical waveguide.
  • the projected video image is diffused adjacent to the optical waveguide.
  • a small gap between a substantial portion of the diffuser and the optical waveguide is provided so that frustration of the internally reflected light by the diffuser is minimized.
  • an electro-switchable diffuser screen synchronized to a rate of projection of the video image is provided, the electro- switchable diffuser screen being non-diffusive according to the shutter periods of a sensor imaging the escaped light.
  • the provided electro-switchable diffuser screen is a directional (e.g. holographic, Lumisty, etc.) that diffuses for certain angles of incidence, and is non-diffuse for others [0087]
  • additional cameras are provided to view through the diffuser when the diffuser is made non-diffusive to observe the touching object.
  • multiple imaging cameras image light escaping from respectively different portions of the optical waveguide.
  • a video image is displayed through the optical waveguide by an LCD display panel that allows light escaping the optical waveguide to pass through it.
  • a method of multi-touch sensing comprises the steps of receiving light within an optical waveguide, internally reflecting the received light within the optical waveguide, depressing a compliant surface overlay disposed adjacent to the optical waveguide, contacting the optical waveguide by the compliant surface overlay upon depressing the compliant surface overlay, frustrating the internally reflected light within the optical waveguide upon contacting the optical waveguide by the compliant surface overlay to cause some of the internally reflected light to escape the optical waveguide, and imaging the escaped light.
  • the method further includes the steps of disposing a small gap between a substantial portion of the compliant surface overlay and the optical waveguide so that frustrating the internally reflected light is minimized when the compliant surface overlay is not depressed.
  • the compliant surface overlay is depressed by a finger of a user.
  • the compliant surface overlay is depressed by passive styluses, gloved hands, and/or arbitrary objects.
  • the compliant surface overlay is depressed simultaneously at multiple locations, the optical waveguide is contacted simultaneously by the compliant surface overlay at multiple corresponding locations, and the internally reflected light within the optical waveguide is frustrated simultaneously at those corresponding locations.
  • At least two of the corresponding locations are disposed apart from one another.
  • light of a predetermined wavelength is received within the optical waveguide, and light escaping the optical waveguide at the predetermined wavelength is imaged.
  • light of two different predetermined wavelengths is received within the optical waveguide, and light substantially only at those wavelengths escaping the optical waveguide is imaged.
  • first and second cameras are provided and each camera images light at a different wavelength.
  • pulses of light are received within the optical waveguide, and an imaging camera having a shutter substantially synchronized to that images the escaped light.
  • an imaging camera having a shutter substantially synchronized to that images the escaped light.
  • a compliant surface overlay having a non- planar contact surface is provided.
  • a two-dimensional video image is projected onto a diffuser disposed on the optical waveguide.
  • an electro-switchable diffuser screen synchronized to a rate of projection of the video image is provided, the electro- switchable diffuser screen being non-diffusive according to the shutter periods of a sensor imaging the escaped light.
  • the provided electro-switchable diffuser screen is a directional (e.g. holographic, Lumisty, etc.) that diffuses for certain angles of incidence, and is non-diffuse for others
  • additional cameras are provided to view through the diffuser when the diffuser is made non-diffusive to observe the touching object.
  • the projected video image is diffused by a diffuser disposed on the optical waveguide.
  • the compliant surface overlay diffuses the projected video image.
  • the compliant surface overlay includes a component that rejects ambient light of the same wavelengths as the light source, a component that reduces friction at the interaction surface, a component that reduces glare, and a component that provides a comfortable cushion for the user to depress.
  • a video image is projected through the optical waveguide onto the compliant surface overlay, and the projected video image is diffused by the compliant surface overlay.
  • Figures IA - ID show several examples of multi-touch sensing in accordance with the present invention.
  • Figure 2 is a schematic illustration of a multi-touch sensing display based on frustrated total internal reflection in accordance with the present invention
  • Figure 3 is a schematic illustration of utilizing two fingers with the multi-touch sensing display of the present invention.
  • Figure 4 is a schematic illustration of a non-planar optical waveguide that may be utilized in accordance with the present invention.
  • Figure 5 is a schematic illustration of a multi-touch sensing display employing a compliant surface overlay in accordance with the present invention
  • Figure 6 A and 6B respectively show images from outputs of a contaminated surface and when employing a compliant surface in accordance with the present invention
  • Figure 7 is a schematic illustration of a non-planar optical waveguide and a non-planar compliant surface overlay that may be utilized in accordance with the present invention
  • Figure 8 is a schematic illustration of a system employing multiple imaging sensors and projectors in accordance with the present invention
  • Figure 9 is a schematic illustration of a multi-touch sensing display employing an LCD in accordance with the present invention.
  • Figure 10 is a schematic illustration of a multi-touch sensing display employing multiple wavelengths of light and multiple imaging sensors in accordance with the present invention.
  • Multi-touch sensing enables a user to interact with a system with more than one finger at a time, as in chording and bi-manual operations. Multi-touch sensing may accommodate multiple users simultaneously, which is especially useful for larger shared-display systems such as interactive walls and tabletops.
  • Figures IA through ID of the drawings show several simple examples of multi-touch sensing of the present invention.
  • Multi-touch sensing in accordance with the present invention is based on frustrated total internal reflection (FTIR).
  • FTIR frustrated total internal reflection
  • TIR total internal reflection
  • the present invention employs FTIR to produce a system that acquires true touch image information at high spatial and temporal resolutions.
  • the multi-touch system of the present invention is scalable to large installations, and is well suited for use with display technologies including rear-projection.
  • FIG. 2 of the drawings is a simplified schematic illustration of a multi-touch sensing display 10 in accordance with the present invention.
  • multi-touch sensing display 10 also called, for convenience, "sensor 10"
  • sensor 10 includes an optical waveguide 12, a light source 14, light absorbing surfaces (or “baffle”) 16, and an imaging sensor 20 (also called imaging camera herein)(other elements in Figure 2 described below).
  • Light source 14 preferably includes multiple high-power infrared LEDs, which are placed directly against a polished edge of optical waveguide 12 so as to maximize coupling into total internal reflection.
  • the LEDs can provide a total optical output of 46OmW at 880nm, although other optical outputs can be employed.
  • Optical waveguide 12 may be a sheet of acrylic whose edges have been polished to admit light, but other suitable materials may be utilized. In one example of a manufactured prototype, a one-quarter inch (6.4mm) thick acrylic sheet having the dimensions of 16 inches by 12 inches (406mm x 305mm) is utilized as the optical waveguide. Common glass generally is not preferred due to its poor optical transmittance. However, clearer glass formulations (e.g. "water white”) may be employed. Though more expensive, such glass is structurally stiffer and is far less easily scratched than acrylic.
  • first and second fingers 30a, 30b contact the optical waveguide 12 at two different locations, thus resulting in the frustration of the total internal reflection within the optical waveguide in two regions.
  • light escapes from such different regions of the optical waveguide, as represented by arrows "B" and "C” in the figure.
  • the imaging sensor is mounted orthogonally to detect the light scattered through the optical waveguide.
  • various optical components e.g., mirrors
  • the imaging sensor preferably is equipped with a band-pass filter matched to the output of the light source (e.g., 14) to minimize background signal.
  • the imaging sensor may be of different types, e.g., CCD, CMOS.
  • the function of the imaging sensor can also be provided by any other means for sensing multiple points of light, such as a 2D array of photodiodes or phototransistors or other light-sensing elements.
  • imaging sensor 20 continuously images the light escaping from non-contact surface 12b of optical waveguide 12. Accordingly, imaging sensor 20 discriminately senses, for each successive instant of time, all points of contact of optical waveguide 12. Hence, for a "single" point of contact, such as contact by one finger of a user as shown in Figure 2, a single "area" of contact corresponding to the surface of the finger actually contacting the optical waveguide is discriminately sensed by the imaging sensor. Likewise, when two or more objects (e.g., two or more fingers of a user) contact the optical waveguide, multiple areas of contact are discriminately (and simultaneously) sensed by the imaging sensor. As used herein, a point of contact, contacting a "location" or other similar phase is understood to mean an area of contact, unless it is clear within the context of the description that another meaning is intended.
  • the senor of the present invention discriminates between a relatively small point of contact and a larger point of contact.
  • a finger contacting the optical waveguide with a relatively small amount of pressure provides an area of contact that generally is smaller than the area of contact when greater pressure is applied (i.e., more of the finger contacts the surface of the optical waveguide when greater pressure is applied).
  • an imaging sensor with a sufficiently high frame capture rate (e.g., 60 frames per second) and a sufficiently high imaging resolution, one, two or more points (areas) of contact of the optical waveguide are continuously sensed to sufficiently track all initial contact, movement and discontinuation of contact by one or more objects (including simultaneous and/or sequential contact/movement), such as by the fingers of a user of the multi-touch sensing display of the present invention.
  • a sufficiently high frame capture rate e.g. 60 frames per second
  • a sufficiently high imaging resolution e.g. 60 frames per second
  • imaging sensor 20 preferably is supplied to a suitable computer (not shown) or other electronic device capable of handling various well-known image-processing operations, such as rectification, background subtraction, noise removal, and analysis for each video frame.
  • a suitable computer not shown
  • image-processing operations such as rectification, background subtraction, noise removal, and analysis for each video frame.
  • Well-known machine vision tracking techniques then may be employed to translate the video sequences into discrete touch events and strokes.
  • An imaging sensor that captures the light at 8-bit monochrome at 60 frames per second at a resolution of 640x480 (corresponding to lmm 2 precision on the surface) is suitable for many multi-touch sense applications.
  • an imaging sensor having greater resolution, a different frame capture rate and/or other characteristics may be employed. Processing may be carried out by any suitable computing system.
  • Multi-touch sensing in accordance with the present invention provides full imaging touch information without occlusion or ambiguity issues.
  • the touch sense is zero-force and true, that is, it accurately discriminates touch from a very slight hover.
  • the multi-touch sensing display of the present invention is capable of sampling at both high temporal and spatial resolutions.
  • the multi-touch sensing display is scalable to relatively large surfaces, such as a wall-sized touch display, although various factors including sensor/camera resolution and amount of illumination should be taken into account for the multi-touch sensing display to cover relatively large areas.
  • the optical waveguide has a non-flat contact surface, i.e., non-planar.
  • the contact surface may be concave, convex or other non-flat design.
  • Figure 4 shows an optical waveguide 32 having a hemispherical shape suitable, for example, for terrestrial body mapping control applications.
  • the multi-touch sensing display of the present invention can be used standalone, but because it is completely visually transparent, it is particularly well suited for use in combination with rear-projection. For example, such a combination avoids the disadvantages of occlusion and shadowing associated with top/front projection.
  • a video projector 22 as shown in Figure 2 may be employed within the multi-touch sensing display of the present invention.
  • Figure 2 shows projector 22 arranged alongside imaging sensor 20, projector 22 may be disposed at other locations and/or away from imaging sensor 20, generally with the aid of suitable optics.
  • a suitable diffuser 18 is disposed on the rear (non-contact) side of optical waveguide 12.
  • Diffuser 18 is disposed alongside optical waveguide 12 with a small gap 24 between the two so that diffuser 18 does not frustrate the total internal reflection of the light output by light source 14.
  • diffuser 18 does not appreciably affect the IR image seen by imaging sensor 20 since diffuser 18 is relatively close to the sources of light (e.g. the user's fingers) being imaged. While this scheme introduces a disparity between the display and interaction surfaces, corresponding to the thickness of the waveguide (one-quarter inch in the example), an optical waveguide having a smaller thickness may be employed if necessary. In such case, rigidity of a relatively large optical waveguide can be increased by employing another layer of transparent material stacked to the rear of the diffuser to add structural support without increasing disparity.
  • optical waveguide 12 includes an anti-reflective coating on the non-contact side (projector side) to minimize reduction in the brightness of the display output by projector 22.
  • the response of the multi-touch sensing display of the present invention may be dependant on the optical qualities of the object being sensed. For example, an article, such as a coffee mug, lying on the contact side of the optical waveguide may not be detected if such contact does not frustrate the total internal reflection of the light.
  • the present invention may be designed (further discussed below) so that FTIR is not dependent on the type of material contacting the contact side of the optical waveguide. In such case, the multi-touch sensing display of the present invention sufficiently detects contact by, for example, gloved hands (or dry skin), passive styluses and arbitrary objects..
  • a compliant surface overlay may be employed with the multi-touch sensing display of the present invention.
  • Figure 5 of the drawings is a simplified schematic illustration of a multi-touch sensing display 40 employing a compliant surface overlay 48 (or "compliant surface”). As shown, compliant surface 48 is disposed adjacent the contact surface of optical waveguide 42. A small gap 54 is disposed between compliant surface 48 and optical waveguide 42 so that total internal reflection of the light output by light source 44 is not frustrated (or is negligibly frustrated) when there is no contact with the compliant surface by, for example, a user's finger 60.
  • compliant surface 48 when compliant surface 48 is depressed at one location or simultaneously depressed at multiple locations by, for example, one or more fingers of a user, the compliant surface contacts the optical waveguide immediately below (i.e., adjacent) such contacted locations thus frustrating total internal reflection at such points of contact which, in turn, causes (simultaneously) light to escape from the optical waveguide at locations that correspond to where the compliant surface was contacted.
  • Imaging sensor 50 thereafter detects the escaped light.
  • the compliant surface may be made of various plastic films and other materials, including common vinyl rear-projection screen material (e.g., Rosco Gray #02015). Various other compliant surfaces in accordance with the present invention are discussed further below.
  • the multi-touch sensing display employing a compliant surface in accordance with the present invention advantageously is immune to contaminants, such as oil and perspiration, which may be deposited on the sensor over extended usage. That is, the existence of oils, dirt, perspiration and other materials on the contact surface of the compliant surface does not degrade or otherwise impact frustrated total internal reflection upon depression of the compliant surface. Likewise, scratches and nicks on the contact surface of the compliant surface do not impact the sensing capability of the multi-touch sensing display of the present invention.
  • Figure 6A shows an image of an output of a contaminated surface in the absence of a compliant surface
  • Figure 6B shows an image of an output when employing a compliant surface. As shown, noise resulting from the contaminants is completely removed in the case of when a compliant surface is employed in the multi-touch system of the present invention.
  • the multi-touch sensing display employing a compliant surface in accordance with the present invention advantageously also now functions based on true force information rather than the effectiveness of the touching object with respect to FTIR. This allows the sensor to indiscriminately detect any object depressing the surface. Thus, a user may utilize passive styluses, or use gloved hands, pens, etc.
  • multiple infrared wavelengths are employed to better discriminate the desired signal from background sources and noise.
  • the multi-touch sensing display employing a compliant surface in accordance with the present invention may be utilized without a rear projector.
  • the multi-touch sensing display employing a compliant surface also employs a rear projector (e.g., projector 52 shown in Figure 5).
  • the compliant surface operates also as a diffuser for the rear-projection.
  • the compliant surface is comprised of a composite of multiple materials, each generally contributing to one or more of the following desired characteristics: i) FTIR effectiveness; ii) function as an optical diffuser for rear-projection; iii) wavelength selective shielding to mitigate interference from external ambient light; iv) anti-glare to enhance visibility of the display; v) the tactile "feel" for a human; and vi) durability — a "hardcoat" wear layer preferably replaceable in the field.
  • the various layers employed may be affixed to one another using well-known index-of-refraction matched optical adhesives.
  • a stack includes (1) a thin layer of rubber, (2) a thin-film PET (polyethylene terephthalate) film with a metal coating, and (3) a thin PET film chemically treated to have a matte surface.
  • the thin layer of rubber provides for FTIR contact, operates as the diffuser for rear-projection, and also provides a comfortable tactile response.
  • the thin-film PET (polyethylene terephthalate) film with the metal coating reflects/absorbs ambient infrared light.
  • the thin PET film treated to have a matte surface provides for a comfortable surface on which a user's finger or fingers can easily glide across, and for durability.
  • the compliant surface may be non-flat, i.e., non-planar. It may concave, convex or have another non-flat design. Similar to the non-flat optical waveguide shown in Figure 4, Figure 7 shows an exemplary non-flat optical waveguide 62 on which a non-flat compliant surface overlay 64 is disposed. As another variation, a non-flat compliant surface overlay may be disposed over a flat optical waveguide.
  • any of the herein-described embodiments and variations may employ a light source (e.g., LED 14 shown in Figure 2 or LED 44 shown in Figure 5) that is pulsed and synchronized to the shutter of the imaging sensor (e.g., video sensor 20 shown in Figure 2 or video sensor 50 shown in Figure 5), to beneficially reduce the amount of ambient light received by the imaging sensor. That is, the imaging sensor's shutter is only exposed to the pulse period of the light source.
  • the light source can be pulsed at a brighter intensity to increase the signal to noise ratio of the system.
  • Other types of light sources e.g. lasers may be used as well.
  • FIG. 8 shows a schematic illustration of an exemplary system 70 employing three imaging sensors 72a, 72b and 72c, along with three projectors 74a, 74b and 74c.
  • each sensor images light escaping a different portion of optical waveguide 76 and each projector projects a respective video image onto one of those portions.
  • the imaging sensors and projectors may be spaced apart from one another along a single axis, multiple axes, along a grid system, or other suitable manner.
  • a compliant surface 78 is employed, but multiple sensors and/or multiple projectors may be employed in a system without a compliant surface overlay, or in any other embodiment described herein.
  • an LCD display panel is used in place of a projector.
  • Figure 9 shows a simplified schematic illustration of a system 80 with a multi-touch sensor employing an LCD panel 82 disposed between an optical waveguide 84 and an imaging sensor 86.
  • LCD panel 82 is adapted/designed to be transparent to infrared light so that infrared light that escapes optical waveguide 84 is imaged by imaging sensor 86. Since LCD panels are well known, further description thereof is omitted herein except where otherwise necessary for an understanding of the present invention.
  • An LCD backlight 90 may be disposed behind LCD panel 82.
  • a compliant surface 88 also may be utilized and, in such case, the compliant surface preferably is adapted/designed to not have optical diffuser properties.
  • use of an LCD display panel advantageously reduces volume and increases portability.
  • one or more additional image sensors are also placed behind the LCD panel.
  • the images from these sensors are processed by computing machines to determine extra information about the touch points and/or the user- e.g. the pose or identification of the user's finger associated with each touch.
  • two wavelengths of light are output from the light source and two imaging sensors are employed, such as schematically illustrated in Figure 10.
  • two sets of LEDs 102a and 102b are employed in system 100.
  • LED 102a which may be an array of LEDs or another type of light source, emits light at, for example, 880nm.
  • LED 102b which also may be an array of LEDs or another type of light source, emits lights at, for example, 950nm.
  • Imaging sensors 104a images light at the first wavelength (e.g., 880nm) and imaging sensor 104b images light at the second wavelength (e.g., 940nm).
  • Suitable filters may be employed.
  • light must be received by both imaging sensors (e.g., at the same time and location) for the system to acknowledge the occurrence of a contact (i.e., FTIR response) at such time/location.
  • Well-known processing methods may be employed to process both video streams in this manner.
  • FTIR response is further discriminated from background light.
  • a live finger is discriminated from latent residues in the event a compliant surface is not utilized.
  • Three or more wavelengths may be employed.
  • a single imaging sensor is employed and designed to image light at multiple selective frequencies or frequency ranges. Multiple wavelengths of light, along with one or more imaging sensors, may be employed in the various systems described herein, including systems that employ a compliant surface overlay as well as those that do not.
  • an electro- switchable diffuser screen is employed with a projector.
  • An LC (liquid crystal) privacy glass window capable of being electrically switchable from a transparent state to a frosty appearance under electronic control is employed.
  • Such an electro- switchable screen is disclosed in Kunz, A. M. and Spagno, C. P., "Technical System for Collaborative Work. In Proceedings of the Workshop on Virtual Environments” (Barcelona, Spain, May 30 - 31, 2002); and in W. St ⁇ rzlinger and S. M ⁇ ller, Eds. ACM International Conference Proceeding Series, vol. 23. Eurographics Association, Aire-la-Ville, Switzerland, 73-80, each of which is incorporated herein by reference.
  • the LC privacy glass window is switched between the two states at a relatively rapid rate (e.g., 60 times a second) and is synchronized to switch with the imaging sensor (e.g., infrared camera) so that a second (or third) sensor view see through the screen, such as to observe the touching object, when the LC privacy glass window is momentarily transparent.
  • the imaging sensor e.g., infrared camera
  • the images from the additional sensors are processed by computing hardware to determine other information about the touches- e.g. the orientation or pose of the user's hand.
  • the diffuser can be a directional-type diffuser that is designed to diffuse for selected angles of incidence, and to not diffuse for other angles of incidence..
  • various other diffusers may be employed to allow additional sensors to view through the screen clearly while simultaneously diffusing light from rear-projection.
  • An exemplary screen includes the holographic film employed by TouchLight, discussed in Wilson, A. D., "TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction," Proceedings of the 6th International Conference on Multimodal Interfaces (State College, PA, USA, October 13 - 15, 2004), ICMI '04, ACM Press, New York, NY, 69-76 (2004), incorporated herein by reference.
  • a less expensive, directionally scattering film is discussed in Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., and Naemura, T., "Lumisight Table: A Face-to-face Collaboration Support System That Optimizes Direction of Projected Information to Each Stakeholder," Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (Chicago, Illinois, USA, November 06 - 10, 2004), CSCW '04, ACM Press, New York, NY, 274-283 (2004), which is incorporated herein by reference.
  • a Rayleigh-scattering material may be employed that that diffuses visible wavelengths, but is substantially transparent to a convenient infrared band.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Optics & Photonics (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Position Input By Displaying (AREA)
  • Switches Operated By Changes In Physical Conditions (AREA)

Abstract

L'invention concerne des systèmes d'affichage à détection à touches multiples adaptables, de haute résolution, et des procédés basés sur une réflexion interne totale frustrée qui emploient un guide d'onde optique (12, 32, 42, 62, 76, 84) qui reçoit de la lumière, telle qu'une lumière infrarouge, qui subit une réflexion interne totale et un capteur d'imagerie (20, 50, 72a, 72b, 72c, 86, 104a, 104b) qui détecte la lumière qui s'échappe du guide d'onde optique (12, 32, 42, 62, 76, 84) provoquée par frustration de la réflexion interne totale en raison du contact par un utilisateur. Le guide d'onde optique (12, 32, 42, 62, 76, 84) peut être ajusté avec un recouvrement de surface conforme (48, 64, 78, 88) pour augmenter de manière importante la performance de détection, rendre minimal l'effet de contaminants et l'endommagement à la surface de contact, pour étendre de manière générale la vie du système et pour fournir d'autres avantages. Les systèmes et les procédés fournissent une capacité véritable de touches multiples (entrées multiples) et de résolutions spatiale et temporelle élevées à cause de l'imagerie en continu de la réflexion interne totale frustrée qui s'échappe du guide d'onde optique entier (12, 32, 42, 62, 76, 84). Entre autres caractéristiques et avantages, les systèmes et les procédés sont adaptables pour de grandes installations et sont appropriés pour une utilisation avec une rétro-projection (22, 52, 74a, 74b, 74c) et d'autres dispositifs d'affichage.
PCT/US2007/075221 2006-08-03 2007-08-03 Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée WO2008017077A2 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP07813780A EP2047308A4 (fr) 2006-08-03 2007-08-03 Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée
JP2009523076A JP2009545828A (ja) 2006-08-03 2007-08-03 全反射妨害による多接触検知表示装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US82132506P 2006-08-03 2006-08-03
US60/821,325 2006-08-03

Publications (3)

Publication Number Publication Date
WO2008017077A2 true WO2008017077A2 (fr) 2008-02-07
WO2008017077A3 WO2008017077A3 (fr) 2008-12-18
WO2008017077A4 WO2008017077A4 (fr) 2009-02-19

Family

ID=38997904

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2007/075221 WO2008017077A2 (fr) 2006-08-03 2007-08-03 Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée

Country Status (4)

Country Link
EP (1) EP2047308A4 (fr)
JP (1) JP2009545828A (fr)
KR (1) KR20090060283A (fr)
WO (1) WO2008017077A2 (fr)

Cited By (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ITMI20081536A1 (it) * 2008-08-26 2010-02-27 Fond Bruno Kessler Sensore di tocco singolo e/o multiplo
EP2194447A1 (fr) 2008-12-08 2010-06-09 IBBT vzw Système de peinture électronique et appareil pour entrée de données dans ce système
EP2212763A1 (fr) * 2007-10-10 2010-08-04 FlatFrog Laboratories AB Pavé tactile et procédé d'exploitation du pavé tactile
WO2011013994A2 (fr) * 2009-07-27 2011-02-03 Hwang Jay-Yeob Procédé pour mise en oeuvre d'une touche d'une guitare virtuelle
WO2011027111A1 (fr) * 2009-09-04 2011-03-10 Cambridge Display Technology Limited Dispositif d'affichage à écran tactile
WO2011033913A1 (fr) * 2009-09-15 2011-03-24 日本電気株式会社 Dispositif d'entrée et système d'entrée
JP2011513828A (ja) * 2008-02-29 2011-04-28 マイクロソフト コーポレーション 切替え可能なディフューザを備える対話型サーフェイスコンピュータ
JP2011108236A (ja) * 2009-11-13 2011-06-02 Samsung Electronics Co Ltd センシングアレイを用いたマルチタッチおよび近接するオブジェクトセンシング装置
EP2348390A1 (fr) * 2010-01-20 2011-07-27 Evoluce Ag Dispositif d'entrée doté d'une caméra
JP2011525635A (ja) * 2008-05-02 2011-09-22 マイクロソフト コーポレーション タンジブル・ユーザ・インタフェース上への画像の投影
FR2958426A1 (fr) * 2010-04-02 2011-10-07 Alain Jutant Ecran interactif multi-points et multi-utilisateurs.
WO2011124648A1 (fr) * 2010-04-07 2011-10-13 Opdi Technologies A/S Dispositif tactile et procédé de détection de contact
JP2011527478A (ja) * 2008-07-07 2011-10-27 ノーテル ネットワークス リミテッド ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
WO2011142889A3 (fr) * 2010-05-14 2012-01-19 International Business Machines Corporation Dispositif d'interface avec cellule(s) solaire(s) intégrée(s) pour la collecte d'énergie
US8125468B2 (en) 2007-07-30 2012-02-28 Perceptive Pixel Inc. Liquid multi-touch sensor and display device
US8144271B2 (en) 2006-08-03 2012-03-27 Perceptive Pixel Inc. Multi-touch sensing through frustrated total internal reflection
US8147316B2 (en) 2006-10-10 2012-04-03 Wms Gaming, Inc. Multi-player, multi-touch table for use in wagering game systems
WO2012105893A1 (fr) 2011-02-02 2012-08-09 Flatfrog Laboratories Ab Intercouplage optique destiné à des systèmes tactiles
US8241912B2 (en) 2008-06-26 2012-08-14 Wms Gaming Inc. Gaming machine having multi-touch sensing device
JP2012529680A (ja) * 2008-07-07 2012-11-22 ノーテル・ネットワークス・リミテッド ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
JP5174962B2 (ja) * 2009-05-28 2013-04-03 シャープ株式会社 タッチパネル、液晶パネル、液晶表示装置、及びタッチパネル一体型の液晶表示装置
US8441467B2 (en) 2006-08-03 2013-05-14 Perceptive Pixel Inc. Multi-touch sensing display through frustrated total internal reflection
WO2013163720A1 (fr) * 2012-05-02 2013-11-07 University Of Manitoba Détection d'identité d'utilisateur sur des surfaces interactives
US8624853B2 (en) 2009-06-01 2014-01-07 Perceptive Pixel Inc. Structure-augmented touch sensing with frustated total internal reflection
US8736581B2 (en) 2009-06-01 2014-05-27 Perceptive Pixel Inc. Touch sensing with frustrated total internal reflection
US8803848B2 (en) 2007-12-17 2014-08-12 Victor Manuel SUAREZ ROVERE Method and apparatus for tomographic touch imaging and interactive system using same
US8947400B2 (en) 2009-06-11 2015-02-03 Nokia Corporation Apparatus, methods and computer readable storage mediums for providing a user interface
US8959459B2 (en) 2011-06-15 2015-02-17 Wms Gaming Inc. Gesture sensing enhancement system for a wagering game
US9086732B2 (en) 2012-05-03 2015-07-21 Wms Gaming Inc. Gesture fusion
US9323396B2 (en) 2009-06-01 2016-04-26 Perceptive Pixel, Inc. Touch sensing
US9389732B2 (en) 2011-09-09 2016-07-12 Flatfrog Laboratories Ab Light coupling structures for optical touch panels
US9864127B2 (en) 2008-10-24 2018-01-09 Tactotek Oy Arrangement for a touchscreen and related method of manufacture
US9874978B2 (en) 2013-07-12 2018-01-23 Flatfrog Laboratories Ab Partial detect mode
US10019113B2 (en) 2013-04-11 2018-07-10 Flatfrog Laboratories Ab Tomographic processing for touch detection
US10126882B2 (en) 2014-01-16 2018-11-13 Flatfrog Laboratories Ab TIR-based optical touch systems of projection-type
US10146376B2 (en) 2014-01-16 2018-12-04 Flatfrog Laboratories Ab Light coupling in TIR-based optical touch systems
US10161886B2 (en) 2014-06-27 2018-12-25 Flatfrog Laboratories Ab Detection of surface contamination
US10168835B2 (en) 2012-05-23 2019-01-01 Flatfrog Laboratories Ab Spatial resolution in touch displays
US10282035B2 (en) 2016-12-07 2019-05-07 Flatfrog Laboratories Ab Touch device
US10318074B2 (en) 2015-01-30 2019-06-11 Flatfrog Laboratories Ab Touch-sensing OLED display with tilted emitters
US10401546B2 (en) 2015-03-02 2019-09-03 Flatfrog Laboratories Ab Optical component for light coupling
US10437389B2 (en) 2017-03-28 2019-10-08 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US10474249B2 (en) 2008-12-05 2019-11-12 Flatfrog Laboratories Ab Touch sensing apparatus and method of operating the same
US10481737B2 (en) 2017-03-22 2019-11-19 Flatfrog Laboratories Ab Pen differentiation for touch display
US10496227B2 (en) 2015-02-09 2019-12-03 Flatfrog Laboratories Ab Optical touch system comprising means for projecting and detecting light beams above and inside a transmissive panel
US10726241B2 (en) 2015-04-06 2020-07-28 Identification International, Inc. Systems and methods for capturing images using a pressure sensitive membrane
US10740902B2 (en) 2016-10-14 2020-08-11 Identification International, Inc. System and method for identifying features of a friction ridge signature based on information representing a topography of friction ridges
US10761657B2 (en) 2016-11-24 2020-09-01 Flatfrog Laboratories Ab Automatic optimisation of touch signal
US11182023B2 (en) 2015-01-28 2021-11-23 Flatfrog Laboratories Ab Dynamic touch quarantine frames
US11256371B2 (en) 2017-09-01 2022-02-22 Flatfrog Laboratories Ab Optical component
US11301089B2 (en) 2015-12-09 2022-04-12 Flatfrog Laboratories Ab Stylus identification
GB2602336A (en) * 2020-12-23 2022-06-29 Uniphy Ltd Optical touch screen
US11474644B2 (en) 2017-02-06 2022-10-18 Flatfrog Laboratories Ab Optical coupling in touch-sensing systems
US11567610B2 (en) 2018-03-05 2023-01-31 Flatfrog Laboratories Ab Detection line broadening
US11600106B2 (en) 2019-03-29 2023-03-07 Identification International, Inc. Systems and methods for generating three-dimensional images of an object based on frustrated total internal reflection
US11893189B2 (en) 2020-02-10 2024-02-06 Flatfrog Laboratories Ab Touch-sensing apparatus
US11943563B2 (en) 2019-01-25 2024-03-26 FlatFrog Laboratories, AB Videoconferencing terminal and method of operating the same
EP4321975A3 (fr) * 2008-06-19 2024-06-05 Massachusetts Institute of Technology Capteur tactile utilisant une imagerie élastomère
US12055969B2 (en) 2018-10-20 2024-08-06 Flatfrog Laboratories Ab Frame for a touch-sensitive device and tool therefor
US12056316B2 (en) 2019-11-25 2024-08-06 Flatfrog Laboratories Ab Touch-sensing apparatus

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100978437B1 (ko) * 2009-10-30 2010-08-26 이익승 멀티터치 제어시스템 및 제어방법
US8384694B2 (en) * 2009-11-17 2013-02-26 Microsoft Corporation Infrared vision with liquid crystal display device
KR100977558B1 (ko) * 2009-12-22 2010-08-23 전자부품연구원 적외선 스크린 방식의 공간 터치 장치
KR101607508B1 (ko) 2010-01-08 2016-03-31 삼성디스플레이 주식회사 터치 모듈, 이를 갖는 표시 장치 및 터치 모듈의 터치 위치 검출 방법
KR101044145B1 (ko) * 2010-02-10 2011-06-24 삼성전기주식회사 광학 포인팅 장치
KR101117644B1 (ko) * 2010-03-17 2012-03-05 삼성모바일디스플레이주식회사 유기 발광 표시 장치 및 이를 포함하는 터치 표시 장치
KR101065408B1 (ko) * 2010-03-17 2011-09-16 삼성모바일디스플레이주식회사 터치 표시 장치
US9535537B2 (en) * 2010-11-18 2017-01-03 Microsoft Technology Licensing, Llc Hover detection in an interactive display device
JPWO2012096269A1 (ja) * 2011-01-13 2014-06-09 株式会社ニコン 映像表示装置、映像表示システム及びスクリーン
KR101148027B1 (ko) * 2011-01-25 2012-05-24 연세대학교 산학협력단 멀티터치 센싱장치와 시스템 및 이를 위한 멀티터치 센싱방법
KR20120102895A (ko) 2011-03-09 2012-09-19 삼성전자주식회사 광센싱 어셈블리 및 이를 갖는 양방향 대화형 표시장치
KR20140022843A (ko) * 2011-04-19 2014-02-25 퍼셉티브 픽셀 인코포레이티드 터치 감지를 위한 광학 필터형 센서인픽셀 기술
TWI543045B (zh) 2012-04-10 2016-07-21 揚明光學股份有限公司 觸控裝置和觸控投影系統
JP2016509275A (ja) * 2012-11-29 2016-03-24 ルノー エス．ア．エス． 物理的タイプの対話性を再現する通信のシステム及び方法
US20140152914A1 (en) * 2012-11-30 2014-06-05 Corning Incorporated Low-Fe Glass for IR Touch Screen Applications
HUE032186T2 (en) * 2013-03-20 2017-09-28 Agc Glass Europe High infrared radiation permeability glass plate
CN105189387A (zh) * 2013-03-20 2015-12-23 旭硝子欧洲玻璃公司 具有高红外线辐射透射率的玻璃片
WO2014180679A1 (fr) * 2013-05-07 2014-11-13 Agc Glass Europe Feuille de verre à haute transmission aux rayonnements infrarouges
WO2015011040A1 (fr) * 2013-07-24 2015-01-29 Agc Glass Europe Plaque de verre à transmission élevée des infrarouges
JP6321474B2 (ja) * 2014-07-04 2018-05-09 京セラ株式会社 センサ、センサ装置および光学式操作装置
FR3025052B1 (fr) * 2014-08-19 2017-12-15 Isorg Dispositif de detection d'un rayonnement electromagnetique en materiaux organiques
JP2016045772A (ja) * 2014-08-25 2016-04-04 日東電工株式会社 位置センサ
CN114840104A (zh) * 2018-08-21 2022-08-02 贝奇克有限公司 用于光学触摸检测装置的阻挡层和触觉表面特征

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4484179A (en) * 1980-04-16 1984-11-20 At&T Bell Laboratories Touch position sensitive surface
US5942761A (en) * 1995-06-07 1999-08-24 Tuli; Raja Singh Enhancement methods and devices for reading a fingerprint image
US6608722B2 (en) * 2000-08-01 2003-08-19 James Cowan Directional diffuser
EP1376872B1 (fr) * 2002-05-07 2007-10-10 Schott Ag Dispositif d'éclairage pour des écrans sensibles à toucher
US6999665B2 (en) * 2002-10-09 2006-02-14 Scram Technologies, Inc. Display panel having dual directional diffusion
US7705835B2 (en) * 2005-03-28 2010-04-27 Adam Eikman Photonic touch screen apparatus and method of use

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of EP2047308A4 *

Cited By (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8259240B2 (en) 2006-08-03 2012-09-04 Perceptive Pixel Inc. Multi-touch sensing through frustrated total internal reflection
US8441467B2 (en) 2006-08-03 2013-05-14 Perceptive Pixel Inc. Multi-touch sensing display through frustrated total internal reflection
US8144271B2 (en) 2006-08-03 2012-03-27 Perceptive Pixel Inc. Multi-touch sensing through frustrated total internal reflection
US8348747B2 (en) 2006-10-10 2013-01-08 Wms Gaming Inc. Multi-player, multi-touch table for use in wagering game systems
US8147316B2 (en) 2006-10-10 2012-04-03 Wms Gaming, Inc. Multi-player, multi-touch table for use in wagering game systems
US8926421B2 (en) 2006-10-10 2015-01-06 Wms Gaming Inc. Multi-player, multi-touch table for use in wagering game systems
US8125468B2 (en) 2007-07-30 2012-02-28 Perceptive Pixel Inc. Liquid multi-touch sensor and display device
US8716614B2 (en) 2007-10-10 2014-05-06 Flatfrog Laboratories Ab Touch pad and a method of operating the touch pad
EP2212763A1 (fr) * 2007-10-10 2010-08-04 FlatFrog Laboratories AB Pavé tactile et procédé d'exploitation du pavé tactile
EP2212763A4 (fr) * 2007-10-10 2012-06-20 Flatfrog Lab Ab Pavé tactile et procédé d'exploitation du pavé tactile
US8803848B2 (en) 2007-12-17 2014-08-12 Victor Manuel SUAREZ ROVERE Method and apparatus for tomographic touch imaging and interactive system using same
US9836149B2 (en) 2007-12-17 2017-12-05 Victor Manuel SUAREZ ROVERE Method and apparatus for tomographic tough imaging and interactive system using same
JP2011513828A (ja) * 2008-02-29 2011-04-28 マイクロソフト コーポレーション 切替え可能なディフューザを備える対話型サーフェイスコンピュータ
JP2011525635A (ja) * 2008-05-02 2011-09-22 マイクロソフト コーポレーション タンジブル・ユーザ・インタフェース上への画像の投影
EP4321975A3 (fr) * 2008-06-19 2024-06-05 Massachusetts Institute of Technology Capteur tactile utilisant une imagerie élastomère
US8241912B2 (en) 2008-06-26 2012-08-14 Wms Gaming Inc. Gaming machine having multi-touch sensing device
JP2011527478A (ja) * 2008-07-07 2011-10-27 ノーテル ネットワークス リミテッド ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
US9268413B2 (en) 2008-07-07 2016-02-23 Rpx Clearinghouse Llc Multi-touch touchscreen incorporating pen tracking
JP2012529680A (ja) * 2008-07-07 2012-11-22 ノーテル・ネットワークス・リミテッド ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
US8842076B2 (en) 2008-07-07 2014-09-23 Rockstar Consortium Us Lp Multi-touch touchscreen incorporating pen tracking
ITMI20081536A1 (it) * 2008-08-26 2010-02-27 Fond Bruno Kessler Sensore di tocco singolo e/o multiplo
US9864127B2 (en) 2008-10-24 2018-01-09 Tactotek Oy Arrangement for a touchscreen and related method of manufacture
US10474249B2 (en) 2008-12-05 2019-11-12 Flatfrog Laboratories Ab Touch sensing apparatus and method of operating the same
WO2010066395A1 (fr) 2008-12-08 2010-06-17 Ibbt Vzw Système de peinture électronique et appareil pour effectuer une entrée dans le système
EP2194447A1 (fr) 2008-12-08 2010-06-09 IBBT vzw Système de peinture électronique et appareil pour entrée de données dans ce système
JP5174962B2 (ja) * 2009-05-28 2013-04-03 シャープ株式会社 タッチパネル、液晶パネル、液晶表示装置、及びタッチパネル一体型の液晶表示装置
US8896576B2 (en) 2009-05-28 2014-11-25 Sharp Kabushiki Kaisha Touch panel, liquid crystal panel, liquid crystal display device, and touch panel-integrated liquid crystal display device
US8624853B2 (en) 2009-06-01 2014-01-07 Perceptive Pixel Inc. Structure-augmented touch sensing with frustated total internal reflection
US8736581B2 (en) 2009-06-01 2014-05-27 Perceptive Pixel Inc. Touch sensing with frustrated total internal reflection
US9323396B2 (en) 2009-06-01 2016-04-26 Perceptive Pixel, Inc. Touch sensing
US8947400B2 (en) 2009-06-11 2015-02-03 Nokia Corporation Apparatus, methods and computer readable storage mediums for providing a user interface
WO2011013994A3 (fr) * 2009-07-27 2011-04-28 Hwang Jay-Yeob Procédé pour mise en oeuvre d'une touche d'une guitare virtuelle
WO2011013994A2 (fr) * 2009-07-27 2011-02-03 Hwang Jay-Yeob Procédé pour mise en oeuvre d'une touche d'une guitare virtuelle
CN102483671A (zh) * 2009-09-04 2012-05-30 剑桥显示技术有限公司 触摸屏显示设备
US9323373B2 (en) 2009-09-04 2016-04-26 Cambridge Display Technology Limited Touch screen display device
WO2011027111A1 (fr) * 2009-09-04 2011-03-10 Cambridge Display Technology Limited Dispositif d'affichage à écran tactile
JP5783045B2 (ja) * 2009-09-15 2015-09-24 日本電気株式会社 入力装置および入力システム
WO2011033913A1 (fr) * 2009-09-15 2011-03-24 日本電気株式会社 Dispositif d'entrée et système d'entrée
JP2011108236A (ja) * 2009-11-13 2011-06-02 Samsung Electronics Co Ltd センシングアレイを用いたマルチタッチおよび近接するオブジェクトセンシング装置
EP2348390A1 (fr) * 2010-01-20 2011-07-27 Evoluce Ag Dispositif d'entrée doté d'une caméra
FR2958426A1 (fr) * 2010-04-02 2011-10-07 Alain Jutant Ecran interactif multi-points et multi-utilisateurs.
US9110540B2 (en) 2010-04-07 2015-08-18 O-Net Wavetouch Limited Touch-sensitive device and method for detection of touch
WO2011124648A1 (fr) * 2010-04-07 2011-10-13 Opdi Technologies A/S Dispositif tactile et procédé de détection de contact
GB2494564B (en) * 2010-05-14 2018-12-19 Ibm Interface device with integrated solar cell(s) for power collection
WO2011142889A3 (fr) * 2010-05-14 2012-01-19 International Business Machines Corporation Dispositif d'interface avec cellule(s) solaire(s) intégrée(s) pour la collecte d'énergie
GB2494564A (en) * 2010-05-14 2013-03-13 Ibm Interface device with integrated solar cell(s) for power collection
EP3173914A1 (fr) 2011-02-02 2017-05-31 FlatFrog Laboratories AB Couplage optique pour systèmes tactiles
US9552103B2 (en) 2011-02-02 2017-01-24 Flatfrog Laboratories Ab Optical incoupling for touch-sensitive systems
WO2012105893A1 (fr) 2011-02-02 2012-08-09 Flatfrog Laboratories Ab Intercouplage optique destiné à des systèmes tactiles
US10151866B2 (en) 2011-02-02 2018-12-11 Flatfrog Laboratories Ab Optical incoupling for touch-sensitive systems
US8959459B2 (en) 2011-06-15 2015-02-17 Wms Gaming Inc. Gesture sensing enhancement system for a wagering game
US9389732B2 (en) 2011-09-09 2016-07-12 Flatfrog Laboratories Ab Light coupling structures for optical touch panels
WO2013163720A1 (fr) * 2012-05-02 2013-11-07 University Of Manitoba Détection d'identité d'utilisateur sur des surfaces interactives
US9086732B2 (en) 2012-05-03 2015-07-21 Wms Gaming Inc. Gesture fusion
US10168835B2 (en) 2012-05-23 2019-01-01 Flatfrog Laboratories Ab Spatial resolution in touch displays
US10019113B2 (en) 2013-04-11 2018-07-10 Flatfrog Laboratories Ab Tomographic processing for touch detection
US9874978B2 (en) 2013-07-12 2018-01-23 Flatfrog Laboratories Ab Partial detect mode
US10126882B2 (en) 2014-01-16 2018-11-13 Flatfrog Laboratories Ab TIR-based optical touch systems of projection-type
US10146376B2 (en) 2014-01-16 2018-12-04 Flatfrog Laboratories Ab Light coupling in TIR-based optical touch systems
US10161886B2 (en) 2014-06-27 2018-12-25 Flatfrog Laboratories Ab Detection of surface contamination
US11182023B2 (en) 2015-01-28 2021-11-23 Flatfrog Laboratories Ab Dynamic touch quarantine frames
US10318074B2 (en) 2015-01-30 2019-06-11 Flatfrog Laboratories Ab Touch-sensing OLED display with tilted emitters
US11029783B2 (en) 2015-02-09 2021-06-08 Flatfrog Laboratories Ab Optical touch system comprising means for projecting and detecting light beams above and inside a transmissive panel
US10496227B2 (en) 2015-02-09 2019-12-03 Flatfrog Laboratories Ab Optical touch system comprising means for projecting and detecting light beams above and inside a transmissive panel
US10401546B2 (en) 2015-03-02 2019-09-03 Flatfrog Laboratories Ab Optical component for light coupling
US10726241B2 (en) 2015-04-06 2020-07-28 Identification International, Inc. Systems and methods for capturing images using a pressure sensitive membrane
US11301089B2 (en) 2015-12-09 2022-04-12 Flatfrog Laboratories Ab Stylus identification
US10740902B2 (en) 2016-10-14 2020-08-11 Identification International, Inc. System and method for identifying features of a friction ridge signature based on information representing a topography of friction ridges
US11302013B2 (en) 2016-10-14 2022-04-12 Identification International, Inc. System and method for identifying features of a friction ridge signature based on information representing a topography of friction ridges
US10761657B2 (en) 2016-11-24 2020-09-01 Flatfrog Laboratories Ab Automatic optimisation of touch signal
US10775935B2 (en) 2016-12-07 2020-09-15 Flatfrog Laboratories Ab Touch device
US12189906B2 (en) 2016-12-07 2025-01-07 Flatfrog Laboratories Ab Touch device
US11579731B2 (en) 2016-12-07 2023-02-14 Flatfrog Laboratories Ab Touch device
US10282035B2 (en) 2016-12-07 2019-05-07 Flatfrog Laboratories Ab Touch device
US11281335B2 (en) 2016-12-07 2022-03-22 Flatfrog Laboratories Ab Touch device
US11740741B2 (en) 2017-02-06 2023-08-29 Flatfrog Laboratories Ab Optical coupling in touch-sensing systems
US12175044B2 (en) 2017-02-06 2024-12-24 Flatfrog Laboratories Ab Optical coupling in touch-sensing systems
US11474644B2 (en) 2017-02-06 2022-10-18 Flatfrog Laboratories Ab Optical coupling in touch-sensing systems
US10481737B2 (en) 2017-03-22 2019-11-19 Flatfrog Laboratories Ab Pen differentiation for touch display
US11016605B2 (en) 2017-03-22 2021-05-25 Flatfrog Laboratories Ab Pen differentiation for touch displays
US11099688B2 (en) 2017-03-22 2021-08-24 Flatfrog Laboratories Ab Eraser for touch displays
US10606414B2 (en) 2017-03-22 2020-03-31 Flatfrog Laboratories Ab Eraser for touch displays
US11269460B2 (en) 2017-03-28 2022-03-08 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US10437389B2 (en) 2017-03-28 2019-10-08 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US11281338B2 (en) 2017-03-28 2022-03-22 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US10845923B2 (en) 2017-03-28 2020-11-24 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US10606416B2 (en) 2017-03-28 2020-03-31 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US10739916B2 (en) 2017-03-28 2020-08-11 Flatfrog Laboratories Ab Touch sensing apparatus and method for assembly
US12086362B2 (en) 2017-09-01 2024-09-10 Flatfrog Laboratories Ab Optical component
US11256371B2 (en) 2017-09-01 2022-02-22 Flatfrog Laboratories Ab Optical component
US11650699B2 (en) 2017-09-01 2023-05-16 Flatfrog Laboratories Ab Optical component
US11567610B2 (en) 2018-03-05 2023-01-31 Flatfrog Laboratories Ab Detection line broadening
US12055969B2 (en) 2018-10-20 2024-08-06 Flatfrog Laboratories Ab Frame for a touch-sensitive device and tool therefor
US11943563B2 (en) 2019-01-25 2024-03-26 FlatFrog Laboratories, AB Videoconferencing terminal and method of operating the same
US11600106B2 (en) 2019-03-29 2023-03-07 Identification International, Inc. Systems and methods for generating three-dimensional images of an object based on frustrated total internal reflection
US12056316B2 (en) 2019-11-25 2024-08-06 Flatfrog Laboratories Ab Touch-sensing apparatus
US11893189B2 (en) 2020-02-10 2024-02-06 Flatfrog Laboratories Ab Touch-sensing apparatus
GB2602336B (en) * 2020-12-23 2024-08-14 Uniphy Ltd Optical touch screen
GB2602336A (en) * 2020-12-23 2022-06-29 Uniphy Ltd Optical touch screen
WO2022136868A1 (fr) * 2020-12-23 2022-06-30 Uniphy Limited Écran tactile optique

Also Published As

Publication number Publication date
WO2008017077A4 (fr) 2009-02-19
WO2008017077A3 (fr) 2008-12-18
EP2047308A4 (fr) 2010-11-24
JP2009545828A (ja) 2009-12-24
EP2047308A2 (fr) 2009-04-15
KR20090060283A (ko) 2009-06-11

Similar Documents

Publication Publication Date Title
US8441467B2 (en) Multi-touch sensing display through frustrated total internal reflection
EP2047308A2 (fr) Dispositif d'affichage à détection à touches multiples par l'intermédiaire de réflexion interne totale frustrée
Han Low-cost multi-touch sensing through frustrated total internal reflection
EP2188701B1 (fr) Détection tactile multipoint par le biais d'une réflexion interne totale frustrée
US8144271B2 (en) Multi-touch sensing through frustrated total internal reflection
US8581852B2 (en) Fingertip detection for camera based multi-touch systems
CA2748881C (fr) Methode de reconnaissance de la gestuelle et systeme d'entree interactif employant ladite methode
US6061177A (en) Integrated computer display and graphical input apparatus and method
JP5346081B2 (ja) ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
JP5693972B2 (ja) 切替え可能なディフューザを備える対話型サーフェイスコンピュータ
JP5411265B2 (ja) ペントラッキングを組み込んだマルチタッチ式タッチスクリーン
EP2284668A2 (fr) Système d'entrée interactif et composants correspondants
US8125468B2 (en) Liquid multi-touch sensor and display device
US20090267919A1 (en) Multi-touch position tracking apparatus and interactive system and image processing method using the same
Hofer et al. FLATIR: FTIR multi-touch detection on a discrete distributed sensor array
TWI502417B (zh) 觸控顯示裝置及其操作方法
KR20100116267A (ko) 터치 패널 및 그를 가지는 터치 표시 장치
Wattanaparinton et al. RFTIRTouch: Touch Sensing Device for Dual-sided Transparent Plane Based on Repropagated Frustrated Total Internal Reflection
Kim et al. Multi-touch tabletop interface technique for HCI
Quigley et al. Face-to-face collaborative interfaces

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07813780

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2007813780

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2009523076

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 1020097004408

Country of ref document: KR

NENP Non-entry into the national phase

Ref country code: RU

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载