+

WO2009061620A1 - Dual-mode projection apparatus and method for locating a light spot in a projected image - Google Patents

Dual-mode projection apparatus and method for locating a light spot in a projected image Download PDF

Info

Publication number
WO2009061620A1
WO2009061620A1 PCT/US2008/080906 US2008080906W WO2009061620A1 WO 2009061620 A1 WO2009061620 A1 WO 2009061620A1 US 2008080906 W US2008080906 W US 2008080906W WO 2009061620 A1 WO2009061620 A1 WO 2009061620A1
Authority
WO
WIPO (PCT)
Prior art keywords
projection
light spot
image sensor
dual
visible light
Prior art date
Application number
PCT/US2008/080906
Other languages
French (fr)
Inventor
Jizhang Shan
Jin Li
Original Assignee
Omnivision Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US11/936,746 external-priority patent/US7862179B2/en
Priority claimed from US11/936,737 external-priority patent/US8188973B2/en
Application filed by Omnivision Technologies, Inc. filed Critical Omnivision Technologies, Inc.
Priority to EP08847625A priority Critical patent/EP2218252A4/en
Priority to CN2008801232795A priority patent/CN101919241B/en
Publication of WO2009061620A1 publication Critical patent/WO2009061620A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G06F3/0386—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry for light pen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3191—Testing thereof
    • H04N9/3194—Testing thereof including sensor feedback
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/22—Setup operations, e.g. calibration, key configuration or button assignment
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/23—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console
    • A63F13/235—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console using a wireless connection, e.g. infrared or piconet
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/24—Constructional details thereof, e.g. game controllers with detachable joystick handles
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1018—Calibration; Key and button assignment
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1025—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals details of the interface with the game device, e.g. USB version detection
    • A63F2300/1031—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals details of the interface with the game device, e.g. USB version detection using a wireless connection, e.g. Bluetooth, infrared connections
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1043—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being characterized by constructional details
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
    • A63F2300/1093—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera using visible light
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60—Methods for processing data by generating or executing the game program
    • A63F2300/6045—Methods for processing data by generating or executing the game program for mapping control signals received from the input arrangement into game commands
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/18—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical projection, e.g. combination of mirror and condenser and objective
    • G02B27/20—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical projection, e.g. combination of mirror and condenser and objective for imaging minute objects, e.g. light-pointer

Definitions

  • This invention relates generally to digital image projectors for projecting a digital image onto a display screen. More particularly, this invention relates to a dual-mode projection apparatus and method capable of both projecting and capturing an image for locating a light spot in the projected image.
  • Digital image projectors are devices that project digital images onto a projection surface for presentation to an audience.
  • the digital images are generated at a personal computer or other image source devices connected to a projector, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others.
  • the projection surface may be a display screen, wall, or other surface in which projected images become visible to the audience.
  • LCD Crystal Display
  • DLP Digital Light Processing
  • LCDoS Liquid Crystal on Silicon
  • a desirable feature during these presentations is to have a presenter or member of the audience draw attention to particular locations or items of interest on the projection surface, such as, for example, by using a laser pointer.
  • Laser pointers are handheld, pen- shaped devices that project a laser beam onto a surface area to generate a laser spot on the surface that is visible to the audience. Their use has become commonplace at venues where presentation materials are shown on a display screen. Most common laser pointers emit red, green, or blue light of at most 1 or 5 mWatts to avoid damage to human eyes.
  • Laser pointers may also include command switches or buttons for enabling remote control of basic computer functions during the presentations.
  • United States Patent Publication Numbers 2007/0030460, 2006/0227107, and 2006/0197756 and United States Patent Number 6,417,840 all describe laser pointers with command switches mounted thereon to enable remote initiation of various commands at the presentation computer.
  • the commands may include, for example, page-up and page-down commands to indicate to the presentation computer that a new image or page of presentation material is to be displayed and projected onto the projection area.
  • the laser spot generated by the laser pointer may function as a control cursor.
  • the presenter may direct the control cursor to a position in the screen and emulate position-dependent commands via the switches.
  • These position-dependent commands require that the laser spot be detected and its position determined on the projection surface.
  • Various approaches have been used to locate laser spots on projection surfaces, including integrating image sensors with the projection surface or with the projection device. The position of the laser spot may be detected by the image sensors and determined by a controller or computer connected to the image sensors.
  • the projection surface is an interactive tiled Organic Light Emitting Diode (“OLED”) display with an array of photosensors integrated into it for detecting the laser spot in a given tile of the display.
  • a display controller is connected to the display screen to generate a signal representing the location of the laser spot. The location is transmitted from the controller to the presentation computer, which is programmed to respond to the signal to selectively change the image being displayed.
  • Image sensors integrated into projection devices are provided, for example, in
  • the image sensors may be used to detect a laser spot generated by a laser pointer on a projection surface.
  • the position of the laser spot may be used together with pulsing signals generated at command switches on the laser pointers to emulate mouse click operations or other commands at the projection device or presentation computer.
  • a user may press a button mounted on the laser pointer to cause it to pulse at a given frequency.
  • the pulsation indicates to the projection device that the laser pointer position is to be monitored for a mouse "dragging" event until the user presses the button again to stop the pulsation.
  • Different frequencies may be used to activate different mouse events.
  • the projection devices and laser pointers available today may only be used to assist in the remote control of presentation computers. They are not able to assist in the remote control of other devices, such as video game players, DVD players, satellite receivers, and so on.
  • a projection apparatus that is capable of emulating a remote control device such as a wireless mouse without the use of command switches.
  • a projection apparatus and method for locating the position of a laser spot generated by a laser pointer and using the position to control a remote device connected to the projection apparatus are desirable.
  • a dual-mode projection apparatus has a projection module for projecting an image onto a projection surface.
  • An image sensor module captures images of the projection surface and determines spatial and temporal characteristics of a visible light spot superimposed on the projection surface.
  • a communications module transmits the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.
  • An embodiment of the invention includes an optical apparatus having a light pointer with on/off power switch for turning the light pointer on/off.
  • a projection apparatus projects an image onto the projection surface.
  • the projection apparatus has an image sensor module for capturing images of the projection surface and determining spatial and temporal characteristics of a visible light spot superimposed on the projection surface and generated by the light pointer.
  • the projection apparatus also has a communications module for transmitting the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.
  • Another embodiment of the invention includes a method for controlling a remote device with a dual-mode projection apparatus and a light pointer.
  • a first image is projected onto a projection surface with a projection module in the dual-mode projection apparatus.
  • a second image is captured from the projection surface with an image sensor module integrated into the dual-mode projection apparatus.
  • a visible light spot is detected in the second image with the image sensor module, the visible light spot generated by the light pointer. Spatial and temporal characteristics of the visible light spot are determined with the image sensor module.
  • the spatial and temporal characteristics of the visible light spot are transmitted to a remote device. The remote device is controlled based on the spatial and temporal characteristics of the visible light spot.
  • FIG. 1 illustrates a dual-mode projection apparatus constructed according to an embodiment of the invention
  • FIG. 2 illustrates an exemplary environment in which the dual-mode projection apparatus may be used according to an embodiment of the invention
  • FIG. 3 illustrates a dual-mode projection apparatus constructed according to another embodiment of the invention
  • FIG. 4 illustrates a detailed view of the dual-mode projection apparatus of
  • FIG. 1 A first figure.
  • FIG. 5 illustrates a more detailed view of the dual-mode projection apparatus of FIG. 1;
  • FIG. 6 illustrates a further detailed view of the dual-mode projection apparatus of FIG. 1;
  • FIG. 7 illustrates a flow chart for controlling a remote device with a dual- mode projection apparatus and a light pointer in accordance with an embodiment of the invention
  • FIG. 8 illustrates an exemplary environment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention
  • FIG. 9 illustrates exemplary images detected by the dual-mode projection apparatus in the exemplary environment of FIG. 8 in accordance with an embodiment of the invention
  • FIG. 10 illustrates another exemplary environment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention
  • FIG. 11 illustrates a remote device being controlled by the dual-mode projection apparatus in the exemplary environment of FIG. 10 in accordance with an embodiment of the invention
  • FIG. 12 illustrates a further exemplary embodiment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention.
  • a dual-mode projection apparatus is provided.
  • a projection device or projector may be a device for projecting digital images onto a projection surface for presentation to an audience.
  • the projection surface area may be, for example, a display screen, wall, or any other surface in which projected images become visible to the audience.
  • the digital images may be generated at a personal computer or any other image source device connected to the projection device, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others.
  • the dual-mode projection apparatus also includes an image sensor module having an image sensor and optionally a processor. The dual-mode projection apparatus is capable of both projecting images onto a projection surface and capturing images from the projection surface.
  • An image sensor may be a device or circuitry having an array of pixels for capturing and processing an optical image into electronic signals.
  • the electronic signals may be processed by the image sensor or the processor to determine a position corresponding to a visible light spot superimposed on top of the digital images at the projection surface.
  • the visible light spot may be generated by a light pointer capable of emitting a light beam onto the projection surface to highlight particular locations or items of interest on the projection surface.
  • Projection device 100 has a projection module 105, an image sensor module 110, a communications module 115, and a calibration module 120.
  • Projection module 105 includes hardware and/or software routines for projecting digital images onto projection surface 125, such as digital image 130.
  • Projection surface 125 may be a display screen, wall, or any other surface in which projected images may become visible to an audience.
  • Digital image 130 may be generated at remote device 135, which may be a personal computer or any other image source device connected to projection device 100, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others. Digital image 130 is transmitted from remote device 135 to projection device
  • Communications module 115 may be a wireless communications module including a wireless receiver for receiving digital images from remote device 135 and a wireless transmitter for transmitting information to remote device 135.
  • Communications module 115 may be, for example, a WiFi or Bluetooth wireless module.
  • communications between projection device 100 and remote device 135 may also be performed through wired connections without deviating from the scope and principles of the invention.
  • communications module 115 may be a communications module providing Ethernet communications between projection device
  • the information transmitted to remote device 135 may include, for example, spatial and temporal characteristics of a visible light spot on projection surface 125, such as light spot 140 generated by light pointer 145.
  • Light pointer 145 may be any light pointer including a light source for emitting a light beam at a given wavelength, for example, wavelengths corresponding to red, green, or blue light, to produce visible light spot 140.
  • the visible light spot 140 can appear on projection surface 125 in different patterns and shapes, such as a circular block, the shape of a butterfly, the shape of a cursor, and so on.
  • light pointer 145 may be a laser pointer including a laser for emitting a coherent light beam.
  • light pointer 145 may be a light pointer including a Light Emitting Diode (“LED”) for emitting an incoherent light beam.
  • LED Light Emitting Diode
  • the spatial and temporal characteristics may include a position of light spot
  • the position of light spot 140 may be detected as a 2D (x,y) coordinate with a corner of projection surface 125 serving as the reference (0,0) point.
  • the position of light spot 140 may be a displacement between the current position and a previous position, with the initial position determined when light pointer 145 is first powered up.
  • image sensor module 110 may be adapted to sense light only at the wavelength of light pointer 145.
  • image sensor module 110 may include an optical filter to capture only the wavelength of visible light spot 140 and reject all other optical information at other wavelengths, including digital image 130, the projection surface 125, the ambient light around projection surface 125, and so on.
  • image sensor module 110 may be able to sense light at all wavelengths. In this case, image sensor module 110 captures both the visible light spot 140 and the digital image 130 displayed on projection surface 125.
  • projection device 100 is a dual-mode projection device as it is capable of both projecting images onto projection surface 125 (with projection module 105) and capturing images from projection surface 125 (with image sensor module
  • Projection device 100 also includes calibration module 120 to align image sensor module 110 with projection module 105 and projection surface 125 so that image sensor module 110 may accurately capture the position of light spot 140 on projection surface
  • Calibration module 120 may include hardware and/or software routines for performing the alignment of image sensor module 110. The alignment may be performed automatically without any user input upon powering up projection device 100 to project images on projection surface 125.
  • Calibration module 120 may also include hardware and/or software routines for automatically calibrating projection module 105 to accurately orient the images to be displayed onto projection surface 125, such as by performing keystone correction and aligning the images to fit properly within the horizontal and vertical boundaries of projection surface 125.
  • the automatic calibration of projection module 105 may be performed in conjunction with the automatic calibration of image sensor module 110.
  • calibration module 120 determines the geometric correspondence between projection module 105, image sensor module 110 and projection surface 125 for accurately locating the positional coordinates of visible light spot 140 on projection surface 125.
  • the spatial and temporal characteristics of light spot 140 are transmitted to remote device 135 by wireless communications module 115.
  • Remote device 135 receives the spatial and temporal characteristics of light spot 140 and uses them to control one or more of its functions.
  • remote device 135 is controlled solely based on the position of light spot 140.
  • light pointer 145 does not have any switches mounted thereon to emulate mouse-click commands or to send different pulsing signals to the remote device to emulate the mouse-click commands.
  • remote device 135 may have a wireless module (not shown) capable of transmitting digital images to and receiving information from projection device 100 via communications module 115. It is also appreciated that remote device 135 may include control routines for controlling the one or more of its functions based on the position of light spot 140.
  • remote device 135 may be a computer generating images for display by projection device 100 at projection surface 125, which may be a display screen for displaying images to an audience, e.g., during a presentation.
  • a user may shine light spot 140 in the display screen to indicate to remote computer 135 that a given command is to be performed, such as a command modifying the presentation of the digital images on projection surface 125.
  • the position of light spot 140 is detected by image sensor module 110 in projection device 100 and transmitted to remote computer 135 via communications module 115.
  • the command may be, for example, a command to go to the next image of the presentation if the position of light spot 140 is in a given location of projection surface 125, such as a region corresponding to an icon indicating an image or page number of the presentation.
  • light pointer 145 may also be moved continuously across projection surface 125 to indicate to remote computer 135 that a new image of the presentation is to be displayed.
  • remote computer 135 may have a scaling routine to scale the position on projection surface 125 to a position on the image displayed on surface 125.
  • the position of light spot 140 on projection surface 125 is determined with 2D coordinates at the scale of projection surface 125.
  • remote computer 135 maps the received position into 2D coordinates at the scale of the image displayed on surface 125 with the use of the scaling routine.
  • remote computer 135 may have a pre-determined mapping from projection surfaces into presentation images. It is also appreciated that remote computer 135 may determine the mapping upon connection with projection device 100 to project the images onto the display screen.
  • light pointer 145 together with projection device 100 effectively operate as a wireless mouse associated with remote computer 135.
  • remote device 135 may be a video game player.
  • Light pointer 145 may be used as a remote control for the video game player by positioning the light spot 140 according to a desired command.
  • the video game player 135 may be an interactive action game and the light pointer 145 may be used to position a target on projection surface 125.
  • video game player 135 is directed to perform an action on the target corresponding to that position, such as, for example, shooting at or driving to the target.
  • remote device 135 may be a television appliance, for example, remote device 135 may be a satellite receiver, cable receiver, or set- top box.
  • Light pointer 145 may be used as a remote control for the television by positioning the light spot 140 on a region of the projection surface 125 indicating a channel of the television, such as, for example, an icon on projection surface 125 corresponding to the channel being displayed by projection device 100.
  • remote device 135 may be a controller connected to a network of appliances.
  • controller 135 may be a presentation computer displaying images for presentation at projection screen 125 to an audience in a presentation room. Controller 135 may be connected to a network of computers outside the presentation room. Light pointer 145 may be used as a remote control for the computers in the network, such as to direct the remote computers to perform actions connected to the presentation in the presentation room, e.g., by positioning the light spot 140 on a given image shown during the presentation to run animations on the computers. When the image sensor module 110 in projection device 100 detects that the light spot 140 is on the image and its position is transmitted to controller 135, controller 135 may send commands to the computers in the network to run the animations.
  • remote device 135 may include control routines to control one or more of its functions based on the position of light spot 140 according to each exemplary embodiment. It is also appreciated that remote device 135 may use the position of light spot 140 to control another device connected to remote device 135, such as, for example, in the exemplary embodiment of the controller described above.
  • remote device 135 may include training and adaptation routines to interpret different positions of light spots generated by light pointer 145 as different commands. For example, remote device 135 may interpret a position on the upper left corner of projection surface 125 to correspond to a page down command, a position on the lower left corner of projection surface 125 to correspond to a page up command, a position on the upper right corner of projection surface 125 to correspond to an animation command, and so on.
  • projection device 100 may be used to display images generated at image source device 200 connected to projection device 100 via communications module 115.
  • the position of visible light spot 140 may be used to control remote device 205 also connected to projection device 100 via communications module 115, as described above with reference to remote device 135 shown in FIG. 1.
  • This embodiment may be used in the case where the device to be controlled is not the device generating images for display by projection module 105.
  • remote device 205 may be a toy, game, or other appliance that may be directed to perform commands according to the position of the visible light spot 140 on projection surface 125, such as a flight simulator or other interactive game that can perform position-dependent actions based on the position of visible light spot 140 on projection surface 125.
  • remote device 135 and remote device 205 are remote in the sense that they are located away from projection surface 125 and light pointer 145.
  • Remote devices 135 and 205 may be integrated with projection device 100 in a single device, such as, for example, a video game player with an integrated projection module, without deviating from the scope and principles of the invention.
  • FIG. 3 illustrates such a dual-mode projection apparatus.
  • Dual-mode projection device 300 includes projection module 305, image sensor module 310, communications module 315, and calibration module 320, similar to dual-mode projection apparatus 100 of FIGS. 1-2.
  • Dual-mode projection device 300 also includes control module 325 for controlling the functions and operation of device 300 in response to the spatial and temporal characteristics of visible light spot 330 on digital image 335 shown on projection surface 340. Visible light spot 330 is generated by light pointer 345, which serves as a remote control for device 300.
  • device 300 may be an integrated computer/projector device.
  • Light pointer 345 may be used as a wireless mouse to control device 300, such as, for example, to control device 300 during presentations of digital images onto projection surface 340.
  • device 300 may be an integrated video player/projector device.
  • Light pointer 345 may be used as a video game console to control video games played at device 300.
  • light pointer 345 may be used to project visible light spot 330 onto projection surface 340 for the purposes of tracking or shooting at an object of a video game played at device 300.
  • device 300 may be used in other contexts to provide control for its functions based on the spatial and temporal characteristics of a visible light spot in a projection surface onto which images from device 300 are projected.
  • Image sensor module 110 in dual-mode projection device 100 includes image sensor 400.
  • Image sensor 400 may be a CMOS or CCD image sensor, or any other type of image sensor capable of capturing optical images.
  • image sensor 400 may include an optical filter for capturing only visible light spots, e.g., visible light spot 140 on projection surface 125 of FIG. 1.
  • image sensor 400 may capture light at all wavelengths.
  • Image sensor 400 may have processing capabilities embedded into it for determining the position of visible light spot 140 on projection surface 125 upon sensing the visible light spot 140 on projection surface 125. In this case, no additional processor is required to determine the position of visible light spot 140. All processing is performed directly by image sensor 400.
  • image sensor module 110 includes both image sensor 500 and processor 505.
  • Image sensor 500 and processor 505 as shown may be mounted on separate chips.
  • Image sensor 500 may be a CMOS or CCD image sensor, or any other type of image sensor capable of capturing optical images.
  • Processor 505 may be a Digital Signal Processor ("DSP"), an Image Signal Processor (“ISP”), or any other kind of processor capable of interfacing with image sensor 500.
  • DSP Digital Signal Processor
  • ISP Image Signal Processor
  • image sensor 500 captures an image of visible light spot 140 on projection surface 125 and transmits the image to processor 505 for analysis and processing.
  • Processor 505 may then execute image processing routines to detect and locate the light spot 140 on the captured image.
  • the 2D coordinates or displacement of the light spot 140 on projection surface 125 can be computed by processor 505 for remote control of a remote device, such as remote device 135 in FIG. 1 and remote device 205 in FIG. 2.
  • Image sensor module 110 in dual-mode projection apparatus 100 includes integrated image sensor/processor chip 600.
  • Integrated chip 600 has both an image sensor and a processor on the same chip.
  • the image sensor may be similar to image sensor 500 and the processor may be similar to processor 505, both shown in FIG. 5.
  • a single integrated chip 600 is used to both capture an image showing the light spot 140 and to detect the position of light spot 140 in the captured image.
  • FIG. 7 a flow chart for controlling a remote device with a dual-mode projection apparatus and a light pointer in accordance with an embodiment of the invention is described.
  • An image is projected onto a projection surface with a projection module in a dual-mode projection device, such as projection module 105 in dual-mode projection device 100.
  • An image is also captured from the projection surface with an image sensor module integrated into the dual-mode projection device, such as image sensor module 110 in dual-mode projection device 100 in step 700.
  • the projection module and the image sensor module in the dual-mode projection device are then calibrated to accurately align the image with the projection surface in step 705.
  • a light spot is then detected on the captured image in step 710. The light spot is generated by a light pointer facing the projection surface.
  • Remote control of the remote device is achieved by first determining spatial and temporal characteristics of the visible light spot on the projection surface with the image sensor module in step 715.
  • the spatial and temporal characteristics are transmitted to the remote device by a communications module in step 720, e.g., communications module 115 in dual-mode projection device 100.
  • the remote device is then controlled based on the spatial and temporal characteristics, e.g., position, velocity, acceleration, of the visible light spot on the projection surface in step 725.
  • Dual- mode projection device 800 may be used to project a digital image, e.g., digital image 805, onto projection surface 810.
  • Projection surface 810 may be a display screen to display images generated by remote computer 815.
  • Light pointer 820 may be used during the presentation to highlight locations or items of interest in digital image 805 with a visible light spot, e.g., visible light spot 825, generated by a light beam emitted by light pointer 820.
  • Light pointer 820 may be any light pointer for generating a visible light spot on a projection surface.
  • dual-mode projection apparatus [0073] According to an embodiment of the invention, dual-mode projection apparatus
  • Projection module 830 projects digital image 805 onto projection surface 810, similar to projection module 105 in projection device 100 shown in FIG. 1.
  • Image sensor module 835 detects visible light spot 825 on projection surface
  • image sensor module 835 may include an image sensor and optionally a processor, such as image sensor module 110 shown in
  • FIGS. 4-5 are identical to FIGS. 4-5.
  • the position of light spot 825 may be transmitted to remote computer 815 by communications module 840, which may be a wired or wireless communications module similar to communications module 115 in projection device 100.
  • Calibration module 845 aligns the projection module 830 with the image sensor module 835 to accurately position and sense digital image 805.
  • Remote computer 815 may use the position of light spot 825 to control one or more of its functions.
  • the remote computer 815 may use the position of light spot 825 to direct an operation to be performed on the presentation, e.g., to go to a previous or next image of the presentation, to start an animation, to superimpose another image on top of the current image 805, and so on.
  • remote computer 815 may have scaling routine 850 to map the position of light spot 825 on display screen 810 into a position at the scale of the image displayed on display screen 810. For example, a 2D coordinate (xi,yi) in the display screen 810 may correspond to a 2D coordinate (x 2 ,y 2 ) in the image being displayed. It is also appreciated that remote computer 815 may have training and adaptation routine 855 to interpret different positions of light spots generated by light pointer 820 as different commands as described above.
  • FIG. 9 shows exemplary images captured and processed by dual-mode projection apparatus 800 shown in FIG. 8.
  • Image 900 shows the optical image that the image sensor in the image sensor module 835 of projection device 800 sees when light pointer 820 projects light spot 825 onto display screen 810.
  • the image sensor in image sensor module 835 may have an optical filter to capture only the wavelength of light spot 825 resulting in the image 900 as shown.
  • the image sensor in image sensor module 835 may also be able to capture image 805 in addition to visible light spot 825.
  • the light spot 825 may be seen by the image sensor as a region 905, which may be a colorful region in the screen that appears different from the background.
  • light spot 825 may appear as a monochromatic (e.g., red, green, or blue), typically nearly round region 905 at a given diameter and luminance.
  • Image sensor module 835 processes the image 900 to detect and locate a 2D coordinate or displacement for the light spot 825.
  • the processed image 910 may include a marking such as cross 915 to indicate the position of light spot 825.
  • image sensor module 835 may include image processing routines for identifying region 905 as light spot 825 and for computing the 2D coordinate or displacement of light spot 825. These routines may include, for example, object and pattern recognition routines, edge detection routines, and other computer vision and image processing routines for differentiating the region 905 as light spot 825 from the rest of the captured image and locating its position.
  • Dual-mode projection device 1000 may be used together with light pointer 1005 as a remote control for remote computer 1010.
  • Dual-mode projection device 1000 connected to remote computer 1010 may be used to project images onto display screen 1015 during a presentation.
  • Light pointer 1005 may be used to project a light beam into light spot 1020 that, for example, superimposes icon 1025 on the image displayed on display screen 1015.
  • the image sensor module in dual-mode projection device 1000 sees the image displayed on display screen 1015 with the light spot 1020 superimposed onto icon 1025 and detects the position of light spot 1020.
  • a transmitter module in projection device 1000 transmits the position to remote computer 1010.
  • remote computer 1010 receives the position of light spot 1020 and determines that light spot 1020 is within icon 1025, remote computer 1010 automatically displays the next image of the presentation for projection onto display screen 1015 by dual-mode projection device 1000.
  • FIG. 11 illustrates the response of remote computer 1010 when controlled by projection device 1000 and light pointer 1005 shown in FIG. 10.
  • remote computer 1010 Upon receiving the position of light spot 1020 on icon 1025 indicating that a new image is to be displayed for presentation onto display screen 1015, remote computer 1010 advances to the next image of the presentation. The next image is then projected onto display screen 1015 by projector 1000 as image 1100.
  • Image 1100 includes page icon 1105 indicating the new page number 2 of the presentation displayed onto display screen 1015. A presenter may then use light pointer 1005 to highlight items of interest on image 1100, such as by pointing light pointer 1005 to display screen 1015 to generate light spot 1110.
  • light pointer 1005 together with projection device 1000 operate as a wireless mouse for remote computer 1010. It is also appreciated that remote computer 1010 may include routines for interpreting the position of light spots generated by light pointer 1005 to correspond to different mouse operations, such as select, right click, left click, drag, and so on.
  • FIG. 12 Another exemplary embodiment in which the light pointer apparatus operates in accordance with an embodiment of the invention is shown in FIG. 12.
  • projection device 1000 is used with a light pointer 1200 that doubles as a video game console for video game player 1205 connected to projection device 1000.
  • Light pointer 1200 that doubles as a video game console for video game player 1205 connected to projection device 1000.
  • video game player 1205 is connected to projection device 1000 to project image 1210 of a "shooting ducks" video game being played on video game player 1205 onto display screen 1215.
  • a user playing the video game may use light pointer 1200 to interact with the game, such as by projecting a light beam into light spot 1220 to target a duck shown on image 1210.
  • the position of light spot 1220 is detected by the image sensor module in projection device 1000 and transmitted to video game player 1205 by the communications module in projection device 1000. Upon receiving the position, video game player 1205 directs the video game to record light spot 1220 as a failed attempt by the user to hit a duck target.
  • light pointer 1200, video game player 1205, and video game image 1210 are shown for illustration purposes only. Other remote devices may be controlled with the use of the projection device of the invention without deviating from the scope and principles of the invention.
  • the dual-mode projection device of the invention enables a user to control a device remotely by simply pointing a light pointer to a desired position and detecting the position with the projection device.
  • the light pointer may be any light pointer, including those without command switches.
  • the remote device may be controlled based solely on the position of visible light spots generated by the light pointer and detected by the dual-mode projection device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Position Input By Displaying (AREA)
  • Projection Apparatus (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Transforming Electric Information Into Light Information (AREA)

Abstract

A dual-mode projection apparatus has a projection module for projecting an image onto a projection surface. An image sensor module captures images of the projection surface and determines spatial and temporal characteristics of a visible light spot superimposed on the projection surface. A communications module transmits the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.

Description

DUAL-MODE PROJECTION APPARATUS AND METHOD FOR LOCATING A LIGHT SPOT IN A PROJECTED IMAGE
BRIEF DESCRIPTION OF THE INVENTION
[0001] This invention relates generally to digital image projectors for projecting a digital image onto a display screen. More particularly, this invention relates to a dual-mode projection apparatus and method capable of both projecting and capturing an image for locating a light spot in the projected image.
BACKGROUND OF THE INVENTION
[0002] Digital image projectors are devices that project digital images onto a projection surface for presentation to an audience. The digital images are generated at a personal computer or other image source devices connected to a projector, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others. The projection surface may be a display screen, wall, or other surface in which projected images become visible to the audience.
[0003] There are several digital image projectors available today, including Liquid
Crystal Display ("LCD") projectors, Digital Light Processing ("DLP") projectors, and Liquid Crystal on Silicon ("LCoS") projectors, among others. These digital image projectors are typically used to project images onto large display screens for viewing by large audiences, such as during presentations at meetings, lectures, conferences, home theater screenings, and other venues requiring large display areas.
[0004] A desirable feature during these presentations is to have a presenter or member of the audience draw attention to particular locations or items of interest on the projection surface, such as, for example, by using a laser pointer. Laser pointers are handheld, pen- shaped devices that project a laser beam onto a surface area to generate a laser spot on the surface that is visible to the audience. Their use has become commonplace at venues where presentation materials are shown on a display screen. Most common laser pointers emit red, green, or blue light of at most 1 or 5 mWatts to avoid damage to human eyes. [0005] Laser pointers may also include command switches or buttons for enabling remote control of basic computer functions during the presentations. For example, United States Patent Publication Numbers 2007/0030460, 2006/0227107, and 2006/0197756 and United States Patent Number 6,417,840 all describe laser pointers with command switches mounted thereon to enable remote initiation of various commands at the presentation computer. The commands may include, for example, page-up and page-down commands to indicate to the presentation computer that a new image or page of presentation material is to be displayed and projected onto the projection area. In some cases, the laser spot generated by the laser pointer may function as a control cursor. The presenter may direct the control cursor to a position in the screen and emulate position-dependent commands via the switches. [0006] These position-dependent commands require that the laser spot be detected and its position determined on the projection surface. Various approaches have been used to locate laser spots on projection surfaces, including integrating image sensors with the projection surface or with the projection device. The position of the laser spot may be detected by the image sensors and determined by a controller or computer connected to the image sensors.
[0007] In United States Patent Publication Number 2004/0012565, for example, the projection surface is an interactive tiled Organic Light Emitting Diode ("OLED") display with an array of photosensors integrated into it for detecting the laser spot in a given tile of the display. A display controller is connected to the display screen to generate a signal representing the location of the laser spot. The location is transmitted from the controller to the presentation computer, which is programmed to respond to the signal to selectively change the image being displayed.
[0008] Image sensors integrated into projection devices are provided, for example, in
United States Patent Number 6,050,690 and United States Patent Publication Numbers 2007/0030460 and 2006/0170874. The image sensors may be used to detect a laser spot generated by a laser pointer on a projection surface. The position of the laser spot may be used together with pulsing signals generated at command switches on the laser pointers to emulate mouse click operations or other commands at the projection device or presentation computer. In one example, a user may press a button mounted on the laser pointer to cause it to pulse at a given frequency. The pulsation indicates to the projection device that the laser pointer position is to be monitored for a mouse "dragging" event until the user presses the button again to stop the pulsation. Different frequencies may be used to activate different mouse events.
[0009] These and other currently-available projection devices either require a custom- based projection surface (such as the tiled OLED display screen described above) or a laser pointer with command switches to provide position-dependent commands to a presentation device. Using command switches on laser pointers to emulate mouse commands may, at best, be cumbersome to the user operating the laser pointer. The user must change focus from giving the presentation to operating the switches. The switches may be prone to error or fail entirely during a presentation.
[0010] Furthermore, the projection devices and laser pointers available today may only be used to assist in the remote control of presentation computers. They are not able to assist in the remote control of other devices, such as video game players, DVD players, satellite receivers, and so on.
[0011] Accordingly, it would be desirable to provide a projection apparatus that is capable of emulating a remote control device such as a wireless mouse without the use of command switches. In particular, it would be desirable to provide a projection apparatus and method for locating the position of a laser spot generated by a laser pointer and using the position to control a remote device connected to the projection apparatus.
SUMMARY OF THE INVENTION
[0012] A dual-mode projection apparatus has a projection module for projecting an image onto a projection surface. An image sensor module captures images of the projection surface and determines spatial and temporal characteristics of a visible light spot superimposed on the projection surface. A communications module transmits the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot. [0013] An embodiment of the invention includes an optical apparatus having a light pointer with on/off power switch for turning the light pointer on/off. A projection apparatus projects an image onto the projection surface. The projection apparatus has an image sensor module for capturing images of the projection surface and determining spatial and temporal characteristics of a visible light spot superimposed on the projection surface and generated by the light pointer. The projection apparatus also has a communications module for transmitting the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.
[0014] Another embodiment of the invention includes a method for controlling a remote device with a dual-mode projection apparatus and a light pointer. A first image is projected onto a projection surface with a projection module in the dual-mode projection apparatus. A second image is captured from the projection surface with an image sensor module integrated into the dual-mode projection apparatus. A visible light spot is detected in the second image with the image sensor module, the visible light spot generated by the light pointer. Spatial and temporal characteristics of the visible light spot are determined with the image sensor module. The spatial and temporal characteristics of the visible light spot are transmitted to a remote device. The remote device is controlled based on the spatial and temporal characteristics of the visible light spot.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The invention is more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
[0016] FIG. 1 illustrates a dual-mode projection apparatus constructed according to an embodiment of the invention;
[0017] FIG. 2 illustrates an exemplary environment in which the dual-mode projection apparatus may be used according to an embodiment of the invention; [0018] FIG. 3 illustrates a dual-mode projection apparatus constructed according to another embodiment of the invention;
[0019] FIG. 4 illustrates a detailed view of the dual-mode projection apparatus of
FIG. 1;
[0020] FIG. 5 illustrates a more detailed view of the dual-mode projection apparatus of FIG. 1;
[0021] FIG. 6 illustrates a further detailed view of the dual-mode projection apparatus of FIG. 1;
[0022] FIG. 7 illustrates a flow chart for controlling a remote device with a dual- mode projection apparatus and a light pointer in accordance with an embodiment of the invention;
[0023] FIG. 8 illustrates an exemplary environment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention; [0024] FIG. 9 illustrates exemplary images detected by the dual-mode projection apparatus in the exemplary environment of FIG. 8 in accordance with an embodiment of the invention;
[0025] FIG. 10 illustrates another exemplary environment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention; [0026] FIG. 11 illustrates a remote device being controlled by the dual-mode projection apparatus in the exemplary environment of FIG. 10 in accordance with an embodiment of the invention; and
[0027] FIG. 12 illustrates a further exemplary embodiment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0028] A dual-mode projection apparatus is provided. As generally used herein, a projection device or projector may be a device for projecting digital images onto a projection surface for presentation to an audience. The projection surface area may be, for example, a display screen, wall, or any other surface in which projected images become visible to the audience. The digital images may be generated at a personal computer or any other image source device connected to the projection device, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others. [0029] The dual-mode projection apparatus also includes an image sensor module having an image sensor and optionally a processor. The dual-mode projection apparatus is capable of both projecting images onto a projection surface and capturing images from the projection surface. An image sensor, as generally used herein, may be a device or circuitry having an array of pixels for capturing and processing an optical image into electronic signals. The electronic signals may be processed by the image sensor or the processor to determine a position corresponding to a visible light spot superimposed on top of the digital images at the projection surface. The visible light spot may be generated by a light pointer capable of emitting a light beam onto the projection surface to highlight particular locations or items of interest on the projection surface.
[0030] A dual-mode projection apparatus constructed according to an embodiment of the invention is illustrated in FIG. 1. Projection device 100 has a projection module 105, an image sensor module 110, a communications module 115, and a calibration module 120. Projection module 105 includes hardware and/or software routines for projecting digital images onto projection surface 125, such as digital image 130. Projection surface 125 may be a display screen, wall, or any other surface in which projected images may become visible to an audience.
[0031] Digital image 130 may be generated at remote device 135, which may be a personal computer or any other image source device connected to projection device 100, such as DVD players, DVRs, VCRs, satellite receivers, cable systems, and video game players, among others. Digital image 130 is transmitted from remote device 135 to projection device
100 via a wireless link through a wireless module (not shown) on remote device 135 and communications module 115 on projection device 100.
[0032] Communications module 115 may be a wireless communications module including a wireless receiver for receiving digital images from remote device 135 and a wireless transmitter for transmitting information to remote device 135. Communications module 115 may be, for example, a WiFi or Bluetooth wireless module.
[0033] It is appreciated that communications between projection device 100 and remote device 135 may also be performed through wired connections without deviating from the scope and principles of the invention. For example, communications module 115 may be a communications module providing Ethernet communications between projection device
100 and remote device 135.
[0034] The information transmitted to remote device 135 may include, for example, spatial and temporal characteristics of a visible light spot on projection surface 125, such as light spot 140 generated by light pointer 145. Light pointer 145 may be any light pointer including a light source for emitting a light beam at a given wavelength, for example, wavelengths corresponding to red, green, or blue light, to produce visible light spot 140. The visible light spot 140 can appear on projection surface 125 in different patterns and shapes, such as a circular block, the shape of a butterfly, the shape of a cursor, and so on.
[0035] In one embodiment, light pointer 145 may be a laser pointer including a laser for emitting a coherent light beam. In another embodiment, light pointer 145 may be a light pointer including a Light Emitting Diode ("LED") for emitting an incoherent light beam.
Other light sources for emitting light beams and producing a visible light spot on projection surface 125 are also within the scope and principles of the invention.
[0036] According to an embodiment of the invention, when light pointer 145, facing projection surface 125, projects a light beam onto it to produce visible light spot 140, light is reflected from projection surface 125 and sensed by image sensor module 110 in projection device 100. Image sensor module 110 sees the projection surface 125 and the visible light spot 140 thereon and captures an image of the projection surface 125 to determine spatial and temporal characteristics of light spot 140.
[0037] The spatial and temporal characteristics may include a position of light spot
140 on projection surface 125, a velocity and an acceleration of light spot 140 as it moves across projection surface 125, or a sequence of positions of light spot 140 over a sequence of images captured by image sensor module 110 over time, among other characteristics. These characteristics may be computed by processing and analyzing the captured images.
[0038] In one exemplary embodiment, the position of light spot 140 may be detected as a 2D (x,y) coordinate with a corner of projection surface 125 serving as the reference (0,0) point. Alternatively, the position of light spot 140 may be a displacement between the current position and a previous position, with the initial position determined when light pointer 145 is first powered up.
[0039] It is appreciated that image sensor module 110 may be adapted to sense light only at the wavelength of light pointer 145. For example, image sensor module 110 may include an optical filter to capture only the wavelength of visible light spot 140 and reject all other optical information at other wavelengths, including digital image 130, the projection surface 125, the ambient light around projection surface 125, and so on. It is also appreciated that image sensor module 110 may be able to sense light at all wavelengths. In this case, image sensor module 110 captures both the visible light spot 140 and the digital image 130 displayed on projection surface 125.
[0040] Further, it is appreciated that projection device 100 is a dual-mode projection device as it is capable of both projecting images onto projection surface 125 (with projection module 105) and capturing images from projection surface 125 (with image sensor module
110).
[0041] Projection device 100 also includes calibration module 120 to align image sensor module 110 with projection module 105 and projection surface 125 so that image sensor module 110 may accurately capture the position of light spot 140 on projection surface
125. Calibration module 120 may include hardware and/or software routines for performing the alignment of image sensor module 110. The alignment may be performed automatically without any user input upon powering up projection device 100 to project images on projection surface 125.
[0042] Calibration module 120 may also include hardware and/or software routines for automatically calibrating projection module 105 to accurately orient the images to be displayed onto projection surface 125, such as by performing keystone correction and aligning the images to fit properly within the horizontal and vertical boundaries of projection surface 125.
[0043] In one embodiment, the automatic calibration of projection module 105 may be performed in conjunction with the automatic calibration of image sensor module 110.
This may be achieved by having image sensor module 110 capture the image displayed by projection module 105 onto projection surface 125 and use the captured image to align both image sensor module 110 and projection module 105 to accurately display the image onto projection surface 125. In doing so, calibration module 120 determines the geometric correspondence between projection module 105, image sensor module 110 and projection surface 125 for accurately locating the positional coordinates of visible light spot 140 on projection surface 125.
[0044] The spatial and temporal characteristics of light spot 140 are transmitted to remote device 135 by wireless communications module 115. Remote device 135 receives the spatial and temporal characteristics of light spot 140 and uses them to control one or more of its functions. According to an embodiment of the invention, remote device 135 is controlled solely based on the position of light spot 140. In this case, unlike currently-available light pointers used in conjunction with projection devices to control a presentation computer, light pointer 145 does not have any switches mounted thereon to emulate mouse-click commands or to send different pulsing signals to the remote device to emulate the mouse-click commands.
[0045] It is appreciated that remote device 135 may have a wireless module (not shown) capable of transmitting digital images to and receiving information from projection device 100 via communications module 115. It is also appreciated that remote device 135 may include control routines for controlling the one or more of its functions based on the position of light spot 140.
[0046] In one exemplary embodiment, remote device 135 may be a computer generating images for display by projection device 100 at projection surface 125, which may be a display screen for displaying images to an audience, e.g., during a presentation. A user may shine light spot 140 in the display screen to indicate to remote computer 135 that a given command is to be performed, such as a command modifying the presentation of the digital images on projection surface 125. The position of light spot 140 is detected by image sensor module 110 in projection device 100 and transmitted to remote computer 135 via communications module 115.
[0047] The command may be, for example, a command to go to the next image of the presentation if the position of light spot 140 is in a given location of projection surface 125, such as a region corresponding to an icon indicating an image or page number of the presentation. Alternatively, light pointer 145 may also be moved continuously across projection surface 125 to indicate to remote computer 135 that a new image of the presentation is to be displayed. [0048] In this exemplary embodiment, remote computer 135 may have a scaling routine to scale the position on projection surface 125 to a position on the image displayed on surface 125. The position of light spot 140 on projection surface 125 is determined with 2D coordinates at the scale of projection surface 125. Upon receiving the position of light spot 140, remote computer 135 maps the received position into 2D coordinates at the scale of the image displayed on surface 125 with the use of the scaling routine.
[0049] It is appreciated that remote computer 135 may have a pre-determined mapping from projection surfaces into presentation images. It is also appreciated that remote computer 135 may determine the mapping upon connection with projection device 100 to project the images onto the display screen.
[0050] Further, it is appreciated that positioning the light spot 140 on the icon or moving the light pointer 145 continuously has the same effect as clicking a button on a mouse associated with remote computer 135 to go to the next image of the presentation. In this exemplary embodiment, light pointer 145 together with projection device 100 effectively operate as a wireless mouse associated with remote computer 135.
[0051] In another exemplary embodiment, remote device 135 may be a video game player. Light pointer 145 may be used as a remote control for the video game player by positioning the light spot 140 according to a desired command. For example, the video game player 135 may be an interactive action game and the light pointer 145 may be used to position a target on projection surface 125. When the position of the light spot 140 is detected by image sensor module 110 in projection device 100 and transmitted to video game player 135 by communications module 115, video game player 135 is directed to perform an action on the target corresponding to that position, such as, for example, shooting at or driving to the target.
[0052] In yet another exemplary embodiment, remote device 135 may be a television appliance, for example, remote device 135 may be a satellite receiver, cable receiver, or set- top box. Light pointer 145 may be used as a remote control for the television by positioning the light spot 140 on a region of the projection surface 125 indicating a channel of the television, such as, for example, an icon on projection surface 125 corresponding to the channel being displayed by projection device 100. When the position of light spot 140 is detected by image sensor module 110 in projection device 100 and transmitted to the television appliance 135 by communications module 115, television appliance 135 is directed to change the television channel. [0053] In a further exemplary embodiment, remote device 135 may be a controller connected to a network of appliances. For example, controller 135 may be a presentation computer displaying images for presentation at projection screen 125 to an audience in a presentation room. Controller 135 may be connected to a network of computers outside the presentation room. Light pointer 145 may be used as a remote control for the computers in the network, such as to direct the remote computers to perform actions connected to the presentation in the presentation room, e.g., by positioning the light spot 140 on a given image shown during the presentation to run animations on the computers. When the image sensor module 110 in projection device 100 detects that the light spot 140 is on the image and its position is transmitted to controller 135, controller 135 may send commands to the computers in the network to run the animations.
[0054] It is appreciated that these exemplary embodiments are given for purposes of illustration only and additional exemplary embodiments are within the scope of the invention. It is also appreciated that remote device 135 may include control routines to control one or more of its functions based on the position of light spot 140 according to each exemplary embodiment. It is also appreciated that remote device 135 may use the position of light spot 140 to control another device connected to remote device 135, such as, for example, in the exemplary embodiment of the controller described above.
[0055] It is further appreciated that remote device 135 may include training and adaptation routines to interpret different positions of light spots generated by light pointer 145 as different commands. For example, remote device 135 may interpret a position on the upper left corner of projection surface 125 to correspond to a page down command, a position on the lower left corner of projection surface 125 to correspond to a page up command, a position on the upper right corner of projection surface 125 to correspond to an animation command, and so on.
[0056] Another exemplary embodiment for using the dual-mode projection apparatus is illustrated in FIG. 2. In this embodiment, projection device 100 may be used to display images generated at image source device 200 connected to projection device 100 via communications module 115. The position of visible light spot 140 may be used to control remote device 205 also connected to projection device 100 via communications module 115, as described above with reference to remote device 135 shown in FIG. 1. [0057] This embodiment may be used in the case where the device to be controlled is not the device generating images for display by projection module 105. For example, remote device 205 may be a toy, game, or other appliance that may be directed to perform commands according to the position of the visible light spot 140 on projection surface 125, such as a flight simulator or other interactive game that can perform position-dependent actions based on the position of visible light spot 140 on projection surface 125.
[0058] It is appreciated that remote device 135 and remote device 205 are remote in the sense that they are located away from projection surface 125 and light pointer 145. Remote devices 135 and 205 may be integrated with projection device 100 in a single device, such as, for example, a video game player with an integrated projection module, without deviating from the scope and principles of the invention.
[0059] FIG. 3 illustrates such a dual-mode projection apparatus. Dual-mode projection device 300 includes projection module 305, image sensor module 310, communications module 315, and calibration module 320, similar to dual-mode projection apparatus 100 of FIGS. 1-2. Dual-mode projection device 300 also includes control module 325 for controlling the functions and operation of device 300 in response to the spatial and temporal characteristics of visible light spot 330 on digital image 335 shown on projection surface 340. Visible light spot 330 is generated by light pointer 345, which serves as a remote control for device 300.
[0060] In one exemplary embodiment, device 300 may be an integrated computer/projector device. Light pointer 345 may be used as a wireless mouse to control device 300, such as, for example, to control device 300 during presentations of digital images onto projection surface 340. In another exemplary embodiment, device 300 may be an integrated video player/projector device. Light pointer 345 may be used as a video game console to control video games played at device 300. For example, light pointer 345 may be used to project visible light spot 330 onto projection surface 340 for the purposes of tracking or shooting at an object of a video game played at device 300.
[0061] It is appreciated that these exemplary uses of device 300 are provided for illustration purposes only. Device 300 may be used in other contexts to provide control for its functions based on the spatial and temporal characteristics of a visible light spot in a projection surface onto which images from device 300 are projected.
[0062] A more detailed view of a dual -mode projection apparatus constructed according to an embodiment of the invention is shown in FIG. 4. Image sensor module 110 in dual-mode projection device 100 includes image sensor 400. Image sensor 400 may be a CMOS or CCD image sensor, or any other type of image sensor capable of capturing optical images. [0063] In one exemplary embodiment, image sensor 400 may include an optical filter for capturing only visible light spots, e.g., visible light spot 140 on projection surface 125 of FIG. 1. In another exemplary embodiment, image sensor 400 may capture light at all wavelengths.
[0064] Image sensor 400 may have processing capabilities embedded into it for determining the position of visible light spot 140 on projection surface 125 upon sensing the visible light spot 140 on projection surface 125. In this case, no additional processor is required to determine the position of visible light spot 140. All processing is performed directly by image sensor 400.
[0065] Another detailed view of the dual-mode projection apparatus constructed according to an embodiment of the invention is shown in FIG. 5. In this case, image sensor module 110 includes both image sensor 500 and processor 505. Image sensor 500 and processor 505 as shown may be mounted on separate chips. Image sensor 500 may be a CMOS or CCD image sensor, or any other type of image sensor capable of capturing optical images. Processor 505 may be a Digital Signal Processor ("DSP"), an Image Signal Processor ("ISP"), or any other kind of processor capable of interfacing with image sensor 500.
[0066] In one exemplary embodiment, image sensor 500 captures an image of visible light spot 140 on projection surface 125 and transmits the image to processor 505 for analysis and processing. Processor 505 may then execute image processing routines to detect and locate the light spot 140 on the captured image. As described above, the 2D coordinates or displacement of the light spot 140 on projection surface 125 can be computed by processor 505 for remote control of a remote device, such as remote device 135 in FIG. 1 and remote device 205 in FIG. 2.
[0067] A further detailed view of a dual-mode projection apparatus constructed according to an embodiment of the invention is illustrated in FIG. 6. Image sensor module 110 in dual-mode projection apparatus 100 includes integrated image sensor/processor chip 600. Integrated chip 600 has both an image sensor and a processor on the same chip. The image sensor may be similar to image sensor 500 and the processor may be similar to processor 505, both shown in FIG. 5. In this exemplary embodiment, a single integrated chip 600 is used to both capture an image showing the light spot 140 and to detect the position of light spot 140 in the captured image.
[0068] Referring now to FIG. 7, a flow chart for controlling a remote device with a dual-mode projection apparatus and a light pointer in accordance with an embodiment of the invention is described. An image is projected onto a projection surface with a projection module in a dual-mode projection device, such as projection module 105 in dual-mode projection device 100. An image is also captured from the projection surface with an image sensor module integrated into the dual-mode projection device, such as image sensor module 110 in dual-mode projection device 100 in step 700. The projection module and the image sensor module in the dual-mode projection device are then calibrated to accurately align the image with the projection surface in step 705. A light spot is then detected on the captured image in step 710. The light spot is generated by a light pointer facing the projection surface. [0069] Remote control of the remote device is achieved by first determining spatial and temporal characteristics of the visible light spot on the projection surface with the image sensor module in step 715. The spatial and temporal characteristics are transmitted to the remote device by a communications module in step 720, e.g., communications module 115 in dual-mode projection device 100. The remote device is then controlled based on the spatial and temporal characteristics, e.g., position, velocity, acceleration, of the visible light spot on the projection surface in step 725.
[0070] It is appreciated that in case the projector apparatus is integrated into the remote device, such as in case of device 300 of FIG. 3, the spatial and temporal characteristics are sent to a control module in the integrated device to control its functions. [0071] An exemplary environment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention is illustrated in FIG. 8. Dual- mode projection device 800 may be used to project a digital image, e.g., digital image 805, onto projection surface 810. Projection surface 810 may be a display screen to display images generated by remote computer 815.
[0072] Light pointer 820 may be used during the presentation to highlight locations or items of interest in digital image 805 with a visible light spot, e.g., visible light spot 825, generated by a light beam emitted by light pointer 820. Light pointer 820 may be any light pointer for generating a visible light spot on a projection surface.
[0073] According to an embodiment of the invention, dual-mode projection apparatus
800 has a projection module 830, an image sensor module 835, a communications module 840, and a calibration module 845. Projection module 830 projects digital image 805 onto projection surface 810, similar to projection module 105 in projection device 100 shown in FIG. 1.
[0074] Image sensor module 835 detects visible light spot 825 on projection surface
810 and determines its position. As described above, image sensor module 835 may include an image sensor and optionally a processor, such as image sensor module 110 shown in
FIGS. 4-5.
[0075] The position of light spot 825 may be transmitted to remote computer 815 by communications module 840, which may be a wired or wireless communications module similar to communications module 115 in projection device 100. Calibration module 845 aligns the projection module 830 with the image sensor module 835 to accurately position and sense digital image 805.
[0076] Remote computer 815 may use the position of light spot 825 to control one or more of its functions. For example, the remote computer 815 may use the position of light spot 825 to direct an operation to be performed on the presentation, e.g., to go to a previous or next image of the presentation, to start an animation, to superimpose another image on top of the current image 805, and so on.
[0077] It is appreciated that remote computer 815 may have scaling routine 850 to map the position of light spot 825 on display screen 810 into a position at the scale of the image displayed on display screen 810. For example, a 2D coordinate (xi,yi) in the display screen 810 may correspond to a 2D coordinate (x2,y2) in the image being displayed. It is also appreciated that remote computer 815 may have training and adaptation routine 855 to interpret different positions of light spots generated by light pointer 820 as different commands as described above.
[0078] FIG. 9 shows exemplary images captured and processed by dual-mode projection apparatus 800 shown in FIG. 8. Image 900 shows the optical image that the image sensor in the image sensor module 835 of projection device 800 sees when light pointer 820 projects light spot 825 onto display screen 810.
[0079] In one embodiment, the image sensor in image sensor module 835 may have an optical filter to capture only the wavelength of light spot 825 resulting in the image 900 as shown. In another embodiment, the image sensor in image sensor module 835 may also be able to capture image 805 in addition to visible light spot 825. The light spot 825 may be seen by the image sensor as a region 905, which may be a colorful region in the screen that appears different from the background.
[0080] For example, light spot 825 may appear as a monochromatic (e.g., red, green, or blue), typically nearly round region 905 at a given diameter and luminance. Image sensor module 835 processes the image 900 to detect and locate a 2D coordinate or displacement for the light spot 825. The processed image 910 may include a marking such as cross 915 to indicate the position of light spot 825. [0081] It is appreciated that image sensor module 835 may include image processing routines for identifying region 905 as light spot 825 and for computing the 2D coordinate or displacement of light spot 825. These routines may include, for example, object and pattern recognition routines, edge detection routines, and other computer vision and image processing routines for differentiating the region 905 as light spot 825 from the rest of the captured image and locating its position.
[0082] Another exemplary embodiment in which the dual-mode projection apparatus operates in accordance with an embodiment of the invention is shown in FIG. 10. Dual-mode projection device 1000 may be used together with light pointer 1005 as a remote control for remote computer 1010. Dual-mode projection device 1000 connected to remote computer 1010 may be used to project images onto display screen 1015 during a presentation. Light pointer 1005 may be used to project a light beam into light spot 1020 that, for example, superimposes icon 1025 on the image displayed on display screen 1015. [0083] The image sensor module in dual-mode projection device 1000 sees the image displayed on display screen 1015 with the light spot 1020 superimposed onto icon 1025 and detects the position of light spot 1020. A transmitter module in projection device 1000 transmits the position to remote computer 1010. When remote computer 1010 receives the position of light spot 1020 and determines that light spot 1020 is within icon 1025, remote computer 1010 automatically displays the next image of the presentation for projection onto display screen 1015 by dual-mode projection device 1000.
[0084] FIG. 11 illustrates the response of remote computer 1010 when controlled by projection device 1000 and light pointer 1005 shown in FIG. 10. Upon receiving the position of light spot 1020 on icon 1025 indicating that a new image is to be displayed for presentation onto display screen 1015, remote computer 1010 advances to the next image of the presentation. The next image is then projected onto display screen 1015 by projector 1000 as image 1100.
[0085] Image 1100 includes page icon 1105 indicating the new page number 2 of the presentation displayed onto display screen 1015. A presenter may then use light pointer 1005 to highlight items of interest on image 1100, such as by pointing light pointer 1005 to display screen 1015 to generate light spot 1110.
[0086] It is appreciated that light pointer 1005 together with projection device 1000 operate as a wireless mouse for remote computer 1010. It is also appreciated that remote computer 1010 may include routines for interpreting the position of light spots generated by light pointer 1005 to correspond to different mouse operations, such as select, right click, left click, drag, and so on.
[0087] Another exemplary embodiment in which the light pointer apparatus operates in accordance with an embodiment of the invention is shown in FIG. 12. In this exemplary embodiment, projection device 1000 is used with a light pointer 1200 that doubles as a video game console for video game player 1205 connected to projection device 1000. Light pointer
1200 together with projection device 1000 are used as a remote control for video game player
1105.
[0088] In the example shown, video game player 1205 is connected to projection device 1000 to project image 1210 of a "shooting ducks" video game being played on video game player 1205 onto display screen 1215. A user playing the video game may use light pointer 1200 to interact with the game, such as by projecting a light beam into light spot 1220 to target a duck shown on image 1210.
[0089] The position of light spot 1220 is detected by the image sensor module in projection device 1000 and transmitted to video game player 1205 by the communications module in projection device 1000. Upon receiving the position, video game player 1205 directs the video game to record light spot 1220 as a failed attempt by the user to hit a duck target.
[0090] It is appreciated that light pointer 1200, video game player 1205, and video game image 1210 are shown for illustration purposes only. Other remote devices may be controlled with the use of the projection device of the invention without deviating from the scope and principles of the invention.
[0091] Advantageously, the dual-mode projection device of the invention enables a user to control a device remotely by simply pointing a light pointer to a desired position and detecting the position with the projection device. The light pointer may be any light pointer, including those without command switches. The remote device may be controlled based solely on the position of visible light spots generated by the light pointer and detected by the dual-mode projection device.
[0092] The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practice the invention. Thus, the foregoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed; obviously, many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications; they thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.

Claims

WHAT IS CLAIMED IS:
1. A dual-mode projection apparatus, comprising: a projection module for projecting an image onto a projection surface; an image sensor module for capturing images of the projection surface and determining spatial and temporal characteristics of a visible light spot superimposed on the projection surface; and a communications module for transmitting the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.
2. The dual-mode projection apparatus of claim 1, wherein the spatial and temporal characteristics are selected from the list comprising: a position of the visible light spot on the projection surface; a velocity of the visible light spot on the projection surface; an acceleration of the visible light spot on the projection surface; and a sequence of positions of the visible light spot on the projection surface over a sequence of images projected onto the projection surface.
3. The dual-mode projection apparatus of claim 2, wherein the remote device is controlled based solely on the position of the visible light spot.
4. The dual-mode projection apparatus of claim 1, further comprising a calibration module for aligning the image sensor module with the projection module.
5. The dual-mode projection apparatus of claim 1, wherein the image sensor module comprises an image sensor.
6. The dual-mode projection apparatus of claim 5, wherein the image sensor is adapted to sense light only at a given wavelength.
7. The dual-mode projection apparatus of claim 5, wherein the image sensor module further comprises a processor.
8. The dual-mode projection apparatus of claim 1, wherein the communications module comprises a wireless communications module.
9. The dual-mode projection apparatus of claim 8, wherein the communications module comprises a receiver for receiving the image from the remote device.
10. The dual-mode projection apparatus of claim 1, wherein the position comprises a displacement position.
11. The dual-mode projection apparatus of claim 10, wherein the displacement position is determined with reference to a reference point in the projection surface.
12. The dual-mode projection apparatus of claim 10, wherein the displacement position is determined with reference to a previous position of the visible light spot.
13. The dual-mode projection apparatus of claim 1, wherein the remote device is selected from the list comprising: a presentation computer; a video game player; a DVD player; a digital video recorder; a set-top box; a cable receiver; a satellite receiver; an entertainment unit; a controller; and a television.
14. An optical apparatus, comprising: a light pointer having an on/off power switch for turning the light pointer on/off; and a projection apparatus for projecting an image onto a projection surface, the projection apparatus comprising: an image sensor module for capturing images of the projection surface and determining spatial and temporal characteristics of a visible light spot superimposed on the projection surface and generated by the light pointer; and a communications module for transmitting the spatial and temporal characteristics of the visible light spot to a remote device for remote control of the device based on the spatial and temporal characteristics of the visible light spot.
15. The optical apparatus of claim 14, wherein the projection apparatus comprises a projection module for projecting the image onto the projection surface.
16. The optical apparatus of claim 15, further comprising a calibration module for aligning the image sensor module with the projection module.
17. The optical apparatus of claim 14, wherein the image sensor module comprises an image sensor.
18. The optical apparatus of claim 17, wherein the image sensor module further comprises a processor.
19. The optical apparatus of claim 18, wherein the image sensor and the processor are mounted on separate chips.
20. The optical apparatus of claim 18, wherein the image sensor and the processor are integrated in a single chip.
21. A method for controlling a remote device with a dual-mode projection apparatus and a light pointer, comprising: projecting a first image onto a projection surface with a projection module in the dual- mode projection apparatus; capturing a second image of the projection surface with an image sensor module integrated into the dual-mode projection apparatus; detecting a visible light spot in the second image with the image sensor module, the visible light spot generated by the light pointer; determining spatial and temporal characteristics of the visible light spot with the image sensor module; transmitting the spatial and temporal characteristics of the visible light spot to a remote device; and controlling the remote device based on the spatial and temporal characteristics of the visible light spot.
22. The method of claim 21 , further comprising calibrating the image sensor module with the projection module.
23. The method of claim 21, further comprising mapping the position of the visible light spot to a position in the first image.
24. A device, comprising: a projection module for projecting an image onto a projection surface; an image sensor module for capturing images of the projection surface and determining spatial and temporal characteristics of a visible light spot superimposed on the projection surface; and a control module for controlling the device based on the spatial and temporal characteristics of the visible light spot.
25. The device of claim 24, wherein the device is selected from the list comprising: a computer; a video game player; a DVD player; a digital video recorder; a set-top box; a cable receiver; a satellite receiver; an entertainment unit; a controller; and a television.
PCT/US2008/080906 2007-11-07 2008-10-23 Dual-mode projection apparatus and method for locating a light spot in a projected image WO2009061620A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP08847625A EP2218252A4 (en) 2007-11-07 2008-10-23 Dual-mode projection apparatus and method for locating a light spot in a projected image
CN2008801232795A CN101919241B (en) 2007-11-07 2008-10-23 Dual-mode projection apparatus and method for locating a light spot in a projected image

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US11/936,746 US7862179B2 (en) 2007-11-07 2007-11-07 Dual-mode projection apparatus and method for locating a light spot in a projected image
US11/936,737 2007-11-07
US11/936,746 2007-11-07
US11/936,737 US8188973B2 (en) 2007-11-07 2007-11-07 Apparatus and method for tracking a light pointer

Publications (1)

Publication Number Publication Date
WO2009061620A1 true WO2009061620A1 (en) 2009-05-14

Family

ID=40626122

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2008/080902 WO2009061619A2 (en) 2007-11-07 2008-10-23 Apparatus and method for tracking a light pointer
PCT/US2008/080906 WO2009061620A1 (en) 2007-11-07 2008-10-23 Dual-mode projection apparatus and method for locating a light spot in a projected image

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/US2008/080902 WO2009061619A2 (en) 2007-11-07 2008-10-23 Apparatus and method for tracking a light pointer

Country Status (4)

Country Link
EP (2) EP2208112A4 (en)
CN (2) CN101896867B (en)
TW (2) TWI400554B (en)
WO (2) WO2009061619A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9423278B1 (en) 2015-03-09 2016-08-23 Laser Projection Technologies, Inc. 3D laser projection, scanning and object tracking

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI400556B (en) * 2009-11-30 2013-07-01 Pegatron Corp Multimedia projecting device and controlling method thereof
HK1147897A2 (en) * 2011-02-14 2011-08-19 Centek Internat Hk Ltd Game apparatus and method of use thereof
CN102509068A (en) * 2011-10-09 2012-06-20 海信集团有限公司 Method and device for projecting
CN102682637A (en) * 2012-05-05 2012-09-19 山东长征教育科技有限公司 Omnibearing interactive wired teaching system
CN102915126B (en) * 2012-09-29 2016-03-02 深圳创维数字技术有限公司 A kind of method, Apparatus and system of light ray remote-control location
CN103595469A (en) * 2013-02-02 2014-02-19 彭海涛 Infrared photoconduction remote control system
US9002112B2 (en) * 2013-08-27 2015-04-07 Trimble Navigation Limited Video alignment system
JP2015158644A (en) * 2014-02-25 2015-09-03 カシオ計算機株式会社 Projection device, projection method, and program
CN104142739B (en) * 2014-08-12 2017-05-17 广东顺德中山大学卡内基梅隆大学国际联合研究院 Laser point tracking system and method based on optical mouse sensing array
TWI684058B (en) * 2016-09-13 2020-02-01 揚明光學股份有限公司 Projection system
CN106610490A (en) * 2016-12-30 2017-05-03 北京大学 Optical positioning method, system and device based on LED and image sensor
JP6941976B2 (en) * 2017-06-06 2021-09-29 株式会社コーエーテクモゲームス User interface processing program, recording medium, user interface processing method
DE102017010079A1 (en) * 2017-10-30 2019-05-02 Michael Kaiser Device with an object and its control
TWI701497B (en) * 2018-10-17 2020-08-11 英錡科技股份有限公司 Projection device with auto-correction of spots
CN112702586B (en) * 2020-12-21 2023-06-30 极米科技股份有限公司 Virtual touch tracking method, device and system for projector based on visible light
CN113440841B (en) * 2021-07-14 2023-11-17 网易（杭州）网络有限公司 Virtual character control method and device, electronic equipment and readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040041786A1 (en) 2002-08-30 2004-03-04 Casio Computer Co., Ltd. Pointed position detection device and pointed position detection method
US20060007142A1 (en) * 2003-06-13 2006-01-12 Microsoft Corporation Pointing device and cursor for use in intelligent computing environments
US20060197742A1 (en) * 2005-03-04 2006-09-07 Gray Robert H Iii Computer pointing input device
KR20070052160A (en) * 2005-11-16 2007-05-21 한양대학교 산학협력단 Mobile communication terminal having laser pointer and presentation providing system using same

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5477284A (en) * 1994-12-15 1995-12-19 Texas Instruments Incorporated Dual mode overhead projection system using field emission device
JPH0980372A (en) * 1995-09-07 1997-03-28 Toshiba Corp Projection display device
US6050690A (en) * 1998-01-08 2000-04-18 Siemens Information And Communication Networks, Inc. Apparatus and method for focusing a projected image
JP2000276297A (en) * 1999-03-25 2000-10-06 Seiko Epson Corp Pointing position detecting device, pointing position detecting method, presentation system, and information storage medium
US6275214B1 (en) * 1999-07-06 2001-08-14 Karl C. Hansen Computer presentation system and method with optical tracking of wireless pointer
US20010010514A1 (en) * 1999-09-07 2001-08-02 Yukinobu Ishino Position detector and attitude detector
US20010030668A1 (en) * 2000-01-10 2001-10-18 Gamze Erten Method and system for interacting with a display
US6704000B2 (en) * 2000-11-15 2004-03-09 Blue Iris Technologies Method for remote computer operation via a wireless optical device
US6736517B2 (en) * 2001-02-28 2004-05-18 Principia Lightworks Inc. Dual mode laser projection system for electronic and film images
TW522283B (en) * 2002-06-27 2003-03-01 Avermedia Tech Inc A boundary indicator of a projector
US6979087B2 (en) * 2002-10-31 2005-12-27 Hewlett-Packard Development Company, L.P. Display system with interpretable pattern detection
US6805451B1 (en) * 2003-11-12 2004-10-19 Cinetron Technology Inc. Horizontal/vertical dual-mode LCD projector
KR20050069828A (en) * 2003-12-31 2005-07-05 엘지전자 주식회사 Projection device of high resolution
CN1674654A (en) * 2004-03-25 2005-09-28 乐金电子（沈阳）有限公司 Rear-projection television and picture protection function releasing method
US7683881B2 (en) * 2004-05-24 2010-03-23 Keytec, Inc. Visual input pointing device for interactive display system
US7746321B2 (en) * 2004-05-28 2010-06-29 Erik Jan Banning Easily deployable interactive direct-pointing system and presentation control system and calibration method therefor
JPWO2006085580A1 (en) * 2005-02-10 2008-06-26 株式会社タクラム・デザイン・エンジニアリング Pointer light tracking method, program and recording medium therefor
TWI305892B (en) * 2005-11-23 2009-02-01 Inst Information Industry Apparatus, computer equipment, method and computer readable media for simultaneously controlling a cursor and an optical pointer
CN100504576C (en) * 2005-12-05 2009-06-24 财团法人资讯工业策进会 Device and method for synchronous operation of cursor and optical projection

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040041786A1 (en) 2002-08-30 2004-03-04 Casio Computer Co., Ltd. Pointed position detection device and pointed position detection method
US20060007142A1 (en) * 2003-06-13 2006-01-12 Microsoft Corporation Pointing device and cursor for use in intelligent computing environments
US20060197742A1 (en) * 2005-03-04 2006-09-07 Gray Robert H Iii Computer pointing input device
KR20070052160A (en) * 2005-11-16 2007-05-21 한양대학교 산학협력단 Mobile communication terminal having laser pointer and presentation providing system using same

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2218252A4

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9423278B1 (en) 2015-03-09 2016-08-23 Laser Projection Technologies, Inc. 3D laser projection, scanning and object tracking
WO2016145079A1 (en) * 2015-03-09 2016-09-15 Laser Projection Technologies, Inc. 3d laser projection, scanning and object tracking
US9826207B2 (en) 2015-03-09 2017-11-21 Faro Technologies, Inc. 3D laser projection, scanning and object tracking
GB2551458A (en) * 2015-03-09 2017-12-20 Faro Tech Inc 3D Laser projection, scanning and object tracking
US10432903B2 (en) 2015-03-09 2019-10-01 Faro Technologies, Inc. 3D laser projection, scanning and object tracking

Also Published As

Publication number Publication date
CN101919241A (en) 2010-12-15
TW200933452A (en) 2009-08-01
EP2208112A4 (en) 2012-06-27
WO2009061619A3 (en) 2009-08-06
TWI400554B (en) 2013-07-01
EP2208112A2 (en) 2010-07-21
TW200935164A (en) 2009-08-16
CN101896867A (en) 2010-11-24
WO2009061619A2 (en) 2009-05-14
CN101919241B (en) 2013-09-18
EP2218252A4 (en) 2013-02-27
CN101896867B (en) 2012-01-25
EP2218252A1 (en) 2010-08-18
TWI375898B (en) 2012-11-01

Similar Documents

Publication Publication Date Title
US7862179B2 (en) Dual-mode projection apparatus and method for locating a light spot in a projected image
WO2009061620A1 (en) Dual-mode projection apparatus and method for locating a light spot in a projected image
US8188973B2 (en) Apparatus and method for tracking a light pointer
CN1290320C (en) Remote control system and method for a television receiver
EP3057297B1 (en) Accessory and information processing system
US20030222849A1 (en) Laser-based user input device for electronic projection displays
US10015402B2 (en) Electronic apparatus
US8957856B2 (en) Systems, methods, and apparatuses for spatial input associated with a display
US20070115254A1 (en) Apparatus, computer device, method and computer program product for synchronously controlling a cursor and an optical pointer
US20130093675A1 (en) Remote controllable image display system, controller, and processing method therefor
US20120262487A1 (en) Interactive multi-display control systems
CN100590577C (en) Touch screen positioning device and positioning method thereof
US9083428B2 (en) Control device
US20160073017A1 (en) Electronic apparatus
JPH1157216A (en) Game device
US20080244466A1 (en) System and method for interfacing with information on a display screen
US8184211B2 (en) Quasi analog knob control method and appartus using the same
US10423282B2 (en) Display apparatus that switches modes based on distance between indicator and distance measuring unit
KR101134245B1 (en) Electronic device including 3-dimension virtualized remote controller and driving methed thereof
US20170357336A1 (en) Remote computer mouse by camera and laser pointer
US20230130815A1 (en) Image processing apparatus, image processing method, and program
CN118411814B (en) Quasi-touch remote control method and system based on projector camera
US20110285624A1 (en) Screen positioning system and method based on light source type
JP2007213197A (en) Coordinate designation device
KR20170057056A (en) Remote Control Apparatus, Driving Method of Remote Control Apparatus, Image Display Apparatus, Driving Method of Image Display Apparatus, and Computer Readable Recording Medium

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200880123279.5

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08847625

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2008847625

Country of ref document: EP

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载