WO2013017991A1

WO2013017991A1 - Remote control with first and second sensors

Info

Publication number: WO2013017991A1
Application number: PCT/IB2012/053771
Authority: WO
Inventors: Petrus Augustinus Maria Van Grinsven
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2011-08-02
Filing date: 2012-07-25
Publication date: 2013-02-07
Also published as: TW201324248A

Abstract

An apparatus (1) converts control information comprising a combination of first and second sensor information from a remote control (10) into a control instruction for a device (20). The remote control (10) comprises a first sensor (11) for converting image information into the first sensor information and a second sensor (12) for converting geometrical information into the second sensor information. The apparatus (1) may form part of the remote control (10) or the device (20) or may be located in between. At least two different sensors (11, 12) used in combination in the remote control(10) are a great advantage. The apparatus (1) may make the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information. The first sensor (11) may comprise a camera, the second sensor may comprise a detector for detecting an acceleration, a motion, a movement, an angle, a tilt, an orientation and/or a rotation.

Description

Remote control with first and second sensors

FIELD OF THE INVENTION

The invention relates to an apparatus for converting control information from a remote control into a control instruction for a device.

The invention further relates to a remote control comprising the apparatus, to a device comprising the apparatus, to a method for converting control information from a remote control into a control instruction for a device, to a computer program product for performing the step of the method, and to a medium for storing and comprising the computer program product.

Examples of such a device are devices with displays and other devices that are to be controlled remotely.

BACKGROUND OF THE INVENTION

WO 2004 / 047011 and US 2010 / 0157033 disclose pointing devices with cameras.

SUMMARY OF THE INVENTION

It is an object of the invention to provide an improved apparatus. Further objects of the invention are to provide a remote control comprising the improved apparatus, a device comprising the improved apparatus, an improved method, a computer program product, and a medium.

According to a first aspect, an apparatus is provided for converting control information from a remote control into a control instruction for a device, the remote control comprising a first sensor for converting image information into first sensor information and comprising a second sensor for converting geometrical information into second sensor information, the control information comprising a combination of the first sensor information and the second sensor information.

The apparatus may form part of the remote control or may form part of the device or may be located between the remote control and the device. By having provided the remote control with at least two different sensors, and by letting the apparatus convert the control information from the remote control into the control instruction for the device, which control information comprises the combination of the first and second sensor information from the different sensors, an improved apparatus has been created. Such an improved apparatus offers more possibilities owing to the fact that different sensors are used in combination.

An embodiment of the apparatus is defined by the apparatus being arranged to make the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information.

According to a first possibility, the control information comprises the first sensor information that has been checked by having used the second sensor information. According to a second possibility, the control information comprises the first sensor information that has been completed by having used the second sensor information.

According to a third possibility, the control information comprises the first sensor information that has been corrected by having used the second sensor information. According to a fourth possibility, the control information comprises the second sensor information that has overruled the first sensor information.

An embodiment of the apparatus is defined by the first sensor comprising a camera, the image information comprising an image showing a single beacon, two or more beacons, a beacon and a non-beacon and/or the device or a part thereof, and the first sensor information comprising a detection of the single beacon, the two or more beacons, the beacon and the non-beacon and/or the device or the part thereof. By locating one beacon, possibly assisted by a non-beacon, such as a noise source or a reflection, or two or more beacons near the device, via the camera of the remote control a position of the remote control with respect to the device can be determined. Alternatively, the position of the remote control with respect to the device can be determined by recognizing the device of a part thereof via the camera of the remote control.

An embodiment of the apparatus is defined by the second sensor comprising an acceleration detector, a motion detector, a movement detector, an angle detector, a tilt detector, an orientation detector and/or a rotation detector, the geometrical information comprising an acceleration in at least one direction, a motion in at least one direction, a movement in at least one direction, an angle with respect to at least one direction, a tilt with respect to at least one direction, an orientation with respect to at least one direction and/or a rotation with respect to at least one direction, and the second sensor information comprising a detection of the acceleration in the at least one direction, the motion in the at least one direction, the movement in the at least one direction, the angle with respect to the at least one direction, the tilt with respect to the at least one direction, the orientation with respect to the at least one direction and/or the rotation with respect to the at least one direction. A combination of a first sensor in the form of a camera and a second sensor in the form of one of the detectors defined above has proven to be advantageous.

An embodiment of the apparatus is defined by the control instruction comprising a pointing position on the device, a distance between the remote control and the device, a location of the remote control with respect to the device, an acceleration of the remote control in at least one direction, a motion of the remote control in at least one direction, a movement of the remote control in at least one direction, an angle of the remote control with respect to at least one direction, a tilt of the remote control with respect to at least one direction, an orientation of the remote control with respect to at least one direction and/or a rotation of the remote control with respect to at least one direction. Such an apparatus forms, together with the remote control and the device, a gesturing control system.

An embodiment of the apparatus is defined by the apparatus providing gesture detection. With control information comprising a combination of the first and second sensor information, an improved gesture detection has become possible.

An embodiment of the apparatus is defined by the apparatus providing tilt compensation. With control information comprising a combination of the first and second sensor information, an improved tilt compensation has become possible.

According to a second aspect, a remote control is provided comprising the apparatus as defined above.

According to a third aspect, a device is provided comprising the apparatus as defined above.

According to a fourth aspect, a method is provided for converting control information from a remote control into a control instruction for a device, the remote control comprising a first sensor for converting image information into first sensor information and comprising a second sensor for converting geometrical information into second sensor information, the method comprising a step of converting the control information comprising a combination of the first sensor information and the second sensor information into the control instruction.

An embodiment of the method is defined by the method further comprising a step of making the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information. According to a fifth aspect, a computer program product is provided for performing the step of the method as defined above.

According to a sixth aspect, a medium is provided for storing and comprising the computer program product as defined above.

An insight could be that a first sensor may provide first sensor information only, and a basic idea could be that a combination of first and second sensors may provide a combination of first and second sensor information.

A problem to provide an improved apparatus has been solved. A further advantage could be that more options may become possible.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

Fig. 1 shows a trajectory of a remote control comprising a first sensor in the form of a camera,

Fig. 2 shows a trajectory of a remote control comprising a second sensor in the form of an acceleration detector,

Fig. 3 shows a remote control comprising an apparatus and shows a device, and

Fig. 4 shows a remote control and shows a device comprising an apparatus.

DETAILED DESCRIPTION OF EMBODIMENTS

In the Fig. 1, a trajectory of a remote control comprising a first sensor in the form of a camera is shown. The device comprises for example a single beacon that is to be detected by the camera or the entire device is to be detected by the camera as a single light point or blob. Clearly, fast left right movements made by a user are not detected as fully horizontal left right movements by the remote control. A first reason might be that the left right movements made by the user are not precisely horizontal. A second reason might be that the user is not making left right movements but is rotating the remote control (e.g.

rotating the remote control such as a handheld around an axis like a line between the handheld and the device when the handheld is kept relatively straight in front of the device) or making circular movements with the remote control (e.g. circles in a plane relatively parallel to a front side of the device). The remote control comprising only the first sensor in the form of the camera cannot distinguish between these two reasons. Similarly, the remote control comprising only the first sensor in the form of the camera cannot distinguish between a rotation and a circular movement.

In the Fig. 2, a trajectory of a remote control comprising a second sensor in the form of an acceleration detector is shown. Fast left right movements made by a user are not detected as fully horizontal left right movements by the remote control at all. The faster the horizontal movements, the more the data from the acceleration detector such as an

accelerometer will show a deviation from a horizontal movement. But also for relatively slow movements, the acceleration detector will not show a straight line.

In the Fig. 3, a remote control 10 comprising an apparatus 1 and a device 20 are shown. The remote control 10 comprises a first sensor 11 and comprises a second sensor 12. The first sensor 11 converts image information into first sensor information provided to the apparatus 1 and the second sensor 12 converts geometrical information into second sensor information provided to the apparatus 1. The apparatus 1 converts control information comprising a combination of the first sensor information and the second sensor information into a control instruction and supplies it to a transmitter 13. The transmitter 13 transmits the control instruction to a receiver 23 in the device 20. The receiver 23 supplies the control instruction to a device controller 24. Alternatively, between the apparatus 1 and one or more of the sensors 11-12, one or more controllers may be present. Further alternatively, these one or more controllers may be located in the sensors 11-12 or in the apparatus 1, or the apparatus 1 may form part of such controller(s).

In the Fig. 4, a remote control 10 and a device 20 comprising an apparatus 1 are shown. The remote control 10 comprises a first sensor 11 and comprises a second sensor 12. The first sensor 11 converts image information into first sensor information provided to a transmitter 13 possibly via a controller 14 and the second sensor 12 converts geometrical information into second sensor information provided to the transmitter 13 possibly via the controller 14. The transmitter 13 transmits the first and second sensor information to a receiver 23 in the device 20. The receiver 23 supplies the first and second sensor information to the apparatus 1. The apparatus 1 converts control information comprising a combination of the first sensor information and the second sensor information into a control instruction and supplies it to a device controller 24. Alternatively, parts of the controller 14 may be located in the sensors 11-12. Further alternatively, the device controller 24 may be located in the apparatus 1, or the apparatus 1 may form part of the device controller 24. Preferably, the apparatus 1 combines the first and second sensor information by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information. In view of the checking, the data from the Fig. 2 could be used to check the data from the Fig. 1. In view of the completing, in case some of the data from the Fig. 1 is missing for some position or for some moments in time, the data from the Fig. 2 could be used to complete the data from the Fig. 1. In view of the correcting, in case some of the data from the Fig. 1 is partly incorrect for some position or for some moments in time, the data from the Fig. 2 could be used to correct the data from the Fig. 1. In view of the overruling, in case the data from the Fig. 1 is fully incorrect for some position or for some moments in time, the data from the Fig. 2 could be used to overrule the data from the Fig. 1.

Preferably, the first sensor 11 comprises a camera, the image information comprises an image showing a single beacon, two or more beacons, a beacon and a non- beacon and/or the device 20 or a part thereof, and the first sensor information comprises a detection of the single beacon, the two or more beacons, the beacon and the non-beacon and/or the device 20 or the part thereof. Here it must be noted that, in a situation with two or more beacons or a combination of a beacon and a non-beacon or a more precise detection of the device or the part thereof, although the remote control 10 comprising only the first sensor 11 in the form of the camera can distinguish between pure left right movements and left right movements accompanied by some rotation, and between a pure rotation and a circular movement, the combination of the two different sensors 11 and 12 will improve that situation too.

Preferably, the second sensor 12 comprises an acceleration detector, a motion detector, a movement detector, an angle detector, a tilt detector, an orientation detector and/or a rotation detector, the geometrical information comprises an acceleration in at least one direction, a motion in at least one direction, a movement in at least one direction, an angle with respect to at least one direction, a tilt with respect to at least one direction, an orientation with respect to at least one direction, and/or a rotation with respect to at least one direction, and the second sensor information comprises a detection of the acceleration in the at least one direction, the motion in the at least one direction, the movement in the at least one direction, the angle with respect to the at least one direction, the tilt with respect to the at least one direction, the orientation with respect to the at least one direction and/or the rotation with respect to the at least one direction. Here it must be noted that, in a situation where the first sensor 11 goes "out-of-reach", the second sensor 12 may compensate for this situation. Preferably, the control instruction comprises a pointing position on the device 20, a distance between the remote control 10 and the device 20, a location of the remote control 10 with respect to the device 20, an acceleration of the remote control 10 in at least one direction, a motion of the remote control 10 in at least one direction, a movement of the remote control 10 in at least one direction, an angle of the remote control 10 with respect to at least one direction, a tilt of the remote control 10 with respect to at least one direction, an orientation of the remote control 10 with respect to at least one direction and/or a rotation of the remote control 10 with respect to at least one direction.

The remote control 10 comprising the first sensor 11 in the form of the camera for example translates a light spot or a blob captured from a beacon or a more precise device detection to for example a cursor position on a display of the device 20. A gesture made by a user who is holding the remote control 10 results in a sequence of light spot samples or blob samples or device detections on the camera sensor. By processing such sequences, a gesture can be detected. However, it is difficult, when using camera data only, to properly distinguish between all kinds of gestures, since the camera might not have a good reference for its orientation in space. In a camera based remote control, while using a single reference point (a single beacon), there is no information about the orientation of the remote control, resulting in difficulties in detecting differences between rotations and circular movements. But also in a camera based remote control, while using several reference points (several beacons) or more precise device detections, problems may occur, resulting in difficulties in detecting different gestures.

Gesture detection based on geometrical data only such as for example acceleration data is relatively unreliable (relatively high probability on false positives and false negatives) due to the combination of gravity and external forces. An acceleration based detection of simple horizontal movements is difficult and may cause latencies in detection. Latencies are typically introduced by filtering sensor data. The filtering is applied to suppress an impact of noise from external forces.

By combining two different sensors 11-12, geometrical data, such as for example acceleration data, and optical data are combined. The more accurate optical trajectory information from the camera can be fully exploited whereby the acceleration data is used to derive an orientation. This increases the correct detection rate of gestures, where acceleration data assists in optical detection and vice versa. The combination of optical and acceleration based detection also results in a richer set of gestures that can be recognized compared to the "optical only" and "acceleration only" situations. Improvements are also achieved in case tilt-correction is implemented. With the combination of two or more different sensors 11-12, more different gestures can be detected more precisely.

Summarizing, an apparatus 1 converts control information comprising a combination of first and second sensor information from a remote control 10 into a control instruction for a device 20. The remote control 10 comprises a first sensor 11 for converting image information into the first sensor information and a second sensor 12 for converting geometrical information into the second sensor information. The apparatus 1 may form part of the remote control 10 or the device 20 or may be located in between. At least two different sensors 11, 12 used in combination in the remote control 10 are a great advantage. The apparatus 1 may make the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information. The first sensor 11 may comprise a camera, the second sensor may comprise a detector for detecting an acceleration, a motion, a movement, an angle, a tilt, an orientation and/or a rotation.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. An apparatus (1) for converting control information from a remote control (10) into a control instruction for a device (20), the remote control (10) comprising a first sensor (11) for converting image information into first sensor information and comprising a second sensor (12) for converting geometrical information into second sensor information, the control information comprising a combination of the first sensor information and the second sensor information.

2. The apparatus (1) as defined in claim 1, the apparatus (1) being arranged to make the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information.

3. The apparatus (1) as defined in claim 1, the first sensor (11) comprising a camera, the image information comprising an image showing a single beacon, two or more beacons, a beacon and a non-beacon and/or the device (20) or a part thereof, and the first sensor information comprising a detection of the single beacon, the two or more beacons, the beacon and the non-beacon and/or the device (20) or the part thereof.

4. The apparatus (1) as defined in claim 1, the second sensor (12) comprising an acceleration detector, a motion detector, a movement detector, an angle detector, a tilt detector, an orientation detector and/or a rotation detector, the geometrical information comprising an acceleration in at least one direction, a motion in at least one direction, a movement in at least one direction, an angle with respect to at least one direction, a tilt with respect to at least one direction, an orientation with respect to at least one direction, and/or a rotation with respect to at least one direction, and the second sensor information comprising a detection of the acceleration in the at least one direction, the motion in the at least one direction, the movement in the at least one direction, the angle with respect to the at least one direction, the tilt with respect to the at least one direction, the orientation with respect to the at least one direction and/or the rotation with respect to the at least one direction.

5. The apparatus (1) as defined in claim 1, the control instruction comprising a pointing position on the device (20), a distance between the remote control (10) and the device (20), a location of the remote control (10) with respect to the device (20), an acceleration of the remote control (10) in at least one direction, a motion of the remote control (10) in at least one direction, a movement of the remote control (10) in at least one direction, an angle of the remote control (10) with respect to at least one direction, a tilt of the remote control (10) with respect to at least one direction, an orientation of the remote control (10) with respect to at least one direction and/or a rotation of the remote control (10) with respect to at least one direction.

6. The apparatus (1) as defined in claim 1, the apparatus (1) providing gesture detection.

7. The apparatus (1) as defined in claim 1, the apparatus (1) providing tilt compensation.

8. A remote control (10) comprising the apparatus (1) as defined in claim 1.

9. A device (20) comprising the apparatus (1) as defined in claim 1.

10. A method for converting control information from a remote control (10) into a control instruction for a device (20), the remote control (10) comprising a first sensor (11) for converting image information into first sensor information and comprising a second sensor (12) for converting geometrical information into second sensor information, the method comprising a step of converting the control information comprising a combination of the first sensor information and the second sensor information into the control instruction.

11. The method as defined in claim 10, the method further comprising a step of making the combination by using the second sensor information for checking, completing, correcting and/or overruling the first sensor information.

12. A computer program product for performing the step of the method as defined in claim 10.

13. A medium for storing and comprising the computer program product as defined in claim 12.