+

WO2008001265A1 - Method of providing object models to a plurality of object recognition devices - Google Patents

Method of providing object models to a plurality of object recognition devices Download PDF

Info

Publication number
WO2008001265A1
WO2008001265A1 PCT/IB2007/052337 IB2007052337W WO2008001265A1 WO 2008001265 A1 WO2008001265 A1 WO 2008001265A1 IB 2007052337 W IB2007052337 W IB 2007052337W WO 2008001265 A1 WO2008001265 A1 WO 2008001265A1
Authority
WO
WIPO (PCT)
Prior art keywords
object model
recognition device
model
object recognition
service platform
Prior art date
Application number
PCT/IB2007/052337
Other languages
French (fr)
Inventor
Thomas Portele
Christian Benien
Original Assignee
Philips Intellectual Property & Standards Gmbh
Koninklijke Philips Electronics N. V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Philips Intellectual Property & Standards Gmbh, Koninklijke Philips Electronics N. V. filed Critical Philips Intellectual Property & Standards Gmbh
Publication of WO2008001265A1 publication Critical patent/WO2008001265A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/94Hardware or software architectures specially adapted for image or video understanding

Definitions

  • the invention describes a method of providing object models to a plurality of object recognition devices, to an object model provision system, and to an object model service platform for use in such a system.
  • Many systems perform tasks using information obtained from image analysis, for example surveillance systems or home dialogue systems.
  • Such systems are equipped with a camera for generating images of their surroundings, or of certain objects in their surroundings.
  • the camera is generally fixed, and the Objects' to be recognised are generally people moving within the field of view of the camera.
  • a home dialogue system can be realised as a kind of 'robot', perhaps even with human or animal appearance or characteristics, and can be used to perform tasks for the user. Such tasks might be of a practical nature, such as locating a certain item or downloading and reading an e-mail, and can serve to assist a user in everyday situations.
  • tasks might be of a practical nature, such as locating a certain item or downloading and reading an e-mail, and can serve to assist a user in everyday situations.
  • To increase the practical usefulness of a home dialogue system such a device will be able to autonomously move about in its environment.
  • the surroundings for such a moveable home dialogue system are typically not fixed or static, and the 'objects' to be recognised by the camera of the home dialogue system can vary in appearance.
  • One kind of object can have many different realisations, for example articles of clothing can vary enormously in colour, size, shape, etc. Devices that perform the same functions can also vary in shape and appearance, for example the different types of telephone in a single household - cordless phone, mobile phone, etc. Furthermore, an object 'seen' by the system's camera can appear entirely different when viewed in poor light or from an unusual aspect or angle.
  • a home dialogue system might have an object model for a user's mobile phone.
  • This mobile phone object model can 'describe' the appearance of the mobile phone using certain parameters or properties so that the home dialogue system can recognise the mobile phone from any direction and under different lighting conditions.
  • Relevant parameters might describe proportions of the device, expected positions of a keypad and display on the device, etc., or may (additionally or alternatively) consist of abstract mathematical representations of features of the device's appearance.
  • the properties of an object in an image can be extracted using image analysis techniques and then compared to the object model to determine if the object in the image belongs to the object type described in the object model.
  • the present invention describes a method of providing object models to a plurality of object recognition devices, which method comprises maintaining a collection of classified object models in an object model service platform, interpreting a request from an object recognition device to retrieve an object model required by the object recognition device, and transferring the requested object model to the object recognition device.
  • An Object recognition device' can be any device that performs object recognition on the basis of images. Such an object recognition device avails of an image acquisition unit, for example a camera, for the purpose of obtaining images of objects for their identification, such as the home dialogue device described above.
  • a 'request' for an object model can be issued by an object recognition device, for example, when it encounters an object which it is unable to recognise.
  • the object model service platform can analyse the request to determine the appropriate object model to be retrieved from the collection of object models, and can then transfer a copy of the object model to the object recognition device, which can use this model for later object recognition purposes.
  • An obvious advantage of the method according to the invention is that the method is not restricted to providing object models to a single object recognition device, but can provide a plurality of object recognition devices with object models.
  • the object models themselves are maintained in a collection in an object model service platform, so that an object recognition device can make use of a large collection of object models without having to maintain such a collection itself.
  • Each object recognition device can retrieve, as necessary, just the object models it needs. In this way, an object recognition device can autonomously and independently 'learn' or adapt to its environment, without a user having to carry out a tedious and time-consuming training process.
  • a suitable object model service platform comprises a maintenance unit for maintaining a collection of object models.
  • the object model service platform comprises a request interface, with which the object model service platform interprets a request from an object recognition device to retrieve a required object model.
  • the object model service platform further comprises an object model transfer interface for transferring the retrieved object model to the object recognition device.
  • Data can be exchanged between the object recognition device and the object model service platform in any suitable manner, e.g. using a WLAN connection, a cable connection, etc.
  • the mode of data transfer may be chosen according to practical considerations such as the network capabilities of the devices involved, the separation between them, etc.
  • a suitable object model provision system comprises a plurality of object recognition devices (for example a plurality of home dialogue systems in the same household or in a number of different households) and an object model service platform.
  • Objects can be classified in a class hierarchy, which can be visualised as an inverted tree structure, with a top-level class and a number of branches or subordinate classes, each of which in turn can have more subordinate classes.
  • the top- level class 'Vessel' can have a number of subordinate classes such as 'Drinking Vessel', 'Bottle', 'Jar', etc.
  • Properties of the top-level 'Vessel' class might be that a vessel is hollow, has an opening at the top and is closed at the bottom, etc. These properties are inherited by all subordinate classes at all levels.
  • the subordinate class 'Drinking Vessel' can have more specific properties such as a limit in size, a height-to -width ratio, etc., and can have a number of subordinate classes for 'Cup', 'Glass', 'Mug', etc., each of which in turn inherit the properties of the 'Drinking Vessel' class.
  • an object can be described at its level in the class hierarchy by means of the class descriptors ('Vessel', 'Cup', etc.) and the corresponding properties. This manner of describing or classifying an object on a hierarchical listing of its properties is sometimes referred to as ontology.
  • an object model might preferably comprise an ontological object description for that object class. Preferably, it might also comprise a collection of images associated with objects belonging to that class, a set of parameters obtained by analysing images of objects in that class, etc. An example of an ontological object description will be given later in the description of the figures.
  • the object models are preferably maintained in the object model service platform according to an object class hierarchy, and object class properties of an object model are preferably inherited by object models of a subordinate class in the object class hierarchy.
  • the object properties are stored for each sub-class separately, and the overall class model consists of a bundle of different submodels.
  • the hierarchy supports the recognition of an object if it is part of one of the pertinent sub-classes.
  • An object recognition device for use in the object model management system described above, comprises an image acquisition unit for obtaining an image of an object.
  • the image acquisition unit can comprise a connection to a separate source of images, or can itself be a camera, e.g. a simple type of camera such as a webcam, which generally delivers images of low resolution, or a more powerful camera for providing images of higher quality.
  • the type of camera used may be chosen to suit the requirements of the system of which the object recognition device is part.
  • an object recognition device might have its own collection of object models for objects which it can be expected to encounter.
  • a home dialogue system might have a collection of object models for objects, which are located in the environment of the home dialogue system and which it can be expected to encounter regularly.
  • the object recognition device can perform suitable image processing steps to extract properties or features of the object from the image, and might compare these to the object models of its own collection in an attempt to identify the object.
  • the object recognition device fails to identify the object, it can issue a request to the object model service platform.
  • the request from an object recognition device to the object model service platform simply comprises an image of the object that the object recognition device failed to identify.
  • the request can equally well comprise more than one image of the object, for example a number of images taken from different angles.
  • the request transfer to the object model service platform can be carried out in any suitable manner, for example by using a predefined communications protocol.
  • the object recognition device according to the invention also comprises an object model receiver interface for receiving the retrieved object model from the object model service platform. The model of the hitherto unknown object can then be added to the object recognition device's own local collection of object models for future reference.
  • the object that the object recognition device has not been able to categorise, or recognise might be a certain type of telephone, perhaps a designer model that has been made to appear 'old-fashioned', with a round dial and with separate headset, and therefore very different from the modern type of telephone, which is generally in one piece, with a keypad, and of simple design.
  • the object recognition device can upload the image of the 'unknown' telephone to the object model service platform, where it can be successfully identified on the basis of a more extensive object model for 'telephone'.
  • the method according to the invention can also be applied to train or extend the collection of object models in an object model service platform.
  • an image of an object is generated by an object recognition device, and uploaded to the object model service platform.
  • Properties or characteristics of the object in the image are extracted using image analysis, and the object class to which the object belongs is identified.
  • the properties obtained from the image analysis can then be compared to the object model description. If any of the properties serve to extend the description of the object in the object model, the object model service platform can then augment or extend the object model using these properties. Thereafter, the extended or augmented object model available to all other object recognition devices with access to that platform.
  • a basic object model can easily be augmented to contain the properties or features of many different versions of the same object.
  • object recognition performed by object recognition devices in an object model provision system according to the invention will be faster and more accurate owing to the greater detail provided by the augmented object models and their availability to all object recognition devices.
  • the manner of augmentation described above is automatic and takes place without requiring any user participation.
  • the object model augmentation can be performed specifically, i.e. by a user deliberately having an object recognition device generate an image of an object and transferring this image to the object model service platform along with the description supplied by the user - for example, for an image of a telephone, the user might supply a written or spoken label or description "This is a telephone".
  • the object model service platform subsequently augments its 'Telephone' object model with features or properties obtained by performing an image analysis on the image supplied by the object recognition device to give an augmented object model for that model class.
  • any new properties identified by the object model service platform for the object model class will be inherited by all subordinate models for that object model class in the class hierarchy.
  • object classification might comprise the creation of a new subordinate object model class.
  • a dedicated subordinate object model class instead of extending the property lists of an existing subordinate object model class such as 'Desk telephone' or 'Mobile telephone'.
  • a top-level class 'Telephone' property 'has a keypad' might be replaced by a broader property such as 'has digits from 0 to 9', and this property is then inherited by the subordinate object models.
  • the object model service platform therefore preferably comprises an object classification module for classifying an object to an object model in an object class hierarchy. This can be done on the basis of image analysis if the request from the object recognition device simply comprises an image. For example, image processing algorithms known to a person skilled in the art can be applied to identify relevant points, edges or surfaces in the image. Using the extracted information, the object in the image can be classified to the appropriate object model.
  • the object model service platform can directly classify the object in the image to the appropriate object model.
  • the object model service platform preferably also comprises an augmentation unit for augmenting the object model using any new properties identified from the image analysis.
  • the initial object model class for 'Cup' in the collection of object models might inherit the 'Vessel' properties, i.e. a vessel is hollow, has an opening at the top and is closed at the bottom, and the 'Drinking Vessel' properties, i.e. a size limit and height-to -width ratio.
  • An existing 'Cup' object model property might be 'Handle', to indicate that a cup has a handle.
  • the additional property regarding the ratio of the upper and lower diameters of the cup learned from the image analysis can be added to the 'Cup' object model, thereby giving an augmented object model for that class.
  • the object models maintained by the object model service platform can become more extensive over time, as object recognition devices supply images with accompanying specifications of hitherto unknown objects, or known objects seen from a different angle or in a different light, etc.
  • this augmented object model is preferably transferred to the object recognition device in response to the request from the object recognition device. In this way, an object recognition device is able to learn to recognise objects that were previously unknown to it.
  • an object recognition device has supplied an image of an object, it is likely that the specific object shown on this image is encountered by the object recognition device in the future, e.g. because it is part of the environment of the object recognition device. In such a case it is advantageous, when providing the object recognition device with an augmented object model, to increase the importance of properties prominent in that specific object in the augmented object model for that device.
  • a dedicated object model can be transferred from the model service platform to an object recognition device in response to a request for that object model, whereby the object model is adapted specially for the requesting object recognition device, for example so that features of the specific object encountered by the object recognition device are made more prominent compared to the general model of that object, available to all other object recognition devices, thus ensuring that the requesting object recognition device will easily be able to recognise this particular type of object.
  • a model for a dial telephone can have a colour property, which, in the general model is a set of colours, but in the special model only contains the colour of the specific object shown on the image submitted by the object recognition device.
  • the features computed for the object on the submitted image may have greater importance, or larger weight, in the special model for that object recognition device, that in the general model.
  • the colour property for the special model is set to 'black' only, and the likelihood of misrecognitions of non-black objects as 'dial telephone' is strongly reduced.
  • a user can supply a description or label for an image of an object to be transferred to the object model service platform for augmentation purposes.
  • the object model service platform will preferably avail of some kind of decision-making module which decides whether the image of the object supplied by the object recognition device does indeed belong to the class specified by the user. In this way, an erroneous description for an image will not result in the specified object model being augmented with entirely unsuitable properties, such as an image of a bottle being provided to the object model service platform along with a label stating that it is a cup.
  • the decision to accept an image for the purposes of object model augmentation on the basis of a label provided by a user might be based on a certain minimum number of properties that the object must feature in order to belong to the specified object class. For example, an image of a bottle with accompanying 'Cup' label can be discarded since the basic proportions of a bottle (narrower at the top than at the bottom, greater height- to-width ratio, etc.) are different to those of a cup. Also, an image intended for object model augmentation purposes might also be discarded or rejected if the image does not provide any properties or features not already in the existing object model. In this way, unnecessary computational effort can be avoided.
  • the object model provision system preferably comprises a billing unit for determining a payment to be levied for an object model transferred to an object recognition device. In this way, the cost for maintaining object model can be carried by all beneficiaries.
  • an object recognition device that delivers images of objects that can be used to augment an object model is actually contributing to the powerfulness of the overall system, this object recognition device might receive its augmented object model at a discount.
  • an object recognition device might subscribe to a service of the object model service platform in order to automatically receive the newest object model for a particular object without having to specifically request it. In this way, the object recognition device always avails of the most up-to-date version of the subscribed object model.
  • Fig. Ia shows two different objects of the same object class
  • Fig. Ib shows a class hierarchy for the objects of Fig. Ia
  • Fig. 2 shows an object recognition device according to an embodiment of the invention
  • Fig. 3 shows an object model provision system according to an embodiment of the invention.
  • Fig. Ia shows two different types of telephone 10, 11.
  • the telephone 10 on the left is a desk telephone with a separate handset attached by a cord to the housing, and features a user interface or keypad with numerous buttons.
  • the telephone 11 on the right is a mobile phone with a small keypad and display.
  • both devices 10, 11 are very different in appearance, they both share properties of a top-level object class 'Telephone'.
  • This object class can be further refined with subordinate classes for the different categories of telephone, as is shown in Fig. Ib, where Mi represents an object model for the 'Telephone' class in a class hierarchy.
  • One subordinate class Mi 0 at a lower level than Mi, is an object model for the desk telephone
  • subordinate class Mn is an object model for a mobile phone.
  • a possible ontological description of a telephone contains functional properties like 'has microphone' and 'has speaker' as well as appearance properties like 'shows 10 digits'.
  • a desk telephone inherits these items, but will have appearance properties like 'has cord', 'has receiver (and/or a description of the appearance of a receiver as being 15-20 cm long with two thicker parts at both ends)', and 'has digits ordered in a 3 by 4 matrix', while a cordless phone has the functional property 'has antenna' and the appearance property 'has digits ordered in a 3 by 4 matrix', and 'can have a display'.
  • an object recognition device Di Di is shown.
  • This object recognition device Di is a home dialogue system Di in the form of a robot which is capable of autonomously moving about and generating images of objects which it wishes to identify.
  • the robot Di has an image analysis module 52 capable of performing image analysis on the images generated by the camera 3 and object classification using object models stored in its own memory 53.
  • the home dialogue system Di has approached a telephone 12 and has generated an image 4 of the telephone 12.
  • Image analysis of the image 4 fails to identify any properties or characteristics corresponding to one of the object models in its own object model collection. Therefore, the robot Di transfers the image 4 to an object model service platform, as will be described below.
  • Fig. 3 shows an object model provision system 1 with an object model service platform 2 and a plurality of object recognition devices Di, D 2 , D 3 .
  • the object recognition device Di is a home dialogue system equipped with a camera 31 and capable of autonomous movement
  • the object recognition device D 2 is a surveillance unit with a camera 32
  • the object recognition device D 3 is a personal computer with an image acquisition unit 33 in the form of a moveable webcam 33.
  • Each of the cameras 31, 32, 33 can be used to generate images of objects.
  • Any number of object recognition devices can communicate with the object model service platform, but only three are shown here for the sake of clarity.
  • the object model service platform 2 maintains a collection of object models Mi, ..., M n in an ontological class hierarchy H, stored in a database 48.
  • object models Mi, M 2 Only two object models Mi, M 2 are shown, but it will be clear that the class hierarchy can comprise any number of object models, conceivably limited only by the amount of memory space or processing power available. In the same way, although only a limited number of subordinate models Mi 0 , ... , M 202 is shown, it will be clear than an object model can comprise an indefinite number of subordinate object model levels, or no subordinate object model levels at all.
  • the object recognition devices Di, D 2 , D 3 can submit requests to the object model service platform 2 and can receive object models Mi, ..., M n from the object model service platform 2 by means of suitable interface signals 301, 302, 303.
  • the object recognition device Di transfers an image of an unidentified object to the object model service platform 2 by means of an image transfer interface 50 and the interface signal 301.
  • the image is that of the old- fashioned telephone described in Fig. 2.
  • a communications interface 40 of the object model service platform 2 receives the request and interprets it in a request interface 41.
  • the request interface 41 concludes that the object recognition device Di requires an object model for the object in the image, and forwards the information to a maintenance unit 43.
  • an image analysis unit 44 performs image analysis of the image and identifies features or properties of the object in the image.
  • an object classification module 46 concludes that the object in the image is a telephone, since it shares features of the object model class Mi for 'Telephone' in the collection of object models in the class hierarchy H, but that it should not be classified to either of the two existing subordinate object classes Mi 0 , Mn for 'Desk telephone' and 'Mobile telephone' respectively. Therefore, the augmentation unit 45 decides to create a new subordinate object class Mi 2 . As already described above, the properties of the top-level model Mi are adjusted if necessary, with a subsequent inheritance of the adjusted top-level properties to the subordinate object models Mi 0 , Mn.
  • the augmented object model Mi for 'Telephone' previously consisting of a bundle of two sub-models now comprises three subordinate models Mi 0 , Mn, Mi 2 .
  • the maintenance unit 43 can forward a copy of the augmented object model Mi including its subordinate object models Mi 0 , Mn, Mi 2 to an object model transfer interface 42 in the communications interface 40 to carry out the transfer to the object recognition device Di.
  • the object recognition device Di Prior to transfer of the object model, it may be converted into a form suitable for interpretation by the object recognition device Di, since different types of object recognition device may work with different types of object model representation.
  • the object recognition device Di receives the information in an object model receiver interface 51. Thereafter, the object recognition device Di can avail of the updated object models and will be able, in future, to recognise the previously unidentifiable telephone and possibly also telephones of similar design.
  • a billing unit 47 is shown in the object model service platform 2.
  • This billing unit 47 can determine any charges to be levied for supplying the object recognition device Di with an augmented model on request or according to subscription, or any discounts to be granted to that object recognition device for images used to augment an object model in the collection of object models.
  • the object model service platform can also be used to provide a set of object models for a new object recognition device such as a home dialog system. Such a system might not be equipped with any object models, or might be equipped with just a basic list of object models.
  • the user of the home dialogue system might simply put together a list of objects that he considers relevant for the functions that his home dialogue system is to carry out, and can order the appropriate object models from the object model service platform by issuing a suitable request.
  • the home dialogue system can autonomously request updates or new object models from the object model service platform, as the necessity arises.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Stored Programmes (AREA)

Abstract

The invention describes a method of providing object models (M1, M10,..., Mn) to a plurality of object recognition devices (D1, D2, D3), which method comprises maintaining a collection of classified object models (M1, M10,..., Mn) in an object model service platform (2), interpreting a request from an object recognition device (D1, D2, D3) to retrieve an object model (M1, M10,..., Mn) required by the object recognition device (D1, D2, D3), and transferring the requested object model (M1, M10,..., Mn) to the object recognition device (D1, D2, D3). The invention further describes an object model provision system (1) comprising a plurality of object recognition devices (D1, D2, D3) and an object model service platform (2) with a maintenance unit (43) for maintaining a collection of object models (M1, M10,..., Mn). The object model service platform also comprises a request interface (41) for interpreting a request from an object recognition device (D1, D2, D) to retrieve an object model (M1, M10,..., Mn) required by the object recognition device (D1, D2, D3), and an object model transfer interface (42) for transferring the object model (M1, M10,..., Mn) to the object recognition device (D1, D2, D3).

Description

Method of providing object models to a plurality of object recognition devices
The invention describes a method of providing object models to a plurality of object recognition devices, to an object model provision system, and to an object model service platform for use in such a system. Many systems perform tasks using information obtained from image analysis, for example surveillance systems or home dialogue systems. Such systems are equipped with a camera for generating images of their surroundings, or of certain objects in their surroundings. In the case of a surveillance system, the camera is generally fixed, and the Objects' to be recognised are generally people moving within the field of view of the camera.
Developments in the field of dialogue systems for use in consumer households can be expected to lead to their widespread use in homes in the near future. A home dialogue system can be realised as a kind of 'robot', perhaps even with human or animal appearance or characteristics, and can be used to perform tasks for the user. Such tasks might be of a practical nature, such as locating a certain item or downloading and reading an e-mail, and can serve to assist a user in everyday situations. To increase the practical usefulness of a home dialogue system, such a device will be able to autonomously move about in its environment. The surroundings for such a moveable home dialogue system are typically not fixed or static, and the 'objects' to be recognised by the camera of the home dialogue system can vary in appearance. One kind of object can have many different realisations, for example articles of clothing can vary enormously in colour, size, shape, etc. Devices that perform the same functions can also vary in shape and appearance, for example the different types of telephone in a single household - cordless phone, mobile phone, etc. Furthermore, an object 'seen' by the system's camera can appear entirely different when viewed in poor light or from an unusual aspect or angle.
When restricted to using techniques of pattern recognition, a system or device that can recognise one realisation of a certain type of object might fail to recognise a different realisation of the same type of object. Object recognition uses
'models' of objects in deciding to which category an object belongs. For example, a home dialogue system might have an object model for a user's mobile phone. This mobile phone object model can 'describe' the appearance of the mobile phone using certain parameters or properties so that the home dialogue system can recognise the mobile phone from any direction and under different lighting conditions. Relevant parameters might describe proportions of the device, expected positions of a keypad and display on the device, etc., or may (additionally or alternatively) consist of abstract mathematical representations of features of the device's appearance. The properties of an object in an image can be extracted using image analysis techniques and then compared to the object model to determine if the object in the image belongs to the object type described in the object model.
Such approaches to object recognition are becoming more and more interesting as developments in home dialogue systems are moving in the direction of autonomous robotic devices for use in the home. Such a device could then, for example, locate and fetch a user's mobile phone in response to a corresponding spoken command. However, such a system is then limited to the object model that it has. If the user purchases a new mobile phone with a different appearance, the home dialogue system might very well no longer recognise it. Furthermore, the system will be unable to recognise any object for which it does not already have an object model. While it is possible to train such systems to recognise objects by compiling an adequate model, the training is a time-consuming, cost-intensive and specialist task, and, for this reason, very unattractive to the average user of a home dialogue system. Since it is foreseeable that many homes in the near future will equipped with such dialogue systems, this problem will be experienced by a growing number of users. Therefore, it is an object of the invention to provide a simple and economical way of obtaining object models.
To this end, the present invention describes a method of providing object models to a plurality of object recognition devices, which method comprises maintaining a collection of classified object models in an object model service platform, interpreting a request from an object recognition device to retrieve an object model required by the object recognition device, and transferring the requested object model to the object recognition device. An Object recognition device' can be any device that performs object recognition on the basis of images. Such an object recognition device avails of an image acquisition unit, for example a camera, for the purpose of obtaining images of objects for their identification, such as the home dialogue device described above. A 'request' for an object model can be issued by an object recognition device, for example, when it encounters an object which it is unable to recognise. The object model service platform can analyse the request to determine the appropriate object model to be retrieved from the collection of object models, and can then transfer a copy of the object model to the object recognition device, which can use this model for later object recognition purposes. An obvious advantage of the method according to the invention is that the method is not restricted to providing object models to a single object recognition device, but can provide a plurality of object recognition devices with object models. The object models themselves are maintained in a collection in an object model service platform, so that an object recognition device can make use of a large collection of object models without having to maintain such a collection itself. Each object recognition device can retrieve, as necessary, just the object models it needs. In this way, an object recognition device can autonomously and independently 'learn' or adapt to its environment, without a user having to carry out a tedious and time-consuming training process.
A suitable object model service platform comprises a maintenance unit for maintaining a collection of object models. To communicate with an object recognition device, the object model service platform comprises a request interface, with which the object model service platform interprets a request from an object recognition device to retrieve a required object model. The object model service platform further comprises an object model transfer interface for transferring the retrieved object model to the object recognition device.
Data can be exchanged between the object recognition device and the object model service platform in any suitable manner, e.g. using a WLAN connection, a cable connection, etc. The mode of data transfer may be chosen according to practical considerations such as the network capabilities of the devices involved, the separation between them, etc.
A suitable object model provision system comprises a plurality of object recognition devices (for example a plurality of home dialogue systems in the same household or in a number of different households) and an object model service platform. The dependent claims and the subsequent description disclose particularly advantageous embodiments and features of the invention.
Objects can be classified in a class hierarchy, which can be visualised as an inverted tree structure, with a top-level class and a number of branches or subordinate classes, each of which in turn can have more subordinate classes. For example, the top- level class 'Vessel' can have a number of subordinate classes such as 'Drinking Vessel', 'Bottle', 'Jar', etc. Properties of the top-level 'Vessel' class might be that a vessel is hollow, has an opening at the top and is closed at the bottom, etc. These properties are inherited by all subordinate classes at all levels. The subordinate class 'Drinking Vessel' can have more specific properties such as a limit in size, a height-to -width ratio, etc., and can have a number of subordinate classes for 'Cup', 'Glass', 'Mug', etc., each of which in turn inherit the properties of the 'Drinking Vessel' class.
An object can be described at its level in the class hierarchy by means of the class descriptors ('Vessel', 'Cup', etc.) and the corresponding properties. This manner of describing or classifying an object on a hierarchical listing of its properties is sometimes referred to as ontology. In the method according to the invention, an object model might preferably comprise an ontological object description for that object class. Preferably, it might also comprise a collection of images associated with objects belonging to that class, a set of parameters obtained by analysing images of objects in that class, etc. An example of an ontological object description will be given later in the description of the figures. In the method according to the invention, the object models are preferably maintained in the object model service platform according to an object class hierarchy, and object class properties of an object model are preferably inherited by object models of a subordinate class in the object class hierarchy.
For objects of sub-classes that have appearances very different from objects in other sub-classes, it is feasible that the object properties are stored for each sub-class separately, and the overall class model consists of a bundle of different submodels. Thus, although it is not possible to deduce a common appearance, the hierarchy supports the recognition of an object if it is part of one of the pertinent sub-classes.
An object recognition device according to the invention, for use in the object model management system described above, comprises an image acquisition unit for obtaining an image of an object. The image acquisition unit can comprise a connection to a separate source of images, or can itself be a camera, e.g. a simple type of camera such as a webcam, which generally delivers images of low resolution, or a more powerful camera for providing images of higher quality. The type of camera used may be chosen to suit the requirements of the system of which the object recognition device is part. Evidently, an object recognition device might have its own collection of object models for objects which it can be expected to encounter. For instance, a home dialogue system might have a collection of object models for objects, which are located in the environment of the home dialogue system and which it can be expected to encounter regularly. To identify an object, the object recognition device can perform suitable image processing steps to extract properties or features of the object from the image, and might compare these to the object models of its own collection in an attempt to identify the object.
In the case that the object recognition device fails to identify the object, it can issue a request to the object model service platform. In a preferred embodiment of the invention, the request from an object recognition device to the object model service platform simply comprises an image of the object that the object recognition device failed to identify. However, the request can equally well comprise more than one image of the object, for example a number of images taken from different angles. The request transfer to the object model service platform can be carried out in any suitable manner, for example by using a predefined communications protocol. The object recognition device according to the invention also comprises an object model receiver interface for receiving the retrieved object model from the object model service platform. The model of the hitherto unknown object can then be added to the object recognition device's own local collection of object models for future reference. For example, the object that the object recognition device has not been able to categorise, or recognise, might be a certain type of telephone, perhaps a designer model that has been made to appear 'old-fashioned', with a round dial and with separate headset, and therefore very different from the modern type of telephone, which is generally in one piece, with a keypad, and of simple design. The object recognition device can upload the image of the 'unknown' telephone to the object model service platform, where it can be successfully identified on the basis of a more extensive object model for 'telephone'.
Since an object model can be further refined, or augmented, by adding to it some additional properties of other objects in its class, the method according to the invention can also be applied to train or extend the collection of object models in an object model service platform. To this end, an image of an object is generated by an object recognition device, and uploaded to the object model service platform. Properties or characteristics of the object in the image are extracted using image analysis, and the object class to which the object belongs is identified. The properties obtained from the image analysis can then be compared to the object model description. If any of the properties serve to extend the description of the object in the object model, the object model service platform can then augment or extend the object model using these properties. Thereafter, the extended or augmented object model available to all other object recognition devices with access to that platform. In this way, a basic object model can easily be augmented to contain the properties or features of many different versions of the same object. In time, object recognition performed by object recognition devices in an object model provision system according to the invention will be faster and more accurate owing to the greater detail provided by the augmented object models and their availability to all object recognition devices. The manner of augmentation described above is automatic and takes place without requiring any user participation. However, the object model augmentation can be performed specifically, i.e. by a user deliberately having an object recognition device generate an image of an object and transferring this image to the object model service platform along with the description supplied by the user - for example, for an image of a telephone, the user might supply a written or spoken label or description "This is a telephone". In a preferred embodiment of the invention, the object model service platform subsequently augments its 'Telephone' object model with features or properties obtained by performing an image analysis on the image supplied by the object recognition device to give an augmented object model for that model class. As described above, any new properties identified by the object model service platform for the object model class will be inherited by all subordinate models for that object model class in the class hierarchy. However, if the object in the image is sufficiently different from the specified object class or its subordinate object classes, object classification might comprise the creation of a new subordinate object model class. For example, in the case described here of an old-fashioned or designer telephone, it might be advantageous to create a dedicated subordinate object model class instead of extending the property lists of an existing subordinate object model class such as 'Desk telephone' or 'Mobile telephone'. Furthermore, it may be necessary to adjust one or more properties of the top- level class. In this example, a top-level class 'Telephone' property 'has a keypad' might be replaced by a broader property such as 'has digits from 0 to 9', and this property is then inherited by the subordinate object models. The more specific property 'has digits from 0 to 9 on a keypad' can be moved down one level to the appropriate subordinate 'Desk telephone' and 'Mobile telephone' object models, whereas the new 'Dial telephone' can be given the property 'has digits from 0 to 9 on a dial'. The object model service platform therefore preferably comprises an object classification module for classifying an object to an object model in an object class hierarchy. This can be done on the basis of image analysis if the request from the object recognition device simply comprises an image. For example, image processing algorithms known to a person skilled in the art can be applied to identify relevant points, edges or surfaces in the image. Using the extracted information, the object in the image can be classified to the appropriate object model. Evidently, if the request from the object recognition device comprises an image and an accompanying description, the object model service platform can directly classify the object in the image to the appropriate object model.
The object model service platform preferably also comprises an augmentation unit for augmenting the object model using any new properties identified from the image analysis. For example, the initial object model class for 'Cup' in the collection of object models might inherit the 'Vessel' properties, i.e. a vessel is hollow, has an opening at the top and is closed at the bottom, and the 'Drinking Vessel' properties, i.e. a size limit and height-to -width ratio. An existing 'Cup' object model property might be 'Handle', to indicate that a cup has a handle. The additional property regarding the ratio of the upper and lower diameters of the cup learned from the image analysis can be added to the 'Cup' object model, thereby giving an augmented object model for that class. In this way, the object models maintained by the object model service platform can become more extensive over time, as object recognition devices supply images with accompanying specifications of hitherto unknown objects, or known objects seen from a different angle or in a different light, etc.
If augmentation has been carried out on an object model, this augmented object model is preferably transferred to the object recognition device in response to the request from the object recognition device. In this way, an object recognition device is able to learn to recognise objects that were previously unknown to it.
If an object recognition device has supplied an image of an object, it is likely that the specific object shown on this image is encountered by the object recognition device in the future, e.g. because it is part of the environment of the object recognition device. In such a case it is advantageous, when providing the object recognition device with an augmented object model, to increase the importance of properties prominent in that specific object in the augmented object model for that device. Therefore, in a preferred embodiment of the invention, a dedicated object model can be transferred from the model service platform to an object recognition device in response to a request for that object model, whereby the object model is adapted specially for the requesting object recognition device, for example so that features of the specific object encountered by the object recognition device are made more prominent compared to the general model of that object, available to all other object recognition devices, thus ensuring that the requesting object recognition device will easily be able to recognise this particular type of object. For example, a model for a dial telephone can have a colour property, which, in the general model is a set of colours, but in the special model only contains the colour of the specific object shown on the image submitted by the object recognition device. For abstract mathematical feature representations of a model, the features computed for the object on the submitted image may have greater importance, or larger weight, in the special model for that object recognition device, that in the general model. Thus, if the only dial telephone encountered by the object recognition device is black, the colour property for the special model is set to 'black' only, and the likelihood of misrecognitions of non-black objects as 'dial telephone' is strongly reduced.
As mentioned above, a user can supply a description or label for an image of an object to be transferred to the object model service platform for augmentation purposes. Naturally, the object model service platform will preferably avail of some kind of decision-making module which decides whether the image of the object supplied by the object recognition device does indeed belong to the class specified by the user. In this way, an erroneous description for an image will not result in the specified object model being augmented with entirely unsuitable properties, such as an image of a bottle being provided to the object model service platform along with a label stating that it is a cup. The decision to accept an image for the purposes of object model augmentation on the basis of a label provided by a user might be based on a certain minimum number of properties that the object must feature in order to belong to the specified object class. For example, an image of a bottle with accompanying 'Cup' label can be discarded since the basic proportions of a bottle (narrower at the top than at the bottom, greater height- to-width ratio, etc.) are different to those of a cup. Also, an image intended for object model augmentation purposes might also be discarded or rejected if the image does not provide any properties or features not already in the existing object model. In this way, unnecessary computational effort can be avoided.
In an object model provision system, where images and object models are traded in the manner described above, it may be advantageous to levy a fee for an object model provided to an object recognition device. Therefore, the object model provision system preferably comprises a billing unit for determining a payment to be levied for an object model transferred to an object recognition device. In this way, the cost for maintaining object model can be carried by all beneficiaries. On the other hand, since an object recognition device that delivers images of objects that can be used to augment an object model is actually contributing to the powerfulness of the overall system, this object recognition device might receive its augmented object model at a discount. In another possible embodiment, an object recognition device might subscribe to a service of the object model service platform in order to automatically receive the newest object model for a particular object without having to specifically request it. In this way, the object recognition device always avails of the most up-to-date version of the subscribed object model.
Other objects and features of the present invention will become apparent from the following detailed descriptions considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for the purposes of illustration and not as a definition of the limits of the invention.
Fig. Ia shows two different objects of the same object class; Fig. Ib shows a class hierarchy for the objects of Fig. Ia; Fig. 2 shows an object recognition device according to an embodiment of the invention;
Fig. 3 shows an object model provision system according to an embodiment of the invention.
In the drawings, like numbers refer to like objects throughout. Fig. Ia shows two different types of telephone 10, 11. The telephone 10 on the left is a desk telephone with a separate handset attached by a cord to the housing, and features a user interface or keypad with numerous buttons. The telephone 11 on the right is a mobile phone with a small keypad and display. Although both devices 10, 11 are very different in appearance, they both share properties of a top-level object class 'Telephone'. This object class can be further refined with subordinate classes for the different categories of telephone, as is shown in Fig. Ib, where Mi represents an object model for the 'Telephone' class in a class hierarchy. One subordinate class Mi0, at a lower level than Mi, is an object model for the desk telephone, and subordinate class Mn is an object model for a mobile phone.
A possible ontological description of a telephone contains functional properties like 'has microphone' and 'has speaker' as well as appearance properties like 'shows 10 digits'. A desk telephone inherits these items, but will have appearance properties like 'has cord', 'has receiver (and/or a description of the appearance of a receiver as being 15-20 cm long with two thicker parts at both ends)', and 'has digits ordered in a 3 by 4 matrix', while a cordless phone has the functional property 'has antenna' and the appearance property 'has digits ordered in a 3 by 4 matrix', and 'can have a display'. In Fig. 2, an object recognition device Di is shown. This object recognition device Di is a home dialogue system Di in the form of a robot which is capable of autonomously moving about and generating images of objects which it wishes to identify. The robot Di has an image analysis module 52 capable of performing image analysis on the images generated by the camera 3 and object classification using object models stored in its own memory 53. In the diagram, the home dialogue system Di has approached a telephone 12 and has generated an image 4 of the telephone 12. Image analysis of the image 4 fails to identify any properties or characteristics corresponding to one of the object models in its own object model collection. Therefore, the robot Di transfers the image 4 to an object model service platform, as will be described below.
Fig. 3 shows an object model provision system 1 with an object model service platform 2 and a plurality of object recognition devices Di, D2, D3. In this embodiment, the object recognition device Di is a home dialogue system equipped with a camera 31 and capable of autonomous movement, the object recognition device D2 is a surveillance unit with a camera 32, and the object recognition device D3 is a personal computer with an image acquisition unit 33 in the form of a moveable webcam 33. Each of the cameras 31, 32, 33 can be used to generate images of objects. Any number of object recognition devices can communicate with the object model service platform, but only three are shown here for the sake of clarity. The object model service platform 2 maintains a collection of object models Mi, ..., Mn in an ontological class hierarchy H, stored in a database 48. Only two object models Mi, M2 are shown, but it will be clear that the class hierarchy can comprise any number of object models, conceivably limited only by the amount of memory space or processing power available. In the same way, although only a limited number of subordinate models Mi0, ... , M202 is shown, it will be clear than an object model can comprise an indefinite number of subordinate object model levels, or no subordinate object model levels at all.
The object recognition devices Di, D2, D3 can submit requests to the object model service platform 2 and can receive object models Mi, ..., Mn from the object model service platform 2 by means of suitable interface signals 301, 302, 303.
In this example, the object recognition device Di transfers an image of an unidentified object to the object model service platform 2 by means of an image transfer interface 50 and the interface signal 301. Let us assume that the image is that of the old- fashioned telephone described in Fig. 2. A communications interface 40 of the object model service platform 2 receives the request and interprets it in a request interface 41. In this case, the request interface 41 concludes that the object recognition device Di requires an object model for the object in the image, and forwards the information to a maintenance unit 43. In the maintenance unit 43, an image analysis unit 44 performs image analysis of the image and identifies features or properties of the object in the image. Using this information, an object classification module 46 concludes that the object in the image is a telephone, since it shares features of the object model class Mi for 'Telephone' in the collection of object models in the class hierarchy H, but that it should not be classified to either of the two existing subordinate object classes Mi0, Mn for 'Desk telephone' and 'Mobile telephone' respectively. Therefore, the augmentation unit 45 decides to create a new subordinate object class Mi2. As already described above, the properties of the top-level model Mi are adjusted if necessary, with a subsequent inheritance of the adjusted top-level properties to the subordinate object models Mi0, Mn. The augmented object model Mi for 'Telephone' previously consisting of a bundle of two sub-models now comprises three subordinate models Mi0, Mn, Mi2. Once the augmentation process is complete, the maintenance unit 43 can forward a copy of the augmented object model Mi including its subordinate object models Mi0, Mn, Mi2 to an object model transfer interface 42 in the communications interface 40 to carry out the transfer to the object recognition device Di. Prior to transfer of the object model, it may be converted into a form suitable for interpretation by the object recognition device Di, since different types of object recognition device may work with different types of object model representation. The object recognition device Di then receives the information in an object model receiver interface 51. Thereafter, the object recognition device Di can avail of the updated object models and will be able, in future, to recognise the previously unidentifiable telephone and possibly also telephones of similar design.
A billing unit 47 is shown in the object model service platform 2. This billing unit 47 can determine any charges to be levied for supplying the object recognition device Di with an augmented model on request or according to subscription, or any discounts to be granted to that object recognition device for images used to augment an object model in the collection of object models. Although the present invention has been disclosed in the form of preferred embodiments and variations thereon, it will be understood that numerous additional modifications and variations could be made thereto without departing from the scope of the invention. For example, the object model service platform can also be used to provide a set of object models for a new object recognition device such as a home dialog system. Such a system might not be equipped with any object models, or might be equipped with just a basic list of object models. In such a situation, the user of the home dialogue system might simply put together a list of objects that he considers relevant for the functions that his home dialogue system is to carry out, and can order the appropriate object models from the object model service platform by issuing a suitable request. Once equipped with a set of models in this way, the home dialogue system can autonomously request updates or new object models from the object model service platform, as the necessity arises.

Claims

CLAIMS:
1. A method of providing object models (Mi, Mi0, ..., Mn) to a plurality of object recognition devices (Di, D2, D3), which method comprises
- maintaining a collection of classified object models (Mi, Mi0, ..., Mn) in an object model service platform (2); - interpreting a request from an object recognition device (Di, D2, D3) to retrieve an object model (Mi, Mi0, ..., Mn) required by the object recognition device (Di, D2, D3);
- transferring the requested object model (Mi, Mi0, ..., Mn) to the object recognition device (Di, D2, D3).
2. A method according to claim 1, wherein a request from an object recognition device (Di, D2) comprises an image (4) of an object (12) for which a corresponding object model (Mi, Mi0, ..., Mn) is to be retrieved from the object model service platform (2).
3. A method according to claim 2, wherein the object model (Mi, Mi0, ..., Mn) corresponding to the object (12) is augmented according to object class properties obtained from analysis of the image (4) of the object (12) to give an augmented object model (Mi, Mi0, ..., Mn).
4. A method according to claim 3, wherein the augmented object model (Mi, Mio, • • ., Mn) is transferred to the object recognition device (Di, D2, D3).
5. A method according to any of the preceding claims, wherein the object models (Mi, Mi0, ..., Mn) are maintained in the object model service platform (2) according to an object class hierarchy (H).
6. A method according to any of claims 2 to 5, wherein a request from an object recognition device (Di, D2, D3) comprises a specification of the object class of an object in an image (4), and the object model (Mi, Mi0, ..., Mn) corresponding to the specified object class is augmented.
7. A method according to any of claims 2 to 6, wherein a dedicated object model (Mi, Mi0, ..., Mn) is transferred from the model service platform (2) to an object recognition device (Di, D2, D3) in response to a request for that object model (Mi, Mi0, ..., Mn).
8. A method according to any of the preceding claims, wherein object class properties of an object model (Mi, Mi0, ..., Mn) are inherited by object models (Mi, Mi0, ..., Mn) of a subordinate class in the object class hierarchy (H).
9. An object model provision system (1) comprising - a plurality of object recognition devices (Di, D2, D3); an object model service platform (2) with a maintenance unit (43) for maintaining a collection of object models (Mi, Mi0, ..., Mn); a request interface (41) for interpreting a request from an object recognition device (Di, D2, D3) to retrieve an object model (Mi, Mi0, ..., Mn) required by the object recognition device (Di, D2, D3); an object model transfer interface (42) for transferring the object model (Mi, Mio, • • ., Mn) to the object recognition device (Di, D2, D3).
10. A system according to claim 9, comprising a billing unit (47) for determining a payment to be levied for an object model (Mi, Mi0, ..., Mn) transferred to an object recognition device (Di, D2, D3).
11. An object model service platform (2) for use in an object model provision system (1), comprising a maintenance unit (43) for maintaining a collection of object models (Mi,
Figure imgf000019_0001
a request interface (41) for interpreting a request from an object recognition device (Di, D2, D3) to retrieve an object model (Mi, Mi0, ...,
Mn) required by the object recognition device (Di, D2, D3); an object model transfer interface (42) for transferring the object model (Mi, Mio, • • ., Mn) to the object recognition device (Di, D2, D3).
12. An object recognition device (Di, D2, D3) for use in an object model provision system (1), comprising
- an image acquisition unit (3) for acquiring an image (4) of an object (12);
- an image transfer interface (50) for transferring the image (4) of the object (12) to an object model service platform (2); - an object model receiver interface (51) for receiving an augmented object model (Mi, Mi0, ..., Mn) from the object model service platform (2).
PCT/IB2007/052337 2006-06-29 2007-06-19 Method of providing object models to a plurality of object recognition devices WO2008001265A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP06116324.2 2006-06-29
EP06116324 2006-06-29

Publications (1)

Publication Number Publication Date
WO2008001265A1 true WO2008001265A1 (en) 2008-01-03

Family

ID=38577274

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2007/052337 WO2008001265A1 (en) 2006-06-29 2007-06-19 Method of providing object models to a plurality of object recognition devices

Country Status (1)

Country Link
WO (1) WO2008001265A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2767914A1 (en) * 2013-02-15 2014-08-20 Samsung Electronics Co., Ltd Electronic device and object recognition method in electronic device
KR20140103029A (en) * 2013-02-15 2014-08-25 삼성전자주식회사 Electronic device and method for recogniting object in electronic device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1324223A2 (en) * 2001-12-29 2003-07-02 Lg Electronics Inc. Apparatus and method for searching multimedia objects
JP2005279804A (en) * 2004-03-29 2005-10-13 Advanced Telecommunication Research Institute International Object recognition device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1324223A2 (en) * 2001-12-29 2003-07-02 Lg Electronics Inc. Apparatus and method for searching multimedia objects
JP2005279804A (en) * 2004-03-29 2005-10-13 Advanced Telecommunication Research Institute International Object recognition device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ESCUDERO-RODRIGO D ET AL: "A distributed 3D vision system", WORLD AUTOMATION CONGRESS, 2004. PROCEEDINGS SEVILLE, SPAIN 28 JUNE - 1 JULY 2004, PISCATAWAY, NJ, USA,IEEE, 28 June 2004 (2004-06-28), pages 163 - 168, XP010808403, ISBN: 1-889335-21-5 *
MARIN R ET AL: "Including Efficient Object Recognition Capabilities in Online Robots: From a Statistical to a Neural-Network Classifier", IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: PART C: APPLICATIONS AND REVIEWS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 35, no. 1, February 2005 (2005-02-01), pages 87 - 96, XP011125670, ISSN: 1094-6977 *
TRIGGS B ET AL.: "The Oxford Robot World Model", PROC. NATO ASI EXPERT SYSTEMS AND ROBOTICS, August 1990 (1990-08-01), pages 1 - 10, XP002455931 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2767914A1 (en) * 2013-02-15 2014-08-20 Samsung Electronics Co., Ltd Electronic device and object recognition method in electronic device
CN103995818A (en) * 2013-02-15 2014-08-20 三星电子株式会社 Electronic device and object recognition method in electronic device
KR20140103029A (en) * 2013-02-15 2014-08-25 삼성전자주식회사 Electronic device and method for recogniting object in electronic device
US9367761B2 (en) 2013-02-15 2016-06-14 Samsung Electronics Co., Ltd. Electronic device and object recognition method in electronic device
CN103995818B (en) * 2013-02-15 2019-06-28 三星电子株式会社 Object identifying method in electronic equipment and electronic equipment
KR102209447B1 (en) 2013-02-15 2021-02-01 삼성전자주식회사 Electronic device and method for recogniting object in electronic device

Similar Documents

Publication Publication Date Title
CN108874967B (en) Dialogue state determining method and device, dialogue system, terminal and storage medium
JP2022504704A (en) Target detection methods, model training methods, equipment, equipment and computer programs
JP6471934B2 (en) Image recognition method, camera system
KR102663375B1 (en) Apparatus and method for automatically focusing the audio and the video
US11410677B2 (en) Adaptive sound event classification
CN110096580A (en) A kind of FAQ dialogue method, device and electronic equipment
WO2010022185A1 (en) Methods and systems for content processing
CN108664651B (en) Pattern recommendation method, device and storage medium
EP3238015A2 (en) First-person camera based visual context aware system
CN111044045A (en) Navigation method, device and terminal device based on neural network
US20220038846A1 (en) Angle of arrival capability in electronic devices
CN113611318A (en) Audio data enhancement method and related equipment
CN110942046A (en) Image retrieval method, device, equipment and storage medium
Ma et al. RF-Siamese: Approaching accurate RFID gesture recognition with one sample
CN114902249A (en) Method, system and classification method, system and medium for generating image recognition model
CN106491322A (en) Blind-man crutch control system and method based on OpenCV image recognitions
CN109993234A (en) A kind of unmanned training data classification method, device and electronic equipment
CN109543579A (en) Recognition methods, device and the storage medium of target object in a kind of image
WO2008001265A1 (en) Method of providing object models to a plurality of object recognition devices
CN106054641A (en) Method, apparatus, and system for turning on intelligent household appliance control interface
CN111626315B (en) Model training method, object recognition method, device, medium and electronic device
CN115309882A (en) Interactive information generation method, system and storage medium based on multi-modal characteristics
CN114610677A (en) Method for determining conversion model and related device
KR20210048271A (en) Apparatus and method for performing automatic audio focusing to multiple objects
CN109858378A (en) Vending machine good selling method and device, medium and equipment based on living things feature recognition

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07789718

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

122 Ep: pct application non-entry in european phase

Ref document number: 07789718

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载