WO2000045293A1 - Method for generating multimedia document descriptions and device associated therewith - Google Patents

Abstract

A method for generating multimedia document descriptions, comprising an analysis step and a step whereby a document is segmented into a plurality of objects that are associated with various plausbilities; a feature description step whereby functions are applied to said objects, the feature descriptions comprise abstract descriptions containing a reference to a described object and functions describing reflex mechanisms; in addition to a step in which the abstract feature descriptions are activated. The feature description generation functions consist of the following: a similarity function with a pre-established collection of prototypes is applied to a multimedia object, exceptions are created by accepting new feature descriptions that contradict the feature descriptions that are already obtained, the feature description generation functions are modified by a user profile comprising a modeled set of knowledge data specific to a user for whom said description is designed in the form of an Or node graph that is linked by means of arcs, stored in a work area containing an organized feature description set with a delete function for said feature descriptions and which can be transmitted by a server to an external system that is capable of responding to a request from a user via a data transmission network.

Description

 METHOD FOR GENERATING MULTIMEDIA DOCUMENT DESCRIPTIONS, AND ASSOCIATED DEVICE

The invention relates to the field of generating descriptions of multimedia documents, such as, in particular, audiovisual documents, video sequences, etc.

It is already known (MPEG-4 standard) to represent an audiovisual document as a set of objects, of behavior of these objects and of relations between these objects, an object being able to be the document itself or a part of this document. Typically, multimedia objects can be video or film sequences, a shot of a film, a musical phrase, etc.

It was proposed in the initial specifications of the draft standard MCDI (Multimedia Content Description Interface) also called MPEG-

7, to define the structure of a multimedia document by means of a description schema which represents the document as a graph of objects, a language for defining the description schema and descriptors, but the contents and the properties description schemes and the definition languages of these schemes have yet to be defined.

The subject of the invention is in particular a method for generating descriptions of multimedia documents in accordance with one or more description schemes, whatever the language used to describe these schemes, provided that said language has the desired expressiveness and properties.

The subject of the invention is also a method of this type which makes it possible to extract a maximum quantity of useful semantic information from a multimedia document from audiovisual content and ontologies, while minimizing the cost and the complexity of the calculation. necessary, this must converge into a finite number (which can be arbitrarily limited) of calculation steps. The invention also relates to a method of the aforementioned type, comprising steps of defining a user profile making it possible to personalize the descriptions of the multimedia documents according to a body of knowledge relating to a user or to a group of users. , so as to be able to offer this user or group of users a personalized selection of a certain number of multimedia documents and allow the production of a document in accordance with the wishes of the user or group of users.

To this end, the invention provides a method for generating descriptions of multimedia documents in a first processing unit (for example housed in a server) comprising a working memory and coupled to a second processing unit (for example housed in a external "client" type system (in this case the coupling is carried out via a data transmission network) also comprising a working memory and making it possible to respond to a request formed by a user with a view to accessing one at least of these documents, and / or to navigate among these documents and / or to interpret them. This method comprises in a known manner a first step of analysis and temporal and / or spatial segmentation of a multimedia document into a plurality of multimedia objects with which likelihoods are associated, and it is characterized in that it also comprises a second step of generating descriptor objects by applying functions to the multimedia objects of the plurality, these descriptor objects being stored in at least one of the working memories of the first and second processing units in correspondence of likelihoods and comprising descriptors information, action descriptors and abstract descriptors, the latter each containing a reference to a described object and functions defining reflex mechanisms, the descriptor generation functions comprising at least one of the functions consisting in:

- apply a similarity function to a multimedia object with a pre-established collection of multimedia object prototypes or descriptors, - create exceptions by accepting new descriptors which contradict descriptors already obtained when the likelihood of a new descriptor is greater than that of the descriptor already obtained and the difference in their likelihood is greater than or equal to a predetermined threshold,

- modify the descriptor generation functions by a user profile comprising a modeled set of knowledge specific to a user receiving a description, the user profile consisting of a graph of nodes connected by arcs, stored in one at less working memories of the first and second processing units, containing an organized set of descriptors with a function for deleting descriptors, and being suitable for being established by one and / or the other of the first and second processing units treatment. According to other characteristics of the invention, which can be taken separately and in combination, the method also consists in:

- in the first step, add to the multimedia objects reference descriptors describing the relationships between the multimedia objects and the objects from which they come, - in a third step, activate abstract descriptors by triggering a reflex mechanism of a abstract descriptor in response to a change in its state or to a request on its state, this activation making it possible to generate other descriptors,

- use one or more ontologies in the reflex mechanisms of abstract descriptors,

- activate abstract descriptors in order to achieve a pseudo-inheritance, consisting of a transfer of a descriptor along a relationship between descriptors, preferably a relationship of type content / container,

- use a procedural code, such as for example an algorithmic method of segmentation or information extraction, for the segmentation of a multimedia document in the first step or the generation of descriptors in the second step, - the triggering of a reflex mechanism of an abstract descriptor causing the triggering of reflex mechanisms of other abstract descriptors, use a mechanism of marking these abstract descriptors to avoid activations in closed loop of two or more of these descriptors abstract,

- create in the first stage a collection of prototypes by application to one or more multimedia objects of interactions (or introduction of data by an external agent) associated with an expert weight or plausibility, - use production rules in the generation functions descriptors of the second step, and possibly create rules by applying meta-rules to other rules,

- use interactions, allowing the production of annotations, in the descriptor generation functions of the second step, - maintain the consistency of the descriptors generated in the second step, by deleting the descriptors from a descriptor contradicted by an exception,

- define a user profile by a graph of nodes connected by arcs and by a working memory containing an organized set of descriptors with a function for deleting descriptors, the establishment of the user profile consisting in filling the working memory of the one and / or the other of the first and second processing units by a set of descriptors, in real time or in deferred time, to delete from memory a set of previous descriptors, to calculate input values of the nodes of the profile, to calculate activation values of the nodes, to generate a new set of descriptors and to enter it into the working memory, each node containing an aggregate function of the inputs, an input value, a function of activation or transition, an output function, a descriptor generation function and a value of interest, digital weighting values being assigned to the arcs connecting the s knots. The method can include an arbitration function determining, among nodes which are candidates for activation, those which are activated at a given time, as a function of a descriptor of the number of nodes which can be activated and of the activation parameters associated with each node.

A weighting value of an arc can for example correspond to the probability that the user who has seen a descriptor contained in a first node of the graph wishes to see another descriptor contained in another node of the graph connected to the first node by the arc in question, or the probability that the user thinks of the other descriptor when he has seen the descriptor contained in the first node.

Advantageously, this method can also consist in adding information of an emotional nature to the user profile, by associating with each node a vector of sensitivity to affects for each of the functions of the node and a vector of increment of global affect, a vector of sensitivity to affects at each arc and a context of affects or vector of affects global to the whole graph, the input, activation or transition and descriptor generation functions contained in the nodes being modified to take into account the global affect vector and affect sensitivity vectors.

The descriptor generation function generates the descriptors from the activation value of the node, from the affect sensitivity vector of this function and from the global affect vector.

The weighting values of the arcs and the vectors of sensitivity to the affects of the nodes can be obtained by automatic capture of the user profile. To do this, we can record the user's reactions to a document viewing session during which he can, each time, move more or less quickly from one document to another or view an entire document. document. The frequency of passage from a document containing a certain descriptor to a document containing another descriptor and the display time of each document are used to weight the graph as follows: - the weighting values of the arcs are initialized to the values given by the transition probabilities between two descriptors obtained from the frequencies of passage between the descriptors, the vectors of sensitivity to affects of the nodes are obtained from the viewing times of the documents by the user, for example by multiplying the context of affect of the user during viewing by the ratio viewing time / total duration of the document.

However, the user profile can be acquired by any other appropriate means. Means can also be provided to anonymize the user profile in particular with a view to its transmission by the second processing unit to the first processing unit. This applies particularly to the case where the second processing unit is housed in an external client-type system and it must be transmitted to an external system such as, for example, a server in a data transmission network. .

In a particular application, the method is arranged to perform the description operations on the first processing unit, the operations for responding to user requests on the second processing unit, the latter being identical to the first processing unit , and document navigation operations in a third processing unit. In this case, the first and second processing units can be housed in a server and the third processing unit in a client. The method according to the invention can be executed by iteration, the constraint of accepting exceptions guaranteeing its convergence.

This process can take into account several user profiles, constituted in the aforementioned manner and of which a copy of the state is saved in working memory, a selection and arbitration function determining the user profiles active at a given time.

In general, the method according to the invention makes it possible to generate personalized descriptions of multimedia documents of all types (films, video sequences, video images supplied by video surveillance cameras, video images supplied by an optoelectronic vision system of a robot, sound signals, etc.) and has the essential advantage of providing an optimized description of a document multimedia, adapted to a given user at a given time.

The invention also relates to a device for generating multimedia document descriptions, comprising in a known manner a first processing unit coupled to a second processing unit arranged to transmit to the first processing unit requests made by a user for the purpose of access at least one of the multimedia documents, and / or navigate among these documents and / or interpret them.

The device according to the invention is characterized in that the first processing unit comprises: ^* clock means making it possible to define cycles,

^* a working memory subdivided into addressable registers and capable of containing descriptors,

^* an auxiliary memory capable of storing collections of examples,

^* a first calculation module making it possible to calculate a similarity function between objects from the outside world, an image of which is contained in a multimedia document, and the collections of examples, and to deliver a primary descriptor representative of the calculated similarity function (some of these primary descriptors may be exceptions introduced by a user), ^* means for managing the addressing, in writing, of the registers of the working memory, arranged to supply all the N cycles (N being greater than or equal at 1) a first address for storing the primary descriptor in the working memory,

^* A second computing module supplied with at least two descriptors stored at selected addresses in the main memory and for generating at least a first and a second secondary descriptors received from these descriptors, and * a third calculation module for calculating a derived descriptor from the first secondary descriptor generated by the second calculation module, the address management means being further capable of addressing the second secondary descriptor and the derived descriptor respectively to registers of the working memory M placed at second and third addresses chosen, with a view to their memorization in the following cycle, and the second calculation module being further capable of determining, from the secondary and derived descriptors, a descriptor d action forming control information for external equipment.

In a particularly advantageous embodiment, the second calculation module is capable of delivering the first secondary descriptor in the form of an affect increment, and the processing unit comprises, between the second and third calculation modules, a adder used to add the increment of affect to the current derivative descriptor in the third module and to transmit the result of this addition to the third calculation module, so that it can determine a new derived descriptor intended to be stored in the register placed at the third address chosen during the next cycle. Furthermore, the second calculation module can be subdivided into a multiplicity of sub-modules, and the first processing unit can include a selection module making it possible to select one of the sub-modules to supply the register with the second secondary descriptor. placed at the second address. This selection module can be arranged so as to select a sub-module by applying a selected function to the increments of affect delivered by each of the sub-modules. Advantageously, these calculation sub-modules and the selection module can be arranged in parallel on a bus.

In one embodiment, the device may also include the second processing unit, which in this case is substantially identical to the first processing unit. The first processing unit is then preferably arranged to carry out the document description operations while the second processing unit is arranged to perform the response operations to user requests. A third processing unit can also be provided to perform document navigation operations.

The first and second processing units can be housed respectively in server and client type systems each comprising a communication interface, the first and second processing units then being coupled by a data transmission network, via the communication interfaces. However, these first and second processing units can also be housed in a server comprising a communication interface, while the third processing unit is housed in an external “client” type system also comprising a communication interface, the clients and server being coupled by a data transmission network, via their respective communication interfaces.

The invention will be better understood and other characteristics, details and advantages thereof will appear more clearly on reading the following description of various embodiments of the invention, given by way of example with reference to the drawings annexed in which:

FIG. 1 illustrates the mode of detection of a movement in images of a film,

FIG. 2 illustrates the concept of pseudo-inheritance,

FIG. 3 is a diagram for weighting the arcs of a graph of user profile,

FIG. 4 schematically represents a node of this profile,

FIG. 5 represents an operating mode of a user profile,

- Figure 6 shows a node of a user profile with the addition of an emotional profile,. FIG. 7 represents an operating mode of the user profile comprising an emotional profile, FIG. 8 illustrates a first embodiment of a processing device implementing the method according to the invention,

- Figure 9 illustrates a variant of the embodiment of Figure

8, FIG. 10 illustrates a variant of the embodiment of the figure

9, and FIG. 11 illustrates a variant of the embodiment of FIG. 10.

The method according to the invention aims to generate a description of multimedia objects, in particular digitized audiovisual documents

(films, documentaries, television programs, etc.), and also other types of objects, of different natures, available on different formats or supports, digital or not, such as video cassettes, books, web pages ", and a priori any object of a real, imaginary or virtual world. A multimedia object is in its most general acceptance an entity of a real, imaginary or virtual world, which can be perceived. The perceptions involved are not limited to visual or auditory perceptions and also include tactile, olfactory and gustatory perceptions. There are systems incorporating odor and touch sensors, to which the method according to the invention is applicable.

This process makes it possible to generate descriptors from basic information (a digitized document, a core of descriptors, etc.) or from an information flow, in this case in "real time", the information for example from a surveillance camera. In the "real time" case, we do not have a database of multimedia objects that we must analyze, but a changing set of multimedia objects and the description must also evolve over time.

The description generated in real time or in deferred time can be used for several classes of applications. A distinction is made between "pull" applications, "push" applications and specialized applications. "Pull" applications respond to a request made explicitly by a user. The push applications anticipate the wishes of the users and make proposals to them. Specialized applications cannot be directly classified into one of the two preceding categories and touch on fields as varied as medical imaging, surveillance of roads or sensitive places, mobile guided systems (missiles, assisted driving cars, etc. .), home automation (intelligent sensors), and robotics: "intelligent" robots analyze the perceived world and must therefore have a description of this world from multimedia objects (sensor images and sounds). In the case of push applications, the system must have a model of the person using it and this model is part of a description of the real world. Almost all computer applications must also incorporate this type of information, including pull and specialized applications. A concrete example can be the case of a robot vacuum cleaner which, in order to disturb its owner as little as possible, will have to use a model describing it. He will then be able to know the ideal moment for its triggering and the paths that its owner conventionally takes in the apartment, to know that when its owner is angry, it is better not to trigger, but that if he is going to receive friends at home, he must go off. The descriptions of the world necessary are very different in nature: model of the user, model of the rooms in the apartment, etc.

The information describing the users (and more generally human beings) is very varied in nature. It is possible to model tastes, moods, habits, associations of classic ideas, etc. With such expressiveness, these models contain very sensitive data relating to privacy. It is therefore necessary to protect them and prevent their disclosure if it is not desired, which conditions the acceptance and use of systems based on these models. This problem is all the more important as these systems will certainly be connected to computer networks. The invention proposes in this context, a method the aim of which is to generate a personalized description of a multimedia object, thanks to a body of knowledge on the object and on the world in which it evolves. We have methods for segmenting multimedia objects, knowledge provided in the form of production rules, knowledge provided by a set of examples (prototypes), and knowledge of users (user profiles).

The description of the process according to the invention calls upon a certain number of specialized expressions, the definitions of which are given below.

An object: either a descriptor object or a multimedia object. A collection of objects is an object.

A multimedia object: a multimedia object is in its most general acceptance an entity of a real, imaginary or virtual world, which can be perceived (tactile, olfactory, gustatory, visual and / or auditory perceptions).

A descriptor object (or descriptor) is an informative descriptor or an action descriptor. An action descriptor triggers an action on the outside world as soon as it is generated (recording a sequence, saving to tape, showing or hiding, etc.). It has the distinction of not being able to be contradicted by an exception. It has a type and a value.

Information descriptors represent a description of an object. They have a type, a target object (a reference to the described object) and a value.

A prototype: a prototype is an object (multimedia or descriptor) which serves as an example.

A function: any application of a set into another set. An aggregate function is a multivalued function that provides a result value. For example, it can be a function of several vectors which returns a vector. The aggregation functions are used in particular to calculate the value of a node from the inputs of this node, or the value of the global affect vector from the increment vectors of the activated nodes. A preferential realization is a sum of the source values.

Collections: a collection is either an object or a collection plus a weight. A collection of multimedia objects is also a multimedia object.

An interaction: any input of data into the program, carried out by an agent external to the system (a user or an IT agent).

A similarity function: it is a function which makes it possible to establish a similarity between two collections of objects. The preferential realization will return a degree of similarity, i.e. a real value between 0 and 1.

An abstract descriptor: it is a descriptor which contains a reference to the described object and a set of functions defining reflex behaviors. He reacts to a change in his state or to a request on his state.

Likelihood: one or more values representing, for example, the credibility, plausibility, usefulness or weight of an object, an interaction or a user. Likelihoods can be combined using logical operators of gradual logic. A preferential realization is to use a real value (likelihood) or two real values (plausibility, importance) with calculations based on fuzzy logic or possibilistic logic.

The steps of the process can be presented in a synthetic form whose operator semantics is as follows: suivante means "associated with", ® means "composed with", => means "generates" and ^* means "one or more" and - means "withdrawal".

Description and descriptors:

A description is defined as a set of descriptors and multimedia objects linked together. A descriptor is a mini-description of one of the objects of the global description. A descriptor is generally composed of a type, a value and a reference to the object described. Abstract descriptors incorporate reflex behaviors. The generation of an action descriptor involves actions on the outside world such as, for example, the recording by a video recorder of a scene from a film. Transient descriptors are removed from the description periodically or on a fixed date. Finally, a likelihood value measures the credibility of each descriptor.

The main process loop: It consists of incrementally completing the description of the multimedia object with a set of functions which, when applied to the description, generate new descriptors or new multimedia objects. The process is broken down into three main stages:

Step 1: The segmentation of multimedia objects

It consists in producing multimedia objects from a multimedia object. The objects produced generally describe the object from which they come. Its synthetic form is:

(Objet_MM Likelihood) (Likelihood Function) (Objet_MM Likelihood) ^*

An extended version also generates a descriptor. This descriptor describes the relationship between multimedia objects: spatial, temporal composition, etc. (Objet_MM Likelihood) (Likelihood Function) ((Objet_MM Vraisemblanœ) (Objet_Descripteur Vraisemblanœ)) ^*

Since a collection of multimedia objects is also a multimedia object, the two preceding rules can therefore generate collections of multimedia objects.

Specialization by a procedural code:

(Objet_MM Vraisemblanœ)

Vraisemblanœ) (Objet_MM Vraisemblanœ) (Objet_Desαipteur Vraisemblanœ)

This specialization of the first stage of the process can be illustrated by the methods of segmenting films into shots, which attempt, algorithmically, to identify all the shots in a film. They present their results in the form of a set of images called characteristic images and they associate the shots of a film with a film by a relation of composition. These methods do produce a set of multimedia objects and a set of descriptors by applying a procedural code to another multimedia object.

Example of segmentation into plans by an algorithmic method:

Each image of a film is characterized by its average intensity and the intensity of each image is compared with those of the n previous images. As soon as you spot too large an irregularity, you assume that there has been a change of plan. This simplistic method allows the detection of clips in a video stream. The planes detected are characterized by one of the images they contain and the film is segmented into a set of characteristic images. Step 2: Generating new descriptors

The purpose of this step is to generate descriptors from other descriptors. Its general synthetic form is:

(Vraisemblanœ object) ^* (Vraisemblanœ function) (Vraisemblanœ descriptor object) A specific version of this synthetic form will be used quite frequently:

(Vraisemblanœ object) ^* (Vraisemblanœ function) (Descripteur_abstrait Vraisemblanœ)

This specific form proposes to create an abstract descriptor, a specific version of a descriptor object defined by the conjunction of a descriptor, a reference to an object and a set of reflex behaviors. These reflexes can be triggered when one modifies his state or makes a request on his state. Specialization by a procedural code:

The function that generates a descriptor object from objects can be a procedural code:

(Likelihood object) (Vraisemblanœ jrccédural code) (Vraisemblanœ descriptor object)

This specialization of the second step of the process can be illustrated by methods of extracting movement information from films. These methods extract, in an algorithmic way, information such as "traveling", "zoom in", "zoom out", etc.

As shown in FIG. 1, each image of a film can be divided into areas of nxn elementary points called "pixels". Each of the areas of image i is searched for in image i + 1 around the initial area. This gives a displacement vector for each pixel area between two images, from which a particularly conventional camera movement can be detected. In the example in FIG. 1, each zone of image i, found in image i + 1, has been numbered. We deduce a map of the movement and we isolate the most probable camera movement, here a "traveling". Specialization by production rules:

The function that generates a descriptor object from objects can be a production rule.

(Vraisemblanœ descriptor) ^* (Vraisemblanœ rule) (Vraisemblanœ descriptor)

Common sense knowledge can be modeled by production rules.

Example of knowledge inference by rules:

For example, the simultaneous presence of the descriptors "sport", "sea", "ski" make it possible to infer the descriptor "water skiing" with good likelihood. The integration of common sense knowledge can be done through production rules which produce descriptors from one or more descriptors already present in the description. Let the descriptors:

(sea 0.99) (sport 0.95)

And the rule: if sea and sport infer (min-likelihood) water-skiing. The description is completed with the descriptor:

(water skiing 0.95)

Specialization by annotations:

The function that generates a descriptor object from objects can be an external function that interacts with the system. An interaction is any input of data into the program, carried out by an agent external to the system (a user or an IT agent).

(Vraisemblanœ descriptor) ^* (Weight_expert interaction) (Vraisemblanœ descriptor) Adding descriptors via a graphical interface is an example. These descriptors are called annotations.

References :

The spatial segmentation of images into objects is a problem dealt with in particular in the MPEG-4 standard. Their segmentation is generally based on movement information (cf. Overview of the MPEG-4 Standard ", Rob Koenen, ISO / IEC JTC1 / SC29 / WG11 N2459). The integration of common sense knowledge is dealt with in the related articles in the field of ontologies, an approach of which is presented in "Formai Ontology in Information Systems", N. Guarino,

Proceedings of the first international conference on Formai Ontology and Information Systems (FOIS'98), IOS Press. The ontologies are based on one or more taxonomies of concepts, object properties (inheritance, composition) and axioms. The method according to the invention provides for the application of ontologies to multimedia descriptions by integrating therein mechanisms of abstract descriptors, annotations and exceptions.

Step 3: activation of the abstract descriptors

An abstract descriptor reacts to a change in its state or to a request on its state. This reaction is comparable to a reflex mechanism. The abstract descriptor then performs an action which can possibly induce reflex mechanisms in other abstract descriptors (creation of descriptor objects, change of values of a descriptor, ...). This action can be carried out either directly by calling another abstract descriptor, or indirectly by creating one or more transient descriptors triggering other abstract descriptors.

A marking mechanism prevents looping of the activation of reflexes, for example: if D_A ₁ stimulates D_A ₂ then if D_A ₂ stimulates D_A., For the same action, the reflex mechanism is stopped. Synthetic shape

(Descripteur_abstrait Vraisemblanœ) (function Vraisemblanœ) (Objet_descripteur Vraisemblanœ)

(Vraisemblanœ descriptor object)

Abstract descriptors provide many very useful properties thanks to this reflex mechanism, such as for example:

- Consistency check: if the value of a descriptor is modified, a check can be carried out: for example, checking that the start date of a plan is well before the end date.

- Special properties of descriptors: for example, the transitivity property of a descriptor can be implemented by an abstract descriptor. The capacity property is: "if contains (x, y) and contains (y, z) then contains (x, z)". In the example of a plan which contains an object X and which is contained in a scene of a film, to the question "do you contain the object X?", The scene will be able to answer positively by propagating the question to the plans it contains. This propagation can be done either during the initial creation of the descriptors (by transitive closure on the abstract descriptors), or to reduce the initial calculation time, during a specific request from a user or the system.

- Integration of ontological knowledge: for example, a descriptor "place" whose value is "Paris", if it is linked to an ontology, can answer "true" to the request "are you in France?".

Exceptions and maintaining consistency:

The descriptors generated during steps 2 and 3 can call into question descriptors already present in the description. These "new" descriptors are called exceptions. To be accepted, an exception must have a higher likelihood than that of the descriptor that it contradicts and the difference of the two likelihoods must be greater than a certain value. This constraint verifies that a descriptor is not going to be contradicted indefinitely and makes it possible to ensure the convergence of the system. In addition, there is a mechanism for maintaining consistency: when a descriptor is generated and it is contradictory with a descriptor D ,, all the descriptors from D are then deleted.

Synthetic form:

Addition of an exception: (Objet_Descripteur Vraisemblanœ _! )

(Exception Likelihood Descriptor ^

if Likelihood, -Likelihood _! >

(Descriptor_exœptJon Likelihood ^

Maintaining consistency: (Objet_Descripteur Vraisemblanœ) *

(Fct_maintien_∞herenœ Vraisemblanœ) - (Descriptor Vraisemblanœ) *

the sign - means that we remove a set of descriptors from the description. Example of exceptions:

The following descriptor is present in the description: (water ski 0.90)

A user with an expert weight of 0.95 adds the following descriptor by annotation: (windsurf 0.95)

This descriptor is an exception and replaces the descriptor "ski-nautique". Example of maintaining consistency:

The descriptor "ski-nautique" had led to the generation of the descriptor "little wind" by the application of the rule "if ski-nautique infer little wind". The arrival of the exception "windsurf" leads to the disappearance of the fact "little wind".

Specific abstract descriptors

Modeling a pseudo-inheritance

A pseudo-inheritance is an inheritance where the descriptors are transferred along the composition relationships. The notion of pseudo-inheritance is opposed to that of inheritance from object languages where inheritance takes place along "class / subclass" relationships. Pseudo-inheritance occurs along relationships between descriptors, preferably of the "content / container" type. However, other relationships between descriptors can be envisaged. Example:

In the example shown in FIG. 2, the descriptor "producer" is distributed according to the property "contains". The plans resulting from the sequences of a film have the descriptor "director = Mr. Dupont", by activation of abstract descriptors.

Specific procedural codes:

Use of prototype collections

In the second step, the function that generates a descriptor object from an object can be replaced by a measure of similarity with a collection of prototypes. A prototype is an object (multimedia or descriptor) or a collection of objects which serves as an example for a concept. Synthetic shape

- Creation of a prototype:

(Objet_MM Vraisemblanœ) (Interaction Poids_expert) (Prototype Vraisemblanœ)

- Use of prototypes:

(Objet_MM Vraisemblanœ) (Fct_de_similarité Vraisemblanœ) (Vraisemblanœ prototypes) * (Objet_Descripteur Vraisemblanœ)

- Creation of a collection

(Vraisemblanœ prototype) ^* (Weight_Expert Interaction) (Vraisemblanœ Prototype)

- Use of collections

(Vraisemblanœ object) (Vraisemblanœ similarity function) ∞llectionjDrototypes (Descriptor Ref_objet_MM Vraisemblanœ)

(collection of prototypes = tree of prototypes + weight). Image research by example is an area that has already been widely covered. The first system to have presented this kind of approach is QBIC (Myron Flicker and al., "Query By Image And Video Content: the QBIC System", Computer vol 28, Number 9, September 1995). In this case only one example was provided to the system. Subsequently, other teams proposed to establish measures of similarities between an object and a set of examples ("Indexing and retrieval of multimedia objects at different levels of granularity", P. Faudemay, G. Durand, C Seyrat and N. Tondre, Multimedia Storage and Archiving Systems III, Proceedings of SPIE, November 1998). The sets of prototypes presented were neither prioritized nor weighted by the weight of the object and the expert.

User profile: The functions of step 2 can take into account knowledge specific to the user for whom the description is intended.

One can model for example the associations of ideas of the user, his centers of interests or his mood. The generation of descriptors then becomes a generation of personalized descriptions.

Synthetic form:

(Vraisemblanœ Descriptor) * ((User Profile User Profile) Vraisemblanœ) => Descriptor Vraisemblanœ

User profile without emotional profile:

The invention proposes a method for modeling user profiles, some of which operating modes and some examples of use are described below.

The profile consists of two main components: a user profile graph and a working memory.

The user profile graph is a set of nodes linked together by arcs. The whole preferably forms a lattice, that is to say an acyclic graph of nodes. The detailed description of the nodes is given in the following paragraph. The working memory is an organized memory of descriptors.

An erase function regularly removes a set of descriptors from this memory. A privileged example of the erase function consists in associating a stamp with each descriptor, to keep in memory the n most recent descriptors. As shown in Figures 3 and 4, each node of the user graph contains an input function g ^ which is an aggregation function of the inputs, an input value e, an activation or transition function fa, a function output g-, a descriptor generation function fd and a value of interest Int ;. The arcs are weighted by numerical values a, a ', b, b'. The node is sensitive to the presence of descriptors in the profile memory.

The general modeling mechanism shown in Figure 5 is broken down into several stages for the sake of clarity. Some of these steps can however take place in parallel.

1: The working memory of the profile is filled with a subset of the description of the document. This filling can be done in real time (when the description of the document changes over time) or in deferred time (when a complete description is already available). 2: The input values of the nodes of the profile are calculated.

3: The activation values of the nodes are calculated.

4: A set of descriptors is generated and placed in the working memory of the profile.

5: Some descriptors are removed from memory by the erase function.

6: The process resumes at step 2.

In step 2 of Figure 5, given a node, its input value is calculated according to the output values of the incoming nodes, the weighting of the arcs and the likelihood of the descriptors to which the node is sensitive:

Value_output_arc_eπtranf Vraisemblanœ_descripteur ^* porκleration_arc_entranf Input_function => Input_value

The activation value of the node is calculated according to its input value by the activation function:

Input value ^* Activation function => Activation value The activation function has an arbitration role. This means that it depends on the input values of all the nodes and that it can decide to activate one of them accordingly. Thus it can activate only a finite number n of nodes. This parameter n is a global system variable. The arbitration function can for example keep only the n nodes with the largest input values. Other examples are given in the following.

Preferably, the activation function determines the activation value in two stages. In a first step, a local transition function at each node determines a possible activation value of the node, for example by comparison of the input value with a transition threshold.

In a second step, a global arbitration function activates a finite number n of nodes, for example by sorting the nodes according to their possible activation value, after having added if necessary a random value.

In a preferred embodiment, the parameters of the arbitration function depend on one or more descriptors.

The vector of descriptors to generate is given by the descriptor generation function. The calculation is done according to the activation of the node and its interest for the user:

Activation_Value Interest Function_generation_descriptor => (Descriptor Vraisemblanœ) *

The descriptors generated can be action descriptors and information descriptors. The first generate an action on the world (for example recording of a sequence by a video recorder). The two types of descriptors are placed in the working memory.

The generation of a descriptor may depend on a second global arbitration function. This function determines which descriptors, among those generated in the previous step, are actually transferred in the working memory. In this case, only the descriptors present in the working memory are used by the system, either as information descriptors, or as action descriptors.

In a preferred embodiment, this second arbitration function chooses a second number, m, of descriptors to be produced, or else a second number m1 of information descriptors and a third number m2 of action descriptors, for example as a function of likelihood of these descriptors.

The method can be extended to the selection of descriptors for any number of descriptor classes.

In a preferred embodiment, the parameters of the second arbitration function depend on one or more descriptors.

The erase function removes certain descriptors from the working memory. This function can for example remove the n oldest descriptors. Other examples are given in the following.

The output vector s, (i.e. the set of values that will be propagated along the arcs) is calculated according to the activation of the node and the output weights:

Activation_value Weight_arc_sortanf Function_output => Value_output_arc_sortanf

The probability of generation of the descriptors generated:

The generation function uses an a priori probability of generation by a given activated node of one or more descriptors. For a given node and descriptor, the probability of generating the descriptor can be a function, for example, of the reference value of this a priori probability, of its previous value and of whether or not descriptors were generated during the 'previous step.

When an activated node generates one or more descriptors, the generation function reduces the a priori probability by a value which can depend on product descriptors. This a priori probability is then brought back to the reference value, for example asymptotically.

This mechanism improves, for example, the quality of audiovisual programs generated from a user profile. In this context, the generation of a sport sequence (descriptor "sport") will weaken the probability that it will be generated again during the same session. However, a sports program will be produced with the same probability the next day.

In this process, the a priori probability can also be replaced by a likelihood coefficient. In this case, the probability of descriptor generation can for example be a non-linear function of the likelihood coefficient.

Preferential examples of the different functions:

Several preferred embodiments are proposed: - The input function performs the scalar product norm of the input weights and output values of the incoming nodes, adds the maximum likelihood among those of the descriptors present in the memory to which the node is sensitive . - The output function is a function which randomly decides to propagate the activation value towards only one of the output nodes. The random drawing is done on the basis of the values of the arcs.

The output function propagates the activation value identically across all arcs. - The output function propagates the activation value on all the arcs in proportion to their value.

The activation function thresholds the input value of the node.

The activation function activates only the n nodes with the largest input values. The activation function chooses n nodes randomly with probabilities calculated according to the input values.

The descriptor generation function only generates descriptors if the weighted sum of the node activation and node interest values is above a certain threshold. The likelihood of the descriptors generated can be equal to the activation value of the node.

The erase function can remove the n oldest descriptors, n being equal to the number of descriptors added to the working memory during step 4.

The erase function can remove the n least recently used descriptors.

The method according to the invention is an extension of the representation methods in the form of a graph such as neural or connectionist networks, Bayesian networks, Sowa graphs or Markov networks which constitute four possible embodiments of the invention. However, it brings three main originalities in relation to these processes: its purpose is to generate descriptors, it has a working memory, and it uses one or more configurable arbitration functions (the activation function or first arbitration function, possibly a second arbitration function), in particular in terms of the number of nodes or of descriptors activated, and of the choice function.

Adding an emotional profile to the user profile The structure described above is enriched with emotional information, which characterizes the mood or mood of the user and has an influence on the generation of descriptors.

The structure of the user profile with affect information:

Each node of the graph described above (figure 4) is enriched as represented in figure 6 by two vectors called vector of sensitivity to affects and vector of increment of affects. Each arc is also enriched with a vector of sensitivity to affects. Another vector called the global affect vector represents the user's current context. The interest value is removed from the node.

The functions ge, fa and fd are modified so that they take into account the different vectors of affects. An affect increment function modifies the value of the global affect according to its old value and all the increment vectors of all the activated nodes. A second time evolution function modifies the value of the global affect as a function of time. General mechanism:

The mechanism shown in Figure 7 has been broken down into several stages for the sake of clarity. Some of these steps can however take place in parallel (references similar to those of FIG. 5 are used to facilitate the comparison of the figures): 1: The memory of the profile is filled with a subset of the description of the document. This filling can be done in real time (when the description of the document changes over time) or in deferred time (when a complete description is already available). A set of descriptors is removed by the erase function. 2: The input values of the nodes of the profile are calculated.

3: The activation values of the nodes are calculated.

4: A set of descriptors is generated and replaced in the user memory.

5: The global affect vector is modified according to all the increment vectors of the activated nodes.

6: The global affect vector is modified.

Some descriptors are removed from memory by the erase function.

7: The process resumes at step 2. Detail of the stages:

2: For a node i, the input vector is calculated according to the output values of the incoming nodes, the weighting of the arcs, the sensitivity to affects of the incoming arcs, the global affect vector and the likelihood vector descriptors to which the node is sensitive:

Entrance_Node_Value

Arc_weighting *

Sensitivity_affects_arc * Affectjgbbal

Really_descriptor *

Input_function => input_value

3: The activation value of node i is calculated as a function of the input values of all nodes, the vector of sensitivity to affects of the node and the global vector of affects:

Inputvalue ^* Sensitivity_affects_node Affectjgbbal

Activation_function

=> Valeur_actJvation 4: The vector of descriptors to generate is given by the trigger function. The calculation is done according to the activation of the node, its vector of sensitivity to affects and the global vector of affects:

Activation_Value

Sensibiiité_affects_nceud Affectjgbbal

Jgenerationjdescripteurs function => (Descriptor Vraisemblanœ) *

5: The global affect vector is modified according to the increment vector of all the activated nodes: Affectjgbbal lncrément_affects Function_incrément => Affectjgbbal 6: The global affect vector changes over time:

Affectjgbbal Function j3volution_temporelle => Affectjgbbal

The probability of generation of the descriptors generated:

The mechanism of variation of generation probabilities is a function of the same type as that described in the absence of affects. However, the evolution of the probability of production of a descriptor as well as the function of modification of this probability depend on the global vector of affects, for example through coefficient of sensitivity to this vector of affects. o We can observe that a similar mechanism can be obtained by decrementing the global affect following the activation of the node.

Preferential examples of the different functions:

The input function can be broken down as follows:

. for each arc, a sensitivity value 'S' of arc 5 is calculated for current affects. S is equal to the norm of a vector V. This vector is obtained by the scalar product of the sensitivity to affects of the arc and the global affect vector.

.. We calculate a weighting vector for the arcs 'P', each component of which is obtained by the average of 'S' and the weight of the arc 0 ... The input value of the node is equal to the product standard scalar of 'P' with the vector of the output values of the incoming nodes, to which we add the maximum likelihood among those of the sensitive descriptors present in the memory.

- The activation function can perform a thresholding of the input value. 5 The thresholding depends on the sensitivity vector of the node and the global affect vector.

- The activation function can arbitrate and allow only the nodes most sensitive to current affects to activate. - The descriptor generation function only generates descriptors if the node's sensitivity to affects is above a certain threshold.

- The global affect increment function adds the average of the node increment vectors to the global affect vector. - The temporal evolution function asymptotically tends the global affect vector towards a particular state.

One of the first articles evoking the integration of information on the moods of the users is "Architecting Personalized Delivery of Multimedia Information", Shoshana Loeb, Communication of the ACM, December 1992. The vocation of this system is to propose pieces of music to an user. This system has only a simplistic model of the user which comprises 5 different moods. In "Affective Computing", MIT Press, 1997, Rosalind Picard deals with systems integrating emotional profiles. She is particularly interested in the affect modification function (step 6). An originality of the invention resides in the sensitivity to affects of the various functions of the process and in their use to control the generation of descriptions.

User profile and emotional profile learning:

The weights of the arcs and the vectors of sensitivity to the affects of the nodes are obtained by automatic capture of the user profile. During a viewing session, the user tends to jump (“zap”) from multimedia documents to multimedia documents. Each time he views all or part of these documents. Since each of these documents has a description, the following two pieces of information can be obtained from an interface for access to multimedia content which monitors the actions of the user, this interface possibly being a video player or music, a web browser, etc:

- the frequency of passage ("zapping") from a document containing a descriptor X to a document containing a descriptor Y - the duration of viewing each document. In the case of a "web" browser, we can distinguish the page loading times, the viewing time after loading, the number of times the page was accessed during a time interval.

5 This information is used to weight the profile graph as follows:

- the weighting of the arcs connecting the nodes is a function of the probability of transition between the two descriptors. This probability is obtained from the frequencies of passage between descriptors: i o Probabilrté_de_transrtion ® function => weighting arcs

- the vectors of sensitivity to the affects of the nodes are obtained from the viewing times of the documents by the user.

15 Affectjgbbal temps_visualisation function =>

Vector affects

Several profiles in a system:

Some systems work in cooperation with more than one user. Several user profiles can be integrated into a single

20 descriptor generation system. To facilitate human-machine interactions, it is sometimes even useful that the application itself has a "user" profile. This particular profile is called a "system" profile. It is built on the same model as user profiles. It can come from the owner's profile or characterize the proper state of the

25 system. The memory space is shared between all user profiles.

An additional function makes it possible to generate descriptors denoting the state of one or more of the user profiles. The description can therefore for example ¹ be supplemented with information on the mood of a user. The "system" profile can use these descriptors. 30 This function is a permanent function. Each modification of the user profile on which the system focuses is immediately reported in the associated descriptors. In a preferred embodiment, the selection function will only focus on user profiles with significant weight.

(User profile Weight User interest) Function ^ selection => (Likewise descriptor) ^*

The values of the interest weights associated with user profiles can be modified by the presence of descriptors.

(Descriptor Vraisemblanœ) ^* Function_modification θids => PoJdsJntinterestj user

In one embodiment of the invention, these descriptors will have been generated by the "system" profile.

Application example:

A household robot like the robot vacuum cleaner described above will often be confronted with several people. He must have a user model for each person whose actions he must plan and know the mood. By allowing the generation of descriptors describing the affects of the user, we make it possible for the robot to take this information into account. Thus, it can be "discreet" if its owner is "upset".

Dynamic adaptation of the profile:

The method allows the dynamic addition or deletion of nodes and the modification of the parameters of each node and each arc in use.

The dynamic addition or deletion of nodes can be carried out by the arbitration function in particular according to the number of descriptors, the presence of new types of descriptors in the working memory, or the creation or not of new descriptors by the system for a period. The modification of the various parameters of the user profile, such as the valuation of the arcs or the probability of transition of a node, is possible in particular by increasing or decreasing these parameters, depending on user interaction or by strengthening the arcs and nodes traversed and weakening of the others, or by a combination of this mechanism with the value of affects, or randomly, or according to the state of the following nodes in the graph.

Securing user profiles:

User profiles are part of the world descriptions. They must be transported on the networks with the descriptions of the multimedia objects. They will also be useful in other contexts, for example for the proposal of personalized programs by multimedia content providers.

These data are sensitive and the resolution of problems related to their confidentiality conditions their acceptance. For this, the invention proposes a network architecture with three components: a server providing a user profile (for example a personal video recorder), an anonymization server and a server consuming a profile (for example a provider of personalized content). The anonymization server maintains an anonymization table, that is to say a correspondence table between user profiles and the network addresses of the people they represent. This anonymization server is a trusted third party.

The protocol between the various servers is as follows: - the profile provider sends the profile server, its network address, personal information and its request to the anonymization server,

- the anonymization server filters this information and transmits the request as well as a part of the personal information, but not the network address of the sender, - the profile consumer uses the information he receives to best respond to the request,

- the result of the request is directed to the anonymization server which, thanks to its correspondence table, redirects the result to the user.

The part of the information transmitted to profile consumers is dependent on the number of profiles registered with the anonymization server. Statistical measurements on samples ensure the maximum transmission of information with the minimum of risk.

The method which has just been described can be implemented in a device for generating descriptions of multimedia documents, of the type of those illustrated in FIGS. 8 to 11. This device comprises at least a first processing unit coupled to a second processing unit comprising at least one working memory and capable of responding to a request formed by a user in order to access at least one document and / or to navigate among these documents and / or to interpret them.

In what follows, it is considered that the first processing unit is housed in a server comprising a communication interface coupled to an external system comprising the second processing unit, via a data transmission network.

The first processing unit comprises a first external calculation module F coupled to a communication interface and to a memory L in which collections of examples X are stored. This first processing unit also comprises a working memory M comprising addressable registers and an address management module C, such as a counter making it possible to increment the current address, in writing, of the working memory M, for example every N cycles (defined by a clock). N is here an integer greater than or equal to 1. In this counter C, when the incrementation reaches a maximum value, an increment causes said value to return to its minimum value. The first processing unit further comprises a second calculation module B intended to generate descriptors from descriptors stored in registers of the working memory M at selected addresses. Finally, the first processing unit also comprises a third calculation module A intended for calculating affects, from descriptors generated by the second calculation module B. This third module A can also serve, alternatively, as a generator of abstract nodes .

Reference is now made, more particularly, to FIG. 8 to describe a first embodiment of the device.

At each cycle defined by a clock, the first external calculation module F calculates a similarity function between objects Oi from the outside world, the image of which is contained in a multimedia document D, and the collections of examples X which are found stored in the memory L. It then delivers a first descriptor e (or primary descriptor) which is stored in the working memory M at the address c provided by the counter C.

The working memory M therefore includes a descriptor in each of its registers. This working memory M supplies descriptors (e ', d', a ') to the input of the second calculation module B. More specifically, the second calculation module B has at least two inputs, preferably three, which are supplied in descriptors respectively by the two or three registers of the working memory M whose addresses are c, d and c2. But this is only an example, B can be fed by more than three descriptors.

In response to this supply of descriptors, the second module B delivers two or more scondary descriptors as output (or two copies of the same secondary descriptor). It is for example an information descriptor d. This information descriptor d is supplied, on the one hand to the third calculation module A and, on the other hand, to the register of the working memory M whose address is d, with a view to its storage in the following cycle. The third calculation module A delivers a derivative descriptor value d1 on its output which feeds the register of the working memory M whose address is c2, with a view to its memorization in the following cycle, at the same time as the memorization of d at address d. The descriptors d and d1 of which used, during the aforementioned following cycle, by the second calculation module B to determine the value of an action descriptor k, which is control information supplied at the input of an external material G, like for example a video recorder.

In this embodiment, the descriptor e which is supplied to the working memory M by the first external calculation module F, can be either a descriptor obtained by similarity between an object and a prototype, as indicated previously, or an exception introduced by a user. In this case, the address management module C can be arranged so that the user can choose the address of the register of the working memory M in which e must be stored.

In FIG. 9 is illustrated a variant of the embodiment illustrated in FIG. 8. In this variant, provision is made upstream of the third calculation module A, that is to say between the second B and third A modules of calculation, an adder H whose usefulness will be specified below.

In this variant embodiment, the second calculation module B delivers at output an increment of affect i (or first secondary descriptor), an information descriptor d, (or second secondary descriptor) and an action descriptor k. The information descriptor d is supplied at the input of the working memory M (address register d) to be stored and used in the next cycle, as indicated above. The action descriptor k is control information delivered at the input of the external hardware G. The increment of affect i is delivered at the input of the adder H where it is added to the value of the current affect in the third calculation module A. The result of this addition (a + i) feeds the input of the third calculation module A. This affect calculation module A then delivers as output a new affect value "a" to the register of the working memory M whose address is c2, with a view to its storage and its use in the following cycle, as explained previously. In another variant illustrated in FIG. 10, the second calculation module B in FIG. 9 is replaced by several modules B1, ..., Bn. There is further provided a selection module S making it possible to determine which of the calculation modules Bi must be used to supply the working memory M with the information descriptor d. This selector S is a specific calculation module which receives as input, for example, the increments of affect coming from each calculation module Bi and which calculates a function of its inputs.

This selection module S outputs the address (or number) of the calculation module Bi, the results of which are written to the working memory M. In other words, only the selected module Bi has write access to the memory of work M.

In the variant illustrated in FIG. 11, the calculation module B is replaced by several modules Bi (here i = 1 to 3) placed in parallel with a selection module S, for example on a bus. In this embodiment, the module Bi chosen by the selection module S becomes the bus master.

The calculation modules A, B and S which have been described previously can be produced in the form of calculation microprocessors equipped with a memory. But this is only an example of implementation. In fact, the calculation modules A and B could be components of neural calculation, the types of which can be chosen according to the specific specification of the structure. Furthermore, the calculation modules A, B and S could also be made up of software modules.

Furthermore, a device has been described comprising a first processing unit, coupled to a second external processing unit. This second processing unit is preferably substantially identical to the first processing unit. It preferably includes in its working memory the user profile which was described in a previous part, relating to the method according to the invention. But of course, in a variant each processing unit may include user profiles. On the other hand, the device may include the first and second units in one place.

The invention is not limited to the embodiments of methods and devices described above, only by way of example, but it encompasses all the variants that a person skilled in the art may envisage within the framework of the claims below. -after. Thus, methods and devices have been described in which the first and second processing units are housed respectively in separate systems of the server and client type communicating via a data transmission network. However, these two processing units can be housed in the same system; in this case they communicate via an internal link, for example of the bus type.

Furthermore, a third processing unit can also be envisaged for carrying out document navigation operations, the first and second processing units then being respectively arranged for carrying out the document description operations and the response operations to the requests of the 'user. In this case, it is advantageous that the first and second processing units are housed in one place, for example the server, while the third unit is housed in another location (outside), for example in the client. Here, the third processing unit acts as a browser, while the second processing unit acts as a search engine.

Claims

1 - Method for generating multimedia document descriptions in a first processing unit comprising a working memory and coupled to at least a second processing unit comprising a working memory and suitable for responding to a request formed by a user with a view to '' access at least one of these documents, and / or navigate among these documents and / or interpret them, and said method comprising a first step of analysis and temporal and / or spatial segmentation of a multimedia document in a plurality of multimedia objects with which likelihoods are associated, characterized in that it also comprises a second step of generating descriptor objects by applying functions to said multimedia objects of the plurality, these descriptor objects being stored in one at least working memories of the first and second likelihood processing units and comprising d information descriptors, action descriptors and abstract descriptors, the latter each containing a reference to a described object and functions defining reflex mechanisms, said descriptor generation functions comprising at least one of the functions consisting in:

- apply a similarity function to a multimedia object with a pre-established collection of multimedia object prototypes or descriptors,

- create exceptions by accepting new descriptors which contradict descriptors already obtained when the likelihood of a new descriptor is greater than that of the descriptor already obtained and the difference in their likelihood is greater than or equal to a predetermined threshold,

- modify the functions ^* for generating descriptors by a user profile comprising a modeled set of knowledge specific to a user receiving a description, the user profile consisting of a graph of nodes connected by arcs, stored in at least one of the working memories of the first and second processing units, containing an organized set of descriptors with a function for deleting descriptors, and being suitable for being established by one and / or the other of the first and second processing units.

2 - Method according to claim 1, characterized in that it is adapted to a client / server type environment in which the server comprises the first processing unit and the client comprises the second processing unit, said first and second processing units. processing then being coupled by a data transmission network.

3 - Method according to one of claims 1 and 2, characterized in that it comprises a third step of activation of abstract descriptors by triggering a reflex mechanism of an abstract descriptor in response to a change in its state or to a request on its state, this activation making it possible to generate other descriptors.

4 - Method according to one of claims 1 to 3, characterized in that, in the first step, reference descriptors describing the relationships between the multimedia objects and the objects from which they are derived are added to the multimedia objects.

5 - Method according to claim 3 or 4, characterized in that, in the third step, the triggering of a reflex mechanism of an abstract descriptor causes the triggering of reflex mechanisms of other abstract descriptors, and in that said method comprises a mechanism for marking these abstract descriptors to avoid closed-loop activations of two or more abstract descriptors.

6 - Method according to any one of the preceding claims, characterized in that it consists in using one or more ontologies in the reflex mechanisms of the abstract descriptors. 7 - Method according to any one of the preceding claims, characterized in that it consists in activating abstract descriptors in view to achieve a pseudo-inheritance, consisting of a transfer of a descriptor along a relationship between descriptors.

8 - Method according to claim 7, characterized in that said relationship is of the content / container type. 9 - Method according to any one of the preceding claims, characterized in that the functions used for the segmentation of a multimedia document in the first step and / or for the generation of descriptors in the second step include a procedural code, such as example as an algorithmic process of segmentation or information extraction.

10 - Method according to any one of the preceding claims, characterized in that, in the first step, a collection of prototypes is created by application to one or more multimedia objects of an interaction, or of data input by a external agent, associated with an expert weight or plausibility.

11 - Method according to one of the preceding claims, characterized in that in the second step the functions of generating descriptors include production rules.

12 - Method according to claim 11, characterized in that it comprises the creation of rules by applying metarules to other rules.

13 - Method according to any one of the preceding claims, characterized in that in the second step the functions of descriptor generation include interactions, or input of data by an external agent, suitable for allowing the production of annotations.

14 - Method according to any one of the preceding claims, characterized in that it comprises a mechanism for maintaining the consistency of the descriptors generated in the second step, consisting in deleting the descriptors originating from a descriptor contradicted by an exception. 15 - Method according to one of claims 1 to 14, characterized in that the establishment of the user profile consists in filling the working memory of one and / or the other of said first and second processing units with a set descriptors of a document, in real time or in deferred time, to delete from this working memory a set of previous descriptors, to calculate input values of the nodes of said profile, to calculate activation values of the nodes, to generate a new set of descriptors and to introduce this new set into the working memory. 16 - Method according to claim 15, characterized in that the above-mentioned node graph is an acyclic lattice graph.

17 - Method according to claim 15 or 16, characterized in that each node contains an aggregate function of the inputs, an input value, an activation or transition function, an output function, a generation function of descriptors and a value of interest, and in that numerical weighting values are assigned to the arcs connecting the nodes, a weighting value of an arc being associated with a probability chosen at least from a probability that the user having seen a descriptor contained in a first node wishes to see another descriptor contained in another node connected by the aforementioned arc to the first node or to a probability that the user thinks of the other descriptor when he has seen the descriptor contained in the first node.

18 - Method according to claim 17, characterized in that it comprises an arbitration function determining, among nodes candidates for activation, those which are activated at a given time, according to a descriptor of the number of nodes which can be activated and activation parameters associated with each node.

19 - Method according to one of claims 15 to 18, characterized in that it comprises an addition of information of an emotional nature to the user profile, by associating with each node a vector of sensitivity to affects for each function of the node and a vector of global affect increment, a vector of sensitivity to affects at each arc and a context affect, or global affect vector, to the whole graph, the input, activation or transition and descriptor generation functions contained in the nodes being modified to take into account the global affect vector and vectors of sensitivity to affects. 20 - Method according to claim 19, characterized in that said descriptor generation function generates the descriptors from the activation value of the node, the vector of sensitivity to affects of the node and the vector of global affects.

21 - Method according to any one of the preceding claims, characterized in that it comprises an iteration of said steps.

22 - Method according to claim 21, characterized in that it includes taking into account several user profiles, constituted as above and of which a copy of the state is saved in said working memory, and a selection function and arbitration determining the active user profiles at a given time.

23 - Method according to any one of the preceding claims, characterized in that it is arranged to anonymize a user profile, in particular for its transmission by the second processing unit to the first processing unit.

24 - Method according to any one of the preceding claims, characterized in that it is arranged to perform the description operations on the first processing unit, the response operations to user requests on the second processing unit, the latter being identical to said first processing unit, and document navigation operations in a third processing unit.

25 - Device for generating multimedia document descriptions, of the type comprising a first processing unit suitable for being coupled to a second processing unit arranged to transmit requests formed by a user to this first processing unit in order to access at least one of the multimedia documents, and / or to navigate among these documents and / or to interpret them, characterized in that said first processing unit comprises:

* clock means suitable for defining cycles, * a working memory (M) subdivided into addressable registers and suitable for containing descriptors,

* an auxiliary memory (L) suitable for storing collections of examples X,

* a first calculation module (F) arranged to calculate a similarity function between objects Oi from the outside world, an image of which is contained in a multimedia document D, and said collections of examples X, and in delivering a primary descriptor ( e) representative of the calculated similarity function,

^* means (C) for managing the addressing, in writing, of the registers of the working memory (M), arranged to supply all the N cycles (N being greater than or equal to 1) a first address (c) for store said primary descriptor (e) in the working memory (M),

^* a second calculation module (B) supplied with at least two descriptors stored at selected addresses (c, d, c2) of the working memory (M) and suitable for generating at least a first (i; d) and a second (d) secondary descriptors from these received descriptors,

^* a third calculation module (A) arranged to calculate a derived descriptor (d1; a) from the first secondary descriptor (i; d) generated by the second calculation module (B), said address management means being further arranged to address respectively said second secondary descriptor (d) and said derived descriptor (d1; a) to registers of the working memory M placed in second (d) and third (c2) chosen addresses, with a view to their storage in the next cycle, and in that said second calculation module (B) is arranged to determine, from said secondary and derived descriptors, an action descriptor (k) forming control information for an external material G. 26 - Device according to claim 25, characterized in that some of said primary descriptors are exceptions introduced by a user.

27 - Device according to one of claims 25 and 26, characterized in that said first and second secondary descriptors are two copies of the same secondary descriptor.

28 - Device according to one of claims 25 and 26, characterized in that said second calculation module (B) is adapted to deliver said first secondary descriptor, in the form of an affect increment (i), and in that said first processing unit comprises, between the second (B) and third (A) calculation modules, an adder (H) suitable for adding said increment of affect (i) to the derived descriptor (a) in progress in the third module (A) and to transmit the result of this addition to said third calculation module (A), so that it determines a new derived descriptor intended to be stored in said register placed at the third chosen address (c2) during the next cycle.

29 - Device according to one of claims 25 to 28, characterized in that said second calculation module (B) is subdivided into a multiplicity of sub-modules (B1 Bn), and in that said first processing unit comprises a selection module (S) arranged to select one of said sub-modules to supply said second secondary descriptor (d) with the register placed at the second address (d).

30 - Device according to claim 29, characterized in that said selection module (S) is arranged to select said sub-module by application of a chosen function to the increments of affect delivered by each of these sub-modules (Bi) .

31 - Device according to one of claims 29 and 30, characterized in that said calculation sub-modules (Bi) and said selection module (S) are arranged in parallel on a bus. 32 - Device according to any one of claims 25 to 31, characterized in that it comprises said second processing unit, and in that said second processing unit is substantially identical to said first processing unit.

33 - Device according to claim 32, characterized in that said first processing unit is arranged to perform document description operations and said second processing unit is arranged to perform operations in response to user requests, and in which it comprises a third processing unit arranged to carry out navigation operations in documents.

34 - Device according to one of claims 32 and 33, characterized in that said first and second processing units are housed in a server comprising a communication interface, and said third processing unit is housed in an external system of type " client ”also comprising a communication interface, said client and server being coupled by a data transmission network, via their respective communication interfaces.

35 - Device according to one of claims 25 to 33, characterized in that said first processing unit is housed in a server comprising a communication interface, and said second processing unit is housed in an external system of “client” type also comprising a communication interface, said first and second processing units being coupled by a data transmission network, via their respective communication interfaces.