CN109033129A - Multi-source Information Fusion knowledge mapping based on adaptive weighting indicates learning method - Google Patents

Abstract

The invention discloses a multi-source information fusion knowledge map representation learning method based on self-adaptive weight. Firstly, the fusion of text information and structured information is considered, and a translation-based model between entity vectors and relationship vectors is adopted. By adjusting the relationship between the two The weight between them is used to optimize the score function, and the type constraint training is performed on the structured information that has been classified in the previous stage without introducing more parameters; then the loss function is used to associate the entity vector and the relationship vector, and the loss is optimized Function, when the optimization goal is achieved, the vector of each entity and the vector of the relationship in the knowledge graph can be learned. The invention solves the problem that the fusion of text information and structured information in the knowledge base does not consider the weight, and utilizes the existing hierarchical information of the structured information in the knowledge base to more accurately represent the interrelationships between entities and relationships, and It is applied to large-scale knowledge graphs.

Description

Multi-source information fusion knowledge graph representation learning method based on adaptive weight

technical field

The invention relates to the technical field of knowledge graph and deep learning, in particular to a multi-source information fusion knowledge graph representation learning method based on adaptive weight.

Background technique

With the rapid development of society, we are slowly entering an information age. Masses of new data and information are generated every day in different forms. The mobile Internet has become the most effective and convenient information acquisition platform in today's society, and users' demand for real information acquisition is also increasing. How to obtain effective information from massive data has become a major problem in many fields. The knowledge graph also came into being.

People usually organize the knowledge in the knowledge base in the form of a network. Each node in the network represents an entity, and each edge represents a relationship between two entities. The form of a triple is (entity 1, relation, entity 2) . Figure 1 is an example diagram of a typical triple in a knowledge graph. The nodes "Shakespeare" and "Romeo and Juliet" represented by the ellipse are entities, and the "author" represented by the edge is a relationship. Therefore, most knowledge can be represented by triples, corresponding to a chain and two linked entities in the knowledge base network, which is the general representation of the knowledge base. In recent years, deep learning has gained widespread attention in the fields of speech recognition, image analysis and natural language processing. Representation learning aims to represent the semantic information of the research object as a dense low-dimensional real-valued vector. In this low-dimensional vector space, the closer the distance between two objects, the higher the semantic similarity. Recently, important progress has been made in this direction, which can efficiently calculate the semantic connection of entities and relationships in low-dimensional space, effectively solve the problem of data sparsity, and significantly improve the performance of knowledge acquisition, fusion and reasoning.

A major challenge in knowledge representation learning is how to achieve multi-source information fusion. The triple structure information of the existing knowledge graph, such as TransE, only uses the triple structure information of the knowledge graph for representation learning, and a large amount of other information related to knowledge has not been effectively utilized, such as other information of the knowledge base, such as Descriptive information, category information, etc. of entities and relationships.

Contents of the invention

Aiming at the problem that existing knowledge map representation learning methods cannot fully utilize the relationship between structured models and text information after fusion with text information, the present invention proposes a multi-source information fusion knowledge map representation learning method based on adaptive weights .

In order to solve the above problems, the present invention is achieved through the following technical solutions:

A multi-source information fusion knowledge map representation learning method based on adaptive weight, the specific steps are as follows:

Step 1. Use adaptive weights to balance the fusion of text information and structured information, and define a total score function f(h, r, t) that correlates text information and structured information:

f(h,r,t)＝(1-λ)(||h _d +rt _d ||+||h _d +rM _rt t _S ||+||M _rh h _S +rt _d ||)+ λ(||M _rh h+r+M _rt t||)

Among them, λ represents the weight, h represents the head entity, t represents the tail entity, r represents the relationship between the head entity h and the tail entity t, h _d represents the text-based representation of the head entity, t _d represents the text-based representation of the tail entity, h _S Indicates that the head entity is based on a structured representation, t _S indicates that the tail entity is based on a structured representation, M _rh is the projection matrix defined according to the head entity, and M _rh is the projection matrix defined according to the tail entity;

Step 2. Based on the total score function f(h,r,t) defined in step 1, establish a loss function based on the fusion of text information and structured information based on adaptive weight, and learn entities and relationships by minimizing the loss function The vector representation of is to achieve the optimization goal.

In the above step 1, the value range of the weight λ is λ∈(0, 1).

In step 2 above, the stochastic gradient descent method is used to minimize the loss function.

In the above step 2, the constructed loss function L is:

Among them, [f(h,r,t)+γ-f(h',r,t')] ₊ ＝max(0,f(h,r,t)+γ-f(h',r,t ')); γ is the set boundary value; (h, r, t) represents the triplet of the knowledge map, that is, the positive triplet, h represents the head entity, t represents the tail entity, and r represents the head entity and tail entity The relationship between, f(h, r, t) represents the score function of positive triples, S(h, r, t) represents the set of positive triples; (h', r, t') represents random Replace the negative example triplet constructed by U-turn entity h and tail entity t, f(h', r, t') represents the score function of the negative example triplet, S'(h, r, t) represents the negative example three A collection of tuples.

Compared with the prior art, the present invention first considers the fusion of text information and structured information, adopts a translation-based model between entity vectors and relationship vectors, optimizes the score function by adjusting the weight between the two, and The structured information that has been classified in the early stage is trained with type constraints, and there is no need to introduce more parameters; then use the loss function to associate the entity vector and the relationship vector, and optimize the loss function. When the optimization goal is achieved, you can learn Get the vector of each entity and the vector of relationship in the knowledge graph. The invention solves the problem that the fusion of text information and structured information in the knowledge base does not consider the weight, and utilizes the existing hierarchical information of the structured information in the knowledge base to more accurately represent the interrelationships between entities and relationships, and It is applied to large-scale knowledge graphs.

Description of drawings

Figure 1 is an example diagram of relational triples in the knowledge graph.

Fig. 2 is a schematic flow chart of the method for learning knowledge map representation of the present invention.

Fig. 3a is an example diagram of a triplet representation obtained according to an existing knowledge graph representation learning method.

Fig. 3b is an example diagram of a triplet representation obtained according to the knowledge graph representation learning method of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in combination with specific examples and with reference to the accompanying drawings.

In view of the fact that the existing technology only considers the fusion of text information and structured information, it does not fully consider how to adjust the weight between the two to achieve the best effect, and does not utilize the existing levels of structured information in the knowledge base Information, and the number of learning parameters is large, so it cannot accurately represent the connection between entities and relationships, and it cannot be well applied to large-scale knowledge graphs.

The invention fully considers the fusion of text information and structured information with self-adaptive weight, and enriches structured information according to unused hierarchical information. When text information and structured information are fused, an adaptive weight is proposed to balance the fusion of text information and structured information, and type-constrained training is performed on structured information that has been classified in advance, and the score is minimized by this method function to achieve the goal of optimization. By adding a multivariate information fusion method with adaptive weights, it can solve the heterogeneity and imbalance of entities and relationships in the knowledge base, more accurately represent entities and relationships and their interconnections, and apply them to large-scale knowledge Atlas.

Specifically, a multi-source information fusion knowledge map representation learning method based on adaptive weight, as shown in Figure 2, includes the following steps:

Step 1. When text information and structured information are fused, an adaptive weight is proposed to balance the fusion of text information and structured information, and type-constrained training is performed on the pre-classified structured information.

Based on the fusion of text information and structured information, define an interrelated total score function f:

f(h,r,t)=(1-λ)f _D (h,r,t)+λf _S (h,r,t)

fD(h,r,t) represents the scoring function based on the text representation:

f _D (h,r,t)=f _DD (h,r,t)+f _DS (h,r,t)+f _SD (h,r,t)

＝||h _d +rt _d ||+||h _d +rM _rt t _S ||+||M _rh h _S +rt _d ||

f _S (h, r, t) represents a scoring function based on structured representations:

f _S (h,r,t)＝||M _rh h+r+M _rt t||

Among them, λ represents the weight, λ∈(0,1), h represents the head entity, t represents the tail entity, r represents the relationship between the head entity h and the tail entity t, h _d represents the text-based representation of the head entity, and t _d represents the tail entity Entity text-based representation, h _S indicates head entity-based structured representation, t _S indicates tail entity-based structured representation, M _rh is the projection matrix defined according to head entity, and M _rh is the projection matrix defined according to tail entity.

Step 2. Propose a loss function based on the fusion of text information and structured information based on adaptive weight, and learn the vector representation of entities and relationships by minimizing the loss function to achieve the optimization goal.

Step 21. Define the loss function as:

Among them, [f(h,r,t)+γ-f(h',r,t')] ₊ ＝max(0,f(h,r,t)+γ-f(h',r,t ')) ₊ ; γ is the set boundary value; (h, r, t) represents the triplet of the knowledge map, that is, the positive example tuple, h represents the head entity, t represents the tail entity, r represents the head entity h and the tail entity The relation of entity t, f(h, r, t) represents the scoring function of positive triples, S(h, r, t) represents the set of positive triples; (h', r, t') represents random The negative triplet constructed by replacing the head entity h and the tail entity t, f(h',r,t') represents the score function of the negative triplet, and S'(h,r,t) represents the negative Set of example triples;

Step 22: Minimize the loss function by using the stochastic gradient descent method, and learn to obtain each entity vector and relationship vector in the knowledge map and the interrelationships between them.

The process of minimizing the loss function is the process of minimizing the score function, and the process of minimizing is the process of achieving the optimization goal. E in the triplet score function uses the energy function in the TransE model, then in the process of minimizing the loss function, when the type of the relationship r is a simple relationship type 1-1 or a complex relationship type 1-N, N-1 , N-N, by constantly adjusting h, t and r, make h+r equal to t as much as possible.

This method learns and obtains a knowledge graph representation learning method based on multi-source information fusion based on adaptive weights, and the type-constrained training model for pre-classified structured information is more accurate and effective.

Adding text information can solve problems that cannot be solved by existing methods: when predicting a new entity (untrained entity), the original method will randomly give it a vector representation, so that its score function and trained The score function of the entity will be poor, and its loss function will become larger, so the prediction effect will be poor. In our unimproved method of adding text information, when a new entity (untrained entity) appears, the structured method will randomly give it a vector representation, but there will be text for this new entity in the knowledge base Description, we can process the text description of the new entity into a vector of text representation, and obtain a new score function by adding the vector of the training structured method and the vector of text representation, so as to achieve the optimization goal. When a new entity (untrained entity) appears, although the score function is optimized by adding text information through text description, in the method of obtaining a new score function by adding the two score functions, the structured information is still It is obviously unreasonable to be able to provide wrong information with a large proportion.

The present invention proposes the fusion of structured information and textual information in adaptive weights. The weight representation of the two is updated through the training times of the entity. When the entity is trained once, the weight of the structured information representation is increased a little, while the weight of the text information representation is decreased a little. This is because when no text information is added, the number of entity training times is also in a long-tail distribution. Frequently used entities and entities with multiple categories appear more often, and the relative number of training times is also sufficient, so entities with enough training times will be closer to each other. Correct representation, the representation of entities with less training times will also be relatively weak. In this case, we think that entities with more occurrences are good enough in the representation of structured information, and we can increase the weight of the structured representation of such entities. The more occurrences, the weight of structured information representation will also be Increases with increasing frequency. On the contrary, for entities that appear less frequently, the representation of structured information is not good enough, so we increase the weight of the text information representation part of these entities. If the entity does not appear once, then the weight of the structured information part is also 0, that is, all are represented by text information, which not only uses text information to represent new entities that have not appeared, but also filters out the structured information that is randomly assigned to the vector.

Fig. 3a is an example diagram of a triplet representation obtained according to an existing knowledge graph representation learning method. In Figure 3a, the hierarchical information of knowledge graph triples is not considered. Hierarchical information means that entities may have different roles in different scenarios. For example, Shakespeare is both a writer and a musician, and Bob also has such attributes. We believe that entities with multiple types should have different representations under different relations. We construct a specific type of projection matrix M _r from the hierarchical structure, and then represent the head entity h and tail entity t through the constructed specific projection matrix. In this way, there will be as many mappings as there are as many relationships as the entity has to represent the special representation of this entity under each relationship. In Figure 3b, we represent the types of entities through specific relations. Entities with the same type tend to be in a cluster and have similar representations in training. In fact, this is the main reason for the error in entity prediction. In the present invention, the training probability of selecting entities of the same type under specific relationship type information can be improved, and the target can be optimized in this way.

The present invention solves the imbalance and heterogeneity of entities and relationships in the prior art, and the calculation is too complicated due to too many parameters, and there is no way to well represent the interconnections between entities and relationships in the knowledge map and It cannot be well applied to problems in large-scale knowledge graphs, and has good practicability.

It should be noted that although the above-mentioned embodiments of the present invention are illustrative, they are not intended to limit the present invention, so the present invention is not limited to the above specific implementation manners. Without departing from the principles of the present invention, all other implementations obtained by those skilled in the art under the inspiration of the present invention are deemed to be within the protection of the present invention.

Claims (4)

1. A multi-source information fusion knowledge map representation learning method based on adaptive weight, characterized in that the specific steps are as follows: Step 1. Use adaptive weights to balance the fusion of text information and structured information, and define a total score function f(h, r, t) that correlates text information and structured information: f(h,r,t)＝(1-λ)(||h _d +rt _d ||+||h _d +rM _rt t _S ||+||M _rh h _S +rt _d ||)+ λ(||M _rh h+r+M _rt t||) Among them, λ represents the weight, h represents the head entity, t represents the tail entity, r represents the relationship between the head entity h and the tail entity t, h _d represents the text-based representation of the head entity, t _d represents the text-based representation of the tail entity, h _S Indicates that the head entity is based on a structured representation, t _S indicates that the tail entity is based on a structured representation, M _rh is the projection matrix defined according to the head entity, and M _rh is the projection matrix defined according to the tail entity; Step 2. Based on the total score function f(h,r,t) defined in step 1, establish a loss function based on the fusion of text information and structured information based on adaptive weight, and learn entities and relationships by minimizing the loss function The vector representation of is to achieve the optimization goal. 2. The multi-source information fusion knowledge map representation learning method based on adaptive weight according to claim 1, characterized in that, in step 1, the value range of weight λ is λ∈(0,1). 3. The adaptive weight-based multi-source information fusion knowledge graph representation learning method according to claim 1, characterized in that in step 2, the stochastic gradient descent method is used to minimize the loss function. 4. The multi-source information fusion knowledge map representation learning method based on adaptive weight according to claim 1, characterized in that, in step 2, the constructed loss function L is: Among them, [f(h,r,t)+γ-f(h',r,t')] ₊ ＝max(0,f(h,r,t)+γ-f(h',r,t ')); γ is the set boundary value; (h, r, t) represents the triplet of the knowledge map, that is, the positive triplet, h represents the head entity, t represents the tail entity, and r represents the head entity and tail entity The relationship between, f(h, r, t) represents the score function of positive triples, S(h, r, t) represents the set of positive triples; (h', r, t') represents random Replace the negative example triplet constructed by U-turn entity h and tail entity t, f(h', r, t') represents the score function of the negative example triplet, S'(h, r, t) represents the negative example three A collection of tuples.