CN115099235B - Text generation method based on entity description - Google Patents

Abstract

The present application proposes a text generation method based on entity description, which relates to the field of natural language processing technology, wherein the method includes: obtaining a to-be-processed set of a target information extraction task; using a pre-trained text generation model to infer the to-be-processed set, applying beam search to decode the inference results, and obtaining all possible paths, wherein the text generation model is generated by training the pre-trained model based on the target information extraction task using the corresponding training set; all possible paths are screened according to probability to obtain candidate paths; according to the type of the target information extraction task, the candidate paths are pruned to obtain the target information. The present application using the above scheme can well complete various information extraction tasks.

Description

Text generation method based on entity description

Technical Field

The present application relates to the technical field of natural language processing, and in particular to a text generation method and device based on entity description.

Background Art

The task of information extraction is to extract specific events or entities from natural language text, and then automatically classify or extract a large amount of content. The extracted information includes entities, relations, events and their categories, etc. Subtasks include named entity recognition, relation extraction, entity linking, semantic role labeling, event extraction, sentiment analysis, etc.

Existing work on various information extraction tasks mainly deals with labeling and classification, and designs different prediction networks according to various specific tasks. Although these methods are effective, they ignore the rich label semantics and require a lot of task-specific designs, making it impossible to construct a general framework.

Summary of the invention

The present application aims to solve one of the technical problems in the related art at least to some extent.

To this end, the first purpose of this application is to propose a text generation method based on entity description, which solves the technical problem that the existing methods are not universal and can well complete various information extraction tasks.

The second objective of this application is to propose a text generation device based on entity description.

To achieve the above-mentioned purpose, the first aspect of the present application proposes a text generation method based on entity description, including: obtaining a to-be-processed set of a target information extraction task; using a pre-trained text generation model to infer the to-be-processed set, applying beam search to decode the inference results, and obtaining all possible paths, wherein the text generation model is generated by training the pre-trained model according to the target information extraction task using the corresponding training set; all possible paths are screened according to probability to obtain candidate paths; and according to the type of the target information extraction task, the candidate paths are pruned to obtain the target information.

The text generation method based on entity description in the embodiment of the present application constructs a general text generation framework, designs different entity descriptions as output sequences for different information extraction tasks. When there are multiple tuples to be extracted in a text, the more effective "sequence to path" approach is used to avoid the order and independence problems between multiple tuples in the generated sequence, thereby completing various information extraction tasks well.

Optionally, in one embodiment of the present application, before using the pre-trained text generation model to perform reasoning on the processing set, the method further includes:

Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;

Use the enhanced dataset to train the pre-trained model to obtain a text generation model.

Optionally, in one embodiment of the present application, constructing an enhanced data set includes:

Get the training set corresponding to the target task;

Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;

According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;

Use the training set to train the pre-trained model and generate a sample generation model;

Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;

The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;

An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.

Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.

To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a text generation device based on entity description, including: an acquisition module, a path generation module, a path screening module, and a pruning module, wherein:

An acquisition module is used to obtain a pending set of target information extraction tasks;

A path generation module is used to use a pre-trained text generation model to infer the processing set, and apply beam search to decode the inference results to obtain all possible paths. The text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;

The path screening module is used to screen all possible paths according to probability to obtain candidate paths;

The pruning module is used to prune candidate paths to obtain target information according to the type of target information extraction task.

Optionally, in one embodiment of the present application, a text generation model training module is further included, which is specifically used to:

Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;

Use the enhanced dataset to train the pre-trained model to obtain a text generation model.

Optionally, in one embodiment of the present application, constructing an enhanced data set includes:

Get the training set corresponding to the target task;

Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;

According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;

Use the training set to train the pre-trained model and generate a sample generation model;

Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;

The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;

An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.

Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.

Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG1 is a schematic diagram of a flow chart of a text generation method based on entity description provided in Example 1 of the present application;

FIG2 is an example diagram of a text generation method based on entity description according to an embodiment of the present application;

FIG3 is a schematic diagram of the structure of a text generation device based on entity description provided in an embodiment of the present application.

DETAILED DESCRIPTION

Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

In order to solve the problem that existing methods are not universal, we consider building a unified generative framework that can solve various information extraction problems. The generative model can make full use of natural language encoding, enrich label semantics, and insert language labels into the target output. In addition, this unified generative model can seamlessly adapt to multiple tasks without introducing additional task-specific model design.

Therefore, all kinds of information extraction tasks can be transformed into a sequence-to-sequence text translation problem. However, the design of the output sequence is crucial. Taking the joint extraction of entities and relations as an example, the output form is designed as (subject, relation, object). Compared with the output form (subject, object, relation), it is more in line with the semantic order of natural language and can obtain better extraction results. Therefore, for more common situations, if a complete sentence is directly used as the output sequence to describe the elements to be extracted, the effect will be further improved.

Although the sequence-to-sequence generation framework has achieved good results on many data sets, it still has limitations: (1) The order problem. Sentiment tuples are parallel and there is no concept of order, but the sequence-to-sequence method outputs them serially in order. (2) The independence problem. According to the concept of the sequence-to-sequence model, the subsequent tuple to be extracted depends on the previous tuple, but in fact the tuples are independent of each other and there is no such dependency. Therefore, the output text of the entity description in this application adopts the "sequence to path" method.

In addition, in order to expand the data set, it is necessary to use negative sample methods for data augmentation.

The following describes the text generation method and device based on entity description in an embodiment of the present application with reference to the accompanying drawings.

FIG1 is a flow chart of a text generation method based on entity description provided in Example 1 of the present application.

As shown in FIG1 , the text generation method based on entity description includes the following steps:

Step 101, obtaining a to-be-processed set of target information extraction tasks;

Step 102, using a pre-trained text generation model to infer the processing set, applying beam search to decode the inference results, and obtaining all possible paths, wherein the text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;

Step 103, screening all possible paths according to probability to obtain candidate paths;

Step 104: Prune the candidate paths according to the type of the target information extraction task to obtain the target information.

The text generation method based on entity description in the embodiment of the present application constructs a general text generation framework, designs different entity descriptions as output sequences for different information extraction tasks. When there are multiple tuples to be extracted in a text, the more effective "sequence to path" approach is used to avoid the order and independence problems between multiple tuples in the generated sequence, thereby completing various information extraction tasks well.

Optionally, in one embodiment of the present application, before using the pre-trained text generation model to perform reasoning on the processing set, the method further includes:

Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;

Use the enhanced dataset to train the pre-trained model to obtain a text generation model.

Optionally, in one embodiment of the present application, constructing an enhanced data set includes:

Get the training set corresponding to the target task;

Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;

According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;

Use the training set to train the pre-trained model and generate a sample generation model;

Use the sample generation model to infer the prediction training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;

The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;

An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.

Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.

This application proposes a text generation method based on entity description, and uses the optimization design method of this application in multiple information extraction tasks such as entity recognition, relationship extraction, entity linking, event extraction, semantic role labeling, and sentiment analysis. In the experiment, Nvidia 1080Ti GPU single graphics card is used for training, and the pre-trained model uses the t5-base model in huggingface. The following introduces the text generation method based on entity description of this application in combination with the text generation process of Figure 2 and taking practical application as an example:

Step 1: Taking the extraction of sentiment element triplets in sentiment analysis as an example, the conventional sequence-to-sequence tuple extraction method generates samples according to the number of input texts:

Input text → "(a1,o1,s1),(a2,o2,s2)"

Based on the sequence-to-path method, samples are generated according to the number of input tuples, and each piece of input text corresponds to an output tuple:

Among them, the entities to be extracted for different subtasks are shown in Table 1.

Table 1 Tuples to be extracted for each information extraction subtask

Step 2: For different subtasks, convert each output tuple as a positive sample into a natural language sentence as an output sequence according to the following method:

(1) Named Entity Recognition Task

The extraction target is the (<entity>, <category>) tuple in the text, where <category> is the category of the entity <entity>. The linearized natural sentence is as follows:

Positive sample <entiiy>isa<category>entity.

Negative sample <entity> is not a <categoty> entity.

(2) Joint extraction of entities and relations

The extraction target is the (<entity1>, <relation>, <entity2>) triple in the text, where <entity1> is the subject, <entity2> is the object, and <relation> is the relationship between the two. The linearized natural sentence is as follows:

Positive example: Therelation between <entity1> and <entity2> is <relation>.

Negative sample The relation between<entity1>and<entity2>is not<relation>.

(3) Semantic role analysis task

For a given predicate (<predicate>), the extraction target is the (<argument>, <role>) tuple in the text. The linearized natural sentence is as follows:

A positive example <argument> is the <role> of <predicate>.

Negative example <argument> is not the <role> of <predicate>.

(4) Event extraction task

Using the pipeline approach, the first step is to extract the trigger, and the second step is to extract the argument.

① The goal of extracting the trigger is the (<event trigger>, <event category>) tuple. The linearized natural language sentence is as follows:

Positive sample <eventtrigger>isa<eventcategory>event.

Negative sample <eventtrigger> is not a <eventcategory> event.

② The goal of extracting argument is the (<event argument>, <entity type>, <entity role>) triple. The linearized natural language sentence is as follows:

Positive Sample

<eventargument>isa<entitytype>type,anditisthe<entityrole>.

Negative samples

<event ar gument>is not a<entity type>type,or it is not the<entityrole>.

(5) Entity Linking Task

For a given entity <entity>, the extracted (<argument>) is linked to <entity>, and the linearized natural language sentence is as follows:

Positive example <argument>refers to <entity>.

Negative example <argument> not refer to <entity>.

(6) Sentiment analysis task

①AOPE mission

Positive sample F _a (a) is _{F o} (o).

Negative sample F _a (a) is not F _o (o).

②ASTE Mission

Positive sample It is F _s (s) because _{F a} (a) is _{F o} (o).

Negative Sample It is F _s (s) not because F _a (a) is F _o (o).

③TASD mission

Positive sample F _a (a)isaF _c (c)categoryandF _a (a)isF _o (o).

Negative sample F _a (a) is not aF _c (c) category or _{F a} (a) is not _{F o} (o).

④UABSA Mission

Positive sample F _a (a) is F _s (s)

Negative sample F _a (a) is not _{F s} (s)

⑤ACOS tasks

Positive sample F _c (c) is F _s (s) because F _a (a) is F _o (o).

Negative sample F _c (c) is F _s (s) not because _{F a} (a) is F _o (o).

in,

Step 3: Based on the pre-trained model T5, train the text generation model _Maug based on the training set for subsequent data enhancement;

Step 4: Use _Maug to predict the training set, construct negative samples D′ ₂ based on the wrong prediction results of _Maug , and then randomly replace the elements of different tuples in the positive sample D to construct negative samples D′ _1. Construction method of D′ ₁ : In order to improve the model's ability to match tuple elements, randomly replace the elements in the tuple; Construction method of D′ ₂ : In order to improve the model's ability to filter out wrong elements, we first use _Maug to generate some wrong examples and add them to the negative samples. The data set after data enhancement is _Daug = D∪D′ = D∪D′ ₁ ∪D′ ₂ . Take sentiment analysis triple extraction as an example: the input text is "Those rolls are big, but not good and sashimi wasn't fresh.", the tuples to be extracted are (rolls, big, positive), (rolls, notgood, negative), (rolls, sashimi, fresh), then the negative samples in D′ ₁ can be (rolls, wasn'tfresh, positive), (sashimi, big, negative, false), and the negative samples in D′ ₂ can be (sashimi, n'tfresh, negative, false).

Step 5: Similar to step 1, train a T5-based model M on _Daug . The loss function of an input sample is the average of the loss functions of each output sequence, expressed as:

in

Step 6: Use the trained model M to infer the test set, apply beam search for decoding, sort the paths in descending order of probability, and the k paths with the highest probability are the candidate paths.

Step 7: Prune k paths for different subtasks to get the final result. During model inference, the decoded words should not belong to the full vocabulary, but should come from the original input text or some other classification tags. For example, in the ASTE task in sentiment analysis, the search range of sentiment polarity s is only in {positive, negative, neutral}. The candidate word set is denoted as T, which consists of the words T(x) of the input text and the classification tag T(task).

Right now

T(x,task)＝T(x)∪T(task)

However, in the actual model reasoning process, the generated words do not necessarily come from the candidate set T, such as the difference between singular and plural or uppercase and lowercase. Therefore, for the case where the generated word e does not belong to the elements in T, the Levenshtein distance from e to all candidate elements can be measured, and the one with the smallest distance can be selected to replace e. A simpler way is to directly limit the generated words to the range of T. Furthermore, the elements to be generated can be divided into extraction-type elements and classification-type elements. The extraction-type elements come from T(x), and the classification-type elements come from T(task). Each type of element has its own limited range.

In summary, this application can effectively complete various information extraction tasks and has good versatility. Table 2 is the experimental results on the sentiment analysis (ACOS subtask) dataset, and Table 3 is the experimental results on the relationship extraction dataset, which proves that this method is effective.

Table 2 Sentiment analysis experimental results (taking ACOS task as an example)

Table 3. Experimental results of relation extraction

In order to implement the above embodiment, the present application also proposes a text generation device based on entity description.

FIG3 is a schematic diagram of the structure of a text generation device based on entity description provided in an embodiment of the present application.

As shown in FIG3 , the text generation device based on entity description includes: an acquisition module, a path generation module, a path screening module, and a pruning module, wherein:

An acquisition module is used to obtain a pending set of target information extraction tasks;

A path generation module is used to use a pre-trained text generation model to infer the processing set, and apply beam search to decode the inference results to obtain all possible paths. The text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;

The path screening module is used to screen all possible paths according to probability to obtain candidate paths;

The pruning module is used to prune candidate paths to obtain target information according to the type of target information extraction task.

Optionally, in one embodiment of the present application, a text generation model training module is further included, which is specifically used to:

Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;

Use the enhanced dataset to train the pre-trained model to obtain a text generation model.

Optionally, in one embodiment of the present application, constructing an enhanced data set includes:

Get the training set corresponding to the target task;

Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;

According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;

Use the training set to train the pre-trained model and generate a sample generation model;

Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;

The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;

An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.

Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.

It should be noted that the aforementioned explanation of the embodiment of the text generation method based on entity description is also applicable to the text generation device based on entity description of this embodiment, and will not be repeated here.

In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims (8)

1. A text generation method based on entity description, comprising the steps of:

acquiring a to-be-processed set of a target information extraction task;

Reasoning the set to be processed by using a pre-trained text generation model, and decoding a reasoning result by using beam search to obtain all possible paths, wherein the text generation model is generated by training the pre-trained model by using a corresponding training set according to the target information extraction task;

screening all possible paths according to probability to obtain candidate paths;

and pruning the candidate paths according to the type of the target information extraction task to obtain target information.

2. The method of claim 1, further comprising, prior to said reasoning the set of pending using a pre-trained text generation model:

Constructing an enhanced data set, wherein the enhanced data set comprises positive and negative samples;

training a pre-training model by using the enhanced data set to obtain the text generation model.

3. The method of claim 2, wherein constructing the enhanced data set comprises:

Acquiring a training set corresponding to a target task;

Preprocessing the set to be processed and the training set to generate positive samples, wherein each text section corresponds to one output tuple in preprocessing, and the number of the positive samples is the same as that of the output tuples;

according to the type of the target task, converting each output tuple into a natural language sentence as an output sequence according to the entity description mode of the positive sample;

Training the pre-training model by using a training set to generate a sample generation model;

reasoning the training set by using the sample generation model, and decoding reasoning results by using beam search to obtain all possible paths and prediction results, wherein the prediction results are combinations of the possible paths;

Taking the error part in the prediction result as a first negative sample, and randomly replacing elements of different tuples in the positive sample by using the first negative sample to generate a second negative sample;

the enhanced data set is constructed from the positive sample, the first negative sample, and the second negative sample.

4. A method according to claim 3, wherein the loss function of an input sample in the enhancement data set is an average of the loss functions of each output sequence when training a pre-training model using the enhancement data set.

5. The text generation device based on entity description is characterized by comprising an acquisition module, a path generation module, a path screening module and a pruning module, wherein,

The acquisition module is used for acquiring a set to be processed of the target information extraction task;

The path generation module is used for reasoning the set to be processed by using a pre-trained text generation model, and decoding a reasoning result by applying beam search to obtain all possible paths, wherein the text generation model is generated by training the pre-trained model by using a corresponding training set according to the target information extraction task;

The path screening module is used for screening all the possible paths according to probability to obtain candidate paths;

And the pruning module is used for pruning the candidate paths according to the type of the target information extraction task to obtain target information.

6. The apparatus of claim 5, further comprising a text generation model training module, in particular for:

Constructing an enhanced data set, wherein the enhanced data set comprises positive and negative samples;

training a pre-training model by using the enhanced data set to obtain the text generation model.

7. The apparatus of claim 6, wherein the constructing an enhanced data set comprises:

Acquiring a training set corresponding to a target task;

preprocessing the set to be processed and the training set to generate positive samples, wherein each text in the training set corresponds to one output tuple, and the number of the positive samples is the same as the number of the output tuples;

according to the type of the target task, converting each output tuple into a natural language sentence as an output sequence according to the entity description mode of the positive sample;

Training the pre-training model by using a training set to generate a sample generation model;

reasoning the training set by using the sample generation model, and decoding reasoning results by using beam search to obtain all possible paths and prediction results, wherein the prediction results are combinations of the possible paths;

Taking the error part in the prediction result as a first negative sample, and randomly replacing elements of different tuples in the positive sample by using the first negative sample to generate a second negative sample;

the enhanced data set is constructed from the positive sample, the first negative sample, and the second negative sample.

8. The apparatus of claim 7, wherein the loss function for one input sample in the enhancement data set is an average of the loss functions for each output sequence when training a pre-training model using the enhancement data set.