CN118227790A - Text classification method, system, device and medium based on multi-label association - Google Patents

Abstract

The invention discloses a text classification method, a system, equipment and a medium based on multi-label association, wherein the method comprises the following steps: acquiring a plurality of texts of known label categories, and constructing a training set and a testing set; the training set and the testing set comprise: a plurality of texts and a plurality of tag categories, wherein the known tag category of each text is a plurality; the training set is divided into two parts: a first training subset and a second training subset; inputting each text and a plurality of label categories in the first training subset into a text classification model, and training the model to obtain a primarily trained text classification model; inputting each text and a plurality of label categories in the second training subset into a text classification model after preliminary training, and training the model to obtain a final trained text classification model; and testing the finally trained text classification model according to the test set, and classifying the text to be classified by using the tested network model.

Description

Text classification method, system, device and medium based on multi-label association

Technical Field

The present invention relates to the technical field of machine learning and natural language processing, and in particular to a text classification method, system, device and medium based on multi-label association.

Background technique

The rapid development of the Internet and social media has led to the generation of a large amount of non-standard text data, which often contains a large amount of informal language, new network words, special symbols and emoticons, reflecting rich social information and personal emotions. The non-standard nature of text is not only reflected in the use of language, but also in the complexity of its structure and semantics, which makes understanding and processing these text data a difficult task.

One of the main challenges of non-standard text is the informality of its language. Although it provides space for the richness of language and the diversity of expression, it also brings unprecedented challenges to traditional text classification methods. The text widely uses new words and abbreviations on the Internet, which are often difficult to find corresponding items in traditional dictionaries or language models. In addition, the frequent use of emoticons and special characters further increases the difficulty of text interpretation.

Another notable feature of non-standard text data is that a single text may involve multiple topics or emotions at the same time, that is, a text may be associated with multiple tags. At the same time, there is a certain correlation between these tags. The existence of tag correlation is both a challenge and an opportunity. If the correlation between tags can be effectively identified and utilized, it can not only improve the accuracy of classification, but also help deepen the understanding of text content and structure. However, traditional text classification methods often ignore this complex relationship between tags, resulting in the inability to fully utilize this information to optimize classification results.

Summary of the invention

In order to address the shortcomings of the prior art, the present invention provides a text classification method, system, device and medium based on multi-label association; it aims to optimize the multi-label classification problem of non-standard text on platforms such as social media. The core lies in deeply exploring and utilizing the complex relationship between labels, and realizing accurate classification of text data by integrating advanced deep learning technology and label association analysis.

On the one hand, a text classification method based on multi-label association is provided;

Text classification methods based on multi-label association include:

Acquire multiple texts of known label categories and construct a training set and a test set; the training set and the test set include: multiple texts and multiple label categories, wherein each text has multiple known label categories; divide the training set into two parts: a first training subset and a second training subset;

Input each text and multiple label categories in the first training subset into the text classification model, train the model, and obtain a preliminarily trained text classification model;

Input each text and multiple label categories in the second training subset into the text classification model after preliminary training, and train the model. During the training process, calculate the conditional probability between labels according to the label category prediction probability of each text, construct a label association matrix based on the conditional probability, construct an association loss function based on the label association matrix, and construct a second loss function of the text classification model based on the association loss function and the cross entropy loss function. When the loss value of the second loss function is not less than a preset threshold, perform a second update on the parameters of the text classification model after preliminary training according to the loss value of the second loss function, and use the text classification model after the second update to obtain the label category prediction probability of each text, until the loss value of the second loss function is calculated to be less than the preset threshold, so as to obtain the final trained text classification model;

According to the test set, the final trained text classification model is tested, and the text to be classified is classified using the tested network model.

On the other hand, a text classification system based on multi-label association is provided;

Text classification system based on multi-label association, including:

The acquisition module is configured to: acquire multiple texts of known label categories, and construct a training set and a test set; the training set and the test set include: multiple texts and multiple label categories, wherein each text has multiple known label categories; divide the training set into two parts: a first training subset and a second training subset;

A primary training module is configured to: input each text and multiple label categories in the first training subset into the text classification model, train the model, and obtain a preliminarily trained text classification model;

A secondary training module, which is configured to: input each text and multiple label categories in the second training subset into the text classification model after preliminary training, train the model, calculate the conditional probability between labels according to the label category prediction probability of each text during the training process, construct a label association matrix based on the conditional probability, construct an association loss function based on the label association matrix, construct a second loss function of the text classification model based on the association loss function and the cross entropy loss function, and when the loss value of the second loss function is not less than a preset threshold, perform a secondary update on the parameters of the text classification model after preliminary training according to the loss value of the second loss function, and use the text classification model after the secondary update to obtain the label category prediction probability of each text, until the loss value of the second loss function is calculated to be less than the preset threshold, and obtain the final trained text classification model;

The testing module is configured to: test the final trained text classification model according to the test set, and classify the text to be classified using the tested network model.

On the other hand, an electronic device is provided, comprising:

a memory for non-transitory storage of computer-readable instructions; and

a processor for executing the computer readable instructions,

When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

On the other hand, a storage medium is provided, which non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is used to implement the method described in the first aspect when running on one or more processors.

The above technical solution has the following advantages or beneficial effects:

The present invention adopts correlation analysis technology and model algorithm among multiple labels, which can more accurately capture the semantic information of non-standard text data, thereby effectively improving the classification efficiency and accuracy of non-standard text.

The present invention adopts the association loss function fusion regularization technology to effectively prevent the model from overfitting, so that the model can maintain good classification performance when facing new and unseen data, and enhance the generalization ability of the model.

The present invention also collects evaluation feedback information of model performance, inputs it into the model again for iteration, and continuously adjusts and optimizes model parameters to make the model more adaptable and better meet actual application scenarios and user needs.

In short, the present invention has not only achieved breakthroughs in technical performance, but also demonstrated great potential in the breadth and depth of practical applications. It plays an important role in promoting the advancement and widespread application of multi-label classification technology. The implementation of this technical solution will build an efficient, accurate and user-friendly multi-label classification system, which can not only provide accurate label prediction services, but also process and understand non-standard text data commonly found on platforms such as social media, and meet users' needs for complex text classification in different fields. In general, the implementation of the present invention will greatly enrich the scope of application of multi-label classification technology, and improve its performance and user experience in practical applications.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

FIG1 is a flow chart of a method according to Embodiment 1.

Detailed ways

It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

Embodiment 1

This embodiment provides a text classification method based on multi-label association;

As shown in Figure 1, the text classification method based on multi-label association includes:

S101: Acquire multiple texts of known label categories, and construct a training set and a test set; the training set and the test set include: multiple texts and multiple label categories, wherein each text has multiple known label categories; divide the training set into two parts: a first training subset and a second training subset;

S102: input each text and multiple label categories in the first training subset into a text classification model, train the model, and obtain a preliminarily trained text classification model;

S103: Input each text and multiple label categories in the second training subset into the text classification model after preliminary training, and train the model. During the training process, according to the label category prediction probability of each text, the conditional probability between labels is calculated, and a label association matrix is constructed based on the conditional probability. The association loss function is constructed based on the label association matrix. The second loss function of the text classification model is constructed based on the association loss function and the cross entropy loss function. When the loss value of the second loss function is not less than a preset threshold, the parameters of the text classification model after preliminary training are updated for the second time according to the loss value of the second loss function. The label category prediction probability of each text is obtained by using the text classification model after the second update, until the loss value of the second loss function is calculated to be less than the preset threshold, so as to obtain the final trained text classification model.

S104: Testing the finally trained text classification model according to the test set, and classifying the text to be classified using the tested network model.

Furthermore, the S101: obtaining multiple texts of known label categories and constructing a training set and a test set specifically includes:

The acquired text is cleaned, normalized, processed with special symbols and emoticons.

The text cleaning includes: removing spaces or punctuation marks;

The text normalization includes: expanding abbreviations in the text into complete words;

The special symbols include: deleting the special symbols, the special symbols are &, ￥;

The emoticon processing includes: replacing the emoticon with a word.

It should be understood that a series of preprocessing steps including text cleaning, text normalization, special symbols and emoji processing are performed on the input text to convert the raw text into a more standardized and cleaner form so that the model can better understand and process the data.

Exemplarily, first, basic cleaning work is performed on the text to remove irrelevant information such as extra spaces, misused punctuation marks, etc. Then, informal expressions and abbreviations in the text are expanded and standardized. For example, the network term "lol" is converted into the full name "laughing out loud". Then, for the common special symbols and emoticons in the text, the present invention adopts a special mapping strategy, specifically, mapping the special symbols and emoticons to predefined tags or words with clear semantic representations, for example, the emoticon ":)" is mapped to the word "happy".

Furthermore, the S101: dividing the training set into two parts: a first training subset and a second training subset, is divided according to a set ratio.

Furthermore, the S102: text classification model comprises: a transformer model, a fully connected layer and an activation function layer connected in sequence; the transformer model comprises: a BERT or RoBERTa model;

Among them, the transformer model is used to extract semantic features from the input text;

Among them, the fully connected layer is used to map the extracted features to obtain a mapping vector;

Among them, the activation function layer is used to output the existence probability of each label.

It should be understood that a pre-trained Transformer Encoder, such as BERT or RoBERTa, is used to encode text sequences to extract deep semantic features of the text. Advanced encoding techniques are used to ensure that the model can effectively handle new words, unknown words, and non-standard terms in non-standard texts, so that the model can fully understand the contextual meaning of the text and provide a solid foundation for subsequent label prediction.

It should be understood that through the fully connected layer and the sigmoid activation function, the semantic representation is mapped to a vector of the same length as the total number of tags, and the existence probability of each tag is output. This step provides an initial judgment for determining whether each tag belongs to the current text.

Exemplarily, a pre-trained transformer model, such as BERT or RoBERTa, is used to encode the input text sequence. In this step, a subword-based encoding method is used, and a dynamic subword splitting strategy is adopted to split words that are not in the model vocabulary (such as new words or proper nouns) into smaller units (subwords) for processing.

The subword-based encoding method includes: first, vocabulary construction: learning and constructing a vocabulary from a large amount of text data, the vocabulary includes subword units such as common words, roots, prefixes, suffixes and single characters. Then, before the text is input into the text classification model, each word in the text to be classified is decomposed into one or more subword units using the constructed vocabulary. If the word in the text to be classified exists directly in the vocabulary, it is used directly; if the word in the text to be classified does not exist in the vocabulary, it is decomposed into smaller subword units until all words can be represented by subword units in the vocabulary. After that, each subword unit corresponds to a unique digital ID. After the words in the text are split into subwords, they are converted into a series of digital IDs for model input.

The dynamic subword splitting strategy means that if there are words in the text to be classified that are not in the vocabulary, the text to be classified will be split into smaller units (subwords). The entire splitting process is dynamic, and the words are decomposed into meaningful subunits as much as possible based on the information in the vocabulary.

Suppose a vocabulary has the word "happy" and sub-word units "un", "ness", etc. If you encounter the encoded word "unhappiness", check whether this word is directly in the vocabulary. If not, you can decompose it into a sequence of sub-word units such as "un", "happy", "ness", and then convert these sub-word units into corresponding numeric IDs.

In addition, the model adjusts the subword embedding representation and enhances the semantic representation according to the context information, and outputs a hidden state vector with a dimension of 768 or 1024 after processing. These vectors capture rich semantic information at the subword level, provide rich context representation for subsequent label prediction, and lay the foundation for the model to understand the semantic information of the text. For example, pre-trained Transformer models such as BERT or RoBERTa use the self-attention mechanism inside the self-attention layer of the pre-trained Transformer model to help the model learn how to adjust the word embedding representation according to the context and how to enhance the semantic representation.

For example, the text semantic representation is passed through a fully connected layer to map it to a vector with a dimension equal to the total number of tags. After that, the sigmoid activation function is used to output the existence probability of each tag. For example, if there are 10 tags in total, the network will output a 10-dimensional probability vector, where each dimension represents the predicted probability of a tag.

Furthermore, the S102: calculates the loss value of the first loss function of the text classification model according to the label category prediction result of each text and the known label category of each text, and the first loss function is implemented by using a cross entropy loss function.

It should be understood that setting a threshold and performing preliminary classification based on the predicted probability provides initial label assignment for subsequent association analysis and model training.

Exemplarily, threshold determination and preliminary classification: a threshold (e.g., 0.5) is set to compare the probability output by the sigmoid function with the threshold, and through this comparison, it is determined whether each label should be assigned to the current text. This step implements a preliminary classification decision, but does not consider the correlation between labels.

Furthermore, the step S102: inputting each text and multiple label categories in the first training subset into a text classification model, training the model, and obtaining a preliminarily trained text classification model specifically includes:

During the training process, the loss value of the first loss function of the text classification model is calculated based on the label category prediction results of each text and the known label category of each text. When the loss value of the first loss function is not less than a preset threshold, the parameters of the text classification model are updated according to the loss value of the first loss function, and the updated text classification model is used to obtain the label category prediction results of each text, until the loss value of the first loss function of the text classification model is calculated to be less than the preset threshold, thereby obtaining a text classification model after preliminary training.

Furthermore, in S103, each text and multiple label categories in the second training subset are input into the text classification model after preliminary training, and the model is trained. During the training process, the conditional probability between labels is calculated according to the label category prediction probability of each text. The specific conditional probability is expressed as:

Among them, P(i) represents the probability that the model predicts that text X belongs to label i for text X. P(i∩j) represents the probability that text X belongs to label i and label j at the same time.

It should be understood that the conditional probability between labels is calculated, which represents the probability of another label appearing when one label has already appeared. The conditional probability is then used to construct a label association matrix to deeply analyze the co-occurrence and dependency relationship between labels. This step is the core of the present invention, which reveals the complex relationship between labels and provides important correlation information for model training.

Tag association analysis: Use statistical methods to calculate the conditional probability between tag pairs and construct a tag association matrix. For tags i and j, calculate P(j|i) and P(i|j), that is, the probability that tag j exists when tag i exists and the probability that tag i exists when tag j exists.

These conditional probabilities reflect the dependency and co-occurrence relationship between labels. These conditional probabilities are combined into a matrix, in which each element represents the degree of association between the corresponding label pairs. This association matrix is used to calculate the association loss function in subsequent model training.

Furthermore, the S103: constructs a tag association matrix based on conditional probability, wherein the tag association matrix A is an n×n matrix, each element A _ij represents the conditional probability A _ij of tag j appearing under the condition that tag i appears, and the formula is expressed as:

_Aij = P(j|i);

This means that the element in the i-th row and j-th column of the matrix represents the probability of label j appearing given label i.

Furthermore, the S103: constructs an association loss function based on the label association matrix. The association loss function is used to measure the difference between the label association degree predicted by the model and the actual label association matrix. The specific association loss function is expressed as follows:

Where n represents the total number of labels, A _ij is an element in the label association matrix, which represents the probability of label j appearing under the condition that label i appears, pi represents the probability that the model predicts that the instance belongs to label i, and p _j represents the probability that the model predicts that the instance belongs to label j. | _pi * _pj - _Aij | represents the absolute error term, which measures the difference between the probability of label i and label j appearing at the same time predicted by the model and the corresponding element _Aii in the label association matrix.

Furthermore, the S103: constructing a second loss function of the text classification model based on the association loss function and the cross entropy loss function, wherein the second loss function L includes:

L＝L _CE +γLassoc

Among them, L _CE represents the cross entropy loss function, Lassoc represents the association loss function, and γ is a balance parameter, which is a positive real number used to adjust the relative importance of the cross entropy loss function and the association loss function in the total loss function.

It should be understood that during the model training phase, in addition to the traditional cross-entropy loss function used to measure classification accuracy, an additional association loss function is introduced. The association loss function is based on the label association matrix. Its purpose is to punish the situation in which the model prediction process does not conform to the association between known labels. The cross-entropy loss function and the association loss function form the final second loss function, which enhances the model's ability to capture label associations. Gradient descent and regularization techniques are used to prevent overfitting and ensure that the model has good generalization capabilities.

At the same time, the Adam optimizer is used for gradient descent. The Adam optimizer has the feature of adaptive learning rate adjustment, which can automatically adjust the learning rate according to different gradient situations, helping to improve the training efficiency and convergence speed of the model.

In addition, a learning rate decay mechanism is set, which helps to gradually reduce the learning rate during training, making the model more stable when approaching the optimal solution. To further prevent overfitting, an L2 regularization term is added to the loss function. Finally, the selection of hyperparameters is done by combining grid search with cross-validation to determine the optimal value of the hyperparameters.

Furthermore, S104: according to the test set, the finally trained text classification model is tested, and the text to be classified is classified using the tested network model, wherein the test passing criterion is that the calculated precision, recall and F1 score indicators are all greater than the set threshold.

It should be understood that during the model evaluation phase, a variety of evaluation indicators are used to measure model performance, test the generalization ability of the model on data sets in different fields, and test the classification effect of the model for different types of labels. After the model training is completed, the test set is predicted, and indicators such as precision, recall, and F1 score are calculated to evaluate the model performance. In order to evaluate the generalization ability of the model, further tests are performed on independent data sets. In addition, the classification performance of different types of labels (such as high-frequency, low-frequency, and medium-frequency labels) is evaluated separately to fully understand the performance of the model.

It should be understood that the scope of application and expansion research will continue to explore the application potential of tag correlation analysis in multiple fields. In addition, attempts will be made to improve the ability to handle complex tag relationships through algorithm optimization and model iteration for more complex tag structures, such as hierarchical tag systems. Iterative improvement and application practice: The model will be iteratively optimized based on the evaluation results, and the actual application scenarios will be considered to further adjust the model parameters and structure to improve user experience and meet specific needs.

Expand research and cross-domain applications: Explore the application potential of label relevance in other fields, such as bioinformatics, scene recognition, etc., try to develop new algorithms to handle more complex label structures, and try to extend this method to other data types, such as classification annotation of image and audio data.

Scenario 1: In a news classification system, each news article needs to be assigned one or more tags, such as "international", "politics", "economy", "sports", etc. There are potential correlations between these tags, for example, "international" and "politics" often appear together.

Implementation steps:

1. Data preparation: A dataset of 10,000 news articles was collected, with an average of 3 relevant tags per article. Text cleaning and preprocessing were performed, including removing stop words, special characters, etc.

2. Model training:

Use BERT as the encoder to encode the articles and obtain the semantic representation of each article. Set the maximum sequence length of BERT to 512 and the batch size to 16.

Design a fully connected layer and sigmoid function to output the probability of existence of each tag. The output dimension of this fully connected layer is 50 (the total number of unique tags in this case).

Calculate the conditional probabilities between all label pairs and construct a label association matrix.

A loss term based on the label association matrix was added to the model training, and the Adam optimizer was used for gradient descent. The initial learning rate was set to 1e-5 and decayed to 0.96 every 10,000 steps. L2 regularization was applied to prevent overfitting, and hyperparameters were selected through grid search and cross-validation.

3. Model evaluation:

Tested on an independent test set (containing 2,000 news articles), the model achieved 85% precision, 80% recall, and 82% F1 score.

By analyzing the classification performance of different types of tags, it was found that the model’s accuracy rate can reach 90% when identifying high-frequency tags (such as “sports”), and the accuracy rate is 75% when identifying low-frequency tags (such as “politics”).

The model was tested on a new news dataset consisting of 500 news articles, and the model maintained similar performance indicators on this dataset.

4. Iterative improvement: Based on the feedback from the model evaluation, it was found that the model had difficulty distinguishing between the "international" and "political" labels. To this end, the weights of these two labels in the association matrix were adjusted and the model was retrained.

5. Deployment and application: Deploy the trained model to the actual news classification system.

Through this implementation case, the news classification system can more accurately label articles and improve the classification quality by combining the analysis of label correlation.

Scenario 2: A social media platform needs to categorize user-generated content in order to recommend it to other interested users or conduct content review. Common content tags include "travel", "food", "technology", "entertainment", etc. These tags are usually related to each other, for example, "food" and "travel" are often tagged at the same time.

Implementation steps:

1. Data preparation: 50,000 labeled social media posts were collected as training data sets, with an average of 2 relevant tags for each post. Text cleaning, text normalization, and special symbols and emoticons were performed on them.

2. Model training:

Use the BERT model as the encoder and set the batch size to 32 to encode the post content.

The fully connected layer output layer maps to 20 possible labels (the total number of unique labels in this case) and predicts the probability of each label using the sigmoid function.

The label association matrix is calculated based on the training data to guide the model to learn the pattern of label co-occurrence.

The model was trained by combining the cross entropy loss and the association loss function. The Adam optimizer was used, and the initial learning rate was set to 3e-5 and decayed by 0.95 after every 5,000 iterations.

To prevent overfitting, weight decay (L2 regularization) is added during model training.

3. Model evaluation:

The model performance was evaluated on an independent test set of 10,000 posts and obtained a precision of 82%, a recall of 79%, and an F1 score of 80%.

For specific tags such as “travel” and “entertainment”, the model demonstrated above-average performance, reaching 85% and 84% precision respectively.

When running the model with the new dataset, it was found that the model was able to maintain stable classification capabilities on different types of content, with the accuracy remaining between 81% and 83%.

4. Iterative improvement: Based on feedback, we found that the model was somewhat confused when distinguishing between "technology" and "entertainment". Therefore, we adjusted the label association matrix and retrained the model to improve the differentiation. The accuracy of the adjusted model on the "technology" and "entertainment" labels increased to 80% and 86% respectively.

5. Deployment and application: Deploying the optimized model to the system of the social media platform can label posts with relevant tags, helping the content operation team to manage and review content more effectively.

In this implementation case, through in-depth mining and application of tag relevance, the content classification system of the social media platform improved the accuracy of classification and enhanced the content management efficiency of the operation team.

This method can effectively handle informal language, special symbols and emoticons, and can identify and utilize the correlation between tags to improve the accuracy and efficiency of classification. In addition, considering the rapid evolution of social media content and language usage trends, this system is also adaptable and flexible, and can continuously optimize itself over time to cope with the changing text characteristics and classification needs.

By introducing and deepening the analysis of label relevance, the present invention significantly improves the accuracy and efficiency of multi-label classification in non-standard text data. The method and system not only show excellent performance in the current social media content classification, but its flexibility and adaptability design also provide a solid foundation for future development and application, and show wide application potential for solving multi-label text classification tasks under complex label relationships.

The present invention is designed to improve the accuracy and efficiency of multi-label classification tasks for text data. First, the method performs a series of preprocessing steps on the input text, then uses a deep learning model to encode it to capture deep semantic information, and then converts the encoded text into prediction probabilities corresponding to all possible labels through a mapping mechanism. Then, the method introduces an innovative label association analysis step, which constructs a label association matrix by quantitatively calculating the conditional probabilities between labels, thereby revealing the dependencies and co-occurrence relationships between labels. In the model training stage, the model parameters are adjusted and optimized in combination with these association information, so that the model can better process and predict a set of related labels. In addition, the present invention includes a complete set of evaluation processes for measuring the performance of the optimized model and iteratively optimizing it according to the evaluation results. In order to enhance the practicality and flexibility of the model, the present invention also deeply explores how to apply the multi-label association classification method and system for non-standard text data to specific scenarios, and looks forward to its wide application prospects in different fields. Overall, this invention, by combining advanced deep learning technology and detailed label correlation analysis, not only demonstrates innovation in theory, but also proves its high practical value in practice, effectively promoting the advancement of natural language processing and machine learning technology.

Embodiment 2

This embodiment provides a text classification system based on multi-label association;

Text classification system based on multi-label association, including:

The acquisition module is configured to: acquire multiple texts of known label categories, and construct a training set and a test set; the training set and the test set include: multiple texts and multiple label categories, wherein each text has multiple known label categories; divide the training set into two parts: a first training subset and a second training subset;

A primary training module is configured to: input each text and multiple label categories in the first training subset into the text classification model, train the model, and obtain a preliminarily trained text classification model;

A secondary training module, which is configured to: input each text and multiple label categories in the second training subset into the text classification model after preliminary training, train the model, calculate the conditional probability between labels according to the label category prediction probability of each text during the training process, construct a label association matrix based on the conditional probability, construct an association loss function based on the label association matrix, construct a second loss function of the text classification model based on the association loss function and the cross entropy loss function, and when the loss value of the second loss function is not less than a preset threshold, perform a secondary update on the parameters of the text classification model after preliminary training according to the loss value of the second loss function, and use the text classification model after the secondary update to obtain the label category prediction probability of each text, until the loss value of the second loss function is calculated to be less than the preset threshold, and obtain the final trained text classification model;

The testing module is configured to: test the final trained text classification model according to the test set, and classify the text to be classified using the tested network model.

It should be noted that the acquisition module, the first training module, the second training module and the test module correspond to steps S101 to S104 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

Embodiment 3

This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.

It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Embodiment 4

This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. The text classification method based on multi-label association is characterized by comprising the following steps:

Acquiring a plurality of texts of known label categories, and constructing a training set and a testing set; the training set and the testing set comprise: a plurality of texts and a plurality of tag categories, wherein the known tag category of each text is a plurality; the training set is divided into two parts: a first training subset and a second training subset;

inputting each text and a plurality of label categories in the first training subset into a text classification model, and training the model to obtain a primarily trained text classification model;

Inputting each text and a plurality of label categories in a second training subset into a preliminarily trained text classification model, training the model, calculating the conditional probability between labels according to the label category prediction probability of each text in the training process, constructing a label association matrix based on the conditional probability, constructing an association loss function based on the label association matrix, constructing a text classification model second loss function based on the association loss function and the cross entropy loss function, carrying out secondary update on parameters of the preliminarily trained text classification model according to the loss value of the second loss function when the loss value of the second loss function is not smaller than a preset threshold, and obtaining the label category prediction probability of each text by using the secondarily updated text classification model until the loss value of the second loss function is smaller than the preset threshold, so as to obtain a finally trained text classification model;

and testing the finally trained text classification model according to the test set, and classifying the text to be classified by using the tested network model.

2. The text classification method based on multi-label association according to claim 1, wherein each text and a plurality of label categories in the first training subset are input into a text classification model, the model is trained, and a primarily trained text classification model is obtained, specifically comprising:

In the training process, according to the label type prediction result of each text and the known label type of each text, calculating the loss value of a first loss function of the text classification model, updating parameters of the text classification model according to the loss value of the first loss function when the loss value of the first loss function is not smaller than a preset threshold value, and obtaining the label type prediction result of each text by using the updated text classification model until the loss value of the first loss function of the text classification model is smaller than the preset threshold value, so as to obtain the primarily trained text classification model.

3. The text classification method based on multi-label association according to claim 1, wherein each text and a plurality of label categories in the second training subset are input into a text classification model after preliminary training, the model is trained, and in the training process, the condition probability between labels is calculated according to the label category prediction probability of each text, and the specific condition probability is expressed as:

Where P (i) represents the probability that the model predicts that text X belongs to tag i for text X, and P (i n j) represents the probability that text X belongs to tag i while belonging to tag j.

4. The multi-tag association based text classification method of claim 1 wherein a tag association matrix is constructed based on conditional probabilities, wherein tag association matrix a is an n x n matrix, wherein each element a _ij represents a conditional probability a _ij of the occurrence of tag j under the condition that tag i occurs, and the formula is:

A_ij＝P(j|i)；

element a _ij of the ith row and jth column of the matrix represents the conditional probability P (j|i) that tag j appears given tag i.

5. The text classification method based on multi-label association according to claim 1, wherein an association loss function is constructed based on a label association matrix, the association loss function is used for measuring the difference between the label association degree predicted by the model and an actual label association matrix, and the specific association loss function is expressed as follows:

Wherein n represents the total number of labels, A _ij is an element in a label association matrix, represents the probability of label j under the condition that label i appears, represents the probability of model prediction instance belonging to label i, p _j represents the probability of model prediction instance belonging to label j, |p _i*p_j-A_ij | represents an absolute error term, and the difference between the probability of simultaneous appearance of label i and label j of model prediction and a corresponding element A _ij in the label association matrix is measured.

6. The text classification method based on multi-label association of claim 1 wherein a text classification model second penalty function is constructed based on the association penalty function and the cross entropy penalty function, wherein the second penalty function L comprises:

L＝L_CE+γLassoc；

where L _CE represents the cross entropy loss function, lassoc represents the associated loss function, and γ is the balance parameter.

7. A text classification system based on multi-tag association, comprising:

An acquisition module configured to: acquiring a plurality of texts of known label categories, and constructing a training set and a testing set; the training set and the testing set comprise: a plurality of texts and a plurality of tag categories, wherein the known tag category of each text is a plurality; the training set is divided into two parts: a first training subset and a second training subset;

A primary training module configured to: inputting each text and a plurality of label categories in the first training subset into a text classification model, and training the model to obtain a primarily trained text classification model;

A secondary training module configured to: inputting each text and a plurality of label categories in a second training subset into a preliminarily trained text classification model, training the model, calculating the conditional probability between labels according to the label category prediction probability of each text in the training process, constructing a label association matrix based on the conditional probability, constructing an association loss function based on the label association matrix, constructing a text classification model second loss function based on the association loss function and the cross entropy loss function, carrying out secondary update on parameters of the preliminarily trained text classification model according to the loss value of the second loss function when the loss value of the second loss function is not smaller than a preset threshold, and obtaining the label category prediction probability of each text by using the secondarily updated text classification model until the loss value of the second loss function is smaller than the preset threshold, so as to obtain a finally trained text classification model;

A test module configured to: and testing the finally trained text classification model according to the test set, and classifying the text to be classified by using the tested network model.

8. An electronic device, comprising:

a memory for non-transitory storage of computer readable instructions; and

A processor for executing the computer-readable instructions,

Wherein the computer readable instructions, when executed by the processor, perform the method of any of the preceding claims 1-6.

9. A storage medium, characterized by non-transitory storage of computer readable instructions, wherein the instructions of the method of any of claims 1-6 are performed when the non-transitory computer readable instructions are executed by a computer.

10. A computer program product comprising a computer program for implementing the method of any of the preceding claims 1-6 when run on one or more processors.