CN117808923A - Image generation method, system, electronic device and readable storage medium - Google Patents

Abstract

The invention discloses an image generation method, system, electronic device and readable storage medium, which relates to the field of image content generation. In order to solve the problem that the solution of generating images from pure text cannot meet the emotional needs in specific task scenarios, the image generation method includes: Obtain semantic guidance text and emotion guidance text; retrieve multiple reference image samples based on the semantic guidance text and emotion guidance text; extract features of multiple reference image samples, and combine at least two features among all features to obtain multiple image combinations Semantic features; obtain the text semantic features corresponding to the semantic guidance text, and generate associated images based on the image with the highest similarity to the text semantic features by combining the semantic features. The invention can improve image generation accuracy, make the generated associated images highly associated with guidance text and emotional text, and satisfy the semantic text requirements of the task scene while also meeting the emotional needs of the task scene.

Description

An image generation method, system, electronic device and readable storage medium

Technical field

The present invention relates to the field of image content generation, and in particular to an image generation method, system, electronic device and readable storage medium.

Background technique

Image content generation is a technique for generating image content based on guidance input in a given modality, such as text, 3D (Three Dimensions, three-dimensional), point clouds, or other forms of information. With the iterative update of technology, AI (Artificial Intelligent, artificial intelligence) image content generation has gradually become an important source of Internet content. As the most important image AIGC (Artificial Intelligent Generated Content) method, generating images based on plain text is the most common task form. However, in some task scenarios, such as text illustration task scenarios, not only the generated The image is the same as the semantic text of the task scene, and the generated image also needs to meet the specified emotion in the task scene. However, the existing pure text-based image generation solution cannot meet the emotional needs of some task scenes.

Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art currently need to solve.

Contents of the invention

The purpose of the present invention is to provide an image generation method, system, electronic device and readable storage medium that can improve the accuracy of image generation and make the generated associated images highly relevant to the guidance text and emotional text, while meeting the semantic text requirements of the task scene. while meeting the emotional needs of the task scenario.

In order to solve the above technical problems, the present invention provides an image generation method, comprising:

Get semantic guidance text and emotion guidance text;

Retrieve multiple reference image samples based on the semantic guidance text and the emotion guidance text;

Extract features of multiple reference image samples, and combine at least two of all the features to obtain multiple image combination semantic features;

Text semantic features corresponding to the semantic guidance text are obtained, and semantic features are combined based on the image with the highest similarity to the text semantic features to generate a related image.

Wherein, the process of retrieving multiple reference image samples based on the semantic guidance text and the emotion guidance text includes:

Perform web page retrieval based on the semantic guidance text and the emotion guidance text;

Construct an associated content set based on the first n retrieved web pages, the associated content set includes associated content corresponding to each of the web pages, the associated content includes the title text and content text of the web page, n is a positive integer;

Select the optimal related content from the related content set that has the strongest comprehensive correlation with the semantic guidance text and the emotional guidance text;

A plurality of reference image samples are retrieved based on the optimal associated content.

Wherein, the process of web page retrieval based on the semantic guidance text and the emotion guidance text includes:

Splice the semantic guidance text and the emotion guidance text to obtain the retrieval text;

The retrieval text is input into the search engine interface to perform web page retrieval on the retrieval text.

Among them, the process of constructing the associated content collection based on the first n retrieved web pages includes:

Extract the title text and content text of the first n web pages retrieved;

The title text and content text corresponding to each web page are stored in the local storage space in the form of a dictionary to obtain an associated content collection; the key of the dictionary is the title text, and the value of the dictionary is the content text.

The process of selecting the optimal associated content with the strongest comprehensive association with the semantic guidance text and the emotion guidance text from the associated content set includes:

For each of the content texts in the associated content set, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, a comprehensive score of the content text is obtained;

The related content including the content text with the highest comprehensive score is determined as the optimal related content with the strongest comprehensive correlation with the semantic guidance text and the emotion guidance text.

Wherein, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, the process of obtaining the comprehensive score of the content text includes:

Determining semantically related texts in the content text that match the semantic guidance text, and emotion-related texts in the semantically related texts that match the emotion guidance text;

Determining a semantic relevance score of the content text based on the semantically related text;

determining an emotion relevance score for the content text based on the emotion-related text;

A comprehensive score of the content text is determined using the semantic relevance score and the emotional relevance score.

Wherein, the process of determining the semantic relevance score of the content text based on the semantically relevant text includes:

Determine the ratio of the character length of the semantically related text to the character length of the content text as the semantic association score of the content text;

The process of determining the emotion relevance score of the content text based on the emotion-related text includes:

The ratio of the character length of the emotion-related text to the character length of the semantic-related text is determined as the emotion correlation score of the content text.

Wherein, the process of using the semantic association score and the emotional association score to determine the comprehensive score of the content text includes:

The product of the semantic association score and the emotional association score is used as the comprehensive score of the content text.

Wherein, the process of retrieving multiple reference image samples based on the optimal associated content includes:

Perform image retrieval based on the optimal associated content to obtain multiple candidate image samples;

Using the emotion guidance text and the semantic guidance text, a plurality of reference image samples are screened out from a plurality of candidate image samples.

Wherein, the process of using the emotional guidance text and the semantic guidance text to select multiple reference image samples from multiple candidate image samples includes:

Extract the image summary text of each candidate image sample;

Perform image element mutual exclusivity calculation on the input text and each image summary text to obtain a visual score of each image summary text, where the input text includes the semantic guidance text and the emotion guidance text;

Candidate image samples whose visual scores exceed a preset value are determined as reference image samples.

Wherein, the process of calculating the mutual exclusivity of image elements between the input text and each image summary text, and obtaining the visual score of each image summary text includes:

Extract the first entity element and the first entity relationship of each image summary text and the second entity element and the second entity relationship of the input text;

The image summary text in which the first entity element does not have an entity element different from the second entity element and the first entity relationship does not include an entity relationship different from the second entity relationship is determined as a candidate summary. text;

The consistency description score of each candidate summary text and the input text is calculated, and the consistency description score is used as the visual score of the candidate summary text.

The process of extracting features of multiple reference image samples and combining at least two of all features to obtain multiple image combination semantic features includes:

Extract features of a plurality of reference image samples;

Clustering all the features to obtain multiple first-level semantic features;

Construct an attention mask matrix according to the number of first-level semantic features,

The first-level semantic features and the attention mask matrix are used to obtain multiple image combination semantic features.

Wherein, the process of obtaining multiple image combined semantic features using the first-level semantic features and the attention mask matrix includes:

The first relational expression is used to obtain the semantic features of multiple image combinations, and the first relational expression is

;

Among them, transformer is a model based on the attention mechanism, g is the first-level semantic feature, softmax is the probability normalization function, W _q is the query parameter weight, W _k is the chain parameter weight, W _v is the value parameter weight, Mask [:,k] is the selection parameter of the k-th column of the attention mask matrix, size (g) is the dimension size of the first-level semantic feature, is the transposition symbol.

Among them, the process of clustering all the features to obtain multiple first-level semantic features includes:

Cluster all the features to obtain multiple first-level semantic features and second-level semantic features under each of the first-level semantic features;

The image generation method also includes:

Construct a semantic feature distribution forest. The semantic feature distribution forest includes multiple tree features. The trunk feature of each tree feature is the first-level semantic feature. The branch feature of each tree trunk feature is the first-level semantic feature. Second-level semantic features under features;

The process of generating an associated image based on the image combination semantic feature having the highest similarity to the text semantic feature comprises:

Determine optimal tree features based on the selection parameters of the attention mask matrix corresponding to the image combination semantic feature with the highest similarity to the text semantic feature;

Obtaining image screening features using the optimal tree features;

A related image is generated based on the image filtering features and the text semantic features.

Wherein, the process of generating associated images based on the image filtering features and the text semantic features includes:

Using the image screening features to obtain a conditional noise initial image;

A correlation image is generated based on the conditional noise initial image and the text semantic features.

Among them, the process of clustering all the features includes:

Calculate the Euclidean distance between any two of the features;

For each feature, determine the number by which the Euclidean distance is less than the first preset distance. When the number is not less than the preset number, divide the features into dense feature subsets. When the number is less than the preset distance, Assuming a quantity, divide the features into non-dense feature subsets;

Determine a subcategory, add any feature in the dense feature subset to the subcategory and remove it from the dense feature subset;

Calculate the minimum Euclidean distance between all features in the subcategory and all features in the dense feature subset, determine whether there is a first feature to be eliminated in the dense feature subset, and if so, remove the first feature to be eliminated Features are added to the subcategory and eliminated from the dense feature subset. Repeat this step until the first feature to be eliminated does not exist in the dense feature subset, and the first feature to be eliminated is the dense feature. Features whose minimum Euclidean distance between the subset and features in the subcategory is less than the second preset distance;

Calculate the minimum Euclidean distance between all features in the subcategory and all features in the non-dense feature subset, determine whether there is a second feature to be eliminated in the non-dense feature subset, and if so, remove the second feature The features to be eliminated are added to the subcategory and eliminated from the non-dense feature subset. This step is repeated until the second feature to be eliminated does not exist in the non-dense feature subset. The second feature to be eliminated is Features whose minimum Euclidean distance between the non-dense feature subset and the features in the subcategory is less than the second preset distance;

The subclass is added to a preset cluster set.

Among them, the process of obtaining multiple first-level semantic features includes:

Calculate the weighted sum of all the features included in the preset cluster set for all the subcategories in the preset cluster set according to the second relational expression, and obtain the first-level semantic feature based on the weighted sum;

The second relational expression is ;

Among them, t is the number of features in the b-th sub-category, f _b is the weighted sum of the b-th sub-category, f _t is the current feature in the b-th sub-category during the traversal of the b-th sub-category. Features, f _p is each feature in the traversal process, is the first preset distance or the second preset distance, dis( _ft , _fp ) is the Euclidean distance between _ft and _fp , Satisfies/> for the b-th subcategory the number of features.

In order to solve the above technical problems, the present invention also provides an image generation system, including:

The acquisition module is used to obtain semantic guidance text and emotional guidance text;

A retrieval module configured to retrieve multiple reference image samples based on the semantic guidance text and the emotion guidance text;

An extraction module, configured to extract features of a plurality of reference image samples, and combine at least two of all the features to obtain multiple image combination semantic features;

A generation module is used to obtain text semantic features corresponding to the semantic guidance text, and generate associated images based on the image with the highest similarity to the text semantic features and the semantic features.

In order to solve the above technical problems, the present invention also provides an electronic device, including:

Memory, used to store computer programs;

A processor is used to implement the steps of the image generation method as described in any one of the above when executing the computer program.

In order to solve the above technical problems, the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, any one of the above mentioned tasks can be implemented. Steps of the image generation method.

The present invention provides an image generation method that searches web pages based on semantic guidance text and emotion guidance text, and obtains multiple reference image samples corresponding to the emotion guidance text and semantic guidance text to facilitate subsequent generation of associations based on multiple reference images. Images, improve the accuracy of image generation, so that the generated associated images are highly related to the guidance text and emotional text, which not only meet the semantic text requirements of the task scene, but also meet the emotional needs of the task scene. The present invention also provides an image generation system, electronic equipment and a computer-readable storage medium, which have the same beneficial effects as the above image generation system.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flowchart of the steps of an image generation method provided by the present invention;

Figure 2 is a schematic diagram of associated content matching and screening provided by the present invention;

Figure 3 is a schematic diagram of semantic association enhancement provided by the present invention;

Figure 4 is a schematic diagram of a semantic feature distribution forest structure provided by an embodiment of the present invention;

Figure 5 is a schematic diagram of an attention mask matrix provided by an embodiment of the present invention;

FIG6 is a schematic diagram of generating an associated image provided by the present invention;

Figure 7 is a schematic structural diagram of an image generation system provided by the present invention;

Figure 8 is a schematic structural diagram of an electronic device provided by the present invention;

FIG. 9 is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention.

Detailed ways

The core of the present invention is to provide an image generation method, system, electronic device and readable storage medium, which can improve the accuracy of image generation, make the generated associated images highly relevant to the guidance text and emotional text, and meet the semantic text requirements of the task scene. while meeting the emotional needs of the task scenario.

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

In the first aspect, please refer to Figure 1, which is a step flow chart of an image generation method provided by the present invention. The image generation method includes:

S101: Obtain semantic guidance text and emotion guidance text;

In this embodiment, considering that in the field of image content generation, image content generation aims to generate accurate images according to text semantics, and the content of the image precisely conforms to the semantic guidance in the text, in certain task scenarios, the image generation task not only requires the content of the generated image to be consistent with the semantic guidance, but also needs to be matched with relevant emotions. For example, in the text illustration task scenario, it is necessary to generate images of specified emotions, or in the psychotherapy task scenario, it is necessary to generate images related to positive emotions as much as possible, and in the sales task scenario, it is necessary to generate images that make people excited and happy as much as possible. In order to make the generated image conform to the semantic guidance and be able to match the relevant emotions, in this embodiment, the semantic guidance text and the emotional guidance text input by the user through the interactive device are obtained. For example, the user's input text includes a puppy on the snow and sadness, the semantic guidance text obtained is a puppy on the snow, and the emotional guidance text obtained is sadness.

S102: Obtain multiple reference image samples based on semantic guidance text and emotion guidance text retrieval;

In this embodiment, the semantic guidance text and the emotion guidance text are input into the search engine for retrieval, and multiple pieces of associated content are obtained. From the multiple pieces of associated content, the optimal associated content with the highest degree of correlation with the semantic guidance text and the emotion guidance text is determined. The purpose of obtaining optimal correlation content in the embodiment is to facilitate subsequent screening of reference image samples.

After determining the optimal associated content, search for relevant images in the search engine based on the optimal associated content, and then filter all retrieved images based on the emotional guidance text and semantic guidance text to select multiple images related to the input text. The image is used as a reference image sample. The reference image sample here is a visually enhanced sample, and the associated image generated based on this can better meet the semantic and emotional needs.

S103: extracting features of multiple reference image samples, and combining at least two features among all the features to obtain multiple image combined semantic features;

In this embodiment, features of multiple reference image samples are extracted and combined in all forms to obtain image combination semantic features. Each image combination semantic feature includes at least two features.

S104: Acquire text semantic features corresponding to the semantic guidance text, and generate associated images based on image combination semantic features with the highest similarity to the text semantic features.

In this embodiment, the text semantic features corresponding to the semantic guidance text are first obtained, the optimal image combination semantic features are determined based on the similarity between the text semantic features and the image combination semantic features, and the image combination semantic features with the highest similarity are determined as the optimal Image combination semantic features generate related images based on optimal image combination semantic features. It can be understood that some specific elements in the reference image sample are referenced based on the similarity, so that the content in the generated associated image is more like the appearance of several referenced images, thereby improving the accuracy of generating the associated image.

It can be seen that in this embodiment, the web page is retrieved based on the semantic guidance text and the emotion guidance text, and multiple reference image samples corresponding to the emotion guidance text and the semantic guidance text are obtained, so as to facilitate subsequent generation of associated images based on the multiple reference images, and improve The accuracy of image generation makes the generated associated images highly relevant to the guidance text and emotional text, which not only meets the semantic text requirements of the task scene, but also meets the emotional needs of the task scene.

Based on the above embodiments:

In an exemplary embodiment, the process of retrieving multiple reference image samples based on semantic guidance text and emotion guidance text includes:

Web page retrieval based on semantic guidance text and emotion guidance text;

Construct an associated content collection based on the first n retrieved web pages. The associated content collection includes the associated content corresponding to each web page. The associated content includes the title text and content text of the web page. n is a positive integer;

Select the optimal related content from the related content collection that has the strongest comprehensive correlation with the semantic guidance text and the emotional guidance text;

Multiple reference image samples are obtained based on optimal associated content retrieval.

In an exemplary embodiment, the process of web page retrieval based on semantic guidance text and emotion guidance text includes:

The semantic guidance text and the emotional guidance text are concatenated to obtain the retrieval text;

Enter the search text into the search engine interface so that the search text can be retrieved from the web.

In an exemplary embodiment, the process of constructing a set of related content based on the first n retrieved web pages includes:

Extract the title text and content text of the first n web pages retrieved;

The title text and content text corresponding to each web page are stored in the local storage space in the form of a dictionary to obtain a collection of related content; the key of the dictionary is the title text, and the value of the dictionary is the content text.

In this embodiment, the emotion guidance text and the semantic guidance text are first spliced together to obtain a combined guidance text. This embodiment does not limit the order in which the emotion guidance text and the semantic guidance text are spliced. The emotion guidance text can be Spliced behind the semantic guidance text, the semantic guidance text can also be spliced behind the emotional guidance text.

After obtaining the merge guidance text, call the search engine interface to retrieve the merge guidance text, open the first n web pages retrieved, and download the associated content in the first n web pages to the local. The associated content includes title text and content text. On the one hand, the related content in the first n web pages has a strong correlation with the guidance text in this embodiment. On the other hand, the amount of data processing can be reduced, and the data processing efficiency can be improved while ensuring the correlation.

After storing the associated content of the first n web pages into the local storage space, construct an associated content collection. The associated content collection is in the form of a dictionary. The title text and content text of each web page are stored in the dictionary, and the key is the retrieved web page. Title text, the content text in the web page.

Of course, in addition to selecting the dictionary type, you can also select other methods to facilitate retrieval. This embodiment is not specifically limited here.

In an exemplary embodiment, the process of selecting the optimal related content with the strongest comprehensive correlation with the semantic guidance text and the emotion guidance text in the related content collection includes:

For each content text in the associated content collection, a comprehensive score of the content text is obtained based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text;

The associated content including the content text with the highest comprehensive score is determined as the optimal associated content with the strongest comprehensive association with the semantic guidance text and the emotion guidance text.

In an exemplary embodiment, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, the process of obtaining the comprehensive score of the content text includes:

Determine the semantically relevant text in the content text that matches the semantic guidance text, and the emotion-related text in the semantically relevant text that matches the emotion guidance text;

determining a semantic relevance score of the content text based on semantically related text;

Determine the emotion relevance score of the content text based on the emotion-related text;

The semantic relevance score and sentiment relevance score are used to determine the comprehensive score of the content text.

In an exemplary embodiment, the process of determining the semantic relevance score of the content text based on the semantically relevant text includes:

The ratio of the character length of the semantically related text to the character length of the content text is determined as the semantic association score of the content text;

The process of determining the emotion relevance score of content text based on emotion-relevant text includes:

The ratio of the character length of the emotion-related text to the character length of the semantically-related text is determined as the emotion relevance score of the content text.

In this embodiment, as shown in Figure 2, each content text in the related content collection is traversed, and the content text is scored for semantic relevance. Specifically, you can choose to use large models such as T5, M6, ChatGPT, etc. for semantic relevance scoring. Obtain the semantic correlation score. Specifically, each text in the content text and the semantic guidance text are spliced and input into the above large model to determine whether they match. All matching texts are output and recorded as semantically relevant texts. The ratio of the character length of the text to the character length of the associated text content is output, recorded as the semantic association score, which represents the proportion of text that is semantically consistent with the input text. At the same time, the content text is scored for emotional relevance. Specifically, you can choose to use large models such as T5, M6, ChatGPT, etc. to score the emotional relevance of the semantically related text to obtain the emotional relevance score. Specifically, each text in the semantically related text is After splicing it with the emotion guidance text, it is input into the above large model to determine whether the two match. If they match, the matching sentence will be determined as the emotion-related text, and the ratio of the character length of the emotion-related text to the character length of the semantic-related text will be output. Record is the emotional correlation score. A comprehensive score is obtained based on the semantic correlation score and the emotional correlation score. The optimal correlation content is determined based on the comprehensive score. Specifically, the content text with the highest comprehensive score is spliced with the title text corresponding to the content text to obtain semantic guidance. Text and emotion guide the optimal associated content corresponding to the text.

In an exemplary embodiment, the process of determining the comprehensive score of the content text using the semantic association score and the emotional association score includes:

The product of the semantic relevance score and the emotional relevance score is used as the comprehensive score of the content text.

In this embodiment, the semantic association score and the sentiment association score may be multiplied to obtain a comprehensive association score. As another optional embodiment, the semantic association score and the sentiment association score may be added to obtain a comprehensive association score, which may be selected according to actual engineering needs, and is not specifically limited in this embodiment.

In an exemplary embodiment, the process of retrieving multiple reference image samples based on optimal associated content includes:

Perform image retrieval based on optimal associated content to obtain multiple candidate image samples;

Emotional guidance text and semantic guidance text are used to screen out multiple reference image samples from multiple candidate image samples.

In an exemplary embodiment, the process of selecting a plurality of reference image samples from a plurality of candidate image samples using the emotion guidance text and the semantic guidance text includes:

Extract the image summary text of each candidate image sample;

Calculate the mutual exclusivity of image elements between the input text and each image summary text to obtain the visual score of each image summary text. The input text includes semantic guidance text and emotion guidance text;

Candidate image samples whose visual scores exceed a preset value are determined as reference image samples.

In an exemplary embodiment, image element mutual exclusivity calculation is performed on the input text and each image summary text, and the process of obtaining the visual score of each image summary text includes:

Extracting a first entity element and a first entity relationship of each image summary text and a second entity element and a second entity relationship of the input text;

Determine the image summary text in which no entity element different from the second entity element exists in the first entity elements and no entity relationship different from the second entity relationship exists in the first entity relationship as a candidate summary text;

The consistency description score between each candidate summary text and the input text is calculated, and the consistency description score is used as the visual score of the candidate summary text.

In this embodiment, as shown in FIG. 3 , the optimal associated content is first extracted using a first preset model to extract the associated summary, and the associated summary is output. The first preset model can use a model such as ChatGPT, with the purpose of reducing it to More refined language text makes it easier to call the search engine for retrieval. Based on the association summary, the search engine is called for image retrieval, and multiple candidate image samples are obtained. A candidate associated image set is established based on multiple candidate image samples. The second method is used for the candidate associated image set. The preset model performs image summary text extraction to obtain the image summary text of each candidate image sample in the candidate associated image set, build an image summary set based on the image summary text of each candidate associated image, traverse the image summary set, and based on the input text The mutual exclusivity of image elements is calculated for each image summary text, and a reference image sample is obtained based on the calculation results.

Image element mutual exclusivity calculation includes extracting entity elements and entity relationships of input text and image summary text. Entity elements include but are not limited to categories and attributes, such as "car", "red", etc., and entity relationships such as "people 'feed' dogs ", record the entity elements and entity relationships in the image summary text as Az, record the entity elements and entity relationships in the input text as Ain, compare the inclusion relationship between Az and Ain, if Az contains other content other than Ain, it will not If it is qualified, it needs to be deleted, otherwise it will be retained. The retained image summary text will be determined as a candidate summary text. The CIDEr (Consensus-based Image Description Evaluation, consensus-based image description score) score of each candidate summary text and the input text will be calculated, that is, they are consistent. Sexual description score, also known as visual score. All candidate summary texts are sorted according to the consistency description score, and candidate image samples corresponding to the retained candidate summary texts are collected as visual enhancement samples, that is, reference image samples in this embodiment.

In this embodiment, the candidate image samples whose visual scores exceed the preset score can be determined as reference image samples, or the candidate image samples corresponding to the top m candidate summary texts with higher scores sorted according to the visual scores can be used as reference images. Sample, m is a positive integer.

In an exemplary embodiment, the process of extracting features of multiple reference image samples and combining at least two features among all features to obtain multiple image combination semantic features includes:

Extract features from multiple reference image samples;

All features are clustered to obtain multiple first-level semantic features;

Construct an attention mask matrix based on the number of first-level semantic features,

The first-level semantic features and attention mask matrix are used to obtain multiple image combined semantic features.

This embodiment is aimed at given several reference image samples that can be referred to, and realizes the reference to certain specific elements in these reference image samples, for example, making the entity elements in the generated associated images closer to the reference images. Solid elements in image samples. Based on this, this embodiment first provides a special representation for characterizing the features of multiple images, that is, constructing a semantic feature distribution forest structure of the image. The semantic feature distribution forest structure of the image is composed of several tree features. Each tree The tree feature consists of two levels of nodes, each node represents a feature (such as a vector of size [1, d], d is the dimension).

Secondly, for the generated associated images, considering that not every reference image sample is helpful, and for helpful reference image samples, not all features are helpful, this embodiment provides a feature screening mechanism based on an attention mask matrix. The attention mask matrix is used to enumerate all possible semantic features of image combinations, so as to select the best features as reference features for generating associated images.

The following describes the semantic feature distribution forest structure and feature screening mechanism for constructing an image.

This embodiment first performs feature extraction on multiple reference image samples. Specifically, multiple reference image samples are input into the image encoder for feature extraction, and a feature set of size [N, j, d] is obtained, which is recorded as visual Enhanced sample feature set, N is the number of reference image samples, j is the number of features extracted from each reference image sample, and d represents the dimension of each feature.

Use a clustering algorithm to cluster each feature in the visual reinforcement sample set. The process of clustering all features includes:

Calculate the Euclidean distance between any two features;

For each feature, determine the number by which the Euclidean distance is less than the first preset distance. When the number is not less than the preset number, the features are divided into dense feature subsets. When the number is less than the preset number, the features are divided into non-dense feature subsets. ;

Determine a subclass, add any feature in the dense feature subset to the subclass and remove it from the dense feature subset;

Calculate the minimum Euclidean distance between all features in the subclass and all features in the dense feature subset, and determine whether there is the first feature to be eliminated in the dense feature subset. If so, add the first feature to be eliminated to the subclass and remove it from the dense feature subset. Eliminate from the feature subset, repeat this step until there is no first feature to be eliminated in the dense feature subset, and the first feature to be eliminated is that the minimum Euclidean distance between the features in the dense feature subset and the subclass is less than the second preset distance Characteristics;

Calculate the minimum Euclidean distance between all features in the subclass and all features in the non-dense feature subset, determine whether there is a second feature to be eliminated in the non-dense feature subset, and if so, add the second feature to be eliminated to the subclass and Eliminate from the non-dense feature subset, and repeat this step until there is no second feature to be eliminated in the non-dense feature subset, and the second feature to be eliminated is that the minimum Euclidean distance between the features in the non-dense feature subset and the subclass is less than Characteristics of the second preset distance;

Add subcategories to the preset cluster set.

The visual enhancement sample feature set is converted into the size of [N×j, d], where L=N×j is the total number of features. The Euclidean distance between every two features is calculated to obtain the distance matrix of [L, L].

A preset number and a preset distance are obtained, where the preset distance includes a first preset distance and a second preset distance, and the first preset distance and the second preset distance may be the same. In this embodiment, the first preset distance and the preset number are used to construct a dense feature subset and a non-dense feature subset.

For each feature in the feature set, calculate the Euclidean distance between the feature and other features in the feature set, determine the number by which all calculated Euclidean distances are less than the first preset distance, and determine whether the number is less than the preset number. If not, quantity, divide the feature into a dense feature subset, let the size of the dense feature subset be [M, d], M≤L, and divide the features that are not divided into the dense feature subset into a non-dense feature subset, let The size of the non-dense feature subset is [L-M, d].

Construct a preset clustering set C (empty set). When the dense feature subset is not empty, traverse each feature f in the dense feature subset. It is assumed that the dense feature subset includes f1, f2, f3, f4 and f5. Initialize a new subclass C _b {f}. Assume that the currently traversed feature is f1, then delete f1 from the dense feature subset, and divide f1 into C _b . For the features in the current dense feature subset, that is, f2, f3, f4, f5, respectively calculate the Euclidean distance between f2 and f1, the Euclidean distance between f3 and f1, the Euclidean distance between f4 and f1, the Euclidean distance between f5 and f1, if there are only f2 and f1 If the Euclidean distance between them is less than the second preset distance, then f2 is divided into C _b , and f2 is deleted from the current dense feature subset, and then all features in the current dense feature subset are calculated with all features in C _b The Euclidean distance is for all features f1 and f2 in f3, f4, f5 and C _b in the current dense feature subset. Since the Euclidean distance between f3, f4, f5 and f1 has been calculated last time, here No more calculations, and then calculate the Euclidean distance between f3 and f2, the Euclidean distance between f4 and f2, and the Euclidean distance between f5 and f2, and determine whether there is a Euclidean distance in the current dense feature subset that is smaller than the second preset distance. characteristics, assuming that the Euclidean distance between f4 and f2 is less than the second preset distance, then divide f4 into C _b , and delete f4 from the dense feature subset, and so on, repeat the above process until the dense feature subset There is no more feature in the set that can be incorporated into C _b . In the same way, traverse all the features in the non-dense feature subset, select the features in the non-dense feature subset whose Euclidean distance is less than the second preset distance from the features in C _b and add them to C _b in the above manner, and add them to C b Eliminate the non-dense feature subset until no feature in the non-dense feature subset can be incorporated into C _b . Add C _b to the preset clustering set and output C = {C ₁ , C ₂ ,..., C _s }, C _b is any one of C ₁ to C _s .

In an exemplary embodiment, the process of obtaining multiple first-level semantic features includes:

Calculating the weighted sum of all features included in all subclasses in the preset clustering set according to the second relational expression, and obtaining the primary semantic feature based on the weighted sum;

The second relational expression is ;

Among them, t is the number of features in the b-th sub-category, f _b is the weighted sum of the b-th sub-category, f _t is the current feature in the b-th sub-category during the traversal of the b-th sub-category, f _p is the traversal Every feature in the process, is the first preset distance or the second preset distance, dis(f _t , f _p ) is the Euclidean distance between f _t and f _p ,/> Satisfies/> for the b-th subcategory the number of features.

In this embodiment, for each subcategory C, the weighted sum of all features included in it is calculated according to the second relational expression, where t represents the number of features in subcategory C _b .

In an exemplary embodiment, the process of clustering all features to obtain multiple first-level semantic features includes:

All features are clustered to obtain multiple first-level semantic features and second-level semantic features under each first-level semantic feature;

Image generation methods also include:

Construct a semantic feature distribution forest. The semantic feature distribution forest includes multiple tree features. The trunk features of each tree feature are first-level semantic features, and the branch features of each trunk feature are second-level semantic features under the first-level semantic features;

The process of generating associated images based on the image with the highest similarity to the semantic features of the text combined with semantic features includes:

Determine the optimal tree feature based on the selection parameters of the attention mask matrix corresponding to the image combination semantic feature with the highest similarity to the text semantic feature;

The image screening features are obtained using the optimal tree features;

Generate associated images based on image screening features and text semantic features.

In this embodiment, a semantic feature distribution forest is constructed, in which the number of tree features is the number of all subclasses in the preset cluster set, and the trunk features of the tree features are first-level semantic features, specifically all f _b outputs , the branch features of each trunk feature are all the features in its corresponding subclass, that is, the features stored in each C _b . For example, assuming that the preset clustering set C={C ₁ , C ₂ , C ₃ , C ₄ , C ₅ } obtained after clustering the features of multiple reference image samples, it can be understood that as shown in Figure 4 The semantic feature distribution forest structure shown is an example. Figure 4 includes five tree features. The first-level semantic feature of the first tree feature is the weighted sum f _b1 of all features in C _1. The second-level semantic feature of the first tree feature is The features are all the features in C ₁ (f _c1 ). The first-level semantic features of the second tree feature are the weighted sum f _b2 of all the features in C ₂ . The second-level semantic features of the second tree feature are all the features in C ₂ features (f _c2 ), the first-level semantic feature of the third tree feature is the weighted sum of all features in C ₃ f _b3 , and the second-level semantic feature of the third tree feature is all the features in C ₃ (f _c3 ) , the first-level semantic feature of the fourth tree feature is the weighted sum f _b4 of all features in C ₄ , the second-level semantic feature of the fourth tree feature is all the features in C ₄ (f _c4 ), and the fifth tree feature The first-level semantic feature is the weighted sum of all features in C ₅ f _b5 , and the second-level semantic feature of the fifth tree feature is all the features in C ₅ (f _c5 ).

Construct an attention mask set and initialize it according to the number of tree features in the semantic feature distribution forest:

; Among them, y is the number of tree features. Combined with the structure of the semantic feature distribution forest shown in Figure 4, the attention mask matrix shown in Figure 5 can be obtained. Each column in the attention mask matrix represents a Feasible feature combinations, 1 indicates that the feature should be selected. For r=2, y=5, there are 10 feature combinations. See the first 10 columns in Figure 5. When r=3, y=5, there are 10 combinations. Feature combinations, see columns 11 to 20 in Figure 5, when r=4, y=5, there are 5 feature combinations, see columns 21 to 25 in Figure 5, when r=5, y=5 , there is 1 feature combination, see column 26 in Figure 5. Use model structures such as transformer to extract features of each first-level semantic feature (denoted as g), and use each column of attention masks in the attention mask set to calculate according to the first relationship to obtain the corresponding first semantic features. The corresponding multiple image combination semantic features are of size [b, d], and the first relational expression is

;

Among them, transformer is a model based on the attention mechanism, g is the first-level semantic feature, softmax is the probability normalization function, _Wq is the query parameter weight, _Wk is the chain parameter weight, _Wv is the value parameter weight, Mask[:,k] is the selection parameter of the kth column of the attention mask matrix, size(g) is the dimension size of the first-level semantic feature, is the transposed symbol. Use the text encoder to encode the input guidance text to obtain the text semantic features, the size is/> .

Calculate the triangular similarity between the text semantic features and each image combination semantic feature, determine the image combination semantic feature with the highest triangular similarity, and obtain the optimal combination based on the position of the image combination semantic feature in the attention mask matrix. Among them, the optimal combination is used to describe which combination of classes after clustering can most accurately express similar semantics in the text. Assume that the position of the semantic feature of the image combination in the attention mask matrix is the 13th column, then the optimal combination is 1, 2, 5, that is, the optimal combination is the first tree feature, the second tree feature and the fifth tree feature in Figure 4. Randomly select a branch feature for the first tree feature, and select a branch feature for the second tree feature. The tree feature randomly selects a branch feature, and randomly selects a branch feature for the fifth tree feature to obtain an image screening feature with a size of [z, d]. In this embodiment, z=3.

In an exemplary embodiment, the process of generating associated images based on image filtering features and text semantic features includes:

Use image filtering features to obtain conditional noise initial images;

Generate associated images based on conditional noise initial images and text semantic features.

In this embodiment, the image filtering features are copied and the dimensions are transformed into the size of [h, w, z × d], where the first two dimensions represent the height and width of the image, which are recorded as the conditional noise initial image, and a diffusion generation model is used , input the conditional noise initial image and text semantic features together, and output the finally generated associated image.

In summary, the associated image generation scheme is shown in Figure 6, which includes inputting the semantic guidance text (size [1, l]) into the text encoder to obtain text semantic features (size [1, l, d]), and converting each A reference image sample (size [N, h, w]) is input to the image encoder to extract the features of the reference image sample, and a feature set (size [N, j, d]) is constructed based on the features of all reference image samples. The concentrated features are used for image semantic clustering, a semantic feature distribution forest is constructed based on the clustering results, and the attention mask matrix is initialized based on the number of tree features in the semantic feature distribution forest, and the semantic feature distribution forest and text semantics are integrated as a whole and The attention mask matrix performs semantic similarity calculation to obtain image screening features. Based on the image screening features, a conditional noise image is generated. The conditional noise image and text semantic features are input into the diffusion model generator to generate a related image.

In the second aspect, please refer to Figure 7, which is a schematic structural diagram of an image generation system provided by the present invention, including:

Acquisition module 11, used to obtain semantic guidance text and emotion guidance text;

A retrieval module 12, configured to retrieve a plurality of reference image samples based on the semantic guidance text and the emotional guidance text;

The extraction module 13 is used to extract features of multiple reference image samples, and combine at least two features among all features to obtain multiple image combination semantic features;

The generation module 14 is used to obtain text semantic features corresponding to the semantic guidance text, and generate associated images based on the image with the highest similarity to the text semantic features by combining the semantic features.

In an exemplary embodiment, the process of retrieving a plurality of reference image samples based on the semantic guidance text and the emotional guidance text includes:

Web page retrieval based on semantic guidance text and emotion guidance text;

Construct an associated content collection based on the first n retrieved web pages. The associated content collection includes the associated content corresponding to each web page. The associated content includes the title text and content text of the web page. n is a positive integer;

Select the optimal related content from the related content collection that has the strongest comprehensive correlation with the semantic guidance text and the emotional guidance text;

Multiple reference image samples are obtained based on optimal associated content retrieval.

In an exemplary embodiment, the process of web page retrieval based on semantic guidance text and emotion guidance text includes:

Splice the semantic guidance text and the emotion guidance text to obtain the retrieval text;

Enter the search text into the search engine interface so that the search text can be retrieved from the web.

In an exemplary embodiment, the process of constructing a related content collection based on the first n retrieved web pages includes:

Extract the title text and content text of the first n web pages retrieved;

Store the title text and content text corresponding to each web page in the form of a dictionary in the local storage space to obtain an associated content collection; the key of the dictionary is the title text, and the value of the dictionary is the content text.

In an exemplary embodiment, the process of selecting the optimal related content with the strongest comprehensive correlation with the semantic guidance text and the emotion guidance text in the related content collection includes:

For each content text in the associated content collection, a comprehensive score of the content text is obtained based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text;

The associated content including the content text with the highest comprehensive score is determined as the optimal associated content with the strongest comprehensive association with the semantic guidance text and the emotion guidance text.

In an exemplary embodiment, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, the process of obtaining the comprehensive score of the content text includes:

determining semantically relevant texts that match the semantically guiding texts in the content texts, and emotionally relevant texts that match the emotionally guiding texts in the semantically relevant texts;

Determine the semantic relevance score of the content text based on the semantically related text;

Determine the emotion relevance score of the content text based on the emotion-related text;

The semantic relevance score and the emotional relevance score are used to determine the overall score of the content text.

In an exemplary embodiment, the process of determining the semantic relevance score of the content text based on the semantically relevant text includes:

The ratio of the character length of the semantically related text to the character length of the content text is determined as the semantic association score of the content text;

The process of determining the emotion relevance score of content text based on emotion-relevant text includes:

The ratio of the character length of the emotion-related text to the character length of the semantically-related text is determined as the emotion relevance score of the content text.

In an exemplary embodiment, the process of determining the comprehensive score of the content text using the semantic association score and the emotional association score includes:

The product of the semantic relevance score and the emotional relevance score is used as the comprehensive score of the content text.

In an exemplary embodiment, the process of retrieving multiple reference image samples based on optimal associated content includes:

Perform image retrieval based on optimal associated content to obtain multiple candidate image samples;

Emotional guidance text and semantic guidance text are used to filter out multiple reference image samples from multiple candidate image samples.

In an exemplary embodiment, the process of filtering out multiple reference image samples from multiple candidate image samples using emotional guidance text and semantic guidance text includes:

Extract the image summary text of each candidate image sample;

Calculate the mutual exclusivity of image elements for the input text and each image summary text to obtain a visual score for each image summary text, where the input text includes a semantic guidance text and an emotional guidance text;

Candidate image samples whose visual scores exceed a preset value are determined as reference image samples.

In an exemplary embodiment, image element mutual exclusivity calculation is performed on the input text and each image summary text, and the process of obtaining the visual score of each image summary text includes:

Extract the first entity element and first entity relationship of each image summary text and the second entity element and second entity relationship of the input text;

Determine the image summary text in which the first entity element does not have an entity element that is different from the second entity element and the first entity relationship does not include an entity relationship that is different from the second entity relationship as a candidate summary text;

Calculate the consistency description score of each candidate summary text and the input text, and use the consistency description score as the visual score of the candidate summary text.

In an exemplary embodiment, the process of extracting features of multiple reference image samples and combining at least two features among all features to obtain multiple image combination semantic features includes:

Extract features from multiple reference image samples;

Cluster all features to obtain multiple first-level semantic features;

Construct an attention mask matrix based on the number of first-level semantic features,

The first-level semantic features and attention mask matrix are used to obtain multiple image combined semantic features.

In an exemplary embodiment, the process of obtaining a plurality of image combination semantic features using the primary semantic features and the attention mask matrix includes:

The first relational expression is used to obtain the semantic features of multiple image combinations. The first relational expression is

;

Among them, transformer is a model based on the attention mechanism, g is the first-level semantic feature, softmax is the probability normalization function, W _q is the query parameter weight, W _k is the chain parameter weight, W _v is the value parameter weight, Mask [:,k] is the selection parameter of the k-th column of the attention mask matrix, size (g) is the dimension size of the first-level semantic feature, is the transpose symbol.

In an exemplary embodiment, the process of clustering all features to obtain multiple first-level semantic features includes:

All features are clustered to obtain multiple first-level semantic features and second-level semantic features under each first-level semantic feature;

The image generation system also includes:

Building module, used to build a semantic feature distribution forest. The semantic feature distribution forest includes multiple tree features. The trunk features of each tree feature are first-level semantic features, and the branch features of each trunk feature are second-level semantic features under the first-level semantic features. Semantic features;

The process of generating associated images based on the semantic features of the image with the highest similarity to the text semantic features includes:

Determine the optimal tree feature based on the selection parameters of the attention mask matrix corresponding to the image combination semantic feature with the highest similarity to the text semantic feature;

Use optimal tree features to obtain image screening features;

Generate associated images based on image filtering features and text semantic features.

In an exemplary embodiment, the process of generating an associated image based on the image screening feature and the text semantic feature includes:

Use image filtering features to obtain conditional noise initial images;

Generate associated images based on conditional noise initial images and text semantic features.

In an exemplary embodiment, the process of clustering all features includes:

Calculate the Euclidean distance between any two features;

For each feature, determine the number by which the Euclidean distance is less than the first preset distance. When the number is not less than the preset number, the features are divided into dense feature subsets. When the number is less than the preset number, the features are divided into non-dense feature subsets. ;

Determine a subclass, add any feature in the dense feature subset to the subclass and remove it from the dense feature subset;

Calculate the minimum Euclidean distance between all features in the subclass and all features in the dense feature subset, and determine whether there is the first feature to be eliminated in the dense feature subset. If so, add the first feature to be eliminated to the subclass and remove it from the dense feature subset. Eliminate from the feature subset, repeat this step until there is no first feature to be eliminated in the dense feature subset, and the first feature to be eliminated is that the minimum Euclidean distance between the features in the dense feature subset and the subclass is less than the second preset distance Characteristics;

Calculate the minimum Euclidean distance between all features in the subclass and all features in the non-dense feature subset, determine whether there is a second feature to be eliminated in the non-dense feature subset, and if so, add the second feature to be eliminated to the subclass and Eliminate from the non-dense feature subset, and repeat this step until there is no second feature to be eliminated in the non-dense feature subset, and the second feature to be eliminated is that the minimum Euclidean distance between the features in the non-dense feature subset and the subclass is less than Characteristics of the second preset distance;

Add subcategories to the preset cluster set.

In an exemplary embodiment, the process of obtaining a plurality of primary semantic features includes:

Calculate the weighted sum of all features included in all subcategories in the preset cluster set according to the second relational expression, and obtain the first-level semantic features based on the weighted sum;

The second relational expression is ;

Among them, t is the number of features in the b-th sub-category, f _b is the weighted sum of the b-th sub-category, f _t is the current feature in the b-th sub-category during the traversal of the b-th sub-category. Features, f _p is each feature in the traversal process, is the first preset distance or the second preset distance, dis( _ft , _fp ) is the Euclidean distance between _ft and _fp , Satisfies/> for the b-th subcategory the number of features.

In a third aspect, refer to FIG. 8 , which is a schematic structural diagram of an electronic device provided by the present invention. The electronic device includes:

A memory 21, used for storing computer programs;

The processor 22 is configured to implement the steps of the image generation method described in any of the above embodiments when executing a computer program.

The electronic device also includes:

The input interface 23 is connected to the processor 22 via the communication bus 26, and is used to obtain the computer programs, parameters and instructions imported from the outside, and save them in the memory 21 under the control of the processor 22. The input interface can be connected to an input device to receive parameters or instructions manually input by the user. The input device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the terminal housing.

The display unit 24 is connected to the processor 22 via the communication bus 26 and is used to display data sent by the processor 22. The display unit may be a liquid crystal display or an electronic ink display.

The network port 25 is connected to the processor 22 via the communication bus 26, and is used to communicate with various external terminal devices. The communication technology used in the communication connection can be a wired communication technology or a wireless communication technology, such as mobile high-definition link technology, universal serial bus, high-definition multimedia interface, wireless fidelity technology, Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

In the fourth aspect, please refer to Figure 9. Figure 9 is a schematic structural diagram of a computer-readable storage medium provided by the present invention. A computer program 31 is stored on the computer-readable storage medium 30. The computer program 31 is implemented when executed by a processor. The steps of the image generation method described in any of the above embodiments.

The computer-readable storage medium 30 may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. medium.

It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or sequence between operations. Furthermore, the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

The above description of the disclosed embodiments enables those skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (20)

1. An image generation method, characterized by comprising: Get semantic guidance text and emotion guidance text; Retrieve multiple reference image samples based on the semantic guidance text and the emotion guidance text; Extract features of a plurality of reference image samples, and combine at least two of all the features to obtain multiple image combination semantic features; Text semantic features corresponding to the semantic guidance text are obtained, and semantic features are combined based on the image with the highest similarity to the text semantic features to generate a related image. 2. The image generation method according to claim 1, wherein the process of retrieving multiple reference image samples based on the semantic guidance text and the emotion guidance text includes: Perform web page retrieval based on the semantic guidance text and the emotion guidance text; Construct an associated content set based on the first n retrieved web pages, the associated content set includes associated content corresponding to each of the web pages, the associated content includes the title text and content text of the web page, n is a positive integer; Select the optimal related content from the related content set that has the strongest comprehensive correlation with the semantic guidance text and the emotional guidance text; A plurality of reference image samples are retrieved based on the optimal associated content. 3. The image generation method according to claim 2, wherein the process of web page retrieval based on the semantic guidance text and the emotion guidance text includes: Splice the semantic guidance text and the emotion guidance text to obtain the retrieval text; The retrieval text is input into the search engine interface to perform web page retrieval on the retrieval text. 4. The image generation method according to claim 2, wherein the process of constructing the associated content collection based on the retrieved first n web pages includes: Extract the title text and content text of the first n web pages retrieved; The title text and content text corresponding to each web page are stored in the local storage space in the form of a dictionary to obtain an associated content collection; the key of the dictionary is the title text, and the value of the dictionary is the content text. 5. The image generation method according to claim 2, characterized by the process of selecting the optimal associated content with the strongest comprehensive correlation with the semantic guidance text and the emotion guidance text in the associated content set. include: For each content text in the associated content set, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, the obtained content text is obtained. The comprehensive score of the content text; The related content including the content text with the highest comprehensive score is determined as the optimal related content with the strongest comprehensive correlation with the semantic guidance text and the emotion guidance text. 6. The image generation method according to claim 5, characterized in that, based on the semantic association score corresponding to the content text and the semantic guidance text and the emotion association score corresponding to the content text and the emotion guidance text, The process of obtaining a comprehensive score for the content text includes: Determining semantically related texts in the content text that match the semantic guidance text, and emotion-related texts in the semantically related texts that match the emotion guidance text; determining a semantic relevance score of the content text based on the semantically relevant text; Determining a sentiment relevance score of the content text based on the sentiment-related text; A comprehensive score of the content text is determined using the semantic relevance score and the emotional relevance score. 7. The image generation method according to claim 6, wherein the process of determining the semantic association score of the content text based on the semantically related text includes: Determine the ratio of the character length of the semantically related text to the character length of the content text as the semantic association score of the content text; The process of determining the emotion correlation score of the content text based on the emotion-related text includes: The ratio of the character length of the emotion-related text to the character length of the semantic-related text is determined as the emotion correlation score of the content text. 8. The image generation method according to claim 6, wherein the process of using the semantic association score and the emotional association score to determine the comprehensive score of the content text includes: The product of the semantic association score and the emotion association score is used as the comprehensive score of the content text. 9. The image generation method according to claim 2, wherein the process of retrieving multiple reference image samples based on the optimal associated content includes: Perform image retrieval based on the optimal associated content to obtain multiple candidate image samples; A plurality of reference image samples are screened out from the plurality of candidate image samples using the emotion guidance text and the semantic guidance text. 10. The image generation method according to claim 9, characterized in that the process of using the emotional guidance text and the semantic guidance text to select multiple reference image samples from a plurality of candidate image samples includes: Extracting image summary text for each candidate image sample; Performing image element mutual exclusivity calculation on an input text and each of the image summary texts to obtain a visual score of each of the image summary texts, wherein the input text includes the semantic guidance text and the emotion guidance text; Candidate image samples whose visual scores exceed a preset value are determined as reference image samples. 11. The image generation method according to claim 10, characterized in that the process of performing image element mutual exclusivity calculation on the input text and each image summary text, and obtaining the visual score of each image summary text includes: : Extract the first entity element and the first entity relationship of each image summary text and the second entity element and the second entity relationship of the input text; The image summary text in which the first entity element does not have an entity element different from the second entity element and the first entity relationship does not include an entity relationship different from the second entity relationship is determined as a candidate summary. text; The consistency description score of each candidate summary text and the input text is calculated, and the consistency description score is used as the visual score of the candidate summary text. 12. The image generation method according to any one of claims 1 to 11, characterized in that features of a plurality of reference image samples are extracted, and at least two of all the features are combined to obtain a plurality of features. The process of combining semantic features of an image includes: Extract features of a plurality of reference image samples; Cluster all the features to obtain multiple first-level semantic features; Construct an attention mask matrix according to the number of first-level semantic features, Multiple image combined semantic features are obtained using the first-level semantic features and the attention mask matrix. 13. The image generation method according to claim 12, characterized in that the process of obtaining a plurality of image combination semantic features using the primary semantic features and the attention mask matrix comprises: The first relational expression is used to obtain the semantic features of multiple image combinations, and the first relational expression is ; Among them, transformer is a model based on the attention mechanism, g is the first-level semantic feature, softmax is the probability normalization function, W _q is the query parameter weight, W _k is the chain parameter weight, W _v is the value parameter weight, Mask [:,k] is the selection parameter of the k-th column of the attention mask matrix, size (g) is the dimension size of the first-level semantic feature, is the transposition symbol. 14. The image generation method according to claim 13, characterized in that the process of clustering all the features to obtain multiple first-level semantic features includes: Clustering all the features to obtain a plurality of first-level semantic features and a second-level semantic feature under each first-level semantic feature; The image generation method also includes: Constructing a semantic feature distribution forest, wherein the semantic feature distribution forest includes a plurality of tree features, wherein a trunk feature of each tree feature is the primary semantic feature, and a branch feature of each trunk feature is a secondary semantic feature under the primary semantic feature; The process of generating associated images based on the image with the highest similarity to the semantic features of the text combined with semantic features includes: Determine optimal tree features based on the selection parameters of the attention mask matrix corresponding to the image combination semantic feature with the highest similarity to the text semantic feature; Using the optimal tree features to obtain image screening features; A related image is generated based on the image filtering features and the text semantic features. 15. The image generation method according to claim 14, characterized in that the process of generating the associated image based on the image screening feature and the text semantic feature comprises: Using the image screening features to obtain a conditional noise initial image; A correlation image is generated based on the conditional noise initial image and the text semantic features. 16. The image generation method according to claim 14, wherein the process of clustering all the features comprises: Calculate the Euclidean distance between any two of the features; For each feature, determine the number by which the Euclidean distance is less than the first preset distance. When the number is not less than the preset number, divide the features into dense feature subsets. When the number is less than the preset distance, Assuming a quantity, divide the features into non-dense feature subsets; Determine a subclass, add any feature in the dense feature subset to the subclass and remove it from the dense feature subset; Calculate the minimum Euclidean distance between all features in the subcategory and all features in the dense feature subset, determine whether there is a first feature to be eliminated in the dense feature subset, and if so, remove the first feature to be eliminated Features are added to the subcategory and eliminated from the dense feature subset. Repeat this step until the first feature to be eliminated does not exist in the dense feature subset, and the first feature to be eliminated is the dense feature. Features whose minimum Euclidean distance between the subset and features in the subcategory is less than the second preset distance; Calculate the minimum Euclidean distance between all features in the subclass and all features in the non-dense feature subset, determine whether there is a second feature to be eliminated in the non-dense feature subset, and if so, add the second feature to be eliminated to the subclass and eliminate it from the non-dense feature subset, repeat this step until the second feature to be eliminated does not exist in the non-dense feature subset, and the second feature to be eliminated is a feature in the non-dense feature subset whose minimum Euclidean distance with the feature in the subclass is less than the second preset distance; Add the subcategories to the preset cluster set. 17. The image generation method according to claim 16, characterized in that the process of obtaining multiple first-level semantic features includes: Calculate the weighted sum of all the features included in the preset cluster set for all the subcategories in the preset cluster set according to the second relational expression, and obtain the first-level semantic feature based on the weighted sum; The second relation is ; Wherein, t is the number of features in the b-th subclass, _fb is the weighted sum of the b-th subclass, _ft is the current feature in the b-th subclass during the traversal of the b-th subclass, and _fp is each of the features during the traversal. is the first preset distance or the second preset distance, dis( _ft , _fp ) is the Euclidean distance between _ft and _fp , For the b-th subclass that satisfies/> The number of features. 18. An image generation system, characterized by comprising: The acquisition module is used to obtain semantic guidance text and emotional guidance text; A retrieval module configured to retrieve multiple reference image samples based on the semantic guidance text and the emotion guidance text; An extraction module, configured to extract features of a plurality of reference image samples, and combine at least two of all the features to obtain multiple image combination semantic features; A generation module is used to obtain text semantic features corresponding to the semantic guidance text, and generate associated images based on the image with the highest similarity to the text semantic features and the semantic features. 19. An electronic device, characterized in that it includes: Memory for storing computer programs; A processor, configured to implement the steps of the image generation method according to any one of claims 1-17 when executing the computer program. 20. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image generation method according to any one of claims 1 to 17 are implemented.