CN119150984A

CN119150984A - Controllable response method, device and storage medium based on large language model

Info

Publication number: CN119150984A
Application number: CN202411063903.4A
Authority: CN
Inventors: 刘艳军; 刘珊杉; 徐亮; 彭桂斌; 赵赟鑫; 任天赠
Original assignee: Tongfang Knowledge Network Digital Publishing Technology Co ltd
Current assignee: Tongfang Knowledge Network Digital Publishing Technology Co ltd
Priority date: 2024-08-05
Filing date: 2024-08-05
Publication date: 2024-12-17

Abstract

The invention discloses a controllable response method, a device and a storage medium based on a large language model, wherein the method comprises the steps of responding to a dialogue instruction aiming at the large language model, determining input information indicated by the dialogue instruction and a controllable response mode; the method comprises the steps of determining input extension information corresponding to input information, carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment, determining at least one search keyword based on the input extension information and documents to which the recall document fragment belongs, carrying out document recall in the preset document database based on each search keyword to obtain keyword recall documents, forming an extension document list by the keyword recall documents and the documents to which the recall document fragment belongs, determining a target extension document indicated by a controllable response mode in the extension document list, and jointly inputting the target extension document and the input extension information into a large language model for information retrieval to obtain response information corresponding to the input information.

Description

Controllable response method, device and storage medium based on large language model

Technical Field

The present invention relates to the field of information technologies, and in particular, to a controllable response method, device and storage medium based on a large language model.

Background

With the rapid development of artificial intelligence technology, AI question-answering technology has been widely applied to various fields, for example, the field of document knowledge retrieval.

Currently, the answer content is generally output directly according to the user's question. However, because the user needs the answer content with stronger individuation degree, and the user often has difficulty in comprehensively expressing the intention when facing complex problems or being difficult to say the appeal of the surface, the answer content directly output according to the user problem is excessively generalized, the pertinence and the professional degree are not high, the actual requirement of the user can not be met, and the output accuracy of the answer content is lower.

Disclosure of Invention

The invention provides a controllable response method, a controllable response device and a storage medium based on a large language model, which mainly aims at enabling output response contents to be more personalized and specialized so as to meet the actual demands of users.

According to a first aspect of the present invention, there is provided a controllable response method based on a large language model, comprising:

responding to a dialogue instruction aiming at a large language model, and determining input information and a controllable response mode indicated by the dialogue instruction;

determining input extension information corresponding to the input information, and carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment;

Determining at least one search keyword based on the input extension information and the document to which the recall document snippet belongs;

based on each search keyword, carrying out document recall in a preset document database to obtain keyword recall documents, and forming an extended document list by the keyword recall documents and documents to which the recall document fragments belong;

and determining a target extension document indicated by the controllable response mode in the extension document list, and inputting the target extension document and the input extension information together into a large language model for information retrieval to obtain response information corresponding to the input information.

Optionally, the determining the input extension information corresponding to the input information includes:

determining a preset expansion information template based on the field and the type of the input information, wherein the preset expansion information template comprises key information filling positions;

determining key information in the input information, and filling the key information filling position in the preset expansion information template based on the key information to obtain initial input expansion information corresponding to the input information;

carrying out semantic analysis on the initial input extension information to obtain an extension semantic information vector, and carrying out semantic analysis on the input information to obtain an input semantic information vector;

calculating the similarity between the initial input expansion information and the input information based on the expansion semantic information vector and the input semantic information vector, and judging whether the initial input expansion information meets expansion requirements or not based on the similarity;

And if the expansion requirement is met, determining the initial input expansion information as input expansion information corresponding to the input information, otherwise, redetermining the input expansion information corresponding to the input information.

Optionally, the determining at least one search keyword based on the input extension information and the document to which the recall document fragment belongs includes:

Performing word segmentation processing on the document to which the recall document fragment belongs to obtain each word segment contained in the document to which the recall document fragment belongs;

determining an expansion feature vector corresponding to the input expansion information and determining a word segmentation feature vector corresponding to each word segmentation;

based on the expansion feature vector and each word segmentation feature vector, carrying out semantic similarity matching on the input expansion information and each word segmentation to obtain a similarity matching result;

And determining at least one search keyword in each segmented word based on the similarity matching result.

Optionally, the determining, in the extended literature list, the target extended literature indicated by the controllable response mode includes:

if the controllable response mode is a text selection controllable response mode, determining a target extension document indicated by the text selection controllable response mode in the extension document list;

if the controllable response mode is a grouping controllable response mode, clustering each extension document in the extension document list aiming at a target cluster type indicated by the grouping controllable response mode to obtain extension documents under different clustering topics corresponding to the target cluster type, wherein the different clustering topics correspond to different subject words;

And determining target subject words indicated by the grouping controllable response mode from the different subject words, carrying out document recall based on the target subject words to obtain a subject recall document, and determining the subject recall document as the target extension document.

Optionally, clustering each extended document in the extended document list to obtain an extended document under different clustering topics corresponding to the target cluster type includes:

determining document feature vectors corresponding to the extended documents;

initializing centroid vectors corresponding to different clusters under the target cluster type;

Calculating cosine similarity between each document feature vector and centroid vectors corresponding to different clusters, and dividing each extended document into different clusters based on the cosine similarity corresponding to the different clusters;

Based on literature feature vectors corresponding to the extended literature in the different clusters, updated centroid vectors corresponding to the different clusters are obtained;

and dividing each extension document into different clusters again based on the updated centroid vector until the updated centroid vector is unchanged, and determining the extension document finally divided into the different clusters as the extension document under different clustering subjects corresponding to the target cluster type.

Optionally, the determining, among the different subject terms, the target subject term indicated by the group controllable answer mode includes:

determining word frequency of each subject term in each extended document corresponding to the extended document list;

determining a preset number of high-frequency subject words in each subject word based on the word frequency;

And determining the target subject words indicated by the grouping controllable response mode from the high-frequency subject words.

Optionally, the method further comprises:

the input extension information and the recall document fragment are input into the large language model together for information retrieval to obtain response information corresponding to the input information, or,

Constructing an extended literature list based on the literature to which the recall literature fragment belongs;

and determining an extended document indicated by a text-selecting controllable response mode in the extended document list, and inputting the extended document and the input extended information together into a large language model for information retrieval to obtain response information corresponding to the input information.

According to a second aspect of the present invention, there is provided a controllable response device based on a large language model, comprising:

a first determining unit, configured to determine input information and a controllable response mode indicated by a dialogue instruction for a large language model in response to the dialogue instruction;

the first recall unit is used for determining input expansion information corresponding to the input information, and carrying out document recall in a preset document fragment database based on the input expansion information to obtain a recall document fragment;

A second determining unit configured to determine at least one search keyword based on the input extension information and a document to which the recall document section belongs;

the second recall unit is used for carrying out document recall in a preset document database based on each search keyword to obtain keyword recall documents, and forming an extended document list by the keyword recall documents and documents to which the recall document fragments belong;

And the information retrieval unit is used for determining a target extension document indicated by the controllable response mode in the extension document list, and inputting the target extension document and the input extension information into a large language model together for information retrieval to obtain response information corresponding to the input information.

According to a third aspect of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above controllable answer method based on a large language model.

According to a fourth aspect of the present invention there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the above controllable answer method based on a large language model when executing the program.

Compared with the conventional method for directly outputting response contents according to user problems, the method, device and storage medium for controllable response based on the large language model are characterized by determining input information and a controllable response mode indicated by the dialog instruction according to the dialog instruction of the large language model, determining input extension information corresponding to the input information, carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment, determining at least one search keyword based on the input extension information and documents to which the recall document fragment belongs, carrying out document recall in the preset document database based on each search keyword to obtain keyword recall documents, forming an extended document list by documents to which the keyword recall document and the recall document fragment belong, determining target extended documents indicated by the controllable response mode in the extended document list, and jointly inputting the target extended documents and the input extension information into the large language model to carry out information retrieval to obtain response information corresponding to the input information. The invention can obtain the input expansion information which completely expresses the intention of the user by expanding the input information of the user, and then carries out information retrieval based on the input expansion information so as to output response information, so that the output response information can more meet the actual requirement of the user, the output accuracy of the response information can be improved, and then, in the process of information retrieval based on the input extension information, the large language model introduces a controllable response mode to realize the control of the retrieval process of the large language model so as to control the large language model to carry out information retrieval in a designated document, so that the retrieved response content is more personalized and specialized, the actual requirement of a user is met, and the output accuracy of the response information is further improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

FIG. 1 shows a flow chart of a controllable response method based on a large language model provided by an embodiment of the invention;

FIG. 2 is a flow chart of another controllable response method based on a large language model according to an embodiment of the present invention;

FIG. 3 is a schematic diagram showing various response information output modes based on a large language model according to an embodiment of the present invention;

FIG. 4 shows a schematic structural diagram of a controllable response device based on a large language model according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of another controllable response device based on a large language model according to an embodiment of the present invention;

fig. 6 shows a schematic physical structure of a computer device according to an embodiment of the present invention.

Detailed Description

The application will be described in detail hereinafter with reference to the drawings in conjunction with embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.

At present, according to the mode of directly outputting response contents according to the user problem, the output response contents are too generalized, the pertinence and the professional degree are not high, the actual demands of the user cannot be met, and therefore the output accuracy of the response contents is lower.

In order to solve the above problems, an embodiment of the present invention provides a controllable response method based on a large language model, as shown in fig. 1, the method includes:

101. In response to a dialog instruction for a large language model, input information and a controllable answer mode indicated by the dialog instruction are determined.

The method for constructing the preset document fragment database comprises the steps of integrating document data of journals, academic papers, newspapers, conferences and the like of all recorded websites, vectorizing the integrated data, splitting the integrated data into vectorized data fragments to form a vector data fragment set, preprocessing the xml structured data, extracting contents such as document titles, sub-titles and sub-title corresponding document texts as training data, setting training parameters such as iteration times and learning rate and the like, training a text embedding model such as a BGE M3 model based on the training data and the set parameters, vectorizing the text, and converting the structured text into 1024-dimensional vector data based on the trained model. And finally, constructing a preset literature segment database by the literature segments corresponding to the vectorized data segments.

The controllable response mode refers to controlling a dialogue system (a large language model question-answer system) according to user requirements in the process of outputting response information, and the controllable response mode comprises a text-selecting controllable response mode, a grouping controllable response mode and the like. The text selection controllable response mode refers to information retrieval in a specified document selected by a user, and the grouping controllable response mode refers to information retrieval in a document under a grouping specified by the user. Specifically, in the process of interaction between the user and the question-answering system, the user can trigger to generate a dialogue instruction by inputting or selecting the content provided by the question-answering system, for example, the user can input a section of text or voice to express the dialogue instruction generated by the current question to be asked, wherein if the user inputs the text in a voice form, the user also inputs or selects a controllable answer mode in the question-answering system to trigger the dialogue instruction, and then the large language model controls the retrieval process of the large language model by using the controllable answer mode in the process of information retrieval based on the input information, so that the large language model can be prevented from outputting generalized information, the output answer information can meet the actual requirement of the user, and the output accuracy of the answer information is improved.

102. And determining input extension information corresponding to the input information, and carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment.

The input expansion information is information which is normalized, semantically similar to or related to the input information, can express the query intention of the user more clearly and comprehensively, and can expand the input information, and comprises expansion information such as research subjects, problem requirements and the like.

For the embodiment of the invention, when the user inputs information to ask questions, firstly, a dialogue system generates the expanded input information of related topics according to the input information, and particularly, the expanded input information can be generated through specific prompt words (the input expanded information is generated, the input expanded information comprises information such as research topics and problem requirements of the input information, and the like).

103. At least one search keyword is determined based on the input extension information and the document to which the recalled document snippet belongs.

According to the embodiment of the invention, after the input information is expanded to obtain the input expanded information, at least one search keyword is determined in the document to which the recall document fragment belongs based on the input expanded information, specifically, word segmentation processing can be performed on the document to which the recall document fragment belongs, and then at least one search keyword with higher similarity with the input expanded information is determined in each word segment.

104. Based on each search keyword, document recall is carried out in a preset document database to obtain keyword recall documents, and an extended document list is formed by the keyword recall documents and documents to which the recall document fragments belong.

Wherein, various fields and various types of documents are recorded in a preset document database. For the embodiment of the invention, in order to enable the output response information to meet the actual demands of users, a literature list meeting the input information demands needs to be constructed in advance, and then a large language model in a dialogue system is controlled to search information based on each literature in the literature list, so that the searched response information can meet the actual demands of the users, and the response information which is widely displayed to the users is avoided. In order to achieve the above-described effects, it is first necessary to construct a document list, a specific dialogue system may perform document retrieval in a built-in document database based on each search keyword to obtain a plurality of documents (keyword recall documents), when information retrieval is performed using each keyword, if each keyword includes keyword 1, keyword 2, and keyword 3, the retrieval is preferably performed in the form of keyword 1and keyword 2and keyword 3, if the above-described retrieval content is empty, conventional retrieval may be performed in the form of "keyword 1and keyword 2" or "keyword 2and keyword 3" or "keyword 1and keyword 3", thereby retrieving a keyword recall document, then an extended document list is formed by the keyword recall document and the document to which the recall document fragment in step 103 belongs, and then the user may designate a part of the documents in the extended document list, and control the dialogue system to perform information retrieval in the part of the documents based on input of the extended information, thereby enabling the user to respond to the actual demand of information by controlling the dialogue system to perform retrieval in a specified range, that is more accurate.

105. And determining a target extension document indicated by the controllable response mode in the extension document list, and inputting the target extension document and the input extension information together into a large language model for information retrieval to obtain response information corresponding to the input information.

For the embodiment of the invention, the controllable response modes comprise a text selection controllable response mode and a grouping controllable response mode, wherein the text selection controllable response mode refers to that a user directly selects at least one target extension document in an extension document list, then a large language model in a control dialogue system directly searches response information in the target extension document based on input extension information, the grouping controllable response mode refers to that each document in the extension document list is grouped in advance, then the user designates at least one target group in each group, and then the large language model in the control dialogue system directly searches response information in the document under the target group based on input extension information. For example, for the text-selecting controllable response mode, a group of documents designated by a user in the text-selecting controllable response mode is determined in an extended document list to form a document data set x= { X ₁………,x_n }, wherein X ₁、x₂...x_n represents the document designated by the user, the large language model performs full-text content learning training based on the data set X to form a knowledge training result set D (X) of the data set X, and D (X) is a knowledge fragmentation vector data set trained by full-text content. the controllable generation function answers the input extension information according to the vectorized data of the full text content in the knowledge training result set D (X). At the same time, aiming at the grouping controllable response mode, each document in the extended document list is subject, published time, research level, Clustering of different clustering topics is carried out on cluster types such as author elements, topic clustering analysis is taken as an example, an extended literature list is clustered into literatures under different clustering topics, then the occurrence frequency of the topics corresponding to the different clustering topics is ranked according to the occurrence frequency of the topics, 20 high-frequency topics with highest occurrence frequency are obtained, a group of topics is selected in the high-frequency topics, a topic data set Z= { Z ₁………,z_n } is formed, wherein Z ₁...z_n represents each selected topic, literature query is carried out based on the topic data set Z to obtain a plurality of search literatures, then a large language model can be controlled to carry out search of response information in the plurality of search literatures based on input extension information, meanwhile, in order to further reduce the search range, so that the searched information can meet the actual requirement of a user, meanwhile, the information search efficiency is improved, a group of literature data sets X= { X ₁………,x_n } can be designated in each search literature by the user, a data set D (X) controllably generated by grouping the topics is taken as a data set D (X), and a group controllable generation function is to be used for generating a summary in the data set D (X) controllably according to the data set D (X) Title, vector data of the keywords are used for inputting answers of the expanded information, so that the retrieval range is narrowed according to the user requirement in a controllable answer mode, the information retrieval efficiency can be improved, and meanwhile, the retrieved information can meet the actual requirement of the user, namely the accuracy of information retrieval can be improved. Furthermore, the embodiment of the invention not only can search information according to the document control dialogue system appointed by the user, but also can search information according to the user appointed grouping, including topic, publishing time, research level, author and other branch control dialogue systems, and can realize diversified search control modes, thereby meeting diversified demands of the user.

Compared with the conventional method for directly outputting response contents according to user problems, the controllable response method based on the large language model comprises the steps of determining input information and a controllable response mode indicated by a dialogue instruction aiming at the large language model, determining input extension information corresponding to the input information, carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment, determining at least one search keyword based on the input extension information and documents to which the recall document fragment belongs, carrying out document recall in the preset document database based on each search keyword to obtain a keyword recall document, forming an extension document list by the keyword recall document and the documents to which the recall document fragment belongs, determining target extension documents indicated by the controllable response mode in the extension document list, and jointly inputting the target extension documents and the input extension information into the large language model to carry out information retrieval to obtain response information corresponding to the input information. The invention can obtain the input expansion information which completely expresses the intention of the user by expanding the input information of the user, and then carries out information retrieval based on the input expansion information so as to output response information, so that the output response information can more meet the actual requirement of the user, the output accuracy of the response information can be improved, and then, in the process of information retrieval based on the input extension information, the large language model introduces a controllable response mode to realize the control of the retrieval process of the large language model so as to control the large language model to carry out information retrieval in a designated document, so that the retrieved response content is more personalized and specialized, the actual requirement of a user is met, and the output accuracy of the response information is further improved.

Further, in order to better illustrate the foregoing process of controllable response based on a large language model, as a refinement and extension to the foregoing embodiment, an embodiment of the present invention provides another controllable response method based on a large language model, as shown in fig. 2, where the method includes:

201. In response to a dialog instruction for a large language model, input information and a controllable answer mode indicated by the dialog instruction are determined.

Specifically, the dialogue system can perform question matching and answer generation according to the vectorized data, meanwhile, the dialogue system also supports a grouping and text selection controllable generation function, a user can select documents, or topics, publishing time, research levels, authors and the like to perform document inquiry according to requirements, and the dialogue system can generate corresponding answers according to document selection results or grouping selection results.

202. And determining input extension information corresponding to the input information, and carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment.

According to the embodiment of the invention, because the user is difficult to comprehensively express the intention when facing the complex problem or the requirement of a difficult-to-say, the user can not clearly and accurately express the intention, in the case, the accuracy of information retrieval is lower only according to the fact that the user directly retrieves the input information in a dialogue system, so that in order to improve the accuracy of information retrieval, firstly, the input expansion information corresponding to the input information needs to be determined, based on the fact that the input expansion information is determined, step 202 specifically comprises the steps of determining a preset expansion information template based on the field and the type of the input information, wherein the preset expansion information template comprises a key information filling position, determining the key information in the input information, filling the key information in the preset expansion information template based on the key information, so as to obtain initial input expansion information corresponding to the input information, carrying out semantic analysis on the initial input expansion information, so as to obtain an expansion semantic information vector, carrying out semantic analysis on the input information, and based on the expansion semantic expansion information vector, determining whether the input expansion information vector and the input expansion information meets the input expansion requirement input expansion information, if the input expansion information meets the initial expansion information and the input expansion requirement, and if the input expansion information meets the input expansion requirement is met, and if the input expansion requirement is met, and the input expansion requirement is met.

Specifically, different expansion information templates are constructed in advance according to different fields and different information types, a series of expansion information templates are designed to cover different retrieval angles and depths according to the characteristics of document retrieval, the expansion information templates can comprise synonym replacement, superword/hyponym expansion, related concept association and the like, synonym dictionary, online resources or professional glossary can be utilized for the templates of the synonym replacement, synonyms or hyponyms of keywords in input information are added into the retrieval, for example, the input information of a user is "machine learning", and the input expansion information after the synonym replacement can comprise "machine learning", "artificial intelligence algorithm", "automatic learning" and the like. For the template of the expansion of the upper level word/lower level word, the expansion of the lower level word can be used for expanding the search range and finding wider related documents, the upper level word can be obtained by looking up a classification method, a subject word list or a professional book, the expansion of the lower level word is used for precisely searching and finding more specific related documents, and also the resources such as the classification method, the subject word list and the like are needed to be used, for example, the input information is taken as 'machine learning', the upper level word can be 'artificial intelligence', the lower level word can comprise 'deep learning', 'support vector machine', 'decision tree' and the like, and the upper level word and the lower level word are combined to obtain the input expansion information corresponding to the input information.

Further, the extended information template may be in a fixed string format containing placeholders (key information pad locations) that may be replaced by key information in the input question during the information extension process. Further, after the input information input by the user in the dialogue system is obtained, firstly, the domain and the type of the input information are determined, then, a preset expansion information template meeting the expansion requirement is determined in different expansion information templates according to the domain and the type, then, key information extraction is carried out on the input information, for example, NLP (Natural Language Processing ) technology (such as named entity recognition, part of speech tagging and the like) can be used for extracting key information from the input information, such as time, place, people, actions and the like, the extracted key information is represented in a structured form (such as a dictionary, a list or a tree) so as to facilitate subsequent expansion processing, then, according to the preset expansion information template and the extracted key information, expansion problems are generated, namely, placeholders in the preset expansion information template are replaced by actual key information values, or the key information values are filled into key information filling positions in the preset expansion information templates, in order to increase the diversity and coverage rate of the expansion information, a plurality of templates can be designed, and a plurality of expansion information can be generated by applying different templates to the same basic input information. Further, in order to verify the accuracy of expanding the input information, firstly, semantic information is needed to be carried out on the input information and the input expansion information respectively to obtain semantic information vectors, then, based on the semantic information vectors, the similarity between the input information and the input expansion information is calculated, finally, the similarity is larger than a preset threshold (a numerical value set according to actual requirements), the input expansion information is determined to be accurate, if the similarity is smaller than the preset threshold, the input expansion information is determined to be inaccurate, and the input information is expanded again until the similarity between the input expansion information and the input information meets the requirements. For example, input information is that what is today's weather is a preset expansion information template is that what is the weather of what is the place, what is the weather of what is the time, what is the weather forecast of what is the question, what is the weather condition of what is the place, what is the question of what is the time, what is the key information extraction, time is today, place, generated expansion information is what is the weather of the XX market, what is the weather forecast of the present is the question, and what is the weather condition of the XX market is the present is the question. Therefore, the embodiment of the invention can make up the defects of insufficient information quantity and ambiguous intention of the input information determined by the user by expanding the input information, thereby improving the accuracy of information retrieval and the retrieval efficiency of the information by carrying out information retrieval according to the expanded information.

Further, after the input information is expanded to obtain the input expansion information, recall document fragments with high similarity with the input expansion information are searched in a vector database of the whole library (wherein the vector database comprises document fragments corresponding to various documents and the corresponding vector fragments thereof).

203. At least one search keyword is determined based on the input extension information and the document to which the recalled document snippet belongs.

In one embodiment of the present invention, after determining the recall document snippet, the retrieval of the response information may be performed only according to the input extension information and the recall document snippet, that is, the input extension information and the recall document snippet are input together into the large language model to perform information retrieval, so as to obtain the response information corresponding to the input information, and at the same time, the large language model may output the document ID described in the recall document snippet. In a further embodiment of the invention, in order to improve the retrieval accuracy of response information, after outputting the document ID of the recall document fragment by the large language model, acquiring corresponding documents according to the document ID, and determining search keywords according to the input extension information and the respective documents, based on the method, the method comprises the steps of performing word segmentation processing on the document of the recall document fragment to obtain each word segment contained in the document of the recall document fragment; the method comprises the steps of determining an expansion feature vector corresponding to input expansion information, determining a word segmentation feature vector corresponding to each word segmentation, respectively carrying out semantic similarity matching on the input expansion information and each word segmentation based on the expansion feature vector and each word segmentation feature vector to obtain a similarity matching result, and determining at least one search keyword in each word segmentation based on the similarity matching result.

Specifically, each word segment corresponding to the document to which the recall document fragment belongs is determined, then, key information in the input extension information is determined, similarity matching is carried out on the key information and each word segment, target word segments with the similarity meeting the requirement are found in each word segment, and finally, the target word segments are determined to be search keywords. And then, information retrieval is carried out according to the search keywords, so that input information can be simplified, the information retrieval efficiency can be improved, meanwhile, the search keywords are determined in the documents, the search keywords are enabled to be closer to the document form, and the information retrieval accuracy can be improved.

204. Based on each search keyword, document recall is carried out in a preset document database to obtain keyword recall documents, and an extended document list is formed by the keyword recall documents and documents to which the recall document fragments belong.

Wherein the preset document database comprises a plurality of documents. Specifically, according to each search keyword, document recall is performed in a preset document database to obtain a keyword recall document, and an extended document list is formed by documents to which the keyword recall document and the recall document fragment belong, and the extended document list contains a plurality of documents corresponding to input extended information, but the information in each document in the extended document list does not meet the actual requirement of a user, because in order to find information meeting the actual requirement of the user, each document in the extended document list needs to be narrowed down, namely, the range of each document in the extended document list is narrowed down by a controllable response mode, and then a large language model is controlled to perform information retrieval in the narrowed down document, so that the information retrieval efficiency and the information retrieval accuracy can be improved by performing information retrieval in a smaller range.

205. If the controllable response mode is a text selection controllable response mode, determining a target extension document indicated by the text selection controllable response mode in the extension document list.

Specifically, different controllable response modes are selected by a user according to wish, if the user selects the text-selecting controllable response mode, a target extension document indicated by the text-selecting controllable response mode is determined in an extension document list, and then, based on input extension information, a large language model searches response information in documents which are smaller in range and designated by the user, so that the searched response information can meet the actual requirements of the user more. In still another embodiment of the present invention, in order to shorten the search time when the text selection controllable response mode is performed, an extended document list may be directly constructed according to documents to which the recall document segments belong, then an extended document indicated by the text selection controllable response mode is determined in the extended document list, and the extended document and the input extended information are input together into a large language model to perform information search, so as to obtain response information corresponding to the input information.

206. If the controllable response mode is a grouping controllable response mode, clustering each extension document in the extension document list aiming at the target cluster type indicated by the grouping controllable response mode to obtain extension documents under different clustering topics corresponding to the target cluster type, wherein the different clustering topics correspond to different subject words.

The target cluster type comprises at least one of a topic keyword cluster type, a publication time cluster type, a research hierarchy cluster type, an author element cluster type and the like.

For the embodiment of the invention, if the user selects the grouping controllable response mode, a target cluster type indicated by the grouping controllable response mode needs to be determined first, then, for the target cluster type, each extended document in the extended document list is clustered, for example, if the target cluster type is an author element cluster type, each extended document in the extended document list is clustered according to different authors, and each document corresponding to the author a, each document corresponding to the author B, each document corresponding to the author C, and the like are clustered, wherein the authors a, B, and C are subject words under different clustering categories. The method comprises the steps of determining document feature vectors corresponding to all extended documents, initializing centroid vectors corresponding to different clusters under the target cluster type, calculating cosine similarity between the document feature vectors and the centroid vectors corresponding to the different clusters, dividing all the extended documents into the different clusters based on the cosine similarity corresponding to the different clusters, obtaining updated centroid vectors corresponding to the different clusters based on the document feature vectors corresponding to the extended documents in the different clusters, and re-dividing all the extended documents into the different clusters based on the updated centroid vectors until the updated centroid vectors are unchanged, and finally dividing the extended documents into the different clusters to determine the extended documents under different clustering subjects corresponding to the target cluster type.

Specifically, document feature vectors of all the extended documents in the extended document list are determined by means of word embedding and the like, then centroid vectors corresponding to initial centroids corresponding to K clusters are selected, cosine similarity between each document feature vector and K centroid vectors is calculated for the document feature vectors corresponding to all the extended documents, and the cosine similarity between each document feature vector and K centroid vectors can be calculated specifically through the following formula:

The cos (θ) represents cosine similarity between any document feature vector and any centroid vector, x _i represents the ith component in the document feature vector corresponding to any extension document, y _i represents the ith component in the centroid vector corresponding to any extension document, n represents the number of components in the document feature vector corresponding to any extension document, so that cosine similarity between each extension document and each cluster centroid vector can be calculated according to the formula, further, after cosine similarity between each extension document and each cluster centroid vector is calculated, each document feature vector is distributed into a cluster corresponding to the centroid vector with the largest cosine similarity, then, for each cluster, the centroid of each cluster and the centroid vector corresponding to each cluster are recalculated, each extension document is divided into different clusters, so that each extension document is continuously divided until the position of the centroid does not change, namely the centroid vector does not change, and finally, the extension documents divided into different clusters are determined as different text clusters.

207. And determining target subject words indicated by the grouping controllable response modes in different subject words, carrying out document recall based on the target subject words to obtain a subject recall document, and determining the subject recall document as a target extension document.

For the embodiment of the invention, different clustering topics correspond to different subject matters, for example, if the extended documents are clustered according to the publication time clustering type to obtain the extended documents under different publication times, for example, the extended document corresponding to 2020, the extended document corresponding to 2021, the extended document corresponding to 2022, and the like, the subject matters are in 2020, 2021, 2022. Further, in order to reduce the search range, it is necessary to determine the target subject words indicated by the group-controllable response method among different subject words, based on which step 207 specifically includes determining word frequencies of the subject words in the respective extension documents corresponding to the extension document list, determining a preset number of high-frequency subject words among the subject words based on the word frequencies, and determining the target subject words indicated by the group-controllable response method among the high-frequency subject words.

Specifically, clustering each extended document in the extended document list to obtain extended documents under different clustering topics, determining the corresponding subject words of different clustering topics, determining the occurrence frequency, namely the word frequency, of each subject word in each extended document corresponding to the extended document list, determining the high-frequency subject word with the word frequency larger than a preset frequency threshold value, displaying the high-frequency subject word to a user, selecting a target subject word in the high-frequency subject word, namely the target subject word indicated by a grouping controllable response mode, finally performing response information retrieval in a document (target extended document) recalled by the target subject word by the large language model based on input extension information, and simultaneously, in order to further reduce the deceleration range and improve the retrieval accuracy, performing document retrieval based on the target subject word to obtain a plurality of documents, determining the target document (target extended document) indicated by a grouping available response mode in the plurality of documents, and finally controlling the large language model to perform response information retrieval in the target document based on the input extension information. Therefore, the embodiment of the invention adopts the grouping controllable retrieval mode of the high-frequency subject words, and because the high-frequency subject words are frequently appearing words in the literature, the high-frequency subject words often represent the core subject or key concept of the literature, and the retrieval range can be rapidly reduced by retrieving the high-frequency subject words, and unnecessary information screening work is reduced, so that the retrieval efficiency is improved; in addition, because the high-frequency words are usually closely related to the main content of the documents, the information is searched in the documents in which the high-frequency words appear, so that the content highly related to the target subject is more likely to be found, and a large number of documents which are irrelevant to the subject or have little relation can be avoided from being searched, thereby improving the accuracy of the search result; in addition, since the appearance of the high-frequency words often reflects research hotspots and trends in a certain field, the latest research results and the front dynamic in the field can be obtained more easily by searching the high-frequency words.

208. And commonly inputting the target extension document and the input extension information into a large language model for information retrieval to obtain response information corresponding to the input information.

Specifically, in one embodiment of the present invention, if a target extension document specified by a user is determined in an extension document list in a text selection controllable manner, the large language model is controlled to perform response information retrieval in the target extension document directly based on input extension information, and the retrieved response information can more satisfy the actual needs of the user due to the information retrieval in the document specified by the user. In still another embodiment of the present invention, if the high-frequency subject word appearing in the extended document list is selected by the grouping controllable response method, and then the response information retrieval is performed by the grouping controllable response method of the high-frequency subject word, the retrieval range can be rapidly narrowed, unnecessary information screening work can be reduced, thereby improving the retrieval efficiency, and the latest research result and the front dynamic in the field can be more easily obtained.

In still another embodiment of the present invention, as shown in fig. 3, fig. 3 shows a method for outputting multiple types of response information, and an embodiment of the present invention may select any one of the methods for outputting response information according to a requirement. In fig. 3, the input question represents input information, the model is a large language model, and the question is input extension information.

According to the method, compared with a mode of directly outputting response content according to a user problem, input information and a controllable response mode indicated by a dialogue instruction are determined in response to the dialogue instruction aiming at the large language model, input extension information corresponding to the input information is determined, document recall is conducted in a preset document segment database based on the input extension information to obtain a recalled document segment, at least one search keyword is determined based on the input extension information and documents to which the recalled document segment belongs, document recall is conducted in the preset document database based on each search keyword to obtain a keyword recalled document, an extended document list is formed by the keyword recalled document and the documents to which the recalled document segment belongs, target extended documents indicated by the controllable response mode are determined in the extended document list, and the target extended documents and the input extension information are input to the large language model together to conduct information retrieval to obtain response information corresponding to the input information. The invention can obtain the input expansion information which completely expresses the intention of the user by expanding the input information of the user, and then carries out information retrieval based on the input expansion information so as to output response information, so that the output response information can more meet the actual requirement of the user, the output accuracy of the response information can be improved, and then, in the process of information retrieval based on the input extension information, the large language model introduces a controllable response mode to realize the control of the retrieval process of the large language model so as to control the large language model to carry out information retrieval in a designated document, so that the retrieved response content is more personalized and specialized, the actual requirement of a user is met, and the output accuracy of the response information is further improved.

Further, as a specific implementation of fig. 1, the embodiment of the present invention provides a controllable response device based on a large language model, as shown in fig. 4, where the device includes a first determining unit 31, a first recall unit 32, a second determining unit 33, a second recall unit 34, and an information retrieving unit 35.

The first determining unit 31 may be configured to determine, in response to a dialogue instruction for a large language model, input information and a controllable answer mode indicated by the dialogue instruction.

The first recall unit 32 may be configured to determine input extension information corresponding to the input information, and perform document recall in a preset document snippet database based on the input extension information, to obtain a recalled document snippet.

The second determining unit 33 may be configured to determine at least one search keyword based on the input extension information and the document to which the recall document section belongs.

The second recall unit 34 may be configured to recall documents in a preset document database based on each of the search keywords, obtain keyword recall documents, and form an extended document list from the keyword recall documents and documents to which the recall document segments belong.

The information retrieval unit 35 may be configured to determine a target extension document indicated by the controllable response mode in the extension document list, and input the target extension document and the input extension information together into a large language model to perform information retrieval, so as to obtain response information corresponding to the input information.

In a specific application scenario, in order to determine the input extension information corresponding to the input information, as shown in fig. 5, the first recall unit 32, the first determining module 321, the filling module 322, the semantic analysis module 323, and the judging module 324 are described.

The first determining module 321 may be configured to determine a preset extension information template based on a domain and a type to which the input information belongs, where the preset extension information template includes a critical information filling position.

The filling module 322 may be configured to determine key information in the input information, and fill the key information filling location in the preset extension information template based on the key information, so as to obtain initial input extension information corresponding to the input information.

The semantic analysis module 323 may be configured to perform semantic analysis on the initial input extension information to obtain an extension semantic information vector, and perform semantic analysis on the input information to obtain an input semantic information vector.

The determining module 324 may be configured to calculate a similarity between the initial input extension information and the input information based on the extension semantic information vector and the input semantic information vector, and determine whether the initial input extension information meets an extension requirement based on the similarity.

The first determining module 321 may specifically be configured to determine the initial input extension information as input extension information corresponding to the input information if the extension requirement is met, and otherwise, re-determine the input extension information corresponding to the input information.

In a specific application scenario, in order to determine the search keyword, the second determining unit 33 includes a word segmentation module 331, a second determining module 332, and a matching module 333.

The word segmentation module 331 may be configured to perform word segmentation processing on a document to which the recall document fragment belongs, so as to obtain each word segment included in the document to which the recall document fragment belongs.

The second determining module 332 may be configured to determine an extended feature vector corresponding to the input extended information, and determine a word segmentation feature vector corresponding to each word segmentation.

The matching module 333 may be configured to perform semantic similarity matching on the input expansion information and each word segment based on the expansion feature vector and each word segment feature vector, so as to obtain a similarity matching result.

The second determining module 332 may specifically be configured to determine at least one search keyword in each of the segmented words based on the similarity matching result.

In a specific application scenario, in order to determine the target extension document indicated by the controllable response manner, the information retrieval unit 35 includes a third determining module 351 and a clustering module 352.

The third determining module 351 may be configured to determine, in the extended document list, a target extended document indicated by the selected text controllable response mode if the controllable response mode is the selected text controllable response mode.

The clustering module 352 may be configured to, if the controllable response manner is a group controllable response manner, cluster each extended document in the extended document list for a target cluster type indicated by the group controllable response manner, to obtain extended documents under different clustering topics corresponding to the target cluster type, where the different clustering topics correspond to different subject words.

The third determining module 351 may be further configured to determine a target subject word indicated by the group controllable answer manner from among the different subject words, and perform document recall based on the target subject word, so as to obtain a subject recall document, and determine the subject recall document as the target extension document.

In a specific application scenario, in order to cluster each extended document in the extended document list, the clustering module 352 may be specifically configured to determine a document feature vector corresponding to each extended document, initialize centroid vectors corresponding to different clusters under the target cluster type, calculate cosine similarity between each document feature vector and the centroid vector corresponding to the different clusters, partition each extended document into the different clusters based on the cosine similarity corresponding to the different clusters, obtain updated centroid vectors corresponding to the different clusters based on the document feature vectors corresponding to the extended documents in the different clusters, and re-partition each extended document into the different clusters based on the updated centroid vectors until the updated centroid vectors do not change, and finally partition the extended document into the different clusters to determine the extended document under the different clustering subject corresponding to the target cluster type.

In a specific application scenario, in order to determine the target subject word indicated by the group controllable response mode, the third determining module 351 may specifically be configured to determine a word frequency of each subject word in each extended document corresponding to the extended document list, determine a preset number of high-frequency subject words in each subject word based on the word frequency, and determine the target subject word indicated by the group controllable response mode in each high-frequency subject word.

In a specific application scenario, for information retrieval, the information retrieval unit 35 may be further configured to input the input extension information and the recall document fragment together into the large language model for information retrieval, so as to obtain response information corresponding to the input information.

The information retrieval unit 35 may be further configured to construct an extended document list based on documents to which the recall document fragment belongs, determine an extended document indicated by a text-selecting controllable response mode in the extended document list, and input the extended document and the input extended information together into a large language model for information retrieval, so as to obtain response information corresponding to the input information.

It should be noted that, other corresponding descriptions of each functional module related to the controllable response device based on the large language model provided in the embodiment of the present invention may refer to corresponding descriptions of the method shown in fig. 1, which are not described herein again.

Based on the method shown in fig. 1, correspondingly, the embodiment of the invention also provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the program is executed by a processor, and the computer readable storage medium comprises the following steps of responding to a dialogue instruction aiming at a large language model, determining input information and a controllable response mode indicated by the dialogue instruction, determining input extension information corresponding to the input information, carrying out document recall in a preset document fragment database based on the input extension information to obtain a recall document fragment, determining at least one search keyword based on the input extension information and a document to which the recall document fragment belongs, carrying out document recall in the preset document database based on each search keyword to obtain a keyword recall document, forming an extension document list by the keyword recall document and the document to which the recall document fragment belongs, determining a target extension document indicated by the controllable response mode in the extension document list, and jointly inputting the target extension document and the input extension information into the large language model to carry out information retrieval to obtain the corresponding information.

Based on the embodiment of the method shown in fig. 1 and the device shown in fig. 4, the embodiment of the invention further provides a physical structure diagram of a computer device, as shown in fig. 6, the computer device comprises a processor 41, a memory 42 and a computer program which is stored in the memory 42 and can run on the processor, wherein the memory 42 and the processor 41 are arranged on a bus 43, when the processor 41 executes the program, the following steps are realized, namely, input information indicated by a dialogue instruction and a controllable response mode are determined in response to the dialogue instruction aiming at a large language model, input extension information corresponding to the input information is determined, document recall is carried out in a preset document fragment database based on the input extension information, at least one search keyword is determined based on documents which the input extension information and the recall document fragment belong, document recall is carried out in the preset document database based on each search keyword, extended documents are obtained, the extended documents are obtained by the keywords and the extended documents and the configuration fragment belong to the extended documents are formed, the input extension information is determined in the corresponding language model, and the input extension information is obtained in the target language model, and the document is input extension information is indicated in the extended document and the target extension information is obtained.

According to the technical scheme, input information and a controllable response mode indicated by the dialogue instruction are determined through responding to the dialogue instruction aiming at a large language model, input extension information corresponding to the input information is determined, document recall is conducted in a preset document segment database based on the input extension information to obtain a recalled document segment, at least one search keyword is determined based on the input extension information and documents to which the recalled document segment belongs, document recall is conducted in the preset document database based on each search keyword to obtain keyword recalled documents, an extension document list is formed by the keyword recalled documents and the documents to which the recalled document segment belongs, target extension documents indicated by the controllable response mode are determined in the extension document list, and the target extension documents and the input extension information are input into the large language model together to conduct information retrieval to obtain response information corresponding to the input information. The invention can obtain the input expansion information which completely expresses the intention of the user by expanding the input information of the user, and then carries out information retrieval based on the input expansion information so as to output response information, so that the output response information can more meet the actual requirement of the user, the output accuracy of the response information can be improved, and then, in the process of information retrieval based on the input extension information, the large language model introduces a controllable response mode to realize the control of the retrieval process of the large language model so as to control the large language model to carry out information retrieval in a designated document, so that the retrieved response content is more personalized and specialized, the actual requirement of a user is met, and the output accuracy of the response information is further improved.

It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A controllable response method based on a large language model, comprising:

2. The method of claim 1, wherein the determining the input extension information corresponding to the input information comprises:

3. The method of claim 1, wherein the determining at least one search keyword based on the input extension information and the document to which the recall document snippet belongs comprises:

4. The method of claim 1, wherein the determining, in the list of extended documents, the target extended document indicated by the controllable answer means comprises:

5. The method of claim 4, wherein clustering each of the extended documents in the extended document list to obtain the extended document under different clustering topics corresponding to the target cluster type comprises:

determining document feature vectors corresponding to the extended documents;

6. The method of claim 4, wherein said determining the target subject word indicated by the group controllable answer mode among the different subject words comprises:

7. The method according to claim 1, wherein the method further comprises:

8. A controllable response device based on a large language model, comprising:

9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program when executed by the processor implements the steps of the method according to any one of claims 1 to 7.