CN117216662A - Risk identification method, system, device, equipment and storage medium - Google Patents

Abstract

The embodiment of the invention provides a risk identification method, a system, a device, equipment and a storage medium, wherein the method comprises the following steps: determining a target position of the potential risk text in the dialogue text, taking the target position as a starting point, and intercepting target text with preset length from the dialogue text, wherein other dialogue texts in the target text are generated before the potential risk text. Then, the target text containing the potential risk text and the risk identification text are used as prompt information, and a generated model is input to further determine whether the potential risk text is at risk or not by the generated model. I.e. the risk recognition means is for further risk recognition of the risk potential text. And during the recognition process, the device can further extract the text of the potential risk based on the target position of the text of the potential risk. The generated model can carry out risk recognition on the potentially-risk text again by means of the potentially-risk text and the potentially-risk text, so that accuracy of text risk recognition is improved.

Description

Risk identification methods, systems, devices, equipment and storage media

Technical field

The present invention relates to the field of artificial intelligence technology, and in particular, to a risk identification method, system, device, equipment and storage medium.

Background technique

Instant Messaging (IM) tool is an application for transmitting instant messages between users and is one of the most popular Internet communication tools. In practice, the conversation text generated when users use IM tools to chat with others may include information related to violation risks, such as non-compliant charging information, false information, fraudulent information, etc. Therefore, how to improve the accuracy of risk identification in dialogue text has become an urgent technical problem to be solved.

Contents of the invention

In view of this, embodiments of the present invention provide a risk identification method, system, device, equipment and storage medium to improve the accuracy of risk identification in dialogue text.

In a first aspect, embodiments of the present invention provide a risk identification method, including:

Determine where potentially risky text should be targeted within the conversation text;

Taking the target position as a starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text;

The target text and the risk identification text containing the potential risk text are input into the generative model as prompt information, so that the generative model determines whether the potential risk text contains risks.

In a second aspect, embodiments of the present invention provide a risk identification system, including: a client and a server deployed with a generative model;

The client is used to determine the target position of the potential risk text in the conversation text;

Taking the target position as the starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text; send the target text to the server ;

The server is configured to input the target text and risk identification text containing the potential risk text as prompt information into the generative model, so that the generative model determines whether the potential risk text exists. risk.

In a third aspect, embodiments of the present invention provide a risk identification system, including: a client and a server deployed with a generative model;

The client is used to send potentially risky text input by the user;

The server is configured to receive the potential risk text to constitute a dialogue text consisting of the potential risk text and the dialogue text generated before the potential risk text;

Determine where potentially risky text should be targeted within the conversation text;

Taking the target position as a starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text;

The target text and risk identification text containing the potential risk text are input into the generative model as prompt information, so that the generative model determines whether the potential risk text contains risks.

In a fourth aspect, embodiments of the present invention provide a risk identification device, including:

The position determination module is used to determine the target position of potentially risky text in the dialogue text;

A target text determination module, configured to take the target position as a starting point and intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text;

A risk determination module is configured to input the target text and risk identification text containing the potential risk text as prompt information into a generative model, so that the generative model determines whether the potential risk text contains risks.

In a fifth aspect, embodiments of the present invention provide an electronic device, including a processor and a memory, the memory being used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor Implement the risk identification method in the first aspect above. The electronic device may also include a communication interface for communicating with other devices or communication systems.

In a sixth aspect, embodiments of the present invention provide a non-transitory machine-readable storage medium. The non-transitory machine-readable storage medium stores executable code. When the executable code is processed by a processor of an electronic device, When executed, the processor is enabled to at least implement the risk identification method in the above first aspect.

In the risk identification method provided by the embodiment of the present invention, after obtaining the dialogue text, the risk identification device can first determine whether there is a potential risk text in the dialogue text, and if so, further determine the target position of the potential risk text in the dialogue text. Then, using the target position as a starting point, a preset length of target text is intercepted from the conversation text. Among them, other dialogue texts in the target text are generated before the potential risk text. Finally, the target text containing the potential risk text and the risk identification text are used as prompt information and input into the generative model, so that the generative model further determines whether the potential risk text contains risks.

It can be seen that in the above method, the risk identification device performs further risk identification on potential risk texts, that is, texts that may be risky. And during the recognition process, based on the target location of the potentially risky text, the device can further extract the upper context of the potentially risky text. Then the generative model can use the potential risk text and its context to identify the risk of the potential risk text again, that is, by contacting the context above to identify the risk of the potential risk text, so as to improve the accuracy of text risk identification.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a risk identification method provided by an embodiment of the present invention;

Figure 2 is a flow chart of another risk identification method provided by an embodiment of the present invention;

Figure 3 is a flow chart of yet another risk identification method provided by an embodiment of the present invention;

Figure 4 is a flow chart of a model training method provided by an embodiment of the present invention;

Figure 5 is a schematic structural diagram of a risk identification system provided by an embodiment of the present invention;

Figure 6 is a schematic diagram of a risk identification process in an enterprise recruitment scenario provided by an embodiment of the present invention;

Figure 7 is a schematic structural diagram of a risk identification device provided by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an electronic device corresponding to the risk identification device provided by the embodiment shown in FIG. 7 .

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.

The terminology used in the embodiments of the present invention is only for the purpose of describing specific embodiments and is not intended to limit the present invention. As used in this embodiment and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. Generally, at least two are included, but at least one is not excluded.

It should be understood that the term "and/or" used in this article is only an association relationship describing related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, and A and A exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

Depending on the context, the words "if" or "if" as used herein may be interpreted as "when" or "when" or "in response to a determination" or "in response to an identification." Similarly, depending on the context, the phrase "if determined" or "if (stated condition or event) is identified" may be interpreted as "when determined" or "in response to determination" or "when (stated condition or event) is identified )" or "in response to identifying (a stated condition or event)."

It should be noted that the terms "include", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a good or system that includes a list of elements includes not only those elements, but also those not expressly listed. Other elements may also include elements inherent to the product or system. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the goods or systems that include the stated element.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention, All are information and data authorized by users or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose and authorize. Or refuse.

Before describing the risk identification methods provided by various embodiments of the present invention in detail, the application scenarios of risk identification can also be schematically described:

As introduced in the background art, the conversation text generated when a user actually uses an instant messaging tool to chat with others may include illegal information, such as non-compliant charging information, false information, fraudulent information, etc.

Taking the corporate recruitment scenario as an example, the dialogue between the corporate recruiter and the applicant is as follows:

Recruiter: "There is a bank less than 100 meters away from our company. Our company has a cooperative relationship with the bank. Company employees can withdraw money, deposit money, transfer money and other services at the bank at any time.";

Applicant: "That's quite convenient.";

Recruiter: "The company stipulates that new employees need to go to the bank to handle a business before joining the company.";

Applicant: "What specific business is it?";

Recruiter: "Onboarding training fee, this is a formal fee."

In the above-mentioned dialogue text between the recruiter and the applicant, if each text is analyzed separately, it can be determined that each text is normal and there are no violations. However, based on the context of the entire conversation, it can be seen that the recruiter requires new employees to go to a designated bank to pay the induction training fee before joining, which means that the banks that pay the induction training fee are restricted. In this case, the recruiter proposes to charge the induction training fee. The credibility is relatively low. Therefore, the text "Induction training fee, this is a formal fee" generated by the recruiter can be regarded as a potential risk text. The content of this text may fall into the category of non-compliant charges, and further risk identification is required.

At this time, the risk identification method provided by the following embodiments of the present invention can be used to further determine whether the potentially risky text in the above conversation is truly risky. And optionally, the risk identification device can perform risk identification during the real-time generation of the dialogue, or can also perform risk identification after the dialogue ends.

Optionally, the conversation text may be all the conversation text generated by both users through instant messaging tools as in the above example, or it may be the conversation text generated between the user and the smart device. Optionally, the smart device may include: any one of a service robot with a conversation function, a mobile terminal with a conversation function, and an intelligent wearable device with a conversation function.

Based on the above description, some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments may be combined with each other as long as there is no conflict between the embodiments. In addition, the sequence of steps in the following method embodiments is only an example and is not strictly limited.

Figure 1 is a flow chart of a risk identification method provided by an embodiment of the present invention. The risk identification method provided by the embodiment of the present invention can be executed by a risk identification device. It can be understood that the risk identification device can be implemented as software, or a combination of software and hardware. As shown in Figure 1, the method may include the following steps:

S101. Determine the target position of the potential risk text in the dialogue text.

Just like the examples in the above embodiments, both users can use the instant messaging tool to implement at least one round of dialogue to generate dialogue text.

In one case, the risk identification device can perform risk identification during the real-time generation of the dialogue. In this case, when either user generates a latest text, the latest text can be obtained by the risk identification device, and the risk identification device can immediately perform preliminary risk identification on the latest text. . If the risk identification device determines that the latest text has a potential risk, the latest text is a potential risk text. Further, the risk identification device may determine the target position of the potential risk text in the currently generated dialogue text.

Alternatively, the risk identification device can also perform risk identification after the entire conversation has ended. In this case, after the dialogue between the two users is completed, the risk identification device can obtain all dialogue texts generated by both users, and perform preliminary risk identification on each text in all dialogue texts. If the risk identification device determines that a certain dialogue text has a potential risk, the text is determined to be a potential risk text. Further, the risk identification device may determine the target position of the potential risk text in the currently generated dialogue text.

In the above two cases, for the determination of potential risk texts, in an optional way, the risk identification device can search from the risk keyword library to determine whether any text containing any risk keyword needs to be preliminary risk identification. text, and treat text containing risk keywords as potential risk text.

Optionally, the potential risk text can be the above-mentioned text related to charging induction training fees, or it can be text related to publishing false and untrue information, such as non-existent job information or untrue housing information, etc. .

When the risk identification device can perform risk identification after the entire conversation is over, if the number of conversation texts generated between the two users is large, in order to improve the positioning efficiency of determining the target location, optionally, the risk identification device can also perform risk identification on all the dialogue texts. Filter the dialogue text, that is, filter out part of the dialogue text generated by both users within a certain period of time, and determine the target location corresponding to the potentially risky text from this part of the dialogue text.

S102, starting from the target position, intercept the target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text.

S103. Enter the target text containing the potential risk text and the risk identification text as prompt information into the generative model, so that the generative model determines whether the potential risk text contains risks.

Based on the target location determined in step S101, the risk identification device can first use the target location as a starting point to intercept a target text of a preset length from the conversation text. Among them, other dialogue texts in the target text are generated before the potential risk text, that is, the target text consists of the potential risk text and its upper text. Then, the risk identification device can input the target text containing the potential risk text and the risk identification text as prompt information into the generative model, so that the generative model determines whether the potential risk text contains risks.

Optionally, the preset length can be the maximum text length threshold that the generative model can process, or it can be any value smaller than the text length threshold. Optionally, the generative model may include a Generalized Linear Model (GLM) or a Generative Pre-training Transformer (GPT) model that has strong text generation capabilities and a large model volume.

Optionally, the risk identification text is used to guide the generative model to perform risk identification, for example, it can be "Detect whether the "onboarding training fee, this is a formal fee" in the following dialogue content involves deception." Among them, "Check whether xx in the following dialogue content involves deception" is the detection command in the risk identification text, which is used to indicate the direction of detection; "Induction training fee, this is a formal fee" is the potential risk text, that is, the text to be detected.

Optionally, the risk identification text can also be, for example, "Check whether the 'onboarding training fee, this is a formal fee' in the dialogue below involves deception, and answer directly yes or no." Among them, "direct answer yes or no" is the answer format, which is used to limit the format of the risk identification results output by the generative model.

In the risk identification method provided by the embodiment of the present invention, after obtaining the dialogue text, the risk identification device can first determine whether there is a potential risk text in the dialogue text, and if so, further determine the target position of the potential risk text in the dialogue text. . Then, using the target position as a starting point, a preset length of target text is intercepted from the conversation text. Among them, other dialogue texts in the target text are generated before the potential risk text. Finally, the target text containing the potential risk text and the risk identification text are used as prompt information and input into the generative model, so that the generative model further determines whether the potential risk text contains risks.

It can be seen that in the above method, the risk identification device performs further risk identification on potential risk texts, that is, texts that may be risky. And during the recognition process, based on the target location of the potentially risky text, the device can further extract the upper context of the potentially risky text. Then the generative model can use the potential risk text and its context to identify the risk of the potential risk text again, that is, by contacting the context above to identify the risk of the potential risk text, so as to improve the accuracy of text risk identification.

In addition, after the generative model determines that the potential risk text is risky, the risk identification device can also display the risk identification result on the display interface of the user terminal to provide a risk reminder to the user.

Taking the example of the above corporate recruitment scenario, assume that the recruiter and the applicant generate a total of 5 conversation texts through instant messaging tools, and the risk identification device determines that "entry training expenses, this is a regular fee" is a potential risk text, and this potential risk text This is the dialogue text of Article 5.

Assuming that the default length is the length corresponding to four dialogue texts, the fifth dialogue text can be used as the starting point, and four dialogue texts can be counted forward, that is, the second dialogue text to the fifth dialogue text can finally be intercepted. Then, the 2nd to 5th dialogue text and the risk identification text are used as prompt information and input into the generative model, so that the generative model outputs "onboarding training expenses, this is a formal expense", which is a risk. Finally, the output result is displayed on the display interface of the user terminal to prompt the user to avoid risks. The prompt form can be seen in Figure 6.

A method of determining the target location of potential risk text based on risk keywords has been mentioned in the embodiment shown in Figure 1 . Optionally, the risk identification device can also determine the target location by analyzing the feature vector.

Specifically, the risk identification device performs feature extraction on each text in the dialogue text generated by the user to obtain a feature vector of each text with a lower dimension. Then, the risk identification device can determine whether there is potential risk text in the dialogue text based on the respective feature vectors of each text. If the risk identification device determines that there is a potential risk text in the dialogue text, the target location of the potential risk text can be further obtained.

And the above process of feature extraction and potential risk text positioning can be performed by the same model, that is, the first model, or can be performed by different models. Optionally, the first model may include a convolutional neural network (Convolutional NeuralNetworks, CNN for short), a Bidirectional Encoder Representation from Transformers (Bert for short) model, and a recurrent neural network (Recurrent Neural Network, RNN for short). And any of the Long Short-Term Memory (LSTM) networks.

The embodiment shown in Figure 1 has disclosed a process in which the risk identification device can use the target position as a starting point to intercept the target text. Optionally, before intercepting the target text, the risk identification device may also divide the conversation text first, and intercept the target text based on the division results. Optionally, the risk identification device may divide the dialogue text generated by the user according to the dialogue topic, or may divide the dialogue text generated by the user according to the dialogue duration.

When the user-generated dialogue text is divided according to dialogue topics, the method shown in Figure 1 can be further refined into Figure 2. Figure 2 is a flow chart of another risk identification method provided by an embodiment of the present invention. As shown in Figure 2, the method may include the following steps:

S201: Determine the target position of the potential risk text in the dialogue text.

For the specific implementation process of the above step S201, please refer to the specific description of the relevant steps in the embodiment shown in Figure 1, and will not be described again here.

S202: Divide the dialogue text into at least one dialogue segment according to the dialogue topic.

S203, starting from the target position, intercept the target text from the target dialogue segment to which the potential risk text belongs according to a preset length, and other dialogue texts in the target text are generated before the potential risk text.

The risk identification device can divide the dialogue text into at least one dialogue segment according to the dialogue topic, and use the target position as a starting point to intercept the target text from the target dialogue segment to which the potential risk text belongs according to a preset length. Among them, other dialogue texts in the target text are generated before the potential risk text, that is, the target text consists of the potential risk text and its upper text.

For the division of dialogue topics, optionally, the semantic analysis module in the risk identification device can perform identification and analysis of dialogue topics based on keywords that can clearly characterize the user's main intention to obtain dialogue segments corresponding to different dialogue topics. Optionally, the semantic analysis module can obtain dialogue segments corresponding to different dialogue topics based on keywords that can clearly represent the switching of dialogue topics, such as: "Let's talk about the next thing" or "Let's talk about the rent yesterday."

After obtaining at least one dialogue segment, the risk identification device can find the target dialogue segment to which the potential risk text belongs from the at least one dialogue segment, and use the target position of the potential risk text as a starting point to intercept a target of a preset length from the target dialogue segment. Text, that is, the intercepted target text is the text under the same conversation topic and meets the preset length.

Optionally, if the length of the target dialogue segment is less than the preset length, there is no need to intercept the target dialogue segment, and the target dialogue segment can be directly determined as the target text. That is, the target text is intercepted based on the dialogue topic first. This method can ensure the semantic integrity of the target text and prevent dialogue texts belonging to other topics from being divided into the target text, causing interference to subsequent risk identification.

Optionally, if the length of the target conversation segment is greater than the preset length, the second model can delete the common sense text in the target conversation segment or the common discourse text that is not directly related to the conversation topic, and replace the remaining text in the target conversation segment with Identified as the target text. And if the length of the target dialogue segment after deleting part of the text is still greater than the preset length, time can also be used as the dimension to further delete text that is far away from the time corresponding to the target position.

S204, input the target text containing the potential risk text and the risk identification text as prompt information into the generative model, so that the generative model determines whether the potential risk text has risks.

For the specific implementation process of the above step S204, please refer to the specific description of the relevant steps in the embodiment shown in FIG. 1, which will not be described again here.

In this embodiment, the risk identification device can divide the dialogue text into at least one dialogue segment according to the dialogue topic, that is, the semantics of each dialogue segment is complete. At this time, the risk identification device uses the target position as a starting point to intercept the target text from the target dialogue segment to which the potential risk text belongs, that is, the intercepted target text and the potential risk text have the same dialogue topic. Moreover, by dividing the dialogue topics, it can also prevent dialogue texts belonging to other topics from being divided into the target text, that is, preventing dialogue texts on other topics from interfering with subsequent risk identification, thereby improving the accuracy of subsequent text risk identification.

In addition, the content not described in detail in this embodiment and the technical effects that can be achieved can be referred to the relevant descriptions in the above embodiments, and will not be described again here.

The process of intercepting the target text mentioned in the embodiment shown in FIG. 2 can be specifically executed by the second model, that is, the method shown in FIG. 2 can be further refined into the method shown in FIG. 3 . Optionally, the second model may include any one of CNN, Bert, RNN, and LSTM. Figure 3 is a flow chart of yet another risk identification method provided by an embodiment of the present invention. As shown in Figure 3, the method may include the following steps:

S301. Determine the target position of the potential risk text in the dialogue text.

For the specific implementation process of the above step S301, please refer to the specific description of the relevant steps in the embodiment shown in FIG. 1, which will not be described again here.

S302. Input the dialogue text, target position and preset length into the second model, so that the second model divides the dialogue text into at least one dialogue segment, and outputs starting from the target position and starting from the preset length from the segment to which the potential risk text belongs. The target text is intercepted from the target dialogue segment, and other dialogue texts in the target text are generated before the potential risk text.

The risk identification device can input the dialogue text, the target location of the potential risk text and the preset length into the built-in second model. The second model can divide the dialogue text into at least one dialogue segment, and according to the preset length requirement, output a target that meets the preset length and is intercepted from the target dialogue segment to which the potential risk text belongs, starting from the target position. text. Among them, other dialogue texts in the target text are generated before the potential risk text, that is, the target text consists of the potential risk text and its upper text.

S303: Enter the target text containing the potential risk text and the risk identification text as prompt information into the generative model, so that the generative model determines whether the potential risk text is risky.

For the specific implementation process of the above step S303, please refer to the specific description of the relevant steps in the embodiment shown in FIG. 1, which will not be described again here.

In this embodiment, through the strong learning ability of the second model, the topic division of the dialogue text can be achieved, and the target text with the same dialogue topic as the potential risk text can be intercepted. Moreover, by dividing the dialogue topics, it can also prevent dialogue texts belonging to other topics from being divided into the target text, that is, preventing dialogue texts on other topics from interfering with subsequent risk identification, thereby improving the accuracy of subsequent text risk identification.

In addition, the content not described in detail in this embodiment and the technical effects that can be achieved can be referred to the relevant descriptions in the above embodiments, and will not be described again here.

According to the use process of the risk identification device described in the embodiment shown in FIG. 3, it can be known that the risk identification device can use the second model when intercepting the target text. In order to ensure the accuracy of the second model in intercepting the target text, the training module in the risk identification device can train the second model using supervised training. Figure 4 is a flow chart of a model training method. As shown in Figure 4, the method may include the following steps:

S401: Obtain the training dialogue text, the training position of the risk text in the training dialogue text, and the reference text containing the risk text in the training dialogue text.

Optionally, the training module can obtain the training dialogue text from the training dialogue text set, and the training dialogue text set can be a pre-collected proprietary text set suitable for any enterprise. Among them, the reference text containing the risk text is included in the training dialogue text, and the training position of the risk text in the training dialogue text can be obtained through pre-processing.

S402: Input the training dialogue text, training position and preset length into the second model, so that the second model outputs predicted text starting from the training position and intercepted from the training text according to the preset length.

S403, perform loss calculation on the reference text and predicted text.

S404, train the second model based on the calculation results.

Based on the training dialogue text, training location and reference text obtained in step S401, the training module can use the training dialogue text, training location and preset length as training data, and use the reference text as supervision information to train the second model to use the second model. The model output starts from the training position and extracts the predicted text from the training text according to the preset length.

Then, the training module can perform loss calculation on the reference text and predicted text, and optimize the second model based on the obtained loss calculation results. Optionally, the loss function used in the loss calculation process may include one of a cross-entropy loss function, a logarithmic loss function, and a square loss function.

In this embodiment, a supervised training method is used, that is, the second model is trained with reference text containing risk text, so that the training effect of the second model is better. Furthermore, during the model use stage, the predicted text output by the second model is also more acurrate.

In addition, the content not described in detail in this embodiment and the technical effects that can be achieved can be referred to the relevant descriptions in the above embodiments, and will not be described again here.

To facilitate understanding, the risk identification methods provided by the above embodiments can also be described from the perspective of the entire system. Figure 5 is a schematic structural diagram of a risk identification system provided by an embodiment of the present invention. As shown in Figure 5, the risk identification system includes: a client and a server deployed with a generative model.

Optionally, in practice, considering the different processing capabilities of the client, the client and the server can perform different steps in the above embodiment to implement text risk identification.

When the client's processing capability is high, after obtaining the dialogue text, the client can first determine whether there is potentially risky text in the dialogue text, and if so, further determine the target location of the potential risk text in the dialogue text. The client can then use this target location as a starting point to intercept a preset length of target text from the conversation text. Among them, other dialogue texts in the target text are generated before the potential risk text. The client can then send the target text to the server.

The server can use the risk identification text and the target text containing the potential risk text sent by the client as prompt information and input it into the generative model deployed by itself, so that the generative model can determine whether the potential risk text is risky.

When the client's processing capability is low, the client can send potential risk text input by the user to the server, so that the server can receive the potential risk text and further identify the risk. Specifically, the server may receive the potential risk text to form a dialogue text consisting of the potential risk text and the dialogue text generated before the potential risk text. Then, the server can determine the target position of the potentially risky text in the conversation text, and use the target position as a starting point to intercept the target text of a preset length from the conversation text. Among them, other dialogue texts in the target text are generated before the potential risk text. Then, the server inputs the target text containing the potential risk text and the risk identification text as prompt information into the generative model, so that the generative model determines whether the potential risk text is risky.

The risk identification system provided by the embodiment of the present invention includes: a client and a server deployed with a generative model. Considering that the actual processing capabilities of the client are different, the client and the server can perform different steps in the above embodiments to implement text risk identification. Based on the target location of the potentially risky text, the system can further extract the context of the potentially risky text. Then the generative model deployed in the server can use the potential risk text and its context to identify the potential risk text again, that is, the potential risk text is risk identified by contacting the context above to improve the effectiveness of text risk identification. accuracy.

In addition, the content not described in detail in this embodiment and the technical effects that can be achieved can be referred to the relevant descriptions in the above embodiments, and will not be described again here.

For ease of understanding, the specific implementation process of the risk identification method provided above can be illustratively described in conjunction with an enterprise recruitment scenario. Figure 6 is a schematic diagram of a risk identification process in an enterprise recruitment scenario provided by an embodiment of the present invention. The process is specifically as follows:

In the corporate recruitment scenario, it is assumed that a total of 7 dialogue texts are generated between the recruiter and the applicant, as follows:

Recruiter: "Hello, I am the manager responsible for this recruitment. Please briefly introduce your work experience.";

Applicant: "Okay, I have worked in Company A for 5 years and have sufficient work experience.";

Recruiter: "Very good, let's talk about the next thing. There is a bank less than a hundred meters away from our company. Our company has a cooperative relationship with the bank. Company employees can withdraw money at the bank at any time. Services such as depositing money and transferring money.";

Applicant: "That's quite convenient.";

Recruiter: "The company stipulates that new employees need to go to the bank to handle a business before joining the company.";

Applicant: "What specific business is it?";

Recruiter: "Onboarding training fee, this is a formal fee.".

After acquiring the above 7 dialogue texts, the risk identification device can input the 7 dialogue texts into the built-in first model, so that the first model can output the potential risk text "Induction Training Fee" based on the respective feature vectors of each text. , this is the position of "regular fee", which is where the dialogue text of Article 7 is located.

Then, the risk identification device can use the built-in second model to divide the seven dialogue texts into dialogue topics, and obtain the first dialogue segment including the 1st to 2nd dialogue texts, and the second dialogue segment including the 3rd to 7th dialogue texts. part.

Then, the risk identification device can find the second dialogue segment to which the seventh dialogue text belongs from the two dialogue segments, and use the position of the seventh dialogue text as a starting point to intercept a preset length of text from the second dialogue segment. Assuming that the preset length is greater than the length of the second dialogue segment, the second model can output the second dialogue segment, that is, the 3rd to 7th dialogue texts composed of the 7th dialogue text and its preceding text. If the preset length is less than the length of the second dialogue segment, you may refer to the processing procedures mentioned in the above embodiments, such as deleting text in the second dialogue segment that is irrelevant to the dialogue topic, which will not be described again here. For the training process of the second model, reference can be made to the descriptions in the above embodiments, and details will not be described again here.

Finally, the risk identification device takes the 3rd to 7th dialogue text and "Check whether the 'entry training fee, this is a formal fee' in the following dialogue content involves deception, and directly answer yes or no" as prompt information, and input it into the generative model , there is a risk in outputting 'onboarding training expenses, this is a formal expense' from the generative model. The risk identification result "This dialogue text is risky" is displayed on the display interface of the user terminal as risk prompt information to prompt the user to avoid risks.

In addition, the content not described in detail in this embodiment and the technical effects that can be achieved can be referred to the relevant descriptions in the above embodiments, and will not be described again here.

The risk identification device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art can understand that these risk identification devices can be constructed using commercially available hardware components and configured through the steps taught in this solution.

Figure 7 is a schematic structural diagram of a risk identification device provided by an embodiment of the present invention. As shown in Figure 7, the device includes:

The position determination module 11 is used to determine the target position of the potential risk text in the dialogue text.

The target text determination module 12 is configured to take the target position as a starting point and intercept a target text of a preset length from the dialogue text. Other dialogue texts in the target text are generated before the potential risk text.

The risk determination module 13 is configured to input the target text containing the potential risk text and the risk identification text as prompt information into a generative model, so that the generative model determines whether the potential risk text contains risks.

Optionally, the location determination module 11 is configured to perform feature extraction on each text in the dialogue text; determine whether there is a potential risk text in the dialogue text based on the respective feature vectors of each text; If the potential risk text exists, the target location of the potential risk text is determined.

Optionally, the location determination module 11 is also configured to input the dialogue text into a first model, so that the first model outputs the potential risk according to the respective feature vectors of each text in the dialogue text. The target location of the text.

Optionally, the target text determination module 12 is configured to divide the dialogue text into at least one dialogue segment according to the dialogue topic; starting from the target position, and starting from the target to which the potential risk text belongs according to a preset length. The target text is intercepted from the dialogue segment.

Optionally, the target text determination module 12 is configured to determine the target dialogue segment as the target text if the length of the target dialogue segment is less than the preset length.

Optionally, the target text determination module 12 is configured to input the dialogue text, the target position and the preset length into a second model, so that the second model divides the dialogue text into at least A dialogue segment, and output the target text that is intercepted from the target dialogue segment to which the potential risk text belongs according to the preset length, starting from the target position.

Optionally, the device further includes: a training module 14 for obtaining the training dialogue text, the training position of the risk text in the training dialogue text, and the reference text containing the risk text in the training dialogue text; The training dialogue text, the training position and the preset length are input to the second model, so that the output of the second model is intercepted from the training text according to the preset length starting from the training position. the predicted text; perform loss calculation on the reference text and the predicted text; and train the second model according to the calculation results.

The device shown in Figure 7 can perform the method of the embodiment shown in Figures 1 to 4. For parts not described in detail in this embodiment, reference can be made to the relevant description of the embodiment shown in Figures 1 to 4. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in Figures 1 to 4, and will not be described again here.

In a possible design, the structure of the risk identification device can be implemented as an electronic device, as shown in FIG. 8 . The electronic device can include: a processor 21 and a memory 22 . The memory 22 is used to store a program that supports the electronic device to execute the risk identification method provided in the embodiment shown in FIGS. 1 to 4 , and the processor 21 is configured to execute the program stored in the memory 22 . program of.

The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 21, the following steps can be implemented:

Determine where potentially risky text should be targeted within the conversation text;

Taking the target position as a starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text;

The target text and the risk identification text containing the potential risk text are input into the generative model as prompt information, so that the generative model determines whether the potential risk text contains risks.

Optionally, the processor 21 is also configured to execute all or part of the steps in the aforementioned embodiments shown in FIGS. 1 to 4 .

Wherein, the structure of the electronic device may also include a communication interface 23 for the electronic device to communicate with other devices or communication networks.

In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used in the above-mentioned electronic device, which includes programs involved in executing the risk identification method in the above-mentioned method embodiments shown in FIGS. 1 to 4 .

Embodiments of the present invention also provide a computer program product. The computer program product includes computer program instructions. When these computer program instructions are read and run by a processor, the risks involved in executing the method embodiments shown in FIGS. 1 to 4 above are eliminated. recognition methods.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be used Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (12)

1. A risk identification method, characterized by including: Determine where potentially risky text should be targeted within the conversation text; Taking the target position as a starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text; The target text and the risk identification text containing the potential risk text are input into the generative model as prompt information, so that the generative model determines whether the potential risk text contains risks. 2. The method according to claim 1, characterized in that determining the target position of the potential risk text in the dialogue text includes: Perform feature extraction on each text in the dialogue text respectively; Determine whether there is potential risk text in the dialogue text according to the respective feature vectors of each text; If the potential risk text exists, the target location of the potential risk text is determined. 3. The method according to claim 1, characterized in that determining the target position of the potential risk text in the dialogue text includes: The dialogue text is input into the first model, so that the first model outputs the target position of the potential risk text according to the respective feature vectors of each text in the dialogue text. 4. The method according to claim 1, characterized in that taking the target position as a starting point and intercepting a target text of a preset length from the conversation text includes: Divide the dialogue text into at least one dialogue segment according to the dialogue topic; Taking the target position as a starting point, the target text is intercepted from the target dialogue segment to which the potential risk text belongs according to a preset length. 5. The method according to claim 4, characterized in that, the method further comprises: If the length of the target dialogue segment is less than the preset length, the target dialogue segment is determined as the target text. 6. The method according to claim 1, characterized in that taking the target position as a starting point and intercepting a target text of a preset length from the conversation text includes: Input the dialogue text, the target position and the preset length into a second model, so that the second model divides the dialogue text into at least one dialogue segment, and outputs the target position as a starting point, The target text is intercepted from the target dialogue segment to which the potential risk text belongs according to the preset length. 7. The method according to claim 6, characterized in that the method further comprises: Obtain the training dialogue text, the training position of the risk text in the training dialogue text, and the reference text containing the risk text in the training dialogue text; The training dialogue text, the training position and the preset length are input to the second model, so that the second model outputs the training text from the training text according to the preset length starting from the training position. The intercepted predictive text; Perform loss calculation on the reference text and the predicted text; The second model is trained according to the calculation results. 8. A risk identification system, characterized in that the system includes: a client and a server deployed with a generative model; The client is used to determine the target position of the potential risk text in the conversation text; Taking the target position as the starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text; send the target text to the server ; The server is configured to input the target text and risk identification text containing the potential risk text as prompt information into the generative model, so that the generative model determines whether the potential risk text exists. risk. 9. A risk identification system, characterized in that the system includes: a client and a server deployed with a generative model; The client is used to send potentially risky text input by the user; The server is configured to receive the potential risk text to constitute a dialogue text consisting of the potential risk text and the dialogue text generated before the potential risk text; Determine where potentially risky text should be targeted within the conversation text; Taking the target position as a starting point, intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text; The target text and risk identification text containing the potential risk text are input into the generative model as prompt information, so that the generative model determines whether the potential risk text contains risks. 10. A risk identification device, characterized in that it includes: The position determination module is used to determine the target position of potentially risky text in the dialogue text; A target text determination module, configured to take the target position as a starting point and intercept a target text of a preset length from the dialogue text, and other dialogue texts in the target text are generated before the potential risk text; A risk determination module is configured to input the target text and risk identification text containing the potential risk text as prompt information into a generative model, so that the generative model determines whether the potential risk text contains risks. 11. An electronic device, characterized in that it includes: a memory and a processor; wherein executable code is stored on the memory, and when the executable code is executed by the processor, the processor is caused to execute The risk identification method according to any one of claims 1 to 7. 12. A non-transitory machine-readable storage medium, characterized in that executable code is stored on the non-transitory machine-readable storage medium. When the executable code is executed by a processor of an electronic device, The processor executes the risk identification method according to any one of claims 1 to 7.