CN112613569B - Image recognition method, image classification model training method and device - Google Patents

Abstract

The application discloses an image recognition method, an image classification model training device and a storage medium, and relates to the field of natural language processing, in particular to the field of image recognition and deep learning. The specific implementation scheme is as follows: classifying the acquired input image by adopting a trained image classification model to determine the label of the input image from at least one abnormal label and the background labels associated with the abnormal labels; in the method, abnormal behavior indicated by an abnormal label of the input image is determined according to the label of the input image, or the background indicated by a background label of the input image is determined. Therefore, the technical problems of high auditing cost, long period and the like in the manual auditing process of the urban management violation cases in the related art are solved, and the workload of case auditing personnel is reduced.

Description

Image recognition method, image classification model training method and device

Technical Field

The present application discloses an image recognition method, a training method, an apparatus, a device and a storage medium for an image classification model, and particularly relates to the technical field of natural language processing, specifically to image recognition and deep learning.

Background technique

The current process of handling urban management cases is that inspectors patrol the city, collect case pictures after discovering a case, upload them to the urban management system, identify the case type through manual review, and store them in the database. After a period of time, the inspectors will go to the case address again to collect pictures, upload them to the urban management system, and manually review to determine whether the current case has been handled. If it has disappeared, the case will be closed, otherwise the case will be updated for subsequent inspections. The core case identification and case closing tasks in the entire process are completely dependent on manual work. For the number of cases at the million level each year, manual review is costly and time-consuming, and different reviewers have different standards and understandings.

Summary of the invention

The present application provides an image recognition method, an image classification model training method, an apparatus, a device, a storage medium and a computer program product.

The first aspect of the present application provides an image recognition method, comprising:

Obtain a trained image classification model and obtain a captured input image;

Using the image classification model to classify the input image, so as to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label;

According to the label of the input image, it is determined that the input image has abnormal behavior indicated by the abnormal label, or it is determined that the input image has a background indicated by the background label.

As a possible implementation manner of the embodiment of the present application, determining, according to the label of the input image, that the input image has an abnormal behavior indicated by the abnormal label, or determining that the input image has a background indicated by the background label, includes:

In a case where the label of the input image is the abnormal label, determining that the input image exhibits abnormal behavior indicated by the abnormal label;

In a case where the label of the input image is the background label, it is determined that the input image shows the background indicated by the background label and does not have abnormal behavior indicated by the abnormal label associated with the background label.

As another possible implementation of the embodiment of the present application, the method further includes:

Querying historical abnormal behaviors displayed in historical images taken before the input image;

When the abnormal label to which the historical abnormal behavior belongs does not match the label of the input image, an attribute recognition model is used to perform attribute recognition on the input image to obtain an attribute value of at least one attribute, wherein the attribute is used to indicate the abnormal behavior, and the attribute value is used to indicate the probability of the existence of the abnormal behavior;

Determining, from the attribute values of the at least one attribute, an attribute value of a target attribute indicating the historical abnormal behavior;

When the attribute value of the target attribute is less than or equal to the probability threshold, the cancellation process of the historical abnormal behavior is executed.

As another possible implementation of the embodiment of the present application, after querying the historical abnormal behavior displayed in the historical image taken before the input image, it also includes:

When the abnormal label to which the historical abnormal behavior belongs matches the label of the input image, an instruction message for continuing to collect images is issued.

As another possible implementation of the embodiment of the present application, after querying the historical abnormal behavior displayed in the historical image taken before the input image, it also includes:

When the attribute value of the target attribute is greater than the probability threshold, an instruction message for continuing to collect images is issued.

The second aspect of the present application provides a training method for an image classification model, where the image classification model is used to perform the image recognition method described in the first aspect of the present application. The training method includes:

Obtain the first sample set and the second sample set used in this round of training;

Using the training samples in the first sample set to train the image classification model;

Using the training samples in the second sample set to test the trained image classification model, and obtaining the predicted label and corresponding confidence of each training sample in the second sample set;

The target samples in the second sample set are moved to the first sample set to obtain the first sample set and the second sample set used for the next round of training; wherein the target samples include training samples whose predicted labels match the labeled labels and whose confidence is greater than a threshold confidence.

As a possible implementation manner of the embodiment of the present application, the threshold confidence level adopted in the current round of training is greater than the threshold confidence level adopted in the next round of training.

As another possible implementation manner of the embodiment of the present application, after using the training samples in the second sample set to test the trained image classification model and obtaining the predicted labels and confidences of each training sample in the second sample set, the method further includes:

Sending prompt information, wherein the prompt information is used to prompt manual review of the training samples in the second sample set, in which the predicted label does not match the labeled label and the confidence level is greater than a threshold confidence level;

In response to a user review operation, the reviewed training samples are moved as the target samples to the first sample set.

As another possible implementation of the embodiment of the present application, the method further includes:

Inputting a plurality of candidate samples into the image classification model to obtain a predicted label of each of the candidate samples;

Generate a target matrix based on the predicted labels and labeled labels of the plurality of candidate samples; wherein the elements in the target matrix represent candidate samples that meet the labeled labels corresponding to the rows and the predicted labels corresponding to the columns;

Obtain a target element from the target matrix, wherein the target element is an element whose labeled label corresponding to a row does not match the predicted label corresponding to a column;

The second sample set used in the first round of training is generated according to the candidate samples represented by the target element.

The third aspect of the present application provides an image recognition device, including:

An acquisition module, used to acquire a trained image classification model and acquire a collected input image;

An input module is used to classify the input image using the image classification model to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label; a determination module is used to determine, based on the label of the input image, whether the input image has abnormal behavior indicated by the abnormal label, or whether the input image has a background indicated by the background label.

The fourth aspect of the present application provides a training device for an image classification model, wherein the image classification model is used to perform the image recognition method described in the first aspect of the present application, and the training device includes:

An acquisition module, used to acquire a first sample set and a second sample set used in this round of training;

A training module, used to train an image classification model using training samples in the first sample set;

A testing module, used to test the trained image classification model using the training samples in the second sample set, and obtain the predicted label and corresponding confidence of each training sample in the second sample set;

The first moving module is used to move the target sample in the second sample set to the first sample set to obtain the first sample set and the second sample set used for the next round of training; wherein the target sample includes the training sample whose predicted label matches the marked label and whose confidence is greater than the threshold confidence.

A fifth aspect of the present application provides an electronic device, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image recognition method described in the first aspect embodiment, or execute the model training method described in the second aspect embodiment.

The sixth aspect embodiment of the present application proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the image recognition method described in the first aspect embodiment, or to execute the model training method described in the second aspect embodiment.

The seventh aspect embodiment of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the image recognition method described in the first aspect embodiment, or executes the model training method described in the second aspect embodiment.

It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

FIG1 is a schematic diagram of a flow chart of an image recognition method provided in an embodiment of the present application;

FIG2 is a schematic diagram of a flow chart of another image recognition method provided in an embodiment of the present application;

FIG3 is a schematic diagram of a flow chart of another image recognition method provided in an embodiment of the present application;

FIG4 is an example diagram of image recognition provided by an embodiment of the present application;

FIG5 is a flow chart of a method for training an image classification model provided in an embodiment of the present application;

FIG6 is a flow chart of another method for training an image classification model provided in an embodiment of the present application;

FIG7 is a flow chart of another method for training an image classification model provided in an embodiment of the present application;

FIG8 is a diagram showing an example of training an image classification model provided in an embodiment of the present application;

FIG9 is a schematic diagram of the structure of an image recognition device provided in an embodiment of the present application;

FIG10 is a schematic diagram of the structure of a training device for an image classification model provided in an embodiment of the present application;

FIG. 11 is a schematic block diagram of an electronic device for implementing an embodiment of the present application.

Detailed ways

The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

In the related technologies, it is completely dependent on manual review and manual entry of case information. However, the number of cases uploaded in a region reaches millions each year, and each case involves multiple pictures and multiple reviews, which brings huge challenges to manual review. In addition, the judgment standards of different reviewers are not unified, which will affect the quality of case identification and cause ambiguity in the cases stored.

To this end, the present application proposes an image recognition method, which obtains a trained image classification model and a collected input image; uses the image classification model to classify the input image to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label; based on the label of the input image, determines whether the input image has abnormal behavior indicated by the abnormal label, or determines whether the input image has a background indicated by the background label.

The following describes the image recognition method, image classification model training method, device, equipment and storage medium of the embodiments of the present application with reference to the accompanying drawings.

FIG1 is a flow chart of an image recognition method provided in an embodiment of the present application.

The embodiment of the present application takes the image recognition method being configured in an image recognition device as an example. The image recognition device can be applied to any electronic device so that the electronic device can perform an image recognition function.

Among them, the electronic device can be a personal computer (PC), a cloud device, a mobile device, etc. The mobile device can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a car device, and other hardware devices with various operating systems.

As shown in FIG1 , the image recognition method may include the following steps:

Step 101, obtaining a trained image classification model and obtaining a collected input image.

The image classification model is trained with training samples and can classify the input image to determine the label of the input image.

In the embodiment of the present application, the input image may be an image captured by law enforcement personnel using an imaging device, or an image captured by a camera installed on the street, etc., which is not limited here.

It is understandable that in the process of urban management, there are illegal cases such as shops occupying the road and littering. After law enforcement officers discover illegal cases, they can collect images of the scene of the crime, that is, collect input images.

Step 102: classify the input image using an image classification model to determine a label of the input image from at least one abnormal label and background labels associated with each abnormal label.

Among them, the abnormal label is used to indicate the abnormal behavior in the input image, such as occupying the road for business, cross-store business, etc.

The background label associated with the abnormal label is used to refer to the background information of the violation type in the input image. For example, if the abnormal label is "garbage exposure", the background label associated with the abnormal label can be "trash can"; if the abnormal label is "cross-store operation", the background label associated with the abnormal label can be "store frontage", and so on.

In the embodiment of the present application, after obtaining the trained image classification model and the input image, the input image can be input into the image classification model, so as to classify the input image using the image classification model to determine the label of the input image. In other words, the image classification model is used to determine whether the label of the input image is an abnormal label or a background label.

Step 103 : determining, based on the label of the input image, whether the input image has abnormal behavior indicated by the abnormal label, or whether the input image has background indicated by the background label.

In one possible case, the image classification model determines that the label of the input image is an abnormal label, and determines that the input image exhibits abnormal behavior indicated by the abnormal label. Therefore, it can be determined that the case in the input image exists and has not been closed.

In another possible case, the input image is classified by using an image classification model to determine that the label of the input image is a background label. Further, the input image is identified to determine that there is a background indicated by the background label in the input image.

The image recognition method of the embodiment of the present application uses a trained image classification model to classify the collected input image to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label; according to the label of the input image, it is determined that the input image has abnormal behavior indicated by the abnormal label, or that the input image has a background indicated by the background label. Therefore, after the label of the input image is determined by the image classification model, it is determined whether there is abnormal behavior in the input image according to the label of the input image, which effectively solves the technical problems of the need for manual review in the process of handling urban management violation cases in the related technology, and there are high review costs and long cycles, etc., which greatly reduces the workload of case reviewers and significantly optimizes the processing efficiency.

Based on the above embodiments, the present application proposes another image recognition method.

FIG2 is a flow chart of another image recognition method provided in an embodiment of the present application.

As shown in FIG2 , the image recognition method may include the following steps:

Step 201, obtaining a trained image classification model and obtaining a collected input image.

Step 202: classify the input image using an image classification model to determine a label of the input image from at least one abnormal label and background labels associated with each abnormal label.

In the embodiment of the present application, the implementation process of step 201 to step 202 can refer to the implementation process of step 101 to step 102 in the above embodiment, which will not be repeated here.

Step 203 : when the label of the input image is an abnormal label, determine whether the input image exhibits abnormal behavior indicated by the abnormal label.

In one possible case, the input image is classified using an image classification model to determine that the label of the input image is an abnormal label, and further, the input image is identified to determine that the input image exhibits abnormal behavior indicated by the abnormal label. Thus, it can be determined that the case in the input image exists and has not been closed.

As an example, suppose the image classification model classifies the input image and determines that the label of the input image is "garbage exposure", then it is determined that the input image shows garbage exposure behavior. Therefore, it can be determined that the case in the input image exists and the case has not been processed. Images of this location can be collected and the image classification model can be used to continue to classify the images.

Step 204 : when the label of the input image is a background label, determine that the input image shows a background indicated by the background label and does not have an abnormal behavior indicated by an abnormal label associated with the background label.

In another possible case, the input image is classified by using an image classification model to determine that the label of the input image is a background label. Further, the input image is identified to determine that the input image shows a background indicated by the background label and does not have an abnormal behavior indicated by an abnormal label associated with the background label.

As an example, assume that the image classification model classifies the input image and determines that the background label of the input image is "trash can". Further, it is determined that there is a trash can in the input image, but there is no abnormal behavior of "trash exposure". Therefore, it can be determined that the illegal case in the input image has disappeared, and law enforcement officers can close the case.

It should be noted that the above steps 203 and 204 are not executed sequentially, but step 203 is actually executed, or step 204 is executed, according to the label of the input image determined by the image classification model in step 202.

The image recognition method of the embodiment of the present application uses a trained image classification model to classify the collected input image to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label; when the label of the input image is an abnormal label, it is determined that the input image shows abnormal behavior indicated by the abnormal label; when the label of the input image is a background label, it is determined that the background indicated by the background label is displayed in the input image, and there is no abnormal behavior indicated by the abnormal label associated with the background label. Therefore, after the label of the input image is determined by the image classification model, it is determined whether there is abnormal behavior in the input image based on the content displayed in the input image, which effectively solves the technical problems of the need for manual review in the process of handling urban management violation cases in the related technology, and there are high review costs and long cycles, etc., which greatly reduces the workload of case reviewers and significantly optimizes processing efficiency.

In an actual scenario, the image classification model classifies the input image and determines that the label of the input image may not match the abnormal label of the abnormal behavior in the historical image taken before. In this application, an attribute recognition model can be used to perform attribute recognition on the input image to determine whether to execute the verification process for the historical abnormal behavior based on the attribute value output by the attribute recognition model. The following is a detailed introduction in conjunction with Figure 3, which is a flow chart of another image recognition method provided in an embodiment of the present application.

As shown in FIG3 , the image recognition method may further include the following steps:

Step 301 , querying historical abnormal behaviors displayed in historical images taken before the input image.

The historical image has the same shooting location and shooting target as the input image.

It is understandable that when an input image is classified using an image classification model, historical abnormal behaviors displayed in historical images taken before the input image can also be queried.

Furthermore, the abnormal label to which the historical abnormal behavior belongs can be determined based on the historical abnormal behavior displayed in the historical image.

As an example, assuming that the historical abnormal behavior in the historical image is occupying the road for business, it can be determined that the abnormal label to which the historical abnormal behavior belongs is "occupying the road for business".

Step 302: When the abnormal label to which the historical abnormal behavior belongs does not match the label of the input image, an attribute recognition model is used to perform attribute recognition on the input image to obtain an attribute value of at least one attribute.

The attribute is used to indicate abnormal behavior, and the attribute value is used to indicate the probability of the existence of abnormal behavior.

In the embodiment of the present application, each abnormal behavior is taken as an attribute of the image, and whether the input image has abnormal behavior is determined according to the attribute value output by the attribute recognition mode.

Among them, the attribute recognition model is trained by using a large number of sample images, and can accurately identify whether there is abnormal behavior in the image.

In an embodiment of the present application, when the abnormal label to which the historical abnormal behavior displayed in the historical image taken before the input image belongs does not match the label of the input image, an attribute recognition model can be used to perform attribute recognition on the input image to obtain the probabilities of various abnormal behaviors existing in the input image.

In one possible case, there may be multiple abnormal behaviors in the input image, and the input image is input into the attribute recognition model, and the attribute recognition model can output the probability of each abnormal behavior existing in the input image.

As an example, assuming that there are three abnormal behaviors in the input image, the attribute recognition model is used to perform attribute recognition on the input image, and three attribute values can be obtained, which are respectively used to indicate the probabilities of the three abnormal behaviors.

In an embodiment of the present application, when it is determined that the abnormal label to which the historical abnormal behavior displayed in the historical image taken before the input image belongs matches the label of the input image, an instruction message to continue collecting images can be issued.

It can be understood that the abnormal behavior in the input image is the same as the abnormal behavior in the historical image, and an instruction message to continue collecting images can be issued so that law enforcement personnel can continue to collect images at the location of the input image to determine whether abnormal behavior still exists in the image based on the re-collected images.

Step 303: Determine the attribute value of the target attribute indicating the historical abnormal behavior from the attribute value of at least one attribute.

Among them, the target attribute is used to indicate the historical abnormal behavior shown in the historical image.

In an embodiment of the present application, the attribute recognition model performs attribute recognition on the input image and obtains the attribute value of at least one attribute. Then, the probability of abnormal behavior that is the same as the historical abnormal behavior shown in the historical image can be determined from the attribute value of at least one attribute.

As an example, assuming that the historical abnormal behavior displayed in the historical image is abnormal behavior A, the attribute recognition model performs attribute recognition on the input image, and after obtaining the attribute values of the three abnormal behaviors, the attribute value corresponding to abnormal behavior A can be determined from the attribute values of the three abnormal behaviors.

Step 304: When the attribute value of the target attribute is less than or equal to the probability threshold, the cancellation process of the historical abnormal behavior is executed.

Among them, the cancellation process refers to the process of deleting abnormal behavior from the case management platform.

The probability threshold is a pre-set probability value.

In the embodiment of the present application, after determining the attribute value of the target attribute indicating the historical abnormal behavior output by the attribute recognition model, a comparison is performed to determine the magnitude relationship between the attribute value of the target attribute and the probability threshold.

In one possible case, it is determined that the attribute value of the target attribute is less than or equal to the probability threshold. In this case, the probability of the abnormal behavior indicated by the target attribute existing in the input image is low, and the verification process of the historical abnormal behavior can be executed.

In another possible case, it is determined that the attribute value of the target attribute is greater than the probability threshold. In this case, an instruction message to continue collecting images can be issued so that the law enforcement personnel can continue to collect images at the location of the input image to determine whether abnormal behavior still exists in the image based on the re-collected images.

In the embodiment of the present application, by querying the historical abnormal behaviors displayed in the historical images taken before the input image, when the abnormal label to which the historical abnormal behavior belongs does not match the label of the input image, the attribute recognition model is used to perform attribute recognition on the input image to obtain the attribute value of at least one attribute, and from the attribute value of at least one attribute, when the attribute value of the target attribute indicating the historical abnormal behavior is determined to be less than or equal to the probability threshold, the historical abnormal behavior cancellation process is executed. Thus, the attribute recognition model is used to perform attribute recognition on the input image to determine the abnormal behaviors present in the input image, thereby avoiding the situation where there are multiple abnormal behaviors in the input image, resulting in the failure of abnormal behavior cancellation.

As an example, as shown in FIG4 , FIG4 is an example diagram of an image recognition method provided in an embodiment of the present application.

As shown in FIG4 , after the collected input image is obtained, the input image can be input into the image classification model to determine the abnormal behavior of the input image. The historical abnormal behavior shown in the historical image taken before the input image is queried. When the abnormal label to which the historical abnormal behavior belongs does not match the label of the input image, the attribute recognition model is used to perform attribute recognition on the input image to obtain the attribute value of at least one attribute.

In one possible case, it is determined that the attribute value of the target attribute indicating the historical abnormal behavior is less than or equal to the probability threshold, and then the cancellation process of the historical abnormal behavior is executed.

In another possible case, based on the attribute value of at least one attribute output by the attribute recognition model, it is determined that the attribute value of the target attribute indicating historical abnormal behavior is greater than a probability threshold, and indication information for continuing to collect images is issued.

The image classification model performs classification processing on the input image, and when determining that the label of the input image is an abnormal label, determines that the input image exhibits abnormal behavior indicated by the abnormal label, and the abnormal behavior is not cancelled.

The image classification model classifies the input image and determines that the label of the input image is a background label. It is determined that the input image displays a background indicated by the background label and does not have abnormal behavior indicated by the abnormal label associated with the background label. Then, it is determined that the abnormal behavior in the input image has been closed.

In the above embodiment, it is introduced that an image classification model is used to classify input images. In order to train the image classification model, a large number of images need to be collected as training samples. However, the collected images include many abnormal labels that cannot be distinguished from the images. It takes a lot of manpower to label the training samples manually. The following describes how to train the image classification model in conjunction with Figure 5. Figure 5 is a flow chart of a training method for an image classification model provided in an embodiment of the present application.

As shown in FIG5 , the training method of the image classification model may include the following steps:

Step 501: Obtain a first sample set and a second sample set used in this round of training.

The training samples in the first sample set and the second sample set are used for training and testing the image classification model, respectively. Each training sample in the second sample set used for testing is a labeled image.

In the embodiment of the present application, the training samples in the first sample set and the second sample set can be images collected by law enforcement personnel using imaging equipment, or images collected by cameras installed on the street, or images obtained from a server, etc., which are not limited here.

After the images for training are acquired, the training samples may be divided into a first sample set and a second sample set by random grouping.

For example, after the training samples are acquired, 20% of the training samples may be divided into the first sample set, and 80% of the training samples may be divided into the second sample set.

In a possible case, the training samples in the first sample set and the second sample set used in the current round of training may be sample sets obtained after the previous round of training process is completed.

Step 502: Use the training samples in the first sample set to train the image classification model.

Step 503: Use the training samples in the second sample set to test the trained image classification model to obtain the predicted label and corresponding confidence of each training sample in the second sample set.

In an embodiment of the present application, after obtaining the first sample set and the second sample set used in this round of training, the training samples in the first sample set can be used to train the image classification model, and the model parameters of the image classification model can be adjusted so that the trained image classification model can recognize the label of the image.

In the present application, after the training of the image classification model using the training samples in the first sample set is completed, the trained image classification model can be tested using the training samples in the second sample set to obtain the predicted labels and corresponding confidences of each training sample in the second sample set output by the image classification model. The confidence is used to indicate the degree of match between the predicted labels of each training sample in the second sample set and the annotated labels of each training sample.

It can be understood that, the higher the confidence of each training sample in the second sample set output by the image classification model, the corresponding training sample is a correct sample, and the lower the confidence, the corresponding training sample is an incorrect sample.

Step 504: Move the target samples in the second sample set to the first sample set to obtain the first sample set and the second sample set used for the next round of training.

The target sample includes a training sample whose predicted label matches the labeled label and whose confidence is greater than a threshold confidence. The threshold confidence is a preset confidence value.

In an embodiment of the present application, the trained image classification model is tested using training samples in the second sample set. After obtaining the predicted labels and corresponding confidence levels of each training sample in the second sample set, the predicted labels can be matched with the labeled labels, and the training samples whose confidence levels are greater than the threshold confidence levels can be used as target samples.

It should be noted that the threshold confidence used in this round of training is greater than the threshold confidence used in the next round of training. The threshold confidence gradually decreases with the number of iterations of model training until the threshold confidence decreases to a preset threshold confidence. All training samples in the second sample set are moved to the first sample set, so that the image classification model can be trained using the training samples in the first sample set.

In the embodiment of the present application, after the target sample is determined from the second sample set, the target sample can be moved to the first sample set to obtain the first sample set and the second sample set used in the next round of training.

The training method of the image classification model of the embodiment of the present application obtains the first sample set and the second sample set used in the current round of training, uses the training samples in the first sample set to train the image classification model, uses the training samples in the second sample set to test the trained image classification model, obtains the predicted label and corresponding confidence of each training sample in the second sample set, matches the predicted label with the annotated label, and uses the training sample with confidence greater than the threshold confidence as the target sample, and moves the target sample to the first sample set to obtain the first sample set and the second sample set used in the next round of training. In this way, automatic iteration is achieved to clean up useless training samples, and the image classification model is trained using training samples with high confidence, which is conducive to improving the accuracy of the image classification model.

In one possible case, in the above step 503, when the trained image classification model is tested using the training samples in the second sample set, there may be training samples whose predicted labels do not match the annotated labels and whose confidence is greater than the threshold confidence. In order to avoid the correct samples with high confidence not being moved to the first sample set, the training samples whose predicted labels do not match the annotated labels and whose confidence is greater than the threshold confidence may be manually reviewed. This is described in detail below in conjunction with FIG6 , which is a flow chart of another training method for an image classification model provided in an embodiment of the present application.

As shown in FIG6 , the training method of the image classification model may further include the following steps:

Step 601, sending a prompt message.

The prompt information is used to prompt manual review of the training samples in the second sample set whose predicted labels do not match the annotated labels and whose confidence is greater than the threshold confidence. Manual review refers to the user manually reviewing the predicted labels and annotated labels of the training samples to determine whether the predicted labels match the annotated labels.

In an embodiment of the present application, after testing the trained image classification model using training samples in the second sample set, it is determined that there are training samples whose predicted labels do not match the annotated labels and whose confidence is greater than a threshold confidence level, and a prompt message can be sent to prompt manual review of the training samples whose predicted labels do not match the annotated labels and whose confidence is greater than the threshold confidence level.

Step 602: In response to a user review operation, the reviewed training samples are moved as target samples to the first sample set.

In one possible case, after the user reviews the training sample whose predicted label does not match the labeled label and whose confidence is greater than a threshold confidence, it is determined that the predicted label of the training sample does not match the labeled label, and the training sample may fail the manual review.

In another possible case, after the user reviews the training samples whose predicted labels do not match the annotated labels and whose confidence is greater than the threshold confidence, it is determined that the predicted labels of the training samples match the annotated labels, and the reviewed training samples can be moved as target samples to the first sample set.

Therefore, by manually reviewing the training samples in the second sample set whose predicted labels do not match the marked labels and whose confidence is greater than the threshold confidence, the reviewed training samples are moved to the first sample set as target samples to avoid screening out the correct samples with high confidence in the second sample set.

In the embodiment of the present application, the pre-sorted training samples can be input into the image classification model to obtain the predicted labels of each training sample, and the target matrix can be generated according to the predicted labels and the labeled labels, and the training samples whose predicted labels do not match the labeled labels can be selected as the samples for predicting the image classification model, so as to screen each training sample. The following is a detailed introduction in conjunction with Figure 7, which is a flow chart of another image classification model training method provided in the embodiment of the present application.

As shown in FIG. 7 , the training method of the image classification model may include the following steps:

Step 701: input multiple candidate samples into an image classification model to obtain a predicted label for each candidate sample.

The candidate samples may be images collected by law enforcement personnel using imaging equipment, images collected by cameras installed on the street, images obtained from a server, etc., and are not limited here. In addition, each candidate sample has been marked with an abnormal label.

In an embodiment of the present application, after obtaining multiple candidate samples, the multiple candidate samples can be input into an image classification model to determine a predicted label for each candidate sample based on an output of the image classification model.

Step 702: Generate a target matrix based on the predicted labels and labeled labels of multiple candidate samples.

Among them, the elements in the target matrix represent candidate samples that meet the annotation labels corresponding to the rows and the predicted labels corresponding to the columns.

It can be understood that after the image classification model is used to classify multiple candidate samples and the predicted labels of each candidate sample are determined, a target matrix can be generated according to the predicted labels and annotation labels of each candidate sample.

As an example, the rows of the target matrix correspond to the annotation labels, and the columns correspond to the predicted labels. Assuming that the annotation labels of the candidate samples corresponding to the first row and first column in the target matrix do not match the predicted labels, the elements of the first row and first column of the target matrix are 0. If the annotation labels of the candidate samples corresponding to the second row and first column in the target matrix are the same as the predicted labels, the elements of the second row and first column of the target matrix are 1.

Step 703, obtaining the target element from the target matrix.

Among them, the target element is the element whose annotation label corresponding to the row does not match the predicted label corresponding to the column.

In an embodiment of the present application, after generating a target matrix based on predicted labels and labeled labels of multiple candidate samples, elements whose labeled labels corresponding to rows do not match predicted labels corresponding to columns can be obtained from the target matrix as target elements.

It can be understood that candidate samples represented by elements in the target matrix whose labeled categories corresponding to the rows do not match the predicted categories corresponding to the columns may be training samples with low confidence.

Step 704: Generate a second sample set used in the first round of training based on the candidate samples represented by the target element.

In an embodiment of the present application, after obtaining the target elements whose labeled labels corresponding to the rows do not match the predicted labels corresponding to the columns from the target matrix, a second sample set used in the first round of training can be generated based on the candidate samples represented by the target elements for testing the trained image classification model.

The training method of the image classification model of the embodiment of the present application inputs multiple candidate samples into the image classification model to obtain the predicted label of each candidate sample, generates a target matrix based on the predicted labels and labeled labels of the multiple candidate samples, obtains the target element from the target matrix, and generates a second sample set used in the first round of training based on the candidate samples represented by the target element. Thus, the candidate samples whose predicted labels and labeled labels do not match are used as the training samples that need to be screened out at present, so as to reduce the workload of manual review.

As an example, as shown in Figure 8, assuming that sample set A and sample set B are initial samples, sample set A and sample set B can be input into the image classification model to obtain the predicted labels of each training sample, and a target matrix is generated according to the predicted labels and annotation labels of each training sample. From the target matrix, the target elements whose annotation labels corresponding to the rows and the predicted labels corresponding to the columns do not match are obtained to generate the second sample set used by the first round of training image classification model.

The image classification model is trained using the first sample set, and the training samples in the second sample set are used to test the trained image classification model to obtain the predicted labels and corresponding confidence levels of each training sample in the second sample set. The predicted labels are matched with the labeled labels, and the training samples whose confidence levels are greater than the threshold confidence level are moved to the first sample set to obtain the first sample set and the second sample set used for the next round of training.

In the embodiment of the present application, after the training of the image classification model is completed, the model can be evaluated. If the evaluation result is stable, the trained image classification model can be used to recognize the image.

In order to implement the above embodiments, the present application proposes an image recognition device.

FIG. 9 is a schematic diagram of the structure of an image recognition device provided in an embodiment of the present application.

As shown in FIG. 9 , the image recognition device 900 may include: an acquisition module 910 , an input module 920 , a first determination module 930 , and a second determination module 940 .

The acquisition module 910 is used to acquire a trained image classification model and acquire a collected input image.

The input module 920 is used to classify the input image using an image classification model to determine the label of the input image from at least one abnormal label and background labels associated with each abnormal label.

The first determination module 930 is configured to determine, when the label of the input image is an abnormal label, that the input image exhibits an abnormal behavior indicated by the abnormal label.

The second determination module 940 is used to determine, when the label of the input image is a background label, that the input image shows a background indicated by the background label and does not have an abnormal behavior indicated by an abnormal label associated with the background label.

As a possible scenario, the image recognition device 900 may further include:

A query module, used to query historical abnormal behaviors displayed in historical images taken before the input image;

A recognition module, configured to use an attribute recognition model to perform attribute recognition on the input image when the abnormal label to which the historical abnormal behavior belongs does not match the label of the input image, so as to obtain an attribute value of at least one attribute, wherein the attribute is used to indicate the abnormal behavior, and the attribute value is used to indicate the probability of the existence of the abnormal behavior;

A third determination module, configured to determine an attribute value of a target attribute indicating historical abnormal behavior from the attribute value of at least one attribute;

The processing module is used to execute the cancellation process of historical abnormal behavior when the attribute value of the target attribute is less than or equal to the probability threshold.

As another possible situation, the image recognition device 900 may further include:

The first sending module is used to send instruction information for continuing to collect images when the abnormal label to which the historical abnormal behavior belongs matches the label of the input image.

As another possible situation, the image recognition device 900 may further include:

The second sending module is used to send instruction information to continue collecting images when the attribute value of the target attribute is greater than the probability threshold.

It should be noted that the above explanations and descriptions of the embodiment of the image recognition method are also applicable to the image recognition device and will not be repeated here.

The image recognition device of the embodiment of the present application uses a trained image classification model to classify the collected input image to determine the label of the input image from at least one abnormal label and the background labels associated with each abnormal label; when the label of the input image is an abnormal label, it is determined that the input image shows abnormal behavior indicated by the abnormal label; when the label of the input image is a background label, it is determined that the input image shows the background indicated by the background label, and does not have abnormal behavior indicated by the abnormal label associated with the background label. Therefore, after determining the label of the input image through the image classification model, it is determined whether there is abnormal behavior in the input image based on the content displayed in the input image, which effectively solves the technical problems of the need for manual review in the process of handling urban management violation cases in the related technology, and there are high review costs and long cycles, etc., which greatly reduces the workload of case reviewers and significantly optimizes processing efficiency.

In order to implement the above embodiment, the present application proposes a training device for an image classification model.

FIG10 is a schematic diagram of the structure of a training device for an image classification model provided in an embodiment of the present application.

Among them, the image classification model is used to execute the image recognition method described in the above embodiment.

As shown in FIG. 10 , the training device 1000 for the image classification model may include: an acquisition module 1010 , a training module 1020 , a testing module 1030 and a first movement module 1040 .

The acquisition module 1010 is used to acquire the first sample set and the second sample set used in this round of training.

The training module 1020 is used to train the image classification model using the training samples in the first sample set.

The testing module 1030 is used to test the trained image classification model using the training samples in the second sample set to obtain the predicted label and the corresponding confidence of each training sample in the second sample set.

The first moving module 1040 is used to move the target sample in the second sample set to the first sample set to obtain the first sample set and the second sample set used for the next round of training; wherein the target sample includes a training sample whose predicted label matches the marked label and whose confidence is greater than a threshold confidence.

As a possible situation, the threshold confidence adopted in this round of training is greater than the threshold confidence adopted in the next round of training.

As another possible situation, the image classification model training device 1000 may further include:

A sending module, used for sending prompt information, wherein the prompt information is used for prompting manual review of the training samples in the second sample set, in which the predicted label does not match the marked label and the confidence is greater than a threshold confidence;

The second moving module is used to move the reviewed training samples as the target samples to the first sample set in response to the user's review operation.

As another possible situation, the image classification model training device 1000 may further include:

An input module, used to input multiple candidate samples into the image classification model to obtain a predicted label of each candidate sample;

A first generating module is used to generate a target matrix according to the predicted labels and labeled labels of the plurality of candidate samples; wherein the elements in the target matrix represent candidate samples that meet the labeled labels corresponding to the rows and meet the predicted labels corresponding to the columns;

An acquisition module, used to acquire a target element from the target matrix, wherein the target element is an element whose labeled label corresponding to a row does not match the predicted label corresponding to a column;

The second generating module is used to generate the second sample set used in the first round of training according to the candidate samples represented by the target element.

It should be noted that the above explanation of the embodiment of the training method for the image classification model is also applicable to the training device for the image classification model, and will not be repeated here.

The training device of the image classification model of the embodiment of the present application obtains the first sample set and the second sample set used in the current round of training, uses the training samples in the first sample set to train the image classification model, uses the training samples in the second sample set to test the trained image classification model, obtains the predicted label and corresponding confidence of each training sample in the second sample set, matches the predicted label with the annotated label, and uses the training sample with confidence greater than the threshold confidence as the target sample, and moves the target sample to the first sample set to obtain the first sample set and the second sample set used in the next round of training. In this way, automatic iteration is achieved to clean up useless training samples, and the image classification model is trained using training samples with high confidence, which is conducive to improving the accuracy of the image classification model.

In order to implement the above embodiment, the present application proposes an electronic device, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image recognition method described in the above embodiment, or execute the model training method described in the above embodiment.

In order to implement the above embodiments, the present application proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the image recognition method described in the above embodiments, or to execute the model training method described in the above embodiments.

In order to implement the above embodiments, the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the image recognition method described in the above embodiments, or executes the model training method described in the above embodiments.

According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

Figure 11 shows a schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and/or required herein.

As shown in FIG. 11 , the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 to a RAM (Random Access Memory) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I/O (Input/Output) interface 1105 is also connected to the bus 1104.

A number of components in the device 1100 are connected to the I/O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

The computing unit 1101 may be a variety of general and/or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as an image recognition method, or a model training method. For example, in some embodiments, the image recognition method, or the model training method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and/or installed on the device 1100 via the ROM 1102 and/or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute an image recognition method, or a training method for an image classification model, in any other appropriate manner (eg, by means of firmware).

Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and/or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions/operations specified in the flow chart and/or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning/deep learning, big data processing technology, knowledge graph technology, and other major directions.

It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims (18)

1. An image recognition method, comprising:

acquiring a trained image classification model and acquiring an acquired input image;

classifying the input image by adopting the image classification model to determine the label of the input image from at least one abnormal label and the background label associated with each abnormal label;

determining that the input image has abnormal behaviors indicated by the abnormal labels or that the input image has the background indicated by the background labels according to the labels of the input image;

wherein the method further comprises:

inquiring historical abnormal behaviors displayed in a historical image shot before the input image;

when the abnormal label of the history abnormal behavior is not matched with the label of the input image, carrying out attribute identification on the input image by adopting an attribute identification model to obtain an attribute value of at least one attribute, wherein the attribute is used for indicating the abnormal behavior, and the attribute value is used for indicating the probability of the abnormal behavior;

determining an attribute value of a target attribute indicating the historical abnormal behavior from among the attribute values of the at least one attribute;

And executing the verification process of the historical abnormal behavior under the condition that the attribute value of the target attribute is smaller than or equal to a probability threshold value.

2. The image recognition method according to claim 1, wherein the determining that the input image has an abnormal behavior indicated by the abnormal label or that the input image has a background indicated by the background label according to the label of the input image comprises:

determining that the input image shows abnormal behaviors indicated by the abnormal label under the condition that the label of the input image is the abnormal label;

and under the condition that the label of the input image is the background label, determining that the background indicated by the background label is shown in the input image and the abnormal behavior indicated by the abnormal label associated with the background label is not shown.

3. The image recognition method according to claim 1, wherein after the querying the historical abnormal behavior exhibited in the historical image captured before the input image, further comprising:

and sending out indication information for continuously collecting the image under the condition that the abnormal label of the history abnormal behavior is matched with the label of the input image.

4. The image recognition method according to claim 1, wherein after the querying the historical abnormal behavior exhibited in the historical image captured before the input image, further comprising:

and sending out indication information for continuing to collect the image under the condition that the attribute value of the target attribute is larger than a probability threshold value.

5. A training method of an image classification model for performing the image recognition method according to any one of claims 1 to 4, the training method comprising:

acquiring a first sample set and a second sample set adopted by the round of training;

training an image classification model by adopting training samples in the first sample set;

testing the trained image classification model by adopting training samples in the second sample set to obtain a prediction label and corresponding confidence coefficient of each training sample in the second sample set;

moving target samples in the second sample set to the first sample set to obtain a first sample set and a second sample set adopted by the next training round; the target sample comprises a training sample, wherein the prediction label is matched with the labeling label, and the confidence coefficient of the training sample is larger than the threshold confidence coefficient.

6. The model training method of claim 5, wherein the threshold confidence level employed by the present round of training is greater than the threshold confidence level employed by the next round of training.

7. The model training method according to claim 5, wherein the testing the trained image classification model using the training samples in the second sample set, after obtaining the prediction labels and the confidence of each training sample in the second sample set, further comprises:

the method comprises the steps of sending prompt information, wherein the prompt information is used for carrying out manual rechecking on training sample prompts, in the second sample set, of which the prediction labels are not matched with the labeling labels and the confidence coefficient is greater than a threshold confidence coefficient;

and responding to the user review operation, and moving the reviewed training sample as the target sample into the first sample set.

8. The model training method of any of claims 5-7, wherein the method further comprises:

inputting a plurality of candidate samples into the image classification model to obtain a prediction label of each candidate sample;

generating a target matrix according to the prediction labels and the labeling labels of the candidate samples; the element characterization in the target matrix accords with the labeling label corresponding to the row and accords with the candidate sample of the prediction label corresponding to the column;

Acquiring target elements from the target matrix, wherein the target elements are elements of which the labeling labels corresponding to the rows are not matched with the prediction labels corresponding to the columns;

and generating the second sample set adopted by the first training according to the candidate samples represented by the target elements.

9. An image recognition apparatus comprising:

the acquisition module is used for acquiring the trained image classification model and acquiring an acquired input image;

the input module is used for classifying the input image by adopting the image classification model so as to determine the label of the input image from at least one abnormal label and the background labels associated with the abnormal labels;

the determining module is used for determining that the input image has abnormal behaviors indicated by the abnormal labels or determining that the input image has the background indicated by the background labels according to the labels of the input image;

wherein the apparatus further comprises:

the query module is used for querying historical abnormal behaviors displayed in the historical images shot before the input image;

the identification module is used for carrying out attribute identification on the input image by adopting an attribute identification model under the condition that the abnormal label to which the historical abnormal behavior belongs is not matched with the label of the input image so as to obtain an attribute value of at least one attribute, wherein the attribute is used for indicating the abnormal behavior, and the attribute value is used for indicating the probability of the abnormal behavior;

A third determining module configured to determine, from among attribute values of the at least one attribute, an attribute value of a target attribute indicating the historical abnormal behavior;

and the processing module is used for executing the verification flow of the historical abnormal behavior under the condition that the attribute value of the target attribute is smaller than or equal to the probability threshold value.

10. The image recognition device of claim 9, wherein the determination module is further configured to:

determining that the input image shows abnormal behaviors indicated by the abnormal label under the condition that the label of the input image is the abnormal label;

and under the condition that the label of the input image is the background label, determining that the background indicated by the background label is shown in the input image and the abnormal behavior indicated by the abnormal label associated with the background label is not shown.

11. The image recognition device of claim 10, wherein the device further comprises:

the first sending module is used for sending out indication information for continuously collecting images under the condition that the abnormal label of the history abnormal behavior is matched with the label of the input image.

12. The image recognition device of claim 10, wherein the device further comprises:

And the second sending module is used for sending out indication information for continuously collecting the image under the condition that the attribute value of the target attribute is larger than a probability threshold value.

13. Training apparatus for an image classification model for performing an image recognition method according to any one of claims 1-5, the training apparatus comprising:

the acquisition module is used for acquiring a first sample set and a second sample set adopted by the round of training;

the training module is used for training the image classification model by adopting training samples in the first sample set;

the test module is used for testing the trained image classification model by adopting the training samples in the second sample set to obtain the prediction labels and the corresponding confidence degrees of the training samples in the second sample set;

the first moving module is used for moving the target samples in the second sample set into the first sample set so as to obtain a first sample set and a second sample set adopted by the next training round; the target sample comprises a training sample, wherein the prediction label is matched with the labeling label, and the confidence coefficient of the training sample is larger than the threshold confidence coefficient.

14. The training apparatus of model of claim 13 wherein the threshold confidence level employed by the present round of training is greater than the threshold confidence level employed by the next round of training.

15. The training device of a model of claim 13, wherein the device further comprises:

the sending module is used for sending prompt information, wherein the prompt information is used for carrying out manual rechecking on training sample prompts, in the second sample set, of which the prediction labels are not matched with the labeling labels and the confidence coefficient is greater than a threshold confidence coefficient;

and the second moving module is used for responding to the user checking operation and moving the checked training sample as the target sample into the first sample set.

16. The training device of a model according to any of claims 13-15, wherein the device further comprises:

the input module is used for inputting a plurality of candidate samples into the image classification model to obtain a prediction label of each candidate sample;

the first generation module is used for generating a target matrix according to the prediction labels and the labeling labels of the plurality of candidate samples; the element characterization in the target matrix accords with the labeling label corresponding to the row and accords with the candidate sample of the prediction label corresponding to the column;

The acquisition module is used for acquiring target elements from the target matrix, wherein the target elements are elements of which the labeling labels corresponding to the rows are not matched with the prediction labels corresponding to the columns;

and the second generation module is used for generating the second sample set adopted by the first round of training according to the candidate samples represented by the target elements.

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image recognition method of any one of claims 1-4 or to perform the training method of the model of any one of claims 5-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the image recognition method according to any one of claims 1-4 or to perform the training method of the model according to any one of claims 5-8.