CN115577768A - Semi-supervised model training method and device - Google Patents

Abstract

The method comprises the steps of performing cross-training by adopting a first model and a second model with the same structure based on labeled data and label-free data processed by different data enhancement methods, determining unsupervised loss according to the consistency of the first model and the second model on prediction results of label-free loss processed by different enhancement methods, realizing noise weighting and filtering, inhibiting noise influence caused by pseudo labels in the semi-supervised model training process, and improving the prediction precision and generalization of the models.

Description

Semi-supervised model training method and device

technical field

The present application relates to the field of artificial intelligence, in particular to a semi-supervised model training method and device.

Background technique

Semi-supervised (Semi-supervised) learning refers to the method of using labeled samples and unlabeled samples to train the deep learning network model (this application is also referred to as "model"). Through semi-supervised learning, it can effectively reduce the number of labeled samples. Quantity, reducing the cost of model training.

In semi-supervised learning, a Teacher model is usually trained with smaller-scale labeled data, and then the Teacher model is used to predict pseudo-labels for larger-scale unlabeled data. Finally, the mixture of pseudo-labeled data and labeled data is used as the Student model. Training data for iterative training of the Student model.

Since the pseudo-label data contains a lot of noise, if the model is overfitted to the noise, it will lead to wrong predictions of the model, and the performance of the trained model will be poor.

Contents of the invention

The present application provides a semi-supervised model training method and device for improving model performance, including prediction accuracy and generalization.

The first aspect of the present application provides a semi-supervised model training method, including: inputting the first enhanced data into the first model to obtain the first prediction result of the first enhanced data, and inputting the first enhanced data into the first The second model obtains the second prediction result of the first enhanced data, the first enhanced data is obtained by processing the first unlabeled data according to the first data enhancement method, and the second model has the same structure as the first model and different initialization parameters; input the second enhanced data into the first model to obtain the third prediction result of the second enhanced data, and input the second enhanced data into the second model to obtain the second The fourth prediction result of the enhanced data, the second enhanced data is obtained by processing the first unlabeled data according to the second data enhancement method, the first data enhancement method is different from the second data enhancement method; according to the The consistency of the first prediction result and the fourth prediction result determines the first unsupervised loss; the second unsupervised loss is determined according to the consistency of the second prediction result and the third prediction result; the first label data Input the first model to obtain a fifth prediction result of the first labeled data; determine a first supervised loss based on the fifth predicted result and the label of the first labeled data; based on the first unsupervised loss , the second unsupervised loss and the first supervised loss update parameters of the first model.

The semi-supervised model training method provided by this application allows the network to input labeled data and unlabeled data at the same time, mix and enhance unlabeled data through different data enhancement methods, and use the first model and the second model of the same structure for cross interaction Training, and determine the unsupervised loss based on the consistency of the prediction results of the two models, which can realize noise weighting and filtering, and can suppress the noise influence caused by pseudo-labels during the training process of the semi-supervised model, and improve the generalization of the model.

In a possible implementation of the first aspect, the first model includes an image semantic segmentation model; the first data enhancement method is a non-color gamut enhancement method, and the non-color gamut enhancement includes flip transformation and mirror transformation . Translation transformation or scale transformation; the second data enhancement method is a color gamut enhancement method, and the color gamut enhancement method includes shading transformation, contrast transformation or image Gaussian noise enhancement.

The semi-supervised model training method provided by this application, for the training of the semantic segmentation model, can adopt two different data enhancement methods for unlabeled data, including the non-color gamut enhancement method and the color gamut enhancement method, which are processed based on different data enhancement methods Model training can increase the generalization performance of the model and improve the robustness of the model.

In another possible implementation, the first model is an image recognition model, the first data enhancement method is a non-color gamut enhancement method, and the non-color gamut enhancement includes flip transformation, mirror transformation, translation transformation or scale transformation ; The second data enhancement method is a color gamut enhancement method, and the color gamut enhancement method includes shading transformation, contrast transformation or image Gaussian noise enhancement.

In a possible implementation manner of the first aspect, the first model includes a semantic segmentation model of an image; the second enhanced data is obtained by processing the first unlabeled data according to a second data enhancement method, including: the The second enhanced data is obtained through copy-paste enhancement according to the first unlabeled data and the first labeled data.

In the semi-supervised model training method provided by the present application, the second enhanced data is copy-pasted and enhanced according to the first unlabeled data and the first labeled data, and the connection between labeled data and unlabeled data can be used to reduce the difference. The problem of excessive noise caused by inter-domain differences between data sets can effectively improve the inter-domain adaptability of the model.

In a possible implementation of the first aspect, the first model is a target detection model; the first data enhancement method includes: one of flip transformation, translation transformation, scale transformation, rotation transformation, scaling transformation, or Various; the second data enhancement method includes: one or more of flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation.

In the semi-supervised model training method provided by this application, for the target detection model, considering that the detection result is usually a marked box, the data is enhanced by flip transformation, translation transformation, scale transformation, rotation transformation or scaling transformation to improve the performance of the model .

In a possible implementation manner of the first aspect, before inputting the first annotation data into the first model, the method further includes: performing enhancement processing on the first annotation data to obtain enhanced first annotation data, The enhanced first labeled data is used to input the first model to obtain the fifth prediction result.

The semi-supervised model training method provided in the present application can also enhance the first labeled data to further enhance the generalization of the first model.

In a possible implementation manner of the first aspect, the first labeled data is input into the second model, and the sixth predicted result of the first labeled data is obtained; and determined according to the sixth predicted result and the label of the first labeled data a second supervised loss; updating parameters of the second model based on the first unsupervised loss, the second unsupervised loss, and the second supervised loss.

The semi-supervised model training method provided in the present application can train the second model synchronously, obtain the second prediction result and the fourth prediction result based on the updated second model, and can improve the reliability of the prediction.

In a possible implementation manner of the first aspect, the inputting the first enhanced data into the first model obtains the first prediction result of the first enhanced data, and inputting the first enhanced data into the second model obtains The second prediction result of the first enhanced data includes: inputting the first enhanced data into a preset feature extraction network to obtain first feature data; inputting the first feature data into the first model to obtain the a first prediction result, and inputting the first feature data into the second model to obtain the second prediction result.

In another possible implementation manner, the inputting the second enhanced data into the first model obtains a third prediction result of the second enhanced data, and inputting the second enhanced data into the second The model obtaining the fourth prediction result of the second enhanced data includes: inputting the second enhanced data into the preset feature extraction network to obtain second feature data; inputting the second feature data into the first A model obtains the third prediction result, and the second characteristic data is input into the second model to obtain the fourth prediction result.

The semi-supervised model training method provided by this application extracts the feature data of the enhanced data through the feature extraction network, and then distributes the feature data into the first model and the second model, which can reduce the amount of data processing, data storage, and training time. Improve training efficiency. In addition, according to the feature data of the same parameters, different model segmentation heads are input, and model training is performed based on data from two different perspectives, which can further improve the generalization of the model.

In a possible implementation manner of the first aspect, the first model is an image semantic segmentation model; the determining the first unsupervised loss according to the consistency between the first prediction result and the fourth prediction result includes : determine the first pseudo label corresponding to the first prediction result, if the first prediction probability corresponding to the first pixel position in the fourth prediction result is greater than the second prediction probability of the first pixel position in the first prediction result , then the weight of the first pseudo-label is 1, and if the first prediction probability is less than or equal to the second prediction probability, the weight of the first pseudo-label is 0; according to the first prediction The results and first pseudo-labels determine the first unsupervised loss.

The semi-supervised model training method provided in this application can determine the pseudo-label through probability comparison, which can improve the reliability of the pseudo-label and improve the prediction accuracy of the training model.

The second aspect of the present application provides a semi-supervised model training device, including: a processing unit, configured to input the first enhanced data into the first model to obtain the first prediction result of the first enhanced data, and the first enhanced data An enhanced data is input into the second model to obtain the second prediction result of the first enhanced data, the first enhanced data is obtained by processing the first unlabeled data according to the first data enhancement method, and the second model and the first The models have the same structure and different initialization parameters; the processing unit is further configured to input second enhanced data into the first model to obtain a third prediction result of the second enhanced data, and input the second enhanced data The enhanced data is input into the second model to obtain a fourth prediction result of the second enhanced data, the second enhanced data is obtained by processing the first unlabeled data according to the second data enhancement method, and the first data enhancement method Different from the second data enhancement method; the determining unit is used to determine the first unsupervised loss according to the consistency of the first prediction result and the fourth prediction result; the determination unit is also used to determine the first unsupervised loss according to the The consistency between the second prediction result and the third prediction result determines a second unsupervised loss; the processing unit is further configured to input the first labeled data into the first model, and obtain the first labeled data of the first labeled data. Five prediction results; the determination unit is further configured to determine a first supervised loss based on the fifth prediction result and the label of the first labeled data; an update unit is configured to based on the first unsupervised loss, the A second unsupervised loss and the first supervised loss update parameters of the first model.

In a possible implementation of the second aspect, the first model includes an image semantic segmentation model; the first data enhancement method is a non-color gamut enhancement method, and the non-color gamut enhancement includes flip transformation and mirror transformation . Translation transformation or scale transformation; the second data enhancement method is a color gamut enhancement method, and the color gamut enhancement method includes shading transformation, contrast transformation or image Gaussian noise enhancement.

In a possible implementation manner of the second aspect, the first model includes a semantic segmentation model of an image; the second enhanced data is obtained by processing the first unlabeled data according to a second data enhancement method, including: the The second enhanced data is obtained through copy-paste enhancement according to the first unlabeled data and the first labeled data.

In a possible implementation manner of the second aspect, the first model is a target detection model; the first data enhancement method includes: one of flip transformation, translation transformation, scale transformation, rotation transformation, scaling transformation, or Various; the second data enhancement method includes: one or more of flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation.

In a possible implementation manner of the second aspect, the processing unit is further configured to: perform enhancement processing on the first annotation data to obtain enhanced first annotation data, and use the enhanced first annotation data to and inputting the first model to obtain the fifth prediction result.

In a possible implementation manner of the second aspect, the processing unit is further configured to input the first label data into the second model, and obtain a sixth prediction result of the first label data; the determining unit, It is further configured to determine a second supervised loss based on the sixth prediction result and the label of the first labeled data; A supervised loss updates the parameters of the second model.

In a possible implementation manner of the second aspect, the processing unit is specifically configured to: input the first enhanced data into a preset feature extraction network to acquire first feature data; input the first feature data into the The first model obtains the first prediction result, and the first feature data is input into the second model to obtain the second prediction result.

In a possible implementation manner of the second aspect, the first model is an image semantic segmentation model; the processing unit is specifically configured to: determine the first pseudo-label corresponding to the first prediction result, if the fourth prediction The first predicted probability corresponding to the first pixel position in the result is greater than the second predicted probability of the first pixel position in the first predicted result, the weight of the first pseudo-label is 1, if the first If the predicted probability is less than or equal to the second predicted probability, the weight of the first pseudo-label is 0; and the first unsupervised loss is determined according to the first predicted result and the first pseudo-label.

The third aspect of the present application provides a semi-supervised model training device, including: a memory, computer readable instructions are stored in the memory; a processor connected to the memory, the computer readable instructions are executed by the processor During execution, the semi-supervised model training device is made to implement the method described in any one of the above first aspect and various possible implementation manners.

The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are run on a computer, the computer executes any one of the above-mentioned first aspect and various possible implementation manners. method described in the item.

The fifth aspect of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer executes the above-mentioned first aspect and various possible Implement the method described in any one of the manners.

The sixth aspect of the present application provides a chip, including a processor. The processor is used to read and execute the computer program stored in the memory, so as to execute the method in any possible implementation manner of any aspect above. Optionally, the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or wires. Further optionally, the chip further includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and/or information to be processed, and the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface. The communication interface may be an input-output interface.

Wherein, the technical effect brought by the second aspect, the third aspect, the fourth aspect, the fifth aspect or the sixth aspect and any one of the implementation methods can refer to the technical effect brought by the corresponding implementation method in the first aspect, I won't repeat them here.

The semi-supervised model training method provided by this application uses different data enhancement methods to perform mixed enhancement on unlabeled data, and uses the first model and the second model of the same structure for cross-training, and predicts the results according to the consistency of the two models Deterministic unsupervised loss can achieve noise weighting and filtering, which can suppress the noise impact caused by pseudo-labels during semi-supervised model training and improve model performance, including prediction accuracy and generalization.

Description of drawings

Fig. 1 is a schematic diagram of an architecture of a semi-supervised model training method;

FIG. 2 is a schematic diagram of a system architecture provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of an embodiment of a semi-supervised model training method in the embodiment of the present application;

4 is a schematic diagram of the architecture of a semi-supervised model training device in the embodiment of the present application;

Fig. 5 is a schematic diagram of obtaining semi-supervised loss in the embodiment of the present application;

6 is a schematic diagram of another embodiment of the semi-supervised model training method in the embodiment of the present application;

Fig. 7 is a schematic diagram of an embodiment of a semi-supervised model training device in the embodiment of the present application;

Fig. 8 is a schematic diagram of another embodiment of the semi-supervised model training device in the embodiment of the present application.

detailed description

The present application provides a semi-supervised model training method and device for improving model generalization.

Embodiments of the present application are described below in conjunction with the accompanying drawings. Apparently, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Those of ordinary skill in the art know that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to the expressly listed Instead, other steps or modules not explicitly listed or inherent to the process, method, product or apparatus may be included. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time/logic sequence indicated by the naming or numbering. The execution order of the technical purpose is changed, as long as the same or similar technical effect can be achieved.

Semantic segmentation is a popular direction in computer vision and digital image processing. As a basic component of many scene understanding systems, it divides images or videos into multiple parts or objects, and is widely used in robot navigation, intelligent video surveillance, safe cities, In many fields such as medical image analysis, assisted driving, and augmented reality, it is of great practical significance to reduce the consumption of human capital through computer vision. Therefore, semantic segmentation has become a research hotspot in theory and application in recent years. Due to the wide application of deep learning, semantic segmentation algorithms have developed relatively rapidly.

Embodiments of the present application are described below in conjunction with the accompanying drawings.

The development of deep learning networks needs to rely on large-scale data samples and finely labeled labels. Usually, manual labeling requires a lot of human and financial resources, which greatly limits the application of semantic segmentation algorithms. In actual scenarios, limited labeled samples will lead to overfitting of deep learning network training and the inability to effectively distinguish samples of different categories, which will eventually lead to wrong predictions of deep learning network. Expensive labeling costs, a large amount of manpower input and a long algorithm development cycle are unfavorable for commercial applications. Therefore, semi-supervised learning aims to train the deep learning network through a small amount of labeled data combined with a large amount of unlabeled data, thereby improving the generalization performance of the deep learning network, effectively mining the relationship between labeled data and unlabeled data, and enhancing Stability and Robustness of Deep Learning Networks in Practical Applications. Please refer to Figure 1, which is a schematic diagram of the architecture of the semi-supervised model training method. Obtaining supervised loss based on labeled data and unsupervised loss training model obtained from unlabeled data can reduce the demand for labeled data and reduce training costs compared with supervised training methods.

Semantic segmentation is a popular direction in computer vision and digital image processing. As a basic component of many scene understanding systems, it divides images or videos into multiple parts or objects, and is widely used in robot navigation, intelligent video surveillance, safe cities, In many fields such as medical image analysis, assisted driving, and augmented reality, it is of great practical significance to reduce the consumption of human capital through computer vision. Therefore, semantic segmentation has become a research hotspot in theory and application in recent years. Due to the wide application of deep learning, semantic segmentation algorithms have developed relatively rapidly. The semantic segmentation network mentioned in this application refers to the semantic segmentation network based on deep learning.

In the traditional semi-supervised training method, since the pseudo-label data contains a lot of noise, the model will be introduced into a large amount of noisy data for training in the later stages of the iteration. If the model is overfitted to the noise, the generalization of the model will be poor.

Using the method of dynamic joint training of two semantic segmentation networks, the weight of the loss function is dynamically updated by comparing the output results of the two semantic segmentation networks, and the pseudo-labels are evaluated using the multi-model voting mechanism by using the prediction inconsistency between the two semantic segmentation networks. fix. This method can improve the generalization of the model to a certain extent, but it is still insufficient. Moreover, the multi-round iteration mechanism will take a long time to train and consume a large amount of storage space when the amount of data is relatively large.

Embodiments of the present application provide a semi-supervised model training method and device. This method can be applied in areas that require semantic segmentation such as unmanned driving, safe cities, and mobile terminals, as well as mobile phone photography or security scenarios.

For example, in semantic segmentation of autonomous driving, semantic segmentation plays a very important role in composition positioning, road feature analysis, etc. Language needs higher semantic segmentation results to provide reliable decision-making for the backend. Using the semi-supervised model training method provided by this application can effectively reduce the demand for labeled data, and at the same time, the problem of degradation in segmentation performance caused by distribution differences between different domains can be significantly alleviated.

Semantic segmentation is inseparable from various features such as blurring the background and picking portraits of mobile phones. However, for users, the shooting scenes, equipment, and environments are very different, so targeted update and optimization of the segmentation system performance It is very necessary and urgent. Using the semi-supervised model training method provided by this application can improve the accuracy of the algorithm and improve the generalization of the model without increasing the labeled data.

The system architecture provided by the embodiment of the present application is introduced below.

Referring to FIG. 2 , an embodiment of the present application provides a system architecture 200 . As shown in system architecture 200, data collection device 260 may be used to collect training data. After the data acquisition device 260 collects the training data, the training data is stored in the database 230 , and the training device 220 obtains the target model/rule 201 based on training data maintained in the database 230 .

The following describes how the training device 220 obtains the target model/rule 201 based on the training data. Exemplarily, the training device 220 outputs the corresponding predicted label for multiple frames of sample images, and calculates the loss between the predicted label and the original label of the sample, and updates the classification network based on the loss until the predicted label is close to the original label of the sample. The difference between the label or the predicted label and the original label is less than a threshold, thereby completing the training of the target model/rule 201 . The label may be a label box involved in this embodiment of the present application. For a detailed description, see the training method in the following text.

The target model/rule 201 in the embodiment of the present application may specifically be a neural network. It should be noted that, in practical applications, the training data maintained in the database 230 may not all be collected by the data collection device 260, but may also be received from other devices. The collected data can be marked and enhanced to obtain the training data. In addition, it should be noted that the training device 220 does not necessarily perform the training of the target model/rules 201 based entirely on the training data maintained by the database 230, and it is also possible to obtain training data from the cloud or other places for model training. Limitations of the Examples.

The target model/rule 201 trained according to the training device 220 can be applied to different systems or devices, such as the execution device 210 shown in FIG. Laptop, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR), vehicle terminal, TV, etc., can also be a server or cloud, etc. In FIG. 2, the execution device 210 is configured with a transceiver 212, which may include an input/output (I/O) interface or other wireless or wired communication interfaces, etc., for data interaction with external devices. , taking the I/O interface as an example, the user can input data to the I/O interface through the client device 240 .

When the execution device 210 preprocesses the input data, or in the execution device 210 computing module 111 executes calculations and other related processing, the execution device 210 can call the data, codes, etc. in the data storage system 250 for corresponding processing , and the correspondingly processed data and instructions may also be stored in the data storage system 250 .

Finally, the I/O interface 212 returns the processing result to the client device 240, thereby providing it to the user.

It is worth noting that the training device 220 can generate corresponding target models/rules 201 based on different training data for different goals or different tasks, and the corresponding target models/rules 201 can be used to achieve the above goals or complete above tasks, thereby providing the desired result to the user.

In the case shown in FIG. 2 , the user can manually specify the input data, and the manual specification can be operated through the interface provided by the transceiver 212 . In another case, the client device 240 can automatically send the input data to the transceiver 212 . If the client device 240 is required to automatically send the input data to obtain the user's authorization, the user can set the corresponding authority in the client device 240 . The user can view the results output by the execution device 210 on the client device 240, and the specific presentation form may be specific ways such as display, sound, and action. The client device 240 can also be used as a data collection terminal, collecting the input data input to the transceiver 212 as shown in the figure and the output results of the output transceiver 212 as new sample data, and storing them in the database 230 . Of course, the client device 240 may not be used for collection, but the transceiver 212 directly stores the input data input into the transceiver 212 and the output result of the output transceiver 212 as new sample data into the database 230 as shown in the figure.

It should be noted that FIG. 2 is only a schematic diagram of a system architecture provided by the embodiment of the present application, and the positional relationship between devices, devices, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG. 2, the data The storage system 250 is an external memory relative to the execution device 210 , and in other cases, the data storage system 250 may also be placed in the execution device 210 .

As shown in FIG. 2 , the target model/rule 201 is obtained by training according to the training device 220. In the embodiment of the present application, the target model/rule 201 may be the first model or the second model involved in the present application, for example, for image processing Semantic segmentation model or target recognition model.

The embodiment of the present application provides a semi-supervised model training method and device, which are used to improve the generalization of the model, and can also improve the prediction accuracy of the model. Referring to FIG. 3 , the embodiment of the present application proposes a semi-supervised model training method.

The method includes:

301. Input first enhanced data into a first model to obtain a first prediction result of the first enhanced data, and input the first enhanced data into a second model to obtain a second prediction result of the first enhanced data.

302. Input the second enhanced data into the first model to obtain a third prediction result of the second enhanced data, and input the second enhanced data into the second model to obtain a fourth prediction result of the second enhanced data.

In the semi-supervised model training method of the present application, the training data involved include labeled data and unlabeled data, wherein the unlabeled data needs to undergo two different enhancement processes before being input into the model for inference. For the first unlabeled data, the first enhanced data is obtained through the first data enhancement method, and the second enhanced data is obtained through the second data enhancement method. It should be noted that the first data enhancement method and the second data different enhancement methods.

The second model and the first model have the same structure and different initialization parameters, for example, both the first model and the second model are semantic segmentation models, or both the first model and the second model are object recognition models.

The specific types of the first augmentation method and the second augmentation method are related to the prediction functions of the first model and the second model: there are many specific types:

In a possible implementation manner, the first model includes an image semantic segmentation model. The first data enhancement method is a non-color gamut enhancement method, and the non-color gamut enhancement includes flip transformation, mirror transformation, translation transformation or scale transformation; the second data enhancement method is a color gamut enhancement method, and the color gamut enhancement method includes shading transformation, contrast Transform or image Gaussian noise enhancement.

In another possible implementation manner, the first model is a target detection model. The first data enhancement method includes: one or more of flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation; the second data enhancement method includes: flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation one or more of them.

That is to say, the first unlabeled data is enhanced from different types of directions through the first data enhancement method and the second data enhancement method. Performing model training based on the processed first enhanced data and second enhanced data can improve model generalization.

In another possible implementation, in order to increase the correlation between labeled data training and unlabeled data and reduce the difference between data domains, the second enhanced data is obtained by processing the first unlabeled data according to the second data enhancement method , the specific second enhanced data is obtained through copy-paste enhancement according to the first unlabeled data and the first labeled data. In one implementation, copy-paste enhancement can be understood as randomly cutting a part of the labeled image and pasting it into the unlabeled data. Using this enhancement method to solve the problem of inconsistency in the data domain can further improve the prediction accuracy of the model.

Input the first enhanced data into the first model and the second model respectively, obtain the first prediction result and the second prediction result respectively; input the second enhanced data into the first model and the second model, obtain the third prediction result respectively, and the fourth predicted result.

In the process of data inference by the first model and the second model, feature extraction is required. Considering that when the amount of training data is large, it will consume more calculations to complete the inference through the first model and the second model respectively. and storage resources, therefore, in a possible implementation, the first enhanced data is input into the preset feature extraction network to obtain the first feature data; the first feature data is input into the first model to obtain the first prediction result, and the The first feature data is input into the second model to obtain a second prediction result. Thus, computing and storage resource consumption can be reduced.

303. Determine a first unsupervised loss according to the consistency between the first prediction result and the fourth prediction result.

304. Determine a second unsupervised loss according to the consistency between the second prediction result and the third prediction result.

The prediction result of the semantic segmentation model is the prediction value and confidence of each pixel in the image. If the confidence of the j-th pixel belonging to the i-th class of the first prediction result is higher than the fourth prediction result, then the first unsupervised loss is based on The loss value of the first prediction result is determined, otherwise it is 0.

305. Input the first labeled data into the first model, and acquire a fifth prediction result of the first labeled data.

The first labeled data may be, for example, an image of RGB channels and a segmentation label (label) corresponding to the image. The first labeled data is input into the first model to obtain a fifth prediction result of the first labeled data.

Before inputting the first annotation data into the first model, the method further includes: performing enhancement processing on the first annotation data to obtain enhanced first annotation data, and the enhanced first annotation data is used for input into the first model to obtain Fifth prediction result. The present application enhances the first labeled data, which can further enhance the generalization of the first model.

The method for performing enhancement processing on the first labeled data is not specifically limited.

306. Determine a first supervision loss according to the fifth prediction result and the label of the first labeled data.

A first supervised loss may be determined based on the difference between the annotation in the first annotated data and the fifth predicted result.

307. Update parameters of the first model based on the first unsupervised loss, the second unsupervised loss, and the first supervised loss.

The parameters of the first model are updated according to the first unsupervised loss, the second unsupervised loss and the first supervised loss, and the first model is trained.

Optionally, input the first labeled data into the second model to obtain the sixth prediction result of the first labeled data; determine the second supervised loss according to the sixth predicted result and the label of the first labeled data; based on the first unsupervised loss, The second unsupervised loss and the second supervised loss update the parameters of the second model. Thus, the second model can be trained synchronously, and the second prediction result and the fourth prediction result can be obtained based on the updated second model, which can improve the reliability of the prediction.

Based on the semi-supervised model training method provided by this application, the unlabeled data is effectively used, which greatly saves the labeling cost.

Referring to FIG. 4 , the architecture of the semi-supervised model training device provided by the embodiment of the present application will be introduced below.

The architecture includes: a weak enhancement module 101, a strong enhancement module 102, a feature extraction module 103, a semantic segmentation header module 104, a semantic segmentation header module 105, and an uncertainty guided re-weight module (UGRM) module 106. .

It should be noted that the semantic segmentation header module 105 and the semantic segmentation header module 104 have the same structure and different initialization parameters.

Weak enhancement module 101: For the labeled data, non-color gamut enhancements are performed on the data and labels, such as random horizontal flipping, that is, the image is flipped in the horizontal direction; random mirroring, that is, the image is mirrored in the vertical direction, and so on.

Enhanced data=random (horizontally flipped image, mirror image horizontally).

The weak enhancement module 101 can be used to enhance the labeled data. Alternatively, performing enhancement processing on the unlabeled data is equivalent to processing through the first data enhancement method in the foregoing embodiments to obtain the first enhanced data.

Strong enhancement module 102: Perform enhancement processing on the unlabeled data, by enhancing the color domain such as image random brightness changes, image random contrast changes, and image Gaussian noise. Optionally, the strong enhancement module 102 can also be copy paste (copy paste) etc. to the image (cross-set) in the cross data set (such as standard data and non-labeled data); wherein, the image Gaussian noise enhancement is for the image The pixels in are randomly added with noise conforming to the Gaussian distribution.

It should be noted that the weak enhancement module 101 is used to process the first unlabeled data according to the first data enhancement method to obtain the first enhanced data in the foregoing embodiments; similarly, the strong enhancement module 102 is used to process the first data enhancement method according to the second data enhancement method. - Unlabeled data Acquire the second enhanced data in the foregoing embodiment.

Feature extraction module 103: a semantic segmentation feature extractor, which mainly uses operations such as convolution and pooling to extract features from pictures. It should be noted that the semantic segmentation network generally includes two parts: encoding (encoder) and decoding (decoder), both of which are composed of a series of operations such as convolution, pooling, and upsampling (upsample). The feature extraction module 103 is equivalent to the encoder part in semantic segmentation; the function of the encoder is to continuously convolve a C×H×W size image to a C'×H'×W', where C<C', H>H ',W>W', that is, compress the image in the H and W directions, and expand the C dimension.

Semantic segmentation header module 104 (Head1): equivalent to the semantic segmentation decoder part, which is used to upsample the features of the feature extraction module 103 and restore them to a specified size; Dimensions (upsampling).

Semantic segmentation head module 105 (Head2): similar to the semantic segmentation head 104, it should be noted that both Head1 and Head2 are segmentation heads with the same structure and different initialization (segmentation head, which can be understood as a decoder).

Head1's unlabeled data weak enhancement prediction (Pred1, weak') and Head2's unlabeled data weak enhancement prediction (Pred2, weak') are the predictions of different heads on the labeled data (labeled date).

It should be noted that the semantic segmentation header module 104 in this embodiment is equivalent to the first model in the foregoing embodiments, and similarly, the semantic segmentation header module 105 is equivalent to the second model in the foregoing embodiments.

UGRM module 106: used to calculate the different uncertainties of the same unlabeled data after passing through module 104 and module 105. As mentioned above, the output of each head has three results, namely:

a. The prediction result of the label data RGB image being input into the network through module 101 weak enhancement;

b. The unlabeled data RGB image is weakly enhanced by module 101 and input to the prediction result of the network;

c. The prediction result input to the network is strongly enhanced by the unlabeled data RGB image through the module 102 .

The output b of head1 (denoted as head1_b) and the output c of head2 (denoted as head2_c) are the prediction results of different segmentation head modules for unlabeled data respectively passed through the weak enhancement module 101 and the strong enhancement module 102 input into the network, There is a problem that the prediction results are not completely consistent between the two. Referring to FIG. 5, it can be seen that the difference between the prediction result output by the first segmentation header and the prediction result output by the second segmentation header.

In this application, the head2_c output by the second mode is not directly used as a pseudo-label to determine the loss function of head1_b, but is judged and weighted based on the credibility of the prediction result. The specific weighting method is as follows:

If ω _2,i,j >ω _1,i,j , then

If ω _2,i,j ≤ω _1,i,j , then

Among them, the value of m is used to distinguish different segmentation heads, Wm,ij represents the maximum probability that the pixel in the i-th row and column j output by the m-th segmentation head in the prediction result belongs to the class c, where Pm,ij is the pixel belonging to each class probability. C The total number of predicted categories. u represents the corrected weight value. For the predicted value of the pixel in row i and column j, if the predicted value output by the second split header is greater than the predicted value output by the first split header, the weight value is 1; otherwise, it is 0.

The unsupervised loss can be obtained according to the weighted prediction result of the UGRM module 106 , combined with the supervised loss obtained from the labeled data, the semantic segmentation header module 104 and the semantic segmentation header module 105 can be updated.

Functional module division is performed based on the architecture of the semi-supervised model training device in FIG. 4 . Please refer to FIG. 6 , which is a schematic diagram of another embodiment of the semi-supervised model training method in the embodiment of the present application.

601. A sensory information input module.

It is mainly responsible for inputting the perceptual information flow, which is used as the signal input module of the whole system. For example, the input of this module is the data collected by the vehicle camera. If it contains the classification label of each pixel, it is the labeled data. If it does not contain the label is unlabeled data.

Labeled data/unlabeled data: As a signal input module of the entire system, this module can be image information.

The label data is a picture of the RGB channel, and the corresponding segmentation label is the RGB data collected by the camera, and the predefined category to which each pixel in the corresponding RGB image belongs (for example, people, cars, lane lines, road surfaces, etc.) label or sky, etc.); unlabeled data is just the RGB data collected by the camera.

602. Strong enhancement and weak enhancement modules.

This module is mainly responsible for data enhancement of input information, including weak enhancement of non-color space and strong enhancement of color space.

603. A feature extraction module.

The feature extraction module is mainly responsible for sending the enhanced image (including labeled data and unlabeled data after strength enhancement) to the encoder to extract features. The encoder part reduces the spatial resolution of the input by downsampling to generate a low-resolution feature map. Feature extraction can make calculations efficient and effectively distinguish between different categories.

604. Divide the decoder module.

Including module 104 and module 105. In this part, the feature extracted by the feature extraction module is up-sampled to a certain size and then output as the result of semantic segmentation. The decoder upsamples these feature descriptions, restores the image resolution, and remaps the obtained features to each pixel in the image for the classification of each pixel.

The output of each head (module 104 and module 105 are respectively a head) has three results, which are respectively: a. the prediction result of the label data RGB image input to the network through module 101 weak enhancement; b. the unlabeled data RGB image after Module 101 weakly enhances the prediction result input to the network; c. The unlabeled data RGB image passes through module 102 and strongly enhances the prediction result input to the network. So the two heads have a total of 6 outputs.

605. UGRM module.

This module will make certain judgments and weights on the inconsistency of predictions between head1_b and head2_c, head1_b and head2_c.

For example: For the prediction result of the i-th position, there are 2 possibilities for the prediction of head1_b and head2_c: First: the prediction result of head1_b is the same as the prediction result of head2_c, both of which are in the kth class, then the weighting of loss for head1_b is Its confidence; second: the prediction result of head1_b is inconsistent with the result of head2_c, then the weight of head1_b is 0. For a specific processing method, reference may be made to the UGRM module 106 in FIG. 4 , which will not be repeated here.

The semi-supervised model training method provided by the present application has been introduced above, and the semi-supervised model training device for realizing the semi-supervised model training method is introduced below, please refer to Fig. 7, which is a semi-supervised model training device in the embodiment of the present application Example schematic.

The semi-supervised model training device 700 includes: a processing unit 701, configured to input the first enhanced data into the first model to obtain a first prediction result of the first enhanced data, and input the first enhanced data into the second The model acquires a second prediction result of the first enhanced data, the first enhanced data is obtained by processing the first unlabeled data according to the first data enhancement method, and the second model has the same structure and Different initialization parameters; the processing unit 701 is further configured to input second enhanced data into the first model to obtain a third prediction result of the second enhanced data, and input the second enhanced data into the The second model obtains the fourth prediction result of the second enhanced data, the second enhanced data is obtained by processing the first unlabeled data according to the second data enhancement method, and the first data enhancement method and the second The data enhancement methods are different; the determination unit 702 is used to determine the first unsupervised loss according to the consistency of the first prediction result and the fourth prediction result; the determination unit 702 is also used to determine the first unsupervised loss according to the second prediction result The consistency between the result and the third prediction result determines a second unsupervised loss; the processing unit 701 is further configured to input the first labeled data into the first model, and obtain a fifth prediction of the first labeled data Result; the determining unit 702 is further configured to determine a first supervised loss according to the fifth predicted result and the label of the first labeled data; an updating unit 703 is configured to determine based on the first unsupervised loss, the A second unsupervised loss and the first supervised loss update parameters of the first model.

In a possible implementation, the first model includes an image semantic segmentation model; the first data enhancement method is a non-color gamut enhancement method, and the non-color gamut enhancement includes flip transformation, mirror transformation, translation transformation or scale transformation; the second data enhancement method is a color gamut enhancement method, and the color gamut enhancement method includes shading transformation, contrast transformation or image Gaussian noise enhancement.

In a possible implementation manner, the first model includes a semantic segmentation model of an image; the second enhanced data is obtained by processing the first unlabeled data according to a second data enhancement method, including: the second enhanced Data is acquired through copy-paste enhancement according to the first unlabeled data and the first labeled data.

In a possible implementation manner, the first model is a target detection model; the first data enhancement method includes: one or more of flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation; The second data enhancement method includes: one or more of flip transformation, translation transformation, scale transformation, rotation transformation, and scaling transformation.

In a possible implementation manner, the processing unit 701 is further configured to: perform enhancement processing on the first annotation data to obtain enhanced first annotation data, and the enhanced first annotation data is used for input the first model to obtain the fifth prediction result.

In a possible implementation manner, the processing unit 701 is further configured to input the first labeled data into the second model, and obtain a sixth prediction result of the first labeled data; the determining unit 702 is also configured to It is used to determine the second supervised loss according to the sixth prediction result and the label of the first labeled data; the updating unit 703 is also used to determine the second supervised loss based on the first unsupervised loss, the second unsupervised loss and the second A supervised loss updates the parameters of the second model.

In a possible implementation manner, the processing unit 701 is specifically configured to: input the first enhanced data into a preset feature extraction network to obtain first feature data; input the first feature data into the first A model obtains the first prediction result, and the first feature data is input into the second model to obtain the second prediction result.

In a possible implementation manner, the first model is an image semantic segmentation model; the processing unit 701 is specifically configured to: determine the first pseudo-label corresponding to the first prediction result, if the fourth prediction result is The first predicted probability corresponding to the first pixel position is greater than the second predicted probability of the first pixel position in the first predicted result, the weight of the first pseudo label is 1, if the first predicted probability is less than or equal to the second prediction probability, the weight of the first pseudo-label is 0; and the first unsupervised loss is determined according to the first prediction result and the first pseudo-label.

It should be understood that the division of each unit of the above device is only a division of logical functions, and may be fully or partially integrated into one physical entity or physically separated during actual implementation. And these units can be implemented in the form of software calling through the processing element; all can be implemented in the form of hardware; some units can also be implemented in the form of software calling through the processing element, and some units can be implemented in the form of hardware. For example, the above units may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits (application specific integrated circuit, ASIC), or, one or more microprocessors (digital signal processor , DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA), etc. For another example, when one of the above units is implemented in the form of a processing element scheduler, the processing element may be a general processor, such as a central processing unit (central processing unit, CPU) or other processors that can call programs. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

Please refer to FIG. 8, which is a schematic diagram of an embodiment of a semi-supervised model training device in the embodiment of the present application;

The semi-supervised model training device provided in this embodiment can be a vehicle-mounted mobile device, a smart phone, a deep learning training platform, an API (Application Programming Interface, an application programming interface), a terminal device, a vehicle-mounted device or a security monitoring device, etc., the present application The embodiment does not limit its specific device form.

The semi-supervised model training device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors 801 and memory 802, and the memory 802 stores programs or data.

Wherein, the memory 802 may be a volatile storage or a non-volatile storage. Optionally, the processor 801 is one or more central processing units (central processing unit, CPU), and the CPU may be a single-core CPU or a multi-core CPU. The processor 801 can communicate with the memory 802 , and execute a series of instructions in the memory 802 on the semi-supervised model training device 800 .

The semi-supervised model training device 800 also includes one or more wired or wireless network interfaces 803, such as Ethernet interfaces.

Optionally, although not shown in Figure 8, the semi-supervised model training device 800 can also include one or more power supplies; one or more input and output interfaces, which can be used to connect display, mouse, keyboard, touch screen devices Or sensing equipment, etc., the input and output interfaces are optional components, which may or may not exist, and are not limited here.

For the process executed by the processor 801 in the semi-supervised model training apparatus 800 in this embodiment, reference may be made to the method process described in the foregoing method embodiments, and details are not repeated here.

For content not described in detail in FIG. 7 or FIG. 8 , reference may be made to the description of relevant parts in FIGS. 3-6 , and details are not repeated here.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other various media that can store program codes. .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions described in each embodiment are modified, or some of the technical features are replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the various embodiments of the application.

Claims (19)

1. A semi-supervised model training method is characterized by comprising the following steps:

inputting first enhancement data into a first model to obtain a first prediction result of the first enhancement data, and inputting the first enhancement data into a second model to obtain a second prediction result of the first enhancement data, wherein the first enhancement data is obtained by processing first label-free data according to a first data enhancement method, and the second model and the first model have the same structure and different initialization parameters;

inputting second enhancement data into the first model to obtain a third prediction result of the second enhancement data, and inputting the second enhancement data into the second model to obtain a fourth prediction result of the second enhancement data, wherein the second enhancement data is obtained by processing the first label-free data according to a second data enhancement method, and the first data enhancement method is different from the second data enhancement method;

determining a first unsupervised loss according to a consistency of the first prediction result and the fourth prediction result;

determining a second unsupervised loss according to consistency of the second predicted result and the third predicted result;

inputting first annotation data into the first model, and acquiring a fifth prediction result of the first annotation data;

determining a first supervision loss according to the fifth prediction result and the label of the first annotation data;

updating parameters of the first model based on the first unsupervised loss, the second unsupervised loss, and the first supervised loss.

2. The method of claim 1, wherein the first model comprises a semantic segmentation model of an image;

the first data enhancement method is a non-color domain enhancement method, and the non-color domain enhancement comprises inversion transformation, mirror transformation, translation transformation or scale transformation;

the second data enhancement method is a color gamut enhancement method that includes a shading transform, a contrast transform, or image gaussian noise enhancement.

3. The method of claim 1 or 2, wherein the first model comprises a semantic segmentation model of an image;

the second enhancement data is obtained by processing the first label-free data according to a second data enhancement method, and comprises the following steps:

and the second enhancement data is obtained by copying and pasting enhancement according to the first unmarked data and the first marked data.

4. The method of claim 1, wherein the first model is an object detection model;

the first data enhancement method includes: one or more of a flip transformation, a translation transformation, a scale transformation, a rotation transformation, and a scaling transformation;

the second data enhancement method includes: one or more of a flipping transformation, a translation transformation, a scaling transformation, a rotation transformation, a scaling transformation.

5. The method of any of claims 1 to 4, wherein prior to entering the first annotation data into the first model, the method further comprises:

and performing enhancement processing on the first annotation data to obtain enhanced first annotation data, wherein the enhanced first annotation data is used for inputting the first model to obtain the fifth prediction result.

6. The method according to any one of claims 1 to 5, further comprising:

inputting the first annotation data into the second model, and obtaining a sixth prediction result of the first annotation data;

determining a second supervision loss according to the sixth prediction result and the label of the first labeling data;

updating parameters of the second model based on the first unsupervised loss, the second unsupervised loss, and the second supervised loss.

7. The method of any one of claims 1 to 6, wherein entering the first enhancement data into a first model obtains a first predicted outcome of the first enhancement data, and entering the first enhancement data into a second model obtains a second predicted outcome of the first enhancement data, comprises:

inputting the first enhanced data into a preset feature extraction network to obtain first feature data;

inputting the first feature data into the first model to obtain the first prediction result, and inputting the first feature data into the second model to obtain the second prediction result.

8. The method according to any one of claims 1 to 7, wherein the first model is a semantic segmentation model of an image;

the determining a first unsupervised loss as a function of the agreement of the first prediction and the fourth prediction comprises:

determining a first pseudo label corresponding to a first prediction result, wherein if a first prediction probability corresponding to a first pixel position in the fourth prediction result is greater than a second prediction probability of the first pixel position in the first prediction result, a weight of the first pseudo label is 1, and if the first prediction probability is less than or equal to the second prediction probability, the weight of the first pseudo label is 0;

determining the first unsupervised loss based on the first prediction and a first pseudo label.

9. A semi-supervised model training device, comprising:

the processing unit is used for inputting first enhancement data into a first model to obtain a first prediction result of the first enhancement data, and inputting the first enhancement data into a second model to obtain a second prediction result of the first enhancement data, wherein the first enhancement data is obtained by processing first label-free data according to a first data enhancement method, and the second model and the first model have the same structure and different initialization parameters;

the processing unit is further configured to input second enhancement data into the first model to obtain a third prediction result of the second enhancement data, and input the second enhancement data into the second model to obtain a fourth prediction result of the second enhancement data, where the second enhancement data is obtained by processing the first label-free data according to a second data enhancement method, and the first data enhancement method is different from the second data enhancement method;

a determining unit, configured to determine a first unsupervised loss according to a consistency of the first prediction result and the fourth prediction result;

the determining unit is further used for determining a second unsupervised loss according to the consistency of the second prediction result and the third prediction result;

the processing unit is further configured to input first annotation data into the first model, and obtain a fifth prediction result of the first annotation data;

the determining unit is further configured to determine a first supervision loss according to the fifth prediction result and the label of the first annotation data;

an updating unit for updating parameters of the first model based on the first unsupervised loss, the second unsupervised loss and the first supervised loss.

10. The apparatus of claim 9, wherein the first model comprises a semantic segmentation model of an image;

the first data enhancement method is a non-color domain enhancement method, and the non-color domain enhancement comprises inversion transformation, mirror transformation, translation transformation or scale transformation;

the second data enhancement method is a color gamut enhancement method that includes a shading transform, a contrast transform, or image gaussian noise enhancement.

11. The apparatus of claim 9 or 10, wherein the first model comprises a semantic segmentation model of an image;

the second enhancement data is obtained by processing the first label-free data according to a second data enhancement method, and comprises the following steps:

and the second enhancement data is obtained by copying and pasting enhancement according to the first unmarked data and the first marked data.

12. The apparatus of claim 9, wherein the first model is an object detection model;

the first data enhancement method includes: one or more of a flip transformation, a translation transformation, a scale transformation, a rotation transformation, and a scaling transformation;

the second data enhancement method includes: one or more of a flipping transformation, a translation transformation, a scaling transformation, a rotation transformation, a scaling transformation.

13. The apparatus of any of claims 9 to 12, wherein the processing unit is further configured to:

and performing enhancement processing on the first annotation data to obtain enhanced first annotation data, wherein the enhanced first annotation data is used for inputting the first model to obtain the fifth prediction result.

14. The apparatus according to any one of claims 9 to 13,

the processing unit is further configured to input first annotation data into the second model, and obtain a sixth prediction result of the first annotation data;

the determining unit is further configured to determine a second supervision loss according to the sixth prediction result and the label of the first annotation data;

the updating unit is further configured to update parameters of the second model based on the first unsupervised loss, the second unsupervised loss, and the second supervised loss.

15. The apparatus according to any one of claims 9 to 14, wherein the processing unit is specifically configured to:

inputting the first enhanced data into a preset feature extraction network to obtain first feature data;

inputting the first feature data into the first model to obtain the first prediction result, and inputting the first feature data into the second model to obtain the second prediction result.

16. The apparatus according to any one of claims 9 to 15, wherein the first model is a semantic segmentation model of an image;

the processing unit is specifically configured to:

determining a first pseudo label corresponding to a first prediction result, wherein if a first prediction probability corresponding to a first pixel position in the fourth prediction result is greater than a second prediction probability of the first pixel position in the first prediction result, a weight of the first pseudo label is 1, and if the first prediction probability is less than or equal to the second prediction probability, the weight of the first pseudo label is 0;

determining the first unsupervised loss based on the first prediction and a first pseudo label.

17. A semi-supervised model training device, comprising:

a memory having computer readable instructions stored therein;

a processor coupled to the memory, the computer readable instructions, when executed by the processor, cause the semi-supervised model training apparatus to implement the method of any of claims 1 to 8.

18. A computer program product comprising computer readable instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 8.

19. A computer-readable storage medium having stored therein instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 8.

Images