CN114692667A - Model training method and related device - Google Patents

Abstract

The present application discloses a model training method, which can be applied to the field of artificial intelligence. The method includes: acquiring a sample pair including noise data and noise-free data; inputting the noise-free data into a noise reduction network and a noise reduction classification network of the classification network to obtain first output data output by the noise reduction network and second output data output by the classification network Output data; input the noise data into the noise reduction classification network to obtain the third output data output by the middle layer of the noise reduction network and the fourth output data output by the classification network. The first loss function is determined according to the first output data and the third output data; the second loss function is determined according to the second output data and the fourth output data; the noise reduction classification network is trained at least according to the first loss function and the second loss function, Until the preset training conditions are met, the target network is obtained. This scheme can enhance the network's ability to suppress local noise and disturbance, and improve the classification and recognition accuracy of the network.

Description

A model training method and related device

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a model training method and related devices.

Background technique

Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that responds in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Sparse time series data is a set of non-dense data sequences arranged in chronological order. Common sparse time series data include, for example, human skeleton keypoint data, electrocardiogram data, and inertial measurement unit (IMU) data. Useful information can be obtained by classifying and identifying sparse time series data. For example, based on the key point data of the human skeleton, the posture and actions of the human body can be identified; based on the electrocardiogram data, the physical condition of the human body can be diagnosed; based on the IMU data on the wearable device, the motion state of the human body can be identified.

In the real environment, sparse time series data is usually interfered by various kinds of noise, resulting in the sparse time series data collected by the device including noise. Based on this, in the related art, the sparse time series data is usually denoised, and then the denoised sparse time series data is classified and identified based on the original classification method.

However, in the related art, the classification and identification accuracy of noisy sparse time series data is low, and it is difficult to ensure the normal identification of sparse time series data. Therefore, there is an urgent need for a method that can effectively classify and identify noisy sparse time series data.

SUMMARY OF THE INVENTION

The present application provides a model training method and related device. In the process of training a noise reduction classification network, the output of the noise reduction network in the noise reduction classification network and the noise data in the middle layer of the noise reduction network are based on the noise-free data. The first loss function is obtained from the output, the second loss function is obtained based on the output of the noise-free data and the noise data in the entire noise reduction classification network, and the noise reduction classification network is trained based on the first loss function and the second loss function. Get the target network. By training the noise reduction classification network, and calculating the loss function based on the noise-free data and the output of the noise reduction network and the output of the noise reduction classification network, the noise reduction target and the classification accuracy target can be guaranteed to be consistent, and the network can be In the noise reduction stage, more complete global features can be learned, which enhances the network's ability to suppress local noise and disturbance, and improves the classification and recognition accuracy of the network.

A first aspect of the present application provides a model training method, the method includes: a terminal acquires a sample pair from a sample set including multiple sample pairs, where the sample pair includes noise data and noise-free data corresponding to the noise data, that is, a sample pair The noise-free data corresponding to the noise data is the noise-free data after removing the noise. The terminal inputs the noise-free data into the noise reduction classification network to obtain the first output data and the second output data. The noise reduction classification network includes a noise reduction network and a classification network, the first output data is the output of the noise reduction network, and the second output data is the output of the classification network. The terminal inputs the noise data into the noise reduction classification network to obtain third output data and fourth output data. The third output data is obtained based on the middle layer of the noise reduction network, and the fourth output data is the output of the classification network.

Then, the terminal determines a first loss function according to the first output data and the third output data, where the first loss function is used to represent the difference between the first output data and the third output data. By obtaining the first loss function between the noise-free data and the noise data between the outputs of the noise reduction network, and training the noise reduction classification network based on the first loss function, the noise reduction classification network can be made in the noise reduction processing stage. Learn global features that are closer to noise-free data, thereby enhancing the ability of the denoising network to suppress local noise and perturbations.

Secondly, the terminal determines a second loss function according to the second output data and the fourth output data, where the second loss function is used to represent the difference between the second output data and the fourth output data and the real class labels of the noise-free data. For example, the second loss function is obtained by calculating the sum of the first difference value of the true class label of the second output data and the noise-free data and the second difference value of the fourth output data and the true class label of the noise-free data.

Finally, the terminal trains the noise reduction classification network at least according to the first loss function and the second loss function, until the preset training conditions are met, and the target network is obtained. Specifically, the terminal may obtain a total loss function based on the first loss function and the second loss function, and the terminal trains the noise reduction classification network based on the total loss function. The total loss function may be the sum of the first loss function and the second loss function , the total loss function may also be obtained by adding the product of the first loss function and the first proportional coefficient to the product of the second loss function and the second proportional coefficient.

In this scheme, the noise reduction classification network is trained, and the loss function is calculated based on the noise-free data and the output of the noise reduction network and the output of the noise reduction classification network, which can ensure that the noise reduction target and the classification accuracy target are consistent. , and enables the network to learn more complete global features in the noise reduction stage, enhances the network's ability to suppress local noise and disturbance, and improves the accuracy of the network's classification and recognition.

Optionally, in a possible implementation manner, the noise reduction network is an auto-encoder, and the auto-encoder includes an encoder and a decoder. Among them, the autoencoder, also known as the autoencoder, is an artificial neural network that can learn an efficient representation of the input data through unsupervised learning. In practical applications, by adding constraints to the self-encoder, the self-encoder can perform noise reduction processing on noisy data. In the self-encoder, the encoder is used to compress and encode the input data, and the decoder is used to reconstruct the data output by the encoder. The first output data is the output of the encoder, and the third output data is based on the encoder. obtained from the middle layer. By using an auto-encoder including an encoder and a decoder as the noise reduction network, changes to the prior art can be reduced, and the practicability of the solution can be improved.

Optionally, in a possible implementation manner, the terminal inputs the noise data into the noise reduction classification network, and obtains the third output data, including: the terminal inputs the noise data into the noise reduction classification network, and obtains the output of the middle layer of the noise reduction network. characteristic data. The terminal divides the feature data into multiple sub-feature data to obtain third output data, where the third output data includes multiple sub-feature data. The terminal determines a difference value between each sub-feature data in the third output data and the first output data. The terminal determines the first loss function according to the difference value between each sub-feature data and the first output data.

In this solution, by dividing the feature data output by the noise data in the middle layer of the noise reduction network into multiple sub-feature data, and establishing a loss function based on the sub-feature data and the first output data, the noise reduction network can be guided to learn rich global information , to improve the noise reduction effect of the noise reduction network.

Optionally, in a possible implementation manner, the terminal evenly divides the feature data into multiple sub-feature data in chronological order to obtain third output data, wherein each sub-feature data in the multiple sub-feature data corresponds to The lengths of the time periods are the same, and the noise data is time series data.

Since time-series data is coherent and adjacent time-series data has a certain correlation, constructing a loss function based on local time-series features corresponding to noise data and global time-series features corresponding to noise-free data can guide the noise reduction network to learn more Rich global timing information, thereby enhancing the ability of the noise reduction network to suppress local noise and disturbance.

Optionally, in a possible implementation manner, since the first output data is the data output by the encoder in the noise reduction network, and the third output data is the data output by the middle layer of the encoder in the noise reduction network. , the two dimensions are not the same. Therefore, before obtaining the first loss function, a dimension alignment operation may be performed on the first output data and the third output data, so that the dimensions of the two are the same, and then the difference value between the two is obtained.

Specifically, the terminal determining a difference value between each sub-feature data in the third output data and the first output data includes: the terminal performs a dimension alignment operation on each sub-feature data in the first output data and the third output data, respectively, Dimensionally aligned first output data and third output data are obtained. The terminal determines a difference value between each sub-feature data in the dimension-aligned third output data and the dimension-aligned first output data. In practical applications, the terminal can construct multiple dimension alignment sub-networks in advance, by inputting the first output data into one of the dimension alignment sub-networks, and inputting each sub-feature data in the third output data into other corresponding dimension alignment sub-networks, Dimensionally aligned first output data and third output data are obtained.

Optionally, in a possible implementation manner, the terminal determines the second loss function according to the second output data and the fourth output data, including: the terminal determines the difference between the second output data and the true category labels of the noise-free data. , to get the first difference value. The terminal determines the difference between the fourth output data and the true category label of the noise-free data, and obtains a second difference value. The terminal obtains the second loss function according to the first difference value and the second difference value. Wherein, the second output data is the prediction result of multi-classification, which is used to represent the prediction result of the classification network.

Optionally, in a possible implementation manner, in the training process of the noise reduction classification network, a binary classifier can also be introduced, and the binary classifier can reduce the input noise based on the features extracted by the noise reduction classification network. Two-class prediction is performed on the data of the noise classification network, that is, to predict whether the data input to the noise reduction classification network is noise data or noise-free data. Then, based on the binary classification result output by the binary classifier and the real binary classification label corresponding to the input data, the terminal determines a third loss function, and the third loss function is used together with the first loss function and the second loss function. The total loss function is obtained, that is, the third loss function is also used for the training of the noise reduction classification network.

Specifically, the method further includes: the terminal acquires the first feature, and predicts the second classification result corresponding to the noise-free data according to the first feature, to obtain the first prediction result. The first feature is extracted by the classification network in the noise reduction classification network after the noise-free data is input into the noise reduction classification network. The terminal acquires the second feature, and predicts a binary classification result corresponding to the noise data according to the second feature, to obtain a second prediction result. The second feature is extracted by the classification network in the noise reduction classification network after the noise data is input into the noise reduction classification network. The terminal determines a third loss function according to the first prediction result and the true binary label of the noise-free data, the second prediction result and the true binary label of the noise data. The terminal trains the noise reduction classification network at least according to the first loss function, the second loss function and the third loss function. The binary classification result corresponding to the noise-free data is the noise-free type or the noise type, and the binary classification result corresponding to the noise data is the noise-free type or the noise type.

In this scheme, a binary classifier is introduced in the training phase, and based on the features extracted by the classification network in the noise reduction classification network, the binary classification result of the input data is predicted by the binary classifier, and the loss corresponding to the binary classification result is obtained. function. By introducing the loss function corresponding to the binary classification result on the basis of the original loss function, an additional evaluation dimension can be introduced, so that the noise reduction classification network obtained by training can have an adaptive noise reduction classification scale for different types of input data. , to improve the classification accuracy of the noise reduction classification network.

Optionally, in a possible implementation manner, the terminal trains the noise reduction classification network at least according to the first loss function and the second loss function, including: the terminal at least according to the first loss function and the second loss function, through the error inverse function. The parameters of the noise reduction classification network are updated by the propagation algorithm. In simple terms, the terminal can correct the size of the parameters in the initial noise reduction classification network through the error back propagation algorithm during the training process of the noise reduction classification network, so that the reconstruction error loss of the noise reduction classification network becomes smaller and smaller. Specifically, forwarding the input signal until the output will generate an error loss, and updating the parameters in the initial noise reduction classification network by back-propagating the error loss information, so that the error loss converges.

Optionally, in a possible implementation manner, the noise data in the sample pair includes sparse time series data, where the sparse time series data includes skeleton point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data.

A second aspect of the present application provides a noise reduction classification method, the method comprising: obtaining data to be classified; inputting the data to be classified into a target network to obtain a prediction result, where the prediction result is a classification result of the data to be classified; Wherein, the target network is used to perform noise reduction processing and classification on the data to be classified, and the target network is obtained by training based on the method described in the first aspect.

A third aspect of the present application provides a model training apparatus, including: an acquisition unit and a processing unit. The acquisition unit is configured to acquire sample pairs, the sample pairs include noise data and noise-free data corresponding to the noise data; the processing unit is configured to input the noise-free data into a noise reduction classification network to obtain the first an output data and a second output data, the noise reduction classification network includes a noise reduction network and a classification network, the first output data is the output of the noise reduction network, and the second output data is the output of the classification network output; the processing unit is further configured to input the noise data into the noise reduction classification network to obtain third output data and fourth output data, the third output data is based on the middle layer of the noise reduction network obtained, the fourth output data is the output of the classification network; the processing unit is further configured to determine a first loss function according to the first output data and the third output data, the first loss The function is used to represent the difference between the first output data and the third output data; the processing unit is further configured to determine a second loss function according to the second output data and the fourth output data, The second loss function is used to represent the difference between the second output data and the fourth output data and the true class labels of the noise-free data; the processing unit is further configured to at least according to the first A loss function and the second loss function are used to train the noise reduction classification network until the preset training conditions are met, and the target network is obtained.

Optionally, in a possible implementation manner, the noise reduction network includes an encoder and a decoder, the encoder is used to compress and encode the input data, and the decoder is used to compress the data output by the encoder. Perform data reconstruction; the first output data is the output of the encoder, and the third output data is obtained based on the middle layer of the encoder.

Optionally, the processing unit is further configured to: input the noise data into the noise reduction classification network to obtain feature data output by the middle layer of the noise reduction network; divide the feature data into multiple sub-features data to obtain the third output data, where the third output data includes the plurality of sub-feature data; determine the difference value between each sub-feature data in the third output data and the first output data; according to The difference value between each sub-feature data and the first output data determines the first loss function.

Optionally, in a possible implementation manner, the processing unit is further configured to evenly divide the feature data into multiple sub-feature data in chronological order to obtain the third output data, the multiple sub-feature data The lengths of time periods corresponding to each sub-feature data in the feature data are the same; wherein, the noise data is time series data.

Optionally, in a possible implementation manner, the processing unit is further configured to: perform a dimension alignment operation on each sub-feature data in the first output data and the third output data, respectively, to obtain a dimension Aligned first output data and third output data. A difference value between each sub-feature data in the dimension-aligned third output data and the dimension-aligned first output data is determined.

Optionally, in a possible implementation manner, the processing unit is further configured to: determine the difference between the second output data and the true category label of the noise-free data, and obtain a first difference value; Determine the difference between the fourth output data and the true category label of the noise-free data to obtain a second difference value; obtain the second loss function according to the first difference value and the second difference value ; wherein, the second output data is a multi-classification prediction result, which is used to represent the prediction result of the classification network.

Optionally, in a possible implementation manner, the obtaining unit is further configured to obtain a first feature, and predict a binary classification result corresponding to the noise-free data according to the first feature, to obtain a first prediction result. , the first feature is extracted by the classification network in the noise reduction classification network after the noise-free data is input into the noise reduction classification network; the obtaining unit is further configured to obtain the second feature, and according to the The second feature predicts the second classification result corresponding to the noise data, and obtains the second prediction result. The second feature is extracted by the classification network in the noise reduction classification network after the noise data is input into the noise reduction classification network. ; The processing unit is also used to determine a third loss function according to the first prediction result and the true binary label of the noise-free data, the second prediction result and the true binary label of the noise data ; The processing unit is further configured to train the noise reduction classification network at least according to the first loss function, the second loss function and the third loss function; wherein, the two corresponding noise-free data The classification result is a noiseless type or a noise type, and the binary classification result corresponding to the noise data is a noiseless type or a noise type.

Optionally, in a possible implementation manner, at least according to the first loss function and the second loss function, the parameters of the noise reduction classification network are updated through an error back-propagation algorithm.

Optionally, in a possible implementation manner, the noise data includes sparse time series data.

Optionally, in a possible implementation manner, the sparse time series data includes skeletal point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data.

A fourth aspect of the present application provides a noise reduction classification device, which includes: an acquisition unit and a processing unit. The obtaining unit is used to obtain the data to be classified. The processing unit is configured to input the data to be classified into the target network to obtain a prediction result, where the prediction result is the classification result of the data to be classified; wherein, the target network is used to perform the data to be classified. Noise reduction processing and classification, the target network is obtained by training based on the method described in the first aspect.

A fifth aspect of the present application provides a model training device, which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and the method described in the first aspect is implemented when the program instructions stored in the memory are executed by the processor . For the steps in each possible implementation manner of the first aspect performed by the processor, reference may be made to the first aspect for details, and details are not repeated here.

A sixth aspect of the present application provides a noise reduction classification device, which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the above-mentioned second aspect is implemented method. For the steps in each possible implementation manner of the second aspect performed by the processor, reference may be made to the second aspect for details, and details are not repeated here.

A seventh aspect of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when it runs on a computer, causes the computer to execute the first aspect or the second aspect. method.

An eighth aspect of the present application provides a computer program product, where a computer program is stored in the computer program product, and when the computer program product runs on a computer, causes the computer to execute the method described in the first aspect or the second aspect.

A ninth aspect of the present application provides a circuit system, the circuit system includes a processing circuit configured to perform the method of the first aspect or the second aspect.

A tenth aspect of the present application provides a chip including one or more processors. Part or all of the processor is used to read and execute the computer program stored in the memory to execute the method in any possible implementation of any of the above aspects. Optionally, the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or a wire. Optionally, the chip further includes a communication interface, and the processor is connected to the communication interface. The communication interface is used for receiving data and/or information to be processed, the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface. The communication interface may be an input-output interface. The method provided by the present application may be implemented by one chip, or may be implemented by multiple chips cooperatively.

Description of drawings

1 is a schematic structural diagram of an artificial intelligence main frame provided by an embodiment of the present application;

Fig. 2a is a kind of data processing system provided by the embodiment of this application;

FIG. 2b is another data processing system provided by an embodiment of the present application;

2c is a schematic diagram of a related device for data processing provided by an embodiment of the present application;

FIG. 3a is a schematic diagram of the architecture of a system 100 provided by an embodiment of the present application;

FIG. 3b is a schematic diagram of an application scenario provided by an embodiment of the present application;

FIG. 3c is a schematic diagram of a specific application of time series data provided by an embodiment of the present application;

4 is a schematic flowchart of a model training method provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a noise reduction classification network provided by an embodiment of the present application;

6 is a schematic structural diagram of a local-global feature association module and a hybrid classifier provided by an embodiment of the present application;

FIG. 7 is a schematic flowchart of training a noise reduction classification network according to an embodiment of the present application;

FIG. 8 is a schematic diagram of generating noise data according to an embodiment of the present application;

9a is a schematic flowchart of constructing a local-global feature association loss provided by an embodiment of the present application;

FIG. 9b is a schematic flowchart of constructing a local-global feature association loss and a mixed classification loss provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of the comparison between the existing solution provided by the embodiment of the present application and the solution of the present application;

11 is a schematic structural diagram of a model training apparatus provided by an embodiment of the application;

12 is a schematic structural diagram of a noise reduction classification device provided by an embodiment of the application;

13 is a schematic structural diagram of an execution device provided by an embodiment of the present application;

14 is a schematic structural diagram of a training device provided by an embodiment of the application;

FIG. 15 is a schematic structural diagram of a chip provided by an embodiment of the present application.

Detailed ways

The embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention. The terms used in the embodiments of the present invention are only used to explain specific embodiments of the present invention, and are not intended to limit the present invention.

The embodiments of the present application will be described below with reference to the accompanying drawings. Those of ordinary skill in the art know that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a distinguishing manner adopted when describing objects with the same attributes in the embodiments of the present application. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product or device comprising a series of elements is not necessarily limited to those elements, but may include no explicit or other units inherent to these processes, methods, products, or devices.

First, the overall workflow of the artificial intelligence system will be described. Please refer to Figure 1. Figure 1 shows a schematic structural diagram of the main frame of artificial intelligence. The above-mentioned artificial intelligence theme framework is explained in two dimensions (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data has gone through the process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecological process of the system.

(1) Infrastructure

The infrastructure provides computing power support for artificial intelligence systems, realizes communication with the outside world, and supports through the basic platform. Communication with the outside world through sensors; computing power is provided by smart chips (hardware acceleration chips such as CPU, NPU, GPU, ASIC, FPGA); the basic platform includes distributed computing framework and network-related platform guarantee and support, which can include cloud storage and computing, interconnection networks, etc. For example, sensors communicate with external parties to obtain data, and these data are provided to the intelligent chips in the distributed computing system provided by the basic platform for calculation.

(2) Data

The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, and text, as well as IoT data from traditional devices, including business data from existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.

(3) Data processing

Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

Among them, machine learning and deep learning can perform symbolic and formalized intelligent information modeling, extraction, preprocessing, training, etc. on data.

Reasoning refers to the process of simulating human's intelligent reasoning method in a computer or intelligent system, using formalized information to carry out machine thinking and solving problems according to the reasoning control strategy, and the typical function is search and matching.

Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

(4) General ability

After the above-mentioned data processing, some general capabilities can be formed based on the results of data processing, such as algorithms or a general system, such as translation, text analysis, computer vision processing, speech recognition, image identification, etc.

(5) Smart products and industry applications

Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall solution of artificial intelligence, and the productization of intelligent information decision-making to achieve landing applications. Its application areas mainly include: intelligent terminals, intelligent transportation, Smart healthcare, autonomous driving, safe city, etc.

Next, several application scenarios of the present application are introduced.

FIG. 2a is a data processing system provided by an embodiment of the present application, where the data processing system includes a user equipment and a data processing device. The user equipment includes smart terminals such as mobile phones, personal computers, or information processing centers. The user equipment is the initiator of data processing, and as the initiator of the data noise reduction classification request, the user usually initiates the request through the user equipment.

The above-mentioned data processing device may be a device or server with data processing functions, such as a cloud server, a network server, an application server, and a management server. The data processing equipment receives the data noise reduction classification request from the intelligent terminal through the interactive interface, and then performs data processing in the form of machine learning, deep learning, search, reasoning, decision-making, etc. through the memory for storing data and the processor for data processing. The memory in the data processing device may be a general term, including local storage and a database for storing historical data. The database may be on the data processing device or on other network servers.

In the data processing system shown in FIG. 2a, the user equipment can receive instructions from the user, for example, the user equipment can obtain a set of data input/selected by the user, and then initiate a request to the data processing equipment, so that the data processing equipment can target the data obtained by the user equipment. The data is subjected to a data denoising classification application, thereby obtaining corresponding processing results for the data. As shown in FIG. 2a, the data processing device may execute the model training method of the embodiment of the present application.

Fig. 2b is another data processing system provided by the embodiment of the application. In Fig. 2b, the user equipment is directly used as a data processing device, and the user equipment can directly obtain the input from the user and directly process it by the hardware of the user equipment itself, The specific process is similar to that of FIG. 2a, and the above description can be referred to, and details are not repeated here.

In the data processing system shown in Figure 2b, the user equipment can receive instructions from the user, for example, the user equipment can acquire a piece of data selected by the user in the user equipment, and then the user equipment itself executes a data processing application for the data, Thus, a corresponding processing result for the data is obtained.

In FIG. 2b, the user equipment itself can execute the model training method of the embodiment of the present application.

FIG. 2c is a schematic diagram of a related device for data processing provided by an embodiment of the present application.

The user equipment in the above-mentioned FIGS. 2a and 2b may specifically be the local device 301 or the local device 302 in FIG. 2c, and the data processing device in FIG. 2a may specifically be the execution device 210 in FIG. 2c, wherein the data storage system 250 may be To store the data to be processed by the execution device 210, the data storage system 250 may be integrated on the execution device 210, or may be set on the cloud or other network servers.

The processors in Figures 2a and 2b may perform data training/machine learning/deep learning through a neural network model or other model (eg, a support vector machine-based model), and use the data to finally train or learn the model to execute against the data Data processing applications to obtain corresponding processing results.

FIG. 3a is a schematic diagram of the architecture of a system 100 provided by an embodiment of the present application. In FIG. 3a, the execution device 110 is configured with an input/output (I/O) interface 112 for performing data interaction with external devices, The user may input data to the I/O interface 112 through the client device 140, and the input data may include: various tasks to be scheduled, callable resources, and other parameters in this embodiment of the present application.

When the execution device 110 preprocesses the input data, or the calculation module 111 of the execution device 110 performs computation and other related processing (for example, performing the function realization of the neural network in this application), the execution device 110 may call the data storage system 150 The data, codes, etc. in the corresponding processing can also be stored in the data storage system 150 .

Finally, the I/O interface 112 returns the processing results to the client device 140 for provision to the user.

It is worth noting that the training device 120 can generate corresponding target models/rules based on different training data for different goals or tasks, and the corresponding target models/rules can be used to achieve the above-mentioned goals or complete the above-mentioned tasks. , which provides the user with the desired result. The training data may be stored in the database 130 and come from training samples collected by the data collection device 160 .

In the case shown in FIG. 3 a , the user can manually specify input data, which can be operated through the interface provided by the I/O interface 112 . In another case, the client device 140 can automatically send the input data to the I/O interface 112 . If the user's authorization is required to request the client device 140 to automatically send the input data, the user can set the corresponding permission in the client device 140 . The user can view the result output by the execution device 110 on the client device 140, and the specific presentation form can be a specific manner such as display, sound, and action. The client device 140 can also be used as a data collection terminal to collect the input data of the input I/O interface 112 and the output result of the output I/O interface 112 as new sample data as shown in the figure, and store them in the database 130 . Of course, it is also possible not to collect through the client device 140, but the I/O interface 112 directly uses the input data input into the I/O interface 112 and the output result of the output I/O interface 112 as shown in the figure as a new sample The data is stored in database 130 .

It is worth noting that FIG. 3a is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG. 3a, the data The storage system 150 is an external memory relative to the execution device 110 , and in other cases, the data storage system 150 may also be placed in the execution device 110 . As shown in FIG. 3a, the neural network can be obtained by training according to the training device 120.

An embodiment of the present application also provides a chip, where the chip includes a neural network processor NPU. The chip can be set in the execution device 110 as shown in FIG. 3 a to complete the calculation work of the calculation module 111 . The chip can also be set in the training device 120 as shown in FIG. 3a to complete the training work of the training device 120 and output the target model/rule.

The neural network processor NPU, the NPU is mounted on the main central processing unit (central processing unit, CPU) (host CPU) as a co-processor, and the main CPU assigns tasks. The core part of the NPU is an arithmetic circuit, and the controller controls the arithmetic circuit to extract the data in the memory (weight memory or input memory) and perform operations.

In some implementations, the arithmetic circuit includes a plurality of process engines (PE) inside. In some implementations, the arithmetic circuit is a two-dimensional systolic array. The arithmetic circuit may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit is a general-purpose matrix processor.

For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit fetches the data corresponding to the matrix B from the weight memory, and buffers it on each PE in the operation circuit. The arithmetic circuit fetches the data of matrix A from the input memory and performs matrix operation on matrix B, and stores the partial result or final result of the matrix in an accumulator.

The vector calculation unit can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on. For example, the vector computing unit can be used for network computation of non-convolutional/non-FC layers in neural networks, such as pooling, batch normalization, local response normalization, etc.

In some implementations, the vector computation unit can store the processed output vector to a unified buffer. For example, the vector computing unit may apply a nonlinear function to the output of the arithmetic circuit, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit generates normalized values, merged values, or both. In some implementations, the vector of processed outputs can be used as activation input to an operational circuit, such as for use in subsequent layers in a neural network.

Unified memory is used to store input data as well as output data.

The weight data directly transfers the input data in the external memory to the input memory and/or the unified memory through the memory unit access controller (direct memory access controller, DMAC), stores the weight data in the external memory into the weight memory, and transfers the unified memory store the data in the external memory.

The bus interface unit (bus interface unit, BIU) is used to realize the interaction between the main CPU, the DMAC and the instruction fetch memory through the bus.

The instruction fetch buffer connected to the controller is used to store the instructions used by the controller;

The controller is used for invoking the instructions cached in the memory to realize and control the working process of the operation accelerator.

Generally, the unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories, and the external memory is the memory outside the NPU, and the external memory can be double data rate synchronous dynamic random access memory (double data rate synchronous random access memory). rate synchronous dynamic random access memory, DDRSDRAM), high bandwidth memory (high bandwidth memory, HBM) or other readable and writable memory.

Since the embodiments of the present application involve a large number of neural network applications, for ease of understanding, related terms and neural networks and other related concepts involved in the embodiments of the present application are first introduced below.

(1) Neural network

A neural network can be composed of neural units, and a neural unit can refer to an operation unit that takes xs and intercept 1 as inputs, and the output of the operation unit can be:

Among them, s=1, 2,...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is an activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected with the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be an area composed of several neural units.

来描述：从物理层面神经网络中的每一层的工作可以理解为通过五种对输入空间(输入向量的集合)的操作，完成输入空间到输出空间的变换(即矩阵的行空间到列空间)，这五种操作包括：1、升维/降维；2、放大/缩小；3、旋转；4、平移；5、“弯曲”。其中1、2、3的操作由

完成，4的操作由+b完成，5的操作则由a()来实现。这里之所以用“空间”二字来表述是因为被分类的对象并不是单个事物，而是一类事物，空间是指这类事物所有个体的集合。其中，W是权重向量，该向量中的每一个值表示该层神经网络中的一个神经元的权重值。该向量W决定着上文所述的输入空间到输出空间的空间变换，即每一层的权重W控制着如何变换空间。训练神经网络的目的，也就是最终得到训练好的神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。因此，神经网络的训练过程本质上就是学习控制空间变换的方式，更具体的就是学习权重矩阵。The work of each layer in a neural network can be expressed mathematically

To describe: From the physical level, the work of each layer in the neural network can be understood as the transformation from the input space to the output space (that is, the row space of the matrix to the column space) through five operations on the input space (set of input vectors). ), the five operations include: 1. Dimension raising/lowering; 2. Enlarging/reducing; 3. Rotation; 4. Translation; 5. "Bending". Among them, the operations of 1, 2, and 3 are determined by

Complete, the operation of 4 is completed by +b, and the operation of 5 is realized by a(). The reason why the word "space" is used here is because the object to be classified is not a single thing, but a type of thing, and space refers to the collection of all individuals of this type of thing. Among them, W is the weight vector, and each value in the vector represents the weight value of a neuron in the neural network of this layer. The vector W determines the space transformation from the input space to the output space described above, that is, the weight W of each layer controls how the space is transformed. The purpose of training the neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by the vectors W of many layers). Therefore, the training process of the neural network is essentially learning the way to control the spatial transformation, and more specifically, learning the weight matrix.

Because we want the output of the neural network to be as close as possible to the value you really want to predict, you can compare the predicted value of the current network with the target value you really want, and then update each layer of the neural network according to the difference between the two. (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network), for example, if the predicted value of the network is high, adjust the weight vector to make it predict low Some, keep adjusting until the neural network can predict the real desired target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which is used to measure the difference between the predicted value and the target value. important equation. Among them, taking the loss function as an example, the higher the output value of the loss function (loss), the greater the difference, then the training of the neural network becomes the process of reducing the loss as much as possible.

(2) Back propagation algorithm

The neural network can use the error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the input signal is passed forward until the output will generate error loss, and the parameters in the initial neural network model are updated by back-propagating the error loss information, so that the error loss converges. The back-propagation algorithm is a back-propagation movement dominated by error loss, aiming to obtain the parameters of the optimal neural network model, such as the weight matrix.

The method provided by the present application will be described below from the training side of the neural network and the application side of the neural network.

The neural network training method provided in the embodiment of the present application involves data processing, and can be specifically applied to data processing methods such as data training, machine learning, and deep learning, and symbolizes and encodes training data (such as sample pairs in the present application). Formal intelligent information modeling, extraction, preprocessing, training, etc., finally obtain a trained noise reduction classification model; and, the data noise reduction classification method provided in the embodiment of the present application can use the above trained noise reduction model to Input data (such as data to be processed in this application) is input into the trained noise reduction classification model to obtain output data. It should be noted that the model training method and the data noise reduction classification method provided by the embodiments of this application are inventions based on the same concept, and can also be understood as two parts in a system, or two stages of an overall process: Such as model training phase and model application phase.

In daily life, sparse time series data is ubiquitous. Sparse time series data is a set of non-dense data sequences arranged in chronological order. Among them, sparseness in sparse time series data is opposite to denseness. Common dense data are images, for example, common sparse time series data include, for example, coordinate data of human skeleton points input from multiple frames in somatosensory games, human electrocardiogram data and posture obtained by IMU data etc. In practical applications, a large amount of useful information can be obtained by classifying and identifying sparse time series data.

For example, human skeleton point coordinate data has a very wide range of applications in the field of somatosensory interaction. In a somatosensory game scene, the virtual interaction of hands is usually realized by using a somatosensory interaction device to obtain the coordinate data of human skeleton points and identify the skeleton points of the hands.

For another example, human electrocardiogram data is one of the important basis for doctors to diagnose cardiovascular diseases. With the popularization of artificial intelligence technology, the application of deep learning based on big data to realize automatic analysis and diagnosis can break through the accuracy and application scope of traditional statistical models. limited. The automatic analysis and diagnosis technology based on the classification and identification of ECG data can interpret the massive data of patients more deeply and realize the accurate classification of patients.

For another example, IMUs have become very popular in smart terminals such as mobile phones and tablet computers, and wearable devices such as virtual reality helmets. The IMU data collected by the IMU can be used as an important basis for the recognition of human motion status. For example, based on the IMU data collected by the gyroscope on the smart bracelet or smart watch, the wearer's motion status can be recognized.

The various application scenarios mentioned above are based on the ability to perform classification and recognition on time series data, but the sparse time series data in the real environment will be disturbed by various noises, which will lead to a significant decrease in the classification accuracy of the sparse time series data. . For example, for the coordinate data of human skeleton points, due to the occlusion of the body, the coordinates of some key points of the skeleton of the human body are shaken or missing, resulting in wrong action category prediction. When it comes to ECG data, whether it's a hospital, ambulance, airplane, ship, clinic, or home, the source of interference is everywhere. For IMU data, the common system errors of IMU include constant zero bias error after IMU is turned on, scale factor error, misalignment and non-orthogonal error, nonlinear error and temperature error. These errors will affect the follow-up to varying degrees. Classification and recognition tasks. Therefore, time series data noise reduction is a problem that cannot be ignored in various application scenarios.

Taking the coordinate data of human skeleton points as an example, the action recognition method based on skeleton points takes the coordinates of human skeleton points as the direct input. Due to the small amount of data, the semantic features of the input data are obvious, and it is highly robust to complex environments. It has a wide range of applications in human-computer interaction, intelligent monitoring, service robots and other fields. Similar to the noise in the field of image processing, the coordinate data of skeleton points in the user scene usually have noise problems such as missing or jittering of skeleton key points due to problems such as occlusion or illumination. However, the existing human action recognition methods require the input information to be complete and free of defects. Therefore, when the above-mentioned skeleton point coordinate data with noise problem is used as input, the phenomenon of human action recognition errors is likely to occur.

Based on this, in the related art, the sparse time series data is usually denoised, and then the denoised sparse time series data is classified and identified based on the original classification method. In order to realize the noise reduction of the sparse time series data, the existing data noise reduction methods usually realize the noise reduction of the sparse time series data by training a specific noise reduction network. For example, when the input is human skeleton point coordinate data, a smooth loss that maintains visual rationality is used in the training phase of the denoising network to guide the update of the network parameters, and finally a network for data denoising is obtained. Generally speaking, the noise reduction network in the related art usually realizes the training of the noise reduction network based on the data without pose ambiguity, time series smoothing, and/or maintaining visual rationality. For an end-to-end noise reduction classification task, the ultimate goal is the classification accuracy of the data. Therefore, there is a gap between the noise reduction network optimization goal and the final classification accuracy goal in the related art. The classification and recognition accuracy of sparse time series data after noise reduction is low, and it is difficult to ensure the normal recognition of sparse time series data.

In view of this, the embodiments of the present application provide a model training method and a related device. In the process of training a noise reduction classification network, the output of the noise reduction network of the noise reduction classification network based on the noiseless data and the noise data in the noise reduction classification network. The first loss function is obtained from the output of the middle layer of the noise network, the second loss function is obtained based on the noise-free data and the noise data in the output of the entire noise reduction classification network, and the noise reduction is performed based on the first loss function and the second loss function. The classification network is trained to obtain the target network. By training the noise reduction classification network, and calculating the loss function based on the noise-free data and the output of the noise reduction network and the output of the noise reduction classification network, the noise reduction target and the classification accuracy target can be guaranteed to be consistent, and the network can be In the noise reduction stage, more complete global features can be learned, which enhances the network's ability to suppress local noise and disturbance, and improves the network's accuracy in classifying and identifying sparse time series data.

The model training method provided in the embodiment of the present application can be applied to a terminal, where the terminal is a device capable of performing model training. After training the target network based on the model training method provided in the embodiment of the present application, the terminal may perform noise reduction classification on the acquired time series data based on the target network. Exemplarily, the terminal can be, for example, a smart TV, a personal computer (PC), a notebook computer, a server, a mobile phone (mobile phone), a tablet computer, a mobile internet device (MID), or a wearable device. , virtual reality (virtual reality, VR) equipment, augmented reality (augmented reality, AR) equipment, wireless terminals in industrial control (industrial control), wireless terminals in unmanned driving (self driving), in remote surgery (remotemedicalsurgery) wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The terminal may be a device running an Android system, an IOS system, a windows system, and other systems.

For ease of understanding, a specific application scenario provided by the embodiments of the present application will be introduced below with reference to the accompanying drawings. Referring to FIG. 3b, FIG. 3b is a schematic diagram of an application scenario provided by an embodiment of the present application. As shown in Figure 3b, a possible application scenario is: the smart TV can classify and identify the user's intention based on time series data such as human body posture, head posture, eye gaze direction, facial expression, gesture action or voice, etc. And based on the user's intention identified by the classification, the corresponding operation is performed to complete the interaction with the user.

Referring to FIG. 3c, FIG. 3c is a schematic diagram of a specific application of time series data provided by an embodiment of the present application. Specifically, a data collection device such as a camera and a microphone may be installed on the smart TV. Through data collection devices such as cameras and microphones, the smart TV can obtain relevant data of users located around the smart TV. For example, the smart TV obtains the hand key point data describing the user's gesture intention, the human body pose data describing the user's body movement intention, the head pose data describing the user's gaze direction intention, and/or the face describing the user's expression intention through the camera. Time series data such as key point data. For another example, a smart TV obtains audio data describing the user's voice intention through a microphone. In addition, a communication device may also be provided on the smart TV, which can receive data sent by an external data collection device. For example, a smart TV is provided with a Bluetooth module, which can receive inertial measurement unit data sent by an external smart watch or smart bracelet. Based on the obtained time series data used to describe the user's intention, the smart TV can classify and identify the user's intention through the built-in classification model, obtain the classification and prediction result of the user's intention, and complete the corresponding interactive response operation.

It can be understood that, in practical applications, the time series data obtained by the smart TV can be one or more types of data as described above, and the smart TV can, based on the one or more types of data obtained, Through the classification model, the user intent is classified and identified. In short, for the classification model preset in the smart TV, the input data of the classification model can be one or more types of data. For example, the input data of the classification model is the above-mentioned head gesture data describing the user's gaze direction intention and the hand key point data describing the user's gesture intention; another example, the input data of the classification model is the audio describing the user's voice intention. data This kind of data. When the input data is various types of data, the classification model can classify and identify the user's intention based on the combination of the various types of data, and obtain the classification prediction result of the user's intention.

Exemplarily, when the input data are head gesture data describing the user's gaze direction intent and hand key point data describing the user's gesture intent, when the head gesture data is specifically represented as the user's gaze direction is the direction where the smart TV is located. And when the hand key point data is specifically represented as waving to the right, the classification model can predict that the user's intention is to switch TV channels.

Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a model training method provided by an embodiment of the present application. As shown in Figure 4, the model training method includes the following steps 401-406.

Step 401: Obtain a sample pair, where the sample pair includes noise data and noise-free data corresponding to the noise data.

In this embodiment, the terminal may acquire a sample set including multiple sample pairs, and each sample pair in the sample set includes a pair of noise data and noise-free data. Among them, the noise-free data in the sample pair corresponds to the noise data, that is, the noise-free data in the sample pair is the noise-free data corresponding to the noise data after the noise is removed. In other words, the noise-free data in the sample pair is added with noise. That is, noise data corresponding to the noise-free data is obtained. Optionally, the noise data in the sample pair may be sparse time series data, for example, the noise data is skeleton point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data. The fault diagnosis data may be, for example, equipment operation data or power grid operation data, such as data such as voltage, current, frequency, or waveform generated in real time during the operation of the power grid.

Exemplarily, the noise data in the sample pair may be one or more types of data, for example, the noise data is only inertial measurement unit data, or the noise data is head pose data and hand key point data. For convenience of description, the following will introduce the model training method in the embodiment of the present application by taking the example that the noise data in the sample pair is one type of data.

In practical applications, after acquiring the noise-free data, the terminal may add random noise to the noise-free data to obtain corresponding noise data, thereby constructing a sample pair. During the training process, the terminal can obtain sample pairs from the sample set one by one to realize the training of the noise reduction classification network.

In a possible example, the terminal for executing the model training method in this embodiment may be, for example, a server, and the trained model is obtained by executing the model training method corresponding to FIG. 4 by the server. Then, the trained model can be deployed in the smart TV before the smart TV leaves the factory; or, after the smart TV leaves the factory, the smart TV can connect to the server through the network, and obtain the training on the server by downloading or updating. good model, so that the trained model can be deployed on the smart TV.

Step 402: Input the noise-free data into the noise reduction classification network to obtain the first output data and the second output data.

In this embodiment, the noise reduction classification network includes a noise reduction network and a classification network, the noise reduction network is connected to the classification network, the input of the noise reduction network is the input of the noise reduction classification network, and the output of the noise reduction network is Input to the classification network. The noise reduction network is used to perform noise reduction processing on the data input to the noise reduction classification network to obtain the noise reduction data. After inputting the noise-free data into the noise reduction classification network, the first output data and the second output data can be obtained. The first output data is the output of the noise reduction network, and the second output data is the output of the classification network, that is, the second output data is the classification result output by the classification network and corresponding to the noise-free data.

Optionally, the noise reduction network may be an auto-encoder, and the auto-encoder includes an encoder and a decoder. Among them, the autoencoder, also known as the autoencoder, is an artificial neural network that can learn an efficient representation of the input data through unsupervised learning. In practical applications, by adding constraints to the self-encoder, the self-encoder can perform noise reduction processing on noisy data. In an autoencoder, the encoder is used to compress the input data, and the decoder is used to reconstruct the data output by the encoder. The key information of the input data is obtained by compressing and encoding the input data by the encoder in the self-encoder, and then the decoder in the self-encoder reconstructs the compressed data to realize the realization based on the key information of the input data. The input data is restored, and the restored data realizes the elimination of noise, that is, the noise reduction processing of the data can be realized based on the self-encoder. Exemplarily, the encoder and the decoder may be a Recurrent Neural Network (RNN) including multiple convolutional layers.

The above-mentioned first output data may be output data obtained after the noise-free data is processed by an encoder in the noise-reduction network after the noise-free data is input into the noise-reduction classification network. In general, the data processed by the encoder in the noise reduction network is also called the content vector. The content vector refers to the output of the encoder in the self-encoder, which is generally a set of high-dimensional feature vectors.

The classification network is used to obtain the data processed by the noise reduction network, and output a set of probability value vectors corresponding to the data input to the classification network, and each element value in the vector is the probability of the corresponding category of the input data. Generally, the category with the highest probability is the category to which the input data belongs. That is to say, the classification network is used to classify the acquired data to obtain the corresponding category of the data. Exemplarily, when the data input to the noise reduction classification network is the IMU data collected by the smart bracelet, the classification network is used to classify the IMU data in four categories of walking, running, cycling, and climbing stairs. For example, when the probability label output by the classification network is {0.1, 0.7, 0.15, 0.05}, it can be determined that the category with probability 0.7 (ie, running) is the category to which the IMU data belongs.

The above-mentioned second output data may be data output by the classification network after the noise-free data is input into the noise reduction classification network and processed by the noise reduction network and the classification network.

Step 403: Input the noise data into the noise reduction classification network to obtain third output data and fourth output data, the third output data is obtained based on the middle layer of the noise reduction network, and the fourth output data is the output of the classification network.

In this embodiment, after inputting the noise data corresponding to the above noise-free data into the noise reduction classification network, the terminal can obtain the third output data extracted by the middle layer of the noise reduction network in the noise reduction classification network. The output data is the feature data extracted by the middle layer of the noise reduction network. In addition, the terminal may also acquire fourth output data output by the classification network in the noise reduction classification network, where the fourth output data is the classification result output by the classification network and corresponding to the noise data.

Optionally, the third output data may be obtained based on an intermediate layer of an encoder in a noise reduction network. Exemplarily, the encoder may include one or more intermediate layers, and the terminal may acquire feature data output by each intermediate layer in the encoder, and use these feature data as third output data. For example, if the encoder is a Recurrent Neural Network (RNN) including three convolutional layers, the middle layer of the encoder is the latter two convolutional layers in the encoder, and the two convolutional layers The data output by the multi-layer is the third output data.

It can be understood that there is no limitation on the execution order of steps 402 and 403. In practical applications, step 402 may be executed first, or step 403 may be executed first. This embodiment does not specifically limit the execution order of step 402 and step 403 .

Step 404: Determine a first loss function according to the first output data and the third output data, where the first loss function is used to represent the difference between the first output data and the third output data.

Since both the first output data and the third output data are obtained based on the noise reduction network, after the first output data and the third output data are obtained, the first loss function can be determined based on the first output data and the third output data, to characterize the difference between the first output data and the third output data. In this way, by obtaining the first loss function between the noise-free data and the noise data respectively between the outputs of the noise reduction network, and training the noise reduction classification network based on the first loss function, the noise reduction classification network can be made in the noise reduction network. The processing stage learns global features closer to the noise-free data, thereby enhancing the ability of the denoising network to suppress local noise and perturbations.

Optionally, when the third output data is obtained based on an intermediate layer of an encoder in a noise reduction network, the third output data includes data output by one or more intermediate layers of the encoder. In this case, the difference value between the data output by each intermediate layer and the first output data can be obtained, and by obtaining the difference value between the data output by the plurality of intermediate layers and the first output data The sum, the first loss function is obtained.

Optionally, in a possible embodiment, in step 403, after the terminal inputs the noise data into the noise reduction classification network, and obtains the feature data output by the middle layer of the noise reduction network, the terminal can divide the obtained feature data into For a plurality of sub-feature data, third output data is obtained, and the third output data includes a plurality of sub-feature data.

Exemplarily, in the case where the noise data is sparse time series data, the terminal may evenly divide the feature data into multiple sub-feature data in chronological order to obtain third output data, where each sub-feature data in the multiple sub-feature data corresponds to The time periods are the same length. For example, when the feature data output by the middle layer of the noise reduction network is the data in the T0-T3 time period, the terminal can divide the feature data in time order to obtain the first sub-feature data in the T0-T1 time period, the T1-T2 time period The second sub-feature data of the segment and the third sub-feature data of the T2-T3 time period, wherein the time segments corresponding to the first sub-feature data, the second sub-feature data, and the third sub-feature data have the same length.

Then, after dividing and obtaining multiple sub-feature data, the terminal obtains the difference value between each sub-feature data in the third output data and the first output data, and based on the difference value between the multiple sub-feature data and the first output data The sum determines the first loss function. In this way, by extracting the feature data corresponding to the noise data output by different intermediate layers in the noise reduction network, and evenly dividing the time series local features, and constructing the time series global features based on the feature data corresponding to the noise-free data output by the noise reduction network, The loss function is constructed based on the time-series local features and the time-series global features, which can guide the noise reduction network to learn the global time-series features more fully, thereby enhancing the ability of the noise reduction network to suppress local noise and disturbance.

For time-series data, time-series data is coherent, and adjacent time-series data has a certain correlation. The loss function is constructed based on local time-series features corresponding to noise data and global time-series features corresponding to noise-free data, which can guide noise reduction. The network learns richer global timing information, thereby enhancing the ability of the denoising network to suppress local noise and disturbance. For example, when the noise data is time series data with missing data for a short period of time, constructing the loss function and training the noise reduction network based on the above method can enable the noise reduction network to restore the missing data based on the learned global time series information. Therefore, the noise reduction of noise data can be achieved with a better noise reduction effect.

Optionally, in a possible embodiment, since the first output data is the data output by the encoder in the noise reduction network, and the third output data is the data output by the middle layer of the encoder in the noise reduction network, The two dimensions are not the same. Therefore, before obtaining the first loss function, a dimension alignment operation may be performed on the first output data and the third output data, so that the dimensions of the two are the same, and then the difference value between the two is obtained.

Exemplarily, the terminal performs a dimension alignment operation on each sub-feature data in the first output data and the third output data, respectively, to obtain dimension-aligned first output data and third output data. Specifically, the terminal may construct multiple dimension alignment sub-networks in advance, and input the first output data into one of the dimension alignment sub-networks, and input each sub-feature data in the third output data into other corresponding dimension alignment sub-networks to obtain the dimension Aligned first output data and third output data. Among them, the dimension alignment sub-network takes the gated recurrent unit (GRU) as the basic unit, and constitutes the dimension alignment sub-network through multiple layers of GRUs. The dimension alignment sub-network can change the dimension of the input data. After the dimensions of the third output data and the first output data are aligned, the terminal determines a difference value between each sub-feature data in the dimension-aligned third output data and the dimension-aligned first output data.

Exemplarily, the process of determining the first loss function based on the first output data and the third output data may be as shown in Equation 1 below.

表示第一损失函数；l表示编码器的第l层中间层；i表示第i个时序局部特征；log()表示对数函数；exp()表示指数函数；

表示对编码器第l层的中间层输出的特征均匀划分后得到的第i个时序局部特征；ρ^l表示编码器第l层中间层时序局部特征对应的维度对齐子网络；

表示编码器输出的内容向量对应的维度对齐子网络，即第一输出数据对应的维度对齐子网络。in,

represents the first loss function; l represents the l-th intermediate layer of the encoder; i represents the i-th time series local feature; log() represents the logarithmic function; exp() represents the exponential function;

Represents the i-th time series local feature obtained after evenly dividing the features of the intermediate layer output of the ^lth layer of the encoder;

Indicates the dimension alignment sub-network corresponding to the content vector output by the encoder, that is, the dimension alignment sub-network corresponding to the first output data.

Specifically, the dimension alignment sub-network using GRU as the basic unit can be formally expressed as GRU(x, num_layers, out_dim), where num_layers is the number of network layers, and out_dim represents the desired output dimension.

Step 405: Determine a second loss function according to the second output data and the fourth output data, where the second loss function is used to represent the difference between the second output data and the fourth output data and the true class labels of the noise-free data.

The second output data is the output of the noise reduction classification network after the noiseless data is input into the noise reduction classification network, that is, the category result corresponding to the noiseless data predicted by the noise reduction classification network. The fourth output data is the output of the noise reduction classification network after the noise data is input into the noise reduction classification network, that is, the category result corresponding to the noise data predicted by the noise reduction classification network. In fact, the real class labels corresponding to the noise-free data and the noise-free data are the same. Therefore, the terminal can obtain the first difference value between the real class labels of the second output data and the noise-free data, and the fourth output data and the noise-free data The second difference value of the true class labels of the data is used to determine the second loss function. For example, the second loss function is obtained by calculating the sum of the first difference value of the true class label of the second output data and the noise-free data and the second difference value of the fourth output data and the true class label of the noise-free data.

Taking the noise-free data as the IMU data as an example, when the category corresponding to the IMU data is running, the true category label corresponding to the IMU data may be {0, 1, 0, 0}. Among them, the four element values in the true category label represent the four categories of walking, running, cycling, and climbing stairs, respectively. Suppose, after inputting the noise-free data into the noise reduction classification network, the obtained second output data is {0.1, 0.7, 0.15, 0.05}. Then, the process of obtaining the first difference value of the real class label between the second output data and the noise-free data is actually to obtain the sum of the vector {0.1,0.7,0.15,0.05} and the vector {0,1,0,0} difference between.

Exemplarily, the process of determining the second loss function based on the second output data and the fourth output data may be as shown in Equation 2 below.

L ₂ =L _mul (X _normal )+L _mul (X _noise ) Formula 2

Wherein, L ₂ represents the second loss function; L _mul (X _normal ) represents the cross-entropy loss between the second output data and the true class label of the noise-free data; L _mul (X _noise ) represents the fourth output data and the noise-free data Cross-entropy loss between the true class labels of the data.

In Equation 2,

Among them, p(X _noise ) represents the probability that the predicted category of the input sample X _noise is y _{noise, i} .

In Equation 2,

Among them, p(X _normal ) represents the probability that the predicted category of the input sample X _normal is y _{normal, i} .

In practical applications, in addition to expressing the difference between the output data and the true category label by calculating the cross entropy loss, other difference measures can also be used to express the difference between the output data and the true category label. This embodiment There is no specific limitation on this.

Step 406, at least according to the first loss function and the second loss function, train the noise reduction classification network until the preset training conditions are met, and the target network is obtained.

After obtaining the first loss function and the second loss function, the terminal may obtain a total loss function based on the first loss function and the second loss function, and the total loss function may be the sum of the first loss function and the second loss function. The total loss function may also be obtained by adding the product of the first loss function and the first scale coefficient to the product of the second loss function and the second scale coefficient. After obtaining the total loss function, the terminal trains the noise reduction classification network based on the total loss function. Wherein, the process that the terminal trains the noise reduction classification network based on the total loss function includes: the terminal adjusts the parameters in the noise reduction classification network (including the noise reduction network and the classification network) based on the value of the total loss function, and repeats steps 401-406 , so as to continuously adjust the parameters in the noise reduction classification network, until the obtained total loss function is less than the preset threshold, it can be determined that the preset training conditions have been met, and the target network can be obtained. The target network is the trained noise reduction classification network, which can be used for subsequent data noise reduction and classification.

Optionally, in the training process of the noise reduction classification network, the terminal may update the parameters of the noise reduction classification network through the error back propagation algorithm. In simple terms, the terminal can correct the size of the parameters in the initial noise reduction classification network through the error back propagation algorithm during the training process of the noise reduction classification network, so that the reconstruction error loss of the noise reduction classification network becomes smaller and smaller. Specifically, forwarding the input signal until the output will generate an error loss, and updating the parameters in the initial noise reduction classification network by back-propagating the error loss information, so that the error loss converges. The back-propagation algorithm is a back-propagation movement dominated by error loss, aiming to obtain the parameters of the optimal neural network model, such as the weight matrix.

Optionally, in a possible embodiment, in the training process of the noise reduction classification network, a binary classifier may also be introduced, and the binary classifier can denoise the input based on the features extracted by the noise reduction classification network. The data of the classification network is subjected to two-class prediction, that is, to predict whether the data input to the denoising classification network is noise data or noise-free data. In short, the binary classification result output by the binary classifier is a probability value vector, and the vector includes two element values, which respectively represent the probability that the input data belongs to the noise-free data category and the noise data category. Then, based on the binary classification result output by the binary classifier and the real binary classification label corresponding to the input data, a third loss function is determined, and the third loss function is used to calculate the first loss function and the second loss function together. Take the total loss function, that is, the third loss function is also used for the training of the noise reduction classification network.

Specifically, the terminal acquires the first feature, and predicts the binary classification result corresponding to the noise-free data according to the first feature, to obtain the first prediction result. The first feature is extracted by the classification network in the noise reduction classification network after the noise-free data is input into the noise reduction classification network. For example, after the noise-free data is input into the noise reduction classification network, the classification network in the noise reduction classification network performs feature extraction on the input data, and performs multi-class prediction based on the extracted features, so as to realize the classification of the noise-free data. Then, the terminal can obtain the first feature by acquiring the feature extracted by the classification network. Then, the terminal inputs the acquired first feature into a binary classifier to predict a binary classification result corresponding to the noise-free data.

The terminal acquires the second feature, and predicts a binary classification result corresponding to the noise data according to the second feature, to obtain a second prediction result. The second feature is extracted by the classification network in the noise reduction classification network after the noise data is input into the noise reduction classification network. Similarly, after the noise data is input into the noise reduction classification network, the classification network in the noise reduction classification network performs feature extraction on the input data, and performs multi-class prediction based on the extracted features. The terminal can obtain the second feature by acquiring the feature extracted by the classification network. Then, the terminal inputs the acquired second feature into a binary classifier to predict a binary classification result corresponding to the noise data.

After obtaining the first prediction result and the second prediction result, the terminal determines the third difference value between the first prediction result and the true binary label of the noise-free data, and the difference between the second prediction result and the true binary label of the noise data The fourth difference value between the two, and the third loss function is determined according to the third difference value and the fourth difference value.

Finally, the terminal trains the noise reduction classification network at least according to the first loss function, the second loss function and the third loss function. The total loss function may be the sum of the first loss function, the second loss function and the third loss function, and the total loss function may also be the product of the first loss function and the first proportional coefficient, the second loss function and the third loss function. The product of the two scale coefficients and the sum of the third loss function and the third scale coefficient. In practical applications, the first proportional coefficient, the second proportional coefficient, and the third proportional coefficient may be adjusted according to the accuracy requirements of noise reduction classification, which are not specifically limited herein.

Exemplarily, the process of determining the third loss function based on the first prediction result and the second prediction result may be as shown in Formula 3 below.

L ₃ =L _bin (X _normal )+L _bin (X _noise ) Equation 3

Among them, L ₃ represents the third loss function; L _bin (X _normal ) represents the cross-entropy loss between the first prediction result and the true binary label of the noise-free data; L _bin (X _noise ) represents the second prediction result and noise Cross-entropy loss between true binary labels of the data.

In Equation 3,

Among them, p(X _normal ) represents the probability that the corresponding input X _normal predicts the label to be _bi , and N is the number of batch samples.

In Equation 3,

Among them, p(X _noise ) represents the probability that the predicted label corresponding to the input X _noise is _bi , and N is the number of batch samples.

In this embodiment, a binary classifier is introduced in the training stage, and based on the features extracted by the classification network in the noise reduction classification network, the binary classifier is used to predict the binary classification result of the input data, and the corresponding binary classification result is obtained. loss function. By introducing the loss function corresponding to the binary classification result on the basis of the original loss function, an additional evaluation dimension can be introduced, so that the noise reduction classification network obtained by training can have an adaptive noise reduction classification scale for different types of input data. , to improve the classification accuracy of the noise reduction classification network. For example, for noiseless data and noisy data, the noise reduction classification network can learn different noise reduction classification scales, so that no matter whether the input data is noiseless data or noisy data, the trained noise reduction classification network can have a higher Noise reduction classification accuracy.

For ease of understanding, the model training method provided by the embodiments of the present application will be described below with reference to specific examples.

Referring to FIG. 5, FIG. 5 is a schematic structural diagram of a noise reduction classification network provided by an embodiment of the present application. As shown in FIG. 5 , a training set 501 and a test set 502 are stored on the server 500 , and a noise reduction classification network 503 is also deployed on the server 500 . The training set 501 includes a plurality of sample pairs for training the noise reduction classification network 503; the test set 502 also includes a plurality of sample pairs for testing the performance of the noise reduction classification network obtained by training.

The noise reduction classification network 503 includes a noise reduction network 5031 and a classification network 5032 . The noise reduction network 5031 includes an encoder 50311 and a decoder 50312, the encoder 50311 is used to compress and encode the input data, and the decoder 50312 is used to reconstruct the data output by the encoder 50311. Based on the encoder 50311 and the decoder 50312, noise reduction processing of the input data can be realized. The classification network 5032 includes a feature extraction module 50321 and a classifier 50322. The feature extraction module 50321 is used to perform feature extraction on the data output by the decoder 50312, and the classifier 50322 is used to perform multi-classification based on the features extracted by the feature extraction module 50321. to obtain the prediction result, that is, input the prediction category corresponding to the noise reduction classification network 503 .

In addition, in order to realize the training of the noise reduction classification network 503 , a local-global feature association module 504 and a hybrid classifier 505 are also deployed on the server 500 . The local-global feature association module 504 is used to obtain the local-global feature association loss based on the output of the noise-free data in the noise reduction network 5031 and the output of the noise data in the noise reduction network 5031, that is, the above-mentioned first loss function.

Specifically, reference may be made to FIG. 6, which is a schematic structural diagram of a local-global feature association module and a hybrid classifier provided by an embodiment of the present application. As shown in FIG. 6 , the local-global feature association module 504 includes a feature division module 5041 and a dimension alignment sub-network 5042. The feature division module 5041 is used to obtain the output data (ie, feature data) corresponding to the noise data in the middle layer of the encoder 50311. , and evenly divide the acquired feature data in chronological order to obtain sub-feature data. The dimension alignment sub-network 5042 is used to perform dimension alignment on the output data of the encoder 50311 for the sub-feature data obtained by uniform division and the noise-free data. In this way, the local-global feature association module 504 can obtain the local-global feature association loss based on the dimension-aligned data, that is, the above-mentioned first loss function.

The hybrid classifier 505 is used to obtain the multi-class classification loss based on the output of the classifier 50322 and the real class label corresponding to the original input data of the noise reduction classification network 503, that is, the above-mentioned second loss function. The hybrid classifier 505 is also used for the output of the feature division module 5041 and the real binary classification label corresponding to the original input data of the noise reduction classification network 503 to obtain the binary classification loss, that is, the above-mentioned third loss function. As shown in FIG. 6 , the hybrid classifier 505 includes a binary classifier 5051 and a multi-class classifier 5052. The binary classifier 5051 is used to obtain the binary classification loss, and the multi-class classifier 5052 is used to obtain the multi-class classification loss. Exemplarily, the binary classifier 5051 may be composed of two fully connected layers plus a softmax layer, and the multi-class classifier 5052 may be a Human Action Recognition (HAR) classifier composed of a spatiotemporal graph neural network.

During the training process, the server 500 classifies the noise reduction through the error back propagation algorithm based on the first loss function obtained by the local-global feature association module 504 and the second loss function and the third loss function obtained by the hybrid classifier 505 The parameters in the network 503 are updated until the preset training conditions are met, and the target network is obtained.

Based on the noise reduction classification network shown in Figure 5, the following will introduce the process of training the noise reduction classification network in detail. Referring to FIG. 7 , FIG. 7 is a schematic flowchart of training a noise reduction classification network according to an embodiment of the present application. As shown in FIG. 7 , in the experiment, the human skeleton key point coordinate data is input into the noise reduction classification network 503, and the corresponding local global feature loss and mixed classification loss are obtained based on the local global feature association module 504 and the hybrid classifier 505, and The total loss function is calculated from the local global feature loss and the mixed classification loss. Then, based on the calculated total loss function, the network parameter update module 802 uses the error back propagation algorithm to update the parameters of the noise reduction network 5031 and the classification network 5032 in the noise reduction classification network 503, thereby realizing the noise reduction classification network 503 training.

Specifically, the process of training the noise reduction classification network will be described in detail below.

1. Before training starts, prepare the training set and test set.

Before starting to train the denoising classification network, you need to prepare the datasets on the server, namely the training set and the test set. The training set and the test set both include multiple sample pairs, each sample pair includes noise-free data and noise data, and the noise-free data and noise data in the sample pair are both time series data. In practical applications, the dataset can be prepared according to the scenario in which the denoising classification network is applied. For example, when the noise reduction classification network is used to classify and recognize human actions, a training set and a test set composed of skeleton point coordinate data are prepared.

Specifically, the process of constructing a training set and a test set can be shown in FIG. 8 . FIG. 8 is a schematic diagram of generating noise data according to an embodiment of the present application. After acquiring the noise-free data, the terminal may add random noise, such as noise of different degrees, to the noise-free data to obtain corresponding noise data, thereby constructing a sample pair. After constructing a plurality of sample pairs, a part of the sample pairs can be divided into a training set, and another part of the sample pairs can be divided into a test set. The ratio of sample pairs included in the training set and the test set may be, for example, 4:1.

Second, construct the local-global feature association loss.

In the training phase, the server obtains the feature data output by the intermediate layer of the encoder when the noise data in the sample pair is input, and divides the obtained feature data evenly to obtain time-series local features. The server also obtains the content vector output by the encoder when the noise-free data in the sample pair is input, that is, the time-series global feature. Then, the server performs dimension alignment on the divided time series local features and the content vector corresponding to the noise-free data through the feature dimension alignment sub-network, and constructs a local-global feature association loss, that is, the above-mentioned first loss function. The server may construct the local-global feature association loss based on the above-mentioned formula 1. For details, please refer to the above-mentioned formula 1.

For example, reference may be made to FIG. 9a, which is a schematic flowchart of constructing a local-global feature association loss provided by an embodiment of the present application. As shown in Figure 9a, X _normal represents the noise-free data in the sample pair, X _noise represents the noise data in the sample pair, E1 represents the encoder, g represents the content vector output by the encoder, and D1 represents the decoder. After inputting noise-free data, the server obtains the content vector g output by the encoder. After inputting the noise data, the server obtains the feature data output by the middle layer of the encoder (that is, the time series feature of the middle layer before division in Figure 9a), and divides the obtained feature data equally to obtain sub-feature data (ie, the divided intermediate layer timing features in Figure 9a). Then, the server performs dimension alignment on the divided intermediate layer timing features and content vector g based on the dimension alignment sub-network, and constructs a local-global feature association loss.

Third, construct the mixed classification loss.

The hybrid classifier constructed by the server includes a multi-class classifier and a binary classifier. For the noise-free data and the noise data in the sample pair, the mixed classifier can calculate the mixed loss corresponding to the noise-free data X _normal and the noise data X _noise , namely the multi-class loss and the binary classification loss. Specifically, the process for the server to calculate the mixed classification loss corresponding to the noise-free data X _normal and the noise data X _noise may be shown in the following formula 4 and formula 5.

L _hc1 =L _bin (X _normal )+aL _mul (X _normal ) Equation 4

L _hc2 =L _bin (X _noise )+αL _mul (X _noise ) Equation 5

Among them, L _hc1 is the mixed classification loss corresponding to the noise-free data X _normal ; L _hc2 is the mixed classification loss corresponding to the noise-free data X _noise ; L _bin (X _normal ) represents the difference between the first prediction result and the true binary label of the noise-free data α is the scale coefficient; L _bin (X _noise ) represents the cross entropy loss between the second prediction result and the true binary label of the noise data; L _mul (X _normal ) represents the second output data and no The cross-entropy loss between the true class labels of the noisy data; L _mul (X _noise ) represents the cross-entropy loss between the fourth output data and the true class labels of the noise-free data.

然后，服务器基于维度对齐子网络对划分后的中间层时序特征以及内容向量g进行维度对齐，并构造局部-全局特征关联损失L_ln。此外，降噪网络对无噪声数据和噪声数据进行降噪处理之后，由分类网络中的特征提取模块继续对降噪网络所输出的数据进行特征提取处理，并且提取得到的特征数据分别输入多类别分类器(binary classifier C1)和二值分类器(HAR classifier C2)，以得到无噪声数据混合分类损失L_hc1以及噪声数据对应的混合分类损失L_hc2。Referring to FIG. 9b, FIG. 9b is a schematic flowchart of constructing a local-global feature association loss and a mixed classification loss provided by an embodiment of the present application. X _normal represents the noise-free data in the sample pair, X _noise represents the noise data in the sample pair, E1 represents the encoder, g represents the content vector output by the encoder, and D1 represents the decoder. After inputting noise-free data, the server obtains the content vector g output by the encoder. After inputting the noise data, the server obtains the feature data output by the middle layer of the encoder, and divides the obtained feature data equally to obtain sub-feature data

Then, based on the dimension alignment sub-network, the server performs dimension alignment on the divided intermediate layer timing features and the content vector g, and constructs a local-global feature association loss L _ln . In addition, after the noise reduction network performs noise reduction processing on the noise-free data and the noise data, the feature extraction module in the classification network continues to perform feature extraction processing on the data output by the noise reduction network, and the extracted feature data are input into multiple categories respectively. A classifier (binary classifier C1) and a binary classifier (HAR classifier C2) are used to obtain a mixed classification loss L _hc1 for noiseless data and a mixed classification loss L _hc2 corresponding to noise data.

Fourth, construct the total loss function of the entire noise reduction classification network, and complete the parameter update of the noise reduction classification network.

After obtaining the local-global feature association loss and the mixed classification loss based on steps 3 and 4, the server takes the ratio between the local-global feature association loss and the mixed classification loss as the total loss function of the entire denoising classification network, and uses The error back-propagation algorithm completes the parameter update of the noise reduction classification network. Among them, the total loss function of the entire noise reduction classification network can be shown in Equation 6 below.

L _total =L _ln +λ(L _hc1 +L _hc2 ) Equation 6

Among them, L _total is the total loss function of the noise reduction classification network; L _ln is the local-global feature association loss; λ is the scale coefficient; L _hc1 is the hybrid classification loss corresponding to the noise-free data X _normal ; L _hc2 is the noise data X _noise The corresponding mixed classification loss.

After the training of the noise reduction classification network is completed, the target network obtained by training can be deployed on a terminal such as a server or a smartphone, so as to realize the application of noise reduction classification of time series data.

Referring to FIG. 10 , FIG. 10 is a schematic diagram of a comparison between the existing solution provided by the embodiment of the present application and the solution of the present application. As shown in FIG. 10 , FIG. 10 is a comparison diagram of the training situation of the classification network in the existing solution and the noise reduction classification network trained by the method of the present application. Among them, the abscissa in Figure 10 represents the number of iterations in the training phase, and its unit is 103; the ordinate represents the classification network in the existing scheme in the training phase and the loss of the noise reduction classification network trained by the method of the present application on the validation set function curve comparison. It can be seen from Fig. 10 that when the number of iterations of the noise reduction classification network trained by the method of the present application is 20000, the loss of the validation set is already lower than that of the classification network in the existing scheme. When the number of iterations is greater than 40,000, the noise reduction classification network trained by the method of the present application begins to converge.

Please refer to Table 1. Table 1 is a comparison of the classification accuracy of the human action classification model ST-GCN in the existing solution and the noise reduction classification network trained by the model training method provided in the embodiment of the present application.

Table 1

noise level = 0 noise level = 1 noise level = 3 noise level = 5 Existing program 81.57% 73.78% 57.76% 42.73% This application plan 84.49% 84.11% 83.28% 82.20%

In Table 1, the noise level n (n=0, 1, 3, 5) indicates that n bone joints are randomly selected on each frame of bone points in the normal human bone point coordinate data and random spatial translation noise is added. Among them, the spatial coordinates of the skeleton points after adding noise are limited to the minimum bounding box determined by the spatial coordinates of the normal skeleton points. When the noise level=0, the noise reduction classification network trained by using the model training method provided in the embodiment of the present application can play a role of data enhancement, and can improve the classification accuracy of the model. As the noise level increases, the gap between the noise reduction classification network of the solution of the present application and the classification network of the existing solution also gradually increases.

The embodiment of the present application further provides a noise reduction classification method, the method includes: acquiring data to be classified; inputting the data to be classified into a target network to obtain a prediction result, where the prediction result is a classification result of the data to be classified. The data to be classified includes sparse time series data, and the sparse time series data includes skeleton point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data. Wherein, the target network is used to perform noise reduction processing and classification on the data to be classified, and the target network is obtained by training based on the model training method described in the above embodiment. For details, please refer to the introduction of the above embodiment. This will not be repeated here. Optionally, the target network may be deployed on a smart TV to perform noise reduction classification on the data obtained by the smart TV to obtain a classification prediction result, where the classification prediction result is specifically used to represent the user's intention. In this way, the smart TV can perform an interactive response operation related to the user's intention based on the obtained classification prediction result, for example, perform a channel switching operation.

The model training method and the noise reduction classification method provided by the embodiments of the present application have been described above, and a device for executing the methods mentioned in the above embodiments will be introduced below.

Referring to FIG. 11 , FIG. 11 is a schematic structural diagram of a model training apparatus provided by an embodiment of the present application. As shown in FIG. 11 , the model training apparatus includes: an acquisition unit 1101 and a processing unit 1102 . The obtaining unit 1101 is configured to obtain sample pairs, and the sample pairs include noise data and noise-free data corresponding to the noise data; the processing unit 1102 is configured to input the noise-free data into the noise reduction classification network, Obtain first output data and second output data, the noise reduction classification network includes a noise reduction network and a classification network, the first output data is the output of the noise reduction network, and the second output data is the classification The output of the network; the processing unit 1102 is further configured to input the noise data into the noise reduction classification network to obtain third output data and fourth output data, the third output data is based on the noise reduction network The fourth output data is the output of the classification network; the processing unit 1102 is further configured to determine a first loss function according to the first output data and the third output data, so The first loss function is used to represent the difference between the first output data and the third output data; the processing unit 1102 is further configured to determine according to the second output data and the fourth output data a second loss function, where the second loss function is used to represent the difference between the second output data and the fourth output data and the true class label of the noise-free data; the processing unit 1102 further uses The noise reduction classification network is trained according to at least the first loss function and the second loss function until a preset training condition is met, and a target network is obtained.

Optionally, in a possible implementation manner, the noise reduction network includes an encoder and a decoder, the encoder is used to compress and encode the input data, and the decoder is used to compress the data output by the encoder. Perform data reconstruction; the first output data is the output of the encoder, and the third output data is obtained based on the middle layer of the encoder.

Optionally, the processing unit 1102 is further configured to: input the noise data into the noise reduction classification network to obtain feature data output by the middle layer of the noise reduction network; divide the feature data into multiple subsections feature data, to obtain the third output data, the third output data includes the plurality of sub-feature data; determine the difference value between each sub-feature data in the third output data and the first output data; The first loss function is determined according to the difference value between each sub-feature data and the first output data.

Optionally, in a possible implementation manner, the processing unit 1102 is further configured to evenly divide the feature data into multiple sub-feature data in chronological order to obtain the third output data, the multiple The lengths of time periods corresponding to each sub-feature data in the pieces of sub-feature data are the same; wherein, the noise data is time series data.

Optionally, in a possible implementation manner, the processing unit 1102 is further configured to: perform a dimension alignment operation on each sub-feature data in the first output data and the third output data, respectively, to obtain Dimensionally aligned first output data and third output data. A difference value between each sub-feature data in the dimension-aligned third output data and the dimension-aligned first output data is determined.

Optionally, in a possible implementation manner, the processing unit 1102 is further configured to: determine the difference between the second output data and the true category label of the noise-free data, and obtain a first difference value ; Determine the difference between the true category label of the fourth output data and the noise-free data, and obtain a second difference value; Obtain the second loss according to the first difference value and the second difference value function; wherein, the second output data is a multi-classification prediction result, which is used to represent the prediction result of the classification network.

Optionally, in a possible implementation manner, the obtaining unit 1101 is further configured to obtain a first feature, and predict a binary classification result corresponding to the noise-free data according to the first feature, to obtain a first prediction. As a result, the first feature is extracted by the classification network in the noise reduction classification network after the noise-free data is input into the noise reduction classification network; the obtaining unit 1101 is further configured to obtain the second feature, and according to The second feature predicts a binary classification result corresponding to the noise data, and obtains a second prediction result, where the second feature is a classification network in the noise reduction classification network after the noise data is input into the noise reduction classification network The processing unit 1102 is further configured to determine the first prediction result according to the first prediction result and the true binary label of the noise-free data, the second prediction result and the true binary label of the noise data Three loss functions; the processing unit 1102 is further configured to train the noise reduction classification network at least according to the first loss function, the second loss function and the third loss function; wherein the noise-free The binary classification result corresponding to the data is a noise-free type or a noise type, and the binary classification result corresponding to the noise data is a noise-free type or a noise type.

Optionally, in a possible implementation manner, at least according to the first loss function and the second loss function, the parameters of the noise reduction classification network are updated through an error back-propagation algorithm.

Optionally, in a possible implementation manner, the noise data includes sparse time series data.

Optionally, in a possible implementation manner, the sparse time series data includes skeletal point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data.

Referring to FIG. 12, FIG. 12 is a schematic structural diagram of a noise reduction classification apparatus provided by an embodiment of the present application. The embodiment of the present application provides a noise reduction classification device, the device includes: an acquisition unit 1201 and a processing unit 1202 . The obtaining unit 1201 is configured to obtain the data to be classified. The processing unit 1202 is configured to input the data to be classified into a target network to obtain a prediction result, where the prediction result is the classification result of the data to be classified; wherein, the target network is used to analyze the data to be classified. Noise reduction processing and classification are performed, and the target network is obtained by training based on the model training method described in the above embodiment.

Next, an execution device provided by an embodiment of the present application is introduced. Please refer to FIG. 13 , which is a schematic structural diagram of the execution device provided by the embodiment of the present application. Smart wearable devices, servers, etc., are not limited here. The data processing apparatus described in the embodiment corresponding to FIG. 13 may be deployed on the execution device 1300 to implement the function of data processing in the embodiment corresponding to FIG. 13 . Specifically, the execution device 1300 includes: a receiver 1301, a transmitter 1302, a processor 1303, and a memory 1304 (wherein the number of processors 1303 in the execution device 1300 may be one or more, and one processor is taken as an example in FIG. 13 ) , wherein the processor 1303 may include an application processor 13031 and a communication processor 13032 . In some embodiments of the present application, the receiver 1301, the transmitter 1302, the processor 1303, and the memory 1304 may be connected by a bus or otherwise.

Memory 1304 may include read-only memory and random access memory, and provides instructions and data to processor 1303 . A portion of memory 1304 may also include non-volatile random access memory (NVRAM). The memory 1304 stores processors and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations.

The processor 1303 controls the operation of the execution device. In a specific application, various components of the execution device are coupled together through a bus system, where the bus system may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus. However, for the sake of clarity, the various buses are referred to as bus systems in the figures.

The methods disclosed in the above embodiments of the present application may be applied to the processor 1303 or implemented by the processor 1303 . The processor 1303 may be an integrated circuit chip, which has signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 1303 or an instruction in the form of software. The above-mentioned processor 1303 may be a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and may further include an application specific integrated circuit (ASIC), a field programmable gate Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 1303 may implement or execute the methods, steps, and logical block diagrams disclosed in the embodiments of this application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 1304, and the processor 1303 reads the information in the memory 1304, and completes the steps of the above method in combination with its hardware.

The receiver 1301 can be used to receive input numerical or character information, and to generate signal input related to performing the relevant setting and function control of the device. The transmitter 1302 can be used to output digital or character information through the first interface; the transmitter 1302 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1302 can also include a display device such as a display screen .

In the embodiment of the present application, in one case, the processor 1303 is configured to execute the training method of the noise reduction model executed by the execution device in the embodiment corresponding to FIG. 4 .

This embodiment of the present application also provides a training device. Please refer to FIG. 14. FIG. 14 is a schematic structural diagram of the training device provided by the embodiment of the present application. Specifically, the training device 1400 is implemented by one or more servers. 1400 may vary greatly depending on configuration or performance, and may include one or more central processing units (CPUs) 1414 (eg, one or more processors) and memory 1432, one or more storage A storage medium 1430 (eg, one or more mass storage devices) for applications 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 may be short-term storage or persistent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instructions to operate on the training device. Further, the central processing unit 1414 may be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the training device 1400 .

The training device 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458; or, one or more operating systems 1441, such as Windows Server ^™ , Mac OS X ^TM , Unix ^TM , Linux ^TM , FreeBSD ^TM and many more.

Embodiments of the present application also provide a computer program product that, when running on a computer, causes the computer to perform the steps performed by the aforementioned execution device, or causes the computer to perform the steps performed by the aforementioned training device.

Embodiments of the present application further provide a computer-readable storage medium, where a program for performing signal processing is stored in the computer-readable storage medium, and when it runs on a computer, the computer executes the steps performed by the aforementioned execution device. , or, causing the computer to perform the steps as performed by the aforementioned training device.

The execution device, training device, or terminal device provided in this embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, pins or circuits, etc. The processing unit can execute the computer executable instructions stored in the storage unit, so that the chip in the execution device executes the data processing method described in the above embodiments, or the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), and the like.

Specifically, please refer to FIG. 15. FIG. 15 is a schematic structural diagram of a chip provided by an embodiment of the present application. The chip may be represented as a neural network processor NPU 1500, and the NPU 1500 is mounted on the main CPU (Host CPU) as a co-processor. CPU), tasks are allocated by the Host CPU. The core part of the NPU is the arithmetic circuit 1503, which is controlled by the controller 1504 to extract the matrix data in the memory and perform multiplication operations.

In some implementations, the arithmetic circuit 1503 includes multiple processing units (Process Engine, PE). In some implementations, the arithmetic circuit 1503 is a two-dimensional systolic array. The arithmetic circuit 1503 may also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, arithmetic circuit 1503 is a general-purpose matrix processor.

For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit fetches the data corresponding to the matrix B from the weight memory 1502 and buffers it on each PE in the arithmetic circuit. The arithmetic circuit fetches the data of matrix A and matrix B from the input memory 1501 to perform matrix operation, and stores the partial result or final result of the matrix in the accumulator 1508 .

Unified memory 1506 is used to store input data and output data. The weight data is directly passed through a storage unit access controller (Direct Memory Access Controller, DMAC) 1505 , and the DMAC is transferred to the weight memory 1502 . Input data is also moved into unified memory 1506 via the DMAC.

The BIU is a Bus Interface Unit (Bus Interface Unit, BIU) 1510 , which is used for the interaction between the AXI bus, the DMAC and the instruction fetch buffer (Instruction Fetch Buffer, IFB) 1509 .

The bus interface unit 1510 is used for the instruction fetch memory 1509 to obtain instructions from the external memory, and is also used for the storage unit access controller 1505 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

The DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 1506 or the weight data to the weight memory 1502 or the input data to the input memory 1501 .

The vector calculation unit 1507 includes a plurality of operation processing units, and further processes the output of the operation circuit 1503, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc., if necessary. It is mainly used for non-convolutional/fully connected layer network computation in neural networks, such as Batch Normalization, pixel-level summation, and upsampling of feature planes.

In some implementations, the vector computation unit 1507 can store the vector of processed outputs to the unified memory 1506 . For example, the vector calculation unit 1507 may apply a linear function; or a non-linear function to the output of the operation circuit 1503, such as linear interpolation of the feature plane extracted by the convolution layer, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 1507 generates normalized values, pixel-level summed values, or both. In some implementations, the vector of processed outputs can be used as activation input to the arithmetic circuit 1503, such as for use in subsequent layers in a neural network.

an instruction fetch buffer 1509 connected to the controller 1504 for storing instructions used by the controller 1504;

The unified memory 1506, the input memory 1501, the weight memory 1502 and the instruction fetch memory 1509 are all On-Chip memories. External memory is private to the NPU hardware architecture.

Wherein, the processor mentioned in any one of the above may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.

In addition, it should be noted that the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be A physical unit, which can be located in one place or distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware. Special components, etc. to achieve. Under normal circumstances, all functions completed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuit, etc. However, a software program implementation is a better implementation in many cases for this application. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art. The computer software products are stored in a readable storage medium, such as a floppy disk of a computer. , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to enable a computer device (which may be a personal computer, training device, or network device, etc.) to execute the various embodiments of the application. method.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be retrieved from a website, computer, training device, or data Transmission from the center to another website site, computer, training facility or data center via wired (eg coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a training device, a data center, or the like that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

Claims (13)

1. a model training method, is characterized in that, comprises: obtaining a sample pair, the sample pair including noise data and noise-free data corresponding to the noise data; Input the noise-free data into a noise reduction classification network to obtain first output data and second output data, the noise reduction classification network includes a noise reduction network and a classification network, and the first output data is the noise reduction network. output, the second output data is the output of the classification network; Input the noise data into the noise reduction classification network to obtain third output data and fourth output data, the third output data is obtained based on the middle layer of the noise reduction network, and the fourth output data is the output of the classification network; determining a first loss function according to the first output data and the third output data, where the first loss function is used to represent the difference between the first output data and the third output data; A second loss function is determined according to the second output data and the fourth output data, and the second loss function is used to represent the second output data and the truth of the fourth output data and the noise-free data Differences between category labels; The noise reduction classification network is trained according to at least the first loss function and the second loss function until a preset training condition is met, and a target network is obtained. 2. The method according to claim 1, wherein the noise reduction network comprises an encoder and a decoder, the encoder is used to compress the input data, and the decoder is used to output the encoder data reconstruction; The first output data is the output of the encoder, and the third output data is obtained based on the middle layer of the encoder. 3. The method according to claim 1 or 2, wherein the noise data is input into the noise reduction classification network to obtain third output data, comprising: Inputting the noise data into the noise reduction classification network to obtain feature data output by the middle layer of the noise reduction network; Dividing the feature data into multiple sub-feature data to obtain the third output data, where the third output data includes the multiple sub-feature data; Determining a first loss function according to the first output data and the third output data includes: determining the difference value between each sub-feature data in the third output data and the first output data; The first loss function is determined according to the difference value between each sub-feature data and the first output data. 4. The method according to claim 3, wherein, dividing the feature data into multiple sub-feature data to obtain the third output data, comprising: The feature data is evenly divided into a plurality of sub-feature data according to the time sequence, and the third output data is obtained, and the length of the time period corresponding to each sub-feature data in the plurality of sub-feature data is the same; Wherein, the noise data is time series data. 5. The method according to claim 3 or 4, wherein the determining the difference value between each sub-feature data in the third output data and the first output data comprises: Perform a dimension alignment operation on each of the sub-feature data in the first output data and the third output data, respectively, to obtain dimension-aligned first output data and third output data; A difference value between each sub-feature data in the dimension-aligned third output data and the dimension-aligned first output data is determined. 6. The method according to any one of claims 1-5, wherein the determining a second loss function according to the second output data and the fourth output data comprises: determining the difference between the second output data and the true category label of the noise-free data to obtain a first difference value; determining the difference between the fourth output data and the true category label of the noise-free data to obtain a second difference value; obtaining the second loss function according to the first difference value and the second difference value; Wherein, the second output data is a multi-classification prediction result, which is used to represent the prediction result of the classification network. 7. The method according to any one of claims 1-6, wherein the method further comprises: Acquire a first feature, and predict the binary classification result corresponding to the noise-free data according to the first feature, and obtain a first prediction result, where the first feature is that after the noise-free data is input into the noise reduction classification network, the extracted from the classification network in the noise reduction classification network; Acquire a second feature, and predict a binary classification result corresponding to the noise data according to the second feature, and obtain a second prediction result, where the second feature is that after the noise data is input into the noise reduction classification network, the Extracted by the classification network in the noise classification network; Determine a third loss function according to the first prediction result and the true binary classification label of the noise-free data, the second prediction result and the true binary classification label of the noise data; The training of the noise reduction classification network at least according to the first loss function and the second loss function includes: training the noise reduction classification network according to at least the first loss function, the second loss function and the third loss function; Wherein, the second classification result corresponding to the noise-free data is a noise-free type or a noise type, and the second-classification result corresponding to the noise data is a noise-free type or a noise type. 8. The method according to any one of claims 1-7, wherein the training of the noise reduction classification network at least according to the first loss function and the second loss function comprises: According to at least the first loss function and the second loss function, the parameters of the noise reduction classification network are updated through an error back-propagation algorithm. 9. The method according to any one of claims 1-8, wherein the noise data comprises sparse time series data. 10. The method according to claim 9, wherein the sparse time series data comprises skeletal point coordinate data, electrocardiogram data, inertial measurement unit data or fault diagnosis data. 11. A terminal, comprising a memory and a processor; the memory stores code, the processor is configured to execute the code, and when the code is executed, the terminal executes the code as claimed in the claims The method of any one of 1 to 10. 12. A computer-readable storage medium, characterized by comprising computer-readable instructions, which, when the computer-readable instructions are executed on a computer, cause the computer to perform the execution of any one of claims 1 to 10 Methods. 13. A computer program product comprising computer readable instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 10.

Images