CN117851750A - An intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines - Google Patents

Abstract

An intelligent cleaning method for low-quality monitoring big data of an intelligent manufacturing production line comprises the steps of adding Gaussian noise to original multi-source monitoring data and covering a certain proportion of subsequences in each sample; mapping sequences visible in the simulated noise data into potential representations by using a transducer network, and reconstructing the potential representations into original data; the model has the capacity of reducing complex noise in the production line monitoring data by minimizing the mean square error between the reconstruction data and the original multi-source monitoring data; finally, the original test data is subjected to noise adding and covering treatment and then is input into a big data intelligent cleaning model to test the cleaning effect of the model; the invention has stronger feature extraction and data reconstruction capability, can effectively reduce noise components in the monitoring data of the intelligent manufacturing production line, and provides data support for downstream tasks such as intelligent diagnosis of subsequent production line equipment.

Description

An intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines

Technical Field

The present invention belongs to the technical field of data cleaning, and in particular relates to an intelligent cleaning method for low-quality monitoring big data of an intelligent manufacturing production line.

Background technique

With the rise of the concept of intelligent manufacturing, enterprises have begun to consciously collect data generated during the processing of production line intelligent equipment to achieve predictive maintenance of production line equipment. However, the processing process of industrial intelligent equipment is complex and multiple devices are usually configured on the production line to work together, resulting in a variety of data recording file types and intricate relationships between files, making it difficult to directly obtain the required information from massive monitoring data; in addition, production line equipment usually moves at high speed during processing and has a harsh service environment. There are many random interference factors, and they are affected by external factors such as foreign body impact, as well as performance degradation of internal data acquisition devices, sensor failure, and data recording anomalies. The monitoring data of its key components will be mixed with noise, abnormal points, missing values and other interference data, which reduces the data quality and affects the performance of subsequent intelligent diagnosis and prediction models. Therefore, it is particularly important to study the cleaning method of low-quality monitoring big data of intelligent manufacturing production lines.

For low-quality monitoring data, traditional processing methods mainly include manual elimination and signal processing technology. Manual elimination is usually done by equipment maintenance personnel relying on their own professional knowledge and experience to identify and eliminate abnormal data. This method is only applicable to situations where the amount of data is small and the degree of data quality damage is more serious. It also has high requirements for the professional level of maintenance personnel. In the actual governance process, human factors will inevitably be introduced, affecting the accuracy of subsequent analysis. Signal processing technology has been widely used in data cleaning methods. Commonly used signal processing methods include wavelet transform and empirical mode decomposition. The theory and implementation process of the wavelet transform method are relatively complex, and it requires high expert knowledge. In addition, its computational complexity is high, and the cleaning effect depends on the selected wavelet basis function. The method based on empirical mode decomposition is difficult to select parameters. Different parameter selections may lead to different decomposition results. In addition, the number of modal functions obtained by decomposition is uncertain, which makes it difficult to determine how many modal functions should be retained in the cleaned data.

In recent years, deep neural networks have shown superior performance in feature extraction and data reconstruction, providing new ideas for low-quality data cleaning methods. Compared with traditional cleaning methods, data-driven intelligent cleaning methods (Qiang Zhang, Yaming Zheng, Qiangqiang Yuan, et al. "Hyperspectral Image Denoising: From Model-Driven, Data-Driven, to Model-Data-Driven", IEEE Transactions on Neural Networks and Learning Systems, 2023.) have strong adaptability, no need for manual feature extraction, and high efficiency; however, this method also has some limitations: on the one hand, this method often only uses a single type of noise, which is difficult to apply to situations where the data of smart manufacturing production lines contains complex noise; on the other hand, this method usually uses convolutional neural networks or long short-term memory networks for feature extraction and data reconstruction, but convolutional neural networks and their variants are difficult to capture long-distance features, and the calculations of long short-term memory networks and their variants are serial, with low computational efficiency, and perform poorly when processing large-scale data; in addition, long short-term memory networks may also be affected by gradient vanishing or gradient exploding problems, resulting in model training difficulties.

Summary of the invention

In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to propose an intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines, which uses a Transformer network to extract robust high-level representations from complex noisy data and reconstruct them back to the original multi-source monitoring data, thereby reducing the impact of noise, outliers and missing values in the data on the subsequent fault diagnosis and life prediction accuracy.

In order to achieve the above object, the technical solution adopted by the present invention is:

An intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines is proposed. The method simulates noisy data by adding multiple noises to the original multi-source monitoring data and masking a certain proportion of subsequences in each sample. The simulated noise data is mapped to potential representations using a Transformer network, and the potential representations are reconstructed into the original multi-source monitoring data, thereby reducing the impact of complex noise in the original multi-source monitoring data on subsequent analysis.

An intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines, comprising the following steps:

Step 1: Obtain the original multi-source monitoring data set during the equipment processing from the data recording file of the production line Where N represents the number of samples, /> is the i-th sample, the sequence length of each sample is L, and the number of channels is C; then the data set X is divided into training sets/> and test set/> Among them, N ₁ represents the number of samples in the training set;

Step 2: Add Gaussian noise _xG , Poisson noise _xP and impulse noise _xS to the sample _xi of the training set _X1 to obtain simulated noise data _xn . The specific calculation formula is as follows:

Where _Ps represents the power of the original vacuum value signal, _Pn represents the power of the Gaussian noise signal, SNR represents the signal-to-noise ratio, Var(·) represents the variance, and _xrand∈N (0,1) represents a random sample sequence that obeys the standard normal distribution, and its sequence length is the same as _xi ;

Step 3: Input the simulated noise data set _Xn into a convolutional neural network to segment the sequence, dividing a single sample into multiple regular, non-overlapping subsequences, and mapping each subsequence into a high-dimensional vector. The noise signal after segmentation and mapping is Where _Np represents the number of blocks, D represents the dimension of the mapped vector, and Embed(·) represents the segmentation mapping operation;

Step 4: Randomly shuffle the subsequences after each sample mapping to obtain the data set Shuffle (Emed (X _n )), then randomly mask (delete) some subsequences, the masking ratio is r, and then use the sine and cosine functions to positionally encode the masked data set. The specific calculation formula is:

Where pos represents the position, i represents the index of the encoding dimension, Shuffle(·) represents the random shuffle operation, Mask(·) represents the masking operation, and _X′n is the noise dataset after adding noise, masking and position encoding;

Step 5: Establish a big data intelligent cleaning model based on the masked autoencoder, whose encoder and decoder both use the Transformer network; input the dataset _X′n into the encoder part of the model and calculate the feature map obtained: Among them,/> A fully connected network is used to map the dimension of z _j to the decoder dimension to obtain/> The decoder input Z′ consists of high-level features Fc(z _j ) and a learnable mask vector set Composition, the specific calculation formula is:

Z′＝Unshuffle([Fc(z _j )；Z _m ])+E _pos (3)

Among them, Unshuffle(·) means canceling the random shuffle operation, and E _pos means the position encoding vector;

Step 6: Input Z′ into the decoder for calculation to obtain the model’s prediction result The denoising result of the model is measured by the mean square error between the predicted result Y ₁ and the training set X ₁ , and the cleaning ability of the model is improved by minimizing the mean square error L _mse of the samples in Y ₁ and X _1. The calculation formula is as follows:

Step 7: Use the reconstruction error L _mse obtained in step 6 as the optimization target of the training phase and use the gradient descent method to update the model parameters θ:

Among them, α represents the learning rate;

Step 8: Repeat steps 2 to 7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations is reached;

Step 9: Test the model. First, use simulated noise data to test the big data intelligent cleaning model. The test set _X2 is processed through steps 2 to 4 to obtain the test sample noise, masking and position encoding result data set _X′n2 ; the data set _X′n2 is input into the trained big data intelligent cleaning model to obtain the model's denoising result _Y2 for the noisy data, and the signal-to-noise ratio between _Y2 and the test set _X2 is calculated as follows:

Wherein, SNR represents signal-to-noise ratio, P _s2 represents the power of the test signal, and P _n2 represents the power of the noise signal;

At the same time, real data is used to test the big data intelligent cleaning model. The test set _X2 is directly input into the trained big data intelligent cleaning model to obtain the denoising result _Y2 ′ of the original multi-source monitoring data set.

In step 5, the big data intelligent cleaning model is divided into two parts: encoder and decoder, and both are composed of Transformer networks; the encoder maps the unmasked subsequence in each sample to a potential representation, which contains three Transformer modules, and the hidden layer dimension of each Transformer module is 256. Each Transformer module consists of layer normalization, multi-head self-attention mechanism, and multi-layer perceptron; the decoder reconstructs the potential representation and the learnable mask vector into the original input, and contains two Transformer modules with the same structure, and the hidden layer dimension of each Transformer module is 64; in addition, a fully connected layer is used to map the encoder hidden layer dimension to the decoder hidden layer dimension.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention proposes an intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines. The method simulates noise data by adding multiple noises to the original multi-source monitoring data and masking a certain proportion of subsequences in each sample. The Transformer network is used to replace the traditional convolutional neural network or the long short-term memory network to map the simulated noise data into potential representations, and the potential representations are reconstructed into the original multi-source monitoring data, so that it can effectively reduce the impact of complex noise in the monitoring data on subsequent analysis, thereby improving the quality of the data. In addition, the structure of the constructed big data intelligent cleaning model is asymmetric, and its encoder part only operates on unmasked subsequences. Compared with the encoder, the decoder is lightweight, narrower and shallower. This design greatly reduces the computational overhead of the model and reduces the training time. Moreover, the decoder is only used to perform data reconstruction tasks during training, and the encoder can be used for downstream diagnosis or prediction tasks after only fine-tuning, thereby improving the generalization ability of the model.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the present invention.

FIG2 is a schematic diagram of the structure of the big data intelligent cleaning model of the present invention.

Detailed ways

The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

As shown in FIG1 , a method for intelligent cleaning of low-quality monitoring big data of an intelligent manufacturing production line includes the following steps:

Step 1: Obtain the original multi-source monitoring data set during the equipment processing from the data recording file of the production line Where N represents the number of samples, /> is the i-th sample, the sequence length of each sample is L, and the number of channels is C; then the data set X is divided into training sets/> and test set/> Among them, N ₁ represents the number of samples in the training set;

Step 2: Add Gaussian noise _xG , Poisson noise _xP and impulse noise _xS to the sample _xi of the training set _X1 to obtain the simulated noise data _xn . The specific calculation formula is as follows:

Where _Ps represents the power of the original vacuum value signal, _Pn represents the power of the Gaussian noise signal, SNR represents the signal-to-noise ratio, Var(·) represents the variance, and _xrand∈N (0,1) represents a random sample sequence that obeys the standard normal distribution, and its sequence length is the same as _xi ;

Since the type and intensity of noise in the production line equipment processing are unknown, the noise data is simulated by adding common Gaussian white noise, Poisson noise and impulse noise to the original multi-source monitoring data;

Step 3: Input the simulated noise data set _Xn into a convolutional neural network to segment the sequence, dividing a single sample into multiple regular, non-overlapping subsequences, and mapping each subsequence into a high-dimensional vector. The noise signal after segmentation and mapping is Where _Np represents the number of blocks, D represents the dimension of the mapped vector, and Embed(·) represents the segmentation mapping operation;

Step 4: Randomly shuffle the subsequences after each sample mapping to obtain the data set Shuffle (Emed (X _n )), then randomly mask (delete) some subsequences, the masking ratio is r, and then use the sine and cosine functions to positionally encode the masked data set. The specific calculation formula is:

Where pos represents the position, i represents the index of the encoding dimension, Shuffle(·) represents the random shuffle operation, Mask(·) represents the masking operation, and _X′n is the noise dataset after adding noise, masking and position encoding;

By randomly masking a certain proportion of subsequences, we simulate the situation where there are missing values in the original multi-source monitoring data. Since Transformer does not have a memory function like recurrent neural networks or convolutional neural networks when processing sequence data, it cannot capture the position information in the input sequence, so position encoding needs to be introduced additionally.

Step 5: Establish a big data intelligent cleaning model based on the masked autoencoder. As shown in Figure 2, the big data intelligent cleaning model is divided into two parts: encoder and decoder, and both are composed of Transformer networks; the encoder maps the unmasked subsequence in each sample to a potential representation, which contains three Transformer modules, and the hidden layer dimension of each Transformer module is 256. Each Transformer module consists of layer normalization, multi-head self-attention mechanism, and multi-layer perceptron; the decoder reconstructs the potential representation and the learnable mask vector into the original input, and contains two Transformer modules with the same structure, and the hidden layer dimension of each Transformer module is 64; in addition, a fully connected layer is used to map the encoder hidden layer dimension to the decoder hidden layer dimension;

The feature map obtained by inputting the dataset _X′n into the encoder part of the model is Among them,/> A fully connected network is used to map the dimension of z _j to the decoder dimension to obtain/> The decoder input Z′ consists of high-level features Fc(z _j ) and a mask vector set/> Composition, the specific calculation formula is:

Z′＝Unshuffle([Fc(z _j )；Z _m ])+E _pos (3)

Among them, Unshuffle(·) means canceling the random shuffle operation, and E _pos means the position encoding vector;

Step 6: Input Z′ into the decoder for calculation to obtain the model’s prediction result The denoising result of the model is measured by the mean square error between the predicted result Y ₁ and the training set X ₁ , and the cleaning ability of the model is improved by minimizing the mean square error L _mse of the samples in Y ₁ and X _1. The calculation formula is as follows:

The mean square error between the output of the big data intelligent cleaning model and the original training data is taken as the loss. By minimizing the loss, the model can reconstruct the original data from the noisy data, that is, the model can have a certain data cleaning ability.

Step 7: Use the reconstruction error L _mse obtained in step 6 as the optimization target of the training phase and use the gradient descent method to update the model parameters θ:

Among them, α represents the learning rate;

Step 8: Repeat steps 2 to 7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations E is reached;

Step 9: Test the model. First, use simulated noise data to test the big data intelligent cleaning model. Process the test set _X2 through steps 2 to 4 to obtain the test sample noise, masking and position encoding result data set _X′n2 ; input the data set _X′n2 into the trained big data intelligent cleaning model to obtain the model's denoising result _Y2 for the noisy data, and calculate the signal-to-noise ratio between _Y2 and the test set _X2 . The calculation formula is:

Among them, SNR represents the signal-to-noise ratio, P _s2 represents the power of the test signal, and P _n2 represents the power of the noise signal. By adding noise and masking to the test set data and inputting it into the big data intelligent cleaning model, and calculating the signal-to-noise ratio of the test set data after the model cleaning, the model cleaning effect and generalization ability can be quantitatively measured.

At the same time, real data is used to test the big data intelligent cleaning model. The test set _X2 is directly input into the trained big data intelligent cleaning model to obtain the denoising result _Y2 ′ of the original multi-source monitoring data set. Since it is impossible to obtain ideal clean data without any noise, it is impossible to directly calculate the signal-to-noise ratio to quantitatively measure the cleaning effect of the model on the real data. However, the real data and the denoised data can be used for downstream tasks such as fault diagnosis or life prediction, and the cleaning effect can be measured by the diagnosis or prediction effect.

Embodiment: Taking the cleaning of multi-source monitoring data collected during mass production of a certain machine processing in a certain intelligent manufacturing production base as an example, the effectiveness of the method of the present invention is verified.

The original multi-source monitoring data of the machine processing process is extracted from the data acquisition system of the intelligent manufacturing production line, and the original multi-source monitoring data is preprocessed. The original multi-source monitoring data in a continuous period of time is divided into a training set and a test set at a ratio of 9:1. The two segments of data are randomly sampled to obtain 2000 training samples and 200 test samples, each of which contains 300 sample points; Gaussian white noise is added to the training set data and the signal-to-noise ratio is fixed to 0; each sample is divided into 50 subsequences, each sequence contains 5 sample points, and about 25% of the subsequences are randomly masked; the preprocessed noise-added data is input into the big data intelligent cleaning model for training. The encoder of the big data intelligent cleaning model stacks three Transformer modules, and the hidden layer dimension of each module is 256. The decoder is narrower and shallower than the encoder, and stacks two Transformer modules, and the hidden layer dimension of each module is 64; the training parameters of the big data intelligent cleaning model are shown in Table 1:

Table 1. Training parameters of big data intelligent cleaning model

After the training of the big data intelligent cleaning model is completed, the test set is subjected to the same data preprocessing and then input into the big data intelligent cleaning model to test the cleaning effect; in order to reduce the randomness of the experiment, the experiment is repeated 5 times, and the statistical values of the cleaning results are calculated; the average signal-to-noise ratios of some samples in the test set after cleaning are 4.10, 6.02, 5.16, and 5.94, respectively. It can be seen that the big data intelligent cleaning model can effectively remove the additional noise added to the data and reconstruct the missing values, and has good generalization ability.

In order to more fully verify the effectiveness of the inventive method, the data cleaned by the inventive method is used to perform fault diagnosis on the machine. The constructed fault diagnosis model includes two parts: feature extraction and state recognition. The feature extraction module is the encoder of the big data intelligent cleaning model in the inventive method. The state recognition module contains two fully connected layers. The activation function of the first fully connected layer uses LeakyReLU and a Dropout layer with a probability of 0.2. The second fully connected layer outputs the predicted state and uses the Softmax function to normalize the predicted value. The training parameters of the fault diagnosis model are shown in Table 2, and the diagnosis results are shown in Table 3. In Table 3, the model structure of comparison method 1 is the same as that of the inventive method, but the weights of its feature extraction module are randomly initialized, that is, the input data is not cleaned. The feature extraction module of comparison method 2 is composed of a convolutional neural network, and the number of network layers is also three. It can be seen from Table 3 that after adopting the intelligent cleaning method of the present invention, the accuracy of intelligent fault diagnosis can reach 92.75%. Comparative method 1 does not clean the original data, so the average diagnostic accuracy can only reach 86.09%, which is much lower than the method of the present invention. In addition, due to the interference of noise, the training is not stable and the test results have large fluctuations. Comparative method 2 uses convolutional neural network for feature extraction, and its feature mining ability is lower than that of Transformer network. The average diagnostic accuracy is only 80.72%, which is much lower than the method of the present invention.

Table 2 Fault diagnosis model training parameters

Table 3 Diagnostic results of different methods

By comparing the diagnostic effects of the method of the present invention with those of comparison method 1 and comparison method 2, it is shown that the method of the present invention can effectively reduce the noise in the original multi-source monitoring data, improve the quality of the data, thereby improving the effect of subsequent intelligent diagnosis, and verifying the generalization ability of the method of the present invention; in addition, the Transformer network used in the present invention has a strong feature extraction capability, and its performance is better than that of the traditional convolutional neural network.

Claims (3)

1. An intelligent cleaning method for low-quality monitoring big data of intelligent manufacturing production lines, characterized by: simulating noise data by adding multiple noises to the original multi-source monitoring data and masking a certain proportion of subsequences in each sample, using a Transformer network to map the simulated noise data into potential representations, and reconstructing the potential representations into the original multi-source monitoring data, thereby reducing the impact of complex noise in the original multi-source monitoring data on subsequent analysis. 2. A smart cleaning method for low-quality monitoring big data of smart manufacturing production lines, characterized by comprising the following steps: Step 1: Obtain the original multi-source monitoring data set during the equipment processing from the data recording file of the production line Where N represents the number of samples, /> is the i-th sample, the sequence length of each sample is L, and the number of channels is C; then the data set X is divided into training sets/> and test set/> Among them, N ₁ represents the number of samples in the training set; Step 2: Add Gaussian noise _xG , Poisson noise _xP and impulse noise _xS to the sample _xi of the training set _X1 to obtain simulated noise data _xn . The specific calculation formula is as follows: Where _Ps represents the power of the original vacuum value signal, _Pn represents the power of the Gaussian noise signal, SNR represents the signal-to-noise ratio, Var(·) represents the variance, and _xrand∈N (0,1) represents a random sample sequence that obeys the standard normal distribution, and its sequence length is the same as _xi ; Step 3: Input the simulated noise data set _Xn into a convolutional neural network to segment the sequence, dividing a single sample into multiple regular, non-overlapping subsequences, and mapping each subsequence into a high-dimensional vector. The noise signal after segmentation and mapping is Where _Np represents the number of blocks, D represents the dimension of the mapped vector, and Embed(·) represents the segmentation mapping operation; Step 4: Randomly shuffle the subsequences after each sample mapping to obtain the data set Shuffle (Emed (X _n )), then randomly mask (delete) some subsequences, the masking ratio is r, and then use the sine and cosine functions to positionally encode the masked data set. The specific calculation formula is: Where pos represents the position, i represents the index of the encoding dimension, Shuffle(·) represents the random shuffle operation, Mask(·) represents the masking operation, and _X′n is the noise dataset after adding noise, masking and position encoding; Step 5: Establish a big data intelligent cleaning model based on the masked autoencoder, whose encoder and decoder both use the Transformer network; input the dataset _X′n into the encoder part of the model and calculate the feature map obtained: Among them,/> A fully connected network is used to map the dimension of z _j to the decoder dimension to obtain/> The decoder input Z′ consists of high-level features Fc(z _j ) and a learnable mask vector set Composition, the specific calculation formula is: Z′＝Unshuffle([Fc(z _j )；Z _m ])+E _pos (3) Among them, Unshuffle(·) means canceling the random shuffle operation, and E _pos means the position encoding vector; Step 6: Input Z′ into the decoder for calculation to obtain the model’s prediction result The denoising result of the model is measured by the mean square error between the predicted result Y ₁ and the training set X ₁ , and the cleaning ability of the model is improved by minimizing the mean square error L _mse of the samples in Y ₁ and X _1. The calculation formula is as follows: Step 7: Use the reconstruction error L _mse obtained in step 6 as the optimization target of the training phase and use the gradient descent method to update the model parameters θ: Among them, α represents the learning rate; Step 8: Repeat steps 2 to 7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations is reached; Step 9: Test the model. First, use simulated noise data to test the big data intelligent cleaning model. Process the test set _X2 through steps 2 to 4 to obtain the test sample noise, masking and position encoding result data set _X′n2 ; input the data set _X′n2 into the trained big data intelligent cleaning model to obtain the model's denoising result _Y2 for the noisy data, and calculate the signal-to-noise ratio between _Y2 and the test set _X2 . The calculation formula is: Wherein, SNR represents signal-to-noise ratio, P _s2 represents the power of the test signal, and P _n2 represents the power of the noise signal; At the same time, real data is used to test the big data intelligent cleaning model. The test set X ₂ is directly input into the trained big data intelligent cleaning model to obtain the denoising result Y ₂ ′ of the original multi-source monitoring data set. 3. According to the method of claim 2, it is characterized in that: in step 5, the big data intelligent cleaning model is divided into two parts, an encoder and a decoder, and both are composed of a Transformer network; the encoder maps the unmasked subsequence in each sample to a potential representation, which includes three Transformer modules, and the hidden layer dimension of each Transformer module is 256. Each Transformer module consists of layer normalization, a multi-head self-attention mechanism, and a multi-layer perceptron; the decoder reconstructs the potential representation and the learnable mask vector into the original input, and includes two Transformer modules with the same structure, and the hidden layer dimension of each Transformer module is 64; in addition, a fully connected layer is used to map the encoder hidden layer dimension to the decoder hidden layer dimension.