CN114692507A - A Soft Sensing Modeling Method for Counting Data Based on Stacked Poisson Autoencoder Networks - Google Patents

Abstract

The invention discloses a counting data soft measurement modeling method based on a stacked Poisson autoencoder network, wherein a stacked Poisson autoencoder network structure is proposed. The encoder introduces count-type quality variables in the pre-training stage to guide feature extraction, and for the discreteness of count data, the quality variables are integrated into the deep stacked autoencoder framework through Poisson regression network layers, so that the model learns The feature representation of is highly correlated with count-type quality variables. The method of the invention not only improves the feature extraction ability of the count data soft measurement model, but also improves the prediction effect of the count quality variable.

Description

A Soft Sensing Modeling Method for Counting Data Based on Stacked Poisson Autoencoder Networks

technical field

The invention belongs to the field of industrial process prediction and soft measurement, and relates to a counting data soft measurement modeling method based on a stacked Poisson autoencoder network.

Background technique

As an important data type, count data has the characteristics of discrete, non-negative integer, and highly skewed distribution. ) and the factors that cause it (called independent variables, input variables, or process variables) to predict the number of occurrences of an event.

In the process industry, soft sensing as a tool to predict product quality or other important variables can be considered for modeling count data. Common data-driven soft-sensor modeling methods are multiple linear regression (MLR) and partial least squares (PLS) regression. They assume that the response variable follows a normal and homoscedastic distribution, which is contrary to the observed highly overdispersed distribution of count data. Also count data are non-negative integers, but MLR and PLS may produce negative values for the dependent variable. However, nonlinear modeling methods such as Support Vector Regression (SVR) and Artificial Neural Network (ANN) methods suffer from poor interpretability and cannot guarantee the non-negativity of predictions.

For count data, Poisson regression model is a typical representative of its modeling. However, in the industrial process, the process data has the characteristics of high dimensionality and nonlinearity, and the Poisson regression has the problem of insufficient data feature mining when it is used in the industrial process. Therefore, extracting deep features of process data is a crucial step in soft-sensor modeling of count data.

As its typical representative, the autoencoder structure has been designed and widely used in complex industrial processes. However, the pre-training of traditional autoencoders adopts an unsupervised learning method, which learns effective feature representation by reconstructing the input and constraining the error minimization. Therefore, the features extracted from the deep network may be related to the soft measurement of the count data. The predicted output of , is irrelevant, making this part of the process inefficient.

When predicting the counting data of an industrial process, since there may be many process variables, and the data has the characteristics of non-linearity and high dimensionality, when the soft sensing model of the counting data is established, the quality variables extracted with the counting data type are highly correlated. features are very necessary. In view of the above problems in the feature extraction stage, if a reasonable way can be designed to introduce quality variables to effectively guide the feature extraction of input data, and at the same time consider the characteristics of count data, then this problem can be easily solved.

SUMMARY OF THE INVENTION

Aiming at the problem that the conventional autoencoder cannot extract the relevant features of the quality variable, and considering the discrete, non-negative and high skew characteristics of the count data, the present invention proposes a count data soft sensing modeling method based on a stacked Poisson autoencoder network . The method of the invention introduces count-type quality variables in the decoding stage of pre-training to guide feature extraction, and integrates the count-type quality variables into the deep stacking encoder structure through the Poisson network layer, so that the feature representation learned by the model is consistent with the count-type quality variables. The variables are highly correlated, which improves the efficiency of feature extraction and improves the prediction effect of count-type quality variables.

The concrete technical scheme of the present invention is as follows:

A soft-sensor modeling method for count data based on stacked Poisson autoencoder network, the method includes the following steps:

其中，x代表输入变量，y代表离散计数数据类型的输出变量，N表示数据样本个数；S1: Collect input and output training datasets for modeling:

Among them, x represents the input variable, y represents the output variable of the discrete count data type, and N represents the number of data samples;

S2: Construct a stacked Poisson autoencoder network. The stacked Poisson autoencoder network is layered by multiple supervised Poisson autoencoders, and the output of the hidden layer of the previous supervised Poisson autoencoder is used as the lower An input to the input layer of a supervised Poisson autoencoder; the supervised Poisson autoencoder includes an input layer, a hidden layer, and an output layer, from the hidden layer to the output layer, including the input reconstruction network layer and the Poisson network layer , the input reconstruction network layer is used to reconstruct the input vector, and the Poisson network layer is used to predict the counted quality data;

Randomly initialize the Poisson network weights, neural network connection weights, and bias parameters of the stacked Poisson autoencoder network.

和隐藏层的输出

将h¹作为第二个监督泊松自编码器的输入层的输入，根据最小化损失函数训练第二个监督泊松自编码器，获得对应的权重和偏置参数，以此层层递进，使用h^{k -1}，根据训练第k个监督泊松自编码器SPAE^k获得参数

和h^k，直到最后一个监督泊松自编码器训练完成；k≤L，其中，L为监督泊松自编码器的数量；S3: Input the training data to the stacked Poisson autoencoder network, train the first supervised Poisson autoencoder according to the minimized loss function, and obtain the weight and bias parameters of the first supervised Poisson autoencoder

and the output of the hidden layer

Take h ¹ as the input of the input layer of the second supervised Poisson autoencoder, train the second supervised Poisson autoencoder according to the minimized loss function, obtain the corresponding weight and bias parameters, and progress through this layer by layer , using h ^{k -1} , parameters obtained from training the k-th supervised Poisson autoencoder SPAE ^k

and h ^k until the last supervised Poisson autoencoder is trained; k≤L, where L is the number of supervised Poisson autoencoders;

S4: After the layer-by-layer training of S3, a Poisson network is established between the output h ^L of the hidden layer of the L-th supervised Poisson autoencoder and the output variable y for regression, and the parameters of the regression network are adjusted and updated according to the prediction error. ; The regression network training ends and the stacked Poisson autoencoder network is saved;

S5: Input the input data to be predicted into the saved stacked Poisson autoencoder network, and the predicted value of the counted quality variable can be obtained through the forward propagation of the stacked Poisson autoencoder network.

Further, in S3, the encoder in the supervised Poisson autoencoder is expressed as:

h=σ(W _e ·x _+be )

where σ represents the sigmoid activation function, x is the input vector of the input layer, h is the output vector of the hidden _layer , and We and be represent the weight and _bias of the encoder, respectively;

The decoder in a supervised Poisson autoencoder is represented as:

表示重构后的输入向量，

分别预测的输出向量；where exp represents the exponential function, W _r and _br represent the weight and bias of the reconstructed input vector in the decoder, respectively; W _p and _bp represent the weight and bias parameters of the Poisson network layer, respectively,

represents the reconstructed input vector,

separately predicted output vectors;

The loss function _Lrec is expressed as:

的含义为二范数，⊙表示哈达玛积。Among them, λ represents the ratio of the weight of the reconstruction error of the input vector to the prediction error of the output vector;

The meaning of is the second norm, and ⊙ represents the Hadamard product.

Further, in S3, the training process of the k-th supervised Poisson autoencoder is expressed as follows:

和

分别是第i个样本在第k个监督泊松自编码器的输入数据和重构的数据，

和

分别是第k层编码器和解码器的权重矩阵以及偏置项；Among them, k=1,2,...L,

and

are the input data and reconstructed data of the i-th sample in the k-th supervised Poisson autoencoder, respectively,

and

are the weight matrices and bias terms of the encoder and decoder of the kth layer, respectively;

This is achieved through the following sub-steps:

The loss function trained by the k-th supervised Poisson autoencoder is as follows:

分别代表第i个样本对应的计数型质量变量实际观测值和其在第k个监督泊松自编码器的预测值。where y _i and

respectively represent the actual observed value of the count-type quality variable corresponding to the ith sample and its predicted value in the kth supervised Poisson autoencoder.

的计算公式如下：Further, in the S4, the predicted output variable

The calculation formula is as follows:

where W _y and b _y represent the weight and bias of the Poisson network, respectively;

The loss function is as follows:

The beneficial effects of the present invention are as follows:

The counting data soft-sensor modeling method based on stacked Poisson autoencoder network proposed in the present invention is used for counting data quality prediction, so as to solve the problem that the feature extraction efficiency of conventional autoencoders is low and is not suitable for counting data modeling. By adding a count-type quality variable to the output layer of the decoding stage, and considering the discreteness and non-negativity of the count data, the count data is integrated into the deep autoencoder framework through a Poisson regression network layer, which improves the loss. function, so that the model can learn features that are highly correlated with the quality variables of the count data, and the prediction effect of the model on the count data is improved.

Description of drawings

Figure 1 is a structural diagram of a deep stacked Poisson autoencoder (SSPAE);

Fig. 2 is a flow chart of soft sensing modeling of counting data based on SSPAE;

Fig. 3 is the flow chart of iron and steel casting and rolling process defect system;

Figure 4 shows the prediction results of SSPAE, STAE and SAE methods, corresponding to sub-figures (c), (b) and (a) respectively, where the abscissa represents the measurement sample, the ordinate represents the value of the quality data, and "+" in the figure represents Model predicted value, "*" represents the true value.

Detailed ways

The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become clearer.

The method of the present invention is based on the stacked Poisson autoencoder network (SSPAE) structure, improves the encoder on the basis of the original autoencoder, and introduces a quality variable in the pre-training decoding stage to guide feature extraction. In addition, considering the count The discreteness of the data and the quality variable are integrated into the deep stacked autoencoder framework through the Poisson regression network layer, so that the feature representation learned by the model is highly correlated with the count data type quality variable, which improves the feature extraction efficiency and improves the accuracy of the data. Soft-sensor accuracy for count data.

As shown in Figure 1, the concrete steps of the method of the present invention are as follows:

其中，x代表输入变量，y代表离散计数数据类型的输出变量，N表示数据样本个数；并将数据集分为训练集、验证集和测试集，根据不同工况进行数据预处理；S1: Collect device data to form input and output training data sets for modeling:

Among them, x represents the input variable, y represents the output variable of the discrete count data type, and N represents the number of data samples; the data set is divided into training set, verification set and test set, and data preprocessing is performed according to different working conditions;

S2: Build a stacked Poisson autoencoder network SSPAE, as shown in Figure 2, SSPAE is composed of multiple supervised Poisson autoencoder SPAE layers stacked, and the output of the hidden layer of the previous supervised Poisson autoencoder is used as the lower An input to the input layer of a supervised Poisson autoencoder; the supervised Poisson autoencoder includes an input layer, a hidden layer, and an output layer, from the hidden layer to the output layer, including the input reconstruction network layer and the Poisson network layer , the input reconstruction network layer is used to reconstruct the input vector, and the Poisson network layer is used to predict the counted quality data;

和

SPAE预测质量数据采用针对计数数据的泊松网络。Suppose x is the input vector, h is the hidden vector, and y is the quality variable. In the decoder, SPAE will simultaneously reconstruct its input data and predict the count data, which are expressed as

and

SPAE predicts quality data using a Poisson network for count data.

{W _{e ,} be } and {W _d ,b _d } are used to denote the parameter sets of the encoder and decoder, respectively. The encoder is represented as:

h=σ(W _e ·x _+be )

where σ represents the sigmoid activation function.

Since the decoder consists of two parts, its decoding weight matrix and decoding bias vector can be decomposed into two parts: input data and count data quality variables. That is, its parameters can be decomposed into:

In the output layer of SPAE, the reconstructed input data is mapped by the hidden data through the input reconstruction network layer:

In particular, for discrete count data modeling, the mapping of hidden features to output data is a Poisson network layer as follows:

Among them, exp represents the exponential function, {W _p , b _p } represents the weight and bias parameters of the Poisson network layer. On the one hand, the error structure of the Poisson distribution of the Poisson network layer enables the data to have nonlinear characteristics and non-constant variance structure; on the other hand, the Poisson network layer can guarantee the non-negativity of the prediction.

So, the decoder output of SPAE can be expressed as:

(2) Given input training data X={x ₁ ,x ₂ ,...,x _N } and corresponding counting quality data Y={y ₁ ,y ₂ ,...,y _N }, the way SPAE network learns parameters is By minimizing the loss function of the output layer as follows:

的含义为二范数，⊙表示哈达玛积。Among them, λ represents the ratio of the weight of the reconstruction error of the input vector to the prediction error of the output vector;

The meaning of is the second norm, and ⊙ represents the Hadamard product.

和第一潜在特征表示

(3) Train the first supervised Poisson autoencoder according to the above minimized loss function to obtain parameters

and the first latent feature representation

Randomly initialize the Poisson network weights, neural network connection weights and bias parameters of the SSPAE model.

和隐藏层的输出

将h¹作为第二个监督泊松自编码器的输入层的输入，根据最小化损失函数训练第二个监督泊松自编码器，获得对应的权重和偏置参数，以此层层递进，使用h^{k -1}，根据训练第k个监督泊松自编码器SPAE^k获得参数

和h^k，直到最后一个监督泊松自编码器训练完成；k≤L，其中，L为监督泊松自编码器的数量；S3: Input the training data to the stacked Poisson autoencoder network, train the first supervised Poisson autoencoder according to the minimized loss function, and obtain the weight and bias parameters of the first supervised Poisson autoencoder

and the output of the hidden layer

Take h ¹ as the input of the input layer of the second supervised Poisson autoencoder, train the second supervised Poisson autoencoder according to the minimized loss function, obtain the corresponding weight and bias parameters, and progress through this layer by layer , using h ^{k -1} , parameters obtained from training the k-th supervised Poisson autoencoder SPAE ^k

and h ^k until the last supervised Poisson autoencoder is trained; k≤L, where L is the number of supervised Poisson autoencoders;

S3 specifically includes the following sub-steps:

的第一隐藏层，生成相应的第一级特征数据

SPAE¹在其输出层使用输入重构网络层重构原始输入数据以及泊松网络层预测质量数据。其预训练是通过最小化训练数据上的重建和预测误差来进行的，如下所示：(1) Build a deep SPAE network by stacking multiple SPAEs in layers, and transmit the input data to the input layer of SSPAE.

The first hidden layer of , generates the corresponding first-level feature data

SPAE ¹ uses the input reconstruction network layer to reconstruct the original input data and the Poisson network layer to predict the quality data in its output layer. Its pre-training is performed by minimizing the reconstruction and prediction errors on the training data as follows:

紧接着，第一层隐藏特征向量会作为第二个SPAE的输入，经过映射获得第二层特征向量。在SPAE²的输出层，第一层特征向量和计数数据质量向量也由第二层特征向量重构和预测。以此获得第二级特征数据

对于其余层次的特征，可以通过类似的方式逐步获得。(2) After SPAE ¹ is pre-trained, its encoder part remains in the entire SPAE network, which can be calculated

Next, the hidden feature vector of the first layer will be used as the input of the second SPAE, and the feature vector of the second layer will be obtained through mapping. At the output layer of SPAE ² , the first-layer feature vectors and count data quality vectors are also reconstructed and predicted from the second-layer feature vectors. In this way, the second-level feature data is obtained

For the features of the remaining levels, it can be obtained step by step in a similar way.

已经学习到，它将通过参数集为

的非线性函数获得第k层特征隐层

然后在输出层通过输入重构网络层映射重构h^k-1，特别地，由泊松网络层预测输出计数数据。Suppose the k-1th feature hidden layer

It has been learned that it will pass the parameter set as

The nonlinear function of get the k-th feature hidden layer

Then at the output layer the reconstruction h ^k-1 is mapped by the input reconstruction network layer, in particular, the output count data is predicted by the Poisson network layer.

The process looks like this:

和

分别是第i个样本在第k个自编码器的输入数据和重构的数据，

和

分别是第k层编码器和解码器的权重矩阵以及偏置项。Among them, k=1,2,...L,

and

are the input data and reconstructed data of the i-th sample in the k-th autoencoder, respectively,

and

are the weight matrices and bias terms of the encoder and decoder of the kth layer, respectively.

S4: After the layer-by-layer training of S3, a Poisson network is established between the output h ^L of the hidden layer of the L-th supervised Poisson autoencoder and the output variable y for regression, and the parameters of the regression network are adjusted and updated according to the prediction error. ; The regression network training ends and the stacked Poisson autoencoder network is saved;

并最小化预测误差的损失函数去更新SSPAE网络的相关参数：After the forward layer-by-layer training, the output mapping network is added at the top layer, and the output data is predicted by using the feature vector h ^L of the last hidden layer

And minimize the loss function of the prediction error to update the relevant parameters of the SSPAE network:

where W _y and b _y represent the weights and biases of the Poisson network, respectively.

S6: Input the input data to be predicted into the saved SSPAE model, and through the forward propagation of the SSPAE network, the predicted value of the counted quality variable can be obtained.

The effectiveness of the present invention will be described below with reference to a specific industrial example. In order to improve product quality and save production costs, it is very important to predict the number of defects in steel plates in real time. For example, based on the online prediction of the number of defects, the operator can change the operating conditions to control the occurrence of defects; in addition, the defect prediction model provides an early measure of the number of defects, which helps the operator to prevent further deterioration of the operating conditions; in addition, Based on the defect prediction model, the key factors affecting the incidence of defects can be further explored.

The data used is a certain type of steel defect data collected from a steelmaking plant during secondary refining, continuous casting, rolling and cooling, as shown in Figure 3. The data contains process variable data and quality variable data. The process variable data includes 146 process operation variables such as heating temperature, which are continuous variables, and the data has strong nonlinearity. The quality variable represents the number of defects and is a discrete count data type. In this experiment, the collected 2500 samples were randomly divided into three data sets, of which 1500 samples were used as training data set for model training, 500 samples were used as validation data set for model parameter selection, and 500 samples were used for testing The dataset is used for model testing. ,

Table 1: Network Structure Parameters

Network layer input layer Hidden layer 1 Hidden layer 2 Hidden layer 3 Hidden layer 4 number of nodes 146 83 43 10 5

For comparative analysis, various methods including Stacked Supervised Poisson Autoencoder SSPAE, Stacked Target Dependent Autoencoder STAE, Stacked Autoencoder SAE and Poisson Regression PS were used for soft sensing construction of count data in this industrial process. mold. The network structure parameters of SSPAE, STAE and SAE are shown in Table 1. The key hyperparameter λ in the SSPAE model is set to 1.5.

Table 2: Comparison of the prediction performance of each comparison method on the test set

Table 2 provides the comparison of the prediction accuracy of each comparison method under the two model evaluation indicators of root mean square error RMSE and correlation coefficient R ² . The smaller the RMSE and the smaller the model prediction error. The larger the R ² , the higher the prediction accuracy of the model. It can be seen from the two indicators that the prediction performance of the method SSPAE of the present invention is the best, with the smallest prediction error and the highest prediction accuracy.

Figure 4 shows the partial prediction effect diagrams of the SSPAE model, the STAE model, and the SAE model, respectively, where the abscissa represents the measurement samples, and the ordinate represents the value of the quality data. Figure 4(c) shows the method of the present invention. It can be seen from the figure that the SSPAE method of the present invention fits the predicted value of the number of steel defects more closely with the actual value, and the prediction result is more accurate.

Those of ordinary skill in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still Modifications are made to the technical solutions described in the foregoing examples, or equivalent replacements are made to some of the technical features. All modifications and equivalent replacements made within the spirit and principle of the invention shall be included within the protection scope of the invention.

Claims (4)

1. a counting data soft-sensor modeling method based on stacked Poisson autoencoder network, is characterized in that, this method comprises the steps: S1: Collect input and output training datasets for modeling:

Among them, x represents the input variable, y represents the output variable of the discrete count data type, and N represents the number of data samples; S2: Construct a stacked Poisson autoencoder network. The stacked Poisson autoencoder network is layered by multiple supervised Poisson autoencoders, and the output of the hidden layer of the previous supervised Poisson autoencoder is used as the lower An input to the input layer of a supervised Poisson autoencoder; the supervised Poisson autoencoder includes an input layer, a hidden layer, and an output layer, from the hidden layer to the output layer, including the input reconstruction network layer and the Poisson network layer , the input reconstruction network layer is used to reconstruct the input vector, and the Poisson network layer is used to predict the counted quality data; Randomly initialize the Poisson network weights, neural network connection weights, and bias parameters of the stacked Poisson autoencoder network. S3: Input the training data to the stacked Poisson autoencoder network, train the first supervised Poisson autoencoder according to the minimized loss function, and obtain the weight and bias parameters of the first supervised Poisson autoencoder

and the output of the hidden layer

Take h ¹ as the input of the input layer of the second supervised Poisson autoencoder, train the second supervised Poisson autoencoder according to the minimized loss function, obtain the corresponding weight and bias parameters, and progress through this layer by layer , using h ^k-1 , parameters obtained from training the k-th supervised Poisson autoencoder SPAE ^k

and h ^k until the training of the last supervised Poisson autoencoder is completed; k≤L, where L is the number of supervised Poisson autoencoders; S4: After the layer-by-layer training of S3, a Poisson network is established between the output h ^L of the hidden layer of the L-th supervised Poisson autoencoder and the output variable y for regression, and the parameters of the regression network are adjusted and updated according to the prediction error. ; The regression network training ends and the stacked Poisson autoencoder network is saved; S5: Input the input data to be predicted into the saved stacked Poisson autoencoder network, and the predicted value of the counted quality variable can be obtained through the forward propagation of the stacked Poisson autoencoder network. 2. the counting data soft measurement modeling method based on stacked Poisson autoencoder network according to claim 1, is characterized in that, in described S3, the encoder in the supervision Poisson autoencoder is expressed as: h=σ(W _e ·x _+be ) where σ represents the sigmoid activation function, x is the input vector of the input layer, h is the output vector of the hidden _layer , and We and be represent the weight and _bias of the encoder, respectively; The decoder in a supervised Poisson autoencoder is represented as:

where exp represents the exponential function, W _r and _br represent the weight and bias of the reconstructed input vector in the decoder, respectively; W _p and _bp represent the weight and bias parameters of the Poisson network layer, respectively,

represents the reconstructed input vector,

represents the predicted output vector; The loss function _Lrec is expressed as:

Among them, λ represents the ratio of the weight of the reconstruction error of the input vector to the prediction error of the output vector;

The meaning of is the second norm, and ⊙ represents the Hadamard product. 3. the counting data soft measurement modeling method based on stacking Poisson autoencoder network according to claim 1, is characterized in that, in described S3, the training process of the kth supervised Poisson autoencoder is represented as follows:

Among them, k=1,2,...L,

and

are the input data and reconstructed data of the i-th sample in the k-th supervised Poisson autoencoder, respectively,

and

are the weight matrices and bias terms of the encoder and decoder of the kth layer, respectively; This is achieved through the following sub-steps: The loss function trained by the k-th supervised Poisson autoencoder is as follows:

where y _i and

represent the actual observed value of the count quality variable corresponding to the i-th sample and its predicted value in the k-th supervised Poisson autoencoder, respectively. 4. the count data soft-sensor modeling method based on stacked Poisson autoencoder network according to claim 1, is characterized in that, in described S4, the output variable of prediction

The calculation formula is as follows:

where W _y and b _y represent the weight and bias of the Poisson network, respectively; The loss function is as follows:

Images