CN108804784A - A kind of instant learning soft-measuring modeling method based on Bayes's gauss hybrid models - Google Patents

Abstract

The invention discloses an instant learning soft sensor modeling method based on a Bayesian Gaussian mixture model, which belongs to the field of complex industrial process modeling and soft sensor. The present invention is used for nonlinear and non-Gaussian time-varying industrial processes. Through an online real-time local update strategy, the Bayesian information criterion is used to determine the optimal number of Gaussian components. When new test data arrives, Calculate the posterior probability that it belongs to each Gaussian component, and find the Mahalanobis distance between it and the training data, and use the fusion of the two as the similarity index; finally, select the one with the largest similarity from the original training samples Group data to establish the current GPR model and predict the output of the model, achieving the effect of improving product quality and reducing production costs.

Description

An Instant Learning Soft Sensor Modeling Method Based on Bayesian Gaussian Mixture Model

technical field

The invention relates to an instant learning soft sensor modeling method based on a Bayesian Gaussian mixture model, which belongs to the fields of complex industrial process modeling and soft sensor.

Background technique

For some non-linear, time-varying and non-Gaussian industrial processes, and the continuous improvement of product quality requirements in the process, it is necessary to strictly monitor and control some process variables that directly determine product quality. However, due to the high price of some measuring instruments or the restriction of technical conditions, these variables cannot be measured by online instruments.

In order to solve these problems, it is possible to estimate and predict by establishing a soft sensor model. Commonly used soft sensor methods include partial least squares (PLS), artificial neural networks (ANN), support vector machine ( support vector machine, SVM), etc. PLS can deal with the linear problem of the process very well, but the actual industrial process is often nonlinear, so the linear method is no longer applicable. Although nonlinear modeling methods such as ANN and SVM can handle the nonlinearity of the process well, there are problems such as many optimization parameters.

In recent years, Gaussian process regression (GPR) has received more and more attention. As a non-parametric probability model, it can not only obtain the predicted value of the model, but also obtain the trust value of the predicted value to the model. Compared with ANN, SVM and other methods, GPR needs fewer parameters to be optimized, and has unique advantages in solving small sample and nonlinear problems. Hence the choice of GPR modeling.

After the model established offline is put into operation, due to some changes in the production environment and continuous changes in product quality requirements, the previously established model may no longer be applicable to the current working conditions, and the predicted results cannot meet the requirements. Accuracy requirements. Aiming at this problem, commonly used solutions are based on sliding window (Moving window, MW) and based on instant learning (Just-in-time learning, JITL) method, but the window length of MW is difficult to determine and is not suitable for the sudden change of the process. According to the principle that similar input produces similar output, the JITL method selects a group of training samples most similar to the test sample to build a local model for prediction, which can better solve the problem of process mutation.

However, for some non-Gaussian time-varying industrial processes, the traditional JITL method selects similar data based on the Euclidean distance or the similarity criterion combined with the angle, which cannot fully consider the non-Gaussian nature of the process data.

Contents of the invention

In order to solve the existing problems, the present invention proposes a real-time learning soft sensor modeling method based on the Bayesian Gaussian mixture model, which not only takes into account the time-varying nature of the process, but also fully considers the Due to the non-Gaussian nature of the data, it is more reasonable to choose similar data. The methods include:

Step 1: Collect input and output data to obtain a historical training data set;

Step 2: X is a known training sample, use Bayesian information criterion BIC to determine the optimal number of Gaussian components K, the description of BIC is as formula (1):

BIC＝-2logp(X|Θ)+dlog(N) (1)

In formula (1), logp(X|Θ) represents the logarithmic likelihood function of the training samples, d represents the number of free parameters of the K Gaussian components, and N represents the number of training samples

Step 3: According to the optimal number K of Gaussian components and the initial parameters of the given Gaussian mixture model GMM, use formulas (5), (6), and (7) to iterate continuously until the difference between the two parameters is less than the set value After setting the threshold, the parameter Θ of the final GMM is obtained. The detailed description of the GMM is as follows:

A data set X{x _i ∈ R ^m ,i=1,2…N} containing N training samples, m represents the dimension of the input data, and the probability density function of the data set is expressed as:

Among them, Θ=[α ₁ , μ ₁ , Σ ₁ ; α ₂ , μ ₂ , Σ ₂ ; ...; α _k , μ _k , Σ _k ] are parameters of GMM, K is the number of Gaussian components, θ _k is the parameter of the kth Gaussian component, θ _k = (μ _k ,Σ _k ), μ _k and Σ _k are the mean value and covariance matrix of the kth Gaussian component respectively, and α _k is the proportion of the kth Gaussian component ratio, and 0<α _k <1, where the probability density function of the kth Gaussian component is:

The unknown parameters in the GMM method are solved by the expectation maximization algorithm. The specific solution process is divided into E step and M step, which are described as follows:

Step E: According to the parameters updated for the current lth time and Calculate the probability that the i-th training sample belongs to the k-th Gaussian component by Bayesian formula where C _k denotes the kth Gaussian component

Step M: update algorithm parameters

Step 4: When a new input data x _q comes, use the real-time learning JITL algorithm to select the most similar set of data from the historical data set to establish a local Gaussian process regression GPR model, the details of the JITL algorithm and GPR modeling method The descriptions are as follows:

JITL algorithm: The JITL method is based on the idea of similar inputs to generate similar outputs. From the training samples, a group of training samples that are most similar to the current incoming test samples are selected to model. The core of JITL is the selection of similarity criteria, based on Euclidean distance The similarity criterion of and angle is a commonly used method, namely:

Among them, the distance d represents the 2-norm between the currently arriving test sample and the training sample, θ represents the angle between the two samples, and γ is a coefficient with a value between 0 and 1;

However, for some non-Gaussian industrial processes, GMM can better describe the non-Gaussian nature of the process. Compared with the traditional similarity criterion, the similarity criterion based on the Bayesian Gaussian mixture model BGMM can better select similar samples to establish a GPR model, the optimal number of Gaussian components K and the parameters Θ of each component obtained by steps 2 and 3 respectively, the corresponding similarity criterion can be expressed as:

where x _q represents the new incoming sample, x _i represents the i-th training sample, p(C _k |x _q ) represents the posterior probability that the new incoming sample x _q belongs to the k-th Gaussian component, is the Mahalanobis distance between two samples, for the current incoming x _q , using the above similarity criterion, select a set of data most similar to x _q to establish the current GPR model

GPR modeling method: the known training sample sets X{ _xi ∈R ^m ,i=1,2…N} and Y{y _i ∈R,i=1,2…N} represent m-dimensional input data and 1 Dimensional output data, the relationship between input and output can be expressed as:

y _i =f(x _i )+ε (10)

Where f represents an unknown function form, ε represents the mean value is 0, and the variance is white noise

For a new test sample x _q , its output prediction value y _q also satisfies the Gaussian distribution, and its mean and variance are expressed as:

y _q (x _q )＝c ^T (x _q )C ^-1 Y (11)

Among them, c(x _q )=[c(x _q ,x ₁ ),…,c(x _q ,x _N )] ^T is the covariance matrix of the test input data and the training input data, is the covariance matrix between the training input data, c(x _q , x _q ) represents the covariance value between the test input data and itself;

GPR selects the radial basis covariance function, and its function is described as follows:

Among them, v represents the overall measure of prior knowledge, ω _t represents the weight corresponding to each dimension of data, and δ _ij is the Kronecher operator, representing the relative importance of each auxiliary variable;

The parameters in formula (13) are obtained by maximum likelihood estimation Its log-likelihood function is:

Use the training set and verification set to try out the parameter θ, and then use the conjugate gradient method to obtain the optimized parameters. After the parameters are determined, for the new test data, the output of the soft sensor model can be obtained by formula (11);

Step 5: Bring the newly arrived sample point x _q into the local GPR model established in step 4 to obtain the final estimated value y _q .

Optionally, the method is a prediction method applied to variables that cannot be directly measured in complex industrial processes.

Optionally, the complex industrial process includes chemical industry, metallurgy and fermentation process.

Optionally, the method is a prediction method for butane concentration applied in the debutanizer process.

The beneficial effects of the present invention are:

Through an online real-time local update strategy, the Bayesian information criterion is used to determine the optimal number of Gaussian components. When new test data arrives, the posterior probability of each Gaussian component is calculated, and its The Mahalanobis distance between the training data and the fusion of the two is used as a similarity index; finally, a set of data with the largest similarity is selected from the original training samples to establish the current GPR model, and the model output prediction is achieved, reaching Improve product quality and reduce production costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

Figure 1 is a flow chart of JITL soft sensor modeling based on BGMM;

Figure 2 shows the BIC values corresponding to different numbers of Gaussian components;

Figure 3 shows the RMSE modeled at different ratios of selected training samples.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.

Example:

This embodiment provides a real-time learning soft sensor modeling method based on the Bayesian Gaussian mixture model. This embodiment takes a common chemical process——butanizer process as an example. The experimental data comes from the debutanizer process, and the butane concentration is predicted, as shown in Figure 1. The method includes:

Step 1: Collect input and output data to obtain a historical training data set.

Step 2: Given the training sample X, use BIC to determine the optimal number K of Gaussian components. The description of BIC is as formula (1):

BIC＝-2logp(X|Θ)+dlog(N) (1)

In formula (1), logp(X|Θ) represents the logarithmic likelihood function of the training samples, d represents the number of free parameters of the K Gaussian components, and N represents the number of training samples

Step 3: After obtaining the optimal number K of Gaussian components, given the initial parameters of the Gaussian mixture model (GMM), the detailed description of the GMM algorithm is as follows:

Given a data set X{ _xi ∈ R ^m , i=1,2…N} containing N training samples, m represents the dimension of the input data, and its probability density function can be expressed as:

Where Θ=[α ₁ , μ ₁ , Σ ₁ ; α ₂ , μ ₂ , Σ ₂ ; ...; α _k , μ _k , Σ _k ] are parameters of GMM, K is the number of Gaussian components, θ _k is The parameters of the k-th Gaussian component, θ _k = (μ _k ,Σ _k ), μ _k and Σ _k are the mean and covariance matrix of the k-th Gaussian component respectively, and α _k is the proportion of the k-th Gaussian component ,and 0<α _k <1, where the probability density function of the kth Gaussian component is:

The unknown parameters in the GMM method are solved by the expectation maximization algorithm. The specific solution process is divided into E step and M step, which are described as follows:

Step E: Use the parameters of the current lth update and The probability that the i-th training sample belongs to the k-th Gaussian component is calculated by the Bayesian formula, as shown in formula (4): where C _k represents the k-th Gaussian component.

Step M: Utilize formulas (5), (6), and (7) to update algorithm parameters.

After obtaining the initial parameters of the GMM, use formulas (5), (6), and (7) to iterate continuously until the difference between the two parameters is less than the set threshold, and the final parameter Θ of the GMM is obtained.

Step 4: When a new input data x _q comes, use the Just-in-time learning (JITL) algorithm to select a set of data most similar to it from the historical data set to establish a local Gaussian process regression (Gaussian process regression) regression, GPR) model. The detailed descriptions of the JITL algorithm and the GPR modeling method are as follows:

JITL algorithm: The JITL method is based on the idea of similar inputs to generate similar outputs, and selects a group of training samples from the training samples that are most similar to the current incoming test samples to model; the core of JITL is the selection of similarity criteria, based on Euclidean distance The similarity criterion of and angle is a commonly used method, namely:

Among them, the distance d represents the 2-norm between the currently arriving test sample and the training sample, θ represents the angle between the two samples, and γ is a coefficient with a value between 0 and 1;

However, for some non-Gaussian industrial processes, GMM can better describe the non-Gaussian nature of the process. Compared with the traditional similarity criterion, the similarity criterion based on the Bayesian Gaussian mixture model BGMM can better select similar samples to establish a GPR model, the optimal number of Gaussian components K and the parameters Θ of each component obtained by steps 2 and 3 respectively, the corresponding similarity criterion can be expressed as:

where x _q represents the new incoming sample, x _i represents the i-th training sample, p(C _k |x _q ) represents the posterior probability that the new incoming sample x _q belongs to the k-th Gaussian component, As the Mahalanobis distance between two samples, for the current incoming x _q , use the above similarity criterion to select a group of data most similar to x _q to establish the current GPR model.

GPR modeling method: the known training sample sets X{ _xi ∈R ^m ,i=1,2…N} and Y{y _i ∈R,i=1,2…N} represent m-dimensional input data and 1 dimension output data. The relationship between input and output can be expressed as formula (10):

y _i =f(x _i )+ε (10)

Where f represents an unknown function form, ε represents the mean value is 0, and the variance is of white noise.

For a new test sample x _q , its output prediction value y _q also satisfies the Gaussian distribution, and its mean and variance can be expressed as formulas (11), (12):

y _q (x _q )＝c ^T (x _q )C ^-1 Y (11)

Where c(x _q )=[c(x _q ,x ₁ ),…,c(x _q ,x _N )] ^T is the covariance matrix of the test input data and the training input data, is the covariance matrix between the training input data, and c(x _q , x _q ) represents the covariance value between the test input data and itself.

GPR can choose different covariance functions. This paper chooses the radial basis covariance function, and its function description is as in formula (12): where v represents the overall measure of prior knowledge, ω _t represents the weight corresponding to each dimension of data, and δ _ij is the Kronecher operator, indicating the relative importance of each auxiliary variable.

Generally, the parameters in formula (13) are obtained by maximum likelihood estimation Its log-likelihood function is:

First set the parameter θ to a reasonable initial value, and then use the conjugate gradient method to obtain the optimized parameters. Generally, use the training set and verification set to try out the parameter θ to make it within a reasonable range; after the parameters are determined, for The new test data can be obtained from the formula (11) to get the output of the soft sensor model.

Step 5: Bring the newly arrived sample point x _q into the local GPR model established in step 4 to obtain the final estimated value y _q .

Fig. 3 is the root mean square error corresponding to building a local model by selecting different proportion data, and using the JITL method based on Euclidean distance and angle to compare with the method proposed in the present invention. It can be seen from the figure that the real-time learning soft sensor modeling method based on the Bayesian Gaussian mixture model can better estimate the butane concentration.

The present invention adopts an online real-time local updating strategy, adopts the Bayesian information criterion to determine the optimal number of Gaussian components, and when new test data arrives, calculates its posterior probability belonging to each Gaussian component, and calculates Find the Mahalanobis distance between it and the training data, and use the fusion of the two as the similarity index; finally, select a set of data with the largest similarity from the original training samples to establish the current GPR model, and predict the model output. The effect of improving product quality and reducing production cost has been achieved.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (4)

1. a kind of instant learning soft sensor modeling method based on Bayesian Gaussian mixture model, it is characterized in that, described method comprises: Step 1: Collect input and output data to obtain a historical training data set; Step 2: X is a known training sample, use Bayesian information criterion BIC to determine the optimal number of Gaussian components K, the description of BIC is as formula (1): BIC＝-2logp(X|Θ)+dlog(N) (1) In formula (1), logp(X|Θ) represents the logarithmic likelihood function of the training sample, d represents the number of free parameters that K Gaussian components have, and N represents the number of training samples; Step 3: According to the optimal number K of Gaussian components and the initial parameters of the given Gaussian mixture model GMM, use formulas (5), (6), and (7) to iterate continuously until the difference between the two parameters is less than the set value After setting the threshold, the parameter Θ of the final GMM is obtained. The detailed description of the GMM is as follows: A data set X{x _i ∈ R ^m ,i=1,2…N} containing N training samples, m represents the dimension of the input data, and the probability density function of the data set is expressed as: Among them, Θ=[α ₁ , μ ₁ , Σ ₁ ; α ₂ , μ ₂ , Σ ₂ ; ...; α _k , μ _k , Σ _k ] are parameters of GMM, K is the number of Gaussian components, θ _k is the parameter of the kth Gaussian component, θ _k = (μ _k ,Σ _k ), μ _k and Σ _k are the mean value and covariance matrix of the kth Gaussian component respectively, and α _k is the proportion of the kth Gaussian component ratio, and 0<α _k <1, where the probability density function of the kth Gaussian component is: The unknown parameters in the GMM method are solved by the expectation maximization algorithm. The specific solution process is divided into E step and M step, which are described as follows: Step E: According to the parameters updated for the current lth time and Calculate the probability that the i-th training sample belongs to the k-th Gaussian component by Bayesian formula where C _k denotes the kth Gaussian component Step M: update algorithm parameters Step 4: When a new input data x _q comes, use the real-time learning JITL algorithm to select the most similar set of data from the historical data set to establish a local Gaussian process regression GPR model, the details of the JITL algorithm and GPR modeling method The descriptions are as follows: JITL algorithm: The JITL method is based on the idea of similar inputs to generate similar outputs. From the training samples, a group of training samples that are most similar to the current incoming test samples are selected to model. The core of JITL is the selection of similarity criteria, based on Euclidean distance The similarity criterion of and angle is a commonly used method, namely: Among them, the distance d represents the 2-norm between the currently arriving test sample and the training sample, θ represents the angle between the two samples, and γ is a coefficient with a value between 0 and 1; However, for some non-Gaussian industrial processes, GMM can better describe the non-Gaussian nature of the process. Compared with the traditional similarity criterion, the similarity criterion based on the Bayesian Gaussian mixture model BGMM can better select similar samples to establish a GPR model, the optimal number of Gaussian components K and the parameters Θ of each component obtained by steps 2 and 3 respectively, the corresponding similarity criterion can be expressed as: where x _q represents the new incoming sample, x _i represents the i-th training sample, p(C _k |x _q ) represents the posterior probability that the new incoming sample x _q belongs to the k-th Gaussian component, is the Mahalanobis distance between two samples, for the current incoming x _q , using the above similarity criterion, select a set of data most similar to x _q to establish the current GPR model GPR modeling method: the known training sample sets X{ _xi ∈R ^m ,i=1,2…N} and Y{y _i ∈R,i=1,2…N} represent m-dimensional input data and 1 Dimensional output data, the relationship between input and output can be expressed as: y _i =f(x _i )+ε (10) Where f represents an unknown function form, ε represents the mean value is 0, and the variance is white noise For a new test sample x _q , its output prediction value y _q also satisfies the Gaussian distribution, and its mean and variance are expressed as: y _q (x _q )＝c ^T (x _q )C ^-1 Y (11) Among them, c(x _q )=[c(x _q ,x ₁ ),…,c(x _q ,x _N )] ^T is the covariance matrix of the test input data and the training input data, is the covariance matrix between the training input data, c(x _q , x _q ) represents the covariance value between the test input data and itself; GPR selects the radial basis covariance function, and its function is described as follows: Among them, v represents the overall measure of prior knowledge, ω _t represents the weight corresponding to each dimension of data, and δ _ij is the Kronecher operator, representing the relative importance of each auxiliary variable; The parameters in formula (13) are obtained by maximum likelihood estimation Its log-likelihood function is: Use the training set and verification set to try out the parameter θ, and then use the conjugate gradient method to obtain the optimized parameters. After the parameters are determined, for the new test data, the output of the soft sensor model can be obtained by formula (11); Step 5: Bring the newly arrived sample point x _q into the local GPR model established in step 4 to obtain the final estimated value y _q . 2. The method according to claim 1, characterized in that, the method is a forecasting method applied to variables that cannot be directly measured in complex industrial processes. 3. The method according to claim 2, characterized in that, the complex industrial process comprises chemical industry, metallurgy and fermentation process. 4. The method according to claim 3, characterized in that, the method is applied to a predictive method for butane concentration in a debutanizer process.