CN110863935B - Recognition method of blade attachments of ocean current machine based on VGG16-SegUnet and dropout - Google Patents

Abstract

The invention belongs to the field of fault diagnosis of ocean current machines, and in particular relates to a method for recognizing the attachments on the blades of ocean current machines based on VGG16-SegUnet and dropout. Collect and perform standardized preprocessing; build VGG16‑SegUnet network; use Adadelta optimizer to train the network; test the trained network, complete the identification of the location and size of the attachments on the blades of the current machine, and estimate the uncertainty of the identification results; finally Calculate the exact percentage of attachment area and the average intersection ratio. The present invention solves the problems that the existing method for diagnosing the attachments of marine current turbine blades based on the image signal cannot locate the attachments, output the accurate proportion of the attachments and estimate the identification uncertainty, and provides conditions for the maintenance of the marine current turbine blades and subsequent fault-tolerant control. Guidance is provided.

Description

Recognition method of blade attachments of ocean current machine based on VGG16-SegUnet and dropout

technical field

The invention relates to the field of fault diagnosis of ocean current machines, in particular to a method for identifying the attachments to the blades of ocean current machines based on VGG16-SegUnet and dropout.

Background technique

Ocean current energy is a kind of renewable and clean energy known as "blue oil field" and "offshore Saudi Arabia". It is mainly formed in the following two ways: relatively stable seawater flow in submarine channels and straits; Regular sea flow. Compared with wind and solar energy, ocean current energy has the advantages of predictability and high energy density. As an ocean current power generation device, the ocean current machine has the advantages of low noise, reliable operation and no strict site selection requirements. Realize grid-connected power generation. Different from the wind turbine installed on land, once the current turbine is officially put into operation, it will be placed underwater for a long time, which will cause the following potential problems: (1) Small marine organisms are likely to be attached to the turbine blades in the form of attachments. (2) The blades of ocean current machines are generally made of metal, so the perennial seawater immersion will corrode the blades, thus affecting the mechanical properties. Specifically, the unbalanced fault caused by the attachment will lead to the reduction of the frequency and amplitude of the generator output voltage, and the distortion of the waveform, which will ultimately affect the quality and efficiency of power generation and even cause grid fluctuations. Therefore, for the ocean current power generation system, it is particularly important to effectively check the corresponding fault status and make an early warning in the "germination" stage of these faults.

At present, there are relatively few methods for fault detection and diagnosis of the current machine, which are mainly divided into two types based on electrical signals (the current and voltage of the current machine stator) and image signals (the underwater image data of the current machine). However, in the face of the complex underwater environment, simply analyzing the stator current and voltage signals is not enough to accurately diagnose the degree of attachment. In addition, the existing methods for diagnosing attachments on ocean current turbine blades based on image signals have the following problems: (1) the location and size of attachments are not identified; (2) the accurate area ratio of attachments is not diagnosed; (3) ) cannot identify the distribution of different attachments, and there is a lack of analysis of the uncertainty of the diagnostic results.

SUMMARY OF THE INVENTION

In order to solve the above-mentioned problems existing in the method for diagnosing the attachments on the blades of the ocean current machine based on the image signal and realize more intuitive and accurate identification of the degree of the attachments, the present invention provides a blade attachment of the ocean current machine based on VGG16-SegUnet and dropout. recognition methods.

The VGG16-SegUnet and dropout-based method for identifying the attachments on a blade of a current machine includes the following steps:

Step 1. First, collect underwater images of current machines with different attachment types, and then use the open source tool labelme for semantic labeling to complete the creation of the original image-semantic label dataset: the background, leaves, and attachments are marked as 0, 1, respectively ,2.

Step 2: Expand the original image-semantic label dataset using the rotation data enhancement technique of [0°, 360°], and then standardize the original image:

Among them, x represents the data of any dimension of R, G, B in the current machine image; x _min , x _max represent the minimum and maximum pixel values in x respectively; x is finally normalized to [-1, 1].

The enhanced data is then divided into training set, validation set and test set according to the ratio of 3:1:1.

Step 3. Build a new semantic segmentation network of VGG16-SegUnet: set the convolution and max pooling models of the first 13 layers of VGG16 as the feature extraction coding layer and use ImageNet pre-training weights to initialize these convolutional structures; the structure of the decoding layer is the same as that of SegNet. The decoding layer is the same, and the inverse max pooling is used for feature recovery; in addition to the forward connection between the encoding layer and the decoding layer, the feature cascade in Unet and the maximum pooling index retention technology in SegNet are also integrated, and in the A 30% dropout layer is inserted in the middle position; the introduction of dropout not only alleviates the phenomenon of training overfitting, but also provides different probabilistic classification results.

Step 4. Input the preprocessed training set image data into VGG16-SegUnet, and output the pixel-by-pixel softmax probability classification result:

p(y⁽ⁱ⁾＝l|x⁽ⁱ⁾；θ)表示x⁽ⁱ⁾的预测结果y⁽ⁱ⁾为语义标签l的概率；N表示待语义标注的类别个数；exp^(·)表示指数函数；h_θ(x⁽ⁱ⁾)为softmax预测结果向量。Among them, x ⁽ⁱ⁾ represents the ith pixel in a training image x; θ is the weight parameter matrix of the softmax classifier, and

p(y ⁽ⁱ⁾ =l|x ⁽ⁱ⁾ ; θ) represents the probability that the prediction result y ⁽ⁱ⁾ of x ⁽ i) is the semantic label 1; N represents the number of categories to be semantically labeled; exp ^{( )} represents Exponential function; h _θ (x ⁽ⁱ⁾ ) is the softmax prediction result vector.

Then, use the Adadelta optimizer to globally train the entire network, reduce the cross-entropy loss until the number of training times reaches the set maximum value, and record the final training weights:

Among them, Loss(θ) represents the cross entropy loss function; N _train represents the number of training data; N _n represents the total number of pixels in the nth image; log( ) represents the logarithmic function; Sex function, when the expression in {·} holds, output 1, otherwise, output 0.

Step 5. Input the preprocessed test set image into the VGG16-SegUnet loaded with training weights, output the semantic segmentation map, and complete the recognition of the background, leaves, and attachment positions and sizes in the image. The uncertainty is estimated, and the specific implementation process is as follows:

i. Input each test image into VGG16-SegUnet fused with 30% dropout, and repeat the test 50 times to obtain 50 softmax probability classification results, which are recorded as Test ₅₀ here.

以及方差

ii. Find the mean of Test ₅₀

and variance

中找出每个像素点的最大概率类别，然后通过可视化技术显示出语义分割图；最大概率类别对应的方差以图像的形式直观地显示出来，即为不确定度图像。iii. From

Find out the maximum probability category of each pixel point, and then display the semantic segmentation map through visualization technology; the variance corresponding to the maximum probability category is visually displayed in the form of an image, which is the uncertainty image.

Step 6. Finally, according to the semantic segmentation map, calculate the exact percentage of attachment area and the recognition accuracy index MIoU:

Among them, AAP is the area ratio of attachments; attachment and blade represent attachments and the entire blade area of the current turbine respectively; Count( ) is used to calculate the number of pixels in the specified area.

Among them, MIoU represents the average intersection ratio; p _ij represents the number of pixels whose true label is i but is misidentified as label j; p _ii represents the number of pixels whose true label is i and is recognized as label i; p _ji represents the true The number of pixels with label j that were misidentified as label i.

beneficial effect

Compared with the prior art, the present invention has the following technical effects:

1) The present invention uses image data with three dimensions (R, G, B), so compared with one-dimensional current and voltage signals, more abundant and intuitive attachment feature information can be extracted.

2) The semantic labeling method adopted in the present invention can effectively improve the network training speed and generalization ability; the rotational data enhancement technology greatly reduces the workload of semantic labeling while simulating the rotational operation of the current machine.

3) The new semantic segmentation network of VGG16-SegUnet proposed by the present invention can effectively segment the current machine image, and complete the identification of the background, leaves, the location and size of the attachment, and output the accurate attachment area ratio.

4) The attachment identification method in the present invention can identify different blade attachment distributions, and can estimate the uncertainty of the attachment identification results, so as to provide guiding suggestions for the subsequent maintenance and fault-tolerant control of the current machine.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

Fig. 1 is the algorithm flow schematic diagram of the method for identifying the attachments to the blades of the ocean current machine based on VGG16-SegUnet and dropout in the present invention;

Fig. 2 is the schematic diagram of the image semantic labeling method in the present invention;

FIG. 3 is a schematic diagram of the architecture of the semantic segmentation network VGG16-SegUnet proposed in the present invention.

Detailed ways

The embodiments of the present invention are described in detail below, and the embodiments are exemplary and intended to explain the present invention, but should not be construed as a limitation of the present invention.

As shown in FIG. 1 , the present invention provides a method for identifying the attachments on a blade of an ocean current machine based on VGG16-SegUnet and dropout, which includes the following steps:

Step 1. First, collect underwater images of current machines with different attachment types, and then use the open source tool labelme for semantic labeling to complete the creation of the original image-semantic label dataset: the background, leaves, and attachments are marked as 0, 1, respectively , 2, as shown in Figure 2.

Step 2: Expand the original image-semantic label dataset using the rotation data enhancement technique of [0°, 360°], and then standardize the original image:

Among them, x represents the data of any dimension of R, G, B in the current machine image; x _min , x _max represent the minimum and maximum pixel values in x respectively; x is finally normalized to [-1, 1].

The enhanced data is then divided into training set, validation set and test set according to the ratio of 3:1:1.

Step 3. Build a new semantic segmentation network of VGG16-SegUnet: set the convolution and max pooling models of the first 13 layers of VGG16 as the feature extraction coding layer and use ImageNet pre-training weights to initialize these convolutional structures; the structure of the decoding layer is the same as that of SegNet. The decoding layer is the same, and the inverse max pooling is used for feature recovery; in addition to the forward connection between the encoding layer and the decoding layer, the feature cascade in Unet and the maximum pooling index retention technology in SegNet are also integrated, and in the A 30% dropout layer is inserted in the middle; the introduction of dropout not only alleviates the phenomenon of training overfitting, but also provides different probability classification results; the specific architecture design of VGG16-SegUnet is shown in Figure 3.

Step 4. Input the preprocessed training set image data into VGG16-SegUnet, and output the pixel-by-pixel softmax probability classification result:

p(y⁽ⁱ⁾＝l|x⁽ⁱ⁾；θ)表示x⁽ⁱ⁾的预测结果y⁽ⁱ⁾为语义标签l的概率；N表示待语义标注的类别个数；exp^(·)表示指数函数；h_θ(x⁽ⁱ⁾)为softmax预测结果向量。Among them, x ⁽ⁱ⁾ represents the ith pixel in a training image x; θ is the weight parameter matrix of the softmax classifier, and

p(y ⁽ⁱ⁾ =l|x ⁽ⁱ⁾ ; θ) represents the probability that the prediction result y ⁽ⁱ⁾ of x ⁽ i) is the semantic label 1; N represents the number of categories to be semantically labeled; exp ^{( )} represents Exponential function; h _θ (x ⁽ⁱ⁾ ) is the softmax prediction result vector.

Then, use the Adadelta optimizer to globally train the entire network, reduce the cross-entropy loss until the number of training times reaches the set maximum value, and record the final training weights:

Among them, Loss(θ) represents the cross entropy loss function; N _train represents the number of training data; N _n represents the total number of pixels in the nth image; log( ) represents the logarithmic function; Sex function, when the expression in {·} holds, output 1, otherwise, output 0.

Step 5. Input the preprocessed test set image into the VGG16-SegUnet loaded with training weights, output the semantic segmentation map, and complete the recognition of the background, leaves, and attachment positions and sizes in the image. The uncertainty is estimated, and the specific implementation process is as follows:

i. Input each test image into VGG16-SegUnet fused with 30% dropout, and repeat the test 50 times to obtain 50 softmax probability classification results, which are recorded as Test ₅₀ here.

以及方差

ii. Find the mean of Test ₅₀

and variance

中找出每个像素点的最大概率类别，然后通过可视化技术显示出语义分割图；最大概率类别对应的方差以图像的形式直观地显示出来，即为不确定度图像。iii. From

Find out the maximum probability category of each pixel point, and then display the semantic segmentation map through visualization technology; the variance corresponding to the maximum probability category is visually displayed in the form of an image, which is the uncertainty image.

Step 6. Finally, according to the semantic segmentation map, calculate the exact percentage of attachment area and the recognition accuracy index MIoU:

Among them, AAP is the area ratio of attachments; attachment and blade represent attachments and the entire blade area of the current turbine respectively; Count( ) is used to calculate the number of pixels in the specified area.

Among them, MIoU represents the average intersection ratio; p _ij represents the number of pixels whose true label is i but is misidentified as label j; p _ii represents the number of pixels whose true label is i and is recognized as label i; p _ji represents the true The number of pixels with label j that were misidentified as label i.

Claims (1)

1. a method for identifying the attachments of ocean current machine blades based on VGG16-SegUnet and dropout, is characterized in that, comprises the following steps: Step 1. First, collect underwater images of current machines with different attachment types, and then use the open source tool labelme for semantic labeling to complete the creation of the original image-semantic label dataset: the background, leaves, and attachments are marked as 0, 1, respectively ,2; Step 2: Expand the original image-semantic label dataset using the [0°, 360°] rotation data enhancement technique, and then standardize the original image:

Among them, x represents the data of any dimension of R, G, B in the current machine image; x _min , x _max represent the minimum and maximum pixel values in x respectively; x is finally normalized to [-1, 1]; Then the enhanced data is divided into training set, validation set and test set according to the ratio of 3:1:1; Step 3. Build a new semantic segmentation network of VGG16-SegUnet: set the convolution and max pooling models of the first 13 layers of VGG16 as the feature extraction coding layer and use ImageNet pre-training weights to initialize these convolutional structures; the structure of the decoding layer is the same as that of SegNet. The decoding layer is the same, and the inverse max pooling is used for feature recovery; in addition to the forward connection between the encoding layer and the decoding layer, the feature cascade in Unet and the maximum pooling index retention technology in SegNet are also integrated, and in the A 30% dropout layer is inserted in the middle position; the introduction of dropout not only alleviates the phenomenon of training overfitting, but also provides different probability classification results; Step 4. Input the preprocessed training set image data into VGG16-SegUnet, and output the pixel-by-pixel softmax probability classification result:

Among them, x ⁽ⁱ⁾ represents the ith pixel in a training image x; θ is the weight parameter matrix of the softmax classifier, and

p(y ⁽ⁱ⁾ =l|x ⁽ⁱ⁾ ; θ) represents the probability that the prediction result y ⁽ⁱ⁾ of x ⁽ i) is the semantic label 1; N represents the number of categories to be semantically labeled; exp ^{( )} represents Exponential function; h _θ (x ⁽ⁱ⁾ ) is the vector of softmax prediction results; Then, use the Adadelta optimizer to globally train the entire network, reduce the cross-entropy loss until the number of training times reaches the set maximum value, and record the final training weights:

Among them, Loss(θ) represents the cross entropy loss function; N _train represents the number of training data; N _n represents the total number of pixels in the nth image; log( ) represents the logarithmic function; Sex function, when the expression in {·} is established, output 1, otherwise, output 0; Step 5. Input the preprocessed test set image into the VGG16-SegUnet loaded with training weights, output the semantic segmentation map, and complete the recognition of the background, leaves, and attachment positions and sizes in the image. The uncertainty is estimated, and the specific implementation process is as follows: i. Input each test image into the VGG16-SegUnet fused with 30% dropout, and repeat the test 50 times to obtain 50 softmax probability classification results, which are denoted as Test ₅₀ here; ii. Find the mean of Test ₅₀

and variance

iii. From

Find out the maximum probability category of each pixel point, and then display the semantic segmentation map through visualization technology; the variance corresponding to the maximum probability category is visually displayed in the form of an image, which is the uncertainty image; Step 6. Finally, according to the semantic segmentation map, calculate the exact percentage of attachment area and the recognition accuracy index MIoU:

Among them, AAP is the percentage of the area of attachments; attachment and blade represent the attachments and the entire blade area of the current turbine respectively; Count( ) is used to calculate the number of pixels in the specified area;

Among them, MIoU represents the average intersection ratio; p _ij represents the number of pixels whose true label is i but is misidentified as label j; p _ii represents the number of pixels whose true label is i and is recognized as label i; p _ji represents the true The number of pixels with label j that were misidentified as label i.

Images