CN114821042A - An R-FCN knife gate detection method combining local and global features - Google Patents

Abstract

The invention provides an R-FCN disconnecting link detection method combining local features and global features. The method comprises the steps that a network camera centralized control software is embedded into a transformer substation auxiliary monitoring system to achieve knife switch image acquisition of cameras in multiple positions and multiple angles in different outdoor weather environments and different backgrounds, and diverse knife switch data are constructed; by parallelly connecting the R-FCN output prediction network with the global feature prediction module, the defect of insufficient receptive field generated by the original network only through predicting the disconnecting link by local features is supplemented, the accuracy of detecting the partially shielded disconnecting link is improved, and the omission factor and the false detection factor of the disconnecting link in a complex background are reduced; the prediction results are accumulated after the local feature prediction results and the global feature prediction results are regularized, so that the local feature prediction results are supplemented by the global feature prediction results, and the requirements of remote detection and unattended operation of the disconnecting link of the transformer substation are met.

Description

An R-FCN knife gate detection method combining local and global features

technical field

The invention relates to the technical field of knife gate detection in a substation auxiliary monitoring system, and relates to an R-FCN knife gate detection method combining local features and global features.

Background technique

The substation is an important transfer place in high-voltage transmission, and there are various vital on-off control devices running in the station. In the past, when the substation was interrupted, the relevant operators needed to check each device to find the fault. The length of the troubleshooting time determines when the substation works normally, which seriously affects the daily life of residents. As an important electrical control switch in a substation, the opening and closing state of the knife switch directly determines whether the entire electrified circuit operates. Traditionally, to check the power failure status of the substation, the operator first needs to enter the high-voltage area of the substation to check whether the knife switch is disconnected, and then check other connecting devices one by one, which is prone to electric shock. With the development demand of unmanned duty in substations, researching the automatic detection of the state of the knife switch in the snapshot image will help to quickly identify the current state of the knife switch, shorten the fault detection efficiency of the substation, ensure the life safety of the operators, and effectively improve the power grid. safe operation.

For the automatic detection of knife gates, there are often problems encountered at present: the working environment of knife gates is located outdoors, and there are many kinds of equipment that are made of the same metal products as the knife gates, with similar colors and are not easy to distinguish; Knife switches are difficult to identify in different environments; due to the presence of various voltage control devices and wires outdoors, as well as the influence of the long rectangular shape of the knife switches, when the auxiliary monitoring system is deployed in the substation, it often occurs that the collected knife switches are affected by other object equipment. Occlusion or incomplete capture of the entire feature.

There are two types of knife gate detection methods: 1) methods based on image processing; 2) methods based on deep learning. Affected by the data collection of the knife gate, most of the current methods are based on image processing. First, the knife gate template is established for the standard image collected at a fixed angle position, and then image processing algorithms such as feature point matching algorithm or template matching algorithm are used to locate and locate. Status recognition, this method can achieve high monitoring accuracy in a short period of good weather conditions, but with frequent changes in weather conditions, the camera position will shift based on the pre-established template image angle, and then The state of the knife gate cannot be recognized, so it is necessary to manually check and adjust the camera position or create a new template. The deep learning method needs to rely on a large number of data sets from different angles and different weathers. The lack of data sets will limit the progress of related work. In addition, most deep learning network models currently make predictions through the global receptive field, such as: Faster-Rcnn series network models, Yolo series network models, etc., when the knife gate is detected, if the knife gate is blocked or the collection object is incomplete, the overall feature information of the knife gate will be missing, resulting in missed detection or false detection; partial depth The learning network model predicts through local features, such as: R-FCN network model, VIT network model, etc. When detecting the knife gate, although the incomplete knife gate can be effectively detected, when the knife gate is full of the whole image , the local feature prediction will also be missed.

SUMMARY OF THE INVENTION

In view of the above technical problems, the purpose of the present invention is: based on the R-FCN network model, to propose a region-based fully convolutional target detection network R-FCN (R-FCN: Object Detection viaRegion-based) that combines local features and global features Fully Convolutional Networks) knife gate detection method, which embeds global feature prediction into the R-FCN network model, and combines the advantages of local features and global feature prediction to improve the accuracy of knife gate detection and the accuracy of existing knife gate detection methods.

In order to achieve the above object, the technical solution adopted in the present invention is: a kind of R-FCN knife gate detection combining local features and global features, comprising the following steps:

Step 1: Build a substation auxiliary monitoring system to collect knife gate images;

Step 2: Divide, clean and label the image dataset;

Step 3: Select the R-FCN backbone feature extraction network;

Step 4: Adjust the backbone feature extraction network of R-FCN;

Step 5: Build a local feature prediction branch;

Step 6: Build a global feature prediction branch;

Step 7: Fusion of local prediction results and global prediction results;

Step 8: Train and save the model.

Further, the substation auxiliary monitoring system in step 1 is the integration of all network monitoring cameras and related control equipment in the substation. By adjusting the angle of each camera, it is possible to collect different time, weather, and backgrounds including knife gates and without knife gates. , and the number of image datasets containing the knife switch should evenly include the two states of the switch.

Further, dividing the data set in step 2 refers to dividing the collected images into images containing knife gates and images not containing knife gates, and dividing the data set containing knife gate images into training sets and test sets with a ratio of 8. : 2; Cleaning the data set refers to removing the blurred data set in the collected data set; the labeling data only labels the images containing the knife gate object.

Further, in step 3, the R-FCN backbone feature extraction network is selected, and the implementation method is as follows: the classification network ResNet101 is used as the backbone feature extraction network, and the region proposal network (RPN: Region Proposal Network) is used after the fourth convolutional layer of ResNet101. Generate a region of interest (for later detection of knife gates), discard the pooling layer and fully connected layer after the fifth set of convolutional layers of ResNet101, and the final output channel number is 2048. The region of interest is an area where a target exists.

Further, in step 4, the backbone feature extraction network of R-FCN is adjusted, and the implementation method is as follows: after selecting the trimmed backbone feature extraction network ResNet101 in step 3, a convolutional layer with a convolution kernel size of 1×1 is added to reduce the number of channels to 1024, in order to reduce the dimension of feature data without changing the size of the feature map, and improve the calculation speed.

Further, the local feature prediction branch is constructed in step 5, and the implementation method is as follows: the original network model R-FCN with local feature prediction is used to output the local prediction result, and the local feature prediction is divided into 7 × 7 local regions by using the RPN recommendation area. predict.

Further, a global feature prediction branch is constructed in step 6, and the implementation method is as follows: perform a pooling operation on the extracted semantic features, and unify the extracted semantic feature sizes; The convolutional layer outputs the global prediction result.

式中x、y为向量，x＝(x₀,x₁,x₂...x_n),表示预测输出结果,y＝(y₀,y₁,y₂...y_n)，表示正则化后的预测输出结果，最终将两种预测结果数值统一缩放至0到1区间，并将两种预测输出结果进行向量相加，然后进行Softmax操作，输出最终预测结果，Softmax的数学表达式为：

R_i为输出结果向量中第i个输出值。Further, the fusion of local prediction results and global prediction results in step 7 is to use L2 regularization for the output prediction results in steps 5 and 6, and the mathematical expression is:

In the formula, x and y are vectors, x=(x ₀ , x ₁ , x ₂ ... x _n ), representing the predicted output result, y=(y ₀ , y ₁ , y ₂ ... y _n ), representing The normalized prediction output results, and finally the two prediction results are uniformly scaled to the range of 0 to 1, and the two prediction output results are vector added, and then the Softmax operation is performed to output the final prediction result, the mathematical expression of Softmax for:

R _i is the ith output value in the output result vector.

Further, in step 8, the model is trained, and the training is performed twice using the divided two data sets: the first training uses only the clear data set containing the knife gate, and the second training uses the knife gate and does not include the knife gate. But a fuzzy dataset with a 1:1 ratio of data quantity like knife gate.

其中L(s,t)表示分类损失和回归损失的总损失，s为类别预测概率，

表示类别c^*的预测概率，t表示模型预测的回归框， L_cls(s)为分类损失，数学表达式为：

λ多任务平衡因子，c^*为类别真实标签，当c^*＝0表示背景类别，c^*≠0表示对应的对别，[c^*＞0]组成指导因子，其值取为1，表示回归框只对非背景类别对象进行调整，L_reg(t,t^*)为回归损失，t表示模型预测的目标框，t^*表示人工标注的真实框。Further, in the training model in step 8, the mathematical expression of the loss function used in the training is:

where L(s,t) represents the total loss of classification loss and regression loss, s is the class prediction probability,

represents the predicted probability of class c ^* , t represents the regression box predicted by the model, L _cls (s) is the classification loss, and the mathematical expression is:

λ multi-task balance factor, c ^* is the true label of the category, when c ^* =0 represents the background category, c ^* ≠0 represents the corresponding pair, [c ^* >0] constitutes the guidance factor, and its value is 1, indicating regression The box is only adjusted for non-background category objects, L _reg (t, t ^* ) is the regression loss, t represents the target box predicted by the model, and t ^* represents the ground-truth box that is manually annotated.

Compared with the prior art, the beneficial effects obtained by the present invention are: for the detection of knife gates with partial feature occlusion and larger features, the R-FCN network based on joint local features and global features is compared with prediction based on original local features. , can be predicted and judged. The R-FCN knife switch detection method based on joint local features and global features is proposed, which effectively reduces the false detection rate and missed detection rate of knife switch detection in complex scenes of substations.

Description of drawings

In order to illustrate the technical solutions of the present invention more clearly, further description will be given below with reference to the accompanying drawings required in the embodiments.

FIG. 1 is a schematic diagram of the operation flow of the R-FCN knife gate detection method combining local features and global features of the present invention.

FIG. 2 is a schematic diagram of a module of an R-FCN model combining local features and global features of the present invention.

FIG. 3 is a schematic diagram of a module for local feature prediction according to the present invention.

FIG. 4 is a schematic diagram of a module for global feature prediction according to the present invention.

Detailed ways

The following describes the embodiments of the present invention in detail with reference to the accompanying drawings 1-4.

As shown in Figure 1, an R-FCN knife gate detection method combining local features and global features, the steps are as follows:

Step 1: When the substation auxiliary monitoring system collects the knife gate images, it is collected from natural factors such as different angles, different fields of view, and different backgrounds, so as to ensure the diversity of the collected images.

Step 2: Clean the data set collected in Step 1, remove damaged image data, and divide it into two types: the image data set that only contains the knife gate and the data set that does not contain the knife gate but is similar to the knife gate. The data set collected by the present invention includes a total of 800 images of the knife gate, and 800 images that do not include the knife gate but are similar to the knife gate. The data set containing only knife gate images is marked with the knife gate position and state. The number of samples used in the first training is shown in Table 1, and the number of samples used in the second training is shown in Table 2.

Step 3: Adjust the R-FCN backbone feature extraction network: use the classification network ResNet101 as the backbone feature extraction network, and use the RPN operation after the fourth convolutional layer of ResNet101 to generate the region of interest (for later detection of knife gates), discard the use of The pooling layer and fully connected layer after the fifth set of convolutional layers of ResNet101, the final output channel number is 2048, as shown in Figure 2.

Step 4: After selecting the trimmed backbone feature extraction network ResNet101 in Step 3, add a convolutional layer with a convolution kernel size of 1×1 to reduce the number of channels to 1024 to reduce the dimension of feature data without changing the size of the feature map, such as shown in Figure 2.

Step 5: Build a local feature prediction branch: use the original network model R-FCN with local feature prediction to output the local prediction result, and the local feature prediction is divided into 7 × 7 local regions for prediction using the RPN recommendation area, as shown in Figure 3 .

Step 6: Build a global feature prediction branch: perform a pooling operation on the extracted semantic features to unify the size of the extracted semantic features; then, use convolutional layers with convolution kernel sizes of 7×7 and 1×1 in series to output the global prediction result ,As shown in Figure 4.

Step 7: Fusion of local prediction results and global prediction results: The local prediction results and the global prediction results are regularized, uniformly scaled to the same numerical range and added to complete the information fusion prediction.

式中x、y为向量， x＝(x₀,x₁,x₂...x_n),表示预测输出结果,y＝(y₀,y₁,y₂...y_n)，表示正则化后的预测输出结果，最终将两种预测结果数值统一缩放至0到1区间，并将两种预测输出结果进行向量相加，然后进行Softmax操作，输出最终预测结果，Softmax的数学表达式为：

R_i为输出结果向量中第i个输出值。Further, the mathematical expression is:

In the formula, x and y are vectors, x=(x ₀ , x ₁ , x ₂ ... x _n ), representing the predicted output result, y=(y ₀ , y ₁ , y ₂ ... y _n ), representing The normalized prediction output results, and finally the two prediction results are uniformly scaled to the range of 0 to 1, and the two prediction output results are vector added, and then the Softmax operation is performed to output the final prediction result, the mathematical expression of Softmax for:

R _i is the ith output value in the output result vector.

Step 8: Train the model: The initialization parameters of the model backbone feature extraction network ResNet101 during training use the model weights trained on the ImageNet dataset, and the loss function used for model training selects the same loss function as the R-FCN network, and conducts two trainings : For the first time, use the data that only contains the knife gate image to train, until the loss function converges, and save the network model; the second time, based on the model parameters of the first training, use the similar knife that contains the knife gate and does not include the knife gate The data of the gate is used for training, and the identification ability of the knife gate of the network model is further characterized, and the test results of the model are finally obtained as shown in Table 3.

其中L(s,t)表示分类损失和回归损失的总损失，s为类别预测概率，

表示类别c^*的预测概率，t表示模型预测的回归框， L_cls(s)为分类损失，数学表达式为：

λ多任务平衡因子，c^*为类别真实标签，当c^*＝0表示背景类别，c^*≠0表示对应的对别，[c^*＞0]组成指导因子，其值取为1，表示回归框只对非背景类别对象进行调整，L_reg(t,t^*)为回归损失，t表示模型预测的目标框，t^*表示人工标注的真实框。Further, in the training model in step 8, the mathematical expression of the loss function used in the training is:

where L(s,t) represents the total loss of classification loss and regression loss, s is the class prediction probability,

represents the predicted probability of class c ^* , t represents the regression box predicted by the model, L _cls (s) is the classification loss, and the mathematical expression is:

λ multi-task balance factor, c ^* is the true label of the category, when c ^* =0 represents the background category, c ^* ≠0 represents the corresponding pair, [c ^* >0] constitutes the guidance factor, and its value is 1, indicating regression The box is only adjusted for non-background category objects, L _reg (t, t ^* ) is the regression loss, t represents the target box predicted by the model, and t ^* represents the ground-truth box that is manually annotated.

Table 1 is the statistical results of the number of samples in the first training set.

Table 2 is the statistical results of the number of samples in the second training set.

Table 3 shows the comparison results of the R-FCN knife gate detection accuracy by combining local features and global features.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principle and spirit of the invention Variations, the scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims (9)

1. an R-FCN knife gate detection method of joint local feature and global feature, is characterized in that, comprises the following steps: Step 1: Build a substation auxiliary monitoring system to collect images from different camera angles and weather conditions at different times; Step 2: Divide the images collected in step 1 into training set images and test set images, and clean and label the image data set; Step 3: Build an R-FCN Knife Gate Detection Model with Joint Local Features and Global Features: Adjust the R-FCN Backbone Feature Extraction Network: Use the classification network ResNet101 as the backbone feature extraction network, and use the region after the fourth set of convolutional layers of ResNet101 It is recommended that the network RPN operation generates a region of interest, and discards the pooling layer and fully connected layer after the fifth set of convolutional layers of ResNet101; Step 4: Based on the backbone feature extraction network adjusted in Step 3, the number of output channels is 2048, and a convolutional layer with a convolution kernel size of 1×1 is attached to reduce the number of channels to 1024; Step 5: Build a local feature prediction branch: Based on the operation in step 4, the extracted semantic feature information is divided into two directions for parallel prediction: local feature prediction and global feature prediction based on the original network, and the local feature prediction directly adopts the original prediction network model Output local prediction results; Step 6: Build a global feature prediction branch: The global feature prediction is based on the semantic features extracted in step 4. First, the extracted semantic features are pooled to unify the size of the extracted semantic features; then, the convolution kernel size is 7× 7 and 1×1 convolutional layers output global prediction results; Step 7: Fusion of local prediction results and global prediction results: Regularize the local prediction results and global prediction results, uniformly scale them to the same numerical range and add them to complete the information fusion prediction; Step 8: Train the model: The loss function used for model training selects the same loss function as the R-FCN network, which is used to guide the optimization of model parameters; the network training parameters are updated until the loss function converges, and the network model is saved. 2. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 1, is characterized in that: the auxiliary monitoring system in the described step 1 is to combine each angle of the substation by a plurality of network cameras Implement comprehensive coverage and real-time monitoring. The auxiliary monitoring system is embedded in the camera centralized control program, and data collection from different angles, different backgrounds, and different weather factors is realized through multiple network cameras. 3. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 1, is characterized in that: the cleaning data set in the described step 2 is specifically: the blurred image caused by camera vibration is carried out. For culling, images containing knife gates are grouped into a dataset, and images that do not contain knife gates but have similar knife gates are grouped into a dataset. 4. The R-FCN knife gate detection method combining local features and global features according to claim 1, wherein the local feature prediction in the step 5 is to divide the RPN suggested area into 7×7 local area for prediction. 5. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 1, is characterized in that: the fusion local prediction result and the global prediction result in the described step 7 are specifically: use regularization The method normalizes the local prediction results and the global prediction results respectively, and uses L2 regularization. The mathematical expression is:

In the formula, x and y are vectors, x=(x ₀ , x ₁ , x ₂ ... x _n ), representing the predicted output result, y=(y ₀ , y ₁ , y ₂ ... y _n ), representing Regularize the prediction output results, and finally scale the prediction results uniformly to the range of 0 to 1. 6 . The R-FCN knife gate detection method combining local features and global features according to claim 6 , wherein the fusion method used is: adding vectors of the same dimension. 7 . 7. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 1, is characterized in that: the data that model training uses in described step 8 is divided into two kinds: the clear that comprises knife gate Datasets and clutter-hard easy-to-sample ratios are 1:1 fuzzy datasets. 8. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 7, it is characterized in that: described training model is specifically: use migration learning thought during training, adopt main feature extraction network to adopt The weight of the model trained on the ImageNet dataset; the clear dataset means that all training images contain at least one knife gate image, and the fuzzy data set refers to the image that contains the knife gate and does not contain the knife gate and is similar to the knife gate 50% of the images each. 9. the R-FCN knife gate detection method of a kind of joint local feature and global feature according to claim 7, is characterized in that: described training model: the loss function mathematical expression that training uses is:

where L(s,t) represents the total loss of classification loss and regression loss, s is the class prediction probability,

Represents the predicted probability of class c ^* , t represents the regression box predicted by the model, L _cls (s) is the classification loss, λ multi-task balance factor, c ^* is the class true label, L _reg (t, t ^* ) is the regression loss, t ^* denotes the ground-truth box annotated by humans.

Images