CN117152139A - Patch inductance defect detection method based on example segmentation technology - Google Patents

Abstract

The invention belongs to the technical field of computer vision detection, and particularly discloses a patch inductance defect detection method based on an example segmentation technology, which comprises the following steps of: collecting various patch inductance images, and then preprocessing and labeling to obtain a patch inductance image data set; constructing a network detection model based on an example segmentation technology, and training the network detection model by using a training set; checking the trained network detection model to obtain a final network detection model; and preprocessing a patch inductance image acquired in real-time production, and inputting the preprocessed patch inductance image into a final network detection model to detect patch inductance defects in real time. The invention can effectively improve the detection efficiency by detecting the defects of the patch inductor through computer vision, can classify the patch inductor with the defects detected, simultaneously improves the detection stability and accuracy, can detect in real time in the production process, and is beneficial to improving the yield of the produced products.

Description

A defect detection method for chip inductors based on instance segmentation technology

Technical field

The invention belongs to the field of computer vision detection technology, and specifically relates to a patch inductor defect detection method based on instance segmentation technology.

Background technique

Chip inductors are key components in electronic equipment, and their performance directly affects the stability, reliability and life of electronic equipment. Therefore, in the manufacturing process of electronic equipment, the quality requirements for chip inductors are relatively high, and they need to be strictly tested. By detecting defects in chip inductors, defective products can be discovered and eliminated in a timely manner, thereby improving the overall yield rate of the product. The traditional chip inductor inspection method mainly relies on manual visual inspection. The operator observes the chip inductors one by one on the production line to identify and mark defective products. The efficiency of manual inspection is low, and it is easily affected by factors such as the fatigue level and experience of the staff, resulting in unstable inspection results and prone to missed detections and false detections. At present, eddy current testing, magnetic flux leakage testing and infrared testing have also appeared. Among them, eddy current detection and magnetic flux leakage detection are prone to false detections when detecting rough surfaces, while infrared detection has many restricted conditions and is usually only used for small-scale offline inspection, which cannot meet the current needs of real-time monitoring of industrial production. The instance segmentation network has achieved good results on standard data sets. The currently widely used yolov7 uses a deeper network structure, a more fine-grained feature pyramid and a more effective loss function, etc., to achieve good results in target detection tasks. A higher accuracy rate. However, due to the wide variety of chip inductor defects and relatively small targets, the above example segmentation network cannot satisfy the defect detection of chip inductors.

Contents of the invention

The technical problem to be solved by the present invention is to provide a chip inductor defect detection method based on instance segmentation technology in view of the above-mentioned shortcomings of the existing technology, which can effectively improve the detection efficiency and at the same time improve the stability and accuracy of detection.

The technical solution adopted by the present invention is: a patch inductor defect detection method based on instance segmentation technology, which includes the following steps:

Step 1: Shoot and collect various chip inductor images through a vision camera, preprocess the chip inductor images, and mark the chip inductor images with chip inductor defects after preprocessing to obtain the chip inductor. Image data set, and divide the patch inductance image data set into a training set and a test set;

Step 2: Construct a network detection model based on instance segmentation technology. The network detection model includes a backbone network and a head network;

The CA-ELAN module is added to the backbone network layer;

In the head network, a layer containing a 3×3 convolution kernel and a 1×1 convolution kernel is added to the final three prediction heads, and batch normalization is added before the LeakyReLu activation function. Residual network with setting operation; it can increase the training speed and improve the prediction accuracy of the model while ensuring the prediction speed;

Step 3: Input the training set in step 1 into the network detection model, train the network detection model, continuously calculate the total loss function during training, and continuously update the detection model parameters through small-batch gradient descent to obtain the trained network. detection model;

Step 4: Input the test set in step 1 into the trained network detection model for verification, and obtain the final network detection model after verification;

Step 5: Preprocess the patch inductor images collected during real-time production and input them into the final network inspection model for visual real-time detection of patch inductor defects;

Step 6: Based on the detection results of Step 5, regularly update the patch inductor image data set of the network detection model, and retrain the network detection model with the updated patch inductor image data set.

Preferably, the preprocessing of the chip inductor image in step 1 includes: scaling, denoising and image enhancement of the chip inductor image in sequence.

As an option, the patch inductor image data set obtained in step 1 also uses mosaic data enhancement, that is, four patch inductor images are randomly selected from the patch inductor image data set and combined into a new patch inductor image, and the new patch inductor image is The patch inductor image is added to the patch inductor image data set.

Preferably, the chip inductor defects described in step 1 include magnetic ring dark crack defects, electrode exposed copper defects, electrode exposed wire defects and magnetic ring damage defects.

Preferably, the CA-ELAN module is introduced into the E-ELAN module of the backbone network layer, and the width and height of the patch inductor image are obtained through the CA module, and the precise position is encoded to enable the network to capture multi-scale information. The contextual information of the chip inductor image is then used for feature fusion through the E-ELAN module to obtain the chip inductor defect image features with contextual information that integrate multi-scale information;

The CA module is the coordinate attention mechanism module, which is a module used to enhance the neural network's perception of position information; in the traditional attention mechanism, the attention weight is calculated only through the channel information of the feature, without considering Position information. However, in target detection and image segmentation of patch inductance images, position information is very important for correctly understanding and processing the image content. Therefore, by introducing the CA module, the network detection model can automatically learn which feature channels are suitable for the target detection task. More importantly, the network detection model can weight different channels according to the importance of the feature channels, so that the network detection model pays more attention to the key features that contribute to target detection, reducing the impact of noise and useless features, that is, by introducing The CA module helps the model automatically select important feature channels, improve target expression capabilities, and improve target detection performance. It also enables the model to use feature information more effectively, thereby improving the accuracy and effect of the target detection task, that is, the model can Locate features useful for defect detection and suppress useless features.

Preferably, the CA module includes the following steps:

In step a, the input size is The chip inductor defect feature map is used, and pooling kernels of size (1, W) and (H, 1) are used to perform global average pooling to obtain the horizontal direction feature vector sum encoded by the global average vector in the channel dimension. Vertical eigenvector:

Among them, H represents the height, W represents the width, and C represents the number of channels. Represents the global average value of the input feature map X in the c-th channel,/> Indicates that the c-th channel coordinates are (i, j) feature information;

Step b: Convolve the horizontal feature vector and vertical feature vector obtained in step a using a convolution kernel of size 1×1, batch normalize and use the LeakyReLu activation function for feature mapping:

in, Represents the attention response of the input feature map X on the c-th channel, /> Represents a 1×1 convolution kernel;

Step c, decompose the features mapped in step b into two independent decomposition features according to the original W and H along the horizontal-vertical direction. , and use two 1×1 size convolution kernel convolution and LeakyReLu activation function to perform feature transformation respectively:

,

in, are the attention weights in the horizontal and vertical directions of the input patch inductor defect feature map, /> It is a convolution kernel of size 1×1;

Step d, obtain the final feature vector:

The final output chip inductor defect characteristic map ;

Preferably, the total loss function in step 3 includes a category loss function, a positioning loss function and a target confidence loss function.

As a preference, the width and height loss is introduced in the positioning loss function to minimize the difference in width and height between the real bounding box and the predicted bounding box, specifically:

Among them, b is the predicted box, b _gt is the real box, IOU is the coverage degree between the predicted bounding box and the real bounding box, is the square of the Euclidean distance between the predicted bounding box and the center point of the actual bounding box;/> is the square of the diagonal length of the smallest closed rectangle containing the predicted bounding box and the actual bounding box;/> is the square of the Euclidean distance between the width of the predicted bounding box and the width of the actual bounding box;/> is the square of the Euclidean distance between the height of the predicted bounding box and the height of the actual bounding box;/> is the square of the width of the smallest closed rectangle containing the predicted bounding box and the actual bounding box;/> is the square of the height of the smallest closed rectangle containing the predicted bounding box and the actual bounding box; w _gt is the width of the actual bounding box, h _gt is the height of the actual bounding box, w _p is the width of the predicted bounding box, and h _p is the predicted boundary The height of the box.

The beneficial effects of the present invention are that: by using computer vision to detect defects in chip inductors, the present invention can effectively improve detection efficiency, classify defective chip inductors, and at the same time improve the stability and stability of detection. It is accurate and can perform real-time detection during the production process, which is beneficial to improving the yield rate of produced products.

Description of the drawings

Figure 1 is a flow chart of the present invention;

Figure 2 is a structural block diagram of the backbone network in the network detection model of the present invention;

Figure 3 is a structural block diagram of the head network in the network detection model of the present invention;

Figure 4 is a structural block diagram of the CA-ELAN module in the network detection model of the present invention;

Figure 5 is a structural block diagram of the CA module in the CA-ELAN module of the present invention.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

As shown in Figures 1, 2, 3, 4 and 5, the patch inductor defect detection method based on instance segmentation technology provided in this embodiment includes the following steps:

Step 1: Take and collect various patch inductor images through the vision camera. The collected patch inductor images include defect-free patch inductor images and defective patch inductor images. The defective patch inductor image defines the rules. It was formulated by consulting the corresponding system of factory production specifications and through detailed consultation with grassroots workers in the field factory, including images of chip inductors with magnetic ring crack defects, images of chip inductors with electrode exposed copper defects, and electrode exposed wire defects. The image of the chip inductor and the image of the chip inductor with the magnetic ring damage defect;

Construct a patch inductor image data set, collect enough defect-free patch inductor images and enough various defective patch inductor images, and preprocess the collected defective patch inductor images, that is, first Scale the image to 640×640 pixels, and perform denoising and image enhancement operations in sequence; label the defective chip inductor image after preprocessing, mark the location, type and defect type of the chip inductor, and obtain the chip Inductor image data set; at the same time, the preprocessed chip inductor image is enhanced with mosaic data, that is, 4 image data are randomly selected from numerous chip inductor images to form a new image, and the combined image is labeled. , after annotation, it is added to the patch inductor image data set as a new training data. Through the combination of images, more contextual information can be provided for the model in the training phase, thereby improving the generalization ability of the model;

Divide the constructed patch inductor image data set into a training set and a test set at a ratio of 8:2;

Step 2: Build a network detection model based on instance segmentation technology, including a backbone network and a head network; input the images in the patch inductor image data set into the backbone network, and introduce the CA module into the E-ELAN module in the backbone network , obtain the width and height of the chip inductor image through the CA module, and encode the precise position, so that the network captures the context information of the chip inductor image with multi-scale information. The specific steps are:

In step a, the input size is The chip inductor defect feature map is used, and pooling kernels of size (1, W) and (H, 1) are used to perform global average pooling to obtain the horizontal direction feature vector sum encoded by the global average vector in the channel dimension. Vertical eigenvector:

Among them, H represents the height, W represents the width, and C represents the number of channels. Represents the global average value of the input feature map X in the c-th channel,/> Indicates that the c-th channel coordinates are (i, j) feature information;

Step b: Convolve the horizontal feature vector and vertical feature vector obtained in step a using a convolution kernel of size 1×1, batch normalize and use the LeakyReLu activation function for feature mapping:

in, Represents the attention response of the input feature map X on the c-th channel, /> Represents a 1×1 convolution kernel;

Step c, decompose the features mapped in step b into two independent decomposition features according to the original W and H along the horizontal-vertical direction. , and use two 1×1 size convolution kernel convolution and LeakyReLu activation function to perform feature transformation respectively:

,

in, are the attention weights in the horizontal and vertical directions of the input patch inductor defect feature map, /> It is a convolution kernel of size 1×1;

Step d, obtain the final feature vector:

The final output chip inductor defect characteristic map ;

The output chip inductor defect feature map is then used for feature fusion through the E-ELAN module to obtain chip inductor defect image features that incorporate multi-scale information and have contextual information;

That is, three layers of feature maps of different sizes are output through the backbone network. After the three layers of feature maps of different sizes are processed by the RepVGG module, each layer contains a 3×3 convolution kernel and a 1×1 convolution kernel and uses LeakyReLu. After adding batch normalization (batch normalization) as a pre-operation residual network layer before the activation function, after passing through a convolution layer, 20×20×27, 40×40×27, and 80×80×27 are output respectively. feature map;

Step 3: Input the training set in step 1 into the network detection model, train the network detection model, and continuously calculate the total loss function during the training. The total loss function includes the category loss function, the positioning loss function, and the target confidence. degree loss function;

The total loss function:

Among them, t _p is the prediction vector, t _gt is the true value vector; K is the output feature map, S ² is the grid, and B is the number of anchor boxes on each grid; L _CIoU is the positioning loss function, and L _obj is the target The confidence loss function, L _cls is the category loss function; is the weight of the corresponding item;/> It is the k-th output feature map, the i-th grid, and whether the j-th anchor box is a positive sample. If it is a positive sample, it is 1, otherwise it is 0;/> In order to balance the weight of the output feature map of each scale, it corresponds to the output feature map of 80×80×27, 40×40×27, and 20×20×27;

The category loss function:

Among them, c _p is the category score of the prediction frame, c _gt is the true category of the target frame, C is the number of categories, is the weight of the i-th category;

The width and height loss is introduced in the positioning loss function, specifically:

Among them, b is the predicted box, b _gt is the real box, IOU is the coverage degree between the predicted bounding box and the real bounding box, is the square of the Euclidean distance between the predicted bounding box and the center point of the actual bounding box;/> is the square of the diagonal length of the smallest closed rectangle containing the predicted bounding box and the actual bounding box;/> is the square of the Euclidean distance between the width of the predicted bounding box and the width of the actual bounding box;/> is the square of the Euclidean distance between the height of the predicted bounding box and the height of the actual bounding box;/> is the square of the width of the smallest closed rectangle containing the predicted bounding box and the actual bounding box;/> is the square of the height of the smallest closed rectangle containing the predicted bounding box and the actual bounding box; w _gt is the width of the actual bounding box, h _gt is the height of the actual bounding box, w _p is the width of the predicted bounding box, and h _p is the predicted boundary The height of the box;

The target confidence loss function:

Among them, p _o is the target confidence score in the prediction box, p _iou is the iou value of the target box corresponding to the prediction box, and w _obj is the weight of the positive sample;

After calculating the total loss function, the detection model is optimized through small-batch gradient descent, that is, a small part of the patch inductance image is used each time to update parameters, update network weights, accelerate network convergence, reduce oscillation during the convergence process, thereby improving detection The accuracy of the model is used to obtain the trained network detection model;

Step 4: Input the test set in step 1 into the trained network detection model for verification, and obtain the final network detection model after verification;

Step 5: Preprocess the image of the chip inductor collected in real-time production and input it into the final network detection model. The network detection model will predict possible defects on the chip inductor in real time and output the prediction results. The prediction results include the position of the bounding box. , size and corresponding defect category; in order to improve the detection accuracy, we can also process the detection results. First, use the non-maximum suppression algorithm to filter out the most credible bounding boxes based on the confidence of the bounding boxes and eliminate overlaps. The bounding box with a higher degree of accuracy is then used to draw the processed detection results on the original image using the drawing frame component of the OpenCV framework; the drawing frame component will draw a bounding box in the patch inductor defect area in the original image, and place it next to the frame. Mark the corresponding defect category;

Step 6: After the network detection model is used for a period of time, new defects may occur due to changes in production equipment, raw materials or processes. In order to ensure that the network detection model can still maintain high recognition performance in the face of new defect characteristics , it is necessary to update the chip inductor image sample set in a timely manner, and retrain and fine-tune the network detection model.

Claims (8)

1. The patch inductance defect detection method based on the example segmentation technology is characterized by comprising the following steps of:

step 1: shooting and collecting various patch inductance images through a vision camera, preprocessing the patch inductance images, marking the patch inductance images with patch inductance defects after preprocessing, obtaining a patch inductance image data set, and dividing the patch inductance image data set into a training set and a test set;

step 2: constructing a network detection model based on an instance segmentation technology, wherein the network detection model comprises a backbone network and a head network;

the CA-ELAN module is added into the backbone network layer;

adding a layer of structure containing a 3×3 convolution kernel and a 1×1 convolution kernel into the final three prediction heads in the head network, and adding batch normalization as a residual network of a pre-operation before a LeakyReLu activation function;

step 3: inputting the training set in the step 1 into a network detection model, training the network detection model, continuously calculating a total loss function in training, and continuously updating detection model parameters through small-batch gradient descent to obtain a trained network detection model;

step 4: inputting the test set in the step 1 into the trained network detection model for verification, and obtaining a final network detection model after verification;

step 5: preprocessing a patch inductance image acquired in real-time production, and inputting the patch inductance image into a final network detection model for visual real-time detection of patch inductance defects;

step 6: and (3) updating the patch inductance image data set of the network detection model at regular intervals according to the detection result in the step (5), and retraining the network detection model with the updated patch inductance image data set.

2. The method for detecting a chip inductance defect based on an instance segmentation technique according to claim 1, wherein the preprocessing of the chip inductance image in step 1 includes: and sequentially scaling, denoising and enhancing the patch inductance image.

3. The method for detecting a chip inductance defect based on an example segmentation technique according to claim 1 or 2, wherein the chip inductance image data set obtained in step 1 is further enhanced by using mosaic data, that is, 4 chip inductance images are randomly extracted from the chip inductance image data set to be combined into a new chip inductance image, and the new chip inductance image is added into the chip inductance image data set.

4. A method for detecting a chip inductor defect based on an instance division technique as claimed in claim 3, wherein the chip inductor defect in step 1 includes a magnetic ring dark crack defect, an electrode copper exposure defect, an electrode line exposure defect, and a magnetic ring breakage defect.

5. The method for detecting the chip inductance defect based on the instance segmentation technology according to claim 1, wherein the CA-ELAN module is characterized in that the CA module is introduced into the E-ELAN module of the backbone network layer, the CA module obtains the width and the height of the chip inductance image in the chip inductance defect feature map of the backbone network layer, and encodes the accurate position, so that the network captures the context information of the chip inductance image of the multi-scale information, and then the E-ELAN module performs feature fusion to obtain the chip inductance defect image feature with the context information, wherein the multi-scale information is fused.

6. The method for detecting patch inductance defects based on the instance division technique according to claim 5, wherein the CA module comprises the steps of:

step a, inputting the input size asAnd using pooling cores with the sizes of (1, W) and (H, 1) to carry out global average pooling to obtain a horizontal direction feature vector and a vertical direction feature vector of global average value vector codes in channel dimension:

，

wherein H represents height, W represents width, C represents channel number,representing the global average of the input profile X at the c-th channel,/or->Representing that the c-th channel coordinates are (i, j) feature information;

step b, carrying out convolution, batch normalization and LeakyReLu activation functions on the horizontal direction feature vector and the vertical direction feature vector obtained in the step a by using a convolution kernel with the size of 1 multiplied by 1 to carry out feature mapping:

，

wherein,representing the attention response of the input profile X on the c-th channel,/for>Representing a 1 x 1 convolution kernel;

c, decomposing the features mapped in the step b into two independent decomposition features along the horizontal-vertical direction according to the original W and HAnd the characteristic conversion is carried out respectively on the two convolution kernel convolutions with the size of 1 multiplied by 1 and the activation function of the LeakyReLu:

，

，

wherein,attention weights in the horizontal and vertical directions of the input patch inductance defect feature map,convolution kernel of size 1 x 1;

step d, obtaining a final feature vector:

，

final output chip inductor defect characteristic diagram。

7. The method of claim 1, wherein the total loss function in step 3 includes a class loss function, a location loss function, and a target confidence loss function.

8. The method for detecting patch inductance defects based on the example segmentation technique according to claim 7, wherein the positioning loss function introduces a wide-high loss, specifically:

，

wherein b is a prediction frame, b _gt For a real bounding box, the IOU is the degree of coverage between the prediction bounding box and the real bounding box,squaring the Euclidean distance between the prediction boundary box and the actual boundary box center point; />Square of the diagonal length of the smallest closed rectangle that contains the prediction bounding box and the actual bounding box; />Squaring the Euclidean distance between the width of the prediction bounding box and the actual bounding box width; />Is the square of the euclidean distance between the height of the prediction bounding box and the height of the actual bounding box; />Square of the width of the smallest closed rectangle that contains the prediction bounding box and the actual bounding box; />Square of the height of the smallest closed rectangle that contains the prediction bounding box and the actual bounding box; w (w) _gt For the width of the actual bounding box, h _gt To the height of the actual bounding box, w _p For predicting the width of the bounding box, h _p Is the height of the prediction bounding box.