CN114639124A - A Pedestrian Re-identification Method Based on Weighted Feature Fusion - Google Patents

Abstract

The invention discloses a pedestrian re-identification method based on weighted feature fusion. The steps are: inputting pedestrian images, and extracting pedestrian features through a backbone network ResNeSt-50; inputting different features obtained from the backbone network into a weighted feature fusion pyramid model In the process, multi-scale features containing high-level semantic information and low-level high-resolution information are obtained; high-level semantic features are used as global features in the global branch network; low-level high-resolution features are used as local features in the local branch network; joint Softmax loss function, triple loss function and center loss function to optimize training on global features and local features, respectively. The present invention performs feature fusion in a weighted feature pyramid network, and uses high-level semantic feature words as global features; jointly optimizes and trains global features and local features by combining Softmax loss function, triple loss function and central loss function, and improves pedestrian performance. Recognition accuracy and accuracy.

Description

A Pedestrian Re-identification Method Based on Weighted Feature Fusion

technical field

The invention belongs to the technical field of pedestrian re-identification, and in particular relates to a pedestrian re-identification method based on weighted feature fusion.

Background technique

Pedestrian re-identification refers to finding a specific pedestrian video sequence or image in the case of cross-camera and cross-scene. With more and more fields of intelligent video systems and intelligent security applications, pedestrian re-identification is also attracting more and more scholars to conduct research. Due to different camera parameters and different scenes, there will be problems such as illumination, occlusion or multi-pose changes, resulting in great differences in pedestrian images captured by the same person, which will increase the difficulty for the network to extract features.

Most of the current pedestrian re-identification methods use the fusion of global features and local features. Generally, feature pyramids are introduced to strengthen the fusion of high-level semantic features and low-level high-resolution features. However, in the process of fusion, different features are not considered during feature fusion. Contribution of Fusion.

SUMMARY OF THE INVENTION

Based on the above deficiencies of the prior art, the technical problem solved by the present invention is to provide a pedestrian re-identification method based on weighted feature fusion. Semantic feature words are used as global features, and low-level high-resolution features are used as local features. Combined with Softmax loss function, triple loss function and center loss function, the global and local features are optimized and trained respectively, which improves the accuracy and accuracy of pedestrian recognition. Rate.

In order to solve the above-mentioned technical problems, the present invention realizes through the following technical solutions:

The pedestrian re-identification method based on weighted feature fusion of the present invention comprises the following steps:

Step S1: Input the pedestrian image, and extract the characteristics of the pedestrian through the backbone network ResNeSt-50;

Step S2: Input different features obtained from the backbone network into the weighted feature fusion pyramid model to obtain multi-scale features including high-level semantic information and low-level high-resolution information;

Step S3: use high-level semantic features as global features in the global branch network; use low-level high-resolution features as local features in the local branch network;

Step S4: Combine the Softmax loss function, the triplet loss function and the center loss function to optimize and train the global features and local features respectively.

Preferably, the pedestrian image in step S1 will be randomly erased and flipped horizontally, which increases the amount of pedestrian data; when using the ResNeSt-50 network to extract pedestrian features, downsampling 3, 4, and 5 times will be obtained. feature map.

Optionally, in step 2, the feature maps downsampled by 3, 4, and 5 times obtained in step 1 are used as input to the weighted feature pyramid network to obtain weighted fusion of downsampling by 3, 4, 5, and 6 times. feature map.

Optionally, in step 3, the feature maps downsampled by 5 times and 6 times in step 2 are connected in the channel dimension, and then global features are obtained through global average pooling; the feature maps are downsampled by 3 times and 4 times. Connect in the channel dimension, divide the feature map horizontally into 4 parts, and then perform generalized mean pooling on these 4 feature maps respectively, and connect the results to obtain local features.

From the above, the pedestrian re-identification method based on weighted feature fusion of the present invention has at least the following beneficial effects:

1. Using the ResNeSt-50 network to extract features can make the network pay more attention to the key areas of pedestrian images and reduce the attention to non-key areas;

2. Using the feature pyramid network with weighted operation can better obtain richer hierarchical feature information, thereby improving the generalization ability and robustness of the network;

3. Using the loss function of the joint Softmax loss function, triple loss function and center loss function, the model can be better optimized.

The above description is only an overview of the technical solution of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other objects, features and advantages of the present invention more obvious and easy to understand , the following detailed description is given in conjunction with the preferred embodiments and in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings of the embodiments will be briefly introduced below.

Fig. 1 is ResNeSt-50 splitting structure diagram;

Fig. 2 is a weighted feature pyramid network structure diagram;

Fig. 3 is a branch network structure diagram;

FIG. 4 is a flow chart of the pedestrian re-identification method based on weighted feature fusion of the present invention.

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. As a part of this specification, the principles of the present invention will be illustrated by examples. Other aspects, features and advantages of the present invention will become apparent from the detailed description. In the figures to which reference is made, the same reference numerals are used for the same or similar parts in different figures.

The pedestrian re-identification method based on weighted feature fusion of the present invention will be described in detail below with reference to FIGS. 1-4 . The pedestrian re-identification method based on weighted feature fusion of the present invention comprises the following steps:

Step 1: Input the pedestrian image, and extract the characteristics of the pedestrian through the backbone network ResNeSt-50;

Step 2: Input the different features obtained from the backbone network into the weighted feature fusion pyramid model to obtain multi-scale features containing high-level semantic information and low-level high-resolution information;

Step 3: In the global branch network, high-level semantic features are used as global features; in the local branch network, low-level high-resolution features are used as local features, and the branch network diagram is shown in Figure 3;

Step 4: Combine the Softmax loss function, the triple loss function and the center loss function to optimize the training of global features and local features respectively.

Specifically, the steps of step 1 are as follows:

Step 1.1: Randomly erase and flip the pedestrian image randomly, and then resize the image to 384×128;

Step 1.2: Input into the ResNeSt-50 network to extract pedestrian features, and finally extract features with downsampling 3 times, downsampling 4 times, and downsampling 5 times, respectively denoted as C ₃ , C ₄ and C ₅ , ResNeSt The split structure in the -50 network is shown in Figure 1. The ResNeSt-50 network can make the network pay more attention to the key areas of pedestrian images and reduce the attention to non-key areas.

Specifically, the steps of step 2 are as follows:

Step 2.1: Input C ₃ , C ₄ and C ₅ obtained in step 1.2 into the weighted feature pyramid network, the weighted feature pyramid network is shown in Figure 2;

Step 2.2: C ₃ , C ₄ and C ₅ obtain feature maps P _{3_in} , P _{4_in} and P _{5_in} with a uniform channel number of 512 after the convolution operation, and then perform a maximum pooling operation on P _{5_in} to obtain 6 times of downsampling Feature map P _{6_in} . Then perform a convolution operation on P _{6_in} to obtain a feature map P _{6_td} , upsample P _{6_td} and add it to P _{5_in} to obtain P _{5_td} , upsample P _{5_td and add it to P 4_in to} obtain P _{4_td} , and so on for each level , and finally obtain feature maps P _{6_out} , P _{5_out} , P _{4_out} and P _{3_out} . Taking P _{5_td} and P _{5_out} as an example, the operation formula is as follows:

Where ω _i is a learnable parameter, 0≤ω _i ≤1, ε is a very small value, the main function is to avoid the situation where the denominator is 0.

Specifically, the steps of step 3 are as follows:

Step 3.1: _{Upsample P6_out} into a feature map of the same size as _{P5_out} , then connect it with _{P5_out} in the channel dimension, and then input it into Global Average Pooling (GAP) to obtain global features of pedestrian images .

Step 3.2: Downsample P _{3_out} to obtain a feature map of the same size as P _{4_out} , and then connect it with P _{4_out} in the channel dimension to obtain a local feature map, and then divide the local feature map into 4 evenly in the horizontal direction, and then Generalized Mean Pooling (GeM) is performed on the four feature maps respectively, and the results are connected. Finally, a 1×1 convolution is performed to reduce the dimension of the feature map to obtain a local feature map. The local feature maps contain high-resolution features of pedestrian images. In the local branch, the formula for generalized mean pooling is:

where p _k is a hyperparameter, f ^(g) is average pooling when p _k = 1, and f ^(g) is max pooling when p _k → ∞. During the training process, generalized mean pooling will automatically learn the hyperparameter p _k , which allows the model to better capture feature differences.

The above are only the preferred embodiments of the present invention, of course, the scope of the rights of the present invention cannot be limited by this. Several improvements and changes are made, and these improvements and changes are also regarded as the protection scope of the present invention.

Claims (4)

1. A pedestrian re-identification method based on weighted feature fusion is characterized by comprising the following steps:

step S1: inputting a pedestrian image, and extracting the characteristics of the pedestrian through a backbone network ResNeSt-50;

step S2: inputting different features acquired from a backbone network into a weighted feature fusion pyramid model to obtain multi-scale features containing high-level semantic information and low-level high-resolution information;

step S3: taking the high-level semantic features as global features in the global branch network; taking the low-level high-resolution features as local features in the local branch network;

step S4: and respectively carrying out optimization training on the global features and the local features by combining the Softmax loss function, the triple loss function and the central loss function.

2. The pedestrian re-recognition method based on weighted feature fusion according to claim 1, wherein the pedestrian image in step S1 is randomly subjected to random erasing and horizontal flipping operations, increasing the pedestrian data amount; when extracting pedestrian features using the resenestt-50 network, downsampled 3, 4, and 5 times feature maps are obtained.

3. The pedestrian re-identification method based on weighted feature fusion as claimed in claim 2, wherein in step 2, the feature maps of the down-sampling 3, 4 and 5 times obtained in step 1 are input into the weighted feature pyramid network to obtain the weighted and fused feature maps of the down-sampling 3, 4, 5 and 6 times.

4. The pedestrian re-identification method based on the weighted feature fusion as claimed in claim 3, wherein in step 3, the feature maps down-sampled by 5 times and 6 times in step 2 are connected in channel dimension, and global features are obtained through global average pooling; and connecting the 3-time and 4-time downsampled feature maps in a channel dimension, dividing the feature maps into 4 parts horizontally and uniformly, then performing generalized mean pooling on the 4 feature maps respectively, and connecting the results to obtain local features.

Images