CN109445453A - A kind of unmanned plane Real Time Compression tracking based on OpenCV - Google Patents

Abstract

A real-time compression tracking method for unmanned aerial vehicles based on OpenCV. The sequence is used for feature point detection and matching, and the feature vector of the target to be detected is extracted; during the tracking process, the processed feature vector is used as the training set to train the classifier, and then each frame of the input image is trained using the previous frame. The classifier is trained, and the target window is obtained to achieve tracking; the difference between the actual horizontal distance between the UAV and the target position is calculated according to the position coordinates of the tracking target, and the single-loop position loop PID controller is input, and the UAV is controlled by the position loop. Carry out position adjustment to achieve tracking of moving targets.

Description

A real-time compression tracking method for UAV based on OpenCV

technical field

The invention relates to the technical field of intelligent information processing and UAV tracking, in particular to a real-time compression tracking method for UAV based on OpenCV.

Background technique

Vision-based moving target tracking technology refers to the analysis of the image sequence obtained by the robot through the vision sensor, usually to detect the target, complete the target recognition, and finally track the recognized target to obtain the real-time spatial position and target size of the target. Information such as size, speed and acceleration can be obtained, and the trajectory of the moving target can be obtained. Due to the mature image-based target recognition and tracking algorithms, low tracking cost, high accuracy and strong anti-interference ability, it has wide and practical applications in many fields.

With the gradual expansion of the application scope of UAVs, the requirements for autonomous control of UAV systems are also increasing. The UAV automatic control system performs intelligent information processing by sensing the external environment, and automatically generates corresponding control strategies to achieve various Demand tasks, and have effective and fast adaptive capabilities, especially the UAV's condescending perspective, which can monitor the ground situation in a large range, and can quickly reach places that are not easily involved by personnel, effectively implement monitoring, and reduce the risk of corresponding personnel , which makes drones widely used in the security industry. As we all know, visual management is basically completed by fixed monitoring equipment. With the in-depth application of equipment by users, the problem of monitoring dead ends cannot be avoided. Therefore, in some places with harsh environments , The installation, wiring and maintenance of cameras and other equipment are big problems. Under the trend of diversified demands in the field of visual management of fixed-point video surveillance, new equipment such as drone aerial photography is required to provide technical support in special circumstances.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a real-time compression tracking method for unmanned aerial vehicles based on OpenCV to solve the problems raised in the above background.

The invention mainly uses the moving object detection algorithm, the target tracking algorithm and the tracking controller level to analyze the demand, designs the target detection and tracking algorithm, adopts the moving object detector based on feature points and the target tracking based on the compressed sensing theory to improve the performance of the algorithm. Tracking accuracy and anti-short-term occlusion ability, a single-loop position loop PID controller is designed, and the actual horizontal distance between the UAV and the target is used as the input of the controller to achieve the UAV tracking the moving target.

A real-time compression tracking method for unmanned aerial vehicles based on OpenCV, the method includes:

Step S1: the drone takes off, starts to collect images, and sends back video data to the PC;

Step S2: After selecting the drone to track the target, convert the video into an image frame;

Step S3: Perform feature point detection and matching on the video image sequence based on openCV, and extract the feature vector of the target to be detected;

Step S4: extract the sample data of the image sequence background and the target, and perform initial frame processing;

Step S5: use the sparse measurement matrix to compress and transform the eigenvectors;

Step S6: in the tracking process, the processed feature vector is used as a training set to train the classifier, and then each frame of image input is trained by using the classifier trained in the previous frame to obtain the target window for tracking;

Step S7: after obtaining the tracking target, calculate the position coordinates of the target;

Step S8: Calculate the horizontal displacement of the two in the vertical direction according to the position coordinates of the tracking target and combined with the height data of the UAV;

Step S9: The difference between the actual horizontal distance between the UAV and the moving target is used as the PID input parameter of the UAV position control, and the UAV adjusts the position through the position loop control to realize the tracking of the moving target.

Further, in step S3, the Fast Feature Detector function of openCV is called to set the detection threshold to extract the corner color, texture and other features of the target to be detected, and a target template is established, and the similarity is judged by feature matching with the corresponding target in the real-time video stream. .

Further, using the surf method, call the Feature Detector interface in openCV to find points of interest, and use the Surf Feature Detector and its function detect to realize the detection process of moving objects.

Further, in step S5, first randomly extract feature point information under images of different scales in different sample areas, and then use the Flann Based Matcher interface and the function FLANN to achieve fast and efficient matching, and reduce the dimension of the high-dimensional feature information. , build a specific appearance model in the corresponding compressed domain.

Further, in the initial frame, several pieces of sample data of the target and the background are sampled, and then multi-scale transformation is performed on them, and then the multi-scale image features are dimensionally reduced through the sparse measurement matrix, and then the dimensionality-reduced features are removed. Train a Naive Bayes classifier.

Further, in step S9, the position loop control includes the following steps:

S1: Determine the focal length of the camera, assuming a target of width w. Place this target at a distance d from the camera and measure the pixel width p of the object to get the camera focal length formula:

f=(p×d)/w

S2: Calculate the horizontal displacement between the drone and the target by calculating the pixel position of the target in the image. Set the pixel position of the target on the image plane as (u, v), then the horizontal displacement between the target and the drone (x, y) is expressed as:

Z is the height of the drone, f is the focal length of the camera, β is the pitch angle of the camera, α is the angle formed by the line connecting the two points of the object image and the optical axis of the camera, and (u0, v0) is the pixel coordinate of the image center point.

S3: The difference between the actual horizontal distance between the UAV and the moving target is used as the PID input parameter of the UAV position control. The UAV adjusts the position through the position loop control. For the position control of the UAV, assume L as the target The difference between the position and the current position, then the output control amount φ _k : and L satisfy the following PID control relationship:

S4: Set the difference between the ideal control amount φ _k of the UAV position control and the actual measured value as the deviation amount φ _{deviation k} , and the deviation amount and the final control output φ _finalk satisfy the following PID control relationship:

The multi-scale image features are dimensionally reduced by a sparse measurement matrix, and then the naive Bayes classifier is trained through the reduced dimensionality features, and the tracking problem is transformed into a binary classification problem using the naive Bayes classifier.

The purpose of the present invention is to provide a set of real-time tracking methods for UAVs based on automatic control algorithms combined with high-precision, strong anti-interference target tracking methods, which can overcome the shortcomings of ordinary visual sensors such as fixed position and small field of view, and can be very effective. to expand the field of view of the vision sensor. Rotor-type UAVs equipped with visual sensors can monitor the ground and air in real time, as well as monitor and track moving targets, and can effectively complete many monitoring tasks.

Beneficial effects:

1. The present invention converts the multi-dimensional difference between the tracking target and the UAV into the difference between the actual horizontal distance between the UAV and the target position through multi-scale image feature dimensionality reduction, and inputs the single-loop position loop PID controller. , to achieve real-time tracking of moving targets, and to simplify the data of target tracking.

2. The present invention establishes a target template based on openCV, extracts feature vectors, improves tracking accuracy and enhances anti-interference, reduces the impact of complex backgrounds on its tracking, adopts compressed sensing algorithm to reduce the amount of data calculation, and accelerates the target tracking speed.

3. Using the combination of embedded hardware and image processing, the data transmission and reception efficiency is high, and the transmission is stable, which is convenient for data observation and analysis.

Description of drawings

FIG. 1 is an overall system frame diagram of the present invention.

Figure 2 is a flowchart of a moving object detection algorithm.

Figure 3 shows the compressed sensing algorithm model.

Figure 4 shows the design framework of the aircraft detection controller.

Detailed ways

In order to make it easy to understand the technical means, creation features, achieved goals and effects of the present invention, the present invention will be further described below with reference to the specific figures.

Example 1

The overall frame diagram is shown in Figure 1. First, the drone is controlled by the remote control to take off and reach the vicinity of the tracking target. After the tracking mode is turned on, the drone obtains the ground image and transmits it back to the PC through the image, and the PC receives the image. Then, based on openCV, the feature point detection and matching of the video image sequence are carried out, and the feature vector of the target to be detected is extracted. After multi-scale transformation, the multi-scale image features are dimensionally reduced by using compressed sensing theory, and then the dimensionality-reduced features are used for training. Naive Bayes classifier, after the naive Bayes classifier is classified to obtain the tracking target window, the pixel coordinates and the actual height distance of the target in the image are converted into the horizontal actual displacement of the two in the vertical direction. The difference between the actual horizontal distance between the UAV and the moving target is used as the PID input parameter of the UAV position control. The UAV adjusts the position through the position loop control to realize the tracking of the moving target.

The specific operation steps of the OpenCV-based UAV real-time compression tracking method are as follows. The moving object detection process and the compressed sensing algorithm model are shown in Figure 2 and Figure 3 respectively.

The specific steps are:

Step S1: the drone takes off, starts to collect images, and sends back video data to the PC;

Step S2: After selecting the drone to track the target, convert the video into an image frame;

Step S3: Perform feature point detection and matching on the video image sequence based on openCV, and extract the feature vector of the target to be detected;

Step S4: extract the sample data of the image sequence background and the target, and perform initial frame processing;

Step S5: use the sparse measurement matrix to compress and transform the eigenvectors;

Step S6: in the tracking process, the processed feature vector is used as a training set to train the classifier, and then each frame of image input is trained by using the classifier trained in the previous frame to obtain the target window for tracking;

Step S7: after obtaining the tracking target, calculate the position coordinates of the target;

Step S8: Calculate the horizontal displacement of the two in the vertical direction according to the position coordinates of the tracking target and combined with the height data of the UAV;

Step S9: The difference between the actual horizontal distance between the UAV and the moving target is used as the PID input parameter of the UAV position control, and the UAV adjusts the position through the position loop control to realize the tracking of the moving target.

It should be pointed out that the feature point detection and matching in the OpenCV-based UAV real-time compression tracking method refers to the extraction of the corner color, texture and other features of the target to be detected by calling the Fast Feature Detector function of opencv to set the detection threshold, and establish the target template. , through feature matching with the corresponding target in the real-time video stream to determine the similarity, and the specific process is shown in Figure 2.

It should be pointed out that the compressed sensing theory in the real-time compression tracking method of UAV based on OpenCV refers to randomly extracting feature point information under images of different scales in different sample areas; then use the Flann BasedMatcher interface and the function FLANN to achieve fast Efficient matching, after dimensionality reduction of high-dimensional feature information, a specific appearance model is established in the corresponding compressed domain.

It should be pointed out that the real-time compressed sensing tracking in the real-time compression tracking method of UAV based on OpenCV refers to the nearest neighbor search for high-dimensional features and large data sets through the Flann function, and then dimensionality reduction and transformation are performed after fast matching. Then, the processed feature vector is used as a training set to train the classifier, and then each frame of the input image is trained using the classifier trained on the previous frame, and the target window is obtained to achieve tracking.

It should be pointed out that in the real-time compression tracking method of UAV based on OpenCV, at the initial frame, several pieces of sample data of the target and background are sampled, and then multi-scale transformation is performed on them, and then the multi-scale data is analyzed by a sparse measurement matrix. The image features are dimensionally reduced, and then the naive Bayes classifier is trained through the dimensionality-reduced features. The sparse dimensionality reduction process is shown in Figure 3.

It should be pointed out that, in the real-time compression tracking method of UAV based on OpenCV, after the tracking target window is obtained after classification by the naive Bayes classifier, the pixel coordinates of the target in the image and the actual height distance are converted into two. For the actual horizontal displacement in the vertical direction, the difference between the actual horizontal distance between the UAV and the moving target is used as the PID input parameter of the UAV position control. The tracking process is shown in Figure 4.

The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. The present invention is not limited by the above-mentioned embodiments. The above-mentioned embodiments and descriptions only illustrate the principles of the present invention without departing from the spirit of the present invention. Under the premise of and scope, the present invention will also have various changes and improvements, and these changes and improvements all fall within the scope of the claimed invention, and the claimed scope of the present invention is defined by the appended claims and their equivalents .

Claims (6)

1. a kind of unmanned plane Real Time Compression tracking based on OpenCV, which is characterized in that the described method includes:

Step S1: unmanned plane takes off, and starts to acquire image, passback video data to the end PC；

Step S2: after selection unmanned plane tracking target, picture frame is converted by video；

Step S3: characteristic point detection and matching are carried out to sequence of video images based on openCV, extract the spy of target to be detected Levy vector；

Step S4: extracting the sample data of image sequence background and target, carries out initial frame processing；

Step S5: feature vector is compressed using sparseness measuring matrix, is converted；

Step S6: during tracking, classifier is trained using treated feature vector as training set, is inputted later Each frame image be all trained using the trained classifier of previous frame, obtain target window realize tracking；

Step S7: after obtaining tracking target, the position coordinates of target are calculated；

Step S8: it according to the position coordinates of tracking target, both is calculated in vertical direction in conjunction with unmanned plane altitude information Horizontal displacement；

Step S9: the difference of unmanned plane and the real standard distance of mobile target is inputted as the PID of unmanned plane position control Parameter, unmanned plane are controlled by position ring and carry out position adjustment, realize the tracking to mobile target.

2. the unmanned plane Real Time Compression tracking according to claim 1 based on OpenCV, which is characterized in that step S3 In, call the Fast Feature Detector function setup detection threshold value of openCV extract target to be detected angle point color, Texture eigenvalue establishes target template, carries out similitude and then with target progress characteristic matching is corresponded in live video stream Judgement.

3. the unmanned plane Real Time Compression tracking according to claim 2 based on OpenCV, which is characterized in that use Surf method calls the Feature Detector interface in openCV to find point-of-interest, uses SurfFeature Detector and its function detect realizes the detection process of mobile object.

4. the unmanned plane Real Time Compression tracking according to claim 3 based on OpenCV, which is characterized in that step S5 In, the characteristic point information under different scale images is randomly selected within the scope of different sample areas first, uses Flann later Based Matcher interface and function FLANN, realization rapidly and efficiently match, after high dimensional feature information is carried out dimensionality reduction, in phase Specific apparent model is established in the compression domain answered.

5. the unmanned plane Real Time Compression tracking according to claim 1 based on OpenCV, which is characterized in that step S5 In, when initial frame, sampling obtains the sample data of several target and backgrounds, multi-scale transform then is carried out to it, Dimensionality reduction is carried out to multi-scale image feature by sparseness measuring matrix again, the simple pattra leaves of training is then gone by the feature after dimensionality reduction This classifier.

6. the unmanned plane Real Time Compression tracking according to claim 1 based on OpenCV, which is characterized in that step S9 In, position ring control the following steps are included:

S1: the focal length of camera is determined, it is assumed that the target that a width is w.This target is placed on to the position for being d apart from camera The pixel wide p for setting and measuring object, obtains camera focus formula:

F=(p × d)/w

S2: the horizontal displacement of unmanned plane and target is calculated by the location of pixels of target in the picture, if target is in image The location of pixels of plane is (u, v), then the horizontal displacement (x, y) between target and unmanned plane indicates are as follows:

It is camera focus that Z, which is unmanned plane height f, and β is the pitch angle of video camera, α be image two o'clock line and camera optical axis at Angle, (u0, v0) be image center pixel coordinate.

S3: inputting parameter as the PID of unmanned plane position control for the difference of unmanned plane and the real standard distance of mobile target, Unmanned plane is controlled by position ring and carries out position adjustment, for the position control of unmanned plane, it is assumed that L is as target position and now The difference for carving position, then export control amount φ_k: meet following PID control relationship with L:

S4: by the ideal control amount φ of unmanned plane position control_kDeparture φ is set as with the difference of actual measured value_{Deviation k}, the deviation Amount and final control export φ_finalkMeet following PID control relationship: