CN108921001B - A video surveillance pan-tilt using artificial intelligence predictive tracking and its tracking method - Google Patents

Abstract

This application proposes a video surveillance pan/tilt using artificial intelligence predictive tracking and a predictive tracking method, which can analyze the differences in scene modes of video images in different scenes, and then, for a certain scene mode, machine learning based on artificial intelligence The method is to determine the state characteristics of the key targets, so as to obtain the identification and tracking standards of key targets in a specific scene mode, and in the scene of a variety of moving targets, it can be based on the correlation between each moving target and the scene mode and the collection purpose. Extract important targets, so as to realize the prediction and tracking of the gimbal according to the movement of the important targets.

Description

A video surveillance pan-tilt using artificial intelligence predictive tracking and its tracking method

technical field

The present application relates to the technical field of video surveillance applied to smart cities, and in particular, to a video surveillance pan-tilt using artificial intelligence prediction and tracking and a tracking method thereof.

Background technique

A smart city is a digital information system that uses IoT technology to connect smart buildings, smart communities, smart streets, and smart homes to form a digital information system with comprehensive coverage, accurate collection, and intelligent analysis. The collection and application of video information plays an indispensable and important role in smart cities, and is the front-end basis for realizing important functions of smart cities such as regional security monitoring, face identification, and traffic information extraction.

In public spaces, video surveillance cameras are the main front-end facilities for the collection of video information. A video surveillance camera is generally composed of a camera and a surveillance pan/tilt. Among them, the monitoring platform is used to carry the camera on the structure, and the carrying camera is rotated to adjust the shooting direction and pitch angle, so that one camera can cover a larger predetermined space with its limited shooting field of view.

Locking a specific moving object and driving the camera to track and shoot the object is the core function of the monitoring gimbal. In order to ensure proper exposure and clear focus, it is generally required that the main object to be photographed be in the central area of the photographic field of view during the video information collection process. Therefore, when the main object to be photographed is a moving object, the monitoring pan/tilt must be able to follow the moving object to drive the camera to adjust the shooting direction and pitch angle, so that the moving object is always kept near the central area of the shooting field of view.

In order to realize the above-mentioned tracking shooting function, the monitoring PTZ needs to extract the moving object from the continuous real-time picture frames shot by the camera. When it is in the central area of the camera, the camera is adaptively driven to adjust, so that the moving object returns to the central area of the shooting field of view.

In order to improve the lag of tracking shooting, the monitoring pan/tilt can also introduce a prediction mechanism, which extracts the moving direction and speed of the moving object from the position change law of the moving object in the previous real-time picture frames, and then according to the moving direction and the moving speed. The speed predicts the expected position deviation of the moving object in the subsequent real-time picture frame, and adjusts the monitoring pan/tilt in advance with reference to the prediction, so that the moving object returns to the central area of the shooting field of view in the subsequent real-time picture frame.

However, in the real application of smart cities, the structure of the public space scene for video information collection is often complex, and there are usually a large number of objects in the scene, including a large number of moving objects. Therefore, for the monitoring PTZ, how to achieve accurate prediction and tracking in the scene structure of complex and multi-moving objects is also an urgent problem to be solved. For example, when there are multiple moving objects in the scene, and there are differences in the moving direction and speed of each moving object, the monitoring platform is required to be able to select and lock the appropriate moving object based on the predetermined purpose of video information collection, so as to Carry a camera for tracking shooting.

In the prior art, when the scene has multiple moving targets, the tracking strategy adopted by the monitoring pan/tilt generally includes: locking the first moving target in the shooting scene for tracking, until the moving target leaves the shooting scene, then update the tracking strategy. Track to the earliest existing moving object in the scene; or, lock the latest moving object in the scene for tracking each time; or, set a key area in the shooting scene, and implement the method for the moving object closest to the key area. Tracking; or, select the moving target that occupies the largest proportion of the screen in the shooting scene to perform tracking.

However, these tracking strategies often cannot fully match the scene characteristics of video information collection and the intended purpose of video information collection. For example, according to the scene characteristics of the video information collection and the purpose of collecting the video information of the scene, the moving target with the greatest importance and relevance to the collection purpose under the scene characteristics should be selected for tracking, and it is not completely affected by the moving target being in the scene. , the size of the moving object, or the timing of its appearance in the scene.

However, because the association between the purpose of video scene acquisition and scene characteristics is often not defined based on some explicit fixed rules, and the characteristics of the acquisition scene are difficult to describe in advance, and the tracking strategy in the video acquisition process needs to be combined Therefore, it is difficult to use the above-mentioned simple, clear and static tracking rules in the prior art to achieve tracking of moving targets with the greatest importance and relevance for acquisition purposes under any scene feature.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present application is to propose a video surveillance pan/tilt using artificial intelligence predictive tracking and a tracking method thereof. The invention uses artificial intelligence methods to realize the recognition of scene features in the process of video information collection, and learns the selection strategy of tracking and locked moving objects under various scene modes, so as to realize the identification of moving objects with the greatest importance and relevance. Automatic extraction and predictive tracking.

The present application provides a video surveillance pan/tilt using artificial intelligence prediction and tracking, which is characterized by comprising: a scene pattern recognition unit, a target area intelligent learning unit, a tracking target extraction unit, and a pan/tilt drive unit; wherein

The scene pattern recognition unit is used to extract the target contained in each scene from the scene video picture captured by the camera of the video surveillance platform, and quantify the target state to generate a state value; according to the state value of the target, determine two adjacent images. Whether the scene video pictures are of the same scene mode, mark the scene video pictures of the same scene mode with the same type of scene mode mark;

The target area intelligent learning unit is used to learn the state value characteristics of important moving targets in each scene mode through a certain number of samples;

The tracking target extraction unit is used for the learning result of the target area intelligent learning unit. According to the state value of each moving target identified from the current scene video screen, it is judged whether the moving target belongs to an important target. When the result indicates that one of the current scene video screen If the moving target is an important target, then the screen position coordinates of the moving target are provided to the PTZ drive unit;

The pan-tilt driving unit obtains the screen position coordinates of the important target in the video picture of the current scene from the tracking target extraction unit, so as to lock the important target in the video screen of the current scene according to the screen position coordinates, and perform tracking shooting.

Preferably, the scene pattern recognition unit compares whether two adjacent scene video pictures contain the same object, and whether the proportion of the same object to all objects in each scene video picture is lower than the first threshold; If the percentage of the same objects in at least one of the video pictures is lower than the first threshold, it is determined that the two scene video pictures belong to different scene modes; if the same objects contained in the two adjacent scene video pictures account for the percentage of all objects are greater than or equal to the first threshold, then the respective state values of the same objects in the two pictures are used to calculate the overall state difference of all the same objects in the two scene video pictures; if the overall state difference is greater than or equal to the second threshold, then It is determined that the two scene video pictures have different scene modes, and if the overall state difference is lower than the second threshold, it is determined that the two scene video pictures have the same scene mode.

Preferably, the target area intelligent learning unit extracts a certain number of scene video pictures as sample pictures for each scene mode, and manually identifies the moving target area that should be locked and tracked among these sample pictures, and determines which of the sample pictures A moving target belonging to an important target; determine the state value of the important target in the sample picture; perform training on the state value of the important target in all sample pictures of each scene mode, and obtain the trained important target recognition SVM classification vector machine ; For all the sample pictures of each scene mode, extract all the moving objects contained in the sample pictures and the state values of each moving object, obtain a list of target state values, and carry out the target state list of all the sample pictures of each scene mode. Training to obtain the trained scene pattern recognition SVM classification vector machine.

Preferably, the tracking target extraction unit is used for substituting the moving target state value list of the current scene video picture into each scene pattern recognition SVM classification vector machine after training, thereby judging which scene mode the current scene video picture belongs to; Further, the state value of each moving object in the scene video picture is substituted into the important object recognition SVM classification vector machine corresponding to the trained scene mode, so as to identify whether each moving object belongs to an important object, when the classification result indicates that the current scene video A moving object in the picture is an important object.

Preferably, the pan-tilt driving unit analyzes the positional change of the important target in each frame of the scene video image, and extracts the movement direction and movement speed of the important target when the important target deviates from the central area of the shooting field of view, and then extracts the movement direction and movement speed of the important target. The predicted position deviation of the important target is predicted according to the moving direction and speed, and the rotating mechanical mechanism of the monitoring platform is driven and monitored in advance with reference to the prediction.

The present invention further provides an artificial intelligence prediction and tracking method applied to a video surveillance platform, characterized in that it includes:

The scene pattern recognition step is used to extract the target contained in each scene from the scene video picture, and quantify the target state to generate a state value; according to the state value of the target, determine whether two adjacent scene video pictures are the same scene pattern , mark the same type of scene mode mark on the scene video images of the same scene mode;

The intelligent learning step of the target area is used to learn the state value characteristics of important moving targets in each scene mode through a certain number of samples;

The tracking target extraction step is used for the learning results of the intelligent learning unit in the target area. According to the state value of each moving target identified from the current scene video screen, it is judged whether the moving target belongs to an important target. When the result indicates that one of the current scene video images If the moving target is an important target, then the screen position coordinates of the moving target are provided to the PTZ drive unit;

In the pan-tilt driving step, the screen position coordinates of the important targets in the video picture of the current scene are obtained from the tracking target extraction unit, so as to lock the important targets in the video screen of the current scene according to the screen position coordinates for tracking shooting.

Preferably, in the scene pattern recognition step, it is compared whether two adjacent scene video pictures contain the same object, and whether the proportion of the same object to all objects in each scene video picture is lower than the first threshold; If the percentage of identical objects in at least one of the scene video pictures is lower than the first threshold, it is determined that the two scene video pictures belong to different scene modes; if the same objects contained in the two adjacent scene video pictures account for all the objects The percentage of all the same objects in the two scenes is greater than or equal to the first threshold, then the respective state values of the same objects in the two pictures are used to calculate the overall state difference of all the same objects in the two scenes; if the overall state difference is greater than or equal to the second threshold , it is determined that the two scene video pictures have different scene modes, and if the overall state difference is lower than the second threshold, it is determined that the two scene video pictures have the same scene mode.

Preferably, in the target area intelligent learning step, a certain number of scene video images are extracted for each scene mode as sample images, and the moving target area that should be locked and tracked among these sample images is manually identified, and each sample image is determined. The moving target that belongs to the important target in the picture; determine the state value of the important target in the sample picture; perform training on the state value of the important target in all the sample pictures of each scene mode, and obtain the trained important target recognition SVM classification Vector machine; for all sample pictures of each scene mode, extract all moving objects contained in the sample pictures and the state values of each moving object, obtain a list of target state values, and combine the target states of all sample pictures of each scene mode The list is trained to obtain the trained scene pattern recognition SVM classification vector machine.

Preferably, in the tracking target extraction step, the moving target state value list of the video picture of the current scene is substituted into each scene pattern recognition SVM classification vector machine after training, so as to determine which scene the video picture of the current scene belongs to. Then, the state value of each moving target in the scene video picture is substituted into the important target recognition SVM classification vector machine corresponding to the trained scene mode, thereby identifying whether each moving target belongs to the important target, when the classification result indicates that the current A moving object in the scene video picture is an important object.

Preferably, in the step of driving the pan-tilt head, the change of the position of the important object in each frame of the scene video picture is analyzed, and when the important object deviates from the central area of the shooting field of view, the moving direction and the moving speed of the important object are extracted. , and then predict the estimated position deviation of the important target according to the movement direction and speed, and drive and monitor the rotating mechanical mechanism of the pan/tilt head in advance with reference to the prediction.

It can be seen that the advantages of the present invention include: the differences in the scene modes of the video images of different scenes can be analyzed, and then for a certain scene mode, the machine learning method based on artificial intelligence can determine the state characteristics of the key targets, so as to obtain specific Recognition and tracking criteria for key targets in scene mode. In the scene of multiple moving targets, important targets can be extracted based on the correlation between each moving target and the scene mode and the collection purpose, so as to realize the pan-tilt according to the movement of the important target. forecast tracking.

Description of drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

1 is a schematic diagram of the overall structure of a video surveillance platform according to an embodiment of the present application;

2 is a schematic diagram of a specific structure of a target area intelligent learning unit according to an embodiment of the present application;

3 is a schematic structural diagram of a tracking target extraction unit according to an embodiment of the present application;

FIG. 4 is a flowchart of an artificial intelligence prediction tracking method according to an embodiment of the present application.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

FIG. 1 is a schematic diagram of the overall structure of a video surveillance pan/tilt using artificial intelligence prediction and tracking proposed by the present invention. The video surveillance pan/tilt is divided into: a scene pattern recognition unit 101 , a target area intelligent learning unit 102 , a tracking target extraction unit 103 , and a pan/tilt driving unit 104 .

First, the video surveillance pan/tilt is connected to the camera via a wired or wireless signal, and each real-time picture frame captured by the camera is obtained, and each real-time picture frame represents a scene video image of the space area that the video surveillance pan/tilt is responsible for shooting.

The scene pattern recognition unit 101 is used for extracting objects contained in each scene from the scene video pictures captured by the cameras of the video surveillance platform, and quantifying the state of the objects to generate a state value. For any two adjacent scene video pictures, first use the edge extraction method of the canny operator to extract the picture area surrounded by each closed edge from the picture as a target, and obtain the color features and texture features of each target. , any one or more of the vector features from the region centroid to the region edge. By comparing these features, it is judged whether two adjacent scene video pictures contain the same object, and whether the proportion of the same object to all objects in each scene video picture is lower than the first threshold; if at least one of the two scene video pictures If the percentage of the same target is lower than the first threshold, it is determined that the two scene video pictures belong to different scene modes respectively. If the percentage of the same objects contained in the two adjacent scene video pictures to the total objects is greater than or equal to the first threshold, then further use the respective state values of the same objects in the two pictures to calculate all the same objects in the two scene video pictures If the overall state difference is greater than or equal to the second threshold, it is determined that the two scene video pictures have different scene modes; if the overall state difference is lower than the second threshold, it is determined that the two scene video pictures have different scene modes Has the same scene mode. The state value of the target here may include the value of a number of parameters, such as the size of the screen area where each target is located in the scene video picture, the aspect ratio, the coordinates of the screen position, and the inclination. According to the above determination result, the scene mode identification unit 101 marks each scene video picture with its scene mode, and the scene video pictures with the same scene mode are marked with the same type of scene mode mark.

For example, for all the scene video pictures S ₁ , S ₂ ......S _n-1 , S _n ...... S _m collected in a time interval, for two adjacent scene video pictures S _n-1 , For _Sn , after target recognition, the set of all moving targets contained in the scene video picture Sn _-1 is Object _n-1 ={O ₁ ,O ₂ ......O _{n-1 ,} On ......O _k }, The set of all moving objects contained in the scene video picture _{Sn is Object n =} {O' ₁ , O' ₂ ... O' _l-1 , O' _l ... O' _m }, then determine two scene video pictures S _{n -1} , the same target contained in Sn, take the same target in the two scene video images as the intersection of the above two sets It is judged that the number of objects in the intersection Object _n-1 ∩ Object _n accounts for the percentages of the number of objects in the set Object n _{- 1} and the set Object _n respectively. _Sn has different scene modes. If the percentages of the number of targets in the intersection Object _n-1 ∩ Object _n respectively account for the number of targets in the set Object _n-1 and the set Object _n , respectively, are greater than or equal to the first threshold, then further for the intersection For each object among them, analyze the absolute difference between the state value of the object in the scene video picture Sn _-1 and the state value of _Sn in the scene video picture (if each object is in the scene video picture Sn _-1 , S There are various types of state values in _n , such as size, aspect ratio, screen position coordinates, inclination, etc., the absolute difference is obtained by using a scoring system for the difference of each type of state value), and then The absolute difference of the state values of all the targets in the intersection Object _n-1 ∩ Object _n is weighted and summed, as the overall difference degree of the state, that is

Wherein DIFF(S _n ,S _n-1 ) represents the overall state difference between the scene video pictures Sn _-1 ,S _n , Represents the common target of the scene video picture Sn _-1 , _Sn The state value in the scene video picture Sn _-1 , Represents the common target of the scene video picture Sn _-1 , _Sn The state value in the scene video picture Sn, α _{i represents} the weighted summation coefficient. Different targets have different degrees of influence on the overall state difference between the analysis scenes, so the weighting coefficient is used to reflect the degree of influence. For the target closer to the center of the foreground shooting angle of view, the weighted summation coefficient has the largest weight. The weighted summation coefficient of the foreground non-central target is smaller than the weighted summation coefficient weight of the foreground central target, and the weighted summation coefficient weight of the background target is the smallest. If the overall state difference degree DIFF(S _n , Sn _-1 ) is greater than or equal to the second threshold, it is determined that the two scene video pictures Sn _-1 , Sn have different scene modes. If the overall state difference degree DIFF(S _n ) _n , Sn _-1 ) is lower than the second threshold, it is determined that Sn _-1 , _Sn have the same scene mode. In this way, the scene mode identification unit ₁₀₁ marks each scene video picture _{S 1} , S _{2 ......} The scene video pictures determined by the algorithm to belong to the same scene mode are marked with the same scene mode mark; if the scene video pictures judged to belong to different scene modes are marked with different scene mode marks.

The target area intelligent learning unit 102 obtains the scene video picture and its scene mode mark from the scene pattern recognition unit 101, and further, the target area intelligent learning unit 102 learns each type of The state characteristics of the moving objects with the greatest importance and relevance to the video capture purpose in the scene mode. As shown in FIG. 2, the target area intelligent learning unit 102 includes an important moving object extraction unit 102A, an important moving object classification training unit 102B, and a scene pattern classification training unit 102C.

First, the formation process of the learning samples used to identify the state characteristics of important moving objects in each scene mode is introduced. The target area intelligent learning unit 102 extracts a certain number of scene video pictures as sample pictures for each scene mode, and manually identifies the moving target area that should be locked and tracked in these sample pictures. Human visual perception has the characteristic of focusing on the important target that you want to pay attention to and ignoring the secondary target in the field of view. As a result, the user's sight is cyclically switched in the mode of pointing to the important target of the scene video screen - pointing to the surrounding secondary target - pointing to the important target again. The duration of time is significantly shorter. Therefore, the important moving object extraction unit 102A can determine the moving object belonging to the important object in each sample picture by photographing and analyzing the picture area to which the eye of the recognizer who manually identifies the sample picture is directed. Of course, the moving objects that belong to important objects in each scene video picture can also be manually marked by a human recognizer.

For the important objects manually identified from the sample images of each scene mode, denote as The important moving target classification training unit 102B of the target area intelligent learning unit 102 determines the state value of the important target in the sample picture, which is denoted as Stat(O _Key ), as described above, the state value Stat(O _Key ) is the important An array composed of the values of the size, aspect ratio, screen position coordinates, inclination and other parameters of the target O _Key in the sample screen. The state value Stat(O _Key ) array of all sample pictures of each scene mode is substituted into the SVM classification vector machine to perform training. After training, the target area intelligent learning unit 102 has obtained several trained important target recognition SVM classification vector machines. Classification identification.

Further, for all the sample pictures of each scene mode, the scene mode classification training unit 102C of the target area intelligent learning unit 102 extracts all the moving objects included in the sample pictures and the state value of each moving object, thereby obtaining {Stat(O _{1 )} . ),Stat(O ₂ ),…Stat(O _n-1 ),Stat(O _n )…Stat(O _k )} of the target status value list, which is composed of the status value of each moving target in the sample picture composed of arrays. Furthermore, the target area intelligent learning unit 102 substitutes the target state list of all sample pictures of each scene mode into the SVM classification vector machine to perform the training of identifying each scene mode. After training, the target area intelligent learning unit 102 has obtained several trained SVM classification vector machines for scene pattern recognition, and each SVM classification vector machine can use the state value of the moving object in the scene video picture to realize whether the picture belongs to this vector machine. The classification and recognition of the corresponding scene mode.

The tracking target extraction unit 103 is used for substituting the moving target state value list of the current scene video picture captured by the camera of the video surveillance platform into each scene pattern recognition SVM classification vector machine after being trained by the scene pattern classification training unit 102C, thereby judging the current scene. Which scene mode the video picture belongs to; and then, the tracking target extraction unit 103 continues to substitute the state value of each moving target in the scene video picture into the important target recognition corresponding to the scene mode trained by the important moving target classification training unit 102B SVM classification vector machine, thereby identifying whether each moving object belongs to an important object, when the classification result indicates that a moving object in the video picture of the current scene belongs to an important object, the tracking object extraction unit 103 provides the picture position coordinates of the moving object to the cloud Table drive unit 104 . Specifically, as shown in FIG. 3 , the tracking target extraction unit 103 obtains the current scene video picture and its scene mode mark from the scene pattern recognition unit 101, and then judges whether the current scene video picture and the previous scene video picture belong to the same A scene mode; if it does not belong to the same scene mode, the scene mode determination unit 103A of the tracking target extraction unit 103 obtains { Stat(O ₁ ), Stat(O ₂ ),...Stat(O _n-1 ), Stat(O _n )... Stat(O _k )} as the moving object state value list, the scene mode determination unit 103A assigns the current scene video picture The list of moving target state values is substituted into each scene pattern recognition SVM classification vector machine after training, and each scene pattern recognition SVM classification vector machine will output a binary result of "Yes" or "No", where the output of "Yes" The vector machine indicates that the current scene video picture belongs to the scene mode corresponding to the vector machine. Furthermore, the key target determination unit 103B of the tracking target extraction unit 103 obtains the status values Stat(O ₁ ), Stat(O ₂ ), . . . Stat(O _{n- 1} ), Stat(O _n )…Stat(O _k ), substitute the state value of each moving target into the important target recognition SVM classification vector machine in the scene mode to which it belongs, and the important target recognition SVM classification vector machine is the state of each input The value outputs a binary result of "Yes" or "No", wherein the output of "Yes" indicates that the moving object corresponding to the state value is an important object in the video picture of the current scene, then the important object determination unit 103B determines the picture position of the moving object. The coordinates are provided to the pan-tilt drive unit 104 .

The pan-tilt driving unit 104 obtains the screen position coordinates of the important objects in the video picture of the current scene from the tracking target extracting unit 103, so as to lock the important objects in the video picture of the current scene according to the picture position coordinates. The pan-tilt driving unit 104 analyzes the change of the position of the important object in each frame of the scene video picture, and when the important object deviates from the central area of the shooting field of view, extracts the movement direction and movement speed of the important object, and then extracts the movement direction and movement speed of the important object according to the movement direction and The speed predicts the estimated position deviation of the important target, and drives the rotating mechanical mechanism of the monitoring head in advance with reference to the prediction, so that the moving object returns to the central area of the shooting field of view as soon as possible in the subsequent scene video images.

As shown in FIG. 4 , the present invention further provides an artificial intelligence prediction and tracking method applied to a video surveillance PTZ. The method includes the following steps:

In the scene pattern recognition step, the target contained in each scene is extracted from the scene video picture captured by the camera of the video surveillance PTZ, and the state value of the target is quantified; it is compared whether two adjacent scene video pictures contain the same target, and Whether the ratio of the same target to all targets in each scene video picture is lower than the first threshold; if the percentage of the same target in at least one of the two scene video pictures is lower than the first threshold, it is determined that the two scene video pictures are respectively belong to different scene modes; if the percentage of the same target contained in the video images of two adjacent scenes to all targets is greater than or equal to the first threshold, then the respective state values of the same targets in the two frames are used to calculate the two scenes The overall state difference degree of all the same objects in the video picture; if the overall state difference degree is greater than or equal to the second threshold, it is determined that the two scene video pictures have different scene modes, and if the overall state difference degree is lower than the second threshold, it is determined that The two scene video images have the same scene mode.

Specifically, for all the scene video pictures S ₁ , S ₂ ......S _n-1 , S _n ...... S _m collected in a time interval, for the two adjacent scene video pictures S _n-1 , For _Sn , after target recognition, the set of all moving targets contained in the scene video picture Sn _-1 is Object _n-1 ={O ₁ ,O ₂ ......O _{n-1 ,} On ......O _k }, The set of all moving objects contained in the scene video picture _{Sn is Object n =} {O' ₁ , O' ₂ ... O' _l-1 , ' _l ... O' _m }, then determine the two scene video pictures _{Sn -1} , the same target contained in _Sn , take the same target in the two scene video images as the intersection of the above two sets It is judged that the number of objects in the intersection Object _n-1 ∩ Object _n accounts for the percentages of the number of objects in the set Object n _{- 1} and the set Object _n respectively. _Sn has different scene modes. If the percentages of the number of targets in the intersection Object _n-1 ∩ Object _n respectively account for the number of targets in the set Object _n-1 and the set Object _n , respectively, are greater than or equal to the first threshold, then further for the intersection For each object among them, analyze the absolute difference between the state value of the object in the scene video picture Sn _-1 and the state value of _Sn in the scene video picture (if each object is in the scene video picture Sn _-1 , S There are various types of state values in _n , such as size, aspect ratio, screen position coordinates, inclination, etc., the absolute difference is obtained by using a scoring system for the difference of each type of state value), and then The absolute difference of the state values of all the targets in the intersection Object _n-1 ∩ Object _n is weighted and summed, as the overall difference degree of the state, that is

Wherein DIFF(S _n ,S _n-1 ) represents the overall state difference between the scene video pictures Sn _-1 ,S _n , Represents the common target of the scene video picture Sn _-1 , _Sn The state value in the scene video picture Sn _-1 , Represents the common target of the scene video picture Sn _-1 , _Sn The state value in the scene video picture Sn, α _{i represents} the weighted summation coefficient. Different targets have different degrees of influence on the overall state difference between the analysis scenes, so the weighting coefficient is used to reflect the degree of influence. For the target closer to the center of the foreground shooting angle of view, the weighted summation coefficient has the largest weight. The weighted summation coefficient of the foreground non-central target is smaller than the weighted summation coefficient weight of the foreground central target, and the weighted summation coefficient weight of the background target is the smallest. If the overall state difference degree DIFF(S _n , Sn _-1 ) is greater than or equal to the second threshold, it is determined that the two scene video pictures Sn _-1 , Sn have different scene modes. If the overall state difference degree DIFF(S _n ) _n , Sn _-1 ) is lower than the second threshold, it is determined that Sn _-1 , _Sn have the same scene mode. In this way, the scene mode identification unit ₁₀₁ marks each scene video picture _{S 1} , S _{2 ......} The scene video pictures determined by the algorithm to belong to the same scene mode are marked with the same scene mode mark; if the scene video pictures judged to belong to different scene modes are marked with different scene mode marks.

The target area intelligent learning step is used to obtain the scene video picture and the scene mode mark, and then through the use of artificial intelligence learning method, through a certain number of samples, learn each scene mode and the purpose of video collection have the greatest importance And the state value feature of the important target of the correlation degree, use the state value feature to train the important target recognition SVM classification vector machine; and use the state value of the sample moving target to train the scene pattern recognition SVM classification vector machine. Specifically, a certain number of scene video pictures are extracted as sample pictures for each scene mode identified in the scene pattern recognition step, and the moving target area that should be locked and tracked in these sample pictures is manually identified, and each sample picture is determined. A moving object belonging to an important object in the picture; determine the state value of the important object in the sample picture, denoted as Stat(O _Key ), as described above, the state value Stat(O _Key ) is the important object O _Key in the sample It is an array composed of the numerical values of multiple parameters such as size, aspect ratio, screen position coordinates, and inclination in the screen. Substitute the state value Stat(O _Key ) array of all sample screens of each scene mode into the SVM classification vector machine for execution. Training; obtained several trained important target recognition SVM classification vector machines; for all sample pictures of each scene mode, extract all moving objects contained in the sample pictures and the state value of each moving object, and obtain {Stat(O ₁ ),Stat(O ₂ ),…Stat(O _n-1 ),Atat(O _n )…Stat(O _k )} target state value list, the list is composed of the state of each moving object in the sample picture The value array is composed together; the target state list of all sample pictures of each scene pattern is substituted into the SVM classification vector machine to perform the training of identifying each scene pattern, and several trained scene pattern recognition SVM classification vector machines are obtained.

The tracking target extraction step is used for substituting the moving target state value list of the current scene video picture into each scene pattern recognition SVM classification vector machine after training, thereby judging which scene pattern the current scene video picture belongs to; The state value of each moving object in the video picture is substituted into the important object recognition SVM classification vector machine corresponding to the trained scene mode, so as to identify whether each moving object belongs to an important object, when the classification result indicates that a movement in the current scene video picture If the target is an important target, the screen position coordinates of the moving target are applied to the gimbal drive.

In the step of driving the gimbal, the screen position coordinates of the important target in the video picture of the current scene are obtained, so as to lock the important target in the video screen of the current scene according to the position coordinates of the screen, and the change of the position of the important target in each frame of the video screen of the scene is analyzed. , when the important target deviates from the central area of the shooting field of view, extract the movement direction and movement speed of the important target, and then predict the expected position deviation of the important target according to the movement direction and speed, and drive the rotation of the monitoring platform in advance with reference to the prediction The mechanical mechanism enables the moving object to return to the central area of the shooting field of view as soon as possible in the subsequent scene video images.

The above description is only a preferred embodiment of the present application and an illustration of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution formed by the specific combination of the above technical features, and should also cover the above technical features or Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above-mentioned features with the technical features disclosed in this application (but not limited to) with similar functions.

Claims (4)

1. a video surveillance PTZ adopting artificial intelligence prediction and tracking, is characterized in that, comprising: scene pattern recognition unit, target area intelligent learning unit, tracking target extraction unit, PTZ drive unit; wherein The scene pattern recognition unit is used to extract the target contained in each scene from the scene video picture captured by the camera of the video surveillance platform, and quantify the target state to generate a state value; according to the state value of the target, determine two adjacent images. Whether the scene video pictures are of the same scene mode, the scene video pictures of the same scene mode are marked with the same type of scene mode mark; wherein, the scene mode recognition unit compares whether two adjacent scene video pictures contain the same target, and whether the same Whether the ratio of the target to all targets in each scene video picture is lower than the first threshold; if the percentage of the same target in at least one of the two scene video pictures is lower than the first threshold, it is determined that the two scene video pictures belong to Different scene modes; if the percentage of the same target contained in the two adjacent scene video images to the total target is greater than or equal to the first threshold, then the two scene videos are calculated by using the respective state values of the same target in the two frames. The overall state difference degree of all the same objects in the picture; if the overall state difference degree is greater than or equal to the second threshold, it is determined that the two scene video pictures have different scene modes, and if the overall state difference degree is lower than the second threshold, it is determined that this The two scene video images have the same scene mode; The target area intelligent learning unit is used to learn the state value characteristics of important moving objects in each scene mode through a certain number of samples; wherein, the target area intelligent learning unit extracts a certain number of scene video images for each scene mode as Sample pictures, and manually identify the moving target areas that should be locked and tracked in these sample pictures, and determine the moving objects that belong to important objects in each sample picture; determine the state value of the important object in the sample pictures; Perform training on the state values of important objects in all sample pictures of the scene mode, and obtain the trained important object recognition SVM classification vector machine; for all sample pictures of each scene mode, extract all moving objects contained in the sample pictures and each The state value of the moving target, obtain the target state value list, train the target state list of all sample pictures of each scene mode, and obtain the trained scene mode recognition SVM classification vector machine; The tracking target extraction unit is used for the learning result of the target area intelligent learning unit. According to the state value of each moving target identified from the current scene video screen, it is judged whether the moving target belongs to an important target. When the result indicates that one of the current scene video screen If the moving target belongs to an important target, then the picture position coordinates of the moving target are provided to the PTZ drive unit; wherein, the tracking target extraction unit is used to substitute the moving target state value list of the video picture of the current scene into each scene after training Pattern recognition SVM classification vector machine, so as to determine which scene mode the current scene video picture belongs to; and then, substitute the state value of each moving object in the scene video picture into the important target recognition SVM classification vector corresponding to the trained scene mode machine, so as to identify whether each moving target belongs to an important target, when the classification result indicates that a moving target in the video picture of the current scene belongs to an important target; The pan-tilt driving unit obtains the screen position coordinates of the important target in the video picture of the current scene from the tracking target extraction unit, so as to lock the important target in the video screen of the current scene according to the screen position coordinates, and perform tracking shooting. 2. The video surveillance pan-tilt according to claim 1, wherein the pan-tilt driving unit analyzes the change of the position of the important target in each frame of scene video picture, and when the important target deviates from the central area of the shooting field of view , extract the movement direction and movement speed of the important target, and then predict the expected position deviation of the important target according to the movement direction and speed, and drive and monitor the rotating mechanical mechanism of the gimbal in advance with reference to the prediction. 3. an artificial intelligence prediction tracking method applied to a video surveillance PTZ, is characterized in that, comprising: The scene pattern recognition step is used to extract the target contained in each scene from the scene video picture, and quantify the target state to generate a state value; according to the state value of the target, determine whether two adjacent scene video pictures are the same scene pattern , the scene video pictures of the same scene mode are marked with the same type of scene mode mark; wherein, in the scene pattern recognition step, it is compared whether two adjacent scene video pictures contain the same target, and the same target accounts for each scene video. Whether the proportion of all objects in the picture is lower than the first threshold; if the percentage of the same object in at least one of the two scene video pictures is lower than the first threshold, it is determined that the two scene video pictures belong to different scene modes; if The percentages of the same objects contained in the two adjacent scene video pictures to all the objects are greater than or equal to the first threshold, and then the respective state values of the same objects in the two pictures are used to calculate the percentage of all the same objects in the two scene video pictures. The overall state difference degree; if the overall state difference degree is greater than or equal to the second threshold, it is determined that the two scene video images have different scene modes; if the overall state difference degree is lower than the second threshold value, it is determined that the two scene video images have the same scene mode; The target area intelligent learning step is used to learn the state value characteristics of important moving objects in each scene mode through a certain number of samples; wherein, in the target area intelligent learning step, a certain number of scenes are extracted for each scene mode. The video picture is used as a sample picture, and the moving target area that should be locked and tracked in these sample pictures is manually identified, and the moving object belonging to the important target in each sample picture is determined; the state value of the important target in the sample picture is determined; Perform training on the state values of important objects in all sample pictures of each scene mode to obtain a trained important object recognition SVM classification vector machine; for all sample pictures of each scene mode, extract all moving objects contained in the sample pictures And the state value of each moving target, obtain the target state value list, carry out training the target state list of all the sample pictures of each kind of scene pattern, obtain the scene pattern recognition SVM classification vector machine trained; The tracking target extraction step is used for the learning results of the intelligent learning unit in the target area. According to the state value of each moving target identified from the current scene video screen, it is judged whether the moving target belongs to an important target. When the result indicates that one of the current scene video images If the moving target belongs to an important target, then the picture position coordinates of the moving target are provided to the pan-tilt drive unit; wherein, in the tracking target extraction step, the moving target state value list of the video picture of the current scene is substituted into the training Each scene pattern recognition SVM classification vector machine of , so as to determine which scene pattern the current scene video picture belongs to; and then, substitute the state value of each moving target in the scene video picture into the important target recognition corresponding to the trained scene pattern SVM classification vector machine, so as to identify whether each moving object belongs to an important object, when the classification result indicates that a moving object in the current scene video picture belongs to an important object; In the pan-tilt driving step, the screen position coordinates of the important targets in the video picture of the current scene are obtained from the tracking target extraction unit, so as to lock the important targets in the video screen of the current scene according to the screen position coordinates for tracking shooting. 4. The artificial intelligence prediction and tracking method according to claim 3, wherein in the step of driving the pan/tilt head, the change of the position of the important target in each frame of scene video picture is analyzed, and when the important target deviates from the shooting field of view When the central area of the important target is extracted, the movement direction and movement speed of the important target are extracted, and then the expected position deviation of the important target is predicted according to the movement direction and speed, and the rotating mechanical mechanism of the monitoring head is driven and monitored in advance with reference to the prediction.