CN111967345A - Method for judging shielding state of camera in real time - Google Patents

Abstract

The invention discloses a method for determining the occlusion state of a camera in real time. The RGB image is converted into chromaticity space to obtain a grayscale image, and then the feature points in the grayscale image are extracted to obtain a set of feature points of the grayscale image; the grayscale image is divided into 4 areas and numbered, and each area is calculated separately If the number of feature points in any of the 4 areas is less than the preset number threshold, the result of the determination that the camera is blocked will be output; if the number of feature points in all the 4 areas is greater than the preset number threshold Set the number threshold, and output the judgment result that the camera is not blocked. The invention has the advantages of simple, efficient and fast method, only one frame of image is needed to accurately determine the occlusion state of the camera, and is suitable for scenarios and embedded devices that require high real-time performance.

Description

A method for real-time determination of camera occlusion state

technical field

The invention relates to the field of computer vision, in particular to a method for determining the occlusion state of a camera in real time.

Background technique

In recent years, due to the rapid development of vision theory and computer science and technology, more and more scholars have devoted themselves to the research in the field of computer vision. Computer vision has shown good development prospects in the application of automatic driving and assisted driving, and has attracted much attention. At present, most of the automatic driving and assisted driving use the camera to obtain the image information in front of the vehicle, but the camera may be partially blocked by objects such as mud, which affects the normal operation of the driving system, or may be accidentally blocked for other special reasons. This situation and handling it will cause very serious security problems.

The Chinese patent (publication number CN103139547.B) titled "Method for Determining the Occlusion State of a Camera Lens Based on Video Image Signals" adopts the occlusion detection method: extracting the background of the image by using the frame difference method, using the background subtraction to obtain the foreground, and Binarize the foreground and divide it into multiple foreground detection units; remove the foreground detection units whose pixel area is less than the threshold, and further screen out candidate occlusion areas; track the pixels of subsequent frames of the candidate occlusion areas, if the grayscale information of the pixels and If the change of the texture information is less than the threshold, it is determined as a suspected occlusion area; the subsequent frames of the suspected occlusion area are tracked and counted, and if it stably exists in the video frame exceeding the preset time threshold, it is determined that the camera is occluded. Although this method can determine the occlusion area on the camera for long-term processing, it needs to process continuous frames for a period of time, and the speed is slow, which is not suitable for application scenarios with high real-time requirements. And it is not recognizable for mobile occluders, such as malicious human operations.

The Chinese patent (publication number CN200710145468.X) titled "A method for detecting video occlusion based on network video surveillance" discloses a occlusion detection method, which needs to determine a reference frame first, and then detect from the motion area. occlude. This method has a certain effect, but the limitation lies in the selection of reference frames. Only qualified reference frames can be used for occlusion detection, and this method also needs to process consecutive frames for a period of time, which is slow and practical.

To sum up, how to provide a method for judging the occlusion state of a camera in application scenarios with high real-time requirements with high occlusion determination accuracy and fast determination speed is a current problem.

SUMMARY OF THE INVENTION

The present invention provides a method for judging the occlusion state of a camera in real time, so as to solve the problem of low accuracy and slow speed of the existing judging method of camera occlusion, which leads to low security of the system.

A method for determining the occlusion state of a camera in real time, comprising the following steps:

Step 1, read a frame of RGB image captured by the camera in real time;

Step 2, the RGB image is first scaled to the target size, and then corrected and de-distorted;

Step 3: Perform chromaticity space conversion on the corrected and dedistorted RGB image obtained in step 2 to obtain a grayscale image, and then extract feature points in the grayscale image to obtain a feature point set of the grayscale image ;

Step 4, the grayscale image is divided into 4 areas and numbered, and the number of feature points in each area is calculated respectively;

Step 5: If the number of feature points in any of the 4 areas is less than the preset number threshold t, output the result of the determination that the camera is blocked; if the number of feature points in all the 4 areas is If the numbers are all greater than the preset number threshold t, the determination result that the camera is not blocked is output.

Further, in an implementation manner, the step 1 includes: before using the camera, calibrating the camera to obtain an internal parameter matrix and a distortion coefficient of the camera;

The step 2 includes: correcting and undistorting the RGB image scaled to the target size according to the internal parameter matrix and the distortion coefficient of the camera, combined with the opencv_undistort algorithm.

Further, in an implementation manner, the step 3 includes extracting image feature points based on a corner detection method, and the corner detection method includes:

If the difference between the gray value of a certain pixel of the grayscale image and the gray value of a certain number of pixels in the area around the pixel is greater than or equal to the preset difference threshold t _p , then determine The pixel points are corner points, that is, the pixel points are image feature points.

Further, in an implementation manner, the extraction of image feature points based on the fast corner detection method includes:

Step 3-1, select a pixel point P from the grayscale image, and the grayscale value of the pixel point _P is IP;

Step 3-2, with the pixel point P as the center and 3 pixels as the radius, set a discrete Bresenham circle, and there are 16 pixels on the discrete Bresenham circle;

Step 3-3, if there are n continuous pixel points on the discretized Bresenham circle, the absolute value of the difference between the gray value of the n continuous pixel points and the gray value of the center of the circle is greater than the preset difference. value threshold t _p , namely:

Among them, I _i is the gray value of the ith pixel in n consecutive pixels, i ₌ 1,2...,n represents the serial number of the pixel, IP is the gray value of the center of the circle, and t _p is the preset difference threshold;

Then, the center of the discrete Bresenham circle is extracted as the image feature point based on the fast corner detection method.

Further, in an implementation manner, the step 4 includes: equally dividing the grayscale image into 4 regions, and the 4 regions are respectively located at the upper left, upper right, lower left and lower right positions of the grayscale image, The specific position of each area is represented by the coordinates, width and height of the upper left corner of each area, namely:

2号区域

3号区域

和4号区域

其中，w为灰度图像的宽，h为灰度图像的高。Area 1

Area 2

Area 3

and area 4

where w is the width of the grayscale image, and h is the height of the grayscale image.

Further, in an implementation manner, the step 4 further includes:

Step 4-1, after obtaining the feature point set of the grayscale image through the step 3, extract the feature point information of the grayscale image, and the feature point information includes: the coordinates of the feature point on the grayscale image;

Step 4-2, according to the coordinates of the feature points on the grayscale image, determine the number r of each of the feature points located in the grayscale image, and count the number of feature points in the four regions respectively:

并且

则确定r＝1，所述特征点P_j位于1号区域内，所述1号区域的特征点个数加1；like

and

Then it is determined that r=1, the feature point P _j is located in the No. 1 area, and the number of feature points in the No. 1 area is increased by 1;

并且

则确定r＝3，所述特征点P_j位于3号区域内，所述3号区域的特征点个数加1；like

and

Then it is determined that r=3, the feature point P _j is located in the No. 3 area, and the number of feature points in the No. 3 area is increased by 1;

并且

则确定r＝2，所述特征点P_j位于2号区域内，所述2号区域的特征点个数加1；like

and

Then it is determined that r=2, the feature point P _j is located in the No. 2 area, and the number of feature points in the No. 2 area is increased by 1;

并且

则确定r＝4，所述特征点P_j位于4号区域内，所述4号区域的特征点个数加1；like

and

Then it is determined that r=4, the feature point P _j is located in the No. 4 area, and the number of feature points in the No. 4 area is increased by 1;

Among them, (x _j , y _j ) are the coordinates of the feature point P _j on the grayscale image.

Further, in an implementation manner, the preset number threshold t needs to be determined before the step 5, including:

Step 5-1, shooting a video when the camera is not blocked;

Step 5-2, according to the video when the camera is not blocked, use the method of random sampling to determine the set of sampling image frame numbers to be used, namely:

S _k =S _k-1 ∪{y _k }

k=1,...,γ

F为摄像头未被遮挡情况下拍摄的视频的帧数，θ为随机变量，在[0,1)区间上满足均匀分布，即θ_k是第k次采样时随机生成的在区间[0,1)上的实数，y_k是第k次采样得到的图像帧序号，γ为采样次数，S_γ即为最终得到的采样结果，即需要使用的采样图像帧序号集合；Among them, _Sk is the set of image frame serial numbers after the kth sampling, set

F is the number of frames of the video shot when the camera is not blocked, θ is a random variable, which satisfies a uniform distribution in the [0,1) interval, that is, θ _k is randomly generated during the kth sampling in the interval [0,1 ), y _k is the image frame serial number obtained by the kth sampling, γ is the number of sampling times, and S _γ is the final sampling result, that is, the set of sampling image frame serial numbers to be used;

Step 5-3, according to the sampling order, read each frame of the sampled image set, and perform steps 1 to 4 for each frame in turn to obtain 4 regions in the grayscale image corresponding to each frame The number of feature points;

Step 5-4, sort the number of feature points of all regions in all the grayscale images from small to large to obtain a sequence a of the number of feature points, a=(a ₁ , a ₂ ,...,a _4×γ ), where a _l represents the number of the lth smallest feature points;

Step 5-5, calculate the preset number threshold t based on the box plot analysis method:

t=Q ₁ -1.5×IQR

IQR=|Q ₁ -Q ₂ |

Among them, t represents the preset number threshold, that is, the lower bound of the normal value of the box plot analysis method, Q ₁ represents the lower quartile, Q ₂ represents the upper quartile, and IQR is the interquartile range, representing the upper quartile The absolute value of the difference between the quantile Q ₂ and the lower quartile Q ₁ ;

The calculation method of the lower quartile Q ₁ and the upper quartile Q ₂ is:

d _o = μ _o -c _o

μ _o =II(o=1)×0.25×(4×γ+1)+II(o=2)×0.75×(4×γ+1)

o=1,2

Among them, μ _o represents the position of the lower quartile Q ₁ or the upper quartile Q ₂ in the feature point number sequence a=(a ₁ ,a ₂ ,...,a _4×γ ) in step 5-4, When o=1, μ ₁ represents the position of the lower quartile Q ₁ in the feature point number sequence a, and when o=2, μ ₂ represents the upper quartile Q ₂ in the feature point number sequence a. position, II is an indicator function, used to distinguish whether the current calculation is the upper quartile Q ₂ or the lower quartile Q ₁ , c _o is the integer part of μ _o , and _do is the fractional part of μ _o .

As can be seen from the above technical solutions, an embodiment of the present invention provides a method for determining the occlusion state of a camera in real time. The method includes: step 1, reading a frame of RGB image captured by the camera in real time; step 2, zooming the RGB image first to the target size, then correct and dedistort; step 3, perform chromaticity space conversion on the corrected and dedistorted RGB image obtained in step 2 to obtain a grayscale image, and then extract the feature points in the grayscale image to obtain The feature point set of the grayscale image; Step 4, divide the grayscale image into 4 regions and number them, respectively calculate the number of feature points in each region; Step 5, if any one of the 4 regions If the number of feature points in the area is less than the preset number threshold t, output the judgment result that the camera is blocked; if the number of feature points in all areas in the four areas is greater than the preset number threshold t, Then output the determination result that the camera is not blocked.

In the prior art, the camera occlusion determination method has low precision and slow speed. The aforementioned method or device can be applied to any device equipped with a camera, and only a single camera can realize occlusion detection; this method extracts feature points based on corner detection, and the feature point extraction speed is fast; this method uses a single frame of image, namely Detection can be achieved without relying on continuous video image frame information or pre-stored information; by dividing images into regions, the method can be well adapted to dynamic scenes, full occlusion and partial occlusion. In summary, this method is a single-camera occlusion detection method suitable for application scenarios with high real-time requirements, and has the advantages of high speed, high accuracy, and no dependence on consecutive frames.

Description of drawings

In order to illustrate the technical solutions of the present invention more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative work, the Additional drawings can be obtained from these drawings.

1 is a schematic work flow diagram of a method for real-time determination of a camera occlusion state provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of the principle of a method for determining the occlusion state of a camera in real time according to an embodiment of the present invention;

3 is a schematic diagram of an image partition in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

4a is a schematic diagram of the first effect of feature extraction in a method for determining a camera occlusion state in real time according to an embodiment of the present invention;

4b is a schematic diagram of a second effect of feature extraction in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

4c is a schematic diagram of a third effect of feature extraction in a method for determining a camera occlusion state in real time provided by an embodiment of the present invention;

5a is a schematic diagram of a first effect of occlusion detection in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

5b is a schematic diagram of a second effect of occlusion detection in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

5c is a schematic diagram of a third effect of occlusion detection in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

5d is a schematic diagram of the fourth effect of occlusion detection in a method for determining a camera occlusion state in real time provided by an embodiment of the present invention;

5e is a schematic diagram of a fifth effect of occlusion detection in a method for determining a camera occlusion state in real time provided in part by an embodiment of the present invention;

FIG. 5f is a schematic diagram of a sixth effect of occlusion detection in a method for determining the occlusion state of a camera in real time provided by an embodiment of the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

The embodiment of the present invention discloses a method for judging the occlusion state of a camera in real time. The method is applied to application scenarios with high real-time requirements, and only a single camera can realize occlusion detection. Because the method extracts feature points based on the fast corner detection method, the feature point extraction speed is fast; in particular, the method can realize detection by using a single frame image, and does not rely on continuous video image frame information, nor does it rely on pre-stored information; By dividing the image into regions, the method can well adapt to dynamic scenes, full occlusion and partial occlusion. Therefore, this method is a single-camera occlusion detection method suitable for application scenarios with high real-time requirements, and has the advantages of high speed, high accuracy, and independent of continuous frames.

1 is a schematic flowchart of occlusion detection according to the present invention, a method for determining the state of a camera in real time based on image feature points, including 5 steps:

Step 1, read a frame of RGB image captured by the camera in real time;

Step 2, the RGB image is first scaled to the target size, and then corrected and de-distorted;

Step 3: Perform chromaticity space conversion on the corrected and dedistorted RGB image obtained in step 2 to obtain a grayscale image, and then extract feature points in the grayscale image to obtain a feature point set of the grayscale image ;

Step 4, the grayscale image is divided into 4 areas and numbered, and the number of feature points in each area is calculated respectively;

In this step, the division of regions is to better detect local occlusions. However, too many divided areas will cause the area of each area to be too small. If the number of feature points in some small areas in the image is too small, it will be mistakenly identified as occlusion. Therefore, the four regions can ensure the realization of partial occlusion detection, and can minimize the occurrence of false positive occlusions.

Step 5: If the number of feature points in any of the 4 areas is less than the preset number threshold t, output the result of the determination that the camera is blocked; if the number of feature points in all the 4 areas is If the numbers are all greater than the preset number threshold t, the determination result that the camera is not blocked is output.

In the method for determining the occlusion state of a camera in real time according to this embodiment, the step 1 includes: before using the camera, calibrating the camera to obtain an internal parameter matrix and a distortion coefficient of the camera;

The step 2 includes: correcting and undistorting the RGB image scaled to the target size according to the internal parameter matrix and the distortion coefficient of the camera, combined with the opencv_undistort algorithm.

As shown in FIG. 2 , in the method for determining the occlusion state of a camera in real time according to this embodiment, the step 3 includes extracting image feature points based on a fast corner detection method, and the fast corner detection method includes:

If the difference between the gray value of a certain pixel of the grayscale image and the gray value of a certain number of pixels in the area around the pixel is greater than or equal to the preset difference threshold t _p , then determine The pixel points are corner points, that is, the pixel points are image feature points.

In this embodiment, the image feature points are specifically image feature points based on corner detection. The corner point is a pixel point containing key information, and the feature point is an extension of the corner point concept. Here, the detected corner point is used as the feature point in the image.

The basic idea of detecting feature points based on corner points is: if a pixel point and a certain number of pixel points in the surrounding area are in different image areas, the pixel point may be a corner point. In particular, for a grayscale image, if the grayscale value of the point is larger or smaller than the grayscale value of a certain number of pixels in its surrounding area, the point may be a corner point.

In the method for determining the occlusion state of a camera in real time according to this embodiment, the extraction of image feature points based on the corner detection method includes:

Step 3-1, select a pixel point P from the grayscale image, and the grayscale value of the pixel point _P is IP;

Step 3-2, with the pixel point P as the center and 3 pixels as the radius to set a discrete Bresenham circle, and there are 16 pixels on the discrete Bresenham circle;

Step 3-3, if there are n continuous pixel points on the discretized Bresenham circle, the absolute value of the difference between the gray value of the n continuous pixel points and the center gray value is greater than the preset difference. value threshold t _p , namely:

Among them, I _i is the gray value of the ith pixel in n consecutive pixels, i ₌ 1,2...,n represents the serial number of the pixel, IP is the gray value of the center of the circle, and t _p is the preset difference threshold;

上述Bresenham圆包含16个像素点，具体的，本实施例中，的值可以设置为12或9，n通常取9较为合适。大多数情况下，为避免检测到的特征点为伪特征点，需要将预设差值阈值t_p设置为较大的数值，所述预设差值阈值t_p一般可以设置为50。Then, the center of the discrete Bresenham circle is extracted as the image feature point based on the corner detection method. In general, if the Bresenham circle contains N pixels, it is necessary to satisfy

The above Bresenham circle includes 16 pixels. Specifically, in this embodiment, the value of , can be set to 12 or 9, and n is usually set to 9. In most cases, in order to prevent the detected feature points from being false feature points, the preset difference threshold t _p needs to be set to a relatively large value, and the preset difference threshold t _p can generally be set to 50.

As shown in FIG. 3 , in the method for determining the occlusion state of a camera in real time according to this embodiment, the step 4 includes: dividing the grayscale image into 4 regions, and the 4 regions are respectively It is located in the upper left, upper right, lower left and lower right positions of the grayscale image, and the specific position of each region is represented by the coordinates, width and height of the upper left corner of each said region, namely:

2号区域

3号区域

和4号区域

其中，w为灰度图像的宽，h为灰度图像的高。Area 1

Area 2

Area 3

and area 4

where w is the width of the grayscale image, and h is the height of the grayscale image.

In the method for determining the occlusion state of a camera in real time according to this embodiment, the step 4 further includes:

Step 4-1, after obtaining the feature point set of the grayscale image through the step 3, extract the feature point information of the grayscale image, and the feature point information includes: the coordinates of the feature point on the grayscale image;

Step 4-2, according to the coordinates of the feature points on the grayscale image, determine the number r of each of the feature points located in the grayscale image, and count the number of feature points in the four regions respectively:

并且

则确定r＝1，所述特征点P_j位于1号区域内，所述1号区域的特征点个数加1；like

and

Then it is determined that r=1, the feature point P _j is located in the No. 1 area, and the number of feature points in the No. 1 area is increased by 1;

并且

则确定r＝3，所述特征点P_j位于3号区域内，所述3号区域的特征点个数加1；like

and

Then it is determined that r=3, the feature point P _j is located in the No. 3 area, and the number of feature points in the No. 3 area is increased by 1;

并且

则确定r＝2，所述特征点P_j位于2号区域内，所述2号区域的特征点个数加1；like

and

Then it is determined that r=2, the feature point P _j is located in the No. 2 area, and the number of feature points in the No. 2 area is increased by 1;

并且

则确定r＝4，所述特征点P_j位于4号区域内，所述4号区域的特征点个数加1；like

and

Then it is determined that r=4, the feature point P _j is located in the No. 4 area, and the number of feature points in the No. 4 area is increased by 1;

Among them, (x _j , y _j ) are the coordinates of the feature point P _j on the grayscale image.

In the method for judging the occlusion state of a camera in real time described in this embodiment, since the number threshold is a key factor for judging occlusion, and the number of feature points in the image has a certain relationship with the environment, in order to ensure that the method has a higher For the occlusion detection capability, the number threshold needs to be preset according to the actual environment. Therefore, before step 5, the preset number threshold t needs to be determined, including:

Step 5-1, shooting a video when the camera is not blocked;

Step 5-2, according to the video under the condition that the camera is not blocked, in order to reduce the chance, the video duration should be as long as possible. In this embodiment, a video of more than 10 minutes is selected, and the random sampling method is used to determine the sampling to be used. The set of image frame serial numbers, namely:

S _k =S _k-1 ∪{y _k }

k=1,...,γ

F为摄像头未被遮挡情况下拍摄的视频的帧数，θ为随机变量，在[0,1)区间上满足均匀分布，即θ_k是第k次采样时随机生成的在区间[0,1)上的实数，y_k是第k次采样得到的图像帧序号，γ为采样次数，S_γ即为最终得到的采样结果，即需要使用的采样图像帧序号集合；Among them, _Sk is the set of image frame serial numbers after the kth sampling, set

F is the number of frames of the video shot when the camera is not blocked, θ is a random variable, which satisfies a uniform distribution in the [0,1) interval, that is, θ _k is randomly generated during the kth sampling in the interval [0,1 ), y _k is the image frame serial number obtained by the kth sampling, γ is the number of sampling times, and S _γ is the final sampling result, that is, the set of sampling image frame serial numbers to be used;

Step 5-3, according to the sampling order, read each frame of the sampled image set, and perform steps 1 to 4 for each frame in turn to obtain 4 regions in the grayscale image corresponding to each frame The number of feature points;

Step 5-4, sort the number of feature points of all regions in all the grayscale images from small to large to obtain a sequence a of the number of feature points, a=(a ₁ , a ₂ ,...,a _4×γ ), where a _l represents the number of the lth smallest feature points;

Step 5-5, calculate the preset number threshold t based on the box plot analysis method:

t=Q ₁ -1.5×IQR

IQR=|Q ₁ -Q ₂ |

Among them, t represents the preset number threshold, that is, the lower bound of the normal value of the box plot analysis method, Q ₁ represents the lower quartile, Q ₂ represents the upper quartile, and IQR is the interquartile range, representing the upper quartile The absolute value of the difference between the quantile Q ₂ and the lower quartile Q ₁ ;

The calculation method of the lower quartile Q ₁ and the upper quartile Q ₂ is:

d _o = μ _o -c _o

μ _o =II(o=1)×0.25×(4×γ+1)+II(o=2)×0.75×(4×γ+1)

o=1,2

Among them, μ _o represents the position of the lower quartile Q ₁ or the upper quartile Q ₂ in the feature point number sequence a=(a ₁ ,a ₂ ,...,a _4×γ ) in step 5-4, When o=1, μ ₁ represents the position of the lower quartile Q ₁ in the feature point number sequence a, and when o=2, μ ₂ represents the upper quartile Q ₂ in the feature point number sequence a. position, II is an indicator function, used to distinguish whether the current calculation is the upper quartile Q ₂ or the lower quartile Q ₁ , c _o is the integer part of μ _o , and _do is the fractional part of μ _o .

Boxplot analysis is often used to detect outliers. When the observed value is greater than the upper bound of the normal value or smaller than the lower bound of the normal value, the observed value can be considered an outlier. Here, based on the box plot analysis method, the lower bound of the normal value can be regarded as the number threshold. When the number of feature points in the area is less than the number threshold, it is considered that an abnormality has occurred, that is, it is determined as occlusion.

In order to verify the effectiveness of the method, an example verification is carried out on the actual collected video. These include completely occluded, partially occluded, and unoccluded images, and occlusion detection is performed on each frame of images in these videos to determine the occlusion status of the camera in real time.

Taking this actual collected video as an example, for each frame of image in the video, follow the steps below to determine the occlusion status of the camera:

Step 1, read a frame of RGB image captured by the camera in real time;

Step 2, the RGB image is first scaled to the target size, and then corrected and de-distorted;

Step 3: Perform chromaticity space conversion on the corrected and dedistorted RGB image obtained in step 2 to obtain a grayscale image, and then extract feature points in the grayscale image to obtain a feature point set of the grayscale image ;

Step 4, the grayscale image is divided into 4 areas and numbered, and the number of feature points in each area is calculated respectively;

Step 5: If the number of feature points in any of the 4 areas is less than the preset number threshold t, output the result of the determination that the camera is blocked; if the number of feature points in all the 4 areas is If the numbers are all greater than the preset number threshold t, the determination result that the camera is not blocked is output.

In Figures 4a-4c, the effect of image feature point extraction is shown, wherein Figure 4a is a schematic diagram of the effect of full occlusion, Figure 4b is a schematic diagram of the effect of partial occlusion, and Figure 4c is a schematic diagram of the effect of no occlusion.

5a to 5f show the occlusion detection effect of the present invention for the camera. Here, for the convenience of description, when the camera is determined to be blocked, the frame number to which the image belongs and the words blocked are output to the figure. Figures 5a and 5b are schematic diagrams of the effect of full occlusion, and Figures 5c and 5d are schematic diagrams of the effect of partial occlusion. It can be seen that the words "Camera is obstructed!" are displayed in Figure 5a, Figure 5b, Figure 5c and Figure 5d; Figure 5e And Figure 5f is a schematic diagram of the unobstructed effect. It can be seen that the words that the camera is blocked are not displayed in Figure 5e and Figure 5f. The verification on the data can show that the accuracy and speed of the method for determining the occlusion state of the camera in real time provided by the present invention both show satisfactory results.

As can be seen from the above technical solutions, an embodiment of the present invention provides a method for determining the occlusion state of a camera in real time. The method includes: step 1, reading a frame of RGB image captured by the camera in real time; step 2, zooming the RGB image first to the target size, then correct and dedistort; step 3, perform chromaticity space conversion on the corrected and dedistorted RGB image obtained in step 2 to obtain a grayscale image, and then extract the feature points in the grayscale image to obtain The feature point set of the grayscale image; Step 4, divide the grayscale image into 4 regions and number them, respectively calculate the number of feature points in each region; Step 5, if any one of the 4 regions If the number of feature points in the area is less than the preset number threshold t, output the judgment result that the camera is blocked; if the number of feature points in all areas in the four areas is greater than the preset number threshold t, Then output the determination result that the camera is not blocked.

In the prior art, the camera occlusion determination method has low precision and slow speed. The aforementioned method or device can be applied to any device equipped with a camera, and only a single camera can realize occlusion detection; this method extracts feature points based on corner detection, and the feature point extraction speed is fast; this method uses a single frame of image, namely Detection can be achieved without relying on continuous video image frame information or pre-stored information; by dividing images into regions, the method can be well adapted to dynamic scenes, full occlusion and partial occlusion. In summary, this method is a single-camera occlusion detection method suitable for application scenarios with high real-time requirements, and has the advantages of high speed, high accuracy, and no dependence on consecutive frames.

In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium can store a program, and when the program is executed, the program can include the methods in the various embodiments of the method for determining a camera occlusion state in real time provided by the present invention. some or all of the steps. The storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, ROM for short) or a random access memory (English: random access memory, RAM for short).

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products may be stored in a storage medium, such as ROM/RAM , magnetic disk, optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

It is sufficient to refer to each other for the same and similar parts among the various embodiments in this specification. The embodiments of the present invention described above do not limit the protection scope of the present invention.

Claims (7)

1. A method for judging the shielding state of a camera in real time is characterized by comprising the following steps:

step 1, reading a frame of RGB image shot by a camera in real time;

step 2, zooming the RGB image to a target size, and then correcting and removing distortion;

step 3, carrying out chromaticity space conversion on the corrected and undistorted RGB image obtained in the step 2 to obtain a gray level image, and extracting feature points in the gray level image to obtain a feature point set of the gray level image;

step 4, dividing the gray level image into 4 areas, numbering the areas, and respectively calculating the number of characteristic points of each area;

step 5, if the number of the characteristic points in any one of the 4 areas is smaller than a preset number threshold t, outputting a judgment result of the shielding of the camera; and if the number of the feature points of all the 4 regions is greater than a preset number threshold t, outputting a judgment result that the camera is not shielded.

2. The method for judging the shielding state of the camera in real time according to claim 1, wherein the step 1 comprises: before the camera is used, calibrating the camera to obtain an internal reference matrix and a distortion coefficient of the camera;

the step 2 comprises the following steps: and correcting and de-distorting the RGB image after being scaled to the target size by combining an opencv _ undistort algorithm according to the internal reference matrix and the distortion coefficient of the camera.

3. The method according to claim 1, wherein the step 3 comprises extracting image feature points based on a fast corner detection method, and the fast corner detection method comprises:

if the difference value between the gray value of a certain pixel point of the gray image and the gray values of a certain number of pixel points in the surrounding field of the pixel point is larger than or equal to a preset difference threshold t_pAnd determining the pixel points as angular points, namely the pixel points are image feature points.

4. The method according to claim 3, wherein the extracting of the image feature points based on the fast corner detection method comprises:

step 3-1, selecting pixel points P from the gray level image, wherein the gray level value of the pixel points P is I_P；

Step 3-2, setting a discretized Bresenham circle by taking the pixel point P as a circle center and 3 pixels as a radius, wherein the discretized Bresenham circle is provided with 16 pixel points;

step 3-3, if n continuous pixels exist on the discretized Bresenham circle, the absolute values of the differences between the gray values of the n continuous pixels and the gray value of the circle center are all larger than a preset difference threshold t_pNamely:

wherein, I_iThe gray value of the ith pixel point in n continuous pixel points is 1,2 …, n represents the serial number of the pixel point, I_PGray value with the center of a circle, t_pIs a preset difference threshold value;

and extracting the circle center of the discrete Bresenham circle as an image feature point based on a fast corner point detection method.

5. The method for judging the shielding state of the camera in real time according to claim 1, wherein the step 4 comprises: equally dividing the gray level image into 4 areas, which are respectively marked as area No. 1, area No. 2, area No. 3 and area No. 4, wherein the 4 areas are respectively positioned at the upper left position, the upper right position, the lower left position and the lower right position of the gray level image, and the specific positions of the areas are represented by the coordinates, the widths and the heights of the upper left corners of the areas, namely:

region No. 1

Region No. 2

Region No. 3

And region No. 4

Where w is the width of the grayscale image and h is the height of the grayscale image.

6. The method for judging the shielding state of the camera in real time according to claim 5, wherein the step 4 further comprises:

step 4-1, after the gray image feature point set is obtained in the step 3, extracting feature point information of the gray image, wherein the feature point information comprises: coordinates of the feature points on the gray level image;

step 4-2, determining the number r of the area where each feature point is located in the gray level image according to the coordinates of the feature points on the gray level image, and counting the number of the feature points in 4 areas respectively:

if it is

And is

Then r is determined to be 1, and the feature point P is determined_jThe number of the characteristic points in the No. 1 area is added with 1;

if it is

And is

Then r is determined to be 3, the feature point P_jThe number of the characteristic points in the No. 3 area is added with 1;

if it is

And is

Then r is determined to be 2, the feature point P_jThe number of the characteristic points in the No. 2 area is added with 1;

if it is

And is

Then r is determined to be 4, and the feature point P is determined_jThe number of the characteristic points in the No. 4 area is added with 1;

wherein (x)_j,y_j) Is a characteristic point P_jCoordinates on the grayscale image.

7. The method for judging the shielding state of the camera in real time according to claim 1, wherein before the step 5, a preset number threshold t needs to be determined, and the method comprises the following steps:

step 5-1, shooting a video under the condition that the camera is not shielded;

step 5-2, according to the video under the condition that the camera is not shielded, determining a sampling image frame number set which needs to be used by using a random sampling method, namely:

S_k＝S_k-1∪{y_k}

k＝1,…,γ

wherein S is_kSetting the image frame sequence number after the kth sampling

F is the frame number of the video shot under the condition that the camera is not shielded, theta is a random variable and meets the uniform distribution in the interval of 0,1, namely theta_kIs a real number, y, over the interval [0,1) randomly generated at the kth sample_kIs the image frame number obtained by the kth sampling, gamma is the sampling frequency, S_γThe sampling result is the finally obtained sampling result, namely a sampling image frame number set which needs to be used;

step 5-3, reading each frame of the sampling image set according to a sampling sequence, and sequentially executing the steps 1 to 4 on each frame to obtain the number of feature points of 4 areas in the gray level image corresponding to each frame;

step 5-4, sorting the number of the feature points of all the areas in all the gray level images from small to large to obtain a sequence a of the number of the feature points, wherein a is (a)₁,a₂,…,a_4×γ) Wherein a is_lThe number of characteristic points representing the ith smallest;

5-5, calculating the preset number threshold t based on a box chart analysis method:

t＝Q₁-1.5×IQR

IQR＝|Q₁-Q₂|

wherein t represents a preset number threshold, i.e., the lower bound of the normal value of the boxplot analysis method, Q₁Denotes the lower quartile, Q₂Representing the upper quartile, IQR is the interquartile range, representing the upper quartile Q₂And lower quartile Q₁The absolute value of the difference of (a);

the lower quartile Q₁And upper quartile Q₂The calculation method comprises the following steps:

d_o＝μ_o-c_o

o＝1,2

wherein, mu_oRepresents the lower quartile Q₁Or upper quartile Q₂In step 5-4, the sequence of feature point numbers a ═ a₁,a₂,…,a_4×γ) When o is 1, mu₁Represents the lower quartile Q₁When the position o in the feature point number sequence a is 2, mu₂Representing the upper quartile Q₂At a position in the sequence a of the number of feature points,

for indicating functions, it is the upper quartile Q that is used to distinguish the current calculation₂Or a lower quartile Q₁，c_oIs mu_oInteger part of (d)_oIs mu_oThe fractional part of (a).

Images