CN115424220A - Traffic light image labeling method and device, electronic equipment and storage medium - Google Patents

Abstract

The present application discloses a traffic light image tagging method and device, electronic equipment and a storage medium. The method includes: obtaining a road image of the current road section and detecting the road image to obtain a traffic light image and vehicle detection results; according to the traffic light image Determine the traffic light display situation, and determine the vehicle driving situation according to the traffic light image and vehicle detection results; determine the traffic light labeling data according to the traffic light display situation and vehicle driving situation; use the traffic light labeling data to mark the traffic light image, and get the traffic Lamp image annotation results. In the embodiment of the present application, on the basis of determining the display status of traffic lights through image detection, the light color of traffic lights is further combined with the vehicle driving conditions determined based on image detection, thereby obtaining more accurate traffic light labeling data, and realizing The self-labeling of traffic light images reduces a lot of manual labeling costs and improves the labeling efficiency of traffic light images.

Description

Traffic light image tagging method and device, electronic device and storage medium

technical field

The present application relates to the technical field of traffic light recognition, and in particular to a traffic light image labeling method and device, electronic equipment and a storage medium.

Background technique

At present, some traffic light equipment will be equipped with a sending device, which can send traffic light signals to road-side equipment or vehicle-side equipment. Most of the time, the current light color displayed by the traffic light can only be obtained through the method of vehicle-side image detection and recognition, but in When the traffic light equipment fails, the color of the traffic light recognized by the car is wrong. If the vehicle drives according to the wrong traffic light, it is easy to cause traffic accidents and lead to traffic jams.

At present, the process of traffic light recognition on the car side relies on deep learning models, and deep learning models rely on a large amount of labeled data, requiring a lot of manpower and material resources. Moreover, the traffic light image acquired by the vehicle end may be wrong due to its own positioning, which may lead to recognition errors, or the captured traffic light image may be detected incorrectly due to angle problems.

Contents of the invention

Embodiments of the present application provide a traffic light image labeling method and device, electronic equipment, and a storage medium, so as to realize self-labeling of traffic light image data and improve labeling efficiency and accuracy.

The embodiment of the application adopts the following technical solutions:

In the first aspect, the embodiment of the present application provides a traffic light image labeling method, wherein the method includes:

Obtain a road image of the current road section and detect the road image to obtain a traffic light image and vehicle detection results;

determining the traffic light display situation according to the traffic light image, and determining the vehicle driving situation according to the traffic light image and the vehicle detection result;

According to the display situation of the traffic light and the driving situation of the vehicle, determine the labeling data of the traffic light;

Using the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

Optionally, the acquiring the road image of the current road section and detecting the road image to obtain the traffic light image and vehicle detection results includes:

Obtain the high-precision map corresponding to the current road section, and the correspondence between the high-precision map and the camera image;

The traffic light image is segmented from the road image according to the high-precision map corresponding to the current road section and the corresponding relationship between the high-precision map and the camera image.

Optionally, the traffic light image includes multiple frames of traffic light images, and the determining the traffic light display situation according to the traffic light image includes:

Perform pixel difference and processing on any two adjacent frames of traffic light images in the multi-frame traffic light images;

According to the pixel difference of two adjacent frames of traffic light images and the processing result, the light color jumping condition and the display duration of the light color are determined as the traffic light display condition.

Optionally, the determining the vehicle driving condition according to the traffic light image and the vehicle detection result includes:

Obtain the high-precision map corresponding to the current road section, and the high-precision map includes the corresponding relationship between the lane and the traffic light;

determining the lane corresponding to the traffic light in the traffic light image according to the correspondence between the lane and the traffic light;

determining the position of the target vehicle corresponding to the lane according to the lane corresponding to the traffic light in the traffic light image and the vehicle detection result;

The vehicle running condition is determined according to the target vehicle position and the stop line position of the current road section.

Optionally, the determining the traffic light labeling data according to the traffic light display situation and the vehicle driving situation includes:

determining whether the traffic light image satisfies a preset traffic light labeling condition according to the traffic light display condition and the vehicle driving condition;

If the traffic light image satisfies the preset traffic light labeling condition, the traffic light labeling data is determined according to the traffic light display condition and the vehicle driving condition.

Optionally, the traffic light image includes a first traffic light image and a second traffic light image, the second traffic light image is obtained before the first traffic light image, and according to the traffic light display conditions and The vehicle driving condition, determining whether the traffic light image satisfies the preset traffic light labeling conditions includes:

Determining whether the first traffic light image satisfies a first preset traffic light labeling condition according to the traffic light display situation and the vehicle driving situation corresponding to the first traffic light image;

If the first traffic light image satisfies a first preset traffic light labeling condition, determine a light color detection result corresponding to the first traffic light image;

Determine the light color detection result corresponding to the second traffic light image according to the light color detection result corresponding to the first traffic light image and the traffic light display situation corresponding to the second traffic light image;

According to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image, determine whether the first traffic light image and the second traffic light image meet a second preset Set traffic light labeling conditions.

Optionally, the determining whether the first traffic light image satisfies the first preset traffic light labeling condition according to the traffic light display situation and the vehicle driving situation corresponding to the first traffic light image includes:

Determine whether the light color of the traffic light remains the same for multiple frames according to the traffic light display situation corresponding to the first traffic light image, and determine whether the corresponding multi-frame target vehicle position is within the stop line position of the current road section according to the vehicle driving situation ;

If so, after determining that the light color of the traffic light jumps according to the traffic light display situation corresponding to the first traffic light image, determine whether the corresponding multi-frame target vehicle position is located at the stop line position according to the vehicle driving situation outside;

If so, determine that the first traffic light image satisfies the first preset traffic light labeling condition.

Optionally, determining the first traffic light image and the second traffic light image according to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image Whether the image satisfies the second preset traffic light labeling condition includes:

According to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image, the light color display duration of the traffic light is determined, and the light color display duration of the traffic light includes red light, red light, The display duration of the green and yellow lights;

If the light color display duration of the red light and the light color display duration of the green light are both longer than the light color display duration of the yellow light, then it is determined that the first traffic light image and the second traffic light image satisfy the second preset traffic light label condition.

In the second aspect, the embodiment of the present application also provides a traffic light image tagging device, wherein the device includes:

an acquisition unit, configured to acquire a road image of the current road section and detect the road image to obtain a traffic light image and a vehicle detection result;

A first determining unit, configured to determine a traffic light display condition according to the traffic light image, and determine a vehicle running condition according to the traffic light image and the vehicle detection result;

A second determining unit, configured to determine traffic light marking data according to the traffic light display conditions and the vehicle driving conditions;

The labeling unit is configured to use the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

In a third aspect, the embodiment of the present application further provides an electronic device, including:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform any one of the preceding methods.

In a fourth aspect, the embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs , causing the electronic device to execute any one of the aforementioned methods.

The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects: the traffic light image labeling method of the embodiment of the present application first obtains the road image of the current road section and detects the road image to obtain the traffic light image and vehicle detection results ; Then determine the display of traffic lights according to the traffic light image, and determine the driving condition of the vehicle according to the traffic light image and vehicle detection results; then determine the traffic light labeling data according to the traffic light display and vehicle driving conditions; finally use the traffic light labeling data to Annotate the traffic light image to obtain the traffic light image annotation result. In the traffic light image labeling method of the embodiment of the present application, on the basis of determining the traffic light display situation through image detection, the light color of the traffic light is further combined with the detected vehicle driving conditions in the image, thereby obtaining more accurate traffic light labeling Data, and in this way realize the self-labeling of traffic light images, which reduces a lot of manual labeling costs and improves the labeling efficiency of traffic light images.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

Fig. 1 is a schematic flow chart of a traffic light image labeling method in the embodiment of the present application;

FIG. 2 is a schematic structural diagram of a traffic light image labeling device in an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

The technical solutions provided by various embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

The embodiment of the present application provides a traffic light image tagging method. As shown in FIG. 1 , it provides a schematic flow chart of a traffic light image tagging method in the embodiment of the present application. The method includes at least the following steps S110 to S140 :

Step S110, acquiring the road image of the current road segment and detecting the road image to obtain traffic light images and vehicle detection results.

The traffic light image labeling method in the embodiment of the present application can be performed by the roadside. The roadside detects the traffic light and sends the result to the vehicle side. The accuracy rate is relatively higher, because for the roadside camera, the traffic light in the image The position and shape in are relatively fixed, and training data can be added in a targeted manner to improve detection accuracy, but the detection and recognition of traffic lights at the road end also need to rely on a large amount of labeling data, so the traffic light image of the embodiment of the present application The labeling method is used to realize the self-labeling of traffic light image data.

When labeling traffic light image data, it is necessary to obtain the road image of the current road segment collected by the roadside camera first, and then use the preset image detection algorithm to detect the road image. The detection here mainly includes the detection of the traffic light area and the The detection of the contained vehicle target, the detection of the traffic light area refers to segmenting the local image of the area where the traffic light is located from the current road image, and the vehicle detection can be realized by using the existing target detection network such as YOLO V5 and other networks. In this way, the location of the vehicle target can be detected from the image.

Step S120, determining the display condition of the traffic light according to the traffic light image, and determining the driving condition of the vehicle according to the traffic light image and the vehicle detection result.

Based on the traffic light image obtained through continuous detection, the display status of the traffic light can be determined in one step through the change of the image pixels, such as the color jump of the traffic light and the display duration of the light color. In addition, since under normal circumstances, signal lights of different light colors and jumping changes of signal lights have a greater impact on the driving of the vehicle, it is also possible to further determine the driving conditions of the vehicle based on the traffic light image and the corresponding vehicle detection results. The driving situation of the vehicle further assists the recognition of the color of the traffic light.

Step S130, determining traffic light marking data according to the traffic light display conditions and the vehicle driving conditions.

The traffic light display status can reflect the color jump of the traffic light and the display duration of the same light color within a period of time. If the traffic light is displayed normally, the red light and green light can be roughly compared with the traffic light change rules based on the traffic light change rules. The yellow lights are distinguished, but when the traffic lights fail, for example, the red lights fail to flicker, at this time, the detection results reflected by the above-mentioned traffic light display conditions will also have errors, and then the light colors cannot be accurately distinguished.

Based on this, the embodiment of the present application further combines the driving conditions of the vehicle to assist in the detection of the light color and change of the traffic light, because in the case of a traffic light failure, most vehicles driven by humans can make accurate judgments and distinctions, so they can Avoid the problem that it cannot be applied to detection scenarios such as traffic light failure and flickering when only relying on traffic light images for detection. Of course, even in the scene where the traffic lights are normally displayed, the detection result of the traffic light image can be verified by combining the driving conditions of the vehicle, thereby further improving the accuracy of traffic light color recognition.

Step S140, using the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

The above traffic light labeling data can be regarded as the accurate labeling data of the currently detected traffic light image, which can specifically include information such as the position of the traffic light and the corresponding light color, and the traffic light image can be self-labeled based on the labeling data. Therefore, it can be used as a sample for subsequent roadside or vehicle side training traffic light color recognition models.

In the traffic light image labeling method of the embodiment of the present application, on the basis of determining the traffic light display situation through image detection, the light color of the traffic light is further combined with the detected vehicle driving conditions in the image, thereby obtaining more accurate traffic light labeling Data, and in this way realize the self-labeling of traffic light images, which reduces a lot of manual labeling costs and improves the labeling efficiency of traffic light images.

In some embodiments of the present application, the acquiring the road image of the current road section and detecting the road image, and obtaining the traffic light image and vehicle detection results include: obtaining the high-precision map corresponding to the current road section, and the high-precision map and Correspondence between camera images: segment the traffic light image from the road image according to the high-precision map corresponding to the current road section and the correspondence between the high-precision map and the camera image.

Since the labeling method of the traffic light image in the embodiment of the present application is performed by the roadside, and the position of the roadside camera is usually relatively fixed, unless it is manually adjusted, generally there will be no major changes. The position in the road image is also basically fixed.

Based on this, in the embodiment of the present application, the joint calibration of the high-precision map and the roadside camera can be performed first, so as to obtain the corresponding relationship between the camera image and the high-precision map. Based on the corresponding relationship between the camera image and the high-precision map, the The provided traffic light position of the current road segment is used to cut out the specific area where the traffic light is located at the corresponding position of the camera image as the traffic light image.

In some embodiments of the present application, the traffic light image includes multiple frames of traffic light images, and the determining the traffic light display situation according to the traffic light images includes: The light image is subjected to pixel difference and processing; according to the pixel difference and processing results of two adjacent frames of traffic light images, the light color jump and the light color display duration are determined as the traffic light display.

The detection of traffic light images at the road end is a real-time and continuous process. Based on the detected multi-frame traffic light images, pixel difference and processing can be performed on any two adjacent frames of traffic light images, that is, using the current frame of traffic light images Each corresponding pixel value in the previous frame of traffic light image is subtracted from each corresponding pixel value in the previous frame of traffic light image, and finally all the obtained pixel value differences are summed as the pixel difference and processing result of two adjacent frames of traffic light images .

The purpose of pixel difference and processing is to determine the jump of the traffic light, that is, to detect whether the traffic light has a light color change in the current frame or is in a normal display of a certain light color. Because if the current frame is in a situation where the light color changes from yellow light to red light, green light to yellow light, or red light to green light, etc., the light color change will inevitably lead to a large difference in pixel values, then the pixel value of the current frame relative to the previous frame The value obtained by the difference sum should be very large, which can be judged by setting an adjustable threshold. And if the current frame and the previous frame are in the continuous display of yellow light, green light or red light, and the pixel values between adjacent frames basically do not change, then the value of the pixel difference sum of adjacent frames should be very small, that is, close to at or equal to 0.

Therefore, through the processing method of the above-mentioned pixel difference sum, the jumping situation of the traffic light can be determined in the continuous detection process of the traffic light image, that is, in which frame the jumping occurs, and at the same time, according to the distance between two adjacent jumps The time interval can also determine the continuous display time of a single light color.

In some embodiments of the present application, the display condition of the traffic light includes the display duration of the light color, and after determining the display condition of the traffic light according to the traffic light image, the method further includes: comparing the display duration of the light color with the preset A duration threshold is set for comparison; if the display duration of the light color exceeds the preset duration threshold, it is determined that the traffic light has failed and a reminder message is sent to the failure platform.

Based on the foregoing embodiments, the continuous display duration of a single light color can be determined. Under normal circumstances, the continuous display duration of the same light color should be within a reasonable duration. However, if the traffic light fails, it may display abnormally. Therefore, this application The embodiment can compare the display duration of the light color with the pre-set preset duration threshold. If the threshold is exceeded, it is considered that the traffic light display is faulty, and then the front and back video can be sent to the fault platform for evidence collection, so as to handle the fault in time. The above-mentioned preset duration threshold can be flexibly set according to actual needs, and is not specifically limited here.

In some embodiments of the present application, the determining the vehicle driving condition according to the traffic light image and the vehicle detection result includes: acquiring a high-precision map corresponding to the current road segment, and the high-precision map includes lanes and traffic lights Correspondence: determining the lane corresponding to the traffic light in the traffic light image according to the correspondence between the lane and the traffic light; determining the lane according to the lane corresponding to the traffic light in the traffic light image and the vehicle detection result The corresponding target vehicle position; determining the vehicle driving condition according to the target vehicle position and the stop line position of the current road section.

In the actual road scene, not every intersection has only one traffic light, and many intersections have different traffic lights corresponding to different lanes. The high-precision map often provides the corresponding relationship between the traffic lights at each intersection and each lane, so the embodiment of the present application can be based on the correspondence between the lanes and the traffic lights in the high-precision map, as well as the lanes and road ends of the high-precision map The corresponding relationship of the camera images determines the specific lane corresponding to each traffic light contained in the traffic light image, that is, it can be determined which area in the image corresponds to which lane, and which traffic light corresponds to the lane. Therefore, the detection of the traffic light in the embodiment of the present application is the detection at the lane level, and the lane information obtained here is also the basis for subsequent determination of the vehicle driving conditions of the lane corresponding to the traffic light.

In some embodiments of the present application, the determining the traffic light labeling data according to the traffic light display and the vehicle driving condition includes: determining the traffic light label data according to the traffic light display and the vehicle driving condition Whether the light image satisfies the preset traffic light labeling condition; if the traffic light image satisfies the preset traffic light labeling condition, determine the traffic light labeling according to the traffic light display situation and the vehicle driving condition data.

In order to ensure the accuracy of the traffic light image labeling data, the embodiment of the present application needs to meet certain preset traffic light labeling conditions when determining the labeling data according to the traffic light display and vehicle driving conditions, because in some special cases, only relying on The traffic light display and vehicle driving conditions still cannot accurately determine the light color of the traffic light. For example, when the traffic light display changes from green light to yellow light or from yellow light to red light, there may still be some The vehicle passes through the intersection, so it is difficult to accurately judge the specific light color before and after the traffic light jumps according to the driving conditions of the vehicle.

Based on this, the embodiment of the present application can first judge whether the current traffic light image satisfies the preset traffic light labeling conditions. If so, the traffic light labeling data can be determined based on the traffic light display and vehicle driving conditions. The image data generated by the traffic light labeling conditions can be obtained by inversion based on the currently marked data and the changing rules of the traffic lights.

In some embodiments of the present application, the traffic light image includes a first traffic light image and a second traffic light image, the second traffic light image is obtained before the first traffic light image, and according to the The traffic light display situation and the vehicle running situation, determining whether the traffic light image meets the preset traffic light labeling conditions includes: determining the second traffic light image according to the traffic light display situation and the vehicle running situation corresponding to the first traffic light image Whether a traffic light image satisfies the first preset traffic light labeling condition; if the first traffic light image satisfies the first preset traffic light labeling condition, determine the light color detection result corresponding to the first traffic light image ; According to the light color detection result corresponding to the first traffic light image and the traffic light display situation corresponding to the second traffic light image, determine the light color detection result corresponding to the second traffic light image; according to the first traffic light image The light color detection result corresponding to the traffic light image and the light color detection result corresponding to the second traffic light image, determine whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling condition .

Since the detection of the traffic light image at the road end is carried out in real time, for a part of the newly detected traffic light images, that is, the above-mentioned first traffic light detection image, it can first be judged whether the first traffic light detection image satisfies the first preset traffic light detection image. Labeling conditions, the first preset traffic light labeling condition can be understood as the situation that the specific light color of a part of the traffic light image can be accurately distinguished based on the vehicle driving situation and the traffic light display situation, for example, when the traffic light display situation changes from red light to When the light is green, the driving conditions of the vehicles at the intersection often change greatly, so this situation can be accurately identified according to the corresponding vehicle driving movements.

Therefore, if the first traffic light image satisfies the first preset traffic light labeling condition, the light color detection result corresponding to the first traffic light image can be determined at this time, such as the current light color jump and the last light color jump The light color corresponding to the traffic light image between should be red, and the light color corresponding to the traffic light image after the current light color jump should be green. And because there is a certain rule in the change of the light color of the traffic light, for example, it usually changes in sequence according to the order of red light-green light-yellow light-red light..., so after the light color of the first traffic light image is determined, it can also be based on The light color detection result corresponding to the above first traffic light image and the light color change rule of the traffic light are backwardly deduced to the light color detection result corresponding to the second traffic light image. An image that has not been labeled after the traffic light image with preset traffic light labeling conditions.

After obtaining the light color detection results corresponding to the first traffic light image and the second traffic light image, it can be further determined whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling condition, the second preset The traffic light labeling condition is mainly used to judge whether the light color display duration of different traffic lights conforms to the normal display law. When the above two conditions are met, it can be considered that the first traffic light image and the second traffic light image obtained this time are The light color detection result corresponding to the image is accurate and can be used as traffic light labeling data.

In some embodiments of the present application, the determining whether the first traffic light image satisfies the first preset traffic light labeling condition according to the traffic light display situation and the vehicle driving situation corresponding to the first traffic light image includes: Determine whether the light color of the traffic light remains the same for multiple frames according to the traffic light display situation corresponding to the first traffic light image, and determine whether the corresponding multi-frame target vehicle position is within the stop line position of the current road section according to the vehicle driving situation If so, after determining that the light color of the traffic light jumps according to the traffic light display situation corresponding to the first traffic light image, determine whether the corresponding multi-frame target vehicle position is located at the stop line according to the vehicle driving situation If not, determine that the first traffic light image satisfies the first preset traffic light labeling condition.

The first traffic light image in the embodiment of the present application can be divided into the first traffic light image before the jump and the first traffic light image after the jump based on the jump situation, and the light color corresponding to the first traffic light image before the jump If multiple frames continue to be the same, at this time, it is judged whether the vehicle position corresponding to the lane is within the stop line and has not driven out. If so, after a light color jump, the vehicle position corresponding to the first traffic light image after the jump Whether to drive out of the stop line, if yes, the light color of the traffic light image corresponding to the first traffic light image before the jump should be red, and the light color of the traffic light image corresponding to the first traffic light image after the jump should be red. is green, continue to trace forward, the light color before jumping to red should be yellow. And if the light color jump conflicts with the driving situation of the vehicle, it is considered that there may be a traffic light failure, and the front and rear video evidence is sent to the fault platform for repair as soon as possible.

The principle of the above embodiment is that during the transition from a red light to a green light, the vehicle movement in the corresponding lane will change greatly, that is, the behavior of stopping before the stop line at the intersection will change to the behavior of driving out of the stop line. Changes in vehicle driving conditions can accurately distinguish the color of traffic lights before and after this jump. Although it is not ruled out that some vehicles violate traffic rules and run red lights, based on the accumulation of a large amount of data, the driving conditions of most vehicles are in line with the above. required.

In some embodiments of the present application, the first traffic light image and the light color detection result corresponding to the second traffic light image are determined according to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image. Whether the second traffic light image satisfies the second preset traffic light labeling condition includes: determining the color of the traffic light according to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image The display duration of the light color, the display duration of the light color of the traffic light includes the display duration of the red light, the green light and the yellow light; display duration, it is determined that the first traffic light image and the second traffic light image satisfy a second preset traffic light labeling condition.

In order to further improve the accuracy of traffic light detection, this embodiment of the present application may further determine whether the light color detection results of the first traffic light image and the second traffic light image obtained in the above embodiment meet the second preset traffic light labeling condition. Under normal circumstances, the display duration of a yellow light is generally only a few seconds, while the display duration of a red light and a green light are longer than that of a yellow light. Therefore, the light color detection result based on the first traffic light image and the second traffic light image can be The continuous display duration of each light color is counted separately. If the display duration of the red light and green light is greater than the display duration of the yellow light, it can be considered that the detection result of the light color is accurate, otherwise it is considered that the traffic light may be malfunctioning , Send before and after video to the faulty platform for evidence collection so as to repair as soon as possible.

The embodiment of the present application also provides a traffic light image tagging device 200. As shown in FIG. 2, a schematic structural diagram of a traffic light image tagging device in the embodiment of the present application is provided. The first determination unit 220, the second determination unit 230, and the labeling unit 240, wherein:

An acquisition unit 210, configured to acquire a road image of the current road section and detect the road image to obtain a traffic light image and vehicle detection results;

The first determining unit 220 is configured to determine the display condition of the traffic light according to the traffic light image, and determine the driving condition of the vehicle according to the traffic light image and the vehicle detection result;

The second determining unit 230 is configured to determine traffic light marking data according to the traffic light display conditions and the vehicle driving conditions;

The labeling unit 240 is configured to use the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

In some embodiments of the present application, the acquisition unit 210 is specifically configured to: acquire the high-precision map corresponding to the current road section, and the correspondence between the high-precision map and the camera image; according to the high-precision map corresponding to the current road section As well as the corresponding relationship between the high-precision map and the camera image, the traffic light image is segmented from the road image.

In some embodiments of the present application, the traffic light image includes multiple frames of traffic light images, and the first determining unit 220 is specifically configured to: pixelate any two adjacent frames of traffic light images in the multiple frames of traffic light images Difference and processing: According to the pixel difference and processing results of two adjacent frames of traffic light images, determine the light color jump and the display duration of the light color as the traffic light display.

In some embodiments of the present application, the first determination unit 220 is specifically configured to: obtain a high-precision map corresponding to the current road segment, the high-precision map includes the correspondence between lanes and traffic lights; Determine the lane corresponding to the traffic light in the traffic light image according to the corresponding relationship of the lights; determine the target vehicle position corresponding to the lane according to the lane corresponding to the traffic light in the traffic light image and the vehicle detection result; The position of the target vehicle and the position of the stop line of the current road section determine the driving condition of the vehicle.

In some embodiments of the present application, the second determination unit 230 is specifically configured to: determine whether the traffic light image satisfies a preset traffic light labeling condition according to the traffic light display condition and the vehicle driving condition; When the traffic light image satisfies the preset traffic light labeling condition, the traffic light labeling data is determined according to the traffic light display condition and the vehicle driving condition.

In some embodiments of the present application, the traffic light image includes a first traffic light image and a second traffic light image, the second traffic light image is obtained before the first traffic light image, and the second determined Unit 230 is specifically configured to: determine whether the first traffic light image satisfies the first preset traffic light labeling condition according to the traffic light display situation and vehicle driving situation corresponding to the first traffic light image; When the light image satisfies the first preset traffic light labeling condition, determine the light color detection result corresponding to the first traffic light image; according to the light color detection result corresponding to the first traffic light image and the second traffic light According to the traffic light display situation corresponding to the light image, determine the light color detection result corresponding to the second traffic light image; according to the light color detection result corresponding to the first traffic light image and the light color corresponding to the second traffic light image As a result of the detection, it is determined whether the first traffic light image and the second traffic light image satisfy a second preset traffic light labeling condition.

In some embodiments of the present application, the second determination unit 230 is specifically configured to: determine whether the light color of the traffic light remains the same for multiple frames according to the traffic light display situation corresponding to the first traffic light image, and according to the Vehicle driving conditions determine whether the corresponding multi-frame target vehicle position is within the stop line position of the current road section; if so, after determining that the light color of the traffic light jumps according to the traffic light display situation corresponding to the first traffic light image, Determine whether the corresponding multi-frame target vehicle position is located outside the stop line position according to the vehicle driving condition; if so, determine that the first traffic light image satisfies the first preset traffic light labeling condition.

In some embodiments of the present application, the second determining unit 230 is specifically configured to: determine the traffic light according to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image The light color display duration of the light, the light color display duration of the traffic light includes the light color display duration of red light, green light and yellow light; if the light color display duration of the red light and the light color display duration of the green light are greater than If the display time of the light color is long, it is determined that the first traffic light image and the second traffic light image meet the second preset traffic light labeling condition.

It can be understood that the above-mentioned traffic light image tagging device can realize each step of the traffic light image tagging method provided in the foregoing embodiments, and relevant explanations about the traffic light image tagging method are applicable to the traffic light image tagging device, and will not be repeated here. .

Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to FIG. 3 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Wherein, the memory may include a memory, such as a high-speed random-access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. Of course, the electronic device may also include hardware required by other services.

The processor, the network interface and the memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnection standard) bus or an EISA (Extended Industry StandardArchitecture, extended industry standard architecture) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one double-headed arrow is used in FIG. 3 , but it does not mean that there is only one bus or one type of bus.

Memory for storing programs. Specifically, the program may include program code, and the program code includes computer operation instructions. Storage, which can include internal memory and nonvolatile storage, provides instructions and data to the processor.

The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a traffic light image labeling device on a logical level. The processor executes the program stored in the memory, and is specifically used to perform the following operations:

Obtain a road image of the current road section and detect the road image to obtain a traffic light image and vehicle detection results;

determining the traffic light display situation according to the traffic light image, and determining the vehicle driving situation according to the traffic light image and the vehicle detection result;

According to the display situation of the traffic light and the driving situation of the vehicle, determine the labeling data of the traffic light;

Using the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

The method performed by the traffic light image labeling device disclosed in the above embodiment shown in FIG. 1 of the present application may be applied to or implemented by a processor. A processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. Above-mentioned processor can be general-purpose processor, comprises central processing unit (Central Processing Unit, CPU), network processor (Network Processor, NP) etc.; It can also be Digital Signal Processor (Digital Signal Processor, DSP), ASIC (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.

The electronic device can also execute the method performed by the traffic light image tagging device in FIG. 1 , and realize the functions of the traffic light image tagging device in the embodiment shown in FIG. 1 , which will not be repeated in this embodiment of the present application.

The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs include instructions, and when the instructions are executed by an electronic device including a plurality of application programs , the electronic device can be made to execute the method executed by the traffic light image tagging device in the embodiment shown in FIG. 1, and is specifically used to execute:

Obtain a road image of the current road section and detect the road image to obtain a traffic light image and vehicle detection results;

determining the traffic light display situation according to the traffic light image, and determining the vehicle driving situation according to the traffic light image and the vehicle detection result;

According to the display situation of the traffic light and the driving situation of the vehicle, determine the labeling data of the traffic light;

Using the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (11)

1. A traffic light image labeling method, wherein the method comprises: Obtain the road image of the current road section and detect the road image to obtain the traffic light image and vehicle detection results; determining the traffic light display situation according to the traffic light image, and determining the vehicle driving situation according to the traffic light image and the vehicle detection result; According to the display situation of the traffic light and the driving situation of the vehicle, determine the labeling data of the traffic light; Using the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result. 2. The method according to claim 1, wherein said obtaining the road image of the current road section and detecting said road image, obtaining traffic light images and vehicle detection results comprises: Obtain the high-precision map corresponding to the current road section, and the correspondence between the high-precision map and the camera image; The traffic light image is segmented from the road image according to the high-precision map corresponding to the current road section and the corresponding relationship between the high-precision map and the camera image. 3. The method according to claim 1, wherein the traffic light image comprises a plurality of frames of traffic light images, and determining the traffic light display situation according to the traffic light images comprises: Perform pixel difference and processing on any two adjacent frames of traffic light images in the multi-frame traffic light images; According to the pixel difference of two adjacent frames of traffic light images and the processing result, the light color jumping condition and the display duration of the light color are determined as the traffic light display condition. 4. The method according to claim 1, wherein said determining vehicle driving conditions according to said traffic light image and said vehicle detection result comprises: Obtain the high-precision map corresponding to the current road section, and the high-precision map includes the corresponding relationship between the lane and the traffic light; determining the lane corresponding to the traffic light in the traffic light image according to the correspondence between the lane and the traffic light; determining the position of the target vehicle corresponding to the lane according to the lane corresponding to the traffic light in the traffic light image and the vehicle detection result; The vehicle running condition is determined according to the target vehicle position and the stop line position of the current road section. 5. The method according to claim 1, wherein said determining traffic light labeling data according to said traffic light display conditions and said vehicle driving conditions comprises: determining whether the traffic light image satisfies a preset traffic light labeling condition according to the traffic light display condition and the vehicle driving condition; If the traffic light image satisfies the preset traffic light labeling condition, the traffic light labeling data is determined according to the traffic light display condition and the vehicle driving condition. 6. The method of claim 5, wherein said traffic light image comprises a first traffic light image and a second traffic light image, said second traffic light image being obtained prior to said first traffic light image, said According to the display condition of the traffic light and the driving condition of the vehicle, determining whether the traffic light image satisfies the preset traffic light labeling condition includes: Determining whether the first traffic light image satisfies a first preset traffic light labeling condition according to the traffic light display condition and the vehicle driving condition corresponding to the first traffic light image; If the first traffic light image satisfies a first preset traffic light labeling condition, determine a light color detection result corresponding to the first traffic light image; Determine the light color detection result corresponding to the second traffic light image according to the light color detection result corresponding to the first traffic light image and the traffic light display situation corresponding to the second traffic light image; According to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image, determine whether the first traffic light image and the second traffic light image meet a second preset Set traffic light labeling conditions. 7. The method according to claim 6, wherein, according to the traffic light display conditions and vehicle driving conditions corresponding to the first traffic light image, it is determined whether the first traffic light image satisfies the first preset traffic light labeling Conditions include: Determine whether the light color of the traffic light remains the same for multiple frames according to the traffic light display situation corresponding to the first traffic light image, and determine whether the corresponding multi-frame target vehicle position is within the stop line position of the current road section according to the vehicle driving situation ; If so, after determining that the light color of the traffic light jumps according to the traffic light display situation corresponding to the first traffic light image, determine whether the corresponding multi-frame target vehicle position is located at the stop line position according to the vehicle driving situation outside; If so, determine that the first traffic light image satisfies the first preset traffic light labeling condition. 8. The method according to claim 6, wherein the first traffic light is determined according to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image Whether the image and the second traffic light image meet the second preset traffic light labeling condition includes: According to the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image, the light color display duration of the traffic light is determined, and the light color display duration of the traffic light includes red light, red light, The display duration of the green and yellow lights; If the light color display duration of the red light and the light color display duration of the green light are both longer than the light color display duration of the yellow light, then it is determined that the first traffic light image and the second traffic light image satisfy the second preset traffic light label condition. 9. A traffic light image labeling device, wherein the device comprises: an acquisition unit, configured to acquire a road image of the current road section and detect the road image to obtain a traffic light image and a vehicle detection result; A first determining unit, configured to determine a traffic light display condition according to the traffic light image, and determine a vehicle running condition according to the traffic light image and the vehicle detection result; A second determining unit, configured to determine traffic light marking data according to the traffic light display conditions and the vehicle driving conditions; The labeling unit is configured to use the traffic light labeling data to label the traffic light image to obtain a traffic light image labeling result. 10. An electronic device comprising: processor; and A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the method of any one of claims 1-8. 11. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform The method according to any one of claims 1-8.

Images