CN117456398A - Image recognition methods, devices, electronic equipment and storage media for high-risk operations in gas stations - Google Patents

Abstract

The invention discloses a gas station high risk operation image recognition method and a device, wherein the method comprises the following steps: A. collecting an original video of a gas station area, and carrying out data preprocessing on a video frame containing a key monitoring target based on a lightweight convolutional neural network; B. extracting characteristics of the preprocessed video frames according to personnel, vehicles and dangerous sources, and performing classification recognition training; C. and respectively carrying out feature fusion on the personnel features, the dangerous source features, the personnel features and the vehicle features according to the model after the classification recognition training, and carrying out violation behavior alarm through personnel track intention analysis and personnel identity analysis. According to the invention, through image preprocessing, feature extraction and feature fusion and combining with a lightweight identification model, the accuracy of identifying high-risk operation can be improved, early warning can be performed in time, and the calculated amount of the edge can be effectively reduced.

Description

Image recognition methods, devices, electronic equipment and storage media for high-risk operations in gas stations

Technical field

The invention relates to the technical field of petrochemical safety engineering, and in particular to a method, device, electronic equipment and storage medium for image recognition of high-risk operations in a gas station.

Background technique

As society's demand for oil continues to increase, accidents at gas stations are becoming more and more frequent, and safety management issues at gas stations have gradually attracted social attention. At present, the number of gas stations across the country has reached nearly 100,000, which has brought great convenience to people's transportation. The risk sources of gas stations are as follows: 1. Vehicle injuries: Vehicles driving too fast in the refueling lane and being blocked and crowded can easily cause personal injuries; 2. Fire and explosion: Due to the volatile characteristics of gasoline and diesel, they will ignite when they are ignited. The main causes of fires are as follows: First, vehicles carry flammable and explosive items, which cause fires in gas stations; Second, oil and gas leakage occurs in the tanker and unloading area, causing oil and gas to accumulate. When encountering an ignition source Fires and explosions are prone to occur. The ignition sources are mainly caused by dangerous behaviors of personnel such as smoking, making phone calls, and insufficient oil stabilization time. Third, when inspection and maintenance operations are carried out in tank areas and sheds, it is easy for personnel to enter without wearing seat belts or entering without invoices. Violations such as inside the tank and failure to wear protective masks. When an accident occurs, the on-site monitoring personnel are not in place and the emergency rescue equipment is missing, which will lead to further expansion of the accident and cause huge economic losses to the gas station. Supervising the behavior of high-risk personnel within gas stations has become particularly important.

To sum up, gas stations need to be equipped with corresponding cameras in various areas to prevent and control risks at gas stations. The real-time nature of the prevention and control method through manual retrieval of surveillance videos has relatively large flaws, and it is impossible to complete the safety warning information at the first time. Sent to managers. With the continuous development of artificial intelligence technology, various machine learning algorithms have been widely used in the field of image recognition. Common machine learning classification algorithms include logistic regression algorithm, support vector machine algorithm, etc. Their advantages include fast calculation speed and small model size. , the disadvantage is that the classification accuracy is not high, and the deep learning algorithm can achieve good recognition results by taking advantage of the strong generalization ability of the neural network. The disadvantage is that the number of parameters of the neural network is large and the training and learning time is long.

Chinese patent application CN109271938A discloses a method for safety monitoring of the oil unloading process at a gas station based on intelligent video analysis technology. This solution performs a neural network deep learning training process and establishes the identification of key characteristic items related to safety operating procedures requirements in the oil unloading scene. Neural network model, all key feature item recognition can share a neural network model. This solution can automatically generate alarm information, which can effectively avoid the current common uncertainty of manual remote video monitoring and reduce the risk of safety accidents caused by the negligence of safety managers.

In addition, because the actual locations of gas stations are widely distributed, many video analysis algorithms are required, and the judgment of certain events requires comprehensive analysis and judgment through multiple algorithms, such as unloading port status detection, pyrotechnics detection, and insufficient monitoring personnel. Detection, etc., can easily lead to insufficient front-end (i.e. edge) computing resources. On the other hand, there is a lot of redundancy in the actual scene data of the gas station. There are no targets in a large number of video image frames. Directly analyzing the original data may lead to a waste of computing resources. In addition, intelligent early warning algorithms for high-risk abnormal events require faster recognition speed and higher recognition accuracy. Only common machine learning algorithms or deep learning algorithms cannot meet the above needs.

Therefore, there is an urgent need for an edge-side image recognition method and device for high-risk gas station operations to better solve the problem of computing resources.

The information disclosed in this Background section is merely intended to enhance an understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art that is already known to a person of ordinary skill in the art.

Contents of the invention

The purpose of the present invention is to provide a method and device for image recognition of high-risk operations in gas stations based on edge analysis. Through image preprocessing, feature extraction and feature fusion, combined with a lightweight recognition model, it can not only improve the accuracy of identifying high-risk operations It can provide timely warning and effectively reduce the amount of calculation at the edge.

In order to achieve the above objectives, according to the first aspect of the present invention, the present invention provides a method for image recognition of high-risk operations in a gas station, which includes the following steps: A. Collect the original video of the gas station area, and use lightweight convolutional neural The network performs data preprocessing on video frames containing key monitoring targets; B. Extract features of the preprocessed video frames according to people, vehicles and hazard sources, and conduct classification and recognition training; C. Based on the trained model for classification recognition, The characteristics of personnel and hazard source characteristics, personnel characteristics and vehicle characteristics are respectively fused, and illegal behavior alarms are carried out through personnel trajectory intention analysis and personnel identity analysis.

Furthermore, in the above technical solution, the feature extraction and recognition training of people in step B may specifically include: detecting and tracking people in video frames by identifying the movement trajectories of human targets; detecting and tracking people in video frames through the positions of key skeleton points of the human body. The movement characteristics of people in video frames are detected; through the recognition of local attribute characteristics of people, it is judged whether the protective measures of the personnel are in place and the identity of the personnel.

Furthermore, in the above technical solution, the vehicle feature extraction and recognition training in step B may specifically include: identifying vehicles in the refueling area and the oil unloading area and labeling the oil unloading vehicles; using the labeled video frames of the oil unloading vehicles to model train.

Furthermore, in the above technical solution, the feature extraction and identification training of hazard sources in step B may specifically include: identifying and labeling potentially risky objects such as tank mouths, operating sparks, and cranes, and using the labeled Object target video frames are used for model training.

Furthermore, in the above technical solution, the feature fusion of human characteristics and hazard source characteristics in step C may specifically include: calibrating around the detected human body pixel set to form an action trajectory of the human target; The direction and distance to the hazard source are used to determine whether the human target is approaching the hazard source.

Furthermore, in the above technical solution, the feature fusion of human characteristics and vehicle characteristics in step C may specifically include: after detecting the arrival of the vehicle, continuously monitoring the human targets around the vehicle and classifying the human identity; at a preset time threshold Within the system, it is judged whether the staff is close to the vehicle and whether to guide the vehicle according to the regulations based on the detection results of the staff's movement characteristics.

Furthermore, in the above technical solution, the personnel identity analysis in step C may specifically include: judging whether the person is a staff member, a guardian or a non-staff member based on the clothing characteristics in the partial attributes of the person.

Furthermore, in the above technical solution, the data preprocessing in step A may include key frame extraction and image compression.

Furthermore, in the above technical solution, key frame extraction may specifically include: using the first frame of the original video as the starting frame, extracting local features of the starting frame, and setting corresponding thresholds based on the local features; sequentially extracting the key frames of subsequent video frames. Local feature points are compared with the feature points of the start frame. When the number of non-similar feature points is greater than the threshold, the frame is set as the end frame; the relationship between the start frame and the end frame and the start frame and the end frame are obtained. The frame with the most similar points is used as the candidate key frame of the original video; the candidate key frames are input into the lightweight convolutional neural network in turn, and the candidate key frames containing the key monitoring targets are detected and retained.

According to the second aspect of the present invention, the present invention provides an image recognition device for high-risk operations in a gas station, including: a collection and preprocessing module, which is used to collect original videos of the gas station area, and is based on lightweight convolutional neural The network performs data preprocessing on video frames containing key monitoring targets; the feature extraction and training module is used to extract features of the preprocessed video frames according to people, vehicles and hazard sources, and perform classification and recognition training; feature fusion and The alarm module is used to fuse personnel characteristics and hazard source characteristics, personnel characteristics and vehicle characteristics based on the model trained by classification recognition, and conduct violation alarms through personnel trajectory intention analysis and personnel identity analysis.

According to a third aspect of the present invention, the present invention provides an electronic device for image recognition of high-risk operations in a gas station, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores information that can be at least Instructions executed by a processor, the instructions being executed by the at least one processor, so that the at least one processor executes the aforementioned high-risk operation image recognition method in a gas station.

According to a fourth aspect of the invention, the invention provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores computer-executable instructions, the computer-executable instructions are used to cause the computer to Implement the image recognition method for high-risk gas station operations as mentioned above.

Compared with the prior art, the present invention has one or more of the following beneficial effects:

1) Through feature extraction and feature fusion, the high-risk gas station operation image recognition method of the present invention can not only realize the classification and individual identification of personnel targets, vehicle targets, and hazard source targets, but also achieve more accurate identification through the combination and fusion of target features. Call the police for violations;

2) In the preprocessing process after video collection, the present invention adopts a method based on similarity discrimination between previous and following frames to select video key frames, and combines it with a target detection algorithm based on a convolutional neural network to further optimize the selection of key frames, which can effectively remove redundancy. Besides, improve calculation efficiency;

3) The present invention compresses video frames that consume large data resources, reducing image quality, length and width, and effectively reducing the size of data;

4) The present invention can embed lightweight recognition models into edge analysis hardware chips, extract local features based on key point locations, perform rapid deduplication and key frame selection, and effectively improve the calculation speed and accuracy of the algorithm;

5) The present invention can effectively reduce the computational complexity of high-resolution images by normalizing the scale of the image during the model training stage.

The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and implement them according to the content of the specification, and to make the above and other objects, technical features and advantages of the present invention easier to understand, One or more preferred embodiments are enumerated below and described in detail with reference to the accompanying drawings.

Description of the drawings

Figure 1 is a schematic flow chart of the high-risk operation image recognition method at a gas station in Embodiment 1 of the present invention.

Figure 2 is a schematic structural diagram of an image recognition device for high-risk operations in a gas station according to Embodiment 2 of the present invention.

Figure 3 is a schematic structural diagram of the image recognition electronic device for high-risk operations at a gas station in Embodiment 4 of the present invention.

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

Unless expressly stated otherwise, throughout the specification and claims, the term "comprises" or its variations such as "includes" or "comprises of," etc., will be understood to include the stated elements or components, and to No other elements or other components are excluded.

In this document, for convenience of description, spatially relative terms, such as "below", "below", "lower", "upper", "upper", "upper", etc., may be used to describe the relationship between one element or feature and another. The relationship of elements or features in the drawing. It will be understood that the spatially relative terms are intended to encompass different orientations of the item in use or operation in addition to the orientation depicted in the figures. For example, if the object in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" the elements or features. Therefore, the exemplary term "below" may include both lower and upper directions. The object may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative terms used herein should be interpreted accordingly.

In this article, the terms "first", "second", etc. are used to distinguish two different elements or parts, but are not used to limit a specific position or relative relationship. In other words, in some embodiments, the terms "first", "second", etc. may also be interchanged with each other.

The method, system, electronic device and storage medium of the present invention will be described in more detail below with specific embodiments. It should be understood that the embodiments are only exemplary and the present invention is not limited thereto.

The present invention can effectively identify high-risk operation images by performing edge analysis at the gas station. Through the steps of video collection, image preprocessing, image feature extraction, edge-end model training, feature fusion, and violation alarm, it can not only realize personnel Targets, vehicle targets, and dangerous source targets can be classified and identified separately. Violation alarms can also be issued through the combination and fusion of target features. At the same time, it can effectively save the calculation amount of the model and meet the image recognition needs of the special application scenario of gas stations. Setting it at the edge creates better conditions.

Example 1

As shown in Figure 1, this embodiment provides a method for image recognition of high-risk operations at a gas station, including the following steps:

Step S101: Collect original videos of the gas station area, and perform data preprocessing on video frames containing key monitoring targets based on a lightweight convolutional neural network. The steps are as follows:

The original video access in this step mainly obtains video image data by connecting to video surveillance or video surveillance platforms. Video access supports onvif and rtsp protocols.

Further, preferably but not limiting, the data preprocessing in this step includes key frame extraction and image compression. Among them, keyframe extraction specifically includes:

1) First, set the first frame of the video as the starting frame (for example, the video frame at time t1 is the starting frame); extract the local features of the starting frame (for example, f(t1)), which have scale invariance, It is mainly based on the points of interest on the target and is insensitive to the scale of the image, image rotation, lighting changes, noise, and slight changes in perspective. Therefore, it is used to extract local features such as the position, scale, and rotation invariants of the video frame and set The corresponding threshold (such as threshold Sthre);

2) Secondly, extract local feature points of subsequent video frames in sequence and compare them with the feature points of the starting frame. When the number of non-similar feature points is greater than the set threshold, the frame is set as the end frame. For example, set the frame interval to Δt, extract the local features of the key points of the subsequent frames tn=t1+(n-1)Δt), and compare the feature similarity with the key points of the starting frame Sn=f(tn)/f( t1), when the similarity is less than the threshold, the frame is set as the end frame;

3) Then, between the start frame and the end frame, find the frame with the most similar key points to the start frame and the end frame as the candidate key frame of the video;

4) Finally, input the candidate key frames into the lightweight convolutional neural network in sequence to detect whether they contain key targets such as people, vehicles, and danger sources. If they do, retain them as key frames and perform the next step. , if not included, discard the image. Specifically, key targets such as vehicles, personnel, oil unloading ports, fire extinguishers, electrostatic clamps, and tank mouths in key frames can be detected based on lightweight convolutional neural networks, and the initially screened video key frames can be further screened. Ignore video frames in which key objects are not detected, and only retain video frames containing key objects, thereby optimizing the selection of video key frames.

Furthermore, image compression in preprocessing is to effectively compress video frame images that consume large amounts of data resources and reduce image storage space and size. That is: the image generation model is used to compress the video frame image, thereby reducing the storage space consumption of the input image; at the same time, a variety of down-sampling algorithms are used to fuse the image to process the image, thereby reducing the size and computational complexity of the input image.

Step S102: Features are extracted from the video frames preprocessed in step S101 according to people, vehicles and hazard sources, and classification and recognition training is performed. This step specifically takes the following approach:

Feature extraction is to extract key frame image feature information. First, scale the preprocessed video frame images to reduce the computational complexity of high-resolution images; then use the scale-normalized images for training data. A model that can be used to detect and identify different key targets. After the training is completed, multiple models are implanted into the edge computing chip. The chip can use a high-performance industrial-grade processor. Its plug-in module expands the number of parallel applications and can support simultaneous Detection of multiple key targets and feature extraction tasks for key behaviors of gas station management such as special operations, refueling, and unloading. Feature extraction can include human features, vehicle features, and hazard source features. Among them, personnel characteristics can include personnel target detection and tracking, personnel action characteristics, personnel local attribute characteristics, etc.

Furthermore, human target detection and tracking can use the YOLO-DeepSort model to identify the movement trajectory of human targets for subsequent judgment of the intention of their trajectory.

Furthermore, human action feature detection can use the HRNet model as the recognition reference network, which can extract the positions of 17 key skeleton points of the human body for detection and recognition. Specifically, you can pay attention to whether there are obvious posture changes in the movements of the person. The angle between the trunk, legs, and arms can be used to represent the specific movements of the human body. For example, assuming a certain joint point is O, and the joint points connected to it are A and B respectively, then the joint angle there can be expressed by the following formula:

Assuming that an action consists of n spatial angles, a frame of image can be expressed as [θ ₁ , θ ₂ , θ ₃ ,...θ _n ]t ₁ , using the labeled human action video frames as the training set, training The action feature detection model provides detailed actions in the video sequence for analysis.

Furthermore, the identification of local attribute features of personnel can be based on the aforementioned detection and tracking of personnel targets, selecting the video frames in which the personnel targets are detected, and further judging their local attribute feature information to determine whether protective measures are in place, that is, whether to wear protective masks, wear Safety helmets analyze clothing characteristics to determine the identity of personnel (including staff, guardians and non-staff), such as whether they are wearing staff uniforms, whether they are wearing guardians' reflective vests, armbands, etc.

Further, vehicle detection and identification is used to identify vehicles working in the refueling area and unloading area, thereby realizing the task of monitoring the refueling and unloading of oil at each point of the gas station. Taking oil unloading vehicles as an example, we collect video image data containing oil unloading vehicles, use LabelImg annotation software to mark the video frames with detection frames of oil unloading vehicles, and label them as oil unloading vehicles; select the target detection network YOLOV5 as the target for identification of oil unloading vehicles. Baseline network; uses annotated vehicle video frames as a training set to train the vehicle recognition model.

Furthermore, hazard source detection classification is used to identify potentially risky objects such as tank mouths, operating sparks, and cranes. The target detection network YOLOV5 is selected as the benchmark network for hazard identification; the labeled video frames of the target of interest are used as training sets to train the hazard detection classification model.

Step S103: Based on the model trained by classification recognition, the personnel characteristics and hazard source characteristics, personnel characteristics and vehicle characteristics are respectively fused, and violation alarms are issued through personnel trajectory intention analysis and personnel identity analysis. This step specifically takes the following approach:

The illegal behavior alarm in this step mainly analyzes the characteristics extracted in step S102 by further analyzing the trajectory intention of the person and classifying the identity of the person. That is, after the characteristics of people (including personnel target detection and tracking, personnel action characteristics, and personnel local attribute characteristics), vehicles, and hazard sources are extracted and trained, the extracted features of different types are further combined and fused for better implementation. Call the police for subsequent violations.

Furthermore, the person trajectory intention analysis can be calibrated around the pixel set of the human body detected by the model. Let the pixel set where the midpoint of the feet is located be P={(x1,y1),(x2,y2),,,,( xp, yp)}, form the action trajectory of the human target, which is judged based on the direction and distance from the danger source. If the person's trajectory has the intention of approaching the danger source target, it is determined that the personnel target is approaching the danger source in a certain direction at this time. That is, at this time, the personnel trajectory intention analysis combines and fuses the personnel characteristics and hazard source characteristics, and comprehensively judges the situation through the combination and fusion of the personnel's movement trajectory and direction, the personnel's local characteristic attributes, and the distance to the hazard source. The intent of the person and the circumstances of the violation.

Furthermore, personnel identity analysis can be based on the local identity attribute characteristics detected by the model. Personnel wearing work clothes are judged to be staff, people wearing work clothes and reflective vests are judged to be guardians, and people not wearing work clothes are judged to be guardians. For non-staff members. For example, combined with the above personnel identity analysis results, when the arrival of a vehicle in the refueling area is identified, the personnel targets around the vehicle are continuously monitored and the personnel identity is classified. Within the set time threshold, if no staff is detected approaching the vehicle, it is considered " "The vehicle has arrived but the people have not arrived" to conduct a behavioral alarm; when the staff is recognized, the staff's action characteristic detection results are used to determine whether the vehicle is guided in accordance with the prescribed vehicle guiding posture. If no vehicle guiding behavior occurs, it is regarded as "work" Personnel failed to guide the vehicle in accordance with the regulations" to call the police. That is, at this time, the personnel trajectory intention analysis combines and fuses the characteristics of the personnel and the characteristics of the vehicle, and comprehensively determines whether there is a violation through the movement trajectory and response time of the personnel, the local characteristic attributes of the personnel, and the combination and fusion with the characteristics of the vehicle.

In this step, feature fusion is performed after obtaining the recognition results of each model and matched with the violation scene rules, thereby achieving more accurate judgment and triggering the identification alarm of real violations.

The image recognition method for high-risk gas station operations provided by this embodiment uses feature extraction and feature fusion to not only classify and individually identify personnel targets, vehicle targets, and hazard source targets, but also detect violations through the combination and fusion of target features. Behavioral alarm; In this embodiment, during the pre-processing process after video collection, a method based on similarity discrimination between previous and following frames is used to select video key frames, and a target detection algorithm based on a convolutional neural network is combined to further optimize the selection of key frames, which can effectively Remove redundancy and improve computing efficiency; this embodiment compresses video frames that consume large data resources, reducing image quality, length and width, and effectively reducing the size of data; this embodiment can embed lightweight recognition models into edge analysis In the hardware chip, local features are extracted based on key point positions, which can quickly deduplicate and select key frames, effectively improving the calculation speed and accuracy of the algorithm; by normalizing the scale of the image in the model training stage, it can effectively reduce high-resolution images. rate image computational complexity.

Example 2

As shown in Figure 2, this embodiment provides a high-risk operation image recognition device for a gas station, including: a collection and preprocessing module 201, a feature extraction and training module 202, and a feature fusion and alarm module 203. Among them, the collection and preprocessing module 201 is used to collect original videos of the gas station area, and perform data preprocessing on video frames containing key monitoring targets based on lightweight convolutional neural networks; the feature extraction and training module 202 is used to convert the preprocessed Features of the processed video frames are extracted according to people, vehicles and hazard sources, and classification and recognition training is performed; the feature fusion and alarm module 203 is used to combine the characteristics of personnel and hazard source characteristics, characteristics of personnel and vehicles based on the trained model of classification recognition. Features are fused separately, and violation alarms are made through personnel trajectory intention analysis and personnel identity analysis.

The image recognition device for high-risk operations in a gas station in this embodiment corresponds to the method in Embodiment 1 and can achieve the same technical effect.

Example 3

Figure 3 is a schematic diagram of the hardware structure of the image recognition electronic device for high-risk operations in a gas station in this embodiment. The device (eg, terminal, server, etc.) includes one or more processors 610 and memory 620. Taking a processor 610 as an example, the device may also include: an input device 630 and an output device 640.

The processor 610, the memory 620, the input device 630 and the output device 640 may be connected through a bus or other means.

As a non-transitory computer-readable storage medium, the memory 620 can be used to store non-transitory software programs, non-transitory computer executable programs and modules. The processor 610 executes various functional applications and data processing of the electronic device by running non-transitory software programs, instructions and modules stored in the memory 620, that is, implementing the processing method of the above method embodiment.

The memory 620 may include a program storage area and a data storage area, where the program storage area may store an operating system and an application program required for at least one function; the data storage area may store data and the like. In addition, memory 620 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 620 optionally includes memory located remotely relative to processor 610, and these remote memories may be connected to the processing device through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

Input device 630 may receive input numeric or character information and generate signal input. The output device 640 may include a display device such as a display screen.

The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, the following steps are performed: A. Collect the original video of the gas station area, and perform the processing based on lightweight convolution The neural network performs data preprocessing on video frames containing key monitoring targets; B. Features are extracted from the preprocessed video frames according to people, vehicles and hazard sources, and classification and recognition training is performed; C. The trained model is based on classification recognition , conduct feature fusion of personnel characteristics and hazard source characteristics, personnel characteristics and vehicle characteristics respectively, and conduct violation alarms through personnel trajectory intention analysis and personnel identity analysis.

The above-mentioned electronic device can execute the method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method. For technical details that are not described in detail in this embodiment, please refer to the methods provided by other embodiments of the present invention.

The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions can be embodied in the form of software products in essence or in part that contribute to related technologies. The computer software products can be stored in computer-readable storage media, such as ROM/RAM, disks. , optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Example 4

This embodiment provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer-executable instructions. The instructions are used to cause the computer to execute the following high-risk operation image recognition method at a gas station: A. , collect the original video of the gas station area, and perform data preprocessing on the video frames containing key monitoring targets based on a lightweight convolutional neural network; B. Extract features of the preprocessed video frames according to people, vehicles and hazard sources , and carry out classification and recognition training; C. Based on the model after classification and recognition training, the personnel characteristics and hazard source characteristics, personnel characteristics and vehicle characteristics are respectively fused, and illegal behavior alarms are carried out through personnel trajectory intention analysis and personnel identity analysis.

The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and illustration. These descriptions are not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teachings. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical applications, thereby enabling others skilled in the art to make and utilize various exemplary embodiments of the invention and various different applications. Choice and change. Any simple modifications, equivalent changes and modifications made to the above exemplary embodiments should fall within the protection scope of the present invention.

Claims (12)

1. The high-risk operation image recognition method for the gas station is characterized by comprising the following steps of:

A. collecting an original video of a gas station area, and carrying out data preprocessing on a video frame containing a key monitoring target based on a lightweight convolutional neural network;

B. extracting the characteristics of the preprocessed video frames according to personnel, vehicles and dangerous sources, and performing classification recognition training;

C. and respectively carrying out feature fusion on the personnel features, the dangerous source features, the personnel features and the vehicle features according to the model after classification and recognition training, and carrying out illegal behavior alarm through personnel track intention analysis and personnel identity analysis.

2. The method for recognizing high risk operation images of gas stations according to claim 1, wherein the feature extraction and recognition training of the person in step B specifically comprises:

detecting and tracking personnel in the video frame by identifying the action track of the personnel target;

detecting the action characteristics of personnel in the video frame through the positions of key skeleton points of the human body;

and judging whether the protective measures of the personnel are in place or not and judging the identity of the personnel through the local attribute feature identification of the personnel.

3. The method for recognizing high risk operation images of gas stations according to claim 1, wherein the feature extraction and recognition training of the vehicle in the step B specifically comprises:

identifying vehicles in the oiling area and the oil unloading area and marking the oil unloading vehicles; and performing model training by using the marked oil discharge vehicle video frames.

4. The method for recognizing high risk operation images of gas stations according to claim 1, wherein the feature extraction and recognition training of the hazard source in the step B specifically comprises:

and identifying and marking object targets with potential risks of the tank mouth of the tank area, the operation electric spark and the crane, and performing model training by using the marked object target video frames.

5. The method for recognizing a high risk job image of a gas station according to claim 1, wherein the feature fusion of the personnel feature and the risk source feature in the step C specifically comprises:

calibrating the periphery of the detected human body pixel set of the person to form a movement track of the person target;

and judging whether the personnel target is approaching to the dangerous source according to the direction of the action track and the distance between the personnel target and the dangerous source.

6. The method for recognizing a high risk job image of a gas station according to claim 1, wherein the feature fusion of the personnel features and the vehicle features in the step C specifically comprises:

when the arrival of the vehicle is detected, continuously monitoring personnel targets around the vehicle and classifying personnel identities;

and judging whether a worker approaches the vehicle within a preset time threshold, and judging whether to carry out regular vehicle guiding according to the action characteristic detection result of the worker.

7. The method for identifying high risk operation images of gas stations according to claim 1, wherein the step C of personnel identity analysis specifically comprises:

and judging whether the personnel are working personnel, guardianship personnel or non-working personnel according to the dressing characteristics in the local personnel attributes.

8. The method for recognizing a high risk job image according to claim 1, wherein the data preprocessing in the step a includes key frame extraction and image compression.

9. The method for identifying a high risk job image of a gas station according to claim 8, wherein the key frame extraction specifically comprises:

taking a first frame of the original video as a starting frame, extracting local characteristics of the starting frame, and setting a corresponding threshold according to the local characteristics;

extracting local characteristic points of a subsequent video frame in sequence, comparing the local characteristic points with the characteristic points of the initial frame, and setting the frame as an end frame when the number of dissimilar characteristic points is greater than the threshold value;

acquiring a frame with the most similar points with the initial frame and the end frame between the initial frame and the end frame as a candidate key frame of the original video;

and sequentially inputting the candidate key frames into the lightweight convolutional neural network, and detecting and retaining the candidate key frames containing the key monitoring targets.

10. A high risk job image recognition apparatus for a gas station, comprising:

the acquisition and preprocessing module is used for acquiring an original video of a gas station area and preprocessing data of a video frame containing a key monitoring target based on a lightweight convolutional neural network;

the feature extraction and training module is used for extracting the features of the preprocessed video frames according to personnel, vehicles and dangerous sources and performing classification recognition training;

and the feature fusion and alarm module is used for respectively carrying out feature fusion on the personnel features, the dangerous source features, the personnel features and the vehicle features according to the model after classification and recognition training, and carrying out illegal behavior alarm through personnel track intention analysis and personnel identity analysis.

11. A high risk job image recognition electronic device of a gas station, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the gas station high risk job image recognition method of any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer-executable instructions for causing the computer to perform the gas station high risk job image recognition method according to any one of claims 1 to 9.