CN118917836A - Intelligent electric power inspection system and method with audio and video integrated - Google Patents

Abstract

The invention discloses an audio-video integrated intelligent power inspection system and an audio-video integrated intelligent power inspection method, wherein the intelligent inspection system comprises front-end sensing equipment, an edge computing integrated machine and intelligent inspection center service software; the inspection method comprises the following steps: the front-end sensing equipment sends the acquired audio and video data of the power production equipment to the edge computing integrated machine, and after the analysis of the edge computing integrated machine, the intelligent computing integrated machine performs recognition processing through an intelligent algorithm to form an intelligent inspection result and uploads the intelligent inspection result to the intelligent inspection center service software; the intelligent inspection center service software stores the inspection result in a database for viewing and downloading. The intelligent inspection system and the intelligent inspection method provided by the invention realize the automation of daily inspection of electric power, replace the traditional manual inspection mode, improve the inspection efficiency and the inspection quality, reduce the workload of inspection personnel, greatly shorten the inspection period of power generation enterprises and systematically solve the problem of pain points of manual inspection of the power generation enterprises.

Description

An audio and video integrated power intelligent inspection system and method

Technical Field

The present invention belongs to the field of electric power intelligent inspection, and specifically discloses an audio-video integrated electric power intelligent inspection system and method.

Background Art

As the scale of my country's power system continues to expand, installed capacity continues to increase, and digital equipment for power production is rapidly updated and developed, the complexity increases. The scope of daily inspections of power generation companies has expanded, the inspection content has become more cumbersome, and inspectors with different skill levels find different problems during the inspection process, resulting in a lack of objectivity in the inspection results. Manual inspection methods have the problems of low inspection efficiency and uneven inspection quality. Digital and intelligent technical means are one direction to solve the problem of manual inspection.

Intelligent inspection is an important area of digital transformation and development for power companies. It uses artificial intelligence, edge computing, the Internet of Things and other technologies to build an intelligent inspection system based on automatic inspection to replace the traditional manual inspection method. The front-end camera, robot and other devices are used to collect on-site video images. The data is processed by an intelligent recognition algorithm based on deep learning to identify the status of the equipment, find problems in time, improve the management level of power production and operation, and improve the economic benefits of power generation companies. At present, the power inspection method based on industrial vision has certain limitations. It is impossible to identify the abnormal status of large rotating equipment such as feed pumps and turbines, and it is difficult to monitor the abnormal operation sound of power equipment such as main transformers and switch cabinets.

In view of this, it is necessary to improve the problems in the prior art to solve the above problems.

Summary of the invention

Purpose of the invention: The present invention discloses an audio and video integrated electric power intelligent inspection system and method, the system includes a front-end sensing device, an edge computing all-in-one machine and an intelligent inspection center service software, wherein the edge computing all-in-one machine is respectively connected to the front-end sensing device and the intelligent inspection center service software; the intelligent inspection method is adopted to replace the manual inspection method, which not only solves the problems of low efficiency and uneven inspection quality of manual inspection, but also breaks through the limitations of the electric power inspection method based on industrial vision based on the intelligent recognition algorithm processing based on deep learning, and can discover and solve many difficult problems of abnormal conditions such as instrument readings, switch status, temperature measurement, leakage, etc.

To achieve the above object, the present invention provides the following technical solutions:

An audio and video integrated power intelligent inspection system, including front-end sensing equipment, edge computing all-in-one machine and intelligent inspection center service software;

The edge computing integrated machine is respectively connected to the front-end sensing device and the intelligent inspection center service software;

The front-end sensing device is used to collect audio and video data representing the operating status and index parameters of the power equipment and send them to the edge computing integrated machine.

The edge computing integrated machine is used to analyze and process the data collected by the front-end sensing device, form inspection results, and upload the inspection results to the intelligent inspection center service software;

After receiving the inspection results, the intelligent inspection center service software stores the inspection results in a database for viewing and downloading.

Furthermore, the front-end sensing equipment includes a high-definition camera, an infrared dual-spectrum camera, an audio sensor, a vibration sensor, a patrol robot and a patrol drone.

Furthermore, the high-definition camera and infrared dual-spectrum camera are used to collect equipment status data of main transformers, circuit breakers, switches, mutual inductors in the station, as well as the nacelle and tower base of the wind turbine, and equipment status data in the main plant and electronic room of the thermal power plant.

Furthermore, the audio sensor and vibration sensor are used to collect status data of various pumps in the pump room of a gas turbine power plant, coal mills in a thermal power plant, and status data of turbines in a turbine room of a hydropower station.

Furthermore, the inspection robots and inspection drones are used to collect status data of switches and measurement and control equipment in switch rooms and relay protection rooms.

Furthermore, the edge computing all-in-one machine is a computing device deployed on the edge side for data collection and intelligent analysis based on domestic GPU/NPU chips, integrating video data collection components, audio data collection software, audio and video data feature extraction software, as well as instrument readings, switch status, temperature measurement and leakage intelligent identification algorithms.

Furthermore, the intelligent inspection center service software has four application modules: inspection tasks, alarm center, inspection management, and system management;

The inspection task module is used for regular inspections, inspection task details viewing, inspection task management, and supports downloading inspection result reports to the local computer;

The alarm center module is used to display inspection alarms, view the details of inspection alarms, and issue alarm processing instructions based on the details of inspection alarms;

The inspection management module is used to add, edit or delete the task bar in the inspection task module;

The system management module is used to add, edit or delete relevant information of the front-end sensing device, and can remotely view the real-time image of the camera.

Furthermore, the system management module has permission management, user organization, process engine basic services and components;

The inspection task, alarm center and inspection management function module respectively integrate relational database, object storage library and middleware, and are used to store system parameters, inspection results and cache data respectively.

An audio and video integrated power intelligent inspection method includes the following inspection steps:

S1. Configure regular inspection tasks in the intelligent inspection center service software, and send inspection task instructions to the edge computing all-in-one machine via the TCP-based remote transmission interface;

S2. After receiving the inspection task instruction, the edge computing machine reads the front-end perception data of the camera, audio sensor and vibration sensor according to the inspection points. The front-end perception data flows to the intelligent algorithm software inside the edge computing machine, and the intelligent algorithm recognizes the audio and video data after data preprocessing.

S3. The result data after the intelligent algorithm recognition is completed is uploaded to the intelligent inspection center service software through the TCP-based remote transmission channel. The algorithm recognition result is based on the JSON format, including the inspection point, result image, and voiceprint feature value information;

S4. After receiving the returned inspection results, the intelligent inspection center service saves the image and audio result data in the object repository, generates a storage path link and saves the inspection result information in the relational database. The inspection result data can be viewed through the inspection task viewing page, and supports PDF report download.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention discloses an audio and video fusion intelligent inspection system and method, the system includes a front-end sensing device, an edge computing integrated machine and an intelligent inspection center service software, wherein the edge computing integrated machine is respectively connected to the front-end sensing device and the intelligent inspection center service software; the system uses front-end sensing devices such as high-definition cameras, infrared dual-spectrum cameras, audio sensors, etc. to collect audio and video multimodal data at the power production site, and the data is uploaded to the edge computing integrated machine. The data is analyzed and processed by the built-in intelligent algorithm of the edge computing integrated machine to form an inspection result, and the inspection result is uploaded to the intelligent inspection center service software. After receiving the inspection result, the intelligent inspection center service software stores the inspection result in an object-based database for retrieval, viewing and downloading at any time. By using audio and video sensing technology, intelligent recognition technology based on deep learning, and developing an intelligent inspection center service based on a microservice framework, the automation of daily power inspection is realized, replacing the traditional manual inspection method.

In the present invention, video data is processed by the intelligent recognition algorithm based on deep learning to identify the equipment status and environmental status, audio data is processed based on voiceprint feature extraction and AI recognition technology, and the edge computing all-in-one machine integrates domestic GPU/NPU chips to provide an efficient computing power foundation. The present invention realizes 24-hour uninterrupted automated continuous inspection of power production equipment and its operating environment, breaking through the limitations of the power inspection method based on industrial vision, and can discover and solve many problems of abnormal conditions such as instrument readings, switch status, temperature measurement, leakage, etc. It improves the inspection efficiency and quality, reduces the workload of personnel, greatly shortens the inspection cycle of power generation enterprises, and systematically solves the pain points of manual inspection methods of power generation enterprises.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 Architecture of the audio and video fusion power intelligent inspection system;

Figure 2 Audio and video fusion intelligent inspection process;

Figure 3. Instrument reading model.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Please refer to Figures 1-2. The present invention provides an audio and video integrated power intelligent inspection system and method. The intelligent inspection system architecture is shown in Figure 1. The intelligent inspection system described in the present invention is mainly aimed at the field of power inspection, integrating technologies such as the Internet of Things, edge computing, artificial intelligence, and microservices to create an edge computing all-in-one machine and build an intelligent inspection software system. By configuring front-end sensing devices such as high-definition cameras, infrared thermal imaging cameras, audio sensors, robots, and drones, the collected audio and video multimodal data is analyzed by the intelligent recognition algorithm deployed in the edge computing all-in-one device to automatically identify the power generation equipment and environmental status. The inspection results are uploaded to the intelligent inspection center application for storage and display, realizing intelligent inspection instead of manual inspection.

The system comprises:

F1. Front-end sensing equipment, mainly including audio and video sensing terminals such as visible light cameras, infrared thermal imaging cameras, sound sensors, vibration sensors, etc.; introduce IoT sensing technology, use front-end sensing terminals such as high-definition visible light cameras, infrared thermal imaging cameras, audio sensors, vibration sensors, inspection robots and inspection drones to develop audio and video multi-modal data collection and data processing software;

The high-definition camera and infrared dual-spectrum camera are used to collect equipment status data of the main transformer, circuit breaker, wind turbine cabin, tower base in the booster station, and equipment status data in the main plant of the thermal power plant;

The audio sensor and vibration sensor are used to collect the status data of various pumps in the pump room of the gas turbine power plant and the status data of the turbines in the turbine room of the hydropower station;

The inspection robots and inspection drones are used to collect status data of switches and measurement and control equipment in switch rooms and relay protection rooms.

F2, edge computing all-in-one machine, is a computing device for edge-side data collection and intelligent analysis based on domestic GPU/NPU chips. It supports AI frameworks such as pytorch and tensorflow, provides basic support for the operation of intelligent recognition algorithms, and integrates intelligent recognition algorithm modules for audio and video fusion. The video data acquisition component is developed based on the computer vision library OpenCV, and the audio data acquisition software is developed based on the audio processing library Soundfile. It integrates intelligent algorithms such as instrument readings, switch status, temperature measurement, leakage, etc.

Among them, various intelligent algorithms such as instrument reading, switch status, temperature measurement, leakage, etc. integrate the classification algorithms of the yolo framework and resnet. Among them, for the instrument reading analysis algorithm based on visible light, the target detection algorithm and the dial key point prediction algorithm are combined to realize the automatic recognition and reading of the instrument. First, collect on-site video data, and establish a pointer instrument target detection data set and a pointer instrument key point prediction data set; then, perform target detection training and key point prediction training respectively; then, first detect and locate the dial, and predict the key points to obtain the scale and pointer key point coordinates; finally, calculate the reading by the angle method. The instrument reading model based on visible light is shown in Figure 3;

For the temperature measurement algorithm of the high-voltage side joint of the main transformer, the detection method is to continuously collect infrared dual-spectrum camera video images of key areas and key points of the equipment, and obtain temperature measurement data such as the maximum, minimum, and average values of the infrared thermal imaging temperature of key areas and key points.

Among them, the instrument reading analysis algorithm based on visible light is combined with the target detection algorithm and the dial key point prediction algorithm to realize the automatic recognition and reading of the instrument. The algorithm training and reasoning process is as follows: first, collect the on-site video data, and establish the pointer instrument target detection data set and the pointer instrument key point prediction data set; then, perform target detection training and key point prediction training respectively; then, first target detection locates the dial, and key point prediction obtains the scale and pointer key point coordinates; finally, calculate the reading by the angle method.

The switch status analysis algorithm based on visible light is based on the ResNet classification algorithm and adapted to domestic GPU/NPU chips. Equipment status models such as substation switches and switch status lights are constructed to achieve efficient identification of equipment status.

Based on the small sample leakage algorithm, the damage of the mechanical structure of the equipment can directly affect the normal operation of the unit. By using a small amount of field data samples, supplemented by small sample data processing and training, water (oil) leakage detection of key equipment can be achieved. The steps are:

Data collection: Collect as many device leakage images as possible through on-site photography and Internet data crawling, and manually annotate them;

Data augmentation: augment the annotated water (oil) leakage images through traditional image geometric transformation methods (including but not limited to rotation, flipping, scaling, cropping, adding noise, color space conversion, etc.);

Data generation: Generate fake water and oil leakage images through adversarial generative networks to further expand the training data set;

4) Algorithm training: Use all collected, amplified, and generated data to train the target detection algorithm so that it has the ability to accurately locate water (oil) leaks.

F3, intelligent inspection center service software, developed based on the Springboot framework, has four application functions: inspection tasks, alarm center, inspection management, and system management. It has basic services and components such as permission management, user organization, and process engine. It integrates relational database, object repository, and middleware to store system parameters, inspection results, cache and other data.

This intelligent inspection center service software builds an intelligent inspection basic platform based on the microservice architecture, with redis as the middleware, postgressql as the relational database, and minio as the object repository. It develops basic services such as permission management and user management, introduces activiti as the process engine, and Quartz as the inspection task scheduling management engine, providing a basis for the smooth operation of each process of intelligent inspection; it develops intelligent inspection application functions, which have four application functions: inspection tasks, alarm center, inspection management, and system management;

Among them, the inspection task function supports the retrieval and details viewing of regular inspection tasks, and supports the downloading of inspection reports to the local computer. The alarm center function supports the detection, details viewing, and alarm processing of inspection alarms, and provides an API interface to support the push of alarms to other systems. The inspection management function supports the addition, editing, and deletion of inspection tasks, and the system management function supports the addition, editing, and deletion of sensing devices such as front-end cameras, and can remotely view the real-time images of cameras.

Table 1 List of intelligent inspection application function pages

;

Based on the above inspection system, an audio and video fusion power intelligent inspection method is developed, which includes the following steps:

S1. Configure regular inspection tasks in the intelligent inspection center service software, and send inspection task instructions to the edge computing all-in-one machine via the TCP-based remote transmission interface;

S2. After receiving the inspection task instruction, the edge computing machine reads the front-end perception data of the camera, audio sensor and vibration sensor according to the inspection points. The front-end perception data flows to the intelligent algorithm software inside the edge computing machine, and the intelligent algorithm recognizes the audio and video data after data preprocessing.

S3. The result data after the intelligent algorithm recognition is completed is uploaded to the intelligent inspection center service software through the TCP-based remote transmission channel. The algorithm recognition result is based on the JSON format, including the inspection point, result image, and voiceprint feature value information;

S4. After receiving the returned inspection results, the intelligent inspection center service saves the image and audio result data in the object repository, generates a storage path link and saves the inspection result information in the relational database. The inspection result data can be viewed through the inspection task viewing page, and supports PDF report download.

After the edge computing all-in-one machine described in the inspection method steps of this embodiment receives the inspection task instruction, it reads the front-end perception data of the camera, audio sensor, vibration sensor, etc. according to the inspection point, and the front-end perception data flows from the edge computing all-in-one machine to the intelligent algorithm software. The collection of its audio and video data mainly uses the Internet of Things technology to convert the analog signals obtained by the audio sensor and vibration sensor into digital signals and transmit them to the edge computing all-in-one machine, where the audio data is in wav format. The camera video data is pushed to the edge computing all-in-one machine based on protocols such as GB/T28181. The data acquisition software integrated in the edge computing all-in-one machine collects pictures based on OpenCV, and the pictures are sent to the intelligent algorithm for recognition.

The present invention provides an audio and video fusion intelligent inspection system and method for power generation enterprises, which mainly utilizes high-definition cameras, infrared dual-spectrum cameras, audio sensors, vibration sensors and other intelligent sensing terminals to collect data such as the operating status and index parameters of power equipment, and timely discovers equipment abnormalities and safety hazards through data processing means such as intelligent analysis and numerical calculation, and realizes an intelligent inspection mode mainly supported by intelligent equipment and software systems, which is applied to the daily inspection business of power generation enterprises such as gas turbine power plants, hydropower plants, and coal-fired power plants.

Compared with the prior art, the beneficial effects of the audio and video integrated power intelligent inspection system and method of the present invention are: introducing Internet of Things perception technology, introducing voiceprint perception technology on the basis of traditional industrial vision, and applying it to power intelligent inspection, creating an edge computing all-in-one machine based on domestic GPU/NPU chips, completing the domestic software and hardware adaptation of yolo and restnet algorithms, integrating intelligent algorithms such as instrument readings, switch status, temperature measurement, leakage, etc.; and constructing an audio and video integrated intelligent inspection application with four main functions: inspection tasks, alarm center, inspection management, and system management; solving the limitations of traditional intelligent inspections that rely only on industrial vision, and expanding the voiceprint-based monitoring methods for rotating equipment and primary power equipment of power generation enterprises.

It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims (9)

1. The intelligent electric power inspection system with the audio and video integrated functions is characterized by comprising front-end sensing equipment, an edge computing integrated machine and intelligent inspection center service software;

the edge computing integrated machine is respectively connected with front-end sensing equipment and intelligent inspection center service software;

The front-end sensing equipment is used for collecting audio and video data representing the running state and index parameters of the power equipment and sending the audio and video data to the edge computing integrated machine,

The edge computing integrated machine is used for analyzing and processing the data acquired by the front-end sensing equipment to form a patrol result, and uploading the patrol result to the intelligent patrol center service software;

after the intelligent inspection center service software receives the inspection result, the inspection result is stored in a database for viewing and downloading.

2. The intelligent power inspection system with audio and video integration according to claim 1, wherein the front-end sensing device comprises a high-definition camera, an infrared double-spectrum camera, an audio sensor, a vibration sensor, an inspection robot and an inspection unmanned aerial vehicle.

3. The intelligent inspection system of audio and video integrated power according to claim 2, wherein the high-definition camera and the infrared double-spectrum camera are used for collecting equipment state data of main transformers, circuit breakers, cabins of wind driven generators and tower foundations in the booster station and equipment state data in a main plant of a thermal power plant.

4. The intelligent power inspection system with audio and video integration according to claim 2, wherein the audio sensor and the vibration sensor are used for collecting state data of various pumps between the power plant and the pump and state data of indoor water turbines of the hydropower station.

5. The intelligent power inspection system with audio and video integration according to claim 2, wherein the inspection robot and the inspection unmanned aerial vehicle are used for collecting state data of a switch room, a relay protection indoor switch and a measurement and control device.

6. The intelligent power inspection system with audio and video fusion according to claim 1, wherein the edge computing integrated machine is an intelligent power computing device which is deployed on the edge side and is based on data acquisition and intelligent analysis of a domestic GPU/NPU chip, and an intelligent recognition algorithm of a video data acquisition component, audio data acquisition software, meter reading, switching state, temperature measurement and running, and leakage is integrated.

7. The audio-video integrated power intelligent patrol system according to claim 1, wherein the intelligent patrol center service software comprises four application modules of patrol task, alarm center, patrol management and system management;

the inspection task module is used for checking details of the inspection task at regular intervals, managing the inspection task and supporting the downloading of an inspection result report to the local;

The alarm center module is used for displaying the patrol alarm, checking the details of the patrol alarm and making an alarm processing instruction according to the details of the patrol alarm;

The inspection management module is used for adding, editing or deleting task bars in the inspection task module;

the system management module is used for adding, editing or deleting the related information of the front-end sensing equipment, and can remotely view the real-time picture of the camera.

8. The intelligent power inspection system with audio and video integration according to claim 7, wherein the system management module comprises rights management, user organization, process engine basic services and components;

The patrol task module, the alarm center module and the patrol management module are respectively integrated with a relational database, an object storage library and middleware and are respectively used for storing system parameters, patrol results and cache data.

9. The intelligent power inspection method with audio and video integration is characterized by comprising the following inspection steps of:

S1, configuring a periodic inspection task in intelligent inspection center service software, and sending an inspection task instruction to an edge computing integrated machine through a TCP-based remote transmission interface;

s2, after the edge computing integrated machine receives the inspection task instruction, respectively reading front end sensing data of the camera, the audio sensor and the vibration sensor according to inspection point positions; the front-end perception data flows to intelligent algorithm software in the edge computing integrated machine, and the intelligent algorithm identifies the audio and video data after data preprocessing;

s3, the intelligent algorithm identifies the finished result data, and the result data is uploaded to intelligent inspection center service software through a remote transmission channel based on TCP, and the algorithm identification result is based on a JSON format and contains inspection point positions, result pictures and voiceprint characteristic value information;

and S4, after receiving the returned inspection result, the intelligent inspection center service stores the picture and audio result data in an object storage library, generates a storage path link and inspection result information to be stored in a relational database, and the inspection result data can be checked through an inspection task check page to support PDF report downloading.