CN103916632B - Real-time panorama video remote monitoring system for runway - Google Patents

Abstract

A real-time panoramic video remote monitoring system for airstrips belongs to the technical field of real-time panoramic video remote monitoring for airstrips, and is characterized in that it includes: a network camera set arranged on an airport tower, a wireless router and a graphics workstation arranged in series in a computer room, an Ethernet network and a remote monitoring center located at the client; the wireless router inputs multiple real-time video images from a group of network cameras in a wired manner, and the program in the graphics workstation inputs a group of multiple real-time video streams wirelessly from the wireless router , use the existing real-time image stitching software or improved real-time video stream stitching software to stitch multiple input real-time images, and get the smooth transition at the stitching line and the registered images are sent to the remote monitoring center via Ethernet, which is different from the traditional Compared with the analog-digital monitoring system DVR, the present invention has the advantages of remote transmission, simple wiring, good real-time performance, panoramic monitoring of the runway and fast image processing speed.

Description

A real-time panoramic video remote monitoring system for airstrip

technical field

The invention relates to a panoramic video remote monitoring system, in particular to a real-time panoramic video remote monitoring system for an aircraft runway

Background technique

Runway safety is a big aviation safety issue. The video surveillance of the runway is very important. With the development of science and technology, the video surveillance system currently used on the runway is mainly "analog-digital" monitoring system (DVR). Meet people's needs in monitoring and management. The "analog-digital" monitoring system is based on the digital hard disk video recorder DVR as the core, a semi-analog-semi-digital solution. From the camera to the DVR, a coaxial cable is used to output video signals. The DVR supports both video recording and playback, and supports network access. , due to the variety of DVR products, there is no standard, so this generation of systems is a non-standard closed system, and there are still a lot of limitations in the DVR system. There are some obvious disadvantages in this product: (1) The transmission distance is limited, which is mainly due to the limited transmission distance of the coaxial cable which affects the transmission distance of the video signal. (2) The wiring is complicated, and each camera must be installed with a separate coaxial cable, resulting in complicated wiring. (3) It is not suitable for remote monitoring, and the real-time performance of remote monitoring is poor, and there is a high network delay. (4) Due to the limited monitoring range of each camera, monitoring the entire runway needs to watch multiple videos at the same time, which is cumbersome to monitor. (5) Multi-channel videos are stored separately, which is not conducive to playback and search for problems. (6) The system deployment is difficult, and a video capture card needs to be installed on the monitoring machine.

Contents of the invention

Purpose of the invention: In order to overcome the deficiencies of the prior art, the present invention provides a real-time panoramic video remote monitoring system for airstrips to realize remote centralized monitoring and control of airstrips.

The present invention includes: at least one group of network cameras, at least one group of wireless routers, at least one group of graphics workstations, Ethernet and at least one remote monitoring center; wherein:

The network camera group is placed on the airport tower, and each group of network cameras has at least one network camera, aiming at different positions on the airstrip;

The wireless router and graphics workstation are located in the computer room. Each wireless router corresponds to a group of network cameras. A graphics workstation is wirelessly connected to a wireless router to input real-time video images corresponding to different positions of an airport runway;

The graphics workstation is connected to Ethernet in a wired manner;

The remote monitoring center is each client PC connected to Ethernet in a wired manner;

Described graphics work station is a computer that is provided with wireless network card and wired network card at the same time, is provided with the NVIDIA graphics card based on parallel programming model instruction set architecture CUDA inside, according to the following steps successively from the multi-channel input of corresponding described network camera Live video image stitching:

Step (1), collecting a batch of video images of three paths partially overlapping in the horizontal direction;

Step (2), performing synchronous mutual exclusion control on the three-way video images, to ensure the correct splicing of the panoramic images of the video stream;

Step (3), judging the real-time performance of the three-way video images:

If it is currently offline, go to step (4),

If it is currently a real-time stage, then go to step (6);

Step (4), sequentially perform feature point extraction, description, matching and feature transformation on the three-way video image input from the offline stage, wherein: use the SURF operator to extract the feature points, and then use the RANSAC algorithm to solve the space transformation of the image space The model obtains 8 parameters of the homography matrix, thereby determining the projection position of a point on a three-dimensional plane in different two-dimensional images, so that two adjacent video images of the same group are registered respectively;

Step (5), sequentially perform color and brightness correction on the two adjacent video images after registration using the final gamma correction parameters, and find the best stitching line, and use the best stitching line to correct the two overlapped images. The video images after registration are panoramically synthesized. On this basis, the distance transformation function is used to calculate all the non-zero The block distance from the pixel point to the nearest zero pixel point adjacent to it, after a smooth transition, obtains the weighted fusion matrix of the two adjacent and partially overlapping video images corresponding to the panoramic image;

In step (6), the three-way real-time video images imported in the real-time stage are spliced in real time according to the following steps:

Use the final gamma correction parameters obtained in the offline stage to perform color brightness correction;

Invoke the CUDA based on the instruction set architecture of the parallel programming model, directly use the homography matrix calculated in the offline stage, realize multi-threaded concurrent image transformation calculation on the image processor NVIDIA GPU card, and register adjacent images input in real time, Determining overlapping regions and optimal stitching lines of the images, returning the results to the CPU of the computer;

The computer uses the weighted fusion matrix obtained in the offline stage to perform weighted fusion, and obtains three real-time video images with a smoother transition at the best seam line, and then uses the mature H264 compression algorithm to compress the stitched panoramic video images, Then output the compressed video stream to the remote monitoring center via Ethernet.

Beneficial effects: Compared with the existing runway video monitoring system, the utility model's real-time panoramic video remote monitoring system for the airstrip has the following advantages: the new airport real-time panoramic video remote monitoring system can make the personnel on duty and the commanding personnel remotely convenient It can monitor the airport runway, and the panoramic video has a better visual effect than the multi-channel video, which makes it more convenient to observe the situation of the airport runway, and it is easier to detect obstacles or abnormal objects on the runway and make corresponding processing. Losses are minimized. Moreover, this system can store the spliced panoramic video on the disk. When it needs to be played back, it can find the problem more quickly than multi-channel video. In addition, this system uses H264 encoder to compress the panoramic video, which can save storage resources and reduce transmission bandwidth requirements. But more importantly, multiple airstrips can be monitored at the same time in a remote monitoring center, thereby reducing monitoring costs, having good practicability, and can generate good economic and social benefits.

Description of drawings

Fig. 1 is a schematic structural diagram of a real-time panoramic video remote monitoring system for an airstrip runway disclosed by the present invention;

Fig. 2 is a schematic diagram of the deployment of the airstrip real-time panoramic video remote monitoring system disclosed in the present invention;

Fig. 3 is a traditional video surveillance display mode;

Fig. 4 is the schematic diagram that the present invention carries out real-time mosaic display to monitoring scene;

Fig. 5 is a schematic diagram of the workflow of the panoramic video remote monitoring system disclosed in the present invention.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings.

The technical scheme that the present invention adopts is as follows:

A real-time panoramic video remote monitoring system for an airstrip. The system equipment is composed of a network camera, a wireless router, and a graphics workstation. Multiple network cameras are connected to one wireless router by wire, and the wireless router and graphics workstation are connected by wireless. The graphics workstation collects the video data of multiple network cameras through the network, splices the video streams from multiple angles in different directions but partially overlapping the field of view, and uses advanced GPU parallel computing to accelerate the video splicing process to achieve real-time smoothness Purpose. Preview and store real-time panoramic video on a graphics workstation. The remote monitoring center establishes a connection with the system through Ethernet.

The network camera needs 2-4 channels, and the wireless router has 2-4 LAN ports for connecting the network camera.

The graphics workstation needs to use an NVIDIA GPU card, and the computing power of the GPU needs to be above 2.0, and it needs to have both a wireless network card and a wired network card.

As shown in Figure 1, the system equipment consists of network cameras, wireless routers, and graphics workstations. The network camera model used in this example is Hikvision DS-2CD883F-E(W), the wireless router model is TP-LINK TL-W841N, the graphics workstation uses Lenovo ThinkStation S30, and the graphics card is NVIDIA Quadro K600. The processor is Intel(R) Xeon(R) E5-1620 3.60GHz, 12GB memory, Windows7 64-bit operating system. The machine used by the remote control center is an ordinary PC.

There are 4 LAN ports on the router. The network camera is deployed on the tower and connected to the wireless router deployed in the computer room through a wired connection. There is a wireless network card and a wired network card on the graphics workstation, and the connection between the workstation and the wireless router is wireless. The graphics workstation is connected to Ethernet through a wired network card. The computer in the remote monitoring center is also connected to the Ethernet through a wired method.

After the graphics workstation is connected to the wireless router, firstly find the IP and port of each network camera. After determining the number of cameras, the IP, port and position sequence of each network camera, initialize the video splicing, and obtain the relative position of each camera and the corresponding transformation parameters.

Start real-time video stitching and port listening on workstations. The remote control center initiates a panoramic video request to the corresponding workstation. The entire workflow is shown in Figure 4. The panoramic stitching system deployed on the workstation first collects multiple videos, then stitches the multiple videos in real time, then uses the mature H264 compression standard to compress the panoramic video, and then uses the RTP protocol to compress the video. The final video data is packaged and sent. After receiving the RTP data packet sent by the server, the program deployed by the remote control center decodes it using the H264 standard, and then displays the decoded image on the screen.

The real-time panoramic video remote monitoring system of the airstrip has passed strict system tests and has high stability. This system is a video monitoring system that makes full use of advanced electronic technology and modern communication technology, and integrates functions such as real-time panoramic preview, screen capture, video compression, panoramic video playback, network real-time transmission, and remote monitoring. It can make real-time monitoring of multiple airstrips in one monitoring center. This system adopts advanced video stitching algorithm and GPU acceleration technology, which can implement friendly panoramic monitoring of monitoring scenes.

The airstrip panoramic video remote monitoring system is to monitor and manage each airstrip in a remote monitoring center. When there are obstacles or abnormal personnel on the airstrip, the management personnel can be notified immediately to take corresponding measures to prevent problems before they happen. , can more effectively avoid the occurrence of accidents. The present invention has the following innovative points:

(1) Remote centralized management: This system enables monitoring and management personnel to simultaneously monitor multiple airstrips in one monitoring center, saving the construction cost of the monitoring system.

(2) Deployment is simple and convenient: the workstation and network camera of this system are connected in a wireless way, and other connection methods are in the form of a network, and there is no need to install and use a capture card and so on.

(3) Panoramic video function: This system splices and processes multi-channel video. Compared with traditional multi-channel video monitoring, it has a more friendly visual effect and is more convenient to find abnormalities in the monitoring scene.

(4) Panoramic recording and playback function: This system stores the real-time spliced video on the disk device. When there is a problem in the monitoring scene, the root cause of the problem can be found out more quickly.

Claims (3)

1. A real-time panoramic video remote monitoring system for an airstrip, characterized in that it includes: at least one group of network cameras, at least one group of wireless routers, at least one group of graphics workstations, Ethernet and at least one remote monitoring center; wherein: The network camera group is placed on the airport tower, and each group of network cameras has at least one network camera, aiming at different positions on the airstrip; The wireless router and graphics workstation are located in the computer room. Each wireless router corresponds to a group of network cameras. A graphics workstation is wirelessly connected to a wireless router to input real-time video images corresponding to different positions of an airport runway; The graphics workstation is connected to Ethernet in a wired manner; The remote monitoring center is each client PC connected to Ethernet in a wired manner; Described graphics work station is a computer that is provided with wireless network card and wired network card at the same time, is provided with the NVIDIA graphics card based on parallel programming model instruction set architecture CUDA inside, according to the following steps successively from the multi-channel input of corresponding described network camera Live video image stitching: Step (1), collecting a batch of video images of three paths partially overlapping in the horizontal direction; Step (2), performing synchronous mutual exclusion control on the three-way video images, to ensure the correct splicing of the panoramic images of the video stream; Step (3), judging the real-time performance of the three-way video images: If it is currently offline, go to step (4), If it is currently a real-time stage, then go to step (6); Step (4), sequentially perform feature point extraction, description, matching and feature transformation on the three-way video image input from the offline stage, wherein: use the SURF operator to extract the feature points, and then use the RANSAC algorithm to solve the space transformation of the image space The model obtains 8 parameters of the homography matrix, thereby determining the projection position of a point on a three-dimensional plane in different two-dimensional images, so that two adjacent video images of the same group are registered respectively; Step (5), sequentially perform color and brightness correction on the two adjacent video images after registration using the final gamma correction parameters, and find the best stitching line, and use the best stitching line to correct the two overlapped images. The video images after registration are panoramically synthesized. On this basis, the distance transformation function is used to calculate all the non-zero The block distance from the pixel point to the nearest zero pixel point adjacent to it, after a smooth transition, obtains the weighted fusion matrix of the two adjacent and partially overlapping video images corresponding to the panoramic image; In step (6), the three-way real-time video images imported in the real-time stage are spliced in real time according to the following steps: Use the final gamma correction parameters obtained in the offline stage to perform color brightness correction; Invoke the CUDA based on the instruction set architecture of the parallel programming model, directly use the homography matrix calculated in the offline stage, realize multi-threaded concurrent image transformation calculation on the image processor NVIDIA GPU card, and register adjacent images input in real time, Determining overlapping regions and optimal stitching lines of the images, returning the results to the CPU of the computer; The computer uses the weighted fusion matrix obtained in the offline stage to perform weighted fusion, and obtains three real-time video images with a smoother transition at the best seam line, and then uses the mature H264 compression algorithm to compress the stitched panoramic video images, Then output the compressed video stream to the remote monitoring center via Ethernet. 2. a kind of runway real-time panoramic video remote monitoring system according to claim 1, is characterized in that, the CPU that graphics workstation adopts is Intel (R) Xeon (R) E5-1620, and computer is Lenovo ThinkStation S30, and graphics card is NVIDIA Quadro K600. 3. The real-time panorama video remote monitoring system of a runway according to claim 1, characterized in that, the splicing program of the graphics workstation is simultaneously equipped with commercially available multi-channel real-time video image splicing software Hikvision The large-screen splicing system is available for optional splicing switching.