KR20190141322A - Crack detection system of road surface using neural network and detection method therefor - Google Patents

Abstract

In a road surface crack detecting method using a neural network, the neural network receives density values for each pixel of an R channel, a G channel and a B channel of a corresponding frame image into an input layer and outputs coordinate information of a window determined that a crack is detected to an output layer.

Description

Crack detection system of road road using neural network and its detection method {CRACK DETECTION SYSTEM OF ROAD SURFACE USING NEURAL NETWORK AND DETECTION METHOD THEREFOR}

The present invention relates to a crack detection system and road detection method using a neural network, and more particularly, to a system and method for detecting a crack by learning a crack image of the road surface in a neural network having a plurality of intermediate layers. It is about.

The crack detection of the road surface is used to measure the pavement condition of the pavement in constructing a pavement management system (PMS) of the road. The basic components of the pavement management system are divided into the evaluation of pavement status, database construction, and rational and professional decision making system. The evaluation of pavement status must be equipped with an 'objective and quantitative evaluation agency' to perform proper pavement management. can do. Accurate pavement assessment is used as a basis for helping road management agencies maintain efficient maintenance during the paving life cycle, while prioritizing maintenance through objective comparisons with other segments or regions. Plays an important role. In addition, it is possible to directly or indirectly predict the degree of damage or reinforcement cost to decision makers by accurately evaluating the pavement status, to provide a basis for calculating the reinforcement cost and to establish a future maintenance cost budget plan in the long term It can be used as a basis.

Therefore, there is a growing demand for a crack detection method on a road surface for accurate evaluation of pavement conditions.

1. Korea Patent Publication No. 10-1651999: Porthole and crack detection system and method. 2. Korean Patent Publication No. 10-2016-0039316: Road crack measurement system and method thereof. 3. Korean Laid-Open Patent Publication No. 10-2009-0022033: Standard measurement system for a subject using a line scan camera.

Disclosure of Invention The present invention has an object of solving the above technical problem, and an object thereof is to provide a crack detection system of a road road surface using a neural network and a method of detecting the same, which enable accurate evaluation of the pavement state of a road. There is this.

In the crack detection method of a road surface using a neural network of the present invention, the neural network receives input and output values of pixels of R, G, and B channels of the frame image as an input layer, and detects cracks in an output layer. And outputting coordinate information of the determined window.

In addition, the crack detection method of the road surface of the present invention, (a) the neural network learning the detection of the cracked window of the road surface; (b) detecting the cracked window of the road surface using the learned neural network of step (a); (c) displaying a window on which the crack is detected on the frame image; And (d) counting the number of windows in which a crack is detected in a plurality of frame images in a predetermined road section, and classifying the class of roads according to the number of counted windows. .

Specifically, in the step (a), the neural network, the center point of the window is included in the crack area, the width of the crack area in the horizontal direction of 1 / (the first value) of the window width and the vertical length of the crack area It is characterized by learning to detect and detect a window when the height of the direction is equal to or larger than 1 / (second value) of the window height as a window with a crack on the road surface. In addition, in the step (a), it is preferable that the neural network uses coordinate information of a window as a teacher signal.

In the crack detection system of the road surface of the present invention, the neural network; includes, the neural network, the input layer receives the shade value of each pixel of the R channel, G channel and B channel of the frame image, the output layer And outputting coordinate information of the window in which the crack is determined to be detected.

In addition, the crack detection system of the road surface of the present invention, the learner for learning the neural network using a window with a crack of the road surface; A detector for detecting a cracked window of a road surface using a neural network learned by the learner; An indicator configured to display, by the detector, a window in which a crack is detected in at least part of an image of the frame image; And a classifier that counts the number of windows in which a crack is detected in a plurality of frame images by the detector in a predetermined road section, and classifies the class of roads according to the number of counted windows. desirable.

Specifically, the learner, the center point of the window is included in the crack area, the width of the crack area in the horizontal direction is at least 1 / (the first value) of the window width and the vertical height of the crack area is 1 of the window height The neural network is trained to determine and detect a window in a case where the value of / (second value) or more is determined as a window with a crack on the road surface. In addition, it is preferable that the learner uses coordinate information of a window used for learning as a teacher signal of the neural network.

According to the crack detection system and road detection method of the road surface using the neural network of the present invention, it is possible to accurately evaluate the pavement state of the road.

1 is a block diagram of a road detection crack detection system using a neural network according to an embodiment of the present invention.
2 is an exemplary diagram of a neural network.
3 is an exemplary view showing a window in which a crack is detected in a corresponding frame image by a display;
4 is an exemplary diagram of classification by a classifier;
5 is an explanatory diagram of coordinates of a window to be used as a teacher signal by the learner.
6 is a flowchart illustrating a crack detection method of a road surface using neural networks according to an embodiment of the present invention.

Hereinafter, a crack detection system and a method of detecting a road surface using a neural network according to embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The following examples of the present invention are intended to embody the present invention, but not to limit or limit the scope of the present invention. From the detailed description and the embodiments of the present invention, those skilled in the art to which the present invention pertains can easily be interpreted as belonging to the scope of the present invention.

First, Figure 1 shows a block diagram of the crack detection system 100 of the road surface using a neural network in accordance with a preferred embodiment of the present invention.

As can be seen from Figure 1, Figure 1 is a road detection surface crack detection system 100 using a neural network according to an embodiment of the present invention, the camera 10, the neural network 20, the learner 30 ), A detector 40, an indicator 50, a classifier 60, and a memory 70.

The neural network 20, the learner 30, the detector 40, the indicator 50, and the classifier 60 may include at least a part of a processor such as an MCU, a DSP, and a CPU. In addition, the memory 70 stores the input / output data of the camera 10, the neural network 20, the learner 30, the detector 40, the display 50, and the classifier 60.

First, the camera 10 continuously captures a road surface of a road that is mounted on a vehicle and operates to acquire an image.

2 shows an exemplary configuration of the neural network 20.

The neural network 20 receives input and output shade values of the R channel, the G channel, and the B channel of the frame image as an input layer, and outputs coordinate information of a window in which a crack is detected to the output layer. Specifically, the coordinate information of the window in which the crack is determined to be detected is characterized in that the first coordinate as the vertex and the second coordinate as the vertex opposite to the first coordinate of the window in which the crack is determined to be detected. In addition, it is preferable that the hidden layer (intermediate layer) of the neural network 20 is many.

The size of the window based on the coordinate information of the window output by the output layer may be constantly set according to the teacher signal, or may be set differently depending on the situation.

 The learner 30 selects a window with a crack of the road surface from the image stored in advance in the memory 70 and learns the neural network 20 using the window with the crack of the selected road surface. In addition, the detector 40 detects a window with a crack of the road surface from the image of the road surface obtained from the camera 10 or the memory 70 by using the neural network 20 learned by the learner 30. It plays a role.

The indicator 50 serves to display a window in which the crack is detected by the detector 40 on at least part of the image of the frame.

3 illustrates an example of display of a window in which a crack is detected in a corresponding frame image by the display unit 50.

The classifier 60 counts the number of windows in which a crack is detected in a plurality of frame images by the detector 40 in a predetermined road section, and classifies the class of roads according to the number of counted windows. Play a role.

4 is a diagram illustrating classification by the classifier 60.

Hereinafter, the learning method of the neural network 20 by the learner 30 will be described in detail.

In the learner 30, the center point of the window is included in the crack area, the width of the crack area in the horizontal direction is equal to or greater than 1 / (first value) of the window width, and the height of the crack area in the vertical direction is equal to 1 / the window height. (2nd value) Selects the window with the crack of the road surface as the window more than. In addition, the learner 30 is characterized in that the neural network 20 is trained to determine and detect the selected window as a window with a crack on the road surface. The center point means the center pixel. The width of the crack area in the horizontal direction reaches the distance between the horizontal start point and the end point of the cracks in which the pixels are connected to each other, and the vertical height of the cracks reaches the distance between the longitudinal start and end points of the cracks in which the pixels are connected to each other.

In this case, the window size may be a multiple of a first value and a multiple of a second value, and the first value and the second value may be different or the same value. For example, the first value and the second value may be set to three.

5 is an explanatory diagram for a coordinate value of a window to be used as a teacher signal by the learner 30.

As can be seen from FIG. 5, the learner 30 preferably uses the coordinate information of the window used for learning as the teacher signal for the output layer of the neural network. Specifically, the coordinate information of the window used for learning is characterized in that the first coordinate as the vertex and the second coordinate as the vertex opposite to the first coordinate of the window in which the crack is detected.

The learning method of the neural network 20 by the learner 30 as described above will be described by the equation.

×

의 윈도우를 설정하여 해당 크랙이 존재하는 부분을 지정하는 것에 의하여 교사 신호를 제공한다. The input layer is composed of three types of images of R, G, and B. When the input is performed, the entire image is inputted to the input node as a pixel-by-pixel value. In addition, the teacher signal of the neural network 20 has a magnitude

×

The teacher signal is provided by setting the window of the window and specifying the part where the crack exists.

The setting criteria of the window including the crack for the teacher signal is as follows.

) 및 윈도우 x 방향 투사값이

×1/N을 넘거나 y 방향 투사값이

×1/N을 넘는 윈도우 집합(

)은 각각, 다음의 [수학식 1] 및 [수학식 2]와 같이 나타낼 수 있다. 여기서, 제 1 값 및 제 2 값을 동일한 N으로 설정하였다.A set of windows if the window center point is a point in the crack (

) And the window x-direction projection value

Exceeds × 1 / N or the projection value in the y direction

Set of windows over × 1 / N (

) Can be represented by the following [Equation 1] and [Equation 2], respectively. Here, the first value and the second value were set to the same N.

The meanings of the variables in [Equation 1] and [Equation 2] are as follows.

: 화면 내

×

크기의 윈도우, k=1, 2, 3, …, n

: In screen

×

Window of magnitude k = 1, 2, 3,... , n

,

: 윈도우의 x 방향 크기와 y 방향 크기

,

: X-direction and y-direction dimensions of the window

: k 윈도우 내의 한 점

: a point in the k window

: 윈도우의 중앙점

: Center point of the window

: 해당 점이 크랙 영역의 점임

: The point is a point in a crack area

: 해당 점이 비크랙 영역의 점임

: The point is a point in the non-cracked area

∈(

∩

) 인 경우, 크랙을 포함한 윈도우로 판정하게 된다.Finally, the learner 30

∈ (

∩

), The window is determined to contain a crack.

Hereinafter, a crack detection method of a road road surface using a neural network according to an exemplary embodiment of the present invention will be described. Crack detection method of the road road surface using a neural network according to an embodiment of the present invention uses the road surface crack detection system 100 using the neural network of the present invention described above, even if there is no separate description Of course, it includes all the features of the crack detection system 100 of the road surface using the neural network of the invention.

Crack detection method of the road surface using a neural network according to an embodiment of the present invention may be implemented in the form of a computer program implemented by at least a portion of a processor such as MCU, DSP, CPU.

6 is a flowchart illustrating a crack detection method of a road road surface using a neural network according to an exemplary embodiment of the present invention.

As can be seen from Figure 6, in the road detection crack detection method using a neural network according to an embodiment of the present invention, the neural network 20, the input layer R channel, G channel and The pixel value of each pixel of the B channel may be input, and the coordinate information of the window determined to have detected cracks may be output to the output layer. Specifically, the coordinate information of the output window is preferably the first coordinate that is the vertex of the window and the second coordinate that is the vertex opposite to the first coordinate of the window in which the crack is detected.

Specifically, the crack detection method of the road road surface using a neural network according to an embodiment of the present invention, the neural network 20 learning the detection of the window with a crack of the road surface (S10); Detecting a window with a crack of the road surface using the learned neural network 20 of step S10 (S20); Displaying a window in which a crack is detected in the corresponding frame image in step S20 (S30); And (d) counting the number of windows in which a crack is detected in the plurality of frame images in step S20 in a predetermined road section, and classifying the class of the road according to the number of counted windows (S40). Characterized in that it comprises a.

In the step S10, the neural network 20, the center point of the dough is included in the crack area, the width of the crack area in the horizontal direction is at least 1 / (the first value) of the window width and the height of the vertical direction of the crack area is It is preferable to learn to determine and detect the window in the case of more than 1 / (second value) of the window height as a window with a crack of the road surface.

In addition, in step S10, the neural network 20, it is preferable to use the coordinate information of the window as a teacher signal for the output layer of the neural network. Specifically, the coordinate information of the window used for learning is characterized in that the first coordinate as the vertex and the second coordinate as the vertex opposite to the first coordinate of the window in which the crack is detected.

As described above, according to the crack detection system 100 and the detection method of the road surface using the neural network of the present invention, it can be seen that it is possible to accurately evaluate the pavement state of the road.

100: crack detection system
10: camera
20: neural network
30: learner
40: detector
50: indicator
60: classifier
70: memory

Claims (12)

In the crack detection method of the road surface using a neural network,
The neural network,
A method for detecting cracks on a road surface, comprising: inputting a gray level value of each pixel of an R channel, a G channel, and a B channel of a corresponding frame image to an input layer, and outputting coordinate information of a window in which a crack is detected to an output layer. The method of claim 1,
The crack detection method of the road surface,
(a) the neural network learning to detect a cracked window of a road surface; And
(b) detecting a window with a crack on the road surface using the learned neural network of step (a). The method of claim 2,
In the step (a), the neural network,
The center point of the window is contained in the crack area, the width of the crack area is at least 1 / (the first value) of the window width, and the height of the crack area in the vertical direction is at least 1 / (the second value) of the window height. A road detection crack detection method characterized by learning to determine and detect a window in a case as a window with cracks on a road surface. The method of claim 3,
In the step (a), the neural network,
A crack detection method for a road surface, using coordinate information of a window as a teacher signal. The method of claim 2,
The crack detection method of the road surface,
and (c) displaying a window in which the crack is determined to be detected on the corresponding frame image. The method of claim 2,
The crack detection method of the road surface,
(d) counting the number of windows determined to have cracks detected in a plurality of frame images in a predetermined road section, and classifying the class of roads according to the number of counted windows; Crack detection method of road road surface to do. In crack detection system of road surface,
Including neural networks;
The neural network,
A road detection crack detection system, characterized in that the input layer receives the shade value of each pixel of the R channel, G channel and B channel of the frame image, and outputs the coordinate information of the window determined that the crack is detected to the output layer. The method of claim 7, wherein
The crack detection system of the road surface,
A learner for learning the neural network using a cracked window of a road surface; And
And a detector for detecting a cracked window of the road surface using neural networks learned by the learner. The method of claim 8,
The learner,
The center point of the window is contained in the crack area, the width of the crack area in the horizontal direction is at least 1 / (the first value) of the window width, and the height of the crack area in the vertical direction is 1 / (the second value of the window height). And the neural network is trained to determine and detect a window in the case of abnormality as a window with a crack on the road surface. The method of claim 9,
The learner,
A road detection crack detection system using coordinate information of a window as a teacher signal of the neural network. The method of claim 8,
The crack detection system of the road surface,
And an indicator for displaying a window on which at least a part of the frame image of the frame image is detected by the detector. The method of claim 8,
The crack detection system of the road surface,
And a classifier for classifying road grades according to the number of counted windows by counting the number of windows in which a crack is detected in a plurality of frame images by the detector in a predetermined road section. Crack detection system of road road surface to assume.

Images