Disclosure of Invention
Aiming at the problems, the invention aims to provide a remote sensing image road breakpoint repairing method, equipment and medium, which are used for effectively repairing broken roads in a remote sensing image and optimizing road boundaries by combining the direction vector of the breakpoints, the distance between the breakpoints and the road width information.
The embodiment of the invention provides a remote sensing image road breakpoint repairing method, equipment and medium.
In a first aspect, a remote sensing image road breakpoint repairing method includes:
S1, converting a road image into a binary image, calculating the size of a communication area of each road, and setting a threshold value to remove part of the communication area to obtain an optimized binary image;
s2, performing skeletonization operation on the optimized binary image, and converting the road into a road skeleton with single pixel width;
s3, performing breakpoint identification processing on the road skeleton, extracting the breakpoints of all roads, and obtaining breakpoint parameter values;
s4, based on the matching value among each group of breakpoint pairs, connecting the breakpoint pairs with the matching value exceeding a threshold value, and obtaining a road skeleton after completion;
s5, repairing the road width of the completed road skeleton, and obtaining the repaired road.
Further, the step S1 includes the steps of:
s11, converting the road gray level image into a road binary image, wherein the formula is as follows:
here, road_image (x, y) is a road grayscale image, 1 is a pixel value of a road region, and 0 is a pixel value of a background region.
And S12, calculating the size of the connected region of the road region connected with each piece in the binary image by using a connected analysis algorithm.
And S13, setting a connected region size threshold value min_size according to priori knowledge, and removing the connected region with the area smaller than the min_size to obtain an optimized binary image.
Further, the step S2 includes:
processing the optimized binary image based on an iterative method of the pixel neighborhood, removing non-skeleton pixel points until only the central line of the road area is left, and reducing all the road area parts into a line structure with single pixel width to obtain a road skeleton;
Further, in the step S3, the breakpoint identification process is performed on the road skeleton by using an eight-neighborhood search method, where the formula is expressed as follows:
The formula is:
Wherein, (0, 0) is the road skeleton pixel position.
Further, the S3 break point parameter values comprise a backtracking coordinate set, a direction vector set and a vertical vector set, wherein:
And (3) tracking along the skeleton from the break point by using a depth-first search algorithm until the bifurcation point or the condition is met, recording the trace coordinate points of the pixel points of the road skeleton in the trace process and forming a trace coordinate set corresponding to the break point, and carrying out fitting normalization on the trace coordinate set to obtain a direction vector set, and orthogonalizing the direction vector set to obtain a vertical vector set.
Further, the matching value is calculated based on the distance and the angle difference between each group of breakpoint pairs, and the formula is as follows:
where d is the Euclidean distance between two break points of the pair, θ is the angle of the direction vector between the pair of break points, and w d and w θ are the weight values of the distance and angle, respectively.
Further, the step S4 also comprises the steps of checking whether each breakpoint accords with a threshold value to form a breakpoint pair, if not, marking the breakpoint pair as an orphan, and determining the angle and the distance for extending the orphan for the orphan, and extending the orphan along the road skeleton until reaching the maximum extension distance or meeting the road pixel.
Further, the step S5 comprises the steps of calculating the width of each pixel point on the vertical vector of the gray-scale road image for each pixel point on the road skeleton section, taking the average value of the road widths corresponding to all the pixel points on the skeleton section as the repairing width corresponding to the break point, repairing the width of the completed road skeleton according to the repairing width, and obtaining the repaired road.
In a second aspect, an electronic device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method as provided in the first aspect when the program is executed.
In a third aspect, a non-transitory computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method as provided in the first aspect.
The invention has the beneficial effects that:
1. according to the road restoration method based on the breakpoint direction vector and the distance information, the continuity of the road skeleton can be maintained by accurately calculating the direction and the distance between the breakpoints, and the accuracy of road extraction is remarkably improved.
2. According to the invention, the repaired road is subjected to regularization treatment by introducing the road width information, so that the width of the repaired road meets the physical rule, the situation of road boundary distortion or excessive connection caused by mismatching is avoided, and the repaired road boundary is more accurate and natural by optimizing the smoothness and regularity of the repairing path, and the risk of misjudgment is reduced.
3. The method can cope with interference caused by shielding, noise and complex topography on road extraction when processing roads in the high-resolution remote sensing image, ensures that a better repairing effect can be obtained even under a complex environment, and can complete road repairing in a shorter time compared with the traditional method, and particularly, the method has higher efficiency when processing large-scale high-resolution images. The method is very suitable for an automatic road extraction system and a large-scale remote sensing data processing task, and can obtain a high-quality road repair result in a short time.
4. The road repaired by the method is enhanced in continuity, the boundary is smoother and more regular, and the repair result has higher consistency with real road data. By comparing the road real form with the marking data, the repairing method can accurately restore the road real form, and effectively improves the overall accuracy of the road extraction system.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar symbols indicate like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
The traditional deep learning method mainly focuses on the whole road extraction result, does not effectively repair broken areas caused by shielding, noise and the like, causes discontinuous extracted road network, influences the integrity and practicality of the road, has lower matching precision in the road repair process, and is easy to introduce misjudgment and error connection.
In order to solve the above problems, the present invention provides a remote sensing image road breakpoint repairing method, fig. 1 is a schematic flow chart of the remote sensing image road breakpoint repairing method provided by the embodiment of the present invention, and fig. 1 is a schematic flow chart of the remote sensing image road breakpoint repairing method provided by the embodiment of the present invention, where the method includes:
s1, converting the road image into a binary image, calculating the size of a communication area of each road, setting a threshold value to remove part of the communication area, and obtaining an optimized binary image.
As shown in fig. 3, the road image is converted into a binary image, the white area is the road area, the pixel value is 1, the black area is the background area, and the pixel value is 0.
The grayscale image road_image (x, y) is converted to a binary image by setting a fixed or adaptive threshold, where the pixel value of the road area is set to 1, the pixel value of the background area is set to 0, and the formula is:
After binarization, a connected region in the image is calculated by using a connected analysis algorithm, each connected road region is marked, and the size of each connected region is calculated. And then setting a threshold value min_size of the size of the connected region according to the priori knowledge, and removing the connected region with the area smaller than the min_size.
For example, if the threshold is set to 64 pixels, the area of the connected area smaller than 64 pixels is removed, and through threshold screening, the interference pixel area can be removed, the road breakpoint data is reduced, the calculation amount is reduced, and redundant breakpoints are prevented from being generated in the subsequent breakpoint detection.
S2, performing skeletonizing operation on the optimized binary image, and converting the road into a road skeleton with single pixel width.
The optimized binary image is processed based on an iterative method of the pixel neighborhood, non-skeleton pixel points are removed until only the central line of a road area is left, the skeletonized output image is the binary image, all the road area parts are reduced to a line structure with single pixel width, the lines keep the topological characteristic and connectivity of an original road, and meanwhile, the road skeleton is obtained.
S3, performing breakpoint identification processing on the road skeleton, extracting the breakpoints of all roads, and obtaining the breakpoint parameter values.
As shown in fig. 6, the breakpoint identification process for the road skeleton uses an eight-neighborhood search method, where an eight-neighborhood refers to a neighborhood composed of eight pixels in the up-down, left-right, and four diagonal directions with a certain road pixel point as the center. For each pixel point in the skeleton, performing AND computation on 8 surrounding neighborhoods by using an eight-neighborhood convolution kernel, traversing each pixel (skeleton pixel) with the value of 1 in the image, and counting the number of 1 in the adjacent 8 neighborhoods by taking the pixel as the center, wherein if the number is 1, the description is a breakpoint.
The center of the kernel matrix is a road skeleton pixel with a pixel value of 1, and eight surrounding areas can have 1,2 and 3 pixels with the value of 1, and if the number is 1, the description is a breakpoint.
The formula is:
Wherein, (0, 0) is the road skeleton pixel position.
Extracting breakpoints of all roads, and obtaining breakpoint parameter values, wherein the breakpoint parameter values comprise a backtracking coordinate set, a direction vector set, a vertical vector set and the like, and the method comprises the following steps of:
And in the backtracking process, the coordinate points of the backtracking of the pixel points of the road skeleton are recorded and the coordinate groups corresponding to the break points are formed.
And then fitting and extracting the direction vector representation by using the coordinate set to obtain a normalized direction vector set, and finally orthogonalizing the direction vector set to obtain a vertical vector set.
And S4, connecting breakpoint pairs with the matched values exceeding a threshold value based on the matched values among the breakpoint pairs of each group, and obtaining the road skeleton after completion.
The two breakpoints form a breakpoint pair, firstly, the Euclidean distance d between the two breakpoints of the breakpoint pair is calculated, then the included angle theta of the direction vectors of the two breakpoints of the breakpoint pair is calculated, and finally, w d and w θ are adopted to respectively represent the weight values of the distance and the angle, the distance d and the included angle theta are regularized, and the calculation formula of the matching value of the breakpoint pair can be expressed as follows:
where d is the Euclidean distance between two break points of the pair, θ is the angle of the direction vector between the pair of break points, and w d and w θ are the weight values of the distance and angle, respectively.
Calculating the matching value of the breakpoint pair through the formula, traversing all the breakpoint pairs, reserving the breakpoint pair with the matching value exceeding a threshold value, and connecting the breakpoint pair to obtain the road skeleton after completion.
For each breakpoint in the breakpoint list, checking whether the breakpoint accords with a threshold value to form a breakpoint pair, if not, marking the breakpoint pair as an orphan point, using a pre-calculated direction vector for each orphan point, normalizing the orphan point, determining the extending direction and distance (step size) of the orphan point, and finally gradually extending the road skeleton until the maximum extending step size is reached or road pixels are encountered.
S5, repairing the road width of the completed road skeleton, and obtaining the repaired road.
The road skeleton after the completion is obtained by the method, and whether the road skeleton is connected between the breakpoint pairs or is single pixel at the moment can be repaired according to the original road width information, so that the road skeleton is more similar to the form of a real road.
Specifically, a backtracking method is used for tracking from the breakpoint to the designated position, and a skeleton segment is obtained. The width of each pixel on the skeleton segment on the vertical vector on the grayscale road image is then calculated for that pixel. Finally, taking an average value of road widths corresponding to all pixels on the skeleton section as a repairing width corresponding to the breakpoint, and repairing the width of the completed road skeleton according to the repairing width to obtain a repaired road area; and finally, carrying out smoothing treatment on the repaired road area, eliminating sharp corners or protrusions possibly existing, and ensuring smooth and natural road shape after width repair.
Based on the above method, the invention also provides a remote sensing image road breakpoint repairing device, as shown in fig. 3, the device comprises:
the image conversion module is used for converting the road image into a binary image, calculating the size of each road communication area, and removing part of the communication areas by setting a threshold value to obtain an optimized binary image;
The framework extraction module is used for performing skeletonizing operation on the optimized binary image and converting the road into a road framework with single pixel width;
The breakpoint acquisition module is used for carrying out breakpoint identification processing on the road skeleton, extracting the breakpoints of all roads and acquiring breakpoint parameter values;
the framework repair module is used for calculating a matching value between each group of breakpoint pairs, connecting the breakpoint pairs with the matching values exceeding a threshold value, and obtaining a road framework after completion;
And the width repair module is used for repairing the width of the road to the road skeleton after the completion and obtaining the repaired road.
The method and the device can maintain the continuity of the road skeleton after repairing and remarkably improve the accuracy of road extraction by accurately calculating the direction and the distance between the break points, and simultaneously, by introducing the road width information, the method and the device not only ensure that the width of the repaired road accords with the physical rule, but also avoid the condition of road boundary distortion or excessive connection caused by mismatching, optimize the smoothness and the regularity of the repairing path, ensure that the repaired road boundary is more accurate and natural, and reduce the risk of misjudgment.
The invention also provides an electronic device, and fig. 7 is a schematic structural diagram of the electronic device provided by the embodiment of the invention, as shown in fig. 7, the electronic device may include a processor (processor), a communication interface (CommunicationsInterface), a memory (memory), and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor may call logic instructions in memory, for example, to perform the following method:
S1, converting a road image into a binary image, calculating the size of a communication area of each road, and setting a threshold value to remove part of the communication area to obtain an optimized binary image;
s2, performing skeletonization operation on the optimized binary image, and converting the road into a road skeleton with single pixel width;
s3, performing breakpoint identification processing on the road skeleton, extracting the breakpoints of all roads, and obtaining breakpoint parameter values;
s4, based on the matching value among each group of breakpoint pairs, connecting the breakpoint pairs with the matching value exceeding a threshold value, and obtaining a road skeleton after completion;
s5, repairing the road width of the completed road skeleton, and obtaining the repaired road.
Further, the logic instructions in the memory described above may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
Embodiments of the present invention also provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the methods provided by the above embodiments, for example, comprising:
S1, converting a road image into a binary image, calculating the size of a communication area of each road, and setting a threshold value to remove part of the communication area to obtain an optimized binary image;
s2, performing skeletonization operation on the optimized binary image, and converting the road into a road skeleton with single pixel width;
s3, performing breakpoint identification processing on the road skeleton, extracting the breakpoints of all roads, and obtaining breakpoint parameter values;
s4, based on the matching value among each group of breakpoint pairs, connecting the breakpoint pairs with the matching value exceeding a threshold value, and obtaining a road skeleton after completion;
s5, repairing the road width of the completed road skeleton, and obtaining the repaired road.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.