CN114266701A - A kind of fan blade image stitching method and device - Google Patents
A kind of fan blade image stitching method and device Download PDFInfo
- Publication number
- CN114266701A CN114266701A CN202111609230.4A CN202111609230A CN114266701A CN 114266701 A CN114266701 A CN 114266701A CN 202111609230 A CN202111609230 A CN 202111609230A CN 114266701 A CN114266701 A CN 114266701A
- Authority
- CN
- China
- Prior art keywords
- fan blade
- image
- blade image
- splicing
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fan blade image splicing method and device. Compared with the prior art, the method utilizes the continuity of the image sequence, realizes image matching by using a sparse optical flow tracking algorithm, overcomes the defects of a classical image splicing method, and effectively solves the problems of less characteristic points and low matching rate. Experiments show that the method has stronger robustness for splicing the wind power blades, and can improve the splicing precision of the images of the fan blades, thereby improving the detection precision of the defects of the follow-up fan blades. The invention solves the problem of fan blade splicing that the image features have rare texture, high local similarity and small difference of light and shade degree by using a sparse optical flow tracking method.
Description
Technical Field
The invention relates to the field of image processing, in particular to a fan blade image splicing method and device.
Background
With the increasingly mature wind power industry in China, the research on the intelligent unmanned aerial vehicle inspection wind field fan is also gradually deepened, the picture acquired by the unmanned aerial vehicle is a local image of the fan blade, and in order to accurately analyze and judge the state and the defect position of the blade, a complete image of the blade needs to be acquired, so that the image segmentation and the blade splicing are necessary to be performed on the video acquired by the unmanned aerial vehicle. Fan blade concatenation is as the leading condition that intelligence was patrolled and examined, and the quality of blade concatenation result has directly decided fan defect testing result.
The following problems exist when the existing image splicing algorithm is applied to blade splicing: (1) the image registration and splicing method based on scale invariant feature transformation (sift, surf and fast) has the problems of few feature points, low registration rate and difficulty in completing blade splicing; (2) the method based on the unmanned aerial vehicle parameters and the flight speed cannot accurately acquire the corresponding relation between the flight speed and the pixel displacement, and has the problems of pixel repetition and pixel loss.
Disclosure of Invention
In order to solve the problems of the existing image splicing algorithm in the background technology, the invention provides a fan blade image splicing method, and the specific technical scheme is as follows.
A fan blade image splicing method comprises the following steps:
s1, extracting characteristic points of the fan blade image of the initial frame, and taking the extracted characteristic points as an initial point set P;
s2, tracking the initial point set P to obtain a tracking point set Q of the next frame of fan blade image; the next frame of fan blade image is an adjacent frame or an interval frame of the initial frame of fan blade image;
s3, rotating P and Q through the rotation transformation matrix M to obtain P 'and Q'; m is a rotation transformation matrix corresponding to the rotation of the fan blade in the fan blade image in the specified direction;
s4, calculating a homography matrix H of P 'and Q' according to the following formula, wherein Q 'is H.P';
s5, according to the homography matrix H, image registration and splicing of the fan blade images of the initial frame and the next frame are completed by adopting an image registration method based on the characteristic points;
and S6, taking the spliced image as an initial frame fan blade image, and repeating the steps S1-S5 until the splicing of the complete fan blade image is completed.
Preferably, the following steps are further included between steps S2 and S3: and eliminating abnormal points in the tracking point set Q by adopting a RANSAC algorithm, and correspondingly eliminating the characteristic points corresponding to the abnormal points in the initial point set P.
Preferably, when the feature point in the tracking point set Q is smaller than a preset value, the tracking is stopped.
Preferably, the following steps are further included between steps S5 and S6: and eliminating splicing lines in the images to be spliced by adopting a multiband fusion method.
Specifically, the rotation transformation matrix M is obtained by the following method:
acquiring pixel coordinates of upper and lower edges of fan blades in a fan blade image;
calculating the inclination slope of the fan blade according to the pixel coordinates of the upper edge and the lower edge to obtain a rotation angle;
and obtaining a rotation transformation matrix M according to the rotation angle, the specified angle and the image rotation center of the fan blade image.
Based on the same inventive concept, the invention also provides a fan blade image splicing device, which comprises:
the characteristic point extraction unit is used for extracting characteristic points of the fan blade image of the initial frame, and taking the extracted characteristic points as an initial point set P;
the tracking unit is used for tracking the initial point set P to obtain a tracking point set Q of the next frame of fan blade image; the next frame of fan blade image is an adjacent frame or an interval frame of the initial frame of fan blade image;
the rotating unit is used for rotating the P and the Q through the rotating transformation matrix M to obtain P 'and Q'; m is a rotation transformation matrix corresponding to the rotation of the fan blade in the fan blade image in the specified direction;
the calculation unit is used for calculating and obtaining a homography matrix H of P 'and Q' according to the following formula, wherein Q 'is H.P';
and the splicing unit is used for finishing image registration and splicing of the fan blade images of the initial frame and the next frame by adopting an image registration method based on the characteristic points according to the homography matrix H.
Preferably, the tracking point set P further comprises an abnormal point removing unit, configured to remove abnormal points in the tracking point set Q by using a RANSAC algorithm, and correspondingly remove feature points in the initial point set P corresponding to the abnormal points.
Preferably, the abnormal rejection unit stops tracking when the feature points in the tracking point set Q are smaller than a preset value.
Preferably, the method further comprises a stitching line elimination unit for eliminating stitching lines in the images to be stitched by adopting a multiband fusion method.
Preferably, the method further comprises a rotation transformation matrix calculation unit, configured to:
acquiring pixel coordinates of upper and lower edges of fan blades in a fan blade image;
calculating the inclination slope of the fan blade according to the pixel coordinates of the upper edge and the lower edge to obtain a rotation angle;
and obtaining a rotation transformation matrix M according to the rotation angle, the specified angle and the image rotation center of the fan blade image.
Preferably, the characteristic point extraction is performed on the image by adopting an orb algorithm.
Preferably, the initial set of points P is tracked using a sparse optical flow tracking algorithm.
Due to the adoption of the technical scheme, compared with the prior art, the method disclosed by the invention realizes image matching by utilizing the continuity of the image sequence and using a sparse optical flow tracking algorithm, overcomes the defects of a classical image splicing method, and effectively solves the problems of few characteristic points and low matching rate. Experiments show that the method has stronger robustness for splicing the wind power blades, and can improve the splicing precision of the images of the fan blades, thereby improving the detection precision of the defects of the follow-up fan blades. The invention solves the problem of fan blade splicing that the image features have rare texture, high local similarity and small difference of light and shade degree by using a sparse optical flow tracking method.
Drawings
FIG. 1 is a schematic flow diagram of the present invention;
FIG. 2 is an initial frame image after feature point extraction;
FIG. 3 is a polygonal profile of the blade of FIG. 2;
FIG. 4 is an initial frame image after edge points are removed;
FIG. 5 is an initial frame image with outliers removed;
FIG. 6 is a second interval frame image after outliers are removed;
FIG. 7 is an initial frame image after being rotated to the horizontal;
FIG. 8 is a second alternate frame image after being rotated to horizontal;
FIG. 9 is an image after the initial frame image and the second interval frame image are spliced;
FIG. 10 is a mosaic of full fan blade images.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
Referring to fig. 1, a fan blade image stitching method includes the following steps:
and S1, extracting characteristic points of the fan blade image of the initial frame by adopting a orb algorithm, and taking the extracted characteristic points as an initial point set P.
Fig. 2 shows an image obtained by extracting feature points from an initial frame image by using the orb algorithm, and a total of 922 feature points are extracted. Then, image erosion and boundary extraction are adopted to obtain a polygonal outline of the blade (as shown in fig. 3), and 330 points are obtained after edge points are removed according to feature point display after polygonal frames are removed, as shown in fig. 4.
S2, tracking the initial point set P by adopting a sparse optical flow tracking algorithm to obtain a tracking point set Q of the next frame of fan blade image; the next frame of fan blade image is an adjacent frame or an alternate frame of the initial frame of fan blade image. The two frames of fan blade images are spaced frames. Because the change of the adjacent frame images is small, the frame interval is preferably adopted in the embodiment, specifically, the frame rate of the video collected by the unmanned aerial vehicle in the embodiment is 30 frames per second, and the frame interval is 15 frames. Removing tracking abnormal points in the sparse optical flow tracking algorithm process: the sparse optical flow tracking algorithm obtains the results of a tracking point set, a matching state and a measurement error, and the tracking point set is rejected according to the matching state (the matching state is a list, the range value is 0 and 1 states, 0 is tracking failure, and 1 is tracking success) and the measurement error (in the embodiment, the threshold value of the measurement error is set to be 2, and the part larger than 2 is rejected).
In this embodiment, the frame rate of video collected by the unmanned aerial vehicle is 30 frames per second, and in order to ensure the tracking accuracy, the designated frame splicing condition is adopted to stop tracking when 15 frames or the tracking point set is less than 15 frames (if tracking is continued, points are always eliminated in the tracking process, and the number of tracking points is less and less, so that the tracking is stopped when the number of points is less than a certain value).
S21, eliminating abnormal points in the tracking point set Q by adopting a RANSAC algorithm, and correspondingly eliminating feature points corresponding to the abnormal points in the initial point set P. And (6) tracking. Fig. 5 (initial frame image) and fig. 6 (second interval frame image) are sets of points from which outliers are removed, and 27 initial points and 27 tracking points are both present.
S3, rotating P and Q through the rotation transformation matrix M to obtain P 'and Q'; m is a rotation transformation matrix corresponding to the rotation of the fan blade in the fan blade image in the specified direction;
specifically, the rotation transformation matrix M is obtained by the following method:
acquiring pixel coordinates of upper and lower edges of fan blades in a fan blade image;
calculating the inclination slope of the fan blade according to the pixel coordinates of the upper edge and the lower edge to obtain a rotation angle;
and obtaining a rotation transformation matrix M according to the rotation angle, the specified angle and the image rotation center of the fan blade image.
According to the blade edge calculation, the following results are obtained:
the rotation matrix of the initial frame picture is
The rotation matrix of the second interval frame picture is
Rotation matrix M for initial frame picture1After rotation the result is
Example of a set of rotation points:
rotating the front point set: [[1099.,558.],[1298.,718.],[1216.,679.]]
Rotating the point set: [[972.5782111974022,1024.8839681620411],[719.0319180038418,994.6304194808515],[809.2395229369383,984.2579553457306]]
Second interval frame picture using rotation matrix M2After rotation the result is
Example of a set of rotation points:
rotating the front point set: [[1184.109,628.28723],[1386.0071,791.116],[1302.1696,751.0744]]
Rotating the point set: [[863.1770171653434,1010.4813353365977],[605.6593729016595,979.485996721354],[697.9723153904574,968.9796672107524]]
Specifically, the designated angle in this embodiment is 0 °, that is, the fan blades in the fan blade image uniformly rotate to the horizontal direction, so as to facilitate subsequent splicing. The initial frame picture after rotation is shown in fig. 7, and the second interval frame picture after rotation is shown in fig. 8.
S4, calculating P according to the following formula′And Q ' are homography matrix H, Q ' ═ H · P ';
and obtaining a homography matrix according to the rotated point set:
s5, according to the homography matrix H, image registration and splicing of the fan blade images of the initial frame and the next frame are completed by adopting an image registration method based on the characteristic points; the stitched picture is shown in fig. 9.
S6, eliminating splicing lines in the images to be spliced by adopting a multiband fusion method;
and S7, repeating the steps S1-S6 by taking the spliced image without the splicing line as the fan blade image of the initial frame until the splicing of the complete fan blade image is completed. Specifically, after the 1 st frame and the 15 th frame of the video are spliced according to the method, the spliced image is spliced with the 30 th frame according to the method, and the process is repeated until the splicing of the complete fan blade image is completed. A mosaic of the full fan blade image is shown in fig. 10.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A fan blade image splicing method is characterized by comprising the following steps:
s1, extracting characteristic points of the fan blade image of the initial frame, and taking the extracted characteristic points as an initial point set P;
s2, tracking the initial point set P to obtain a tracking point set Q of the next frame of fan blade image; the next frame of fan blade image is an adjacent frame or an interval frame of the initial frame of fan blade image;
s3, rotating P and Q through the rotation transformation matrix M to obtain P 'and Q'; m is a rotation transformation matrix corresponding to the rotation of the fan blade in the fan blade image in the specified direction;
s4, calculating the homography matrix H of the P 'and the Q' according to the following formula: q '═ H · P';
s5, according to the homography matrix H, image registration and splicing of the fan blade images of the initial frame and the next frame are completed by adopting an image registration method based on the characteristic points;
and S6, taking the spliced image as an initial frame fan blade image, and repeating the steps S1-S5 until the splicing of the complete fan blade image is completed.
2. The fan blade image stitching method according to claim 1, wherein the steps S2 and S3 further include the following steps: and eliminating abnormal points in the tracking point set Q by adopting a RANSAC algorithm, and correspondingly eliminating the characteristic points corresponding to the abnormal points in the initial point set P.
3. The fan blade image stitching method according to claim 2, characterized in that: and when the characteristic points in the tracking point set Q are smaller than a preset value, stopping tracking.
4. The fan blade image stitching method according to any one of claims 1 to 3, wherein the steps S5 and S6 further include the following steps: and eliminating splicing lines in the images to be spliced by adopting a multiband fusion method.
5. The fan blade image stitching method according to claim 4, characterized in that: the rotation transformation matrix M is obtained by the following method:
acquiring pixel coordinates of upper and lower edges of fan blades in a fan blade image;
calculating the inclination slope of the fan blade according to the pixel coordinates of the upper edge and the lower edge to obtain a rotation angle;
and obtaining a rotation transformation matrix M according to the rotation angle, the specified angle and the image rotation center of the fan blade image.
6. A fan blade image stitching device, comprising:
the characteristic point extraction unit is used for extracting characteristic points of the fan blade image of the initial frame, and taking the extracted characteristic points as an initial point set P;
the tracking unit is used for tracking the initial point set P to obtain a tracking point set Q of the next frame of fan blade image; the next frame of fan blade image is an adjacent frame or an interval frame of the initial frame of fan blade image;
the rotating unit is used for rotating the P and the Q through the rotating transformation matrix M to obtain P 'and Q'; m is a rotation transformation matrix corresponding to the rotation of the fan blade in the fan blade image in the specified direction;
the calculation unit is used for calculating and obtaining the homography matrix H of the P 'and the Q' according to the following formula: q '═ H · P';
and the splicing unit is used for finishing image registration and splicing of the fan blade images of the initial frame and the next frame by adopting an image registration method based on the characteristic points according to the homography matrix H.
7. The fan blade image stitching device according to claim 6, wherein: the system also comprises an abnormal point removing unit which is used for removing the abnormal points in the tracking point set Q by adopting RANSAC algorithm and correspondingly removing the characteristic points corresponding to the abnormal points in the initial point set P.
8. The fan blade image stitching device according to claim 7, wherein: and the abnormal point eliminating unit stops tracking when the characteristic points in the tracking point set Q are smaller than a preset numerical value.
9. The fan blade image stitching device according to any one of claims 6 to 8, wherein: the device also comprises a splicing line eliminating unit which is used for eliminating the splicing lines in the images to be spliced by adopting a multiband fusion method.
10. The fan blade image stitching device of claim 9, wherein: further comprising a rotation transformation matrix calculation unit for:
acquiring pixel coordinates of upper and lower edges of fan blades in a fan blade image;
calculating the inclination slope of the fan blade according to the pixel coordinates of the upper edge and the lower edge to obtain a rotation angle;
and obtaining a rotation transformation matrix M according to the rotation angle, the specified angle and the image rotation center of the fan blade image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111609230.4A CN114266701A (en) | 2021-12-27 | 2021-12-27 | A kind of fan blade image stitching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111609230.4A CN114266701A (en) | 2021-12-27 | 2021-12-27 | A kind of fan blade image stitching method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114266701A true CN114266701A (en) | 2022-04-01 |
Family
ID=80830127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111609230.4A Pending CN114266701A (en) | 2021-12-27 | 2021-12-27 | A kind of fan blade image stitching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114266701A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106952225A (en) * | 2017-02-15 | 2017-07-14 | 山东科技大学 | A panorama mosaic method for forest fire protection |
| CN109961398A (en) * | 2019-02-18 | 2019-07-02 | 鲁能新能源(集团)有限公司 | Fan blade image segmentation and grid optimization joining method |
| CN110084743A (en) * | 2019-01-25 | 2019-08-02 | 电子科技大学 | Image mosaic and localization method based on more air strips starting track constraint |
| CN110136059A (en) * | 2019-04-04 | 2019-08-16 | 哈尔滨理工大学 | Image mosaic method of wind turbine blades based on straight line features |
| CN111899179A (en) * | 2019-12-04 | 2020-11-06 | 华南农业大学 | Unmanned aerial vehicle image splicing method and device based on Harris-SURF |
-
2021
- 2021-12-27 CN CN202111609230.4A patent/CN114266701A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106952225A (en) * | 2017-02-15 | 2017-07-14 | 山东科技大学 | A panorama mosaic method for forest fire protection |
| CN110084743A (en) * | 2019-01-25 | 2019-08-02 | 电子科技大学 | Image mosaic and localization method based on more air strips starting track constraint |
| CN109961398A (en) * | 2019-02-18 | 2019-07-02 | 鲁能新能源(集团)有限公司 | Fan blade image segmentation and grid optimization joining method |
| CN110136059A (en) * | 2019-04-04 | 2019-08-16 | 哈尔滨理工大学 | Image mosaic method of wind turbine blades based on straight line features |
| CN111899179A (en) * | 2019-12-04 | 2020-11-06 | 华南农业大学 | Unmanned aerial vehicle image splicing method and device based on Harris-SURF |
Non-Patent Citations (2)
| Title |
|---|
| 张楚东等: "基于稀疏光流的道路航拍图像拼接算法", 《舰船电子工程》, vol. 38, no. 8, 20 August 2018 (2018-08-20), pages 31 - 36 * |
| 王传宇;赵明;阎建河;周顺利;张英华;: "基于双目立体视觉技术的玉米叶片三维重建", 农业工程学报, no. 04, 30 April 2010 (2010-04-30) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109525194B (en) | Photovoltaic panel fault spot detection and identification method and system | |
| CN106290388B (en) | A kind of insulator breakdown automatic testing method | |
| WO2022027931A1 (en) | Video image-based foreground detection method for vehicle in motion | |
| CN114266895A (en) | Fan blade image segmentation and splicing method and device | |
| CN102254149B (en) | Method for detecting and identifying raindrops in video image | |
| CN109632808B (en) | Edge defect detection method and device, electronic equipment and storage medium | |
| CN109559324B (en) | A method of target contour detection in line image | |
| CN109146832B (en) | Video image splicing method and device, terminal equipment and storage medium | |
| CN110390696A (en) | A visual detection method of circular hole pose based on image super-resolution reconstruction | |
| CN109946304A (en) | On-line detection system and detection method for surface defects of parts based on feature matching | |
| CN107133969A (en) | A kind of mobile platform moving target detecting method based on background back projection | |
| CN106198569A (en) | A kind of LTPS/IGZO glass substrate broken hole method for quick | |
| CN107423737A (en) | The video quality diagnosing method that foreign matter blocks | |
| CN116758425A (en) | Automatic acceptance checking method and device for large-base photovoltaic power station | |
| Zhang et al. | Automatic diagnosis system of transmission line abnormalities and defects based on UAV | |
| CN107784652A (en) | A kind of shaft tower quick determination method based on unmanned plane image | |
| CN105913464A (en) | Multi-body target online measurement method based on videos | |
| CN114037682A (en) | A two-dimensional automatic detection method for surface defects of optical components | |
| CN112991374A (en) | Canny algorithm-based edge enhancement method, device, equipment and storage medium | |
| CN114897699B (en) | Infrared image stitching method and device for fan blade | |
| CN110807416A (en) | Digital instrument intelligent recognition device and method suitable for mobile detection device | |
| CN113570587A (en) | Photovoltaic cell broken grid detection method and system based on computer vision | |
| CN113192061A (en) | LED package appearance detection image extraction method and device, electronic equipment and storage medium | |
| CN105574860B (en) | A kind of rudder piece deflection polarity recognition methods | |
| CN114092385A (en) | Industrial machine fault detection method and device based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |