CN109859202A - A kind of deep learning detection method based on the tracking of USV water surface optical target - Google Patents
A kind of deep learning detection method based on the tracking of USV water surface optical target Download PDFInfo
- Publication number
- CN109859202A CN109859202A CN201910120133.5A CN201910120133A CN109859202A CN 109859202 A CN109859202 A CN 109859202A CN 201910120133 A CN201910120133 A CN 201910120133A CN 109859202 A CN109859202 A CN 109859202A
- Authority
- CN
- China
- Prior art keywords
- target
- usv
- distance
- water surface
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 59
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 28
- 238000013135 deep learning Methods 0.000 title claims abstract description 15
- 230000003287 optical effect Effects 0.000 title claims abstract description 15
- 230000033001 locomotion Effects 0.000 claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 24
- 230000008569 process Effects 0.000 claims abstract description 13
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 12
- 238000004891 communication Methods 0.000 claims description 21
- 230000006399 behavior Effects 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 239000000284 extract Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000009434 installation Methods 0.000 claims description 5
- 238000005316 response function Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 2
- 238000013459 approach Methods 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 6
- 230000008447 perception Effects 0.000 abstract description 4
- 239000013535 sea water Substances 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000007613 environmental effect Effects 0.000 abstract 1
- 230000008859 change Effects 0.000 description 8
- 230000003068 static effect Effects 0.000 description 8
- 238000012795 verification Methods 0.000 description 6
- 238000005286 illumination Methods 0.000 description 5
- 238000011835 investigation Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000001537 neural effect Effects 0.000 description 3
- 230000000452 restraining effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 239000003653 coastal water Substances 0.000 description 2
- 230000034994 death Effects 0.000 description 2
- 231100000517 death Toxicity 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 210000004218 nerve net Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
本发明属于水面无人平台环境感知和控制系统交叉技术领域，具体涉及一种基于USV水面光学目标跟踪的深度学习检测方法：USV装载摄像机实时采集视频，并将视频信号通过图像采集卡传给USV内部嵌入式计算机，计算机先每隔n帧选定一个关键帧，再对其进行去模糊化处理，最后利用卷积神经网络检测出水面目标；本发明在水面目标检测与定位过程中，加入了基于卷积神经网络的图像去模糊技术，改善了海水波动、目标物运动和USV航行造成的目标模糊问题，且基于24个卷积层和2个全连接层的回归方式检测目标具有速度快、背景误检率低的优点，使USV目标检测具有实时性，且不易受海面阳光折射等干扰。
The invention belongs to the technical field of the intersection of environmental perception and control systems for unmanned platforms on water, and in particular relates to a deep learning detection method based on USV water surface optical target tracking: the USV mounts a camera to collect video in real time, and transmits the video signal to the USV through an image capture card Internal embedded computer, the computer first selects a key frame every n frames, then de-blurs it, and finally detects the water surface target by using the convolutional neural network; the present invention adds the water surface target detection and positioning process. The image deblurring technology based on convolutional neural network improves the target blur problem caused by seawater fluctuation, target motion and USV navigation, and the regression method based on 24 convolutional layers and 2 fully connected layers detects targets with high speed, The advantage of low background false detection rate makes USV target detection real-time, and it is not easy to be disturbed by the refraction of sunlight on the sea surface.
Description
Technical field
The invention belongs to the unmanned platform environment perception of the water surface and control system interleaving techniques fields, and in particular to one kind is based on
The deep learning detection method of USV water surface optical target tracking.
Background technique
Unmanned water surface ship (Unmanned Surface Vehicle, abbreviation USV), is increasingly becoming the supplement of Ship platform
Or substitute, have many advantages, such as small in size, high speed, preferable stealthy, intelligent, unmanned injures and deaths, sea can be completed at lower cost
On a large scale, a variety of military and non-military operations also can be performed in scientific investigation and engineering duty for a long time, as sea area searches and rescues, leads
Boat and hydro_geography prospecting；Hydrographic information monitoring, marine meterologal prediction, aquatic organism research, exploration of ocean resources and region
Sea chart is drawn；Coastal waters zone defence；Scouting, search, detection and the removal of mines of specified sea areas；Anti-submarine warfare, anti-special operations and
Hit pirate, anti-terrorism attack etc..
USV needs autonomous navigation in the case where unmanned intervene and completes various tasks, so must have to ocean ring
Border and the entirely autonomous perception and understandability of all kinds of realizations of goal therein, wherein target detection is to can be USV with tracking
The information such as position, posture and the motion profile of waterborne target are provided.Video camera detected by receiving target ontology radiation energy and
Target is tracked, there is anti-reconnaissance capability and the stronger advantage of anti-electronic jamming capability, and the target information observed is abundant, directly
The property seen and reliability are relatively high.Currently, countries in the world, which have had a profound understanding of, develops the object detecting and tracking based on light vision
The importance and urgency of technology.It searched and rescued for sea area, specified sea areas investigation, hit the tasks such as pirate, need closely to detect
Even strike target.Therefore, the target tracking strategy for studying USV also has having very important significance.
Do detection and tracking problems faced using light vision system in USV at present to be mainly reflected in: illumination and weather become
Change will affect picture quality, the movement of object often results in objective fuzzy, this just largely reduces target detection tracking
Accuracy rate；The shake of camera caused by seawater fluctuation and USV high speed operation, not only increases challenge to target detection, to USV
Motion control will also result in very big influence.
Summary of the invention
Present invention aim to address deficiency in the prior art, provide a kind of based on the tracking of USV water surface optical target
Deep learning detection method solves the problems, such as the high false alarm rate of target detection when distant object tracking, reduces objective fuzzy and light
Interference according to variation to object detecting and tracking.
A kind of deep learning detection method based on the tracking of USV water surface optical target is detected by waterborne target and is positioned, closes
Three target following of key interframe, USV path planning and behaviour control system compositions, comprising the following steps:
Waterborne target detection and positioning system step are as follows:
(1.1) USV loads video camera and acquires video in real time, and vision signal is transmitted to inside USV by image pick-up card
Embedded computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes volume
Product neural network detects waterborne target；
(1.2) it is calculated using the binocular camera binocular distance measuring method on USV in the target detected in step (1.1)
Heart point to left lens camera distance, by this position of the distance as target relative to video camera；The IMU carried further according to USV
Installation site of the Angle of Trim and roll angle, left lens camera of measured USV on USV, it is opposite by calculating acquisition target
The position of USV and angle information；
(1.3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe Target Tracking System step are as follows:
(2.1) target detected according to step (1.1) extracts the LBP feature in key frame target region, according to feature
The probability density of model establishes object module；
(2.2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(2.3) target position of Meanshift vector iterative search present frame is used；
(2.4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame mesh is exported
Cursor position；
(2.5) communication protocol comprising object location information is sent to USV path planning with network communication；
(2.6) step (2.2)-(2.5) are repeated until the tracking of crucial interframe terminates, after return step (1.1)；
USV path planning and behaviour control system step are as follows:
(3.1) path planning layer circulation waiting step (1.3), (2.5) send communication protocol, once receiving, are just added
And verification, by the location information for extracting object after verification；
(3.2) information such as target range, angle extracted according to step (3.1), start to carry out path planning, and by result
It is sent to behaviour control layer；
(3.3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
The USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target, comprising:
(1.1.1) generates the principle of blurred picture, the M frame image captured during exposure, after being averaged according to sensor
It is mapped as blurred picture with camera response function, clear image and blurred picture pair are produced with this, passes through training convolutional nerve net
Network learns the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(1.1.2) makes waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
The prediction of classification；
(1.1.3) first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate.
The target detected according to step (1.1) extracts the LBP feature in key frame target region, according to feature
The probability density of model establishes object module, comprising:
According to the target that step (1.1) detects, the LBP feature of target area is extracted, calculation is as follows:
Wherein, gcRepresent central pixel point (xc,yc) gray value；For p using central pixel point as origin, R is the field of radius
Inside there are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, 0 is taken when x is less than 0, otherwise
Take 1.
LBP feature space is quantified as m grade, calculates the core histogram probability density of each grade, generation is
Object module.
The information such as target range, the angle that the basis (3.1) is extracted start to carry out path planning, and result are sent
To behaviour control layer, comprising:
(3.2.1) considers that false-alarm feelings may occur for the influence of uncertain factor in sensor detection process, target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin, when USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change rule at this time
The path drawn；
(3.2.2) is when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor
It can detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from image at this time
Heart abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point
Abscissa is away from picture centre abscissa mThWithin a pixel；
(3.2.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, is delayed at this time
Slowly close-target is leaned on, until certain safe distance DsWhen stop motion, the state of observed object: use Kalman Filter Estimation target
Speed of related movement v, if be less than setting threshold value vTh, then it is considered as static target, otherwise it is assumed that being moving target.For quiet
Only target sends to control layer and instructs, and so that USV is done rotary motion around it in the form of face object, prevents target unexpected
It is mobile；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
The beneficial effects of the present invention are:
In waterborne target detection with position fixing process, it joined the Smear-eliminated technique of image based on convolutional neural networks, change
It has been apt to objective fuzzy problem caused by seawater fluctuation, object movement and USV are navigated by water, and has been connected entirely based on 24 convolutional layers and 2
The recurrence mode for connecing layer, which detects target, has the advantages that speed is fast, background false detection rate is low, and USV target detection is made to have real-time,
And not vulnerable to interference such as sea sunlight refractions.In crucial interframe object tracking process, with LBP operator extraction image object area
The textural characteristics in domain reduce illumination variation interference caused by target following as object module.In USV path planning and
During behaviour control, so that target is maintained at center to angle using remote adjustment USV bow, closely chased after according to location information
The mode of track target solves the problems, such as that distant object position error is big, target detection false alarm rate is high.
Detailed description of the invention
Fig. 1 is overall flow figure of the invention；
Fig. 2 is step 2 of embodiment of the present invention key interframe target following program flow chart；
Fig. 3 is binocular ranging flow diagram；
Fig. 4 is the simplification relational expression model of sensor detection probability and target relative distance；
Specific embodiment
The present invention is described further with reference to the accompanying drawing.
The present invention discloses a kind of deep learning detection method based on the tracking of USV water surface optical target, includes the following steps:
S1) video camera acquires video in real time, and first selecting key frame, de-fuzzy is handled again, finally detects the water surface with convolutional neural networks
Target, and with binocular ranging localization its with respect to USV position, location information is transmitted to path planning layer；S2) crucial interframe
Target following, according to the similarity iterative search target of object module and target candidate model based on LBP feature in present frame
Position, equally transmission position, return to S1 after tracking；S3) path planning layer planning path tracks target, and setting is maximum
Threshold value and minimum threshold, current goal distance do not change path when being more than max-thresholds；Current goal distance is in max-thresholds
Between minimum threshold, adjustment bow keeps object central point to deviate picture centre axis no more than m to angleThA pixel；Currently
It is slowly close to safe distance D when target range is less than minimum thresholds, Kalman Filter Estimation target relative movement speed v,
If being less than threshold value vThThen it is considered as static, and is turned round around target, be otherwise considered as movement, stablizing under safe distance is kept to track.This
Invention is big to illumination variation, the scene detection effect of camera shake is good, strong real-time, and tracking when enhancing different distance
Reliability.
A kind of deep learning detection method based on the tracking of USV water surface optical target, this method include waterborne target detection
With positioning, three crucial interframe target following, USV path planning and behaviour control systems, waterborne target detection and positioning system
Step are as follows:
(a1) USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target；
(a2) it is arrived using the central point that the binocular camera binocular distance measuring method on USV calculates the target detected in (a1)
The distance of left lens camera, by this position of the distance as target relative to video camera；Measured by the IMU carried further according to USV
USV installation site on USV of Angle of Trim and roll angle, left lens camera, pass through to calculate and obtain target with respect to USV's
Position and angle information；
(a3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe Target Tracking System step are as follows:
(b1) target detected according to (a1) extracts LBP (the Local Binary in key frame target region
Pattern, local binary patterns) feature, object module is established according to the probability density of feature submodel；
(b2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(b3) target position of Meanshift vector iterative search present frame is used；
(b4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame target is exported
Position；
(b5) communication protocol comprising object location information is sent to USV path planning with network communication；
(b6) repeat (b2)~(b5) until the tracking of crucial interframe terminates, after return to (a1)；
USV path planning and behaviour control system step are as follows:
(c1) path planning layer circulation waits (a3), (b5) to send communication protocol, once receiving, just sums up verification,
By the location information for extracting object after verification；
(c2) according to information such as (c1) target range, angles extracted, start to carry out path planning, and result is sent to
Behaviour control layer；
(c3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
The step (a1) includes the following steps:
(a1.1) principle of blurred picture is generated according to sensor, the M frame image captured during exposure is used after being averaged
Camera response function is mapped as blurred picture, produces clear image and blurred picture pair with this.Pass through training convolutional neural networks
Learn the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(a1.2) waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT are made in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
The prediction of classification；
(a1.3) it first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate；
The step (b1) includes the following steps:
(b1.1) target detected according to (a1), extracts the LBP feature of target area, calculation is as follows:
Wherein gcRepresent central pixel point (xc,yc) gray value；Using central pixel point as origin, R is in the field of radius
There are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, and 0 is taken when x is less than 0, is otherwise taken
1。
(b1.2) it is quantified as m grade in LBP feature space, calculates the core histogram probability density of each grade, generated
Be object module.
The step (c2) includes the following steps:
(c2.1) in view of the influence of uncertain factor in sensor detection process, false-alarm feelings may occur for target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin.When USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change planning at this time
Path；
(c2.2) when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor energy
Detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from picture centre at this time
Abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point horizontal
Coordinate is away from picture centre abscissa mThWithin a pixel；
(c2.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time slowly
By close-target, until certain safe distance DsWhen stop motion, the state of observed object: with Kalman Filter Estimation target
Speed of related movement v, if being less than the threshold value v of settingTh, then it is considered as static target, otherwise it is assumed that being moving target.For static
Target sends to control layer and instructs, and so that USV is done rotary motion around it in the form of face object, prevents target from moving suddenly
It is dynamic；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
Unmanned water surface ship (Unmanned Surface Vehicle, abbreviation USV), is increasingly becoming the supplement of Ship platform
Or substitute, have many advantages, such as small in size, high speed, preferable stealthy, intelligent, unmanned injures and deaths, sea can be completed at lower cost
On a large scale, a variety of military and non-military operations also can be performed in scientific investigation and engineering duty for a long time, as sea area searches and rescues, leads
Boat and hydro_geography prospecting；Hydrographic information monitoring, marine meterologal prediction, aquatic organism research, exploration of ocean resources and region
Sea chart is drawn；Coastal waters zone defence；Scouting, search, detection and the removal of mines of specified sea areas；Anti-submarine warfare, anti-special operations and
Hit pirate, anti-terrorism attack etc..
USV needs autonomous navigation in the case where unmanned intervene and completes various tasks, so must have to ocean ring
Border and the entirely autonomous perception and understandability of all kinds of realizations of goal therein, wherein target detection is to can be USV with tracking
The information such as position, posture and the motion profile of waterborne target are provided.Video camera detected by receiving target ontology radiation energy and
Target is tracked, there is anti-reconnaissance capability and the stronger advantage of anti-electronic jamming capability, and the target information observed is abundant, directly
The property seen and reliability are relatively high.Currently, countries in the world, which have had a profound understanding of, develops the object detecting and tracking based on light vision
The importance and urgency of technology.It searched and rescued for sea area, specified sea areas investigation, hit the tasks such as pirate, need closely to detect
Even strike target.Therefore, the target tracking strategy for studying USV also has having very important significance.
Do detection and tracking problems faced using light vision system in USV at present to be mainly reflected in: illumination and weather become
Change will affect picture quality, the movement of object often results in objective fuzzy, this just largely reduces target detection tracking
Accuracy rate；The shake of camera caused by seawater fluctuation and USV high speed operation, not only increases challenge to target detection, to USV
Motion control will also result in very big influence.
Present invention aim to address deficiency in the prior art, provide a kind of based on the tracking of USV water surface optical target
Deep learning detection method solves the problems, such as the high false alarm rate of target detection when distant object tracking, reduces objective fuzzy and light
Interference according to variation to object detecting and tracking.
In order to achieve the above object, the technical solution adopted by the present invention are as follows: it is a kind of based on the USV water surface optical target tracking
Deep learning detection method, this method include waterborne target detection with positioning, crucial interframe target following, USV path planning and
Three systems of behaviour control, waterborne target detection and positioning system step are as follows:
(a1) USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target；
(a2) central point of the target that (a1) is detected is calculated using the binocular camera binocular distance measuring method on USV to left
The distance of lens camera, by this position of the distance as target relative to video camera；Measured by the IMU carried further according to USV
Installation site of the Angle of Trim and roll angle, left lens camera of USV on USV obtains position of the target with respect to USV by calculating
It sets and angle information；
(a3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe target following step are as follows:
(b1) target detected according to (a1) extracts LBP (the Local Binary in key frame target region
Pattern, local binary patterns) feature, object module is established according to the probability density of feature submodel；
(b2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(b3) target position of Meanshift vector iterative search present frame is used；
(b4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame target is exported
Position；
(b5) communication protocol comprising object location information is sent to USV path planning with network communication；
(b6) repeat (b2)~(b5) until the tracking of crucial interframe terminates, after return to (a1)；
USV path planning and behaviour control system step are as follows:
(c1) path planning layer circulation waits (a3), (b5) to send communication protocol, once receiving, just sums up verification,
By the location information for extracting object after verification；
(c2) according to information such as (c1) target range, angles extracted, start to carry out path planning, and result is sent to
Behaviour control layer；
(c3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
Further, the step (a1) includes the following steps:
(a1.1) principle of blurred picture is generated according to sensor, the M frame image captured during exposure is used after being averaged
Camera response function is mapped as blurred picture, produces clear image and blurred picture pair with this.Pass through training convolutional neural networks
Learn the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(a1.2) waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT are made in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
Class prediction；
(a1.3) it first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate；
Further, the step (b1) includes the following steps:
(b6.1) target detected according to (a1), extracts the LBP feature of target area, calculation is as follows:
Wherein gcRepresent central pixel point (xc,yc) gray value；Using central pixel point as origin, R is in the field of radius
There are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, and 0 is taken when x is less than 0, is otherwise taken
1。
(b6.2) it is quantified as m grade in LBP feature space, calculates the core histogram probability density of each grade, generated
Be object module.
Further, the step (c3) includes the following steps:
(c3.1) in view of the influence of uncertain factor in sensor detection process, false-alarm feelings may occur for target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin.When USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change planning at this time
Path；
(c3.2) when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor energy
Detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from picture centre at this time
Abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point horizontal
Coordinate is away from picture centre abscissa mThWithin a pixel；
(c3.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time with slow
Slow speed leans on close-target, until certain safe distance is DsWhen stop movement, the state of observed object: use Kalman filtering
The speed of related movement v of target is estimated, if being less than the threshold value v of settingTh, then it is considered as static target, otherwise it is assumed that being movement mesh
Mark.It for static target, sends and instructs to control layer, USV is made to do rotary motion around it in the form of face object, prevent
Only target moves suddenly；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
A kind of deep learning detection method based on the tracking of USV water surface optical target proposed by the present invention, including walk as follows
It is rapid:
Step 1: the detection of key frame waterborne target and positioning are carried out to the video of input, save target object location and classification
Input of the information as next step, and transmit these information to foundation of the path planning layer as decision.Especially by following
Step is implemented:
1.1, first in the preparation stage, need: 1) the convolutional neural networks model of training image deblurring, make clear figure
The image pair of picture and blurred picture, wherein blurred picture uses for reference the principle that camera generates blurred picture: camera is indirect in exposure period
Light is received, clear image can accumulate stimulation every time to generate blurred picture, be simulated and be generated by following formula:
In above formula, M is the sampling frame number captured during exposure, and s [i] is i-th of articulating frame, and g is to reflect clear image
Penetrate into the camera response function of observation image.The non-linear of blurred picture and clear image is fitted by convolutional neural networks
Mapping relations realize deblurring.
2) the convolutional neural networks model of training objective detection.Production includes passenger boat, sailing boat, yacht, warship, five class of buoy
The data set of target, totally 1000, wherein training set, verifying collection and test set account for 60%, 10%, 30% respectively.Pass through migration
Learning trained convolutional layer and two full articulamentums, loss function expression formula based on GoogLeNeT end-to-endly is
In above formula, s2Indicate that image is divided into s × s lattice, B represents the bounding box number of each prediction, wherein the s of this paper
7, B is taken to take 2.I indicates that the sequence number of grid, j indicate bounding box sequence number,Indicate whether there is object on i-th of grid,WithJ-th of bounding box for respectively indicating i-th of grid is responsible for and not responsible detection object, and x, y, w, h are in object
Heart point coordinate, width and height, CiAnd piIt (c) is the probability of i-th of quadrille object and the probability for belonging to c class.
1.2 input key frame in network, obtain one 7 × 7 × 15 vector, 15 represent the p of each gridi(c) and 2
The coordinate (x, y, w, h) and C of the bounding box of a predictioni, find out pi(c)×CiGreater than the bounding box of threshold value, and with non-very big suppression
System rejects the bounding box for repeating to predict same target, and finally obtained is the target detected.
The center position of the 1.3 object boundary frame baselines obtained according to 1.2 calculate using binocular distance measuring method and be obtained
The central point of object boundary frame baseline is obtained to the distance of left lens camera, by this position of the distance as target relative to video camera
It sets；Installation of the Angle of Trim θ and roll angle Ф, left lens camera of USV measured by the IMU carried further according to USV on USV
Position obtains position of the target with respect to USV and angle information by calculating.Binocular ranging process is as shown in Figure 3.
To obtain target relative position in a manner of agreement by network communication and transmission to path planning layer as decision
Foundation.
Step 2: for crucial interframe target following, color characteristic is converted into textural characteristics, reduces illumination variation to mesh
The influence for marking tracking, implements especially by following steps:
The 2.1 coordinates of targets information obtained according to step 1.2, calculate the LBP feature of target area, calculation is as follows:
In above formula, LBPP,R(xc,yc) and gcIt is central pixel point (xc,yc) LBP characteristic value and gray value, R be middle imago
For vegetarian refreshments at a distance from neighbor pixel, P is neighbor pixel number, gpRepresent the gray value of neighbor pixel p, the value of s (x)
For two-valued function, otherwise it is 1 that when x is less than 0, value, which is 0,.
2.2 for object module and target candidate model, the probability density of u=1,2 ..., m grades of LBP feature submodels
Are as follows:
In above formula,K (x) is kernel function, and h is kernel function bandwidth, and y is mesh
Candidate region central point is marked, n is pixel number in region, xiIt is the normalization position of pixel in region, b (xi) it is xi?
Aspect indexing value in histogram.
2.3 introduce the similarity between Bhattacharyya coefficient metric objective model and target candidate model, use
The target position of Meanshift iterative search present frame makes similarity reach maximum, iteration once after kernel function central point by
y0It is moved to y1:
2.4 set primary iteration number as 0, and every iteration an iteration number adds one, when the drift distance of target's center position
When exceeding threshold value lower than threshold value or the number of iterations, the corresponding target position of the secondary iteration result and road is transferred to according to 1.3 outputs
Diameter planning layer.
Step 3: during USV path planning and behaviour control, according to target detection in step 1 and step 2 with
The target relative position positioned when tracking does different decisions to remote and short distance respectively and tracks target, specific logical
Cross following steps implementation:
3.1 in view of uncertain factor in sensor detection process influence, sensor detection probability it is opposite with target away from
From simplification relational expression it is as follows, model is as shown in Figure 4.
P in above formulaD∈ [0,1] is the detection probability of sensor, PF∈ [0,1] is the false-alarm probability of sensor, p (b (k) |
dk) it is detection probability of the sensor at the k moment, otherwise it is 0, d that b (k), which is two-value vector, and it is then 1 that the k moment, which detects target,kIt is
The relative distance of k moment USV and target.
Set the max-thresholds d an of distancemaxWith minimum threshold dmin.When being more than d with target rangemaxWhen, sensor
Detection probability is false-alarm probability, is difficult accurately to detect dbjective state at this time, detects target if still reporting, then it is assumed that is empty
It is alert, do not change the path of planning at this time.
When being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor can detect
The presence of target, but detection probability is observed with detection range linear decline and still has uncertainty in the section.At this time
If the abscissa of object central point is more than m away from picture centre abscissaThA pixel, then according to the pixel number exceeded
USV bow is adjusted to angle, keeps object central point abscissa away from picture centre abscissa mThWithin a pixel, bow is adjusted
Frequency to angle is excessively high larger to USV loss, therefore threshold value mThWhat should not be arranged is too small.
When with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time with slowly speed
Degree leans on close-target, until certain safe distance is DsWhen stop motion, with the speed of related movement of Kalman Filter Estimation target
V is with object observing motion state, the tasks such as completion close-ups even strike target.If being less than the threshold value v of settingTh, then regard
For static target, otherwise it is assumed that being moving target.For static target, sends and instruct to control layer, make it with face object
Form do rotary motion around it, prevent target from moving suddenly；For moving target, then USV is enabled to keep certain safe distance
For DsStablize tracking.
Claims (4)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910120133.5A CN109859202B (en) | 2019-02-18 | 2019-02-18 | A deep learning detection method based on USV water surface optical target tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910120133.5A CN109859202B (en) | 2019-02-18 | 2019-02-18 | A deep learning detection method based on USV water surface optical target tracking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859202A true CN109859202A (en) | 2019-06-07 |
| CN109859202B CN109859202B (en) | 2022-04-12 |
Family
ID=66898273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910120133.5A Active CN109859202B (en) | 2019-02-18 | 2019-02-18 | A deep learning detection method based on USV water surface optical target tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859202B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110398720A (en) * | 2019-08-21 | 2019-11-01 | 深圳耐杰电子技术有限公司 | A kind of anti-unmanned plane detection tracking interference system and photoelectric follow-up working method |
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| CN110930433A (en) * | 2019-11-21 | 2020-03-27 | 华南理工大学 | High-speed moving object machine vision target detection method based on image restoration |
| CN111538059A (en) * | 2020-05-11 | 2020-08-14 | 东华大学 | Self-adaptive rapid dynamic positioning system and method based on improved Boltzmann machine |
| CN117292103A (en) * | 2023-09-05 | 2023-12-26 | 海南大学 | Marine target ranging method based on embedded monocular vision |
| CN118135550A (en) * | 2024-04-03 | 2024-06-04 | 武汉船讯科技有限公司 | Water surface multi-target distance measurement method, system and storage medium based on deep learning and monocular vision |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140266851A1 (en) * | 2013-03-14 | 2014-09-18 | Counter Echo Solutions LLC | System and Methods for Countering Satellite-Navigated Munitions |
| CN106773741A (en) * | 2017-03-02 | 2017-05-31 | 华南理工大学 | A kind of unmanned boat dynamic positioning system and method |
| CN107818571A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | Ship automatic tracking method and system based on deep learning network and average drifting |
| CN108303078A (en) * | 2018-01-29 | 2018-07-20 | 哈尔滨工程大学 | A kind of early warning of omnidirection shipping anti-collision and navigation system based on stereoscopic vision |
| CN108459602A (en) * | 2018-02-28 | 2018-08-28 | 哈尔滨工程大学 | The autonomous method of docking of drive lacking unmanned boat under more obstacle complex environments |
| CN108803632A (en) * | 2018-09-19 | 2018-11-13 | 哈尔滨工程大学 | Unmanned surface vehicle total state constrained trajectory tracking and controlling method based on saturation compensation technology |
| CN109242019A (en) * | 2018-09-01 | 2019-01-18 | 哈尔滨工程大学 | A kind of water surface optics Small object quickly detects and tracking |
-
2019
- 2019-02-18 CN CN201910120133.5A patent/CN109859202B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140266851A1 (en) * | 2013-03-14 | 2014-09-18 | Counter Echo Solutions LLC | System and Methods for Countering Satellite-Navigated Munitions |
| CN106773741A (en) * | 2017-03-02 | 2017-05-31 | 华南理工大学 | A kind of unmanned boat dynamic positioning system and method |
| CN107818571A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | Ship automatic tracking method and system based on deep learning network and average drifting |
| CN108303078A (en) * | 2018-01-29 | 2018-07-20 | 哈尔滨工程大学 | A kind of early warning of omnidirection shipping anti-collision and navigation system based on stereoscopic vision |
| CN108459602A (en) * | 2018-02-28 | 2018-08-28 | 哈尔滨工程大学 | The autonomous method of docking of drive lacking unmanned boat under more obstacle complex environments |
| CN109242019A (en) * | 2018-09-01 | 2019-01-18 | 哈尔滨工程大学 | A kind of water surface optics Small object quickly detects and tracking |
| CN108803632A (en) * | 2018-09-19 | 2018-11-13 | 哈尔滨工程大学 | Unmanned surface vehicle total state constrained trajectory tracking and controlling method based on saturation compensation technology |
Non-Patent Citations (2)
| Title |
|---|
| GREGORY P. SCOTT ET.AL: "Autonomous robotic refueling of an unmanned surface vehicle in varying sea states", 《2015 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)》 * |
| 周鸿祥: "无人艇的目标跟踪策略研究", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110393165B (en) * | 2019-07-11 | 2021-06-25 | 浙江大学宁波理工学院 | Open sea aquaculture net cage bait feeding method based on automatic bait feeding boat |
| CN110398720A (en) * | 2019-08-21 | 2019-11-01 | 深圳耐杰电子技术有限公司 | A kind of anti-unmanned plane detection tracking interference system and photoelectric follow-up working method |
| CN110398720B (en) * | 2019-08-21 | 2024-05-03 | 深圳耐杰电子技术有限公司 | Anti-unmanned aerial vehicle detection tracking interference system and working method of photoelectric tracking system |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| CN110930433A (en) * | 2019-11-21 | 2020-03-27 | 华南理工大学 | High-speed moving object machine vision target detection method based on image restoration |
| CN111538059A (en) * | 2020-05-11 | 2020-08-14 | 东华大学 | Self-adaptive rapid dynamic positioning system and method based on improved Boltzmann machine |
| CN117292103A (en) * | 2023-09-05 | 2023-12-26 | 海南大学 | Marine target ranging method based on embedded monocular vision |
| CN118135550A (en) * | 2024-04-03 | 2024-06-04 | 武汉船讯科技有限公司 | Water surface multi-target distance measurement method, system and storage medium based on deep learning and monocular vision |
| CN118135550B (en) * | 2024-04-03 | 2024-09-06 | 武汉船讯科技有限公司 | Water surface multi-target distance measurement method, system and storage medium based on deep learning and monocular vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859202B (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109859202A (en) | A kind of deep learning detection method based on the tracking of USV water surface optical target | |
| Liu et al. | Detection and pose estimation for short-range vision-based underwater docking | |
| Lyu et al. | Sea-surface object detection based on electro-optical sensors: A review | |
| CN109919026B (en) | Surface unmanned ship local path planning method | |
| Zhou et al. | Collision-free waterway segmentation for inland unmanned surface vehicles | |
| Liu et al. | Ship recognition and tracking system for intelligent ship based on deep learning framework | |
| CN111723632B (en) | Ship tracking method and system based on twin network | |
| Petraglia et al. | Pipeline tracking and event classification for an automatic inspection vision system | |
| CN117218380A (en) | Dynamic target detection tracking method for unmanned ship remote sensing image | |
| KR20220045762A (en) | System for automatic recognition and monitoring of vessel using artificial intelligence image processing and method for providing the same | |
| CN119723049A (en) | A method for detecting and identifying marine targets based on the combination of radar and optoelectronic system | |
| Zhang et al. | Nearshore vessel detection based on Scene-mask R-CNN in remote sensing image | |
| CN114913409A (en) | Camouflage target identification method for marine organisms | |
| Dang et al. | Rcyolo: An efficient small target detector for crack detection in tubular topological road structures based on unmanned aerial vehicles | |
| CN116704688A (en) | Ocean buoy passive defense method and system | |
| CN116246139A (en) | Target identification method based on multi-sensor fusion for unmanned ship navigation environment | |
| Heng et al. | A review of intelligent detection methods for underwater targets in sonar images | |
| Zhu et al. | Saliency detection for underwater moving object with sonar based on motion estimation and multi-trajectory analysis | |
| CN115601397A (en) | Ship track tracking and predicting method based on monocular camera | |
| CN115080903A (en) | Offshore multi-target multi-modal matching fusion method based on intelligent optimization algorithm | |
| CN109887004A (en) | A TLD Algorithm-Based Method for Target Tracking of Unmanned Ships in Sea Areas | |
| CN113822297A (en) | A device and method for identifying a target of a marine vessel | |
| Zhang et al. | A review of research on light visual perception of unmanned surface vehicles | |
| CN117593700A (en) | Water rescue method and system combining machine vision and dynamic positioning | |
| He et al. | A coastal obstacle detection framework of dual USVs based on dual-view color fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |